Versions Compared

Old Version 35

changes.mady.by.user Daniela Bourges-Waldegg

Saved on

compared with

New Version 36

changes.mady.by.user Daniela Bourges-Waldegg

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Warning

The ETLer expects data to be entered into one of the generated templates, and a few conventions to be respected (see Data preparation and ETL Workflow) . A data curator usually makes sure that the template is correctly filled. In particular, the location of the resources to be ETLd (e.g. Lab or Core facility name) must be provided in every row of data.

  • Place your input files (i.e. the completed templates) in a directory of your choice, e.g. dataDirectory. All files contained in this directory will be processed by the ETLer.

  • To run an ETL, execute one of the

...

  • two following commands. 

    For creating new resources:

    No Format
    ./ETLer.sh -d dataDirectory [-p DRAFT|CURATION|PUBLISH] -c username:password -r repositoryURL

    Note that all resources will be uploaded in the requested workflow state - we recommend to choose CURATION, verify the resources were ETLd correctly, and then publish using the bulk workflow command (see below). If you've already ran a test ETL in a staging environment, choose PUBLISH directly.

    For replacing existing resources:

    No Format
    ./ETLer.sh -d dataDirectory [-p DRAFT|CURATION|PUBLISH] -c username:password -r repositoryURL -eid property-uri

     

     

     

    Info

    If you are practicing the ETL process, you may wish to upload your data to the common eagle-i training node. In this case, if your directory is named dataDirectory, the script would be executed as follows (default workflow state is DRAFT):

    No Format
    ./ETLer.sh -d dataDirectory  -c L4:Level4 -r https://training.eagle-i.net

    Note that the data that is uploaded to the training node CAN be viewed and modified by others even in a draft state (even if you subsequently lock the records). Note also that the information in the training node is not persistent as the node is refreshed periodically.

  •  

  •  

  1. A detailed report of the ETL results is generated in the ./logs directory; please inspect it to verify that all rows were correctly uploaded. The RDF version of generated resources is also logged in this directory.
  2. To further verify the data upload, log on to the SWEET application and select the lab to which the ETLd resources belong.

...