Example application illustrating structure for Unity App Generator application repositories
unity-app-generator | app-pack-generator
- Demonstrates Unity application structure
- Demonstrates Unity Jupyter notebook parameterization
- Demonstrates use of Unity-py for interacting with STAC files
- Python 3.9
- Jupyter
- Papermill
Install required Python packages:
pip install -r requirements.txt
The process.ipynb notebook file is designed to work either as an independent notebook or as part of an OGC application package. It is meant for illustrative purposes and doesn't do any "real" work. What it does is show the parameterization of application arguments using Papermill and the use of Unity stage-in and stage-out mechanisms. The actions taken in the notebok are simply to extract some metadata from a set of netCDF files and place them into a textual table that is written out as well as displayed in the notebook.
Unity OGC applications rely upon using Papermill parameritzation of arguments. One of the cells is tagged with the parameters tag, indicating to Papermill which cell to inspect for insertion of values from the command line. See the app-pack-generator for more information on the formatting of parameters and the use of type hints.
to run this without the stage-in or out parameters, simply call the process.cwl contained in this repo:
# run stage in
cwltool --outdir stage_in --copy-output stage_in.cwl test/ogc_app_package/stage_in.yml
#For my run, it ended up in a directory called z7ai3uj8
# For now, the current stage_in creates an invalid 'root' in catalog.json. you'll need to set the root from /<someting>/catalog.json to catalog.json
vim z7ai3uj8/catalog.json
# Pass that as the directory into the process.py runs
cwltool process.cwl --example_argument_empty '' --input z7ai3uj8/ --output_collection <mycollection>
#The output files and catalog.json from this run were stored in 02kajdto
#run stageout
cwltool stage_out.cwl --output_dir 02kajdto/ --staging_bucket <mybucket>
This notebook is connected to a Unity stage-in process through the input_stac_collection_file variable. This variable contains the location of a STAC feature collection file. That feature collection points to the input files used by the notebook. In our example notebook we use Unity-py to parse the file and obtain the full paths to the input files.
In the example notebook output_stac_catalog_dir is the variable where the directory where a STAC catalog listing the output files should be placed. In the example we use Unity-py to create the output STAC catalog. The example gives the minimal amount of metadata necessary for writing a valid catalog file and corresponding item collection files.
The test/ directory contains some useful scripts for testing the example notebook with unity-app-generator. It requires that package to be set up in your environment.
The test/ogc_app/build_app.sh script will use unity-app-generator to build a Docker image and CWL files from the repository.
The test/ogc_app/test_cwl.sh script can be used to test execution of the resulting CWL files. It requres that you first copy the cwl_job_input.yml.template file to be named cwl_job_input.yml in the same directory where it is located. Fill out the edl_username and edl_password values with base 64 encoded versions of your Earthdata Login credentials.
See our CHANGELOG.md for a history of our changes.
See our releases page for our key versioned releases.
See our: LICENSE