Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deploy OGC-API process using CWL payload #443

Merged
merged 18 commits into from
Jul 5, 2022
Merged

Deploy OGC-API process using CWL payload #443

merged 18 commits into from
Jul 5, 2022

Conversation

fmigneault
Copy link
Collaborator

@fmigneault fmigneault commented Jun 9, 2022

This PR provides important support of CWL directly specified as request content during process deployment (instead of nested under executionUnit), aligning with OGC Best Practices document.

Changes

  • Add support of official CWL IANA types to allow Process deployment with the relevant Content-Type header
    for the submitted payload.
  • Support Process deployment using only CWL content provided it contains an id field representing the target
    Process ID as per recommendation in OGC Best Practice for Earth Observation Application Package, CWL Document <https://docs.ogc.org/bp/20-089r1.html#toc26>.
  • Support Process deployment with a payload using YAML content instead of JSON. This YAML content
    MUST be submitted in the request with a Content-Type header either equal to application/x-yaml or
    application/ogcapppkg+yaml for the OGC Application Package <https://github.com/opengeospatial/ogcapi-processes/blob/master/extensions/deploy_replace_undeploy/standard/openapi/schemas/ogcapppkg.yaml> schema, or using application/cwl+yaml for a CWL-only definition. The definition will be loaded and converted to JSON for schema validation. Otherwise, JSON contents is assumed to be directly provided in the request payload for validation as previously accomplished.
  • Add weaver.cwl_processes_dir configuration setting for preloading, registering or updating a set of
    known Process definitions from CWL files stored in a nested directory structure. This allows a service provider
    that uses Weaver to offer their Processes to directly maintain their definitions from the set of CWL files and
    upload changes in the web application at startup without need to manually undeploy and redeploy each Process.

Fixes

  • Fix Process deployment using a WPS-1/2 URL reference defining a GetCapabilities request to resolve
    the corresponding DescribeProcess request if the Process ID can be inferred from other known locations
    (some items checked off from functional tests for process deployment #11).

References

@fmigneault fmigneault added the feature/CWL Issue related to CWL support label Jun 9, 2022
@fmigneault fmigneault self-assigned this Jun 9, 2022
@fmigneault fmigneault changed the title update to docs about existing (though undocumented) features Deploy OGC-API process using CWL payload Jun 9, 2022
@github-actions github-actions bot added ci/doc Issue related to documentation of the package ci/tests Tests of the package and features feature/db Related to database or datatype manipulation. feature/job Issues related to job execution, reporting and logging. feature/oas Issues related to OpenAPI specifications. feature/providers Issue related to providers convertion to WPS-REST processes. feature/vault Issue related to Vault storage feature. process/builtin Issue related to builtin application processes process/wps3 Issue related to WPS 3.x (REST-JSON) processes support labels Jun 9, 2022
@fmigneault fmigneault marked this pull request as ready for review June 9, 2022 23:57
@github-actions github-actions bot added the feature/opensearch Issue related to OpenSearch functionalities. label Jun 10, 2022
@codecov
Copy link

codecov bot commented Jun 10, 2022

Codecov Report

Merging #443 (8a61134) into master (2f04613) will increase coverage by 0.13%.
The diff coverage is 93.20%.

@@            Coverage Diff             @@
##           master     #443      +/-   ##
==========================================
+ Coverage   81.50%   81.64%   +0.13%     
==========================================
  Files          76       75       -1     
  Lines       14541    14668     +127     
  Branches     3158     3188      +30     
==========================================
+ Hits        11851    11975     +124     
- Misses       1973     1975       +2     
- Partials      717      718       +1     
Impacted Files Coverage Δ
weaver/base.py 100.00% <ø> (ø)
weaver/config.py 80.76% <ø> (ø)
weaver/datatype.py 74.97% <0.00%> (ø)
weaver/exceptions.py 91.96% <ø> (ø)
weaver/processes/builtin/__init__.py 83.33% <ø> (ø)
weaver/processes/convert.py 80.12% <ø> (ø)
weaver/tweens.py 77.94% <ø> (ø)
weaver/wps_restapi/providers/providers.py 80.64% <ø> (ø)
weaver/wps_restapi/quotation/quotes.py 68.36% <ø> (ø)
weaver/processes/wps_package.py 75.57% <87.09%> (+0.57%) ⬆️
... and 17 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 75f6cdd...8a61134. Read the comment docs.

@github-actions github-actions bot added the ci/operations Related to CI operations (actions, execution, install, builds, etc.) label Jun 10, 2022
@fmigneault fmigneault merged commit afd4782 into master Jul 5, 2022
@fmigneault fmigneault deleted the cwl-deploy branch July 5, 2022 14:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci/doc Issue related to documentation of the package ci/operations Related to CI operations (actions, execution, install, builds, etc.) ci/tests Tests of the package and features feature/CWL Issue related to CWL support feature/db Related to database or datatype manipulation. feature/job Issues related to job execution, reporting and logging. feature/oas Issues related to OpenAPI specifications. feature/opensearch Issue related to OpenSearch functionalities. feature/providers Issue related to providers convertion to WPS-REST processes. feature/vault Issue related to Vault storage feature. process/builtin Issue related to builtin application processes process/wps3 Issue related to WPS 3.x (REST-JSON) processes support
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Deploy process directly with CWL content
1 participant