Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release/1.15.0 #154

Merged
merged 30 commits into from
Feb 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
7605599
initial implementaiton of subset code
mike-gangl Mar 16, 2023
c69ed98
added tests, more updates
mike-gangl Mar 22, 2023
65737b9
removed skip_errors (not implemented yet)
mike-gangl Mar 30, 2023
ba90bd1
Upgrade to harmony-py 0.4.9
skorper Aug 4, 2023
13b9ece
subsetting capability initial commit
skorper Aug 4, 2023
658ba05
update spacing
skorper Aug 4, 2023
c27363c
Merge remote-tracking branch 'origin' into issues/36
skorper Aug 4, 2023
364dc40
Update docs
skorper Aug 7, 2023
0b30966
update poetry lock
skorper Aug 18, 2023
c9513c5
upgrade to poetry 1.5.1 (latest)
skorper Aug 18, 2023
3decb92
fix failing checksum test
skorper Aug 18, 2023
285970b
fixed regression test issues
skorper Aug 18, 2023
d5dccea
bump poetry version in second gh action file
skorper Aug 21, 2023
ceceae8
Update poetry version to fix build issue
skorper Aug 21, 2023
6319250
bump poetry version
skorper Aug 22, 2023
12ca7b4
bump poetry version
skorper Aug 22, 2023
2e4565b
Merge pull request #142 from podaac/issues/36
skorper Aug 22, 2023
cd217eb
bump version to 1.15.0
skorper Aug 22, 2023
7c12601
bump version to 1.15.0
skorper Aug 22, 2023
cb849d4
Fixed bug introduced in issues/36 where valid bboxes were raising an …
skorper Aug 24, 2023
0119cfc
Merge pull request #149 from podaac/issue/148
skorper Aug 24, 2023
9486bcd
bump version to 1.15.0-a.2
skorper Aug 24, 2023
f77f689
Add error if both cycle and subset are provided
skorper Sep 14, 2023
12b71ae
If verbose, output Harmony job details. Also allow temporal and spati…
skorper Sep 14, 2023
904950b
Added log message to indicate harmony job is submitted and waiting
skorper Sep 14, 2023
f6b2bfe
fixed cycle check typo
skorper Sep 14, 2023
07e6e29
dry run + subset
skorper Sep 14, 2023
dc81efa
idl check
skorper Sep 16, 2023
9c07563
bump version to a3
skorper Sep 16, 2023
7f01419
fix failing unit test to correct value returned by CMR (#155)
skorper Feb 3, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/python-app.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ jobs:
max-parallel: 2
matrix:
python-version: [ "3.7", "3.8", "3.9", "3.10" ]
poetry-version: [ "1.1.14" ]
poetry-version: [ "1.5.1" ]
os: [ ubuntu-22.04, macos-latest, windows-latest ]
runs-on: ${{ matrix.os }}
steps:
Expand Down Expand Up @@ -63,7 +63,7 @@ jobs:
max-parallel: 1
matrix:
python-version: [ "3.7", "3.8", "3.9", "3.10" ]
poetry-version: [ "1.1.14" ]
poetry-version: [ "1.5.1" ]
os: [ ubuntu-22.04, macos-latest, windows-latest ]
runs-on: ${{ matrix.os }}
steps:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:
- name: Install Poetry
uses: abatilo/[email protected]
with:
poetry-version: 1.1.14
poetry-version: '1.5.1'
- name: Install dependencies
run: |
poetry install
Expand Down
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,10 @@ All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)

## [1.15.0]
### Added
- Added support for Harmony subsetting

## [1.14.0]
### Added
- Added support for wildcard search patterns in podaac-data-downloader when executed with the -gr option (i.e. search/download by CMR Granule Ur/Id). Also, added usage details to Downloader.md to describe this new feature [138](https://github.com/podaac/data-subscriber/pull/138).
Expand Down
3 changes: 3 additions & 0 deletions Downloader.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ optional arguments:
Specify a provider for collection search. Default is POCLOUD.
--limit LIMIT Integer limit for number of granules to download. Useful in testing. Defaults to no limit.
--dry-run Search and identify files to download, but do not actually download them
--subset Flag to enable subsetting on the specified collection

```

Expand Down Expand Up @@ -236,7 +237,9 @@ and

Using the `--process` option, you can run a simple command agaisnt the "just" downloaded file. This will take the format of "<command> <path/to/file>". This means you can run a command like `--process gzip` to gzip all downloaded files. We do not support more advanced processes at this time (piping, running a process on a directory, etc).

### Granule subsetting

To enable granule subsetting, include the `--subset` flag in your request. This will invoke the NASA Harmony API to subset the granules in the specified collection. The collection must have subsetting enabled for this feature to function. If it does not, the data will be downloaded normally.

### In need of Help?
The PO.DAAC User Services Office is the primary point of contact for answering your questions concerning data and information held by the PO.DAAC. User Services staff members are knowledgeable about both the data ordering system and the data products themselves. We answer questions about data, route requests to other DAACs, and direct questions we cannot answer to the appropriate information source.
Expand Down
4 changes: 4 additions & 0 deletions Subscriber.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ $> podaac-data-subscriber -h
-p PROVIDER, --provider PROVIDER
Specify a provider for collection search. Default is POCLOUD.
--dry-run Search and identify files to download, but do not actually download them
--subset Flag to enable subsetting on the specified collection
```

## Run the Script
Expand Down Expand Up @@ -212,6 +213,9 @@ and

Using the `--process` option, you can run a simple command agaisnt the "just" downloaded file. This will take the format of "<command> <path/to/file>". This means you can run a command like `--process gzip` to gzip all downloaded files. We do not support more advanced processes at this time (piping, running a process on a directory, etc).

### Granule subsetting

To enable granule subsetting, include the `--subset` flag in your request. This will invoke the NASA Harmony API to subset the granules in the specified collection. The collection must have subsetting enabled for this feature to function. If it does not, the data will be downloaded normally.

### In need of Help?
The PO.DAAC User Services Office is the primary point of contact for answering your questions concerning data and information held by the PO.DAAC. User Services staff members are knowledgeable about both the data ordering system and the data products themselves. We answer questions about data, route requests to other DAACs, and direct questions we cannot answer to the appropriate information source.
Expand Down
577 changes: 332 additions & 245 deletions poetry.lock

Large diffs are not rendered by default.

3 changes: 2 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[tool.poetry]
name = "podaac-data-subscriber"
version = "1.14.0"
version = "1.15.0-alpha.3"
description = "PO.DAAC Data Subscriber Command Line Tool"
authors = ["PO.DAAC <[email protected]>"]
readme = "README.md"
Expand All @@ -16,6 +16,7 @@ python = "^3.7"
requests = "^2.27.1"
tenacity = "^8.0.1"
packaging = "^23.0"
harmony-py = "^0.4.9"

[tool.poetry.dev-dependencies]
pytest = "^7.1.2"
Expand Down
Loading
Loading