Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SDSS caches files despite cache=False #3121

Open
JohannesBuchner opened this issue Oct 19, 2024 · 1 comment · May be fixed by #3123
Open

SDSS caches files despite cache=False #3121

JohannesBuchner opened this issue Oct 19, 2024 · 1 comment · May be fixed by #3123

Comments

@JohannesBuchner
Copy link

JohannesBuchner commented Oct 19, 2024

On request by @keflavich, in #1239 (comment) I am report a bug.

I expected `SDSS.get_spectra(..., cache=False)' to not cache on disk.

But it does:

$ rm -rf ~/.astropy/cache/download/url/
$ python3 -c 'from astroquery.sdss import SDSS; SDSS.get_spectra(plate=827, fiberID=23, mjd=52312, cache=False)'
Downloading https://data.sdss.org/sas/dr17/sdss/spectro/redux/26/spectra/0827/spec-0827-52312-0023.fits
|===================================================================================================================================================================| 892k/892k (100.00%)         0s
$ ls ~/.astropy/cache/download/url/
0ad9ac9e9a4231884297d457accb4027

Also setting SDSS.cache_location does not help

$ rm -rf ~/.astropy/cache/download/url/
$ python3 -c 'from astroquery.sdss import SDSS; SDSS.cache_location = "/tmp/"; SDSS.get_spectra(plate=827, fiberID=23, mjd=52312, cache=False)'
Downloading https://data.sdss.org/sas/dr17/sdss/spectro/redux/26/spectra/0827/spec-0827-52312-0023.fits
|===================================================================================================================================================================| 892k/892k (100.00%)         0s
$ ls ~/.astropy/cache/download/url/
0ad9ac9e9a4231884297d457accb4027

version:

$ python3 -c 'import astroquery; print(astroquery.__version__)'
0.4.7
@keflavich
Copy link
Contributor

keflavich commented Oct 19, 2024

Thanks @JohannesBuchner , this clarifies the issue. The cache keyword simply isn't being passed, and on top of that, astropy's default file download caching system is being used rather than astroquery's, so the cached files are being put in the wrong place.

I think I can solve this pretty quickly, first by passing the cache kwarg here:

results.append(commons.FileContainer(link,

and then, possibly, by modifying FileContainer to allow a module-specified path:

target = aud.download_file(target_key, cache=True, sources=[])

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants