Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tracking "Connection reset by peer" issue with Copernicus zipper API #30

Open
sharkinsspatial opened this issue Nov 6, 2023 · 0 comments
Assignees
Labels
bug Something isn't working

Comments

@sharkinsspatial
Copy link
Collaborator

In an attempt to bypass the continual issues with incorrect MD5 checksums #29, I deployed an updated version of the downloader with checksum validation disabled. But we immediately began to see high rates of errors for

[ERROR] ProtocolError: ("Connection broken: ConnectionResetError(104, 'Connection reset by peer')", ConnectionResetError(104, 'Connection reset by peer'))
Traceback (most recent call last):
  File "/var/task/handler.py", line 74, in handler
    download_file(
  File "/var/task/handler.py", line 213, in download_file
    Body=response.raw.read(),
  File "/var/task/urllib3/response.py", line 593, in read
    raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
  File "/var/lang/lib/python3.8/contextlib.py", line 131, in __exit__
    self.gen.throw(type, value, traceback)
  File "/var/task/urllib3/response.py", line 461, in _error_catcher
    raise ProtocolError("Connection broken: %r" % e, e)

I submitted the following issue to the Copernicus support system

We are seeing high rates of our downloads being interrupted with "Connection reset by peer" errors indicating TCP interruptions by the ESA Copernicus zipper API. Our requests are concurrency limited so they do not exceed the 15 connections we have been allotted.

ref https://github.com/NASA-IMPACT/hls_development/issues/148

@sharkinsspatial sharkinsspatial added the bug Something isn't working label Nov 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants