-
Notifications
You must be signed in to change notification settings - Fork 176
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix cohttp EOF read #1035
base: v5-backports
Are you sure you want to change the base?
Fix cohttp EOF read #1035
Conversation
@@ -76,4 +76,6 @@ module Make (Channel : Mirage_channel.S) = struct | |||
| Read_exn e -> Lwt.return_error (Read_error e) | |||
| Write_exn e -> Lwt.return_error (Write_error e) | |||
| ex -> Lwt.fail ex) | |||
|
|||
let wait_eof_or_closed _conn _ic _sleep_fn = assert false |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here we ask for the advice of the maintainers of the library, as we advocate that this function will never be called from this library, but we want to receive a second opinion.
Ping @rgrinberg and @mefyl who might be able to review. |
Thank you for the detailed report and reproduction case! TLDR: this is IMO more a design choice than a bug. But it's a good design question. There is a preliminary question to all this: is it a good idea to kill the server request handler if the client disconnects? I'm not really sure it is, as one could have some heavy processing server side after sending the response headers, and abruptly killing this processing if the client disconnects before the end of the body is a very opinionated choice. In the test example, the "resource leak" mentioned exists because the handler never ever returns, so the handler is a user-created resource leak in itself. Now I do reckon that your point has merit, and stopping processing of a request when the client disconnects is a desirable feature in some if not most cases - it's just that failing to do so is a sub-optimal while doing so inappropriately is a straight bug. With that in mind I think it would be better to provide the expert request handler with the possibility to monitor client disconnection, and maybe provide some simpler default behavior in the simple interface. Happy to discuss this. |
Thank you for your response!
The new callback should accept
Please let us know what you think, we would be happy to chat about it. We can also showcase this adaptation in a fixup commit, if that is clearer for you. Would you prefer to continue this discussion in mails/comments here, or would it be easier for you to have a call? |
@art-w Do you know how we can move forward on this? |
This work has been done together by the following people:
@savvadia
@vect0r-vicall
@picojulien
@johnyob
@raphael-proust
Motivation
We have discovered that there is a resource leak for particular cases. Here are two possible scenarios:
callback
never resolves,EOF
is never detected, leading to a resource leak.callback
is very slow and resource-intensive,EOF
is not detected until thecallback
resolves. Consequently, client resources are wasted even after the connection has been closed by the peer.Solution
A new
wait_eof_or_closed
function is introduced, which repeatedly peeks (usingMSG_PEEK
) into the data stream for anEOF
, in which case it closes the connection.MSG_PEEK
used for this does not disturb reading from the stream by thehandle_client
. If the connection is closed locally before this happens, the function stops waiting forEOF
.If provided to the
wait_eof_or_closed
, thesleep_fn
argument will be used to dictate the periodicity of the checks forEOF
from the peer, yielding control periodically. Ifsleep_fn
is not provided, the server keeps its legacy behaviour.PR format
The format of the PR is as follows:
[WIP]
commit adds atest_leak.ml
test, which showcases theEOF
leak; the[WIP]
tag highlights that this test is not something that we want to be added necessarily, but we would be happy if you find it desirable or helpful and choose to incorporate itTesting
You can observe the initial leaking behaviour by cherry picking only the first commit:
Now, the server is listening, so you can call on a separate console
curl -s 'localhost:8080/sleep'
and observe that the server keeps printing messages ofI slept
. Now, when you close thecurl
request withCTRL+C
, the server does not stop.To solve this problem, add the second commit as well, and re-do this test, and after stopping the
curl
request, you will see this:So, the leak is now fixed.
One can also monitor the resources used by the executable thanks to a tool such as
lsof
: