[Errno 9] Bad file descriptor when run multi workers with multiprocessing module

[jmkim]

Hi,

I am having an issue when I use Uvicorn with Python multiprocessing module. When I set the number of worker to multiple (2+), then [Errno 9] Bad file descriptor error has been occur.

Issue

Let's assume that run this simple sample app using uvicorn:

• Code 1: sample app

  async def sample_app(scope, receive, send):
      await send({"type": "http.response.start", "status": 204})
      await send({"type": "http.response.body", "body": b""})

This app works well with Code 2 when I use uvicorn solely:

• Code 2: uvicorn (working)

  import uvicorn
  
  if __name__ == "__main__":
      uvicorn.run(app="demo:sample_app", host="localhost", port=8000, workers=2)

Now I rewrote Code 2 to Code 3. They have same semantics. The difference is it has multiprocessing.

• Code 3: uvicorn + multiprocessing (not working)

  import uvicorn
  import multiprocessing
  
  if __name__ == '__main__':
      proc = multiprocessing.Process(
          target=uvicorn.run,
          kwargs={
              "app":      "demo:sample_app",
              "host":     "localhost",
              "port":     8000,
              "workers":  2
          }
      )
      proc.start()

Code 3 works well with single worker.
However, it gets failure when the number of workers goes higher (2+).

• Error message shown by Code 3:

  INFO:     Uvicorn running on http://localhost:8000 (Press CTRL+C to quit)
  INFO:     Started parent process [96305]
  
  Process SpawnProcess-1:1:
  Traceback (most recent call last):
    File "/Users/jmkim/miniforge3/envs/do-nand/lib/python3.10/multiprocessing/process.py", line 315, in _bootstrap
      self.run()
    File "/Users/jmkim/miniforge3/envs/do-nand/lib/python3.10/multiprocessing/process.py", line 108, in run
      self._target(*self._args, **self._kwargs)
    File "/Users/jmkim/miniforge3/envs/do-nand/lib/python3.10/site-packages/uvicorn/subprocess.py", line 70, in subprocess_started
      sys.stdin = os.fdopen(stdin_fileno)
    File "/Users/jmkim/miniforge3/envs/do-nand/lib/python3.10/os.py", line 1029, in fdopen
      return io.open(fd, mode, buffering, encoding, *args, **kwargs)
  OSError: [Errno 9] Bad file descriptor
  
  Process SpawnProcess-1:2:
  Traceback (most recent call last):
    File "/Users/jmkim/miniforge3/envs/do-nand/lib/python3.10/multiprocessing/process.py", line 315, in _bootstrap
      self.run()
    File "/Users/jmkim/miniforge3/envs/do-nand/lib/python3.10/multiprocessing/process.py", line 108, in run
      self._target(*self._args, **self._kwargs)
    File "/Users/jmkim/miniforge3/envs/do-nand/lib/python3.10/site-packages/uvicorn/subprocess.py", line 70, in subprocess_started
      sys.stdin = os.fdopen(stdin_fileno)
    File "/Users/jmkim/miniforge3/envs/do-nand/lib/python3.10/os.py", line 1029, in fdopen
      return io.open(fd, mode, buffering, encoding, *args, **kwargs)
  OSError: [Errno 9] Bad file descriptor
  

Diagnostic snippets

• List of installed packages on my machine: System & Python (GitHub Gist)

[jmkim]

uvicorn/uvicorn/_subprocess.py

Lines 67 to 71 in 6cf6aba

	"""
	# Re-open stdin.
	if stdin_fileno is not None:
	sys.stdin = os.fdopen(stdin_fileno)

Skipping the line 70 in uvicorn/_subprocess.py can fix the issue.

This is a temporary fix, I do not know the purpose of that line. Please let me know the effect of this.

[jmkim]

Arbitrary skipping patch:

--- subprocess.py
+++ subprocess.py
@@ -67,7 +67,7 @@
     """
     # Re-open stdin.
     if stdin_fileno is not None:
-        sys.stdin = os.fdopen(stdin_fileno)
+        pass
 
     # Logging needs to be setup again for each child.
     config.configure_logging()

[Kludex]

• Related Fix Bad file descriptor error when using reload or workers flag and multiprocessing #883

[jmkim]

Yes, definitely same issue. I now read the discussions and proposed codes there. Hope it got fixed soon \o/