add socket-load-balance flag #2472
Conversation
uvicorn/_subprocess.py
Outdated
| def get(self) -> socket.socket: | ||
| return self._sock | ||
|
|
||
| if (sys.platform == "linux" and hasattr(socket, "SO_REUSEPORT")) or hasattr(socket, "SO_REUSEPORT_LB"): |
There was a problem hiding this comment.
Needs python/cpython#124761
graingert
force-pushed
the
socket-load-balance
branch
from
c873d00 to
3e2854a
Compare
|
There's a tradeoff where the default unfair load balancing favours lower latency and the SO_REUSEPORT(_LB) approach favours an even load between processes I suspect most people do want the fair load balancing, however I think it should bed in a bit before we suddenly turn it on by default for everyone. |
| started = m.Event() | ||
| d = m.dict() | ||
| app = functools.partial(lb_app, d, started) | ||
| config = Config(app=app, workers=2, socket_load_balance=True, port=0, interface="asgi3") |
There was a problem hiding this comment.
this needs a test for two independent calls to Multiprocess(...).run() running on the same port:
def test_bind_to_used_port():
port = ephemeral_port_reserve.reserve()
config = Config(app=app, workers=2, socket_load_balance=True, port=port, interface="asgi3")
with multiprocessing.get_context("spawn").Pool(max_workers=1) as pool:
try:
f1 = pool.apply_async(uvicorn.main, ...)
# wait for started
with pytest.raises(OSError, match="already bound"):
uvicorn.main(...)
finally: pool.terminate()
|
|
||
| class SocketShareRebind: | ||
| def __init__(self, sock: socket.socket): | ||
| if not (sys.platform == "linux" and hasattr(socket, "SO_REUSEPORT")) or hasattr(socket, "SO_REUSEPORT_LB"): |
There was a problem hiding this comment.
Wouldn't it be better to put the logic here in config.bind_socket?
Just add it to the else, uds and fd don't seem to support SO_REUSEPORT.
There was a problem hiding this comment.
hmm I'm not sure about this, I want to make sure the SO_REUSEPORT flag isn't set before the socket is bound so we still fail if another uvicorn is using this port
There was a problem hiding this comment.
How about this?
vvanglro@f2b1037
|
@graingert , do you have any updates on this work? It would be great to support socket LB for a simpler deployment that can leverage multiple CPU cores more effectively without a dependency on another process manager. Is there any blockers? How can I help? Cheers. |
|
It's feature complete, it just need a decision on how coverage for subprocesses will be managed, eg pytest-cov? |
|
When we use File "/python3.13/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/python3.13/site-packages/uvicorn/_subprocess.py", line 117, in subprocess_started
target(sockets=[s.get() for s in sockets])
~~~~~^^
File "/python3.13/site-packages/uvicorn/_subprocess.py", line 46, in get
sock.bind(self._sockname)
~~~~~~~~~^^^^^^^^^^^^^^^^
OSError: [Errno 98] Address already in use |
Summary
Fixes #2467
Checklist