More predictable timeout-notify with slow sync calls #2625
Conversation
|
Can you reproduce a real example with this actually being a problem? Will this amount of calls be a problem, performance-wise? |
|
Sorry for the delayed response! I can give you an example, how real it is is a matter of definition. pyproject.tomlgunicorn.conf.pyapp.pyimport time
from fastapi import FastAPI
app = FastAPI()
@app.get("/slow_sync")
async def slow_sync():
time.sleep(1)
return "slept sync"
@app.get("/fast_async")
async def fast_async():
return "slept async"demo.shRunning Performance-wise, the effect should be negligible, it'll be 9 extra calls to |
Summary
The timeout-notify mechanism is less predictable than necessary when slow synchronous calls are run on the event loop thread. This can lead to uvicorn workers being killed by (for example) gunicorn while they're still responsive.
The uvicorn servers
main_looptriggerson_tickand sleeps asynchronously for 0.1 seconds. Every 10 tickson_tickchecks if it's time to runcfg.callback_notify. The comments in the code imply that the check is supposed to run once every second, but if there are synchronous calls blocking the asynchronous sleeps from triggering in a timely manner, the actual cadence can be a lot slower.Of course, you're not supposed to block the event loop like that, but checking the need to notify on each tick (every 0.1+ seconds) improves the situation at a negligible cost.
Checklist