fix: don't block the event loop on synchronous storage calls#35
Merged
Conversation
paradoc-serve runs a single uvicorn worker (one asyncio event loop). Many
route handlers were `async def` but called SYNCHRONOUS DocStore S3 methods
(list_doc_ids, get_static_*, get_bundle_manifest, put_bundle_file, ...)
directly — each blocked the whole event loop for the duration of the S3
round-trip, so one slow request stalled every other request AND the health
probe.
Worst offender: GET /api/health returned doc_store.list_doc_ids() — a sync S3
list on EVERY 15s liveness probe. Under Garage/node slowness it blew the 5s
probe timeout and crashlooped the pod while blocking all traffic. Health is
now trivial ({"status": "ok"}) — it must never touch external I/O.
For the doc-serving endpoints: the delegating handlers that only call sync
helpers are now plain `def`, so FastAPI runs them in its threadpool (async
deps like current_user still resolve on the loop). Handlers that genuinely
await (streaming, DB, request.body) stay `async def` and offload their sync
S3 work via starlette run_in_threadpool (get_landing fan-out, bundle PUT).
Net: the event loop stays responsive under concurrent load and slow S3; a
single in-flight doc fetch no longer freezes the server or trips liveness.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Krande
changed the title
|
👋 Hi there! I have checked your PR and found the following: PR Review:I found no pr-related issues.
Python Review:I found some python-related issues: Python Linting results:
Python Packaging results: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
paradoc-serveruns a single uvicorn worker — one asyncio event loop. Manyroute handlers are declared
async defbut call synchronousDocStoremethods (
list_doc_ids,get_static_*,get_bundle_manifest,put_bundle_file, …) directly. Each of those does blocking object-store I/O, soit blocks the entire event loop for the duration of the round-trip: one slow
request stalls every other request, and the health endpoint stops responding.
The worst offender was
GET /api/health, which returneddoc_store.list_doc_ids()— a synchronous object-store listing on everyhealth check. Under slow storage that can exceed the health-check timeout and
get the process restarted, all while blocking live traffic.
Fix
/api/healthis now trivial ({"status": "ok"}). A health check mustprove the process/event loop is alive and never touch external I/O.
def,so the framework runs them in its threadpool (async dependencies such as the
auth dependency still resolve on the loop). Only handlers that genuinely
awaitstayasync def.async defhandlers that do synchronous storage work offload itvia
starlette.concurrency.run_in_threadpool(the landing aggregator'sper-scope fan-out, and the bundle-file upload PUT).
Result
The event loop stays responsive under concurrent load and slow storage; a single
in-flight fetch can no longer freeze the server or trip the health check.
Notes
fix.
blocked worker can't monopolise a request slot.