livekit-plugins-fishaudio: pooled websocket reuse + prebuffer to fix cold-start crackle#6368
Open
cshape wants to merge 2 commits into
Open
livekit-plugins-fishaudio: pooled websocket reuse + prebuffer to fix cold-start crackle#6368cshape wants to merge 2 commits into
cshape wants to merge 2 commits into
Conversation
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
…cold-start crackle - reuse/pre-warm the /v1/tts/live websocket via utils.ConnectionPool (like cartesia/rime) instead of opening a fresh WSS per utterance, so replies skip the ~330ms TLS+WS handshake after the first. - prebuffer the first two audio chunks before starting playout, so webrtc's real-time playout doesn't underrun in the ~250ms gap after fish's small first chunk (the cold-start crackle). internal constant _PREBUFFER_CHUNKS=2, a stopgap until fish inference streams the opening chunks smoothly (then -> 1).
cshape
force-pushed
the
cale/fishaudio-pool-prebuffer
branch
from
4a2b264 to
c5a9a0b
Compare
model is sent as a ws connection header (not the per-request body), so a pooled socket keeps the old model after update_options(model=...). invalidate the pool on model change so the next stream reconnects with the new model. other options ride in the request body and need no reconnect. addresses PR review feedback.
cshape
added a commit
to cshape/agents-expressive
that referenced
this pull request
Jul 9, 2026
…ol on model change - prebuffer_chunks is now an internal constant (_PREBUFFER_CHUNKS=2), not a public option, matching the upstream PR (livekit#6368). - shorten the pool/prebuffer comments. - invalidate the connection pool when update_options(model=...) changes the model, since the model rides as a ws connection header (other opts are per-request body).
tinalenguyen
approved these changes
Jul 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
two changes to the fish plugin:
reuse/pre-warm the /v1/tts/live websocket via utils.ConnectionPool (same
pattern as cartesia/rime). the plugin used to open a fresh WSS per utterance,
paying ~330ms of TLS+WS handshake on every reply. now one socket is reused per
session and pre-warmed via prewarm(), so the first reply skips the handshake.
general latency win, independent of the crackle fix.
prebuffer the first two audio chunks before starting playout. fish streams a
small first chunk (~460ms) then, ~250ms later, a much larger second chunk.
starting webrtc's real-time playout at the first chunk leaves only ~190ms of
buffer headroom; app/network jitter tips it into an underrun the receiver
conceals as an audible click/gap ~0.5s into the first utterance (repro'd ~1 in
5). waiting for the second chunk starts playout with ~1.3s buffered and
eliminates it.
the prebuffer is an internal constant (_PREBUFFER_CHUNKS = 2), not a public
option. it costs a little TTFB and is a stopgap for fish's cold-start chunk
pacing; once the inference side streams the opening chunks smoothly we'll drop it
to 1 to start on the first chunk. working with the fish inference team on that now.
verified no crackles on livekit-demo-web.onrender.com; chunk-2 headroom measured
before/after and the negative tail is gone.