Add Boson realtime plugin#6340
Open
helin-mktech wants to merge 5 commits into
Open
Conversation
|
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
…ntime
Addresses the review finding that a single WebSocket failure was
terminal. Instead of extending the forked connection machinery, drop it
(~400 lines) and inherit the parent OpenAI plugin's
_main_task/_run_ws/_reconnect loop, xAI-style; Boson-specific behavior
now lives in small overrides and hooks:
- Reconnection follows the server's documented contract (the voice-chat
server keeps no state across connections): the base _reconnect re-sends
the full session config and replays the local chat-context mirror via
conversation.item.create, with _create_session_update_event /
_create_tools_update_event / _create_update_chat_ctx_events overridden
to emit Boson-shaped payloads. Retry count/backoff come from
conn_options, and a session_reconnected event is emitted on resume.
- _run_ws wraps the base loop only to reclassify closes a reconnect
cannot fix as non-retryable: close code 3000 (invalid API key) and
server-announced session ends (session.idle_timeout,
session.max_duration_reached), which must not be resurrected.
- Boson-only server events are handled off the base's
openai_server_event_received hook instead of a forked dispatch;
_main_task fails pending response futures immediately on terminal
failure instead of leaving them to their own timeouts.
- Stop advertising auto_tool_reply_generation: the server treats
conversation.item.create as a pure insert now, so the framework must
send response.create after tool outputs — previously tool calls
completed client-side but no reply audio ever came back.
- Treat conversation.item.added for a known id as an in-place update:
the server merges consecutive same-role turns into one item and
re-emits added with cumulative content, which the base insert rejected
("already exists"), leaving the mirror with stale content.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The _handle_response_created override closed the current generation and recorded _current_response_id before the base handler could check _discarded_event_ids. A response that timed out or was interrupted before the server created it would therefore cut off the streams of a newer in-progress generation, and a subsequent interrupt() would cancel the stale response id instead of the active one. Check the discard set first and let the base handler cancel/discard the stale response without touching the in-progress generation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
The previous fix stopped closing the active generation when a discarded response's response.created arrived, but the base handler still parked a _DiscardedGeneration marker in the generation slot, so every subsequent delta of the legitimate in-progress response was silently dropped — and plain save/restore would only defer the damage, because the stale response's response.done (which follows the cancel almost immediately) would close whatever generation sits in the slot: neither the slot nor _handle_response_done is response-id aware. When a legitimate generation is streaming, restore it after the base discards the stale response AND remember the stale response id; _handle_response_done ignores terminal events for remembered ids instead of closing the active generation or clearing _current_response_id. With no generation in progress (the common serial case) the base discard marker is kept so the stale response's trailing events are still eaten. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a Boson realtime plugin for LiveKit Agents.
livekit-plugins-bosonwithboson.realtime.RealtimeModel.Notes
["audio"]by default, or["text"]for text-only responses.turn_detection=None.input_audio_transcriptioncontains a non-emptymodel.