feat(google): support gemini-3.5-live-translate-preview via translation_config#6358
feat(google): support gemini-3.5-live-translate-preview via translation_config#6358sahal200 wants to merge 2 commits into
Conversation
|
|
83e1239 to
110f7c3
Compare
There was a problem hiding this comment.
π Initial chat context is still sent to translate models on session connect
When mutable_chat_context is False (as it is for translate models), _build_connect_config sets history_config=HistoryConfig(initial_history_in_client_content=True) (realtime_api.py:1162-1164). The _main_task then unconditionally sends any stored chat context via send_client_content (realtime_api.py:908-912). For translate models that are audio-to-audio only, sending text-based chat history could cause an API error if the chat context is non-empty. In practice this is unlikely since the framework checks mutable_chat_context before calling update_chat_ctx in most paths, and a fresh session starts with an empty context. This follows the same pre-existing pattern as 3.1 models, but it's worth verifying that the translate API gracefully handles an empty initial_history_in_client_content config.
(Refers to lines 1162-1164)
Was this helpful? React with π or π to provide feedback.
There was a problem hiding this comment.
π Translate models still accumulate local chat history from generations
When a generation completes, _mark_current_generation_done (realtime_api.py:1336-1384) unconditionally appends input transcription and output text to self._chat_ctx. For translate models, this means the local chat context grows over time even though it's never sent to the API (since mutable_chat_context=False prevents mid-session updates). On session reconnect, this accumulated context would be sent via send_client_content in _main_task (realtime_api.py:908-912), which could cause issues for a translate-only model. This is the same behavior as 3.1 models and is unlikely to cause problems in practice since translate sessions are typically short-lived, but it's worth being aware of.
(Refers to lines 1336-1384)
Was this helpful? React with π or π to provide feedback.
Summary
Google launched Gemini 3.5 Live Translate (
gemini-3.5-live-translate-preview), a low-latency speech-to-speech translation model on the Live API. Using it requires settingtranslation_configonLiveConnectConfig, which the Google plugin'sRealtimeModelpreviously had no way to pass.This PR:
translation_config: NotGivenOr[types.TranslationConfig]parameter toRealtimeModel(threaded through_RealtimeOptionsinto_build_connect_config(), following the same pattern asrealtime_input_config/context_window_compression)gemini-3.5-live-translate-previewto theLiveAPIModelsLiteral and toKNOWN_GEMINI_API_MODELS(it is a Gemini API model, so the existing model/API mismatch validation applies)The required
types.TranslationConfigandLiveConnectConfig.translation_configare available ingoogle-genai2.8.0 (the version currently locked inuv.lock; the plugin requiresgoogle-genai >= 1.67).Usage
Input language is auto-detected; audio is 16kHz PCM in / 24kHz PCM out, matching what the plugin already sends/expects.
Testing
uv run ruff format/uv run ruff checkβ cleanuv run python scripts/check_types.py(mypy strict) β no issuesuv run pytest --unit -k googleβ 34 passedRealtimeModel(model="gemini-3.5-live-translate-preview", translation_config=types.TranslationConfig(target_language_code="es-US", echo_target_language=True))constructs, and_build_connect_config()produces aLiveConnectConfigcarrying the translation configNote: I don't currently have access to the preview model, so this is not verified against the live API end-to-end β the change is config plumbing only and existing behavior is unchanged when
translation_configis not given.