feat: add experimental Gemini Omni video provider (-p omni)#268
Merged
Conversation
Add an opt-in, experimental client for Google's Gemini Omni preview video model (`gemini-omni-flash-preview`). Omni uses the new stateful `POST /v1beta/interactions` endpoint (not Veo's `:predictLongRunning`) with a `generation_config.video_config.task` block, and reuses the existing GOOGLE_API_KEY — no new credential. - New `OmniProvider` in ai-providers (defensive parse of the preview interactions schema), registered via `defineProvider` with NO resolverPriority so it sorts last and is never auto-selected — `-p omni` only. - Wire `omni` into both video paths: the `vibe generate video` CLI command and the `executeVideoGenerate` executor (pipeline / scene-build / MCP tool). - Expose `omni` in the generate_video agent/MCP schema, flagged experimental. - Document the model id, endpoint, tasks, and preview/EEA/audio-ref limits in MODELS.md (replaces the old "watchlist" note). - Update the provider-surface guard tests + CLI reference to include omni. Experimental and opt-in: the stable video path (seedance/veo/kling/runway/grok) is unchanged.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add an opt-in, experimental client for Google's Gemini Omni preview video
model (
gemini-omni-flash-preview). Omni uses the new statefulPOST /v1beta/interactionsendpoint (not Veo's:predictLongRunning) with ageneration_config.video_config.taskblock, and reuses the existingGOOGLE_API_KEY — no new credential.
OmniProviderin ai-providers (defensive parse of the previewinteractions schema), registered via
defineProviderwith NO resolverPriorityso it sorts last and is never auto-selected —
-p omnionly.omniinto both video paths: thevibe generate videoCLI command andthe
executeVideoGenerateexecutor (pipeline / scene-build / MCP tool).omniin the generate_video agent/MCP schema, flagged experimental.MODELS.md (replaces the old "watchlist" note).
Experimental and opt-in: the stable video path (seedance/veo/kling/runway/grok)
is unchanged.