refactor(llm): add AI SDK, reroute Gemini calls to it#165
Open
LoneRifle wants to merge 3 commits into
Open
Conversation
LoneRifle
force-pushed
the
refactor/ai-sdk/google
branch
3 times, most recently
from
54d5922 to
e67cd29
Compare
Provide the plumbing needed to interact with providers through Vercel's AI SDK. Use it to direct Gemini-driven `completeText()` calls (mostly invoked for titles and summaries) - Define `MODEL_PROVIDER_CONFIGS`, each Provider mapping to: - a AI SDK ProviderV3 factory function - the Provider default API key, or an empty str if not set - the env var name for the key, for error messages - Implement `resolveModel()`, using the above - Replace the `gemini` branch of `completeText()` with the fallthrough to `resolveModel()` and `generateText()`
LoneRifle
force-pushed
the
refactor/ai-sdk/google
branch
10 times, most recently
from
f1c5121 to
e1f172c
Compare
Provide an implementation for stream chat with tools using AI SDK, wiring Gemini calls to this. Take reference from the other LLM integrations and faithfully reproduce impl as close as possible. To discuss - at the end, all models would append responses to what effectively are transformed copies of `params.messages`, although the intent of that is uncertain. - Map `streamChatWithTools()` to `streamText()` - Wire all relevant events from the stream to callbacks - Collate full text response as well as all tool calls - Append `assistant` and `tool` messages at the end
LoneRifle
force-pushed
the
refactor/ai-sdk/google
branch
from
e1f172c to
dc7f773
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Provide the plumbing needed to interact with providers through Vercel's
AI SDK. Use it to direct Gemini-driven
completeText()calls(mostly invoked for titles and summaries)
Provide an implementation for stream chat with tools using AI SDK,
wiring Gemini calls to this. Take reference from the other LLM
integrations and faithfully reproduce impl as close as possible.
Define
MODEL_PROVIDER_CONFIGS, each Provider mapping to:Implement
resolveModel(), using the aboveReplace the
geminibranch ofcompleteText()with the fallthroughto
resolveModel()andgenerateText()Map
streamChatWithTools()tostreamText()assistantandtoolmessages at the endPart 1/3 for #160