feat: smart channel routing based on request API type#5017
Open
xemxx wants to merge 4 commits into
Open
Conversation
added 3 commits
May 21, 2026 15:29
When multiple channels share the same priority for a model, prefer channels whose native API type matches the client's request format (OpenAI/Claude/Gemini). This avoids unnecessary request/response conversion and improves relay performance. Changes: - Add RelayFormatToAPIType() mapping in types/relay_format.go - Pass RelayFormat through RetryParam to channel selection - Smart routing in model/ability.go (DB query path, memory cache disabled) - Smart routing in model/channel_cache.go (memory cache enabled) - Default to OpenAI format in middleware/distributor.go (pre-detection phase) Fixes QuantumNous#4982
When the smart channel routing query joins abilities with channels, both tables expose 'group' and 'weight' columns, causing SQLite to fail with 'ambiguous column name' errors. - Qualify the group filter with abilities. prefix in JOINed queries - Wrap the priority subquery in parentheses so the generated SQL is valid (priority = (SELECT MAX(priority) ...) instead of priority = SELECT ...) - Order by abilities.weight DESC explicitly, and select abilities.* on the filtered query to avoid column collision when scanning rows
The Distribute middleware runs before the per-route handler that sets the actual RelayFormat, so smart channel routing was always seeing RelayFormatOpenAI and could never prefer a Claude- or Gemini-native channel for /v1/messages or Gemini paths. Add InferRelayFormatFromPath, which mirrors the routing table in router/relay-router.go, and use it in the distributor so requests hitting /v1/messages get RelayFormatClaude (and so on) before channel selection runs.
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
WalkthroughThis PR infers a request's relay format from its HTTP path, threads that format through service RetryParam into model selection, and uses it to prefer channels whose native API type matches the inferred relay format during channel selection. ChangesRelay Format-Based Smart Channel Routing
Sequence DiagramsequenceDiagram
participant Client
participant Middleware as middleware/distributor
participant Service as service/channel_select
participant Model as model/ability & model/channel_cache
participant DB as Database
Client->>Middleware: HTTP request (path)
Middleware->>Middleware: InferRelayFormatFromPath(path)
Middleware->>Service: CacheGetRandomSatisfiedChannel(retryParam{RelayFormat})
Service->>Model: GetRandomSatisfiedChannel(relayFormat)
Model->>Model: RelayFormatToAPIType(relayFormat)
Model->>DB: Query candidate channels by (group, model)
DB-->>Model: candidate channels
Model->>Model: Partition/filter by matching API type
Model-->>Service: selected channel
Service-->>Middleware: channel
Middleware-->>Client: route to selected backend
Estimated Code Review Effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly Related PRs
Suggested Reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@types/relay_format.go`:
- Around line 70-71: The case in types/relay_format.go that matches routes with
strings.HasPrefix(path, "/v1/engines/") && strings.HasSuffix(path,
"/embeddings") currently returns RelayFormatGemini which misclassifies legacy
OpenAI-style embeddings; update that branch to return the OpenAI embeddings
relay format constant (e.g., RelayFormatOpenAIEmbeddings or the project’s
embeddings-specific RelayFormat) instead of RelayFormatGemini so
/v1/engines/*/embeddings is routed to the embeddings backend.
- Around line 50-76: The InferRelayFormatFromPath function currently returns
RelayFormatOpenAI in the default case which biases routing for unknown paths;
change the default return to RelayFormatUnknown to preserve neutral behavior,
and add a new case branch so paths starting with "/v1beta/models/" return
RelayFormatGemini (similar to the existing "/v1/models/" handling); update the
switch in InferRelayFormatFromPath accordingly and ensure RelayFormatUnknown is
used downstream where absence of an API-type hint is intended.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: a466504f-b7f0-43c6-98eb-6743b74a7024
📒 Files selected for processing (6)
controller/relay.gomiddleware/distributor.gomodel/ability.gomodel/channel_cache.goservice/channel_select.gotypes/relay_format.go
…d unknown paths Address review feedback on smart channel routing: - /v1/engines/*/embeddings is OpenAI's legacy embeddings endpoint, not Gemini. Return RelayFormatEmbedding so it routes to the correct backend. - /v1beta/models/ is the Gemini API path used by upstream clients (see middleware/distributor.go handling); add it alongside /v1/models/ so Gemini routes are inferred consistently. - Default branch now returns an empty RelayFormat instead of RelayFormatOpenAI. Downstream GetChannel treats empty as 'no API-type hint' and falls back to the original priority/weight-based selection, which is the truly neutral behaviour for unknown paths.
Author
关于 #4759@seefs001 在 #4759 提了同主题的修复(早本 PR 10 天),思路一致:同模型多渠道时优先选原生格式渠道。两者实现路径不同,简单对比:
另:#4759 引用了 我都可以配合 — 关闭本 PR、或在 #4759 基础上补齐路径覆盖、或在本 PR 引入 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Important
📝 变更描述 / Description
当前 new-api 支持同一模型配置多个渠道（如 Claude、OpenAI、Gemini 等），但渠道选择仅按优先级 + 权重随机，未考虑客户端请求的 API 格式。这导致：
当请求模型例如 mimo-v2.5-pro 的时候，小米官方提供两种接口，但是 new-api 这块默认渠道只能配置 openai 或者 claude，如果我同时需要两种接口访问，当前只能配置一种然后做转换，但是转换有时候在 claude code 或者 hermes agent 会失败
这时需要一个策略来自动选择，比如配置两个小米官方的渠道，分别是 openai 和 claude，当用户访问的时候，自动选择对应的官方接口去直接访问，这样避免转换带来的误差以及问题出现，但是也要考虑现有逻辑，所以需要当两个渠道同优先级的时候再进入这种策略，否则依旧保证高优先级的先访问，走默认的转换逻辑
兼容性：
RelayFormat时跳过智能路由，保持原有行为🚀 变更类型 / Type of change
🔗 关联任务 / Related Issue
✅ 提交前检查项 / Checklist
Bug fix，我已提交或关联对应 Issue，且不会将设计取舍、预期不一致或理解偏差直接归类为 bug。📸 运行证明 / Proof of Work
Summary by CodeRabbit
New Features
Refactor