feat: add key-level channel affinity#5006
Open
bubbleatgit wants to merge 3 commits into
Open
Conversation
Contributor
WalkthroughPersist ChannelAffinitySelection (channel_id + key_index) in cache, restore selected key index in middleware via service helpers and GetEnabledKeyByIndex, update affinity context/logging, and surface channel/key metadata in admin UI components. ChangesChannel Affinity Key Index Tracking
Sequence DiagramsequenceDiagram
participant Middleware as middleware/distributor
participant Service as service/channel_affinity
participant Model as model/channel
participant Cache as Redis Cache
participant Context as Gin Context
Middleware->>Service: GetPreferredChannelByAffinity(modelName, using_group)
Service->>Cache: Get ChannelAffinitySelection
Cache-->>Service: {channel_id, key_index}
Service-->>Middleware: ChannelAffinitySelection
Middleware->>Service: GetChannelAffinityKeyIndex(channel_id)
Service->>Context: read channel_affinity_key_index
Context-->>Service: key_index
Service-->>Middleware: key_index
Middleware->>Model: GetEnabledKeyByIndex(key_index)
Model-->>Middleware: key or error
alt key found and enabled
Middleware->>Service: UpdateChannelAffinitySelectedKeyIndex(channel_id, key_index)
else error or disabled
Middleware->>Model: GetNextEnabledKey() (fallback)
end
Middleware->>Service: MarkChannelAffinityUsed(selectedGroup, selection)
Service->>Context: store channel_affinity_channel_id and channel_affinity_key_index
Service->>Cache: RecordChannelAffinity(selection)
Estimated Code Review Effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly Related PRs
Suggested Reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
middleware/distributor.go (1)
383-389:⚠️ Potential issue | 🟠 Major | ⚡ Quick winClear the previous multi-key index when switching to a single-key channel.
Line 387 already notes that this context survives retries. With the new affinity recording, leaving
ContextKeyChannelMultiKeyIndexuntouched lets a stale index from an earlier multi-key attempt get cached for the final single-key channel, so later affinity restores and usage logs carry the wrong key index.Proposed fix
if channel.ChannelInfo.IsMultiKey { common.SetContextKey(c, constant.ContextKeyChannelIsMultiKey, true) common.SetContextKey(c, constant.ContextKeyChannelMultiKeyIndex, index) } else { // 必须设置为 false，否则在重试到单个 key 的时候会导致日志显示错误 common.SetContextKey(c, constant.ContextKeyChannelIsMultiKey, false) + common.SetContextKey(c, constant.ContextKeyChannelMultiKeyIndex, 0) }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@middleware/distributor.go` around lines 383 - 389, When channel.ChannelInfo.IsMultiKey is false you must clear any previous multi-key index so stale values don't survive retries; in the else branch (where you currently call common.SetContextKey(c, constant.ContextKeyChannelIsMultiKey, false)) also call common.SetContextKey(c, constant.ContextKeyChannelMultiKeyIndex, nil) (or the sentinel you use for "no index") to remove the old ContextKeyChannelMultiKeyIndex value so affinity/restores/logs cannot pick up a stale index.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@web/default/src/features/system-settings/general/channel-affinity/cache-stats-dialog.tsx`:
- Around line 129-132: The displayed label currently concatenates a hardcoded
English fragment `#${keyIndex + 1} (index ${keyIndex})`; replace this with a
translated string using t() and interpolation so both the prefix/number and the
“index” token are localizable. Update the data.push value construction (where
data.push is called) to call t(...) with placeholders (e.g. number and index)
and pass keyIndex and keyIndex+1 as values so translators can render the whole
label.
In
`@web/default/src/features/usage-logs/components/columns/common-logs-columns.tsx`:
- Around line 74-79: formatKeyIndexLabel currently emits user-facing English
text ("index") directly; update it so all display text is i18n-able by either
(a) accepting the translation function t (from useTranslation) as an argument
and using t() to build the label inside formatKeyIndexLabel, or (b) change
formatKeyIndexLabel to return structured data (e.g., {labelNumber: number,
index: number} or null) and perform the final string assembly with t() inside
the React component that renders the value; modify callers of
formatKeyIndexLabel accordingly (e.g., in the component that uses the function)
and ensure the translation key(s) are defined and used via t('...') rather than
hardcoded English.
---
Outside diff comments:
In `@middleware/distributor.go`:
- Around line 383-389: When channel.ChannelInfo.IsMultiKey is false you must
clear any previous multi-key index so stale values don't survive retries; in the
else branch (where you currently call common.SetContextKey(c,
constant.ContextKeyChannelIsMultiKey, false)) also call common.SetContextKey(c,
constant.ContextKeyChannelMultiKeyIndex, nil) (or the sentinel you use for "no
index") to remove the old ContextKeyChannelMultiKeyIndex value so
affinity/restores/logs cannot pick up a stale index.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 1b7dd7b6-ec37-4239-8fe8-1a592daba8aa
📒 Files selected for processing (10)
middleware/distributor.gomodel/channel.gomodel/channel_key_affinity_test.goservice/channel_affinity.goservice/channel_affinity_template_test.goservice/channel_affinity_usage_cache_test.goweb/default/src/features/system-settings/general/channel-affinity/cache-stats-dialog.tsxweb/default/src/features/usage-logs/components/columns/common-logs-columns.tsxweb/default/src/features/usage-logs/index.tsxweb/default/src/features/usage-logs/types.ts
Comment on lines
+129
to
+132
| data.push({ | ||
| key: t('Channel key'), | ||
| value: `#${keyIndex + 1} (index ${keyIndex})`, | ||
| }) |
Contributor
There was a problem hiding this comment.
Localize the key-index label text.
(index ${keyIndex}) introduces hardcoded English in UI output. Use t() for the full display string (or at least the “index” token) so it can be translated.
As per coding guidelines: web/default/**/*.{tsx,ts}: “All user-facing text content must support i18n using the t() function from useTranslation() in React components”.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@web/default/src/features/system-settings/general/channel-affinity/cache-stats-dialog.tsx`
around lines 129 - 132, The displayed label currently concatenates a hardcoded
English fragment `#${keyIndex + 1} (index ${keyIndex})`; replace this with a
translated string using t() and interpolation so both the prefix/number and the
“index” token are localizable. Update the data.push value construction (where
data.push is called) to call t(...) with placeholders (e.g. number and index)
and pass keyIndex and keyIndex+1 as values so translators can render the whole
label.
Comment on lines
+74
to
+79
| function formatKeyIndexLabel(keyIndex?: number): string | null { | ||
| if (keyIndex == null || !Number.isFinite(keyIndex) || keyIndex < 0) { | ||
| return null | ||
| } | ||
| return `#${keyIndex + 1} (index ${keyIndex})` | ||
| } |
Contributor
There was a problem hiding this comment.
Move key-index display text under i18n.
formatKeyIndexLabel returns a user-facing English phrase (index) directly. Pass t into this formatter (or return structured values and format in component with t) so the full label is translatable.
As per coding guidelines: web/default/**/*.{tsx,ts}: “All user-facing text content must support i18n using the t() function from useTranslation() in React components”.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@web/default/src/features/usage-logs/components/columns/common-logs-columns.tsx`
around lines 74 - 79, formatKeyIndexLabel currently emits user-facing English
text ("index") directly; update it so all display text is i18n-able by either
(a) accepting the translation function t (from useTranslation) as an argument
and using t() to build the label inside formatKeyIndexLabel, or (b) change
formatKeyIndexLabel to return structured data (e.g., {labelNumber: number,
index: number} or null) and perform the final string assembly with t() inside
the React component that renders the value; modify callers of
formatKeyIndexLabel accordingly (e.g., in the component that uses the function)
and ensure the translation key(s) are defined and used via t('...') rather than
hardcoded English.
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@web/default/src/features/usage-logs/components/dialogs/details-dialog.tsx`:
- Around line 138-143: formatKeyIndexLabel currently returns hard-coded English;
update it so the component uses the i18n translation function instead. Either
modify formatKeyIndexLabel to accept a translation function (t: TFunction) and
return t('keyIndexLabel', {n: keyIndex+1, index: keyIndex}) or move the string
construction into the React component that already calls useTranslation() and
call t(...) there (e.g., replace direct calls to formatKeyIndexLabel(...) with
t('...', {n: ..., index: ...})). Ensure you use the existing useTranslation/t()
from the component and add a matching translation key (with interpolation) in
locale files instead of hard-coded English.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 7cb3c294-3960-4dd2-83a2-c72636dad3dd
📒 Files selected for processing (1)
web/default/src/features/usage-logs/components/dialogs/details-dialog.tsx
Comment on lines
+138
to
+143
| function formatKeyIndexLabel(keyIndex?: number): string | null { | ||
| if (keyIndex == null || !Number.isFinite(keyIndex) || keyIndex < 0) { | ||
| return null | ||
| } | ||
| return `#${keyIndex + 1} (index ${keyIndex})` | ||
| } |
Contributor
There was a problem hiding this comment.
Localize key-index display text instead of hard-coded English.
formatKeyIndexLabel returns a user-facing English string directly, so this part won’t translate with locale changes. Route it through t(...) in the component path.
Suggested patch
-function formatKeyIndexLabel(keyIndex?: number): string | null {
+function formatKeyIndexLabel(
+ keyIndex: number | undefined,
+ t: (key: string, options?: Record<string, unknown>) => string
+): string | null {
if (keyIndex == null || !Number.isFinite(keyIndex) || keyIndex < 0) {
return null
}
- return `#${keyIndex + 1} (index ${keyIndex})`
+ return t('#{{displayIndex}} (index {{rawIndex}})', {
+ displayIndex: keyIndex + 1,
+ rawIndex: keyIndex,
+ })
}- const affinityKeyDisplay = formatKeyIndexLabel(affinity?.key_index)
+ const affinityKeyDisplay = formatKeyIndexLabel(affinity?.key_index, t)As per coding guidelines, "All user-facing text content must support i18n using the t() function from useTranslation() in React components".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@web/default/src/features/usage-logs/components/dialogs/details-dialog.tsx`
around lines 138 - 143, formatKeyIndexLabel currently returns hard-coded
English; update it so the component uses the i18n translation function instead.
Either modify formatKeyIndexLabel to accept a translation function (t:
TFunction) and return t('keyIndexLabel', {n: keyIndex+1, index: keyIndex}) or
move the string construction into the React component that already calls
useTranslation() and call t(...) there (e.g., replace direct calls to
formatKeyIndexLabel(...) with t('...', {n: ..., index: ...})). Ensure you use
the existing useTranslation/t() from the component and add a matching
translation key (with interpolation) in locale files instead of hard-coded
English.
bubbleatgit
force-pushed
the
channel-affinity-key-index
branch
2 times, most recently
from
0b61177 to
798cd98
Compare
bubbleatgit
force-pushed
the
channel-affinity-key-index
branch
from
798cd98 to
d633de0
Compare
Author
|
测试验收截图 渠道亲和性配置截图 [
{
"name": "codex cli trace",
"model_regex": [
"^gpt-.*$"
],
"path_regex": [
"/v1/responses"
],
"key_sources": [
{
"type": "gjson",
"path": "prompt_cache_key"
}
],
"param_override_template": {
"operations": [
{
"mode": "pass_headers",
"value": [
"Originator",
"Session_id",
"User-Agent",
"X-Codex-Beta-Features",
"X-Codex-Turn-Metadata"
],
"keep_origin": true
}
]
},
"value_regex": "",
"ttl_seconds": 0,
"skip_retry_on_failure": true,
"include_using_group": true,
"include_model_name": false,
"include_rule_name": true
},
{
"name": "claude cli trace",
"model_regex": [
"^claude-.*$"
],
"path_regex": [
"/v1/messages"
],
"key_sources": [
{
"type": "gjson",
"path": "metadata.user_id"
}
],
"param_override_template": {
"operations": [
{
"mode": "pass_headers",
"value": [
"X-Stainless-Arch",
"X-Stainless-Lang",
"X-Stainless-Os",
"X-Stainless-Package-Version",
"X-Stainless-Retry-Count",
"X-Stainless-Runtime",
"X-Stainless-Runtime-Version",
"X-Stainless-Timeout",
"User-Agent",
"X-App",
"Anthropic-Beta",
"Anthropic-Dangerous-Direct-Browser-Access",
"Anthropic-Version"
],
"keep_origin": true
}
]
},
"value_regex": "",
"ttl_seconds": 0,
"skip_retry_on_failure": true,
"include_using_group": true,
"include_model_name": false,
"include_rule_name": true
},
{
"name": "domestic codex cache trace",
"model_regex": [
"(?i)^(kimi|glm|minimax|deepseek|doubao-seed|qwen3|mimo)([-_/.:].*)?$"
],
"path_regex": [
"/v1/responses"
],
"user_agent_include": [],
"key_sources": [
{
"type": "gjson",
"key": "",
"path": "prompt_cache_key"
},
{
"type": "request_header",
"key": "Session_id",
"path": ""
}
],
"value_regex": "",
"ttl_seconds": 7200,
"skip_retry_on_failure": true,
"include_using_group": true,
"include_model_name": true,
"include_rule_name": true,
"param_override_template": {
"operations": [
{
"mode": "pass_headers",
"value": [
"Originator",
"Session_id",
"User-Agent",
"X-Codex-Beta-Features",
"X-Codex-Turn-Metadata"
],
"keep_origin": true
}
]
}
},
{
"name": "domestic claude cache trace",
"model_regex": [
"(?i)^(kimi|glm|minimax|deepseek|doubao-seed|qwen3|mimo)([-_/.:].*)?$"
],
"path_regex": [
"/v1/messages"
],
"user_agent_include": [],
"key_sources": [
{
"type": "gjson",
"key": "",
"path": "metadata.user_id"
},
{
"type": "gjson",
"key": "",
"path": "metadata.session_id"
},
{
"type": "request_header",
"key": "X-Session-Id",
"path": ""
}
],
"value_regex": "",
"ttl_seconds": 7200,
"skip_retry_on_failure": true,
"include_using_group": true,
"include_model_name": true,
"include_rule_name": true,
"param_override_template": {
"operations": [
{
"mode": "pass_headers",
"value": [
"X-Stainless-Arch",
"X-Stainless-Lang",
"X-Stainless-Os",
"X-Stainless-Package-Version",
"X-Stainless-Retry-Count",
"X-Stainless-Runtime",
"X-Stainless-Runtime-Version",
"X-Stainless-Timeout",
"User-Agent",
"X-App",
"Anthropic-Beta",
"Anthropic-Dangerous-Direct-Browser-Access",
"Anthropic-Version"
],
"keep_origin": true
}
]
}
}
]日志页面增加显示命中的规则详情
日志详情增加显示命中的规则详情
redis 缓存命中
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Important
📝 变更描述 / Description
🚀 变更类型 / Type of change
🔗 关联任务 / Related Issue
无关联任务
✅ 提交前检查项 / Checklist
Bug fix，我已提交或关联对应 Issue，且不会将设计取舍、预期不一致或理解偏差直接归类为 bug。📸 运行证明 / Proof of Work
测试验收截图
渠道亲和性配置
截图
配置示列：
[ { "name": "codex cli trace", "model_regex": [ "^gpt-.*$" ], "path_regex": [ "/v1/responses" ], "key_sources": [ { "type": "gjson", "path": "prompt_cache_key" } ], "param_override_template": { "operations": [ { "mode": "pass_headers", "value": [ "Originator", "Session_id", "User-Agent", "X-Codex-Beta-Features", "X-Codex-Turn-Metadata" ], "keep_origin": true } ] }, "value_regex": "", "ttl_seconds": 0, "skip_retry_on_failure": true, "include_using_group": true, "include_model_name": false, "include_rule_name": true }, { "name": "claude cli trace", "model_regex": [ "^claude-.*$" ], "path_regex": [ "/v1/messages" ], "key_sources": [ { "type": "gjson", "path": "metadata.user_id" } ], "param_override_template": { "operations": [ { "mode": "pass_headers", "value": [ "X-Stainless-Arch", "X-Stainless-Lang", "X-Stainless-Os", "X-Stainless-Package-Version", "X-Stainless-Retry-Count", "X-Stainless-Runtime", "X-Stainless-Runtime-Version", "X-Stainless-Timeout", "User-Agent", "X-App", "Anthropic-Beta", "Anthropic-Dangerous-Direct-Browser-Access", "Anthropic-Version" ], "keep_origin": true } ] }, "value_regex": "", "ttl_seconds": 0, "skip_retry_on_failure": true, "include_using_group": true, "include_model_name": false, "include_rule_name": true }, { "name": "domestic codex cache trace", "model_regex": [ "(?i)^(kimi|glm|minimax|deepseek|doubao-seed|qwen3|mimo)([-_/.:].*)?$" ], "path_regex": [ "/v1/responses" ], "user_agent_include": [], "key_sources": [ { "type": "gjson", "key": "", "path": "prompt_cache_key" }, { "type": "request_header", "key": "Session_id", "path": "" } ], "value_regex": "", "ttl_seconds": 7200, "skip_retry_on_failure": true, "include_using_group": true, "include_model_name": true, "include_rule_name": true, "param_override_template": { "operations": [ { "mode": "pass_headers", "value": [ "Originator", "Session_id", "User-Agent", "X-Codex-Beta-Features", "X-Codex-Turn-Metadata" ], "keep_origin": true } ] } }, { "name": "domestic claude cache trace", "model_regex": [ "(?i)^(kimi|glm|minimax|deepseek|doubao-seed|qwen3|mimo)([-_/.:].*)?$" ], "path_regex": [ "/v1/messages" ], "user_agent_include": [], "key_sources": [ { "type": "gjson", "key": "", "path": "metadata.user_id" }, { "type": "gjson", "key": "", "path": "metadata.session_id" }, { "type": "request_header", "key": "X-Session-Id", "path": "" } ], "value_regex": "", "ttl_seconds": 7200, "skip_retry_on_failure": true, "include_using_group": true, "include_model_name": true, "include_rule_name": true, "param_override_template": { "operations": [ { "mode": "pass_headers", "value": [ "X-Stainless-Arch", "X-Stainless-Lang", "X-Stainless-Os", "X-Stainless-Package-Version", "X-Stainless-Retry-Count", "X-Stainless-Runtime", "X-Stainless-Runtime-Version", "X-Stainless-Timeout", "User-Agent", "X-App", "Anthropic-Beta", "Anthropic-Dangerous-Direct-Browser-Access", "Anthropic-Version" ], "keep_origin": true } ] } } ]日志页面增加显示命中的规则详情
日志详情增加显示命中的规则详情
redis 缓存命中
Summary: Adds channel+key affinity, bumps channel affinity usage-cache stats namespace to v2, and exposes the matched channel key index in the new usage logs UI. Tests: go test ./service ./model ./middleware -count=1; docker buildx build --output type=docker --provenance=false --sbom=false --platform linux/amd64 -t new-api:key-affinity-ui-check .
Summary by CodeRabbit
New Features
Bug Fixes
Tests