fix: add basic inline file support for Claude relay#3505
Merged
Calcium-Ion merged 2 commits intoQuantumNous:mainfrom Apr 2, 2026
Merged
fix: add basic inline file support for Claude relay#3505Calcium-Ion merged 2 commits intoQuantumNous:mainfrom
Calcium-Ion merged 2 commits intoQuantumNous:mainfrom
Conversation
Contributor
WalkthroughAdded helpers to parse and convert file attachments into Claude-compatible media (PDF → document, text files → text), refactored message parsing to a switch over content types, and added nil guards to avoid panics when non-image media lack image metadata. Changes
Sequence DiagramsequenceDiagram
participant Client
participant MessageParser as Message Parser
participant ContentSwitch as Content Type Handler
participant FileService as File Service
participant ClaudeFormatter as Claude Formatter
participant Claude
Client->>MessageParser: Send message with content[]
MessageParser->>ContentSwitch: Iterate content items
alt Text item
ContentSwitch->>ClaudeFormatter: Pass text
ClaudeFormatter->>Claude: Add text content
else Image URL item
ContentSwitch->>FileService: GetImageMedia()
FileService-->>ContentSwitch: imageUrl or nil
alt imageUrl present
ContentSwitch->>FileService: service.GetBase64Data(url)
FileService-->>ContentSwitch: base64 data
ContentSwitch->>ClaudeFormatter: Create image source
ClaudeFormatter->>Claude: Add image content
else nil
ContentSwitch->>ContentSwitch: continue (skip)
end
else File item
ContentSwitch->>FileService: service.GetBase64Data(fileRef)
FileService-->>ContentSwitch: base64 data
ContentSwitch->>ContentSwitch: detect MIME from name or data
alt application/pdf
ContentSwitch->>ClaudeFormatter: Build document media (base64)
ClaudeFormatter->>Claude: Add document media
else text/plain
ContentSwitch->>ContentSwitch: decode base64 to text
ContentSwitch->>ClaudeFormatter: Build text media
ClaudeFormatter->>Claude: Add text media
else Unsupported MIME
ContentSwitch->>ContentSwitch: log & skip
end
end
Claude-->>Client: Return response
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
relay/channel/claude/relay-claude.go (1)
401-461:⚠️ Potential issue | 🟠 MajorReject messages that become empty after media filtering.
If every part is skipped (
unsupported file,nil imageUrl, etc.),claudeMediaMessagesstays empty and still gets assigned toclaudeMessage.Content. That bypasses the fallback at Lines 318-320, so a request with only an unsupported attachment no longer panics but still leaves this relay with an empty Claude message instead of a clear local error.🛠️ Suggested patch
if message.ToolCalls != nil { for _, toolCall := range message.ParseToolCalls() { inputObj := make(map[string]any) if err := json.Unmarshal([]byte(toolCall.Function.Arguments), &inputObj); err != nil { common.SysLog("tool call function arguments is not a map[string]any: " + fmt.Sprintf("%v", toolCall.Function.Arguments)) continue } claudeMediaMessages = append(claudeMediaMessages, dto.ClaudeMediaMessage{ Type: "tool_use", Id: toolCall.ID, Name: toolCall.Function.Name, Input: inputObj, }) } } + if len(claudeMediaMessages) == 0 { + return nil, fmt.Errorf("message contains no Claude-compatible content") + } claudeMessage.Content = claudeMediaMessages🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@relay/channel/claude/relay-claude.go` around lines 401 - 461, After filtering media into claudeMediaMessages, ensure you reject/return an error when all parts were skipped so we don't assign an empty slice to claudeMessage.Content; specifically, after the loop that builds claudeMediaMessages (which uses message.ParseContent(), message.ParseToolCalls(), buildClaudeFileMessage, and service.GetBase64Data), check if len(claudeMediaMessages) == 0 and return a clear local error (or trigger the existing fallback) instead of assigning the empty slice to claudeMessage.Content; this prevents creating an empty Claude payload when the original message contained only unsupported or nil media.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@relay/channel/claude/relay-claude.go`:
- Around line 49-102: createClaudeFileSource currently returns
types.NewURLFileSource(file.FileData) without preserving a filename-derived MIME
and buildClaudeFileMessage matches mime strings strictly (so values like
"text/plain; charset=utf-8" are skipped); fix by populating the
FileSource.MimeType for URL-backed files in createClaudeFileSource (create the
URL source into a variable, detect extension via service.GetMimeTypeByExtension
using filepath.Ext(file.FileName) and set source.MimeType when not
"application/octet-stream", then return that source) and update
buildClaudeFileMessage to normalize the mimeType from service.GetBase64Data by
trimming and lowercasing and stripping any parameters after ';' before the
switch so "text/plain; charset=utf-8" and similar values match the supported
branches.
- Around line 3-10: Replace the direct uses of encoding/json's json.Marshal and
json.Unmarshal in relay-claude.go with the codebase wrapper functions
common.Marshal and common.Unmarshal: find the four call sites that currently
call json.Unmarshal(...) and json.Marshal(...) and swap them to
common.Unmarshal(...) and common.Marshal(...), keep the same error handling and
argument shapes (pass the byte slice and target pointer to common.Unmarshal and
use the returned []byte from common.Marshal), add an import for the common
package and remove encoding/json if it becomes unused so the file compiles.
---
Outside diff comments:
In `@relay/channel/claude/relay-claude.go`:
- Around line 401-461: After filtering media into claudeMediaMessages, ensure
you reject/return an error when all parts were skipped so we don't assign an
empty slice to claudeMessage.Content; specifically, after the loop that builds
claudeMediaMessages (which uses message.ParseContent(),
message.ParseToolCalls(), buildClaudeFileMessage, and service.GetBase64Data),
check if len(claudeMediaMessages) == 0 and return a clear local error (or
trigger the existing fallback) instead of assigning the empty slice to
claudeMessage.Content; this prevents creating an empty Claude payload when the
original message contained only unsupported or nil media.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 498da071-2726-4be8-8dae-47b3b59e8485
📒 Files selected for processing (2)
relay/channel/claude/relay-claude.gorelay/channel/claude/relay_claude_test.go
Comment on lines
3
to
10
| import ( | ||
| "encoding/base64" | ||
| "encoding/json" | ||
| "fmt" | ||
| "io" | ||
| "net/http" | ||
| "path/filepath" | ||
| "strings" |
Contributor
There was a problem hiding this comment.
🛠️ Refactor suggestion | 🟠 Major
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Direct encoding/json usage in relay/channel/claude/relay-claude.go:"
rg -n -C1 'encoding/json|json\.(Marshal|Unmarshal)\b' relay/channel/claude/relay-claude.go
echo
echo "Approved wrappers in common/json.go:"
rg -n -C1 'func (Marshal|Unmarshal|UnmarshalJsonStr|DecodeJson|GetJsonType)\b' common/json.goRepository: QuantumNous/new-api
Length of output: 1442
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "=== All direct encoding/json calls in relay/channel/claude/ ==="
rg 'json\.(Marshal|Unmarshal|MarshalIndent|UnmarshalString)\b' relay/channel/claude/ -n || true
echo
echo "=== Checking specific lines in relay-claude.go ==="
# Show context around each flagged line
for line_num in 145 449 583 932; do
echo "--- Line $line_num ---"
sed -n "$((line_num-2)),$((line_num+2))p" relay/channel/claude/relay-claude.go
done
echo
echo "=== Count of direct json calls ==="
rg 'json\.(Marshal|Unmarshal)\b' relay/channel/claude/relay-claude.go | wc -lRepository: QuantumNous/new-api
Length of output: 1790
Replace direct json.Marshal and json.Unmarshal calls with wrapper functions from common/json.go.
Lines 145, 449, 583, and 932 use direct json.Unmarshal and json.Marshal calls. Replace with common.Unmarshal() and common.Marshal() respectively to align with codebase standards.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@relay/channel/claude/relay-claude.go` around lines 3 - 10, Replace the direct
uses of encoding/json's json.Marshal and json.Unmarshal in relay-claude.go with
the codebase wrapper functions common.Marshal and common.Unmarshal: find the
four call sites that currently call json.Unmarshal(...) and json.Marshal(...)
and swap them to common.Unmarshal(...) and common.Marshal(...), keep the same
error handling and argument shapes (pass the byte slice and target pointer to
common.Unmarshal and use the returned []byte from common.Marshal), add an import
for the common package and remove encoding/json if it becomes unused so the file
compiles.
Comment on lines
+49
to
+102
| func createClaudeFileSource(file *dto.MessageFile) *types.FileSource { | ||
| if file == nil || file.FileData == "" { | ||
| return nil | ||
| } | ||
| if strings.HasPrefix(file.FileData, "http://") || strings.HasPrefix(file.FileData, "https://") { | ||
| return types.NewURLFileSource(file.FileData) | ||
| } | ||
| mimeType := "" | ||
| if ext := strings.TrimPrefix(strings.ToLower(filepath.Ext(file.FileName)), "."); ext != "" { | ||
| if detected := service.GetMimeTypeByExtension(ext); detected != "application/octet-stream" { | ||
| mimeType = detected | ||
| } | ||
| } | ||
| return types.NewBase64FileSource(file.FileData, mimeType) | ||
| } | ||
|
|
||
| func buildClaudeFileMessage(c *gin.Context, file *dto.MessageFile) (*dto.ClaudeMediaMessage, error) { | ||
| source := createClaudeFileSource(file) | ||
| if source == nil { | ||
| return nil, nil | ||
| } | ||
| base64Data, mimeType, err := service.GetBase64Data(c, source, "formatting document for Claude") | ||
| if err != nil { | ||
| return nil, fmt.Errorf("get file data failed: %w", err) | ||
| } | ||
| switch strings.ToLower(mimeType) { | ||
| case "application/pdf": | ||
| return &dto.ClaudeMediaMessage{ | ||
| Type: "document", | ||
| Source: &dto.ClaudeMessageSource{ | ||
| Type: "base64", | ||
| MediaType: mimeType, | ||
| Data: base64Data, | ||
| }, | ||
| }, nil | ||
| case "text/plain": | ||
| decodedData, err := base64.StdEncoding.DecodeString(base64Data) | ||
| if err != nil { | ||
| return nil, fmt.Errorf("decode text file data failed: %w", err) | ||
| } | ||
| return &dto.ClaudeMediaMessage{ | ||
| Type: "text", | ||
| Text: common.GetPointer(string(decodedData)), | ||
| }, nil | ||
| default: | ||
| msg := fmt.Sprintf("claude: skip unsupported file content, filename=%q, mime=%q", file.FileName, mimeType) | ||
| if c != nil { | ||
| logger.LogInfo(c, msg) | ||
| } else { | ||
| common.SysLog(msg) | ||
| } | ||
| return nil, nil | ||
| } | ||
| } |
Contributor
There was a problem hiding this comment.
Harden MIME detection for URL-backed file content.
Valid URL-backed PDFs/text files can still get dropped here. createClaudeFileSource returns types.NewURLFileSource before the filename-derived MIME is attached, even though types.FileSource already has MimeType in types/file_source.go, Lines 20-31, and types.NewURLFileSource in types/file_source.go, Lines 140-154 only preserves the URL. On top of that, buildClaudeFileMessage compares the raw content type string, so common values like text/plain; charset=utf-8 miss the supported branch and get skipped.
🛠️ Suggested patch
func createClaudeFileSource(file *dto.MessageFile) *types.FileSource {
if file == nil || file.FileData == "" {
return nil
}
- if strings.HasPrefix(file.FileData, "http://") || strings.HasPrefix(file.FileData, "https://") {
- return types.NewURLFileSource(file.FileData)
- }
mimeType := ""
if ext := strings.TrimPrefix(strings.ToLower(filepath.Ext(file.FileName)), "."); ext != "" {
if detected := service.GetMimeTypeByExtension(ext); detected != "application/octet-stream" {
mimeType = detected
}
}
+ if strings.HasPrefix(file.FileData, "http://") || strings.HasPrefix(file.FileData, "https://") {
+ return &types.FileSource{
+ Type: types.FileSourceTypeURL,
+ URL: file.FileData,
+ MimeType: mimeType,
+ }
+ }
return types.NewBase64FileSource(file.FileData, mimeType)
}- switch strings.ToLower(mimeType) {
+ normalizedMimeType := strings.ToLower(strings.TrimSpace(strings.SplitN(mimeType, ";", 2)[0]))
+ switch normalizedMimeType {🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@relay/channel/claude/relay-claude.go` around lines 49 - 102,
createClaudeFileSource currently returns types.NewURLFileSource(file.FileData)
without preserving a filename-derived MIME and buildClaudeFileMessage matches
mime strings strictly (so values like "text/plain; charset=utf-8" are skipped);
fix by populating the FileSource.MimeType for URL-backed files in
createClaudeFileSource (create the URL source into a variable, detect extension
via service.GetMimeTypeByExtension using filepath.Ext(file.FileName) and set
source.MimeType when not "application/octet-stream", then return that source)
and update buildClaudeFileMessage to normalize the mimeType from
service.GetBase64Data by trimming and lowercasing and stripping any parameters
after ';' before the switch so "text/plain; charset=utf-8" and similar values
match the supported branches.
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@relay/channel/claude/relay-claude.go`:
- Around line 434-442: The switch branch for dto.ContentTypeFile is commented
out causing file attachments to be skipped; restore the case for
dto.ContentTypeFile in the same switch where mediaMessage is handled, call
buildClaudeFileMessage(c, mediaMessage.GetFile()), check and return any error,
and if the returned claudeFileMessage is non-nil append it to
claudeMediaMessages (the same slice used for other media). Ensure you use the
existing identifiers dto.ContentTypeFile, buildClaudeFileMessage,
mediaMessage.GetFile(), and claudeMediaMessages so the file handling integrates
with the surrounding conversion flow.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: de480dec-4b0e-428c-b4f9-0e3de98fb31a
📒 Files selected for processing (1)
relay/channel/claude/relay-claude.go
Comment on lines
+434
to
+442
| // FIXME | ||
| //case dto.ContentTypeFile: | ||
| // claudeFileMessage, err := buildClaudeFileMessage(c, mediaMessage.GetFile()) | ||
| // if err != nil { | ||
| // return nil, err | ||
| // } | ||
| // if claudeFileMessage != nil { | ||
| // claudeMediaMessages = append(claudeMediaMessages, *claudeFileMessage) | ||
| // } |
Contributor
There was a problem hiding this comment.
dto.ContentTypeFile handling is still disabled.
Line 435-442 is commented out, so file attachments fall through to default (Line 443-444) and are skipped. That leaves inline file support inactive in the main conversion path.
Suggested patch
- // FIXME
- //case dto.ContentTypeFile:
- // claudeFileMessage, err := buildClaudeFileMessage(c, mediaMessage.GetFile())
- // if err != nil {
- // return nil, err
- // }
- // if claudeFileMessage != nil {
- // claudeMediaMessages = append(claudeMediaMessages, *claudeFileMessage)
- // }
+ case dto.ContentTypeFile:
+ claudeFileMessage, err := buildClaudeFileMessage(c, mediaMessage.GetFile())
+ if err != nil {
+ return nil, err
+ }
+ if claudeFileMessage != nil {
+ claudeMediaMessages = append(claudeMediaMessages, *claudeFileMessage)
+ }📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| // FIXME | |
| //case dto.ContentTypeFile: | |
| // claudeFileMessage, err := buildClaudeFileMessage(c, mediaMessage.GetFile()) | |
| // if err != nil { | |
| // return nil, err | |
| // } | |
| // if claudeFileMessage != nil { | |
| // claudeMediaMessages = append(claudeMediaMessages, *claudeFileMessage) | |
| // } | |
| case dto.ContentTypeFile: | |
| claudeFileMessage, err := buildClaudeFileMessage(c, mediaMessage.GetFile()) | |
| if err != nil { | |
| return nil, err | |
| } | |
| if claudeFileMessage != nil { | |
| claudeMediaMessages = append(claudeMediaMessages, *claudeFileMessage) | |
| } |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@relay/channel/claude/relay-claude.go` around lines 434 - 442, The switch
branch for dto.ContentTypeFile is commented out causing file attachments to be
skipped; restore the case for dto.ContentTypeFile in the same switch where
mediaMessage is handled, call buildClaudeFileMessage(c, mediaMessage.GetFile()),
check and return any error, and if the returned claudeFileMessage is non-nil
append it to claudeMediaMessages (the same slice used for other media). Ensure
you use the existing identifiers dto.ContentTypeFile, buildClaudeFileMessage,
mediaMessage.GetFile(), and claudeMediaMessages so the file handling integrates
with the surrounding conversion flow.
VeryGoodUser1
pushed a commit
to xmz-ai/new-api
that referenced
this pull request
Apr 3, 2026
…pport fix: add basic inline file support for Claude relay
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fix #3481
Summary by CodeRabbit
New Features
Tests