This repository was archived by the owner on Feb 18, 2026. It is now read-only.
fix(openai): transform non-streaming tool_calls to Anthropic tool_use format#12
Closed
elidickinson wants to merge 3 commits into9j:mainfrom
Closed
fix(openai): transform non-streaming tool_calls to Anthropic tool_use format#12elidickinson wants to merge 3 commits into9j:mainfrom
elidickinson wants to merge 3 commits into9j:mainfrom
Conversation
…plete streams This fix addresses two critical issues with OpenAI-compatible provider streaming: 1. **Tool Calls Transformation**: OpenAI streaming sends tool_calls in a different format than Anthropic. This commit implements full transformation: - Detects tool_calls in OpenAI streaming chunks - Transforms to Anthropic tool_use format with proper event structure - Closes text content blocks before tool_use blocks - Sends content_block_start/delta/stop for each tool 2. **Incomplete Stream Handling**: Some OpenAI-compatible providers (notably Cerebras) close the stream without sending a finish_reason chunk. This commit adds: - Stream finalization that detects incomplete streams - Automatic end event generation (content_block_stop, message_delta, message_stop) - Prevents duplicate end events when finish_reason IS sent The fix ensures: - ✅ Streaming works with tool calls (TodoWrite, etc.) - ✅ No duplicate messages - ✅ Graceful handling of provider-specific streaming bugs - ✅ Full Anthropic API compatibility Tested with: Cerebras (zai-glm-4.6), streaming tool calls work perfectly.
…-streaming responses
The transform_response function was only extracting text content and ignoring
tool_calls from OpenAI responses. This caused tool calling to fail silently.
Now properly transforms OpenAI tool_calls format:
- OpenAI: message.tool_calls[].function.{name, arguments}
- Anthropic: content[].tool_use.{id, name, input}
Streaming already had this transformation, now non-streaming matches.
elidickinson
force-pushed
the
fix-non-streaming-tool-calls
branch
from
52f32c8 to
67cca74
Compare
The workaround was added to handle Cerebras closing streams without finish_reason. However, the root cause was the SSE event queue bug in streaming.rs that dropped finish_reason events when multiple events arrived in the same TCP chunk. Now that the event queue bug is fixed, the workaround is no longer needed. This simplifies the streaming implementation by removing: - Stream finalization logic that generated synthetic end events - Arc<Mutex> variables used to track stream state for cleanup The streaming implementation now relies on providers sending proper finish_reason events, which works correctly with the fixed event queue.
Author
|
ok ok nevermind I got a full fix in #14 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The transform_response function was only extracting text content and ignoring tool_calls from OpenAI responses. This caused tool calling to fail silently.
Now properly transforms OpenAI tool_calls format:
Streaming already had this transformation, now non-streaming matches.