-
Notifications
You must be signed in to change notification settings - Fork 512
Feat/preserve thought signature for gemini3 #1227
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Feat/preserve thought signature for gemini3 #1227
Conversation
- Add thoughtSignature field to ToolUse TypedDict as optional field - Add thoughtSignature to ContentBlockStartToolUse TypedDict - Preserve thoughtSignature during streaming event processing - Fixes compatibility with Gemini 3 Pro thinking mode This change enables proper multi-turn function calling with Gemini 3 Pro, which requires thought_signature to be passed back in subsequent requests. Resolves: Gemini 3 Pro 400 error for missing thought_signature See: https://ai.google.dev/gemini-api/docs/thought-signatures
- Capture thought_signature from Gemini function call responses - Base64 encode thought_signature for storage in message history - Decode and pass thought_signature back to Gemini in subsequent requests - Configure thinking_config to disable thinking text but preserve signatures - Add NotRequired import to content.py for type safety This complements the framework changes by implementing Gemini-specific handling of thought signatures for proper multi-turn function calling with Gemini 3 Pro. See: https://ai.google.dev/gemini-api/docs/thought-signatures
- Fix variable name conflict with thought_signature - Break long lines to comply with 120 character limit - Use explicit type annotations for thought signature variables
ed1837d to
a20b7ea
Compare
|
@dpolistwm I see that you have created two new test files. I dont recommend that, the tools test file already exists here and creating new test files for testing small logic is something the maintainers wouldn't like I think. Make sure you only add tests for the fix you are proposing. The best option I think here would be just waiting for a maintainer to see what approach they would take. Thanks. |
- Rename test_tools.py -> test_tool_use.py (tests ToolUse TypedDict) - Rename test_content.py -> test_content_block_start_tool_use.py (tests ContentBlockStartToolUse TypedDict) This makes the test file names more descriptive and avoids confusion with tests/strands/tools/test_tools.py
|
The above commit enhances coverage of Gemini models, including lines that implement the new thoughtSignature preservation functionality |
|
Guys, do I need to do something else here? Let me know... |
|
hey @dpolistwm , thank you for the PR :) I am hesitant to merge in this PR right now, because Given the bug that keeps Gemin 3 Pro working with Strands, I will instead merge a change to use |
|
Hey @mkmeral , I understand your concern, but implementing this workaround and degrading the tought process of such a popular model would make strands agents a "second class citizen" among other frameworks in the market. Many users may stop using Strands Agents with Gemini 3 Pro (one of the most popular, fast and powerful LLMs in the market) if this is the official solution and probably choose ADK instead. While it does not support AWS Agentcore directly (memory, etc.), it does support Google's equally feature-rich AgentEngine counterpart. We would certainly consider that. Please take this risk into account when making your decision. |
|
@dpolistwm actually I think I found a way to get the feature working with your PR, and leave the tool use event type as is. Gemini 3 Pro only validates the latest thought signature, not all of them. So what we can do instead is keep track of the thought signatures in Can you update your PR following this approach? One other thing to note: we would also still want to apply Does this make sense? |
|
I'll try and make this work ;) |
This refactors the thoughtSignature feature to be self-contained within the GeminiModel class instead of spreading it across the SDK: Changes: - Removed thoughtSignature from ContentBlockStartToolUse (content.py) - Removed thoughtSignature from ToolUse (tools.py) - Removed thoughtSignature handling from streaming.py - Added self.last_thought_signature instance attribute to GeminiModel - GeminiModel now stores thought_signature from responses and uses it for subsequent requests automatically Benefits: - Cleaner SDK types without Gemini-specific fields - Simpler implementation without base64 encoding/decoding in messages - thought_signature is automatically tracked per model instance - No changes required to SDK-level streaming pipeline Tests updated accordingly to verify internal storage behavior.
…ini.py Reverted content.py, tools.py, streaming.py and test files to their original state. thoughtSignature handling is now self-contained in GeminiModel using self.last_thought_signature.
|
Now the only changed file in this PR is gemini.py with the last thought signature. I tested it locally against a real project and it is working fine 💯 |
|
Working on "0 tools" implementation (getting a 400 error)... |
|
Looks fine now to me. Please let me know if you need anything else |
|
Hello @mkmeral, could my PR also be used in some way, since @dpolistwm mentions gemini 3 doesnt work in my PR, maybe we can figure out a way to fix it. Or if this PR is already correct, I will close my PR then ig. |
Description
This PR adds support for preserving Gemini's
thoughtSignaturefield during function calling, which is required by Gemini 3 Pro for multi-turn conversations with tools.Problem
When using Gemini 3 Pro with function calling, the model returns a
thought_signaturefield that must be passed back in subsequent requests. Currently, Strands drops this field during streaming and message reconstruction, causing 400 errors:Unable to submit request because function call tool_name in the 2. content block is missing a thought_signature
This affects multi-turn conversations, nested agents, and any workflow requiring multiple function call exchanges with Gemini 3 Pro.
Solution
Framework Changes:
src/strands/types/tools.py: AddedthoughtSignature: NotRequired[str]toToolUseTypedDict (changed tototal=Falseto support optional field)src/strands/types/content.py: AddedthoughtSignature: NotRequired[str]toContentBlockStartToolUseTypedDict and addedNotRequiredimportsrc/strands/event_loop/streaming.py: PreservethoughtSignaturewhen processing streaming chunks (2 locations: extracting from tool use data and passing through to ToolUse object creation)Gemini Provider Changes:
4.
src/strands/models/gemini.py:thought_signaturefrom Gemini function call responsesthinking_configto disable thinking text while preserving signaturesTechnical Details
The
thoughtSignatureis an encrypted token provided by Gemini that preserves the model's reasoning context. It's:NotRequired[str]) that doesn't affect other providersReference: Gemini Thought Signatures Documentation
Related Issues
None - this is a new feature to support Gemini 3 Pro's function calling requirements.
Documentation PR
N/A - No documentation changes needed (inline code comments added for clarity)
Type of Change
New feature
Testing
How have you tested the change?
Tested with:
gemini-3-pro-preview) with function callingResults:
thought_signatureVerified in consuming repositories:
Note on CI: There is 1 pre-existing mypy error in
src/strands/tools/tools.py:47(unused type: ignore comment) that is unrelated to these changes. All type checking passes for the files modified in this PR.hatch run prepareChecklist
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Related Issues
Fixes #1199