Skip to content

Conversation

@ryx2
Copy link
Contributor

@ryx2 ryx2 commented Dec 3, 2025

Summary

This adds support for Anthropic's programmatic tool calling feature, allowing tools to be called from within code execution blocks.

Part 3 of 3 - Following the review recommendations on #3550, splitting into:

  1. PR Add examples field to ToolDefinition and send it to Anthropic #3619: input_examples feature
  2. PR feat: Add ToolSearchTool and defer_loading for dynamic tool discovery #3620: Tool Search + defer_loading feature
  3. This PR: Programmatic Tool Calling (programmatically_callable)

Changes

  • Add programmatically_callable field to ToolDefinition dataclass

    • False (default): Tool can only be called directly by the model
    • True: Tool callable both directly and from code execution
    • 'only': Tool only callable from code execution
  • Add programmatically_callable parameter to:

    • Tool class constructor
    • FunctionToolset.add_function() and @tool decorator
    • Agent.tool() and Agent.tool_plain() decorators
  • Update Anthropic model:

    • Map programmatically_callable to Anthropic's allowed_callers API
    • Auto-add CodeExecutionTool when programmatically_callable is used
    • Use newer code execution tool (20250825) when PTC is enabled
    • Store caller and container_id in ToolCallPart.provider_details for tools called from code execution

Example Usage

from pydantic_ai import Agent
from pydantic_ai.builtin_tools import CodeExecutionTool

agent = Agent('anthropic:claude-sonnet-4-5', builtin_tools=[CodeExecutionTool()])

# Tool callable both directly by model and from code execution
@agent.tool_plain(programmatically_callable=True)
def add_numbers(x: int, y: int) -> int:
    """Add two numbers."""
    return x + y

# Tool only callable from code execution (not directly by model)
@agent.tool_plain(programmatically_callable='only')
def calculate_pi(precision: int) -> float:
    """Calculate pi to given precision."""
    # Only code execution can call this
    return 3.14159265358979

Test plan

  • Added unit tests for programmatically_callable values (True, 'only', False)
  • Added test for auto-adding CodeExecutionTool when PTC is used
  • Added tests verifying correct code execution tool version is used
  • All existing tests pass

🤖 Generated with Claude Code

This adds support for Anthropic's programmatic tool calling feature,
allowing tools to be called from within code execution blocks.

Changes:
- Add `programmatically_callable` field to `ToolDefinition` dataclass
  - `False` (default): Tool can only be called directly by the model
  - `True`: Tool callable both directly and from code execution
  - `'only'`: Tool only callable from code execution

- Add `programmatically_callable` parameter to:
  - `Tool` class constructor
  - `FunctionToolset.add_function()` and `@tool` decorator
  - `Agent.tool()` and `Agent.tool_plain()` decorators

- Update Anthropic model:
  - Map `programmatically_callable` to Anthropic's `allowed_callers` API
  - Auto-add `CodeExecutionTool` when `programmatically_callable` is used
  - Use newer code execution tool (20250825) when PTC is enabled
  - Store `caller` and `container_id` in `ToolCallPart.provider_details`
    for tools called from code execution

- Add comprehensive tests for the feature

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@DouweM
Copy link
Collaborator

DouweM commented Dec 8, 2025

@ryx2 Thanks for splitting this out!

I wrote some notes in #3666 (comment) about how I'd like to support this feature for all models by implementing it in Pydantic AI, while leveraging provider-native support if possible, so while we don't need to do all of that right in this PR, we do need to make sure the API we commit to here is compatible with that next step.

Specifically, let's think through how the programatically_callable field interacts with the proposed ProgrammaticallyCallableToolset (name TBD!), which can both explicitly wrap another toolset (which would typically have tools with programmatically_callable set to the default value), and can be created automatically when any of the agent's tools have programmatically_callable set explicitly.

That suggests that the default value for programmatically_callable should be None, which ProgrammaticallyCallableToolset would then interpret as 'only' when wrapping, as the expectation of creating the toolset would be that these are only available from code, and toolset will expose just a single run_code tool.

If the field is set explicitly to False, the toolset will return it directly from get_tools (and not make it available to code). If it's set to True, it'll be returned from get_tools and be available from code. (If Anthropic is used, the tool def will be returned from get_tools and the value will be mapped by AnthropicModel)

If we detect any ToolDefinition with programmatically_callable set to a value other than the default of None, we automatically create a ProgrammaticallyCallableToolset for it, with the behavior as above.

I don't love the programmatically_callable name, and it's not clear that True means "both directly and programmatically".

I'm thinking that Anthropic's allowed_callers does make more sense, which in our case would become None | set[Literal['direct', 'code']] = None, with None effectively meaning {'direct'} for top-level tools (that wouldn't get any special handling), and None meaning {'code'} for tools passed into ProgrammaticallyCallableToolset. An empty set is a valid value and I suppose should mean that the tool is not callable at all.

Can you make those changes to field name and default please?

I don't like the ProgrammaticallyCallableToolset name one bit, but we can figure out something better later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants