Skip to content

Consider pruning failed structured output and tool call generations #495

@eb8680

Description

@eb8680

When Templates generate malformed tool calls or structured outputs detectable statically at the level of the tool signature or output schema, we should probably be removing the failures early from the message sequence once a corrected version is generated in order to minimize context usage.

Note this is distinct from semantic failures in tool calling, where we may generate a correctly typed tool call that fails when executing the body of the tool. These need to persist for the duration of a Template call.

Metadata

Metadata

Assignees

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions