to_provider_format crashes with JSONDecodeError when replaying a tool call whose arguments never parsed (Anthropic / Google / AWS)

[georgeglarson]

Bug Description

FunctionCall.arguments holds the model's raw tool-call output. It gets canonicalized to valid JSON only when the arguments parse successfully: parse_function_arguments runs json_repair before a tool executes and writes the repaired form back to history. When the arguments can't be parsed at all, the raw string stays in chat history verbatim. That happens two ways: a small or open-weight model emits output json_repair also can't recover (the tool call fails as a ToolError and the raw string persists), or the history is built outside the execution path (ChatContext.from_dict, a custom llm_node, restored session state).

The Anthropic, Google, and AWS chat-context formatters call json.loads(msg.arguments) directly in to_chat_ctx. A later, healthy turn that formats that history for the provider then raises json.JSONDecodeError and the whole request aborts. The OpenAI and Mistral formatters pass arguments through as an opaque string, so they don't hit this.

Expected Behavior

Formatting stored history should not crash on a tool call the framework itself already recorded. A call with unparseable arguments should degrade gracefully (an empty input object) instead of taking down an unrelated later turn.

Reproduction Steps

from livekit.agents.llm import ChatContext, FunctionCall, FunctionCallOutput

ctx = ChatContext.empty()
# A tool call whose arguments never parsed: unrecoverable small/open-weight
# model output, or history restored via ChatContext.from_dict / a custom
# llm_node. The raw string persists in history verbatim.
ctx.insert(FunctionCall(call_id="c1", name="lookup", arguments="not-a-json-object"))
ctx.insert(FunctionCallOutput(call_id="c1", name="lookup", output="tool error", is_error=True))

# A later, healthy turn formats the whole history for the provider:
ctx.to_provider_format(format="anthropic")   # raises json.JSONDecodeError

On current main this raises, from inside to_provider_format:

json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

The same crash reproduces with format="google" and format="aws". No live model or network needed; the history is built directly.

Operating System

Fedora Linux 43 (the path is pure Python and OS-independent).

Models Used

Any LLM routed through the Anthropic, Google, or AWS chat-context formatter. The repro needs no model; it builds the ChatContext directly.

Package Versions

livekit-agents==1.6.4
python==3.13
# also reproduces on current main (222bd0f7)

Proposed Solution

# a shared helper the three JSON-object formatters call instead of json.loads:
def parse_tool_call_arguments(fnc_call: llm.FunctionCall) -> dict[str, Any]:
    arguments = fnc_call.arguments
    if not arguments:
        return {}
    try:
        parsed = json.loads(arguments)
    except json.JSONDecodeError:
        parsed = None
    if isinstance(parsed, dict):
        return parsed
    logger.warning(
        "could not parse stored tool call arguments ...",
        extra={"call_id": fnc_call.call_id, "tool_name": fnc_call.name},
    )
    return {}

{} is preferred over a best-effort repair: the historical tool call already produced its output, so fabricating arguments the model never sent would risk feeding wrong data back to the provider. A tool call's arguments are always a named-parameter object, so JSON that parses to a non-object (array, scalar, null) degrades the same way.

I've a branch ready with this fix plus tests: the three providers against malformed, truncated, and template-token-leaked arguments, a valid-args regression, non-object inputs, and the fallback's warning. All repo CI gates pass (unit tests, ruff, mypy strict). Happy to open the PR if you're open to it.

Additional Context

This is distinct from #4240 and the closed #4974. Those are execution-time: a tool fails now because its arguments were truncated, and the proposed fix tried to recover the arguments so the tool could still run. That direction was declined, since the framework shouldn't guess parameters from broken model output.

This bug points the other way. The malformed call already failed correctly and became a tool error. Nothing here recovers anything. The framework just shouldn't crash a valid later request while formatting its own stored history. The {} fallback respects the same position #4974 was closed on: it never invents arguments, it only keeps a self-recorded bad call from aborting an unrelated turn.

On "why not json_repair here": repair already runs at execution time and writes its result back to history, so anything still unparseable at format time either defeated repair (nothing to recover) or arrived through history the framework never executed. Repairing it there would be the exact recovery #4974 declined.

The same unguarded pattern lives in a couple of other spots on live tool output (browser_agent.py) and in a test assertion helper (run_result.py). Glad to fold those into separate follow-ups if useful.