fix(langchain): mark handled tool errors as errors

[vismaytiwari]

What does this PR do?

Fixes handled LangChain tool errors being recorded as successful tool observations.

When a LangChain tool is configured with handle_tool_error=True, LangChain can call on_tool_end instead of on_tool_error, while still passing a ToolMessage with status="error". The callback handler previously ended the observation without checking that status, so Langfuse did not mark the observation as an error.

This updates on_tool_end to inspect ToolMessage.status and set the observation level to ERROR when the message status is "error". It also stores the handled error content as the observation status message.

Fixes langfuse/langfuse#12867

Type of change

• Bug fix
• New feature
• Breaking change
• Refactor
• Documentation update
• Tooling, CI, or repo maintenance

Verification

List the main commands you ran:

uv run --frozen pytest tests/unit/test_langchain.py::test_handled_tool_error_marks_observation_error
uv run --frozen pytest tests/unit/test_langchain.py
uv run --frozen ruff check .
uv run --frozen mypy langfuse --no-error-summary
uv run --frozen pytest -n auto --dist worksteal tests/unit

I also reproduced the issue locally before the fix. Before this change, a handled tool error with ToolMessage(status="error") had no observation level or status message. After the change, the same path records level=ERROR and status_message from the handled tool error content.

Checklist

• I self-reviewed the diff using code_review.md.
• I added or updated tests for behavior changes.
• I updated docs, examples, or .env.template if needed — N/A, this is covered by the LangChain callback unit test and does not change documented setup or examples.
• I did not hand-edit generated files; if generated files changed, I used the upstream regeneration path.
• I did not commit secrets or credentials.

Greptile Summary

This PR fixes a bug where LangChain tool errors handled via handle_tool_error=True were silently recorded as successful observations. LangChain routes these through on_tool_end with a ToolMessage(status="error") rather than calling on_tool_error, so the previous code never set an error level.

• on_tool_end now widens its output parameter from str to Any and inspects the value: if it is a ToolMessage with status="error", the observation is updated with level="ERROR" and status_message set from the message content.
• A focused unit test is added that drives on_tool_start → on_tool_end(ToolMessage(status="error")) and asserts the resulting span has the correct level, status message, and preserved output fields.

Confidence Score: 4/5

The change is a small, well-targeted fix with a direct unit test. It does not alter any existing code paths — only the new ToolMessage branch is affected.

The fix is correct and the test covers the new path end-to-end. The only open question is whether cost_details should be zeroed in the new error branch to match on_tool_error behavior; if it should, users could see non-zero costs attributed to errored tool spans until that is addressed.

langfuse/langchain/CallbackHandler.py — specifically whether the new error branch in on_tool_end should zero out cost_details to match on_tool_error

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant LC as LangChain
    participant CB as CallbackHandler
    participant OB as LangfuseObservation

    Note over LC,OB: Normal tool success path
    LC->>CB: on_tool_start(tool_name, input)
    CB->>OB: create tool observation
    LC->>CB: on_tool_end(output: str)
    CB->>OB: "update(output=output).end()"

    Note over LC,OB: Handled tool error path (handle_tool_error=True) NEW
    LC->>CB: on_tool_start(tool_name, input)
    CB->>OB: create tool observation
    LC->>CB: "on_tool_end(ToolMessage(status=error, content=...))"
    CB->>CB: "check isinstance(output, ToolMessage) AND status==error"
    CB->>OB: "update(output=msg, level=ERROR, status_message=msg.content).end()"

    Note over LC,OB: Unhandled tool error path (pre-existing)
    LC->>CB: on_tool_start(tool_name, input)
    CB->>OB: create tool observation
    LC->>CB: on_tool_error(exception)
    CB->>OB: "update(level=..., status_message=...).end()"

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant LC as LangChain
    participant CB as CallbackHandler
    participant OB as LangfuseObservation

    Note over LC,OB: Normal tool success path
    LC->>CB: on_tool_start(tool_name, input)
    CB->>OB: create tool observation
    LC->>CB: on_tool_end(output: str)
    CB->>OB: "update(output=output).end()"

    Note over LC,OB: Handled tool error path (handle_tool_error=True) NEW
    LC->>CB: on_tool_start(tool_name, input)
    CB->>OB: create tool observation
    LC->>CB: "on_tool_end(ToolMessage(status=error, content=...))"
    CB->>CB: "check isinstance(output, ToolMessage) AND status==error"
    CB->>OB: "update(output=msg, level=ERROR, status_message=msg.content).end()"

    Note over LC,OB: Unhandled tool error path (pre-existing)
    LC->>CB: on_tool_start(tool_name, input)
    CB->>OB: create tool observation
    LC->>CB: on_tool_error(exception)
    CB->>OB: "update(level=..., status_message=...).end()"

Loading

Prompt To Fix All With AI

Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 1
langfuse/langchain/CallbackHandler.py:1114-1123
The new error branch in `on_tool_end` doesn't set `cost_details={"total": 0}`, while the parallel `on_tool_error` handler does. Without this, any accumulated cost tracking may still be attributed even when the tool ended with a handled error. If the intent is that handled errors should behave consistently with unhandled ones in terms of cost reporting, `cost_details` should be zeroed here too.

```suggestion
                if (
                    isinstance(output, ToolMessage)
                    and getattr(output, "status", None) == "error"
                ):
                    update_kwargs["level"] = "ERROR"
                    update_kwargs["status_message"] = (
                        output.content
                        if isinstance(output.content, str)
                        else str(output.content)
                    )
                    update_kwargs["cost_details"] = {"total": 0}
```

_{Reviews (1): Last reviewed commit: "fix(langchain): mark handled tool errors..." | Re-trigger Greptile}

Greptile also left 1 inline comment on this PR.

[CLAassistant]

All committers have signed the CLA.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[vismaytiwari]

I signed the CLA. The GitHub Actions workflows are awaiting maintainer approval.

[hassiebp]

Thanks for your contribution, @vismaytiwari !