Skip to content

fix(report): tint a target trace red only when its tool calls differ - #66

Merged
babaliauskas merged 1 commit into
mainfrom
fix/trace-tone
Oct 22, 2026
Merged

babaliauskas merged 1 commit into
mainfrom
fix/trace-tone

Conversation

@babaliauskas

Copy link
Copy Markdown
Collaborator

What this changes

A follow-up to #65 from looking at a real converter report.

  • The target's trace box is red only when its tool calls differ from the source's. A card flagged for its wording alone (semantic similarity) showed both sides making the same convert_currency call, with the target's in a red box. That read as a tool failure when there wasn't one. "Differ" means the pair's tool diff is not empty: same rule as the card's "Tool diffs" list, so the box and the list agree. Applies to imported agent traces too (their trace_diff).
  • A target's "(no tool calls)" is red only when the source did call tools. When neither side called a tool, it was red anyway.
  • A within-threshold finding sits in a neutral notice. The plain notice style uses the warning tint, so a "Within threshold" card still looked like a warning.

Template and CSS only (report.html.j2, report.css); no data or report.json change.

Test plan

  • New tests:

    • the target trace is red only when its calls differ (same call vs a different call);
    • "(no tool calls)" isn't flagged as missing when neither side called a tool, and is when the target dropped one;
    • a within-threshold finding uses notice-neutral and regressed cards don't.

    Each failed before the change.

  • make ci: ruff, ruff format, mypy --strict, and pytest (3396 passed, 95.3% coverage)

  • pre-commit run --all-files

  • Re-rendered the real converter run and checked headless-Chromium screenshots: a regressed card with matching calls and an opened within-threshold card are both neutral where nothing differs

🤖 Generated with Claude Code

A card flagged for its wording alone showed both sides making the same tool call with the target's in a red box, which read as a tool failure. The box and a target's "(no tool calls)" are now red only when the pair's tool diff is non-empty, and a within-threshold finding sits in a neutral notice instead of the warning tint.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@babaliauskas
babaliauskas merged commit 1547e65 into main Oct 22, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant