[babaliauskas]

Summary

Two bugs found while reading a real planner-suite run (Sonnet 4.6 → 5.5) that came back inconclusive.

1. semantic / llm_judge scored narration as an answer.
Without tool-result fixtures the replay stops at the first tool call, so any text a model writes beside that call is narration ("Let me look up your notes!"), not the answer a user would read. The source narrated beside 4 of its calls, the target did not, and both evaluators scored each as a −1.0 loss: 8 of the 20 regressions behind the verdict.

• New ToolTrace.ends_in_tool_calls (last round has calls).
• _score_one returns no record for semantic / llm_judge when either side ends in a tool call; coverage books the pair as not applicable. structural and the tool evaluators are unchanged.

Re-scoring the same run: equivalence 72.2% → 83.9% (52/62), regression rate 27.8% → 16.1%. On a first-round-only agent suite the two text evaluators now honestly report Nothing measured (the existing recommendation explains it), instead of fabricated drops. captures.rounds: all is the way to get them real answers to compare.

2. The evaluator table printed a skipped test's placeholders.
A uniform −1.000 drop rendered as 0.00 negligible [0.00, 0.00]. When test == "skipped", Effect and 95% range now read —, and the executive summary drops the magnitude word. Effect is now shown signed (-0.44, not 0.44), so it sits inside its own range [-0.70, -0.22]; the slice table matches.

Docs (DOCS.md, llms-full.txt, docs/evaluators.md, docs/methodology.md) and CHANGELOG.md updated.

Test plan

• New failing-first tests: TestToolTraceRounds (ends_in_tool_calls), TestTextEvaluatorsSkipUnfinishedTurns (both/either side, both answered, structural unaffected), report tests for untested rows and signed effect.
• ruff check, ruff format --check, mypy --strict clean; full pytest passes apart from test_online_sync_says_it_ignores_limits_it_cannot_read, which fails identically on main locally on macOS (Rich wraps the long temp path), CI-green on Linux.
• Re-scored the real planner run with this branch: semantic/judge rows gone, report shows — / signed effects.

🤖 Generated with Claude Code