Skip to content

Brittle exact-string assertions in tests will break on any formatting change #62

Description

@devinoldenburg

Location

Multiple test files:

tests/view.test.mjs:29-36 — exact multiline string match
tests/view.test.mjs:43 — exact lines[1].length, 24 (hardcoded sparkWidth)
tests/view.test.mjs:46assert.ok(lines[1].endsWith("█")) (assumes max glyph)
tests/view.test.mjs:87,119-120includes("150") / includes("146") substring matches
tests/integration.test.mjs:78-80 — exact multiline string match

Examples

// view.test.mjs:29-36
assert.equal(renderText(v), [
  "TPS  200 tok/s",
  "last 200 tok/s  ttft 500ms · −4s wait",
  "avg  200 tok/s  peak 200"
].join("\n"));

// integration.test.mjs:78-80
assert.equal(renderText(v),
  ["TPS  200 tok/s", "last 200 tok/s  ttft 500ms · −4s wait", "avg  200 tok/s  peak 200"].join("\n"));

Problem

Any change to spacing, punctuation, ordering, or formatting will break these tests even if the change is intentional and correct. This includes:

  • Changing " − " to " -- " (different dash characters)
  • Adding or removing a trailing space
  • Changing peak 200 to peak 200 tok/s
  • Changing sparkWidth default from 24 to another value
  • Changing formatting of milliseconds (stripping ms suffix)

The substring matches (includes("150")) are even more fragile — they match unintended text and break on formatting changes like fmtRate(146.2) rounding differently.

Expected Behavior

  • Use structural assertions: check line counts, segment presence, tone assignments, and numeric ranges
  • Or use regex with tolerance for whitespace
  • Or define a stable serialization format separate from human-facing rendering

Impact

Any formatting change (even intentional UX improvements) requires updating 5+ tests. High maintenance cost.

Severity

Low — affects only developer experience, not end users.

Related

These tests are also duplicated across view.test.mjs and integration.test.mjs, testing the same rendering pipeline with nearly identical assertions.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions