Skip to content

test(mutation): mutate sql_guard live, not just declared#109

Merged
brownjuly2003-code merged 1 commit into
mainfrom
test/mutate-sql-guard-serving
Jun 29, 2026
Merged

test(mutation): mutate sql_guard live, not just declared#109
brownjuly2003-code merged 1 commit into
mainfrom
test/mutate-sql-guard-serving

Conversation

@brownjuly2003-code

Copy link
Copy Markdown
Owner

What

The mutation gate (scripts/mutation_report.py) mutated only the duckdb-free
sdk/agentflow/retry.py. Every serving module was declared-only in
[tool.mutmut], documented as blocked by "mutmut x duckdb" — so a surviving
mutant in sql_guard's NL→SQL denylist would have gone unnoticed (the policy was
decorative for the serving surfaces).

The real blocker was narrower than duckdb: mutmut's trampoline asserts a module
name does not start with src., and the serving unit tests also pull the
duckdb-backed query engine into mutmut's mutants/ workspace.

Change

sql_guard imports only sqlglot, so it can be mutated as a top-level
serving package (exactly how retry.py is mutated as agentflow.retry, not
src.*) against a narrow duckdb-free test:

  • mutation_report.py: serving-target workspace handling (copy
    src/servingserving; also_copy = serving/config/scripts) + sql_guard
    in MODULE_TARGETS (threshold 0.90).
  • tests/unit/test_sql_guard_mutation.py: narrow duckdb-free test,
    dual-context import (serving under mutmut, src.serving under pytest). Every
    pytest.raises pins the message so error-text mutants die; the
    Anonymous-vs-typed forbidden-function cases both exercise the .lower()
    casing — a surviving mutant there is a denylist bypass, which mutation testing
    surfaced, so the test is stronger than the original suite.
  • Correct the now-stale "serving cannot be mutated" notes in pyproject
    [tool.mutmut] and test_mutmut_policy.py.

Verification

Ran the real mutation.yml workflow on this branch (workflow_dispatch, Python
3.11 — the same job the weekly gate runs):

Overall: killed=59, survived=5, total=64
retry.py:     score=75.0%  threshold=75% (killed=15, survived=5)
sql_guard.py: score=100.0% threshold=90% (killed=44)
Mutation scores meet thresholds

sql_guard now has live mutation coverage (44/44 killed). The remaining
serving modules stay declared-only until each gets its own duckdb-free unit
test (the blocker is the test import chain, not the module).

🤖 Generated with Claude Code

The mutation gate (scripts/mutation_report.py) ran only the duckdb-free
sdk/agentflow/retry.py; every serving module was declared-only, documented as
blocked by "mutmut x duckdb". The real blocker was narrower: mutmut's trampoline
asserts a module name does not start with "src.", and the serving unit tests
also drag the duckdb-backed query engine into mutmut's mutants/ workspace.

sql_guard imports only sqlglot, so it can be mutated as a top-level `serving`
package (like retry.py is mutated as agentflow.retry, not src.*) against a narrow
duckdb-free test. Add that path:

- mutation_report.py: serving-target workspace handling (copy src/serving ->
  serving, also_copy = serving/config/scripts) + sql_guard in MODULE_TARGETS
  (threshold 0.90).
- tests/unit/test_sql_guard_mutation.py: narrow duckdb-free test, dual-context
  import (serving under mutmut, src.serving under pytest). Every pytest.raises
  pins the message so error-text mutants die; the Anonymous-vs-typed forbidden-
  function cases both exercise the .lower() casing (a surviving mutant there is a
  denylist bypass -- mutation testing surfaced that gap).
- Correct the now-stale "serving cannot be mutated / duckdb limitation" notes in
  pyproject [tool.mutmut] and test_mutmut_policy.py.

Verified in a Linux venv (mutmut 3.6): sql_guard scores 44/44 mutants killed
(100%). The remaining serving modules stay declared-only until each gets its own
duckdb-free unit test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

DORA Metrics

  • Window: last 30 days
  • Branch: main
  • Deployment frequency: 160 total / 37.33 per week
  • Lead time for changes: avg 0.28h / median 0.0h
  • Change failure rate: 58.13% (93/160)
  • MTTR: 0.25h across 3 incident(s)

@brownjuly2003-code brownjuly2003-code merged commit 4beae1f into main Jun 29, 2026
21 checks passed
@brownjuly2003-code brownjuly2003-code deleted the test/mutate-sql-guard-serving branch June 29, 2026 17:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants