test(mutation): mutate sql_guard live, not just declared#109
Merged
Conversation
The mutation gate (scripts/mutation_report.py) ran only the duckdb-free sdk/agentflow/retry.py; every serving module was declared-only, documented as blocked by "mutmut x duckdb". The real blocker was narrower: mutmut's trampoline asserts a module name does not start with "src.", and the serving unit tests also drag the duckdb-backed query engine into mutmut's mutants/ workspace. sql_guard imports only sqlglot, so it can be mutated as a top-level `serving` package (like retry.py is mutated as agentflow.retry, not src.*) against a narrow duckdb-free test. Add that path: - mutation_report.py: serving-target workspace handling (copy src/serving -> serving, also_copy = serving/config/scripts) + sql_guard in MODULE_TARGETS (threshold 0.90). - tests/unit/test_sql_guard_mutation.py: narrow duckdb-free test, dual-context import (serving under mutmut, src.serving under pytest). Every pytest.raises pins the message so error-text mutants die; the Anonymous-vs-typed forbidden- function cases both exercise the .lower() casing (a surviving mutant there is a denylist bypass -- mutation testing surfaced that gap). - Correct the now-stale "serving cannot be mutated / duckdb limitation" notes in pyproject [tool.mutmut] and test_mutmut_policy.py. Verified in a Linux venv (mutmut 3.6): sql_guard scores 44/44 mutants killed (100%). The remaining serving modules stay declared-only until each gets its own duckdb-free unit test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
DORA Metrics
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The mutation gate (
scripts/mutation_report.py) mutated only the duckdb-freesdk/agentflow/retry.py. Every serving module was declared-only in[tool.mutmut], documented as blocked by "mutmut x duckdb" — so a survivingmutant in sql_guard's NL→SQL denylist would have gone unnoticed (the policy was
decorative for the serving surfaces).
The real blocker was narrower than duckdb: mutmut's trampoline asserts a module
name does not start with
src., and the serving unit tests also pull theduckdb-backed query engine into mutmut's
mutants/workspace.Change
sql_guardimports onlysqlglot, so it can be mutated as a top-levelservingpackage (exactly howretry.pyis mutated asagentflow.retry, notsrc.*) against a narrow duckdb-free test:mutation_report.py: serving-target workspace handling (copysrc/serving→serving;also_copy = serving/config/scripts) +sql_guardin
MODULE_TARGETS(threshold 0.90).tests/unit/test_sql_guard_mutation.py: narrow duckdb-free test,dual-context import (
servingunder mutmut,src.servingunder pytest). Everypytest.raisespins the message so error-text mutants die; theAnonymous-vs-typed forbidden-function cases both exercise the
.lower()casing — a surviving mutant there is a denylist bypass, which mutation testing
surfaced, so the test is stronger than the original suite.
[tool.mutmut]andtest_mutmut_policy.py.Verification
Ran the real
mutation.ymlworkflow on this branch (workflow_dispatch, Python3.11 — the same job the weekly gate runs):
sql_guard now has live mutation coverage (44/44 killed). The remaining
serving modules stay declared-only until each gets its own duckdb-free unit
test (the blocker is the test import chain, not the module).
🤖 Generated with Claude Code