fix: mutation testing#14848
Draft
zerosnacks wants to merge 2 commits into
Draft
Conversation
…compound assignment mutation, reject incompatible flags - runner: thread FilterArgs and --isolate through MutationRunConfig into compile_and_test_inner; rebuild ProjectPathsAwareFilter against the per-mutant temp config so --match-test/--match-contract/--match-path work and mutants exercise the same test set/execution model as baseline - mod: add runtime_context_digest folded into the per-file cache key so cached results aren't reused when filters, isolation, fork URL/block, sender, or initial_balance change - orchestrator: compute runtime_context_digest once per run and bind it to every MutationHandler - binary_op_mutator: stop mutating compound assignments (a += b would silently be rewritten to a - b dropping the assignment); add regression tests - cmd/test: bail when --mutate is combined with --list/--debug/ --flamegraph/--flamechart/--junit instead of silently mixing modes - fs_permissions guard now narrowed to permissions whose path can reach symlinked dependency trees (lib/node_modules/dependencies) Amp-Thread-ID: https://ampcode.com/threads/T-019e4567-e7ca-717e-bcc0-bb67a3667c4d Co-authored-by: Amp <amp@ampcode.com>
Blockers
- filter parity: pass materialized baseline filter (filter.args().clone())
into MutationRunConfig so positional 'forge test <path>' and --rerun
reach mutant runs too; raw self.filter dropped them silently
- bail when baseline matched zero tests; without it every compileable
mutant was reported as 'Alive'
- negative literal mutation: add OwnedLiteral::NegatedNumber(U256)
formatted as '-{val}'; the old Number(-*val) wrapped via two's
complement and produced wrong-source mutants
- mutator test harness: wrap snippets in valid Solidity, panic on parse
failure, and check emitted replacement text. Fixed 6 stale test
expectations and 1 unmatched delegate-pattern case the old vacuous
harness was hiding
Mediums
- survived-span resume: load retrieve_survived_spans BEFORE mutant
generation/cached load and filter cached mutants through
should_skip_span so cross-run adaptive skipping actually works
- runtime context digest: hash the full serialized EvmOpts (networks,
env, gas-limit toggles, fork, sender, balance, ...) instead of a
hand-picked 4-field subset that missed several mutation-affecting
knobs
- deterministic output: BTreeMap for JSON survived_mutants with sorted
entries; reporter sorts survived list by (path, line, col, span,
mutation); persisted results_vec sorted by span
- --mutate-contract: intersect with explicit --mutate <paths> instead of
silently overriding them; explicit paths are now respected
- workspace: copy 'script/' when present and distinct from src/test so
projects that import script-side helpers don't see false Invalid
mutants
Amp-Thread-ID: https://ampcode.com/threads/T-019e4567-e7ca-717e-bcc0-bb67a3667c4d
Co-authored-by: Amp <amp@ampcode.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Numerous findings