[Experimental] Classify each SMT-query#966
Draft
jcp19 wants to merge 11 commits into
Draft
Conversation
…iming Adds `ProofQueryKind` (5 categories: Consistency, Heap, FunctionalCorrectness, Axiomatization, PathInfeasibility) and extends `Decider.assert`, `Decider.check`, and `Decider.checkSmoke` to accept `kind`, `pos`, and `member` parameters with defaults so all existing call sites compile unchanged. When `--recordProofQueries` is enabled, each SMT query is recorded into a thread-safe `ProofQueryCollector` with its kind, member name, source position, wall-clock duration (ms), and success flag. Results can be exported to a CSV file via `--proofQueryCsvFile <path>`. All ~59 call sites across rules, resources, and decider have been updated to pass the appropriate kind, source position, and current member name. https://claude.ai/code/session_01Guw2LiiZWESXZ5toNLpbQ8
- Evaluator.scala: ePerm is Option[ast.Exp], use .map(_.pos).getOrElse(NoPosition) - QuantifiedChunkSupport.scala: resource not in scope in assertReadPermission, use permsExp.map(_.pos).getOrElse(NoPosition) instead https://claude.ai/code/session_01Guw2LiiZWESXZ5toNLpbQ8
…x duration formatting - --recordProofQueries now takes the output CSV path directly; recording is enabled iff the option is provided (removes the separate --proofQueryCsvFile flag) - Duration values in the CSV are now formatted as fixed-point decimal with 3 decimal places instead of scientific notation https://claude.ai/code/session_01Guw2LiiZWESXZ5toNLpbQ8
…o State ConsumptionResult.apply, NonQuantifiedPropertyInterpreter, and the QCS heuristic/findChunk functions did not receive a State and so always emitted member = None in recorded queries. Fix by: - Adding member: Option[String] = None to ConsumptionResult.apply and its callers - Adding member to NonQuantifiedPropertyInterpreter's constructor and all 5 instantiation sites (ChunkSupporter, MoreCompleteExhaleSupporter, StateConsolidator ×2, QuantifiedChunkSupport) - Adding member to qpAppChunkOrderHeuristics, singleReceiverChunkOrderHeuristic, and findChunk (trait + impl) in QuantifiedChunkSupport; threading it through StateConsolidator.singleMerge → findMatchingChunk → findChunk https://claude.ai/code/session_01Guw2LiiZWESXZ5toNLpbQ8
…nd annotation
Callers can pass description = Some("...") at specific call sites for extra
clarity; all existing call sites default to None.
https://claude.ai/code/session_01Guw2LiiZWESXZ5toNLpbQ8
jcp19
changed the title
- deciderAssert: do not record queries where isKnownToBeTrue short-circuits (no prover call was made, so timing and results are meaningless noise) - ConsumptionResult: derive pos from the permission expression argument - qpAppChunkOrderHeuristics: add pos parameter, thread resource.pos from caller - singleReceiverChunkOrderHeuristic: add pos parameter, thread resourceAccess.pos from caller Three internal helpers (findChunkWithProver, NonQuantifiedPropertyInterpreter.buildCheck, QuantifiedChunkSupport.findChunk) still emit NoPosition because they operate at the pure term level with no AST resource node in scope. https://claude.ai/code/session_01Guw2LiiZWESXZ5toNLpbQ8
findChunk (ChunkSupportRules trait + chunkSupporter impl) and the private findChunkWithProver were the only remaining call paths without member or pos. All five callers have State in scope and can supply both: - consumeGreedy: passes s.currentMember.map(_.name) - lookupGreedy: passes member + resource.pos - MoreCompleteExhaleSupporter (x2): passes member + resource.pos - StateConsolidator.findMatchingChunk: passes the already-threaded member findChunkWithProver now also passes kind = ProofQueryKind.Heap, eliminating the remaining Consistency-default misclassification for alias checks. https://claude.ai/code/session_01Guw2LiiZWESXZ5toNLpbQ8
Populate description: Option[String] at every non-trivial decider.assert / check / checkSmoke call site so CSV rows are self-explanatory without consulting the codebase. 60 call sites across 14 files: - PermissionSupporter, HavocSupporter: permission consistency (non-neg, positive, injectivity) - QuantifiedChunkSupport: QP produce/consume non-neg & injectivity; QP tookEnough, chunkDepleted, chunkUnused; QP chunk-ordering heuristics (singleton alias, invertible alias, receiver-forall match, singleton arg equality) - ChunkSupporter: read-only read, chunk-depletion, chunk-has-some-permission, zero-permission retry, lookup positivity, smoke checks, alias-via-prover - MoreCompleteExhaleSupporter: MCE alias filter, argument equality, smoke check, total-permission-positive (w/ and w/o snap), read-unneeded, candidate-positivity, split-time depletion/enough, snap-condition, sufficient-permission, took-some - Evaluator: sequence index/update bounds checks (with retry variants), map key in domain, non-zero divisor, perm-expression non-negative, quantifier smoke - Consumer, Producer, HeapSupporter, Brancher, Executor, MagicWandSupporter: user assertions, path infeasibility, heap consistency, branch feasibility, smoke checks, magic wand - ConsumptionResult, NonQuantifiedPropertyInterpreter: permission consumed, chunk property condition All changes are purely additive named parameters; no behavior change. https://claude.ai/code/session_01Guw2LiiZWESXZ5toNLpbQ8
The singleReceiverChunkOrderHeuristic call in execFieldAssign was using the default member=None and pos=NoPosition, producing unidentified heap-kind queries in the CSV output. https://claude.ai/code/session_01Guw2LiiZWESXZ5toNLpbQ8
…sistency Dropping the default forces every call site to state its intent explicitly, making it easy to audit classification decisions at a glance. kind is moved before timeout in assert so that the required parameter precedes the optional one (Scala 2 restriction). All existing call sites already pass kind as a named argument so no functional change. https://claude.ai/code/session_01Guw2LiiZWESXZ5toNLpbQ8
jcp19
commented
Apr 24, 2026
Comment on lines
+125
to
+127
| if (verifier.decider.check(conditionTerm, Verifier.config.checkTimeout(), | ||
| kind = ProofQueryKind.Heap, member = member, | ||
| description = Some("chunk property condition"))) { |
Contributor
Author
There was a problem hiding this comment.
TODO: improve description
Comment on lines
+200
to
+201
| if (v.decider.checkSmoke(member = sOut.currentMember.map(_.name), | ||
| description = Some("smoke check: magic wand"))) { |
| v1.decider.assert(pNeeded === NoPerm, | ||
| kind = ProofQueryKind.Heap, pos = resource.pos, | ||
| member = s1.currentMember.map(_.name), | ||
| description = Some("MCE sufficient permission (split)")) { |
| // TODO: Replace by QP-analogue of permissionSupporter.assertNotNegative | ||
| v.decider.assert(nonNegTerm) { | ||
| v.decider.assert(nonNegTerm, | ||
| kind = ProofQueryKind.Consistency, pos = forall.pos, |
Contributor
Author
There was a problem hiding this comment.
Probably should be of type Heap
| // TODO: Replace by QP-analogue of permissionSupporter.assertNotNegative | ||
| v.decider.assert(nonNegTerm) { | ||
| v.decider.assert(nonNegTerm, | ||
| kind = ProofQueryKind.Consistency, pos = resource.pos, |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is not meant to be merged or reviewed as is. Architecturally, the way he record the per-query data might not be the most adequate; perhaps the symbex logger would be a better place for that.
This PR introduces support for recording and classifying SMT queries (assertions and checks) issued during verification. Each query may be classified as one of the following:
Consistency: Well-formedness checks (permissions, injectivity)Heap: Heap access and permission correctnessFunctionalCorrectness: User-visible proof obligations (pre/postconditions, assertions)Axiomatization: Axiom consistency checksPathInfeasibility: Branch feasibility and smoke checksUnknown: Unclassified queriesThis feature is enabled by the CLI option
--recordProofQueries, which takes the path to the file storing the results of the analysis as an argument.Notable Implementation Details
Verifier.config.recordProofQueriesis definedNoPosition