Merge functions and procedures in Laurel

[keyboardDrummer]

Builds on:

• Formatting and debugging improvements strata-org/Strata#1115

Functional changes

1. Merge functions and procedures:
   1. Allow postconditions on functions
   2. Allow calling procedures from contracts
   3. Allow transparent procedures IF they have an expressionish body
   4. Allow assertions and assumptions in functions
2. Improve diagnostics related to contracts, using the correct verbiage "precondition" and "postcondition" instead of "assertion"

Implementation

Add these passes:

• [New] EliminateReturnStatements: rewrite return to exit statements, needed for the next pass.
• [New] ContractPass: translate away pre and postconditions entirely by introducing assertion and assumptions at call sites and at procedure starts and ends
• [New] TransparencyPass: for each Core procedure, generate a function with the same signature and name suffixed with $asFunction. If a Core procedure is marked as transparent, attempt to generate a functional version of it, where assertions are erased and all calls are to functional versions. Tie the functional version to the procedure using a free postcondition.
• [Updated] Lift assertions, assumptions and procedure calls when they occur in expressions.

Why can't we have a PR with just the contract or just the transparency pass?

• Just the contract pass means that the preconditions which are currently supported on functions are no longer supported, because assertions and assumptions in functions are not supported.
• Just the transparency pass means that making calls from contracts is no longer supported, since in the procedure world, all calls are to procedures and we can't lift procedure calls in Core contracts

Follow-up work

• Remove the 'functional' keyword from the Laurel grammar
• Create WF proofs for quantifier bodies
• Lift assumptions in expressions to axioms.
• In the transparency phase, if something has no asserts and only calls functions, only create a function and no procedure
• Move the transparency phase so its Core->Core instead of UnorderedCoreWithLaurelTypes -> UnorderedCoreWithLaurelTypes

[keyboardDrummer]

@keyboardDrummer-bot please merge strata-org#1023 and #22 into this PR

[keyboardDrummer-bot]

I've merged both strata-org/Strata#1023 and #22 into this PR. The build passes, but there are test failures.

What was done:

• Resolved merge conflicts between PR Generalize multi-target assign so it supports intertwined declarations and local assignments and field assignments #22's AST refactoring (LocalVariable → Var (.Declare ...) + Assign) and this PR's contract/transparency passes
• Updated ContractPass.lean, EliminateMultipleOutputs.lean, EliminateReturnStatements.lean, InlineLocalVariablesInExpressions.lean, and TransparencyPass.lean to use the new AST
• Applied PR Generalize multi-target assign so it supports intertwined declarations and local assignments and field assignments #22's grammar change (removed opaqueSpec, made ensures/modifies direct fields of procedure/function) and added modifiesWildcard support
• Updated PythonRuntimeLaurelPart.lean to remove opaque keywords (no longer needed with the new grammar)
• Adapted API signatures (Condition type for preconditions, LaurelVerifyOptions vs separate params)
• Merged PR Enable local variables in functions strata-org/Strata#1023's local-variable-in-functions implementation using the new AST (Assign [Declare ...] instead of LocalVariable)

Test failures to investigate:
Many Laurel fundamental tests are failing (T4b, T5, T6, T8, T10, T12, T14, T15, T17, T18, T22, and others). These are likely caused by the contract pass changing verification behavior (inserting assert/assume for contracts). Some Python tests also fail with "block expression should have been lowered in a separate pass".

[keyboardDrummer]

@keyboardDrummer-bot can you investigate why ~/repos/Strata/StrataTest/Languages/Python$ ./run_py_analyze.sh --filter test_bug_finding_unreachable fails? What is the difference in the emitted Core in formatting-and-debugging-improvements and issue-924-contract-and-proof-pass ? Use the --keep-all-files flag to emit intermediate programs so you can easily find the produces Core.

[keyboardDrummer-bot]

Investigation: test_bug_finding_unreachable failure

Symptom

On issue-924-contract-and-proof-pass, all assertions in the test become ❓ unknown (inconclusive) instead of being properly verified/falsified.

Root Cause

The transparency pass converts functions (like PNEq, PGt, PLt, Any_to_bool) into procedures with isFunctional := false, but keeps their body as .Transparent expr. When the Laurel-to-Core translator processes a procedure with a Transparent body, it calls translateStmt bodyExpr. The body expression (e.g., from_bool(normalize_any(v) != normalize_any(|v'|))) ends up in the catch-all case of translateStmt:

| _ =>
    -- Expression in statement position: preserve as an unused variable init
    exprAsUnusedInit stmt md

This creates var $unused_N := expr instead of assigning the expression to the output variable (result). As a result, after procedure inlining, the output variable is never assigned, making it unconstrained. The solver can't determine anything about the results of these calls.

Comparison of emitted Core

Base branch (formatting-and-debugging-improvements): PNEq is an inline function that directly returns the expression:

inline function PNEq (v : Any, |v'| : Any) : Any {
  from_bool(!(normalize_any(v) == normalize_any(v')))
}

PR branch (issue-924-contract-and-proof-pass): PNEq is a procedure where the return value goes into $unused_54:

procedure PNEq (v : Any, |v'| : Any, out result : Any)
spec {
  free ensures [postcondition]: result == PNEq$asFunction(v, v');
} {
  $body: {
    $return: {
      var $cndtn_25 : Any;
      call normalize_any(v', out $cndtn_25);
      var $cndtn_26 : Any;
      call normalize_any(v, out $cndtn_26);
      var $unused_54 : Any := from_bool(!($cndtn_26 == $cndtn_25));  // ← should be `result :=`
    }
  }
};

After inlining, PNEq_result_181 is never assigned, so $cndtn_29 := PNEq_result_181 gives an unconstrained value.

Suggested Fix Options

1. In the transparency pass (TransparencyPass.lean): When converting a function to a procedure, change the body from .Transparent expr to .Opaque [] (some (Block [Assign [output] expr])) [] so the expression is explicitly assigned to the output parameter.

2. In EliminateReturnStatements: For procedures with a Transparent body, convert the trailing expression into an explicit assignment to the output variable before wrapping in the $return block.

3. In the Laurel-to-Core translator (LaurelToCoreTranslator.lean): When translating a procedure (not function) with a Transparent body, detect the trailing expression and assign it to the output variable instead of creating a $unused_N variable.

[keyboardDrummer]

In the transparency pass (TransparencyPass.lean): When converting a function to a procedure, change the body from .Transparent expr to .Opaque [] (some (Block [Assign [output] expr])) [] so the expression is explicitly assigned to the output parameter.

@keyboardDrummer-bot let's do that

[keyboardDrummer]

@keyboardDrummer-bot can you figure out why ~/repos/Strata/StrataTest/Languages/Python$ ./run_py_analyze.sh --filter test_bug_finding_unreachable is failing?

[keyboardDrummer-bot]

I found and fixed the root cause of the test_bug_finding_unreachable failure.

Root cause: When the transparency pass promotes a transparent procedure body to opaque (via addFreePostcondition), the body is an expression tree representing the return value (e.g., from_bool(normalize_any(v) != normalize_any(|v'|)) for PNEq). Without an explicit Return wrapper, the EliminateReturnStatements pass cannot assign this expression to the output variable. The Laurel-to-Core translator then treats it as "expression in statement position" and assigns it to a throwaway $unused_N variable, leaving the actual output variable (result) uninitialized.

After inlining, the output variable remains nondet, so the solver can't determine the assertion outcomes → everything comes back as "unknown".

Fix: In addFreePostcondition, when promoting .Transparent body to .Opaque, wrap the body in a Return node (unless it already contains Return statements). This allows EliminateReturnStatements to properly convert it to result := <body>; exit "$return".

The fix also resolves test_class_field_use and test_class_mixed_init which had the same underlying issue.

[keyboardDrummer-bot]

CI Failure Investigation

I investigated the 4 test failures from the latest CI run. Here's what I found:

1. DDM.Integration.Java.TestGen — Environment issue

Missing jar file (ion-java-1.11.11.jar). This is a test environment dependency, not a code issue.

2-4. Languages.Python.AnalyzeLaurelTest, VerifyPythonTest, DictNoneTest — Core type checking error

Root cause: After the transparency pass + contract pass + lift pass pipeline, there are still references to List_len (without the $asFunction suffix) that end up as Core function operations (.op "List_len"), but List_len only exists as a Core procedure (not a function). The Core type checker fails because it can't find List_len in the function table.

How it happens:

• List_len is a transparent function in the Laurel prelude
• The transparency pass creates List_len$asFunction (a Core function) and moves List_len to coreProcedures (a Core procedure)
• The LiftImperativeExpressionsInCore pass should lift List_len(...) calls from expression context to statement-level calls in procedure bodies
• However, somewhere in the pipeline a reference to List_len (not List_len$asFunction) survives in expression context and gets translated as a Core function operation

Possible locations where List_len leaks through:

1. The CallElim pass inlines procedure postconditions as assume statements. If a postcondition expression contains List_len (not $asFunction), it would appear in expression context after inlining.
2. The lift pass doesn't process postconditions of core procedures — only the implementation body. If a non-free postcondition somehow survives with List_len references, it would be translated with isPureContext := true.
3. The PrecondElim pass generates well-formedness procedures from procedure contracts. If a postcondition contains List_len, the generated WF procedure body would reference it.

Additional issue: AnalyzeLaurelTest also reports "Resolution failed: 'Any..isfrom_str' is not defined" which appears to be a separate FilterPrelude dependency tracking issue.

VerifyPythonTest panics with "key is not present in hash table" which is likely a downstream consequence of the same issue.

I was unable to pinpoint the exact location where List_len leaks into expression context. The interaction between the transparency pass, contract pass, lift pass, and Core-level CallElim/PrecondElim passes is complex. A debugging approach would be to emit the intermediate UnorderedCoreWithLaurelTypes after each core pipeline pass and inspect where List_len (without $asFunction) appears in function bodies or postconditions.