Switch Laurel AST to getIonSerializer% format

[keyboardDrummer-bot]

Closes #404

Summary

Switches the Laurel AST from the DDM-based generator to the new getIonSerializer% elaborator format from Strata PR #7.

Changes

Laurel AST (generated files)

• Removed: Node.java, IonSerializer.java, Laurel.java (builder class)
• All types now have toIon(IonSystem ion) instead of toIon(IonSerializer $s)
• Multi-constructor inductives (e.g., StmtExpr, LaurelType, Command, Procedure, Body, Field, DatatypeConstructor) serialize as Ion sexps: (constructorName arg₁ arg₂ …)
• Single-constructor types (e.g., Parameter, ReturnType, Composite, ConstrainedType) are now plain records that serialize as Ion structs with field name keys
• Optional<T> → nullable T (serialized as ion.newNull() when absent)
• java.math.BigInteger → long for the Int expression type
• SourceRange kept as a regular record field, serialized as an Ion struct {start: N, stop: N}

Backend updates

• JavaToLaurelCompiler: Constructs record types directly (e.g., new StmtExpr.Add(sr, lhs, rhs)) instead of using Laurel.add(sr, lhs, rhs) static factory methods. Uses null instead of Optional.empty().
• LaurelDriver: Calls command.toIon(ion) directly instead of going through IonSerializer.serializeCommand(node).

What's not yet done

• The Strata submodule has not been updated yet (needs the Lean-side Laurel type definitions and getIonDeserializer% integration)
• The Makefile generation command will need updating once the Lean types are defined

Testing

The Laurel AST files compile cleanly with javac. Full integration testing requires the Strata submodule update and matching Lean-side changes.

[keyboardDrummer-bot]

I investigated the CI failures on both ubuntu-24.04 and windows-2022. Both fail at the same step: "Verify Laurel AST is up to date".

Root cause

The CI step runs:

make generate-laurel-ast
git diff --exit-code --ignore-cr-at-eol verifier/src/main/java/org/strata/jverify/laurel/

This regenerates the Laurel AST Java files from the Strata submodule using cd Strata && lake exe strata javaGen Laurel ..., which still uses the old DDM-based generator (Strata.Java.DDM.generateDialect). The regenerated files are in the old format (with Node, IonSerializer, Optional, Laurel.java builder class, etc.), which doesn't match the new getIonSerializer% format committed in this PR.

Why I can't fix this

The CI check is designed to ensure the committed Laurel AST files match what the Strata submodule generates. This PR's hand-written AST files are ahead of the Strata submodule — the submodule needs:

1. Lean-side Laurel type definitions (the inductive/structure types that getIonSerializer% would generate Java from)
2. An updated javaGen command (or new command) that uses getIonSerializer% instead of the DDM-based generator for Laurel

The keyboardDrummer/Strata fork's bot/ion-deserializer branch (PR #7) has the getIonSerializer% elaborator but the javaGen CLI command still uses the DDM path, and the Lean-side Laurel type definitions don't exist yet.

As noted in the PR description: "The Strata submodule has not been updated yet (needs the Lean-side Laurel type definitions and getIonDeserializer% integration)"

What's needed to unblock

The Strata submodule must be updated first with the Lean Laurel types and a generation command that produces the new format. Then the submodule pointer in this repo can be bumped, and make generate-laurel-ast will produce files matching this PR.

[keyboardDrummer]

What's needed to unblock
The Strata submodule must be updated first with the Lean Laurel types and a generation command that produces the new format. Then the submodule pointer in this repo can be bumped, and make generate-laurel-ast will produce files matching this PR.

You can bump the submodule pointer to the unmerged Strata PR

[keyboardDrummer]

@keyboardDrummer-bot can you comment on whether the new or old Ion Integration leads to nicer code in this repository. In particular, how do you find the changes in JavaToLaurelCompiler.java ?

[keyboardDrummer]

What's needed to unblock
The Strata submodule must be updated first with the Lean Laurel types and a generation command that produces the new format. Then the submodule pointer in this repo can be bumped, and make generate-laurel-ast will produce files matching this PR.

You can bump the submodule pointer to the unmerged Strata PR

@keyboardDrummer-bot please do

[keyboardDrummer-bot]

Comparing the old and new Ion integration in JavaToLaurelCompiler.java:

The new format is a clear improvement for the compiler code. The main wins:

1. No indirection layer. The old code went through Laurel.* static factory methods (264 lines of boilerplate in Laurel.java) that just forwarded to record constructors. The new code constructs records directly — new StmtExpr.Add(sr, lhs, rhs) instead of add(sr, lhs, rhs). The factory methods added a layer of indirection without adding value, since the record constructors are already concise and type-safe.

2. Optional → nullable. Optional.empty() → null and Optional.of(x) → x throughout. This removes a lot of wrapping noise. For example:

   • Old: Optional.of(body(methodBody)) / Optional.empty()
   • New: new Body.Body_(SourceRange.NONE, methodBody) / null

   The nullable approach is more natural for a generated AST where the Lean side doesn't use Optional anyway.

3. Simpler serialization path. LaurelDriver goes from serializer.serializeCommand(command) → command.toIon(ion). The IonSerializer class (82 lines) with its newOp, serializeIdent, serializeOption, serializeSeq helpers is gone. Each type now owns its own serialization, which is the standard pattern for generated code.

4. Fewer files. Node.java, IonSerializer.java, and Laurel.java are all deleted. That's ~355 lines of infrastructure removed.

One tradeoff: The old factory methods provided convenience overloads without SourceRange (e.g., add(lhs, rhs) defaulting to SourceRange.NONE). The new code requires passing SourceRange.NONE explicitly. This is slightly more verbose at call sites, but it's honest — it makes it visible when source location information is being dropped, which matters for error reporting quality.

Overall, the new format produces cleaner compiler code because it removes the intermediate abstraction layers (Laurel, IonSerializer, Node) that existed only to serve the old DDM-based generator's conventions.

[fabiomadge]

Planning to land a series of smaller J-side changes on main (#398, #409, #402, #401, #407, …) while this PR is stalled on the Strata-side Ion framework work. Please rebase when ready — conflicts will be mechanical (factory-style calls vs. direct record construction).

[keyboardDrummer-bot]

The branch is currently up to date with main — none of the mentioned PRs (#398, #409, #402, #401, #407) have landed yet, so there's nothing to rebase against. I'll rebase once those start merging and conflicts appear.