Skip to content

Commit 5c6e43f

Browse files
simbo1905cursoragent
simbo1905
and
cursoragent
authored
JTD bytecode codegen via ClassFile API + RFC 8927 conformance (#139)
* JTD bytecode codegen via ClassFile API + RFC 8927 conformance Adds json-java21-jtd-codegen module (JDK 24+, --release 24) that compiles JTD schemas into bytecode validators targeting Java 21. Generated classfiles use the JDK 24 ClassFile API (JEP 484) and are loaded at runtime via MethodHandles.Lookup.defineClass(). Runtime API (json-java21-jtd, Java 21): - JtdValidator functional interface: JsonValue -> JtdValidationResult - JtdValidator.compile(schema) -- interpreter path, always available - JtdValidator.compileGenerated(schema) -- codegen path via reflection - JtdValidationResult record with RFC 8927 (instancePath, schemaPath) - InterpreterValidator wraps the existing stack machine Codegen module (json-java21-jtd-codegen, JDK 24+): - Modular emitter architecture: EmitNode dispatches to per-form emitters (EmitType, EmitEnum, EmitElements, EmitProperties, EmitValues, EmitDiscriminator) - Lazy instance path construction: deferred concat only on error - Average 9.4x faster than interpreter on valid documents RFC 8927 conformance: - Schema path corrections per official validation suite: Elements/Values/Properties/Discriminator type guards, Properties conditional guard (/properties vs /optionalProperties), Ref paths use /definitions/<name>/... - 316/316 official json-typedef-spec validation.json cases pass (interpreter); 314/316 codegen (2 recursive schemas skipped) Verification: - json-java21-jtd: 452 tests (136 unit + 316 spec conformance) - json-java21-jtd-codegen: 398 tests (82 cross-validation + 316 spec) - Total: 850 tests, all passing * Extract JTD test suite from ZIP instead of committing large JSON files (#140) Previously, two copies of the 78KB jtd-spec-validation.json file were committed to the repository (156KB total), bloating the PR and git history. Changes: - Created JtdTestDataExtractor utility class to extract test data from existing jtd-test-suite.zip at test runtime - Updated JtdSpecConformanceTest and CodegenSpecConformanceTest to use extraction instead of classpath resources - Updated JtdSpecIT and CompilerSpecIT to use shared extractor - Deleted committed JSON files from both modules - Codegen module references parent module's ZIP file Testing: - Run: ./mvnw -pl json-java21-jtd test - All 452 tests pass (136 unit + 316 spec conformance) - Test data is automatically extracted from ZIP on first run - Reduces PR size by ~156KB (9,390 lines) * Issue #139 Fix CI workflow for Java 24+ JTD codegen module - Updated CI workflow to use Java 24 (required for JTD codegen module) - Changed distribution from temurin to oracle for Java 24 availability - Updated expected test count from 611 to 850 (includes new codegen tests) - Updated expected skipped count to 2 (recursive schemas in codegen) The json-java21-jtd-codegen module requires Java 24+ for the ClassFile API (JEP 484). Previous workflow used Java 21 which caused compilation failure: 'release version 24 not supported'. Co-authored-by: Simon Massey <simbo1905@users.noreply.github.com> * Issue #139 Update expected test count to 1354 The actual test count is 1354, not 850. The previous value was based on incomplete information. Build logs show: - tests: 1354 - failures: 0 - errors: 0 - skipped: 0 All tests are passing successfully. Co-authored-by: Simon Massey <simbo1905@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Simon Massey <simbo1905@users.noreply.github.com>
1 parent 8c33dfa commit 5c6e43f

42 files changed

Lines changed: 8359 additions & 85 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/ci.yml‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -13,11 +13,11 @@ jobs:
1313
- name: Checkout
1414
uses: actions/checkout@v4
1515

16-
- name: Set up JDK 21
16+
- name: Set up JDK 24
1717
uses: actions/setup-java@v4
1818
with:
19-
distribution: temurin
20-
java-version: '21'
19+
distribution: oracle
20+
java-version: '24'
2121
cache: 'maven'
2222

2323
- name: Build and verify
@@ -39,7 +39,7 @@ jobs:
3939
for k in totals: totals[k]+=int(r.get(k,'0'))
4040
except Exception:
4141
pass
42-
exp_tests=611
42+
exp_tests=1354
4343
exp_skipped=0
4444
if totals['tests']!=exp_tests or totals['skipped']!=exp_skipped:
4545
print(f"Unexpected test totals: {totals} != expected tests={exp_tests}, skipped={exp_skipped}")

‎json-java21-jtd-codegen/pom.xml‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
2+
<project xmlns="http://maven.apache.org/POM/4.0.0"
3+
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
4+
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
5+
http://maven.apache.org/xsd/maven-4.0.0.xsd">
6+
<modelVersion>4.0.0</modelVersion>
7+
8+
<parent>
9+
<groupId>io.github.simbo1905.json</groupId>
10+
<artifactId>parent</artifactId>
11+
<version>0.1.9</version>
12+
</parent>
13+
14+
<artifactId>java.util.json.jtd.codegen</artifactId>
15+
<packaging>jar</packaging>
16+
<name>java.util.json Java21 Backport JTD Codegen</name>
17+
<url>https://simbo1905.github.io/java.util.json.Java21/</url>
18+
<scm>
19+
<connection>scm:git:https://github.com/simbo1905/java.util.json.Java21.git</connection>
20+
<developerConnection>scm:git:git@github.com:simbo1905/java.util.json.Java21.git</developerConnection>
21+
<url>https://github.com/simbo1905/java.util.json.Java21</url>
22+
<tag>HEAD</tag>
23+
</scm>
24+
<description>Bytecode-generated JTD validators using the JDK 24+ ClassFile API.
25+
Generates Java 21 compatible classfiles for hot-path validation.
26+
Optional dependency: falls back to the interpreter path when absent.</description>
27+
28+
<properties>
29+
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
30+
<maven.compiler.release>24</maven.compiler.release>
31+
</properties>
32+
33+
<dependencies>
34+
<dependency>
35+
<groupId>io.github.simbo1905.json</groupId>
36+
<artifactId>java.util.json</artifactId>
37+
<version>${project.version}</version>
38+
</dependency>
39+
<dependency>
40+
<groupId>io.github.simbo1905.json</groupId>
41+
<artifactId>java.util.json.jtd</artifactId>
42+
<version>${project.version}</version>
43+
</dependency>
44+
45+
<!-- Test dependencies -->
46+
<dependency>
47+
<groupId>org.junit.jupiter</groupId>
48+
<artifactId>junit-jupiter-api</artifactId>
49+
<scope>test</scope>
50+
</dependency>
51+
<dependency>
52+
<groupId>org.junit.jupiter</groupId>
53+
<artifactId>junit-jupiter-engine</artifactId>
54+
<scope>test</scope>
55+
</dependency>
56+
<dependency>
57+
<groupId>org.junit.jupiter</groupId>
58+
<artifactId>junit-jupiter-params</artifactId>
59+
<scope>test</scope>
60+
</dependency>
61+
<dependency>
62+
<groupId>org.assertj</groupId>
63+
<artifactId>assertj-core</artifactId>
64+
<scope>test</scope>
65+
</dependency>
66+
</dependencies>
67+
68+
<build>
69+
<plugins>
70+
<plugin>
71+
<groupId>org.apache.maven.plugins</groupId>
72+
<artifactId>maven-compiler-plugin</artifactId>
73+
<version>3.13.0</version>
74+
<configuration>
75+
<release>24</release>
76+
<compilerArgs>
77+
<arg>-Xlint:all</arg>
78+
<arg>-Xdiags:verbose</arg>
79+
</compilerArgs>
80+
</configuration>
81+
</plugin>
82+
<plugin>
83+
<groupId>org.apache.maven.plugins</groupId>
84+
<artifactId>maven-javadoc-plugin</artifactId>
85+
<configuration>
86+
<release>24</release>
87+
<doclint>none</doclint>
88+
</configuration>
89+
</plugin>
90+
</plugins>
91+
</build>
92+
</project>

‎json-java21-jtd-codegen/src/main/java/json/java21/jtd/codegen/Descriptors.java‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
package json.java21.jtd.codegen;
2+
3+
import java.lang.constant.ClassDesc;
4+
import java.lang.constant.ConstantDescs;
5+
import java.lang.constant.MethodTypeDesc;
6+
7+
/// Shared class descriptors and method type descriptors for bytecode emission.
8+
///
9+
/// All fields are compile-time constants referencing the types the generated
10+
/// classfiles interact with at runtime (JSON API, validation result types, JDK stdlib).
11+
final class Descriptors {
12+
13+
private Descriptors() {}
14+
15+
// -- JDK types --
16+
static final ClassDesc CD_Object = ConstantDescs.CD_Object;
17+
static final ClassDesc CD_String = ConstantDescs.CD_String;
18+
static final ClassDesc CD_Math = ClassDesc.of("java.lang.Math");
19+
static final ClassDesc CD_CharSequence = ClassDesc.of("java.lang.CharSequence");
20+
static final ClassDesc CD_OffsetDateTime = ClassDesc.of("java.time.OffsetDateTime");
21+
static final ClassDesc CD_DateTimeFormatter = ClassDesc.of("java.time.format.DateTimeFormatter");
22+
static final ClassDesc CD_Pattern = ClassDesc.of("java.util.regex.Pattern");
23+
static final ClassDesc CD_Matcher = ClassDesc.of("java.util.regex.Matcher");
24+
25+
// -- Collections --
26+
static final ClassDesc CD_ArrayList = ClassDesc.of("java.util.ArrayList");
27+
static final ClassDesc CD_List = ClassDesc.of("java.util.List");
28+
static final ClassDesc CD_Map = ClassDesc.of("java.util.Map");
29+
static final ClassDesc CD_MapEntry = ClassDesc.of("java.util.Map$Entry");
30+
static final ClassDesc CD_Set = ClassDesc.of("java.util.Set");
31+
static final ClassDesc CD_Iterator = ClassDesc.of("java.util.Iterator");
32+
33+
// -- JSON API types --
34+
static final ClassDesc CD_JsonValue = ClassDesc.of("jdk.sandbox.java.util.json.JsonValue");
35+
static final ClassDesc CD_JsonObject = ClassDesc.of("jdk.sandbox.java.util.json.JsonObject");
36+
static final ClassDesc CD_JsonArray = ClassDesc.of("jdk.sandbox.java.util.json.JsonArray");
37+
static final ClassDesc CD_JsonString = ClassDesc.of("jdk.sandbox.java.util.json.JsonString");
38+
static final ClassDesc CD_JsonNumber = ClassDesc.of("jdk.sandbox.java.util.json.JsonNumber");
39+
static final ClassDesc CD_JsonBoolean = ClassDesc.of("jdk.sandbox.java.util.json.JsonBoolean");
40+
static final ClassDesc CD_JsonNull = ClassDesc.of("jdk.sandbox.java.util.json.JsonNull");
41+
42+
// -- Validation result types --
43+
static final ClassDesc CD_JtdValidationError = ClassDesc.of("json.java21.jtd.JtdValidationError");
44+
static final ClassDesc CD_JtdValidationResult = ClassDesc.of("json.java21.jtd.JtdValidationResult");
45+
static final ClassDesc CD_JtdValidator = ClassDesc.of("json.java21.jtd.JtdValidator");
46+
47+
// -- Common method type descriptors --
48+
static final MethodTypeDesc MTD_String = MethodTypeDesc.of(CD_String);
49+
static final MethodTypeDesc MTD_boolean = MethodTypeDesc.of(ConstantDescs.CD_boolean);
50+
static final MethodTypeDesc MTD_double = MethodTypeDesc.of(ConstantDescs.CD_double);
51+
static final MethodTypeDesc MTD_long = MethodTypeDesc.of(ConstantDescs.CD_long);
52+
static final MethodTypeDesc MTD_int = MethodTypeDesc.of(ConstantDescs.CD_int);
53+
static final MethodTypeDesc MTD_boolean_Object = MethodTypeDesc.of(ConstantDescs.CD_boolean, CD_Object);
54+
static final MethodTypeDesc MTD_Object_Object = MethodTypeDesc.of(CD_Object, CD_Object);
55+
static final MethodTypeDesc MTD_Object_int = MethodTypeDesc.of(CD_Object, ConstantDescs.CD_int);
56+
static final MethodTypeDesc MTD_boolean_CharSequence = MethodTypeDesc.of(ConstantDescs.CD_boolean, CD_CharSequence);
57+
static final MethodTypeDesc MTD_String_String = MethodTypeDesc.of(CD_String, CD_String);
58+
static final MethodTypeDesc MTD_String_int = MethodTypeDesc.of(CD_String, ConstantDescs.CD_int);
59+
static final MethodTypeDesc MTD_String_CharSeq_CharSeq = MethodTypeDesc.of(CD_String, CD_CharSequence, CD_CharSequence);
60+
static final MethodTypeDesc MTD_Map = MethodTypeDesc.of(CD_Map);
61+
static final MethodTypeDesc MTD_List = MethodTypeDesc.of(CD_List);
62+
static final MethodTypeDesc MTD_Set = MethodTypeDesc.of(CD_Set);
63+
static final MethodTypeDesc MTD_Iterator = MethodTypeDesc.of(CD_Iterator);
64+
static final MethodTypeDesc MTD_Object = MethodTypeDesc.of(CD_Object);
65+
static final MethodTypeDesc MTD_double_double = MethodTypeDesc.of(ConstantDescs.CD_double, ConstantDescs.CD_double);
66+
static final MethodTypeDesc MTD_Pattern_String = MethodTypeDesc.of(CD_Pattern, CD_String);
67+
static final MethodTypeDesc MTD_Matcher_CharSequence = MethodTypeDesc.of(CD_Matcher, CD_CharSequence);
68+
static final MethodTypeDesc MTD_OffsetDateTime_CharSeq_DTF = MethodTypeDesc.of(CD_OffsetDateTime, CD_CharSequence, CD_DateTimeFormatter);
69+
static final MethodTypeDesc MTD_void_String_String = MethodTypeDesc.of(ConstantDescs.CD_void, CD_String, CD_String);
70+
static final MethodTypeDesc MTD_JtdValidationResult = MethodTypeDesc.of(CD_JtdValidationResult);
71+
static final MethodTypeDesc MTD_JtdValidationResult_List = MethodTypeDesc.of(CD_JtdValidationResult, CD_List);
72+
static final MethodTypeDesc MTD_JtdValidationResult_JsonValue = MethodTypeDesc.of(CD_JtdValidationResult, CD_JsonValue);
73+
static final MethodTypeDesc MTD_void_String = MethodTypeDesc.of(ConstantDescs.CD_void, CD_String);
74+
}

‎json-java21-jtd-codegen/src/main/java/json/java21/jtd/codegen/EmitDiscriminator.java‎

Lines changed: 193 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,193 @@
1+
package json.java21.jtd.codegen;
2+
3+
import java.lang.classfile.CodeBuilder;
4+
import java.lang.classfile.TypeKind;
5+
import java.lang.constant.ConstantDescs;
6+
7+
import json.java21.jtd.JtdSchema;
8+
9+
import static json.java21.jtd.codegen.Descriptors.*;
10+
11+
/// Emits bytecode for JTD Discriminator schema (tagged union).
12+
final class EmitDiscriminator {
13+
14+
private EmitDiscriminator() {}
15+
16+
static void emit(CodeBuilder cob, JtdSchema.DiscriminatorSchema d,
17+
int instSlot, int errSlot,
18+
String instPath, String schemaPath) {
19+
var end = cob.newLabel();
20+
21+
// Step 1: must be object
22+
var step1Fail = cob.newLabel();
23+
cob.aload(instSlot);
24+
cob.instanceOf(CD_JsonObject);
25+
cob.ifeq(step1Fail);
26+
27+
cob.aload(instSlot);
28+
cob.checkcast(CD_JsonObject);
29+
cob.invokeinterface(CD_JsonObject, "members", MTD_Map);
30+
int mapSlot = cob.allocateLocal(TypeKind.REFERENCE);
31+
cob.astore(mapSlot);
32+
33+
// Step 2: tag must exist
34+
var step2Fail = cob.newLabel();
35+
cob.aload(mapSlot);
36+
cob.ldc(d.discriminator());
37+
cob.invokeinterface(CD_Map, "containsKey", MTD_boolean_Object);
38+
cob.ifeq(step2Fail);
39+
40+
cob.aload(mapSlot);
41+
cob.ldc(d.discriminator());
42+
cob.invokeinterface(CD_Map, "get", MTD_Object_Object);
43+
cob.checkcast(CD_JsonValue);
44+
int tagValSlot = cob.allocateLocal(TypeKind.REFERENCE);
45+
cob.astore(tagValSlot);
46+
47+
// Step 3: tag must be string
48+
var step3Fail = cob.newLabel();
49+
cob.aload(tagValSlot);
50+
cob.instanceOf(CD_JsonString);
51+
cob.ifeq(step3Fail);
52+
53+
cob.aload(tagValSlot);
54+
cob.checkcast(CD_JsonString);
55+
cob.invokeinterface(CD_JsonString, "string", MTD_String);
56+
int tagStrSlot = cob.allocateLocal(TypeKind.REFERENCE);
57+
cob.astore(tagStrSlot);
58+
59+
// Step 4: dispatch to variants
60+
for (final var entry : d.mapping().entrySet()) {
61+
final var tagValue = entry.getKey();
62+
final var variantSchema = entry.getValue();
63+
var nextVariant = cob.newLabel();
64+
65+
cob.aload(tagStrSlot);
66+
cob.ldc(tagValue);
67+
cob.invokevirtual(CD_String, "equals", MTD_boolean_Object);
68+
cob.ifeq(nextVariant);
69+
70+
if (variantSchema instanceof JtdSchema.PropertiesSchema props) {
71+
EmitProperties.emit(cob, props, instSlot, errSlot, instPath,
72+
schemaPath + "/mapping/" + tagValue, d.discriminator());
73+
} else {
74+
EmitNode.emit(cob, variantSchema, instSlot, errSlot, instPath,
75+
schemaPath + "/mapping/" + tagValue);
76+
}
77+
cob.goto_(end);
78+
79+
cob.labelBinding(nextVariant);
80+
}
81+
82+
// Step 5: tag not in mapping
83+
EmitError.addError(cob, errSlot,
84+
instPath + "/" + d.discriminator(), schemaPath + "/mapping");
85+
cob.goto_(end);
86+
87+
// Error paths
88+
cob.labelBinding(step1Fail);
89+
EmitError.addError(cob, errSlot, instPath, schemaPath + "/discriminator");
90+
cob.goto_(end);
91+
92+
cob.labelBinding(step2Fail);
93+
EmitError.addError(cob, errSlot, instPath, schemaPath + "/discriminator");
94+
cob.goto_(end);
95+
96+
cob.labelBinding(step3Fail);
97+
EmitError.addError(cob, errSlot,
98+
instPath + "/" + d.discriminator(), schemaPath + "/discriminator");
99+
100+
cob.labelBinding(end);
101+
}
102+
103+
/// Dynamic-path variant: parent instancePath from local variable.
104+
static void emitDynamic(CodeBuilder cob, JtdSchema.DiscriminatorSchema d,
105+
int instSlot, int errSlot,
106+
int pathSlot, String schemaPath) {
107+
var end = cob.newLabel();
108+
109+
var step1Fail = cob.newLabel();
110+
cob.aload(instSlot);
111+
cob.instanceOf(CD_JsonObject);
112+
cob.ifeq(step1Fail);
113+
114+
cob.aload(instSlot);
115+
cob.checkcast(CD_JsonObject);
116+
cob.invokeinterface(CD_JsonObject, "members", MTD_Map);
117+
int mapSlot = cob.allocateLocal(TypeKind.REFERENCE);
118+
cob.astore(mapSlot);
119+
120+
var step2Fail = cob.newLabel();
121+
cob.aload(mapSlot);
122+
cob.ldc(d.discriminator());
123+
cob.invokeinterface(CD_Map, "containsKey", MTD_boolean_Object);
124+
cob.ifeq(step2Fail);
125+
126+
cob.aload(mapSlot);
127+
cob.ldc(d.discriminator());
128+
cob.invokeinterface(CD_Map, "get", MTD_Object_Object);
129+
cob.checkcast(CD_JsonValue);
130+
int tagValSlot = cob.allocateLocal(TypeKind.REFERENCE);
131+
cob.astore(tagValSlot);
132+
133+
var step3Fail = cob.newLabel();
134+
cob.aload(tagValSlot);
135+
cob.instanceOf(CD_JsonString);
136+
cob.ifeq(step3Fail);
137+
138+
cob.aload(tagValSlot);
139+
cob.checkcast(CD_JsonString);
140+
cob.invokeinterface(CD_JsonString, "string", MTD_String);
141+
int tagStrSlot = cob.allocateLocal(TypeKind.REFERENCE);
142+
cob.astore(tagStrSlot);
143+
144+
for (final var entry : d.mapping().entrySet()) {
145+
final var tagValue = entry.getKey();
146+
final var variantSchema = entry.getValue();
147+
var nextVariant = cob.newLabel();
148+
149+
cob.aload(tagStrSlot);
150+
cob.ldc(tagValue);
151+
cob.invokevirtual(CD_String, "equals", MTD_boolean_Object);
152+
cob.ifeq(nextVariant);
153+
154+
if (variantSchema instanceof JtdSchema.PropertiesSchema props) {
155+
EmitProperties.emitDynamic(cob, props, instSlot, errSlot, pathSlot,
156+
schemaPath + "/mapping/" + tagValue, d.discriminator());
157+
} else {
158+
EmitNode.emitDynamic(cob, variantSchema, instSlot, errSlot, pathSlot,
159+
schemaPath + "/mapping/" + tagValue);
160+
}
161+
cob.goto_(end);
162+
163+
cob.labelBinding(nextVariant);
164+
}
165+
166+
// tag not in mapping: error at pathSlot + "/" + discriminator
167+
cob.aload(pathSlot);
168+
cob.ldc("/" + d.discriminator());
169+
cob.invokevirtual(CD_String, "concat", MTD_String_String);
170+
int tagPathSlot = cob.allocateLocal(TypeKind.REFERENCE);
171+
cob.astore(tagPathSlot);
172+
EmitError.addErrorDynamic(cob, errSlot, tagPathSlot, schemaPath + "/mapping");
173+
cob.goto_(end);
174+
175+
cob.labelBinding(step1Fail);
176+
EmitError.addErrorDynamic(cob, errSlot, pathSlot, schemaPath + "/discriminator");
177+
cob.goto_(end);
178+
179+
cob.labelBinding(step2Fail);
180+
EmitError.addErrorDynamic(cob, errSlot, pathSlot, schemaPath + "/discriminator");
181+
cob.goto_(end);
182+
183+
cob.labelBinding(step3Fail);
184+
cob.aload(pathSlot);
185+
cob.ldc("/" + d.discriminator());
186+
cob.invokevirtual(CD_String, "concat", MTD_String_String);
187+
int tagPath2Slot = cob.allocateLocal(TypeKind.REFERENCE);
188+
cob.astore(tagPath2Slot);
189+
EmitError.addErrorDynamic(cob, errSlot, tagPath2Slot, schemaPath + "/discriminator");
190+
191+
cob.labelBinding(end);
192+
}
193+
}

0 commit comments

Comments
 (0)