Skip to content

Request for comment: System.Text.Json compatibility#10

Open
arendvw wants to merge 3 commits intobertt:masterfrom
arendvw:feature/system-text-json
Open

Request for comment: System.Text.Json compatibility#10
arendvw wants to merge 3 commits intobertt:masterfrom
arendvw:feature/system-text-json

Conversation

@arendvw
Copy link
Copy Markdown
Contributor

@arendvw arendvw commented Dec 23, 2025

Since cityjson files tend to be huge and system.text.json promises to be more performant in many places, here's a refactor of the code that uses system.text.json.

At this point, all unit tests run ok, but there are breaking changes in the api or usage.

Most prominently:

  • System.Text.JSON is case sensitive by default, and Newtonsoft is not
  • Decoding requires non-case sensitivity passed as options
  • Marking things as required has a different impact

TODO:

  • Benchmarking to see if it actually matters
  • More thorough testing.

So the main question: is System.Text.Json interesting to you as well? There were also other clear reasons, upstream or downstream, to choose Newtonsoft.Json over stj.

Happy holidays!

- Updated to .netstandard 2.0 for cityjson
- Added small fix for net48 compatiblity in cj2glb.csproj
- Note: cj2glb is not compatible with net48 because of signing issues with SharpGLTF and AnyOf (SharpGLTF is signed, AnyOf is not), but the code on this project's side is fully compatible if built against a locally checked out Gltf project
System.Text.JSON behaves a bit differently and does not allow global overrides, so deserialization should be done with CityJsonSystemTextJson.Options.

All tests run completely, but there might be other trouble.
@bertt
Copy link
Copy Markdown
Owner

bertt commented Dec 23, 2025

yes interested in this, I shall add a basic benchmark test so we can do a comparison

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors the CityJSON library and related tools/tests to use System.Text.Json instead of Newtonsoft.Json for (de)serialization, aiming for better performance and aligning behavior via shared serializer options.

Changes:

  • Replace Newtonsoft.Json usage across src/, tests/, and tools/ with System.Text.Json, including new STJ converters/options.
  • Update model annotations to STJ attributes (JsonPropertyName, JsonRequired) and refactor custom converters to STJ (Utf8JsonReader/Utf8JsonWriter).
  • Minor tool/test/project updates (dependency bump, net48 dev note, .gitignore, test csproj cleanup).

Reviewed changes

Copilot reviewed 27 out of 28 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
tools/cj2glb/cj2glb.csproj Bumps triangulator dependency version.
tools/cj2glb/TexturedGltfCreator.cs Adds TF-specific Vector2 construction to support older targets.
tools/cj2glb/README.net48.md Documents .NET Framework 4.8 strong-name runtime issue/workaround.
tools/cj2glb/Program.cs Switches CityJSON read + tileset write paths to System.Text.Json.
tools/cj2glb.tests/UnitTest1.cs Updates cj2glb tests to deserialize with STJ options.
tests/tests.csproj Updates language version settings (currently duplicated).
tests/SimpleGeometriesTests.cs Switches deserialization to STJ options.
tests/CityJsonWriteTests.cs Switches read/write roundtrip tests to STJ options.
tests/CityJsonTests.cs Switches deserialization to STJ options and adjusts an attribute assertion for STJ value types.
src/cityjson.csproj Replaces Newtonsoft.Json package with System.Text.Json.
src/Vertex.cs Updates usings for STJ + keeps custom converter via attribute.
src/Transform.cs Migrates JSON attributes to STJ (JsonPropertyName, JsonRequired).
src/Texture.cs Migrates JSON attributes to STJ (JsonPropertyName, JsonRequired).
src/Geometry/SolidGeometry.cs Migrates texture JSON property naming to STJ.
src/Geometry/MultiSurfaceGeometry.cs Migrates texture JSON property naming to STJ.
src/Geometry/MultiSolidGeometry.cs Migrates texture JSON property naming to STJ.
src/Geometry/Geometry.cs Switches to STJ and keeps polymorphic converter via [JsonConverter].
src/Geometry/CompositeSurfaceGeometry.cs Migrates texture JSON property naming to STJ.
src/Geometry/CompositeSolidGeometry.cs Migrates texture JSON property naming to STJ.
src/Convertors/VertexConvertor.cs Rewrites VertexConverter from Newtonsoft to STJ.
src/Convertors/GeometryConverter.cs Rewrites polymorphic geometry converter from Newtonsoft to STJ.
src/CityObject.cs Replaces JToken address with JsonNode?.
src/CityJsonWriter.cs Replaces Newtonsoft-based writer with STJ options.
src/CityJsonSystemText.cs Introduces shared STJ JsonSerializerOptions.
src/CityJsonSeqWriter.cs Replaces Newtonsoft-based JSON-seq writer with STJ options.
src/CityJsonSeqReader.cs Replaces Newtonsoft-based JSON-seq reader with STJ options.
src/Appearance.cs Migrates vertices-texture JSON property naming to STJ.
.gitignore Ignores .idea/*.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@@ -0,0 +1,19 @@
# .NET Framework 4.8 Runtime issue
Note, this tool does not need to be compiled against net48, and is not released as such,
this is purely for development or other adoptation purposes.
Copy link

Copilot AI Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo in documentation: "adoptation" should be "adaptation".

Suggested change
this is purely for development or other adoptation purposes.
this is purely for development or other adaptation purposes.

Copilot uses AI. Check for mistakes.
Comment thread src/CityJsonSystemText.cs
DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull,
Converters =
{
new JsonStringEnumConverter(JsonNamingPolicy.CamelCase, allowIntegerValues: true)
Copy link

Copilot AI Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JsonStringEnumConverter(JsonNamingPolicy.CamelCase, ...) will serialize enums as camelCase (e.g., buildingInstallation), while the fixtures and typical CityJSON use PascalCase values (e.g., BuildingInstallation). If this options instance is intended for writing as well, consider using the default naming (no policy) to preserve casing, while keeping case-insensitive reads.

Suggested change
new JsonStringEnumConverter(JsonNamingPolicy.CamelCase, allowIntegerValues: true)
new JsonStringEnumConverter(allowIntegerValues: true)

Copilot uses AI. Check for mistakes.
Comment thread src/CityJsonWriter.cs
Comment on lines 10 to +14
{
Formatting = Formatting.Indented,
ContractResolver = new DefaultContractResolver
{
NamingStrategy = new CamelCaseNamingStrategy
{
ProcessDictionaryKeys = false,
OverrideSpecifiedNames = false
}
},
NullValueHandling = NullValueHandling.Ignore
WriteIndented = true, // Formatting.Indented
PropertyNamingPolicy = JsonNamingPolicy.CamelCase,
// DictionaryKeyPolicy = null; // keep dictionary keys unchanged
DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull
Copy link

Copilot AI Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CityJsonWriter defines its own JsonSerializerOptions instead of reusing CityJsonSystemTextJson.Options, which can lead to subtle serialization differences (notably enum handling via JsonStringEnumConverter, trailing commas tolerance, case-insensitive reads, etc.). Consider reusing the shared options (or explicitly mirroring the relevant settings/converters) so read/write behavior stays consistent across the library.

Copilot uses AI. Check for mistakes.
Comment thread src/CityJsonSeqWriter.cs
Comment on lines 12 to +16
{
Formatting = Formatting.None,
ContractResolver = new DefaultContractResolver
{
NamingStrategy = new CamelCaseNamingStrategy
{
ProcessDictionaryKeys = false,
OverrideSpecifiedNames = false
}
},
NullValueHandling = NullValueHandling.Ignore
WriteIndented = false, // Formatting.None
PropertyNamingPolicy = JsonNamingPolicy.CamelCase, // CamelCaseNamingStrategy
// DictionaryKeyPolicy = null; // keep dictionary keys unchanged (matches ProcessDictionaryKeys = false)
DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull // NullValueHandling.Ignore
Copy link

Copilot AI Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CityJsonSeqWriter duplicates JsonSerializerOptions that may drift from the library’s shared CityJsonSystemTextJson.Options (e.g., enum string handling). Consider reusing the shared options (or a shared base options factory) to keep JSON output consistent across writers.

Copilot uses AI. Check for mistakes.
Boundaries = ReadRequired(root, "boundaries").Deserialize<int[][][][][]>(options)
?? throw new JsonException("Invalid 'boundaries' for CompositeSolid.")
},
_ => null
Copy link

Copilot AI Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read(...) returns null for unknown geometry type (_ => null). This can lead to returning a null Geometry instance (and a subsequent NullReferenceException when lod is set) instead of failing fast. Consider throwing a JsonException for unsupported types so deserialization errors are explicit and non-nullable return contracts are honored.

Suggested change
_ => null
_ => throw new JsonException($"Unsupported geometry type '{type}'.")

Copilot uses AI. Check for mistakes.
Comment on lines +46 to +53
if (root.TryGetProperty("lod", out var lodEl) && lodEl.ValueKind != JsonValueKind.Null)
{
case "solid":
{
var boundaries = jsonObject["boundaries"].ToObject<int[][][][]>();
obj = new SolidGeometry() { Type = GeometryType.Solid, Boundaries = boundaries };

if (textureObject != null)
{
((SolidGeometry)obj).Texture = ReadTextureSolid(jsonObject["texture"].ToObject<Dictionary<string, object>>());
}
break;
}

case "compositesurface":
{
var boundaries = jsonObject["boundaries"].ToObject<int[][][]>();
obj = new CompositeSurfaceGeometry() { Type = GeometryType.CompositeSurface, Boundaries = boundaries };
break;
}

case "multisurface":
{
var boundaries = jsonObject["boundaries"].ToObject<int[][][]>();
obj = new MultiSurfaceGeometry() { Type = GeometryType.MultiSurface, Boundaries = boundaries };
if(textureObject != null)
{
((MultiSurfaceGeometry)obj).Texture = ReadTexture(jsonObject["texture"].ToObject<Dictionary<string, object>>());
}
break;
}
case "multisolid":
{
var boundaries = jsonObject["boundaries"].ToObject<int[][][][][]>();
obj = new MultiSolidGeometry() { Type = GeometryType.MultiSolid, Boundaries = boundaries };
break;
}
case "compositesolid":
{
var boundaries = jsonObject["boundaries"].ToObject<int[][][][][]>();
obj = new CompositeSolidGeometry() { Type = GeometryType.CompositeSolid, Boundaries = boundaries };
break;
}

}

if (jsonObject["lod"] != null)
{
obj.Lod = jsonObject["lod"].ToString();
obj.Lod = lodEl.ValueKind switch
{
JsonValueKind.String => lodEl.GetString(),
JsonValueKind.Number => lodEl.GetRawText(), // "2"
_ => throw new JsonException("'lod' must be string or number.")
};
Copy link

Copilot AI Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

obj.Lod = ... assumes obj is non-null, but obj can be null if an unsupported geometry type is encountered. After changing the default case to throw, this becomes safe; otherwise add a null-check/throw before assigning Lod.

Copilot uses AI. Check for mistakes.
Comment thread src/CityJsonSeqReader.cs
Comment on lines 14 to 18
while (!reader.EndOfStream)
{
var line = reader.ReadLine();
var cityJsonSecond = JsonConvert.DeserializeObject<CityJsonDocument>(line);
var cityJsonSecond = JsonSerializer.Deserialize<CityJsonDocument>(line, CityJsonSystemTextJson.Options);
result.Add(cityJsonSecond);
Copy link

Copilot AI Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

StreamReader isn’t disposed, and JsonSerializer.Deserialize(...) can return null (or ReadLine() can return null/empty), which will lead to nulls in the returned list or a NullReferenceException later. Use a using/using var for the reader and skip/guard empty lines; also validate the deserialize result before adding to result.

Copilot uses AI. Check for mistakes.
Comment thread tests/tests.csproj

<IsPackable>false</IsPackable>

<LangVersion>latest</LangVersion>
Copy link

Copilot AI Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are two <LangVersion>latest</LangVersion> entries in the same PropertyGroup, and one includes trailing whitespace. Remove the duplicate to avoid confusion about which value should apply.

Suggested change
<LangVersion>latest</LangVersion>

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants