Skip to content

Commit a0cdf96

Browse files
[CBD-2166] Document ContentLayout.json and publish its schema
Add Documentation/contentlayout.md describing the ContentLayout.json file produced by content directory builds, and publish the C# schema as a copy of the Unity source type. Move the JSON model classes (BuildLayout, ContentLayout) into a new dependency-free UnityDataModels project so they are easier to find and can be reused by tools that don't depend on the analyzer or SQLite.
1 parent 99ce157 commit a0cdf96

11 files changed

Lines changed: 306 additions & 4 deletions

Analyzer/Analyzer.csproj

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@
2525
<ItemGroup>
2626
<ProjectReference Include="..\UnityBinaryFormat\UnityBinaryFormat.csproj" />
2727
<ProjectReference Include="..\UnityFileSystem\UnityFileSystem.csproj" />
28+
<ProjectReference Include="..\UnityDataModels\UnityDataModels.csproj" />
2829
</ItemGroup>
2930

3031
<ItemGroup>

Analyzer/AnalyzerTool.cs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,8 @@
44
using System.IO;
55
using UnityDataTools.Analyzer.SQLite.Handlers;
66
using UnityDataTools.Analyzer.SQLite.Parsers;
7-
using UnityDataTools.Analyzer.SQLite.Parsers.Models;
87
using UnityDataTools.Analyzer.SQLite.Writers;
8+
using UnityDataTools.Models;
99
using UnityDataTools.BinaryFormat;
1010
using UnityDataTools.FileSystem;
1111

Analyzer/SQLite/Parsers/AddressablesBuildLayoutParser.cs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
using Newtonsoft.Json;
55
using Newtonsoft.Json.Linq;
66
using UnityDataTools.Analyzer.SQLite.Handlers;
7-
using UnityDataTools.Analyzer.SQLite.Parsers.Models;
7+
using UnityDataTools.Models;
88
using UnityDataTools.Analyzer.SQLite.Writers;
99

1010
namespace UnityDataTools.Analyzer.SQLite.Parsers

Analyzer/SQLite/Writers/AddressablesBuildLayoutSQLWriter.cs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
using Microsoft.Data.Sqlite;
55
using Newtonsoft.Json;
66
using UnityDataTools.Analyzer.SQLite.Commands.AddressablesBuildReport;
7-
using UnityDataTools.Analyzer.SQLite.Parsers.Models;
7+
using UnityDataTools.Models;
88

99
namespace UnityDataTools.Analyzer.SQLite.Writers
1010
{

Documentation/contentlayout.md

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
# ContentLayout.json
2+
3+
`ContentLayout.json` describes the content that a content directory build produced. It is written by [`BuildPipeline.BuildContentDirectory`](https://docs.unity3d.com/6000.6/Documentation/ScriptReference/BuildPipeline.BuildContentDirectory.html) into the build report directory, alongside the other build report files. For an overview of the build report directory and the other files it contains, see [Build report and build history](https://docs.unity3d.com/6000.6/Documentation/Manual/build-reporting.html) in the Unity Manual.
4+
5+
This page explains what the file contains conceptually to aid in creation of build-analysis tooling or inspection of content directory build output. The C# types that define the schema are published alongside this documentation in [`ContentLayout.cs`](../UnityDataModels/ContentLayout.cs), which is the authoritative reference for the individual fields.
6+
7+
## What it is for
8+
9+
Use `ContentLayout.json` to:
10+
11+
* Find which source assets a build included, and which serialized file each one ended up in.
12+
* See the dependencies between the files the build produced.
13+
* Understand what contributes to the size or loading footprint of the content in the runtime.
14+
15+
## Relationship to the build manifest
16+
17+
`ContentLayout.json` is a superset of the build manifest that Unity ships with the content. The build manifest is a minimal summary that contains only what is required to load the content at runtime. `ContentLayout.json` is not shipped, and adds a more complete picture, including a mapping from the built content back to the source assets in the project.
18+
19+
## What it does not contain
20+
21+
* **Packaging information.** It does not record how the content is stored, for example whether the artifacts are packed inside Unity Archive files. It describes the logical content, not its on-disk packaging.
22+
* **Object-level detail.** It does not describe individual Unity objects inside the files, and contains no type information. This is intentional, to keep the file size reasonable. For object-level analysis, run [`UnityDataTool analyze`](command-analyze.md) on the build output. For per-type and per-asset size statistics, use the `ContentSummary` of the [BuildReport](buildreport.md).
23+
* **Non-deterministic data.** To avoid non-deterministic data this file contains no timestamps, and no information about the build process that created it. Two builds that produce identical content produce an identical `ContentLayout.json`.
24+
25+
## Terminology
26+
27+
The file uses a few terms consistently:
28+
29+
* **Artifact** — a unit of build output. The term is used instead of "file" because the build output is not necessarily written as individual files (for example, it could be packed inside Unity Archives, or served from key/value storage).
30+
* **SerializedFile** — a Unity binary file containing serialized objects (the `.cf` content files of the build). See [Unity Content Format](unity-content-format.md).
31+
* **Loadable** — an object that can be loaded on demand, identified independently of the serialized file that happens to contain it. Loadable will reference a specific object, for example the root GameObject of a prefab, but loading that object will load the entire SerializedFile.
32+
* **LoadableSceneId** — Similar to a Loadable, but referencing a Unity Scene.
33+
* **Source asset** — an asset in the source project (identified by GUID and asset path) that contributed to the build output.
34+
35+
## Top-level structure
36+
37+
`ContentLayout.json` is a single JSON object with the following members. Entries in several of the arrays cross-reference each other by array index or by hash, so the file describes a graph rather than a flat list.
38+
39+
| Member | Description |
40+
|--------|-------------|
41+
| `Version` | Schema version of the file. See [Schema versioning](#schema-versioning). |
42+
| `BuildManifestHash` | Hash of the build manifest this layout corresponds to. |
43+
| `SerializedFiles` | One entry per serialized file in the build. Each entry records the source assets it contains, its content hash, and its dependencies on other serialized files, loadables, and loadable scenes. The same source asset can appear in more than one serialized file (for example, a single FBX file can be split into multiple output files). |
44+
| `RootAssets` | The `ObjectIdHash` of each root asset the build was made from. Each one has a corresponding entry in `LoadableObjectIds`. |
45+
| `LoadableObjectIds` | The objects that can be loaded on demand. Each entry records its `ObjectIdHash`, where the object lives in the built content (which serialized file) and where it came from in the source project (GUID, asset path, local file ID, and identifier type). |
46+
| `LoadableSceneIds` | The scenes in the build, each with its source project path and GUID, and the serialized file that contains it. |
47+
| `BinaryArtifacts` | The artifacts that make up the build output. See [Binary artifacts](#binary-artifacts). |
48+
49+
### Binary artifacts
50+
51+
`BinaryArtifacts` is essentially the list of files in the build output. Each entry has a `Category` and a `Size`, and lists its direct dependencies in `ArtifactReferences`:
52+
53+
* The entry with category `manifest` is the root of the graph.
54+
* Entries with category `contentfile` each have a matching entry in `SerializedFiles` (matched by content hash).
55+
* `BinaryArtifacts` also reports the additional data files that hold audio, video, texture, and mesh data (the `.resource` and `.resS` files).
56+
* BinaryArtifacts are identified by the hash of their content. When saved as a file, the filename is the hash and the file extension is based on the category.
57+
58+
`ArtifactReferences` lists only direct dependencies. Dependencies that go through a loadable or loadable scene are not included, and the dependency graph is never cyclical. Together, this makes it possible to see every artifact required to load a particular serialized file, excluding data that is loaded on demand through a loadable or loadable scene.
59+
60+
## Schema versioning
61+
62+
The schema is subject to change. The `Version` field records the schema version of the file, independently of the Unity version that produced it. When the schema changes, the version number increments.
63+
64+
[`ContentLayout.cs`](../UnityDataModels/ContentLayout.cs) always represents the latest schema version (currently version 1).
65+
66+
## Related documentation
67+
68+
| Topic | Description |
69+
|-------|-------------|
70+
| [Build report and build history](https://docs.unity3d.com/6000.6/Documentation/Manual/build-reporting.html) | The build report directory and the files in it (Unity Manual). |
71+
| [BuildReport Support](buildreport.md) | Analyzing Unity build report files with UnityDataTool. |
72+
| [Unity Content Format](unity-content-format.md) | SerializedFiles, Unity Archives, and how build output maps back to source assets. |
73+
| [`analyze` command](command-analyze.md) | Object-level analysis of build output. |

Documentation/unity-content-format.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -124,3 +124,5 @@ You can include a Unity BuildReport file when running `UnityDataTools analyze`.
124124
`UnityDataTools analyze` can also import Addressables build layout files, which include source asset information. See [Addressable Build Reports](./addressables-build-reports.md).
125125

126126
For AssetBundles built by [BuildPipeline.BuildAssetBundles()](https://docs.unity3d.com/ScriptReference/BuildPipeline.BuildAssetBundles.html), Unity creates a .manifest file for each AssetBundle that has source information. This is a text-based format.
127+
128+
For content directory builds, the [ContentLayout.json](contentlayout.md) file maps the build output back to the source assets in the project.

Documentation/unitydatatool.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,3 +99,4 @@ If you see a warning about `UnityFileSystemApi.dylib` not being verified, go to
9999
| [Analyze Examples](analyze-examples.md) | Practical database queries |
100100
| [Comparing Builds](comparing-builds.md) | Strategies for build comparison |
101101
| [Unity Content Format](unity-content-format.md) | TypeTrees and file formats |
102+
| [ContentLayout.json](contentlayout.md) | The content layout file produced by content directory builds |

Analyzer/SQLite/Parsers/Models/BuildLayout.cs renamed to UnityDataModels/BuildLayout.cs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
// this file comes from Addressables and is used to serialize and deserialize
44
// build layout information to JSON
5-
namespace UnityDataTools.Analyzer.SQLite.Parsers.Models
5+
namespace UnityDataTools.Models
66
{
77
public class BuildLayout
88
{

UnityDataModels/ContentLayout.cs

Lines changed: 194 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,194 @@
1+
// This file defines the structure of the ContentLayout.json file produced by BuildPipeline.BuildContentDirectory
2+
// (available starting in Unity 6.6).
3+
//
4+
// See Documentation/contentlayout.md for further details.
5+
//
6+
// ContentLayout always represents the latest schema version (currently 1). If the schema changes
7+
// in future, older versions can be preserved under a version-specific namespace while this type
8+
// continues to track the latest version.
9+
namespace UnityDataTools.Models
10+
{
11+
/// <summary>
12+
/// Category strings used by <see cref="BinaryArtifact.Category"/>.
13+
/// </summary>
14+
public static class BuildArtifactCategory
15+
{
16+
/// <summary>Texture data, written to a streaming resource (.resS) file.</summary>
17+
public const string Texture = "texture";
18+
19+
/// <summary>Mesh data, written to a streaming resource (.resS) file.</summary>
20+
public const string Mesh = "mesh";
21+
22+
/// <summary>Audio data, written to a resource (.resource) file.</summary>
23+
public const string Audio = "audio";
24+
25+
/// <summary>Video data, written to a resource (.resource) file.</summary>
26+
public const string Video = "video";
27+
28+
/// <summary>A SerializedFile / Content File (.cf).</summary>
29+
public const string ContentFile = "contentfile";
30+
31+
/// <summary>A manifest file (.json).</summary>
32+
public const string Manifest = "manifest";
33+
}
34+
35+
/// <summary>
36+
/// Describes a SerializedFile in the build output.
37+
/// </summary>
38+
public struct SerializedFileLayout
39+
{
40+
/// <summary>Index of this entry inside <see cref="ContentLayout.SerializedFiles"/>.</summary>
41+
public int Index;
42+
43+
/// <summary>Stable identifier used to reference this SerializedFile from other SerializedFiles
44+
/// in a way that doesn't break when the content changes. Currently based on the cluster or guid of the source.</summary>
45+
public string ID;
46+
47+
/// <summary>True for synthetic entries representing built-in Unity resources that are not produced
48+
/// by the build (currently only "Library/unity default resources"). Such entries have no ContentHash.</summary>
49+
public bool IsBuiltIn;
50+
51+
/// <summary>The source assets included in this SerializedFile.</summary>
52+
public string[] SourceAssets;
53+
54+
/// <summary>Indices into the containing <see cref="ContentLayout.SerializedFiles"/> array, identifying
55+
/// other SerializedFiles that need to be loaded prior to loading this SerializedFile.</summary>
56+
public int[] SerializedFileDependencies;
57+
58+
/// <summary>ObjectIdHash values for loadable objects referenced from this SerializedFile.</summary>
59+
public string[] LoadableDependencies;
60+
61+
/// <summary>Scene paths for scenes referenced from this SerializedFile.</summary>
62+
public string[] LoadableSceneDependencies;
63+
64+
/// <summary>xxhash3 hash of the content, used for the filename (+".cf") and for lookup into UDS.
65+
/// Matches the <see cref="BinaryArtifact.ContentHash"/> of the corresponding entry in
66+
/// <see cref="ContentLayout.BinaryArtifacts"/>.</summary>
67+
public string ContentHash;
68+
}
69+
70+
/// <summary>
71+
/// Records a loadable object in the build. Listed at the top level of the ContentLayout so that a
72+
/// loadable's identity is described independently of the SerializedFile that happens to contain it.
73+
/// </summary>
74+
public class LoadableObjectIdLayout
75+
{
76+
/// <summary>Hash of the GUID, LFID and IdentifierType.</summary>
77+
public string ObjectIdHash;
78+
79+
/// <summary>AssetDatabase GUID of the source asset.</summary>
80+
public string GUID;
81+
82+
/// <summary>Path of the source asset.</summary>
83+
public string AssetPath;
84+
85+
/// <summary>Local file id of the source object.</summary>
86+
public long LFID;
87+
88+
/// <summary>Identifier type of the source object.</summary>
89+
public int IdentifierType;
90+
91+
/// <summary>Index into <see cref="ContentLayout.SerializedFiles"/> for the file that contains this
92+
/// loadable, or -1 if it was dropped (e.g. server build shader references).</summary>
93+
public int SerializedFile = -1;
94+
}
95+
96+
/// <summary>
97+
/// Records a scene exposed as loadable in the build, and the SerializedFile that contains it.
98+
/// </summary>
99+
public class LoadableSceneIdLayout
100+
{
101+
/// <summary>AssetDatabase GUID of the scene.</summary>
102+
public string GUID;
103+
104+
/// <summary>Scene path.</summary>
105+
public string Path;
106+
107+
/// <summary>Index into <see cref="ContentLayout.SerializedFiles"/>, or -1 if not mapped.</summary>
108+
public int SerializedFile = -1;
109+
}
110+
111+
/// <summary>
112+
/// Describes a binary artifact produced by the build (SerializedFile, streaming resource, manifest, etc.).
113+
/// </summary>
114+
public class BinaryArtifact
115+
{
116+
/// <summary>Index of this entry inside <see cref="ContentLayout.BinaryArtifacts"/>.</summary>
117+
public int Index;
118+
119+
/// <summary>Content addressable hash. For ContentFile artifacts, the matching
120+
/// <see cref="SerializedFileLayout"/> can be found by ContentHash.</summary>
121+
public string ContentHash;
122+
123+
/// <summary>One of the strings in <see cref="BuildArtifactCategory"/>.</summary>
124+
public string Category;
125+
126+
/// <summary>Size in bytes.</summary>
127+
public ulong Size;
128+
129+
/// <summary>Indices into <see cref="ContentLayout.BinaryArtifacts"/> identifying additional artifacts
130+
/// referenced from this one (e.g. mesh, audio).
131+
/// Note: this does not track references to other ContentFiles — those are recorded in
132+
/// <see cref="SerializedFileLayout.SerializedFileDependencies"/>.</summary>
133+
public int[] ArtifactReferences;
134+
135+
/// <summary>The on-disk file extension for this artifact, derived from <see cref="Category"/>.
136+
/// For unrecognized categories, returns the category name.</summary>
137+
public string FileExtension
138+
{
139+
get
140+
{
141+
switch (Category)
142+
{
143+
case BuildArtifactCategory.Texture:
144+
case BuildArtifactCategory.Mesh:
145+
return ".resS";
146+
case BuildArtifactCategory.Audio:
147+
case BuildArtifactCategory.Video:
148+
return ".resource";
149+
case BuildArtifactCategory.ContentFile:
150+
return ".cf";
151+
case BuildArtifactCategory.Manifest:
152+
return ".json";
153+
default:
154+
return string.IsNullOrEmpty(Category) ? "" : "." + Category;
155+
}
156+
}
157+
}
158+
}
159+
160+
/// <summary>
161+
/// In-memory representation of the ContentLayout.json file written by the build.
162+
///
163+
/// The Layout is a companion to the BuildManifest, recording additional details about the build
164+
/// (including source assets and information about which object an ObjectId hash refers to).
165+
/// It is not shipped with the build; it exists for tools and tests that analyze build output.
166+
/// The schema is subject to change and there is currently no backward compatibility.
167+
/// </summary>
168+
public class ContentLayout
169+
{
170+
/// <summary>The schema version this type represents.</summary>
171+
public const int CurrentVersion = 1;
172+
173+
/// <summary>Schema version of the ContentLayout.json file.</summary>
174+
public int Version;
175+
176+
/// <summary>Hash of the BuildManifest this layout corresponds to.</summary>
177+
public string BuildManifestHash;
178+
179+
/// <summary>The SerializedFiles in the build output.</summary>
180+
public SerializedFileLayout[] SerializedFiles;
181+
182+
/// <summary>ObjectIdHash values of the root assets; resolve via <see cref="LoadableObjectIds"/>.</summary>
183+
public string[] RootAssets;
184+
185+
/// <summary>Loadable objects in the build.</summary>
186+
public LoadableObjectIdLayout[] LoadableObjectIds;
187+
188+
/// <summary>Loadable scenes in the build.</summary>
189+
public LoadableSceneIdLayout[] LoadableSceneIds;
190+
191+
/// <summary>Binary artifacts produced by the build.</summary>
192+
public BinaryArtifact[] BinaryArtifacts;
193+
}
194+
}
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
<Project Sdk="Microsoft.NET.Sdk">
2+
3+
<PropertyGroup>
4+
<OutputType>Library</OutputType>
5+
<TargetFramework>net9.0</TargetFramework>
6+
<LangVersion>latest</LangVersion>
7+
</PropertyGroup>
8+
9+
<PropertyGroup Condition=" '$(Configuration)' == 'Debug' ">
10+
<PlatformTarget>AnyCPU</PlatformTarget>
11+
</PropertyGroup>
12+
13+
<PropertyGroup Condition=" '$(Configuration)' == 'Release' ">
14+
<PlatformTarget>AnyCPU</PlatformTarget>
15+
</PropertyGroup>
16+
17+
</Project>

0 commit comments

Comments
 (0)