Consolidate bundle schema annotations into a single tree-structured file#5574
Open
pietern wants to merge 9 commits into
Open
Consolidate bundle schema annotations into a single tree-structured file#5574pietern wants to merge 9 commits into
pietern wants to merge 9 commits into
Conversation
Drop the checked-in annotations_openapi.yml: the schema generator now derives those annotations from .codegen/cli.json in memory on every run. Merge annotations.yml and annotations_openapi_overrides.yml into a single annotations.yml whose structure mirrors the bundle configuration tree. Docs absent from the file inherit from cli.json; entries override them. The generated jsonschema.json and jsonschema_for_docs.json are unchanged. Co-authored-by: Isaac
Contributor
Approval status: pending
|
Collaborator
Integration test reportCommit: 5aba24b
25 interesting tests: 15 SKIP, 7 RECOVERED, 2 FAIL, 1 flaky
Top 30 slowest tests (at least 2 minutes):
|
Replace the "_" entry inside "fields" with an explicit "type" key on the node, sibling to "fields". For map and sequence fields it documents each entry; for scalar fields it documents the value's type (enum values live here). Purely a file-format spelling change; the in-memory representation and the generated schema are unchanged. Co-authored-by: Isaac
Fold the near-identical inline-descriptor and type-docs serialization paths into one descriptorToMap helper and hoist the shared key order to a package var. Document RootTypeKey as the in-memory sentinel. No change to the generated annotations file or schema. Co-authored-by: Isaac
The in-memory annotations keyed each type's docs by field name plus a
magic "_" entry for the type itself, forcing "_" to dodge real field
names (config has fields named type, fields, description). Model it
instead as TypeAnnotation{Self, Fields}: Self is the type's own docs
(applied to its $defs entry; also where enum values live), Fields is the
per-field docs. The "_" sentinel and RootTypeKey are gone.
No change to the generated annotations file or schema.
Co-authored-by: Isaac
structFieldNames re-implemented the generator's readonly/internal/ exported/json-tag skip rules and then cross-checked the result against the emitted schema. The schema's property set is already authoritative, so order the names by struct declaration and keep only those the schema emitted; reflection now supplies order alone. A count check still catches a struct walk that fails to reach a property. Co-authored-by: Isaac
A config field named "type" (artifacts.*.type, variables.*.type) was told apart from the structural type-docs key only by nesting level. Prefix the two structural keys with "$" — matching the JSON-Schema convention this file documents — so they can never be confused with or collide with a field of the same name, which always appears as a bare key inside "$fields". Also tidy the saver: a takeSelf accessor mirrors takeField, and the ordering line numbers are named constants. No change to the generated schema. Co-authored-by: Isaac
It wrapped clijson.Parse with an empty-schemas check and was called once. Fold it into generateSchema and drop cli_json.go. Co-authored-by: Isaac
The type graph was built from the unpruned schema, so fields the generator deletes (jobs.GitSource.git_snapshot, resources.Pipeline.dry_run) stayed valid graph edges and kept stale PLACEHOLDER entries alive in annotations.yml even though they can never appear in the schema. Build the graph with the structural prune transforms (removeJobsFields, removePipelineFields) applied, via a shared configTypeGraph used by both the generator and TestNoDetachedAnnotations, so the file only documents fields the schema actually emits. The two stale entries are dropped. Also: surface all type-graph drift errors via errors.Join instead of keeping only the last; add a direct test for annotation.File's SetField/SetSelf; document why descriptorToMap/descriptorEmpty rely on a nil reference; and align the two "Dropping annotation" warnings. No change to the generated schema. Co-authored-by: Isaac
Reconciles #5443 (expose launch stage/release status in the schema) with the consolidated single-file annotation system on this branch: - Brought in unchanged: annotation.Descriptor's new LaunchStage / EnumLaunchStages / EnumDescriptions fields, the PreviewTag helper (preview.go), jsonschema.Extension's enumDescriptions, and assignAnnotation's launch-stage description prefixing + enumDescriptions emission (these merged cleanly onto our annotations.go). - Reapplied in our in-memory extractor: parser.go now populates the launch stage on each field and the enum launch stages/descriptions on the type's Self descriptor via SetField/SetSelf, instead of the deleted annotations_openapi.yml. The launch-stage data is inherited from cli.json at generation time and never persisted to annotations.yml. - Kept annotations_openapi.yml deleted (our system computes that layer in memory). bundle/schema/jsonschema.json and jsonschema_for_docs.json are byte-for-byte identical to origin/main after regeneration. Co-authored-by: Isaac
pietern
changed the title
pietern
requested review from
andrewnester,
denik,
janniklasrose and
shreyas-goenka
and removed request for
janniklasrose
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
annotations_openapi.yml,annotations_openapi_overrides.yml,annotations.yml) with oneannotations.ymlstructured like the bundle config tree..codegen/cli.jsonin memory at generation time, instead of checking in ~7k lines of generated YAML. The file now holds only CLI-owned docs and overrides.$type/$fields; absent entries inherit from cli.json, anddescription: PLACEHOLDERflags undocumented fields.task generate-schemakeeps the file in lockstep with the config structure: it adds placeholders for new fields and drops entries for fields the schema no longer emits.Why
The old files duplicated cli.json data that went stale between regenerations and were keyed by Go type name, which hid where a field actually lives. One structured file gives a single place for CLI-owned docs, no duplicated upstream content, and a layout that reads like the config it documents.
Tests
bundle/schema/jsonschema.jsonandjsonschema_for_docs.jsonare byte-for-byte identical to before — this is purely an internal representation change.This PR was written by Isaac.