Skip to content

Simplify bundle JSON schema annotation system#4670

Draft
shreyas-goenka wants to merge 4 commits intomainfrom
simplify-schema-annotations
Draft

Simplify bundle JSON schema annotation system#4670
shreyas-goenka wants to merge 4 commits intomainfrom
simplify-schema-annotations

Conversation

@shreyas-goenka
Copy link
Contributor

@shreyas-goenka shreyas-goenka commented Mar 5, 2026

Summary

  • Read OpenAPI descriptions directly from the spec at runtime instead of pre-extracting them into YAML files. The OpenAPI spec is now a requirement for schema generation.
  • Remove annotations_openapi.yml and annotations_openapi_overrides.yml which flip-flopped on every SDK bump. Merge manual overrides into a single annotations.yml.
  • Use bundle paths (e.g. resources.jobs.*) as annotation keys instead of Go type paths.
  • Remove bundle/docsgen. It was never used beyond the initial documentation scaffolding for our public docs.

Test plan

  • All schema tests pass
  • JSON schema output is identical (0 descriptions lost, 81 gained)
  • Schema generation is idempotent

🤖 Generated with Claude Code

@eng-dev-ecosystem-bot
Copy link
Collaborator

eng-dev-ecosystem-bot commented Mar 5, 2026

Commit: 6225f44

Run: 22783508355

Env 🪲​BUG ❌​FAIL 🟨​KNOWN 🔄​flaky 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
🟨​ aws linux 7 1 7 268 780 6:26
🟨​ aws windows 7 1 7 270 778 6:02
💚​ aws-ucws linux 8 7 365 695 7:23
🔄​ aws-ucws windows 2 7 7 366 693 6:40
🪲​ azure linux 1 8 1 1 9 262 778 143:55
🪲​ azure windows 1 8 1 1 9 264 776 144:01
🪲​ azure-ucws linux 1 8 1 6 1 9 355 691 145:15
🪲​ azure-ucws windows 1 8 1 4 9 360 689 146:13
🪲​ gcp linux 1 8 1 1 9 258 781 143:51
🪲​ gcp windows 1 8 1 1 9 260 779 144:12
33 interesting tests: 10 flaky, 8 FAIL, 7 KNOWN, 7 SKIP, 1 BUG
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
🟨​ TestAccept 🟨​K 🟨​K 💚​R 💚​R 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K
🪲​ TestAccept/bundle/deployment/bind/alert 🙈​s 🙈​s 🙈​s 🙈​s 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B
❌​ TestAccept/bundle/deployment/bind/alert/DATABRICKS_BUNDLE_ENGINE=direct ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/deployment/bind/alert/DATABRICKS_BUNDLE_ENGINE=terraform ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/generate/alert ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/generate/alert/DATABRICKS_BUNDLE_ENGINE=direct ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/generate/alert/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/resources/alerts/with_file ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/resources/alerts/with_file/DATABRICKS_BUNDLE_ENGINE=direct ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/resources/alerts/with_file/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
🔄​ TestAccept/bundle/resources/jobs/check-metadata ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p
🔄​ TestAccept/bundle/resources/jobs/check-metadata/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p
🙈​ TestAccept/bundle/resources/permissions 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🔄​ TestAccept/bundle/resources/permissions/jobs/delete_one/cloud 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s 🔄​f ✅​p 🙈​s 🙈​s
🔄​ TestAccept/bundle/resources/permissions/jobs/delete_one/cloud/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p 🔄​f ✅​p
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions 🟨​K 🟨​K 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions 🟨​K 🟨​K 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 🟨​K 🟨​K 💚​R 💚​R
🙈​ TestAccept/bundle/resources/postgres_branches/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/recreate 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/update_protected 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/without_branch_id 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_endpoints/recreate 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/synced_database_tables/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🔄​ TestAccept/selftest/record_cloud/basic ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
🔄​ TestAccept/selftest/record_cloud/basic/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
🔄​ TestAccept/ssh/connect-serverless-gpu 🙈​s 🙈​s ✅​p 🔄​f 🙈​s 🙈​s ✅​p 🔄​f 🙈​s 🙈​s
🔄​ TestAccept/ssh/connection 💚​R 💚​R 💚​R 🔄​f 💚​R 💚​R 💚​R 🔄​f 💚​R 💚​R
🔄​ TestFilerWorkspaceNotebook ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
🔄​ TestFilerWorkspaceNotebook/rJupyterNb.ipynb ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
Top 20 slowest tests (at least 2 minutes):
duration env testname
3:39 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:27 aws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:20 aws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:13 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:11 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:08 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:04 aws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:55 aws-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:54 aws-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:52 aws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:50 aws-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:50 aws-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:17 azure linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:16 azure linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:16 azure windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:14 azure-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:13 azure-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:09 azure windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:08 azure-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:06 azure-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct

shreyas-goenka and others added 4 commits March 6, 2026 22:40
Read OpenAPI descriptions directly from the spec at runtime instead of
pre-extracting them into annotation YAML files. This makes the OpenAPI
spec a requirement for schema generation and eliminates annotation files
that flip-flopped on every SDK bump.

Changes:
- Remove annotations_openapi.yml and annotations_openapi_overrides.yml
- Merge manual overrides into a single annotations.yml
- Use bundle paths (e.g. resources.jobs.*) as annotation keys instead
  of Go type paths
- Add path_mapping.go to resolve bundle paths to Go types via reflection
- Remove bundle/docsgen (was the only other consumer of annotations)
- Makefile schema targets now require DATABRICKS_OPENAPI_SPEC

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix copyloopvar lint warnings in annotations.go
- Fix gofmt issue in path_mapping_test.go
- Use maps.Copy instead of manual loop
- Restore permission level enums, lifecycle descriptions,
  markdownDescription, and SDK type overrides from the deleted
  annotations_openapi_overrides.yml into annotations.yml
- Update getOpenApiAnnotations to check embedded types for
  promoted field descriptions from OpenAPI spec
- Update TestNoDetachedAnnotations to handle Go type path keys

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…qlWarehouse, and SDK types

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
After rebase on main, adapt to #4666 which switched grants from custom
per-resource types to SDK catalog.PrivilegeAssignment. Remove stale
per-resource grant annotation entries (descriptions now come from
OpenAPI), regenerate schema and python codegen, and clean up .wsignore
references to deleted docsgen files.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@shreyas-goenka shreyas-goenka force-pushed the simplify-schema-annotations branch from 2f2a2d7 to 6225f44 Compare March 6, 2026 21:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants