Skip to content

direct: Fix state serialization for zero-value fields on deeply-embedded SDK specs#5782

Open
pietern wants to merge 1 commit into
mainfrom
enable-pg-native-login-bug
Open

direct: Fix state serialization for zero-value fields on deeply-embedded SDK specs#5782
pietern wants to merge 1 commit into
mainfrom
enable-pg-native-login-bug

Conversation

@pietern

@pietern pietern commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

The dyn→typed ForceSendFields routing only handled one level of struct embedding. When a field declared in an SDK spec embedded two levels deep (e.g. PostgresProjectPostgresProjectConfigProjectSpec) was set to its zero value, its name was recorded in the wrong struct's ForceSendFields. The direct engine then failed to serialize the plan state with field X cannot be found in struct Y.

Fix routes each field's ForceSendFields entry to the struct that actually declares it, at any embedding depth. This affected postgres_projects (enable_pg_native_login: false), postgres_branches, and postgres_endpoints (replace_existing: false). Terraform was unaffected (it serializes via dyn, not the SDK marshaler).

Adds a unit regression test for the deep-embedding case and a systemic guard that round-trips every registered resource type's zero-value fields, so any future resource using the embedded-spec wrapper pattern is covered automatically.

This pull request and its description were written by Isaac.

The dyn->typed ForceSendFields routing only handled one level of struct
embedding. When a field declared in an SDK spec embedded two levels deep
(e.g. PostgresProject -> PostgresProjectConfig -> ProjectSpec) was set to
its zero value, its name was recorded in the wrong struct's
ForceSendFields. The direct engine then failed to serialize the plan
state with "field X cannot be found in struct Y".

Route each field's ForceSendFields entry to the struct that actually
declares it, at any embedding depth. This affected postgres_projects
(enable_pg_native_login: false), postgres_branches, and postgres_endpoints
(replace_existing: false). Terraform was unaffected (it serializes via
dyn, not the SDK marshaler).

Add a unit regression test for the deep-embedding case and a systemic
guard that round-trips every registered resource type's zero-value
fields, so any future resource using the embedded-spec wrapper pattern is
covered automatically.

Co-authored-by: Isaac
@pietern pietern requested a review from denik June 30, 2026 18:47
@pietern pietern temporarily deployed to test-trigger-is June 30, 2026 18:47 — with GitHub Actions Inactive
@pietern pietern temporarily deployed to test-trigger-is June 30, 2026 18:47 — with GitHub Actions Inactive
@github-actions

Copy link
Copy Markdown
Contributor

Approval status: pending

/acceptance/bundle/ - needs approval

Files: acceptance/bundle/resources/postgres_projects/basic/databricks.yml.tmpl, acceptance/bundle/resources/postgres_projects/basic/out.requests.json
Suggested: @denik
Also eligible: @janniklasrose, @anton-107, @andrewnester, @shreyas-goenka, @lennartkats-db

/bundle/ - needs approval

Files: bundle/config/resources_types_test.go
Suggested: @denik
Also eligible: @janniklasrose, @anton-107, @andrewnester, @shreyas-goenka, @lennartkats-db

General files (require maintainer)

4 files changed
Based on git history:

  • @denik -- recent work in ./, libs/dyn/convert/, bundle/config/

Any maintainer (@andrewnester, @anton-107, @denik, @shreyas-goenka, @simonfaltum, @renaudhartert-db) can approve all areas.
See OWNERS for ownership rules.

@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: b3fa0b9

Run: 28468121968

Env 🟨​KNOWN 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
🟨​ aws linux 7 1 13 232 1037 5:30
🟨​ aws windows 7 1 13 234 1035 6:42
💚​ aws-ucws linux 8 13 316 955 5:13
💚​ aws-ucws windows 8 13 318 953 3:52
💚​ azure linux 2 15 232 1036 3:57
💚​ azure windows 2 15 234 1034 3:45
💚​ azure-ucws linux 2 15 318 952 5:08
💚​ azure-ucws windows 2 15 320 950 3:32
💚​ gcp linux 2 15 231 1038 3:55
💚​ gcp windows 2 15 233 1036 3:31
21 interesting tests: 13 SKIP, 7 KNOWN, 1 RECOVERED
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
🟨​ TestAccept 🟨​K 🟨​K 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🙈​ TestAccept/bundle/invariant/no_drift 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/permissions 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions 🟨​K 🟨​K 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions 🟨​K 🟨​K 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 🟨​K 🟨​K 💚​R 💚​R
🙈​ TestAccept/bundle/resources/postgres_branches/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/recreate 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/replace_existing 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/update_protected 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/without_branch_id 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_endpoints/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_projects/update_display_name 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/synced_database_tables/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/ssh/connection 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestFetchRepositoryInfoAPI_FromRepo 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 4 slowest tests (at least 2 minutes):
duration env testname
2:51 azure windows TestAccept
2:45 gcp windows TestAccept
2:32 aws-ucws windows TestAccept
2:31 azure-ucws windows TestAccept

ForceEmpty: make(map[string]bool),
GolangNames: make(map[string]string),
ForceSendFieldsStructKey: make(map[string]int),
ForceSendFieldsStructKey: make(map[string]string),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why can't we keep precalculated index? the structs are static, so it should always resolve to the same index, right?

my_project:
project_id: test-pg-proj-$UNIQUE_NAME
display_name: "Test Postgres Project"
enable_pg_native_login: false

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we also add this to invariant tests?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants