Skip to content

Reproduce stale-published-content dashboard bug as acceptance test#5597

Merged
denik merged 14 commits into
mainfrom
denik/dashboard-published-bug
Jun 15, 2026
Merged

Reproduce stale-published-content dashboard bug as acceptance test#5597
denik merged 14 commits into
mainfrom
denik/dashboard-published-bug

Conversation

@denik

@denik denik commented Jun 14, 2026

Copy link
Copy Markdown
Contributor

Adds an acceptance test for the bug where a dashboard rename succeeds (PATCH) but the subsequent publish fails, leaving the user unable to recover by simply re-running deploy.

What the test shows:

  1. First deploy: dashboard created and published as "my dashboard"
  2. Second deploy (fault injected on POST /published): PATCH renames the draft to "my dashboard renamed" and bumps the remote etag, but publish fails. SaveState is only called on success, so state retains the pre-PATCH etag.
  3. Third deploy: detects state.etag ≠ remote.etag and errors with "dashboard has been modified remotely" — no API writes are attempted, so the stale published version ("my dashboard") is never updated.

Also includes:

  • testserver: bump dashboard etag on every PATCH (previously only bumped when SerializedDashboard changed), matching actual cloud behavior
  • Dashboard acceptance tests: replace generic ETAG regex replacements in test.toml with per-test add_repl.py calls so etag value changes are visible in diffs

denik added 5 commits June 14, 2026 08:32
Co-authored-by: Denis Bilenko <denis.bilenko@databricks.com>
… calls

Co-authored-by: Denis Bilenko <denis.bilenko@databricks.com>
Previously the fake workspace only bumped the etag when SerializedDashboard
changed. Cloud bumps it on every write. Update the test server and regenerate
the stale-content acceptance test output accordingly.

Co-authored-by: Isaac
SaveState is only called on success; the failing second deploy leaves state
with the pre-PATCH etag. The third deploy detects the etag mismatch and
errors with "modified remotely" instead of silently skipping publish.
Wrap the third deploy in errcode so the subsequent get/get-published calls
run and show the draft renamed while the published version is still stale.

Co-authored-by: Isaac
@denik denik temporarily deployed to test-trigger-is June 14, 2026 16:02 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is June 14, 2026 16:02 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is June 14, 2026 16:12 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is June 14, 2026 16:12 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is June 14, 2026 16:14 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is June 14, 2026 16:14 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is June 14, 2026 16:18 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is June 14, 2026 16:18 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is June 14, 2026 16:21 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is June 14, 2026 16:21 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is June 14, 2026 16:31 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is June 14, 2026 16:31 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is June 14, 2026 16:37 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is June 14, 2026 16:37 — with GitHub Actions Inactive
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Jun 14, 2026

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 0133f23

Run: 27550514896

Env 🔄​flaky 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
💚​ aws linux 7 15 264 979 5:55
💚​ aws windows 7 15 266 977 6:35
💚​ aws-ucws linux 7 15 360 893 14:52
💚​ aws-ucws windows 7 15 362 891 9:46
💚​ azure linux 1 17 267 977 6:09
💚​ azure windows 1 17 269 975 6:33
💚​ azure-ucws linux 1 17 365 889 10:09
💚​ azure-ucws windows 1 17 367 887 9:58
🔄​ gcp linux 2 1 17 261 980 7:01
💚​ gcp windows 1 17 265 978 6:42
24 interesting tests: 15 SKIP, 7 RECOVERED, 2 flaky
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🙈​ TestAccept/bundle/invariant/no_drift 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/permissions 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions 💚​R 💚​R 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions 💚​R 💚​R 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R 💚​R 💚​R
🙈​ TestAccept/bundle/resources/postgres_branches/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/recreate 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/replace_existing 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/update_protected 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/without_branch_id 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_endpoints/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_endpoints/recreate 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_projects/update_display_name 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/synced_database_tables/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_indexes/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_indexes/grants/select 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/ssh/connection 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🔄​ TestFilerWorkspaceNotebook ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p
🔄​ TestFilerWorkspaceNotebook/rJupyterNb.ipynb ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p
Top 24 slowest tests (at least 2 minutes):
duration env testname
4:58 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
4:45 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
4:20 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:19 aws-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
4:09 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:46 aws-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:40 azure windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:24 aws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:20 azure windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:19 azure linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:12 azure-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:10 aws-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:09 aws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:06 aws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:04 azure-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:03 azure-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:59 azure-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:48 aws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:46 aws-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:41 aws-ucws linux TestAccept/bundle/deploy/files/no-snapshot-sync/DATABRICKS_BUNDLE_ENGINE=direct
2:27 azure linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:21 aws-ucws linux TestAccept/bundle/deploy/files/no-snapshot-sync/DATABRICKS_BUNDLE_ENGINE=terraform
2:07 aws-ucws windows TestAccept/bundle/resources/volumes/recreate/DATABRICKS_BUNDLE_ENGINE=terraform
2:01 aws-ucws windows TestAccept/bundle/resources/permissions/dashboards/create/DATABRICKS_BUNDLE_ENGINE=terraform

…r DoUpdate

DoUpdate may mutate newState in place (e.g. etag) and server-generated fields
may change on every write. Update the test to compare DoUpdate's return value
against a fresh DoRead, and update remappedState to the post-update value so
the subsequent field checks stay consistent.

Co-authored-by: Isaac
@denik denik temporarily deployed to test-trigger-is June 14, 2026 20:02 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is June 14, 2026 20:02 — with GitHub Actions Inactive
Replace add_repl.py calls with direct echo in tests that run with
MSYS_NO_PATHCONV=1. When this env var is set, MSYS does not convert
Unix-style paths to Windows format before passing them to Python.exe,
causing the shebang invocation to fail with the wrong path. Direct
echo to ACC_REPLS avoids Python entirely and works on all platforms.

Co-authored-by: Denis Bilenko <denis.bilenko@databricks.com>
@denik denik temporarily deployed to test-trigger-is June 14, 2026 22:41 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is June 14, 2026 22:41 — with GitHub Actions Inactive
@@ -0,0 +1,15 @@
Cloud = false
Local = true
RecordRequests = true

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing Badness?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added

Co-authored-by: Denis Bilenko <denis.bilenko@databricks.com>
@denik denik temporarily deployed to test-trigger-is June 15, 2026 13:42 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is June 15, 2026 13:42 — with GitHub Actions Inactive
@denik denik enabled auto-merge June 15, 2026 13:52
@denik denik added this pull request to the merge queue Jun 15, 2026
Merged via the queue into main with commit 7011cac Jun 15, 2026
25 checks passed
@denik denik deleted the denik/dashboard-published-bug branch June 15, 2026 14:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants