Add bin/update-observe.js to automate post-release Observe updates

[alfonso-noriega]

WHY are these changes introduced?

The CLI minor release runbook includes three manual post-release steps to point Observe at the just-released CLI version:

• 4 SLOs
• 1 Error Alert rule (Spike in Errors (vX.Y.Z))
• 1 Error Management Rule

Today these are 3 manual checkbox steps in the runbook. They're easy to forget and easy to fat-finger. This PR collapses them to one command.

WHAT is this pull request doing?

Adds bin/update-observe.js (no new deps) and a pnpm update-observe script:

# Update everything
pnpm update-observe -- --version=3.94.2

# Or update a single resource by key
pnpm update-observe -- --version=3.94.2 --resource=slo-p50-latency

# Print the GraphQL payloads without sending
pnpm update-observe -- --version=3.94.2 --dry-run

--version is required and must be concrete X.Y.Z. --resource is optional; valid keys live in bin/observe-cli-resources.json (slo-correctness-app-deploy, slo-correctness, slo-p50-latency, slo-p75-latency, alert-spike-errors, error-project-cli).

How it works (live-first, no templates)

For each managed resource the script:

1. Fetches the live config from shopify-monitoring.shopifycloud.com/query (sloDefinition, ruleConfig, errorProject).
2. Patches in place every X.Y.Z literal it finds in the version-touching fields:
   • SLO name
   • For Raw SLOs: sli.expression (and weightExpression if present)
   • For Histogram SLOs: the cli_version= filter value
   • For the alert rule: name, expression, and the app.version filter inside customErrorFilters
   • For the error project: name and the app.version filter inside errorFilters
3. Re-upserts the full resource, so non-version fields (objective, alertType, vaultTeamId, notificationPolicyId, useRecordingRules, severityThresholds, etc.) are preserved verbatim from what's currently live.

Because patching is done against the live values (not against templates in JSON), any manual edits made directly in Observe — wording tweaks, extra labels in the name, etc. — round-trip untouched.

Wildcard preservation

If a live resource pins a partial version like 3.*.* or 4.90.*, the script preserves the wildcard components and only rewrites the numeric ones. Given --version=4.95.3:

Live value	Result
3.94.2	4.95.3
3.*.*	4.*.*
4.90.*	4.95.*
*.*.*	*.*.*

Wildcards are intentionally not allowed in the --version argument — the input is always concrete; only Observe owns the wildcards.

Auth

No external tooling required. On first run the script opens https://shopify-monitoring.shopifycloud.com/gql in your browser, prompts you to paste the MINERVA_TOKEN cookie value, and caches it at ~/Library/Caches/shopify-cli/observe-cookie (mode 0600) only after verifying the cookie works against the API. Subsequent runs reuse the cache; if the cookie expires, the script detects the SSO redirect and re-prompts.

Where to find the MINERVA_TOKEN cookie

1. After the browser opens to https://shopify-monitoring.shopifycloud.com/gql, sign in via Okta if prompted. You'll land on either a GraphQL playground page or a JSON 404/405 — both are fine; the cookie has been set.
2. Open DevTools (Cmd+Option+I on macOS, F12 elsewhere).
3. Navigate to:
   • Chrome / Arc / Brave / Edge: Application tab → Storage → Cookies → https://shopify-monitoring.shopifycloud.com
   • Firefox: Storage tab → Cookies → https://shopify-monitoring.shopifycloud.com
   • Safari: Storage tab → Cookies → shopify-monitoring.shopifycloud.com
4. Find the row where Name = MINERVA_TOKEN.
5. Copy the entire Value (long opaque string).
6. Paste it into the script prompt (input is hidden).

Make sure you're looking at cookies for shopify-monitoring.shopifycloud.com, not observe.shopify.io — those are different.

Non-interactive use

For CI or scripted runs, set $MINERVA_TOKEN directly to skip the prompt:

MINERVA_TOKEN=… pnpm update-observe -- --version=3.94.2

Files

• bin/update-observe.js — the script (no new deps)
• bin/observe-cli-resources.json — declarative list of managed resources (key, kind, id, optional versionFilterColumn); no templates
• pnpm update-observe entry in root package.json

How to test your changes?

• ✅ node bin/update-observe.js --version=3.94.2 --dry-run — payloads look correct for all 6 resources
• ✅ node bin/update-observe.js --dry-run — fails with "--version is required"
• ✅ node bin/update-observe.js --version=garbage --dry-run — rejects non-semver
• ✅ node bin/update-observe.js --version=3.*.* --dry-run — rejects wildcards in --version
• ✅ node bin/update-observe.js --version=3.94.2 --resource=slo-p50-latency --dry-run — updates only the selected resource
• ✅ node bin/update-observe.js --version=3.94.2 --resource=nope --dry-run — fails and lists valid keys
• ✅ MINERVA_TOKEN=bogus pnpm update-observe -- --version=3.94.2 — fails with "refresh it from your browser"; cache is not written
• ✅ No cookie + non-TTY stdin (CI-like) — fails with "set $MINERVA_TOKEN to skip the interactive prompt"
• ✅ Wildcard preservation unit-checked: live 3.*.* + --version=4.95.3 → 4.*.*; live 4.90.* → 4.95.*
• ⏳ Interactive first-run flow (browser open + paste prompt) — to be exercised in real release
• ⏳ End-to-end (real upsert) — to be run during the next release with a real token

Post-release steps

• Update the release runbook issue to replace the three manual checkboxes with one pnpm update-observe step.
• Optionally wire pnpm update-observe into bin/post-release so it runs alongside the homebrew/docs/notification PR creation. Skipped for now because it requires CI auth setup; the runbook step can run locally with the developer's own monitoring token.

Checklist

• I've considered possible cross-platform impacts (Mac, Linux, Windows) — cache path branches by platform; chmod 0600 is best-effort on Windows
• I've considered possible documentation changes — runbook update tracked in Post-release steps
• I've considered analytics changes to measure impact — N/A (release-tooling script)
• The change is user-facing — N/A (internal release tooling, no changeset needed)

[alfonso-noriega]

• Add bin/update-observe.js to automate post-release Observe updates #7407 👈 (View in Graphite)
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.