Problem
ORB's #4899 tracks a fleet rollout mechanism to version and roll back many centrally-hosted deployments at once. AMS has no design for the equivalent problem: how a fleet of hosted AMS tenant instances would receive updates, be staged into a rollout, or be rolled back safely if a new version misbehaves. Without this, any future hosted-AMS release process defaults to manual, per-tenant intervention — which does not scale and does not give a safe rollback path when something breaks across many tenants at once.
Area
AMS / Fleet & ops management
Proposal
Design a rollout/versioning approach for hosted AMS instances — staged rollout, canary tenant selection, rollback triggers — built around the same core packages being reused across all tenant instances per the architecture spec. The goal is for a rollout to be a control-plane operation (flip a version pointer, promote a canary cohort, roll back a cohort) rather than a per-tenant code change or manual deploy.
Deliverables
- A design doc covering: rollout staging (how a new version moves from canary to full fleet), canary tenant selection criteria, automatic and manual rollback triggers, and how the shared core package is version-pinned per tenant during a rollout
Resources
Boundaries
- Depends on the control-plane architecture spec settling how tenant instances are represented and addressed — this design should reference, not re-derive, that model.
- Design only: no rollout tooling, no CI/CD pipeline changes, no actual canary infrastructure in this issue.
- Must not assume single-tenant deploy semantics carry over unchanged — a rollback that is safe for one operator's own infra is not automatically safe across many tenants sharing a control plane, and the design must call out where that assumption breaks.
- Fleet-wide rollout/rollback is a blast-radius-affecting decision across every tenant on the platform; any open question in the design should block downstream implementation issues rather than be resolved ad hoc during a later PR.
Problem
ORB's #4899 tracks a fleet rollout mechanism to version and roll back many centrally-hosted deployments at once. AMS has no design for the equivalent problem: how a fleet of hosted AMS tenant instances would receive updates, be staged into a rollout, or be rolled back safely if a new version misbehaves. Without this, any future hosted-AMS release process defaults to manual, per-tenant intervention — which does not scale and does not give a safe rollback path when something breaks across many tenants at once.
Area
AMS / Fleet & ops management
Proposal
Design a rollout/versioning approach for hosted AMS instances — staged rollout, canary tenant selection, rollback triggers — built around the same core packages being reused across all tenant instances per the architecture spec. The goal is for a rollout to be a control-plane operation (flip a version pointer, promote a canary cohort, roll back a cohort) rather than a per-tenant code change or manual deploy.
Deliverables
Resources
Boundaries