Skip to content

Blue/green or canary deploy: smoke test suite and automatic rollback on stream errors #138

@greatest0fallt1me

Description

@greatest0fallt1me

Post-deploy smoke tests for health, read path, and a synthetic stream op

    ## Description

    After each **deploy** to **staging** (and **production** if applicable), run an

automated smoke suite: /readyz, GET /streams with auth, and a
synthetic create + settle on testnet or mocks to catch catastrophic
misconfiguration before users hit Soroban errors at scale.

    ## Requirements and context

    - **Idempotent** synthetic user or token for smoke only; separate from real users.
  • Gating: fail pipeline or auto-rollback hook if SLO not met.

  • No production money movement; testnet or chain mock only.

  • Timeout and clear logs on failure.

  • Docs for re-running smokes by on-call.

      ## Suggested execution
    
      1. `git checkout -b ci/deploy-smoke-streams`
    
  1. Add scripts/smoke.sh and wire to CD tool (Argo, GitHub Environments, etc.).

  2. PR with at least 3 checks in the smoke; sample green log.

  3. Security note: protect synthetic credentials in CI secrets; rotate.

  4. 96h; extend to multi-region in follow-up.

     ## Test and commit
    
  • Run the full test suite; add or update tests until the agreed coverage bar is met.
  • Cover edge cases listed in this issue; document any intentional exclusions with brief rationale in the PR.
  • Include relevant test output (e.g. test runner summary) or a link to a passing CI run in the pull request.
  • Add security notes for auth, keys, PII, chain settlement, or money movement (assumptions verified, out-of-scope items).

Example commit message

ci(ops): add post-deploy smoke with synthetic stream op on Stellar testnet or mocks

Guidelines

  • Target: at least 95% coverage on new or meaningfully changed code (per the repo’s standard tooling).
  • Documentation: update contributor-facing or API documentation where a reviewer would be blocked without it.
  • Timeframe: 96 hours to ready-for-review (surface blockers early).

Metadata

Metadata

Assignees

No one assigned

    Labels

    area-infrastructureStreamPay ghit: area-infrastructuredomain-operationsStreamPay ghit: domain-operationspriority-p2StreamPay ghit: priority-p2type-ciStreamPay ghit: type-ci

    Type

    No fields configured for Task.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions