fix(flaky): propagate auth trigger errors

[JerryNee]

What does this PR do?

test flaky documented auth failures as exit code 3, but the rerun-trigger catch path only rethrew NOT_FOUND; other ApiErrors were recorded as { outcome: "error" } attempts and eventually exited as flaky/failing with code 1.

This changes runFlaky to propagate auth-family trigger errors (AUTH_REQUIRED, AUTH_INVALID, AUTH_FORBIDDEN) before scoring per-attempt errors, so credentials problems keep the standard auth envelope/exit code.

Related issue

Fixes #199

Type of change

• Bug fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• Breaking change (fix or feature that changes existing behavior)
• Documentation only
• Build / CI / chore

Checklist

• PR targets the main branch.
• Commits follow Conventional Commits (feat(...), fix(...), docs(...), …).
• npm run lint and npm run format:check pass.
• npm run typecheck passes.
• npm test passes and coverage stays at or above the 80% gate.
• New behavior is covered by unit tests (mock-based; no network or credentials required).
• No secrets, API keys, internal endpoints, or personal data are included.
• User-facing changes are reflected in README.md / DOCUMENTATION.md where relevant.

Notes for reviewers

Added focused runFlaky coverage for AUTH_REQUIRED, AUTH_INVALID, and AUTH_FORBIDDEN during the rerun trigger. Existing NOT_FOUND behavior for missing replayable runs is unchanged.

Validation run locally:

• npm test -- src/commands/test.flaky.spec.ts
• npm run typecheck
• npm run format:check
• npm run lint
• NO_COLOR= npm test

My shell exports NO_COLOR=1; the full suite was rerun with it cleared so the existing ticker TTY ANSI tests exercise their expected raw-write path.

Summary by CodeRabbit

• Bug Fixes
  • test flaky now stops immediately on authentication-related trigger failures instead of treating them as normal retry errors.
  • Authentication errors now return the correct failure status and no rerun is attempted.
  • Added coverage for these auth failure cases to ensure they behave consistently.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 974ed02a-5d18-4930-8b2d-59d90dfc3b9b

📥 Commits

Reviewing files that changed from the base of the PR and between 3305dfa and 6d9eb7c.

📒 Files selected for processing (2)

• src/commands/test.flaky.spec.ts
• src/commands/test.ts

---

Walkthrough

Adds detection of auth-related ApiError codes (AUTH_REQUIRED, AUTH_INVALID, AUTH_FORBIDDEN) during the rerun-trigger step in runFlaky, rethrowing these as fatal instead of recording them as flaky attempts. Extends the test mock and adds parameterized tests validating the new propagation behavior.

Changes

Flaky rerun auth failure propagation

Layer / File(s)	Summary
Fatal auth error detection and rethrow src/commands/test.ts	Adds isFlakyFatalTriggerError helper identifying AUTH_REQUIRED, AUTH_INVALID, AUTH_FORBIDDEN ApiErrors; rethrows them in runFlaky's rerun-trigger catch block instead of recording as errored attempts.
Mock and test coverage src/commands/test.flaky.spec.ts	Adds TriggerAuthErrorCode type and triggerAuthError option to makeFlakyFetch mock returning 401/403 without enqueuing a rerun; adds it.each tests asserting runFlaky throws ApiError with exitCode: 3 and zero trigger attempts.

Estimated code review effort: 2 (Simple) | ~10 minutes

Sequence Diagram(s)

sequenceDiagram
  participant runFlaky
  participant triggerRerun
  participant isFlakyFatalTriggerError

  runFlaky->>triggerRerun: POST /runs/rerun
  triggerRerun-->>runFlaky: ApiError (auth code)
  runFlaky->>isFlakyFatalTriggerError: check(err)
  isFlakyFatalTriggerError-->>runFlaky: true
  runFlaky-->>runFlaky: rethrow (exitCode 3)

Loading

Possibly related PRs

• TestSprite/testsprite-cli#132: Introduced the original runFlaky/makeFlakyFetch implementation in src/commands/test.ts and src/commands/test.flaky.spec.ts that this PR extends.

Suggested reviewers: ruili-testsprite, zeshi-du

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title is concise and accurately summarizes the main change: propagating auth trigger errors in flaky test handling.
Linked Issues check	✅ Passed	The implementation matches issue #199 by rethrowing auth trigger errors, preserving NOT_FOUND handling, and adding focused tests.
Out of Scope Changes check	✅ Passed	The changed files stay within flaky test handling and its tests, with no unrelated or extra-scope modifications evident.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}