Skip to content

feat(proxy): add opportunistic traffic class#567

Closed
Komzpa wants to merge 16 commits into
Soju06:mainfrom
Komzpa:codex/opportunistic-traffic-class
Closed

feat(proxy): add opportunistic traffic class#567
Komzpa wants to merge 16 commits into
Soju06:mainfrom
Komzpa:codex/opportunistic-traffic-class

Conversation

@Komzpa
Copy link
Copy Markdown
Collaborator

@Komzpa Komzpa commented May 9, 2026

Summary

  • Add API-key traffic_class with foreground default and opportunistic routing.
  • Add Codex opportunistic admission preflight at /backend-api/codex/opportunistic/admission.
  • Return OpenAI-style 429 denials with Retry-After when the opportunistic burn window is closed.
  • Add dynamic opportunistic burn floors for burn_first, normal, and preserve accounts.
  • Route real opportunistic Codex requests through the existing Codex proxy path while keeping Fixer-style preflight as a scheduling hint.
  • Split sticky reallocation thresholds for primary and secondary budget windows.
  • Apply account health-tier filtering before routing-policy selection so unhealthy accounts cannot win manual/opportunistic routing.
  • Preserve additional-quota usage when stale standard-quota resets clear only the standard window.
  • Keep foreground routing falling through from exhausted burn_first accounts to usable normal/preserve accounts instead of surfacing a global no_accounts 503.

Stack

Together with #553 and the already-merged #561, this addresses #578.

Live regression note

A live-stack deployment from an older local working copy reproduced the failure mode this PR now guards: all burn_first accounts were exhausted for the weekly window while a preserve account still had usable capacity, and foreground Codex requests received 503 no_accounts with Rate limit exceeded. The PR branch contains test_budget_safe_selection_falls_back_when_burn_first_unavailable to keep that fallback behavior covered.

Validation

  • /home/kom/proj/codex-lb/.venv/bin/ruff check app/modules/proxy/load_balancer.py tests/unit/test_load_balancer.py
  • /home/kom/proj/codex-lb/.venv/bin/python -m pytest -q tests/unit/test_load_balancer.py (98 passed)
  • uv run --frozen pytest tests/unit/test_load_balancer.py tests/unit/test_select_with_stickiness.py tests/integration/test_settings_api.py tests/integration/test_migrations.py -q (146 passed, 3 skipped)
  • uv run --frozen ruff check .
  • uv run --frozen ruff format --check .
  • uv run --frozen ty check app tests
  • codex review --base github/main completed clean after the health-tier and additional-quota review fixes.
  • GitHub Actions on 6af32ede are all green and the PR is mergeable/clean.

@Komzpa Komzpa force-pushed the codex/opportunistic-traffic-class branch from c21f588 to 288ab6a Compare May 9, 2026 08:42
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c21f58874b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread app/core/balancer/logic.py Outdated
Comment on lines +189 to +191
if _routing_policy(other) == ROUTING_POLICY_PRESERVE:
if _preserve_allows_opportunistic_burn(other, current, preserve_count=preserve_count):
return True
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Treat preserve capacity as foreground in zero-burn eligibility

The zero-burn guard for normal/burn-first accounts only counts preserve peers when _preserve_allows_opportunistic_burn(...) returns true. That conflates “foreground capacity exists” with “preserve is also opportunistically burnable”: a preserve account at its opportunistic floor can still retain foreground reserve, but this code treats it as unavailable and blocks draining expendable accounts to zero. As a result, opportunistic admission can close earlier than intended whenever the only remaining foreground reserve is on preserve accounts.

Useful? React with 👍 / 👎.

@Komzpa Komzpa force-pushed the codex/opportunistic-traffic-class branch 3 times, most recently from e4a5f8f to 42d3f9f Compare May 9, 2026 09:17
@Komzpa Komzpa force-pushed the codex/opportunistic-traffic-class branch 6 times, most recently from c0e5d81 to 8042bfe Compare May 10, 2026 17:06
@Komzpa Komzpa force-pushed the codex/opportunistic-traffic-class branch from e2284ab to 6af32ed Compare May 10, 2026 20:06
@Komzpa Komzpa force-pushed the codex/opportunistic-traffic-class branch 4 times, most recently from 6455170 to 8fef828 Compare May 12, 2026 17:06
@Komzpa Komzpa force-pushed the codex/opportunistic-traffic-class branch from 8fef828 to 5b2b995 Compare May 15, 2026 08:27
@Komzpa Komzpa requested a review from Soju06 as a code owner May 15, 2026 08:27
@Komzpa
Copy link
Copy Markdown
Collaborator Author

Komzpa commented May 16, 2026

Closing this draft because the opportunistic traffic class work was split into reviewable replacement PRs: #657 for the core traffic-class behavior and #659 for independent sticky-threshold settings. The active stack should use those split PRs.

@Komzpa Komzpa closed this May 16, 2026
@Komzpa Komzpa deleted the codex/opportunistic-traffic-class branch May 16, 2026 05:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant