feat(proxy): add fill-first routing strategy by ramhaidar · Pull Request #831 · Soju06/codex-lb

ramhaidar · 2026-05-27T08:10:00Z

Summary

Add a new fill_first routing strategy that deterministically picks the eligible account with the lowest primary 5h used_percent, tiebroken by account_id ascending. Operators get a single-account-at-a-time mode for prompt-cache locality and predictable cost attribution, without changing the default routing strategy.

Type of change

feat: — new user-facing feature or capability

Linked issue:

OpenSpec

This PR includes / updates an OpenSpec change

Change directory: openspec/changes/add-fill-first-routing-strategy/

The change adds:

proposal.md — problem, solution, scope, impact
tasks.md — implementation breakdown (T1–T19)
specs/proxy-admission-control/spec.md — ## ADDED Requirements delta with normative scenarios for deterministic ranking, stable selection, transition-on-pool-exit, and account_id tiebreak

Changes

app/core/balancer/logic.py: extend RoutingStrategy literal to include "fill_first"; add _fill_first_sort_key and _select_fill_first helpers; add new dispatch branch in select_account() that applies _prefer_earlier_reset_candidates the same way as capacity_weighted. Determinism is a hard property — no randomness, only (used_percent, account_id).
app/modules/proxy/service.py: extend _routing_strategy() to propagate the new value to LoadBalancer.select_account(). Without this, the dashboard would accept fill_first but the proxy would silently fall back to capacity_weighted.
app/modules/settings/schemas.py: widen Pydantic regex on both response and update models.
frontend/src/features/settings/schemas.ts: extend RoutingStrategySchema; align the optional default with the backend (capacity_weighted).
frontend/src/features/settings/components/routing-settings.tsx: add <SelectItem value="fill_first">Fill first</SelectItem> directly under "Capacity weighted"; widen the onValueChange literal cast.
frontend/src/components/layout/status-bar.tsx: widen getRoutingLabel(...) to render fill_first with sticky / prefer-earlier-reset compositions.
frontend/src/utils/constants.ts: add fill_first: "fill first" to ROUTING_LABELS.
frontend/src/test/mocks/handlers.ts: widen the routing-strategy Zod enum.
Tests: 11 new backend unit tests, 1 new backend integration test (cycle through accounts as they saturate), 2 new settings-API round-trip tests, plus frontend assertions in routing-settings.test.tsx, schemas.test.ts, and constants.test.ts.

No alembic migration: dashboard_settings.routing_strategy is a free-form String column with app-level Pydantic validation only; the existing column accepts the new value as-is. Default routing strategy is unchanged (capacity_weighted).

Test plan

uvx ruff check .                                          # All checks passed
uvx ruff format --check .                                 # 499 files clean
uv run ty check                                           # All checks passed

uv run pytest tests/unit/test_load_balancer.py -q
# 93 passed (includes 11 new fill_first tests)

uv run pytest tests/integration/test_settings_api.py -q
# 3 passed (includes new fill_first round-trip + rejection)

uv run pytest tests/integration/test_load_balancer_integration.py::test_load_balancer_fill_first_cycles_through_accounts -q
# 1 passed

uv run pytest tests/unit -q --timeout=60
# 2076 passed, 40 skipped, 3 unrelated Windows-only failures in
# tests/unit/test_conversation_archive.py that reproduce on main

cd frontend
bun run lint        # clean
bun run typecheck   # clean
bun run test        # 449 tests across 76 files, all passing

Checklist

Title is in Conventional Commits format.
Added or updated tests covering the change.
Did not edit CHANGELOG.md by hand.
OpenSpec change folder included for the new behavior.
Default routing strategy unchanged.
openspec validate --specs will run in CI; the local CLI was unavailable in the dev environment.

Soju06 · 2026-05-27T15:42:53Z

@codex review

chatgpt-codex-connector · 2026-05-27T15:43:00Z

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

Komzpa · 2026-05-31T21:16:41Z

@codex review

Komzpa · 2026-05-31T23:06:18Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 119aceed59

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Komzpa · 2026-05-31T23:12:50Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9231178ffb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Komzpa · 2026-05-31T23:31:51Z

@codex review

chatgpt-codex-connector · 2026-05-31T23:34:25Z

Codex Review: Didn't find any major issues. Nice work!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Add a new `fill_first` routing strategy that deterministically picks the eligible account with the lowest primary 5h `used_percent`, tiebreaking on `account_id` ascending. The 'fill' behavior emerges naturally from the existing `effective_pool` ladder: an account stays selected while it remains the lowest-usage candidate, and traffic only moves once the upstream marks that account rate-limited or quota-exceeded (or it transitions to draining). Touches: - balancer: extend `RoutingStrategy`, add `_select_fill_first`, dispatch - settings: widen Pydantic regex; `_routing_strategy()` propagates `fill_first` - frontend: schema, dropdown, status-bar label, mocks - tests: unit, integration, settings-API round-trip, frontend - openspec: change folder + proxy-admission-control delta No alembic migration: `dashboard_settings.routing_strategy` is a free-form String column with app-level Pydantic validation only. Default routing strategy unchanged (capacity_weighted).

Soju06 · 2026-06-01T07:35:06Z

@codex review

Komzpa · 2026-06-01T07:35:07Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 56e3555970

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 56e3555970

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Soju06 · 2026-06-01T07:58:34Z

@codex review

Komzpa · 2026-06-01T07:58:37Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 060783be27

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Soju06 · 2026-06-01T08:24:14Z

@codex review

chatgpt-codex-connector · 2026-06-01T08:27:04Z

Codex Review: Didn't find any major issues. What shall we delve into next?

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

ramhaidar force-pushed the feat/fill-first-routing-strategy branch from e94fd6f to 119acee Compare May 27, 2026 10:44

chatgpt-codex-connector Bot reviewed May 31, 2026

View reviewed changes

Comment thread app/core/balancer/logic.py Outdated

Soju06 added the 🤖 codex: needs work [@codex review] raised an issue label May 31, 2026

Soju06 removed the 🤖 codex: needs work [@codex review] raised an issue label May 31, 2026

chatgpt-codex-connector Bot reviewed May 31, 2026

View reviewed changes

Comment thread app/core/balancer/logic.py

Soju06 added 🤖 codex: needs work [@codex review] raised an issue and removed 🤖 codex: needs work [@codex review] raised an issue labels May 31, 2026

ramhaidar and others added 5 commits June 1, 2026 11:10

fix(balancer): keep fill-first on filling account

3de3934

fix(balancer): restore fill-first lowest usage order

b7a4ff7

fix(balancer): keep fill-first on selected healthy account

3466093

fix(settings): preserve routing defaults after rebase

56e3555

Komzpa force-pushed the feat/fill-first-routing-strategy branch from 3c122cf to 56e3555 Compare June 1, 2026 07:19

chatgpt-codex-connector Bot reviewed Jun 1, 2026

View reviewed changes

Comment thread app/core/balancer/logic.py Outdated

chatgpt-codex-connector Bot reviewed Jun 1, 2026

View reviewed changes

Comment thread app/core/balancer/logic.py Outdated

Komzpa added the 🤖 codex: needs work [@codex review] raised an issue label Jun 1, 2026

Soju06 removed the 🤖 codex: needs work [@codex review] raised an issue label Jun 1, 2026

chatgpt-codex-connector Bot reviewed Jun 1, 2026

View reviewed changes

Comment thread app/core/balancer/logic.py Outdated

Komzpa added the 🤖 codex: needs work [@codex review] raised an issue label Jun 1, 2026

fix(balancer): honor fill-first usage ranking

45a10fc

Komzpa force-pushed the feat/fill-first-routing-strategy branch from 060783b to 45a10fc Compare June 1, 2026 08:06

Soju06 removed the 🤖 codex: needs work [@codex review] raised an issue label Jun 1, 2026

Soju06 added the 🤖 codex: ok [@codex review] says no issues found. label Jun 1, 2026

Conversation

ramhaidar commented May 27, 2026

Summary

Type of change

OpenSpec

Changes

Test plan

Checklist

Uh oh!

Soju06 commented May 27, 2026

Uh oh!

chatgpt-codex-connector Bot commented May 27, 2026

Uh oh!

Komzpa commented May 31, 2026

Uh oh!

Komzpa commented May 31, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Komzpa commented May 31, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Komzpa commented May 31, 2026

Uh oh!

chatgpt-codex-connector Bot commented May 31, 2026

Uh oh!

Soju06 commented Jun 1, 2026

Uh oh!

Komzpa commented Jun 1, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Soju06 commented Jun 1, 2026

Uh oh!

Komzpa commented Jun 1, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Soju06 commented Jun 1, 2026

Uh oh!

chatgpt-codex-connector Bot commented Jun 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants