Skip to content

perf(dev-server): fetch flags concurrently on startup#738

Closed
davidbrackbill wants to merge 1 commit into
mainfrom
perf/parallel-flag-fetch
Closed

perf(dev-server): fetch flags concurrently on startup#738
davidbrackbill wants to merge 1 commit into
mainfrom
perf/parallel-flag-fetch

Conversation

@davidbrackbill

@davidbrackbill davidbrackbill commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Problem

The dev server paginates the flag list serially (following next links at 100/page) during startup, taking up to ~50s in CI on large projects (e.g. Catamorphic) before the health check passes.

Fix

Fetch the pages concurrently instead: pull page 0, and while pages come back full, fetch the rest in bounded concurrent batches until a short page marks the end of the list.

  • New internal.FetchPagesConcurrently paginator (unit-tested), replacing the serial GetPaginatedItems. It never trusts the API's optional total count — that field reads back as 0 when absent and would silently truncate large projects, so a short page is the only end-of-list signal.
  • Retry429s now guards against a nil *http.Response before dereferencing StatusCode, so a transport-level failure (DNS, connection refused) returns an error instead of panicking.

Api.GetAllFlags keeps its signature, so nothing downstream changes — this is a contained, behavior-preserving speedup. Variation names are still fully resolved before the server reports healthy (just fetched concurrently); the health-check contract is unchanged.

Scope

This is the base of a two-PR split. A follow-up adds an opt-in --stream-flag-startup mode (values off the SDK stream + background name fill) for ~1s health checks; that machinery is reviewed separately, on top of this.

Outcome

  • Startup flag fetch drops from ~50s to ~12s on large projects.
  • Net −72 lines (removes the serial paginator and its test, adds a smaller tested one).

Claude claude@anthropic.com


Note

Medium Risk
Changes startup data-fetch behavior (ordering preserved, but more parallel API load and different pagination semantics) plus error-handling in shared Retry429s; contained to dev-server adapters with unit tests.

Overview
Replaces serial LaunchDarkly flag list pagination (following next links) with bounded concurrent offset paging so dev-server startup on large projects is much faster while GetAllFlags stays the same for callers.

getFlags now uses new FetchPagesConcurrently (page size 100, concurrency 6): load page 0, then pull further pages in parallel batches until a short page ends the list—explicitly not using optional API total counts that can read as 0 and truncate results. The old GetPaginatedItems / parseHref path is removed.

Retry429s only returns early when *http.Response is nil (transport failures), so 429s that arrive with both error and response still retry; tests cover nil-response and 429+error cases.

Reviewed by Cursor Bugbot for commit e6e9ddc. Bugbot is set up for automated code reviews on this repo. Configure here.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit f764bb3. Configure here.

Comment thread internal/dev_server/adapters/internal/api_util.go
The dev server paginated the flag list serially (following next-links at
100/page), taking up to ~50s in CI on large projects. Fetch pages concurrently
instead: pull page 0, and while pages come back full, fetch the rest in bounded
concurrent batches until a short page ends the list.

- New internal.FetchPagesConcurrently paginator (with tests), replacing the
  serial GetPaginatedItems. It never trusts the API's optional total count
  (which reads back as 0 when absent and would truncate large projects) - a
  short page is the only end-of-list signal.
- Retry429s: guard against a nil response before dereferencing StatusCode, so a
  transport-level failure returns an error instead of panicking.

Api.GetAllFlags keeps its signature, so nothing downstream changes.

Claude <claude@anthropic.com>
@davidbrackbill

Copy link
Copy Markdown
Contributor Author

Folding into #739 as a single PR — the concurrent-fetch base and the opt-in --stream-flag-startup mode ship together there (the risky path is gated behind the flag, so one PR is fine). This branch's commit is the first commit in #739.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant