perf: Speed up dev server startup (two flavors: parallelism and streaming)

[davidbrackbill]

Context

Some valid concerns were brought up around breaking changes. I've changed the code to use parallel, blocking boot by default and added the streaming endpoint as an opt-in. Respectively, startup becomes:

• 10s on parallel boot
• 1s on streaming boot

Streaming demo

streaming_sync.mov

Summary

On large projects the dev server took ~50s to pass its health check because it walked the flag list one page at a time. It now fetches the pages concurrently (~12s for 3,125 flags), and a new --stream-flag-startup flag drops that to ~1s by serving flag state off the SDK stream and filling in variation display names afterward, in the background.

1) Parallelize the blocking flag GET startup (non-breaking)

This reduces the startup time by about ~40s and is behavior-preserving: the flag list used to be paginated one page at a time, and now it's fetched concurrently. The only thing that really needs scrutiny is the paginator (FetchPagesConcurrently and its tests).

2) Use streaming to start up the server

Uses the streaming endpoint via --stream-flag-startup mode. This fills flag values and backfills variation names using the parallel load in the background. The parts worth reading are model/fill_variations.go and the streaming branch in model/project.go.

Blast radius: dev server startup only. We could update our CLI calls internally to use this if we want, without any breaking changes for customers.

Testing

• FetchPagesConcurrently: single page, batch boundaries, ordering, a missing total count, and error propagation. Retry429s: a 429 returned with a non-nil error still retries, and a nil response returns without panicking.
• FillVariations and the streaming sync path: the fill replaces variations from REST; streaming sync defers the fetch and schedules the fill after overrides; a resync with empty variations keeps live overrides.
• Ran against the real Catamorphic default project (3,125 flags): default mode syncs in ~12s with names present; resync ~11.8s. Full Go suite, UI tests, lint, and dist build pass.

Raised by	Concern	Status
Dan	Ship a cross-cutting change safely — new major version or experimental flag	Streaming is opt-in behind --stream-flag-startup; default is a behavior-preserving speedup
Dan	"Healthy" must mean SDKs can get all flags	Held — flag state loads before the server listens, in both modes
Dan	Overrides applied before healthy	Held — applied during sync, before listen; the fill is scheduled after them
Drew	Unrecovered goroutine panic crashes the server (nil variation id, nil Retry429s response)	Fixed — recover() in the fill, nil-id fallback, and the Retry429s nil-response guard
Drew	429 retry skipped when the client returns an error with the response	Fixed — bail only on a nil response, never on err alone
Drew	Missing/zero totalCount truncates large projects	Fixed — the paginator never trusts the count; it stops on a short page
Drew	Fill-vs-resync race reverts or permanently poisons names	Gone — no placeholder ids or name merge; the fill replaces wholesale, resync preserves, and a singleflight prevents concurrent fills
Drew	Value-match merge assigns duplicate ids and drops a variation	Gone — the merge step no longer exists
Drew	UI poll stops forever after one failed request	Fixed — it reschedules on failure
Drew	UI poll gives up silently after ~40s	Fixed — backoff with a higher cap, and the server retries too
Drew	UI poll applies a stale response after switching projects	Fixed — re-checks cancellation after awaiting the body
Drew	Flag-fetch failure downgraded to a log line with no retry	Fixed — the default fetch fails loudly; the background fill retries with backoff
Drew	Holdout handling forked into a fragile key-suffix heuristic in three places	Gone — no stream value capture; holdouts are excluded server-side by the REST purpose filter
Drew	Dead Api.GetFlag, duplicated upsert loop, unshared pending- constant, redundant offset := offset	Gone — none of that code exists in this version
Bugbot	Empty variations prune stored overrides in streaming mode	Fixed — override pruning keys off flag_state, not available_variations

Supersedes #733 and #738.

---

Note

Medium Risk
Changes dev-server sync, REST pagination, and override pruning; default path is a behavior-preserving speedup, while streaming mode defers variation metadata and relies on background fills and updated prune logic.

Overview
Dev server startup is faster on large projects by paging the LaunchDarkly REST flag list with bounded concurrency instead of walking next links one page at a time, plus a small fix so 429 retries still run when the API client returns an error alongside the response.

Optional --stream-flag-startup loads flag values from the SDK stream before listen (health ~1s on huge projects) and fills variation display names in the background via FillVariations / SetAvailableVariationsForProject. Sync and add/patch project paths schedule that fill when the mode is on; refreshExternalState skips blocking GetAllFlags and keeps stored variations until the fill completes.

Override pruning on resync now keys off flag_state (via json_each) instead of available_variations, so overrides are not dropped while variations are still empty or lagging in streaming mode. Nil variation IDs from REST get a safe per-index fallback when building stored variations.

^{Reviewed by Cursor Bugbot for commit 7cd06ee. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit c0ed5dd. Configure here.}