Add RateLimiter.create(double, double) for pre-filled burst capacity#8425
Add RateLimiter.create(double, double) for pre-filled burst capacity#8425c-wchen wants to merge 1 commit into
Conversation
…illed burst capacity. The new factory method creates a `SmoothBursty` `RateLimiter` whose token bucket is pre-filled to capacity on creation, so it can immediately absorb an initial traffic surge without throttling the first wave of requests. The existing `RateLimiter.create(double)` always starts with an empty bucket that fills up over time (capped at 1 second worth of permits). The new overload lets callers set both the bucket size and its initial fill level via `maxBurstSeconds`. Implementation detail: `SmoothBursty` gains a package-private `prefillStoredPermits()` method; the factory method calls it after `setRate`. No new fields are introduced. RELNOTES=`RateLimiter`: added `create(double permitsPerSecond, double maxBurstSeconds)` factory method that creates a bursty rate limiter with a pre-filled token bucket for handling startup traffic surges.
If you don't need the warm up time, then can't you use RateLimiter.create(permitsPerSecond, /* warmupPeriod= */ Duration.ZERO) instead? |
First of all, that Secondly, we do have an internal overload that does pretty much exactly that. @kluever do we want to open source |
|
Thanks for the review! A few clarifications on the points raised: 1. That overload creates a 2. Is there an open-source plan for I noticed the internal overload was mentioned. If the team is considering opening it up, that would be a great alternative to this PR. Happy to close this in favour of that approach if there is a concrete plan to expose it publicly. 3. The parameter name The constructor signature is |
Summary
This PR adds a new factory method
RateLimiter.create(double permitsPerSecond, double maxBurstSeconds)that creates aSmoothBurstyRateLimiterwith its token bucket pre-filled to capacity at creation time.Motivation
The existing
RateLimiter.create(double)always starts with an empty token bucket that fills up gradually over time (capped at 1 second of permits). This means that on startup or after a long idle period, the first batch of requests cannot immediately use the full burst capacity — they must wait for the bucket to fill.The new overload solves the "cold start" problem: callers that know they will face an initial traffic surge (e.g., application startup, cache warming) can pre-fill the bucket so the first wave of requests is served without throttling.
Changes
RateLimiter.java(JRE + Android): addspublic static RateLimiter create(double permitsPerSecond, double maxBurstSeconds)and a@VisibleForTestingpackage-private overload that accepts aSleepingStopwatch.SmoothRateLimiter.java(JRE + Android): addsvoid prefillStoredPermits()onSmoothBursty, which setsstoredPermits = maxPermits. No new fields are introduced.RateLimiterTest.java(JRE + Android): adds tests covering pre-filled bucket behavior, comparison with the unfilled variant, stable rate after burst, accumulation after idle, and parameter validation.API
The returned
RateLimiterbehaves identically to the result ofcreate(double)after the initial burst is consumed.Testing
All existing
RateLimiterTestcases continue to pass. New test methods added:testCreateWithMaxBurstSeconds_prefillsBuckettestCreateWithMaxBurstSeconds_vsCreateWithoutPrefilltestCreateWithMaxBurstSeconds_rateAfterBurstIsStabletestCreateWithMaxBurstSeconds_accumulatesAfterIdletestCreateWithMaxBurstSeconds_parameterValidationRELNOTES=
RateLimiter: addedcreate(double permitsPerSecond, double maxBurstSeconds)factory method that creates a bursty rate limiter with a pre-filled token bucket for handling startup traffic surges.