Skip to content

Add RateLimiter.create(double, double) for pre-filled burst capacity#8425

Open
c-wchen wants to merge 1 commit into
google:masterfrom
c-wchen:feature/rate-limiter-burst-init
Open

Add RateLimiter.create(double, double) for pre-filled burst capacity#8425
c-wchen wants to merge 1 commit into
google:masterfrom
c-wchen:feature/rate-limiter-burst-init

Conversation

@c-wchen

@c-wchen c-wchen commented May 14, 2026

Copy link
Copy Markdown

Summary

This PR adds a new factory method RateLimiter.create(double permitsPerSecond, double maxBurstSeconds) that creates a SmoothBursty RateLimiter with its token bucket pre-filled to capacity at creation time.

Motivation

The existing RateLimiter.create(double) always starts with an empty token bucket that fills up gradually over time (capped at 1 second of permits). This means that on startup or after a long idle period, the first batch of requests cannot immediately use the full burst capacity — they must wait for the bucket to fill.

The new overload solves the "cold start" problem: callers that know they will face an initial traffic surge (e.g., application startup, cache warming) can pre-fill the bucket so the first wave of requests is served without throttling.

Changes

  • RateLimiter.java (JRE + Android): adds public static RateLimiter create(double permitsPerSecond, double maxBurstSeconds) and a @VisibleForTesting package-private overload that accepts a SleepingStopwatch.
  • SmoothRateLimiter.java (JRE + Android): adds void prefillStoredPermits() on SmoothBursty, which sets storedPermits = maxPermits. No new fields are introduced.
  • RateLimiterTest.java (JRE + Android): adds tests covering pre-filled bucket behavior, comparison with the unfilled variant, stable rate after burst, accumulation after idle, and parameter validation.

API

// Creates a bursty RateLimiter at 5 permits/sec with a 2-second pre-filled burst bucket (10 permits).
RateLimiter limiter = RateLimiter.create(5.0, 2.0);

The returned RateLimiter behaves identically to the result of create(double) after the initial burst is consumed.

Testing

All existing RateLimiterTest cases continue to pass. New test methods added:

  • testCreateWithMaxBurstSeconds_prefillsBucket
  • testCreateWithMaxBurstSeconds_vsCreateWithoutPrefill
  • testCreateWithMaxBurstSeconds_rateAfterBurstIsStable
  • testCreateWithMaxBurstSeconds_accumulatesAfterIdle
  • testCreateWithMaxBurstSeconds_parameterValidation

RELNOTES=RateLimiter: added create(double permitsPerSecond, double maxBurstSeconds) factory method that creates a bursty rate limiter with a pre-filled token bucket for handling startup traffic surges.

…illed burst capacity.

The new factory method creates a `SmoothBursty` `RateLimiter` whose token bucket is
pre-filled to capacity on creation, so it can immediately absorb an initial traffic
surge without throttling the first wave of requests.

The existing `RateLimiter.create(double)` always starts with an empty bucket that
fills up over time (capped at 1 second worth of permits). The new overload lets
callers set both the bucket size and its initial fill level via `maxBurstSeconds`.

Implementation detail: `SmoothBursty` gains a package-private `prefillStoredPermits()`
method; the factory method calls it after `setRate`. No new fields are introduced.

RELNOTES=`RateLimiter`: added `create(double permitsPerSecond, double maxBurstSeconds)`
factory method that creates a bursty rate limiter with a pre-filled token bucket for
handling startup traffic surges.
@chaoren

chaoren commented May 14, 2026

Copy link
Copy Markdown
Member

This means that on startup or after a long idle period, the first batch of requests cannot immediately use the full burst capacity — they must wait for the bucket to fill.

If you don't need the warm up time, then can't you use

RateLimiter.create(permitsPerSecond, /* warmupPeriod= */ Duration.ZERO) 

instead?

@chaoren

chaoren commented May 14, 2026

Copy link
Copy Markdown
Member

RateLimiter.create(double permitsPerSecond, double maxBurstSeconds)

First of all, that maxBurstSeconds should be a Duration instead of a double with a hard coded time unit.

Secondly, we do have an internal overload that does pretty much exactly that. @kluever do we want to open source RateLimiter.createWithCapacity?

@c-wchen

c-wchen commented May 15, 2026

Copy link
Copy Markdown
Author

@chaoren @kluever

Thanks for the review! A few clarifications on the points raised:

1. RateLimiter.create(permitsPerSecond, Duration.ZERO) does not pre-fill the bucket

That overload creates a SmoothWarmingUp limiter, not a SmoothBursty one. When warmupPeriod = 0, the formulas yield thresholdPermits = 0 and maxPermits = 0, so the bucket has zero capacity and no tokens are ever stored.

2. Is there an open-source plan for RateLimiter.createWithCapacity?

I noticed the internal overload was mentioned. If the team is considering opening it up, that would be a great alternative to this PR. Happy to close this in favour of that approach if there is a concrete plan to expose it publicly.

3. The parameter name maxBurstSeconds is taken directly from the SmoothBursty constructor

The constructor signature is SmoothBursty(SleepingStopwatch stopwatch, double maxBurstSeconds). Using the same name keeps the API consistent with the existing internal design and makes the parameter's purpose self-evident. That said, if the team prefers a Duration-based overload for the public API, I am happy to update the signature accordingly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants