Skip to content

feat: add JMH benchmarks module + BENCHMARKS.md#11

Merged
jlc488 merged 1 commit into
mainfrom
feat/jmh-benchmarks
May 23, 2026
Merged

feat: add JMH benchmarks module + BENCHMARKS.md#11
jlc488 merged 1 commit into
mainfrom
feat/jmh-benchmarks

Conversation

@jlc488
Copy link
Copy Markdown
Collaborator

@jlc488 jlc488 commented May 23, 2026

Summary

Adds a ssrf-guard-benchmarks Gradle subproject (JMH-based) measuring the two hot paths consumers pay for at runtime, plus a BENCHMARKS.md documenting the numbers, methodology, and how to interpret them.

What's measured

Module under test Method Why it matters
ssrf-guard-core UrlPolicy.validate(URI) What every HTTP-client interceptor (RestClient, RestTemplate, WebClient, Feign, OkHttp) invokes once per request
ssrf-guard-llm JsonToolInputGuard.checkOrFormatError(json) What -springai and -langchain4j invoke once per LLM tool call

Results (JDK 21, single-fork, 5×1s warmup + 5×1s measurement)

UrlPolicyBenchmark
  allowed              5,285 ± 924   ns/op   (~5 μs)
  blockedIpLiteral     4,888 ± 750   ns/op   (~5 μs, early-exit)
  blockedHost         11,822 ± 2,035 ns/op   (~12 μs, full path + exception)

JsonToolInputGuardBenchmark
  small_allowed        5,629 ± 486   ns/op   (~6 μs, one URL top-level)
  medium_blocked       6,722 ± 290   ns/op   (~7 μs, nested + format error)
  large_allowed       24,228 ± 4,958 ns/op   (~24 μs, ~2 KB / 3 URLs)

Practical: for a 100 ms remote API call the allowed path adds 0.005% overhead. For LLM tool calls (typically 100 ms - 5 s end-to-end including LLM round-trip + actual fetch), JsonToolInputGuard is invisible.

Findings worth noting in the docs:

  • blockedIpLiteral is faster than allowed (~5 μs vs ~5 μs) because the IP-literal check fires before the whitelist lookup. Early exit on the cheapest failure path.
  • blockedHost is the most expensive (~12 μs) — passes scheme/port/IP-literal/userinfo, then fails whitelist; SsrfGuardException construction is in the measurement.
  • The JSON guard scales roughly as Jackson parse + N × UrlPolicy.validatelarge_allowed (3 URLs) is ~24 μs.

Build wiring

  • New module ssrf-guard-benchmarks listed in settings.gradle.kts.
  • Filtered out of the root subprojects { } block — no publishing, no jacoco, no -Werror (JMH-generated code emits "unused" warnings). One-line change: configure(subprojects.filter { it.name != "ssrf-guard-benchmarks" }).
  • The benchmarks module applies io.spring.dependency-management itself so the :ssrf-guard-core / :ssrf-guard-llm transitives (Spring / SLF4J / Jackson — all versioned via the Boot BOM) resolve on the jmh classpath.

How to run

./gradlew :ssrf-guard-benchmarks:jmh

Output lands at ssrf-guard-benchmarks/build/results/jmh/results.txt.

For canonical numbers (e.g. a blog post), bump fork = 3 in ssrf-guard-benchmarks/build.gradle.kts to get JIT-variance data.

Test plan

  • ./gradlew :ssrf-guard-benchmarks:compileJmhJava — clean compile
  • ./gradlew :ssrf-guard-benchmarks:jmh — full run, BUILD SUCCESSFUL in 1m 27s
  • No effect on other subprojects (filtered out of subprojects { })
  • CI verifies the compile path (jmh full-run is too slow for every PR; documented as a local concern in BENCHMARKS.md)

What's deliberately out of scope

  • DNS-time gates (SafeDnsResolver, OkHttp Dns, reactor-netty AddressResolverGroup) — dominated by I/O; meaningless to micro-benchmark
  • Spring auto-config startup cost — one-time tax, not per-request
  • Micrometer metrics overhead — benchmarks use NoOpSsrfGuardMetrics
  • GraalVM native-image performance — different shape; run the same JMH harness under nativeRun for AOT numbers

Related

A new `ssrf-guard-benchmarks` Gradle subproject measuring the two hot
paths consumers pay for at runtime:

- `UrlPolicy.validate(URI)` — what every HTTP-client interceptor (RestClient,
  RestTemplate, WebClient, Feign, OkHttp) invokes once per request.
- `JsonToolInputGuard.checkOrFormatError(json)` — what `-springai` and
  `-langchain4j` invoke once per LLM tool call.

Results (JDK 21, single-fork, 5x1s warmup + 5x1s measurement; quote
your own machine before benchmarks-as-marketing):

  UrlPolicyBenchmark
    allowed              5,285 ± 924   ns/op   (~5 μs)
    blockedIpLiteral     4,888 ± 750   ns/op   (~5 μs, early-exit)
    blockedHost         11,822 ± 2,035 ns/op   (~12 μs, full path + exception)

  JsonToolInputGuardBenchmark
    small_allowed        5,629 ± 486   ns/op   (~6 μs, one URL top-level)
    medium_blocked       6,722 ± 290   ns/op   (~7 μs, nested + format error)
    large_allowed       24,228 ± 4,958 ns/op   (~24 μs, ~2 KB / 3 URLs)

For a 100 ms remote API call, the `allowed` path adds 0.005% overhead.
For LLM tool calls (typically 100 ms - 5 s end-to-end including the
LLM round-trip), `JsonToolInputGuard` is invisible.

Build wiring:
- The benchmarks module is intentionally filtered out of the root build's
  subprojects { } block — no publishing, no jacoco, no -Werror (JMH-
  generated code emits unused warnings). The filter is the single-line
  `configure(subprojects.filter { it.name != "ssrf-guard-benchmarks" })`.
- The module applies `io.spring.dependency-management` itself so the
  `:ssrf-guard-core` / `:ssrf-guard-llm` transitives (Spring, SLF4J,
  Jackson — versioned via the Boot BOM) resolve on the jmh classpath.

Run with: `./gradlew :ssrf-guard-benchmarks:jmh`

Output: `ssrf-guard-benchmarks/build/results/jmh/results.txt`

Methodology, caveats, and how-to-read are documented in `BENCHMARKS.md`.
@jlc488 jlc488 merged commit 4196886 into main May 23, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant