feat: add JMH benchmarks module + BENCHMARKS.md#11
Merged
Conversation
A new `ssrf-guard-benchmarks` Gradle subproject measuring the two hot
paths consumers pay for at runtime:
- `UrlPolicy.validate(URI)` — what every HTTP-client interceptor (RestClient,
RestTemplate, WebClient, Feign, OkHttp) invokes once per request.
- `JsonToolInputGuard.checkOrFormatError(json)` — what `-springai` and
`-langchain4j` invoke once per LLM tool call.
Results (JDK 21, single-fork, 5x1s warmup + 5x1s measurement; quote
your own machine before benchmarks-as-marketing):
UrlPolicyBenchmark
allowed 5,285 ± 924 ns/op (~5 μs)
blockedIpLiteral 4,888 ± 750 ns/op (~5 μs, early-exit)
blockedHost 11,822 ± 2,035 ns/op (~12 μs, full path + exception)
JsonToolInputGuardBenchmark
small_allowed 5,629 ± 486 ns/op (~6 μs, one URL top-level)
medium_blocked 6,722 ± 290 ns/op (~7 μs, nested + format error)
large_allowed 24,228 ± 4,958 ns/op (~24 μs, ~2 KB / 3 URLs)
For a 100 ms remote API call, the `allowed` path adds 0.005% overhead.
For LLM tool calls (typically 100 ms - 5 s end-to-end including the
LLM round-trip), `JsonToolInputGuard` is invisible.
Build wiring:
- The benchmarks module is intentionally filtered out of the root build's
subprojects { } block — no publishing, no jacoco, no -Werror (JMH-
generated code emits unused warnings). The filter is the single-line
`configure(subprojects.filter { it.name != "ssrf-guard-benchmarks" })`.
- The module applies `io.spring.dependency-management` itself so the
`:ssrf-guard-core` / `:ssrf-guard-llm` transitives (Spring, SLF4J,
Jackson — versioned via the Boot BOM) resolve on the jmh classpath.
Run with: `./gradlew :ssrf-guard-benchmarks:jmh`
Output: `ssrf-guard-benchmarks/build/results/jmh/results.txt`
Methodology, caveats, and how-to-read are documented in `BENCHMARKS.md`.
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a
ssrf-guard-benchmarksGradle subproject (JMH-based) measuring the two hot paths consumers pay for at runtime, plus aBENCHMARKS.mddocumenting the numbers, methodology, and how to interpret them.What's measured
ssrf-guard-coreUrlPolicy.validate(URI)ssrf-guard-llmJsonToolInputGuard.checkOrFormatError(json)-springaiand-langchain4jinvoke once per LLM tool callResults (JDK 21, single-fork, 5×1s warmup + 5×1s measurement)
Practical: for a 100 ms remote API call the
allowedpath adds 0.005% overhead. For LLM tool calls (typically 100 ms - 5 s end-to-end including LLM round-trip + actual fetch),JsonToolInputGuardis invisible.Findings worth noting in the docs:
blockedIpLiteralis faster thanallowed(~5 μs vs ~5 μs) because the IP-literal check fires before the whitelist lookup. Early exit on the cheapest failure path.blockedHostis the most expensive (~12 μs) — passes scheme/port/IP-literal/userinfo, then fails whitelist;SsrfGuardExceptionconstruction is in the measurement.Jackson parse + N × UrlPolicy.validate—large_allowed(3 URLs) is ~24 μs.Build wiring
ssrf-guard-benchmarkslisted insettings.gradle.kts.subprojects { }block — no publishing, no jacoco, no-Werror(JMH-generated code emits "unused" warnings). One-line change:configure(subprojects.filter { it.name != "ssrf-guard-benchmarks" }).io.spring.dependency-managementitself so the:ssrf-guard-core/:ssrf-guard-llmtransitives (Spring / SLF4J / Jackson — all versioned via the Boot BOM) resolve on thejmhclasspath.How to run
Output lands at
ssrf-guard-benchmarks/build/results/jmh/results.txt.For canonical numbers (e.g. a blog post), bump
fork = 3inssrf-guard-benchmarks/build.gradle.ktsto get JIT-variance data.Test plan
./gradlew :ssrf-guard-benchmarks:compileJmhJava— clean compile./gradlew :ssrf-guard-benchmarks:jmh— full run, BUILD SUCCESSFUL in 1m 27ssubprojects { })What's deliberately out of scope
SafeDnsResolver, OkHttpDns, reactor-nettyAddressResolverGroup) — dominated by I/O; meaningless to micro-benchmarkNoOpSsrfGuardMetricsnativeRunfor AOT numbersRelated