@@ -116,28 +116,6 @@ detect names and locations the regex engine does not target at all. The
116116comparison is a fair _ speed_ comparison on identical inputs and entity
117117types, not an accuracy study.
118118
119- ## How these map to published claims
120-
121- - ** "~ 31µs per request" (LiteLLM guardrail, project README /
122- datafog.ai)** — not reproduced _ as worded_ . ~ 31–43 µs is the cost of
123- scanning/redacting ** one short message** ; a full 2-message request
124- through ` async_pre_call_hook ` is ~ 120 µs clean / ~ 220 µs with redaction
125- and litellm logging. Still 100–1000x below a sidecar's network hop, but
126- the claim should be reworded to per-message, e.g. _ "~ 40µs per message —
127- a request clears the guardrail in well under a millisecond"_ . Reproduce:
128- ` python benchmarks/run.py --suite guardrail ` .
129- - ** "~ 70ms per invocation including process startup" (Claude Code
130- hook)** — reproduced: 69–89 ms median across runs on this machine
131- (~ 70 ms idle, higher under load), dominated by interpreter startup.
132- _ "~ 70–90ms"_ (or "under 100ms") is the defensible phrasing. Reproduce: ` python benchmarks/run.py --suite hook ` .
133- - ** "190x performance advantage" (PyPI tagline)** — not reproduced at
134- 190x by this suite. Measured: 103–170x vs Presidio and 114–140x vs spaCy
135- NER, varying by payload. The historical 190x came from a different
136- (13.3 KB, entity-dense) document and engine configuration. Honest
137- phrasings this suite supports: _ "100x+ faster than NER-based PII
138- detection"_ or _ "up to 170x faster than Presidio on identical
139- payloads"_ . Reproduce: ` python benchmarks/run.py --suite spacy,presidio ` .
140-
141119## Fairness notes / known limitations
142120
143121- Single-threaded, single-machine, synthetic payloads. Real chat traffic
0 commit comments