Bump actions/checkout from 4 to 6#4
Closed
dependabot[bot] wants to merge 249 commits into
Closed
Conversation
mod_botshield is an Apache 2.4 module for tiered bot detection. This first commit puts the scaffolding in place: - Per-directory config with BotShieldEnabled and BotShieldDebug flags, usable in server config, VirtualHost, Directory, Location, and Files scopes (not .htaccess, intentionally). - Request handler registered at APR_HOOK_FIRST that, when debug mode is active in the current scope, returns 403 with a "Hello World" body so we can verify the module is intercepting requests end-to-end. - Makefile wrapping apxs: build, install, enable (a2enmod), disable, reload, clean. - Development Apache vhost (apache/botshield-dev.conf) serving the local testsite over HTTPS on localhost, with BotShieldEnabled globally and BotShieldDebug scoped to /debug. - .gitignore excluding the testsite docroot (not part of the module) and the usual apxs/libtool build artifacts.
Replaces the debug-only handler with a working PoW interstitial. A
cookieless request gets a self-contained HTML page with an inline
JavaScript worker that finds a SHA-256 hash with N leading zeros,
sets _bs_verified=<ts>:<hex>, and reloads; subsequent requests with a
well-formed, fresh cookie pass through. Static assets (CSS/JS/images/
fonts) are always let through so the first cookieless page still
renders its resources.
The widget is reCAPTCHA-shaped: checkbox + "I'm not a robot" + brand
column with an embedded Guardian shield SVG. All visible content is
configurable:
- BotShieldPromptText — label next to the checkbox
- BotShieldLogoFile — SVG file served inline as the logo
- BotShieldLogoLabel — small caption under the logo
- BotShieldCookieTTL — verified-cookie lifetime (seconds)
- BotShieldDifficulty — required PoW leading hex zeros
- BotShieldHelp — off | on | button; controls the explainer
- BotShieldHelpFile — HTML fragment for the help panel
- BotShieldChallengeFile — full HTML page shell; the widget splices
in at a <!-- BOTSHIELD --> marker, which
apachectl configtest rejects if missing
File-backed directives read their files once at config-parse time via
a shared helper, so no per-request I/O. Content sourced from config is
treated as trusted HTML; prompt and logo-label strings are HTML-escaped
at render. All widget CSS is scoped under .bs-stack/.bs-widget so a
custom challenge page's own layout isn't clobbered.
Accessibility: the interstitial passes axe-core WCAG 2.1 AA with 0
violations — landmarks, visible focus, prefers-reduced-motion,
aria-live status, aria-describedby on the button.
dev-vhost (apache/botshield-dev.conf) exercises each feature: /debug
keeps the 403 "Hello World" path, /about.html shows the customize-
everything case, /search.html shows help=off, and /login.html shows
a custom ChallengeFile that wraps the widget in a site-themed shell.
Three flag directives that strip the widget's visual chrome
independently, so admins can dial the default reCAPTCHA-style
widget down to a lone checkbox that they then style around in
their own page:
- BotShieldShowLogo (default on) — drops the brand column
- BotShieldShowLabel (default on) — hides the prompt text and
promotes it to the button's
aria-label so screen readers
still have an accessible name
- BotShieldShowBox (default on) — adds .bs-bare which clears
border/background/shadow/
min-width/padding
All three combined with BotShieldHelp=off reduces the rendered
widget to `<div class="bs-widget bs-bare" id="c"><button id="btn"
aria-label="..." aria-describedby="msg"><span class="bs-check"
id="cb"></span></button></div>` plus the live-status paragraph.
axe-core WCAG 2.1 AA audit stays at 0 violations across the three
configurations (full / nobox / bare). The dev vhost adds demo
scopes on /api/users.json (nobox) and /search.html (bare) to
exercise both code paths under the existing test site.
README covers what ships today: build and install, minimal vhost, directive table, per-URL scoping, the three customization layers (content / chrome / page shell), the WCAG posture, the baseline cookie's forgeability caveat, and the dev-vhost demo paths. Deliberately doesn't advertise unshipped tiers (HMAC, heuristics, SHM, Turnstile). MIT license attributes copyright to Purdue University; adjust if the owning entity is different.
First of two commits for the signed-envelope protocol. This one lands the machinery without wiring it into the request path: code compiles, new directives parse and validate at configtest time, but the handler still uses the existing M1 regex-based cookie check. Commit two flips the switch. - Link against libcrypto in the Makefile. - EVP_Digest-based SHA-256 wrapper, HMAC-SHA-256 wrapper, constant-time comparison, hex encode/decode. OpenSSL 3.x clean (no deprecated-API warnings). - Challenge struct and canonical serializer (pipe-delimited ASCII: "v|alg|salthex|noncehex|difficulty|expires_at") — deterministic input for the HMAC. - Static algorithm registry with function-pointer dispatch. sha256-zeros implemented today; sha384-zeros, sha512-zeros, pbkdf2-sha256, and argon2id are reserved slots with implemented=0 so config references fail fast. - bs_issue_challenge / bs_verify_cookie top-level functions: base64 cookie with 8 pipe-delimited fields, HMAC + expiry + alg-dispatch. Marked __attribute__((unused)) for this commit; commit two uses them. - BotShieldSecretFile directive: reads the HMAC key once at startup, refuses group/world-accessible files and secrets <16 bytes, strips one trailing newline. - BotShieldAlgorithm directive: registry lookup; rejects names that aren't recognized AND names that exist in the registry but aren't built into this module, with distinct messages for each. All six config-parse paths exercised manually (happy, bad perms, short secret, missing file, reserved-but-not-built algo, unknown algo) — each produces a clear AH00526 syntax error naming the offending input. Existing baseline (`curl https://localhost/` → interstitial) is unchanged.
Wires the M2 plumbing into the request path. The unsigned client-set
cookie is retired; every verified cookie now carries an HMAC-SHA-256
signature over a canonical form of the challenge it proves.
- Handler issues a fresh bs_issue_challenge() on every cookieless
request and embeds the full signed envelope as `window.__bsChallenge`
in the interstitial, right before the PoW worker script.
- JS worker reads the embedded challenge instead of generating its own
salt/fingerprint. Hashes salt_bytes || nonce_bytes || counter_ascii
exactly the way the server verifies. On success it base64-encodes
the 8-field pipe-delimited payload and sets _bs_verified.
- Handler's cookie path uses the new bs_get_cookie_value extractor +
bs_verify_cookie. Rejects log a specific reason:
signature mismatch / expired / insufficient leading zeros /
malformed timestamp / wrong field count / bad <field> / ...
- The old bs_has_valid_cookie (regex + TTL) function is deleted.
There is no unsigned code path post-M2.
- Handler returns 503 + misconfigured X-Botshield header when
BotShieldEnabled is on in a scope without both SecretFile and
Algorithm, so misconfig is loud rather than silently weaker.
- Dev vhost globally sets BotShieldSecretFile and BotShieldAlgorithm.
End-to-end verified: happy path through Playwright (PoW solves in
~1.6s, real page loads on reload), axe-core still 0 WCAG violations,
and curl probes confirm the four rejection modes each log their
distinct reason string.
Introduces the bs_score_add(r, penalty, ttl, reason) primitive that
every future feature — rate limits, honeypots, fake-Googlebot,
failed-captcha bumps — will feed. For M3 the state lives in
r->request_config and dies with the request; M4 will swap in an
SHM-backed store without changing the call sites.
- bs_score_entry + bs_request_score structs, bs_tier enum, threshold
defaults (silent=20 / hard=50 / captcha=80) and heuristic penalties
(missing UA=40, missing Accept-Language=15, scraper UA token=50).
- bs_get_score / bs_score_add / bs_score_reasons_joined: request-scoped
accumulator, capped at 16 reasons to prevent pathological inflation.
Reason strings must outlive the request (static or r->pool-allocated).
- bs_run_builtin_heuristics: missing User-Agent, missing Accept-Language,
scraper-UA-token matches (curl/wget/python/httpx/aiohttp/Go-http-client/
okhttp/axios/scrapy/java/okhttp/libwww/lwp-request). Case-sensitive on
purpose — we list both casings where both occur in the wild.
- bs_decide_tier computes pass/silent/hard/captcha from the running
score. Tier is logged but not acted on; every challenged request
still gets the form-PoW tier we shipped in M2 until M7/M8 land the
silent and captcha paths.
- Three new directives: BotShieldScoreSilent, BotShieldScoreHard,
BotShieldScoreCaptcha. Setters share a bs_set_score_int helper that
rejects non-integers or values outside 0..10000 with a clear message
at configtest time.
- Handler's challenge log line now includes score, tier, and a bracketed
reason list with per-reason penalties:
"challenging / (alg=sha256-zeros, difficulty=4, ttl=300)
score=65 tier=hard reasons=[missing-accept-language:15,
scraper-ua-curl:50]"
Scoring runs after the cookie check fails (or is absent), so a valid
cookied user skips it entirely. Three client shapes verified end-to-end:
empty-UA curl → 55/hard, default curl → 65/hard, Chrome-like UA +
Accept-Language → 0/pass. Routing unchanged — everyone still goes
through form-PoW, which is the plan for M3.
The _bs_verified cookie now holds a rep envelope alongside the M2 PoW
proof. Re-issues on expired-but-HMAC-valid cookies carry score and
flags forward, minus the tier's forgiveness amount. Routing is
unchanged — this ships only the cookie-schema change. M4b will make
the decoded rep feed effective_score.
Wire format grows from 8 to 14 pipe-delimited fields:
v | alg | salt | nonce | difficulty | expires_at
| score | flags | passes_silent | passes_form | passes_captcha
| challenged_at
| sig | counter
The 12-field canonical form (everything before sig+counter) is what
the HMAC now covers — a client editing any rep byte fails signature
check.
- bs_rep_state struct; embedded into bs_challenge.
- bs_challenge_canonical and bs_challenge_json extended in matching
field order.
- bs_issue_challenge takes an optional rep_in pointer. NULL means
first-ever issue; non-NULL copies fields verbatim (handler applies
forgiveness before calling).
- bs_verify_cookie grows an out_ch parameter so a handler can salvage
rep from a sig-verified but expired / bad-PoW cookie. Signature
mismatch still yields no rep.
- bs_flag_penalty placeholder (returns 0 today). E1/E4 wire real
bits here; used as the forgiveness floor.
- Three forgiveness directives (BotShieldForgivenessSilent /
ForgivenessForm / ForgivenessCaptcha) default 10 / 25 / 50.
Configtest-validated integer 0..10000.
- BotShieldCookieDomain directive with light character validation;
populates a cookie_domain field in the challenge JSON so the JS
worker can append Domain= to document.cookie.
- BotShieldCookieTTL default bumps from 300 to 3600.
- JS worker assembles the 14-field cookie payload in matching order
and honours the new cookie_domain field.
- Handler: on verify failure that isn't signature mismatch, extract
rep, apply form-tier forgiveness (score - forgive_form with floor
max(0, flag_penalty)), increment passes_form. Fresh issues start
with score=0 passes_form=1.
- Log line adds cookie_score, cookie_flags, passes=[s:… f:… c:…].
cookie_score=-1 indicates no prior rep was carried.
Gate verified end-to-end:
* Fresh browser → cookie with score=0, passes_form=1, ttl=3600.
* Hand-minted expired cookie score=40 → re-issue with score=15
(40 - 25), passes_form=prior+1.
* Expired cookie score=10 → floored at 0, not -15.
* Tampered score field → signature mismatch, no rep salvage.
* axe-core WCAG still 0 violations.
Stale code comments updated in the same pass: ttl_seconds comment
redirects to M5's flagged-IP table (M4 no longer wires anything to
SHM); handler cookie-check comment calls out the M4b follow-up.
Help string claimed 'default: 60' from the pre-M4a era; the actual constant is 3600. Pure operator-facing text fix, no behavior change. Also spells out the valid range (1..86400) while we're here, to match what the setter enforces.
The cookie rep from M4a now feeds the tier decision, and the tier
decision now affects routing: score below the silent threshold returns
DECLINED without issuing a cookie. Legitimate users with realistic
request signatures experience mod_botshield as invisible.
Handler restructure (still narrow, no server-memory changes):
- Cookie parse no longer short-circuits. A fully valid cookie used
to go straight to DECLINED; now it's just another signal feeding
effective_score. A cookied user whose request picks up a fresh
heuristic hit can land on a re-challenge.
- Heuristics run on every request regardless of cookie state —
bs_table_get-based checks, so the cost is negligible (µs).
- effective_score = heuristic_total + cookie_score + flag_penalty.
(IP-flag contribution reserved for M5.)
- bs_decide_tier(cfg, effective_score) → BS_TIER_PASS or higher.
PASS returns DECLINED with a debug log; higher tiers fall through
to the existing form-PoW issue path.
- Silent and captcha tiers still stub to form-PoW until M7 and M8
ship; form-tier forgiveness is applied on any re-issue.
Log line format tightened for clarity:
effective=N tier=X heuristic=N reasons=[…]
cookie_score=N cookie_flags=0xN cookie_ok=0|1 passes=[…]
Pass-tier requests log at DEBUG; challenged requests at INFO. The
new cookie_ok flag disambiguates "rep salvaged from expired cookie"
from "cookie fully validated and still a re-challenge was needed."
Six gate cases verified:
1. Real-browser curl → DECLINED, no cookie issued.
2. Scraper UA curl → challenged.
3. Playwright Chromium headless → real page, no interstitial,
no cookie in jar.
4. Valid cookie + scraper UA → re-challenge (heuristic pushed
effective_score over hard threshold).
5. Valid cookie with cookie_score=70 + real UA → re-challenge
(cookie score alone pushed over threshold).
6. Valid cookie with cookie_score=0 + real UA → DECLINED.
axe-core WCAG: 0 violations.
Reviewer's scope fence honored: no server memory touched, no
flagged-IP table, no Bloom filter. That's all M5.
Critical deployment note: M4b alone is rollback-vulnerable against
determined scrapers — an attacker who snapshots a lower-score cookie
can replay it to launder later flags. M5's flagged-IP table is the
mitigation. Fine for dev; don't ship to prod without M5.
Reviewer flagged that the three-outcomes comment in the handler read like a half-abandoned thought. No behavior change — just spelling out what each verify return means and what the handler does with it.
First real shared-memory feature. Closes M4b's rollback vulnerability:
an attacker who replays a snapshot of their own low-score cookie can
launder the cookie's contribution but not an IP flag that lives in
SHM and is checked on every request.
SHM layout — one segment, header + flagged-IP slot array, with room
reserved for M5b's Bloom buffers at the tail. A module-global
bs_shm_runtime struct exposes named pointers so M5b doesn't need to
re-plumb anything.
Slot layout is 32 bytes, cache-line friendly:
version : seqlock counter (even=quiet, odd=mid-write)
flags : uint32 bitmap, 0 = empty
ip : 16 bytes (IPv6-mapped v4 or v6)
expires_at : unix sec, past means stale
Reads are lockless via seqlock; writes take a short apr_global_mutex
and bump the per-slot version odd→fields→even. Readers see either
the old or new fully-consistent slot, never torn — exactly what the
reviewer asked for.
Overflow policy simplified per review: within a bounded 10-slot linear
probe, pick exact-IP match, else empty slot, else first stale slot,
else first slot in the probe (rate-limited warning logs the pressure).
Hashing is SipHash-2-4 with a key generated at startup via RAND_bytes
and stored in the SHM header. Attackers can't precompute IPs that
collide into the same bucket to evict stored entries.
Lifecycle:
- post-config creates the segment, initializes the header, randomizes
the SipHash key, creates apr_global_mutex, sets perms on systems
that need it. Runs once on the second post-config pass.
- child-init re-attaches the global mutex so each worker can lock.
- pool cleanup zeroes the runtime pointers on graceful restart.
Four flag bits with real penalties (bs_flag_penalty graduates from
stub):
honeypot_hit → +60 scanner_probe → +50
fake_crawler → +80 pow_fail_streak → +30
New directives:
BotShieldShmSize — total SHM budget, 128K..256M, default 8M.
BotShieldFlaggedIPCapacity — slot count, 1024..1000000, default 50k.
BotShieldFlagIP <bits>[,bits…] [ttl] — flag client IP on any request
that matches this scope. Intended for honeypot Locations and
scanner-trap paths; will be the integration point E4 uses.
Handler wiring:
- Client IP parsed from r->useragent_ip on every request.
- Flagged-IP lookup contributes its penalty via bs_score_add
(reason 'flagged-ip'), which composes into heuristic_total and
then effective_score.
- If the matched scope has flag_on_match set, bs_flagged_ip_add
merges the bits (refreshing TTL to the later of existing and new).
Dev vhost gets a /admin/.env honeypot scope to exercise the flow.
Gate verified:
1. Clean browser UA → pass, no cookie (no M4b regression).
2. First hit to /admin/.env → 404, IP silently flagged, log shows
'flagged IP ::1 bits=0x1 ttl=3600 scope=/admin/.env'.
3. Same IP's next request to / with clean headers → re-challenged,
log shows 'effective=60 tier=hard reasons=[flagged-ip:60]'.
This is the rollback-proof signal M4b was missing.
4. Subsequent /admin/.env hits merge the flag (no duplication),
refresh TTL.
5. SHM size cap enforced at config-parse time (128K..256M);
capacity validated 1024..1000000.
Configtest catches sizing mistakes before reload. Startup logs a
NOTICE line with the realized SHM size and capacity.
What's deliberately not in M5a: the Bloom filter (M5b), state
persistence to disk (M6), sliding-window rate counters for E2/E3
(different data structure, different milestone).
IPv6 subscribers routinely get a /64 allocation (home ISP, cloud
account). Keying the flagged-IP table on /128 lets a determined
attacker rotate through 2^64 addresses in their own prefix to shed
flags at zero cost. /64 aggregation makes the subscriber allocation
the identity unit, which matches how v6 is actually deployed.
- bs_mask_ipv6_prefix: in-place prefix mask. IPv4-mapped addresses
(the v6-mapped form from v4 clients) are detected by their
prefix (::ffff:0:0/96) and left untouched — the v4 economy is
per-/32, not per-/24.
- BotShieldIPv6PrefixLen directive, server-scope, 0..128, default
64. 128 disables aggregation.
- Handler applies the mask immediately after bs_parse_client_ip,
so both bs_flagged_ip_lookup and bs_flagged_ip_add see the
masked key.
- Dev vhost gets mod_remoteip trust for loopback so XFF-based
tests can simulate varied client addresses without fancy
network plumbing.
Gate verified with six curl probes through mod_remoteip / XFF:
1. Clean visits from same-/64 A, same-/64 B, diff-/64, v4_A,
v4_B all pass.
2. Honeypot hit from 2001:db8:dead:beef::1 flags the /64.
3. 2001:db8:dead:beef::cafe (different /128, same /64) is
re-challenged — aggregation works.
4. 2001:db8:dead:bef0::1 (different /64) is NOT challenged —
no over-aggregation.
5. Honeypot hit from 203.0.113.10 flags /32, not /24 —
203.0.113.99 stays clean.
6. 203.0.113.10 itself stays challenged — v4 flag persists.
Log lines preserve the original full address in the "flagged IP"
and "client" fields, so operators can still see the specific /128
that tripped the flag.
Operators behind an LB or reverse proxy need mod_remoteip configured so r->useragent_ip is the real client IP rather than the proxy's. Without it, the flagged-IP table and every IP-based signal will key on the edge hop and challenge every legitimate visitor. Also bumps the BotShieldCookieTTL default in the directive table from the stale 300 to the actual 3600.
Two bit-array buffers at the tail of the existing SHM segment; one
active for writes, both scanned for queries. Rotate-on-insert via CAS
on bloom_next_rotate guarantees correctness even during quiet periods
(first insert after the window boundary triggers the flip). No
mod_watchdog dependency today — the only thing it would buy is
rotation during zero-traffic gaps, which doesn't change semantics.
Sizing: ~10 bits/IP at 1% FP, k=7. Default 1M working-set IPs =
1.25MB per buffer (2.5MB total) plus the existing 1.6MB flagged-IP
table comfortably fits the 8M default SHM budget. Two new server-
scope directives:
BotShieldBloomIPs 1000000 range 1000..10000000
BotShieldBloomWindow 604800 range 3600..2592000 (rotation at /2)
SipHash reuses the key generated at startup for the flagged-IP table,
then Kirsch-Mitzenmacher double hashing derives the 7 bit indices
(two SipHash rounds per insert/query regardless of k).
Rotation lifetime: guaranteed minimum window/2 (3.5 days @ default),
max window (7 days). Semi-graceful aging: an IP inserted right after
a rotation lives the full window; an IP inserted right before a
rotation lives window/2.
Feed/read policy matches the plan exactly:
- Feed: only on the challenge-issue branch (after tier > pass)
- Read: only when !have_prior_rep (cookieless or sig-mismatch)
— sig-verified-but-expired cookies skip the read because we
already know this is a returning client
Handler integration adds one score entry 'first-sight-ip:5' in the
reasons list, small enough to never tip a clean request over the
silent threshold alone but additive with scraper-UA and missing-
header heuristics.
Verified by curl+XFF probes through mod_remoteip:
1. Fresh v4 scraper → first-sight-ip:5 in reasons, tier=hard.
2. Same v4 IP again → first-sight-ip gone from reasons.
3. Different v4 IP → first-sight-ip back.
4. v6 IP → first-sight-ip works independently.
5. Clean UA on fresh IP → passes (+5 alone below silent=20).
Startup log now reports Bloom geometry alongside flagged-IP capacity:
"SHM ready: 8388608 bytes, flagged-IP capacity 50000,
Bloom 1000000 IPs per 604800 s (2x 1250000 bytes)".
Rotation correctness isn't exercised by a short test — the code path
is minimal and the CAS serialization is the same pattern as M5a's
flagged-IP updates. Treat as plausible-not-battle-proven until a long
run demonstrates clean half-window boundaries.
Narrow, boring format. Goal is deployment continuity — hard-won sparse
state survives a systemctl reload/restart without operator work. Not
designed as generalizable persistence infrastructure.
Scope per reviewer guardrails:
- Persist flagged-IP entries (dropping ones whose TTL has passed
at load time) and both Bloom buffers with active_index plus
next_rotate_at.
- Atomic write-rename at graceful shutdown via pool cleanup.
- Any load error — missing, too small, bad magic, bad version,
too old, bad checksum, dimension mismatch — degrades to "start
fresh" with a NOTICE. Never fatal.
- No periodic snapshots. Crashes lose state since last graceful
shutdown. Add later only if operationally needed.
- No generic serializer, no TLV/record framing, no migration
between format versions.
File format (x86-64 little-endian, not cross-arch portable):
header magic 'BSHD', version=1, saved_at
siphash 16 B key — persisted so flagged-IP entries land in the
same buckets on restart (without this, lookups miss
every preserved entry)
flagged capacity + raw slot array
bloom buf_bytes + active_index + next_rotate + buf0 + buf1
trailer FNV-1a-64 over everything above
New directive:
BotShieldStateFile /var/lib/botshield/state.bin
(server-scope; must be at main-server, not inside <VirtualHost>;
ignored-without-warning in vhost — the module-global SHM footgun
we've already documented as a pending warn-at-post-config item)
Slot seqlock versions are zeroed at load time so live readers see
a clean 'fresh slot' state post-restore.
Gate verified end-to-end: flag ::1 via honeypot → systemctl reload
(save) → systemctl restart (load) → ::1 still carries flagged-ip:60
on the next request, exactly as expected. Log shows
'state loaded from ... (age 1 s): flagged kept 1, dropped-stale 0,
bloom buffers restored'.
Two bugs found and fixed during the gate run:
* SipHash key wasn't persisted — post-config always generates a
random one, so restored flagged entries ended up in "wrong"
buckets after restart. Added 16-byte key section to the file
(between header and flagged section); restored key takes
precedence over the freshly randomized one.
* %lld format wasn't supported by APR's vformatter — passed
through literally in log output. Switched to APR_INT64_T_FMT.
Persistence is optional: unset BotShieldStateFile leaves state
purely in-memory. Default is unset so upgrading from M5 is a no-op.
Reviewer found three real issues in the M6 as-shipped. Addresses two code-level findings; the third (PLAN/code drift on periodic saves) is a doc-only change handled in the untracked PLAN.md. 1. Concurrency: the flagged-IP memcpy during save was lockless. A racing bs_flagged_ip_add writer could be captured with version-odd mid-write state (half-written ip[]/flags). Load forcibly resets version to 0, which would publish the half- written entry as a legitimately authored slot — worse than a torn read; creates false flags after restart. Take the global mutex briefly during the flagged-table memcpy. Bloom buffers remain lockless — they're mutated by single-byte atomic OR, and per-byte reads are inherently torn-free, so memcpy captures a well-defined snapshot with no post-load forgery risk. 2. Durability: fsync on the temp file is necessary but not sufficient. On some filesystems the rename's directory entry can still be lost on crash/power-loss if the containing directory's inode hasn't been flushed. Added bs_fsync_parent_dir: open(dir,O_RDONLY) + fsync + close after the successful rename. Failure is logged at INFO and non-fatal (the rename already succeeded; FS will flush eventually). APR doesn't expose a directory-fsync helper, so this is plain POSIX. Persistence gate re-run after the fixes: flag ::1 → graceful reload (save) → systemctl restart (load) → log shows 'state loaded from ... (age 1 s): flagged kept 1, dropped-stale 0, bloom buffers restored', subsequent ::1 request still gets 'effective=60 tier=hard reasons=[flagged-ip:60]'. As a side-effect of the new format-string discipline, the %lld → APR_INT64_T_FMT fix from the initial M6 commit is reconfirmed.
Closes the last gap the reviewer flagged on M6. The on-graceful- shutdown save bounded state loss to "since last clean restart" — fine for patch-reboot cycles, useless against a crash or kill -9. mod_watchdog gives us a timer callback that re-invokes the existing bs_state_save path at a configured cadence. Integration is intentionally thin — no scheduler framework, no internal tick infrastructure. Standard APR optional-function pattern: look up ap_watchdog_get_instance and ap_watchdog_register_callback at post-config; if either is NULL we log NOTICE and degrade to shutdown-only. On Ubuntu's apache2 build mod_watchdog is compiled into the main binary, so the degraded path is correct-by-inspection but not runtime-testable here. - New directive BotShieldStateSaveInterval, default 300 s. 0 disables periodic saves (graceful-shutdown save still runs); non-zero is range-validated 30..86400. - Callback reuses bs_state_save unchanged — same mutex-protected flagged copy, same fsync + rename + parent-dir fsync, same load/save format. - Dev vhost set to 30 s for observation; production sites keep the 300 s default. Gate: Startup → "periodic state save enabled via mod_watchdog every 30 s" ~28 s later → "state saved to /var/lib/botshield/state.bin (4100060 bytes)" from the watchdog tick, no operator action. Edge-case defense: if ap_watchdog_get_instance returns APR_SUCCESS with a NULL watchdog pointer (docs say it shouldn't, but defensive coding) we treat it as a registration failure rather than log "enabled" without registering.
This is a single catch-up commit covering ~5 milestones of work that
didn't get committed at their own natural boundaries. Per-milestone
granularity resumes starting with M10.2. The milestones and main
changes, in the order they shipped:
M7 — silent-PoW tier via auto-submit interstitial
* auto_tier field added to bs_challenge envelope (HMAC-covered);
cookie grows from 14 to 15 pipe-delimited fields
* silent-tier route: renders a minimal "checking your browser"
splash + fires the SHA-256 PoW on DOMContentLoaded instead of
waiting for a click
* bs_tier_name(BS_TIER_HARD) returns "form" (was "hard") to match
the decision-log enum documented later in M9.1
* BotShieldForgivenessSilent applied when prior cookie carried
auto_tier=1; passes_silent / passes_form counters bookkeeping
M8 — captcha tier with six third-party providers
* libcurl + json-c linkage (Makefile)
* bs_captcha_provider registry: Turnstile, hCaptcha, reCAPTCHA v2,
reCAPTCHA v3 (with BotShieldRecaptchaV3MinScore), Friendly Captcha
(body-field variant solution/response), GeeTest v4 (HMAC-signed
siteverify body via provider-specific siteverify_fn)
* captcha-turnstile / -hcaptcha / -recaptcha-v2 / -recaptcha-v3 /
-friendly / -geetest all register as cookie-alg entries (no-op
PoW verify — provider already did the work)
* Three interstitial templates: render pattern, execute pattern
(reCAPTCHA v3 grecaptcha.execute), initGeetest4 pattern
* Per-provider verify URL: /<prefix>/captcha-verify/<name>, so
multiple providers cohabit one vhost
* BotShieldEndpointPrefix directive (default /botshield) for the
module-owned URL namespace
* Response parser uses json-c (replaced earlier strstr approach);
falls back from "error-codes" to "errors" for Friendly Captcha v1
* Fail-open policy on timeout / network error / non-2xx HTTP
M8.1 — captcha-verify endpoint hardening
* HMAC-signed _bs_captcha_pending cookie at interstitial render,
required at verify before any libcurl call
* Per-IP rate limit ring (cv_rate_slots) + directive
BotShieldCaptchaRateLimit (default 30/min)
* Global in-flight semaphore (cv_inflight) + directive
BotShieldCaptchaMaxInFlight (default 64)
* Per-IP log throttle (cv_log_slots) — REJECTED/WARNING lines
collapse to one per 60s with "x N in last 60s" suffix
* Content-Type prefilter (415 on non-form-urlencoded)
* Body size cap lowered to 8 KB
* Cache-Control: no-store on every verify response
M9 — observability
M9.1 structured decision log: one key=value line per terminal
handler outcome, enum values for tier/outcome/cookie/provider,
parseable by a 50-line awk script. Vocabulary frozen.
M9.2 SHM-backed counters: 26 counters mapped mechanically 1:1 to
M9.1 enum strings (tier_<t>_total, outcome_<o>_total, etc.)
plus persistence gauges and on-demand Bloom popcount / flagged
IP usage. No parallel vocabulary.
M9.3 /<prefix>/metrics Prometheus 0.0.4 text export (deterministic
ordering, hardcoded names, 41 metrics) plus mod_status
optional hook (both HTML and ?auto modes). Parity gate asserts
delta(counters) == count(decision log) per enum.
M10.1 — sanitizer readiness + ASan/UBSan run
* Pre-M10 fixes prompted by reviewer findings:
- curl_global_init moved to bs_post_config (thread-safety; the
old lazy-init static guard raced under mpm_event). Failure is
now checked and fails post_config loud.
- __thread storage for bs_gauges (eliminates same-process races
on the metrics gauge cache)
- __atomic_load_n on CV ring observation reads (rate-limit +
log-throttle); CAS below still validates
- __atomic_load_n per u64 on the Bloom popcount scan path
- Bloom rotation: atomic-store loop (not memset) + brief global
mutex hold — the atomic loop is what TSAN actually wants; the
mutex is a cheap additional serialization point
* make sanitize / install-sanitize targets (ASan + UBSan, -O1,
frame pointers, -g). object-size check explicitly disabled with
a comment — it produces spurious reports against APR pool
sub-allocations and isn't useful for this code.
* systemd drop-in template (apache/sanitize.conf style — dev
scratch, not committed) for LD_PRELOAD runs
* bs_state_save duration: clamp to >=0 (apr_time_now is wall-clock
and can go backward on NTP adjustment); also removes dead-code
first calculation that was immediately overwritten
Docs
* README.md: status paragraph updated through M10.1; how-it-works
extended to cover captcha tier + module-owned endpoints; five-
category directives table covering all 33 shipped directives;
new sections on "Module-owned endpoints" and "Observability";
dev-vhost paths table split into Widget / Captcha / Endpoints
* Milestone names normalized from M#letter to M#.# throughout
source comments and docs
* Several stale forward-references in source comments fixed
("M9 will add counters" -> "M9.2 counts these as ..."; "reserved
for M5b" -> accurate current-state descriptions; etc.)
PLAN.md continues to live untracked per its own "Living doc. Not
committed." convention — it captures the forward-looking plan, not
the shipped state; README is the source of truth for what ships.
Per the M11 plan rewrite (replacing the old M11 packaging section),
this commit lays out the regression + acceptance test framework and
ports four example tests to prove the shape. tests/ replaces the
ad-hoc scripts that previously lived in /tmp across milestones.
Framework is deliberately light — bash + curl + python3 + awk, no
test-runner dependency. Each test sources tests/lib/common.sh and
emits PASS/FAIL/SKIP lines; tests/run walks categories and reports
a summary.
Layout:
tests/
├── README.md prerequisites, setup, how to run, how
│ provider secrets work, how to debug a
│ failing test
├── run dispatcher with --only / --match / --list
│ / --verbose flags
├── lib/
│ ├── common.sh bs_curl, bs_curl_split, metrics_snapshot,
│ │ metrics_delta, log_mark/log_slice,
│ │ fetch_pending_cookie, assert_* helpers,
│ │ enum sets (TIERS/OUTCOMES/COOKIES/PROVIDERS)
│ └── decision_gate.awk M9.1 validator, ported from /tmp
├── setup/
│ ├── provision.sh idempotent one-shot: apt packages, build,
│ │ module install, self-signed cert,
│ │ /etc/botshield/* secrets at 0600,
│ │ /var/lib/botshield/ owned by www-data,
│ │ dev vhost enabled, mod_status/remoteip/ssl,
│ │ mpm_event selected, configtest + reload
│ └── reset-state.sh delete state.bin, restart apache — for
│ tests that need a known-clean start
├── tools/
│ └── solve_pow.py SHA-256 leading-zero PoW solver + cookie
│ payload builder, for PoW round-trip tests
├── unit/ no-Apache-needed tests
├── integration/ running-Apache tests
├── acceptance/ end-to-end user journeys (empty; M11.3)
└── stress/ wrk, MPM matrix, soak (empty; M11.3)
Initial ports (one per category pattern):
unit/decision_format.sh
Drives 7 requests across multiple outcome classes, captures the
log slice, runs decision_gate.awk. Asserts every decision line
parses with zero enum violations.
integration/m7_silent_tier.sh
Cookieless silent-band probe → extract __bsChallenge →
solve_pow.py → base64 15-field cookie → replay → assert tier
drops to pass (no X-Botshield: challenge). Catches cookie-
envelope, canonical HMAC, and silent-tier routing regressions.
integration/m8_captcha_turnstile.sh
Three outcomes against live Cloudflare: always-pass → 303 +
captcha-ok + _bs_verified set; always-fail → 403 + error-codes
in log; 100ms timeout → fail-open + 'failing open' WARNING.
Self-skips if challenges.cloudflare.com unreachable. Safely
swaps the secret file/timeout in the dev config and reverts
before asserting (so a mid-test failure doesn't leave the box
mis-configured).
integration/m9_3_metrics_parity.sh
Snapshot /metrics, drive a diverse traffic mix, snapshot again,
for every enum value in tier/outcome/cookie/provider assert
delta(counters) == count(decision log lines). M9.3's core claim:
counters never drift from the log vocabulary.
Provider secrets without published dummies (Friendly, GeeTest, real
reCAPTCHA v3) read from env vars; tests that need them t_skip when
the env isn't set.
Smoke run: tests/run → 4 passed, 0 skipped, 0 failed.
M11.2 will port the remaining per-milestone gates (M2, M5.1, M5.2,
M6, M6.1, M8.1, M9.1, M9.2). M11.3 adds acceptance flows + CI hook.
Rate-limited soak driver for the overnight-run form of M10 hardening.
Three files in tests/stress/:
soak_load.py
stdlib-only load generator (urllib + threading). Target rps is
exact — 1/rps interval between sends, each request on its own
short-lived daemon thread so server latency doesn't throttle the
sender. Mix is 70/20/10 pass/form/captcha-render against the dev
vhost; all internal (no third-party egress).
Rate limiting matters for an overnight run because wrk's default
concurrency-based throughput can hit 7000+ rps on localhost HTTPS,
which would dump tens of GB of logs. A real rate-limited loader
keeps log growth predictable (~30 KB/s at 50 rps) and lets the
analyzer's log-growth budget be scaled by duration rather than
a hand-waved absolute number.
soak.sh
Driver. Parses --duration (2m / 30m / 8h etc.), --rps (default 50),
--report. Launches soak_load.py, samples /metrics + worker RSS +
log file size at an interval scaled to total duration (every 15s
for <=5m smoke runs, every 5m for <=1h, every 30m for overnight).
Each sample is one line in the report file, all counters compressed
to a single field for post-hoc parsing. Runs the analyzer on
completion and returns its exit code.
Designed to be nohup'd and left alone overnight — progress is
visible via tail -f on the report file.
soak-analyze.sh
Post-mortem on the report. Asserts:
- RSS growth from first to last sample < 200 MB
- log file growth < 100 MB per hour (MAX_LOG_GROWTH_PER_HOUR_MB
env override; floor at 50 MB for short smoke runs)
- five critical counters (tier_{pass,form,captcha}_total,
outcome_{declined,challenged}_total) are monotonically non-
decreasing across the sample series — any decrease means SHM
reset mid-soak (graceful restart, crash, etc.)
- zero 'metrics: unknown' WARNING lines in the apache error log
since soak start (M9.2 vocabulary-drift guard)
- zero crash signatures (SIGSEGV, segfault, core dumped,
AddressSanitizer) in journalctl since soak start
Exits 0 on clean pass, 1 on any failure. Intended as the M10.4
overnight gate: "did the module stay healthy for hours?"
Smoke: 2-minute run at 50 rps completed clean. RSS grew 128 KB, log
grew 2.4 MB (72 MB/hr, inside budget), all five counters monotonic,
no drift, no crashes. Ready for overnight invocation.
To run tonight:
nohup tests/stress/soak.sh --duration 8h > /tmp/bs_soak_stdout.log 2>&1 &
The report file is timestamped (/tmp/bs_soak_<ts>.report) so multiple
runs don't collide; the analyzer reads it in the morning.
Ten new integration tests covering every shipped milestone from M2
onward, plus framework fixes surfaced by the full-suite run.
Ports (alphabetical — matches dispatcher run order):
m2_cookie_hmac
Issue a silent-tier challenge, solve the PoW, mint a valid
15-field cookie, flip one hex char in the signature, replay,
assert the log shows "signature mismatch" rejection. Proves the
M2 HMAC envelope still rejects tampered cookies.
m5_1_flagged_ip
Trip /admin/.env (honeypot) from a fresh XFF IP, then hit / from
the same IP, assert the second decision carries reason=flagged-ip.
Proves the flagged-IP SHM table's mutex-guarded write + lockless
read path still works end-to-end.
m5_2_bloom_first_sight
Bloom-fresh IP + missing Accept-Language → first hit carries
first-sight-ip, second hit does not. Proves M5.2 rotating Bloom
records correctly and doesn't mistakenly re-penalize repeat IPs.
m6_state_round_trip
Flag an IP, systemctl restart apache, assert the flag and gauge
survived via state-file save/load. Exercises the full snapshot-
write-rename-fsync-load dance.
m6_1_periodic_save
Wait 40 seconds (30s BotShieldStateSaveInterval + slack), assert
botshield_state_saves_total bumped. Also asserts the duration
column in the log line is non-negative, as a regression guard
for the apr_time_now wall-clock-rollback fix landed in M10.1.
m8_1_content_type
Verify POSTs with Content-Type: application/json or no CT header
return 415 + X-Botshield: captcha-bad-content-type, before libcurl.
Form-urlencoded bodies pass the prefilter.
m8_1_pending_cookie
No cookie → 403 + captcha-pending-missing. Tampered cookie (wrong
HMAC) → 403. Valid cookie gets past the guard. Proves the HMAC-
signed challenge-origin cookie works.
m8_1_rate_limit
40 parallel POSTs from one time-salted XFF IP → at least one 429,
plus the log throttle collapses the burst into a single "captcha-
verify rate limit" log line. Serial POSTs to Cloudflare's
siteverify can span the 1-minute rate window boundary and reset
the count mid-test; parallel issue is a sub-second burst that
always clears the 30/min cap cleanly.
m9_1_enum_coverage
Drive traffic to hit every reachable outcome value (declined,
challenged, verified, rejected, failopen, rate_limited,
pending_missing) and assert each appears >=1 time in the
decision log slice for this test.
m9_2_vocab_sync
Drive mixed traffic and assert zero 'metrics: unknown' WARNING
lines in the apache error log. Any such line means a decision-
log emission used an enum string the counter lookup didn't know
about — vocabulary drift between M9.1 and M9.2, which the
framework must catch before it merges.
Framework fixes (surfaced by running the full suite and watching
things flake):
tests/run:
- stress/ moved out of the DEFAULT_CATEGORIES list. Opt in via
--only stress (or --only all to include alongside). Prior
default included stress, which ran tests/stress/soak.sh with
its default --duration 8h in the middle of a tests/run call.
Very bad.
- --only all resolves to the full set (unit+integration+
acceptance+stress).
tests/tools/soak-analyze.sh (moved from tests/stress/):
soak-analyze is a helper, not a test. Moved to tools/ so the
dispatcher doesn't list/run it. soak.sh's invocation path
updated to ../tools/soak-analyze.sh.
tests/integration/m7_silent_tier, m2_cookie_hmac, m5_2_bloom_first_sight:
Time-salted XFF IPs across two octets. $RANDOM alone gave a
small enough range that re-runs within the Bloom window would
occasionally pick an already-seen IP, and first-sight-ip
wouldn't fire, and the score dropped below silent so no
challenge was served. Time-salted IPs (203.<range>.X.Y with
X.Y derived from epoch seconds) give ~65k unique IPs, well
past any weekly re-run cadence.
tests/integration/m8_captcha_turnstile (timeout branch):
Accept either "failing open" WARNING or "captcha OK" log —
Cloudflare occasionally responds under 100ms and the fail-open
branch doesn't fire. Either outcome proves the code path is
exercised; only a silent "neither" is a real failure.
Smoke: tests/run → 14 passed, 0 skipped, 0 failed. Covers M2, M5.1,
M5.2, M6, M6.1, M7, M8.1, M8 (Turnstile), M9.1, M9.2, M9.3 plus the
unit decision-log format validator. Remaining ports (hCaptcha /
reCAPTCHA v2 / Friendly / GeeTest) follow the same pattern and can
land in M11.3 alongside acceptance flows + CI hook.
tests/acceptance/:
- pass_tier.sh — clean browser → 200, no challenge (the happy path)
- form_tier.sh — cookieless suspicious UA → interstitial → PoW solve →
cookie → pass tier
- captcha_tier.sh — /captcha-demo → interstitial → Turnstile solve →
verified cookie → normal-path pass
tests/integration/m8_captcha_*.sh: ports for the five providers the
Turnstile test had been standing in for. Turnstile / hCaptcha /
reCAPTCHA v2 have published always-pass test keys so OK and REJECTED
are both exercised. reCAPTCHA v3 / Friendly / GeeTest don't publish
test keys — those tests do a plumbing smoke (body-field extraction,
siteverify round-trip, decision-log shape) using placeholder secrets
and SKIP the OK branch unless BS_{RECAPTCHA_V3_TOKEN,FRIENDLY_SOLUTION,
GEETEST_TOKEN} is set.
m8_captcha_turnstile.sh: REJECTED assertion moved from the "captcha
REJECTED" prose line (throttled 1/IP/60s) to the decision line
(never throttled) — rapid re-runs were flaking on log throttle.
Same fix applied to hCaptcha + v2 tests.
tests/stress/soak.sh: new --long flag for the 8h overnight run.
Default is now a 60s/25rps smoke that finishes in a minute so
`tests/run --only stress` can run it in CI on every PR without
camping for hours.
.github/workflows/ci.yml: two jobs.
- test: per-PR, runs unit + integration + acceptance + the 60s
soak smoke on ubuntu-24.04. Live-provider secrets from repo
secrets; tests SKIP gracefully when absent.
- nightly: scheduled 06:00 UTC + manual trigger. Runs the full
--long 8h soak and uploads the report as an artifact.
Full suite: 22 passed, 0 skipped, 0 failed.
ci.yml: header and nightly job comments claimed sanitizer + full stress matrix, but the job only runs the 8h soak. Rewrote the comments and renamed the job "8h soak" so the wiring matches the copy. Pointed the forward-looking references at PLAN.md (M10.1 sanitizer is a local build target, MPM matrix is planned for M11.7). README.md: line 22 still said "production hardening is the next planned milestone" — stale since M10.1–M10.3 shipped and the M11 test tree landed. Rewrote the status paragraph to reflect what's actually in the tree (sanitizer-clean, load-tested, MPM-matrix- verified, 22-test suite, CI gate, 60s soak smoke per PR) and point at PLAN.md for the pytest+Playwright rebuild in M11.4+. tests/run: zero-exit with no PASS: or SKIP: markers was counted as a pass. A script that silently returns 0 (truncated, stubbed, early returned) would score green. Now requires at least one marker and fails loud with a clear diagnostic otherwise. Verified with a mock empty test; real suite still 22 green.
New package tests/botshield_test/ with the reusable primitives the
bash suite had scattered across common.sh and per-test boilerplate:
- client.py — httpx wrapper with defaults for self-signed cert,
timeouts, XFF injection. Cookies go out as a raw
Cookie header; httpx's deprecated jar is bypassed.
- apache.py — reload(), restart(), reset_state(), and crucially
config_override() as a context manager that mutates
the dev vhost, reloads, yields, and ALWAYS reverts
on exit. No more sed-swap-revert that leaks broken
config on test failure.
- logs.py — log_slice() context manager + _LogSlice class with
.decision_lines(**filters) returning structured
dicts. Replaces grep -q on prose log lines.
- metrics.py — snapshot()/delta()/value() for /metrics. Returns
dicts; tests assert on named fields.
- cookies.py — fetch_pending_cookie, solve_pow, build_cookie,
tamper_signature, extract_challenge (from HTML).
Absorbs tests/tools/solve_pow.py.
- ips.py — fresh_ip() allocator in 100.64.0.0/10 (CGN space,
never touched by the legacy bash suite so Bloom
slots stay clean across a mixed run). rate_slot=True
uses 198.51.100.0/24 with rate-ring-aware entropy.
pytest-xdist worker id factors into the mix.
- enums.py — single source of truth for TIERS/OUTCOMES/COOKIES/
PROVIDERS.
- config.py — paths + constants, all overridable via env vars
for CI or non-default vhost layouts.
Pytest infra:
- pyproject.toml — PEP 621 + tool.pytest.ini_options with markers
(serial, live_network, live_provider, slow),
timeout=120s, xfail_strict=true, warnings=error.
- requirements-test.txt pins httpx / pytest / xdist / timeout.
- conftest.py fixtures: apache (session), fresh_ip, rate_slot_ip,
log_slice, config_override, pending_cookie,
clean_state.
Three POC tests proving the framework handles the hardest cases:
- test_cookie_hmac.py — silent-tier PoW solve + cookie
tamper + signature-mismatch assert
(port of m2_cookie_hmac.sh).
- test_bloom_first_sight.py — fresh IP fires first-sight-ip,
repeat doesn't (port of m5_2).
- test_rate_limit.py — 40 parallel POSTs via ThreadPoolExecutor,
assert 429s + log throttle (port of
m8_1_rate_limit.sh).
tests/run: new 'pytests' category runs pytest once per invocation
and folds its passed/failed/skipped counts into the suite summary.
Default category list now (unit integration acceptance pytests).
tests/setup/provision.sh: apt installs python3-venv; creates
tests/.venv owned by the unprivileged user; installs
requirements-test.txt and the botshield_test package in editable
mode. Idempotent.
tests/README.md: documents the bash-and-pytest-side-by-side shape
for the M11.4–M11.5 transition window.
.gitignore: tests/.venv/, pytest caches, egg-info excluded.
Full suite: 25 passed, 0 skipped, 0 failed (22 bash + 3 pytest).
Pytest-only: 3 passed in 0.55s.
Pytest becomes the canonical test framework. Bash scripts move to
tests/bash-legacy/ as a reference, not executed. Full coverage
preserved: 32 pytest tests pass in the default run, 33 with --slow.
Tests ported (one pytest file per bash script):
unit/decision_format → test_decision_format.py
m5_1_flagged_ip → test_flagged_ip.py
m6_state_round_trip → test_state_round_trip.py (serial)
m6_1_periodic_save → test_periodic_save.py (serial + slow)
m7_silent_tier → test_silent_tier.py
m8_1_content_type → test_verify_content_type.py
m8_1_pending_cookie → test_verify_pending_cookie.py
m9_1_enum_coverage → test_enum_coverage.py (serial + live)
m9_2_vocab_sync → test_vocab_sync.py
m9_3_metrics_parity → test_metrics_parity.py (serial + live)
acceptance/pass_tier → test_acceptance_pass_tier.py
acceptance/form_tier → test_acceptance_form_tier.py
acceptance/captcha_tier → test_acceptance_captcha_tier.py
Six captcha-provider bash scripts collapse into one parametrized
pytest file (test_captcha_providers.py) driven by a ProviderSpec
dict in botshield_test/providers.py. Four parametrized tests across
the six providers:
- test_captcha_ok (3 expansions: turnstile, hcaptcha, v2)
- test_captcha_plumbing_smoke (6 expansions: all providers)
- test_captcha_rejected_via_bad_secret (2 expansions: turnstile, v2)
- test_captcha_body_field_name_stable (2 expansions: friendly, geetest)
Framework additions:
- botshield_test/providers.py — per-provider specs (field names,
ok tokens, bad-secret paths, env
vars for real-token opt-ins)
- botshield_test/logs.py — validate_decision() replaces
decision_gate.awk; parses required
keys + enum membership
- botshield_test/apache.py — config_override() loosened to
replace-all by default (most
directives appear in multiple
Location blocks); optional count=N
for strict one-match caller intent
- botshield_test/config.py — APACHE_ERROR_LOG (main apache log,
where state-save lines land per
mod_watchdog callback)
- pyproject.toml — register acceptance + browser
markers (M11.6 placeholder)
periodic_save test asserts against the log's timestamps rather than
the state_saves_total counter, because any neighbouring test that
calls config_override triggers reload → SHM counter reset (apr_shm
bound to pconf). Log survives reloads; counter doesn't. Marked
serial + slow.
tests/run dispatcher rewritten:
- pytests is the only default category (bash dirs gone)
- --parallel flag: two-phase xdist (non-serial -n auto, then
serial sequential). Default stays sequential for reliability.
- --slow flag: opt-in for @slow tests (periodic_save's 40s wait)
- stress/ still runs the bash soak driver via --only stress
bash-legacy/README.md documents the coverage map and the deletion
criteria (end of M11.7, once pytest has been stable in CI for a
week and Playwright acceptance is in).
.github/workflows/ci.yml: test step now `tests/run --parallel
--verbose`. tests/README.md rewritten for the pytest-only shape;
removed dead references to log_mark / common.sh / m9_3 template.
run_pytests() used to print a skip message and return when
tests/.venv was missing, without bumping fail / fails[]. Summary
then said 0/0/0 and exit was 0 — a literal false green if the
canonical runner ever ran without provisioning. Now prints
"FAIL: no pytest venv" with the remedy and fails the run.
_one_pytest_invocation() treated any nonzero pytest exit as a
failure. pytest returns 5 for "no tests collected," which is
expected during --parallel when one phase's marker filter
intersected with --match/--slow yields an empty selection
(e.g. --match state_round_trip only matches serial tests, so
the "not serial" phase collects zero). Exit 5 is now treated
as benign with a comment referencing pytest's documented exit
codes. Verified both edge cases: state_round_trip-only (serial
match) and cookie_hmac-only (non-serial match) both exit 0 under
--parallel.
Docs caught up:
- README.md line 22–30: status paragraph was frozen at M11.3
("Next up (M11.4+) is rebuilding..."). Rewritten to describe
what actually shipped through M11.5 — 32-test pytest suite,
botshield_test framework components, bash-legacy archive.
- tests/pyproject.toml: comment still referenced the transition
period; now just points at bash-legacy for the exclusion.
M11.5 moved tests/lib/ into tests/bash-legacy/lib/ but stress/soak.sh
still did `source "$HERE/../lib/common.sh"`. Any invocation died
immediately with "No such file or directory" and the analyzer then
"passed" on the empty 2-sample report (all counters read -1).
Solves both:
- New tests/stress/lib.sh: minimal self-contained bash helpers for
the soak driver. Exports BASE, ERROR_LOG, and bs_curl. No other
dependencies; soak no longer touches bash-legacy/.
- soak.sh sources it at the new path.
Verified with a 60s smoke: 547 pass-tier + 246 challenged outcomes,
RSS delta 4.8 MB, log delta 414 KB, analyzer reports PASS on
real numbers.
Separate concern worth flagging: the analyzer's PASS check on the
empty 2-sample report (monotonic check trivially passes when every
counter reads -1) is a false-green. Filed mentally for M11.7 — it
belongs alongside the soak driver's port to pytest.
Seven browser-driven tests catch regressions no request library can
see: interstitial JS execution, cookie attribute enforcement, DOM
presence of captcha widgets, auto-submit form semantics.
Framework additions:
- pytest-playwright + playwright pinned in requirements-test.txt
(browser binary into ~/.cache/ms-playwright, cached across runs).
- conftest.py:
* browser_context_args overrides the stock fixture to trust the
self-signed dev cert (ignore_https_errors=True).
* bs_browser_context: Chromium context pinned to one Bloom-fresh
IP via X-Forwarded-For, Accept-Language deliberately BLANK so
cookieless requests reliably land in silent/form tier. Every
test that wants to exercise the challenge → cookie → pass
cycle uses this.
* bs_browser_context_pass: a "real-human" shape (AL set, fresh
IP, default Chromium UA). Used for the pass-tier assertion.
Tests (all chromium-headless, all marked @browser):
test_browser_pass_tier.py
test_pass_tier_no_challenge — browser-shaped visitor goes
through /, no interstitial, no challenge header, no cookie.
test_browser_form_tier.py
test_silent_tier_js_pow_round_trip — cookieless suspicious
request lands the silent-tier interstitial. Real Chromium runs
the PoW worker, the auto-submit fires, reload brings real page.
Waits on title change (not cookie appearance — the JS sets the
cookie before the 250ms setTimeout(location.reload)).
test_verified_cookie_clears_subsequent_challenge — after the
silent round-trip, a fresh / navigates without re-challenge.
test_browser_captcha_tier.py (@serial, @live_network)
test_captcha_tier_end_to_end — /captcha-demo renders interstitial
+ Turnstile widget DOM; test injects an always-pass token into
cf-turnstile-response and submits the form (rather than waiting
for Turnstile's widget to auto-solve, which isn't reliable in
headless). return_to is rewritten to /, so post-solve the user
lands on the real site. Asserts _bs_verified cookie in jar.
test_captcha_cookie_clears_subsequent_challenge — same, then
replay / with AL set: no challenge.
test_browser_cookie_attributes.py
test_verified_cookie_attributes_on_silent_solve — after a silent
PoW solve, _bs_verified in the jar has secure=True and
sameSite in (Lax, Strict). HttpOnly is deliberately false (the
challenge JS has to read the cookie).
test_pending_cookie_path_scoped — _bs_captcha_pending has
path=/botshield/captcha-verify, secure=True, httpOnly=True.
Chromium-level assertion: document.cookie at root path does NOT
include the pending cookie (browser enforces the Path scope).
Serial markers added:
test_rate_limit (@serial now) — was hitting RemoteProtocolError
under xdist because the 40-way parallel burst would contend with
other tests' verify calls through the global inflight
semaphore. It's inherently a "saturate one IP's rate slot"
test; quarantining from other verify traffic fixes the flake.
test_browser_captcha_tier (@serial now) — Turnstile's always-pass
sitekey is rate-limited upstream; concurrent siteverify calls
from multiple captcha tests would occasionally flake. M11.7
will wire pytest-rerunfailures for @live_network so this can
relax back to parallel.
Provision (tests/setup/provision.sh):
apt-installs Chromium's shared-lib deps (libnss3, libatk-bridge2.0-0,
libxkbcommon0, libcups2t64, libxcomposite1, libxdamage1, libxfixes3,
libxrandr2, libgbm1, libpango-1.0-0, libcairo2, libasound2t64,
fonts-liberation). Falls back to the pre-noble package names if the
"t64" variants aren't available, for Debian 12 / Ubuntu 22.04.
Runs `playwright install chromium` to populate ~/.cache/ms-playwright.
Idempotent — 0.4s on a warm cache.
README: marker docs updated (@browser explained, @Acceptance no
longer says "becomes browser-driven in M11.6"). Provisioning paragraph
notes the Chromium binary + system-lib install.
Full suite: 39 passed parallel in 19s (+7 tests over M11.5's 32), 40
passed with --slow in ~60s. Browser tests add ~1.5s each, dominated
by Chromium cold-start.
Bumped pyproject.toml so `pip install -e '.[browser]'` installs the
browser layer separately if someone wants the framework without
pulling Chromium.
The Path-scope assertion on _bs_captcha_pending was vacuous. Earlier
version asserted the cookie wasn't in document.cookie at the root
path — but the cookie is HttpOnly, so it's hidden from JS regardless
of Path. The assertion would still pass if Path widened to /.
Rewritten around Playwright's ctx.cookies(urls=...), which returns
the cookies Chromium would attach to a request for a given URL —
the same Path filter the browser applies before sending. Two asserts:
cookies_for_root = ctx.cookies(urls="https://localhost/")
cookies_for_verify = ctx.cookies(urls=".../botshield/captcha-verify/...")
Assert _bs_captcha_pending is absent from the first and present in
the second. Regression gate: a Path widening shows up by the cookie
appearing in cookies_for_root.
Verified the new assertion bites by fault-injecting `Path=/` into
the module source, rebuilding, and running the test:
would send to /: {'_bs_captcha_pending'}
would send to /botshield/captcha-verify/...: {'_bs_captcha_pending'}
pending-in-root? True (regression caught)
Source reverted + module reinstalled; tests pass again.
pyproject.toml markers: "acceptance: becomes browser-driven in M11.6"
and "browser: reserved for M11.6" were promissory lines from M11.4;
now stale. Rewritten to describe present state.
… results Performance section rewritten to merge the wrk-driven single- connection / saturation numbers with the new oha-driven fixed-rate sweep into a single table. The older reference-points subsection (L3 cache miss / loopback RTT / framework bootstrap) is dropped — the fixed-rate row tells the production-load story directly, without needing analogies. Updates the operator-facing reading too: the worst-looking RPS percentage is the saturation-bench tiny-static-file ceiling, not production representative; the fixed-rate row at 10k rps is the better proxy and shows median latency barely moves.
Five sections were stale enough to mislead a new contributor:
- Layout block: claimed pytests/ contains only test_soak.py
(actual: 55 test files plus assets/axe.min.js); missing
fuzz_robots.c + corpus-robots/ + seeds-robots/; missing
tests/site/ (the new committed dev docroot); missing
tests/bench/ entirely.
- Fuzzing section: only documented fuzz_cookie. Now covers
both targets, the right run.sh invocation
(`--target {cookie|robots} <seconds>`), and the
workflow_dispatch-only fuzz-nightly CI job.
- Setup section: provision.sh now also seeds
/etc/botshield/test-robots/, /etc/botshield/load.state.test,
/var/lib/botshield/bots/ from apache/bots/*.txt, and refuses
to build if the repo path is group/world-writable. Added
sudoers.d.example as an operator-optional convenience.
- Provider secrets table: env-var column missing the _TOKEN
and _SOLUTION variants the CI workflow passes as repo
secrets.
- Added a Benchmarks section: tests/bench/ is a real surface
(run-bench.sh + run-rate-bench.sh) and was previously
invisible from the test README.
The committed vhost previously hardcoded
/home/su-nkisseberth/mod_botshield/... in 8 places — anyone else
cloning the repo needed manual sed before `provision.sh` would
work. Replaced with ${BS_REPO} throughout, with a fallback default
in an <IfDefine !BS_REPO> block so the file still parses standalone
on the original developer's box.
provision.sh now writes /etc/apache2/conf-available/botshield-repo.conf
containing `Define BS_REPO $REPO` (where $REPO is the actual
clone path the script resolved) and a2enconf's it before installing
the vhost. Apache loads conf-enabled/ before sites-enabled/, so the
Define is in scope when the vhost reads ${BS_REPO}.
Net effect: any developer can `git clone <repo>; sudo
tests/setup/provision.sh` and the dev vhost works against their
checkout path with no manual edits.
Three stale claims fixed:
- Cookie envelope: HMAC-13-field → AES-256-GCM-15-field. The
HMAC era ended with E8 (cookie confidentiality migration);
forgiveness-cap state added two more fields in E15.
- Soak/fuzz cadence: was 'kicked off nightly via GitHub Actions';
those jobs are now workflow_dispatch-only.
- Test count: was '60+ pytest tests'; actual is ~250 cases
across 54 test files.
Front-matter rewritten as a what's-shipped bullet list instead of
a milestone-history paragraph. Public-facing readers don't need
'M10.4' / 'M11.8' framing; CHANGELOG.md has the chronological
detail.
Development section pruned: removed the per-URL Crestline-themed
demo tables (the testsite content moved to ~/mod_botshield-testsite/
on 2026-04-29 and is no longer the project's docroot). Replaced
with a clean pointer to provision.sh + tests/README.md.
Cascade: docs/guide/index.html rebuilt from the slimmed-down
README sections (-150 lines).
The vhost previously defaulted BS_REPO to one developer's checkout path so the file would parse standalone. That meant the path was both committed (visible to anyone reading the repo) and a silent- wrong-path footgun if provision.sh hadn't run. Replaced with an Error directive that aborts startup with a clear message pointing at provision.sh. No personal-path data committed; broken setup fails loudly instead of silently serving from the wrong DocumentRoot.
Cut from 696 to 150 lines. The big in-line sections (How it works, Directives, Per-URL scoping, Staging, Understanding scoring, Multi- vhost, Customizing the challenge page, Security note, Observability) were mostly duplicating content the operator handbook in docs-src/ already owns — readers had two sources of truth and a long front-page scroll before they could find what they were looking for. Replaced with a table linking each topic to its docs-src/<file>.md home. Kept what belongs on a GitHub landing page: tagline, what's- shipped bullets, quick-start build + minimal vhost config, the module-endpoints inventory (cheap to know without leaving), and a pointer to tests/README.md for the dev workflow. Cascade: docs/guide/index.html drops by ~820 lines since it templates from the now-slimmed README content.
Six badges at the top — CI + License + Documentation are the ones that carry verifiable signal; Apache 2.4 + C + status:beta give a passing reader a quick mental hook without crossing into vanity. Skipped 'PRs welcome', stars/forks duplicates, and Made-With-Love shapes.
Centered the project's header lockup (logo + wordmark) at the top, followed by the tagline in italics, with the badges centered on the third row. Banner SVG already lives at docs/assets/logo-botshield-header.svg, so the README references the same file the docs site uses (no asset duplication). Note: kept '# mod_botshield' as an H1 below the centered block because GitHub's auto-generated table of contents and anchor linking expect a real heading; otherwise the rendered page has no H1 for the repo name. The wordmark in the SVG handles the visual title; the H1 handles the structural one.
'Disciplined Judgment. Proportionate Response.' is the real tagline; the descriptive sentence I had at the top was a working description, not a tagline. Moved the descriptive sentence into the H1 body where it belongs as the 'what is this' opener.
'Adaptive bot mitigation for the Apache HTTP Server' as the elevator-pitch line, followed by the pass/challenge/slow/block ladder framing.
Was 'C — 99' with a logo=c stylized C icon, producing three C glyphs in a row (icon + label + value). Collapsed the label into the value so it reads as [logo-icon] [C99].
Was solid blue (badge/C99-blue with no slash). Re-introduced the 'language' label so it renders as [logo-icon] gray-language | blue-C99, matching the two-tone shape of the other badges.
When I trimmed testsite/ down to a 4-file tests/site/ fixture (commit bf489f8), I missed three files the pytest suite hits as real-content paths: - about.html — test_app_feedback.py (FEEDBACK_PATH_1, hit ~7 times across multiple feedback tests). The test asserts 200, so the file has to exist. - login.html — test_app_feedback.py (FEEDBACK_PATH_2, used in the credit-vs-penalty composition test). - embedded-test.html — test_browser_embedded.py (EMBEDDED_PATH, used by every browser-tier embedded-mode test). Each is a minimal valid HTML page with no __bsChallenge marker, so the safeguard tests still distinguish challenged-vs-passed correctly. Local 'tests/run --parallel --mark not browser' goes from 240 passed / 1 failed to 240 passed / 0 failed after this. Likely accounts for the bulk of the CI mass-failure on the first push to GitHub: the fast-pytest job hit all the test_app_feedback tests against a 404 docroot.
The original logo-botshield-header.svg uses near-white wordmark colors (#bdc7d4 + #f5f9fc) for a dark background — fits the docs site's dark theme but renders nearly invisibly on GitHub's white README. Added logo-botshield-header-light.svg with dark wordmark colors (#566676 + #1d2732) for light backgrounds. README uses a <picture> element so GitHub serves the right one based on the reader's prefers-color-scheme. Docs site stays unchanged (it references the dark-theme original via build_site.py's HERO_LOGO constant). Only the two wordmark fills change between variants; the shield emblem and cartridge are identical.
The '# mod_botshield' heading + the horizontal rule GitHub draws under it duplicated what the centered logo banner already shows. With the SVG wordmark + tagline + badges in the header block, the H1 was just visual noise — removing it lets the body open straight into the descriptor sentence. Re-pointed the status:beta badge anchor (was #mod_botshield, no longer exists) to #whats-shipped, which is now the first body heading.
logo=c rendered a stylized C glyph on the left of the badge, next to a 'language | C99' text where 'C99' already carries the same information. Two C glyphs in a row reads as duplication. Plain two-tone badge without the icon.
Status:beta is already declared (badge + body 'Status: beta' sentence); enumerating sanitizer / MPM / soak / fuzz / test count details on the front page over-promised given the real state. The same content lives in DESIGN.md and CHANGELOG.md where it can be more precise about what's actually been exercised vs what's just been wired.
Apache runs as www-data; the dev vhost serves from ${BS_REPO}/
tests/site/. For Apache to read that, every ancestor needs +x for
others.
On a typical developer box /home/<user>/ is 0755 (or 0750 with
www-data in the group), so it works out of the box. On GitHub
Actions runners /home/runner/ defaults to 0700 — www-data gets
EACCES on every request and Apache 403s before the file handler
can serve. That manifests as 150-of-211 pytest failures with the
'status=pass must not 403; got 403' pattern: BotShield correctly
DECLINEs, Apache tries to serve, fails on permission, returns
403.
provision.sh now walks up from $REPO and adds o+x to each
ancestor (chmod is idempotent and only adds the traversal bit,
not read or write).
Two CI-only failures the local box masked:
(1) test_browser_embedded.py asserts the page title contains
'embedded-mode test page'. My minimal stub from the earlier
commit had title 'Embedded test — mod_botshield' instead.
Restored the originals from the moved-out testsite content
for about.html, login.html, and embedded-test.html so titles
+ content match what the suite was written against.
(2) test_env_trigger_from_rewrite_producer uses RewriteEngine On
+ RewriteRule, which requires mod_rewrite. provision.sh was
a2enmod'ing 'botshield status remoteip ssl headers' but not
rewrite. Added it. The reload-fail in this test cascaded into
the next two captcha-rejected[turnstile|recaptcha_v2] tests
because Apache stopped after the bad reload — those should
recover once rewrite is enabled.
Locally everything was green because (a) my home dir is 0751 so
www-data can traverse, and (b) mod_rewrite was already enabled
on this box from earlier ad-hoc work.
Two more CI gaps the local box masked because they pre-existed:
(1) test_captcha_rejected_via_bad_secret swaps the secret-file
path to a 'bad' variant the test expects to be installed at
/etc/botshield/{turnstile-fail-secret,recaptcha-v2-badsecret}.
These are well-formed values the provider rejects on
siteverify (Turnstile's published always-fail 2x000...AA;
reCAPTCHA v2 doesn't publish a fail-key so a malformed string
that fails clean). Local box had them from past manual
setup, CI didn't. provision.sh now installs both.
(2) test_form_widget_injects_per_provider_markup loads
/form-widget-test.html — has a [data-bs-form-captcha] slot
the form-widget JS targets. I missed this file in the
earlier fixture restore. Copied from
~/mod_botshield-testsite/.
Three GitHub-conventional repo-meta files:
- SECURITY.md: points reporters at GitHub's private security
advisory flow (already enabled at repo level), describes
scope and the bug classes most worth a careful look — cookie
auth bypass, replay against the nonce/strike tables,
score-laundering paths, fuzzer-discoverable crashes,
HTTP-level confusion, unsafe directive defaults.
- CONTRIBUTING.md: short pointer doc for outside contributors —
quick start (provision.sh + tests/run), how to file bugs and
questions, PR expectations (CI must pass, rebuild docs/ if
docs-src/ changes), and a brief code-layout map. Defers
framework details to tests/README.md.
- .github/dependabot.yml: weekly auto-PRs for github-actions
(.github/workflows/), pip in tests/ (pytest + Playwright +
Hypothesis), pip in tools/ (markdown-it-py). Limited to a
handful of open PRs each so updates land in batches.
GitHub auto-discovers all three: SECURITY.md and CONTRIBUTING.md
appear as tabs in the file-list header next to README and MIT
license; dependabot.yml gets its own scheduling logic.
Bumps [actions/checkout](https://github.com/actions/checkout) from 4 to 6. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v4...v6) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>
dependabot
Bot
added
dependencies
Author
|
OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let me know by commenting If you change your mind, just re-open this PR and I'll resolve any conflicts on it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bumps actions/checkout from 4 to 6.
Release notes
Sourced from actions/checkout's releases.
... (truncated)
Changelog
Sourced from actions/checkout's changelog.
... (truncated)
Commits
de0fac2Fix tag handling: preserve annotations and explicit fetch-tags (#2356)064fe7fAdd orchestration_id to git user-agent when ACTIONS_ORCHESTRATION_ID is set (...8e8c483Clarify v6 README (#2328)033fa0dAdd worktree support for persist-credentials includeIf (#2327)c2d88d3Update all references from v5 and v4 to v6 (#2314)1af3b93update readme/changelog for v6 (#2311)71cf226v6-beta (#2298)069c695Persist creds to a separate file (#2286)ff7abcdUpdate README to include Node.js 24 support details and requirements (#2248)08c6903Prepare v5.0.0 release (#2238)Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting
@dependabot rebase.Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
@dependabot rebasewill rebase this PR@dependabot recreatewill recreate this PR, overwriting any edits that have been made to it@dependabot show <dependency name> ignore conditionswill show all of the ignore conditions of the specified dependency@dependabot ignore this major versionwill close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)@dependabot ignore this minor versionwill close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)@dependabot ignore this dependencywill close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)