feat: syslog/SIEM export (CEF + JSON over udp/tcp/tcp-tls)#2217
Merged
Conversation
Daemon-side SIEM/syslog exporter — Issue #2199. Forwards ClawMetry events to Splunk / QRadar / ArcSight / Elastic SIEM or any RFC 5424 collector. Off by default; activates when `CLAWMETRY_SIEM_HOST` is set. ## What - `clawmetry/siem.py` (~330 LOC): pure formatter functions (`format_cef`, `format_json`, `format_syslog_line`) + a bounded-queue `SIEMExporter` with a single background sender thread + env-var driven singleton getter (`get_default_exporter` / `forward_event`). - `clawmetry/local_store.py` (+9 LOC): SIEM forward hook in `ingest()`, called *after* the redaction pass (#2204) so secrets never reach syslog either, and *after* the hash chain stamp (#2210) so the SIEM line carries the same audit-grade payload as DuckDB. Composes cleanly with both. - `tests/test_siem.py` (~250 LOC, 21 tests): CEF formatting for tool call/result/LLM-usage/security-threat/unknown-fallthrough; CEF escaping (\, =, |, \n); JSON formatting + non-serialisable fallback; RFC 5424 priority + ISO-8601 timestamp coercion; `SIEMExporter` drains queue / drops without blocking / counts writer errors; singleton wiring + env-var dispatch / unknown protocol disables. ## Configuration (env vars) | var | default | purpose | |-------------------------------|-------------|---------------------------------| | `CLAWMETRY_SIEM_HOST` | (unset) | enables the exporter when set | | `CLAWMETRY_SIEM_PORT` | `514` | syslog port | | `CLAWMETRY_SIEM_PROTOCOL` | `udp` | `udp` / `tcp` / `tcp-tls` | | `CLAWMETRY_SIEM_FORMAT` | `cef` | `cef` / `json` | | `CLAWMETRY_SIEM_FACILITY` | `16` | syslog facility (local0) | | `CLAWMETRY_SIEM_APPNAME` | `clawmetry` | RFC 5424 APP-NAME | ## Event taxonomy (CEF signature IDs) Stable map: tool.call/tool_call → 1001, tool.result → 1002, mcp_call → 1003, channel.in → 2001, channel.out → 2002, model.completed → 3001, session.started → 4001, budget_exceeded → 5001, security_threat → 6001, approval_required → 7001, cron_run → 8001, daemon.error → 9002, unknown → 9999. New event types fall through to 9999 rather than crashing, so adding a new event type does not require a SIEM-side change to be observed. ## Local verify (FLYWHEEL §3 acceptance) UDP — `nc -ul 15514` listener received 3 CEF lines from `forward_event(...)` with `sent=3 dropped=0 errors=0`. TCP — `nc -l 15515` listener received 2 JSON-format events carrying the hash-chain `chain_prev_hash`/`chain_hash` fields, one line each. Pytest — 21 new tests pass (`tests/test_siem.py`); 42 tests pass across the security cluster (redaction + integrity + siem). ## Done bar - [x] Off by default (no env var → no behaviour change, zero overhead). - [x] `ingest()` never blocks on socket IO (bounded queue, async worker). - [x] Collector unreachable does not crash ingest (counted, dropped). - [x] Composes with #2204 redaction and #2210 hash chain. - [x] CEF lines parse cleanly in a real syslog collector (verified with netcat). Daemon-side only; no cloud route or pin change. Closes #2199. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
vivekchand
added
the
persona-skip
5 tasks
vivekchand
added a commit
that referenced
this pull request
May 28, 2026
+ #2222) CHANGELOG entries for two changes already on main: 1. #2217 — syslog/SIEM export (CEF + JSON over udp/tcp/tcp-tls). New Enterprise-grade exporter; off by default. Closes #2199. 2. #2222 — verify-integrity CLI daemon-proxy crash fix. Caught by FLYWHEEL §7 live verification immediately after 0.12.342 shipped: the new CLI subscript-crashed against any standard install because verify_integrity wasn't in the daemon-proxy method allowlist. Two- layer fix (allowlist + CLI graceful-None) plus 3 regression tests. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #2199.
Summary
Daemon-side SIEM/syslog exporter that forwards every event to a syslog
collector (Splunk / QRadar / ArcSight / Elastic SIEM / any RFC 5424
receiver) as CEF or JSON. Off by default; activates when
CLAWMETRY_SIEM_HOSTis set. Composes with the redaction (#2204) andhash-chain (#2210) cluster — by the time an event reaches the SIEM hook
it has been scrubbed of secrets and stamped with
chain_prev_hash/chain_hash.Files
clawmetry/siem.py(new)clawmetry/local_store.pyforward_eventhook after redactiontests/test_siem.py(new)Configuration (env vars)
CLAWMETRY_SIEM_HOSTCLAWMETRY_SIEM_PORT514CLAWMETRY_SIEM_PROTOCOLudpudp/tcp/tcp-tlsCLAWMETRY_SIEM_FORMATcefcef/jsonCLAWMETRY_SIEM_FACILITY16CLAWMETRY_SIEM_APPNAMEclawmetryEvent taxonomy (CEF signature IDs)
Stable map: tool.call → 1001, tool.result → 1002, mcp_call → 1003,
channel.in → 2001, channel.out → 2002, model.completed → 3001,
session.started → 4001, budget_exceeded → 5001, security_threat →
6001, approval_required → 7001, cron_run → 8001, daemon.error →
9002, unknown → 9999. New event types fall through to 9999 rather than
crashing, so adding a new event type does not require a SIEM-side
change.
Local verify (FLYWHEEL §3)
Done bar
ingest()never blocks on socket IO (bounded queue + worker thread).cs5=chain_hash).[RELEASE]PR + PyPI propagation (follow-up).Daemon-side only; no cloud route or pin change.
🤖 Generated with Claude Code