Skip to content

Generic webhook adapter transforms#123

Merged
lance0 merged 1 commit into
mainfrom
feature/webhook-transforms
May 12, 2026
Merged

Generic webhook adapter transforms#123
lance0 merged 1 commit into
mainfrom
feature/webhook-transforms

Conversation

@lance0
Copy link
Copy Markdown
Owner

@lance0 lance0 commented May 12, 2026

Summary

Adds per-field transforms to the generic webhook adapter (ADR 020). Operators can now reshape extracted values without writing a shim detector or pre-processing payloads outside prefixd.

Closes ROADMAP item: "Generic adapter transform functions (unit conversion, regex extract, computed fields)" under Signal Adapters.

Three transform variants

Variant Applies to Use case
`unit_conversion` `bps`, `pps`, `confidence` Detector reports Mbps but prefixd wants bps. `multiplier: 1_000_000`.
`regex_extract` `vector` Detector embeds the attack type inside a free-form `description` string. Pull a capture group out.
`computed` `bps`, `pps`, `confidence` Detector doesn't report `bps` but does report `packets` and `avg_size`. Derive it as `packets × avg_size × 8`.

Example YAML:

```yaml
transforms:
bps:
type: unit_conversion
multiplier: 1000000
vector:
type: regex_extract
pattern: "(\\w+)_flood"
group: 0
pps:
type: computed
paths:
- "$.metrics.packets"
- "$.metrics.duration_inv"
scale: 1.0
```

Validation

All validation happens at config load (`POST /v1/config/reload`), not at request time:

  • Regex compiles successfully.
  • Multipliers and `computed.scale` are finite (NaN/inf rejected).
  • Transform variant matches field type (`unit_conversion` / `computed` only on numeric fields; `regex_extract` only on `vector`).
  • Transform key is one of the four whitelisted fields (`bps`, `pps`, `confidence`, `vector`).

Misconfigurations fail fast at reload time so an operator sees the error in the admin UI immediately rather than silently producing zero events at request time.

Pipeline order

For `confidence`: JSONPath extract → transform → `confidence_scale` divisor → clamp to `[0, 1]`. The clamp is the last step, so a transform that pushes the value out of range is still safely contained.

For `computed`, the field's primary JSONPath is bypassed (it doesn't need to exist).

Tests

  • 14 new unit tests in `src/correlation/webhook.rs`:
    unit_conversion on each numeric field, regex_extract with default + named groups, computed product, missing-value passthrough, compile-time validation (invalid regex, NaN multiplier, unsupported field, type mismatch), YAML round-trip.
  • 3 new end-to-end integration tests in `tests/integration.rs` exercising the live HTTP endpoint and verifying the ingested event surfaced via `GET /v1/events` has the transformed value.

Full suite: 250 unit + 133 integration + 16 postgres + 87 frontend all pass. `cargo fmt --check` + `cargo clippy --all-targets --features test-utils -- -D warnings` clean. Frontend build green.

Migration

None. `transforms` defaults to an empty map; adapters without it behave exactly as before.

Dependencies

Adds `regex = "1"` as a top-level dependency.

Add per-field transforms applied after JSONPath extraction in the
generic webhook adapter (ADR 020). Three variants cover the common
shapes detectors emit in the wild:

- unit_conversion: multiply an extracted numeric value (bps, pps,
  confidence) by a constant. Useful for Mbps->bps, kpps->pps, % -> ratio.
- regex_extract: pull a capture group out of an extracted vector
  string. Lets operators map free-form alert descriptions onto
  prefixd vector names without writing a shim.
- computed: replace a numeric field's value with scale * product of
  one or more JSONPath extractions. Lets operators derive fields not
  present directly in the payload (bps = packets * avg_size * 8).

All transforms are validated at config load: regex compiles,
multipliers are finite, transform variant matches field type
(unit_conversion/computed -> numeric, regex_extract -> string),
field name is one of the four whitelisted ones. Misconfigurations
fail fast on POST /v1/config/reload rather than silently producing
zero events.

Confidence pipeline order: JSONPath extract -> transform ->
confidence_scale divisor -> clamp to [0, 1].

Adds regex 1.x as a top-level dependency.

Tests: 17 new (14 unit + 3 end-to-end integration) covering each
variant, compile-time validation, type-mismatch rejection, missing-
value passthrough, and clamping interaction with confidence_scale.
@lance0 lance0 merged commit 814db7a into main May 12, 2026
5 checks passed
@lance0 lance0 deleted the feature/webhook-transforms branch May 12, 2026 14:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant