Generic webhook adapter transforms#123
Merged
Merged
Conversation
Add per-field transforms applied after JSONPath extraction in the generic webhook adapter (ADR 020). Three variants cover the common shapes detectors emit in the wild: - unit_conversion: multiply an extracted numeric value (bps, pps, confidence) by a constant. Useful for Mbps->bps, kpps->pps, % -> ratio. - regex_extract: pull a capture group out of an extracted vector string. Lets operators map free-form alert descriptions onto prefixd vector names without writing a shim. - computed: replace a numeric field's value with scale * product of one or more JSONPath extractions. Lets operators derive fields not present directly in the payload (bps = packets * avg_size * 8). All transforms are validated at config load: regex compiles, multipliers are finite, transform variant matches field type (unit_conversion/computed -> numeric, regex_extract -> string), field name is one of the four whitelisted ones. Misconfigurations fail fast on POST /v1/config/reload rather than silently producing zero events. Confidence pipeline order: JSONPath extract -> transform -> confidence_scale divisor -> clamp to [0, 1]. Adds regex 1.x as a top-level dependency. Tests: 17 new (14 unit + 3 end-to-end integration) covering each variant, compile-time validation, type-mismatch rejection, missing- value passthrough, and clamping interaction with confidence_scale.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds per-field transforms to the generic webhook adapter (ADR 020). Operators can now reshape extracted values without writing a shim detector or pre-processing payloads outside prefixd.
Closes ROADMAP item: "Generic adapter transform functions (unit conversion, regex extract, computed fields)" under Signal Adapters.
Three transform variants
Example YAML:
```yaml
transforms:
bps:
type: unit_conversion
multiplier: 1000000
vector:
type: regex_extract
pattern: "(\\w+)_flood"
group: 0
pps:
type: computed
paths:
- "$.metrics.packets"
- "$.metrics.duration_inv"
scale: 1.0
```
Validation
All validation happens at config load (`POST /v1/config/reload`), not at request time:
Misconfigurations fail fast at reload time so an operator sees the error in the admin UI immediately rather than silently producing zero events at request time.
Pipeline order
For `confidence`: JSONPath extract → transform → `confidence_scale` divisor → clamp to `[0, 1]`. The clamp is the last step, so a transform that pushes the value out of range is still safely contained.
For `computed`, the field's primary JSONPath is bypassed (it doesn't need to exist).
Tests
unit_conversion on each numeric field, regex_extract with default + named groups, computed product, missing-value passthrough, compile-time validation (invalid regex, NaN multiplier, unsupported field, type mismatch), YAML round-trip.
Full suite: 250 unit + 133 integration + 16 postgres + 87 frontend all pass. `cargo fmt --check` + `cargo clippy --all-targets --features test-utils -- -D warnings` clean. Frontend build green.
Migration
None. `transforms` defaults to an empty map; adapters without it behave exactly as before.
Dependencies
Adds `regex = "1"` as a top-level dependency.