Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

## [0.18.1] - 2026-05-11

### Added

- **Generic webhook adapter transforms.** Per-field transforms applied after JSONPath extraction in the generic webhook adapter (ADR 020). Three variants cover the common shapes:
- `unit_conversion` — multiply an extracted numeric value (`bps`, `pps`, `confidence`) by a constant. Useful for `Mbps→bps`, `kpps→pps`, percentage → ratio.
- `regex_extract` — pull a capture group out of an extracted `vector` string. Lets operators map free-form alert descriptions ("Detected: udp_flood at 250Gbps") onto prefixd vector names without writing a shim.
- `computed` — replace a numeric field's value with `scale * Π(extract(path_i))`. Lets operators derive fields not directly present in the payload (e.g. `bps = packets × avg_size × 8`).
- All transforms are validated at config load (regex compiles, multipliers are finite, transform shape matches field type) so misconfigurations fail fast on `POST /v1/config/reload` rather than silently producing zero events.
- 17 new tests (14 unit + 3 end-to-end integration) covering each variant, compile-time validation, type-mismatch rejection, missing-value passthrough, and interaction with the existing `confidence_scale` clamp pipeline.

## [0.18.0] - 2026-05-11

### Added
Expand Down Expand Up @@ -920,7 +931,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Safelist prevents mitigation of protected infrastructure
- Guardrails block overly broad mitigations

[Unreleased]: https://github.com/lance0/prefixd/compare/v0.18.0...HEAD
[Unreleased]: https://github.com/lance0/prefixd/compare/v0.18.1...HEAD
[0.18.1]: https://github.com/lance0/prefixd/compare/v0.18.0...v0.18.1
[0.18.0]: https://github.com/lance0/prefixd/compare/v0.17.1...v0.18.0
[0.17.1]: https://github.com/lance0/prefixd/compare/v0.17.0...v0.17.1
[0.17.0]: https://github.com/lance0/prefixd/compare/v0.16.0...v0.17.0
Expand Down
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ hmac = "0.13"
hex = "0.4"
subtle = "2"
serde_json_path = "0.7"
regex = "1"
ipnet = { version = "2", features = ["serde"] }
clap = { version = "4", features = ["derive", "env"] }
async-trait = "0.1"
Expand Down
2 changes: 1 addition & 1 deletion ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -280,7 +280,7 @@ Example: FastNetMon says UDP flood at 0.6 confidence + router CPU spiking + host
- [x] FastNetMon webhook adapter (`POST /v1/signals/fastnetmon`) — classifies vectors from traffic breakdown, configurable confidence mapping
- [x] Generic webhook adapter (`POST /v1/signals/webhook/{name}`) — operator-configured JSONPath mapping, HMAC/bearer/none auth, array batching via root_path (ADR 020)
- [ ] Router telemetry adapter (JTI, gNMI)
- [ ] Generic adapter transform functions (unit conversion, regex extract, computed fields)
- [x] Generic adapter transform functions (unit conversion, regex extract, computed fields)

### Correlation Engine

Expand Down
25 changes: 25 additions & 0 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -426,6 +426,20 @@ correlation:
default_vector: unknown
confidence_scale: 100 # divide extracted value by this
source_id_prefix: "radware-"
transforms: # optional per-field transforms
bps:
type: unit_conversion # Mbps -> bps
multiplier: 1000000
vector:
type: regex_extract # pull "udp_flood" out of free-form text
pattern: "(\\w+)_flood"
group: 0 # 0=whole match, 1+=capture group N
pps:
type: computed # derive pps from packets/second fields
paths:
- "$.metrics.packets"
- "$.metrics.duration_inv"
scale: 1.0
```

**Adapter schema:**
Expand All @@ -449,6 +463,17 @@ correlation:
| `default_vector` | no | string | Fallback when vector missing or not in map |
| `confidence_scale` | no | float | Divisor (e.g. `100` for 0–100 scales) |
| `source_id_prefix` | no | string | Prefix prepended to extracted `source_id` |
| `transforms` | no | map | Per-field transforms applied post-extraction (see below) |

**Field transforms.** Each entry in `transforms` maps a field name to a single transform. Only `bps`, `pps`, `confidence` (numeric) and `vector` (string) accept transforms; the daemon rejects the adapter at load time if a transform is attached to a different field or with the wrong shape.

| `transforms.<field>.type` | Applies to | Other keys | Behavior |
|---|---|---|---|
| `unit_conversion` | `bps`, `pps`, `confidence` | `multiplier` (float, must be finite) | Multiplies the extracted value. Use for `Mbps→bps` (`1_000_000`), `kpps→pps` (`1000`), `%→ratio` (`0.01`). Missing values stay `None`. |
| `regex_extract` | `vector` | `pattern` (string regex), `group` (int, default `0`) | Replaces the extracted string with `captures.get(group).as_str()`. No match ⇒ field is treated as missing and falls through to `vector_map` / `default_vector`. The regex is compiled once at config load. |
| `computed` | `bps`, `pps`, `confidence` | `paths` (array of JSONPath), `scale` (float, default `1.0`) | Bypasses the field's primary `fields.<name>` JSONPath. Evaluates `scale * Π(extract(path_i))`. Any path that resolves to null or non-number ⇒ field is `None`. Useful for fields not directly present in the payload (e.g. derive `bps = packets × avg_size × 8`). |

For `confidence`, the order of operations is: JSONPath extract → transform → `confidence_scale` divisor → clamp to `[0, 1]`.

**Authentication:**

Expand Down
1 change: 1 addition & 0 deletions src/correlation/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1082,6 +1082,7 @@ sources:
default_vector: None,
confidence_scale: None,
source_id_prefix: None,
transforms: HashMap::new(),
}
}

Expand Down
4 changes: 2 additions & 2 deletions src/correlation/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,6 @@ pub mod webhook;
pub use config::*;
pub use engine::*;
pub use webhook::{
CompiledAdapter, MapError, WebhookAdapter, WebhookAuth, WebhookFieldMap, is_valid_name,
map_payload, verify_hmac_sha256,
CompiledAdapter, CompiledTransform, MapError, WebhookAdapter, WebhookAuth, WebhookFieldMap,
WebhookTransform, is_valid_name, map_payload, verify_hmac_sha256,
};
Loading
Loading