Skip to content

Follow-up guardrails for custom pattern recognizer #317

Description

@martsokha

Adds deeper defense layers on top of what already ships.

What's already shipped

  • Wire-side source-length cap (nvisy-schema): rejects any regex source over 512 bytes at deserialize.
  • Per-request rule count cap (nvisy-engine): rejects custom.len() + custom_dictionaries.len() > 32 at analyzer-compile.
  • Aggregate dictionary limits (elide's PatternRecognizerBuilder::with_term_count_limit / with_term_bytes_limit): 100 000 terms / 8 MiB across all dictionaries — builtin and custom.

Remaining follow-ups

(A) Split regex automaton budgets

Elide today exposes a single `with_size_limit` / `with_dfa_size_limit` pair applied uniformly to (i) every individual regex NFA and (ii) the shared `RegexSet` union that the recognizer builds across every registered pattern.

Those two wants are incompatible:

  • Per-regex NFA: a caller-supplied rule should compile to well under 1 MiB, else it's pathological.
  • Shared `RegexSet` union: the shipped builtins union past 4 MiB routinely, and setting the budget below that breaks the whole builtin path.

Any single budget either lets pathological custom rules through or breaks builtins. Runtime needs elide to expose separate knobs — `with_variant_size_limit(bytes)` for the per-variant NFA and the current `with_size_limit` (or a renamed `with_set_size_limit`) for the union — so nvisy can enforce a tight per-regex cap without breaking the union.

File: elide-side issue tracking this API split.

(B) AST-level catastrophic-shape refuse-list

Parse each caller-supplied regex with the `regex-syntax` crate (no compile) and reject known-catastrophic patterns:

  • Nested repetition on the same alternation: `(a+)+`, `(a*)*`, `(a|a)+`
  • Alternation with common-prefix repeat: `(a|ab)*`
  • Deeply nested quantifiers past a per-request depth budget

Best implemented inside elide as `PatternRecognizerBuilder::with_refuse_list(...)`; interim implementation can live in the nvisy engine as a preflight.

(C) Per-match timeout on the pattern-recognition path

Wrap regex matching in a hard deadline. A regex that hangs synchronously inside `elide::PatternRecognizer::analyze` won't yield to `tokio::time::timeout`; only spawn_blocking + a hard deadline stops it.

Best implemented inside elide as `PatternRecognizerBuilder::with_match_timeout(Duration)`.

(D) Per-document total budget

Cap total pattern-recognition wall-clock per document. Once exceeded, abort the analyze with a Validation error carrying the `correlation_id`. Catches multi-pattern amplification that no single-regex bound catches.

Configuration

All should expose a per-deployment override so paranoid deployments can lock down harder and permissive internal SDKs can relax.

Conservative starting defaults:

  • (A) per-variant NFA limit: ~1 MiB; RegexSet union limit: current elide default
  • (B) refuse-list: on
  • (C) per-match timeout: 500 ms
  • (D) document budget: 10 s

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions