From 8ba647b37d923be72fbb77d91bf10d5098f0aedf Mon Sep 17 00:00:00 2001 From: InauguralPhysicist Date: Wed, 1 Jul 2026 23:34:08 -0500 Subject: [PATCH] ERE parity: escapes + POSIX classes (S6), {n,m} intervals (S7), re_replace + builtin-shaped compat layer (S8) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes the four gaps between the Pike-VM engine and EigenScript's libc-backed regex builtins, so lib/regex_compat.eigs can drop in for regex_match/regex_find/regex_replace in the freestanding (EigenOS) profile where libc regcomp is gone: - parser: backslash escapes (\. \* \\ etc. literal; \w \W \s \S classes; \d stays a literal 'd' — glibc ERE parity, verified against the builtin oracle; trailing backslash errors) - parser: POSIX [[:alpha:]]-style classes (12 names + [:word:]), composable with ranges and negation; unknown names error - parser: {n} {n,} {n,m} intervals with lazy '?' variants, desugared to existing AST forms (no new VM ops). The repeated subtree is shared — safe because the compiler never mutates AST nodes — which reproduces glibc's last-repetition-wins capture semantics. Invalid/unclosed specs are parse errors like glibc; bounds cap at 255. - lib/regex.eigs: re_replace (literal replacement, zero-width matches pass one char through — byte-matches the builtin's "-a-b-c-" shape) - lib/regex_compat.eigs (new): regex_match/regex_find/regex_replace with the builtins' exact names, arg shapes, substring returns, and quirks (group list truncates at first non-participating group, caps at 16; bad pattern → []/input). Known divergences documented in the header: POSIX leftmost-longest vs Pike-VM leftmost-first, no \b. Tests: S6 (40) + S7 (39) + S8 (48, including a 42-check differential suite running shim-vs-builtin over shared inputs with the libc builtins as the oracle). Suite 347/347 green on EigenScript v0.23.0. Docs: README + CLAUDE.md feature/status/layout refresh; stale EIGS_REF note corrected to v0.23.0. Co-Authored-By: Claude Fable 5 --- CLAUDE.md | 41 +++++--- README.md | 49 ++++++--- lib/regex.eigs | 32 ++++++ lib/regex_compat.eigs | 76 ++++++++++++++ lib/regex_parse.eigs | 170 ++++++++++++++++++++++++++++++- tests/test_s6_escapes_posix.eigs | 101 ++++++++++++++++++ tests/test_s7_curly.eigs | 98 ++++++++++++++++++ tests/test_s8_compat.eigs | 83 +++++++++++++++ 8 files changed, 621 insertions(+), 29 deletions(-) create mode 100644 lib/regex_compat.eigs create mode 100644 tests/test_s6_escapes_posix.eigs create mode 100644 tests/test_s7_curly.eigs create mode 100644 tests/test_s8_compat.eigs diff --git a/CLAUDE.md b/CLAUDE.md index f01ac30..0f8ebcd 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -31,9 +31,14 @@ libc-POSIX-backed builtins. EigenRegex prefixes its public API with | backend | libc POSIX ERE in C | Pike-VM in EigenScript | | speed | native, fast | interpreted, ~100–1000× slower | | worst case | can backtrack catastrophically | guaranteed `O(n·m)` | -| features | `{n,m}`, POSIX classes | MVP: `* + ? . [] ^ $ ( )` | +| features | POSIX ERE + GNU `\w \s \b` | ERE parity minus `\b`; lazy quantifiers extra | +| match rule | leftmost-longest (POSIX) | leftmost-first (Pike-VM priority) | | return shape | substring list | positional spans `[s, e, ...]` | +`lib/regex_compat.eigs` re-exposes the builtins' exact names/shapes on +top of the Pike VM (for freestanding/WASM); divergences are documented +in its header — chiefly leftmost-longest vs leftmost-first, and no `\b`. + **Pick the builtin for hot paths; pick `re_*` when you need a linear worst-case guarantee or libc isn't available** (e.g. the WASM playground build). @@ -43,7 +48,7 @@ playground build). EigenScript is **not** vendored. Pin v0.13.0 minimum (strings needed `<`/`<=` comparison and `ord of s`, both of which shipped in v0.13.0 — see GAPS.md for the fix history). CI pins the runtime via -`.devcontainer/Dockerfile`'s `EIGS_REF` (currently **v0.21.0**) and +`.devcontainer/Dockerfile`'s `EIGS_REF` (currently **v0.23.0**) and builds it from source — bump that to move the tested runtime. ## Run / test @@ -64,7 +69,7 @@ $EIGS tests/test_s1_literals.eigs # ... s2 alt, s3 repeat, s4 classes, $EIGS tests/test_s5_anchors_groups.eigs # s5 anchors/groups, test_smoke ``` -220 test checks across S1–S5, all green. (`tests/bench_search.eigs` is +347 test checks across S1–S8, all green. (`tests/bench_search.eigs` is a manual timing bench, not part of the gate.) ## Layout @@ -75,7 +80,8 @@ a manual timing bench, not part of the gate.) | `lib/regex_parse.eigs` | Pattern string → AST | | `lib/regex_compile.eigs` | AST → instruction list | | `lib/regex_vm.eigs` | Pike-VM executor (parallel-thread simulation) | -| `tests/test_s{1..5}_*.eigs` | Per-stage tests (literals → alt → repeat → classes → anchors/groups) | +| `lib/regex_compat.eigs` | Builtin-shaped shim (`regex_match`/`regex_find`/`regex_replace` over the Pike VM) | +| `tests/test_s{1..8}_*.eigs` | Per-stage tests (literals → alt → repeat → classes → anchors/groups → escapes/POSIX → intervals → compat/differential) | | `tests/test_smoke.eigs` | S0 end-to-end load + API smoke | | `tests/run.sh` | Suite runner — runs every test, exits non-zero on any FAIL/crash (the CI gate) | | `tests/bench_search.eigs` | Manual scaling bench for `re_search` (not a pass/fail gate) | @@ -97,18 +103,22 @@ a manual timing bench, not part of the gate.) separated. Adding a feature usually touches all three of parse, compile, vm. -## Supported features (MVP, stable) +## Supported features (stable) -Literals, concat, `|`, `( )`, `* + ?` (greedy + lazy), `.`, -`[abc]`, `[^abc]`, `[a-z]`, `^`, `$`, numbered capture groups. +Literals, escaped metachars (`\.` `\*` …), concat, `|`, `( )`, +`* + ?` and `{n} {n,} {n,m}` (all greedy + lazy), `.`, `[abc]`, +`[^abc]`, `[a-z]`, POSIX `[[:alpha:]]`-style classes, `\w \W \s \S` +(`\d` is a literal `d` — glibc ERE parity, verified against the +oracle), `^`, `$`, numbered capture groups, `re_replace`, and the +builtin-shaped compat layer. ## Out of scope - Backreferences (force backtracking — incompatible with Pike-VM guarantee) -- Lookahead / lookbehind +- Lookahead / lookbehind; `\b` word boundaries (would need text access + in the VM's epsilon-closure — doable, deferred until something needs it) - Named groups -- `{n,m}` quantifiers - Unicode classes (`\p{...}`) - Case-insensitive flags (yet) @@ -127,9 +137,16 @@ Literals, concat, `|`, `( )`, `* + ?` (greedy + lazy), `.`, ## Current state -**S5 complete.** All 220 checks across S1–S5 green. Next is **S6: -fold into corpus for retraining** — feeding the engine's own source -+ tests into the iLambdaAi self-training pipeline as a real workload. +**S8 complete (ERE parity, 2026-07-01).** All 347 checks across S1–S8 +green. S6 added escapes + POSIX classes, S7 added `{n,m}` intervals +(desugared in the parser — no new VM ops; shared-subtree repetition +gives glibc's last-repetition-wins capture semantics), S8 added +`re_replace` + `lib/regex_compat.eigs` and a differential suite that +runs shim-vs-builtin over shared inputs with the libc builtins as the +oracle. Built for EigenScript's freestanding profile (EigenOS): the +compat layer is the planned regex story when libc's regcomp is gone. +Still open: fold into corpus for retraining — feeding the engine's own +source + tests into the iLambdaAi self-training pipeline. **Search is now genuinely O(n·m).** `re_search` used to loop over every start position and re-run the VM from each — O(n²), which silently diff --git a/README.md b/README.md index 9edb40f..4fd3a18 100644 --- a/README.md +++ b/README.md @@ -8,33 +8,41 @@ Dual purpose: ## Status -S5 complete. Working: literals, concat, `|`, `( )`, `* + ?` (greedy + lazy), -`.`, `[abc]`, `[^abc]`, `[a-z]`, `^`, `$`, numbered capture groups. -220 test checks across S1–S5, all green. Next: fold into corpus for retraining (S6). - -## Supported (target MVP) - -- Literal characters +S8 complete (ERE parity). Working: literals, concat, `|`, `( )`, `* + ?` +and `{n} {n,} {n,m}` (all greedy + lazy), `.`, `[abc]`, `[^abc]`, `[a-z]`, +POSIX `[[:alpha:]]`-style classes, escapes (`\.` `\w` `\W` `\s` `\S`), +`^`, `$`, numbered capture groups, `re_replace`, and a builtin-shaped +compat layer (`lib/regex_compat.eigs`) that drops in for the libc-backed +`regex_match` / `regex_find` / `regex_replace` builtins. +347 test checks across S1–S8, all green — including a differential suite +run against the live libc builtins as the oracle. + +## Supported + +- Literal characters and escaped metacharacters (`\.` `\*` `\\` …) - Concatenation - Alternation `|` - Grouping `( ... )` -- Repetition `*`, `+`, `?` (greedy and lazy) -- Char classes: `.`, `[abc]`, `[^abc]`, `[a-z]` +- Repetition `*`, `+`, `?`, `{n}`, `{n,}`, `{n,m}` (greedy and lazy) +- Char classes: `.`, `[abc]`, `[^abc]`, `[a-z]`, POSIX `[[:alpha:]]` etc., + `\w` `\W` `\s` `\S` (note: `\d` is a literal `d`, matching glibc ERE) - Anchors `^`, `$` - Capturing groups (numbered) ## Not supported - Backreferences (would force backtracking; out of scope for Pike-VM) -- Lookahead / lookbehind +- Lookahead / lookbehind; `\b` word boundaries - Named groups -- `{n,m}` quantifiers - Unicode classes (`\p{...}`) - Case-insensitive flags (yet) ## Layout -- `lib/regex.eigs` — public API +- `lib/regex.eigs` — public API (spans-shaped: `re_compile`, `re_match`, + `re_search`, `re_find_all`, `re_replace`) +- `lib/regex_compat.eigs` — builtin-shaped shim (`regex_match` / + `regex_find` / `regex_replace` over the Pike VM; shadows the builtins) - `lib/regex_parse.eigs` — pattern string → AST - `lib/regex_compile.eigs` — AST → instruction list - `lib/regex_vm.eigs` — Pike-VM executor @@ -51,11 +59,14 @@ uses the `re_*` prefix so both can be used in the same script: | backend | libc POSIX ERE in C | Pike-VM in EigenScript | | speed | native, fast | interpreted, ~100–1000× slower | | worst case | can backtrack catastrophically | guaranteed O(n·m) | -| features | `{n,m}`, POSIX classes | MVP: `* + ? . [] ^ $ ( )` | +| features | POSIX ERE + GNU `\w \s \b` | ERE parity minus `\b`; lazy quantifiers extra | +| match rule | leftmost-longest (POSIX) | leftmost-first (Pike-VM priority) | | return shape | substring list | positional spans `[s, e, ...]` | Pick the builtin for hot paths; pick `re_*` when you need a linear-time -guarantee or are running in a no-libc environment. +guarantee or are running in a no-libc environment. For the latter, +`lib/regex_compat.eigs` re-exposes the builtins' exact names and shapes +on top of the Pike VM (divergences documented in its header). ## Usage @@ -69,10 +80,16 @@ m is re_match of [prog, "ef"] # → 0 prog is re_compile of "[a-zA-Z_][a-zA-Z0-9_]*" r is re_search of [prog, " foo_2 bar"] # → [2, 7] -prog is re_compile of "(\\w+) (\\w+)" # (not yet — use [a-z]+) -prog is re_compile of "([a-z]+) ([a-z]+)" +prog is re_compile of "(\\w+) (\\w+)" r is re_search of [prog, "hello world"] # → [0, 11, 0, 5, 6, 11] prog is re_compile of "a+" all is re_find_all of [prog, "aaabaaab"] # → [[0,3], [4,7]] + +prog is re_compile of "[0-9]{2,4}" +out is re_replace of [prog, "year 2026!", "Y"] # → "year Y!" + +# Builtin-shaped drop-ins (substring lists, not spans): +load_file of "lib/regex_compat.eigs" +regex_match of ["hello world", "h([a-z]+)"] # → ["hello", "ello"] ``` diff --git a/lib/regex.eigs b/lib/regex.eigs index 11034a5..e5d3e3e 100644 --- a/lib/regex.eigs +++ b/lib/regex.eigs @@ -55,6 +55,38 @@ define re_search(args) as: return null return regex_vm_run of [prog, text, 0, 0] +# ---- re_replace: replace every non-overlapping match with a literal string ---- +# args = [prog, text, replacement]. The replacement is inserted literally +# (no $1 backreferences — same as the regex_replace builtin). Zero-width +# matches insert the replacement, then pass one input char through, so an +# empty-matching pattern behaves like the builtin ("x*" over "abc" gives +# "-a-b-c-"). Returns text unchanged when prog is null. +define re_replace(args) as: + local prog is args[0] + local text is args[1] + local rep is args[2] + if prog == null: + return text + local out is "" + local pos is 0 + local n is len of text + loop while pos <= n: + local caps is regex_vm_run of [prog, text, pos, 0] + if caps == null: + break + local ms is caps[0] + local me is caps[1] + out is out + text[pos:ms] + rep + if me > ms: + pos is me + else: + if me < n: + out is out + text[me:me + 1] + pos is me + 1 + if pos <= n: + out is out + text[pos:] + return out + # ---- re_find_all: all non-overlapping matches ---- # Returns a list of capture lists (same shape as re_search results). define re_find_all(args) as: diff --git a/lib/regex_compat.eigs b/lib/regex_compat.eigs new file mode 100644 index 0000000..105e104 --- /dev/null +++ b/lib/regex_compat.eigs @@ -0,0 +1,76 @@ +# ============================================================ +# EigenRegex — builtin-shaped compat layer +# ============================================================ +# +# Drop-in implementations of EigenScript's libc-backed builtins +# (`regex_match` / `regex_find` / `regex_replace`) on top of the Pike-VM +# engine — same names, same argument shapes, same substring-list returns. +# Loading this file SHADOWS the builtins in the current scope; that is the +# point. Intended for environments without libc regex (the freestanding / +# EigenOS profile, WASM) or when the linear-time guarantee matters more +# than raw speed. To differential-test against the live builtins, capture +# them first: bi is regex_match … then load this file. +# +# Faithfully replicated builtin quirks (verified against the oracle): +# - regex_match returns [full, g1, ...] but STOPS at the first +# non-participating group and caps at 16 entries. +# - a pattern that fails to compile returns [] (match/find) or the +# input unchanged (replace) — indistinguishable from no-match. +# - zero-width replace passes one char through per empty match +# ("x*" over "abc" → "-a-b-c-"). +# +# Known divergences from the libc builtins (documented, deliberate): +# - POSIX ERE picks the leftmost-LONGEST alternative; the Pike VM picks +# the leftmost-FIRST (greedy priority). "a|ab" on "ab" differs. +# - \b / \B word boundaries are not supported (parse error). +# - The builtins re-run regexec on the remainder after each match +# without REG_NOTBOL, so "^" can re-anchor mid-string in +# regex_find/regex_replace; here "^" means position 0 only. +# - builtin regex_find drops a zero-width match at end-of-string. + +load_file of "lib/regex.eigs" + +# ---- regex_match: [string, pattern] → [full, g1, ...] or [] ---- +define regex_match(args) as: + local s is args[0] + local pattern is args[1] + local prog is re_compile of pattern + if prog == null: + return [] + local caps is re_search of [prog, s] + if caps == null: + return [] + local out is [] + local i is 0 + loop while i < (len of caps): + if (len of out) >= 16: + break + local ms is caps[i] + if ms < 0: + break + append of [out, s[ms:caps[i + 1]]] + i is i + 2 + return out + +# ---- regex_find: [string, pattern] → list of matched substrings ---- +define regex_find(args) as: + local s is args[0] + local pattern is args[1] + local prog is re_compile of pattern + if prog == null: + return [] + local out is [] + local spans is re_find_all of [prog, s] + for m in spans: + append of [out, s[m[0]:m[1]]] + return out + +# ---- regex_replace: [string, pattern, replacement] → string ---- +define regex_replace(args) as: + local s is args[0] + local pattern is args[1] + local rep is args[2] + local prog is re_compile of pattern + if prog == null: + return s + return re_replace of [prog, s, rep] diff --git a/lib/regex_parse.eigs b/lib/regex_parse.eigs index fefb0cf..d4b5ac7 100644 --- a/lib/regex_parse.eigs +++ b/lib/regex_parse.eigs @@ -43,6 +43,12 @@ define _advance(st) as: st["pos"] is st["pos"] + 1 return c +# ---- _peek2: char after the current one, or "" ---- +define _peek2(st) as: + if (st["pos"] + 1) >= (len of st["src"]): + return "" + return char_at of [st["src"], st["pos"] + 1] + # ---- _is_metachar: is c a reserved metacharacter? (S4 set) ---- define _is_metachar(c) as: if c == "|": @@ -69,6 +75,37 @@ define _is_metachar(c) as: return 1 return 0 +# ---- _posix_ranges: [:name:] → list of [lo, hi] ranges (ASCII), or null ---- +define _posix_ranges(name) as: + if name == "alpha": + return [[65, 90], [97, 122]] + if name == "digit": + return [[48, 57]] + if name == "alnum": + return [[48, 57], [65, 90], [97, 122]] + if name == "upper": + return [[65, 90]] + if name == "lower": + return [[97, 122]] + if name == "space": + return [[9, 13], [32, 32]] + if name == "blank": + return [[9, 9], [32, 32]] + if name == "xdigit": + return [[48, 57], [65, 70], [97, 102]] + if name == "punct": + return [[33, 47], [58, 64], [91, 96], [123, 126]] + if name == "cntrl": + return [[0, 31], [127, 127]] + if name == "graph": + return [[33, 126]] + if name == "print": + return [[32, 126]] + if name == "word": + return [[48, 57], [65, 90], [95, 95], [97, 122]] + # Unknown class name — parse error (glibc regcomp errors here too). + return null + # Forward declarations via mutual recursion are fine in EigenScript # (load order doesn't matter for function refs resolved at call time). @@ -91,6 +128,29 @@ define _parse_class(st) as: if c == "]": _advance of st break + if c == "[": + if (_peek2 of st) == ":": + # POSIX class: [:name:] — expand to ranges and continue. + _advance of st + _advance of st + name is "" + loop while 1: + d is _peek of st + if d == "": + return null + if d == ":": + break + name is name + (_advance of st) + _advance of st + if (_peek of st) != "]": + return null + _advance of st + prs is _posix_ranges of name + if prs == null: + return null + for pr in prs: + append of [ranges, pr] + continue lo_c is _advance of st lo is ord of lo_c if lo < 0: @@ -145,12 +205,118 @@ define _parse_atom(st) as: return ["anchor", "end"] if c == "[": return _parse_class of st + if c == "\\": + _advance of st + e is _advance of st + if e == "": + return null + if e == "w": + return ["class", 0, [[48, 57], [65, 90], [95, 95], [97, 122]]] + if e == "W": + return ["class", 1, [[48, 57], [65, 90], [95, 95], [97, 122]]] + if e == "s": + return ["class", 0, [[9, 13], [32, 32]]] + if e == "S": + return ["class", 1, [[9, 13], [32, 32]]] + # Any other escaped char is that literal char. This matches glibc + # ERE, including its trap: \d is a literal 'd' there, NOT digits — + # verified against the builtin oracle. Use [0-9] or [[:digit:]]. + return ["lit", e] if _is_metachar of c: return null _advance of st return ["lit", c] -# ---- _parse_quant: atom optionally followed by *, *?, +, +?, ?, ?? ---- +# ---- _read_digits: consume a (possibly empty) run of ASCII digits ---- +define _read_digits(st) as: + out is "" + loop while 1: + c is _peek of st + if c == "": + break + if c >= "0": + if c <= "9": + out is out + (_advance of st) + else: + break + else: + break + return out + +# ---- _parse_curly: '{n}' '{n,}' '{n,m}' (optional lazy '?') after an atom ---- +# Desugars to the existing AST forms so the compiler/VM need no new ops: +# e{n} → e repeated n times +# e{n,} → e ×n then e* (star) +# e{n,m} → e ×n then the nested form (e (e (…)?)?)? with m−n levels +# The atom subtree is SHARED between repetitions — safe because the compiler +# never mutates AST nodes. A shared capture group re-saves the same slots +# each repetition, so the LAST repetition wins — matching glibc ERE +# ("^(a|b){2}$" on "ab" captures "b"; verified against the builtin oracle). +# Invalid or unclosed specs are parse errors, as in glibc; bounds cap at 255. +define _parse_curly(st, atom) as: + _advance of st + lo_s is _read_digits of st + if lo_s == "": + return null + lo is num of lo_s + hi is lo + unbounded is 0 + if (_peek of st) == ",": + _advance of st + if (_peek of st) == "}": + unbounded is 1 + else: + hi_s is _read_digits of st + if hi_s == "": + return null + hi is num of hi_s + if (_peek of st) != "}": + return null + _advance of st + lazy is 0 + if (_peek of st) == "?": + _advance of st + lazy is 1 + if lo > 255: + return null + if unbounded == 0: + if hi > 255: + return null + if hi < lo: + return null + parts is [] + i is 0 + loop while i < lo: + append of [parts, atom] + i is i + 1 + if unbounded == 1: + if lazy == 1: + append of [parts, ["star_lazy", atom]] + else: + append of [parts, ["star", atom]] + else: + # Optional tail, innermost-first: m−n nested (e …)? levels. + tail is null + k is hi - lo + loop while k > 0: + if tail == null: + inner is atom + else: + inner is ["concat", [atom, tail]] + if lazy == 1: + tail is ["quest_lazy", inner] + else: + tail is ["quest", inner] + k is k - 1 + if tail != null: + append of [parts, tail] + if (len of parts) == 0: + return ["empty"] + if (len of parts) == 1: + return parts[0] + return ["concat", parts] + +# ---- _parse_quant: atom optionally followed by *, *?, +, +?, ?, ??, {n,m} ---- # Greedy variants prefer to consume; lazy variants prefer to stop. # Encoded by the order of split targets in the compiler. define _parse_quant(st) as: @@ -176,6 +342,8 @@ define _parse_quant(st) as: _advance of st return ["quest_lazy", atom] return ["quest", atom] + if c == "{": + return _parse_curly of [st, atom] return atom # ---- _parse_concat: zero or more quantified atoms, stop at |, ), EOF ---- diff --git a/tests/test_s6_escapes_posix.eigs b/tests/test_s6_escapes_posix.eigs new file mode 100644 index 0000000..638452c --- /dev/null +++ b/tests/test_s6_escapes_posix.eigs @@ -0,0 +1,101 @@ +# S6: backslash escapes (\. \* \\ \w \W \s \S) and POSIX [:classes:] + +load_file of "lib/regex.eigs" + +define check(label, got, want) as: + if got == want: + print of (label + " OK") + else: + print of (label + " FAIL") + print of " got:" + print of got + print of " want:" + print of want + return 0 + +# ---- escaped metacharacters are literals ---- +prog is re_compile of "a\\.c" +check of ["\\. literal dot matches", re_match of [prog, "a.c"], 1] +check of ["\\. literal dot rejects x", re_match of [prog, "axc"], 0] + +prog is re_compile of "a\\*b" +check of ["\\* literal star", re_match of [prog, "a*b"], 1] +check of ["\\* rejects repetition", re_match of [prog, "aab"], 0] + +prog is re_compile of "\\(x\\)" +check of ["\\( \\) literal parens", re_match of [prog, "(x)"], 1] + +prog is re_compile of "a\\\\b" +check of ["\\\\ literal backslash", re_match of [prog, "a\\b"], 1] + +prog is re_compile of "\\$5" +check of ["\\$ literal dollar", re_match of [prog, "$5"], 1] + +# ---- escape classes ---- +prog is re_compile of "\\w+" +check of ["\\w+ letters digits underscore", re_match of [prog, "abc_123"], 1] +check of ["\\w+ rejects space", re_match of [prog, "a b"], 0] + +prog is re_compile of "\\W" +check of ["\\W matches space", re_match of [prog, " "], 1] +check of ["\\W rejects word char", re_match of [prog, "a"], 0] + +prog is re_compile of "a\\sb" +check of ["\\s matches space", re_match of [prog, "a b"], 1] +check of ["\\s matches tab", re_match of [prog, "a\tb"], 1] +check of ["\\s rejects x", re_match of [prog, "axb"], 0] + +prog is re_compile of "\\S+" +check of ["\\S+ nonspace run", re_match of [prog, "abc"], 1] +check of ["\\S+ rejects embedded space", re_match of [prog, "a c"], 0] + +# glibc-parity trap: \d is a literal 'd' in ERE, not a digit class. +prog is re_compile of "\\d+" +check of ["\\d is literal d (glibc parity)", re_match of [prog, "ddd"], 1] +check of ["\\d does not match digits", re_match of [prog, "123"], 0] + +check of ["trailing backslash is a parse error", re_compile of "ab\\", null] + +# ---- POSIX classes ---- +prog is re_compile of "[[:alpha:]]+" +check of ["[:alpha:] letters", re_match of [prog, "AbC"], 1] +check of ["[:alpha:] rejects digit", re_match of [prog, "a1"], 0] + +prog is re_compile of "[[:digit:]]+" +check of ["[:digit:] digits", re_match of [prog, "0129"], 1] +check of ["[:digit:] rejects letter", re_match of [prog, "12a"], 0] + +prog is re_compile of "[[:alnum:]_]+" +check of ["[:alnum:] plus underscore union", re_match of [prog, "id_9"], 1] + +prog is re_compile of "[[:alpha:]0-9]+" +check of ["posix class + range union", re_match of [prog, "a5Z"], 1] + +prog is re_compile of "[^[:digit:]]" +check of ["negated posix matches letter", re_match of [prog, "a"], 1] +check of ["negated posix rejects digit", re_match of [prog, "5"], 0] + +prog is re_compile of "[[:xdigit:]]+" +check of ["[:xdigit:] hex digits", re_match of [prog, "9aF"], 1] +check of ["[:xdigit:] rejects g", re_match of [prog, "g"], 0] + +prog is re_compile of "[[:space:]]" +check of ["[:space:] tab", re_match of [prog, "\t"], 1] +check of ["[:space:] rejects letter", re_match of [prog, "x"], 0] + +prog is re_compile of "[[:upper:]][[:lower:]]+" +check of ["[:upper:][:lower:]+ capitalized word", re_match of [prog, "Word"], 1] +check of ["[:upper:][:lower:]+ rejects lowercase", re_match of [prog, "word"], 0] + +prog is re_compile of "[[:punct:]]" +check of ["[:punct:] comma", re_match of [prog, ","], 1] +check of ["[:punct:] rejects letter", re_match of [prog, "a"], 0] + +check of ["unknown class name errors", re_compile of "[[:bogus:]]", null] +check of ["unterminated posix class errors", re_compile of "[[:alpha:]", null] + +# glibc-parity: backslash inside a bracket class is a literal backslash. +prog is re_compile of "[\\w]" +check of ["class backslash literal (glibc parity)", re_match of [prog, "\\"], 1] +check of ["[\\w] matches w", re_match of [prog, "w"], 1] +check of ["[\\w] does not match a", re_match of [prog, "a"], 0] diff --git a/tests/test_s7_curly.eigs b/tests/test_s7_curly.eigs new file mode 100644 index 0000000..3ef08b1 --- /dev/null +++ b/tests/test_s7_curly.eigs @@ -0,0 +1,98 @@ +# S7: {n} {n,} {n,m} interval quantifiers (greedy + lazy) via desugaring + +load_file of "lib/regex.eigs" + +define check(label, got, want) as: + if got == want: + print of (label + " OK") + else: + print of (label + " FAIL") + print of " got:" + print of got + print of " want:" + print of want + return 0 + +# ---- exact {n} ---- +prog is re_compile of "a{3}" +check of ["a{3} exact", re_match of [prog, "aaa"], 1] +check of ["a{3} too few", re_match of [prog, "aa"], 0] +check of ["a{3} too many", re_match of [prog, "aaaa"], 0] + +prog is re_compile of "a{1}" +check of ["a{1} single", re_match of [prog, "a"], 1] + +prog is re_compile of "a{0}" +check of ["a{0} matches empty", re_match of [prog, ""], 1] +check of ["a{0} rejects a", re_match of [prog, "a"], 0] + +# ---- open-ended {n,} ---- +prog is re_compile of "a{2,}" +check of ["a{2,} minimum", re_match of [prog, "aa"], 1] +check of ["a{2,} many", re_match of [prog, "aaaaaaa"], 1] +check of ["a{2,} too few", re_match of [prog, "a"], 0] + +prog is re_compile of "a{0,}" +check of ["a{0,} is star (empty)", re_match of [prog, ""], 1] +check of ["a{0,} is star (many)", re_match of [prog, "aaaa"], 1] + +# ---- bounded {n,m} ---- +prog is re_compile of "a{2,4}" +check of ["a{2,4} low", re_match of [prog, "aa"], 1] +check of ["a{2,4} mid", re_match of [prog, "aaa"], 1] +check of ["a{2,4} high", re_match of [prog, "aaaa"], 1] +check of ["a{2,4} below", re_match of [prog, "a"], 0] +check of ["a{2,4} above", re_match of [prog, "aaaaa"], 0] + +prog is re_compile of "a{0,2}" +check of ["a{0,2} empty", re_match of [prog, ""], 1] +check of ["a{0,2} two", re_match of [prog, "aa"], 1] +check of ["a{0,2} three", re_match of [prog, "aaa"], 0] + +# ---- compound atoms ---- +prog is re_compile of "(ab){2}" +check of ["(ab){2} matches abab", re_match of [prog, "abab"], 1] +check of ["(ab){2} rejects ab", re_match of [prog, "ab"], 0] + +prog is re_compile of "[a-z]{2,3}" +check of ["[a-z]{2,3} two", re_match of [prog, "xy"], 1] +check of ["[a-z]{2,3} three", re_match of [prog, "xyz"], 1] +check of ["[a-z]{2,3} four", re_match of [prog, "wxyz"], 0] + +prog is re_compile of "\\.{2}" +check of ["escape + interval", re_match of [prog, ".."], 1] +check of ["escape + interval rejects chars", re_match of [prog, "ab"], 0] + +prog is re_compile of "a{2}b{2}" +check of ["adjacent intervals", re_match of [prog, "aabb"], 1] + +# ---- greedy vs lazy spans ---- +prog is re_compile of "a{1,3}" +r is re_search of [prog, "aaa"] +check of ["greedy {1,3} takes maximum", [r[0], r[1]], [0, 3]] + +prog is re_compile of "a{1,3}?" +r is re_search of [prog, "aaa"] +check of ["lazy {1,3}? takes minimum", [r[0], r[1]], [0, 1]] + +# ---- captures: shared group slots, last repetition wins (glibc parity) ---- +prog is re_compile of "(a|b){2}" +r is re_search of [prog, "ab"] +check of ["(a|b){2} full span", [r[0], r[1]], [0, 2]] +check of ["(a|b){2} last repetition wins", [r[2], r[3]], [1, 2]] + +prog is re_compile of "(a){0,2}" +r is re_search of [prog, ""] +check of ["optional group unset reports -1", [r[2], r[3]], [-1, -1]] + +# ---- invalid specs are parse errors (glibc parity) ---- +check of ["unclosed brace errors", re_compile of "a{", null] +check of ["non-numeric bound errors", re_compile of "a{x}", null] +check of ["unterminated spec errors", re_compile of "a{2", null] +check of ["reversed bounds error", re_compile of "a{5,2}", null] +check of ["count over cap errors", re_compile of "a{300}", null] +check of ["upper bound over cap errors", re_compile of "a{1,300}", null] + +# A brace with no preceding atom is a literal character. +prog is re_compile of "{x}" +check of ["literal brace at atom position", re_match of [prog, "{x}"], 1] diff --git a/tests/test_s8_compat.eigs b/tests/test_s8_compat.eigs new file mode 100644 index 0000000..019c29a --- /dev/null +++ b/tests/test_s8_compat.eigs @@ -0,0 +1,83 @@ +# S8: builtin-shaped compat layer (re_replace + regex_* shims) and a +# differential run against the live libc-backed builtins. +# +# The differential cases stay inside the territory where POSIX +# leftmost-longest and Pike-VM leftmost-first agree (no ambiguous +# alternations), and avoid the builtins' NOTBOL re-anchoring quirks — +# both divergences are documented in lib/regex_compat.eigs. + +# Capture the live builtins BEFORE the compat layer shadows them. +bi_match is regex_match +bi_find is regex_find +bi_replace is regex_replace + +load_file of "lib/regex_compat.eigs" + +define check(label, got, want) as: + if got == want: + print of (label + " OK") + else: + print of (label + " FAIL") + print of " got:" + print of got + print of " want:" + print of want + return 0 + +# ---- re_replace core behavior ---- +prog is re_compile of "[0-9]+" +check of ["re_replace basic", re_replace of [prog, "a1 b22 c", "#"], "a# b# c"] +check of ["re_replace no match", re_replace of [prog, "abc", "#"], "abc"] +check of ["re_replace whole string", re_replace of [prog, "123", "#"], "#"] +prog is re_compile of "x*" +check of ["re_replace zero-width passes chars through", re_replace of [prog, "abc", "-"], "-a-b-c-"] +check of ["re_replace null prog returns input", re_replace of [null, "abc", "-"], "abc"] + +# ---- shim shapes match the builtins' shapes ---- +check of ["shim match groups", regex_match of ["hello world", "h([a-z]+) ([a-z]+)"], ["hello world", "ello", "world"]] +check of ["shim match no-match is []", regex_match of ["xyz", "[0-9]+"], []] +check of ["shim match bad pattern is []", regex_match of ["x", "(unclosed"], []] +check of ["shim match truncates at unset group", regex_match of ["b", "(a)|(b)"], ["b"]] +check of ["shim find all", regex_find of ["a1 b2 c3", "[a-z][0-9]"], ["a1", "b2", "c3"]] +check of ["shim find none is []", regex_find of ["abc", "[0-9]"], []] +check of ["shim replace", regex_replace of ["a1 b2", "[0-9]", "#"], "a# b#"] +check of ["shim replace bad pattern returns input", regex_replace of ["x", "(", "-"], "x"] + +# ---- differential: shim vs live builtin, match + find ---- +cases is [] +append of [cases, ["abc", "[a-z]+"]] +append of [cases, ["a1 b2 c3", "[a-z][0-9]"]] +append of [cases, ["user@example.com", "^([a-zA-Z0-9.]+)@([a-zA-Z0-9.]+)$"]] +append of [cases, ["hello world", "h([a-z]+) ([a-z]+)"]] +append of [cases, [" extra spaces ", " +"]] +append of [cases, ["abc def_9", "\\w+"]] +append of [cases, ["a.b then axb", "a\\.b"]] +append of [cases, ["0x1F and 0x2a", "0x[[:xdigit:]]+"]] +append of [cases, ["foo12bar7", "[[:digit:]]+"]] +append of [cases, ["aaab aab ab", "a{2}b"]] +append of [cases, ["aaaa", "a{2,3}"]] +append of [cases, ["b", "(a)|(b)"]] +append of [cases, ["no digits here", "[0-9]+"]] +append of [cases, ["Word And word", "[[:upper:]][[:lower:]]+"]] + +for tc in cases: + s is tc[0] + p is tc[1] + check of ["diff match " + p, regex_match of [s, p], bi_match of [s, p]] + check of ["diff find " + p, regex_find of [s, p], bi_find of [s, p]] + +# ---- differential: replace ---- +rcases is [] +append of [rcases, ["a1 b2", "[0-9]", "#"]] +append of [rcases, ["hello world", "o", "0"]] +append of [rcases, ["xxx", "x{2}", "-"]] +append of [rcases, ["abc", "x*", "-"]] +append of [rcases, ["a-b-c", "-", " "]] +append of [rcases, ["tabs\tand spaces", "[[:space:]]+", "_"]] +append of [rcases, ["nothing", "[0-9]", "#"]] + +for tc in rcases: + s is tc[0] + p is tc[1] + rep is tc[2] + check of ["diff replace " + p, regex_replace of [s, p, rep], bi_replace of [s, p, rep]]