ERE parity: escapes, POSIX classes, {n,m} intervals, re_replace + builtin-shaped compat layer

CLAUDE.md

@@ -31,9 +31,14 @@ libc-POSIX-backed builtins. EigenRegex prefixes its public API with
 | backend        | libc POSIX ERE in C | Pike-VM in EigenScript |
 | speed          | native, fast        | interpreted, ~100–1000× slower |
 | worst case     | can backtrack catastrophically | guaranteed `O(n·m)` |
-| features       | `{n,m}`, POSIX classes | MVP: `* + ? . [] ^ $ ( )` |
+| features       | POSIX ERE + GNU `\w \s \b` | ERE parity minus `\b`; lazy quantifiers extra |
+| match rule     | leftmost-longest (POSIX) | leftmost-first (Pike-VM priority) |
 | return shape   | substring list      | positional spans `[s, e, ...]` |
 
+`lib/regex_compat.eigs` re-exposes the builtins' exact names/shapes on
+top of the Pike VM (for freestanding/WASM); divergences are documented
+in its header — chiefly leftmost-longest vs leftmost-first, and no `\b`.
+
 **Pick the builtin for hot paths; pick `re_*` when you need a linear
 worst-case guarantee or libc isn't available** (e.g. the WASM
 playground build).
@@ -43,7 +48,7 @@ playground build).
 EigenScript is **not** vendored. Pin v0.13.0 minimum (strings
 needed `<`/`<=` comparison and `ord of s`, both of which shipped in
 v0.13.0 — see GAPS.md for the fix history). CI pins the runtime via
-`.devcontainer/Dockerfile`'s `EIGS_REF` (currently **v0.21.0**) and
+`.devcontainer/Dockerfile`'s `EIGS_REF` (currently **v0.23.0**) and
 builds it from source — bump that to move the tested runtime.
 
 ## Run / test
@@ -64,7 +69,7 @@ $EIGS tests/test_s1_literals.eigs   # ... s2 alt, s3 repeat, s4 classes,
 $EIGS tests/test_s5_anchors_groups.eigs   # s5 anchors/groups, test_smoke
 ```
 
-220 test checks across S1–S5, all green. (`tests/bench_search.eigs` is
+347 test checks across S1–S8, all green. (`tests/bench_search.eigs` is
 a manual timing bench, not part of the gate.)
 
 ## Layout
@@ -75,7 +80,8 @@ a manual timing bench, not part of the gate.)
 | `lib/regex_parse.eigs` | Pattern string → AST |
 | `lib/regex_compile.eigs` | AST → instruction list |
 | `lib/regex_vm.eigs` | Pike-VM executor (parallel-thread simulation) |
-| `tests/test_s{1..5}_*.eigs` | Per-stage tests (literals → alt → repeat → classes → anchors/groups) |
+| `lib/regex_compat.eigs` | Builtin-shaped shim (`regex_match`/`regex_find`/`regex_replace` over the Pike VM) |
+| `tests/test_s{1..8}_*.eigs` | Per-stage tests (literals → alt → repeat → classes → anchors/groups → escapes/POSIX → intervals → compat/differential) |
 | `tests/test_smoke.eigs` | S0 end-to-end load + API smoke |
 | `tests/run.sh` | Suite runner — runs every test, exits non-zero on any FAIL/crash (the CI gate) |
 | `tests/bench_search.eigs` | Manual scaling bench for `re_search` (not a pass/fail gate) |
@@ -97,18 +103,22 @@ a manual timing bench, not part of the gate.)
   separated. Adding a feature usually touches all three of parse,
   compile, vm.
 
-## Supported features (MVP, stable)
+## Supported features (stable)
 
-Literals, concat, `|`, `( )`, `* + ?` (greedy + lazy), `.`,
-`[abc]`, `[^abc]`, `[a-z]`, `^`, `$`, numbered capture groups.
+Literals, escaped metachars (`\.` `\*` …), concat, `|`, `( )`,
+`* + ?` and `{n} {n,} {n,m}` (all greedy + lazy), `.`, `[abc]`,
+`[^abc]`, `[a-z]`, POSIX `[[:alpha:]]`-style classes, `\w \W \s \S`
+(`\d` is a literal `d` — glibc ERE parity, verified against the
+oracle), `^`, `$`, numbered capture groups, `re_replace`, and the
+builtin-shaped compat layer.
 
 ## Out of scope
 
 - Backreferences (force backtracking — incompatible with Pike-VM
   guarantee)
-- Lookahead / lookbehind
+- Lookahead / lookbehind; `\b` word boundaries (would need text access
+  in the VM's epsilon-closure — doable, deferred until something needs it)
 - Named groups
-- `{n,m}` quantifiers
 - Unicode classes (`\p{...}`)
 - Case-insensitive flags (yet)
 
@@ -127,9 +137,16 @@ Literals, concat, `|`, `( )`, `* + ?` (greedy + lazy), `.`,
 
 ## Current state
 
-**S5 complete.** All 220 checks across S1–S5 green. Next is **S6:
-fold into corpus for retraining** — feeding the engine's own source
-+ tests into the iLambdaAi self-training pipeline as a real workload.
+**S8 complete (ERE parity, 2026-07-01).** All 347 checks across S1–S8
+green. S6 added escapes + POSIX classes, S7 added `{n,m}` intervals
+(desugared in the parser — no new VM ops; shared-subtree repetition
+gives glibc's last-repetition-wins capture semantics), S8 added
+`re_replace` + `lib/regex_compat.eigs` and a differential suite that
+runs shim-vs-builtin over shared inputs with the libc builtins as the
+oracle. Built for EigenScript's freestanding profile (EigenOS): the
+compat layer is the planned regex story when libc's regcomp is gone.
+Still open: fold into corpus for retraining — feeding the engine's own
+source + tests into the iLambdaAi self-training pipeline.
 
 **Search is now genuinely O(n·m).** `re_search` used to loop over every
 start position and re-run the VM from each — O(n²), which silently

README.md

@@ -8,33 +8,41 @@ Dual purpose:
 
 ## Status
 
-S5 complete. Working: literals, concat, `|`, `( )`, `* + ?` (greedy + lazy),
-`.`, `[abc]`, `[^abc]`, `[a-z]`, `^`, `$`, numbered capture groups.
-220 test checks across S1–S5, all green. Next: fold into corpus for retraining (S6).
-
-## Supported (target MVP)
-
-- Literal characters
+S8 complete (ERE parity). Working: literals, concat, `|`, `( )`, `* + ?`
+and `{n} {n,} {n,m}` (all greedy + lazy), `.`, `[abc]`, `[^abc]`, `[a-z]`,
+POSIX `[[:alpha:]]`-style classes, escapes (`\.` `\w` `\W` `\s` `\S`),
+`^`, `$`, numbered capture groups, `re_replace`, and a builtin-shaped
+compat layer (`lib/regex_compat.eigs`) that drops in for the libc-backed
+`regex_match` / `regex_find` / `regex_replace` builtins.
+347 test checks across S1–S8, all green — including a differential suite
+run against the live libc builtins as the oracle.
+
+## Supported
+
+- Literal characters and escaped metacharacters (`\.` `\*` `\\` …)
 - Concatenation
 - Alternation `|`
 - Grouping `( ... )`
-- Repetition `*`, `+`, `?` (greedy and lazy)
-- Char classes: `.`, `[abc]`, `[^abc]`, `[a-z]`
+- Repetition `*`, `+`, `?`, `{n}`, `{n,}`, `{n,m}` (greedy and lazy)
+- Char classes: `.`, `[abc]`, `[^abc]`, `[a-z]`, POSIX `[[:alpha:]]` etc.,
+  `\w` `\W` `\s` `\S` (note: `\d` is a literal `d`, matching glibc ERE)
 - Anchors `^`, `$`
 - Capturing groups (numbered)
 
 ## Not supported
 
 - Backreferences (would force backtracking; out of scope for Pike-VM)
-- Lookahead / lookbehind
+- Lookahead / lookbehind; `\b` word boundaries
 - Named groups
-- `{n,m}` quantifiers
 - Unicode classes (`\p{...}`)
 - Case-insensitive flags (yet)
 
 ## Layout
 
-- `lib/regex.eigs` — public API
+- `lib/regex.eigs` — public API (spans-shaped: `re_compile`, `re_match`,
+  `re_search`, `re_find_all`, `re_replace`)
+- `lib/regex_compat.eigs` — builtin-shaped shim (`regex_match` /
+  `regex_find` / `regex_replace` over the Pike VM; shadows the builtins)
 - `lib/regex_parse.eigs` — pattern string → AST
 - `lib/regex_compile.eigs` — AST → instruction list
 - `lib/regex_vm.eigs` — Pike-VM executor
@@ -51,11 +59,14 @@ uses the `re_*` prefix so both can be used in the same script:
 | backend | libc POSIX ERE in C | Pike-VM in EigenScript |
 | speed | native, fast | interpreted, ~100–1000× slower |
 | worst case | can backtrack catastrophically | guaranteed O(n·m) |
-| features | `{n,m}`, POSIX classes | MVP: `* + ? . [] ^ $ ( )` |
+| features | POSIX ERE + GNU `\w \s \b` | ERE parity minus `\b`; lazy quantifiers extra |
+| match rule | leftmost-longest (POSIX) | leftmost-first (Pike-VM priority) |
 | return shape | substring list | positional spans `[s, e, ...]` |
 
 Pick the builtin for hot paths; pick `re_*` when you need a linear-time
-guarantee or are running in a no-libc environment.
+guarantee or are running in a no-libc environment. For the latter,
+`lib/regex_compat.eigs` re-exposes the builtins' exact names and shapes
+on top of the Pike VM (divergences documented in its header).
 
 ## Usage
 
@@ -69,10 +80,16 @@ m is re_match of [prog, "ef"]                    # → 0
 prog is re_compile of "[a-zA-Z_][a-zA-Z0-9_]*"
 r is re_search of [prog, "  foo_2 bar"]          # → [2, 7]
 
-prog is re_compile of "(\\w+) (\\w+)"            # (not yet — use [a-z]+)
-prog is re_compile of "([a-z]+) ([a-z]+)"
+prog is re_compile of "(\\w+) (\\w+)"
 r is re_search of [prog, "hello world"]          # → [0, 11, 0, 5, 6, 11]
 
 prog is re_compile of "a+"
 all is re_find_all of [prog, "aaabaaab"]         # → [[0,3], [4,7]]
+
+prog is re_compile of "[0-9]{2,4}"
+out is re_replace of [prog, "year 2026!", "Y"]   # → "year Y!"
+
+# Builtin-shaped drop-ins (substring lists, not spans):
+load_file of "lib/regex_compat.eigs"
+regex_match of ["hello world", "h([a-z]+)"]      # → ["hello", "ello"]
 ```

lib/regex.eigs

@@ -55,6 +55,38 @@ define re_search(args) as:
         return null
     return regex_vm_run of [prog, text, 0, 0]
 
+# ---- re_replace: replace every non-overlapping match with a literal string ----
+# args = [prog, text, replacement]. The replacement is inserted literally
+# (no $1 backreferences — same as the regex_replace builtin). Zero-width
+# matches insert the replacement, then pass one input char through, so an
+# empty-matching pattern behaves like the builtin ("x*" over "abc" gives
+# "-a-b-c-"). Returns text unchanged when prog is null.
+define re_replace(args) as:
+    local prog is args[0]
+    local text is args[1]
+    local rep is args[2]
+    if prog == null:
+        return text
+    local out is ""
+    local pos is 0
+    local n is len of text
+    loop while pos <= n:
+        local caps is regex_vm_run of [prog, text, pos, 0]
+        if caps == null:
+            break
+        local ms is caps[0]
+        local me is caps[1]
+        out is out + text[pos:ms] + rep
+        if me > ms:
+            pos is me
+        else:
+            if me < n:
+                out is out + text[me:me + 1]
+            pos is me + 1
+    if pos <= n:
+        out is out + text[pos:]
+    return out
+
 # ---- re_find_all: all non-overlapping matches ----
 # Returns a list of capture lists (same shape as re_search results).
 define re_find_all(args) as:

lib/regex_compat.eigs

@@ -0,0 +1,76 @@
+# ============================================================
+# EigenRegex — builtin-shaped compat layer
+# ============================================================
+#
+# Drop-in implementations of EigenScript's libc-backed builtins
+# (`regex_match` / `regex_find` / `regex_replace`) on top of the Pike-VM
+# engine — same names, same argument shapes, same substring-list returns.
+# Loading this file SHADOWS the builtins in the current scope; that is the
+# point. Intended for environments without libc regex (the freestanding /
+# EigenOS profile, WASM) or when the linear-time guarantee matters more
+# than raw speed. To differential-test against the live builtins, capture
+# them first:  bi is regex_match  … then load this file.
+#
+# Faithfully replicated builtin quirks (verified against the oracle):
+#   - regex_match returns [full, g1, ...] but STOPS at the first
+#     non-participating group and caps at 16 entries.
+#   - a pattern that fails to compile returns [] (match/find) or the
+#     input unchanged (replace) — indistinguishable from no-match.
+#   - zero-width replace passes one char through per empty match
+#     ("x*" over "abc" → "-a-b-c-").
+#
+# Known divergences from the libc builtins (documented, deliberate):
+#   - POSIX ERE picks the leftmost-LONGEST alternative; the Pike VM picks
+#     the leftmost-FIRST (greedy priority). "a|ab" on "ab" differs.
+#   - \b / \B word boundaries are not supported (parse error).
+#   - The builtins re-run regexec on the remainder after each match
+#     without REG_NOTBOL, so "^" can re-anchor mid-string in
+#     regex_find/regex_replace; here "^" means position 0 only.
+#   - builtin regex_find drops a zero-width match at end-of-string.
+
+load_file of "lib/regex.eigs"
+
+# ---- regex_match: [string, pattern] → [full, g1, ...] or [] ----
+define regex_match(args) as:
+    local s is args[0]
+    local pattern is args[1]
+    local prog is re_compile of pattern
+    if prog == null:
+        return []
+    local caps is re_search of [prog, s]
+    if caps == null:
+        return []
+    local out is []
+    local i is 0
+    loop while i < (len of caps):
+        if (len of out) >= 16:
+            break
+        local ms is caps[i]
+        if ms < 0:
+            break
+        append of [out, s[ms:caps[i + 1]]]
+        i is i + 2
+    return out
+
+# ---- regex_find: [string, pattern] → list of matched substrings ----
+define regex_find(args) as:
+    local s is args[0]
+    local pattern is args[1]
+    local prog is re_compile of pattern
+    if prog == null:
+        return []
+    local out is []
+    local spans is re_find_all of [prog, s]
+    for m in spans:
+        append of [out, s[m[0]:m[1]]]
+    return out
+
+# ---- regex_replace: [string, pattern, replacement] → string ----
+define regex_replace(args) as:
+    local s is args[0]
+    local pattern is args[1]
+    local rep is args[2]
+    local prog is re_compile of pattern
+    if prog == null:
+        return s
+    return re_replace of [prog, s, rep]