feat(observer): surface coherence — unity horizon + a real 'how' gradient (#412)

CHANGELOG.md

@@ -4,6 +4,21 @@ All notable changes to EigenScript are documented here.
 
 ## [Unreleased]
 
+### Changed
+- **Observer surface coherence (#412), two settled decisions.** (1)
+  **Unity is the horizon**: the `|x| == 1.0` entropy special case is
+  gone — the binary-entropy formula is smooth and maximal there
+  (`H = 1.0`), so a value placed exactly at `1.0` reads like its
+  neighbors (a flat run at `1.0` classifies `equilibrium`, never
+  `converged`); `|x| == 0` keeps `H = 0`, the formula's own home-point
+  limit. (2) **`how` is a real 0–1 gradient**: deadband-normalized
+  settledness of the last observed step, `1 - min(1, |dH|/dh_zero)` —
+  `1.0` = entropy unmoved, `0.0` = moved by the settle deadband or
+  more. Pure in the recorded `dH`, so `how is x at L` reads identically
+  from tape history (no tape format change). The old
+  `1 - entropy/last_entropy` was degenerate (always `0` or `1`).
+  OBSERVER.md's "Rough edges" caveats are closed as settled decisions.
+
 ### Added
 - **JIT loop back-edges poll the async abort flag (#410)** — hard
   timeouts (`eigs_set_abort_flag`) now hold at full speed. The gap was

docs/OBSERVER.md

@@ -50,7 +50,7 @@ history — never from an external reference:
 | `when is x` | assignment count | how many times it has been set |
 | `where is x`| information content | how much information it carries |
 | `why is x`  | change in that content | how fast that is changing |
-| `how is x`  | a stability reading | (see *Rough edges* — currently coarse) |
+| `how is x`  | settledness in `[0, 1]` | how settled the last step left it |
 
 Two of these are the load-bearing pair:
 
@@ -145,7 +145,11 @@ infinity. So the value's information-landscape is a **watershed**:
 
 A value can *ride over* the ridge by ordinary motion (going `1.5 → 0.9`
 just passes the peak and comes down the other side). Landing **exactly**
-on `|x| = 1` is different — see *Rough edges*.
+on `|x| = 1` is not special (#412 decided this): the formula is smooth
+and maximal there (`H = 1.0`), so an exactly-placed `1.0` reads like its
+neighbors — maximally entropic, never `converged` (a flat run at `1.0`
+classifies `equilibrium`: steady, but not at a low-entropy home). Unity
+is the **horizon**, not a home point; `0` is the home point (`H = 0`).
 
 A consequence worth internalizing: because `where` is not monotonic, a
 value whose magnitude is *shrinking* does not always read as `improving`.
@@ -303,25 +307,32 @@ explicit `ensure of [(abs of x) < LIMIT]`. The contracts document this and file
 it rather than hiding the instrument's edge behind an ad-hoc heuristic — the
 forcing-function model: a gap surfaces upstream instead of being papered over.
 
-## Rough edges (current implementation)
-
-Honest notes about where the code, as of this writing, does not yet fully
-honor the model above. These are observations, not yet fixes.
-
-- **The unity horizon is special-cased to zero.** `compute_entropy_impl`
-  returns `0` for `|x|` exactly `0` or `1`. At `1` that is the *opposite*
-  of the formula's limit there (which approaches the maximum). The
-  defensible reading is that the ridge can be *approached* by motion but
-  only *occupied* by direct assignment, so an exactly-placed `1` is marked
-  as a boundary event. The practical effect: a value sitting at exactly
-  `1.0` reports as `converged` immediately. Worth a deliberate decision:
-  is unity the horizon (drop the special case) or a home point (re-center
-  the formula)?
-
-- **`how` is degenerate.** It computes `1 - entropy/last_entropy`, but the
-  observer sets those two equal on every refresh, so `how` is effectively
-  always `0` (or `1` when entropy is `0`). It does not currently provide
-  the `0–1` gradient its name suggests. Treat it as not-yet-implemented.
+## Settled decisions (formerly "Rough edges")
+
+Two long-standing caveats were decided and closed by #412:
+
+- **Unity is the horizon.** `compute_entropy_impl` used to special-case
+  `|x| == 1.0` to entropy `0` — the *opposite* of the formula's value
+  there (`1.0`, the maximum) — so a value placed exactly at `1.0`
+  reported `converged` immediately. The special case is gone: the
+  formula is smooth at the ridge and an exactly-placed `1.0` now reads
+  like its neighbors. `|x| == 0` keeps entropy `0`, which *is* the
+  formula's limit at the home point.
+
+- **`how` is a real gradient now.** It reads the deadband-normalized
+  settledness of the last observed step: `1 - min(1, |dH| / dh_zero)`,
+  where `dh_zero` is the same settle threshold the `converged` window
+  uses (`set_observer_thresholds`). `1.0` = the last assignment left the
+  entropy unmoved; `0.0` = it moved by the deadband or more; linear in
+  between. It is a pure function of the recorded `dH`, so
+  `how is x at L` reads identically from tape history, and it measures
+  the **entropy** trajectory like `where`/`why` — a value-magnitude step
+  that lands at the same information content is settled by construction
+  (the value-channel reading is `report_value`'s job, #294). The old
+  `1 - entropy/last_entropy` was degenerate (the observer refreshes
+  `last_entropy` on every push, so it read `0` or `1` only).
+
+One observation stands (not a defect — a property to know):
 
 - **Trajectories are sampled at every *assignment*, not at reads.** Since the
   slot-keyed rewrite (#262), a binding that is interrogated *anywhere* in the

src/eigenscript.c

@@ -219,13 +219,32 @@ static int env_hash_find(const EnvHash *ht, const char *name, uint32_t h, char *
  * Value (the default path no longer promotes observed nums to tracked
  * pointers). Single source of truth for the VAL_NUM entropy formula. */
 static double entropy_of_num(double num) {
+    /* #412: unity is the HORIZON, not a home point. The binary-entropy
+     * formula is smooth and maximal at |x| = 1 (p = 0.5, H = 1.0); the old
+     * `x == 1.0 -> 0.0` special case inverted it, so a value placed exactly
+     * at 1.0 read as converged while 1.0000001 read as maximally entropic —
+     * a discontinuity the formula never had. |x| = 0 keeps H = 0 (p = 1,
+     * the guard below), which IS the formula's limit there. */
     double x = fabs(num);
-    if (x == 0.0 || x == 1.0) return 0.0;
+    if (x == 0.0) return 0.0;
     double p = 1.0 / (1.0 + x);
     if (p <= 0.0 || p >= 1.0) return 0.0;
     return -(p * log2(p) + (1.0-p) * log2(1.0-p));
 }
 
+/* #412: `how` — deadband-normalized settledness of the last observed step.
+ * 1.0 = the last assignment moved entropy not at all; 0.0 = it moved by the
+ * settle deadband (g_obs_dh_zero, the same threshold the converged window
+ * uses) or more; linear in between. A pure function of the recorded dH, so
+ * `how is x at L` reads identically from tape history — no new record kind.
+ * (The old 1 - entropy/last_entropy was degenerate: the observer refreshes
+ * last_entropy to entropy on every push, so it read 0 or 1 only.) */
+double observer_settledness(double dH) {
+    double r = fabs(dH) / g_obs_dh_zero;
+    if (r > 1.0) r = 1.0;
+    return 1.0 - r;
+}
+
 static double compute_entropy_impl(Value *v, int depth) {
     if (!v || depth > 64) return 0.0;
     switch (v->type) {

src/eigenscript.h

@@ -999,6 +999,12 @@ void eigs_module_cache_clear(void);
 /* Observer thresholds are EigsState fields — set via set_observer_thresholds;
  * read through g_obs_dh_zero / g_obs_dh_small / g_obs_h_low (macros above). */
 
+/* #412: `how` — deadband-normalized settledness of the last observed step,
+ * 1.0 (unmoved) .. 0.0 (moved by >= the settle deadband). Pure function of
+ * the recorded dH, shared by the live INTERROGATE paths (vm.c) and the tape
+ * history reader (trace.c) so `how is x at L` matches the live reading. */
+double observer_settledness(double dH);
+
 /* ---- Cross-file functions for MODEL tensor builtins ---- */
 /* When MODEL is enabled, these are defined in model_infer.c.
  * When disabled, eigenscript.c provides static stubs. */

src/trace.c

@@ -343,8 +343,7 @@ int trace_query_at(int kind, const char *interned_name, int line, EigsSlot *out)
         if (!o->valid) return 0;
         double r = (kind == 3) ? o->entropy
                  : (kind == 4) ? o->dH
-                 : (o->last_entropy > 0 ? 1.0 - o->entropy / o->last_entropy
-                                        : 1.0);
+                 : observer_settledness(o->dH);   /* #412: how = f(dH) */
         *out = slot_from_num(r);
         return 1;
     }

src/vm.c

@@ -4202,7 +4202,7 @@ static Value *vm_run(EigsChunk *chunk, Env *env) {
         }
         case 3: case 4: case 5: {
             /* #262 Phase-3 A: where/why/how on a bound name read the binding's
-             * SLOT trajectory (entropy / dH / 1 - entropy/last_entropy). Falls
+             * SLOT trajectory (entropy / dH / settledness-of-dH, #412). Falls
              * back to the Value's observer fields if the slot isn't populated
              * yet, so the answer matches the value path. */
             int oidx = -1, odepth = 0;
@@ -4211,8 +4211,7 @@ static Value *vm_run(EigsChunk *chunk, Env *env) {
                 const ObserverSlot *s = &oe->obs[oidx];
                 if (kind == 3)      result = make_num(s->entropy);
                 else if (kind == 4) result = make_num(s->dH);
-                else                result = make_num(s->last_entropy > 0
-                                            ? 1.0 - s->entropy / s->last_entropy : 1.0);
+                else                result = make_num(observer_settledness(s->dH));
             } else {
                 /* #262 Step E: unobserved binding — no trajectory. */
                 result = make_num(kind == 5 ? 1.0 : 0.0);

tests/observer_corpus/golden/EigenScript__structural_observer.out

@@ -1,6 +1,6 @@
 equilibrium
 stable
-["stable", 0.7219280948873623, -0.19636773916712724, 0.9182958340544896]
+["stable", 0.7219280948873623, -0.19636773916712724, -0.08170416594551044]
 0.00032292460179985625
 4
 1

tests/observer_corpus/golden/dynamics__physics.out

@@ -2,9 +2,9 @@
 observe ENERGY (Lyapunov) vs DISPLACEMENT (oscillation); flags = regime visited
 
   zeta     | E:osc conv div grew | x:osc conv div | reading
-  -0.05	| E:Y - Y Y     | x:Y - Y      | DIVERGES (energy grew)
-  0	| E:Y - - -     | x:Y - Y      | x OSCILLATES, energy settles
-  0.1	| E:- Y - -     | x:Y - Y      | x OSCILLATES, energy settles
+  -0.05	| E:Y - Y Y     | x:Y - -      | DIVERGES (energy grew)
+  0	| E:Y - - -     | x:Y - -      | x OSCILLATES, energy settles
+  0.1	| E:- Y - -     | x:Y - -      | x OSCILLATES, energy settles
   0.4	| E:- Y - -     | x:Y Y -      | x OSCILLATES, energy settles
   1	| E:- Y - -     | x:- Y -      | settles
   2	| E:- Y - -     | x:- Y -      | settles

tests/observer_corpus/golden/dynamics__solve.out

@@ -4,10 +4,10 @@ not a magnitude tolerance, decides when to stop. (iters include the hold)
 
 Jacobi        Ax=b -> [0.9999995231628418, 0.9999995231628418, 0.9999995231628418]
               14 iters
-Gauss-Seidel  Ax=b -> [0.9999999953433871, 0.9999999976716936, 0.9999999994179234]
-              10 iters  (fresh values -> fewer than Jacobi)
+Gauss-Seidel  Ax=b -> [0.9999999994179234, 0.9999999997089617, 0.9999999999272404]
+              11 iters  (fresh values -> fewer than Jacobi)
 
-Power iteration dominant eigenvalue -> 2  (5 iters, expect ~2)
+Power iteration dominant eigenvalue -> 2  (6 iters, expect ~2)
 
 PageRank stationary distribution -> [0.39998372395833337, 0.20001220703125, 0.4000040690104167]  (27 iters)
 

tests/observer_corpus/golden/iLambdaAi__test_interrogative_spec.out

@@ -43,11 +43,11 @@ WHY returns gradient (number):
 
 --- HOW: stability (0-1) ---
 HOW returns stability (0-1 range):
-0
+1
 
 --- ARITHMETIC COMPOSITION ---
 Arithmetic on interrogatives works:
-101
+102
 
 --- ASSIGNMENT THEN INTERROGATIVE ---
 256

tests/run_all_tests.sh

@@ -2395,6 +2395,11 @@ echo ""
 # full-window single call still converges on the reused env.
 echo "[Observer] Parked call-env observer reset"
 check_eigs_suite "observer park-env reset" test_observer_park.eigs "OBS_PARK_OK" 1
+
+# #412: the settled observer-surface decisions — unity horizon (entropy at
+# |x|=1.0 is the formula max, never converged) and `how` as the
+# deadband-normalized settledness gradient.
+check_eigs_suite "observer coherence (#412)" test_observer_coherence.eigs "All tests passed" 10
 echo ""
 
 # #366: frameless leaf-accessor call fast path — results, borrows from

tests/test_observer_coherence.eigs

@@ -0,0 +1,72 @@
+# test_observer_coherence.eigs — #412: the two settled decisions.
+#
+# 1. Unity is the HORIZON: entropy(|x| = 1.0) is the formula's smooth
+#    maximum (1.0), not the old special-cased 0 — so a value placed exactly
+#    at 1.0 never reads `converged` (a flat run there is `equilibrium`:
+#    steady, but not at a low-entropy home). entropy(0) stays 0 (home point).
+# 2. `how` is deadband-normalized settledness of the last step:
+#    1 - min(1, |dH| / dh_zero) — a real 0..1 gradient of the ENTROPY
+#    trajectory (value-channel readings are report_value's job, #294),
+#    pure in the recorded dH so `how is x at L` matches the live reading.
+
+pass_count is 0
+fail_count is 0
+
+define check(args) as:
+    label is args[0]
+    cond is args[1]
+    if cond:
+        pass_count is pass_count + 1
+    else:
+        fail_count is fail_count + 1
+        print of ("FAIL: " + label)
+
+# --- unity horizon ---
+x is 1.0
+check of ["entropy at exactly 1.0 is the formula max", (where is x) == 1.0]
+nx is 0 - 1.0
+check of ["entropy at exactly -1.0 too", (where is nx) == 1.0]
+z is 0.0
+check of ["entropy at 0 stays 0 (home point)", (where is z) == 0.0]
+
+y is 1.0
+i is 0
+loop while i < 12:
+    y is 1.0
+    i is i + 1
+check of ["flat run AT 1.0 is equilibrium, not converged", (report of y) == "equilibrium"]
+check of ["converged predicate stays false at the horizon", (converged of y) == 0]
+
+# a genuine convergence to the home region still classifies converged
+c is 100.0
+j is 0
+loop while j < 40:
+    c is c * 0.5
+    j is j + 1
+check of ["decay toward 0 still converges", (report of c) == "converged"]
+
+# --- how gradient ---
+s is 5.0
+k is 0
+loop while k < 12:
+    s is 5.0
+    k is k + 1
+check of ["settled binding: how == 1", (how is s) == 1.0]
+
+b is 5.0
+b is 1.0
+check of ["entropy step >= deadband: how == 0", (how is b) == 0.0]
+check of ["how tracks the recorded dH", (why is b) > 0.001]
+
+# in [0, 1] always
+q is 3.0
+q is 3.0001
+hq is how is q
+check of ["how stays in [0,1]", (hq >= 0.0) and (hq <= 1.0)]
+
+total is pass_count + fail_count
+print of f"Observer coherence tests: {pass_count}/{total} passed"
+if fail_count == 0:
+    print of "All tests passed"
+else:
+    assert of [0, "Some observer coherence tests failed"]

tests/test_observer_park.eigs

@@ -5,15 +5,19 @@
 # silently "converges" on its 2nd invocation by inheriting the 1st call's
 # samples. The window is 10.
 
+# #412 note: the settling value is 0.001, NOT 1.0 — unity is the entropy
+# HORIZON now (H = 1.0 there, >= h_low), so nothing parked at exactly 1.0 can
+# classify converged; 0.001 sits in the low-entropy home region (H ~ 0.011).
+
 # (a) DRIFT GUARD: 6 observed assigns/call (< 10) must NEVER fill the window,
 #     no matter how many times the function is called on a recycled env.
 define few() as:
-    local x is 1.0
-    x is 1.0
-    x is 1.0
-    x is 1.0
-    x is 1.0
-    x is 1.0
+    local x is 0.001
+    x is 0.001
+    x is 0.001
+    x is 0.001
+    x is 0.001
+    x is 0.001
     return converged of x
 local a1 is few of null
 local a2 is few of null
@@ -23,18 +27,18 @@ local a3 is few of null
 #     every time, including on the reused env — so the park-reset doesn't
 #     over-correct and suppress legitimate convergence.
 define many() as:
-    local y is 1.0
-    y is 1.0
-    y is 1.0
-    y is 1.0
-    y is 1.0
-    y is 1.0
-    y is 1.0
-    y is 1.0
-    y is 1.0
-    y is 1.0
-    y is 1.0
-    y is 1.0
+    local y is 0.001
+    y is 0.001
+    y is 0.001
+    y is 0.001
+    y is 0.001
+    y is 0.001
+    y is 0.001
+    y is 0.001
+    y is 0.001
+    y is 0.001
+    y is 0.001
+    y is 0.001
     return converged of y
 local b1 is many of null
 local b2 is many of null