becktothefuture · becktothefuture · Sep 13, 2025 · Aug 31, 2025 · Sep 2, 2025 · Sep 2, 2025
diff --git a/docs/spec/thresholds.yaml → .!46637!.DS_Store b/docs/spec/thresholds.yaml → .!46637!.DS_Store
diff --git a/.DS_Store b/.DS_Store
diff --git a/.cursor/rules.md b/.cursor/rules.md
@@ -15,7 +15,7 @@
 
 1. Follow the plan, in order
 
-- Execute tasks strictly in the order listed under “Critical LM Task Execution Order” in `docs/implementation.md`.
+- Execute tasks strictly in the order listed under “Critical LM Task Execution Order” in `docs/02-implementation/02-Implementation.md`.
 - Announce any intentional deviation with justification; otherwise, do not re-order.
 
 2. Enforce Quality Gates & Definition of Done
@@ -49,7 +49,7 @@
 7. Tests & docs as deliverables
 
 - Add unit/integration tests for new logic and update `docs/qa/README.md` test mapping.
-- Keep `docs/guide/reference/lm-behavior.md` and `docs/implementation.md` in sync with behaviour changes.
+- Keep `docs/06-guides/06-03-reference/lm-behavior.md` and `docs/02-implementation/02-Implementation.md` in sync with behaviour changes.
 
 8. Communication discipline
 
@@ -100,7 +100,7 @@
 
 16. Docs & Questions discipline
 
-- Update `docs/implementation.md`, `docs/guide/reference/lm-behavior.md`, and `docs/qa/README.md` for any behaviour change.
+- Update `docs/02-implementation/02-Implementation.md`, `docs/06-guides/06-03-reference/lm-behavior.md`, and `docs/qa/README.md` for any behaviour change.
 - Log uncertainties in `docs/questions.md`; proceed on safe defaults; revisit once answered.
 
 17. Observability & safety
@@ -114,7 +114,7 @@
 
 ### References
 
-- Plan and task order: `docs/implementation.md`
+- Plan and task order: `docs/02-implementation/02-Implementation.md`
 - QA matrix and CI gates: `docs/qa/README.md`
-- LM policy/behaviour: `docs/guide/reference/lm-behavior.md`
+- LM policy/behaviour: `docs/06-guides/06-03-reference/lm-behavior.md`
 - Questions log: `docs/questions.md`
diff --git a/.cursor/rules/doc2code.mdc b/.cursor/rules/doc2code.mdc
@@ -22,7 +22,7 @@ alwaysApply: true
 # Doc2Code Authoring Rules
 
 - Load these docs when editing `core/**`, `engines/**`, `ui/**`, or `tests/**`:
-  - `docs/implementation.md`, `docs/PRD.md`, `docs/system_principles.md`, `docs/adr/**`.
+  - `docs/02-implementation/02-Implementation.md`, `docs/01-prd/01-PRD.md`, `docs/system_principles.md`, `docs/adr/**`.
 - Do not hand-edit the Swiss-grid header in source files; run `pnpm doc:sync` instead.
 - When introducing new behavior, add a SPEC block (REQ or CONTRACT) in the appropriate doc.
 - PRs must pass `pnpm doc:check`; missing SPECs should block.

diff --git a/.cursor/rules/doc_links.mdc b/.cursor/rules/doc_links.mdc
@@ -6,11 +6,11 @@ alwaysApply: true
 Links to critical files by name for Cursor memory and quick reference.
 
 ## Core Documentation
-- [Implementation Tasks](docs/implementation.md)
-- [PRD](docs/PRD.md)
-- [Project Structure](docs/project_structure.md)
-- [Glossary](docs/README.md#glossary)
-- [Project Overview](docs/README.md)
+- [Implementation Tasks](docs/02-implementation/02-Implementation.md)
+- [PRD](docs/01-prd/01-PRD.md)
+- [Project Structure](docs/14-project-structure/14-project_structure.md)
+- [Glossary](docs/00-index/00-README.md#glossary)
+- [Project Overview](docs/00-index/00-README.md)
 - [System Principles](docs/system_principles.md)
 
 ## Key Directories
@@ -19,10 +19,10 @@ Links to critical files by name for Cursor memory and quick reference.
 - [UI Components](ui/)
 - [Utilities](utils/)
 - [Tests](tests/)
- - [Architecture](docs/architecture/)
+ - [Architecture](docs/04-architecture/)
  - [ADRs](docs/adr/)
  - [Acceptance (BDD)](docs/qa/acceptance/)
- - [Guides](docs/guide/)
+ - [Guides](docs/06-guides/)
  - [Accessibility](docs/a11y/)
 
 ## Entry Point

diff --git a/.cursor/rules/workflow.mdc b/.cursor/rules/workflow.mdc
@@ -5,7 +5,7 @@
 ╚═══════════════════════════════════════════════════════════════════╝
   • WHAT ▸ End-to-end rules for planning, execution, and library use
   • WHY  ▸ Ensure predictable, test-first changes and safe merges
-  • HOW  ▸ Pick tasks from docs/implementation.md, enforce gates, use Context7
+  • HOW  ▸ Pick tasks from docs/02-implementation/02-Implementation.md, enforce gates, use Context7
 -->
 
 ## 0) Preload & Tools
@@ -22,14 +22,14 @@
 ```
 
 ## 1) Task Intake (single source of truth)
-- Read `docs/implementation.md` and pick the **first unchecked task** (top–down) from the highest active Stage.
+- Read `docs/02-implementation/02-Implementation.md` and pick the **first unchecked task** (top–down) from the highest active Stage.
 - If no tasks exist, switch to **PLAN_ONLY** and propose tasks using the schema below.
 
 ## 2) Modes
 
 ### A. PLAN_ONLY (no code edits)
 1) Scan repo and constraints to fulfil the requested outcome.  
-2) **Append tasks** into `docs/implementation.md` under the correct Stage using the **Task Schema**.  
+2) **Append tasks** into `docs/02-implementation/02-Implementation.md` under the correct Stage using the **Task Schema**.  
 3) Keep tasks atomic; each must have **AC** (acceptance criteria).  
 4) Stop and wait for approval.
 
@@ -56,7 +56,7 @@
 - One branch per task; never commit to `main`.  
 - Protected branch checks (typecheck, tests, lint, format) must be required before merge.
 
-## 5) Task Schema (docs/implementation.md)
+## 5) Task Schema (docs/02-implementation/02-Implementation.md)
 - `- [ ] (P1) [FT-123] Title  — path/to/file.ts`  
   **AC:** concrete acceptance criteria (facts, numbers, paths)  
   **Owner:** @alex (optional) • **DependsOn:** FT-122 (optional) • **Source:** Questionnaire #NN / ADR-00X / PRD line

diff --git a/.github/workflows/demo-headers.yml b/.github/workflows/demo-headers.yml
@@ -0,0 +1,20 @@
+name: Demo Header Check
+on:
+  pull_request:
+    paths:
+      - 'web-demo/public/demo/**/index.html'
+      - 'scripts/docs-verify-demo-headers.mjs'
+
+jobs:
+  verify-demo-headers:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: pnpm/action-setup@v4
+        with:
+          version: 9
+      - name: Install deps
+        run: pnpm install --frozen-lockfile
+      - name: Verify header includes in demo pages
+        run: node scripts/docs-verify-demo-headers.mjs
+
diff --git a/.github/workflows/docs-guards.yml b/.github/workflows/docs-guards.yml
@@ -0,0 +1,45 @@
+name: Docs Guards
+on:
+  pull_request:
+    paths:
+      - 'docs/**'
+      - 'scripts/**'
+      - '.github/workflows/docs-guards.yml'
+
+jobs:
+  check-docs-structure:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: pnpm/action-setup@v4
+        with:
+          version: 9
+      - name: Install deps
+        run: pnpm install --frozen-lockfile
+      - name: Enforce numbered folders
+        run: |
+          set -e
+          offending=$(find docs -maxdepth 1 -type f \( -name '*.md' -o -name '*.json' -o -name '*.yaml' -o -name '*.yml' \) -not -name 'index.md' -print)
+          if [ -n "$offending" ]; then
+            echo "Flat files detected at docs root:" >&2
+            echo "$offending" >&2
+            exit 1
+          fi
+      - name: Generate folder indices
+        run: node scripts/docs-generate-indices.mjs
+      - name: Check Swiss-grid header presence
+        run: |
+          set -e
+          missing=$(grep -L "^<!--══════════════════════════════════════════════════" $(git ls-files 'docs/**/*.md' | tr '\n' ' ')) || true
+          if [ -n "$missing" ]; then
+            echo "Files missing Swiss-grid header:" >&2
+            echo "$missing" >&2
+            exit 1
+          fi
+      - name: Markdown link check
+        uses: lycheeverse/lychee-action@v2
+        with:
+          args: --no-progress --verbose --include-fragments 'docs/**/*.md'
+        env:
+          GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}}
+
diff --git a/.gitignore b/.gitignore
@@ -22,3 +22,9 @@ web-demo/node_modules/
 
 
 web-demo/public/models/
+
+# Todo2 - AI-powered task management directories
+.cursor/rules/todo2-overview.mdc
+.cursor/rules/todo2.mdc
+.cursor/mcp.json
+.todo2/
diff --git a/.husky/post-checkout b/.husky/post-checkout
@@ -0,0 +1,3 @@
+#!/bin/sh
+command -v git-lfs >/dev/null 2>&1 || { printf >&2 "\n%s\n\n" "This repository is configured for Git LFS but 'git-lfs' was not found on your path. If you no longer wish to use Git LFS, remove this hook by deleting the 'post-checkout' file in the hooks directory (set by 'core.hookspath'; usually '.git/hooks')."; exit 2; }
+git lfs post-checkout "$@"
diff --git a/.husky/post-commit b/.husky/post-commit
@@ -0,0 +1,3 @@
+#!/bin/sh
+command -v git-lfs >/dev/null 2>&1 || { printf >&2 "\n%s\n\n" "This repository is configured for Git LFS but 'git-lfs' was not found on your path. If you no longer wish to use Git LFS, remove this hook by deleting the 'post-commit' file in the hooks directory (set by 'core.hookspath'; usually '.git/hooks')."; exit 2; }
+git lfs post-commit "$@"
diff --git a/.husky/post-merge b/.husky/post-merge
@@ -0,0 +1,3 @@
+#!/bin/sh
+command -v git-lfs >/dev/null 2>&1 || { printf >&2 "\n%s\n\n" "This repository is configured for Git LFS but 'git-lfs' was not found on your path. If you no longer wish to use Git LFS, remove this hook by deleting the 'post-merge' file in the hooks directory (set by 'core.hookspath'; usually '.git/hooks')."; exit 2; }
+git lfs post-merge "$@"
diff --git a/.husky/pre-push b/.husky/pre-push
@@ -0,0 +1,3 @@
+#!/bin/sh
+command -v git-lfs >/dev/null 2>&1 || { printf >&2 "\n%s\n\n" "This repository is configured for Git LFS but 'git-lfs' was not found on your path. If you no longer wish to use Git LFS, remove this hook by deleting the 'pre-push' file in the hooks directory (set by 'core.hookspath'; usually '.git/hooks')."; exit 2; }
+git lfs pre-push "$@"
diff --git a/.playwright-mcp/web-demo-initial-state.png b/.playwright-mcp/web-demo-initial-state.png
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -20,6 +20,36 @@ All notable changes to this project will be documented in this file.
 
 The format is based on Keep a Changelog and this project adheres to Semantic Versioning (pre‑1.0). Dates are in YYYY‑MM‑DD.
 
+## [0.4.0] - 2025-09-09
+
+### Added
+
+- Core‑owned LM orchestration in Context stage with dual‑context windowing:
+  - `core/lm/contextManager.ts` for Close (2–5 sentences) and Wide (document) contexts
+  - Validation of Close‑context proposals against Wide context prior to commit
+- Web Worker integration for Transformers.js with robust adapter:
+  - `core/lm/workerAdapter.ts` (timeouts, error propagation, health logs)
+  - `core/lm/transformersRunner.ts` (CDN wasmPaths default; `/wasm/` fallback)
+- Workbench in web demo showing LM health, context windows, logs, and metrics
+- Data‑testid hooks in LM Lab for E2E verification; presets JSON support
+
+### Changed
+
+- `engines/contextTransformer.ts` updated to drive LM span selection, prompting, and band‑bounded merges
+- `core/sweepScheduler.ts` passes LM adapter/context to Context stage; removed duplicate LM stage
+- Docs consolidated: canonical LM reference at `docs/06-guides/06-03-reference/lm.md`; architecture updated with v0.4 pipeline
+
+### Fixed
+
+- Stabilized ONNX Runtime Web asset loading (CDN by default; local fallback)
+- Improved worker error handling and abort on new keystroke
+
+### Docs
+
+- Architecture (C1/C2/C3 and Overview) updated to show LM Worker + dual‑context
+- Guides updated: LM reference now includes dual‑context and worker runtime details
+- Root docs index clarified that v0.4 content is consolidated; removed redundant v0.4 files
+
 ## [0.0.1-alpha] - 2025-08-08
 
 ### Added
@@ -31,7 +61,7 @@ The format is based on Keep a Changelog and this project adheres to Semantic Ver
 
 - Prettier config aligned with workspace gates (`.prettierrc`).
 - ESLint flat config polish and consistent single quotes (`eslint.config.js`).
-- `docs/implementation.md` updated with TODOs covering WASM bindings, LLM adapter, engine rules, A11Y, benches, and traceability.
+- `docs/02-implementation/02-Implementation.md` updated with TODOs covering WASM bindings, LLM adapter, engine rules, A11Y, benches, and traceability.
 
 ### Removed
 
@@ -51,14 +81,14 @@ The format is based on Keep a Changelog and this project adheres to Semantic Ver
 
 ### Added
 
-- FT-212: Punctuation normalization in `engines/tidySweep.ts` (spaces around commas/periods, em dash spacing).
+- FT-212: Punctuation normalization in `engines/noiseTransformer.ts` (spaces around commas/periods, em dash spacing).
 - FT-214: Whitespace normalization (collapse multi-spaces/tabs; trim trailing whitespace before newline).
 - FT-216: Capitalization rules (sentence-start capitalization; standalone 'i' → 'I').
 - Web demo: active region alignment and newline safety improvements; `SecurityContext` gating hooks.
 
 ### Tests
 
-- Expanded unit tests across tidySweep rules, diffusion controller, and sweep scheduler; integration harness proves end-to-end flow.
+- Expanded unit tests across noise transformer rules, diffusion controller, and sweep scheduler; integration harness proves end-to-end flow.
 - Added branch-edge tests to lift global branch coverage ≥90%; utils guard at 100% branches.
 
 ### CI / Quality Gates