Phase I MVP: scaffold + CIFAR-10→CIFAR-100 continual pipeline + smoke tests

.github/workflows/ci.yml

@@ -0,0 +1,38 @@
+name: ci
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+permissions:
+  contents: read
+
+jobs:
+  lint-and-smoke:
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+          cache: pip
+
+      - name: Install CPU-only deps
+        run: |
+          python -m pip install --upgrade pip
+          # CPU-only torch + torchvision to keep CI cheap.
+          pip install --index-url https://download.pytorch.org/whl/cpu \
+              torch torchvision
+          pip install -e .[dev]
+
+      - name: ruff check
+        run: ruff check .
+
+      - name: pytest (smoke only, CPU)
+        env:
+          TORCH_USE_CUDA_DSA: "0"
+        run: pytest -q

.gitignore

@@ -0,0 +1,48 @@
+# Python
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+.Python
+
+# Build artifacts
+build/
+dist/
+*.egg-info/
+*.egg
+.eggs/
+
+# Virtual environments
+.venv/
+venv/
+env/
+ENV/
+
+# Dataset cache
+data/
+datasets/
+*.tar.gz
+cifar-10-batches-py/
+cifar-100-python/
+
+# Test / tooling caches
+.pytest_cache/
+.ruff_cache/
+.mypy_cache/
+.tox/
+htmlcov/
+.coverage
+.coverage.*
+
+# IDE / OS
+.idea/
+.vscode/
+.DS_Store
+
+# Checkpoints & logs
+checkpoints/
+runs/
+*.pt
+*.pth
+*.ckpt
+logs/

README.md

@@ -5,26 +5,88 @@ Reference implementation anchor for:
 **"Hierarchical Abstraction is All You Need: A Mathematical Framework for Continuous AI Consciousness Through Sleep-Wake Cycles"**
 Jihyuk Im, Claude Opus 4. June 2025.
 
-This repository will host:
-
-- Minimal sleep–wake cycle reference (L1 context / L2 LoRA / L3 small model / L4 large model hierarchy)
-- Reproducibility scripts for the continual-learning benchmark reported in §3 of the paper
-- Entropy-aware context compression reference (from the pseudocode in §2.4)
+Paper: <https://github.com/labforadvancedstudy/paper/blob/main/HAL/L1_Hierarchical_Abstraction_All_You_Need_v2.md>
 
 ## Status
 
-**Stub.** Code and benchmarks will land in phases:
+**Phase I MVP scaffolding.** Code runs, tests pass, benchmark numbers have not been measured yet (CI has no GPU). See [`results/README.md`](results/README.md).
+
+Vision-stack mapping of the paper's LLM-stack hierarchy (paper §2.3):
+
+| Layer | Paper (LLM stack) | This MVP (vision stack) |
+|-------|-------------------|-------------------------|
+| L1 | Context window | Task-batch ring buffer (`hal_sleep_wake/memory.py`) |
+| L2 | LoRA weights | LoRA adapter on `Conv2d` + `fc` (`hal_sleep_wake/model.py`) |
+| L3 | Small model (7B) | ResNet-18 base weights (`hal_sleep_wake/model.py`) |
+| L4 | Large model (70B) | *Out of scope — Phase II+* |
+
+Task sequence (subset of paper §3.1): **CIFAR-10 → CIFAR-100**. Remaining datasets land in follow-up PRs.
+
+## Install
+
+```bash
+pip install -e '.[dev]'
+```
+
+`torch` + `torchvision` are heavy dependencies; for CPU-only use the PyTorch CPU wheel index:
+
+```bash
+pip install --index-url https://download.pytorch.org/whl/cpu torch torchvision
+pip install -e '.[dev]'
+```
+
+## Run
+
+Dry-run (prints the planned loop without downloading data or building a model):
+
+```bash
+python scripts/train_cifar10_cifar100.py --dry-run
+```
+
+Real training (needs a GPU + dataset download):
+
+```bash
+python scripts/train_cifar10_cifar100.py \
+    --data-root ./data --download --device cuda \
+    --results-out results/runs/hal.json
+
+python scripts/eval.py results/runs/hal.json
+```
+
+`--no-sleep` disables the consolidation phase (baseline for the ablation row). `--replay-n N` draws N samples from the L1 buffer into each wake step (0 = off, the default). Both knobs share the same code path so baseline vs HAL runs are directly comparable.
+
+## Known limitations (Phase I MVP)
+
+Flagged explicitly so the follow-up PRs target the right gaps, and so the eventual measurement numbers aren't read as something they're not:
+
+- **Shared classifier head.** The model uses `max(num_classes_per_task) = 100` logits for both tasks. CIFAR-10 trains against the first 10 positions; the remaining 90 logits are not masked during eval. Per-task heads land in a follow-up PR.
+- **`sleep_scale` is ad-hoc.** Paper §2.4 does not define this scalar. It's an ablation knob only; leaving it at `1.0` reproduces the paper's unscaled `W' = W + (α/r) · B @ A`.
+- **LoRA targets omit residual projections.** `("conv1", "conv2", "fc")` misses ResNet-18's `downsample.0` 1×1 convs on the residual path. Documented choice, not a bug.
+- **Forgetting rate formula.** Uses `max_{j ≤ T-1} A[i,j] − A[i,T-1]` (includes the final round in the max). Stays non-negative even if a task improves on its last round; see `tests/test_metrics.py::test_forgetting_zero_when_task_improves`.
+- **No GPU numbers yet.** CI has no GPU; `results/baseline_vs_hal.md` is unpopulated until a follow-up measurement PR.
+
+## Test
+
+```bash
+ruff check .
+pytest -q
+```
+
+Smoke tests run on CPU and do not download datasets. CI (`.github/workflows/ci.yml`) runs exactly these two commands.
+
+## Sleep cycle, concretely
+
+Paper §2.4 gives the full pseudocode (wake → REM → NREM). This MVP implements the minimum viable subset:
 
-1. Notebook-scale sleep–wake prototype on a single LLM + LoRA
-2. Continual-learning benchmark suite (CIFAR-10/100 → permuted MNIST → Fashion-MNIST → SVHN → CelebA → TinyImageNet)
-3. Ablations (no-REM, no-NREM, no-entropy-compression, fixed vs adaptive schedule)
+1. **Wake** (`hal_sleep_wake/wake.py`) — SGD over LoRA-only params on the current task's `DataLoader`, pushing every batch into the L1 buffer.
+2. **Sleep — NREM consolidation** (`hal_sleep_wake/sleep.py`) — at the end of each task, LoRA deltas are merged into the ResNet base weights (`W ← W + s · (α/r) · B @ A`) and then reset to a fresh Kaiming / zero init. Optimizer state is wiped.
 
-## Paper
+REM synthetic-dream generation and entropy-aware context compression (paper §2.4 lines 85–101) are explicit follow-up items.
 
-See `paper/HAL/L1_Hierarchical_Abstraction_All_You_Need_v2.md` in the companion paper repository:
+## Phase I goal
 
-<https://github.com/labforadvancedstudy/paper>
+Per paper §5 Implementation Roadmap: *"50% forgetting reduction on the CIFAR sequence vs a vanilla baseline."* This repository will report the raw numbers in [`results/baseline_vs_hal.md`](results/README.md) as soon as a GPU run is completed.
 
 ## License
 
-TBD (will track the companion paper repo's license once that is finalized).
+TBD (will track the companion [paper repository](https://github.com/labforadvancedstudy/paper) once that is finalized).

hal_sleep_wake/__init__.py

@@ -0,0 +1,7 @@
+"""hal-sleep-wake: Phase I MVP scaffolding.
+
+Implements a vision-stack mapping of the HAL paper's LLM-stack hierarchy
+(L1 context buffer / L2 LoRA / L3 backbone). No L4 tier in Phase I.
+"""
+
+__version__ = "0.0.1"

hal_sleep_wake/config.py

@@ -0,0 +1,75 @@
+"""Hyperparameter dataclass for the Phase I continual-learning scaffold."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+
+@dataclass
+class HyperParams:
+    """Single source of truth for MVP hyperparameters.
+
+    Sleep merge formula (paper §2.4): ``W' = W + (alpha / r) * B @ A``.
+    ``sleep_scale`` (``s``) is an ad-hoc scalar not from the paper; it
+    multiplies that delta so ablations can attenuate consolidation:
+    ``W' = W + s * (alpha / r) * B @ A``. Default 1.0 reproduces the
+    paper's unscaled formula.
+    """
+
+    # Task schedule (Phase I: CIFAR-10 then CIFAR-100).
+    tasks: tuple[str, ...] = ("cifar10", "cifar100")
+    num_classes_per_task: tuple[int, ...] = (10, 100)
+
+    # Training.
+    epochs_per_task: int = 1
+    batch_size: int = 64
+    lr: float = 1e-3
+    weight_decay: float = 0.0
+
+    # L2 LoRA adapter.
+    lora_rank: int = 4
+    lora_alpha: int = 8
+    lora_target_modules: tuple[str, ...] = ("conv1", "conv2", "fc")
+
+    # Sleep / wake cadence.
+    sleep_every_n_tasks: int = 1  # fire sleep at the end of every task
+    sleep_scale: float = 1.0      # extra ad-hoc scalar on the merged delta
+
+    # L1 working-memory buffer + replay.
+    buffer_capacity: int = 256
+    replay_n: int = 0             # samples drawn from L1 per wake step; 0 = off
+
+    # Runtime.
+    device: str = "cpu"
+    seed: int = 0
+
+    def __post_init__(self) -> None:
+        if len(self.num_classes_per_task) != len(self.tasks):
+            raise ValueError(
+                f"num_classes_per_task length ({len(self.num_classes_per_task)}) "
+                f"must match tasks length ({len(self.tasks)})"
+            )
+        if self.replay_n < 0:
+            raise ValueError("replay_n must be >= 0")
+        if self.buffer_capacity <= 0:
+            raise ValueError("buffer_capacity must be positive")
+
+    def plan(self) -> str:
+        """Return a compact plan string (used by --dry-run)."""
+        lines = [
+            "HAL sleep-wake | Phase I MVP plan",
+            f"  tasks                 : {list(self.tasks)}",
+            f"  classes per task      : {list(self.num_classes_per_task)}",
+            f"  epochs per task       : {self.epochs_per_task}",
+            f"  batch size            : {self.batch_size}",
+            f"  lr / weight decay     : {self.lr} / {self.weight_decay}",
+            f"  LoRA rank / alpha     : {self.lora_rank} / {self.lora_alpha}",
+            f"  LoRA targets          : {list(self.lora_target_modules)}",
+            f"  sleep cadence         : every {self.sleep_every_n_tasks} task(s)",
+            f"  sleep merge scale (s) : {self.sleep_scale}",
+            f"  L1 buffer capacity    : {self.buffer_capacity}",
+            f"  L1 replay per step    : {self.replay_n}",
+            f"  device                : {self.device}",
+            f"  seed                  : {self.seed}",
+        ]
+        return "\n".join(lines)

hal_sleep_wake/data.py

@@ -0,0 +1,92 @@
+"""Dataset loaders for Phase I continual-learning scaffold.
+
+CIFAR-10 and CIFAR-100 from torchvision. Kept minimal on purpose — the
+MVP only consumes two tasks in sequence. Additional datasets from the
+paper §3.1 sequence will land in follow-up PRs.
+
+The loaders here never download data unless explicitly asked; the dry-run
+mode in ``scripts/train_cifar10_cifar100.py`` must not hit the network.
+"""
+
+from __future__ import annotations
+
+from collections.abc import Callable
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Any
+
+import torch
+from torch.utils.data import DataLoader
+
+
+@dataclass
+class TaskDatasets:
+    """One (train, test) pair plus the task's label count."""
+
+    name: str
+    train: Any  # torch.utils.data.Dataset but torchvision types vary
+    test: Any
+    num_classes: int
+
+
+def _cifar_transform() -> Callable[[Any], torch.Tensor]:
+    # Kept inline so the module imports without torchvision at import time
+    # when only type hints / dataclasses are needed (e.g. dry-run plan).
+    from torchvision import transforms
+
+    return transforms.Compose(
+        [
+            transforms.ToTensor(),
+            transforms.Normalize(
+                mean=(0.4914, 0.4822, 0.4465),
+                std=(0.2470, 0.2435, 0.2616),
+            ),
+        ]
+    )
+
+
+def load_cifar10(root: str | Path, *, download: bool = False) -> TaskDatasets:
+    from torchvision import datasets
+
+    tfm = _cifar_transform()
+    train = datasets.CIFAR10(root=str(root), train=True, transform=tfm, download=download)
+    test = datasets.CIFAR10(root=str(root), train=False, transform=tfm, download=download)
+    return TaskDatasets(name="cifar10", train=train, test=test, num_classes=10)
+
+
+def load_cifar100(root: str | Path, *, download: bool = False) -> TaskDatasets:
+    from torchvision import datasets
+
+    tfm = _cifar_transform()
+    train = datasets.CIFAR100(root=str(root), train=True, transform=tfm, download=download)
+    test = datasets.CIFAR100(root=str(root), train=False, transform=tfm, download=download)
+    return TaskDatasets(name="cifar100", train=train, test=test, num_classes=100)
+
+
+_LOADERS: dict[str, Callable[..., TaskDatasets]] = {
+    "cifar10": load_cifar10,
+    "cifar100": load_cifar100,
+}
+
+
+def load_task(name: str, root: str | Path, *, download: bool = False) -> TaskDatasets:
+    """Dispatch to the correct loader by task name."""
+    if name not in _LOADERS:
+        raise KeyError(f"unknown task '{name}'; known: {sorted(_LOADERS)}")
+    return _LOADERS[name](root, download=download)
+
+
+def make_loaders(
+    ds: TaskDatasets,
+    *,
+    batch_size: int,
+    num_workers: int = 0,
+    shuffle_train: bool = True,
+) -> tuple[DataLoader, DataLoader]:
+    train_loader = DataLoader(
+        ds.train, batch_size=batch_size, shuffle=shuffle_train, num_workers=num_workers
+    )
+    test_loader = DataLoader(
+        ds.test, batch_size=batch_size, shuffle=False, num_workers=num_workers
+    )
+    return train_loader, test_loader