From 5a759467e356b111e1917f19bcbbc4baba76bb47 Mon Sep 17 00:00:00 2001 From: Joshua Natarajan Date: Sun, 10 May 2026 19:37:34 -0400 Subject: [PATCH 1/2] Updated roadmap for a scalable approach to the ecosystem engine --- README.md | 55 +- docs/LILA_PROJECT_STATE.md | 621 +++++++++++++++++ docs/TRAIT_TRANSITION_PLAN.md | 1178 ++++++++++++++++++++++++++++++++ docs/TWO_POOL_NUTRIENT_SPEC.md | 685 +++++++++++++++++++ 4 files changed, 2536 insertions(+), 3 deletions(-) create mode 100755 docs/LILA_PROJECT_STATE.md create mode 100644 docs/TRAIT_TRANSITION_PLAN.md create mode 100644 docs/TWO_POOL_NUTRIENT_SPEC.md diff --git a/README.md b/README.md index 3894508..c1bfb4e 100644 --- a/README.md +++ b/README.md @@ -261,15 +261,64 @@ For the full argument, see ["The Unseen Hand"](https://postcorporate.substack.com/p/the-unseen-hand) on Substack. +## Roadmap + +The current engine encodes ecological knowledge as **per-species rules** — +each species has hand-tuned guard thresholds, hard-coded interaction logic, +and type-specific flow equations. This works for five species. It won't +scale to fifty. + +The next major architecture shift replaces species-specific rules with +**functional traits and allometric scaling**. A species becomes a point +in trait space — body mass, diet type, metabolic class, locomotion mode — +and the engine derives all behavior parameters from established ecological +scaling laws (Kleiber's Law, metabolic theory of ecology). Adding a wolf +means writing a JSON trait vector, not new Python code. The interaction +templates (herbivory, predation, pollination, decomposition) handle the +combinatorics. + +This also makes līlā a compelling **substrate for automated search**. +Recent work on [Automated Search for Artificial Life](https://asal.sakana.ai/) +(ASAL) uses foundation models to discover interesting simulations across +substrates like Boids, Particle Life, and Lenia. Those substrates produce +visually interesting emergence, but the emergence has no ecological +semantics — a Lenia pattern that looks like a cell isn't modeling nutrient +uptake. līlā's trait-based engine would be a substrate where the search +space is biologically meaningful: "what community of trait-defined +organisms produces the most open-ended dynamics on this biome?" is a +question with real ecological content. + +**Near-term:** +- Trait-based species definitions (body mass → derived behavior) +- Two-pool soil nutrient system (fast/slow pools, mineralization, decomposition) +- New species by JSON trait vector only — wolf, songbird, decomposer fungi +- Headless renderer for FM-guided evaluation + +**Medium-term:** +- ASAL substrate protocol (Init/Step/Render interface) +- FM-guided ecosystem search (CMA-ES over trait space, CLIP evaluation) +- Simulation atlas — UMAP of discovered ecosystem configurations +- Godot 3D client with latent-driven skeletal animation + +See [LILA_PROJECT_STATE.md](LILA_PROJECT_STATE.md) for detailed milestones. + ## Contributing līlā is in early alpha. Contributions welcome — especially: -- **New species** — add entity metadata, tune flow equations, submit a PR +- **New species** — today: entity metadata + flow equation tuning. + Soon: a JSON trait vector and the engine derives the rest - **Motor adapters** — train a model, export weights, share it - **Biome presets** — new environments with tuned simulation constants -- **Client work** — the Godot client needs skeleton rigs, shaders, and scene work -- **Bug reports** — the [known issues](docs/lessons_learned.md) are documented, but there are certainly more +- **Ecological modeling** — allometric scaling, interaction templates, + soil nutrient dynamics. If you know metabolic theory of ecology, + there's real work here +- **Client work** — the Godot client needs skeleton rigs, shaders, + and scene work +- **ALife/search integration** — if you've worked with ASAL, Lenia, + or similar frameworks, the substrate protocol is being designed +- **Bug reports** — the [known issues](docs/lessons_learned.md) are + documented, but there are certainly more ## Acknowledgments diff --git a/docs/LILA_PROJECT_STATE.md b/docs/LILA_PROJECT_STATE.md new file mode 100755 index 0000000..7fc902a --- /dev/null +++ b/docs/LILA_PROJECT_STATE.md @@ -0,0 +1,621 @@ +# līlā — Project State (v0.0.1-alpha) + +## Current Status + +**Tagged release: v0.0.1-alpha** — published, repo public on GitHub. + +līlā is a BYOM (Bring Your Own Model) ecosystem simulation engine. Users define a world in JSON — species, biome, soil, water — and the engine grows an autonomous ecosystem from simple rules. The server runs the hybrid automaton (ecology, physics, ML inference); clients render the result via WebSocket at 10 Hz. + +The project thesis — explored in ["The Unseen Hand"](https://postcorporate.substack.com/p/the-unseen-hand) — is that the most impactful AI is small, specialized, and invisible. Tiny ML models guide lifelike motion and behavior; the user never sees inference happening, they just see a world that feels alive. + +The name comes from the Sanskrit concept of [līlā](https://www.embodiedphilosophy.com/what-is-lila/) — the spontaneous, purposeless creative unfolding of reality. There's no win condition. The world plays as itself. + +**Current direction:** The engine is transitioning from hand-crafted per-species rules to a **trait-based architecture** using allometric scaling laws (Metabolic Theory of Ecology). Species become points in trait space; the engine derives all behavior parameters from body mass and functional traits. This also makes līlā a compelling **substrate for automated ALife search** (ASAL framework) — an ecologically-grounded simulation where FM-guided search discovers interesting ecosystem configurations. + +**Copyright:** BioSynthArt Studios LLC. **License:** Apache 2.0. +**Source control:** GitHub at `github.com/hellolifeforms/lila` (org: hellolifeforms). +**CI:** GitHub Actions (`.github/workflows/test.yml`) — pytest + ruff, Python 3.11/3.12. Badge in README. +**Social:** @hellolifeforms on Bluesky, Postcorporate on Substack. + +--- + +## Architecture Overview + +``` +┌─────────────────────────┐ +│ Browser Visualizer │ ← v0.0.1-alpha (shipped, single HTML file) +│ Godot 4.x Client │ ← deferred to Milestone 4 +│ Headless Renderer │ ← Milestone 3 (for ASAL search) +└──────────┬──────────────┘ + │ WebSocket (delta-encoded tick packets) +┌──────────▼──────────────┐ +│ Worker │ ← Shipped. HTTP + WS on single port +│ (one per session) │ Serves viz HTML, streams ticks +└──────────┬──────────────┘ + │ +┌──────────▼──────────────────────────────────────────┐ +│ ecosim (Python package, stdlib only) │ +│ ┌────────────────┐ ┌────────────────────────────┐ │ +│ │ Hybrid Automaton│ │ Trait System (Milestone 2) │ │ +│ │ Flow + Guards │ │ TraitVector + Compiler │ │ +│ ├────────────────┤ │ Allometric Derivations │ │ +│ │ Voxel Manager │ │ Interaction Templates │ │ +│ │ 5 layers (M2) │ ├────────────────────────────┤ │ +│ ├────────────────┤ │ BYOM Adapters │ │ +│ │ Water System │ │ mlp/static/random │ │ +│ │ Dynamic levels │ ├────────────────────────────┤ │ +│ ├────────────────┤ │ World Randomizer │ │ +│ │ Two-Pool Soil │ │ D4 transforms │ │ +│ │ Fast/Slow (M2) │ └────────────────────────────┘ │ +│ └────────────────┘ │ +└──────────────────────────────────────────────────────┘ + │ +┌──────────▼──────────────────────────────────────────┐ +│ search/ (Milestone 3, separate package) │ +│ ASAL Substrate Protocol (Init/Step/Render) │ +│ FM Evaluator (CLIP/DINOv2) │ +│ CMA-ES Search (target, open-ended, illumination) │ +│ Simulation Atlas Visualization │ +└──────────────────────────────────────────────────────┘ +``` + +--- + +## Repository Structure + +``` +lila/ +├── server/ +│ ├── pyproject.toml # lila-ecosim package, stdlib only +│ ├── .python-version # uv Python version (3.12) +│ ├── uv.lock # deterministic dependency lockfile +│ ├── ecosim/ # core simulation library +│ │ ├── __init__.py +│ │ ├── engine.py # hybrid automaton (EcosystemEngine) +│ │ ├── entities.py # entity schemas, init_entity() +│ │ ├── biome.py # biome presets → BiomeConfig +│ │ ├── voxel_manager.py # sparse 3D grid, delta tracking +│ │ ├── model_adapter.py # MotorAdapter protocol, ContextSpec +│ │ ├── worker.py # async WS tick loop + HTTP viz server +│ │ ├── traits.py # [M2] TraitVector, allometric derivations +│ │ ├── interactions.py # [M2] InteractionTemplate grammar +│ │ ├── trait_compiler.py # [M2] TraitCompiler: traits → engine params +│ │ └── adapters/ +│ │ ├── __init__.py # create_adapter() factory +│ │ ├── mlp.py # reference MLP (~500 params, pure Python) +│ │ ├── static.py # hand-tuned latent per state +│ │ └── random.py # random latents for testing +│ ├── examples/ +│ │ ├── demo_world.json # temperate meadow with randomization +│ │ └── temperate_meadow_8sp.json # [M3] 8-species trait-based world +│ ├── tests/ +│ │ ├── smoke_test.py # 50-tick integration test +│ │ ├── test_ecosim.py # unit tests (12 tests) +│ │ ├── test_traits.py # [M2] allometric derivation tests +│ │ ├── test_nutrients.py # [M2] two-pool nutrient flow tests +│ │ └── test_regression.py # [M2] 2000-tick baseline comparison +│ └── weights/ +│ └── (motion_v0.json) # placeholder for trained weights +│ +├── client/ +│ ├── browser/ +│ │ └── index.html # canvas-based 2D ecosystem visualizer +│ └── godot/ # [M4] Godot 4.x client +│ +├── search/ # [M3] ASAL substrate + search +│ ├── pyproject.toml # deps: torch, clip, cma, umap, pillow +│ ├── substrate.py # ALifeSubstrate protocol + LilaSubstrate +│ ├── renderer.py # headless PIL renderer (256×256) +│ ├── evaluator.py # FM evaluation (CLIP, DINOv2) +│ ├── search.py # supervised, open-ended, illumination +│ ├── theta.py # θ parameterization (EcoRates/Topology/Adapt) +│ ├── atlas.py # simulation atlas UMAP visualization +│ ├── constraints.py # physical plausibility validation +│ └── examples/ +│ ├── search_target.py +│ ├── search_openended.py +│ └── search_illuminate.py +│ +├── training/ # ML training pipeline (not core) +│ ├── pyproject.toml +│ ├── data/ +│ ├── scripts/ +│ └── notebooks/ +│ +├── deploy/ +│ └── compose/ # ← primary getting-started path +│ ├── docker-compose.yml +│ ├── Dockerfile.worker +│ └── README.md +│ +├── docs/ +│ ├── model_adapter_spec.md # BYOM guide — how to build adapters +│ ├── data_contract.md # v0.2 protocol spec +│ ├── architecture.md +│ ├── species_spec.md # 0.1-alpha species + skeleton rigs +│ ├── lessons_learned.md # debugging war stories +│ ├── trait_species_guide.md # [M2] how biologists add species +│ └── asal_substrate_guide.md # [M3] how to use līlā with ASAL +│ +├── .github/workflows/ +│ └── test.yml # CI: pytest + ruff, Python 3.11/3.12 +│ +├── DEVELOPING.md # uv workflow, dev setup +├── LICENSE # Apache 2.0 +├── README.md # project overview, quick start, roadmap +├── TRAIT_TRANSITION_PLAN.md # detailed Phase 1-3 implementation plan +└── TWO_POOL_NUTRIENT_SPEC.md # two-pool soil nutrient spec +``` + +Items marked `[M2]`, `[M3]`, `[M4]` indicate which milestone introduces them. + +--- + +## What Shipped in v0.0.1-alpha + +### Core Engine (ecosim) + +**Hybrid automaton** — seven-phase tick loop: flow → interactions → guards → voxel effects → water replenishment → soil evaporation → motor inference → removals → spawns. + +**Entity types:** ANIMAL, BIRD, INSECT, PLANT, TREE, MICROORGANISM. Each has type-specific flow equations, guard conditions with hysteresis, and valid state sets. + +**Behavioral intelligence** (no ML required): +- Purposeful movement — entities seek food, water, flowers, and mates based on state +- Grazing chain — deer seek nearest grass, fall back to wildflowers when grass is gone +- Pollination chain — butterflies seek FRUITING wildflowers, linger 1.5–3s, then seek next bloom. Flower cooldown prevents re-pollination +- Water seeking — thirsty animals walk to nearest pond, drink, drain the source +- Mate seeking — grid-wide search when reproductive drive is high, proximity check for actual reproduction +- Flee response — prey flees from carnivores with clamped escape targets + +**Guard hysteresis bands:** +- Hydration: enter DRINKING at 0.2, exit at 0.6 +- Energy: enter RESTING at 0.2, exit at 0.5 (animals) / 0.15→0.4 (insects) +- Hunger: enter FORAGING at 0.3, exit at 0.15 +- Reproduction: drive > 0.8 AND mate within sensory range (animals) / > 0.7 (insects) + +**Plant ecology:** +- Vegetative spreading — grass (range 2, frequent) and flowers (range 3.5, less frequent) with soil checks, density limits, and parent resource cost +- Dormancy — plants go DORMANT at health 0 instead of dying. Roots persist. Recovery when soil moisture > 0.25 and nutrients > 0.15 +- Dormancy timeout — 2000 ticks without recovery → permanent death +- FRUITING threshold — growth ≥ 0.5 and health > 0.4 + +**Water system:** +- Dynamic water levels — each source tracks `water_level` (0–1), controls effective radius +- Evaporation drains water sources, groundwater replenishes, drinking animals deplete +- Background soil evaporation across the full grid +- Dried-up sources (< 5%) skipped by pathfinding + +**Ecosystem collapse:** +- Tree collapse pressure when support_count (non-tree, non-insect, non-dormant) ≤ 2 +- Generational decline — children inherit parent stress (hunger × 0.3, energy × 0.9, colony_health × 0.9) +- Reproduction costs parent colony_health (insects) +- Starvation acceleration — colony_health drain scales with hunger level + +**Rain system:** +- `apply_rain(intensity)` — boosts soil moisture (+0.24), nutrients (+0.024), water source levels (+0.32), plant hydration (+0.16), plant health (+0.08), animal hydration (+0.08) +- Suppresses soil evaporation and plant evapotranspiration for 80 ticks +- Triggered via WebSocket control message `{"type": "rain", "intensity": 0.8}` + +**Rate multipliers** (configurable per world): +- `consumption`, `hunger`, `thirst`, `growth`, `reproduction`, `water_replenishment` +- All default to 1.0. Stress testing via JSON, no code changes. + +**World randomization** (JSON-driven): +- D4 symmetry transforms (4 rotations × 2 flips = 8 orientations) +- Position jitter (configurable range) +- Extra grass (0–4) and wildflower (0–2) spawns +- Water source position and radius variation +- State variable jitter (±5%) +- Plants pushed out of water sources post-randomization +- Opt-in: omit `"randomize"` key for exact JSON positions + +**BYOM adapter system:** +- `MotorAdapter` protocol — `context_spec()` + `infer()` +- `ContextSpec` with typed fields, source routing, normalization +- Type-specific specs via `context_spec_for(entity_type)` +- Three built-in: `mlp` (500 params, Xavier init), `static` (per-state latents), `random` (testing) +- `create_adapter()` factory + +**Voxel manager:** +- Sparse 3D grid, four layers: nutrients, moisture, temperature, organic_matter +- Threshold-gated dirty tracking for delta packets +- `initialize_from_soil` — correctly initializes all three computed layers (break bug fixed) + +### Browser Visualizer + +- Canvas-based 2D renderer at 60fps with 10Hz tick interpolation +- Moisture heatmap (subtle teal→amber gradient) +- Water sources with radial gradient, animated ripples, dynamic radius/opacity tracking water level +- Deer as directional triangles with state labels and motion latent halos +- Butterflies with animated wing flaps and pollination glows +- Oaks with canopy radius shadows +- Grass clusters scaling with growth, tinting with hydration +- Wildflowers pulsing golden during FRUITING +- Dormant plants as faded brown root markers +- Event particles (grazing=green, pollination=gold, death=dark) +- Stats panel (tick, entities, events, fps) + scrolling event log +- All state transitions logged from tick 1 +- Session started message on connect +- Entity type inference from ID prefixes +- **☔ Rain button** — sends rainfall control message, visual feedback +- **⏺ Record button** — 10-second canvas capture via MediaRecorder, codec fallback (VP9→VP8→WebM→MP4), auto-download +- Legend with all entity types + water + +### Worker + +- Combined HTTP + WebSocket on single port +- `process_request` compatible with websockets 13+ (`connection, request` → `Response` objects) +- `SimulationSession` with pause/resume/stop/rain controls +- Control message dispatch table +- Drift-compensated tick loop +- File resolution for viz and world (repo, Docker, env vars) +- CLI headless mode for benchmarking + +### Infrastructure + +- **Docker Compose** — single command: `docker compose up --build` +- **Dockerfile** — python:3.12-slim, `pip install ".[worker]"` +- **GitHub CI** — pytest (12 tests) + ruff lint, Python 3.11/3.12 +- **uv workflow** — `uv sync` for local dev, deterministic lockfile +- **pyproject.toml** — setuptools backend (Docker-compatible), optional dep groups (worker, gateway, dev, all), ruff/pytest/pyright config, script entry points + +### Documentation + +- **README.md** — positioning (engine, not game), quick start, architecture diagram, BYOM examples, species table, interaction chains, roadmap, controls, contributing note, CI badge +- **DEVELOPING.md** — uv workflow, pip fallback, dependency groups, project layout +- **docs/model_adapter_spec.md** — protocol, context spec, state codes, full worked example, type-specific specs, training/weights, built-in adapter comparison +- **docs/lessons_learned.md** — debugging war stories from the build session +- **deploy/compose/README.md** — Docker quick start with controls + +--- + +## 0.1-Alpha Species Set + +Five species, two skeletons, five interaction chains: + +| Species | Type | Skeleton | Role | +|--------------|--------|------------------|---------------------------------------| +| Deer | ANIMAL | quadruped_medium | Grazer, seeks grass → flowers → water → mates | +| Butterfly | INSECT | insect_wing | Pollinator, seeks flowers → water fallback | +| Oak | TREE | none | Structure, shade, collapse indicator | +| Meadow Grass | PLANT | none | Ground cover, spreads via runners | +| Wildflower | PLANT | none | Bloom cycle, pollination target | + +**Interaction chains:** +1. **Grazing** — deer hunger → forages nearest grass → consumption → grass spreads if soil is moist +2. **Pollination** — wildflower FRUITING → butterfly flies to it → pollinates → lingers → seeks next +3. **Water** — thirst → walk to pond → drink → pond level drops → soil dries +4. **Stress cascade** — overgrazing → flowers consumed → butterflies lose food → cluster at ponds → ponds dry → collapse +5. **Dormancy & recovery** — plants die to roots → rain → soil moisture rises → roots revive → flowers bloom → butterflies return + +--- + +## Completed Milestones + +### Milestone 0 — Engine Foundation ✅ + +1. ✅ WebSocket `process_request` fix for websockets 13+ +2. ✅ Smoke test imports verified (`ecosim.*`) +3. ✅ Voxel `initialize_from_soil` break bug fixed +4. ✅ Docker build verified end-to-end +5. ✅ Dev requirements (uv + pyproject.toml + lockfile) + +### Milestone 1 — v0.0.1-alpha Release ✅ + +6. ✅ README with positioning, quick start, controls, CI badge +7. ✅ `docs/model_adapter_spec.md` — BYOM guide +8. ✅ GitHub CI (pytest + ruff, Python 3.11/3.12) +9. ✅ Tagged v0.0.1-alpha, repo public + +### Bonus — Simulation Tuning ✅ + +10. ✅ Purposeful movement (food/flower/water/mate seeking) +11. ✅ Water sources with dynamic levels and drought +12. ✅ Plant dormancy and rain-triggered recovery +13. ✅ Rain control (button + WebSocket + engine) +14. ✅ Record button for GIF/video capture +15. ✅ Butterfly pollination lifecycle (linger, cooldown, skip dormant) +16. ✅ Generational decline and reproduction costs +17. ✅ Ecosystem collapse cascade +18. ✅ Rate multipliers for stress testing +19. ✅ World randomization (D4 transforms, jitter, extra plants) +20. ✅ `docs/lessons_learned.md` + +--- + +## Pending — Milestone 2: Trait-Based Architecture + Two-Pool Nutrients + +**Goal:** Replace per-species hard-coded rules with functional trait derivations. Split the single nutrient layer into fast/slow pools with mineralization. All existing tests must still pass. The hybrid automaton tick loop does not change. + +**Motivation:** The current engine encodes ecological knowledge as per-species rules. Every new species requires hand-tuned guard thresholds, interaction logic, and flow equations — O(n²) design effort. The trait-based approach encodes knowledge as allometric scaling laws and interaction templates, making new species a JSON definition rather than new code. This is informed by the Madingley General Ecosystem Model (Harfoot et al. 2014) and the Metabolic Theory of Ecology (Brown et al. 2004). + +**Reference documents:** `TRAIT_TRANSITION_PLAN.md` (Phase 1), `TWO_POOL_NUTRIENT_SPEC.md` + +### Step 2.1 — Audit Current Hard-Coded Parameters +Extract every species-specific constant from `engine.py`, `entities.py`, and `biome.py` into a reference table. This is the calibration target for the derivation layer. + +### Step 2.2 — Define TraitVector Schema +Dataclass capturing functional traits: body_mass_kg, diet_type, diet_breadth, locomotion, thermoregulation, reproductive_strategy, thermal_range, drought_tolerance, sensory_range_multiplier, spread_mode, root_persistence, etc. A species is a point in trait space. + +### Step 2.3 — Allometric Derivation Functions +Pure functions in `ecosim/traits.py` (stdlib only): `TraitVector → DerivedParams`. Core equations: +- Metabolic rate: BMR = B₀ × M^0.75 (endotherm) / M^0.69 (ectotherm) — Kleiber 1932, Gillooly 2001 +- Cruising speed: v = v₀ × M^0.25 (terrestrial) / M^0.17 (insect flight) — Peters 1983, Dudley 2000 +- Sensory range: ∝ M^0.5 — derived from home range scaling (McNab 1963) +- Flow rates (hunger, thirst, energy): proportional to metabolic rate +- Guard thresholds: hysteresis bands scaled by normalized metabolic rate +- Consumption rate: proportional to metabolic rate + +Calibration constants chosen so that deer traits (80 kg, endotherm, quadruped) produce values matching the current hard-coded parameters within 5%. + +### Step 2.4 — Interaction Template Grammar +Six parameterized templates replace per-species-pair code: +- **Herbivory** — actor diet_breadth matches target resource_tags +- **Predation** — actor diet_breadth matches target functional_group, body mass ratio constraints (0.1–2× for mammalian, 1–1000× for insectivory) +- **Pollination** — actor floral_affinity matches target pollination_syndrome, target must be FRUITING +- **Decomposition** — actor diet_type "decomposer", targets voxel organic_matter layer (unique: interacts with voxels, not entities) +- **Competition** — implicit via shared resource depletion, no explicit template +- **Water access** — all mobile non-autotroph entities, thirst derived from metabolic rate + +### Step 2.5 — TraitCompiler +Runs once at world initialization. Takes list of TraitVectors + BiomeConfig, produces: per-entity DerivedParams, sparse interaction matrix, resource tag registry. + +### Step 2.6 — Two-Pool Nutrient Refactor +Split `nutrients` voxel layer into `nutrients_fast` and `nutrients_slow` (voxel layers 4 → 5): +- **nutrients_fast** (plant-available): quick turnover, depleted by plant growth, refilled by rain and dissolution from slow pool +- **nutrients_slow** (mineralized reserve): long-term soil health, fed by decomposition of organic_matter, slowly dissolves into fast pool +- **organic_matter** (existing): dead entity biomass deposited here on death, converted to slow nutrients via mineralization + +New per-tick fluxes in voxel effects phase: +- Mineralization: organic_matter → nutrients_slow (rate 0.002/tick, accelerated by decomposer entities in M3) +- Dissolution: nutrients_slow → nutrients_fast (rate 0.005/tick) +- Leaching: nutrients_fast drains slowly (rate 0.001/tick) + +Updated touchpoints: rain split (0.020 fast + 0.004 slow), dormancy recovery uses weighted effective nutrients (fast + slow × 0.3), plant spreading checks fast pool only, entity death deposits biomass to organic_matter layer. + +Three new rate multipliers: `mineralization`, `dissolution`, `nutrient_leaching` (all default 1.0). + +### Step 2.7 — Refactor engine.py +Replace `if entity["type"] ==` branches with DerivedParams lookups. Seven tick phases refactored one at a time with tests after each: flow → guards → interactions → movement → spawning → voxel effects → motor inference. + +### Step 2.8 — Write Trait Vectors for Existing Species +Express deer, butterfly, oak, meadow grass, wildflower as TraitVectors in JSON. When compiled, must produce parameters matching the Step 2.1 audit within 5%. + +### Step 2.9 — Calibration & Regression Testing +- Compare DerivedParams output against audit table +- Run full test suite (12 existing + new trait/nutrient tests) +- 2000-tick regression: population curves, state transitions, event counts within ±10–15% of baseline +- Backward compatibility: worlds without `species_definitions` key fall back to legacy code paths + +### Milestone 2 Deliverables +- `ecosim/traits.py` — TraitVector, DerivedParams, allometric derivation functions +- `ecosim/interactions.py` — InteractionTemplate base + 4 concrete templates +- `ecosim/trait_compiler.py` — TraitCompiler class +- Refactored `engine.py` — reads from DerivedParams +- Refactored `voxel_manager.py` — 5 layers, inter-pool fluxes, death deposits +- Updated `examples/demo_world.json` — species_definitions + 3 new rate multipliers +- `tests/test_traits.py`, `tests/test_nutrients.py`, `tests/test_regression.py` +- `docs/trait_species_guide.md` — how to add species via trait vectors + +**New files: 4. Modified files: 4. No new external dependencies.** + +--- + +## Pending — Milestone 3: New Species + ASAL Substrate + +**Goal:** Validate the trait architecture by adding three species with zero engine code. Build the ASAL substrate protocol and FM-guided search pipeline. + +**Reference documents:** `TRAIT_TRANSITION_PLAN.md` (Phases 2–3) + +### New Species (Trait Vectors Only) + +**Wolf** — completes the food chain: grass → deer → wolf. diet_type: carnivore, diet_breadth: ["herbivore"], body_mass_kg: 40. Predation template matches wolf→deer automatically. Deer flee response triggers from carnivore detection. Expected emergent dynamic: Lotka-Volterra oscillations, trophic cascade (wolves reduce deer → grass recovers). + +**Songbird** — new trophic niche: insectivore + frugivore. diet_breadth: ["pollinator", "forb:fruiting"], body_mass_kg: 0.025. Tests insectivory mass-ratio window (predator >> prey, unlike mammalian predation). Expected: songbird-butterfly predation reduces pollination pressure. + +**Mushroom** — closes the nutrient loop. diet_type: decomposer, targets organic_matter voxel layer. r_selected (clutch_size 5, fast generation). Accelerates mineralization rate locally. Expected: measurably faster soil recovery near decomposer clusters, 3–4× reduction in ecosystem recovery time after collapse events. + +### Emergent Dynamics Validation +With 8 species, run 10,000-tick simulations documenting which interaction chains emerge without being coded: +- Wolf-deer predation with population oscillations +- Trophic cascade: wolves reduce deer → grass recovers → wildflowers bloom +- Songbird-butterfly predation reducing pollination rates +- Mushroom decomposition accelerating soil recovery after death events +- Cross-trophic competition: songbirds and butterflies competing for fruiting flowers +- Thermal range exclusions in extreme biome settings + +### ASAL Substrate Protocol + +Formalize līlā as an ASAL-compatible substrate with the three-function interface: +- `Init(θ)` — parameterized world initialization from trait vectors + biome config +- `Step(θ)` — one tick of the hybrid automaton +- `Render(θ)` — headless 256×256 RGB image (PIL, no browser) + +### θ Parameterization (Three Variants) + +**EcoRates** (~15 dimensions) — rate multipliers + biome parameters. Answers: "what metabolic tuning produces the most interesting dynamics?" + +**EcoTopology** (~50–80 dimensions) — rates + species composition + spatial layout + water sources. Answers: "what ecosystem configurations produce the most diverse dynamics?" + +**EcoAdapt** (~550–600 dimensions) — topology + MLP adapter weights. Answers: "what learned behaviors, in what ecological contexts, produce the most lifelike dynamics?" + +### FM Evaluation Pipeline +- CLIP (ViT-B/32) and DINOv2 for embedding rendered simulation frames +- Three ASAL search modes: + - **Supervised target** — "find parameters matching these ecological prompts" (e.g., "thriving meadow" → "overgrazing" → "rain recovery") + - **Open-endedness** — maximize trajectory novelty in FM embedding space over long rollouts + - **Illumination** — discover maximally diverse set of ecosystem configurations +- CMA-ES optimization (gradient-free, handles 600-dim search spaces) +- Physical plausibility constraints (square-cube law, thermal homeostasis limits, trophic sanity) + +### Simulation Atlas +UMAP projection of all discovered ecosystems with rendered thumbnails. "The atlas of possible ecologies" — what does the space of all possible temperate meadows look like? + +### Milestone 3 Deliverables +- Three new species as JSON trait vectors (zero engine code) +- Updated interaction templates with parameterized mass-ratio windows +- `examples/temperate_meadow_8sp.json` — 8-species trait-based world +- Emergent dynamics validation report +- `search/` directory with own pyproject.toml +- `search/substrate.py`, `renderer.py`, `evaluator.py`, `search.py`, `theta.py`, `atlas.py`, `constraints.py` +- `docs/asal_substrate_guide.md` + +--- + +## Pending — Milestone 4: Godot Client + Trained Motion Model + +**Goal:** 3D visualization of trait-based ecosystems with latent-driven skeletal animation. Built against the stable trait-based engine from Milestone 2, not the current hand-coded species. + +**Why deferred:** The Godot client should be built against the trait-based engine, not the current per-species architecture — building it now means refactoring it after the trait transition. The engine work (Milestones 2–3) generates more interesting content (trophic cascades, FM-discovered ecosystems) than the Godot client would at this stage. The browser visualizer is sufficient for validating all near-term work. + +### 3D Assets + +1. **Blender models** — low-poly faceted deer (200–400 faces, quadruped_medium rig), butterfly (<50 faces, insect_wing rig), oak tree, grass clump, wildflower. Flat shading, no smoothing. Asset pipeline documented in `LILA_ASSET_PIPELINE_CONTEXT.md`. + +### Godot Project + +2. **Project scaffolding** — project.godot, autoloads (session_manager, skeleton_registry, event_bus), base_entity.gd. +3. **WebSocket + tick receiver** — connect to worker, parse packets (5 voxel layers), dispatch to subsystems. +4. **Position interpolation** — smooth between 10Hz ticks at 60fps render. +5. **Motion retargeter** — latent vector → bone transforms via per-bone weight matrix. `R_final(bone) = R_base + Σ(latent[i] × W[bone][i])`. This is the thesis demo. +6. **Voxel renderer** — ImageTexture3D for moisture layer, ground plane shader. Optional soil health overlay (nutrients_slow as subtle gradient). +7. **Event particles** — CONSUMPTION (leaf burst), POLLINATION (golden trail), RAIN (droplets). +8. **Water rendering** — shader-based pond with dynamic radius from tick packets. + +### Trained Motion Model + +9. **Motion data acquisition** — source animation clips for deer locomotion (walk, trot, graze, drink, rest) and butterfly flight (cruise, hover, land). +10. **Feature extraction** — `training/scripts/extract_features.py`: animation clips → context→motion training pairs. +11. **Training** — `training/scripts/train.py`: PyTorch training loop targeting the MLP architecture (10→16→12→8→4). Context spec from trait-based engine (richer context vectors than v0.0.1). +12. **Evaluation** — `training/scripts/evaluate.py`: latent space visualization, reconstruction quality. +13. **Weight export** — `training/scripts/export_weights.py`: PyTorch → ecosim JSON format. +14. **Integration** — load trained weights in demo world, compare against static/random adapters. + +### Server + +15. **Minimal gateway** — FastAPI, accepts WS connections, proxies to worker. `SessionOrchestrator` protocol + `LocalOrchestrator`. Proves multi-session architecture. + +--- + +## Future (v0.2+) + +### Ecosystem Richness +- Reproduction with genetic variation (trait inheritance with mutation) +- Seasonal cycles — temperature/rainfall oscillations driving phenology +- Multiple biome presets (desert, arctic, tropical) with biome-specific trait constraints +- L-systems for procedural plant clusters and terrain objects + +### Engine Scaling +- Behavior-level adapter (ML-influenced guard conditions via trait context) +- Narrative-level adapter (ecosystem-scale intelligence, event injection) +- Bounded fields for dense actor clusters (insect swarms, grass patches) +- Spatial hash for O(1) neighbor queries (current brute-force is O(n²)) +- Tick-rate/bandwidth optimization + +### ASAL Extensions +- Video-language FM evaluation (temporal dynamics without frame sampling) +- 3D FM evaluation (via Godot renderer output) +- Substrate contribution to ASAL codebase (JAX port of core dynamics, or Python bridge) +- Cross-substrate comparison: līlā vs Boids vs Lenia on same ASAL search objectives +- Automated trait vector discovery — FM-guided search for novel functional groups + +### World Building +- Scene editor UI — click to place entities, drag sliders for rates and traits +- Trait database import from PanTHERIA (mammals), TRY (plants), EltonTraits (diet/foraging) + +### Deployment +- Cloud-agnostic orchestration (ECS/K8s/Fly adapters) +- Multi-session gateway with Redis session state +- Spectator mode — read-only WebSocket for observers + +--- + +## Key Gotchas (see docs/lessons_learned.md for details) + +1. WebSocket `process_request` signature varies between websockets versions — check return types first. +2. `elif` chains in Python are exclusive — combine guard conditions with `and` to avoid blocking downstream branches. +3. Entities must seek targets purposefully, not wander randomly. +4. Pollination needs cooldowns on flowers, not memory on insects. +5. Children must inherit parent stress or populations become immortal. +6. Reproductive drive needs a dead zone between build and decay conditions. +7. Rain must work at multiple levels (soil, plants, water sources, evaporation suppression) or it's too weak. +8. Plants should go dormant, not die — root persistence enables recovery. +9. World randomization must be JSON-driven and opt-in. +10. Don't use hatchling in Docker — setuptools is pre-installed everywhere. +11. **(New)** Allometric scaling laws are well-validated for animals but weaker for plants. Keep plant-specific traits (spread_range, spread_mode, root_persistence) as explicit fields rather than deriving from body mass. +12. **(New)** The two-pool nutrient split must preserve `initialize_from_soil` correctness — all five layers initialized. The original break bug (skipped layers 2–3) was from an `elif` chain; verify with 5 layers. + +--- + +## Design Decisions (Locked) + +- **BYOM adapter architecture.** Engine accepts adapters via dict. Three built-in (mlp, static, random). Custom adapters implement `MotorAdapter` protocol. +- **Model level hierarchy.** Motor (implemented), Behavior (reserved), Narrative (reserved). +- **ecosim is stdlib-only.** Zero external dependencies in the core package. Trait system, interactions, and trait compiler use only stdlib math + dataclasses. +- **Docker Compose is the primary path.** Clone, compose up, open browser. +- **Skeleton mapping is client-side.** Server sends `skeleton_id` + motion latent. Client owns rigs. +- **Voxel layers: 5.** nutrients_fast, nutrients_slow, moisture, temperature, organic_matter. Updated from 4 → 5 per two-pool nutrient decision. +- **Motion latent dimensions: 4.** Expandable later. +- **Grid: 32³ default, cell_size 1.0.** +- **Tick rate: ~100ms (10 Hz).** +- **Randomization is opt-in via JSON.** No `"randomize"` key = deterministic positions. +- **Plants go dormant, not dead.** Root persistence is ecologically accurate and enables recovery narratives. +- **Solo creative project.** Contributions welcome, creative direction maintained by author. +- **(New) Trait-based species architecture.** Species defined as functional trait vectors in JSON. Engine derives behavior parameters from allometric scaling laws. Interaction templates handle combinatorics. Per-species engine code is a legacy path. +- **(New) Two-pool nutrient system.** Fast pool (plant-available, quick turnover) + slow pool (mineralized reserve, long-term soil health). Mineralization, dissolution, and leaching fluxes run per tick. Decomposer entities accelerate mineralization locally. +- **(New) ASAL substrate compatibility.** Engine exposes Init/Step/Render protocol for FM-guided search over trait space. search/ package has its own dependencies (torch, clip, cma); ecosim core stays clean. +- **(New) Engine-first priority.** Godot client deferred until trait-based engine is stable. Browser visualizer sufficient for validating trait system, two-pool nutrients, and ASAL search. + +--- + +## Planning Documents + +- **TRAIT_TRANSITION_PLAN.md** — Detailed implementation plan for the Phase 1–3 transition from hand-crafted rules to trait-based architecture with ASAL substrate integration. Includes TraitVector schema, allometric derivation functions, interaction template grammar, TraitCompiler design, engine refactor sequence, calibration strategy, and ASAL search loop implementations. + +- **TWO_POOL_NUTRIENT_SPEC.md** — Implementation spec for the two-pool nutrient system. Covers pool dynamics equations, rate constants, voxel manager changes (4→5 layers), every engine touchpoint that reads/writes nutrients, rain split ratios, dormancy recovery update, death→organic_matter deposits, timescale analysis for three recovery scenarios, test plan, and backward compatibility. + +- **LILA_ASSET_PIPELINE_CONTEXT.md** — AI-generated 3D asset pipeline research. Covers Flux.1 Schnell → BiRefNet → Hunyuan 3D v2.1 pipeline, deer mesh prototype results, rigging plan. Relevant to Milestone 4 (Godot client). + +--- + +## Key References + +**Ecological theory:** +- Kleiber, M. (1932). Body size and metabolism. *Hilgardia*. — BMR = B₀ × M^0.75 +- Brown, J.H. et al. (2004). Toward a metabolic theory of ecology. *Ecology*. — Metabolic Theory of Ecology +- Gillooly, J.F. et al. (2001). Effects of size and temperature on metabolic rate. *Science*. — Ectotherm scaling exponent 0.69 +- Peters, R.H. (1983). *The Ecological Implications of Body Size*. Cambridge. — Movement speed scaling +- Harfoot, M.B.J. et al. (2014). Emergent global patterns from a mechanistic general ecosystem model. *PLoS Biology*. — The Madingley Model + +**ALife/search:** +- Kumar, A. et al. (2024). Automating the Search for Artificial Life with Foundation Models. *Artificial Life* (MIT Press). — ASAL framework: FM-guided search across ALife substrates +- Project page: https://asal.sakana.ai/ — Code: https://github.com/SakanaAI/asal + +**Allometric scaling:** +- Hirt, M.R. et al. (2017). A general scaling law reveals why the largest animals are not the fastest. *Nature Ecology & Evolution*. — Hump-shaped speed-mass relationship +- McNab, B.K. (1963). Bioenergetics and the determination of home range size. *American Naturalist*. — Home range ∝ M^0.75 +- Dudley, R. (2000). *The Biomechanics of Insect Flight*. Princeton. — Insect flight speed scaling + +--- + +## Project Links + +- **GitHub:** https://github.com/hellolifeforms/lila +- **Substack essay:** https://postcorporate.substack.com/p/the-unseen-hand +- **Series:** "The Geometry Beneath" on postcorporate.substack.com +- **Bluesky:** https://bsky.app/profile/hellolifeforms.bsky.social +- **līlā concept:** https://www.embodiedphilosophy.com/what-is-lila/ +- **ASAL:** https://asal.sakana.ai/ +- **Madingley Model:** https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1001841 + +--- + +## Performance + +Worker benchmarks (23 entities, 32³ grid): +- Step time: 0.60–0.83ms per tick +- Throughput: 1200–1680 Hz (120× headroom above 10Hz target) +- Browser viz: 60fps with tick interpolation +- Docker image: ~50MB (python:3.12-slim + websockets) + +**Performance note for Milestone 2:** The trait compiler runs once at init, not per tick. Per-tick lookups into DerivedParams are dict access — O(1). Two-pool nutrient fluxes add three multiply-and-clamp operations per active voxel cell per tick. Expected impact: negligible relative to the ~1ms step time budget. diff --git a/docs/TRAIT_TRANSITION_PLAN.md b/docs/TRAIT_TRANSITION_PLAN.md new file mode 100644 index 0000000..037c0d0 --- /dev/null +++ b/docs/TRAIT_TRANSITION_PLAN.md @@ -0,0 +1,1178 @@ +# līlā — Trait-Based Architecture Transition Plan + +> From hand-crafted species rules to allometrically-derived ecological dynamics, +> with an ASAL-compatible substrate protocol for FM-guided ecosystem search. + +--- + +## Why This Transition + +The current engine encodes ecological knowledge as **per-species rules**. Every species +has hand-tuned guard thresholds, hard-coded interaction logic, and type-specific flow +equations. This worked for five species and twenty tuning milestones, but the cost of +a sixth species is not additive — it's multiplicative, because every new species +potentially interacts with every existing one. The engine needs O(n²) *design effort* +per species, which is intractable. + +The solution, validated by the Madingley General Ecosystem Model and the Metabolic +Theory of Ecology, is to encode knowledge as **functional traits** and **derive** the +rules from well-established allometric scaling laws. A species becomes a point in +trait space; the engine computes its behavior from first principles. + +This transition also enables ASAL-style automated search (Akarsh Kumar et al., 2024). +Once the engine accepts parameterized trait vectors, the search space is biologically +meaningful — searching over body masses, diet types, and thermal tolerances, not +arbitrary rate multipliers. Foundation models evaluate rendered simulation output for +open-endedness, target phenomena, or diversity, and evolutionary search optimizes the +trait configurations that produce the most interesting ecological dynamics. + +--- + +## Phase 1 — Trait Derivation Layer (Refactor, No Rewrite) + +**Goal:** Express the existing five species as trait vectors. Build a derivation layer +that produces the *same* engine parameters currently hard-coded. All 12 existing tests +must still pass. The hybrid automaton tick loop does not change. + +### Step 1.1 — Audit Current Hard-Coded Parameters + +Walk through `engine.py`, `entities.py`, and `biome.py` and extract every species-specific +constant into a reference table. This is the target output of the derivation layer. + +**Per-entity-type parameters to extract:** + +| Parameter | Current Location | Example (Deer) | Example (Butterfly) | +|-----------|-----------------|----------------|---------------------| +| Guard: hunger_enter | engine.py guards | 0.3 | 0.3 | +| Guard: hunger_exit | engine.py guards | 0.15 | 0.15 | +| Guard: hydration_enter | engine.py guards | 0.2 | — | +| Guard: hydration_exit | engine.py guards | 0.6 | — | +| Guard: energy_enter | engine.py guards | 0.2 | 0.15 | +| Guard: energy_exit | engine.py guards | 0.5 | 0.4 | +| Guard: repro_drive_threshold | engine.py guards | 0.8 | 0.7 | +| Flow: hunger_rate | engine.py flow | per-tick Δ | per-tick Δ | +| Flow: thirst_rate | engine.py flow | per-tick Δ | — | +| Flow: energy_decay | engine.py flow | per-tick Δ | per-tick Δ | +| Flow: repro_drive_build | engine.py flow | per-tick Δ | per-tick Δ | +| Movement: speed | engine.py movement | units/tick | units/tick | +| Movement: sensory_range | engine.py movement | grid units | grid units | +| Interaction: consumption_rate | engine.py interactions | per-event Δ | — | +| Interaction: diet_targets | engine.py interactions | [grass, wildflower] | [wildflower:FRUITING] | +| Interaction: flee_from | engine.py interactions | [carnivore types] | — | +| Plant: spread_range | engine.py spawns | 2.0 (grass) | 3.5 (flower) | +| Plant: spread_frequency | engine.py spawns | high (grass) | low (flower) | +| Plant: dormancy_recovery_moisture | engine.py guards | 0.25 | 0.25 | +| Plant: dormancy_recovery_nutrients | engine.py guards | 0.15 | 0.15 | +| Plant: fruiting_growth_threshold | engine.py guards | 0.5 | 0.5 | + +**Deliverable:** A Python dict or dataclass per species containing every parameter the +engine currently reads from `if entity_type ==` branches. This is the "ground truth" +the derivation layer must reproduce. + +### Step 1.2 — Define the Trait Schema + +Each species is described by a trait vector in JSON. The traits are chosen to be +(a) biologically meaningful, (b) measurable from real-world databases, and +(c) sufficient to derive all parameters from Step 1.1. + +```python +@dataclass +class TraitVector: + """Functional traits for a species. All derivations flow from these.""" + + # === Identity === + species_id: str # "deer", "butterfly", "oak", etc. + functional_group: str # "herbivore", "pollinator", "producer", "decomposer" + entity_class: str # maps to current EntityType: ANIMAL, INSECT, PLANT, TREE + + # === Body Plan === + body_mass_kg: float # THE key trait. Most rates derive from this. + locomotion: str # "quadruped", "flight_insect", "sessile", "rooted" + skeleton_id: str | None # "quadruped_medium", "insect_wing", None + + # === Metabolism === + thermoregulation: str # "endotherm", "ectotherm", "autotroph" + mass_specific_bmr: float | None # override if known; otherwise derived from body_mass + + # === Diet & Trophic === + diet_type: str # "herbivore", "nectarivore", "carnivore", "omnivore", + # "autotroph", "decomposer" + diet_breadth: list[str] # resource tags consumed: ["graminoid", "forb"], + # ["forb:fruiting"], ["herbivore_medium"] + trophic_level: float # 1.0 = producer, 2.0 = primary consumer, 3.0 = predator + + # === Reproduction === + reproductive_strategy: str # "K_selected", "r_selected" + clutch_size: int # offspring per reproduction event + generation_time_ticks: int # minimum ticks between reproduction events + + # === Environmental Tolerances === + thermal_range: tuple[float, float] # (min_C, max_C) — viable temperature range + drought_tolerance: float # 0.0 (needs constant water) to 1.0 (desert-adapted) + shade_tolerance: float # 0.0 (full sun) to 1.0 (understory specialist) + + # === Sensory & Movement === + sensory_range_multiplier: float # 1.0 = default for body size; >1 = enhanced + movement_budget: float # fraction of energy allocated to movement (0–1) + + # === Plant-Specific (ignored for animals) === + spread_mode: str | None # "runner", "seed_wind", "seed_animal", None + spread_range: float | None # max spread distance in grid units + root_persistence: bool # True = goes dormant instead of dying + canopy_radius: float | None # shade footprint for trees +``` + +### Step 1.3 — Write the Allometric Derivation Functions + +These are pure functions: `TraitVector → DerivedParams`. They live in a new file +`ecosim/traits.py`. They use **no external dependencies** (stdlib math only, honoring +the ecosim constraint). + +**Core allometric equations to implement:** + +```python +import math + +# === Metabolic Rate (Kleiber's Law) === +# BMR = B0 * M^0.75 +# B0 ≈ 70 for mammals in kcal/day, but we normalize to per-tick rates. +# The exponent 0.75 is the consensus value (Kleiber 1932, Brown et al. 2004). +# For ectotherms, use 0.69 (Gillooly et al. 2001). + +def derive_metabolic_rate(mass_kg: float, thermoregulation: str) -> float: + """Returns normalized metabolic rate (arbitrary units, per tick).""" + exponent = 0.75 if thermoregulation == "endotherm" else 0.69 + # B0 chosen so that an 80kg deer ≈ current engine hunger_rate + return B0_NORMALIZED * (mass_kg ** exponent) + + +# === Movement Speed === +# Cruising speed scales as M^0.25 for terrestrial animals (Peters 1983). +# Maximum speed has a hump-shaped relationship (Hirt et al. 2017), but +# cruising/foraging speed is well-approximated by the power law. +# Insects: flight speed scales as M^0.17 (Dudley 2000). + +def derive_speed(mass_kg: float, locomotion: str) -> float: + """Returns movement speed in grid units per tick.""" + if locomotion == "flight_insect": + return SPEED_BASE_INSECT * (mass_kg ** 0.17) + elif locomotion in ("quadruped", "biped"): + return SPEED_BASE_TERRESTRIAL * (mass_kg ** 0.25) + else: # sessile, rooted + return 0.0 + + +# === Sensory Range === +# Scales with home range, which scales as M^0.75 (McNab 1963) to M^1.0 +# (Kelt & Van Vuren 2001). We use M^0.5 as a moderate estimate for +# sensory detection range (not home range) within our 32³ grid. + +def derive_sensory_range(mass_kg: float, multiplier: float) -> float: + """Returns detection radius in grid units.""" + return SENSORY_BASE * (mass_kg ** 0.5) * multiplier + + +# === Hunger / Thirst / Energy Rates === +# All consumption rates scale with metabolic rate. +# Hunger rate = metabolic_rate * hunger_fraction +# Thirst rate = metabolic_rate * water_fraction (endotherms need more water) + +def derive_flow_rates(metabolic_rate: float, traits: 'TraitVector') -> dict: + """Returns per-tick flow deltas for hunger, thirst, energy.""" + hunger_rate = metabolic_rate * HUNGER_METABOLIC_FRACTION + thirst_rate = metabolic_rate * WATER_METABOLIC_FRACTION + if traits.thermoregulation == "ectotherm": + thirst_rate *= 0.3 # ectotherms lose less water + energy_decay = metabolic_rate * ENERGY_METABOLIC_FRACTION + return { + "hunger_rate": hunger_rate, + "thirst_rate": thirst_rate, + "energy_decay": energy_decay, + } + + +# === Guard Thresholds === +# Hysteresis bands scale inversely with metabolic rate — smaller/faster +# metabolisms hit thresholds sooner (tighter margins). +# Enter threshold = base_enter * (1 + metabolic_adjustment) +# Exit threshold = base_exit * (1 - metabolic_adjustment) +# where metabolic_adjustment compresses bands for high-metabolism species. + +def derive_guard_thresholds(metabolic_rate: float, traits: 'TraitVector') -> dict: + """Returns hysteresis enter/exit pairs for each guard condition.""" + # Normalize metabolic rate relative to reference species (deer = 1.0) + m_norm = metabolic_rate / REFERENCE_METABOLIC_RATE + adjustment = min(0.15, 0.1 * math.log(m_norm + 0.01) + 0.1) + + if traits.reproductive_strategy == "r_selected": + repro_threshold = 0.7 # lower bar, reproduce more readily + else: + repro_threshold = 0.8 # K-selected, higher bar + + return { + "hunger_enter": 0.3 * (1 + adjustment), + "hunger_exit": 0.15 * (1 - adjustment), + "hydration_enter": 0.2, + "hydration_exit": 0.6, + "energy_enter": 0.2 * (1 + adjustment), + "energy_exit": 0.5 * (1 - adjustment), + "repro_drive_threshold": repro_threshold, + } + + +# === Consumption Rate === +# How much resource an entity consumes per feeding event. +# Scales with metabolic rate — larger animals eat more per bite. + +def derive_consumption_rate(metabolic_rate: float) -> float: + return metabolic_rate * CONSUMPTION_METABOLIC_FRACTION +``` + +**Calibration constants** (`B0_NORMALIZED`, `SPEED_BASE_TERRESTRIAL`, etc.) are chosen +so that when you plug in deer traits (80 kg, endotherm, quadruped), the derivation +produces values matching the current hard-coded parameters from Step 1.1. This is the +"same dynamics" guarantee. The constants live in `ecosim/traits.py` as module-level +values with comments explaining the calibration. + +### Step 1.4 — Define the Interaction Template Grammar + +Replace per-species-pair interaction code with a small set of parameterized templates. + +```python +@dataclass +class InteractionTemplate: + """A class of ecological interaction, parameterized by trait compatibility.""" + interaction_type: str # "herbivory", "predation", "pollination", + # "competition", "mutualism", "decomposition" + + def matches(self, actor: TraitVector, target: TraitVector) -> bool: + """Does this interaction apply between actor and target?""" + ... + + def compute_rates(self, actor: TraitVector, target: TraitVector) -> dict: + """Returns interaction-specific parameters (consumption, linger time, etc.).""" + ... +``` + +**Six templates cover the current five interaction chains plus future expansions:** + +**Herbivory** — Actor: `diet_type in (herbivore, omnivore)`. Target: `entity_class in (PLANT, TREE)`. +Match condition: any tag in actor's `diet_breadth` matches target's resource tags +(`graminoid` for grass, `forb` for wildflower, `mast` for oak acorns). +Derived params: consumption rate (from actor metabolic rate), preference ordering +(match specificity — `graminoid` before `forb` for a grazer with both in diet_breadth). + +**Predation** — Actor: `diet_type in (carnivore, omnivore)`. Target: any tag in +actor's `diet_breadth` matches target's functional group. Additional constraint: +body mass ratio between 0.1× and 2× (predators don't take prey much larger than +themselves). Derived params: capture probability (speed ratio), consumption rate, +flee trigger on target. + +**Pollination** — Actor: `diet_type == nectarivore` or `pollinator` in functional roles. +Target: PLANT with `pollination_syndrome` matching actor's `floral_affinity`. +Match condition: target must be in FRUITING state (growth ≥ threshold, health > threshold). +Derived params: linger time (inversely proportional to actor metabolic rate), +cooldown on target (prevents re-pollination). + +**Competition** — Implicit. When two entities share `diet_breadth` tags and forage +in the same area, resource depletion creates competition without explicit code. +The engine already handles this through resource tracking — no template needed, +just ensure resource tags are checked consistently. + +**Water Access** — All mobile entities with `thermoregulation != autotroph` seek +water when hydration drops below threshold. This is already trait-derivable +(thirst rate from metabolic rate), not a species-specific interaction. + +**Decomposition** — Actor: `diet_type == decomposer`. Target: dead organic matter. +Converts dead entity biomass into soil nutrients. Template params derived from +actor metabolic rate. + +### Step 1.5 — Build the Trait Compiler + +The `TraitCompiler` runs once at world initialization. It takes the list of +`TraitVector` objects from the world JSON and produces: + +1. **Per-entity derived params** — a `DerivedParams` dataclass for each entity, + containing all the values the tick loop needs (guard thresholds, flow rates, + speed, sensory range, consumption rate). + +2. **Interaction matrix** — for each entity pair, which interaction templates + apply and with what parameters. Stored as a sparse structure (most pairs + don't interact). This replaces the current `if entity_type ==` dispatch in + the interaction phase. + +3. **Resource tag registry** — maps plant species to their resource tags, + so the herbivory template can match diet_breadth against available food. + +```python +class TraitCompiler: + """Compiles trait vectors into engine-ready parameters.""" + + def __init__(self, trait_vectors: list[TraitVector], biome: BiomeConfig): + self.traits = {tv.species_id: tv for tv in trait_vectors} + self.biome = biome + + def compile(self) -> CompiledEcology: + """Returns all derived parameters for the engine.""" + derived = {} + for sid, tv in self.traits.items(): + metabolic = derive_metabolic_rate(tv.body_mass_kg, tv.thermoregulation) + derived[sid] = DerivedParams( + metabolic_rate=metabolic, + speed=derive_speed(tv.body_mass_kg, tv.locomotion), + sensory_range=derive_sensory_range(tv.body_mass_kg, + tv.sensory_range_multiplier), + flow_rates=derive_flow_rates(metabolic, tv), + guard_thresholds=derive_guard_thresholds(metabolic, tv), + consumption_rate=derive_consumption_rate(metabolic), + ) + + interactions = self._build_interaction_matrix() + + return CompiledEcology( + derived_params=derived, + interaction_matrix=interactions, + resource_tags=self._build_resource_tags(), + ) + + def _build_interaction_matrix(self) -> dict: + """For each (actor, target) pair, find matching templates.""" + matrix = {} + templates = [Herbivory(), Predation(), Pollination(), Decomposition()] + for actor_id, actor_tv in self.traits.items(): + for target_id, target_tv in self.traits.items(): + if actor_id == target_id: + continue + matches = [] + for tmpl in templates: + if tmpl.matches(actor_tv, target_tv): + params = tmpl.compute_rates(actor_tv, target_tv) + matches.append((tmpl.interaction_type, params)) + if matches: + matrix[(actor_id, target_id)] = matches + return matrix +``` + +### Step 1.6 — Refactor engine.py to Read Derived Params + +This is the most delicate step. The tick loop structure stays identical. What changes +is *where* each phase gets its constants. + +**Before:** +```python +# In the guard phase +if entity["type"] == "ANIMAL": + if entity["hydration"] < 0.2: # hard-coded + entity["state"] = "DRINKING" +``` + +**After:** +```python +# In the guard phase +params = compiled.derived_params[entity["species_id"]] +if entity["hydration"] < params.guard_thresholds["hydration_enter"]: + entity["state"] = "DRINKING" +``` + +Each `if entity["type"] ==` branch becomes a lookup into `DerivedParams`. The seven +tick phases are refactored one at a time, with tests run after each: + +1. **Flow phase** — replace hard-coded hunger/thirst/energy deltas with + `params.flow_rates`. Run tests. +2. **Guard phase** — replace hard-coded thresholds with `params.guard_thresholds`. + Run tests. +3. **Interaction phase** — replace species-specific interaction code with + interaction matrix lookups. Run tests. +4. **Movement phase** — replace hard-coded speeds with `params.speed` and + `params.sensory_range`. Run tests. +5. **Spawning/reproduction phase** — replace hard-coded clutch sizes and + spread ranges with derived values. Run tests. +6. **Voxel effects** — these are mostly biome-driven, not species-driven. + Minimal change expected. +7. **Motor inference** — no change. BYOM adapters already use the protocol. + +### Step 1.7 — Write Trait Vectors for Existing Species + +Express each current species as a trait vector in the world JSON. These vectors, +when compiled, must produce parameters matching the audit from Step 1.1. + +```json +{ + "species_definitions": [ + { + "species_id": "deer", + "functional_group": "herbivore", + "entity_class": "ANIMAL", + "body_mass_kg": 80.0, + "locomotion": "quadruped", + "skeleton_id": "quadruped_medium", + "thermoregulation": "endotherm", + "diet_type": "herbivore", + "diet_breadth": ["graminoid", "forb"], + "trophic_level": 2.0, + "reproductive_strategy": "K_selected", + "clutch_size": 1, + "generation_time_ticks": 5000, + "thermal_range": [0, 40], + "drought_tolerance": 0.3, + "shade_tolerance": 0.3, + "sensory_range_multiplier": 1.0, + "movement_budget": 0.4, + "resource_tags": [] + }, + { + "species_id": "butterfly", + "functional_group": "pollinator", + "entity_class": "INSECT", + "body_mass_kg": 0.0005, + "locomotion": "flight_insect", + "skeleton_id": "insect_wing", + "thermoregulation": "ectotherm", + "diet_type": "nectarivore", + "diet_breadth": ["forb:fruiting"], + "trophic_level": 2.0, + "reproductive_strategy": "r_selected", + "clutch_size": 3, + "generation_time_ticks": 2000, + "thermal_range": [10, 35], + "drought_tolerance": 0.1, + "shade_tolerance": 0.5, + "sensory_range_multiplier": 1.2, + "movement_budget": 0.6, + "resource_tags": [], + "floral_affinity": ["forb"] + }, + { + "species_id": "oak", + "functional_group": "producer", + "entity_class": "TREE", + "body_mass_kg": 5000.0, + "locomotion": "rooted", + "skeleton_id": null, + "thermoregulation": "autotroph", + "diet_type": "autotroph", + "diet_breadth": [], + "trophic_level": 1.0, + "reproductive_strategy": "K_selected", + "clutch_size": 1, + "generation_time_ticks": 20000, + "thermal_range": [-10, 40], + "drought_tolerance": 0.5, + "shade_tolerance": 0.2, + "sensory_range_multiplier": 0.0, + "movement_budget": 0.0, + "canopy_radius": 3.0, + "root_persistence": true, + "resource_tags": ["mast"] + }, + { + "species_id": "meadow_grass", + "functional_group": "producer", + "entity_class": "PLANT", + "body_mass_kg": 0.01, + "locomotion": "sessile", + "skeleton_id": null, + "thermoregulation": "autotroph", + "diet_type": "autotroph", + "diet_breadth": [], + "trophic_level": 1.0, + "reproductive_strategy": "r_selected", + "clutch_size": 2, + "generation_time_ticks": 500, + "thermal_range": [5, 35], + "drought_tolerance": 0.2, + "shade_tolerance": 0.4, + "sensory_range_multiplier": 0.0, + "movement_budget": 0.0, + "spread_mode": "runner", + "spread_range": 2.0, + "root_persistence": true, + "resource_tags": ["graminoid"] + }, + { + "species_id": "wildflower", + "functional_group": "producer", + "entity_class": "PLANT", + "body_mass_kg": 0.05, + "locomotion": "sessile", + "skeleton_id": null, + "thermoregulation": "autotroph", + "diet_type": "autotroph", + "diet_breadth": [], + "trophic_level": 1.0, + "reproductive_strategy": "r_selected", + "clutch_size": 1, + "generation_time_ticks": 800, + "thermal_range": [5, 35], + "drought_tolerance": 0.15, + "shade_tolerance": 0.3, + "sensory_range_multiplier": 0.0, + "movement_budget": 0.0, + "spread_mode": "runner", + "spread_range": 3.5, + "root_persistence": true, + "resource_tags": ["forb"], + "pollination_syndrome": "insect_generalist" + } + ] +} +``` + +### Step 1.8 — Calibration & Regression Testing + +1. Run `TraitCompiler` on the five trait vectors above. +2. Compare every value in `DerivedParams` against the audit table from Step 1.1. +3. Adjust calibration constants (`B0_NORMALIZED`, `SPEED_BASE_*`, etc.) until + derived values match hard-coded values within 5%. +4. Run the full test suite (12 unit tests + smoke test). +5. Run the demo_world for 2000 ticks and compare entity population curves, + state transition counts, and event counts against a baseline recording + from the current engine. Acceptable drift: ±10% on population counts, + identical state machine topology (same states reachable in same order). + +### Step 1.9 — Backward Compatibility + +The old world JSON format (without `species_definitions`) must still work. +If no trait vectors are present, the engine falls back to the current hard-coded +paths. This is a deprecation bridge, not a permanent design. + +```python +if "species_definitions" in world_config: + compiled = TraitCompiler(parse_traits(world_config), biome).compile() +else: + compiled = LegacyParams() # wraps current hard-coded values +``` + +### Phase 1 Deliverables + +- `ecosim/traits.py` — TraitVector, DerivedParams, allometric derivation functions +- `ecosim/interactions.py` — InteractionTemplate base + 4 concrete templates +- `ecosim/trait_compiler.py` — TraitCompiler class +- Refactored `engine.py` — reads from DerivedParams instead of hard-coded constants +- Refactored `voxel_manager.py` — 5 layers (nutrients_fast, nutrients_slow, + moisture, temperature, organic_matter), mineralization/dissolution/leaching + fluxes, death→organic_matter deposits +- Updated `examples/demo_world.json` — includes `species_definitions` key + + 3 new rate multipliers (mineralization, dissolution, nutrient_leaching) +- New tests in `tests/test_traits.py` — unit tests for every derivation function +- New tests in `tests/test_nutrients.py` — two-pool flow tests (see nutrient spec) +- New test: `tests/test_regression.py` — 2000-tick comparison against baseline + +**New files: 4. Modified files: 4. No new external dependencies.** + +--- + +## Phase 2 — New Species by Trait Vector Only + +**Goal:** Add three new species to the demo world by writing trait vectors in JSON. +Zero new engine code. The interaction templates and allometric derivations handle +everything. + +### Step 2.1 — Wolf (Predator) + +The first real test of the architecture. A wolf completes the food chain: +grass → deer → wolf. + +```json +{ + "species_id": "wolf", + "functional_group": "predator", + "entity_class": "ANIMAL", + "body_mass_kg": 40.0, + "locomotion": "quadruped", + "skeleton_id": "quadruped_medium", + "thermoregulation": "endotherm", + "diet_type": "carnivore", + "diet_breadth": ["herbivore"], + "trophic_level": 3.0, + "reproductive_strategy": "K_selected", + "clutch_size": 3, + "generation_time_ticks": 8000, + "thermal_range": [-15, 35], + "drought_tolerance": 0.4, + "shade_tolerance": 0.5, + "sensory_range_multiplier": 1.5, + "movement_budget": 0.5, + "resource_tags": [] +} +``` + +**What should happen automatically:** + +- Predation template matches: wolf `diet_breadth` ["herbivore"] overlaps with + deer's `functional_group` "herbivore". Body mass ratio 40/80 = 0.5 is within + the 0.1–2.0 predation window. +- Deer's flee response triggers: the engine sees a carnivore within sensory range + with body mass sufficient to be threatening. No deer-specific flee code needed. +- Wolf hunger rate derived from 40 kg endotherm: `70 * 40^0.75` ≈ lower than + deer's rate (smaller body), meaning wolves hunt frequently. +- Wolf speed derived from `40^0.25` ≈ 2.51 relative units. Deer speed from + `80^0.25` ≈ 2.99. Deer are faster but wolves have higher sensory range + multiplier, creating a pursuit dynamic. +- No wolf-butterfly interaction: wolf `diet_breadth` ["herbivore"] doesn't match + butterfly's functional_group "pollinator". + +**Validation:** Run 5000 ticks. Verify: wolves hunt deer, deer population declines +to a sustainable level, grass recovers (reduced grazing pressure), trophic cascade +emerges without any new engine code. + +### Step 2.2 — Songbird (New Trophic Niche) + +An insectivore that introduces a new diet pathway. Eats insects (butterflies), +disperses seeds. + +```json +{ + "species_id": "songbird", + "functional_group": "insectivore", + "entity_class": "BIRD", + "body_mass_kg": 0.025, + "locomotion": "flight_bird", + "skeleton_id": "bird_small", + "thermoregulation": "endotherm", + "diet_type": "omnivore", + "diet_breadth": ["pollinator", "forb:fruiting"], + "trophic_level": 2.5, + "reproductive_strategy": "r_selected", + "clutch_size": 4, + "generation_time_ticks": 3000, + "thermal_range": [5, 35], + "drought_tolerance": 0.2, + "shade_tolerance": 0.6, + "sensory_range_multiplier": 2.0, + "movement_budget": 0.5, + "resource_tags": [] +} +``` + +**Architecture test:** The predation template should match songbird → butterfly +(diet_breadth includes "pollinator", body mass ratio 0.025/0.0005 = 50×, but +this exceeds the 2× cap — so we either adjust the predation window for +insectivory or add body-mass-ratio rules per diet category). This is a real +design question the architecture must handle. Insectivory has different mass +ratios than mammalian predation. Solution: the predation template's mass ratio +window is parameterized per `diet_type` or `diet_breadth` category: + +- Carnivore hunting herbivores: ratio 0.1–2.0× +- Insectivore hunting insects: ratio 1.0–1000× (predator is always much larger) + +This is one parameterized constant, not a species-specific rule. + +### Step 2.3 — Mushroom (Decomposer) + +Closes the nutrient loop. Decomposes dead organic matter, enriches soil. + +```json +{ + "species_id": "mushroom", + "functional_group": "decomposer", + "entity_class": "MICROORGANISM", + "body_mass_kg": 0.001, + "locomotion": "sessile", + "skeleton_id": null, + "thermoregulation": "ectotherm", + "diet_type": "decomposer", + "diet_breadth": ["dead_organic_matter"], + "trophic_level": 1.0, + "reproductive_strategy": "r_selected", + "clutch_size": 5, + "generation_time_ticks": 300, + "thermal_range": [5, 30], + "drought_tolerance": 0.1, + "shade_tolerance": 0.9, + "sensory_range_multiplier": 0.0, + "movement_budget": 0.0, + "spread_mode": "spore", + "spread_range": 4.0, + "root_persistence": false, + "resource_tags": [] +} +``` + +**Architecture test:** This requires the decomposition interaction template to +work with the voxel system. When an entity dies, instead of just disappearing, +it leaves a "dead_organic_matter" marker. Nearby mushrooms consume it and +boost soil nutrients. This connects the entity lifecycle to the voxel layer +in a general way — not a mushroom-specific way. + +### Step 2.4 — Emergent Dynamics Validation + +With 8 species, run a suite of 10,000-tick simulations with varied initial +conditions. Document which interaction chains emerge *without being coded*: + +- [ ] Wolf-deer predation with population oscillations (Lotka-Volterra dynamics) +- [ ] Trophic cascade: wolves reduce deer → grass recovers → wildflowers bloom +- [ ] Songbird-butterfly predation reducing pollination rates +- [ ] Mushroom decomposition accelerating soil recovery after death events +- [ ] Cross-trophic competition: songbirds and butterflies competing for fruiting flowers +- [ ] Thermal range exclusions: some species drop out in extreme biome settings + +### Phase 2 Deliverables + +- Three new species as JSON trait vectors (zero new engine code) +- Updated interaction templates with parameterized mass-ratio windows +- Decomposition template + dead_organic_matter voxel integration +- Extended demo world: `examples/temperate_meadow_8sp.json` +- Emergent dynamics validation report +- `docs/trait_species_guide.md` — how a biologist adds a new species + +--- + +## Phase 3 — ASAL Substrate Protocol & FM-Guided Search + +**Goal:** Formalize līlā as an ASAL-compatible substrate. Build the headless +rendering pipeline, parameterize the search space, and implement the three +ASAL search modes (supervised target, open-endedness, illumination). + +### Step 3.1 — Substrate Protocol + +Define the three-function interface that ASAL expects. This lives in a new +top-level `search/` directory with its own `pyproject.toml` (depends on ecosim +core, plus torch, CLIP, and search/viz libraries). + +```python +from typing import Protocol +import numpy as np + +class ALifeSubstrate(Protocol): + """ASAL-compatible substrate interface.""" + + def init(self, theta: np.ndarray, seed: int = 0) -> dict: + """Initialize simulation state from parameter vector theta.""" + ... + + def step(self, state: dict) -> dict: + """Advance simulation by one tick. Returns new state.""" + ... + + def render(self, state: dict) -> np.ndarray: + """Render current state as RGB image (H, W, 3) uint8.""" + ... + + def theta_spec(self) -> ThetaSpec: + """Describes the parameter space: names, ranges, types.""" + ... +``` + +### Step 3.2 — θ Parameterization (Three Variants) + +Each variant exposes a different slice of the ecological parameter space: + +**EcoRates** (~15 dimensions) +- 6 rate multipliers (consumption, hunger, thirst, growth, reproduction, water_replenishment) +- Biome base values (soil_nutrients, soil_moisture, temperature — 3 dims) +- Water source config (count, mean_radius, mean_water_level — 3 dims) +- Rain frequency and intensity (2 dims) +- Entity count scaling factor (1 dim) + +**EcoTopology** (~50–80 dimensions) +- Everything in EcoRates +- Per-species: count (how many to spawn), spatial distribution parameters +- Species presence vector (binary: which of the 8 species are included) +- Water source positions (2D × count) +- Initial state variable perturbations + +**EcoAdapt** (~550–600 dimensions) +- Everything in EcoTopology +- MLP adapter weights (500 params for the reference MLP) +- Per-species motor adapter selection + +Each variant implements `theta_to_world_config(theta) -> dict` which converts +the flat parameter vector into a valid world JSON that the engine can load. + +### Step 3.3 — Headless Renderer + +A lightweight Python renderer (PIL or pure numpy) that takes `EcosystemEngine` +state and produces a top-down 2D image. No browser, no WebSocket. + +**What to render (semantically important for CLIP embedding):** + +- Soil moisture as background gradient (teal → amber, matching browser viz) +- Water sources as blue circles with radius proportional to water_level +- Each species as a distinct colored shape at its grid position: + - Animals/birds: directional triangles (colored by species) + - Insects: small dots with wing indicators + - Plants: circles scaled by growth, colored by health + - Trees: large circles with canopy halos + - Dormant plants: faded brown markers +- State labels not needed (CLIP works on visual patterns, not text) + +Target: 256×256 px, ~1ms render time. Lives in `search/renderer.py`. +Uses PIL (Pillow) which is the only new dependency for this module. + +```python +def headless_render(engine: EcosystemEngine, size: int = 256) -> np.ndarray: + """Render engine state as RGB numpy array.""" + img = Image.new("RGB", (size, size)) + draw = ImageDraw.Draw(img) + + # Background: soil moisture heatmap + _draw_moisture_background(draw, engine.voxel_manager, size) + + # Water sources + for ws in engine.water_sources: + _draw_water_source(draw, ws, size) + + # Entities by species + for entity in engine.entities: + _draw_entity(draw, entity, engine.compiled.traits, size) + + return np.array(img) +``` + +### Step 3.4 — FM Evaluation Pipeline + +The evaluation loop renders periodic frames from a simulation rollout and +embeds them with a vision-language foundation model. + +```python +import torch +import clip + +class FMEvaluator: + """Evaluates simulation rollouts using a vision-language foundation model.""" + + def __init__(self, model_name: str = "ViT-B/32", device: str = "cuda"): + self.model, self.preprocess = clip.load(model_name, device) + self.device = device + + def embed_frames(self, frames: list[np.ndarray]) -> torch.Tensor: + """Embed a sequence of rendered frames into FM space.""" + images = [self.preprocess(Image.fromarray(f)) for f in frames] + batch = torch.stack(images).to(self.device) + with torch.no_grad(): + embeddings = self.model.encode_image(batch) + return embeddings / embeddings.norm(dim=-1, keepdim=True) + + def supervised_target_score(self, embeddings: torch.Tensor, + prompts: list[str]) -> float: + """Score: how well does the rollout match a sequence of text prompts?""" + text_tokens = clip.tokenize(prompts).to(self.device) + with torch.no_grad(): + text_emb = self.model.encode_text(text_tokens) + text_emb = text_emb / text_emb.norm(dim=-1, keepdim=True) + # Match each prompt to the corresponding temporal frame + scores = (embeddings @ text_emb.T).diag() + return scores.mean().item() + + def open_endedness_score(self, embeddings: torch.Tensor) -> float: + """Score: how much novel territory does the trajectory cover?""" + novelties = [] + archive = [embeddings[0]] + for i in range(1, len(embeddings)): + distances = [1 - (embeddings[i] @ a).item() for a in archive] + novelty = min(distances) # nearest neighbor distance + novelties.append(novelty) + archive.append(embeddings[i]) + return sum(novelties) / len(novelties) + + def illumination_distance(self, embedding: torch.Tensor, + archive: list[torch.Tensor]) -> float: + """Score: how far is this simulation from its nearest neighbor?""" + if not archive: + return float("inf") + distances = [1 - (embedding @ a).item() for a in archive] + return min(distances) +``` + +### Step 3.5 — Search Loop Implementations + +Three search modes, matching ASAL's framework: + +**Supervised Target Search** — "Find the ecological parameters that produce +a sequence of events matching these prompts." + +```python +def search_supervised(substrate, evaluator, prompts, generations=100, + population=50, rollout_ticks=2000, render_every=100): + """CMA-ES search for theta that matches prompt sequence.""" + import cma + theta_spec = substrate.theta_spec() + es = cma.CMAEvolutionStrategy(theta_spec.initial, theta_spec.sigma0, + {"bounds": [theta_spec.lower, theta_spec.upper], + "popsize": population}) + for gen in range(generations): + candidates = es.ask() + scores = [] + for theta in candidates: + state = substrate.init(np.array(theta)) + frames = [] + for t in range(rollout_ticks): + state = substrate.step(state) + if t % render_every == 0: + frames.append(substrate.render(state)) + embeddings = evaluator.embed_frames(frames) + score = evaluator.supervised_target_score(embeddings, prompts) + scores.append(-score) # CMA-ES minimizes + es.tell(candidates, scores) + return es.result.xbest +``` + +Example ecological prompts: + +- Single target: `"a thriving meadow with grazing animals"` +- Temporal sequence: `["a lush green meadow", "overgrazing and bare soil", + "rain falling on dry ground", "new growth emerging from soil"]` +- Open-ended: no prompt needed — maximizes trajectory novelty + +**Open-Endedness Search** — "Find the ecosystem configuration that stays +interesting the longest." + +Uses the same CMA-ES loop but with `open_endedness_score` as the objective. +Longer rollouts (5000–10000 ticks) to test sustained novelty. + +**Illumination** — "Map the space of possible ecosystems." + +Uses a genetic algorithm with diversity pressure. Maintains an archive of +diverse solutions. New candidates are evaluated on how different they are +from everything in the archive. + +### Step 3.6 — Simulation Atlas Visualization + +After illumination, produce a 2D UMAP projection of all discovered ecosystems, +with rendered thumbnails at each point. This is the "atlas of possible ecologies" +visualization. + +```python +def build_simulation_atlas(archive: list[dict], output_path: str): + """Generate UMAP visualization of discovered ecosystems.""" + import umap + embeddings = np.stack([a["embedding"] for a in archive]) + reducer = umap.UMAP(n_components=2, metric="cosine") + coords = reducer.fit_transform(embeddings) + + # Create atlas image with thumbnails at UMAP coordinates + fig, ax = plt.subplots(figsize=(20, 20)) + for i, (x, y) in enumerate(coords): + thumbnail = archive[i]["final_frame"] + # ... plot thumbnail at (x, y) + fig.savefig(output_path, dpi=150) +``` + +### Step 3.7 — Physical Plausibility Constraints + +Unlike abstract ASAL substrates, līlā can reject physically impossible +configurations before evaluation, saving compute. + +```python +def validate_theta(theta: np.ndarray, spec: ThetaSpec) -> bool: + """Reject biologically impossible configurations.""" + world = theta_to_world_config(theta) + for species in world["species_definitions"]: + mass = species["body_mass_kg"] + locomotion = species["locomotion"] + thermo = species["thermoregulation"] + + # Square-cube law: flying insects can't exceed ~0.1 kg + if locomotion == "flight_insect" and mass > 0.1: + return False + + # Endotherms below ~2g can't thermoregulate + if thermo == "endotherm" and mass < 0.002: + return False + + # Terrestrial animals above ~10,000 kg are structurally implausible + if locomotion == "quadruped" and mass > 10000: + return False + + # Trophic sanity: carnivores need prey species present + if species["diet_type"] == "carnivore": + prey_present = any( + s["functional_group"] in species["diet_breadth"] + for s in world["species_definitions"] + if s["species_id"] != species["species_id"] + ) + if not prey_present: + return False + + return True +``` + +### Phase 3 Deliverables + +- `search/` directory with own pyproject.toml +- `search/substrate.py` — ALifeSubstrate protocol + LilaSubstrate implementation +- `search/renderer.py` — headless PIL renderer +- `search/evaluator.py` — FM evaluation pipeline (CLIP + DINOv2) +- `search/search.py` — three search mode implementations +- `search/theta.py` — θ parameterization for EcoRates, EcoTopology, EcoAdapt +- `search/atlas.py` — simulation atlas visualization +- `search/constraints.py` — physical plausibility validation +- `examples/search_configs/` — example search configurations +- `docs/asal_substrate_guide.md` — how to use līlā as an ASAL substrate + +--- + +## File Layout After All Three Phases + +``` +lila/ +├── server/ +│ ├── ecosim/ +│ │ ├── engine.py # refactored: reads DerivedParams +│ │ ├── entities.py # updated: species_id field +│ │ ├── traits.py # NEW: TraitVector, allometric derivations +│ │ ├── interactions.py # NEW: InteractionTemplate grammar +│ │ ├── trait_compiler.py # NEW: TraitCompiler +│ │ ├── biome.py # unchanged +│ │ ├── voxel_manager.py # minor: dead_organic_matter support +│ │ ├── model_adapter.py # unchanged +│ │ ├── worker.py # unchanged +│ │ └── adapters/ # unchanged +│ ├── tests/ +│ │ ├── test_ecosim.py # existing, must still pass +│ │ ├── smoke_test.py # existing, must still pass +│ │ ├── test_traits.py # NEW: allometric derivation tests +│ │ └── test_regression.py # NEW: 2000-tick baseline comparison +│ └── examples/ +│ ├── demo_world.json # updated: species_definitions key +│ └── temperate_meadow_8sp.json # NEW: 8-species world +│ +├── search/ # NEW: entire directory +│ ├── pyproject.toml # deps: torch, clip, cma, umap, pillow, matplotlib +│ ├── substrate.py +│ ├── renderer.py +│ ├── evaluator.py +│ ├── search.py +│ ├── theta.py +│ ├── atlas.py +│ ├── constraints.py +│ └── examples/ +│ ├── search_target.py # example: find a thriving meadow +│ ├── search_openended.py # example: find the most open-ended ecosystem +│ └── search_illuminate.py # example: map the ecology space +│ +├── docs/ +│ ├── trait_species_guide.md # NEW: how biologists add species +│ └── asal_substrate_guide.md # NEW: how to use līlā with ASAL +│ +└── (everything else unchanged) +``` + +--- + +## Critical Constraints Preserved + +- **ecosim remains stdlib-only.** `traits.py`, `interactions.py`, and + `trait_compiler.py` use only `math`, `dataclasses`, and typing from stdlib. + All FM/search dependencies live in `search/`. +- **Docker Compose still works.** The trait system is internal to ecosim. + No new containers needed. +- **Tick rate budget preserved.** The trait compiler runs once at init, not per tick. + Per-tick lookups into `DerivedParams` are dict access — O(1), no regression. +- **32³ grid, 4D motion latent, 10Hz tick rate** — all locked, unchanged. +- **Voxel layers: 5.** nutrients_fast, nutrients_slow, moisture, temperature, + organic_matter. Updated from 4 → 5 per two-pool nutrient decision. +- **Randomization remains opt-in.** Trait vectors are deterministic; randomization + still controlled by the `"randomize"` key. +- **Plants still go dormant.** `root_persistence: true` in the trait vector + maps to the existing dormancy logic. +- **BYOM adapter protocol unchanged.** Adapters don't know about traits. + They receive the same context spec and return the same 4D latent. + +--- + +## Allometric References + +These are the scaling laws used in the derivation functions: + +| Relationship | Equation | Source | +|---|---|---| +| Basal metabolic rate | BMR = B₀ × M^0.75 (endotherm) | Kleiber 1932, Brown et al. 2004 | +| Metabolic rate (ectotherm) | BMR = B₀ × M^0.69 | Gillooly et al. 2001 | +| Cruising speed (terrestrial) | v = v₀ × M^0.25 | Peters 1983 | +| Flight speed (insect) | v = v₀ × M^0.17 | Dudley 2000 | +| Home range / sensory | HR ∝ M^0.75 | McNab 1963, Kelt & Van Vuren 2001 | +| Max speed (hump-shaped) | v_max = v_theor × (1 - e^(-k×τ)) | Hirt et al. 2017 | +| Consumption rate | ∝ BMR | Brown et al. 2004 (MTE) | +| Generation time | T_gen ∝ M^0.25 | Western 1979 | + +**Key reference for the overall approach:** + +- Harfoot et al. 2014. "Emergent Global Patterns of Ecosystem Structure and + Function from a Mechanistic General Ecosystem Model." PLoS Biology. + (The Madingley Model — trait-based, allometric, no species-specific code.) + +--- + +## Sequence & Dependencies + +``` +Phase 1.1 Audit hard-coded params ← no dependencies, start here +Phase 1.2 Define TraitVector schema ← informs 1.3 +Phase 1.3 Allometric derivations ← needs 1.1 for calibration targets +Phase 1.4 Interaction templates ← needs 1.2 for trait matching +Phase 1.5 TraitCompiler ← needs 1.3 + 1.4 +Phase 1.5a Two-pool nutrient refactor ← see TWO_POOL_NUTRIENT_SPEC.md + (voxel layers 4→5, mineralization/dissolution/leaching, + rain split, dormancy check update, death→organic_matter, + 3 new rate multipliers) — do before 1.6 so the engine + refactor picks up the new layer indices +Phase 1.6 Refactor engine.py ← needs 1.5 + 1.5a, most delicate step +Phase 1.7 Write trait vectors ← needs 1.2 schema +Phase 1.8 Calibration & regression ← needs 1.6 + 1.7, blocks Phase 2 +Phase 1.9 Backward compatibility ← safety net, do alongside 1.6 + +Phase 2.1 Wolf trait vector ← needs Phase 1 complete +Phase 2.2 Songbird trait vector ← reveals mass-ratio edge cases +Phase 2.3 Mushroom trait vector ← connects to two-pool decomposition +Phase 2.4 Emergent dynamics report ← validates the architecture + +Phase 3.1 Substrate protocol ← needs Phase 1 (trait-parameterized engine) +Phase 3.2 θ parameterization ← needs 3.1 (includes mineralization/ + dissolution/leaching as searchable dims) +Phase 3.3 Headless renderer ← independent, can start during Phase 2 +Phase 3.4 FM evaluation pipeline ← needs 3.3 +Phase 3.5 Search loop implementations ← needs 3.2 + 3.4 +Phase 3.6 Simulation atlas viz ← needs 3.5 results +Phase 3.7 Plausibility constraints ← needs 1.2 trait schema +``` + +--- + +## Open Questions (Decisions Needed Before or During Implementation) + +1. **Allometric exponent for ectotherm metabolism:** Literature ranges from 0.69 + to 0.75. Does līlā use a single exponent with a thermoregulation coefficient, + or separate exponents? The Madingley Model uses separate. Recommend: separate, + matching Gillooly et al. 2001. + +2. **Predation mass-ratio windows by diet category:** Mammalian predation (0.1–2×), + insectivory (1–1000×), piscivory (0.01–10×). Should these be hard constants or + part of the trait vector? Recommend: hard constants per diet category, stored in + `interactions.py`. They're well-established ecological relationships, not tunable + parameters. + +3. **Dead organic matter representation:** ✅ **DECIDED.** Entity death deposits + biomass into the organic_matter voxel layer at the death position. Amount + proportional to body mass. Organic matter mineralizes into the new + nutrients_slow pool, which dissolves into nutrients_fast. Decomposer entities + accelerate mineralization locally. See `TWO_POOL_NUTRIENT_SPEC.md` for full + implementation spec. **Design decision update: Voxel layers 4 → 5.** + +4. **Plant trait derivations:** Allometric scaling is best validated for animals. + Plant growth rates, spread distances, and dormancy thresholds are less cleanly + allometric. Recommend: keep plant-specific traits (spread_range, spread_mode, + root_persistence) as explicit trait fields rather than deriving them from + body mass. The trait system still eliminates per-species engine code; it just + doesn't pretend plant ecology follows mammalian allometry. + +5. **FM choice for Phase 3:** CLIP (ViT-B/32) is the ASAL default and runs on + 16GB VRAM. DINOv2 is vision-only (no text prompts for supervised target). + SigLIP or newer VLMs may be better but are heavier. Recommend: start with + CLIP ViT-B/32, add DINOv2 as an option for open-endedness/illumination where + text prompts aren't needed. + +6. **JAX port consideration:** ASAL's codebase is JAX-native for end-to-end + differentiability. līlā is pure Python. Porting the engine to JAX would + enable gradient-based search but is a major rewrite. Recommend: don't port. + CMA-ES (gradient-free) works well for ASAL's non-NCA substrates and handles + 600-dimensional spaces fine. The ecological grounding of the search space + may make gradient-free search more efficient anyway, since parameters have + meaningful directions. + +7. **Narrative for the blog series:** This transition is a natural continuation + of "The Unseen Hand" thesis. The story arc: manual design → trait-based + derivation → FM-guided discovery. The AI isn't just driving motion (motor + adapters) — it's searching for the ecological configurations that produce + the most lifelike dynamics. The unseen hand operates at two levels now. diff --git a/docs/TWO_POOL_NUTRIENT_SPEC.md b/docs/TWO_POOL_NUTRIENT_SPEC.md new file mode 100644 index 0000000..00fffec --- /dev/null +++ b/docs/TWO_POOL_NUTRIENT_SPEC.md @@ -0,0 +1,685 @@ +# līlā — Two-Pool Nutrient Split: Implementation Spec + +> **Decision:** Split the single `nutrients` voxel layer into `nutrients_fast` and +> `nutrients_slow`. Add a mineralization flux between pools. This unlocks +> meaningful temporal separation in soil recovery and gives the Phase 2 +> decomposer species a mechanistically distinct role. +> +> **Design decision update:** Voxel layers changes from 4 → 5. +> `[nutrients, moisture, temperature, organic_matter]` → +> `[nutrients_fast, nutrients_slow, moisture, temperature, organic_matter]` + +--- + +## Conceptual Model + +``` + ┌─────────────────────┐ + │ Dead Entities │ + │ (animal/plant) │ + └──────────┬──────────┘ + │ death event: +biomass + ▼ + ┌─────────────────────┐ + │ organic_matter │ Layer 5 (existing, unchanged) + │ (dead biomass) │ Slow decay, spatial + └──────────┬──────────┘ + │ decomposition rate (accelerated by decomposers) + ▼ +┌──────────┐ ┌─────────────────────┐ +│ Rain │──────▶│ nutrients_slow │ Layer 2 (NEW) +│ (mineral │ │ (mineralized pool) │ Stable, slow-release +│ input) │ │ Long-term soil │ Represents soil health +└──────────┘ │ health indicator │ + │ └──────────┬──────────┘ + │ │ dissolution rate + │ ▼ + │ ┌─────────────────────┐ + └─────────────▶│ nutrients_fast │ Layer 1 (replaces old "nutrients") + │ (plant-available) │ Quick turnover + │ Dissolved, labile │ Plants consume from here + └──────────┬──────────┘ + │ plant uptake + ▼ + ┌─────────────────────┐ + │ Plant Growth │ + │ (health, growth) │ + └─────────────────────┘ +``` + +**Why two pools, not one:** + +With a single nutrient pool, rain and decomposition are interchangeable — they +both increment the same number. This means: + +- A heavily overgrazed meadow recovers at the same rate from one rain event as + a meadow with active decomposition. That's ecologically wrong. +- There's no concept of "soil health" distinct from "nutrients available right now." + A lush meadow and a depleted one with recent rain look identical. +- The decomposer species (Phase 2 mushroom) has no distinct mechanistic role — + it just does what rain does, slower. + +With two pools: + +- **Fast pool** (nutrients_fast): what plants actually eat. Depletes quickly + under heavy growth. Refills quickly from rain (mineral nutrients washed in) + and from dissolution of the slow pool. This is the short-term signal. +- **Slow pool** (nutrients_slow): long-term soil health. Builds up from + decomposition of organic matter. Depletes slowly. Feeds the fast pool via + mineralization/dissolution. This is the memory of the soil — a meadow that's + been healthy for thousands of ticks has a deep slow pool that buffers against + short-term stress. + +This creates two recovery timescales: +- **Rain recovery** (fast): rain → nutrients_fast. Plants respond within tens + of ticks. But if the slow pool is depleted, the fast pool drains quickly again + once rain stops. The meadow "bounces" but doesn't sustain. +- **Decomposition recovery** (slow): dead matter → organic_matter → + nutrients_slow → nutrients_fast. Takes hundreds of ticks but builds lasting + soil health. This is the ecological role of decomposers. + +--- + +## Pool Dynamics (Per-Tick Equations) + +All values are clamped to [0.0, 1.0] after each tick. + +### Mineralization: organic_matter → nutrients_slow + +Organic matter slowly converts to mineralized nutrients. The base rate is +constant; decomposer entities accelerate it locally (Phase 2). + +```python +# Base mineralization (always active, represents microbial background) +delta_slow = organic_matter[cell] * MINERALIZATION_RATE +nutrients_slow[cell] += delta_slow +organic_matter[cell] -= delta_slow + +# Decomposer acceleration (Phase 2, when decomposer entities exist) +# Each nearby decomposer multiplies the local rate +# decomposer_factor = 1.0 + (nearby_decomposer_count * DECOMPOSER_BOOST) +# delta_slow *= decomposer_factor +``` + +**MINERALIZATION_RATE:** 0.002 per tick (tuned so organic_matter has a half-life +of ~350 ticks without decomposers, ~70 ticks with 3 nearby decomposers) + +### Dissolution: nutrients_slow → nutrients_fast + +Slow pool feeds fast pool at a steady rate. This is the "background fertility" +that makes healthy soil valuable. + +```python +delta_fast = nutrients_slow[cell] * DISSOLUTION_RATE +nutrients_fast[cell] += delta_fast +nutrients_slow[cell] -= delta_fast +``` + +**DISSOLUTION_RATE:** 0.005 per tick (slow pool half-life ~140 ticks, meaning +a fully charged slow pool sustains the fast pool for hundreds of ticks even +with no new input) + +### Plant Uptake: nutrients_fast → plant growth + +Plants consume from the fast pool only. Uptake rate is proportional to growth +rate (larger/faster-growing plants consume more). + +```python +# Already exists conceptually in the flow phase. +# Currently reads "nutrients" — change to read "nutrients_fast" +uptake = plant_growth_rate * UPTAKE_FRACTION +nutrients_fast[cell] -= uptake +# Clamped: if nutrients_fast < 0, growth is limited +``` + +### Rain Input + +Rain delivers mineral nutrients directly to the fast pool (dissolved in +rainwater) and a smaller amount to the slow pool (particulate deposition). + +```python +# Currently: nutrients += 0.024 * intensity +# New: +nutrients_fast[cell] += 0.020 * intensity # ~83% of original rain nutrient boost +nutrients_slow[cell] += 0.004 * intensity # ~17% — builds long-term health +``` + +The split preserves the total nutrient input from rain (0.024) while directing +most of it to the immediately available pool. + +### Soil Evaporation / Nutrient Leaching + +The fast pool should experience slow leaching (nutrients wash deeper into soil +or are lost). The slow pool is stable. + +```python +# Fast pool leaching (new, runs in the soil evaporation phase) +nutrients_fast[cell] -= nutrients_fast[cell] * NUTRIENT_LEACH_RATE + +# Slow pool: no leaching (stable soil organic matter doesn't leach significantly) +``` + +**NUTRIENT_LEACH_RATE:** 0.001 per tick (very slow, but creates long-term +pressure to maintain inputs) + +### Death Event: entity → organic_matter + +When an entity dies, its biomass is deposited into the organic_matter layer at +its grid position. The amount deposited is proportional to body mass (from the +trait vector in the trait-based architecture, or a constant per entity type +in the current architecture). + +```python +# In the removals phase, when an entity is removed: +cell = grid_position(entity) +biomass_deposit = entity_biomass(entity) # body_mass_kg normalized to 0–1 scale +organic_matter[cell] += biomass_deposit +# Clamped to 1.0 +``` + +**Normalization:** body_mass_kg mapped to organic_matter deposit via a scaling +constant. An 80 kg deer deposits ~0.15 organic_matter. A 0.01 kg grass blade +deposits ~0.002. A 5000 kg oak deposits 0.5+ (capped at cell max, spillover +to neighbors). + +--- + +## Voxel Manager Changes + +### Layer Index Update + +```python +# Before: +LAYER_NUTRIENTS = 0 +LAYER_MOISTURE = 1 +LAYER_TEMPERATURE = 2 +LAYER_ORGANIC_MATTER = 3 +NUM_LAYERS = 4 + +# After: +LAYER_NUTRIENTS_FAST = 0 +LAYER_NUTRIENTS_SLOW = 1 +LAYER_MOISTURE = 2 +LAYER_TEMPERATURE = 3 +LAYER_ORGANIC_MATTER = 4 +NUM_LAYERS = 5 +``` + +### initialize_from_soil Update + +The biome's `soil_nutrients` value now initializes both pools: + +```python +def initialize_from_soil(self, soil_config: dict): + """Initialize voxel grid from biome soil configuration.""" + base_nutrients = soil_config.get("nutrients", 0.5) + moisture = soil_config.get("moisture", 0.3) + temperature = soil_config.get("temperature", 0.5) + + for cell in self._all_cells(): + # Fast pool: 40% of base nutrients (immediately available) + self.set(cell, LAYER_NUTRIENTS_FAST, base_nutrients * 0.4) + # Slow pool: 60% of base nutrients (long-term reserve) + self.set(cell, LAYER_NUTRIENTS_SLOW, base_nutrients * 0.6) + # Rest unchanged + self.set(cell, LAYER_MOISTURE, moisture) + self.set(cell, LAYER_TEMPERATURE, temperature) + # organic_matter starts at 0 (no dead biomass yet) + self.set(cell, LAYER_ORGANIC_MATTER, 0.0) +``` + +The 40/60 split means a new world starts with some immediate fertility (fast) +and a deeper reserve (slow). The ratio is a biome parameter that could vary: +desert = 20/80 (little available, deep mineral reserve), tropical = 60/40 +(lots available, less mineral reserve due to leaching). + +**IMPORTANT:** The `initialize_from_soil` break bug (documented in gotchas) +was that it silently skipped layers 2-3 due to a `break` in an `elif` chain. +With 5 layers, this must be verified: all five layers must be initialized. + +### Dirty Tracking + +The threshold-gated dirty tracking works identically for 5 layers — each cell×layer +pair is tracked independently. No structural change needed, just the expanded +layer count. + +### Delta Packets + +Delta packets sent over WebSocket include the layer index. Clients that don't +understand the new layer indices will ignore them (forward compatibility). +The browser visualizer currently renders the moisture heatmap — it doesn't render +the nutrient layer visually, so adding a fifth layer doesn't break the existing +client. + +If/when the visualizer shows nutrients, it should display `nutrients_fast + +nutrients_slow * 0.3` as an "effective fertility" value — weighted toward the +immediately available pool but acknowledging the reserve. + +--- + +## Engine Touchpoints (Every Place "nutrients" Is Currently Read/Written) + +These are the exact code locations that need to change. Each one is a +find-and-replace scoped to specific semantic meaning. + +### 1. Plant Dormancy Recovery Check + +```python +# BEFORE (engine.py, guard phase): +if soil_moisture > 0.25 and nutrients > 0.15: + # Plant recovers from dormancy + +# AFTER: +# Recovery requires BOTH some immediate nutrients AND some soil health. +# This means rain alone isn't sufficient for recovery if the soil is +# completely depleted — you need either time (dissolution from slow pool) +# or decomposer activity. +effective_nutrients = nutrients_fast + nutrients_slow * 0.3 +if soil_moisture > 0.25 and effective_nutrients > 0.15: + # Plant recovers from dormancy +``` + +**Why the weighted sum:** Pure fast-pool check would let rain alone trigger +recovery (which is the current behavior — preserve it for now). Pure slow-pool +check would make recovery impossibly slow. The weighted sum means: "rain helps +a lot, but a depleted soil can't fully recover from rain alone." + +The `0.3` weight on the slow pool means: +- Fast = 0.15 alone → recovery (rain was enough) +- Fast = 0.05, Slow = 0.33 → effective = 0.05 + 0.1 = 0.15 → recovery + (modest fast + decent soil health) +- Fast = 0, Slow = 0.3 → effective = 0.09 → no recovery + (soil has reserves but nothing plant-available yet) + +### 2. Plant Spreading Soil Check + +```python +# BEFORE: +if nutrients > threshold and moisture > threshold: + # Allow vegetative spreading + +# AFTER: +# Spreading only needs immediate nutrients — the plant is investing +# energy now, not building long-term reserves. +if nutrients_fast > threshold and moisture > threshold: + # Allow vegetative spreading +``` + +### 3. Rain Application (apply_rain) + +```python +# BEFORE: +nutrients += 0.024 * intensity + +# AFTER: +nutrients_fast += 0.020 * intensity # dissolved mineral input +nutrients_slow += 0.004 * intensity # particulate/sediment deposition +``` + +### 4. Voxel Effects Phase (Per-Tick Soil Processes) + +This is where the new inter-pool fluxes run. Add to the existing voxel +effects phase, after moisture updates: + +```python +# NEW: Nutrient pool dynamics (runs every tick for every active cell) +for cell in active_cells: + om = self.voxel_manager.get(cell, LAYER_ORGANIC_MATTER) + slow = self.voxel_manager.get(cell, LAYER_NUTRIENTS_SLOW) + fast = self.voxel_manager.get(cell, LAYER_NUTRIENTS_FAST) + + # 1. Mineralization: organic_matter → nutrients_slow + mineralized = om * MINERALIZATION_RATE + om -= mineralized + slow += mineralized + + # 2. Dissolution: nutrients_slow → nutrients_fast + dissolved = slow * DISSOLUTION_RATE + slow -= dissolved + fast += dissolved + + # 3. Leaching: nutrients_fast slowly drains + leached = fast * NUTRIENT_LEACH_RATE + fast -= leached + + # Clamp all to [0, 1] + self.voxel_manager.set(cell, LAYER_ORGANIC_MATTER, max(0, min(1, om))) + self.voxel_manager.set(cell, LAYER_NUTRIENTS_SLOW, max(0, min(1, slow))) + self.voxel_manager.set(cell, LAYER_NUTRIENTS_FAST, max(0, min(1, fast))) +``` + +### 5. Plant Growth Flow Phase + +```python +# BEFORE: +# Plant growth influenced by nutrients (exact code depends on engine.py) +growth_factor = nutrients * moisture * ... + +# AFTER: +# Growth draws from fast pool only +growth_factor = nutrients_fast * moisture * ... +# AND: successful growth depletes the fast pool +nutrients_fast -= growth_amount * UPTAKE_FRACTION +``` + +### 6. Entity Death (Removals Phase) + +```python +# BEFORE: +# Entity simply removed from entity list + +# AFTER: +# Deposit biomass into organic_matter layer +cell = self._grid_cell(entity["x"], entity["y"], entity["z"]) +deposit = self._biomass_deposit(entity) +current_om = self.voxel_manager.get(cell, LAYER_ORGANIC_MATTER) +self.voxel_manager.set(cell, LAYER_ORGANIC_MATTER, + min(1.0, current_om + deposit)) +# Then remove entity as before +``` + +```python +def _biomass_deposit(self, entity: dict) -> float: + """Convert entity to organic matter deposit amount.""" + # Phase 1 (pre-trait): use fixed values per entity type + DEPOSITS = { + "ANIMAL": 0.15, # deer-sized + "BIRD": 0.01, + "INSECT": 0.002, + "PLANT": 0.005, + "TREE": 0.4, + "MICROORGANISM": 0.001, + } + return DEPOSITS.get(entity.get("type", ""), 0.01) + + # Phase 2 (trait-based): derive from body_mass_kg + # return min(0.5, entity["body_mass_kg"] * BIOMASS_DEPOSIT_SCALE) +``` + +--- + +## Rate Constants Summary + +| Constant | Value | Unit | Ecological Meaning | +|----------|-------|------|-------------------| +| MINERALIZATION_RATE | 0.002 | per tick | organic_matter → nutrients_slow conversion | +| DISSOLUTION_RATE | 0.005 | per tick | nutrients_slow → nutrients_fast release | +| NUTRIENT_LEACH_RATE | 0.001 | per tick | nutrients_fast drainage | +| UPTAKE_FRACTION | 0.01 | per growth event | fast pool consumed by plant growth | +| BIOMASS_DEPOSIT_SCALE | 0.002 | per kg | body_mass_kg → organic_matter deposit | +| RAIN_FAST_FRACTION | 0.020 | per rain × intensity | rain → nutrients_fast | +| RAIN_SLOW_FRACTION | 0.004 | per rain × intensity | rain → nutrients_slow | +| INIT_FAST_RATIO | 0.4 | ratio | fraction of base nutrients → fast pool | +| INIT_SLOW_RATIO | 0.6 | ratio | fraction of base nutrients → slow pool | + +**Calibration strategy:** Run the existing demo_world for 2000 ticks with these +constants. Compare plant population dynamics, dormancy/recovery timing, and +post-rain behavior against the baseline from the single-pool engine. Adjust +DISSOLUTION_RATE and INIT ratios until the two-pool system produces dynamics +that feel qualitatively similar but with the new temporal separation visible +in extended runs (5000+ ticks). + +These constants should also be exposed as rate multipliers in the world JSON +(alongside the existing six rate multipliers) for stress testing: + +```json +{ + "rate_multipliers": { + "consumption": 1.0, + "hunger": 1.0, + "thirst": 1.0, + "growth": 1.0, + "reproduction": 1.0, + "water_replenishment": 1.0, + "mineralization": 1.0, + "dissolution": 1.0, + "nutrient_leaching": 1.0 + } +} +``` + +--- + +## Timescale Analysis + +At the default constants, here's what the pool dynamics look like over time: + +**Scenario 1: Rain event on depleted soil** +- Tick 0: nutrients_fast = 0, nutrients_slow = 0.1, organic_matter = 0 +- Rain (intensity 0.8): nutrients_fast jumps to 0.016, nutrients_slow to 0.103 +- Ticks 1-50: nutrients_fast slowly drains via leaching and plant uptake, + but dissolution from slow pool feeds it at ~0.0005/tick +- Tick 100: nutrients_fast ≈ 0.05 (enough for some growth, declining) +- Tick 200: nutrients_fast ≈ 0.02 (slow pool nearly drained too) +- **Result:** Brief green-up, then decline. Rain alone doesn't sustain. + +**Scenario 2: Healthy soil, no disturbance** +- Tick 0: nutrients_fast = 0.2, nutrients_slow = 0.3, organic_matter = 0.05 +- Per tick: dissolution adds ~0.0015 to fast, leaching removes ~0.0002, + mineralization adds ~0.0001 to slow from organic_matter +- Steady state: fast pool stays high (~0.18-0.22), slow pool slowly + declines but very gradually +- **Result:** Sustained growth. The slow pool acts as a battery. + +**Scenario 3: Overgrazing then recovery with decomposers (Phase 2)** +- Tick 0: Heavy grazing has depleted fast pool to 0.02, slow pool to 0.05 +- Ticks 1-200: Dead grass deposits organic_matter (OM accumulates to ~0.1) +- Without decomposers: OM → slow at 0.002/tick = 0.0002/tick. Very slow. + Recovery takes 1000+ ticks. +- With 3 decomposers nearby: OM → slow rate × 4 = 0.0008/tick. + Slow pool rebuilds to 0.15 by tick 200. Dissolution feeds fast pool. + Plants can recover by tick 300. +- **Result:** Decomposers cut recovery time by 3-4×. Mechanistically distinct + from rain. + +--- + +## Phase 2 Connection: Decomposer Integration + +When the mushroom species arrives in Phase 2, it connects to this system +through the interaction template: + +```python +class Decomposition(InteractionTemplate): + interaction_type = "decomposition" + + def matches(self, actor: TraitVector, target_cell: VoxelCell) -> bool: + """Decomposers 'interact' with voxel cells, not entities.""" + return (actor.diet_type == "decomposer" + and target_cell.organic_matter > 0.01) + + def apply(self, actor: TraitVector, cell: GridPosition, + voxel_manager: VoxelManager): + """Accelerate mineralization at this cell.""" + om = voxel_manager.get(cell, LAYER_ORGANIC_MATTER) + boost = actor.body_mass_kg * DECOMPOSER_METABOLIC_FACTOR + extra_mineralized = om * MINERALIZATION_RATE * boost + voxel_manager.add(cell, LAYER_NUTRIENTS_SLOW, extra_mineralized) + voxel_manager.add(cell, LAYER_ORGANIC_MATTER, -extra_mineralized) +``` + +Note that decomposition is unique among interaction templates: the "target" +is a voxel cell, not another entity. The decomposer senses high organic_matter +cells (like herbivores sense plants) and moves toward them. Its presence +accelerates the OM → slow pool conversion. This is the only interaction +template that operates on voxel state rather than entity state. + +The mushroom's trait vector drives this behavior: +- `diet_type: "decomposer"` → triggers decomposition template +- `diet_breadth: ["dead_organic_matter"]` → seeking behavior targets high-OM cells +- `body_mass_kg: 0.001` → small boost per individual, but they reproduce fast + (r_selected, clutch_size 5), so local clusters amplify the effect + +--- + +## ASAL Substrate Impact (Phase 3) + +The two-pool system adds three new dimensions to the θ parameter space: + +**EcoRates variant:** MINERALIZATION_RATE, DISSOLUTION_RATE, NUTRIENT_LEACH_RATE +as searchable parameters. "What soil chemistry turnover rates produce the most +open-ended ecosystem dynamics?" + +**EcoTopology variant:** Initial fast/slow ratio as a per-cell or per-biome +parameter. "What starting soil health distribution produces the most interesting +trophic cascades?" + +**Headless renderer:** The nutrient visualization becomes a two-channel signal. +Fast pool maps to brightness (immediate fertility visible to CLIP). Slow pool +maps to a subtle undertone (soil health visible over time). The FM can +distinguish "looks green because it rained" from "looks green because the soil +is healthy" — and these are ecologically different things. + +--- + +## Browser Visualizer Impact + +The current moisture heatmap (teal→amber) is the only voxel-layer visualization. +Nutrients aren't currently rendered. Two options: + +**Option A (minimal):** No change. The nutrient pools affect entity behavior +(plant growth, dormancy recovery) which the visualizer already shows through +plant appearance (size, color, dormancy markers). The pools are "felt, not seen." +Aligned with the "unseen hand" thesis. + +**Option B (if desired later):** Add a toggleable "soil health" overlay that +visualizes `nutrients_slow` as a subtle brown→dark green gradient underneath +the moisture heatmap. This would make long-term soil depletion visible before +plants go dormant — an early warning signal. + +**Recommendation:** Option A for now. The emergent plant behavior is the +visualization. Add Option B when/if the Godot client needs richer terrain shading. + +--- + +## Implementation Sequence + +This work slots into Phase 1 of the transition plan between Steps 1.5 and 1.6. +The trait compiler needs to know about two nutrient pools when deriving plant +growth thresholds. + +``` +Step 1.5a — Two-pool voxel refactor + 1. Update layer indices in voxel_manager.py (4 → 5 layers) + 2. Update initialize_from_soil with fast/slow split + 3. Add mineralization + dissolution + leaching to voxel effects phase + 4. Update rain application (split nutrient boost) + 5. Update dormancy recovery check (weighted effective nutrients) + 6. Update plant spreading soil check (fast pool only) + 7. Add biomass deposit on entity death + 8. Run existing test suite — all 12 tests must pass + 9. Run 2000-tick regression test — calibrate rate constants + +Step 1.5b — Rate multiplier exposure + 1. Add mineralization, dissolution, nutrient_leaching to rate_multipliers + 2. Update demo_world.json with defaults (1.0) + 3. Test: world JSON without new multipliers still works (defaults to 1.0) +``` + +**Time estimate:** The voxel refactor is 1-2 sessions of focused work. Most +changes are mechanical (layer index updates, splitting a single addition into +two additions). The calibration step is where iteration lives — getting the +rate constants to produce dynamics that feel right takes experimentation. + +--- + +## Test Plan + +### Unit Tests (test_ecosim.py additions) + +```python +def test_two_pool_initialization(): + """Verify both nutrient pools initialized from soil config.""" + vm = VoxelManager(grid_size=4) + vm.initialize_from_soil({"nutrients": 0.5, "moisture": 0.3}) + cell = (2, 2, 0) + assert abs(vm.get(cell, LAYER_NUTRIENTS_FAST) - 0.2) < 0.01 # 40% of 0.5 + assert abs(vm.get(cell, LAYER_NUTRIENTS_SLOW) - 0.3) < 0.01 # 60% of 0.5 + +def test_mineralization_flow(): + """Organic matter converts to slow nutrients over time.""" + vm = VoxelManager(grid_size=4) + cell = (2, 2, 0) + vm.set(cell, LAYER_ORGANIC_MATTER, 0.5) + vm.set(cell, LAYER_NUTRIENTS_SLOW, 0.0) + # Simulate 100 ticks of mineralization + for _ in range(100): + om = vm.get(cell, LAYER_ORGANIC_MATTER) + mineralized = om * MINERALIZATION_RATE + vm.set(cell, LAYER_ORGANIC_MATTER, om - mineralized) + vm.set(cell, LAYER_NUTRIENTS_SLOW, + vm.get(cell, LAYER_NUTRIENTS_SLOW) + mineralized) + # OM should have decayed, slow should have grown + assert vm.get(cell, LAYER_ORGANIC_MATTER) < 0.42 + assert vm.get(cell, LAYER_NUTRIENTS_SLOW) > 0.08 + +def test_dissolution_flow(): + """Slow nutrients dissolve into fast pool.""" + vm = VoxelManager(grid_size=4) + cell = (2, 2, 0) + vm.set(cell, LAYER_NUTRIENTS_SLOW, 0.5) + vm.set(cell, LAYER_NUTRIENTS_FAST, 0.0) + for _ in range(100): + slow = vm.get(cell, LAYER_NUTRIENTS_SLOW) + dissolved = slow * DISSOLUTION_RATE + vm.set(cell, LAYER_NUTRIENTS_SLOW, slow - dissolved) + vm.set(cell, LAYER_NUTRIENTS_FAST, + vm.get(cell, LAYER_NUTRIENTS_FAST) + dissolved) + assert vm.get(cell, LAYER_NUTRIENTS_FAST) > 0.15 + assert vm.get(cell, LAYER_NUTRIENTS_SLOW) < 0.35 + +def test_rain_splits_nutrients(): + """Rain adds to both pools with correct ratio.""" + vm = VoxelManager(grid_size=4) + cell = (2, 2, 0) + vm.set(cell, LAYER_NUTRIENTS_FAST, 0.0) + vm.set(cell, LAYER_NUTRIENTS_SLOW, 0.0) + # Simulate rain at intensity 1.0 + vm.set(cell, LAYER_NUTRIENTS_FAST, 0.020) + vm.set(cell, LAYER_NUTRIENTS_SLOW, 0.004) + assert abs(vm.get(cell, LAYER_NUTRIENTS_FAST) - 0.020) < 0.001 + assert abs(vm.get(cell, LAYER_NUTRIENTS_SLOW) - 0.004) < 0.001 + +def test_death_deposits_organic_matter(): + """Entity death adds biomass to organic_matter layer.""" + # Integration test with engine + ... + +def test_dormancy_recovery_effective_nutrients(): + """Dormancy recovery uses weighted sum of both pools.""" + fast = 0.10 + slow = 0.20 + effective = fast + slow * 0.3 # = 0.16 + assert effective > 0.15 # Should allow recovery + fast_only = 0.10 + effective_no_slow = fast_only + 0.0 * 0.3 # = 0.10 + assert effective_no_slow < 0.15 # Should NOT allow recovery +``` + +### Regression Test (test_regression.py) + +Run 2000-tick simulation with both single-pool (legacy) and two-pool engines. +Compare: +- Plant population count at ticks 500, 1000, 1500, 2000 (within ±15%) +- Number of dormancy → active transitions (within ±20%) +- Number of death events (within ±20%) +- Post-rain recovery timing: ticks from rain event to first dormancy recovery + (two-pool may be slightly faster due to fast pool, acceptable) + +--- + +## Backward Compatibility + +Worlds without the new rate multipliers use defaults (1.0 for all three new +constants). The only breaking change is the layer index shift: + +- Code that references `LAYER_NUTRIENTS` by name → update to `LAYER_NUTRIENTS_FAST` +- Code that references layers by numeric index (0, 1, 2, 3) → update to new indices +- Client code parsing delta packets with layer indices → forward-compatible if + it ignores unknown indices, breaking if it assumes exactly 4 layers + +**WebSocket protocol:** The delta packet format includes layer index as an integer. +Existing clients seeing layer index 4 (organic_matter, unchanged) are fine. +New layer index 1 (nutrients_slow) might be unexpected. The browser visualizer +doesn't render nutrient layers, so it ignores these deltas. The Godot client +(not yet built) will be designed for 5 layers from the start. From 774dfb7c41ab0bfbfa9e6e9142c4ee090e9dab26 Mon Sep 17 00:00:00 2001 From: Joshua Natarajan Date: Sun, 10 May 2026 19:43:49 -0400 Subject: [PATCH 2/2] doc cleanup --- docs/LILA_PROJECT_STATE.md | 23 +++++++++++++++-------- docs/TRAIT_TRANSITION_PLAN.md | 7 +++++++ docs/TWO_POOL_NUTRIENT_SPEC.md | 31 +++++++++++++++++++------------ 3 files changed, 41 insertions(+), 20 deletions(-) diff --git a/docs/LILA_PROJECT_STATE.md b/docs/LILA_PROJECT_STATE.md index 7fc902a..0044664 100755 --- a/docs/LILA_PROJECT_STATE.md +++ b/docs/LILA_PROJECT_STATE.md @@ -1,3 +1,10 @@ + + # līlā — Project State (v0.0.1-alpha) ## Current Status @@ -25,7 +32,7 @@ The name comes from the Sanskrit concept of [līlā](https://www.embodiedphiloso ┌─────────────────────────┐ │ Browser Visualizer │ ← v0.0.1-alpha (shipped, single HTML file) │ Godot 4.x Client │ ← deferred to Milestone 4 -│ Headless Renderer │ ← Milestone 3 (for ASAL search) +│ Headless Renderer │ ← Milestone 3 (for ASAL search) └──────────┬──────────────┘ │ WebSocket (delta-encoded tick packets) ┌──────────▼──────────────┐ @@ -33,24 +40,24 @@ The name comes from the Sanskrit concept of [līlā](https://www.embodiedphiloso │ (one per session) │ Serves viz HTML, streams ticks └──────────┬──────────────┘ │ -┌──────────▼──────────────────────────────────────────┐ +┌──────────▼───────────────────────────────────────────┐ │ ecosim (Python package, stdlib only) │ -│ ┌────────────────┐ ┌────────────────────────────┐ │ +│ ┌─────────────────┐ ┌────────────────────────────┐ │ │ │ Hybrid Automaton│ │ Trait System (Milestone 2) │ │ │ │ Flow + Guards │ │ TraitVector + Compiler │ │ -│ ├────────────────┤ │ Allometric Derivations │ │ +│ ├─────────────────┤ │ Allometric Derivations │ │ │ │ Voxel Manager │ │ Interaction Templates │ │ │ │ 5 layers (M2) │ ├────────────────────────────┤ │ -│ ├────────────────┤ │ BYOM Adapters │ │ +│ ├─────────────────┤ │ BYOM Adapters │ │ │ │ Water System │ │ mlp/static/random │ │ │ │ Dynamic levels │ ├────────────────────────────┤ │ -│ ├────────────────┤ │ World Randomizer │ │ +│ ├─────────────────┤ │ World Randomizer │ │ │ │ Two-Pool Soil │ │ D4 transforms │ │ │ │ Fast/Slow (M2) │ └────────────────────────────┘ │ -│ └────────────────┘ │ +│ └─────────────────┘ │ └──────────────────────────────────────────────────────┘ │ -┌──────────▼──────────────────────────────────────────┐ +┌──────────▼───────────────────────────────────────────┐ │ search/ (Milestone 3, separate package) │ │ ASAL Substrate Protocol (Init/Step/Render) │ │ FM Evaluator (CLIP/DINOv2) │ diff --git a/docs/TRAIT_TRANSITION_PLAN.md b/docs/TRAIT_TRANSITION_PLAN.md index 037c0d0..ffc9148 100644 --- a/docs/TRAIT_TRANSITION_PLAN.md +++ b/docs/TRAIT_TRANSITION_PLAN.md @@ -1,3 +1,10 @@ + + # līlā — Trait-Based Architecture Transition Plan > From hand-crafted species rules to allometrically-derived ecological dynamics, diff --git a/docs/TWO_POOL_NUTRIENT_SPEC.md b/docs/TWO_POOL_NUTRIENT_SPEC.md index 00fffec..78fa84c 100644 --- a/docs/TWO_POOL_NUTRIENT_SPEC.md +++ b/docs/TWO_POOL_NUTRIENT_SPEC.md @@ -1,3 +1,10 @@ + + # līlā — Two-Pool Nutrient Split: Implementation Spec > **Decision:** Split the single `nutrients` voxel layer into `nutrients_fast` and @@ -14,36 +21,36 @@ ## Conceptual Model ``` - ┌─────────────────────┐ + ┌──────────────────────┐ │ Dead Entities │ │ (animal/plant) │ - └──────────┬──────────┘ + └──────────┬───────────┘ │ death event: +biomass ▼ - ┌─────────────────────┐ + ┌──────────────────────┐ │ organic_matter │ Layer 5 (existing, unchanged) │ (dead biomass) │ Slow decay, spatial - └──────────┬──────────┘ + └──────────┬───────────┘ │ decomposition rate (accelerated by decomposers) ▼ -┌──────────┐ ┌─────────────────────┐ -│ Rain │──────▶│ nutrients_slow │ Layer 2 (NEW) +┌──────────┐ ┌──────────────────────┐ +│ Rain │─────> │ nutrients_slow │ Layer 2 (NEW) │ (mineral │ │ (mineralized pool) │ Stable, slow-release │ input) │ │ Long-term soil │ Represents soil health └──────────┘ │ health indicator │ - │ └──────────┬──────────┘ + │ └──────────┬───────────┘ │ │ dissolution rate │ ▼ │ ┌─────────────────────┐ - └─────────────▶│ nutrients_fast │ Layer 1 (replaces old "nutrients") - │ (plant-available) │ Quick turnover - │ Dissolved, labile │ Plants consume from here + └─────────────>│ nutrients_fast │ Layer 1 (replaces old "nutrients") + │ (plant-available) │ Quick turnover + │ Dissolved, labile │ Plants consume from here └──────────┬──────────┘ │ plant uptake ▼ ┌─────────────────────┐ - │ Plant Growth │ - │ (health, growth) │ + │ Plant Growth │ + │ (health, growth) │ └─────────────────────┘ ```