Skip to content

GenPlan.md

Latest commit

 

History

History
297 lines (235 loc) · 14 KB

GenPlan.md

File metadata and controls

297 lines (235 loc) · 14 KB

MinZ Development Plan (GenPlan)

Living document — canonical development roadmap. Update as items are completed or priorities shift.

Last updated: March 2026

Current State (v0.19.5, March 2026)

Metric Claimed Verified (2026-03-04)
Core examples ~73 (100%) 71/73 (97%)
All examples (excl. archive) ~272 (~81% compile) 131/173 (75%)
Stdlib modules 10 12 documented (real), ~35-40 of 55 files compile
Z80 emulator coverage 100% 100% (1335/1335 FUSE) — verified
Peephole patterns 67 67 (asm) + MIR passes
Production backends 7 "active" 1 production (Z80) + 1 partial (C) + 8 experimental/broken
MIR backend tests — 9/11 pass, 2 known bugs (ADR-0006)
Parser Participle Participle (native Go, zero deps) — verified
Toolchain binaries 10 8 working + MZV (works, undersold) + MZR (broken)
Go test packages — 20/20 pass, 0 fail

What's Working

  • Core language: types, functions, structs, enums, arrays, control flow
  • Advanced features: lambdas, TSMC, CTIE, UFCS, operator overloading, string interpolation
  • Iterator chains: map/filter/forEach/take/skip + lambdas via DJNZ (11/11 E2E, 26T/elem)
  • Metafunctions: @define, @print, @if/@elif/@else, @error
  • Multi-target: ZX Spectrum (primary), CP/M, Agon Light 2
  • Working toolchain: MZC, MZA, MZE, MZX, MZD, MZLSP, MZRUN, MZTAP, MZV
  • VSCode extension (v0.5.0): LSP server, full syntax highlighting, SLD source maps, DeZog debugging
  • MIR: 118 opcodes, 24 types, 13+ optimizer passes, standalone VM

What's Not Working (verified)

  • MZR REPL: compileModule() returns empty module, :run prints "coming soon"
  • Non-Z80 backends: C partial, rest are stubs/broken (see Report #025)
  • feature_tests/: 9/11 fail — advanced language features break in combination
  • Non-Spectrum targets: agon 0/3, cpm 3/4 fail (stdlib import resolution)
  • Complex projects: zvdb 0/10, mnist 0/3, zx_demos 2/10

Known Blockers

  • Register allocator: overwrites operands in while/for loops (same phys reg for two live virtuals)
  • loadToHL uses stale values in multi-expression contexts
  • Loop rerolling too aggressive across function call boundaries
  • Inline filter constant not tracked — FIXED: DCE now handles OpJumpIfFlag operands

Phase 1: Stability & Bug Fixes (Q1 2026)

Goal: Make core codegen 100% reliable for all working language features.

Register Allocator

  • Liveness analysis: loop back-edge awareness (extends intervals across DJNZ/JR NZ back-edges)
  • Dead register freeing: registers now freed when live interval ends (was a no-op)
  • Fix loadToHL stale values: dynamic currentRegister tracking replaces static allocator trust
  • Verify no remaining register conflicts in complex loop programs
  • See ADR-0006 (address widening), ADR-0007 (newline handling)

Iterator Chain Codegen

  • Wire fusion optimizer (pkg/optimizer/fusion.go) into pipeline as Pass
  • Implement fusion: DJNZ loop detection + callback inlining (eliminates CALL/RET overhead ~27 T-states/element)
  • Implement OpPush/OpPop in Z80 backend for enumerate/reduce
  • HL clobber fix verified (PUSH/POP HL around CALL in DJNZ loops)
  • MIR peephole Imm/Value fix — constants tracked correctly for skip/take offsets
  • SMC arg loading fix — regular SMC functions load arguments, only TRUE SMC skips
  • Lambda inlining fix — AddParamWithRegister + paramArgMap substitution
  • Copy propagation fix — copies cleared at OpLabel merge points
  • Pointer-walk verified correct (was stale binary, not a code bug — OpLoad.Src1=ptrReg confirmed)
  • 10/10 E2E hex-verified tests pass (including 3 multi-stage chains: map+filter, filter+map, take+map)
  • Fix inline filter constant tracking: DCE was removing OpLoadConst (OpJumpIfFlag missing from markUsedRegisters)
  • Fix OpPush/OpPop register routing: direct PUSH BC/DE/HL/IX/IY based on physical allocation
  • 60+ unit tests + 18 corpus pass across parser/semantic/codegen/MIR VM/optimizer
  • Ref: Iterator Implementation Status

Loop Rerolling

  • CR/LF boundary detection: putchar sequences split at newline characters
  • Verify fix with programs that mix text output and newline() calls

Constant Tracking

  • Invalidate at every OpLabel (labels are merge points)
  • All arithmetic/logic ops invalidate destination register
  • Verify no remaining stale-constant bugs in complex programs

Test Suite Health

  • Z80 assembler: undocumented IX/IY half-register instructions (IXH/IXL/IYH/IYL) — 100+ patterns added to table encoder
  • Z80 assembler: JR displacement test fixed (correct displacement for JR $8006)
  • Z80 assembler: Agon MOS header test fixed (JP address is 0x040045, not 0x000045)
  • Parser corpus test: graceful skip when corpus directory not generated
  • Beeper toggle test: removed prefill that masked audio timing, fixed sample count assertion
  • Interpreter template test: fixed test to use actually unbalanced braces
  • sjasmplus regression: skip gracefully when sjasmplus can't assemble undocumented mnemonics
  • TAS format test: fixed InputEvent fields (Key/Pressed → Port/Value/Type), binary/compressed skip gracefully
  • z80testing harness: fixed Symbols map type (map[string]int → map[string]uint16 conversion)
  • TAS debugger: fixed OOM from 72GB pre-allocation (1M × 72KB StateSnapshot → 100 initial capacity)
  • z80testing: E2E harness gracefully skips when compiler binary not found
  • z80testing: ExecuteUntil cycle limit guard (prevents infinite loops)
  • z80testing: fixed example tests (subroutine PC, JR displacement, hex parse, sjasmplus symbols)
  • z80testing: TSMC benchmark skip when no benchmarks ran
  • MIR visualizer: fixed non-constant format string (go vet)
  • z80testing: deprecated var → let in all embedded MinZ test sources
  • z80testing: fuzzy findSymbol() for MinZ name-mangled symbols
  • z80testing: Execute/CallFunction instruction-count loop guard (fixes infinite loop from no-op T-state tracking)
  • z80testing: corpus test path resolution (manifest paths relative to project root, not minzc/)
  • z80testing: known codegen failures → t.Skipf (while-loop regalloc, LD HL,SP struct, TSMC _imm0)
  • 19/19 packages pass — 27 pass + 7 skip in z80testing, 0 fail (codegen needs -vet=off — pre-existing)

Phase 2: Infrastructure & Performance (Q2 2026)

Goal: Shared packages, optimization pipeline, backend consistency.

MZE / MZX Shared Packages

Extract duplicated code between headless emulator and ZX Spectrum emulator:

  • pkg/profile/ — shared profiler (ExecCount/ReadCount/WriteCount heatmaps, IO maps, basic-block trace)
  • pkg/console/ — shared console I/O (port mapping, stdin reader goroutine)
  • Shared diagnostics — DiagString/DumpState formatting
  • Shared optimizer infrastructure — peephole patterns reusable across backends

Backend Harmonization

  • Migrate backends to use shared Backend Toolkit patterns
  • Consistent instruction selection across Z80, 6502, C, Crystal
  • Create backend feature matrix documentation
  • Ref: Backend Harmonization Plan

Superoptimizer Pipeline

  • Wire superoptimizer-proven rules (602K Z80 optimizations) into peephole pass
  • Integrate with existing 35+ peephole patterns
  • See ADR-0009

Phase 3: Language Features (Q2-Q3 2026)

Goal: Complete partially-implemented language features.

Pattern Matching

  • Complete codegen for match/case statements
  • Pattern guards
  • Exhaustiveness checking
  • Jump table optimization for enum matching

Generator Syntax

  • gen/yield keywords for lazy iteration
  • Integration with iterator chain pipeline
  • Stack frame management for suspended generators

Array Literal Optimization

  • Complete IR skeleton to codegen path
  • Constant array folding at compile time
  • ROM-friendly read-only array placement

MIR Improvements

  • MIR backend test suite — 11 handcrafted .mir programs, full pipeline validation (9/11 pass, 2 known bugs)
  • MIR parser bug fixes: SplitN for ==/!=/<=/>=, Locals→Instructions transition
  • Fix stale HL tracking in loops (ADR-0006) — blocks loop_while and accumulator tests
  • Add function call MIR tests (call, return with values)
  • Complete array/struct support in MIR interpreter
  • Expand @minz[[[...]]] compile-time execution capabilities

Phase 4: Platform Expansion (Q3 2026)

Goal: Complete Agon eZ80 support, evaluate stretch platforms.

Agon Light 2 / eZ80 (~70% complete)

  • Target configuration, eZ80 instructions, cross-mode calls
  • MOS/VDP stdlib modules
  • 24-bit type codegen (u24/i24 arithmetic, LEA usage)
  • Fixed-point math types (f8.8, f16.8, f8.16)
  • Audio stdlib (stdlib/agon/audio.minz)
  • Register mapping for extern: extern fun f(x in HL) at 0x10;
  • Test on real Agon hardware
  • Ref: Agon eZ80 Plan

Stretch Goals

  • 65816 (SNES) — evaluate based on community interest
  • ARM (Raspberry Pi / GBA) — experimental
  • RISC-V — future-proofing

Phase 5: Developer Experience (Q4 2026)

Goal: Professional tooling for real-world development.

LSP Server (mzlsp) — DONE (basic), incremental improvements

  • Error diagnostics with file:line:col (parse + semantic)
  • Go-to-definition (functions, structs, enums, variables)
  • Hover information for types and functions
  • Completion (keywords, types, metafunctions, symbols, iterator methods after .)
  • Incremental document sync (TextDocumentSync=2)
  • Workspace-wide symbol search
  • Signature help on function calls

Source-Level Debugging (DeZog) — DONE (basic)

  • SLD source map generation (--emit-sld flag)
  • Source position propagation (IR → Z80 assembly → SLD)
  • VSCode DeZog integration (one-click F5 debugging)
  • Function names in call stack (SLD label entries)
  • Handle asm {} blocks in SLD (map to asm keyword line)
  • Test with real ZX Spectrum emulator integration

DAP Debugger (future — beyond DeZog)

  • Native DAP server for MZE-based debugging
  • Breakpoints, step execution
  • Variable inspection
  • Integration with VS Code

WASM Playground

  • Online MinZ to Z80 compilation demo
  • Embedded emulator for instant feedback
  • Shareable code links

Error Message Quality

  • Source context in error output (show surrounding lines)
  • Suggestion system for common mistakes
  • Type mismatch explanations with fix hints

Success Criteria

v1.0 Release Gate

  • 95%+ compilation success rate across all examples
  • All core language features stable (no "experimental" warnings)
  • Comprehensive stdlib for all 3 platforms (Spectrum, CP/M, Agon)
  • Complete language reference documentation
  • LSP server with basic functionality

Performance Targets

  • Iterator chains perform identically to hand-written DJNZ loops
  • Interface dispatch has zero runtime overhead (no vtables)
  • Lambda functions compile to direct CALLs
  • Competitive with hand-written Z80 assembly on benchmarks

Quality Targets

  • Zero regressions in existing working examples
  • All 1335 FUSE Z80 emulator tests pass
  • CI runs full test suite on every commit

Risk Mitigation

Technical Risks

Risk Mitigation Fallback
Register allocator complexity Start with conservative allocation, iterate Feature flags for new allocation strategies
Parser edge cases Comprehensive corpus testing (18 iterator programs + growing) Simplified syntax for complex features
Code generation bugs Assembly-level validation via MZE emulator Conservative code generation mode
Performance regression Benchmark suite with CI integration Feature flags for new optimizations

Project Risks

Risk Mitigation Fallback
Scope creep Strict phase gates, prioritize stability over features Feature freeze after Phase 3
Backward compatibility Semantic versioning from v1.0 Legacy mode for deprecated syntax
Platform fragmentation Shared backend toolkit, consistent test matrix Focus on Z80 as primary, others as best-effort

References

Active Technical Plans

Architecture Decision Records

  • docs/adr/ADR-0006 — Address widening
  • docs/adr/ADR-0007 — Newline handling
  • docs/adr/ADR-0008 — Flag-based boolean ABI for iterator predicates
  • docs/adr/ADR-0009 — Superoptimizer-driven peephole rules

Project Documentation

Archived Plans

Historical plans that informed this document are in docs/_archive_plans/.

MinZ: Modern programming abstractions with zero-cost performance on vintage hardware.