Redesign terminal output system with composable, non-allocating architecture

MIGRATION_PLAN.md

@@ -0,0 +1,186 @@
+# Migration Plan: Terminal Output Redesign
+
+## Overview
+
+This document outlines the migration plan from the current messy terminal output system (`treenest.zig`, `dank.zig`, `AnsiWriter.zig`) to the new clean, composable architecture.
+
+## Current Usage Analysis
+
+Based on code analysis, the current system is used in:
+
+### 1. Tracing/Logging (`src/layout.zig`, `src/dom.zig`, etc.)
+- **Current**: `Trace = @import("ansi").FileTrace`
+- **Pattern**: Hierarchical debug output with `trace.enter()`, `trace.exit()`, `trace.put()`, `trace.fields()`
+- **Volume**: Heavy usage across layout engine, DOM manipulation
+
+### 2. Test Runner (`src/lib/test_runner.zig`)
+- **Current**: Complex usage of `TreeNest` and `Dank` for formatted test output
+- **Pattern**: Test results, stack traces, progress indicators, colored output
+- **Volume**: Moderate usage in test infrastructure
+
+### 3. Stack Trace Dumping (`src/lib/libansi.zig`)
+- **Current**: `dumpConciseStackTrace()` using `tree.dk()` methods
+- **Pattern**: Source code display, file paths, syntax highlighting
+- **Volume**: Error handling paths
+
+### 4. General Debug Output (various files)
+- **Current**: Ad-hoc usage of `tree.info()`, `tree.note()`, etc.
+- **Pattern**: Simple styled output for debugging
+- **Volume**: Light usage scattered throughout
+
+## Migration Strategy
+
+### Phase 1: Foundation (Week 1)
+**Goal**: Establish new system alongside old system
+
+1. **Create new terminal module** ✅
+   - [x] `src/lib/terminal/` directory structure
+   - [x] Core components (`AnsiStreamer`, `TreeFormatter`, `StyleApplier`, `BufferedTerminal`)
+   - [x] Main module with convenience functions and helpers
+   - [x] Comprehensive test coverage
+
+2. **Integration with existing build system**
+   - [ ] Update `build.zig` to include new terminal module
+   - [ ] Add new module to `src/lib/libansi.zig` exports
+   - [ ] Ensure compatibility with existing imports
+
+### Phase 2: New Tracing API (Week 2)
+**Goal**: Create drop-in replacement for tracing functionality
+
+1. **Create trace wrapper**
+   ```zig
+   // src/lib/terminal/TraceWrapper.zig
+   pub fn TraceWrapper(comptime Writer: type) type {
+       return struct {
+           terminal: BufferedTerminal(Writer, 4096),
+
+           pub fn enter(self: *Self) !void { try self.terminal.enterTree(); }
+           pub fn exit(self: *Self) void { self.terminal.exitTree(); }
+           pub fn put(self: *Self, comptime key: []const u8, value: anytype) !void {
+               // Format key-value pair and write as tree line
+           }
+           pub fn fields(self: *Self, comptime label: []const u8, data: anytype) !void {
+               // Write structured data section
+           }
+       };
+   }
+   ```
+
+2. **Update import aliases**
+   ```zig
+   // src/lib/libansi.zig
+   pub const FileTrace = terminal.TraceWrapper(std.fs.File.Writer);
+   // Keep old aliases working during migration
+   pub const nest = @import("treenest.zig"); // deprecated
+   ```
+
+### Phase 3: Test Runner Migration (Week 3)  
+**Goal**: Convert test runner to new API
+
+1. **Create test output helpers**
+   ```zig
+   // src/lib/terminal/TestHelpers.zig
+   pub fn writeTestSuite(terminal: anytype, name: []const u8) !void { ... }
+   pub fn writeTestCase(terminal: anytype, name: []const u8, result: TestResult) !void { ... }
+   pub fn writeStackTrace(terminal: anytype, stack_trace: std.builtin.StackTrace) !void { ... }
+   ```
+
+2. **Update test runner**
+   - Replace `TreeNest` usage with `BufferedTerminal`
+   - Replace `Dank` helpers with new helper functions  
+   - Maintain exact same output format for compatibility
+
+### Phase 4: Stack Trace Migration (Week 4)
+**Goal**: Convert stack trace dumping
+
+1. **Create stack trace helpers**
+   ```zig
+   // src/lib/terminal/StackTraceHelpers.zig  
+   pub fn writeSourceBlock(terminal: anytype, code: []const u8, highlight_line: ?u64, highlight_col: ?u64) !void { ... }
+   pub fn writeStackFrame(terminal: anytype, file: []const u8, line: u64, col: u64, func: []const u8) !void { ... }
+   ```
+
+2. **Update libansi.zig functions**
+   - Convert `dumpConciseStackTrace()` to use new API
+   - Maintain visual output format
+
+### Phase 5: General Usage Migration (Week 5)
+**Goal**: Convert remaining usage sites
+
+1. **Create migration helpers**
+   ```zig
+   // Temporary compatibility shims
+   pub fn legacyInfo(terminal: anytype, message: []const u8) !void {
+       try terminal.helpers.writeInfo(terminal, message);
+   }
+   ```
+
+2. **Convert each usage site**
+   - Replace `trace.info()`, `trace.note()` calls
+   - Update import statements  
+   - Test each conversion
+
+### Phase 6: Cleanup (Week 6)
+**Goal**: Remove old system
+
+1. **Remove deprecated files**
+   - [ ] Delete `src/lib/treenest.zig` (650 lines removed)
+   - [ ] Delete `src/lib/dank.zig` (471 lines removed)  
+   - [ ] Update `src/lib/AnsiWriter.zig` (remove redundant parts)
+
+2. **Clean up imports**
+   - Remove deprecated exports from `libansi.zig`
+   - Update all import statements
+   - Remove compatibility shims
+
+3. **Final testing**
+   - Run full test suite
+   - Visual regression testing for output formats
+   - Performance benchmarking
+
+## Risk Mitigation
+
+### Compatibility Risks
+- **Risk**: Output format changes break automation/scripts
+- **Mitigation**: Exact format preservation during migration, extensive testing
+
+### Performance Risks  
+- **Risk**: New system slower than old system
+- **Mitigation**: Benchmarking at each phase, optimize BufferedTerminal buffer sizes
+
+### Integration Risks
+- **Risk**: Build system issues with new module structure
+- **Mitigation**: Gradual integration, maintain old system until fully migrated
+
+## Success Criteria
+
+1. **Code Quality**
+   - [ ] Reduce total lines of code by >50% (current: ~1400 lines, target: <700)
+   - [ ] All components have >90% test coverage
+   - [ ] Zero allocation in core hot paths (tracing)
+
+2. **API Quality**
+   - [ ] Simple APIs: core components have <10 public methods each
+   - [ ] Composable: can mix and match components easily
+   - [ ] Consistent error handling throughout
+
+3. **Performance**
+   - [ ] No performance regression in tracing benchmarks
+   - [ ] Memory usage reduced (no allocations in core paths)
+   - [ ] Build time not significantly impacted
+
+4. **Maintainability**
+   - [ ] Clear separation of concerns
+   - [ ] Easy to add new domain-specific helpers
+   - [ ] Comprehensive documentation and examples
+
+## Timeline Summary
+
+- **Week 1**: Foundation established ✅
+- **Week 2**: Tracing API replacement
+- **Week 3**: Test runner migration  
+- **Week 4**: Stack trace migration
+- **Week 5**: General usage migration
+- **Week 6**: Cleanup and removal of old system
+
+Total effort: ~6 weeks for complete migration while maintaining compatibility.

TERMINAL_OUTPUT_REDESIGN.md

@@ -0,0 +1,182 @@
+# Terminal Output Redesign Specification
+
+## Problem Analysis
+
+The current terminal output system (`treenest.zig`, `dank.zig`, `AnsiWriter.zig`) has several issues:
+
+### Current Issues
+- **Memory allocation**: Components allocate memory for formatting operations
+- **Mixed responsibilities**: Tree structure, styling, ANSI output, and domain formatting intertwined
+- **Complex APIs**: Many methods with overlapping functionality  
+- **Inconsistent error handling**: Some methods ignore errors, others propagate
+- **Hard to compose**: Monolithic components that try to do everything
+- **Not testable**: Complex state makes unit testing difficult
+
+### Usage Patterns
+- Tracing/logging with hierarchical structure
+- Test runner output with formatted results
+- Stack trace dumping with source code snippets
+- Debug output with tree-like visualization
+
+## New Design Principles
+
+1. **Separation of Concerns**: Separate tree structure, styling, and ANSI output
+2. **Non-allocating**: Use stack buffers, streaming, or caller-provided buffers
+3. **Composable**: Small, focused components that can be combined
+4. **Simple APIs**: Few methods with clear responsibilities
+5. **Correct Error Handling**: Consistent error propagation throughout
+6. **Zero-cost Abstractions**: No runtime overhead for unused features
+
+## Core Architecture
+
+### 1. AnsiStreamer - ANSI Escape Sequence Generation
+**Responsibility**: Generate ANSI escape sequences without allocation
+
+```zig
+pub const AnsiStreamer = struct {
+    no_color: bool = false,
+
+    // Non-allocating methods that write directly to provided writer
+    pub fn resetStyle(writer: anytype) !void
+    pub fn setForegroundRgb(writer: anytype, r: u8, g: u8, b: u8) !void
+    pub fn setBold(writer: anytype) !void
+    pub fn moveCursor(writer: anytype, row: u32, col: u32) !void
+    // ... other ANSI operations
+};
+```
+
+### 2. TreeFormatter - Tree Structure Rendering  
+**Responsibility**: Format hierarchical tree structures without allocation
+
+```zig
+pub const TreeFormatter = struct {
+    depth: u8 = 0,
+    stack: [32]LevelState, // Fixed-size stack
+
+    pub const LevelState = struct { has_more: bool };
+
+    pub fn enter(self: *Self) !void
+    pub fn exit(self: *Self) !void
+    pub fn writeIndent(self: *Self, writer: anytype, is_last: bool) !void
+    pub fn writeContinuation(self: *Self, writer: anytype) !void
+};
+```
+
+### 3. StyleApplier - Style Management
+**Responsibility**: Apply styles without allocation using streaming approach
+
+```zig
+pub const Style = packed struct {
+    fg_r: u8 = 0, fg_g: u8 = 0, fg_b: u8 = 0, has_fg: bool = false,
+    bg_r: u8 = 0, bg_g: u8 = 0, bg_b: u8 = 0, has_bg: bool = false,  
+    bold: bool = false,
+    italic: bool = false,
+    underline: bool = false,
+};
+
+pub const StyleApplier = struct {
+    current: Style = .{},
+
+    pub fn apply(self: *Self, writer: anytype, ansi: *AnsiStreamer, new_style: Style) !void
+    pub fn reset(self: *Self, writer: anytype, ansi: *AnsiStreamer) !void
+};
+```
+
+### 4. BufferedTerminal - Output Coordination
+**Responsibility**: Coordinate the other components with optional buffering
+
+```zig
+pub fn BufferedTerminal(comptime Writer: type, comptime buffer_size: usize) type {
+    return struct {
+        writer: Writer,
+        buffer: [buffer_size]u8,
+        pos: usize = 0,
+
+        ansi: AnsiStreamer,
+        tree: TreeFormatter,
+        style: StyleApplier,
+
+        pub fn init(writer: Writer, no_color: bool) Self
+        pub fn flush(self: *Self) !void
+        pub fn writeStyled(self: *Self, text: []const u8, style: Style) !void
+        pub fn enterTree(self: *Self) !void
+        pub fn exitTree(self: *Self) !void
+        pub fn writeTreeLine(self: *Self, text: []const u8, style: Style, is_last: bool) !void
+    };
+}
+```
+
+## API Usage Examples
+
+### Basic Styled Output
+```zig
+var terminal = BufferedTerminal(File.Writer, 4096).init(stdout.writer(), false);
+
+// Simple styled text
+try terminal.writeStyled("Hello", .{ .fg_r = 255, .has_fg = true, .bold = true });
+
+// Tree structure  
+try terminal.enterTree();
+try terminal.writeTreeLine("Root", .{}, false);
+try terminal.enterTree();
+try terminal.writeTreeLine("Child 1", .{ .fg_g = 200, .has_fg = true }, false);
+try terminal.writeTreeLine("Child 2", .{ .fg_g = 200, .has_fg = true }, true);
+try terminal.exitTree();
+try terminal.exitTree();
+```
+
+### Domain-Specific Extensions
+```zig
+// Extensions built on top of core components
+pub fn writeTestResult(terminal: anytype, name: []const u8, passed: bool) !void {
+    const style = if (passed) 
+        Style{ .fg_g = 200, .has_fg = true } 
+    else 
+        Style{ .fg_r = 200, .has_fg = true };
+    const icon = if (passed) "✓" else "✗";
+
+    try terminal.writeStyled(icon, style);
+    try terminal.writeStyled(" ", .{});
+    try terminal.writeStyled(name, .{});
+}
+```
+
+## Migration Strategy
+
+### Phase 1: Core Components
+1. Implement `AnsiStreamer` 
+2. Implement `TreeFormatter`
+3. Implement `StyleApplier`
+4. Implement `BufferedTerminal`
+5. Write comprehensive tests
+
+### Phase 2: Domain Extensions
+1. Create domain-specific helper functions (test output, tracing, etc.)
+2. Port existing `dank.zig` functionality as composable functions
+
+### Phase 3: Incremental Migration  
+1. Update `libansi.zig` to expose new API alongside old
+2. Port test runner to new API
+3. Port tracing usage sites  
+4. Remove old components
+
+## Benefits
+
+1. **Memory Efficient**: No allocations in core paths
+2. **Composable**: Components can be mixed and matched
+3. **Testable**: Simple, focused components are easy to test
+4. **Fast**: Direct streaming with minimal overhead
+5. **Flexible**: Easy to extend with domain-specific functionality
+6. **Maintainable**: Clear separation of concerns
+
+## File Structure
+
+```
+src/lib/terminal/
+├── AnsiStreamer.zig      # ANSI escape sequences
+├── TreeFormatter.zig     # Tree structure rendering  
+├── StyleApplier.zig      # Style management
+├── BufferedTerminal.zig  # Coordinating component
+├── helpers.zig           # Domain-specific helpers
+└── terminal.zig          # Main module exports
+```