Skip to content

Latest commit

 

History

History
347 lines (297 loc) · 14.6 KB

File metadata and controls

347 lines (297 loc) · 14.6 KB

MAINFRAME Roadmap

The AI-Native Bash Runtime - V10.0

This roadmap outlines planned features and improvements for MAINFRAME. Items are organized by priority and status.

Current Stats: 6,200+ functions | 177 libraries | 11,300+ tests | Pure Bash + Bindings


In Progress

V10.1 - Enhanced AI Integration (Current)

  • Function Discovery CLI (implemented)
    • mainframe fzf - Interactive fuzzy search via fzf
    • mainframe fzf <library> - Filter by library
    • mainframe explore - TUI browser via dialog/whiptail
    • Preview pane with function signatures
  • Security hardening of eval sites (8 critical, 4 high, 3 medium fixed)
    • events.sh - Callback validation, subprocess isolation
    • pipe.sh - Command validation, pattern blocking
    • streams.sh - Expression subprocess isolation
    • testing.sh - Variable/function name validation, printf -v
    • idempotent.sh - bash -c isolation for check commands
  • TOCTOU race condition fixes in atomic operations
    • ensure_line - flock-based atomic check-and-append
    • ensure_symlink - temp-link + rename pattern
    • ensure_file - temp-file + atomic rename
  • Telemetry System (lib/telemetry.sh)
    • Opt-in with MAINFRAME_TELEMETRY=1
    • Local aggregation to ~/.mainframe/telemetry/
    • Functions: telemetry_enabled, telemetry_init, telemetry_track, telemetry_flush, telemetry_report
    • No PII collection, privacy-first design
  • New ext/ Libraries (6 libraries, 62 functions)
    • ext/go.sh - Go project analysis (10 functions)
    • ext/rust.sh - Rust/Cargo project analysis (10 functions)
    • ext/terraform.sh - Terraform CLI wrapper (12 functions)
    • ext/aws.sh - AWS CLI wrapper with structured output (10 functions)
    • ext/gcp.sh - GCP CLI wrapper with gsutil fallback (10 functions)
    • All USOP-compliant with comprehensive tests
  • Language Bindings (bindings/)
    • Python package (pip install mainframe-bash) - 126 tests
    • Node.js/Bun package (npm install mainframe-bash) - 142 tests
    • Subprocess wrapper pattern with USOP JSON parsing
    • Full type hints (Python) and TypeScript definitions
  • Property-Based Testing (lib/proptest.sh) - 67 tests
    • Built-in generators: gen_int, gen_string, gen_email, gen_uuid, etc.
    • Automatic shrinking for minimal failing examples
    • Reproducible via PROPTEST_SEED
  • Benchmark Tracking (lib/benchmark.sh) - 71 tests
    • Statistical analysis: mean, median, stddev, p95, p99
    • Baseline comparison with regression detection
    • Multi-format reports (text/JSON/markdown)
    • Initial benchmarks in benchmarks/
  • AWM Usage Patterns Cookbook (docs/AWM_COOKBOOK.md)
    • Quick reference with essential functions
    • Real-world examples (audit, research, handoff)
    • Best practices and troubleshooting guide
  • Developer Pain Point Libraries (4 libraries, 208 tests)
    • lib/workpool.sh - Parallel work pool (1,014 lines)
      • Background job + temp file IPC pattern
      • Functions: pool_create, pool_submit, pool_wait, pool_map, pool_results
      • No external dependencies (pure bash parallel processing)
    • lib/schema.sh - JSON Schema validation (1,079 lines)
      • Draft 2020-12 subset implementation
      • Functions: schema_define, schema_validate, schema_errors
      • Supports: type, properties, required, enum, pattern, min/max
    • lib/forensics.sh - Error context capture (911 lines)
      • Stack traces, variable watches, forensic bundles
      • Functions: forensics_capture, forensics_stack, forensics_bundle
      • Integrates with bash ERR trap for auto-capture
    • lib/compat.sh - Enhanced BSD/GNU compatibility (453 lines)
      • 20+ underscore-style wrappers
      • Functions: compat_sed_inplace, compat_date_parse, compat_stat_size
      • macOS vs Linux transparent compatibility
  • Test coverage expansion (52.3% → 80% target) - 10,000+ tests (in progress)
  • Distributed task scheduling across multiple hosts
  • Agent capability negotiation protocol
  • Orchestration metrics and dashboards

Planned

Additional Libraries

Priority: Low

  • ansible.sh - Ansible integration

Testing Improvements

Priority: Low (property-based and benchmark tracking complete)

  • Mutation testing for coverage gaps

Documentation

Priority: Low (AWM cookbook complete)

  • Video tutorials
  • Interactive playground (WebAssembly bash)
  • More AI agent integration examples

Completed

V10.0 - AI-Native Bash Runtime

Major milestone: Transforming bash into a robust AI execution environment

  • Command Verification Engine (lib/verify.sh)
    • Pre-execution static analysis (verify_command)
    • Typo detection (sl→ls, gti→git, etc.)
    • Dangerous pattern detection (rm -rf /)
    • Pipeline validation (verify_pipeline)
    • Resource estimation (verify_estimate)
  • Intent Parser (lib/intent.sh)
    • Natural language to structured intent (intent_parse)
    • Intent to bash code generation (intent_to_bash)
    • Code explanation (intent_explain)
    • Auto-complete suggestions (intent_complete)
  • Code Generation (lib/generate.sh)
    • Generate bash from descriptions (generate_function)
    • Template-based code generation
    • Self-documenting output
  • Graph Execution Engine (lib/graph.sh)
    • DAG workflow definition (graph_define)
    • Auto-parallelization of independent tasks
    • Progress tracking per node
    • ASCII visualization (graph_visualize)
  • Self-Healing System (lib/heal.sh)
    • Error classification (permission, notfound, network, etc.)
    • Confidence-scored fix suggestions
    • Auto-retry with backoff (heal_wrap)
    • Integration with verify for pre-flight checks
  • Predictive Resource Management (lib/predict.sh)
    • Time/memory/CPU prediction (predict_resources)
    • Risk assessment (predict_risk)
    • Success probability estimation (predict_success)
    • 148 command profiles
  • Command Optimization (lib/optimize.sh)
    • Automatic command rewriting (optimize_command)
    • Pattern detection (cat|grep → grep)
    • Performance suggestions
  • Temporal Query Engine (lib/temporal.sh)
    • SQL-like history queries (temporal_query)
    • Pattern detection (temporal_detect_pattern)
    • Anomaly detection (temporal_anomaly_detect)
    • SQLite backend with bash fallback
  • Persistent Agent Loops (lib/agent_loop.sh)
    • Background agent processes (agent_loop_start)
    • Goal-oriented execution
    • Checkpoint/resume functionality
    • Human-in-the-loop support
    • Parent-child agent coordination
  • State Machine Workflows (lib/state_machine.sh)
    • Visual state machine definition
    • ASCII state diagrams
    • State transitions with guards
    • Workflow execution with rollback
  • UAP v2 - Agent Messaging (lib/uap_v2.sh)
    • ZeroMQ-like RPC (uap_v2_call)
    • Bidirectional streaming (uap_v2_stream)
    • Broadcast messaging (uap_v2_broadcast)
    • Unix sockets, named pipes, file fallback
    • Service discovery with heartbeats
  • 11,825 lines of new V10 code
  • ~289 new functions across 12 libraries
  • Full USOP compliance
  • Zero external dependencies (pure bash)
  • Comprehensive test coverage

v7.2 - Multi-Agent Team Orchestration

Major milestone: Complete orchestration system for coordinated agent teams

  • Multi-Agent Orchestration (lib/orchestrate.sh)
    • Team lifecycle management (orch_team_register, orch_team_dissolve)
    • Agent spawning with TMUX windows (orch_agent_spawn, orch_agent_terminate)
    • Sub-agent delegation with limits (orch_subagent_spawn, orch_subagent_list)
    • Task distribution with priority queues (orch_task_assign, orch_task_complete)
    • Redis pub/sub for real-time coordination
    • File-based fallback when Redis unavailable
    • USOP v4 message protocol (orch_message_create, orch_message_send)
    • Health monitoring with heartbeats (orch_agent_heartbeat, orch_agent_healthy)
    • Stale agent pruning (orch_prune_stale)
    • Failed agent recovery with task re-queuing (orch_agent_recover)
    • Discovery broadcasting for knowledge sharing
    • Graceful shutdown with cleanup (orch_shutdown)
  • Predefined Team Types: default, research, implementation, review, testing
  • Agent Status Tracking: pending, initializing, ready, busy, blocked, completed, failed, terminated
  • Task Status Tracking: queued, assigned, running, completed, failed, cancelled
  • 80 exported functions and constants
  • Full documentation in docs/ORCHESTRATION.md

v7.1 - Security Hardening

  • Capability-based security model (lib/capability.sh)
    • Capability token format: cap://domain/action/resource
    • Grant, revoke, and check capabilities
    • Wildcard pattern matching
    • Capability profiles: minimal, readonly, developer, network, admin
    • Guarded operations: cap_read_file, cap_write_file, cap_exec, cap_env_get
    • Audit logging with JSON export
    • Capability delegation for agent handoffs
    • 38 unit tests

v6.1 - Multi-Agent Coordination

Major milestone: Infinite agent memory with tiered storage and inter-agent protocols

  • AWM v2 - Infinite Memory Architecture (awm_*.sh)
    • Storage abstraction layer (awm_storage.sh) - Unified interface for file/Redis/ChromaDB
    • Auto-detection with graceful fallback (no dependencies required)
    • Context streaming engine (awm_stream.sh) - Prevent context overflow
    • Memory pointer system - Store large data, pass references (7x token reduction)
    • Semantic chunking for code, prose, JSON, markdown
    • Dynamic token budget based on model detection
    • Pre-rot threshold management (compress at 75% capacity)
  • Agent Communication Protocol (awm_protocol.sh)
    • USOP v4 message envelope for agent-to-agent messaging
    • Agent cards for capability discovery
    • Message types: request, response, discovery, handoff, heartbeat
    • contextId for session continuity across agent boundaries
    • Handoff protocol for sub-agent delegation with context inheritance
  • Tiered Memory Manager (awm_tiers.sh)
    • Hot tier (in-memory, fastest)
    • Warm tier (file/Redis, 1hr TTL)
    • Cold tier (persistent archive with semantic search)
    • Automatic promotion/eviction based on access patterns
    • Importance-based retention (critical items never evicted)
  • 50 new unit tests for AWM v2 components

v6.0 - Agent Working Memory

Major milestone: Persistent memory for AI agents outside context window

  • Agent Working Memory (AWM) - Full external memory system (awm.sh)
    • Session lifecycle management (awm_init, awm_resume, awm_close)
    • Key-value checkpoints (awm_checkpoint, awm_get)
    • Discovery logging (awm_discovery, awm_list_discoveries)
    • Sub-agent inheritance model (awm_context_for)
    • Namespace isolation for concurrent agents
    • Token budget estimation
    • Automatic compression of old entries
    • Session summary generation (awm_summary)
  • bURL library - AI-native HTTP client (burl.sh)
    • Automatic retry with exponential backoff
    • USOP-formatted responses
    • Request/response logging
    • Header management
  • MCP Server - Model Context Protocol integration (mcp/)
    • Direct function exposure to AI agents
    • Structured JSON request/response
    • Function metadata and documentation via MCP
    • Integration with Claude, GPT, and LLM tooling ecosystems
  • LSP Server - Language Server Protocol support (lsp/)
    • IDE integration (VS Code, Neovim, etc.)
    • Function completion and documentation
    • Signature help and hover info
  • Bun package manager support (bun.sh)
    • Bun detection and version checking
    • Package management wrappers
    • Script execution helpers
  • Agent execution library (agent.sh)
    • Agent spawn primitives
    • Execution context management
    • Retry with backoff
  • CI/CD working on all platforms (Linux, macOS, Windows WSL)
  • 117 libraries (up from 114)
  • 4,003 functions (up from 3,400+)
  • 6,538 tests (up from 5,500+)

v5.0 - AI Agent Runtime

  • Agent Safety library (agent_safety.sh)
  • Agent Communication library (agent_comm.sh)
  • USOP - Universal Structured Output Protocol
  • Idempotent operations (ensure_* functions)
  • Atomic file operations with rollback
  • Memoization/caching (cache.sh)
  • Lazy loading engine
  • Context budget management (context.sh)
  • Diff/patch operations for surgical editing (diff.sh)

v4.0 - Language Analysis

  • TypeScript analysis (typescript.sh)
    • Import graph analysis
    • Circular dependency detection
    • Breaking change detection
    • Bundle size estimation
  • Python analysis (python.sh)
    • Import analysis
    • Framework detection
    • Dependency management
    • Code metrics

v3.0 - AI Optimization

  • Idempotent operations library (idempotent.sh)
  • Atomic operations library (atomic.sh)
  • Observability/tracing library (observe.sh)
  • Project detection library (project.sh)
  • Design-by-Contract library (contract.sh)
  • Performance/feature gates library (perf.sh)
  • Network scanning library (netscan.sh)
  • Parser library (parsers.sh)

v2.0 - Extended Libraries

  • DateTime operations (datetime.sh)
  • HTTP client - pure bash (http.sh)
  • CSV parsing (csv.sh)
  • Git helpers (git.sh)
  • Cryptography (crypto.sh)
  • Process management (proc.sh)
  • Path manipulation (path.sh)
  • Validation & sanitization (validation.sh)
  • Environment management (env.sh)
  • Docker/Compose helpers (docker.sh)

v1.0 - Core Libraries

  • String operations (pure-string.sh)
  • Array operations (pure-array.sh)
  • JSON generation (json.sh)
  • File operations (pure-file.sh)
  • ANSI colors (ansi.sh)
  • Logging (log.sh)
  • CLI utilities (cli.sh)

Contributing

Have ideas for the roadmap?


Last updated: February 2026