Roadmap: Dynamic Compose Architecture

[joshrotenberg]

Dynamic Compose Architecture Roadmap

Vision

Transform docker-wrapper from a Docker CLI wrapper into a comprehensive tool for programmatic Docker orchestration through dynamic compose generation.

Core Concept

Instead of the original 3-tier system (Templates → Groups → Stacks), we're pivoting to a simpler, more powerful approach:

• Templates: Domain experts that handle both direct container execution AND compose service generation
• Dynamic Compose: Programmatic docker-compose.yml generation with logic, conditionals, and type safety

Architecture Components

1. docker-wrapper (Core)

• Excellent Docker CLI wrapper (already complete)
• All Docker commands with type-safe builders
• Integration with compose-builder for seamless workflow

2. Templates

Templates are domain experts that encapsulate all knowledge about running a service:

• Dual-mode operation:
  • Direct execution with full setup (including HTTP APIs, configuration, etc.)
  • Compose service generation
  • Setup script generation for compose users
• Current templates to update:
  • Redis (add compose generation)
  • PostgreSQL (add compose generation)
• Future templates:
  • MongoDB
  • Elasticsearch
  • Kafka
  • MySQL
  • RabbitMQ

3. Compose Builder

The new star feature - programmatic compose generation:

• Full Compose spec support
• Fluent builder API
• Dynamic service generation based on:
  • Environment (dev/test/staging)
  • Feature flags
  • Runtime conditions
• YAML serialization
• Integration with templates
• Validation and type safety

Implementation Phases

Phase 1: Foundation ✅

• Workspace restructure
• Remove Groups concept
• Create compose-builder crate

Phase 2: Compose Builder Core

• Implement full Compose spec as Rust types
• Builder pattern for services, networks, volumes
• YAML generation and validation
• Basic patterns library (common service configurations)

Phase 3: Template Integration

• Update Template trait for dual-mode operation
• Add to_compose_service() to existing templates
• Add generate_setup_script() for complex setup
• Create template registry for discovery

Phase 4: Advanced Features

• Environment-based composition
• Service dependency resolution
• Health check generation
• Resource limit calculations
• Secret management integration

Phase 5: Developer Experience

• CLI tool for compose generation
• VS Code extension for template discovery
• Documentation and examples
• Migration guide from static compose files

Example Use Cases

Dynamic Environment Configuration

let compose = match env {
    "development" => dev_stack()
        .with_debugging()
        .with_hot_reload(),
    "testing" => test_stack()
        .with_isolated_databases()
        .with_test_data(),
    "staging" => staging_stack()
        .with_monitoring()
        .with_ssl(),
    _ => panic!("Unknown environment")
};

Conditional Services

let mut compose = base_stack();
if feature_flags.needs_redis {
    compose = compose.add_service(Redis::new());
}
if feature_flags.needs_search {
    compose = compose.add_service(Elasticsearch::new());
}

Template with Complex Setup

// Direct execution - full automation
let redis = RedisEnterprise::new()
    .with_cluster_setup()
    .start()
    .await?;

// Compose generation - container + setup script
let compose = redis.to_compose();
let setup = redis.generate_setup_script();

Benefits Over Static Compose

1. Type Safety: Catch configuration errors at compile time
2. Logic & Conditionals: Use if/else, loops, functions
3. Environment Awareness: Different configs for dev/test/prod
4. DRY Principle: Reuse common patterns
5. IDE Support: Autocomplete, refactoring, go-to-definition
6. Testing: Unit test your infrastructure

Success Metrics

• Can generate any valid docker-compose.yml programmatically
• Templates work seamlessly in both direct and compose modes
• Significant reduction in compose file duplication
• Improved developer experience over static YAML

Open Questions

1. Should compose-builder be part of docker-wrapper or separate crate?
2. How to handle compose features that require external files (configs, secrets)?
3. Should we support docker-compose extensions (x- fields)?
4. How to handle compose profiles?

Related Issues

• Migrate Core Docker Container Commands to Unified Pattern #97-100: DockerCommandV2 migration (in progress)
• This issue: Dynamic Compose Architecture

Let's discuss and refine this roadmap!