CORE-005: Implement run command

[AliiiBenn]

CORE-005: Implement run Command (Full Pipeline)

Overview

Implement the run command to orchestrate the complete data pipeline: import → analyze → export in a single command.

Description

The run command executes all three core commands in sequence:

1. import - Load and transform data
2. analyze - Calculate KPIs and insights
3. export - Generate Excel reports

This provides a one-click experience for users to update their warehouse reports.

Technical Approach

Sequential Execution

class PipelineRunner:
    def __init__(self, project_dir: Path):
        self.project_dir = project_dir

    def run_full_pipeline(self) -> dict:
        results = {
            "import": None,
            "analyze": None,
            "export": None,
            "status": "success",
            "errors": []
        }

        # Step 1: Import
        try:
            importer = Importer(self.project_dir)
            results["import"] = importer.run()
        except Exception as e:
            results["errors"].append(f"Import failed: {e}")
            results["status"] = "failed"
            return results

        # Step 2: Analyze
        try:
            analyzer = Analyzer(self.project_dir / "warehouse.db")
            results["analyze"] = analyzer.run_all()
        except Exception as e:
            results["errors"].append(f"Analyze failed: {e}")
            # Continue to export anyway

        # Step 3: Export
        try:
            exporter = Exporter(self.project_dir / "warehouse.db")
            results["export"] = exporter.run()
        except Exception as e:
            results["errors"].append(f"Export failed: {e}")
            results["status"] = "partial"

        return results

Error Handling Strategy

Fail fast on import: If import fails, stop immediately (no data to analyze/export)

Continue on analyze failure: If analyze fails, still export raw data

Always report status: Show user what succeeded/failed

Implementation Plan

Phase 1: Pipeline Orchestration (1 day)

• Create run/ module
• Implement PipelineRunner class
• Execute commands in sequence
• Collect results from each step

Phase 2: Error Handling & Reporting (1 day)

• Implement fail-fast for import
• Continue on analyze failure
• Aggregate error messages
• Display comprehensive status report

CLI Usage

# Run full pipeline
wareflow run

# Run with specific steps
wareflow run --steps import,export  # Skip analyze

# Dry-run (show what would happen)
wareflow run --dry-run

# Verbose output
wareflow run --verbose

# Continue on errors
wareflow run --continue-on-error

Output Format

$ wareflow run

🚀 Starting Wareflow Pipeline...
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Step 1/3: Import
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  ✓ produits: 1,234 rows imported     [2.3s]
  ✓ mouvements: 45,678 rows imported   [8.7s]
  ✓ commandes: 789 rows imported       [1.1s]

Step 2/3: Analyze
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  ✓ Database overview                 [0.2s]
  ✓ Movement analysis                  [0.8s]
  ✓ Product performance                [0.5s]
  ✓ Order statistics                   [0.3s]

Step 3/3: Export
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  ✓ Summary dashboard                  [0.4s]
  ✓ Movements analysis & chart         [0.8s]
  ✓ Products performance               [0.5s]
  ✓ Orders statistics & chart          [0.4s]

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

✅ Pipeline completed successfully!

📊 Results:
  Import: 47,701 rows imported (12.8s)
  Analyze: 4 analyses completed (1.8s)
  Export: warehouse_report_20250121_143045.xlsx (2.1s)

📄 Report: output/warehouse_report_20250121_143045.xlsx
⏱️  Total time: 16.7 seconds

💡 Next steps:
  open output/warehouse_report_*.xlsx    # View report
  wareflow status                         # Check database state

Error Scenarios

Import Fails

$ wareflow run

Step 1/3: Import
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  ❌ produits: Import FAILED

❌ Pipeline failed at Import step

Error: File not found: data/produits.xlsx

💡 Solution:
  Place your Excel files in the data/ directory:
    - data/produits.xlsx
    - data/mouvements.xlsx
    - data/commandes.xlsx

Pipeline stopped. 0/3 steps completed.

Analyze Fails (but continues)

$ wareflow run

Step 1/3: Import
  ✓ All imports completed successfully

Step 2/3: Analyze
  ⚠️  Analysis completed with errors
  ⚠️  Movement analysis failed: table 'mouvements' empty

Step 3/3: Export
  ✓ Export completed (raw data only)

⚠️  Pipeline completed with warnings

Warnings:
  - Movement analysis failed (empty table)
  - Exported raw data only (no analysis)

📄 Report: output/warehouse_report_20250121_143045.xlsx

Success Criteria

• Execute import → analyze → export in sequence
• Fail fast if import fails
• Continue if analyze fails (export raw data)
• Display progress for each step
• Show comprehensive status report
• Report total execution time
• Provide next steps recommendations
• Support --steps flag for partial pipeline

File Structure

src/wareflow_analysis/run/
├── __init__.py
└── pipeline.py      # Pipeline orchestration

Dependencies

Required

• CORE-001 (import)
• CORE-002 (analyze)
• CORE-004 (export)

Related Issues

• Depends on: CORE-001, CORE-002, CORE-004
• Enables: Automation and scheduling

Scheduling Integration

Once run is implemented, can be scheduled:

Windows Task Scheduler

<Task>
  <Triggers>
    <CalendarTrigger>
      <StartAt 02:00:00</StartAt>
      <DaysInterval>1</DaysInterval>
    </CalendarTrigger>
  </Triggers>
  <Actions>
    <Exec>
      <Command>wareflow</Command>
      <Arguments>run</Arguments>
    </Exec>
  </Actions>
</Task>

Linux Cron

# Run daily at 2 AM
0 2 * * * cd /path/to/warehouse && wareflow run >> logs/pipeline.log 2>&1

Future Enhancements

• Parallel execution (import multiple files concurrently)
• Incremental pipeline (only analyze new data)
• Notification on failure (email, Slack)
• Automatic retry with exponential backoff
• Pipeline history and logs

References

• Pipeline architecture: docs/ARCHITECTURE.md
• Individual commands: CORE-001 to CORE-004

Notes

This command provides the complete user experience - a single command that transforms raw Excel files into professional reports. It's the "happy path" for most users.