Multi-Agent System (Planner + Executor + Evaluator)

A small Python orchestration loop where three specialized agents collaborate on a single goal:

Agent	Role
Planner	Breaks the goal into ordered steps with explicit success criteria
Executor	Produces work for each step (using prior step outputs as context)
Evaluator	Scores each output against criteria; failed steps can retry or trigger replan

flowchart LR
  G[User Goal] --> P[Planner]
  P --> E[Executor]
  E --> V[Evaluator]
  V -->|pass| E
  V -->|fail retry| E
  V -->|fail replan| P
  V -->|all steps pass| R[Run Report]

Loading

Quick start

cd multi-agent-system
python -m venv .venv

# Windows
.venv\Scripts\activate

pip install -e ".[dev]"  # or: pip install -r requirements.txt pytest
cp .env.example .env     # optional: set OPENAI_API_KEY

# Demo without API key
set MOCK_LLM=1
python -m multi_agent_system.cli "Design a REST API for a todo app"

# With OpenAI (or any OpenAI-compatible endpoint)
set OPENAI_API_KEY=sk-...
python -m multi_agent_system.cli "Write a migration plan for PostgreSQL 15"

JSON output for scripting:

python -m multi_agent_system.cli --json "Summarize our onboarding docs"

Configuration

Variable	Description
OPENAI_API_KEY	API key for the LLM provider
OPENAI_BASE_URL	Base URL (default https://api.openai.com/v1)
OPENAI_MODEL	Model id (default gpt-4o-mini)
MOCK_LLM	Set to 1 for deterministic offline demo

Project layout

src/multi_agent_system/
  agents/          # planner, executor, evaluator
  orchestrator.py  # main control loop
  llm.py           # OpenAI + mock providers
  models.py        # Pydantic schemas
  cli.py           # entrypoint
tests/

Push to GitHub

1. Authenticate (one-time):

   gh auth login

2. From this directory:

   git init
   git add .
   git commit -m "Initial multi-agent planner/executor/evaluator system"
   gh repo create multi-agent-system --public --source=. --remote=origin --push

   Or create the repo on github.com, then:

   git remote add origin https://github.com/YOUR_USER/multi-agent-system.git
   git branch -M main
   git push -u origin main

License

MIT