Skip to content

Commit c1fd3cb

Browse files
committed
feat: update example configurations and remove deprecated files for improved clarity and usability
1 parent 6b05270 commit c1fd3cb

8 files changed

Lines changed: 139 additions & 52 deletions

File tree

.env.example

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# ── AutoPrompt Core ─────────────────────────────────────
22
# This is the only required key for the optimizer loop to run.
3-
# AutoPrompt does NOT need access to your agent's actual database (Supabase/Postgres)
4-
# or Slack. It runs completely isolated using local SQLite for its own logging.
3+
# AutoPrompt does NOT need access to your agent's actual database
4+
# It runs completely isolated using local SQLite for its own logging.
55
OPENROUTER_API_KEY=your_openrouter_api_key_here
66
OPENROUTER_MODEL=deepseek/deepseek-v3.2

README.md

Lines changed: 27 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -39,23 +39,40 @@ Copy env example and fill out:
3939
cp .env.example .env
4040
```
4141

42-
Validate your config:
42+
The fastest way to try AutoPrompt is with the multi-turn example — it runs entirely offline, no API key required for the agent itself:
4343

4444
```bash
45-
autoprompt validate autoprompt.yaml
45+
autoprompt run examples/multi_turn/autoprompt.yaml --dry-run
46+
autoprompt run examples/multi_turn/autoprompt.yaml
4647
```
4748

48-
Run in dry-run mode:
49+
For a real LLM agent (requires `OPENROUTER_API_KEY`):
4950

5051
```bash
51-
autoprompt run autoprompt.yaml --dry-run
52+
autoprompt run examples/simple/autoprompt.yaml
5253
```
5354

54-
Execute the optimization loop:
55+
## Choosing an Adapter
5556

56-
```bash
57-
autoprompt run autoprompt.yaml
58-
```
57+
AutoPrompt connects to your agent via one of three built-in adapters:
58+
59+
| Adapter | When to use | Config field |
60+
|---|---|---|
61+
| `python_callable` | Your agent is a Python function in the same repo | `import_path: "my_module:my_function"` |
62+
| `http` | Your agent runs as an HTTP service | `endpoint: "http://localhost:8000/chat"` |
63+
| `cli` | Your agent is a CLI tool that reads from stdin | `command: "python my_agent.py"` |
64+
65+
**You need a custom adapter** only when the built-in ones don't fit — for example, if your HTTP endpoint requires JWT authentication or a non-standard request format. Custom adapters subclass `AgentAdapter` from [`autoprompt/adapters/base.py`](autoprompt/adapters/base.py) and implement two methods: `send()` and `health_check()`.
66+
67+
### What your callable needs to accept
68+
69+
For `python_callable`, AutoPrompt calls your function with either:
70+
- `handle_message(message: str, context: dict)` — single-turn
71+
- `chat(messages: list[dict], context: dict)` — multi-turn
72+
73+
Return `{"content": "..."}` or just a plain `str`.
74+
75+
For `http`, AutoPrompt POSTs `{"message": "..."}` (or `{"messages": [...]}` for multi-turn) and expects `{"content": "..."}` in the response.
5976

6077
## Example Config
6178

@@ -108,13 +125,8 @@ autoprompt diff autoprompt.yaml
108125

109126
| Example | Description |
110127
|---|---|
111-
| [`examples/simple/`](examples/simple/) | Minimal setup with a Python callable adapter |
112-
| [`examples/multi_turn/`](examples/multi_turn/) | Multi-turn agent with offline mock (no API key needed) |
113-
| [`examples/ora/`](examples/ora/) | HTTP adapter with custom JWT authentication |
114-
115-
## Creating Custom Adapters
116-
117-
If your agent runs inside Python without an HTTP endpoint, use `python_callable` or write a custom class subclassing `AgentAdapter` (see [`autoprompt/adapters/base.py`](autoprompt/adapters/base.py) and [`examples/ora/adapter.py`](examples/ora/adapter.py) for a pattern).
128+
| [`examples/multi_turn/`](examples/multi_turn/) | Complete multi-turn agent — runs fully offline, no API key needed for the agent |
129+
| [`examples/simple/`](examples/simple/) | Minimal single-turn LLM agent using `python_callable` |
118130

119131
## Development
120132

autoprompt.yaml

Lines changed: 29 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -1,51 +1,49 @@
1+
# AutoPrompt configuration template
2+
# Copy this file and adjust it for your agent.
3+
# All LLM calls go through OpenRouter — set OPENROUTER_API_KEY in .env
4+
15
agent:
2-
adapter: "http"
6+
adapter: "python_callable" # "python_callable" | "http" | "cli"
37
name: "My Agent"
48
description: "Describe what this agent does and who it serves."
5-
endpoint: "http://localhost:8000/chat"
9+
10+
# python_callable: import path to your handler function
11+
import_path: "my_module:handle_message"
12+
13+
# http: uncomment if your agent runs as an HTTP service
14+
# endpoint: "http://localhost:8000/chat"
15+
16+
# cli: uncomment if your agent is a CLI tool
17+
# command: "python my_agent.py"
18+
619
optimizable:
720
- type: system_prompt
8-
path: "./prompts/system.md"
9-
- type: memory_config
10-
path: "./config/memory.yaml"
11-
- type: tool_config
12-
path: "./config/tools.yaml"
21+
path: "./system.md" # AutoPrompt will read, mutate, and roll back this file
1322

1423
rubric:
15-
path: "./rubrics/program.md"
16-
scoring_model: "claude-sonnet-4-20250514"
24+
path: "./rubric.md" # Markdown file describing what a good response looks like
25+
scoring_model: "${OPENROUTER_MODEL:-deepseek/deepseek-v3.2}"
1726
score_range: [1.0, 10.0]
1827
dimensions:
1928
- name: "helpfulness"
20-
weight: 0.3
29+
weight: 0.6
2130
- name: "accuracy"
22-
weight: 0.3
23-
- name: "tone"
24-
weight: 0.2
25-
- name: "conciseness"
26-
weight: 0.2
31+
weight: 0.4
2732

2833
tests:
29-
mode: "dynamic"
30-
static_suite: "./tests/suite.yaml"
31-
generator_model: "claude-sonnet-4-20250514"
32-
tests_per_iteration: 10
34+
mode: "mix" # "static" | "dynamic" | "mix"
35+
static_suite: "./tests.yaml" # your hand-written test cases (required for "static" and "mix")
36+
generator_model: "${OPENROUTER_MODEL:-deepseek/deepseek-v3.2}"
37+
tests_per_iteration: 6
3338
categories:
34-
- "general_knowledge"
35-
- "personal_context"
36-
- "task_execution"
39+
- "general"
3740
- "edge_cases"
3841

3942
loop:
4043
max_iterations: 10
41-
budget_limit_usd: 5.00
42-
improvement_threshold: 0.05
43-
mutation_strategy: "targeted"
44-
mutation_model: "claude-sonnet-4-20250514"
45-
keep_top_n: 3
44+
budget_limit_usd: 2.00
45+
mutation_model: "${OPENROUTER_MODEL:-deepseek/deepseek-v3.2}"
46+
improvement_threshold: 0.05 # minimum score gain to accept a mutation
4647

4748
logging:
48-
backend: "supabase"
49-
supabase_url: "${SUPABASE_URL}"
50-
supabase_key: "${SUPABASE_KEY}"
51-
49+
backend: "sqlite" # "sqlite" | "jsonl" | "supabase"

examples/simple/agent.py

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
"""
2+
Minimal single-turn LLM agent for the AutoPrompt simple example.
3+
4+
This is the agent AutoPrompt will optimize. It loads its system prompt
5+
from system.txt — that file is what gets mutated each iteration.
6+
7+
Run the optimization loop with:
8+
autoprompt run examples/simple/autoprompt.yaml
9+
"""
10+
11+
import os
12+
from pathlib import Path
13+
import openai
14+
15+
SYSTEM_PROMPT_PATH = Path(__file__).parent / "system.txt"
16+
17+
_client: openai.OpenAI | None = None
18+
19+
20+
def _get_client() -> openai.OpenAI:
21+
global _client
22+
if _client is None:
23+
_client = openai.OpenAI(
24+
base_url="https://openrouter.ai/api/v1",
25+
api_key=os.environ["OPENROUTER_API_KEY"],
26+
)
27+
return _client
28+
29+
30+
def health_check() -> bool:
31+
return bool(os.getenv("OPENROUTER_API_KEY"))
32+
33+
34+
def handle_message(message: str, context: dict | None = None) -> dict:
35+
system = SYSTEM_PROMPT_PATH.read_text(encoding="utf-8")
36+
response = _get_client().chat.completions.create(
37+
model=os.getenv("OPENROUTER_MODEL", "deepseek/deepseek-v3.2"),
38+
messages=[
39+
{"role": "system", "content": system},
40+
{"role": "user", "content": message},
41+
],
42+
max_tokens=512,
43+
)
44+
return {"content": response.choices[0].message.content}

examples/simple/autoprompt.yaml

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,21 @@
11
agent:
22
adapter: "python_callable"
33
name: "Simple Example Agent"
4-
description: "A minimal example agent for testing the AutoPrompt optimization loop."
5-
import_path: "my_agent.core:handle_message"
4+
description: "A minimal single-turn LLM agent. AutoPrompt will optimize its system prompt."
5+
import_path: "examples.simple.agent:handle_message"
66
optimizable:
77
- type: system_prompt
88
path: "./system.txt"
99

1010
rubric:
1111
path: "./simple-rubric.md"
12-
scoring_model: "claude-sonnet-4-20250514"
12+
scoring_model: "${OPENROUTER_MODEL:-deepseek/deepseek-v3.2}"
1313
score_range: [1.0, 10.0]
14+
dimensions:
15+
- name: "helpfulness"
16+
weight: 0.6
17+
- name: "clarity"
18+
weight: 0.4
1419

1520
tests:
1621
mode: "static"
@@ -19,7 +24,7 @@ tests:
1924
loop:
2025
max_iterations: 3
2126
budget_limit_usd: 1.0
22-
mutation_model: "claude-sonnet-4-20250514"
27+
mutation_model: "${OPENROUTER_MODEL:-deepseek/deepseek-v3.2}"
2328

2429
logging:
2530
backend: "jsonl"

examples/simple/simple-rubric.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
# Evaluation Rubric
2+
3+
## Helpfulness
4+
Does the response actually answer the question? Is the information correct and useful to the user?
5+
6+
## Clarity
7+
Is the response easy to understand? Is it appropriately concise — no unnecessary filler, no excessive length?

examples/simple/system.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
You are a helpful assistant. Answer questions clearly and concisely.

examples/simple/tests.yaml

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
tests:
2+
- prompt: "What is the capital of France?"
3+
category: "factual"
4+
expected_behavior: "Correctly states Paris as the capital of France"
5+
6+
- prompt: "Explain what a neural network is in one or two sentences."
7+
category: "explanation"
8+
expected_behavior: "Clear, jargon-free explanation accessible to a non-technical reader"
9+
10+
- prompt: "How do I reverse a list in Python?"
11+
category: "coding"
12+
expected_behavior: "Shows a working Python example, e.g. list[::-1] or list.reverse()"
13+
14+
- prompt: "What are three tips for better sleep?"
15+
category: "advice"
16+
expected_behavior: "Three practical, concrete tips — not vague platitudes"
17+
18+
- prompt: "Summarize the water cycle in two sentences."
19+
category: "explanation"
20+
expected_behavior: "Accurate summary covering evaporation, condensation, and precipitation"

0 commit comments

Comments
 (0)