Description
Before each benchmark scenario begins, reset the bench-session Qdrant collection and SQLite database so earlier scenarios cannot contaminate later ones.
Part of epic #2827. See spec: .local/specs/zeph-bench/spec.md NFR-001, FR-002.
Scope
BenchIsolation::reset async function that:
- Deletes and recreates the Qdrant collection named
bench_<dataset>_<run_id>
- Clears or recreates the SQLite conversation-history tables for the bench session DB path (
bench-<run_id>.db)
- Bench session config overrides:
memory.qdrant.collection and DB path are set to bench-namespaced values before agent construction
- Verify production collection names (
zeph_memory, zeph_skills, etc.) are never touched
- Isolation overhead must be < 2 seconds (NFR-007)
Acceptance Criteria
Description
Before each benchmark scenario begins, reset the bench-session Qdrant collection and SQLite database so earlier scenarios cannot contaminate later ones.
Part of epic #2827. See spec:
.local/specs/zeph-bench/spec.mdNFR-001, FR-002.Scope
BenchIsolation::resetasync function that:bench_<dataset>_<run_id>bench-<run_id>.db)memory.qdrant.collectionand DB path are set to bench-namespaced values before agent constructionzeph_memory,zeph_skills, etc.) are never touchedAcceptance Criteria