Skip to content

feat(bench): implement --resume for interrupted benchmark runs #2835

@bug-ops

Description

@bug-ops

Description

When --resume is passed and a partial results.json exists in the output directory, skip already-completed scenarios and continue from where the run stopped.

Part of epic #2827. See spec: .local/specs/zeph-bench/spec.md FR-008.

Scope

  • On startup with --resume: read results.json if it exists, extract completed scenario IDs
  • Skip scenarios whose IDs appear in the existing results
  • Append new results to existing list; update aggregate on completion
  • If results.json does not exist, behave as a fresh run
  • Warn if the existing results are from a different dataset or model

Acceptance Criteria

  • --resume on a partial results file skips completed scenarios
  • Final results.json contains all scenarios (original + resumed)
  • --resume with no existing file behaves as fresh run
  • Warning printed if dataset or model mismatch between existing and current run

Metadata

Metadata

Assignees

Labels

P2High value, medium complexityenhancementNew feature or request

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions