An implementation of the Anthropic's paper and essay on "A statistical approach to model evaluations"

AI agent evolving strategies through automated self-play overnight. Generic framework with GEPA-inspired feedback loop and Elo tracking.

Create your self-hosted, open-source Operator model.

LOAB: A benchmark for evaluating LLM agents on end-to-end mortgage lending operations under real regulatory constraints.

Legal Action Boundary Eval (LABE): public proxy eval for legal AI workflows at the action boundary

Build a private evaluation dataset to optimize your organization's token costs.

Portable evaluation bundles for agents and agent-shaped workflows: bounded, reproducible, regression-aware proof surfaces for quality claims.

Isomorphic agent tooling: author once, run anywhere. Build, lint, route, fan out, eval, trace, guard, contract, and ledger AI-agent workflows across Cursor, Claude Code, Codex, and OpenCode.

Reproducible evaluation harness for hidden coordination variables in multi-agent LLM systems.

𝘈 𝘔𝘶𝘭𝘵𝘪-𝘈𝘨𝘦𝘯𝘵 𝘚𝘺𝘴𝘵𝘦𝘮 𝘧𝘰𝘳 𝘊𝘳𝘰𝘴𝘴-𝘊𝘩𝘦𝘤𝘬𝘪𝘯𝘨 𝘗𝘩𝘪𝘴𝘩𝘪𝘯𝘨 𝘜𝘙𝘓𝘴.

The node-level tracing library for agentic software.

Horizon-Eval: evaluation-integrity framework for portable long-horizon agent benchmarks, with QA gates, trajectory auditing, replayable run bundles, and safety-gap analysis.

Reasoning quality analysis for autonomous agents — detect silent reasoning failures using optimal transport, information theory, and algorithmic complexity.

A practical workbench for prompt, model, and mocked workflow evaluation with repeatable benchmarks, structured graders, and agent episode traces.

Automate AI agent behavior tuning with human feedback, test small mutations, and keep what improves performance over time

Alpha benchmark for repo continuation intelligence

Trace-to-eval control plane that turns production failures into promptfoo-ready eval packs.