Skip to content

Add RunLedger replay gate for agent regressions#1354

Closed
ZackMitchell910 wants to merge 2 commits intoUndertone0809:mainfrom
ZackMitchell910:runledger/replay-gate
Closed

Add RunLedger replay gate for agent regressions#1354
ZackMitchell910 wants to merge 2 commits intoUndertone0809:mainfrom
ZackMitchell910:runledger/replay-gate

Conversation

@ZackMitchell910
Copy link
Copy Markdown

@ZackMitchell910 ZackMitchell910 commented Dec 19, 2025

Summary

  • add a replay-only RunLedger eval suite (suite/case/schema/cassette + stub agent)
  • add a baseline file for regression gating
  • add a GitHub Actions workflow using runledger/Runledger@v0.1
  • add a small README note + ignore runledger_out/

How to run locally

runledger run evals/runledger --mode replay --baseline baselines/runledger-demo.json

Notes

  • no external calls; replay-only cassette
  • feel free to remove the suite/workflow if it is not desired
  • GitHub Actions note: workflows from first-time contributors/forks may require a maintainer to click “Approve and run” before checks will execute.

@vercel
Copy link
Copy Markdown

vercel Bot commented Dec 19, 2025

@ZackMitchell910 is attempting to deploy a commit to the zeeland's projects Team on Vercel.

A member of the Team first needs to authorize it.

Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @ZackMitchell910, thanks for opening your first pull request 😊! We really appreciate your work. Happy Coding 🎉🎊 !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants