Skip to content

fix: discover nested eval result directories#131

Open
EfeDurmaz16 wants to merge 1 commit into
vercel-labs:mainfrom
EfeDurmaz16:efe/recursive-eval-discovery
Open

fix: discover nested eval result directories#131
EfeDurmaz16 wants to merge 1 commit into
vercel-labs:mainfrom
EfeDurmaz16:efe/recursive-eval-discovery

Conversation

@EfeDurmaz16

Copy link
Copy Markdown

Summary

Fixes result discovery for nested eval ids such as ui-development/design-system/layout-props.

saveResults already writes nested eval result directories correctly, but housekeeping and reusable-result scanning only looked at immediate children of a timestamp directory. That caused nested results to be treated as incomplete or missed for reuse because summary.json lives deeper in the tree.

Changes

  • Add a shared recursive discoverEvalResultDirs helper.
  • Use it in housekeep so nested eval result directories are deduped/kept by their full relative eval id.
  • Use it in scanReusableResults so nested evals can match fingerprint keys and be reused.
  • Add regression coverage for nested housekeeping and reusable-result scanning.
  • Add a patch changeset.

Validation

  • npm run test -w packages/agent-eval -- src/lib/housekeeping.test.ts src/lib/results.test.ts
  • npm run build -w packages/agent-eval
  • npm run lint -w packages/agent-eval

Closes #129.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support for eval dirs more than one level deep

1 participant