Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 16 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,20 @@ documented with migration notes.
- Additional docs for suite-specific adoption.
- Better compatibility tests for Promptfoo variable contracts.

## [0.1.7] - 2026-07-04

### Fixed

- Generic bootstrap now writes `evals/results/.gitignore` so raw result bundles stay local by
default while preserving `README.md` and the ignore file.
- Updated bootstrap tests to cover the generated results ignore policy.

### Notes

- No runner, schema, prompt, or consumer semantic behavior changed.
- No npm package is published.
- Consumers may pin `github:agentic-workflow-kit/eval-kit#v0.1.7`.

## [0.1.6] - 2026-07-04

### Fixed
Expand Down Expand Up @@ -121,7 +135,8 @@ documented with migration notes.
- Suite-specific presets remain deferred.
- Consumer repos own their own semantics, prompts, cases, and pass/fail policies.

[Unreleased]: https://github.com/agentic-workflow-kit/eval-kit/compare/v0.1.6...main
[Unreleased]: https://github.com/agentic-workflow-kit/eval-kit/compare/v0.1.7...main
[0.1.7]: https://github.com/agentic-workflow-kit/eval-kit/compare/v0.1.6...v0.1.7
[0.1.6]: https://github.com/agentic-workflow-kit/eval-kit/compare/v0.1.5...v0.1.6
[0.1.5]: https://github.com/agentic-workflow-kit/eval-kit/compare/v0.1.4...v0.1.5
[0.1.4]: https://github.com/agentic-workflow-kit/eval-kit/compare/v0.1.3...v0.1.4
Expand Down
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Shared evaluation infrastructure for `agentic-workflow-kit` repositories.
```json
{
"devDependencies": {
"@agentic-workflow-kit/eval-kit": "github:agentic-workflow-kit/eval-kit#v0.1.6"
"@agentic-workflow-kit/eval-kit": "github:agentic-workflow-kit/eval-kit#v0.1.7"
}
}
```
Expand Down Expand Up @@ -70,7 +70,7 @@ Install from a Git tag in a consumer repo:
```json
{
"devDependencies": {
"@agentic-workflow-kit/eval-kit": "github:agentic-workflow-kit/eval-kit#v0.1.6"
"@agentic-workflow-kit/eval-kit": "github:agentic-workflow-kit/eval-kit#v0.1.7"
},
"scripts": {
"eval:doctor": "eval-kit doctor --config evals/eval-kit.config.json",
Expand Down Expand Up @@ -195,6 +195,7 @@ v0.1.3
v0.1.4
v0.1.5
v0.1.6
v0.1.7
v0.2.0
```

Expand Down
2 changes: 1 addition & 1 deletion docs/design/consumer-integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Consumer repos should adopt eval-kit through a pinned Git tag and keep their eva
```json
{
"devDependencies": {
"@agentic-workflow-kit/eval-kit": "github:agentic-workflow-kit/eval-kit#v0.1.6"
"@agentic-workflow-kit/eval-kit": "github:agentic-workflow-kit/eval-kit#v0.1.7"
}
}
```
Expand Down
2 changes: 1 addition & 1 deletion docs/guides/consumer-integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ If you cannot state the eval goal, do not bootstrap a suite yet. Empty harnesses
```json
{
"devDependencies": {
"@agentic-workflow-kit/eval-kit": "github:agentic-workflow-kit/eval-kit#v0.1.6"
"@agentic-workflow-kit/eval-kit": "github:agentic-workflow-kit/eval-kit#v0.1.7"
}
}
```
Expand Down
2 changes: 1 addition & 1 deletion docs/guides/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ This guide adds a generic deterministic eval suite to a consumer repo.
```json
{
"devDependencies": {
"@agentic-workflow-kit/eval-kit": "github:agentic-workflow-kit/eval-kit#v0.1.6"
"@agentic-workflow-kit/eval-kit": "github:agentic-workflow-kit/eval-kit#v0.1.7"
}
}
```
Expand Down
18 changes: 9 additions & 9 deletions docs/reference/release-process.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ Consumers depend on tags like:
Title:

```text
chore(release): v0.1.6
chore(release): v0.1.7
```

Required changes:
Expand Down Expand Up @@ -63,18 +63,18 @@ git checkout main
git pull --ff-only
git rev-parse HEAD

git tag -a v0.1.6 -m "v0.1.6"
git push origin v0.1.6
git tag -a v0.1.7 -m "v0.1.7"
git push origin v0.1.7
```

Verify:

```bash
git rev-parse v0.1.6^{}
git show --no-patch --decorate v0.1.6
git rev-parse v0.1.7^{}
git show --no-patch --decorate v0.1.7
```

`v0.1.6^{}` must point to the release commit. With an annotated tag, `git rev-parse v0.1.6`
`v0.1.7^{}` must point to the release commit. With an annotated tag, `git rev-parse v0.1.7`
returns the tag object; `^{}` dereferences to the commit.

## GitHub Release
Expand All @@ -93,7 +93,7 @@ For each consumer repo:

```json
{
"@agentic-workflow-kit/eval-kit": "github:agentic-workflow-kit/eval-kit#v0.1.6"
"@agentic-workflow-kit/eval-kit": "github:agentic-workflow-kit/eval-kit#v0.1.7"
}
```

Expand All @@ -108,7 +108,7 @@ pnpm check
3. Run consumer smoke commands, for example in `technical-design`:

```bash
pnpm eval:case -- --case case-tiny-laundry-pickup-v1 --candidate evals/cases/case-tiny-laundry-pickup-v1/reference-design.md --run-id verify-eval-kit-v0.1.6
pnpm eval:case -- --case case-tiny-laundry-pickup-v1 --candidate evals/cases/case-tiny-laundry-pickup-v1/reference-design.md --run-id verify-eval-kit-v0.1.7
```

4. Open a PR with dependency, lockfile, and any compatibility fixes.
Expand All @@ -120,7 +120,7 @@ Do not move the tag.
Create a new patch release:

```text
v0.1.6 -> v0.1.7
v0.1.7 -> v0.1.8
```

Then open consumer bump PRs.
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/results.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ Current schema:
"run_type": "deterministic",
"runner": {
"id": "generic-eval-case",
"version": "0.1.6"
"version": "0.1.7"
},
"case_ids": ["case-example-v1"],
"started_at": "2026-07-03T00:00:00.000Z",
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@agentic-workflow-kit/eval-kit",
"version": "0.1.6",
"version": "0.1.7",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Use a minor version for layout changes

This release includes the new generated evals/results/.gitignore layout noted in the changelog, but docs/reference/versioning-policy.md says “changed bootstrap file layout” belongs in a minor release. If consumers bump patch tags expecting only patch-compatible changes, they will miss that a versioned generated-file contract changed; release this as the next minor version (with matching tag/docs) or keep the layout change out of the patch release.

Useful? React with 👍 / 👎.

"description": "Portable eval runner primitives for local eval suites.",
"private": true,
"type": "module",
Expand Down
2 changes: 1 addition & 1 deletion src/sdk.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ import { assertContainedPath, assertSafeId, toPosixPath } from "./paths.mjs";
const DEFAULT_SANDBOX_MODE = "read-only";
const DEFAULT_APPROVAL_POLICY = "never";
const RANDOMIZATION_METHOD = "sha256-seed-parity-v1";
const EVAL_KIT_VERSION = "0.1.6";
const EVAL_KIT_VERSION = "0.1.7";

const escapeRegExp = (value) => value.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");

Expand Down