diff --git a/docs/adoption-status.md b/docs/adoption-status.md new file mode 100644 index 0000000..4b91813 --- /dev/null +++ b/docs/adoption-status.md @@ -0,0 +1,43 @@ +# `rivet-validate` adoption status + +This file records the most recent run of +[`scripts/audit-rivet-validate-adoption.sh`](../scripts/audit-rivet-validate-adoption.sh) +across PulseEngine repositories. It is the audit deliverable for +[issue #187](https://github.com/pulseengine/rivet/issues/187). + +## How to refresh + +The audit script needs sibling clones of every PulseEngine repository +(it walks a workspace directory and does not call out to the network). +A typical refresh looks like: + +```sh +mkdir -p /tmp/pe-workspace +cd /tmp/pe-workspace +for r in kiln loom meld relay rivet sigil spar synth gale wohl; do + git clone --depth 1 https://github.com/pulseengine/$r.git +done + +cd /path/to/rivet +scripts/audit-rivet-validate-adoption.sh /tmp/pe-workspace \ + > docs/adoption-status.md +``` + +Replace this placeholder with the script output and commit. The +intention is for a CI job (Phase 4 of the V&V coverage initiative) to +do the refresh periodically and open a PR when the status changes; that +job is not yet wired up. + +## Latest recorded run + +> **placeholder** — replace this section with the output of +> `scripts/audit-rivet-validate-adoption.sh` against a fresh workspace. + +## Related + +- [`docs/pre-commit.md`](pre-commit.md) — canonical hook configuration + and adoption recipe. +- [Issue #187](https://github.com/pulseengine/rivet/issues/187) — + enforcement tracking issue. +- [Issue #184](https://github.com/pulseengine/rivet/issues/184) — + pulseengine-wide V&V coverage initiative (Phase 4). diff --git a/docs/pre-commit.md b/docs/pre-commit.md index b9f6881..c34c6b5 100644 --- a/docs/pre-commit.md +++ b/docs/pre-commit.md @@ -83,15 +83,101 @@ in a trailing comment so an adopter can grep-trim the file to their tier. ## Installing `rivet` for hooks -Two tested install paths: +Three tested install paths, in increasing order of reproducibility: + +- **GitHub release binary** (recommended for CI and end-user machines): + download the prebuilt `rivet` binary attached to the `vX.Y.Z` release + from and place it on + `PATH`. Tags are immutable; pinning to a tag pins the toolchain. +- **`cargo install --git`** (recommended when matching a specific + commit, e.g. an unreleased fix): + ```sh + cargo install --git https://github.com/pulseengine/rivet \ + --tag v0.8.0 \ + rivet-cli + ``` + Always pass `--tag` or `--rev` — the bare command picks `HEAD` of the + default branch and silently drifts between developer setups. +- **Workspace build** (rivet itself, monorepo developers): + `cargo run --release -p rivet-cli -- validate`. The + template's `entry:` for `rivet-validate` already does this so rivet's + own checkout works without an external install. + +### Version-pinning + +For adopter repositories, pin the rivet version your hooks use so the +diff between developer machines and CI stays empty. + +Two equivalent patterns work today: + +```yaml +# Option 1 — `cargo install` step in CI, then a system-language local hook. +# Suited to CI/CD pipelines. +- id: rivet-validate + name: rivet validate (dogfood) + entry: rivet validate + language: system # rivet must already be on PATH + pass_filenames: false + files: '(artifacts/.*\.yaml|schemas/.*\.yaml|safety/.*\.yaml|rivet\.yaml)$' +``` + +```yaml +# Option 2 — explicit additional_dependencies on the local hook. +# Suited to developer pre-commit installs once a binary release wrapper +# is published. (Tracking: pulseengine/rivet#187.) +- id: rivet-validate + name: rivet validate (dogfood) + entry: rivet validate + language: system + additional_dependencies: ['rivet==0.8.0'] # pinned binary version + pass_filenames: false + files: '(artifacts/.*\.yaml|schemas/.*\.yaml|safety/.*\.yaml|rivet\.yaml)$' +``` + +Pin to the **same** version your CI uses. If you bump in CI, bump here in +the same PR — drift between the two is the failure mode this hook +exists to prevent. + +## Failure mode (what the developer sees) + +When `rivet validate` finds a problem, the pre-commit hook fails the +commit and prints the validator's report verbatim. A typical failure +looks like: + +``` +rivet validate (dogfood)..................................................Failed +- hook id: rivet-validate +- exit code: 1 + +Loaded 142 artifacts from artifacts +ERROR: [REQ-042] link 'verifies' requires at least 1 target, found 0 +ERROR: [REQ-091] broken cross-ref: missing artifact 'TST-007' +WARN: [DD-019] field 'status' has no declared value + +Result: FAIL (2 errors, 1 warnings, 1 broken cross-refs) +``` + +Each error line carries the artifact id, the specific link or field at +fault, and the reason. Fix the underlying artifact YAML, re-stage, and +re-commit. + +> **`--no-verify` is not a fix.** The hook gates *local* drift, not +> upstream policy. CI runs the same `rivet validate` as a required +> status check; bypassing the hook only delays the failure to the PR. + +## Adoption audit + +The repo ships an audit script that surveys a workspace of repository +clones and reports which of them register the `rivet-validate` hook: + +```sh +scripts/audit-rivet-validate-adoption.sh /path/to/workspace > docs/adoption-status.md +``` -- **Cargo:** `cargo install --git https://github.com/pulseengine/rivet - rivet-cli` (pin a tag/sha for reproducibility). -- **Pre-commit `additional_dependencies`** (preferred for adopters): once a - binary release exists, the `rivet-validate` and `rivet-commit-msg` - entries can be wrapped in a pre-commit `local` repo with - `additional_dependencies: [rivet@]`. Tracking issue: - [#187](https://github.com/pulseengine/rivet/issues/187). +The script exits non-zero when a gap is present, so it can be wired +into a periodic CI job that publishes the latest gap list. See +[`docs/adoption-status.md`](adoption-status.md) for the most recent +recorded run. ## Drift policy @@ -109,6 +195,8 @@ adopter's `docs/pre-commit.md` as an explicit opt-out with rationale. ## See also - [`templates/pre-commit/.pre-commit-config.yaml`](../templates/pre-commit/.pre-commit-config.yaml) — the template itself +- [`scripts/audit-rivet-validate-adoption.sh`](../scripts/audit-rivet-validate-adoption.sh) — workspace audit script +- [`docs/adoption-status.md`](adoption-status.md) — most recent audit run - [Issue #186](https://github.com/pulseengine/rivet/issues/186) — canonical-template tracking issue - [Issue #187](https://github.com/pulseengine/rivet/issues/187) — `rivet-validate` enforcement across repos - [Issue #185](https://github.com/pulseengine/rivet/issues/185) — `cargo-mutants` adoption (T3) diff --git a/scripts/audit-rivet-validate-adoption.sh b/scripts/audit-rivet-validate-adoption.sh new file mode 100755 index 0000000..5d9b869 --- /dev/null +++ b/scripts/audit-rivet-validate-adoption.sh @@ -0,0 +1,95 @@ +#!/usr/bin/env bash +# Audit rivet-validate hook adoption across a workspace of repository clones. +# +# Surveys every directory under WORKSPACE that contains a rivet.yaml and +# checks whether its .pre-commit-config.yaml registers the rivet-validate +# hook. Emits a markdown gap report to stdout suitable for pasting into +# docs/adoption-status.md or an issue body. +# +# Tracking: pulseengine/rivet#187 (V&V coverage initiative — Phase 4). +# +# Usage: +# scripts/audit-rivet-validate-adoption.sh [WORKSPACE] +# +# WORKSPACE defaults to the parent of the rivet checkout. Set RIVET_AUDIT_QUIET=1 +# to suppress per-repo progress output on stderr. + +set -euo pipefail + +WORKSPACE="${1:-$(cd "$(dirname "$0")/../.." && pwd)}" +QUIET="${RIVET_AUDIT_QUIET:-0}" + +if [[ ! -d "$WORKSPACE" ]]; then + echo "error: workspace '$WORKSPACE' is not a directory" >&2 + exit 2 +fi + +log() { [[ "$QUIET" -eq 1 ]] || echo "$@" >&2; } + +declare -a HOOKED_REPOS=() +declare -a GAP_REPOS=() +declare -a NO_HOOK_CONFIG_REPOS=() + +while IFS= read -r -d '' rivet_yaml; do + repo_dir="$(dirname "$rivet_yaml")" + repo_name="$(basename "$repo_dir")" + log "scanning $repo_name" + + config="$repo_dir/.pre-commit-config.yaml" + if [[ ! -f "$config" ]]; then + NO_HOOK_CONFIG_REPOS+=("$repo_name") + continue + fi + + # Match either a `- id: rivet-validate` line or a `name: rivet validate ...` + # line; either form indicates the hook is wired up. + if grep -Eq '^[[:space:]]*-[[:space:]]+id:[[:space:]]*rivet-validate[[:space:]]*$' "$config" \ + || grep -Eq '^[[:space:]]*name:[[:space:]]+rivet validate' "$config"; then + HOOKED_REPOS+=("$repo_name") + else + GAP_REPOS+=("$repo_name") + fi +done < <(find "$WORKSPACE" -mindepth 2 -maxdepth 3 -name rivet.yaml -not -path '*/.git/*' -print0) + +total=$(( ${#HOOKED_REPOS[@]} + ${#GAP_REPOS[@]} + ${#NO_HOOK_CONFIG_REPOS[@]} )) + +cat < 0 )); then + echo "## Adopted" + echo + for r in "${HOOKED_REPOS[@]}"; do echo "- $r"; done + echo +fi + +if (( ${#GAP_REPOS[@]} > 0 )); then + echo "## Gap (has \`.pre-commit-config.yaml\` but no \`rivet-validate\` hook)" + echo + for r in "${GAP_REPOS[@]}"; do echo "- $r"; done + echo +fi + +if (( ${#NO_HOOK_CONFIG_REPOS[@]} > 0 )); then + echo "## No pre-commit framework" + echo + for r in "${NO_HOOK_CONFIG_REPOS[@]}"; do echo "- $r"; done + echo +fi + +# Exit 1 if any gap is present so this script can be wired into CI as a gate. +if (( ${#GAP_REPOS[@]} > 0 )); then + exit 1 +fi