From d7cc713eab800b03a4a100d328b0f81c07cb780b Mon Sep 17 00:00:00 2001 From: Calvin Smithu Date: Thu, 19 Mar 2026 12:59:49 -0600 Subject: [PATCH 01/15] initial onboarding plugin w/improved skills --- plugins/onboarding/agent-onboarding/SKILL.md | 151 ++++++++++++++++++ .../agent-onboarding/references/criteria.md | 144 +++++++++++++++++ plugins/onboarding/agent-readiness/SKILL.md | 121 ++++++++++++++ .../agent-readiness/references/criteria.md | 144 +++++++++++++++++ .../scripts/scan_agent_instructions.sh | 88 ++++++++++ .../agent-readiness/scripts/scan_build_env.sh | 114 +++++++++++++ .../scripts/scan_feedback_loops.sh | 133 +++++++++++++++ .../agent-readiness/scripts/scan_policy.sh | 113 +++++++++++++ .../agent-readiness/scripts/scan_workflows.sh | 127 +++++++++++++++ 9 files changed, 1135 insertions(+) create mode 100644 plugins/onboarding/agent-onboarding/SKILL.md create mode 100644 plugins/onboarding/agent-onboarding/references/criteria.md create mode 100644 plugins/onboarding/agent-readiness/SKILL.md create mode 100644 plugins/onboarding/agent-readiness/references/criteria.md create mode 100755 plugins/onboarding/agent-readiness/scripts/scan_agent_instructions.sh create mode 100755 plugins/onboarding/agent-readiness/scripts/scan_build_env.sh create mode 100755 plugins/onboarding/agent-readiness/scripts/scan_feedback_loops.sh create mode 100755 plugins/onboarding/agent-readiness/scripts/scan_policy.sh create mode 100755 plugins/onboarding/agent-readiness/scripts/scan_workflows.sh diff --git a/plugins/onboarding/agent-onboarding/SKILL.md b/plugins/onboarding/agent-onboarding/SKILL.md new file mode 100644 index 0000000..9662923 --- /dev/null +++ b/plugins/onboarding/agent-onboarding/SKILL.md @@ -0,0 +1,151 @@ +--- +name: agent-onboarding +description: Use an agent readiness report to identify and implement improvements that make a repository more agent-friendly. Proposes repo-appropriate fixes for each pillar, then implements them on request. Use after running `/readiness-report` or when a user wants to improve their repo's AI-readiness. +triggers: +- /agent-onboarding +--- + +# Agent Onboarding + +Take a readiness report and turn its gaps into concrete, repo-appropriate fixes. + +## Prerequisites + +This skill expects an agent readiness report to already exist — either from +running `/readiness-report` on this repo or provided by the user. If no report +exists, run the readiness report skill first. + +## How to run + +### Step 1: Read the report + +Identify every **✗** (missing) feature across all five pillars. + +### Step 2: Propose high-impact fixes + +Don't try to fix everything. Pick the **5–10 changes that would help agents the +most** across all pillars, ranked by impact. Not every pillar needs to be +represented — focus on what matters most for this repo. The goal is to +meaningfully improve the score, not to hit 74/74. + +Proposals should fit **this specific repo**: + +- Look at the languages, frameworks, and tools already in use +- Look at the project structure (monorepo vs single package, etc.) +- Look at what's already partially in place (don't reinvent what exists) +- Propose additions that match the repo's existing conventions and style + +For each proposal, include: +- What to add or change +- Where it goes (file path) +- Which pillar(s) it improves +- **Why it's high impact** — use the feature descriptions in + `references/criteria.md` to explain what goes wrong for agents without this + feature. Don't just say "improves Agent Instructions" — say what the agent + can't do today and what it'll be able to do after the fix. + +### Step 3: Implement on request + +When the user approves fixes (all or a subset), implement them, then **update the +readiness report** to reflect the new state. Flip each addressed feature from ✗ +to ✓ and update the pillar counts and summary. This ensures the next run of this +skill won't re-propose fixes that have already been applied. + +Follow these rules: + +- **Don't generate boilerplate** — content should be specific to this repo +- **Match existing style** — if the repo uses tabs, use tabs; if docs are in + RDoc, don't write Markdown +- **Don't over-generate** — a concise, accurate AGENTS.md beats a long generic + one +- **Commit atomically** — one commit per logical fix, not one giant commit + +## Pillar-specific guidance + +### Agent Instructions fixes + +The highest-impact fix is almost always an **agent instruction file** at the +root. A good one includes: + +- Build / test / lint commands (copy from CI config or Makefile, not invented) +- Project structure overview (what's in each top-level directory) +- Key conventions (naming, architecture patterns, things to avoid) +- Where to find more documentation + +If the repo is a monorepo, also consider per-component instruction files that +cover component-specific conventions. + +Other common fixes: +- Add AI IDE configuration if the team uses Cursor, Copilot, or Claude Code +- Create a contributing guide if one doesn't exist +- Add `.env.example` if the project uses environment variables + +### Feedback Loops fixes + +Focus on what gives agents the fastest signal: + +- **Pre-commit hooks** are the single fastest feedback loop — configure them with + the linter and formatter the project already uses +- **Test run documentation** in the agent instruction file — agents need to know + *exactly* which command runs which tests +- If the project has no linter or formatter configured, propose one that matches + the language ecosystem (don't propose Prettier for a Go project) + +### Workflows & Automation fixes + +Focus on structure that helps agents understand the process: + +- **PR template** with a checklist — agents follow checklists well +- **Issue templates** for bugs and features — gives agents structured input +- **CI concurrency control** — prevents agent-triggered CI from piling up + +### Policy & Governance fixes + +Focus on boundaries the agent needs to know: + +- **CODEOWNERS** — agents should know who owns what before making changes +- **Security policy** — agents need to know not to file public security issues +- Agent-aware **.gitignore** entries — prevent agent config files from being + committed accidentally + +### Build & Dev Environment fixes + +Focus on reproducibility: + +- **Dependency lockfile** if one doesn't exist — agents can't work with + non-deterministic installs +- **Tool version pinning** — prevents "works on my machine" failures +- **Single-command setup** — document or script the full bootstrap + +## Output format + +Present proposals like this: + +``` +# Agent Onboarding Proposals: {repo name} + +Ranked by impact. Implementing all of these would improve: +Agent Instructions (+4), Feedback Loops (+2), Policy & Governance (+1) + +1. **Create AGENTS.md** — Agent Instructions + - Include build commands from Makefile, test commands from CI, project structure + - Path: `./AGENTS.md` + - Right now agents have no way to learn this repo's conventions, banned + patterns, or how to build/test — they'll guess and get it wrong. + +2. **Add pre-commit hooks** — Feedback Loops + - Configure with ruff (already in pyproject.toml) and mypy + - Path: `./.pre-commit-config.yaml` + - Agents currently don't find out about lint/type errors until CI runs. + Pre-commit hooks catch these in seconds instead of minutes. + +3. **Add .env.example** — Agent Instructions, Build & Dev Environment + - Document the 3 env vars referenced in docker-compose.yml + - Path: `./.env.example` + - Agents can't start the dev server without knowing which env vars to set. + They'll either skip setup or hallucinate values. + +... + +Ready to implement? Reply with "all" or specify which proposals to apply. +``` diff --git a/plugins/onboarding/agent-onboarding/references/criteria.md b/plugins/onboarding/agent-onboarding/references/criteria.md new file mode 100644 index 0000000..1fe2788 --- /dev/null +++ b/plugins/onboarding/agent-onboarding/references/criteria.md @@ -0,0 +1,144 @@ +# Agent Readiness Criteria + +Features that make a codebase ready for AI-assisted development, organized into +five pillars. Derived from cluster analysis of 123 repositories, but written to +be tool-agnostic. Every feature answers one question: *if this is missing, what +goes wrong for the agent?* + +--- + +## Pillar 1 · Agent Instructions + +How the repo tells AI agents what to do, what to avoid, and how the codebase +works. This is the highest-signal pillar — it's the difference between an agent +that understands the project and one that's guessing. + +| # | Feature | What to look for | Evidence | +|---|---------|------------------|----------| +| 1 | **Agent instruction file** | A dedicated file telling agents how to work in this repo — conventions, banned patterns, common commands | `AGENTS.md`, `CLAUDE.md`, `COPILOT.md`, `CONVENTIONS.md` at root | +| 2 | **AI IDE configuration** | Settings or rules for AI-powered editors/IDEs | `.cursor/rules/`, `.cursorrules`, `.github/copilot-instructions.md`, `.github/instructions/`, `.claude/settings.json` | +| 3 | **Multi-model support** | Instructions that work across different AI models/tools, not locked to one vendor | 2+ distinct agent config types from features 1–2 present in same repo | +| 4 | **Agent skills or capabilities** | Packaged, reusable abilities the agent can invoke | `.claude/skills/`, `.factory/skills/`, `skill.md` files, tool definition files | +| 5 | **Tool server configuration** | Config for agent tool protocols (lets agents use external tools) | `.mcp.json`, `mcp.config.js`, tool server manifests | +| 6 | **Agent prompt library** | Pre-built prompts for common tasks in this repo | `.github/prompts/`, `prompts/` directory, prompt template files | +| 7 | **Component-level agent guidance** | Different parts of the codebase have their own agent instructions | `AGENTS.md` or instruction files in subdirectories (e.g. `frontend/AGENTS.md`, `api/CLAUDE.md`) | +| 8 | **README with build/run/test** | README includes the commands to build, run, and test the project | `README.md` containing code blocks with build/install/test commands | +| 9 | **Contributing guide** | How to contribute — code style, PR process, commit conventions | `CONTRIBUTING.md`, `docs/contributing.md`, contributing section in README | +| 10 | **Architecture documentation** | High-level overview of how the system is structured and why | `ARCHITECTURE.md`, `docs/architecture/`, Mermaid/PlantUML diagrams, `doc/design/` | +| 11 | **API documentation** | Reference docs for the project's interfaces | `openapi.yaml`, generated HTML docs, `doc.go` files, Swagger UI, `api-docs/` | +| 12 | **Inline code documentation** | Doc comments, docstrings — agents read these to understand intent | JSDoc `/** */` blocks, Python docstrings, GoDoc comments, RDoc `#` blocks, Rust `///` | +| 13 | **Runnable examples** | Working example code the agent can study and imitate | `examples/` directory, `_examples/`, example apps with their own READMEs | +| 14 | **Changelog** | History of what changed and how entries should be written | `CHANGELOG.md`, `CHANGES.md`, `HISTORY.md`, release notes in GitHub Releases | +| 15 | **Environment variable documentation** | Template or docs for required env vars | `.env.example`, `.env.template`, env var table in README or docs | +| 16 | **Documentation site or directory** | Organized docs beyond the README | `docs/` directory, Docusaurus/Sphinx/MkDocs/VitePress config, published doc site | +| 17 | **Decision records** | Documented reasoning behind past architectural choices | `doc/adr/`, `decisions/`, `rfcs/`, numbered markdown decision files | +| 18 | **Module-level READMEs** | Individual packages/modules have their own READMEs | `packages/*/README.md`, `libs/*/README.md`, per-crate/per-module READMEs | + +--- + +## Pillar 2 · Feedback Loops + +How quickly and clearly the agent learns whether its changes are correct. Fast, +clear feedback is the difference between an agent that converges on a solution +and one that spirals. + +| # | Feature | What to look for | Evidence | +|---|---------|------------------|----------| +| 19 | **Linter** | Static analysis that catches bugs and style issues | `.eslintrc.*`, `ruff.toml`, `.golangci.yml`, `clippy.toml`, `pylintrc`, lint config in `pyproject.toml` | +| 20 | **Formatter** | Auto-formatter that enforces consistent style | `.prettierrc`, `rustfmt.toml`, `[tool.black]` in pyproject.toml, `gofmt`/`goimports` in CI | +| 21 | **Type checking** | Static type system or type checker | `tsconfig.json` with `strict`, `mypy.ini`, `[tool.mypy]`, `py.typed` marker, Go (inherent) | +| 22 | **Pre-commit hooks** | Checks that run before commit — catches issues before CI | `.pre-commit-config.yaml`, `.husky/`, `lefthook.yml`, `lint-staged` config | +| 23 | **Unit tests** | Tests for individual components | `*_test.go`, `*_test.py`, `*.spec.ts`, `test/`, `__tests__/`, test runner config | +| 24 | **Integration tests** | Tests that verify components work together | `test/integration/`, `tests/e2e/`, API test suites, test files with service dependencies | +| 25 | **End-to-end tests** | Full system/browser tests | Playwright config, Cypress config, Selenium tests, `e2e/` directory | +| 26 | **Test coverage measurement** | Coverage tracking so the agent knows if new code is tested | `.codecov.yml`, `coverageThreshold` in jest config, `--cov` in pytest, `cover` profile in Go | +| 27 | **CI pipeline** | Automated checks on every push or PR | `.github/workflows/ci.yml`, `.circleci/config.yml`, `.gitlab-ci.yml`, `Jenkinsfile` | +| 28 | **Fast CI feedback** | CI completes quickly enough for agent iteration | CI workflow with documented expected duration, parallel jobs, test splitting | +| 29 | **Test run documentation** | Agent knows exactly how to run which tests | Test commands in README, AGENTS.md, CONTRIBUTING.md, or Makefile help target | +| 30 | **Config/schema validation** | YAML, JSON, config files validated automatically | `yamllint` config, JSON schema `$schema` refs, `actionlint` in CI, `taplo` for TOML | +| 31 | **Snapshot or golden-file tests** | Tests that detect unexpected output changes | `__snapshots__/` directories, `.snap` files, `testdata/` golden files, VCR cassettes | +| 32 | **Benchmark suite** | Performance tests the agent can run to check for regressions | `bench/`, `benchmarks/`, `*_bench_test.go`, pytest-benchmark, Criterion.rs | +| 33 | **Warnings-as-errors** | Compiler/runtime warnings treated as failures | `-Werror`, `warningsAsErrors` in build config, `filterwarnings = error` in pytest | +| 34 | **Spell/typo checking** | Automated spelling checks in CI or hooks | `.cspell.json`, `codespell` in pre-commit, `typos.toml`, spell check CI step | + +--- + +## Pillar 3 · Workflows & Automation + +The processes that support agent-driven development — how work is structured, +tracked, and shipped. + +| # | Feature | What to look for | Evidence | +|---|---------|------------------|----------| +| 35 | **Issue templates** | Structured templates for bug reports, feature requests | `.github/ISSUE_TEMPLATE/` with `bug_report.md`, `feature_request.md`, `config.yml` | +| 36 | **PR template** | Template that guides PR descriptions | `.github/pull_request_template.md`, PR template with checklist items | +| 37 | **Dependency update automation** | Automated PRs for dependency updates | `.github/dependabot.yml`, `renovate.json`, `.renovaterc`, Renovate preset configs | +| 38 | **Release automation** | Automated release pipeline (build, tag, publish) | Release workflow in CI, `semantic-release` config, GoReleaser config, `release-please` | +| 39 | **Branch protection** | Protected main branch with required checks | Branch protection rules (inferred from merge queue config, required status checks in CI) | +| 40 | **Merge automation** | Merge queue, auto-merge, or merge bot | `merge_group` trigger in CI, Mergify config, auto-merge labels, `gh pr merge --auto` | +| 41 | **Task runner** | Single entry point for common commands | `Makefile`, `Justfile`, `Taskfile.yml`, `package.json` scripts section, `Rakefile` | +| 42 | **Structured change tracking** | Changesets, conventional commits, or similar discipline | `.changeset/` directory, `commitlint` config, conventional commit enforcement in CI | +| 43 | **CI concurrency control** | Cancel-in-progress, concurrency groups to avoid CI pile-up | `concurrency:` blocks in GitHub Actions, `cancel-in-progress: true` | +| 44 | **Automated release notes** | Changelog/release notes generated from commits or PRs | `release-please` config, `auto-changelog`, `git-cliff`, `conventional-changelog` | +| 45 | **Stale issue/PR management** | Automation to close or label stale items | `.github/workflows/stale.yml`, `stale` bot config, issue lifecycle labels | +| 46 | **Label automation** | Automatic PR/issue labeling based on paths or content | `.github/labeler.yml`, label-sync config, auto-label workflows | +| 47 | **Multi-platform CI** | CI matrix covering multiple OS, arch, or runtime versions | `matrix:` in CI with `os: [ubuntu, macos, windows]` or multiple language versions | +| 48 | **Deployment automation** | Automated deployment pipeline | Deploy workflow triggered on merge/tag, staging + production environments in CI | +| 49 | **Automated code review checks** | Bot-driven review checks beyond CI | Danger.js config, review bot config, required review assignments, CODEOWNERS + required reviews | + +--- + +## Pillar 4 · Policy & Governance + +Rules, ownership, and constraints the agent must know about and respect. + +| # | Feature | What to look for | Evidence | +|---|---------|------------------|----------| +| 50 | **Comprehensive .gitignore** | Covers secrets, build artifacts, IDE files, agent artifacts | `.gitignore` with entries for `.env`, `node_modules/`, build dirs, `.cursor/`, `.claude/` | +| 51 | **License** | Clear license at root | `LICENSE`, `MIT-LICENSE`, `COPYING`, `LICENSE.md` | +| 52 | **Code ownership** | File/directory ownership mapping | `CODEOWNERS`, `.github/CODEOWNERS`, docs describing area owners or maintainer teams | +| 53 | **Security policy** | How to report vulnerabilities | `SECURITY.md`, `.github/security.md`, security reporting instructions | +| 54 | **Code of conduct** | Community standards | `CODE_OF_CONDUCT.md`, conduct link in contributing guide | +| 55 | **AI usage policy** | Documented guidelines for AI/agent contributions | AI policy in AGENTS.md, CONTRIBUTING.md, or standalone doc; agent boundary definitions | +| 56 | **Secrets management** | Secrets handled via environment/vault, not hardcoded | References to `${{ secrets.* }}` in CI, vault config, `.env.example` without values, `git-secrets` | +| 57 | **Security scanning** | Automated vulnerability scanning in CI | `.github/workflows/codeql.yml`, Snyk config, `gosec` in CI, Trivy, Dependabot security alerts | +| 58 | **Git attributes** | Line endings, diff drivers, LFS, linguist overrides | `.gitattributes` with `text=auto`, `linguist-generated`, LFS tracking patterns | +| 59 | **Contributor agreement** | DCO sign-off or CLA process | DCO bot config, `Signed-off-by` requirement in contributing guide, CLA-assistant config | +| 60 | **Governance model** | Documented maintainer roles, decision-making process | `GOVERNANCE.md`, `MAINTAINERS.md`, governance section in docs, team/role descriptions | +| 61 | **CI workflow validation** | CI config itself is linted/validated | `actionlint` step in CI, `circleci config validate`, CI config schema validation | +| 62 | **Environment separation** | Distinct configs for dev/test/prod | `.env.test`, `.env.production`, environment-specific config directories, config per deploy target | + +--- + +## Pillar 5 · Build & Dev Environment + +Can the agent actually build, run, and iterate on the project? Reproducibility +and speed matter — an agent that can't build the project can't do anything. + +| # | Feature | What to look for | Evidence | +|---|---------|------------------|----------| +| 63 | **Dependency lockfile** | Pinned dependency versions for reproducible installs | `package-lock.json`, `yarn.lock`, `pnpm-lock.yaml`, `uv.lock`, `Cargo.lock`, `go.sum`, `Gemfile.lock`, `poetry.lock` | +| 64 | **Single-command build** | One documented command to build the entire project | `make build`, `npm run build`, `cargo build` documented in README or agent file | +| 65 | **Single-command dev setup** | One command to bootstrap a working dev environment | `bin/setup`, `make dev`, `scripts/bootstrap.sh`, `just setup` | +| 66 | **Dev container** | Containerized dev environment definition | `.devcontainer/devcontainer.json`, `devcontainer.json` at root | +| 67 | **Containerized services** | Docker-based local development stack | `Dockerfile`, `docker-compose.yml`, `compose.yaml` with dev services | +| 68 | **Reproducible environment** | Declarative, reproducible dev environment | `flake.nix`, `shell.nix`, `devbox.json`, hermetic build definitions | +| 69 | **Tool version pinning** | Runtime/tool versions pinned to a file | `.tool-versions`, `mise.toml`, `.node-version`, `.python-version`, `.ruby-version`, `rust-toolchain.toml` | +| 70 | **Monorepo orchestration** | Tooling for multi-package repositories | Workspace config in `package.json`, `pnpm-workspace.yaml`, Cargo workspace, Go multi-module, Nx/Turborepo/Bazel config | +| 71 | **Build caching** | Caching for faster rebuilds | CI cache steps (`actions/cache`), Turborepo remote cache, ccache/sccache config, layer caching in Docker | +| 72 | **Cross-platform support** | Builds on multiple OS/arch | CI matrix with multiple OS entries, cross-compilation configs, multi-arch Docker builds | +| 73 | **Cloud dev environment** | Cloud-based workspace configuration | `.devcontainer/` with Codespaces features, `.gitpod.yml`, cloud workspace config | +| 74 | **Package manager configuration** | Custom registry, auth, resolution settings | `.npmrc`, `pip.conf`, `.cargo/config.toml`, registry overrides, resolution overrides | + +--- + +## Summary + +| Pillar | Features | What it answers | +|--------|----------|-----------------| +| Agent Instructions | 18 | Does the agent know what to do? | +| Feedback Loops | 16 | Does the agent know if it's right? | +| Workflows & Automation | 15 | Does the process support agent work? | +| Policy & Governance | 13 | Does the agent know the rules? | +| Build & Dev Environment | 12 | Can the agent build and run the project? | +| **Total** | **74** | | diff --git a/plugins/onboarding/agent-readiness/SKILL.md b/plugins/onboarding/agent-readiness/SKILL.md new file mode 100644 index 0000000..6acd816 --- /dev/null +++ b/plugins/onboarding/agent-readiness/SKILL.md @@ -0,0 +1,121 @@ +--- +name: agent-readiness-report +description: Evaluate how well a codebase supports autonomous AI-assisted development. Analyzes repositories across five pillars (Agent Instructions, Feedback Loops, Workflows & Automation, Policy & Governance, Build & Dev Environment) covering 74 features. Use when users request `/readiness-report` or want to assess how agent-ready a repository is. +triggers: +- /readiness-report +--- + +# Agent Readiness Report + +Evaluate how well a repository supports autonomous AI-assisted development. + +## What this does + +Assess a codebase across five pillars that determine whether an AI agent can +work effectively in a repository. The output is a structured report identifying +what's present and what's missing. + +## Five Pillars + +| Pillar | Question | Features | +|--------|----------|----------| +| **Agent Instructions** | Does the agent know what to do? | 18 | +| **Feedback Loops** | Does the agent know if it's right? | 16 | +| **Workflows & Automation** | Does the process support agent work? | 15 | +| **Policy & Governance** | Does the agent know the rules? | 13 | +| **Build & Dev Environment** | Can the agent build and run the project? | 12 | + +74 features total. See `references/criteria.md` for the full list with +descriptions and evidence examples. + +## How to run + +### Step 1: Run the scanner scripts + +Five shell scripts gather filesystem signals — file existence, config patterns, +directory structures. They surface what's present so you don't have to run +dozens of `find` commands manually. + +```bash +bash scripts/scan_agent_instructions.sh /path/to/repo +bash scripts/scan_feedback_loops.sh /path/to/repo +bash scripts/scan_workflows.sh /path/to/repo +bash scripts/scan_policy.sh /path/to/repo +bash scripts/scan_build_env.sh /path/to/repo +``` + +Or scan all five at once: + +```bash +for s in scripts/scan_*.sh; do bash "$s" /path/to/repo; echo; done +``` + +**Important**: The scripts are helpers, not scorers. They find files and +patterns but do not evaluate quality. Many features require judgment that only +reading the actual files can provide — for example, whether a README includes +real build commands or just badges, whether inline documentation is systematic or +scattered, whether an AI usage policy has meaningful boundaries. + +### Step 2: Evaluate each feature + +Walk through `references/criteria.md` pillar by pillar. For each feature: + +1. Check the scanner output for relevant signals +2. For features the scanner can't fully evaluate, inspect the files yourself +3. Mark each feature: **✓** (present), **✗** (missing), or **—** (not applicable) + +Features that require judgment (not fully covered by scanners): +- Does the README actually contain build/run/test commands? (not just that it exists) +- Is inline documentation systematic across the public API? +- Are examples actually runnable? +- Does the contributing guide include code standards? +- Is there a meaningful AI usage policy? +- Is the architecture documentation current? +- Are tests documented well enough for an agent to run them? + +### Step 3: Write the report + +Structure the output as: + +``` +# Agent Readiness Report: {repo name} + +## Summary +- Features present: X / 74 +- Strongest pillar: {pillar} +- Weakest pillar: {pillar} + +## Pillar 1 · Agent Instructions (X / 18) +✓ Agent instruction file — AGENTS.md at root +✓ AI IDE configuration — .cursor/rules/ with 3 rule files +✗ Multi-model support — only Cursor configured +... + +## Pillar 2 · Feedback Loops (X / 16) +... + +## Pillar 3 · Workflows & Automation (X / 15) +... + +## Pillar 4 · Policy & Governance (X / 13) +... + +## Pillar 5 · Build & Dev Environment (X / 12) +... + +``` + +For each passing feature, briefly note what evidence you found. +For each failing feature, note what's missing. + +## What makes these features useful + +Every feature answers: *if this is missing, what goes wrong for the AI agent?* +Features like "agent instruction file" and "tool server configuration" exist +because agents need them. Features like "linter" and "CI pipeline" exist because +agents need fast, clear feedback on whether their changes are correct — not +because they're general best practices. + +The criteria were derived from analysis of 123 real repositories across five +AI-readiness categories, then filtered for features that actually affect agent +effectiveness. diff --git a/plugins/onboarding/agent-readiness/references/criteria.md b/plugins/onboarding/agent-readiness/references/criteria.md new file mode 100644 index 0000000..1fe2788 --- /dev/null +++ b/plugins/onboarding/agent-readiness/references/criteria.md @@ -0,0 +1,144 @@ +# Agent Readiness Criteria + +Features that make a codebase ready for AI-assisted development, organized into +five pillars. Derived from cluster analysis of 123 repositories, but written to +be tool-agnostic. Every feature answers one question: *if this is missing, what +goes wrong for the agent?* + +--- + +## Pillar 1 · Agent Instructions + +How the repo tells AI agents what to do, what to avoid, and how the codebase +works. This is the highest-signal pillar — it's the difference between an agent +that understands the project and one that's guessing. + +| # | Feature | What to look for | Evidence | +|---|---------|------------------|----------| +| 1 | **Agent instruction file** | A dedicated file telling agents how to work in this repo — conventions, banned patterns, common commands | `AGENTS.md`, `CLAUDE.md`, `COPILOT.md`, `CONVENTIONS.md` at root | +| 2 | **AI IDE configuration** | Settings or rules for AI-powered editors/IDEs | `.cursor/rules/`, `.cursorrules`, `.github/copilot-instructions.md`, `.github/instructions/`, `.claude/settings.json` | +| 3 | **Multi-model support** | Instructions that work across different AI models/tools, not locked to one vendor | 2+ distinct agent config types from features 1–2 present in same repo | +| 4 | **Agent skills or capabilities** | Packaged, reusable abilities the agent can invoke | `.claude/skills/`, `.factory/skills/`, `skill.md` files, tool definition files | +| 5 | **Tool server configuration** | Config for agent tool protocols (lets agents use external tools) | `.mcp.json`, `mcp.config.js`, tool server manifests | +| 6 | **Agent prompt library** | Pre-built prompts for common tasks in this repo | `.github/prompts/`, `prompts/` directory, prompt template files | +| 7 | **Component-level agent guidance** | Different parts of the codebase have their own agent instructions | `AGENTS.md` or instruction files in subdirectories (e.g. `frontend/AGENTS.md`, `api/CLAUDE.md`) | +| 8 | **README with build/run/test** | README includes the commands to build, run, and test the project | `README.md` containing code blocks with build/install/test commands | +| 9 | **Contributing guide** | How to contribute — code style, PR process, commit conventions | `CONTRIBUTING.md`, `docs/contributing.md`, contributing section in README | +| 10 | **Architecture documentation** | High-level overview of how the system is structured and why | `ARCHITECTURE.md`, `docs/architecture/`, Mermaid/PlantUML diagrams, `doc/design/` | +| 11 | **API documentation** | Reference docs for the project's interfaces | `openapi.yaml`, generated HTML docs, `doc.go` files, Swagger UI, `api-docs/` | +| 12 | **Inline code documentation** | Doc comments, docstrings — agents read these to understand intent | JSDoc `/** */` blocks, Python docstrings, GoDoc comments, RDoc `#` blocks, Rust `///` | +| 13 | **Runnable examples** | Working example code the agent can study and imitate | `examples/` directory, `_examples/`, example apps with their own READMEs | +| 14 | **Changelog** | History of what changed and how entries should be written | `CHANGELOG.md`, `CHANGES.md`, `HISTORY.md`, release notes in GitHub Releases | +| 15 | **Environment variable documentation** | Template or docs for required env vars | `.env.example`, `.env.template`, env var table in README or docs | +| 16 | **Documentation site or directory** | Organized docs beyond the README | `docs/` directory, Docusaurus/Sphinx/MkDocs/VitePress config, published doc site | +| 17 | **Decision records** | Documented reasoning behind past architectural choices | `doc/adr/`, `decisions/`, `rfcs/`, numbered markdown decision files | +| 18 | **Module-level READMEs** | Individual packages/modules have their own READMEs | `packages/*/README.md`, `libs/*/README.md`, per-crate/per-module READMEs | + +--- + +## Pillar 2 · Feedback Loops + +How quickly and clearly the agent learns whether its changes are correct. Fast, +clear feedback is the difference between an agent that converges on a solution +and one that spirals. + +| # | Feature | What to look for | Evidence | +|---|---------|------------------|----------| +| 19 | **Linter** | Static analysis that catches bugs and style issues | `.eslintrc.*`, `ruff.toml`, `.golangci.yml`, `clippy.toml`, `pylintrc`, lint config in `pyproject.toml` | +| 20 | **Formatter** | Auto-formatter that enforces consistent style | `.prettierrc`, `rustfmt.toml`, `[tool.black]` in pyproject.toml, `gofmt`/`goimports` in CI | +| 21 | **Type checking** | Static type system or type checker | `tsconfig.json` with `strict`, `mypy.ini`, `[tool.mypy]`, `py.typed` marker, Go (inherent) | +| 22 | **Pre-commit hooks** | Checks that run before commit — catches issues before CI | `.pre-commit-config.yaml`, `.husky/`, `lefthook.yml`, `lint-staged` config | +| 23 | **Unit tests** | Tests for individual components | `*_test.go`, `*_test.py`, `*.spec.ts`, `test/`, `__tests__/`, test runner config | +| 24 | **Integration tests** | Tests that verify components work together | `test/integration/`, `tests/e2e/`, API test suites, test files with service dependencies | +| 25 | **End-to-end tests** | Full system/browser tests | Playwright config, Cypress config, Selenium tests, `e2e/` directory | +| 26 | **Test coverage measurement** | Coverage tracking so the agent knows if new code is tested | `.codecov.yml`, `coverageThreshold` in jest config, `--cov` in pytest, `cover` profile in Go | +| 27 | **CI pipeline** | Automated checks on every push or PR | `.github/workflows/ci.yml`, `.circleci/config.yml`, `.gitlab-ci.yml`, `Jenkinsfile` | +| 28 | **Fast CI feedback** | CI completes quickly enough for agent iteration | CI workflow with documented expected duration, parallel jobs, test splitting | +| 29 | **Test run documentation** | Agent knows exactly how to run which tests | Test commands in README, AGENTS.md, CONTRIBUTING.md, or Makefile help target | +| 30 | **Config/schema validation** | YAML, JSON, config files validated automatically | `yamllint` config, JSON schema `$schema` refs, `actionlint` in CI, `taplo` for TOML | +| 31 | **Snapshot or golden-file tests** | Tests that detect unexpected output changes | `__snapshots__/` directories, `.snap` files, `testdata/` golden files, VCR cassettes | +| 32 | **Benchmark suite** | Performance tests the agent can run to check for regressions | `bench/`, `benchmarks/`, `*_bench_test.go`, pytest-benchmark, Criterion.rs | +| 33 | **Warnings-as-errors** | Compiler/runtime warnings treated as failures | `-Werror`, `warningsAsErrors` in build config, `filterwarnings = error` in pytest | +| 34 | **Spell/typo checking** | Automated spelling checks in CI or hooks | `.cspell.json`, `codespell` in pre-commit, `typos.toml`, spell check CI step | + +--- + +## Pillar 3 · Workflows & Automation + +The processes that support agent-driven development — how work is structured, +tracked, and shipped. + +| # | Feature | What to look for | Evidence | +|---|---------|------------------|----------| +| 35 | **Issue templates** | Structured templates for bug reports, feature requests | `.github/ISSUE_TEMPLATE/` with `bug_report.md`, `feature_request.md`, `config.yml` | +| 36 | **PR template** | Template that guides PR descriptions | `.github/pull_request_template.md`, PR template with checklist items | +| 37 | **Dependency update automation** | Automated PRs for dependency updates | `.github/dependabot.yml`, `renovate.json`, `.renovaterc`, Renovate preset configs | +| 38 | **Release automation** | Automated release pipeline (build, tag, publish) | Release workflow in CI, `semantic-release` config, GoReleaser config, `release-please` | +| 39 | **Branch protection** | Protected main branch with required checks | Branch protection rules (inferred from merge queue config, required status checks in CI) | +| 40 | **Merge automation** | Merge queue, auto-merge, or merge bot | `merge_group` trigger in CI, Mergify config, auto-merge labels, `gh pr merge --auto` | +| 41 | **Task runner** | Single entry point for common commands | `Makefile`, `Justfile`, `Taskfile.yml`, `package.json` scripts section, `Rakefile` | +| 42 | **Structured change tracking** | Changesets, conventional commits, or similar discipline | `.changeset/` directory, `commitlint` config, conventional commit enforcement in CI | +| 43 | **CI concurrency control** | Cancel-in-progress, concurrency groups to avoid CI pile-up | `concurrency:` blocks in GitHub Actions, `cancel-in-progress: true` | +| 44 | **Automated release notes** | Changelog/release notes generated from commits or PRs | `release-please` config, `auto-changelog`, `git-cliff`, `conventional-changelog` | +| 45 | **Stale issue/PR management** | Automation to close or label stale items | `.github/workflows/stale.yml`, `stale` bot config, issue lifecycle labels | +| 46 | **Label automation** | Automatic PR/issue labeling based on paths or content | `.github/labeler.yml`, label-sync config, auto-label workflows | +| 47 | **Multi-platform CI** | CI matrix covering multiple OS, arch, or runtime versions | `matrix:` in CI with `os: [ubuntu, macos, windows]` or multiple language versions | +| 48 | **Deployment automation** | Automated deployment pipeline | Deploy workflow triggered on merge/tag, staging + production environments in CI | +| 49 | **Automated code review checks** | Bot-driven review checks beyond CI | Danger.js config, review bot config, required review assignments, CODEOWNERS + required reviews | + +--- + +## Pillar 4 · Policy & Governance + +Rules, ownership, and constraints the agent must know about and respect. + +| # | Feature | What to look for | Evidence | +|---|---------|------------------|----------| +| 50 | **Comprehensive .gitignore** | Covers secrets, build artifacts, IDE files, agent artifacts | `.gitignore` with entries for `.env`, `node_modules/`, build dirs, `.cursor/`, `.claude/` | +| 51 | **License** | Clear license at root | `LICENSE`, `MIT-LICENSE`, `COPYING`, `LICENSE.md` | +| 52 | **Code ownership** | File/directory ownership mapping | `CODEOWNERS`, `.github/CODEOWNERS`, docs describing area owners or maintainer teams | +| 53 | **Security policy** | How to report vulnerabilities | `SECURITY.md`, `.github/security.md`, security reporting instructions | +| 54 | **Code of conduct** | Community standards | `CODE_OF_CONDUCT.md`, conduct link in contributing guide | +| 55 | **AI usage policy** | Documented guidelines for AI/agent contributions | AI policy in AGENTS.md, CONTRIBUTING.md, or standalone doc; agent boundary definitions | +| 56 | **Secrets management** | Secrets handled via environment/vault, not hardcoded | References to `${{ secrets.* }}` in CI, vault config, `.env.example` without values, `git-secrets` | +| 57 | **Security scanning** | Automated vulnerability scanning in CI | `.github/workflows/codeql.yml`, Snyk config, `gosec` in CI, Trivy, Dependabot security alerts | +| 58 | **Git attributes** | Line endings, diff drivers, LFS, linguist overrides | `.gitattributes` with `text=auto`, `linguist-generated`, LFS tracking patterns | +| 59 | **Contributor agreement** | DCO sign-off or CLA process | DCO bot config, `Signed-off-by` requirement in contributing guide, CLA-assistant config | +| 60 | **Governance model** | Documented maintainer roles, decision-making process | `GOVERNANCE.md`, `MAINTAINERS.md`, governance section in docs, team/role descriptions | +| 61 | **CI workflow validation** | CI config itself is linted/validated | `actionlint` step in CI, `circleci config validate`, CI config schema validation | +| 62 | **Environment separation** | Distinct configs for dev/test/prod | `.env.test`, `.env.production`, environment-specific config directories, config per deploy target | + +--- + +## Pillar 5 · Build & Dev Environment + +Can the agent actually build, run, and iterate on the project? Reproducibility +and speed matter — an agent that can't build the project can't do anything. + +| # | Feature | What to look for | Evidence | +|---|---------|------------------|----------| +| 63 | **Dependency lockfile** | Pinned dependency versions for reproducible installs | `package-lock.json`, `yarn.lock`, `pnpm-lock.yaml`, `uv.lock`, `Cargo.lock`, `go.sum`, `Gemfile.lock`, `poetry.lock` | +| 64 | **Single-command build** | One documented command to build the entire project | `make build`, `npm run build`, `cargo build` documented in README or agent file | +| 65 | **Single-command dev setup** | One command to bootstrap a working dev environment | `bin/setup`, `make dev`, `scripts/bootstrap.sh`, `just setup` | +| 66 | **Dev container** | Containerized dev environment definition | `.devcontainer/devcontainer.json`, `devcontainer.json` at root | +| 67 | **Containerized services** | Docker-based local development stack | `Dockerfile`, `docker-compose.yml`, `compose.yaml` with dev services | +| 68 | **Reproducible environment** | Declarative, reproducible dev environment | `flake.nix`, `shell.nix`, `devbox.json`, hermetic build definitions | +| 69 | **Tool version pinning** | Runtime/tool versions pinned to a file | `.tool-versions`, `mise.toml`, `.node-version`, `.python-version`, `.ruby-version`, `rust-toolchain.toml` | +| 70 | **Monorepo orchestration** | Tooling for multi-package repositories | Workspace config in `package.json`, `pnpm-workspace.yaml`, Cargo workspace, Go multi-module, Nx/Turborepo/Bazel config | +| 71 | **Build caching** | Caching for faster rebuilds | CI cache steps (`actions/cache`), Turborepo remote cache, ccache/sccache config, layer caching in Docker | +| 72 | **Cross-platform support** | Builds on multiple OS/arch | CI matrix with multiple OS entries, cross-compilation configs, multi-arch Docker builds | +| 73 | **Cloud dev environment** | Cloud-based workspace configuration | `.devcontainer/` with Codespaces features, `.gitpod.yml`, cloud workspace config | +| 74 | **Package manager configuration** | Custom registry, auth, resolution settings | `.npmrc`, `pip.conf`, `.cargo/config.toml`, registry overrides, resolution overrides | + +--- + +## Summary + +| Pillar | Features | What it answers | +|--------|----------|-----------------| +| Agent Instructions | 18 | Does the agent know what to do? | +| Feedback Loops | 16 | Does the agent know if it's right? | +| Workflows & Automation | 15 | Does the process support agent work? | +| Policy & Governance | 13 | Does the agent know the rules? | +| Build & Dev Environment | 12 | Can the agent build and run the project? | +| **Total** | **74** | | diff --git a/plugins/onboarding/agent-readiness/scripts/scan_agent_instructions.sh b/plugins/onboarding/agent-readiness/scripts/scan_agent_instructions.sh new file mode 100755 index 0000000..7df58f3 --- /dev/null +++ b/plugins/onboarding/agent-readiness/scripts/scan_agent_instructions.sh @@ -0,0 +1,88 @@ +#!/usr/bin/env bash +# Scan for Agent Instructions signals (Pillar 1) +# Helps the agent find relevant files — not a substitute for judgment. + +REPO="${1:-.}" +cd "$REPO" 2>/dev/null || { echo "Cannot access $REPO"; exit 1; } + +echo "=== Pillar 1: Agent Instructions ===" +echo "" + +echo "-- Agent instruction files --" +find . -maxdepth 3 -iname 'AGENTS.md' -o -iname 'CLAUDE.md' -o -iname 'COPILOT.md' \ + -o -iname 'CONVENTIONS.md' -o -iname 'CODING_GUIDELINES.md' 2>/dev/null | sort + +echo "" +echo "-- AI IDE configuration --" +for f in .cursor .cursor/rules .cursorrules .github/copilot-instructions.md \ + .github/instructions .claude .claude/settings.json; do + [ -e "$f" ] && echo "./$f" +done + +echo "" +echo "-- Agent skills --" +for d in .claude/skills .factory/skills; do + [ -d "$d" ] && echo "./$d/ ($(find "$d" -type f | wc -l | tr -d ' ') files)" +done +find . -maxdepth 3 -iname 'skill.md' 2>/dev/null | sort + +echo "" +echo "-- Tool server configuration --" +find . -maxdepth 2 -name '.mcp.json' -o -name 'mcp.config.*' 2>/dev/null | sort + +echo "" +echo "-- Agent prompt library --" +for d in .github/prompts prompts; do + [ -d "$d" ] && echo "./$d/ ($(find "$d" -type f | wc -l | tr -d ' ') files)" +done + +echo "" +echo "-- README --" +[ -f README.md ] && echo "./README.md ($(wc -l < README.md) lines)" || echo "(not found)" + +echo "" +echo "-- Contributing guide --" +find . -maxdepth 2 -iname 'CONTRIBUTING*' 2>/dev/null | sort + +echo "" +echo "-- Architecture docs --" +find . -maxdepth 3 -iname 'ARCHITECTURE*' -o -iname 'DESIGN*' 2>/dev/null | sort +for d in doc/adr docs/adr decisions rfcs doc/design; do + [ -d "$d" ] && echo "./$d/ ($(find "$d" -type f | wc -l | tr -d ' ') files)" +done + +echo "" +echo "-- API documentation --" +find . -maxdepth 3 -name 'openapi.yaml' -o -name 'openapi.json' -o -name 'swagger.yaml' \ + -o -name 'swagger.json' 2>/dev/null | sort +find . -maxdepth 2 -name 'doc.go' 2>/dev/null | head -5 +api_doc_count=$(find . -maxdepth 2 -name 'doc.go' 2>/dev/null | wc -l | tr -d ' ') +[ "$api_doc_count" -gt 5 ] && echo " ... and $((api_doc_count - 5)) more doc.go files" + +echo "" +echo "-- Changelog --" +find . -maxdepth 1 -iname 'CHANGELOG*' -o -iname 'CHANGES*' -o -iname 'HISTORY*' 2>/dev/null | sort + +echo "" +echo "-- Environment variable docs --" +find . -maxdepth 2 -name '.env.example' -o -name '.env.template' -o -name '.env.sample' 2>/dev/null | sort + +echo "" +echo "-- Documentation site / directory --" +[ -d docs ] && echo "./docs/ ($(find docs -type f | wc -l | tr -d ' ') files)" +for f in docusaurus.config.* mkdocs.yml conf.py .vitepress/config.*; do + find . -maxdepth 2 -name "$(basename "$f")" 2>/dev/null +done + +echo "" +echo "-- Examples directory --" +for d in examples _examples; do + [ -d "$d" ] && echo "./$d/ ($(find "$d" -type f | wc -l | tr -d ' ') files)" +done + +echo "" +echo "-- Module-level READMEs --" +find . -mindepth 2 -maxdepth 3 -name 'README.md' 2>/dev/null | head -10 +readme_count=$(find . -mindepth 2 -maxdepth 3 -name 'README.md' 2>/dev/null | wc -l | tr -d ' ') +[ "$readme_count" -gt 10 ] && echo " ... and $((readme_count - 10)) more" +echo " Total: $readme_count module READMEs" diff --git a/plugins/onboarding/agent-readiness/scripts/scan_build_env.sh b/plugins/onboarding/agent-readiness/scripts/scan_build_env.sh new file mode 100755 index 0000000..f854d0b --- /dev/null +++ b/plugins/onboarding/agent-readiness/scripts/scan_build_env.sh @@ -0,0 +1,114 @@ +#!/usr/bin/env bash +# Scan for Build & Dev Environment signals (Pillar 5) +# Helps the agent find relevant files — not a substitute for judgment. + +REPO="${1:-.}" +cd "$REPO" 2>/dev/null || { echo "Cannot access $REPO"; exit 1; } + +echo "=== Pillar 5: Build & Dev Environment ===" +echo "" + +echo "-- Dependency lockfiles --" +for f in package-lock.json yarn.lock pnpm-lock.yaml bun.lockb \ + Cargo.lock go.sum Gemfile.lock poetry.lock uv.lock \ + Pipfile.lock composer.lock pubspec.lock; do + [ -f "$f" ] && echo "./$f" +done + +echo "" +echo "-- Build commands --" +# Check for documented build commands in common files +for f in README.md AGENTS.md CONTRIBUTING.md Makefile Justfile; do + if [ -f "$f" ]; then + build_refs=$(grep -ci 'make build\|npm run build\|cargo build\|go build\|gradle build\|mvn.*package\|bundle exec' "$f" 2>/dev/null) + [ "$build_refs" -gt 0 ] && echo " $build_refs build command references in $f" + fi +done + +echo "" +echo "-- Setup scripts --" +for f in bin/setup script/setup scripts/setup.sh scripts/bootstrap.sh \ + script/bootstrap Makefile Justfile; do + [ -f "$f" ] && echo "./$f" +done + +echo "" +echo "-- Dev container --" +if [ -d .devcontainer ]; then + echo ".devcontainer/:" + ls -1 .devcontainer/ 2>/dev/null | while read f; do echo " $f"; done +elif [ -f devcontainer.json ]; then + echo "./devcontainer.json" +else + echo " (not found)" +fi + +echo "" +echo "-- Containerized services --" +find . -maxdepth 2 -name 'Dockerfile*' -o -name 'docker-compose*.yml' \ + -o -name 'docker-compose*.yaml' -o -name 'compose.yml' -o -name 'compose.yaml' 2>/dev/null | sort + +echo "" +echo "-- Reproducible environment --" +for f in flake.nix shell.nix default.nix devbox.json devbox.lock; do + [ -f "$f" ] && echo "./$f" +done + +echo "" +echo "-- Tool version pinning --" +for f in .tool-versions mise.toml .mise.toml .node-version .nvmrc \ + .python-version .ruby-version .go-version rust-toolchain.toml \ + rust-toolchain .java-version .sdkmanrc; do + [ -f "$f" ] && echo "./$f" +done + +echo "" +echo "-- Monorepo orchestration --" +# Check for workspace configs +if [ -f package.json ]; then + grep -q '"workspaces"' package.json 2>/dev/null && echo " workspaces in package.json" +fi +for f in pnpm-workspace.yaml lerna.json nx.json turbo.json rush.json; do + [ -f "$f" ] && echo "./$f" +done +if [ -f Cargo.toml ]; then + grep -q '\[workspace\]' Cargo.toml 2>/dev/null && echo " Cargo workspace in Cargo.toml" +fi +if [ -f go.work ]; then + echo "./go.work" +fi + +echo "" +echo "-- Build caching --" +if [ -d .github/workflows ]; then + cache_count=$(grep -rl 'actions/cache\|buildx.*cache\|turbo.*cache\|ccache\|sccache' .github/workflows/ 2>/dev/null | wc -l | tr -d ' ') + echo " $cache_count workflows with cache configuration" +fi +[ -f turbo.json ] && grep -q 'cache' turbo.json 2>/dev/null && echo " Turborepo cache config" + +echo "" +echo "-- Cross-platform support --" +if [ -d .github/workflows ]; then + for f in .github/workflows/*.yml .github/workflows/*.yaml; do + [ -f "$f" ] || continue + if grep -q 'matrix:' "$f" 2>/dev/null; then + os_line=$(grep -A10 'matrix:' "$f" | grep -i 'os:' | head -1 | tr -d ' ') + [ -n "$os_line" ] && echo " $(basename "$f"): $os_line" + fi + done +fi + +echo "" +echo "-- Cloud dev environment --" +[ -f .gitpod.yml ] && echo "./.gitpod.yml" +if [ -d .devcontainer ]; then + grep -q 'codespaces\|ghcr.io' .devcontainer/devcontainer.json 2>/dev/null \ + && echo " Codespaces support in devcontainer.json" +fi + +echo "" +echo "-- Package manager configuration --" +for f in .npmrc .yarnrc .yarnrc.yml .pnpmrc pip.conf .cargo/config.toml \ + .cargo/config gradle.properties; do + [ -f "$f" ] && echo "./$f" +done diff --git a/plugins/onboarding/agent-readiness/scripts/scan_feedback_loops.sh b/plugins/onboarding/agent-readiness/scripts/scan_feedback_loops.sh new file mode 100755 index 0000000..c312562 --- /dev/null +++ b/plugins/onboarding/agent-readiness/scripts/scan_feedback_loops.sh @@ -0,0 +1,133 @@ +#!/usr/bin/env bash +# Scan for Feedback Loops signals (Pillar 2) +# Helps the agent find relevant files — not a substitute for judgment. + +REPO="${1:-.}" +cd "$REPO" 2>/dev/null || { echo "Cannot access $REPO"; exit 1; } + +echo "=== Pillar 2: Feedback Loops ===" +echo "" + +echo "-- Linter configuration --" +find . -maxdepth 2 \( \ + -name '.eslintrc*' -o -name 'eslint.config.*' \ + -o -name 'ruff.toml' -o -name '.golangci.yml' -o -name '.golangci.yaml' \ + -o -name 'clippy.toml' -o -name '.clippy.toml' \ + -o -name '.pylintrc' -o -name 'pylintrc' \ + -o -name 'biome.json' -o -name 'biome.jsonc' \ + -o -name '.swiftlint.yml' -o -name '.ktlint*' \ + \) 2>/dev/null | sort +# Check pyproject.toml for ruff/pylint +if [ -f pyproject.toml ]; then + grep -l -i '\[tool\.ruff\]\|\[tool\.pylint\]\|\[tool\.flake8\]' pyproject.toml 2>/dev/null \ + && echo " (also: linter config in pyproject.toml)" +fi + +echo "" +echo "-- Formatter configuration --" +find . -maxdepth 2 \( \ + -name '.prettierrc*' -o -name 'prettier.config.*' \ + -o -name 'rustfmt.toml' -o -name '.rustfmt.toml' \ + -o -name '.clang-format' \ + -o -name '.editorconfig' \ + \) 2>/dev/null | sort +if [ -f pyproject.toml ]; then + grep -l -i '\[tool\.black\]\|\[tool\.ruff\.format\]\|\[tool\.isort\]' pyproject.toml 2>/dev/null \ + && echo " (also: formatter config in pyproject.toml)" +fi + +echo "" +echo "-- Type checking --" +find . -maxdepth 2 -name 'tsconfig.json' -o -name 'tsconfig.*.json' 2>/dev/null | sort +if [ -f tsconfig.json ]; then + grep -q '"strict"' tsconfig.json 2>/dev/null && echo " (strict mode in tsconfig.json)" +fi +find . -maxdepth 2 -name 'mypy.ini' -o -name '.mypy.ini' 2>/dev/null | sort +if [ -f pyproject.toml ]; then + grep -q '\[tool\.mypy\]\|\[tool\.pyright\]' pyproject.toml 2>/dev/null \ + && echo " (type checker config in pyproject.toml)" +fi +find . -maxdepth 1 -name 'py.typed' 2>/dev/null + +echo "" +echo "-- Pre-commit hooks --" +for f in .pre-commit-config.yaml .husky lefthook.yml .lefthook.yml; do + [ -e "$f" ] && echo "./$f" +done +if [ -f package.json ]; then + grep -q 'lint-staged' package.json 2>/dev/null && echo " (lint-staged in package.json)" +fi + +echo "" +echo "-- Test directories --" +for d in test tests __tests__ spec test/unit test/integration tests/unit tests/integration \ + test/e2e tests/e2e e2e cypress playwright; do + [ -d "$d" ] && echo "./$d/ ($(find "$d" -maxdepth 1 -type f | wc -l | tr -d ' ') top-level files)" +done + +echo "" +echo "-- Test file count --" +test_files=$(find . -maxdepth 5 \( \ + -name '*_test.go' -o -name '*_test.py' -o -name 'test_*.py' \ + -o -name '*.spec.ts' -o -name '*.test.ts' -o -name '*.spec.js' -o -name '*.test.js' \ + -o -name '*_test.rb' -o -name '*_spec.rb' \ + -o -name '*_test.rs' \ + \) 2>/dev/null | wc -l | tr -d ' ') +echo " $test_files test files found" + +echo "" +echo "-- Test coverage --" +find . -maxdepth 2 \( \ + -name '.codecov.yml' -o -name 'codecov.yml' \ + -o -name '.coveragerc' -o -name 'coverage.config.*' \ + -o -name 'jest.config.*' \ + \) 2>/dev/null | sort +if [ -f pyproject.toml ]; then + grep -q '\[tool\.coverage\]\|\[tool\.pytest\.ini_options\]' pyproject.toml 2>/dev/null \ + && echo " (coverage/pytest config in pyproject.toml)" +fi +if [ -f jest.config.js ] || [ -f jest.config.ts ]; then + grep -l 'coverageThreshold' jest.config.* 2>/dev/null +fi + +echo "" +echo "-- CI pipelines --" +if [ -d .github/workflows ]; then + echo ".github/workflows/:" + ls -1 .github/workflows/*.yml .github/workflows/*.yaml 2>/dev/null | while read f; do + echo " $(basename "$f")" + done +fi +for f in .circleci/config.yml .gitlab-ci.yml Jenkinsfile .travis.yml; do + [ -f "$f" ] && echo "./$f" +done + +echo "" +echo "-- Config/schema validation --" +find . -maxdepth 2 -name '.yamllint*' -o -name 'taplo.toml' 2>/dev/null | sort +if [ -d .github/workflows ]; then + grep -rl 'actionlint\|yamllint\|schema.*validate' .github/workflows/ 2>/dev/null | head -3 +fi + +echo "" +echo "-- Snapshot tests --" +snap_count=$(find . -maxdepth 5 -name '__snapshots__' -o -name '*.snap' 2>/dev/null | wc -l | tr -d ' ') +golden_count=$(find . -maxdepth 4 -name 'testdata' -type d 2>/dev/null | wc -l | tr -d ' ') +echo " $snap_count snapshot dirs/files, $golden_count testdata dirs" + +echo "" +echo "-- Benchmark suite --" +for d in bench benchmarks benchmark; do + [ -d "$d" ] && echo "./$d/ ($(find "$d" -type f | wc -l | tr -d ' ') files)" +done +bench_files=$(find . -maxdepth 4 -name '*_bench_test.go' -o -name '*benchmark*' -type f 2>/dev/null | wc -l | tr -d ' ') +echo " $bench_files benchmark files found" + +echo "" +echo "-- Spell checking --" +find . -maxdepth 2 -name '.cspell.json' -o -name 'cspell.json' -o -name 'typos.toml' \ + -o -name '.typos.toml' 2>/dev/null | sort +if [ -f .pre-commit-config.yaml ]; then + grep -q 'codespell\|cspell\|typos' .pre-commit-config.yaml 2>/dev/null \ + && echo " (spell checker in pre-commit config)" +fi diff --git a/plugins/onboarding/agent-readiness/scripts/scan_policy.sh b/plugins/onboarding/agent-readiness/scripts/scan_policy.sh new file mode 100755 index 0000000..6b10a8d --- /dev/null +++ b/plugins/onboarding/agent-readiness/scripts/scan_policy.sh @@ -0,0 +1,113 @@ +#!/usr/bin/env bash +# Scan for Policy & Governance signals (Pillar 4) +# Helps the agent find relevant files — not a substitute for judgment. + +REPO="${1:-.}" +cd "$REPO" 2>/dev/null || { echo "Cannot access $REPO"; exit 1; } + +echo "=== Pillar 4: Policy & Governance ===" +echo "" + +echo "-- .gitignore --" +if [ -f .gitignore ]; then + lines=$(wc -l < .gitignore | tr -d ' ') + echo "./.gitignore ($lines lines)" + # Check for agent-aware entries + agent_entries=$(grep -ci 'cursor\|claude\|copilot\|\.agent\|\.mcp' .gitignore 2>/dev/null) + [ "$agent_entries" -gt 0 ] && echo " ($agent_entries agent-related ignore entries)" +else + echo " (not found)" +fi + +echo "" +echo "-- License --" +find . -maxdepth 1 -iname 'LICENSE*' -o -iname 'COPYING*' -o -iname 'MIT-LICENSE' 2>/dev/null | sort + +echo "" +echo "-- Code ownership --" +find . -maxdepth 3 -name 'CODEOWNERS' 2>/dev/null | sort +if [ -f CODEOWNERS ] || [ -f .github/CODEOWNERS ] || [ -f docs/CODEOWNERS ]; then + f=$(find . -maxdepth 3 -name 'CODEOWNERS' 2>/dev/null | head -1) + rules=$(grep -c '^[^#]' "$f" 2>/dev/null | tr -d ' ') + echo " ($rules ownership rules)" +fi + +echo "" +echo "-- Security policy --" +find . -maxdepth 3 -iname 'SECURITY*' 2>/dev/null | sort + +echo "" +echo "-- Code of conduct --" +find . -maxdepth 2 -iname 'CODE_OF_CONDUCT*' -o -iname 'CONDUCT*' 2>/dev/null | sort + +echo "" +echo "-- AI usage policy --" +# Check common locations for AI policy mentions +for f in AGENTS.md CLAUDE.md CONTRIBUTING.md; do + if [ -f "$f" ]; then + ai_mentions=$(grep -ci 'ai policy\|ai usage\|agent boundar\|ai contribut\|llm\|copilot' "$f" 2>/dev/null) + [ "$ai_mentions" -gt 0 ] && echo " $ai_mentions AI policy references in $f" + fi +done + +echo "" +echo "-- Secrets management --" +if [ -d .github/workflows ]; then + secrets_refs=$(grep -roh '\${{ secrets\.[A-Z_]*' .github/workflows/ 2>/dev/null | sort -u | wc -l | tr -d ' ') + echo " $secrets_refs distinct secrets referenced in CI" +fi +find . -maxdepth 2 -name '.env.example' -o -name '.env.template' 2>/dev/null | sort +find . -maxdepth 2 -name 'vault.hcl' -o -name '.vault-token' 2>/dev/null | sort + +echo "" +echo "-- Security scanning --" +if [ -d .github/workflows ]; then + for scanner in codeql snyk trivy gosec semgrep; do + grep -ril "$scanner" .github/workflows/ 2>/dev/null | while read f; do + echo " $scanner in $(basename "$f")" + done + done +fi +find . -maxdepth 2 -name '.snyk' -o -name '.trivyignore' 2>/dev/null | sort + +echo "" +echo "-- Git attributes --" +if [ -f .gitattributes ]; then + lines=$(wc -l < .gitattributes | tr -d ' ') + echo "./.gitattributes ($lines lines)" + linguist=$(grep -c 'linguist' .gitattributes 2>/dev/null) + [ "$linguist" -gt 0 ] && echo " ($linguist linguist overrides)" + lfs=$(grep -c 'filter=lfs' .gitattributes 2>/dev/null) + [ "$lfs" -gt 0 ] && echo " ($lfs LFS-tracked patterns)" +else + echo " (not found)" +fi + +echo "" +echo "-- Contributor agreement --" +find . -maxdepth 2 -iname 'DCO*' -o -iname 'CLA*' 2>/dev/null | sort +for f in CONTRIBUTING.md .github/workflows/*.yml; do + [ -f "$f" ] && grep -qi 'signed-off-by\|DCO\|CLA\|contributor license' "$f" 2>/dev/null \ + && echo " DCO/CLA reference in $(basename "$f")" +done + +echo "" +echo "-- Governance model --" +find . -maxdepth 2 -iname 'GOVERNANCE*' -o -iname 'MAINTAINERS*' -o -iname 'OWNERS*' 2>/dev/null | sort + +echo "" +echo "-- CI workflow validation --" +if [ -d .github/workflows ]; then + grep -rl 'actionlint' .github/workflows/ 2>/dev/null | head -3 +fi +if [ -f .pre-commit-config.yaml ]; then + grep -q 'actionlint' .pre-commit-config.yaml 2>/dev/null && echo " actionlint in pre-commit" +fi + +echo "" +echo "-- Environment separation --" +find . -maxdepth 2 -name '.env.test' -o -name '.env.production' -o -name '.env.staging' \ + -o -name '.env.development' 2>/dev/null | sort +for d in config/environments environments; do + [ -d "$d" ] && echo "./$d/ ($(ls "$d" | wc -l | tr -d ' ') environments)" +done diff --git a/plugins/onboarding/agent-readiness/scripts/scan_workflows.sh b/plugins/onboarding/agent-readiness/scripts/scan_workflows.sh new file mode 100755 index 0000000..ec48ffc --- /dev/null +++ b/plugins/onboarding/agent-readiness/scripts/scan_workflows.sh @@ -0,0 +1,127 @@ +#!/usr/bin/env bash +# Scan for Workflows & Automation signals (Pillar 3) +# Helps the agent find relevant files — not a substitute for judgment. + +REPO="${1:-.}" +cd "$REPO" 2>/dev/null || { echo "Cannot access $REPO"; exit 1; } + +echo "=== Pillar 3: Workflows & Automation ===" +echo "" + +echo "-- Issue templates --" +if [ -d .github/ISSUE_TEMPLATE ]; then + echo ".github/ISSUE_TEMPLATE/:" + ls -1 .github/ISSUE_TEMPLATE/ 2>/dev/null | while read f; do echo " $f"; done +else + echo " (not found)" +fi + +echo "" +echo "-- PR template --" +find . -maxdepth 3 -iname 'pull_request_template*' 2>/dev/null | sort +[ -f .github/pull_request_template.md ] || echo " (not found)" + +echo "" +echo "-- Dependency update automation --" +for f in .github/dependabot.yml .github/dependabot.yaml renovate.json .renovaterc \ + .renovaterc.json renovate.json5; do + [ -f "$f" ] && echo "./$f" +done + +echo "" +echo "-- Release automation --" +if [ -d .github/workflows ]; then + grep -ril 'release\|publish\|deploy' .github/workflows/ 2>/dev/null | while read f; do + echo " $(basename "$f")" + done +fi +for f in .releaserc .releaserc.json .releaserc.yml release-please-config.json \ + .goreleaser.yml .goreleaser.yaml; do + [ -f "$f" ] && echo "./$f" +done + +echo "" +echo "-- Branch protection signals --" +# Can't check GitHub settings from filesystem, but look for merge queue triggers +if [ -d .github/workflows ]; then + grep -rl 'merge_group' .github/workflows/ 2>/dev/null | while read f; do + echo " merge_group trigger in $(basename "$f")" + done +fi + +echo "" +echo "-- Merge automation --" +find . -maxdepth 2 -name '.mergify.yml' -o -name 'mergify.yml' 2>/dev/null | sort +if [ -d .github/workflows ]; then + grep -rl 'auto-merge\|automerge\|gh pr merge' .github/workflows/ 2>/dev/null | head -3 +fi + +echo "" +echo "-- Task runner --" +for f in Makefile GNUmakefile makefile Justfile justfile Taskfile.yml taskfile.yml \ + Rakefile Earthfile; do + [ -f "$f" ] && echo "./$f" +done +if [ -f package.json ]; then + script_count=$(python3 -c "import json; d=json.load(open('package.json')); print(len(d.get('scripts',{})))" 2>/dev/null) + [ -n "$script_count" ] && echo " package.json: $script_count scripts" +fi + +echo "" +echo "-- Structured change tracking --" +[ -d .changeset ] && echo ".changeset/ ($(ls .changeset/*.md 2>/dev/null | wc -l | tr -d ' ') pending changesets)" +find . -maxdepth 2 -name 'commitlint.config.*' -o -name '.commitlintrc*' 2>/dev/null | sort +if [ -d .github/workflows ]; then + grep -rl 'conventional-commits\|commitlint\|semantic-pull-request' .github/workflows/ 2>/dev/null | head -3 +fi + +echo "" +echo "-- CI concurrency control --" +if [ -d .github/workflows ]; then + concurrency_count=$(grep -rl 'concurrency:' .github/workflows/ 2>/dev/null | wc -l | tr -d ' ') + cancel_count=$(grep -rl 'cancel-in-progress' .github/workflows/ 2>/dev/null | wc -l | tr -d ' ') + echo " $concurrency_count workflows with concurrency groups, $cancel_count with cancel-in-progress" +fi + +echo "" +echo "-- Automated release notes --" +for f in release-please-config.json .github/release.yml cliff.toml .cliff.toml; do + [ -f "$f" ] && echo "./$f" +done +if [ -d .github/workflows ]; then + grep -rl 'auto-changelog\|conventional-changelog\|git-cliff\|release-please' .github/workflows/ 2>/dev/null | head -3 +fi + +echo "" +echo "-- Stale issue management --" +if [ -d .github/workflows ]; then + grep -rl 'stale' .github/workflows/ 2>/dev/null | while read f; do + echo " $(basename "$f")" + done +fi +[ -f .github/stale.yml ] && echo ".github/stale.yml" + +echo "" +echo "-- Label automation --" +find . -maxdepth 3 -name 'labeler.yml' -o -name '.github/labeler.yml' -o -name 'label-sync*' 2>/dev/null | sort + +echo "" +echo "-- Multi-platform CI --" +if [ -d .github/workflows ]; then + grep -rl 'matrix:' .github/workflows/ 2>/dev/null | while read f; do + os_line=$(grep -A5 'matrix:' "$f" | grep -i 'os:' | head -1) + [ -n "$os_line" ] && echo " $(basename "$f"): $os_line" + done +fi + +echo "" +echo "-- Deployment automation --" +if [ -d .github/workflows ]; then + grep -ril 'deploy' .github/workflows/ 2>/dev/null | while read f; do + echo " $(basename "$f")" + done +fi +[ -f vercel.json ] && echo "./vercel.json" +[ -f netlify.toml ] && echo "./netlify.toml" +[ -f fly.toml ] && echo "./fly.toml" +[ -f render.yaml ] && echo "./render.yaml" From 8a2311500fd7e4ee9f0a7d2044e193f66c277081 Mon Sep 17 00:00:00 2001 From: Calvin Smithu Date: Mon, 23 Mar 2026 07:29:48 -0600 Subject: [PATCH 02/15] agents-md skill, updating onboarding skill, moving pr review skill --- plugins/onboarding/agent-onboarding/SKILL.md | 15 ++ plugins/onboarding/agents-md/SKILL.md | 172 ++++++++++++++++++ .../onboarding}/github-pr-review/README.md | 0 .../onboarding}/github-pr-review/SKILL.md | 0 4 files changed, 187 insertions(+) create mode 100644 plugins/onboarding/agents-md/SKILL.md rename {skills => plugins/onboarding}/github-pr-review/README.md (100%) rename {skills => plugins/onboarding}/github-pr-review/SKILL.md (100%) diff --git a/plugins/onboarding/agent-onboarding/SKILL.md b/plugins/onboarding/agent-onboarding/SKILL.md index 9662923..a4cd276 100644 --- a/plugins/onboarding/agent-onboarding/SKILL.md +++ b/plugins/onboarding/agent-onboarding/SKILL.md @@ -28,6 +28,21 @@ most** across all pillars, ranked by impact. Not every pillar needs to be represented — focus on what matters most for this repo. The goal is to meaningfully improve the score, not to hit 74/74. +**Ranking heuristic** — rank by how directly the change helps an agent complete a +coding task in this repo: + +1. Things that unblock agents from working at all (AGENTS.md, build commands, + bootstrap scripts, dev environment setup) +2. Things that give agents faster feedback on their work (pre-commit hooks, test + documentation, PR templates with checklists) +3. Things that improve quality or process (CI caching, label automation, spell + checking, merge queues) +4. Things that improve governance or compliance (SECURITY.md, CODEOWNERS, + actionlint, CodeQL) + +A single change that lets an agent build and test the project outranks a 2-for-1 +that addresses minor gaps. + Proposals should fit **this specific repo**: - Look at the languages, frameworks, and tools already in use diff --git a/plugins/onboarding/agents-md/SKILL.md b/plugins/onboarding/agents-md/SKILL.md new file mode 100644 index 0000000..45c149b --- /dev/null +++ b/plugins/onboarding/agents-md/SKILL.md @@ -0,0 +1,172 @@ +--- +name: agents-md +description: Generate a high-quality, repo-specific AGENTS.md file that tells AI agents how to work effectively in a repository. Reads the actual repo to extract commands, structure, conventions, and guardrails — never generates generic boilerplate. Use when a user wants to create or improve their AGENTS.md, or after a readiness report identifies a missing agent instruction file. +triggers: +- /agents-md +--- + +# Generate AGENTS.md + +Create a repo-specific AGENTS.md that gives AI agents the context they need to +work effectively. Every section should reference real commands, real paths, and +real conventions from the repository — not generic advice. + +## Why this matters + +An AGENTS.md is the single highest-impact addition for agent readiness. It +directly addresses features across multiple pillars: + +- **Agent Instructions**: agent instruction file, README with build/run/test, + contributing guide, environment variable documentation +- **Feedback Loops**: test run documentation — agents need exact commands +- **Policy & Governance**: guardrails the agent must follow + +Without it, agents guess at build commands, miss project conventions, run the +wrong test suite, and don't know what's dangerous. The best AGENTS.md files +we've studied (airflow, CopilotKit, grafana, swc) all share the same core +pattern: real commands, clear structure, explicit boundaries. + +## How to run + +### Step 1: Read the repo + +Before writing anything, gather the actual information. Check these sources: + +**Commands** (most important — agents need to know how to build, test, lint): +- `Makefile` / `justfile` / `Taskfile.yml` — look for build/test/lint/format targets +- `package.json` `scripts` — npm/yarn/pnpm run targets +- `pyproject.toml` `[tool.prek]` / `[tool.pytest]` — Python tooling config +- `.github/workflows/*.yml` — CI steps reveal the real commands +- `Cargo.toml` — Rust build/test commands +- `build.gradle` / `pom.xml` — Java/Kotlin build commands +- `Rakefile` — Ruby task definitions +- `docker-compose.yml` — service orchestration commands + +**Project structure**: +- Top-level directory listing — what's in each major directory +- Monorepo indicators: workspace configs, multiple package.json/Cargo.toml/pyproject.toml +- Source vs test layout (where does code live, where do tests live) + +**Conventions**: +- Existing linter/formatter configs (.eslintrc, .prettierrc, rustfmt.toml, .rubocop.yml, ruff.toml) +- CONTRIBUTING.md — often contains coding standards +- Existing README — architecture descriptions, setup instructions +- `.editorconfig` — indentation/style basics + +**Guardrails**: +- Database configs — identify destructive commands to warn about +- CI enforcement — what must pass before merge +- Secrets/env patterns — `.env.example`, vault configs +- Branch protection or merge requirements + +### Step 2: Write the AGENTS.md + +Use this structure. Every section is optional — include only what's relevant to +this repo. A 30-line file with real commands beats a 200-line file with generic +advice. + +```markdown +# AGENTS.md + +## Project overview + +One or two sentences: what this project is, what language/framework, and the +high-level architecture (monorepo? client-server? library?). + +## Commands + +Organized by category. Use the exact commands from the repo — copy from +Makefile/CI/package.json, don't invent them. + +### Build +### Test +### Lint & format + +## Project structure + +What's in each top-level directory. Focus on what an agent needs to navigate +the codebase — not an exhaustive tree. + +## Coding standards + +Language-specific conventions that aren't captured by linter config. Things +like: naming patterns, import ordering preferences, test file location +conventions, preferred patterns over anti-patterns. + +## Testing + +Where tests live, how they're organized, how to run a single test vs the full +suite, any test database setup needed. + +## Guardrails + +Things the agent must NOT do. Destructive commands, files that shouldn't be +edited by hand, operations that require human approval. +``` + +### Key principles + +**Be specific, not generic.** Don't write "run the linter before committing" — +write `uv run ruff check --fix `. Don't write "follow the project's +coding standards" — write "use `snake_case` for variables, no `assert` in +production code." + +**Commands are king.** If the agent only reads one section, it should be +Commands. Every command block should be copy-pasteable. + +**Link, don't repeat.** If there's a detailed CONTRIBUTING.md or architecture +doc, link to it rather than duplicating the content. The AGENTS.md is an index +and quick reference, not a manual. + +**Include dangerous operations.** If `make db-reset` destroys the dev database, +say so. If `git push -f` to main is forbidden, say so. Agents are literal — +they need explicit guardrails. + +**Keep it maintainable.** A short, accurate AGENTS.md is better than a long one +that drifts out of date. Reference CI configs and Makefiles by path so a human +(or agent) can update the AGENTS.md when the underlying commands change. + +### What to leave out + +- Generic software engineering advice ("write clean code", "use meaningful names") +- Process documentation that belongs in CONTRIBUTING.md +- Full API documentation that belongs in doc comments or a docs site +- Operational runbooks that belong in an ops directory + +### Monorepo considerations + +If the repo is a monorepo, consider whether sub-directory AGENTS.md files are +warranted. The root AGENTS.md should cover repo-wide commands and conventions. +Per-package files should only exist if a package has substantially different +commands or conventions from the root. + +CopilotKit does this well: the root file covers Nx monorepo basics and +architecture, then links to `.claude/docs/` for deeper topic-specific guidance. +Grafana versions its root file and explicitly lists sub-directory AGENTS.md paths. + +## Examples from real repos + +These patterns come from studying the best AGENTS.md files across 125 +repositories: + +**Concise and effective** (CopilotKit, 32 lines): One-sentence architecture, +"Essentials" section with 4 bullet points, then links to detailed docs. An +agent can read the whole thing in seconds and knows where to find more. + +**Thorough command reference** (airflow, 215 lines): Every test command has a +real example with placeholders (``, ``). Repository +structure section maps paths to purposes. Architecture boundaries section +explains what talks to what. + +**Categorized commands** (grafana, 480 lines): Build, test, lint commands +separated into backend/frontend blocks with exact shell commands. Sub-directory +AGENTS.md files linked from root for specialized areas (docs, alerting). + +**Performance-first philosophy** (swc, 75 lines): States the project's core +value ("always prefer performance") upfront so every decision the agent makes +is anchored to it. Verification checklist tells the agent exactly what to run +before finishing. + +**Safety-critical guardrails** (Lychee, 122 lines): Explicit "NEVER RUN THESE +COMMANDS" section with database-destroying operations called out. Test database +isolation explained so the agent doesn't accidentally use production. diff --git a/skills/github-pr-review/README.md b/plugins/onboarding/github-pr-review/README.md similarity index 100% rename from skills/github-pr-review/README.md rename to plugins/onboarding/github-pr-review/README.md diff --git a/skills/github-pr-review/SKILL.md b/plugins/onboarding/github-pr-review/SKILL.md similarity index 100% rename from skills/github-pr-review/SKILL.md rename to plugins/onboarding/github-pr-review/SKILL.md From a940aba565234fac6e0469f565b51e13b3f8530f Mon Sep 17 00:00:00 2001 From: Calvin Smithu Date: Mon, 23 Mar 2026 07:41:21 -0600 Subject: [PATCH 03/15] slight re-org, adding readmes --- plugins/onboarding/README.md | 42 ++++++++++++ .../skills/agent-onboarding/README.md | 51 ++++++++++++++ .../{ => skills}/agent-onboarding/SKILL.md | 0 .../agent-onboarding/references/criteria.md | 0 .../skills/agent-readiness/README.md | 67 +++++++++++++++++++ .../{ => skills}/agent-readiness/SKILL.md | 0 .../agent-readiness/references/criteria.md | 0 .../scripts/scan_agent_instructions.sh | 0 .../agent-readiness/scripts/scan_build_env.sh | 0 .../scripts/scan_feedback_loops.sh | 0 .../agent-readiness/scripts/scan_policy.sh | 0 .../agent-readiness/scripts/scan_workflows.sh | 0 plugins/onboarding/skills/agents-md/README.md | 52 ++++++++++++++ .../{ => skills}/agents-md/SKILL.md | 0 .../github-pr-review/README.md | 0 .../github-pr-review/SKILL.md | 0 16 files changed, 212 insertions(+) create mode 100644 plugins/onboarding/README.md create mode 100644 plugins/onboarding/skills/agent-onboarding/README.md rename plugins/onboarding/{ => skills}/agent-onboarding/SKILL.md (100%) rename plugins/onboarding/{ => skills}/agent-onboarding/references/criteria.md (100%) create mode 100644 plugins/onboarding/skills/agent-readiness/README.md rename plugins/onboarding/{ => skills}/agent-readiness/SKILL.md (100%) rename plugins/onboarding/{ => skills}/agent-readiness/references/criteria.md (100%) rename plugins/onboarding/{ => skills}/agent-readiness/scripts/scan_agent_instructions.sh (100%) rename plugins/onboarding/{ => skills}/agent-readiness/scripts/scan_build_env.sh (100%) rename plugins/onboarding/{ => skills}/agent-readiness/scripts/scan_feedback_loops.sh (100%) rename plugins/onboarding/{ => skills}/agent-readiness/scripts/scan_policy.sh (100%) rename plugins/onboarding/{ => skills}/agent-readiness/scripts/scan_workflows.sh (100%) create mode 100644 plugins/onboarding/skills/agents-md/README.md rename plugins/onboarding/{ => skills}/agents-md/SKILL.md (100%) rename {plugins/onboarding => skills}/github-pr-review/README.md (100%) rename {plugins/onboarding => skills}/github-pr-review/SKILL.md (100%) diff --git a/plugins/onboarding/README.md b/plugins/onboarding/README.md new file mode 100644 index 0000000..f64c597 --- /dev/null +++ b/plugins/onboarding/README.md @@ -0,0 +1,42 @@ +# Onboarding Plugin + +Assess how well a repository supports AI-assisted development and implement targeted improvements. This plugin bundles skills that evaluate agent readiness, propose high-impact fixes, and generate repo-specific `AGENTS.md` files — everything needed to take a codebase from agent-unfriendly to agent-ready. + +## Quick Start + +``` +/readiness-report +``` + +Run the readiness report on any repository to get a scored evaluation across five pillars covering 74 features. Then use `/agent-onboarding` to turn gaps into concrete fixes, or `/agents-md` to generate the single highest-impact file for agent readiness. + +## Features + +- **Readiness Assessment** — Evaluates 74 features across five pillars with scanner scripts and manual judgment +- **Prioritized Improvement Plans** — Ranks fixes by agent impact, not checklist completeness +- **AGENTS.md Generation** — Produces repo-specific agent instruction files from real commands and conventions + +## Workflow + +1. **Assess** — `/readiness-report` scans the repo and produces a structured report +2. **Plan** — `/agent-onboarding` reads the report and proposes the 5–10 highest-impact fixes +3. **Implement** — Approved fixes are applied with atomic commits and the report is updated +4. **Generate** — `/agents-md` creates or improves the AGENTS.md using real repo data + +## Skills + +| Skill | Trigger | Description | +|-------|---------|-------------| +| [agent-readiness](skills/agent-readiness/) | `/readiness-report` | Evaluate a repo across five pillars (74 features) | +| [agent-onboarding](skills/agent-onboarding/) | `/agent-onboarding` | Propose and implement fixes from a readiness report | +| [agents-md](skills/agents-md/) | `/agents-md` | Generate a repo-specific AGENTS.md | + +## Five Pillars + +| Pillar | Question | Features | +|--------|----------|----------| +| **Agent Instructions** | Does the agent know what to do? | 18 | +| **Feedback Loops** | Does the agent know if it's right? | 16 | +| **Workflows & Automation** | Does the process support agent work? | 15 | +| **Policy & Governance** | Does the agent know the rules? | 13 | +| **Build & Dev Environment** | Can the agent build and run the project? | 12 | \ No newline at end of file diff --git a/plugins/onboarding/skills/agent-onboarding/README.md b/plugins/onboarding/skills/agent-onboarding/README.md new file mode 100644 index 0000000..15b7d13 --- /dev/null +++ b/plugins/onboarding/skills/agent-onboarding/README.md @@ -0,0 +1,51 @@ +# Agent Onboarding + +Take an agent readiness report and turn its gaps into concrete, repo-appropriate fixes. + +## Triggers + +This skill is activated by the following keywords: + +- `/agent-onboarding` + +## Prerequisites + +An agent readiness report must already exist — either from running `/readiness-report` on the target repo or provided directly by the user. If no report exists, run the readiness report skill first. + +## How It Works + +1. **Read the report** — identify every missing (✗) feature across all five pillars +2. **Propose 5–10 high-impact fixes** — ranked by how directly each change helps an agent complete coding tasks, not by checklist coverage +3. **Implement on request** — apply approved fixes with atomic commits, then update the readiness report to reflect the new state + +## Ranking Heuristic + +Proposals are ranked by agent impact: + +1. **Unblocking** — things that let agents work at all (AGENTS.md, build commands, bootstrap scripts) +2. **Fast feedback** — things that tell agents whether their changes are correct (pre-commit hooks, test documentation) +3. **Quality & process** — things that improve agent output quality (CI caching, label automation) +4. **Governance** — things that define boundaries (SECURITY.md, CODEOWNERS, CodeQL) + +A single change that lets an agent build and test a project outranks a two-for-one that addresses minor gaps. + +## Pillar-Specific Guidance + +The skill includes targeted advice for each pillar: + +- **Agent Instructions** — AGENTS.md, AI IDE config, contributing guides, `.env.example` +- **Feedback Loops** — pre-commit hooks, test run documentation, linter/formatter setup +- **Workflows & Automation** — PR templates, issue templates, CI concurrency control +- **Policy & Governance** — CODEOWNERS, security policy, agent-aware `.gitignore` +- **Build & Dev Environment** — dependency lockfiles, tool version pinning, single-command setup + +## Implementation Rules + +- Content is specific to the target repo — no boilerplate +- Matches existing style (tabs/spaces, doc format, naming conventions) +- One commit per logical fix +- Readiness report is updated after implementation + +## References + +- `references/criteria.md` — full 74-feature evaluation criteria with descriptions and evidence examples diff --git a/plugins/onboarding/agent-onboarding/SKILL.md b/plugins/onboarding/skills/agent-onboarding/SKILL.md similarity index 100% rename from plugins/onboarding/agent-onboarding/SKILL.md rename to plugins/onboarding/skills/agent-onboarding/SKILL.md diff --git a/plugins/onboarding/agent-onboarding/references/criteria.md b/plugins/onboarding/skills/agent-onboarding/references/criteria.md similarity index 100% rename from plugins/onboarding/agent-onboarding/references/criteria.md rename to plugins/onboarding/skills/agent-onboarding/references/criteria.md diff --git a/plugins/onboarding/skills/agent-readiness/README.md b/plugins/onboarding/skills/agent-readiness/README.md new file mode 100644 index 0000000..013a689 --- /dev/null +++ b/plugins/onboarding/skills/agent-readiness/README.md @@ -0,0 +1,67 @@ +# Agent Readiness Report + +Evaluate how well a repository supports autonomous AI-assisted development. + +## Triggers + +This skill is activated by the following keywords: + +- `/readiness-report` + +## What This Does + +Assesses a codebase across five pillars covering 74 features that determine whether an AI agent can work effectively in a repository. The output is a structured report identifying what's present, what's missing, and which pillars are strongest and weakest. + +## Five Pillars + +| Pillar | Question | Features | +|--------|----------|----------| +| **Agent Instructions** | Does the agent know what to do? | 18 | +| **Feedback Loops** | Does the agent know if it's right? | 16 | +| **Workflows & Automation** | Does the process support agent work? | 15 | +| **Policy & Governance** | Does the agent know the rules? | 13 | +| **Build & Dev Environment** | Can the agent build and run the project? | 12 | + +## How It Works + +### 1. Run the scanner scripts + +Five shell scripts gather filesystem signals — file existence, config patterns, directory structures: + +```bash +bash scripts/scan_agent_instructions.sh /path/to/repo +bash scripts/scan_feedback_loops.sh /path/to/repo +bash scripts/scan_workflows.sh /path/to/repo +bash scripts/scan_policy.sh /path/to/repo +bash scripts/scan_build_env.sh /path/to/repo +``` + +Or scan all five at once: + +```bash +for s in scripts/scan_*.sh; do bash "$s" /path/to/repo; echo; done +``` + +The scripts are helpers, not scorers. They find files and patterns but many features require judgment that only reading the actual files can provide. + +### 2. Evaluate each feature + +Walk through `references/criteria.md` pillar by pillar and mark each feature: **✓** (present), **✗** (missing), or **—** (not applicable). + +### 3. Write the report + +Structured output with per-pillar scores, evidence for passing features, and notes on what's missing. + +## Scanner Scripts + +| Script | Pillar | +|--------|--------| +| `scan_agent_instructions.sh` | Agent Instructions | +| `scan_feedback_loops.sh` | Feedback Loops | +| `scan_workflows.sh` | Workflows & Automation | +| `scan_policy.sh` | Policy & Governance | +| `scan_build_env.sh` | Build & Dev Environment | + +## References + +- `references/criteria.md` — full 74-feature evaluation criteria derived from analysis of 123 repositories diff --git a/plugins/onboarding/agent-readiness/SKILL.md b/plugins/onboarding/skills/agent-readiness/SKILL.md similarity index 100% rename from plugins/onboarding/agent-readiness/SKILL.md rename to plugins/onboarding/skills/agent-readiness/SKILL.md diff --git a/plugins/onboarding/agent-readiness/references/criteria.md b/plugins/onboarding/skills/agent-readiness/references/criteria.md similarity index 100% rename from plugins/onboarding/agent-readiness/references/criteria.md rename to plugins/onboarding/skills/agent-readiness/references/criteria.md diff --git a/plugins/onboarding/agent-readiness/scripts/scan_agent_instructions.sh b/plugins/onboarding/skills/agent-readiness/scripts/scan_agent_instructions.sh similarity index 100% rename from plugins/onboarding/agent-readiness/scripts/scan_agent_instructions.sh rename to plugins/onboarding/skills/agent-readiness/scripts/scan_agent_instructions.sh diff --git a/plugins/onboarding/agent-readiness/scripts/scan_build_env.sh b/plugins/onboarding/skills/agent-readiness/scripts/scan_build_env.sh similarity index 100% rename from plugins/onboarding/agent-readiness/scripts/scan_build_env.sh rename to plugins/onboarding/skills/agent-readiness/scripts/scan_build_env.sh diff --git a/plugins/onboarding/agent-readiness/scripts/scan_feedback_loops.sh b/plugins/onboarding/skills/agent-readiness/scripts/scan_feedback_loops.sh similarity index 100% rename from plugins/onboarding/agent-readiness/scripts/scan_feedback_loops.sh rename to plugins/onboarding/skills/agent-readiness/scripts/scan_feedback_loops.sh diff --git a/plugins/onboarding/agent-readiness/scripts/scan_policy.sh b/plugins/onboarding/skills/agent-readiness/scripts/scan_policy.sh similarity index 100% rename from plugins/onboarding/agent-readiness/scripts/scan_policy.sh rename to plugins/onboarding/skills/agent-readiness/scripts/scan_policy.sh diff --git a/plugins/onboarding/agent-readiness/scripts/scan_workflows.sh b/plugins/onboarding/skills/agent-readiness/scripts/scan_workflows.sh similarity index 100% rename from plugins/onboarding/agent-readiness/scripts/scan_workflows.sh rename to plugins/onboarding/skills/agent-readiness/scripts/scan_workflows.sh diff --git a/plugins/onboarding/skills/agents-md/README.md b/plugins/onboarding/skills/agents-md/README.md new file mode 100644 index 0000000..58887f2 --- /dev/null +++ b/plugins/onboarding/skills/agents-md/README.md @@ -0,0 +1,52 @@ +# Generate AGENTS.md + +Generate a high-quality, repo-specific AGENTS.md file that tells AI agents how to work effectively in a repository. + +## Triggers + +This skill is activated by the following keywords: + +- `/agents-md` + +## What This Does + +Reads the actual repository — Makefiles, CI configs, package.json scripts, linter configs, directory structure — and produces an AGENTS.md with real commands, real paths, and real conventions. Never generates generic boilerplate. + +## Why AGENTS.md Matters + +An AGENTS.md is the single highest-impact addition for agent readiness. Without it, agents guess at build commands, miss project conventions, run the wrong test suite, and don't know what's dangerous. + +## Generated Sections + +All sections are optional — only what's relevant to the repo is included: + +| Section | Contents | +|---------|----------| +| **Project overview** | What the project is, language/framework, high-level architecture | +| **Commands** | Build, test, lint, and format commands copied from the repo's actual configs | +| **Project structure** | What's in each top-level directory | +| **Coding standards** | Conventions not captured by linter config | +| **Testing** | Where tests live, how to run them, any setup needed | +| **Guardrails** | Things the agent must NOT do — destructive commands, protected files | + +## Key Principles + +- **Be specific, not generic** — `uv run ruff check --fix `, not "run the linter" +- **Commands are king** — every command block is copy-pasteable +- **Link, don't repeat** — reference existing docs rather than duplicating them +- **Include dangerous operations** — agents are literal and need explicit guardrails +- **Keep it maintainable** — a short, accurate file beats a long one that drifts + +## Monorepo Support + +For monorepos, the root AGENTS.md covers repo-wide commands and conventions. Per-package files are created only when a package has substantially different commands or conventions. + +## Real-World Examples + +The skill references patterns from studied AGENTS.md files: + +- **CopilotKit** (32 lines) — concise essentials with links to detailed docs +- **airflow** (215 lines) — thorough command reference with real placeholders +- **grafana** (480 lines) — categorized frontend/backend commands with sub-directory links +- **swc** (75 lines) — performance-first philosophy anchoring every decision +- **Lychee** (122 lines) — safety-critical guardrails with explicit "never run" section diff --git a/plugins/onboarding/agents-md/SKILL.md b/plugins/onboarding/skills/agents-md/SKILL.md similarity index 100% rename from plugins/onboarding/agents-md/SKILL.md rename to plugins/onboarding/skills/agents-md/SKILL.md diff --git a/plugins/onboarding/github-pr-review/README.md b/skills/github-pr-review/README.md similarity index 100% rename from plugins/onboarding/github-pr-review/README.md rename to skills/github-pr-review/README.md diff --git a/plugins/onboarding/github-pr-review/SKILL.md b/skills/github-pr-review/SKILL.md similarity index 100% rename from plugins/onboarding/github-pr-review/SKILL.md rename to skills/github-pr-review/SKILL.md From e072b29d1c0aae3cdae95d0fb7ed9b08cc9035a2 Mon Sep 17 00:00:00 2001 From: Calvin Smithu Date: Mon, 23 Mar 2026 07:44:20 -0600 Subject: [PATCH 04/15] minor metadata --- marketplaces/default.json | 13 +++++++++++++ plugins/onboarding/.claude-plugin/plugin.json | 13 +++++++++++++ 2 files changed, 26 insertions(+) create mode 100644 plugins/onboarding/.claude-plugin/plugin.json diff --git a/marketplaces/default.json b/marketplaces/default.json index 60c03b7..8fa5337 100644 --- a/marketplaces/default.json +++ b/marketplaces/default.json @@ -483,6 +483,19 @@ "verification", "sample" ] + }, + { + "name": "onboarding", + "source": "./plugins/onboarding", + "description": "Assess repository agent-readiness across five pillars, propose high-impact fixes, and generate repo-specific AGENTS.md files.", + "category": "productivity", + "keywords": [ + "onboarding", + "readiness", + "agents-md", + "agent-readiness", + "assessment" + ] } ] } diff --git a/plugins/onboarding/.claude-plugin/plugin.json b/plugins/onboarding/.claude-plugin/plugin.json new file mode 100644 index 0000000..fc97ab0 --- /dev/null +++ b/plugins/onboarding/.claude-plugin/plugin.json @@ -0,0 +1,13 @@ +{ + "name": "onboarding", + "version": "1.0.0", + "description": "Assess repository agent-readiness across five pillars, propose high-impact fixes, and generate repo-specific AGENTS.md files", + "author": { + "name": "OpenHands", + "email": "contact@all-hands.dev" + }, + "homepage": "https://github.com/OpenHands/extensions", + "repository": "https://github.com/OpenHands/extensions", + "license": "MIT", + "keywords": ["onboarding", "readiness", "agents-md", "agent-readiness", "assessment"] +} From 0c797ef3d892326326e6ebae6250e2780976dfac Mon Sep 17 00:00:00 2001 From: Calvin Smithu Date: Mon, 23 Mar 2026 07:50:14 -0600 Subject: [PATCH 05/15] removing unused readiness report skill --- marketplaces/default.json | 819 ++++------ skills/readiness-report/README.md | 132 -- skills/readiness-report/SKILL.md | 127 -- .../readiness-report/references/criteria.md | 185 --- .../references/maturity-levels.md | 249 --- .../readiness-report/scripts/analyze_repo.py | 1396 ----------------- .../scripts/generate_report.py | 297 ---- 7 files changed, 320 insertions(+), 2885 deletions(-) delete mode 100644 skills/readiness-report/README.md delete mode 100644 skills/readiness-report/SKILL.md delete mode 100644 skills/readiness-report/references/criteria.md delete mode 100644 skills/readiness-report/references/maturity-levels.md delete mode 100755 skills/readiness-report/scripts/analyze_repo.py delete mode 100755 skills/readiness-report/scripts/generate_report.py diff --git a/marketplaces/default.json b/marketplaces/default.json index 8fa5337..434597d 100644 --- a/marketplaces/default.json +++ b/marketplaces/default.json @@ -1,501 +1,322 @@ { - "name": "default", - "owner": { - "name": "OpenHands", - "email": "contact@all-hands.dev" - }, - "metadata": { - "description": "Official OpenHands skills marketplace - reusable guidelines that customize agent behavior", - "version": "1.0.0", - "pluginRoot": "./skills" - }, - "plugins": [ - { - "name": "add-skill", - "source": "./add-skill", - "description": "Add (import) an OpenHands skill from a GitHub repository into the current workspace.", - "category": "productivity", - "keywords": [ - "skill", - "import", - "github", - "sparse-checkout" - ] - }, - { - "name": "agent-memory", - "source": "./agent-memory", - "description": "Persist and retrieve repository-specific knowledge using AGENTS.md files. Use when you want to save important information about a codebase (build commands, code style, workflows) for future sessions.", - "category": "productivity", - "keywords": [ - "memory", - "knowledge", - "persistence", - "agents" - ] - }, - { - "name": "agent-sdk-builder", - "source": "./agent-sdk-builder", - "description": "Guided workflow for building custom AI agents using the OpenHands Software Agent SDK. Use when you want to create a new agent through an interactive interview process that gathers requirements and generates implementation plans.", - "category": "development", - "keywords": [ - "agent", - "sdk", - "builder", - "openhands" - ] - }, - { - "name": "azure-devops", - "source": "./azure-devops", - "description": "Interact with Azure DevOps repositories, pull requests, and APIs using the AZURE_DEVOPS_TOKEN environment variable. Use when working with code hosted on Azure DevOps or managing Azure DevOps resources.", - "category": "integration", - "keywords": [ - "azure", - "devops", - "git", - "pull-request" - ] - }, - { - "name": "babysit-pr", - "source": "./babysit-pr", - "description": "Babysit a GitHub pull request by monitoring CI checks, workflow runs, review comments, and mergeability until it is ready to merge.", - "category": "productivity", - "keywords": [ - "github", - "pull-request", - "ci", - "actions", - "review", - "monitoring" - ] - }, - { - "name": "bitbucket", - "source": "./bitbucket", - "description": "Interact with Bitbucket repositories and pull requests using the BITBUCKET_TOKEN environment variable. Use when working with code hosted on Bitbucket or managing Bitbucket resources via API.", - "category": "integration", - "keywords": [ - "bitbucket", - "git", - "pull-request" - ] - }, - { - "name": "code-review", - "source": "./code-review", - "description": "Structured code review covering style, readability, and security concerns with actionable feedback. Use when reviewing pull requests or merge requests to identify issues and suggest improvements.", - "category": "code-quality", - "keywords": [ - "code-review", - "quality", - "security", - "style" - ] - }, - { - "name": "codereview-roasted", - "source": "./codereview-roasted", - "description": "Brutally honest code review in the style of Linus Torvalds, focusing on data structures, simplicity, and pragmatism. Use when you want critical, no-nonsense feedback that prioritizes engineering fundamentals over style preferences.", - "category": "code-quality", - "keywords": [ - "code-review", - "quality", - "linus-torvalds" - ] - }, - { - "name": "datadog", - "source": "./datadog", - "description": "Query and analyze Datadog logs, metrics, APM traces, and monitors using the Datadog API. Use when debugging production issues, monitoring application performance, or investigating alerts.", - "category": "monitoring", - "keywords": [ - "datadog", - "monitoring", - "logs", - "metrics", - "apm" - ] - }, - { - "name": "deno", - "source": "./deno", - "description": "Common project operations using Deno (tasks, run/test/lint/fmt, and dependency management).", - "category": "development", - "keywords": [ - "deno", - "typescript", - "javascript", - "runtime" - ] - }, - { - "name": "discord", - "source": "./discord", - "description": "Build and automate Discord integrations (bots, webhooks, slash commands, and REST API workflows). Use when the user mentions Discord, a Discord server/guild, channels, webhooks, bot tokens, slash commands/application commands, discord.js, or discord.py.", - "category": "integration", - "keywords": [ - "discord", - "bot", - "webhook", - "automation" - ] - }, - { - "name": "docker", - "source": "./docker", - "description": "Run Docker commands within a container environment, including starting the Docker daemon and managing containers. Use when building, running, or managing Docker containers and images.", - "category": "infrastructure", - "keywords": [ - "docker", - "container", - "images" - ] - }, - { - "name": "flarglebargle", - "source": "./flarglebargle", - "description": "A test skill that responds to the magic word 'flarglebargle' with a compliment. Use for testing skill activation and trigger functionality.", - "category": "testing", - "keywords": [ - "test", - "demo" - ] - }, - { - "name": "frontend-design", - "source": "./frontend-design", - "description": "Create distinctive, production-grade frontend interfaces with high design quality. Use this skill when the user asks to build web components, pages, artifacts, posters, or applications. Generates creative, polished code and UI design that avoids generic AI aesthetics.", - "category": "design", - "keywords": [ - "frontend", - "design", - "ui", - "web", - "react", - "html", - "css" - ] - }, - { - "name": "github", - "source": "./github", - "description": "Interact with GitHub repositories, pull requests, issues, and workflows using the GITHUB_TOKEN environment variable and GitHub CLI. Use when working with code hosted on GitHub or managing GitHub resources.", - "category": "integration", - "keywords": [ - "github", - "git", - "pull-request", - "issues", - "workflows" - ] - }, - { - "name": "github-pr-review", - "source": "./github-pr-review", - "description": "Post structured PR reviews to GitHub with inline comments/suggestions in a single API call.", - "category": "code-quality", - "keywords": [ - "github", - "pull-request", - "review", - "code-review" - ] - }, - { - "name": "gitlab", - "source": "./gitlab", - "description": "Interact with GitLab repositories, merge requests, and APIs using the GITLAB_TOKEN environment variable. Use when working with code hosted on GitLab or managing GitLab resources.", - "category": "integration", - "keywords": [ - "gitlab", - "git", - "merge-request" - ] - }, - { - "name": "init", - "source": "./init", - "description": "Scaffold an AGENTS.md contributor guide for a repository or subdirectory.", - "category": "documentation", - "keywords": [ - "agents", - "agents-md", - "documentation", - "guidelines", - "init" - ] - }, - { - "name": "jupyter", - "source": "./jupyter", - "description": "Read, modify, execute, and convert Jupyter notebooks programmatically. Use when working with .ipynb files for data science workflows, including editing cells, clearing outputs, or converting to other formats.", - "category": "data-science", - "keywords": [ - "jupyter", - "notebook", - "ipynb", - "data-science" - ] - }, - { - "name": "kubernetes", - "source": "./kubernetes", - "description": "Set up and manage local Kubernetes clusters using KIND (Kubernetes IN Docker). Use when testing Kubernetes applications locally or developing cloud-native workloads.", - "category": "infrastructure", - "keywords": [ - "kubernetes", - "k8s", - "kind", - "cloud-native" - ] - }, - { - "name": "learn-from-code-review", - "source": "./learn-from-code-review", - "description": "Distill code review feedback from GitHub PRs into reusable skills and guidelines. Use when users ask to learn from code reviews, extract review patterns, or generate coding standards from historical PR feedback.", - "category": "code-quality", - "keywords": [ - "code-review", - "learning", - "skills", - "guidelines", - "pr-feedback" - ] - }, - { - "name": "notion", - "source": "./notion", - "description": "Create, search, and update Notion pages/databases using the Notion API. Use for documenting work, generating runbooks, and automating knowledge base updates.", - "category": "productivity", - "keywords": [ - "notion", - "documentation", - "knowledge-base" - ] - }, - { - "name": "npm", - "source": "./npm", - "description": "Handle npm package installation in non-interactive environments by piping confirmations. Use when installing Node.js packages that require user confirmation prompts.", - "category": "development", - "keywords": [ - "npm", - "nodejs", - "packages" - ] - }, - { - "name": "onboarding-agent", - "source": "./onboarding-agent", - "description": "Interactive onboarding workflow that interviews users to understand their coding goals and generates PR-ready implementation plans. Use when starting a new development task to ensure clear requirements and structured execution.", - "category": "productivity", - "keywords": [ - "onboarding", - "planning", - "requirements" - ] - }, - { - "name": "openhands-api", - "source": "./openhands-api", - "description": "Use the OpenHands REST API (V0) to create and manage agent conversations programmatically. Includes minimal Python and TypeScript clients under scripts/.", - "category": "development", - "keywords": [ - "openhands", - "api", - "conversations", - "automation" - ] - }, - { - "name": "openhands-api-v1", - "source": "./openhands-api-v1", - "description": "Use the OpenHands Cloud REST API (V1) to create and manage app conversations and to access sandbox agent-server endpoints. Includes minimal Python and TypeScript clients under scripts/.", - "category": "development", - "keywords": [ - "openhands", - "api", - "cloud", - "v1", - "agent-server", - "sandbox", - "conversations", - "automation" - ] - }, - { - "name": "pdflatex", - "source": "./pdflatex", - "description": "Install and use pdflatex to compile LaTeX documents into PDFs on Linux. Use when generating academic papers, research publications, or any documents written in LaTeX.", - "category": "documentation", - "keywords": [ - "latex", - "pdf", - "academic", - "documents" - ] - }, - { - "name": "readiness-report", - "source": "./readiness-report", - "description": "Assess repository readiness for autonomous AI development and generate prioritized recommendations.", - "category": "productivity", - "keywords": [ - "readiness", - "report", - "repo", - "quality" - ] - }, - { - "name": "releasenotes", - "source": "./releasenotes", - "description": "Generate formatted changelogs from git history since the last release tag. Use when preparing release notes that categorize changes into breaking changes, features, fixes, and other sections.", - "category": "development", - "keywords": [ - "release", - "changelog", - "git" - ] - }, - { - "name": "security", - "source": "./security", - "description": "Security best practices for secure coding, authentication, authorization, and data protection. Use when developing features that handle sensitive data, user authentication, or require security review.", - "category": "security", - "keywords": [ - "security", - "authentication", - "authorization", - "encryption" - ] - }, - { - "name": "skill-creator", - "source": "./skill-creator", - "description": "Guide for creating effective skills. This skill should be used when users want to create a new skill (or update an existing skill) that extends Claude's capabilities with specialized knowledge, workflows, or tool integrations.", - "category": "development", - "keywords": [ - "skill", - "plugin", - "create" - ] - }, - { - "name": "ssh", - "source": "./ssh", - "description": "Establish and manage SSH connections to remote machines, including key generation, configuration, and file transfers. Use when connecting to remote servers, executing remote commands, or transferring files via SCP.", - "category": "infrastructure", - "keywords": [ - "ssh", - "remote", - "scp", - "server" - ] - }, - { - "name": "swift-linux", - "source": "./swift-linux", - "description": "Install and configure Swift programming language on Debian Linux for server-side development. Use when building Swift applications on Linux or setting up a Swift development environment.", - "category": "development", - "keywords": [ - "swift", - "linux", - "server-side" - ] - }, - { - "name": "theme-factory", - "source": "./theme-factory", - "description": "Toolkit for styling artifacts with a theme. These artifacts can be slides, docs, reportings, HTML landing pages, etc. There are 10 pre-set themes with colors/fonts that you can apply to any artifact that has been creating, or can generate a new theme on-the-fly.", - "category": "design", - "keywords": [ - "theme", - "styling", - "design", - "slides", - "documents" - ] - }, - { - "name": "uv", - "source": "./uv", - "description": "Common project, dependency, and environment operations using uv.", - "category": "development", - "keywords": [ - "uv", - "python", - "dependencies", - "venv" - ] - }, - { - "name": "vercel", - "source": "./vercel", - "description": "Deploy and manage applications on Vercel, including preview deployments and deployment protection.", - "category": "integration", - "keywords": [ - "vercel", - "deployment", - "preview", - "hosting" - ] - }, - { - "name": "linear", - "source": "./linear", - "description": "Interact with Linear project management - query issues, update status, create tickets using the Linear GraphQL API.", - "category": "integration", - "keywords": [ - "linear", - "ticket", - "issue", - "project-management", - "graphql" - ] - }, - { - "name": "city-weather", - "source": "./plugins/city-weather", - "description": "Get current weather, time, and precipitation forecast for any city using the free Open-Meteo API. Provides slash command /city-weather:now .", - "category": "utilities", - "keywords": [ - "weather", - "time", - "forecast", - "temperature", - "precipitation", - "sample" - ] - }, - { - "name": "magic-test", - "source": "./plugins/magic-test", - "description": "A simple test plugin for verifying plugin loading. Triggers on magic words (alakazam, abracadabra) and returns a specific phrase to confirm plugins are working.", - "category": "testing", - "keywords": [ - "test", - "magic", - "plugin", - "verification", - "sample" - ] - }, - { - "name": "onboarding", - "source": "./plugins/onboarding", - "description": "Assess repository agent-readiness across five pillars, propose high-impact fixes, and generate repo-specific AGENTS.md files.", - "category": "productivity", - "keywords": [ - "onboarding", - "readiness", - "agents-md", - "agent-readiness", - "assessment" - ] - } - ] + "name": "default", + "owner": { + "name": "OpenHands", + "email": "contact@all-hands.dev" + }, + "metadata": { + "description": "Official OpenHands skills marketplace - reusable guidelines that customize agent behavior", + "version": "1.0.0", + "pluginRoot": "./skills" + }, + "plugins": [ + { + "name": "add-skill", + "source": "./add-skill", + "description": "Add (import) an OpenHands skill from a GitHub repository into the current workspace.", + "category": "productivity", + "keywords": ["skill", "import", "github", "sparse-checkout"] + }, + { + "name": "agent-memory", + "source": "./agent-memory", + "description": "Persist and retrieve repository-specific knowledge using AGENTS.md files. Use when you want to save important information about a codebase (build commands, code style, workflows) for future sessions.", + "category": "productivity", + "keywords": ["memory", "knowledge", "persistence", "agents"] + }, + { + "name": "agent-sdk-builder", + "source": "./agent-sdk-builder", + "description": "Guided workflow for building custom AI agents using the OpenHands Software Agent SDK. Use when you want to create a new agent through an interactive interview process that gathers requirements and generates implementation plans.", + "category": "development", + "keywords": ["agent", "sdk", "builder", "openhands"] + }, + { + "name": "azure-devops", + "source": "./azure-devops", + "description": "Interact with Azure DevOps repositories, pull requests, and APIs using the AZURE_DEVOPS_TOKEN environment variable. Use when working with code hosted on Azure DevOps or managing Azure DevOps resources.", + "category": "integration", + "keywords": ["azure", "devops", "git", "pull-request"] + }, + { + "name": "babysit-pr", + "source": "./babysit-pr", + "description": "Babysit a GitHub pull request by monitoring CI checks, workflow runs, review comments, and mergeability until it is ready to merge.", + "category": "productivity", + "keywords": [ + "github", + "pull-request", + "ci", + "actions", + "review", + "monitoring" + ] + }, + { + "name": "bitbucket", + "source": "./bitbucket", + "description": "Interact with Bitbucket repositories and pull requests using the BITBUCKET_TOKEN environment variable. Use when working with code hosted on Bitbucket or managing Bitbucket resources via API.", + "category": "integration", + "keywords": ["bitbucket", "git", "pull-request"] + }, + { + "name": "code-review", + "source": "./code-review", + "description": "Structured code review covering style, readability, and security concerns with actionable feedback. Use when reviewing pull requests or merge requests to identify issues and suggest improvements.", + "category": "code-quality", + "keywords": ["code-review", "quality", "security", "style"] + }, + { + "name": "codereview-roasted", + "source": "./codereview-roasted", + "description": "Brutally honest code review in the style of Linus Torvalds, focusing on data structures, simplicity, and pragmatism. Use when you want critical, no-nonsense feedback that prioritizes engineering fundamentals over style preferences.", + "category": "code-quality", + "keywords": ["code-review", "quality", "linus-torvalds"] + }, + { + "name": "datadog", + "source": "./datadog", + "description": "Query and analyze Datadog logs, metrics, APM traces, and monitors using the Datadog API. Use when debugging production issues, monitoring application performance, or investigating alerts.", + "category": "monitoring", + "keywords": ["datadog", "monitoring", "logs", "metrics", "apm"] + }, + { + "name": "deno", + "source": "./deno", + "description": "Common project operations using Deno (tasks, run/test/lint/fmt, and dependency management).", + "category": "development", + "keywords": ["deno", "typescript", "javascript", "runtime"] + }, + { + "name": "discord", + "source": "./discord", + "description": "Build and automate Discord integrations (bots, webhooks, slash commands, and REST API workflows). Use when the user mentions Discord, a Discord server/guild, channels, webhooks, bot tokens, slash commands/application commands, discord.js, or discord.py.", + "category": "integration", + "keywords": ["discord", "bot", "webhook", "automation"] + }, + { + "name": "docker", + "source": "./docker", + "description": "Run Docker commands within a container environment, including starting the Docker daemon and managing containers. Use when building, running, or managing Docker containers and images.", + "category": "infrastructure", + "keywords": ["docker", "container", "images"] + }, + { + "name": "flarglebargle", + "source": "./flarglebargle", + "description": "A test skill that responds to the magic word 'flarglebargle' with a compliment. Use for testing skill activation and trigger functionality.", + "category": "testing", + "keywords": ["test", "demo"] + }, + { + "name": "frontend-design", + "source": "./frontend-design", + "description": "Create distinctive, production-grade frontend interfaces with high design quality. Use this skill when the user asks to build web components, pages, artifacts, posters, or applications. Generates creative, polished code and UI design that avoids generic AI aesthetics.", + "category": "design", + "keywords": ["frontend", "design", "ui", "web", "react", "html", "css"] + }, + { + "name": "github", + "source": "./github", + "description": "Interact with GitHub repositories, pull requests, issues, and workflows using the GITHUB_TOKEN environment variable and GitHub CLI. Use when working with code hosted on GitHub or managing GitHub resources.", + "category": "integration", + "keywords": ["github", "git", "pull-request", "issues", "workflows"] + }, + { + "name": "github-pr-review", + "source": "./github-pr-review", + "description": "Post structured PR reviews to GitHub with inline comments/suggestions in a single API call.", + "category": "code-quality", + "keywords": ["github", "pull-request", "review", "code-review"] + }, + { + "name": "gitlab", + "source": "./gitlab", + "description": "Interact with GitLab repositories, merge requests, and APIs using the GITLAB_TOKEN environment variable. Use when working with code hosted on GitLab or managing GitLab resources.", + "category": "integration", + "keywords": ["gitlab", "git", "merge-request"] + }, + { + "name": "init", + "source": "./init", + "description": "Scaffold an AGENTS.md contributor guide for a repository or subdirectory.", + "category": "documentation", + "keywords": ["agents", "agents-md", "documentation", "guidelines", "init"] + }, + { + "name": "jupyter", + "source": "./jupyter", + "description": "Read, modify, execute, and convert Jupyter notebooks programmatically. Use when working with .ipynb files for data science workflows, including editing cells, clearing outputs, or converting to other formats.", + "category": "data-science", + "keywords": ["jupyter", "notebook", "ipynb", "data-science"] + }, + { + "name": "kubernetes", + "source": "./kubernetes", + "description": "Set up and manage local Kubernetes clusters using KIND (Kubernetes IN Docker). Use when testing Kubernetes applications locally or developing cloud-native workloads.", + "category": "infrastructure", + "keywords": ["kubernetes", "k8s", "kind", "cloud-native"] + }, + { + "name": "learn-from-code-review", + "source": "./learn-from-code-review", + "description": "Distill code review feedback from GitHub PRs into reusable skills and guidelines. Use when users ask to learn from code reviews, extract review patterns, or generate coding standards from historical PR feedback.", + "category": "code-quality", + "keywords": [ + "code-review", + "learning", + "skills", + "guidelines", + "pr-feedback" + ] + }, + { + "name": "notion", + "source": "./notion", + "description": "Create, search, and update Notion pages/databases using the Notion API. Use for documenting work, generating runbooks, and automating knowledge base updates.", + "category": "productivity", + "keywords": ["notion", "documentation", "knowledge-base"] + }, + { + "name": "npm", + "source": "./npm", + "description": "Handle npm package installation in non-interactive environments by piping confirmations. Use when installing Node.js packages that require user confirmation prompts.", + "category": "development", + "keywords": ["npm", "nodejs", "packages"] + }, + { + "name": "onboarding-agent", + "source": "./onboarding-agent", + "description": "Interactive onboarding workflow that interviews users to understand their coding goals and generates PR-ready implementation plans. Use when starting a new development task to ensure clear requirements and structured execution.", + "category": "productivity", + "keywords": ["onboarding", "planning", "requirements"] + }, + { + "name": "openhands-api", + "source": "./openhands-api", + "description": "Use the OpenHands REST API (V0) to create and manage agent conversations programmatically. Includes minimal Python and TypeScript clients under scripts/.", + "category": "development", + "keywords": ["openhands", "api", "conversations", "automation"] + }, + { + "name": "openhands-api-v1", + "source": "./openhands-api-v1", + "description": "Use the OpenHands Cloud REST API (V1) to create and manage app conversations and to access sandbox agent-server endpoints. Includes minimal Python and TypeScript clients under scripts/.", + "category": "development", + "keywords": [ + "openhands", + "api", + "cloud", + "v1", + "agent-server", + "sandbox", + "conversations", + "automation" + ] + }, + { + "name": "pdflatex", + "source": "./pdflatex", + "description": "Install and use pdflatex to compile LaTeX documents into PDFs on Linux. Use when generating academic papers, research publications, or any documents written in LaTeX.", + "category": "documentation", + "keywords": ["latex", "pdf", "academic", "documents"] + }, + { + "name": "releasenotes", + "source": "./releasenotes", + "description": "Generate formatted changelogs from git history since the last release tag. Use when preparing release notes that categorize changes into breaking changes, features, fixes, and other sections.", + "category": "development", + "keywords": ["release", "changelog", "git"] + }, + { + "name": "security", + "source": "./security", + "description": "Security best practices for secure coding, authentication, authorization, and data protection. Use when developing features that handle sensitive data, user authentication, or require security review.", + "category": "security", + "keywords": ["security", "authentication", "authorization", "encryption"] + }, + { + "name": "skill-creator", + "source": "./skill-creator", + "description": "Guide for creating effective skills. This skill should be used when users want to create a new skill (or update an existing skill) that extends Claude's capabilities with specialized knowledge, workflows, or tool integrations.", + "category": "development", + "keywords": ["skill", "plugin", "create"] + }, + { + "name": "ssh", + "source": "./ssh", + "description": "Establish and manage SSH connections to remote machines, including key generation, configuration, and file transfers. Use when connecting to remote servers, executing remote commands, or transferring files via SCP.", + "category": "infrastructure", + "keywords": ["ssh", "remote", "scp", "server"] + }, + { + "name": "swift-linux", + "source": "./swift-linux", + "description": "Install and configure Swift programming language on Debian Linux for server-side development. Use when building Swift applications on Linux or setting up a Swift development environment.", + "category": "development", + "keywords": ["swift", "linux", "server-side"] + }, + { + "name": "theme-factory", + "source": "./theme-factory", + "description": "Toolkit for styling artifacts with a theme. These artifacts can be slides, docs, reportings, HTML landing pages, etc. There are 10 pre-set themes with colors/fonts that you can apply to any artifact that has been creating, or can generate a new theme on-the-fly.", + "category": "design", + "keywords": ["theme", "styling", "design", "slides", "documents"] + }, + { + "name": "uv", + "source": "./uv", + "description": "Common project, dependency, and environment operations using uv.", + "category": "development", + "keywords": ["uv", "python", "dependencies", "venv"] + }, + { + "name": "vercel", + "source": "./vercel", + "description": "Deploy and manage applications on Vercel, including preview deployments and deployment protection.", + "category": "integration", + "keywords": ["vercel", "deployment", "preview", "hosting"] + }, + { + "name": "linear", + "source": "./linear", + "description": "Interact with Linear project management - query issues, update status, create tickets using the Linear GraphQL API.", + "category": "integration", + "keywords": ["linear", "ticket", "issue", "project-management", "graphql"] + }, + { + "name": "city-weather", + "source": "./plugins/city-weather", + "description": "Get current weather, time, and precipitation forecast for any city using the free Open-Meteo API. Provides slash command /city-weather:now .", + "category": "utilities", + "keywords": [ + "weather", + "time", + "forecast", + "temperature", + "precipitation", + "sample" + ] + }, + { + "name": "magic-test", + "source": "./plugins/magic-test", + "description": "A simple test plugin for verifying plugin loading. Triggers on magic words (alakazam, abracadabra) and returns a specific phrase to confirm plugins are working.", + "category": "testing", + "keywords": ["test", "magic", "plugin", "verification", "sample"] + }, + { + "name": "onboarding", + "source": "./plugins/onboarding", + "description": "Assess repository agent-readiness across five pillars, propose high-impact fixes, and generate repo-specific AGENTS.md files.", + "category": "productivity", + "keywords": [ + "onboarding", + "readiness", + "agents-md", + "agent-readiness", + "assessment" + ] + } + ] } diff --git a/skills/readiness-report/README.md b/skills/readiness-report/README.md deleted file mode 100644 index c567920..0000000 --- a/skills/readiness-report/README.md +++ /dev/null @@ -1,132 +0,0 @@ -# Readiness Report - -Evaluate how well a codebase supports autonomous AI development. Analyzes repositories across eight technical pillars (Style & Validation, Build System, Testing, Documentation, Dev Environment, Debugging & Observability, Security, Task Discovery) and five maturity levels. Use when users request `/readiness-report` or want to assess agent readiness, codebase maturity, or identify gaps preventing effective AI-assisted development. - -## Triggers - -This skill is activated by the following keywords: - -- `/readiness-report` - -## Details - -# Agent Readiness Report - -Evaluate how well a repository supports autonomous AI development by analyzing it across eight technical pillars and five maturity levels. - -## Overview - -Agent Readiness measures how prepared a codebase is for AI-assisted development. Poor feedback loops, missing documentation, or lack of tooling cause agents to waste cycles on preventable errors. This skill identifies those gaps and prioritizes fixes. - -## Quick Start - -The user will run `/readiness-report` to evaluate the current repository. The agent will then: -1. Clone the repo, scan repository structure, CI configs, and tooling -2. Evaluate 81 criteria across 9 technical pillars -3. Determine maturity level (L1-L5) based on 80% threshold per level -4. Provide prioritized recommendations - -## Workflow - -### Step 1: Run Repository Analysis - -Execute the analysis script to gather signals from the repository: - -```bash -python scripts/analyze_repo.py --repo-path . -``` - -This script checks for: -- Configuration files (.eslintrc, pyproject.toml, etc.) -- CI/CD workflows (.github/workflows/, .gitlab-ci.yml) -- Documentation (README, AGENTS.md, CONTRIBUTING.md) -- Test infrastructure (test directories, coverage configs) -- Security configurations (CODEOWNERS, .gitignore, secrets management) - -### Step 2: Generate Report - -After analysis, generate the formatted report: - -```bash -python scripts/generate_report.py --analysis-file /tmp/readiness_analysis.json -``` - -### Step 3: Present Results - -The report includes: -1. **Overall Score**: Pass rate percentage and maturity level achieved -2. **Level Progress**: Bar showing L1-L5 completion percentages -3. **Strengths**: Top-performing pillars with passing criteria -4. **Opportunities**: Prioritized list of improvements to implement -5. **Detailed Criteria**: Full breakdown by pillar showing each criterion status - -## Nine Technical Pillars - -Each pillar addresses specific failure modes in AI-assisted development: - -| Pillar | Purpose | Key Signals | -|--------|---------|-------------| -| **Style & Validation** | Catch bugs instantly | Linters, formatters, type checkers | -| **Build System** | Fast, reliable builds | Build docs, CI speed, automation | -| **Testing** | Verify correctness | Unit/integration tests, coverage | -| **Documentation** | Guide the agent | AGENTS.md, README, architecture docs | -| **Dev Environment** | Reproducible setup | Devcontainer, env templates | -| **Debugging & Observability** | Diagnose issues | Logging, tracing, metrics | -| **Security** | Protect the codebase | CODEOWNERS, secrets management | -| **Task Discovery** | Find work to do | Issue templates, PR templates | -| **Product & Analytics** | Error-to-insight loop | Error tracking, product analytics | - -See `references/criteria.md` for the complete list of 81 criteria per pillar. - -## Five Maturity Levels - -| Level | Name | Description | Agent Capability | -|-------|------|-------------|------------------| -| L1 | Initial | Basic version control | Manual assistance only | -| L2 | Managed | Basic CI/CD and testing | Simple, well-defined tasks | -| L3 | Standardized | Production-ready for agents | Routine maintenance | -| L4 | Measured | Comprehensive automation | Complex features | -| L5 | Optimized | Full autonomous capability | End-to-end development | - -**Level Progression**: To unlock a level, pass ≥80% of criteria at that level AND all previous levels. - -See `references/maturity-levels.md` for detailed level requirements. - -## Interpreting Results - -### Pass vs Fail vs Skip - -- ✓ **Pass**: Criterion met (contributes to score) -- ✗ **Fail**: Criterion not met (opportunity for improvement) -- — **Skip**: Not applicable to this repository type (excluded from score) - -### Priority Order - -Fix gaps in this order: -1. **L1-L2 failures**: Foundation issues blocking basic agent operation -2. **L3 failures**: Production readiness gaps -3. **High-impact L4+ failures**: Optimization opportunities - -### Common Quick Wins - -1. **Add AGENTS.md**: Document commands, architecture, and workflows for AI agents -2. **Configure pre-commit hooks**: Catch style issues before CI -3. **Add PR/issue templates**: Structure task discovery -4. **Document single-command setup**: Enable fast environment provisioning - -## Resources - -- `scripts/analyze_repo.py` - Repository analysis script -- `scripts/generate_report.py` - Report generation and formatting -- `references/criteria.md` - Complete criteria definitions by pillar -- `references/maturity-levels.md` - Detailed level requirements - -## Automated Remediation - -After reviewing the report, common fixes can be automated: -- Generate AGENTS.md from repository structure -- Add missing issue/PR templates -- Configure standard linters and formatters -- Set up pre-commit hooks - -Ask to "fix readiness gaps" to begin automated remediation of failing criteria. \ No newline at end of file diff --git a/skills/readiness-report/SKILL.md b/skills/readiness-report/SKILL.md deleted file mode 100644 index 1b26766..0000000 --- a/skills/readiness-report/SKILL.md +++ /dev/null @@ -1,127 +0,0 @@ ---- -name: readiness-report -description: Evaluate how well a codebase supports autonomous AI development. Analyzes repositories across eight technical pillars (Style & Validation, Build System, Testing, Documentation, Dev Environment, Debugging & Observability, Security, Task Discovery) and five maturity levels. Use when users request `/readiness-report` or want to assess agent readiness, codebase maturity, or identify gaps preventing effective AI-assisted development. -triggers: -- /readiness-report ---- - -# Agent Readiness Report - -Evaluate how well a repository supports autonomous AI development by analyzing it across eight technical pillars and five maturity levels. - -## Overview - -Agent Readiness measures how prepared a codebase is for AI-assisted development. Poor feedback loops, missing documentation, or lack of tooling cause agents to waste cycles on preventable errors. This skill identifies those gaps and prioritizes fixes. - -## Quick Start - -The user will run `/readiness-report` to evaluate the current repository. The agent will then: -1. Clone the repo, scan repository structure, CI configs, and tooling -2. Evaluate 81 criteria across 9 technical pillars -3. Determine maturity level (L1-L5) based on 80% threshold per level -4. Provide prioritized recommendations - -## Workflow - -### Step 1: Run Repository Analysis - -Execute the analysis script to gather signals from the repository: - -```bash -python scripts/analyze_repo.py --repo-path . -``` - -This script checks for: -- Configuration files (.eslintrc, pyproject.toml, etc.) -- CI/CD workflows (.github/workflows/, .gitlab-ci.yml) -- Documentation (README, AGENTS.md, CONTRIBUTING.md) -- Test infrastructure (test directories, coverage configs) -- Security configurations (CODEOWNERS, .gitignore, secrets management) - -### Step 2: Generate Report - -After analysis, generate the formatted report: - -```bash -python scripts/generate_report.py --analysis-file /tmp/readiness_analysis.json -``` - -### Step 3: Present Results - -The report includes: -1. **Overall Score**: Pass rate percentage and maturity level achieved -2. **Level Progress**: Bar showing L1-L5 completion percentages -3. **Strengths**: Top-performing pillars with passing criteria -4. **Opportunities**: Prioritized list of improvements to implement -5. **Detailed Criteria**: Full breakdown by pillar showing each criterion status - -## Nine Technical Pillars - -Each pillar addresses specific failure modes in AI-assisted development: - -| Pillar | Purpose | Key Signals | -|--------|---------|-------------| -| **Style & Validation** | Catch bugs instantly | Linters, formatters, type checkers | -| **Build System** | Fast, reliable builds | Build docs, CI speed, automation | -| **Testing** | Verify correctness | Unit/integration tests, coverage | -| **Documentation** | Guide the agent | AGENTS.md, README, architecture docs | -| **Dev Environment** | Reproducible setup | Devcontainer, env templates | -| **Debugging & Observability** | Diagnose issues | Logging, tracing, metrics | -| **Security** | Protect the codebase | CODEOWNERS, secrets management | -| **Task Discovery** | Find work to do | Issue templates, PR templates | -| **Product & Analytics** | Error-to-insight loop | Error tracking, product analytics | - -See `references/criteria.md` for the complete list of 81 criteria per pillar. - -## Five Maturity Levels - -| Level | Name | Description | Agent Capability | -|-------|------|-------------|------------------| -| L1 | Initial | Basic version control | Manual assistance only | -| L2 | Managed | Basic CI/CD and testing | Simple, well-defined tasks | -| L3 | Standardized | Production-ready for agents | Routine maintenance | -| L4 | Measured | Comprehensive automation | Complex features | -| L5 | Optimized | Full autonomous capability | End-to-end development | - -**Level Progression**: To unlock a level, pass ≥80% of criteria at that level AND all previous levels. - -See `references/maturity-levels.md` for detailed level requirements. - -## Interpreting Results - -### Pass vs Fail vs Skip - -- ✓ **Pass**: Criterion met (contributes to score) -- ✗ **Fail**: Criterion not met (opportunity for improvement) -- — **Skip**: Not applicable to this repository type (excluded from score) - -### Priority Order - -Fix gaps in this order: -1. **L1-L2 failures**: Foundation issues blocking basic agent operation -2. **L3 failures**: Production readiness gaps -3. **High-impact L4+ failures**: Optimization opportunities - -### Common Quick Wins - -1. **Add AGENTS.md**: Document commands, architecture, and workflows for AI agents -2. **Configure pre-commit hooks**: Catch style issues before CI -3. **Add PR/issue templates**: Structure task discovery -4. **Document single-command setup**: Enable fast environment provisioning - -## Resources - -- `scripts/analyze_repo.py` - Repository analysis script -- `scripts/generate_report.py` - Report generation and formatting -- `references/criteria.md` - Complete criteria definitions by pillar -- `references/maturity-levels.md` - Detailed level requirements - -## Automated Remediation - -After reviewing the report, common fixes can be automated: -- Generate AGENTS.md from repository structure -- Add missing issue/PR templates -- Configure standard linters and formatters -- Set up pre-commit hooks - -Ask to "fix readiness gaps" to begin automated remediation of failing criteria. diff --git a/skills/readiness-report/references/criteria.md b/skills/readiness-report/references/criteria.md deleted file mode 100644 index 5dc68dc..0000000 --- a/skills/readiness-report/references/criteria.md +++ /dev/null @@ -1,185 +0,0 @@ -# Agent Readiness Criteria - -Complete reference of all 81 criteria evaluated across nine technical pillars. - -## Criteria Format - -Each criterion is binary (pass/fail) and includes: -- **ID**: Unique identifier (snake_case) -- **Level**: Maturity level (1-5) where criterion is evaluated -- **Detection**: How to check if the criterion is met -- **Impact**: What happens when this criterion fails - ---- - -## 1. Style & Validation - -Automated tools that catch bugs instantly. Without them, agents waste cycles on syntax errors and style drift. - -| Criterion | Level | Detection | Impact | -|-----------|-------|-----------|--------| -| `formatter` | 1 | Prettier, Black, Ruff format, gofmt config exists | Agent submits code with formatting issues, waits for CI, fixes blindly | -| `lint_config` | 1 | ESLint, Ruff, golangci-lint, or language-specific linter configured | Style inconsistencies accumulate, harder to review | -| `type_check` | 1 | TypeScript strict, mypy, Go (static by default) | Type errors caught late in CI instead of immediately | -| `strict_typing` | 2 | TypeScript strict:true, mypy strict=true | Partial typing allows bugs to slip through | -| `pre_commit_hooks` | 2 | .pre-commit-config.yaml or Husky config | Checks run in CI instead of locally, slower feedback | -| `naming_consistency` | 2 | Linter rules or documented conventions | Inconsistent names make codebase harder to navigate | -| `large_file_detection` | 2 | Git LFS, pre-commit check-added-large-files | Large files bloat repository, slow clones | -| `code_modularization` | 3 | import-linter, Nx boundaries, Bazel modules | Architecture degrades over time | -| `cyclomatic_complexity` | 3 | gocyclo, lizard, radon, SonarQube | Complex functions harder to understand and modify | -| `dead_code_detection` | 3 | vulture, knip, deadcode in CI | Unused code clutters codebase | -| `duplicate_code_detection` | 3 | jscpd, PMD CPD, SonarQube | Duplicated code increases maintenance burden | -| `tech_debt_tracking` | 4 | TODO scanner, SonarQube, linter TODO rules | Tech debt accumulates without visibility | -| `n_plus_one_detection` | 4 | nplusone, bullet gem, query analyzer | Performance issues in database queries | - ---- - -## 2. Build System - -Fast, reliable builds enable rapid iteration. Slow CI kills agent productivity. - -| Criterion | Level | Detection | Impact | -|-----------|-------|-----------|--------| -| `build_cmd_doc` | 1 | Build commands documented in README/AGENTS.md | Agent doesn't know how to build | -| `deps_pinned` | 1 | Lockfile exists (package-lock.json, uv.lock, go.sum) | Non-reproducible builds | -| `vcs_cli_tools` | 1 | gh/glab CLI authenticated | Can't interact with PRs/issues | -| `fast_ci_feedback` | 2 | CI completes in <10 minutes | Agent waits too long for feedback | -| `single_command_setup` | 2 | One command to set up dev environment | Agent can't bootstrap quickly | -| `release_automation` | 2 | Automated release workflow exists | Manual releases slow deployment | -| `deployment_frequency` | 2 | Regular releases (weekly+) | Infrequent deploys signal process issues | -| `release_notes_automation` | 3 | Auto-generated changelogs/release notes | Manual release notes are error-prone | -| `agentic_development` | 3 | AI agent commits visible in history | No prior agent integration | -| `automated_pr_review` | 3 | Danger.js, automated review bots | Reviews require human intervention | -| `feature_flag_infrastructure` | 3 | LaunchDarkly, Statsig, Unleash, custom system | Hard to ship incrementally | -| `build_performance_tracking` | 4 | Build caching, timing metrics | Build times creep up unnoticed | -| `heavy_dependency_detection` | 4 | Bundle size analysis (webpack-bundle-analyzer) | Bundle bloat goes unnoticed | -| `unused_dependencies_detection` | 4 | depcheck, deptry in CI | Bloated dependency tree | -| `dead_feature_flag_detection` | 4 | Stale flag detection tooling | Abandoned flags clutter code | -| `monorepo_tooling` | 4 | Nx, Turborepo, Bazel for monorepos | Cross-package changes are error-prone | -| `version_drift_detection` | 4 | Version consistency checks | Packages diverge silently | -| `progressive_rollout` | 5 | Canary deploys, gradual rollouts | All-or-nothing deployments are risky | -| `rollback_automation` | 5 | One-click rollback capability | Slow recovery from bad deploys | - ---- - -## 3. Testing - -Tests verify that changes work. Without them, agents can't validate their own work. - -| Criterion | Level | Detection | Impact | -|-----------|-------|-----------|--------| -| `unit_tests_exist` | 1 | Test files present (*_test.*, *.spec.*) | No way to verify basic correctness | -| `unit_tests_runnable` | 1 | Test command documented and works | Agent can't run tests | -| `test_naming_conventions` | 2 | Consistent test file naming | Tests hard to find | -| `test_isolation` | 2 | Tests can run in parallel | Slow test runs | -| `integration_tests_exist` | 3 | E2E/integration test directory | Only unit-level coverage | -| `test_coverage_thresholds` | 3 | Coverage enforcement in CI | Coverage drifts down | -| `flaky_test_detection` | 4 | Test retry, quarantine, or tracking | Flaky tests erode trust | -| `test_performance_tracking` | 4 | Test timing metrics | Slow tests accumulate | - ---- - -## 4. Documentation - -Documentation tells the agent what it needs to know. Missing docs mean wasted exploration. - -| Criterion | Level | Detection | Impact | -|-----------|-------|-----------|--------| -| `readme` | 1 | README.md exists with setup instructions | Agent doesn't know project basics | -| `agents_md` | 2 | AGENTS.md or CLAUDE.md exists | Agent lacks operational guidance | -| `documentation_freshness` | 2 | Key docs updated in last 180 days | Stale docs mislead agent | -| `api_schema_docs` | 3 | OpenAPI spec, GraphQL schema, or API docs | Agent must reverse-engineer APIs | -| `automated_doc_generation` | 3 | Doc generation in CI | Docs drift from code | -| `service_flow_documented` | 3 | Architecture diagrams (mermaid, PlantUML) | Agent lacks system context | -| `skills` | 3 | Skills directory (.claude/skills/, .factory/skills/) | No specialized agent instructions | -| `agents_md_validation` | 4 | CI validates AGENTS.md commands work | AGENTS.md becomes stale | - ---- - -## 5. Dev Environment - -Reproducible environments prevent "works on my machine" issues. - -| Criterion | Level | Detection | Impact | -|-----------|-------|-----------|--------| -| `env_template` | 2 | .env.example or documented env vars | Agent guesses at configuration | -| `devcontainer` | 3 | .devcontainer/devcontainer.json exists | Environment setup is manual | -| `devcontainer_runnable` | 3 | Devcontainer builds and works | Devcontainer is broken | -| `database_schema` | 3 | Schema files or migration directory | Database structure undocumented | -| `local_services_setup` | 3 | docker-compose.yml for dependencies | External services need manual setup | - ---- - -## 6. Debugging & Observability - -When things go wrong, observability helps diagnose issues quickly. - -| Criterion | Level | Detection | Impact | -|-----------|-------|-----------|--------| -| `structured_logging` | 2 | Logging library with structured output | Logs hard to parse | -| `code_quality_metrics` | 2 | Coverage reporting in CI | No visibility into code quality | -| `error_tracking_contextualized` | 3 | Sentry, Bugsnag with context | Errors lack context for debugging | -| `distributed_tracing` | 3 | OpenTelemetry, trace IDs | Can't trace requests across services | -| `metrics_collection` | 3 | Prometheus, Datadog, or custom metrics | No runtime visibility | -| `health_checks` | 3 | Health/readiness endpoints | Can't verify service status | -| `profiling_instrumentation` | 4 | CPU/memory profiling tools | Performance issues hard to diagnose | -| `alerting_configured` | 4 | PagerDuty, OpsGenie, alert rules | Issues discovered late | -| `deployment_observability` | 4 | Deploy tracking, dashboards | Can't correlate issues to deploys | -| `runbooks_documented` | 4 | Runbooks directory or linked docs | No guidance for incident response | -| `circuit_breakers` | 5 | Resilience patterns implemented | Cascading failures | - ---- - -## 7. Security - -Security criteria protect the codebase and data. - -| Criterion | Level | Detection | Impact | -|-----------|-------|-----------|--------| -| `gitignore_comprehensive` | 1 | .gitignore excludes secrets, build artifacts | Sensitive files committed | -| `secrets_management` | 2 | GitHub secrets, vault, cloud secrets | Hardcoded secrets | -| `codeowners` | 2 | CODEOWNERS file exists | No clear ownership | -| `branch_protection` | 2 | Protected main branch | Unreviewed changes to main | -| `dependency_update_automation` | 3 | Dependabot, Renovate configured | Dependencies go stale | -| `log_scrubbing` | 3 | Log sanitization for PII | Sensitive data in logs | -| `pii_handling` | 3 | PII redaction mechanisms | PII exposure risk | -| `automated_security_review` | 4 | CodeQL, Snyk, SonarQube in CI | Security issues caught late | -| `secret_scanning` | 4 | GitHub secret scanning enabled | Leaked credentials | -| `dast_scanning` | 5 | Dynamic security testing | Runtime vulnerabilities missed | -| `privacy_compliance` | 5 | GDPR/privacy tooling | Compliance gaps | - ---- - -## 8. Task Discovery - -Structured task management helps agents find and understand work. - -| Criterion | Level | Detection | Impact | -|-----------|-------|-----------|--------| -| `issue_templates` | 2 | .github/ISSUE_TEMPLATE/ exists | Inconsistent issue quality | -| `issue_labeling_system` | 2 | Consistent labels on issues | Issues hard to categorize | -| `pr_templates` | 2 | pull_request_template.md exists | PRs lack context | -| `backlog_health` | 3 | Issues have descriptive titles, labels | Unclear what to work on | - ---- - -## 9. Product & Analytics - -Connect errors to insights and understand user behavior. - -| Criterion | Level | Detection | Impact | -|-----------|-------|-----------|--------| -| `error_to_insight_pipeline` | 5 | Sentry-GitHub issue creation automation | Errors don't become actionable issues | -| `product_analytics_instrumentation` | 5 | Mixpanel, Amplitude, PostHog, Heap | No user behavior data to inform decisions | - ---- - -## Skipped Criteria - -Some criteria are skipped based on repository type: -- **Libraries**: Skip deployment-related criteria (progressive_rollout, health_checks) -- **CLI tools**: Skip web-specific criteria (dast_scanning) -- **Database projects**: Skip N+1 detection (they ARE the database) -- **Single apps**: Skip monorepo tooling criteria - -The analysis script automatically determines which criteria to skip based on detected repository characteristics. diff --git a/skills/readiness-report/references/maturity-levels.md b/skills/readiness-report/references/maturity-levels.md deleted file mode 100644 index d00e79b..0000000 --- a/skills/readiness-report/references/maturity-levels.md +++ /dev/null @@ -1,249 +0,0 @@ -# Maturity Levels - -Repositories progress through five levels. Each level represents a qualitative shift in what autonomous agents can accomplish. - -## Level Progression Rules - -1. **80% Threshold**: To unlock a level, pass ≥80% of criteria at that level -2. **Cumulative**: Must also pass all previous levels at 80%+ -3. **Binary Criteria**: Each criterion is pass/fail (no partial credit) -4. **Skipped Criteria**: Not applicable criteria are excluded from the calculation - ---- - -## Level 1: Initial - -**Status**: Basic version control -**Agent Capability**: Manual assistance only - -### Requirements -The bare minimum for any collaborative development. - -| Criterion | Pillar | -|-----------|--------| -| `formatter` | Style & Validation | -| `lint_config` | Style & Validation | -| `type_check` | Style & Validation | -| `build_cmd_doc` | Build System | -| `deps_pinned` | Build System | -| `vcs_cli_tools` | Build System | -| `unit_tests_exist` | Testing | -| `unit_tests_runnable` | Testing | -| `readme` | Documentation | -| `gitignore_comprehensive` | Security | - -### What Agents Can Do -- Read and understand code -- Make simple edits with manual verification -- Cannot reliably validate their own changes - -### What's Missing -Without L1, agents operate blind—no way to verify builds, run tests, or ensure code quality. - ---- - -## Level 2: Managed - -**Status**: Basic CI/CD and testing -**Agent Capability**: Simple, well-defined tasks - -### Requirements -Foundational automation and documentation. - -| Criterion | Pillar | -|-----------|--------| -| `strict_typing` | Style & Validation | -| `pre_commit_hooks` | Style & Validation | -| `naming_consistency` | Style & Validation | -| `large_file_detection` | Style & Validation | -| `fast_ci_feedback` | Build System | -| `single_command_setup` | Build System | -| `release_automation` | Build System | -| `deployment_frequency` | Build System | -| `test_naming_conventions` | Testing | -| `test_isolation` | Testing | -| `agents_md` | Documentation | -| `documentation_freshness` | Documentation | -| `env_template` | Dev Environment | -| `structured_logging` | Observability | -| `code_quality_metrics` | Observability | -| `secrets_management` | Security | -| `codeowners` | Security | -| `branch_protection` | Security | -| `issue_templates` | Task Discovery | -| `issue_labeling_system` | Task Discovery | -| `pr_templates` | Task Discovery | - -### What Agents Can Do -- Fix simple, well-scoped bugs -- Make straightforward changes with fast feedback -- Run tests and verify basic correctness - -### What's Missing -Without L2, feedback loops are too slow. Agents wait minutes for CI instead of seconds for local checks. - ---- - -## Level 3: Standardized - -**Status**: Production-ready for agents -**Agent Capability**: Routine maintenance - -### Requirements -Clear processes defined and enforced. This is the target for most teams. - -| Criterion | Pillar | -|-----------|--------| -| `code_modularization` | Style & Validation | -| `cyclomatic_complexity` | Style & Validation | -| `dead_code_detection` | Style & Validation | -| `duplicate_code_detection` | Style & Validation | -| `release_notes_automation` | Build System | -| `agentic_development` | Build System | -| `automated_pr_review` | Build System | -| `feature_flag_infrastructure` | Build System | -| `integration_tests_exist` | Testing | -| `test_coverage_thresholds` | Testing | -| `api_schema_docs` | Documentation | -| `automated_doc_generation` | Documentation | -| `service_flow_documented` | Documentation | -| `skills` | Documentation | -| `devcontainer` | Dev Environment | -| `devcontainer_runnable` | Dev Environment | -| `database_schema` | Dev Environment | -| `local_services_setup` | Dev Environment | -| `error_tracking_contextualized` | Observability | -| `distributed_tracing` | Observability | -| `metrics_collection` | Observability | -| `health_checks` | Observability | -| `dependency_update_automation` | Security | -| `log_scrubbing` | Security | -| `pii_handling` | Security | -| `backlog_health` | Task Discovery | - -### What Agents Can Do -- Bug fixes and routine maintenance -- Test additions and documentation updates -- Dependency upgrades with confidence -- Feature work with clear specifications - -### Example Repositories at L3 -- FastAPI -- GitHub CLI -- pytest - ---- - -## Level 4: Measured - -**Status**: Comprehensive automation -**Agent Capability**: Complex features - -### Requirements -Advanced tooling and metrics for optimization. - -| Criterion | Pillar | -|-----------|--------| -| `tech_debt_tracking` | Style & Validation | -| `n_plus_one_detection` | Style & Validation | -| `build_performance_tracking` | Build System | -| `unused_dependencies_detection` | Build System | -| `dead_feature_flag_detection` | Build System | -| `monorepo_tooling` | Build System | -| `version_drift_detection` | Build System | -| `flaky_test_detection` | Testing | -| `test_performance_tracking` | Testing | -| `agents_md_validation` | Documentation | -| `profiling_instrumentation` | Observability | -| `alerting_configured` | Observability | -| `deployment_observability` | Observability | -| `runbooks_documented` | Observability | -| `automated_security_review` | Security | -| `secret_scanning` | Security | - -### What Agents Can Do -- Complex multi-file refactors -- Performance optimization with data -- Architecture improvements -- Security hardening - -### Example Repositories at L4 -- CockroachDB -- Temporal - ---- - -## Level 5: Optimized - -**Status**: Full autonomous capability -**Agent Capability**: End-to-end development - -### Requirements -Comprehensive observability, security, and automation. - -| Criterion | Pillar | -|-----------|--------| -| `progressive_rollout` | Build System | -| `rollback_automation` | Build System | -| `circuit_breakers` | Observability | -| `dast_scanning` | Security | -| `privacy_compliance` | Security | -| `error_to_insight_pipeline` | Task Discovery | -| `product_analytics_instrumentation` | Task Discovery | - -### What Agents Can Do -- Full feature development with minimal oversight -- Incident response and remediation -- Autonomous triage and prioritization -- Continuous improvement of the codebase itself - -### What L5 Looks Like -Very few repositories achieve L5. It requires mature DevOps, comprehensive observability, and sophisticated automation. - ---- - -## Calculating Scores - -### Repository Score -``` -Pass Rate = (Passed Criteria) / (Total Applicable Criteria) × 100% -``` - -### Level Determination -``` -For each level L (1 to 5): - If (pass rate at L ≥ 80%) AND (all lower levels ≥ 80%): - Repository is at level L -``` - -### Organization Score -``` -Org Level = floor(average of all repository levels) -Key Metric = % of active repos at L3+ -``` - ---- - -## Priority Order for Improvement - -1. **Fix L1 failures first**: These block basic operation -2. **Then L2**: Enable fast feedback loops -3. **Aim for L3**: Production-ready target -4. **L4+ is optimization**: Nice to have, not essential - -### Quick Wins by Level - -**L1 → L2**: -- Add pre-commit hooks -- Document setup commands -- Add AGENTS.md - -**L2 → L3**: -- Add integration tests -- Set up devcontainer -- Configure automated reviews - -**L3 → L4**: -- Add complexity analysis -- Set up flaky test detection -- Enable security scanning diff --git a/skills/readiness-report/scripts/analyze_repo.py b/skills/readiness-report/scripts/analyze_repo.py deleted file mode 100755 index 482da82..0000000 --- a/skills/readiness-report/scripts/analyze_repo.py +++ /dev/null @@ -1,1396 +0,0 @@ -#!/usr/bin/env python3 -""" -Repository Readiness Analyzer - -Analyzes a repository across eight technical pillars to determine -agent readiness. Outputs a JSON file with detailed criteria evaluation. - -Usage: - python analyze_repo.py --repo-path /path/to/repo - python analyze_repo.py --repo-path . --output /tmp/analysis.json -""" - -import argparse -import json -import os -import re -import subprocess -from dataclasses import dataclass, field, asdict -from pathlib import Path -from typing import Optional -from enum import Enum - - -class CriterionStatus(str, Enum): - PASS = "pass" - FAIL = "fail" - SKIP = "skip" - - -@dataclass -class CriterionResult: - id: str - pillar: str - level: int - status: CriterionStatus - score: str # "1/1", "0/1", or "—/—" - reason: str - - -@dataclass -class PillarResult: - name: str - passed: int - total: int - criteria: list[CriterionResult] = field(default_factory=list) - - @property - def percentage(self) -> int: - if self.total == 0: - return 100 - return int((self.passed / self.total) * 100) - - -@dataclass -class AnalysisResult: - repo_path: str - repo_name: str - pillars: dict[str, PillarResult] = field(default_factory=dict) - level_scores: dict[int, float] = field(default_factory=dict) - achieved_level: int = 1 - pass_rate: float = 0.0 - total_passed: int = 0 - total_criteria: int = 0 - repo_type: str = "application" # library, cli, database, monorepo, application - languages: list[str] = field(default_factory=list) - - -class RepoAnalyzer: - """Analyzes repository for agent readiness criteria.""" - - def __init__(self, repo_path: str): - self.repo_path = Path(repo_path).resolve() - self.result = AnalysisResult( - repo_path=str(self.repo_path), - repo_name=self.repo_path.name - ) - self._file_cache: dict[str, bool] = {} - self._content_cache: dict[str, str] = {} - - def analyze(self) -> AnalysisResult: - """Run full analysis and return results.""" - self._detect_repo_type() - self._detect_languages() - self._evaluate_all_pillars() - self._calculate_levels() - return self.result - - def _file_exists(self, *patterns: str) -> bool: - """Check if any of the given file patterns exist.""" - for pattern in patterns: - cache_key = f"exists:{pattern}" - if cache_key in self._file_cache: - if self._file_cache[cache_key]: - return True - continue - - # Handle glob patterns - if "*" in pattern: - matches = list(self.repo_path.glob(pattern)) - exists = len(matches) > 0 - else: - exists = (self.repo_path / pattern).exists() - - self._file_cache[cache_key] = exists - if exists: - return True - return False - - def _read_file(self, path: str) -> Optional[str]: - """Read file content, with caching.""" - if path in self._content_cache: - return self._content_cache[path] - - full_path = self.repo_path / path - if not full_path.exists(): - return None - - try: - content = full_path.read_text(errors='ignore') - self._content_cache[path] = content - return content - except Exception: - return None - - def _search_files(self, pattern: str, content_pattern: str = None) -> bool: - """Search for files matching pattern, optionally with content.""" - matches = list(self.repo_path.glob(pattern)) - if not matches: - return False - if content_pattern is None: - return True - - regex = re.compile(content_pattern, re.IGNORECASE) - for match in matches[:10]: # Limit for performance - try: - content = match.read_text(errors='ignore') - if regex.search(content): - return True - except Exception: - continue - return False - - def _run_command(self, cmd: list[str], timeout: int = 10) -> tuple[int, str]: - """Run a command and return (exit_code, output).""" - try: - result = subprocess.run( - cmd, - cwd=self.repo_path, - capture_output=True, - text=True, - timeout=timeout - ) - return result.returncode, result.stdout + result.stderr - except Exception as e: - return -1, str(e) - - def _detect_repo_type(self): - """Detect repository type for criterion skipping.""" - # Check for library indicators - if self._file_exists("setup.py", "setup.cfg") and not self._file_exists("Dockerfile"): - if "library" in str(self._read_file("setup.py") or "").lower(): - self.result.repo_type = "library" - return - - if self._file_exists("pyproject.toml"): - content = self._read_file("pyproject.toml") or "" - if "[project]" in content and "Dockerfile" not in os.listdir(self.repo_path): - # Likely a library - readme = self._read_file("README.md") or "" - if "pip install" in readme.lower() and "docker" not in readme.lower(): - self.result.repo_type = "library" - return - - # Check for CLI tool - if self._file_exists("**/cli.py", "**/main.py", "**/cmd/**"): - readme = self._read_file("README.md") or "" - if any(x in readme.lower() for x in ["command line", "cli", "usage:"]): - self.result.repo_type = "cli" - return - - # Check for database project - if "database" in self.result.repo_name.lower() or "db" in self.result.repo_name.lower(): - self.result.repo_type = "database" - return - - # Check for monorepo - if self._file_exists("packages/*", "apps/*", "lerna.json", "pnpm-workspace.yaml"): - self.result.repo_type = "monorepo" - return - - self.result.repo_type = "application" - - def _detect_languages(self): - """Detect primary programming languages.""" - languages = [] - - if self._file_exists("*.py", "**/*.py", "pyproject.toml", "setup.py"): - languages.append("Python") - if self._file_exists("*.ts", "**/*.ts", "tsconfig.json"): - languages.append("TypeScript") - if self._file_exists("*.js", "**/*.js", "package.json"): - if "TypeScript" not in languages: - languages.append("JavaScript") - if self._file_exists("*.go", "**/*.go", "go.mod"): - languages.append("Go") - if self._file_exists("*.rs", "**/*.rs", "Cargo.toml"): - languages.append("Rust") - if self._file_exists("*.java", "**/*.java", "pom.xml", "build.gradle"): - languages.append("Java") - if self._file_exists("*.rb", "**/*.rb", "Gemfile"): - languages.append("Ruby") - if self._file_exists("*.cpp", "*.cc", "**/*.cpp", "CMakeLists.txt"): - languages.append("C++") - - self.result.languages = languages if languages else ["Unknown"] - - def _should_skip(self, criterion_id: str) -> tuple[bool, str]: - """Determine if a criterion should be skipped based on repo type.""" - repo_type = self.result.repo_type - - skip_rules = { - "library": [ - ("health_checks", "Library, not a deployed service"), - ("progressive_rollout", "Not applicable for a library"), - ("rollback_automation", "Not applicable for a library"), - ("dast_scanning", "Library, not a web service"), - ("alerting_configured", "Library without runtime"), - ("deployment_observability", "Library without deployments"), - ("metrics_collection", "Library without runtime"), - ("profiling_instrumentation", "Library where profiling not meaningful"), - ("circuit_breakers", "Library without external dependencies"), - ("distributed_tracing", "Library without runtime"), - ("local_services_setup", "Library without external dependencies"), - ("database_schema", "Library without database"), - ("n_plus_one_detection", "Library without database/ORM"), - ("privacy_compliance", "Library without user data"), - ("pii_handling", "Library without user data"), - ], - "database": [ - ("n_plus_one_detection", "Database project IS the database layer"), - ("dast_scanning", "Database server, not web application"), - ], - "cli": [ - ("dast_scanning", "CLI tool, not web application"), - ("health_checks", "CLI tool, not a service"), - ("progressive_rollout", "CLI tool without deployments"), - ], - } - - for rule_type, rules in skip_rules.items(): - if repo_type == rule_type: - for crit_id, reason in rules: - if criterion_id == crit_id: - return True, reason - - # Skip monorepo criteria for non-monorepos - if repo_type != "monorepo": - if criterion_id in ["monorepo_tooling", "version_drift_detection"]: - return True, "Single-application repository, not a monorepo" - - # Skip prerequisites - if criterion_id == "devcontainer_runnable": - if not self._file_exists(".devcontainer/devcontainer.json"): - return True, "No devcontainer to test (prerequisite failed)" - - if criterion_id == "agents_md_validation": - if not self._file_exists("AGENTS.md", "CLAUDE.md"): - return True, "No AGENTS.md exists (prerequisite failed)" - - if criterion_id == "dead_feature_flag_detection": - # Check if feature flags exist first - if not self._check_feature_flags(): - return True, "No feature flag infrastructure (prerequisite failed)" - - return False, "" - - def _check_feature_flags(self) -> bool: - """Check if feature flag infrastructure exists.""" - # Check for common feature flag services - patterns = [ - "launchdarkly", "statsig", "unleash", "growthbook", - "feature.flag", "featureflag", "feature_flag" - ] - - for pattern in ["package.json", "requirements.txt", "go.mod", "Gemfile"]: - content = self._read_file(pattern) - if content: - for flag_pattern in patterns: - if flag_pattern in content.lower(): - return True - return False - - def _evaluate_all_pillars(self): - """Evaluate all criteria across all pillars.""" - pillars = { - "Style & Validation": self._evaluate_style_validation, - "Build System": self._evaluate_build_system, - "Testing": self._evaluate_testing, - "Documentation": self._evaluate_documentation, - "Dev Environment": self._evaluate_dev_environment, - "Debugging & Observability": self._evaluate_observability, - "Security": self._evaluate_security, - "Task Discovery": self._evaluate_task_discovery, - "Product & Analytics": self._evaluate_product_analytics, - } - - for pillar_name, evaluate_func in pillars.items(): - criteria = evaluate_func() - passed = sum(1 for c in criteria if c.status == CriterionStatus.PASS) - total = sum(1 for c in criteria if c.status != CriterionStatus.SKIP) - - self.result.pillars[pillar_name] = PillarResult( - name=pillar_name, - passed=passed, - total=total, - criteria=criteria - ) - - self.result.total_passed += passed - self.result.total_criteria += total - - if self.result.total_criteria > 0: - self.result.pass_rate = round( - (self.result.total_passed / self.result.total_criteria) * 100, 1 - ) - - def _make_result( - self, - criterion_id: str, - pillar: str, - level: int, - passed: bool, - reason: str - ) -> CriterionResult: - """Create a criterion result, handling skips.""" - should_skip, skip_reason = self._should_skip(criterion_id) - - if should_skip: - return CriterionResult( - id=criterion_id, - pillar=pillar, - level=level, - status=CriterionStatus.SKIP, - score="—/—", - reason=skip_reason - ) - - return CriterionResult( - id=criterion_id, - pillar=pillar, - level=level, - status=CriterionStatus.PASS if passed else CriterionStatus.FAIL, - score="1/1" if passed else "0/1", - reason=reason - ) - - def _evaluate_style_validation(self) -> list[CriterionResult]: - """Evaluate Style & Validation pillar.""" - pillar = "Style & Validation" - results = [] - - # L1: formatter - formatter_found = self._file_exists( - ".prettierrc", ".prettierrc.json", ".prettierrc.js", "prettier.config.js", - "pyproject.toml", ".black.toml", # Black/Ruff - ".gofmt", # Go uses gofmt by default - "rustfmt.toml", ".rustfmt.toml" - ) - if not formatter_found: - # Check pyproject.toml for ruff format - pyproject = self._read_file("pyproject.toml") or "" - formatter_found = "ruff" in pyproject.lower() or "black" in pyproject.lower() - results.append(self._make_result( - "formatter", pillar, 1, formatter_found, - "Formatter configured" if formatter_found else "No formatter config found" - )) - - # L1: lint_config - lint_found = self._file_exists( - ".eslintrc", ".eslintrc.js", ".eslintrc.json", ".eslintrc.yaml", - "eslint.config.js", "eslint.config.mjs", - ".pylintrc", "pylintrc", - "golangci.yml", ".golangci.yml", ".golangci.yaml" - ) - if not lint_found: - pyproject = self._read_file("pyproject.toml") or "" - lint_found = "ruff" in pyproject.lower() or "pylint" in pyproject.lower() - results.append(self._make_result( - "lint_config", pillar, 1, lint_found, - "Linter configured" if lint_found else "No linter config found" - )) - - # L1: type_check - type_check = False - if "Go" in self.result.languages or "Rust" in self.result.languages: - type_check = True # Statically typed by default - elif self._file_exists("tsconfig.json"): - type_check = True - elif self._file_exists("pyproject.toml"): - content = self._read_file("pyproject.toml") or "" - type_check = "mypy" in content.lower() - results.append(self._make_result( - "type_check", pillar, 1, type_check, - "Type checking configured" if type_check else "No type checking found" - )) - - # L2: strict_typing - strict_typing = False - if "Go" in self.result.languages or "Rust" in self.result.languages: - strict_typing = True - elif self._file_exists("tsconfig.json"): - content = self._read_file("tsconfig.json") or "" - strict_typing = '"strict": true' in content or '"strict":true' in content - elif self._file_exists("pyproject.toml"): - content = self._read_file("pyproject.toml") or "" - strict_typing = "strict = true" in content or "strict=true" in content - results.append(self._make_result( - "strict_typing", pillar, 2, strict_typing, - "Strict typing enabled" if strict_typing else "Strict typing not enabled" - )) - - # L2: pre_commit_hooks - pre_commit = self._file_exists( - ".pre-commit-config.yaml", ".pre-commit-config.yml", - ".husky", ".husky/*" - ) - results.append(self._make_result( - "pre_commit_hooks", pillar, 2, pre_commit, - "Pre-commit hooks configured" if pre_commit else "No pre-commit hooks found" - )) - - # L2: naming_consistency - naming = False - eslint = self._read_file(".eslintrc.json") or self._read_file(".eslintrc") or "" - if "naming" in eslint.lower() or "@typescript-eslint/naming" in eslint: - naming = True - agents_md = self._read_file("AGENTS.md") or self._read_file("CLAUDE.md") or "" - if "naming" in agents_md.lower() or "convention" in agents_md.lower(): - naming = True - # Go uses stdlib naming by default - if "Go" in self.result.languages: - naming = True - results.append(self._make_result( - "naming_consistency", pillar, 2, naming, - "Naming conventions enforced" if naming else "No naming convention enforcement" - )) - - # L2: large_file_detection - large_file = self._file_exists(".gitattributes", ".lfsconfig") - if not large_file: - pre_commit_cfg = self._read_file(".pre-commit-config.yaml") or "" - large_file = "check-added-large-files" in pre_commit_cfg - results.append(self._make_result( - "large_file_detection", pillar, 2, large_file, - "Large file detection configured" if large_file else "No large file detection" - )) - - # L3: code_modularization - modular = self._file_exists( - ".importlinter", "nx.json", "BUILD.bazel", "BUILD" - ) - results.append(self._make_result( - "code_modularization", pillar, 3, modular, - "Module boundaries enforced" if modular else "No module boundary enforcement" - )) - - # L3: cyclomatic_complexity - complexity = False - for config in [".golangci.yml", ".golangci.yaml", "pyproject.toml"]: - content = self._read_file(config) or "" - if any(x in content.lower() for x in ["gocyclo", "mccabe", "complexity", "radon"]): - complexity = True - break - results.append(self._make_result( - "cyclomatic_complexity", pillar, 3, complexity, - "Complexity analysis configured" if complexity else "No complexity analysis" - )) - - # L3: dead_code_detection - dead_code = False - workflows = list(self.repo_path.glob(".github/workflows/*.yml")) + \ - list(self.repo_path.glob(".github/workflows/*.yaml")) - for wf in workflows[:5]: - content = wf.read_text(errors='ignore') - if any(x in content.lower() for x in ["vulture", "knip", "deadcode"]): - dead_code = True - break - results.append(self._make_result( - "dead_code_detection", pillar, 3, dead_code, - "Dead code detection enabled" if dead_code else "No dead code detection" - )) - - # L3: duplicate_code_detection - duplicate = False - for wf in workflows[:5]: - content = wf.read_text(errors='ignore') - if any(x in content.lower() for x in ["jscpd", "pmd cpd", "sonarqube"]): - duplicate = True - break - results.append(self._make_result( - "duplicate_code_detection", pillar, 3, duplicate, - "Duplicate detection enabled" if duplicate else "No duplicate detection" - )) - - # L4: tech_debt_tracking - tech_debt = False - for wf in workflows[:5]: - content = wf.read_text(errors='ignore') - if any(x in content.lower() for x in ["todo", "fixme", "sonar"]): - tech_debt = True - break - results.append(self._make_result( - "tech_debt_tracking", pillar, 4, tech_debt, - "Tech debt tracking enabled" if tech_debt else "No tech debt tracking" - )) - - # L4: n_plus_one_detection - n_plus_one = False - deps = (self._read_file("requirements.txt") or "") + \ - (self._read_file("Gemfile") or "") + \ - (self._read_file("package.json") or "") - if any(x in deps.lower() for x in ["nplusone", "bullet", "query-analyzer"]): - n_plus_one = True - results.append(self._make_result( - "n_plus_one_detection", pillar, 4, n_plus_one, - "N+1 detection enabled" if n_plus_one else "No N+1 query detection" - )) - - return results - - def _evaluate_build_system(self) -> list[CriterionResult]: - """Evaluate Build System pillar.""" - pillar = "Build System" - results = [] - - # L1: build_cmd_doc - readme = self._read_file("README.md") or "" - agents_md = self._read_file("AGENTS.md") or self._read_file("CLAUDE.md") or "" - build_doc = any(x in (readme + agents_md).lower() for x in [ - "npm run", "yarn", "pnpm", "make", "cargo build", "go build", - "pip install", "python setup.py", "gradle", "mvn" - ]) - results.append(self._make_result( - "build_cmd_doc", pillar, 1, build_doc, - "Build commands documented" if build_doc else "Build commands not documented" - )) - - # L1: deps_pinned - lockfile = self._file_exists( - "package-lock.json", "yarn.lock", "pnpm-lock.yaml", - "uv.lock", "poetry.lock", "Pipfile.lock", - "go.sum", "Cargo.lock", "Gemfile.lock" - ) - results.append(self._make_result( - "deps_pinned", pillar, 1, lockfile, - "Dependencies pinned with lockfile" if lockfile else "No lockfile found" - )) - - # L1: vcs_cli_tools - code, output = self._run_command(["gh", "auth", "status"]) - vcs_cli = code == 0 - if not vcs_cli: - code, output = self._run_command(["glab", "auth", "status"]) - vcs_cli = code == 0 - results.append(self._make_result( - "vcs_cli_tools", pillar, 1, vcs_cli, - "VCS CLI authenticated" if vcs_cli else "VCS CLI not authenticated" - )) - - # L2: fast_ci_feedback - # Check for CI config existence as proxy - ci_exists = self._file_exists( - ".github/workflows/*.yml", ".github/workflows/*.yaml", - ".gitlab-ci.yml", ".circleci/config.yml", - "Jenkinsfile", ".travis.yml" - ) - results.append(self._make_result( - "fast_ci_feedback", pillar, 2, ci_exists, - "CI workflow configured" if ci_exists else "No CI configuration found" - )) - - # L2: single_command_setup - single_cmd = any(x in (readme + agents_md).lower() for x in [ - "make install", "npm install", "yarn install", "pip install -e", - "docker-compose up", "./dev", "make setup", "just" - ]) - results.append(self._make_result( - "single_command_setup", pillar, 2, single_cmd, - "Single command setup documented" if single_cmd else "No single command setup" - )) - - # L2: release_automation - release_auto = self._search_files( - ".github/workflows/*.yml", - r"(release|publish|deploy)" - ) or self._search_files( - ".github/workflows/*.yaml", - r"(release|publish|deploy)" - ) - results.append(self._make_result( - "release_automation", pillar, 2, release_auto, - "Release automation configured" if release_auto else "No release automation" - )) - - # L2: deployment_frequency (check for recent releases) - release_auto_exists = release_auto # Use same check as proxy - results.append(self._make_result( - "deployment_frequency", pillar, 2, release_auto_exists, - "Regular deployments" if release_auto_exists else "Deployment frequency unclear" - )) - - # L3: release_notes_automation - release_notes = self._search_files( - ".github/workflows/*.yml", - r"(changelog|release.notes|latest.changes)" - ) - results.append(self._make_result( - "release_notes_automation", pillar, 3, release_notes, - "Release notes automated" if release_notes else "No release notes automation" - )) - - # L3: agentic_development - code, output = self._run_command(["git", "log", "--oneline", "-50"]) - agentic = any(x in output.lower() for x in [ - "co-authored-by", "droid", "copilot", "claude", "gpt", "ai agent" - ]) - results.append(self._make_result( - "agentic_development", pillar, 3, agentic, - "AI agent commits found" if agentic else "No AI agent commits detected" - )) - - # L3: automated_pr_review - pr_review = self._file_exists("danger.js", "dangerfile.js", "dangerfile.ts") - if not pr_review: - workflows = list(self.repo_path.glob(".github/workflows/*.yml")) - for wf in workflows[:5]: - content = wf.read_text(errors='ignore') - if any(x in content.lower() for x in ["review", "danger", "lint-pr"]): - pr_review = True - break - results.append(self._make_result( - "automated_pr_review", pillar, 3, pr_review, - "Automated PR review configured" if pr_review else "No automated PR review" - )) - - # L3: feature_flag_infrastructure - feature_flags = self._check_feature_flags() - results.append(self._make_result( - "feature_flag_infrastructure", pillar, 3, feature_flags, - "Feature flags configured" if feature_flags else "No feature flag system" - )) - - # L4: build_performance_tracking - build_perf = False - if self._file_exists("turbo.json", "nx.json"): - build_perf = True - results.append(self._make_result( - "build_performance_tracking", pillar, 4, build_perf, - "Build caching configured" if build_perf else "No build performance tracking" - )) - - # L4: heavy_dependency_detection (for JS bundles) - heavy_deps = False - pkg_json = self._read_file("package.json") or "" - if any(x in pkg_json.lower() for x in ["webpack-bundle-analyzer", "bundlesize", "size-limit"]): - heavy_deps = True - results.append(self._make_result( - "heavy_dependency_detection", pillar, 4, heavy_deps, - "Bundle size tracking configured" if heavy_deps else "No bundle size tracking" - )) - - # L4: unused_dependencies_detection - unused_deps = False - workflows = list(self.repo_path.glob(".github/workflows/*.yml")) - for wf in workflows[:5]: - content = wf.read_text(errors='ignore') - if any(x in content.lower() for x in ["depcheck", "deptry", "go mod tidy"]): - unused_deps = True - break - results.append(self._make_result( - "unused_dependencies_detection", pillar, 4, unused_deps, - "Unused deps detection enabled" if unused_deps else "No unused deps detection" - )) - - # L4: dead_feature_flag_detection - dead_flags = False # Requires feature flag infra first - results.append(self._make_result( - "dead_feature_flag_detection", pillar, 4, dead_flags, - "Dead flag detection enabled" if dead_flags else "No dead flag detection" - )) - - # L4: monorepo_tooling - monorepo_tools = self._file_exists("lerna.json", "nx.json", "turbo.json", "pnpm-workspace.yaml") - results.append(self._make_result( - "monorepo_tooling", pillar, 4, monorepo_tools, - "Monorepo tooling configured" if monorepo_tools else "No monorepo tooling" - )) - - # L4: version_drift_detection - version_drift = False # Complex to detect - results.append(self._make_result( - "version_drift_detection", pillar, 4, version_drift, - "Version drift detection enabled" if version_drift else "No version drift detection" - )) - - # L5: progressive_rollout - progressive = False - for pattern in ["*.yml", "*.yaml"]: - if self._search_files(f".github/workflows/{pattern}", r"canary|gradual|rollout"): - progressive = True - break - results.append(self._make_result( - "progressive_rollout", pillar, 5, progressive, - "Progressive rollout configured" if progressive else "No progressive rollout" - )) - - # L5: rollback_automation - rollback = False - results.append(self._make_result( - "rollback_automation", pillar, 5, rollback, - "Rollback automation configured" if rollback else "No rollback automation" - )) - - return results - - def _evaluate_testing(self) -> list[CriterionResult]: - """Evaluate Testing pillar.""" - pillar = "Testing" - results = [] - - # L1: unit_tests_exist - tests_exist = self._file_exists( - "tests/**/*.py", "test/**/*.py", "*_test.py", "*_test.go", - "**/*.spec.ts", "**/*.spec.js", "**/*.test.ts", "**/*.test.js", - "spec/**/*.rb", "tests/**/*.rs" - ) - results.append(self._make_result( - "unit_tests_exist", pillar, 1, tests_exist, - "Unit tests found" if tests_exist else "No unit tests found" - )) - - # L1: unit_tests_runnable - readme = self._read_file("README.md") or "" - agents_md = self._read_file("AGENTS.md") or self._read_file("CLAUDE.md") or "" - test_cmd = any(x in (readme + agents_md).lower() for x in [ - "pytest", "npm test", "yarn test", "go test", "cargo test", - "make test", "rake test", "rspec", "jest" - ]) - results.append(self._make_result( - "unit_tests_runnable", pillar, 1, test_cmd, - "Test commands documented" if test_cmd else "Test commands not documented" - )) - - # L2: test_naming_conventions - naming = False - if self._file_exists("pyproject.toml"): - content = self._read_file("pyproject.toml") or "" - naming = "pytest" in content.lower() - if self._file_exists("jest.config.js", "jest.config.ts"): - naming = True - if "Go" in self.result.languages: - naming = True # Go has standard _test.go convention - results.append(self._make_result( - "test_naming_conventions", pillar, 2, naming, - "Test naming conventions enforced" if naming else "No test naming conventions" - )) - - # L2: test_isolation - isolation = False - if self._file_exists("pyproject.toml"): - content = self._read_file("pyproject.toml") or "" - isolation = "pytest-xdist" in content or "-n auto" in content - workflows = list(self.repo_path.glob(".github/workflows/*.yml")) - for wf in workflows[:5]: - content = wf.read_text(errors='ignore') - if "matrix" in content.lower(): - isolation = True - break - if "Go" in self.result.languages: - isolation = True # Go tests run in parallel by default - results.append(self._make_result( - "test_isolation", pillar, 2, isolation, - "Tests support isolation/parallelism" if isolation else "No test isolation" - )) - - # L3: integration_tests_exist - integration = self._file_exists( - "tests/integration/**", "integration/**", "e2e/**", - "tests/e2e/**", "cypress/**", "playwright.config.*" - ) - results.append(self._make_result( - "integration_tests_exist", pillar, 3, integration, - "Integration tests found" if integration else "No integration tests found" - )) - - # L3: test_coverage_thresholds - coverage = False - for wf in workflows[:5]: - content = wf.read_text(errors='ignore') - if any(x in content.lower() for x in ["coverage", "codecov", "coveralls"]): - coverage = True - break - if self._file_exists(".coveragerc", "coverage.xml", "codecov.yml"): - coverage = True - results.append(self._make_result( - "test_coverage_thresholds", pillar, 3, coverage, - "Coverage thresholds enforced" if coverage else "No coverage thresholds" - )) - - # L4: flaky_test_detection - flaky = False - for wf in workflows[:5]: - content = wf.read_text(errors='ignore') - if any(x in content.lower() for x in ["retry", "flaky", "quarantine", "rerun"]): - flaky = True - break - results.append(self._make_result( - "flaky_test_detection", pillar, 4, flaky, - "Flaky test handling configured" if flaky else "No flaky test detection" - )) - - # L4: test_performance_tracking - test_perf = False - for wf in workflows[:5]: - content = wf.read_text(errors='ignore') - if any(x in content.lower() for x in ["durations", "timing", "benchmark"]): - test_perf = True - break - results.append(self._make_result( - "test_performance_tracking", pillar, 4, test_perf, - "Test performance tracked" if test_perf else "No test performance tracking" - )) - - return results - - def _evaluate_documentation(self) -> list[CriterionResult]: - """Evaluate Documentation pillar.""" - pillar = "Documentation" - results = [] - - # L1: readme - readme = self._file_exists("README.md", "README.rst", "README.txt", "README") - results.append(self._make_result( - "readme", pillar, 1, readme, - "README exists" if readme else "No README found" - )) - - # L2: agents_md - agents_md = self._file_exists("AGENTS.md", "CLAUDE.md") - results.append(self._make_result( - "agents_md", pillar, 2, agents_md, - "AGENTS.md exists" if agents_md else "No AGENTS.md found" - )) - - # L2: documentation_freshness - freshness = False - code, output = self._run_command([ - "git", "log", "-1", "--format=%ci", "--", "README.md" - ]) - if code == 0 and output.strip(): - freshness = True - results.append(self._make_result( - "documentation_freshness", pillar, 2, freshness, - "Documentation recently updated" if freshness else "Documentation may be stale" - )) - - # L3: api_schema_docs - api_docs = self._file_exists( - "openapi.yaml", "openapi.json", "swagger.yaml", "swagger.json", - "schema.graphql", "*.graphql", - "docs/api/**", "api-docs/**" - ) - results.append(self._make_result( - "api_schema_docs", pillar, 3, api_docs, - "API documentation exists" if api_docs else "No API documentation found" - )) - - # L3: automated_doc_generation - doc_gen = self._search_files( - ".github/workflows/*.yml", - r"(docs|documentation|mkdocs|sphinx|typedoc)" - ) - results.append(self._make_result( - "automated_doc_generation", pillar, 3, doc_gen, - "Doc generation automated" if doc_gen else "No automated doc generation" - )) - - # L3: service_flow_documented - diagrams = self._file_exists( - "**/*.mermaid", "**/*.puml", "docs/architecture*", - "docs/**/*.md" - ) - results.append(self._make_result( - "service_flow_documented", pillar, 3, diagrams, - "Architecture documented" if diagrams else "No architecture documentation" - )) - - # L3: skills - skills = self._file_exists( - ".claude/skills/**", ".factory/skills/**", ".skills/**" - ) - results.append(self._make_result( - "skills", pillar, 3, skills, - "Skills directory exists" if skills else "No skills directory" - )) - - # L4: agents_md_validation - agents_validation = False - workflows = list(self.repo_path.glob(".github/workflows/*.yml")) - for wf in workflows[:5]: - content = wf.read_text(errors='ignore') - if any(x in content.lower() for x in ["agents.md", "claude.md"]): - agents_validation = True - break - results.append(self._make_result( - "agents_md_validation", pillar, 4, agents_validation, - "AGENTS.md validation in CI" if agents_validation else "No AGENTS.md validation" - )) - - return results - - def _evaluate_dev_environment(self) -> list[CriterionResult]: - """Evaluate Dev Environment pillar.""" - pillar = "Dev Environment" - results = [] - - # L2: env_template - env_template = self._file_exists(".env.example", ".env.template", ".env.sample") - if not env_template: - readme = self._read_file("README.md") or "" - agents_md = self._read_file("AGENTS.md") or "" - env_template = "environment variable" in (readme + agents_md).lower() - results.append(self._make_result( - "env_template", pillar, 2, env_template, - "Environment template exists" if env_template else "No environment template" - )) - - # L3: devcontainer - devcontainer = self._file_exists(".devcontainer/devcontainer.json") - results.append(self._make_result( - "devcontainer", pillar, 3, devcontainer, - "Devcontainer configured" if devcontainer else "No devcontainer found" - )) - - # L3: devcontainer_runnable - devcontainer_valid = False - if devcontainer: - content = self._read_file(".devcontainer/devcontainer.json") - if content and "image" in content.lower(): - devcontainer_valid = True - results.append(self._make_result( - "devcontainer_runnable", pillar, 3, devcontainer_valid, - "Devcontainer appears valid" if devcontainer_valid else "Devcontainer not runnable" - )) - - # L3: database_schema - db_schema = self._file_exists( - "migrations/**", "db/migrations/**", "alembic/**", - "prisma/schema.prisma", "schema.sql", "db/schema.rb" - ) - results.append(self._make_result( - "database_schema", pillar, 3, db_schema, - "Database schema managed" if db_schema else "No database schema management" - )) - - # L3: local_services_setup - local_services = self._file_exists( - "docker-compose.yml", "docker-compose.yaml", - "compose.yml", "compose.yaml" - ) - results.append(self._make_result( - "local_services_setup", pillar, 3, local_services, - "Local services configured" if local_services else "No local services setup" - )) - - return results - - def _evaluate_observability(self) -> list[CriterionResult]: - """Evaluate Debugging & Observability pillar.""" - pillar = "Debugging & Observability" - results = [] - - # L2: structured_logging - logging_found = False - deps = (self._read_file("package.json") or "") + \ - (self._read_file("requirements.txt") or "") + \ - (self._read_file("go.mod") or "") - if any(x in deps.lower() for x in [ - "pino", "winston", "bunyan", "structlog", "loguru", - "zerolog", "zap", "slog" - ]): - logging_found = True - if "Python" in self.result.languages: - if self._search_files("**/*.py", r"import logging"): - logging_found = True - results.append(self._make_result( - "structured_logging", pillar, 2, logging_found, - "Structured logging configured" if logging_found else "No structured logging" - )) - - # L2: code_quality_metrics - quality_metrics = self._search_files( - ".github/workflows/*.yml", - r"(coverage|sonar|quality)" - ) - results.append(self._make_result( - "code_quality_metrics", pillar, 2, quality_metrics, - "Code quality metrics tracked" if quality_metrics else "No quality metrics" - )) - - # L3: error_tracking_contextualized - error_tracking = any(x in deps.lower() for x in [ - "sentry", "bugsnag", "rollbar", "honeybadger" - ]) - results.append(self._make_result( - "error_tracking_contextualized", pillar, 3, error_tracking, - "Error tracking configured" if error_tracking else "No error tracking" - )) - - # L3: distributed_tracing - tracing = any(x in deps.lower() for x in [ - "opentelemetry", "jaeger", "zipkin", "datadog", "x-request-id" - ]) - results.append(self._make_result( - "distributed_tracing", pillar, 3, tracing, - "Distributed tracing configured" if tracing else "No distributed tracing" - )) - - # L3: metrics_collection - metrics = any(x in deps.lower() for x in [ - "prometheus", "datadog", "newrelic", "statsd", "cloudwatch" - ]) - results.append(self._make_result( - "metrics_collection", pillar, 3, metrics, - "Metrics collection configured" if metrics else "No metrics collection" - )) - - # L3: health_checks - health = self._search_files( - "**/*.py", r"health|ready|alive" - ) or self._search_files( - "**/*.ts", r"health|ready|alive" - ) or self._search_files( - "**/*.go", r"health|ready|alive" - ) - results.append(self._make_result( - "health_checks", pillar, 3, health, - "Health checks implemented" if health else "No health checks found" - )) - - # L4: profiling_instrumentation - profiling = any(x in deps.lower() for x in [ - "pyinstrument", "py-spy", "pprof", "clinic" - ]) - results.append(self._make_result( - "profiling_instrumentation", pillar, 4, profiling, - "Profiling configured" if profiling else "No profiling instrumentation" - )) - - # L4: alerting_configured - alerting = self._file_exists( - "**/alerts*.yml", "**/alertmanager*", "monitoring/**" - ) - results.append(self._make_result( - "alerting_configured", pillar, 4, alerting, - "Alerting configured" if alerting else "No alerting configuration" - )) - - # L4: deployment_observability - deploy_obs = self._search_files( - ".github/workflows/*.yml", - r"(datadog|grafana|newrelic|deploy.*notify)" - ) - results.append(self._make_result( - "deployment_observability", pillar, 4, deploy_obs, - "Deployment observability configured" if deploy_obs else "No deployment observability" - )) - - # L4: runbooks_documented - runbooks = self._file_exists("runbooks/**", "docs/runbooks/**", "ops/**") - results.append(self._make_result( - "runbooks_documented", pillar, 4, runbooks, - "Runbooks documented" if runbooks else "No runbooks found" - )) - - # L5: circuit_breakers - circuit = any(x in deps.lower() for x in [ - "opossum", "resilience4j", "hystrix", "cockatiel" - ]) - results.append(self._make_result( - "circuit_breakers", pillar, 5, circuit, - "Circuit breakers configured" if circuit else "No circuit breakers" - )) - - return results - - def _evaluate_security(self) -> list[CriterionResult]: - """Evaluate Security pillar.""" - pillar = "Security" - results = [] - - # L1: gitignore_comprehensive - gitignore = self._file_exists(".gitignore") - comprehensive = False - if gitignore: - content = self._read_file(".gitignore") or "" - comprehensive = any(x in content.lower() for x in [ - ".env", "node_modules", "__pycache__", ".idea", ".vscode" - ]) - results.append(self._make_result( - "gitignore_comprehensive", pillar, 1, comprehensive, - "Comprehensive .gitignore" if comprehensive else "Incomplete .gitignore" - )) - - # L2: secrets_management - secrets_mgmt = False - workflows = list(self.repo_path.glob(".github/workflows/*.yml")) - for wf in workflows[:5]: - content = wf.read_text(errors='ignore') - if "secrets." in content: - secrets_mgmt = True - break - results.append(self._make_result( - "secrets_management", pillar, 2, secrets_mgmt, - "Secrets properly managed" if secrets_mgmt else "No secrets management" - )) - - # L2: codeowners - codeowners = self._file_exists("CODEOWNERS", ".github/CODEOWNERS") - results.append(self._make_result( - "codeowners", pillar, 2, codeowners, - "CODEOWNERS configured" if codeowners else "No CODEOWNERS file" - )) - - # L2: branch_protection - branch_rules = self._file_exists(".github/branch-protection.yml", ".github/rulesets/**") - results.append(self._make_result( - "branch_protection", pillar, 2, branch_rules, - "Branch protection configured" if branch_rules else "Branch protection unclear" - )) - - # L3: dependency_update_automation - dep_updates = self._file_exists( - ".github/dependabot.yml", "renovate.json", ".renovaterc" - ) - results.append(self._make_result( - "dependency_update_automation", pillar, 3, dep_updates, - "Dependency updates automated" if dep_updates else "No dependency automation" - )) - - # L3: log_scrubbing - log_scrub = False - deps = (self._read_file("package.json") or "") + \ - (self._read_file("requirements.txt") or "") - if any(x in deps.lower() for x in ["pino", "redact", "scrub"]): - log_scrub = True - results.append(self._make_result( - "log_scrubbing", pillar, 3, log_scrub, - "Log scrubbing configured" if log_scrub else "No log scrubbing" - )) - - # L3: pii_handling - pii = self._search_files( - "**/*.py", r"(redact|sanitize|mask|pii)" - ) or self._search_files( - "**/*.ts", r"(redact|sanitize|mask|pii)" - ) - results.append(self._make_result( - "pii_handling", pillar, 3, pii, - "PII handling implemented" if pii else "No PII handling found" - )) - - # L4: automated_security_review - security_scan = self._search_files( - ".github/workflows/*.yml", - r"(codeql|snyk|sonar|security)" - ) - results.append(self._make_result( - "automated_security_review", pillar, 4, security_scan, - "Security scanning enabled" if security_scan else "No security scanning" - )) - - # L4: secret_scanning - secret_scan = self._search_files( - ".github/workflows/*.yml", - r"(gitleaks|trufflehog|secret)" - ) - results.append(self._make_result( - "secret_scanning", pillar, 4, secret_scan, - "Secret scanning enabled" if secret_scan else "No secret scanning" - )) - - # L5: dast_scanning - dast = self._search_files( - ".github/workflows/*.yml", - r"(zap|dast|owasp|burp)" - ) - results.append(self._make_result( - "dast_scanning", pillar, 5, dast, - "DAST scanning enabled" if dast else "No DAST scanning" - )) - - # L5: privacy_compliance - privacy = self._file_exists( - "PRIVACY.md", "docs/privacy/**", "gdpr/**" - ) - results.append(self._make_result( - "privacy_compliance", pillar, 5, privacy, - "Privacy compliance documented" if privacy else "No privacy documentation" - )) - - return results - - def _evaluate_task_discovery(self) -> list[CriterionResult]: - """Evaluate Task Discovery pillar.""" - pillar = "Task Discovery" - results = [] - - # L2: issue_templates - issue_templates = self._file_exists( - ".github/ISSUE_TEMPLATE/**", ".github/ISSUE_TEMPLATE.md" - ) - results.append(self._make_result( - "issue_templates", pillar, 2, issue_templates, - "Issue templates configured" if issue_templates else "No issue templates" - )) - - # L2: issue_labeling_system - labels = False - if issue_templates: - templates = list(self.repo_path.glob(".github/ISSUE_TEMPLATE/*.md")) - for t in templates[:5]: - content = t.read_text(errors='ignore') - if "labels:" in content.lower(): - labels = True - break - results.append(self._make_result( - "issue_labeling_system", pillar, 2, labels, - "Issue labels configured" if labels else "No issue labeling system" - )) - - # L2: pr_templates - pr_template = self._file_exists( - ".github/pull_request_template.md", - ".github/PULL_REQUEST_TEMPLATE.md", - "pull_request_template.md" - ) - results.append(self._make_result( - "pr_templates", pillar, 2, pr_template, - "PR template configured" if pr_template else "No PR template" - )) - - # L3: backlog_health - backlog = self._file_exists("CONTRIBUTING.md", ".github/CONTRIBUTING.md") - results.append(self._make_result( - "backlog_health", pillar, 3, backlog, - "Contributing guidelines exist" if backlog else "No contributing guidelines" - )) - - return results - - def _evaluate_product_analytics(self) -> list[CriterionResult]: - """Evaluate Product & Analytics pillar.""" - pillar = "Product & Analytics" - results = [] - - # L5: error_to_insight_pipeline - error_pipeline = False - workflows = list(self.repo_path.glob(".github/workflows/*.yml")) - for wf in workflows[:5]: - content = wf.read_text(errors='ignore') - if any(x in content.lower() for x in ["sentry", "create.*issue", "error.*issue"]): - error_pipeline = True - break - # Also check for Sentry-GitHub integration - deps = (self._read_file("package.json") or "") + \ - (self._read_file("requirements.txt") or "") + \ - (self._read_file("go.mod") or "") - if "sentry" in deps.lower(): - # Check for issue creation automation - for wf in workflows[:5]: - content = wf.read_text(errors='ignore') - if "issue" in content.lower() and "sentry" in content.lower(): - error_pipeline = True - break - results.append(self._make_result( - "error_to_insight_pipeline", pillar, 5, error_pipeline, - "Error-to-issue pipeline exists" if error_pipeline else "No error-to-issue pipeline" - )) - - # L5: product_analytics_instrumentation - analytics = False - if any(x in deps.lower() for x in [ - "mixpanel", "amplitude", "posthog", "heap", "segment", "ga4", "google-analytics" - ]): - analytics = True - results.append(self._make_result( - "product_analytics_instrumentation", pillar, 5, analytics, - "Product analytics configured" if analytics else "No product analytics" - )) - - return results - - def _calculate_levels(self): - """Calculate maturity level based on criteria pass rates.""" - level_criteria: dict[int, list[CriterionResult]] = {i: [] for i in range(1, 6)} - - for pillar in self.result.pillars.values(): - for criterion in pillar.criteria: - if criterion.status != CriterionStatus.SKIP: - level_criteria[criterion.level].append(criterion) - - for level in range(1, 6): - criteria = level_criteria[level] - if not criteria: - self.result.level_scores[level] = 100.0 - continue - - passed = sum(1 for c in criteria if c.status == CriterionStatus.PASS) - self.result.level_scores[level] = round((passed / len(criteria)) * 100, 1) - - achieved = 0 - for level in range(1, 6): - if self.result.level_scores[level] >= 80: - achieved = level - else: - break - - # Only set achieved level if at least L1 is passed - self.result.achieved_level = achieved if achieved > 0 else 0 - - -def main(): - parser = argparse.ArgumentParser( - description="Analyze repository for agent readiness" - ) - parser.add_argument( - "--repo-path", "-r", - default=".", - help="Path to the repository to analyze" - ) - parser.add_argument( - "--output", "-o", - default="/tmp/readiness_analysis.json", - help="Output file for analysis results" - ) - parser.add_argument( - "--quiet", "-q", - action="store_true", - help="Suppress progress output" - ) - - args = parser.parse_args() - - if not args.quiet: - print(f"🔍 Analyzing repository: {args.repo_path}") - - analyzer = RepoAnalyzer(args.repo_path) - result = analyzer.analyze() - - output = { - "repo_path": result.repo_path, - "repo_name": result.repo_name, - "repo_type": result.repo_type, - "languages": result.languages, - "pass_rate": result.pass_rate, - "total_passed": result.total_passed, - "total_criteria": result.total_criteria, - "achieved_level": result.achieved_level, - "level_scores": result.level_scores, - "pillars": {} - } - - for pillar_name, pillar in result.pillars.items(): - output["pillars"][pillar_name] = { - "name": pillar.name, - "passed": pillar.passed, - "total": pillar.total, - "percentage": pillar.percentage, - "criteria": [asdict(c) for c in pillar.criteria] - } - - output_path = Path(args.output) - output_path.parent.mkdir(parents=True, exist_ok=True) - output_path.write_text(json.dumps(output, indent=2)) - - if not args.quiet: - print(f"✅ Analysis complete: {result.total_passed}/{result.total_criteria} criteria passed ({result.pass_rate}%)") - print(f"📊 Achieved Level: L{result.achieved_level}") - print(f"📄 Results saved to: {args.output}") - - return result - - -if __name__ == "__main__": - main() diff --git a/skills/readiness-report/scripts/generate_report.py b/skills/readiness-report/scripts/generate_report.py deleted file mode 100755 index 8f7c966..0000000 --- a/skills/readiness-report/scripts/generate_report.py +++ /dev/null @@ -1,297 +0,0 @@ -#!/usr/bin/env python3 -""" -Report Generator for Agent Readiness - -Generates formatted markdown reports from analysis JSON. - -Usage: - python generate_report.py --analysis-file /tmp/readiness_analysis.json - python generate_report.py --analysis-file /tmp/readiness_analysis.json --format markdown -""" - -import argparse -import json -from pathlib import Path - - -def format_level_bar(level_scores: dict, achieved: int) -> str: - """Generate a visual level progress bar.""" - bars = [] - for level in range(1, 6): - score = level_scores.get(str(level), level_scores.get(level, 0)) - if level <= achieved: - indicator = "█" * 4 - status = f"L{level} {score:.0f}%" - else: - indicator = "░" * 4 - status = f"L{level} {score:.0f}%" - bars.append(f"{indicator} {status}") - return " | ".join(bars) - - -def format_criterion_row(criterion: dict) -> str: - """Format a single criterion as a table row.""" - status = criterion["status"] - crit_id = criterion["id"] - score = criterion["score"] - reason = criterion["reason"] - - if status == "pass": - icon = "✓" - elif status == "fail": - icon = "✗" - else: # skip - icon = "—" - - return f"{icon} `{crit_id}` | {score} | {reason}" - - -def get_top_strengths(data: dict, n: int = 3) -> list[tuple[str, int, list[str]]]: - """Get top performing pillars with example passing criteria.""" - pillar_scores = [] - for pillar_name, pillar in data["pillars"].items(): - if pillar["total"] > 0: - pct = pillar["percentage"] - passing = [c["id"] for c in pillar["criteria"] if c["status"] == "pass"][:3] - pillar_scores.append((pillar_name, pct, passing)) - - # Sort by percentage descending - pillar_scores.sort(key=lambda x: x[1], reverse=True) - return pillar_scores[:n] - - -def get_top_opportunities(data: dict, n: int = 5) -> list[tuple[str, str, str]]: - """Get highest priority improvement opportunities.""" - opportunities = [] - - # Prioritize by level (lower levels first), then by pillar importance - for pillar_name, pillar in data["pillars"].items(): - for criterion in pillar["criteria"]: - if criterion["status"] == "fail": - opportunities.append(( - criterion["id"], - criterion["level"], - criterion["reason"], - pillar_name - )) - - # Sort by level (ascending) to prioritize foundational issues - opportunities.sort(key=lambda x: x[1]) - return [(o[0], o[2], o[3]) for o in opportunities[:n]] - - -def generate_markdown_report(data: dict) -> str: - """Generate a full markdown report from analysis data.""" - repo_name = data["repo_name"] - pass_rate = data["pass_rate"] - achieved = data["achieved_level"] - total_passed = data["total_passed"] - total = data["total_criteria"] - languages = data.get("languages", ["Unknown"]) - repo_type = data.get("repo_type", "application") - level_scores = data["level_scores"] - - lines = [] - - # Header - lines.append(f"# Agent Readiness Report: {repo_name}") - lines.append("") - lines.append(f"**Languages**: {', '.join(languages)} ") - lines.append(f"**Repository Type**: {repo_type} ") - lines.append(f"**Pass Rate**: {pass_rate}% ({total_passed}/{total} criteria) ") - if achieved > 0: - lines.append(f"**Achieved Level**: **L{achieved}**") - else: - lines.append(f"**Achieved Level**: **Not yet L1** (need 80% at L1)") - lines.append("") - - # Level Progress - lines.append("## Level Progress") - lines.append("") - lines.append("| Level | Score | Status |") - lines.append("|-------|-------|--------|") - for level in range(1, 6): - score = level_scores.get(str(level), level_scores.get(level, 0)) - if achieved > 0 and level <= achieved: - status = "✅ Achieved" - elif score >= 80: - status = "✅ Passed" - else: - status = f"⬜ {100-score:.0f}% to go" - lines.append(f"| L{level} | {score:.0f}% | {status} |") - lines.append("") - - # Summary - lines.append("## Summary") - lines.append("") - - # Strengths - strengths = get_top_strengths(data) - if strengths: - lines.append("### Strengths") - lines.append("") - for pillar_name, pct, passing in strengths: - if passing: - passing_str = ", ".join(f"`{p}`" for p in passing) - lines.append(f"- **{pillar_name}** ({pct}%): {passing_str}") - else: - lines.append(f"- **{pillar_name}** ({pct}%)") - lines.append("") - - # Opportunities - opportunities = get_top_opportunities(data) - if opportunities: - lines.append("### Priority Improvements") - lines.append("") - lines.append("| Criterion | Issue | Pillar |") - lines.append("|-----------|-------|--------|") - for crit_id, reason, pillar in opportunities: - lines.append(f"| `{crit_id}` | {reason} | {pillar} |") - lines.append("") - - # Detailed Results - lines.append("## Detailed Results") - lines.append("") - - for pillar_name, pillar in data["pillars"].items(): - pct = pillar["percentage"] - passed = pillar["passed"] - total = pillar["total"] - - lines.append(f"### {pillar_name}") - lines.append(f"**Score**: {passed}/{total} ({pct}%)") - lines.append("") - lines.append("| Status | Criterion | Score | Details |") - lines.append("|--------|-----------|-------|---------|") - - for criterion in pillar["criteria"]: - status = criterion["status"] - if status == "pass": - icon = "✓" - elif status == "fail": - icon = "✗" - else: - icon = "—" - - crit_id = criterion["id"] - score = criterion["score"] - reason = criterion["reason"] - lines.append(f"| {icon} | `{crit_id}` | {score} | {reason} |") - - lines.append("") - - # Recommendations - lines.append("## Recommended Next Steps") - lines.append("") - - if achieved < 2: - lines.append("**Focus on L1/L2 Foundations:**") - lines.append("1. Add missing linter and formatter configurations") - lines.append("2. Document build and test commands in README") - lines.append("3. Set up pre-commit hooks for fast feedback") - lines.append("4. Create AGENTS.md with project context for AI agents") - elif achieved < 3: - lines.append("**Progress to L3 (Production Ready):**") - lines.append("1. Add integration/E2E tests") - lines.append("2. Set up test coverage thresholds") - lines.append("3. Configure devcontainer for reproducible environments") - lines.append("4. Add automated PR review tooling") - else: - lines.append("**Optimize for L4+:**") - lines.append("1. Implement complexity analysis and dead code detection") - lines.append("2. Set up flaky test detection and quarantine") - lines.append("3. Add security scanning (CodeQL, Snyk)") - lines.append("4. Configure deployment observability") - - lines.append("") - lines.append("---") - lines.append(f"*Report generated from repository analysis*") - - return "\n".join(lines) - - -def generate_brief_report(data: dict) -> str: - """Generate a brief summary report.""" - repo_name = data["repo_name"] - pass_rate = data["pass_rate"] - achieved = data["achieved_level"] - total_passed = data["total_passed"] - total = data["total_criteria"] - - lines = [] - lines.append(f"## Agent Readiness: {repo_name}") - lines.append("") - level_str = f"Level {achieved}" if achieved > 0 else "Not yet L1" - lines.append(f"**{level_str}** | {pass_rate}% ({total_passed}/{total})") - lines.append("") - - # Quick level summary - for level in range(1, 6): - score = data["level_scores"].get(str(level), data["level_scores"].get(level, 0)) - bar = "█" * int(score / 10) + "░" * (10 - int(score / 10)) - check = "✅" if achieved > 0 and level <= achieved else "⬜" - lines.append(f"L{level} {check} [{bar}] {score:.0f}%") - - lines.append("") - - # Top opportunities - opps = get_top_opportunities(data, 3) - if opps: - lines.append("**Quick Wins:**") - for crit_id, reason, _ in opps: - lines.append(f"- {crit_id}: {reason}") - - return "\n".join(lines) - - -def main(): - parser = argparse.ArgumentParser( - description="Generate Agent Readiness report from analysis" - ) - parser.add_argument( - "--analysis-file", "-a", - default="/tmp/readiness_analysis.json", - help="Path to analysis JSON file" - ) - parser.add_argument( - "--output", "-o", - help="Output file (default: stdout)" - ) - parser.add_argument( - "--format", "-f", - choices=["markdown", "brief", "json"], - default="markdown", - help="Output format" - ) - - args = parser.parse_args() - - # Load analysis - analysis_path = Path(args.analysis_file) - if not analysis_path.exists(): - print(f"❌ Analysis file not found: {args.analysis_file}") - print("Run analyze_repo.py first to generate the analysis.") - return 1 - - data = json.loads(analysis_path.read_text()) - - # Generate report - if args.format == "markdown": - report = generate_markdown_report(data) - elif args.format == "brief": - report = generate_brief_report(data) - else: # json - report = json.dumps(data, indent=2) - - # Output - if args.output: - Path(args.output).write_text(report) - print(f"✅ Report saved to: {args.output}") - else: - print(report) - - return 0 - - -if __name__ == "__main__": - exit(main()) From a40967558e5aa94c6a9aa376a1be60f9ee12a786 Mon Sep 17 00:00:00 2001 From: Calvin Smithu Date: Mon, 23 Mar 2026 07:54:38 -0600 Subject: [PATCH 06/15] moving shared refs --- plugins/onboarding/README.md | 12 +- plugins/onboarding/references/criteria.md | 144 +++++++++++++++++ .../agent-onboarding/references/criteria.md | 145 +----------------- .../skills/agent-readiness/README.md | 2 +- .../agent-readiness/references/criteria.md | 145 +----------------- 5 files changed, 150 insertions(+), 298 deletions(-) create mode 100644 plugins/onboarding/references/criteria.md mode change 100644 => 120000 plugins/onboarding/skills/agent-onboarding/references/criteria.md mode change 100644 => 120000 plugins/onboarding/skills/agent-readiness/references/criteria.md diff --git a/plugins/onboarding/README.md b/plugins/onboarding/README.md index f64c597..3ab911f 100644 --- a/plugins/onboarding/README.md +++ b/plugins/onboarding/README.md @@ -31,12 +31,6 @@ Run the readiness report on any repository to get a scored evaluation across fiv | [agent-onboarding](skills/agent-onboarding/) | `/agent-onboarding` | Propose and implement fixes from a readiness report | | [agents-md](skills/agents-md/) | `/agents-md` | Generate a repo-specific AGENTS.md | -## Five Pillars - -| Pillar | Question | Features | -|--------|----------|----------| -| **Agent Instructions** | Does the agent know what to do? | 18 | -| **Feedback Loops** | Does the agent know if it's right? | 16 | -| **Workflows & Automation** | Does the process support agent work? | 15 | -| **Policy & Governance** | Does the agent know the rules? | 13 | -| **Build & Dev Environment** | Can the agent build and run the project? | 12 | \ No newline at end of file +## Shared References + +The `agent-readiness` and `agent-onboarding` skills both use the same 74-feature evaluation criteria. The canonical file lives at `references/criteria.md` at the plugin root, and each skill symlinks to it from its own `references/` directory. Edit the plugin-level copy — the symlinks keep both skills in sync automatically. diff --git a/plugins/onboarding/references/criteria.md b/plugins/onboarding/references/criteria.md new file mode 100644 index 0000000..1fe2788 --- /dev/null +++ b/plugins/onboarding/references/criteria.md @@ -0,0 +1,144 @@ +# Agent Readiness Criteria + +Features that make a codebase ready for AI-assisted development, organized into +five pillars. Derived from cluster analysis of 123 repositories, but written to +be tool-agnostic. Every feature answers one question: *if this is missing, what +goes wrong for the agent?* + +--- + +## Pillar 1 · Agent Instructions + +How the repo tells AI agents what to do, what to avoid, and how the codebase +works. This is the highest-signal pillar — it's the difference between an agent +that understands the project and one that's guessing. + +| # | Feature | What to look for | Evidence | +|---|---------|------------------|----------| +| 1 | **Agent instruction file** | A dedicated file telling agents how to work in this repo — conventions, banned patterns, common commands | `AGENTS.md`, `CLAUDE.md`, `COPILOT.md`, `CONVENTIONS.md` at root | +| 2 | **AI IDE configuration** | Settings or rules for AI-powered editors/IDEs | `.cursor/rules/`, `.cursorrules`, `.github/copilot-instructions.md`, `.github/instructions/`, `.claude/settings.json` | +| 3 | **Multi-model support** | Instructions that work across different AI models/tools, not locked to one vendor | 2+ distinct agent config types from features 1–2 present in same repo | +| 4 | **Agent skills or capabilities** | Packaged, reusable abilities the agent can invoke | `.claude/skills/`, `.factory/skills/`, `skill.md` files, tool definition files | +| 5 | **Tool server configuration** | Config for agent tool protocols (lets agents use external tools) | `.mcp.json`, `mcp.config.js`, tool server manifests | +| 6 | **Agent prompt library** | Pre-built prompts for common tasks in this repo | `.github/prompts/`, `prompts/` directory, prompt template files | +| 7 | **Component-level agent guidance** | Different parts of the codebase have their own agent instructions | `AGENTS.md` or instruction files in subdirectories (e.g. `frontend/AGENTS.md`, `api/CLAUDE.md`) | +| 8 | **README with build/run/test** | README includes the commands to build, run, and test the project | `README.md` containing code blocks with build/install/test commands | +| 9 | **Contributing guide** | How to contribute — code style, PR process, commit conventions | `CONTRIBUTING.md`, `docs/contributing.md`, contributing section in README | +| 10 | **Architecture documentation** | High-level overview of how the system is structured and why | `ARCHITECTURE.md`, `docs/architecture/`, Mermaid/PlantUML diagrams, `doc/design/` | +| 11 | **API documentation** | Reference docs for the project's interfaces | `openapi.yaml`, generated HTML docs, `doc.go` files, Swagger UI, `api-docs/` | +| 12 | **Inline code documentation** | Doc comments, docstrings — agents read these to understand intent | JSDoc `/** */` blocks, Python docstrings, GoDoc comments, RDoc `#` blocks, Rust `///` | +| 13 | **Runnable examples** | Working example code the agent can study and imitate | `examples/` directory, `_examples/`, example apps with their own READMEs | +| 14 | **Changelog** | History of what changed and how entries should be written | `CHANGELOG.md`, `CHANGES.md`, `HISTORY.md`, release notes in GitHub Releases | +| 15 | **Environment variable documentation** | Template or docs for required env vars | `.env.example`, `.env.template`, env var table in README or docs | +| 16 | **Documentation site or directory** | Organized docs beyond the README | `docs/` directory, Docusaurus/Sphinx/MkDocs/VitePress config, published doc site | +| 17 | **Decision records** | Documented reasoning behind past architectural choices | `doc/adr/`, `decisions/`, `rfcs/`, numbered markdown decision files | +| 18 | **Module-level READMEs** | Individual packages/modules have their own READMEs | `packages/*/README.md`, `libs/*/README.md`, per-crate/per-module READMEs | + +--- + +## Pillar 2 · Feedback Loops + +How quickly and clearly the agent learns whether its changes are correct. Fast, +clear feedback is the difference between an agent that converges on a solution +and one that spirals. + +| # | Feature | What to look for | Evidence | +|---|---------|------------------|----------| +| 19 | **Linter** | Static analysis that catches bugs and style issues | `.eslintrc.*`, `ruff.toml`, `.golangci.yml`, `clippy.toml`, `pylintrc`, lint config in `pyproject.toml` | +| 20 | **Formatter** | Auto-formatter that enforces consistent style | `.prettierrc`, `rustfmt.toml`, `[tool.black]` in pyproject.toml, `gofmt`/`goimports` in CI | +| 21 | **Type checking** | Static type system or type checker | `tsconfig.json` with `strict`, `mypy.ini`, `[tool.mypy]`, `py.typed` marker, Go (inherent) | +| 22 | **Pre-commit hooks** | Checks that run before commit — catches issues before CI | `.pre-commit-config.yaml`, `.husky/`, `lefthook.yml`, `lint-staged` config | +| 23 | **Unit tests** | Tests for individual components | `*_test.go`, `*_test.py`, `*.spec.ts`, `test/`, `__tests__/`, test runner config | +| 24 | **Integration tests** | Tests that verify components work together | `test/integration/`, `tests/e2e/`, API test suites, test files with service dependencies | +| 25 | **End-to-end tests** | Full system/browser tests | Playwright config, Cypress config, Selenium tests, `e2e/` directory | +| 26 | **Test coverage measurement** | Coverage tracking so the agent knows if new code is tested | `.codecov.yml`, `coverageThreshold` in jest config, `--cov` in pytest, `cover` profile in Go | +| 27 | **CI pipeline** | Automated checks on every push or PR | `.github/workflows/ci.yml`, `.circleci/config.yml`, `.gitlab-ci.yml`, `Jenkinsfile` | +| 28 | **Fast CI feedback** | CI completes quickly enough for agent iteration | CI workflow with documented expected duration, parallel jobs, test splitting | +| 29 | **Test run documentation** | Agent knows exactly how to run which tests | Test commands in README, AGENTS.md, CONTRIBUTING.md, or Makefile help target | +| 30 | **Config/schema validation** | YAML, JSON, config files validated automatically | `yamllint` config, JSON schema `$schema` refs, `actionlint` in CI, `taplo` for TOML | +| 31 | **Snapshot or golden-file tests** | Tests that detect unexpected output changes | `__snapshots__/` directories, `.snap` files, `testdata/` golden files, VCR cassettes | +| 32 | **Benchmark suite** | Performance tests the agent can run to check for regressions | `bench/`, `benchmarks/`, `*_bench_test.go`, pytest-benchmark, Criterion.rs | +| 33 | **Warnings-as-errors** | Compiler/runtime warnings treated as failures | `-Werror`, `warningsAsErrors` in build config, `filterwarnings = error` in pytest | +| 34 | **Spell/typo checking** | Automated spelling checks in CI or hooks | `.cspell.json`, `codespell` in pre-commit, `typos.toml`, spell check CI step | + +--- + +## Pillar 3 · Workflows & Automation + +The processes that support agent-driven development — how work is structured, +tracked, and shipped. + +| # | Feature | What to look for | Evidence | +|---|---------|------------------|----------| +| 35 | **Issue templates** | Structured templates for bug reports, feature requests | `.github/ISSUE_TEMPLATE/` with `bug_report.md`, `feature_request.md`, `config.yml` | +| 36 | **PR template** | Template that guides PR descriptions | `.github/pull_request_template.md`, PR template with checklist items | +| 37 | **Dependency update automation** | Automated PRs for dependency updates | `.github/dependabot.yml`, `renovate.json`, `.renovaterc`, Renovate preset configs | +| 38 | **Release automation** | Automated release pipeline (build, tag, publish) | Release workflow in CI, `semantic-release` config, GoReleaser config, `release-please` | +| 39 | **Branch protection** | Protected main branch with required checks | Branch protection rules (inferred from merge queue config, required status checks in CI) | +| 40 | **Merge automation** | Merge queue, auto-merge, or merge bot | `merge_group` trigger in CI, Mergify config, auto-merge labels, `gh pr merge --auto` | +| 41 | **Task runner** | Single entry point for common commands | `Makefile`, `Justfile`, `Taskfile.yml`, `package.json` scripts section, `Rakefile` | +| 42 | **Structured change tracking** | Changesets, conventional commits, or similar discipline | `.changeset/` directory, `commitlint` config, conventional commit enforcement in CI | +| 43 | **CI concurrency control** | Cancel-in-progress, concurrency groups to avoid CI pile-up | `concurrency:` blocks in GitHub Actions, `cancel-in-progress: true` | +| 44 | **Automated release notes** | Changelog/release notes generated from commits or PRs | `release-please` config, `auto-changelog`, `git-cliff`, `conventional-changelog` | +| 45 | **Stale issue/PR management** | Automation to close or label stale items | `.github/workflows/stale.yml`, `stale` bot config, issue lifecycle labels | +| 46 | **Label automation** | Automatic PR/issue labeling based on paths or content | `.github/labeler.yml`, label-sync config, auto-label workflows | +| 47 | **Multi-platform CI** | CI matrix covering multiple OS, arch, or runtime versions | `matrix:` in CI with `os: [ubuntu, macos, windows]` or multiple language versions | +| 48 | **Deployment automation** | Automated deployment pipeline | Deploy workflow triggered on merge/tag, staging + production environments in CI | +| 49 | **Automated code review checks** | Bot-driven review checks beyond CI | Danger.js config, review bot config, required review assignments, CODEOWNERS + required reviews | + +--- + +## Pillar 4 · Policy & Governance + +Rules, ownership, and constraints the agent must know about and respect. + +| # | Feature | What to look for | Evidence | +|---|---------|------------------|----------| +| 50 | **Comprehensive .gitignore** | Covers secrets, build artifacts, IDE files, agent artifacts | `.gitignore` with entries for `.env`, `node_modules/`, build dirs, `.cursor/`, `.claude/` | +| 51 | **License** | Clear license at root | `LICENSE`, `MIT-LICENSE`, `COPYING`, `LICENSE.md` | +| 52 | **Code ownership** | File/directory ownership mapping | `CODEOWNERS`, `.github/CODEOWNERS`, docs describing area owners or maintainer teams | +| 53 | **Security policy** | How to report vulnerabilities | `SECURITY.md`, `.github/security.md`, security reporting instructions | +| 54 | **Code of conduct** | Community standards | `CODE_OF_CONDUCT.md`, conduct link in contributing guide | +| 55 | **AI usage policy** | Documented guidelines for AI/agent contributions | AI policy in AGENTS.md, CONTRIBUTING.md, or standalone doc; agent boundary definitions | +| 56 | **Secrets management** | Secrets handled via environment/vault, not hardcoded | References to `${{ secrets.* }}` in CI, vault config, `.env.example` without values, `git-secrets` | +| 57 | **Security scanning** | Automated vulnerability scanning in CI | `.github/workflows/codeql.yml`, Snyk config, `gosec` in CI, Trivy, Dependabot security alerts | +| 58 | **Git attributes** | Line endings, diff drivers, LFS, linguist overrides | `.gitattributes` with `text=auto`, `linguist-generated`, LFS tracking patterns | +| 59 | **Contributor agreement** | DCO sign-off or CLA process | DCO bot config, `Signed-off-by` requirement in contributing guide, CLA-assistant config | +| 60 | **Governance model** | Documented maintainer roles, decision-making process | `GOVERNANCE.md`, `MAINTAINERS.md`, governance section in docs, team/role descriptions | +| 61 | **CI workflow validation** | CI config itself is linted/validated | `actionlint` step in CI, `circleci config validate`, CI config schema validation | +| 62 | **Environment separation** | Distinct configs for dev/test/prod | `.env.test`, `.env.production`, environment-specific config directories, config per deploy target | + +--- + +## Pillar 5 · Build & Dev Environment + +Can the agent actually build, run, and iterate on the project? Reproducibility +and speed matter — an agent that can't build the project can't do anything. + +| # | Feature | What to look for | Evidence | +|---|---------|------------------|----------| +| 63 | **Dependency lockfile** | Pinned dependency versions for reproducible installs | `package-lock.json`, `yarn.lock`, `pnpm-lock.yaml`, `uv.lock`, `Cargo.lock`, `go.sum`, `Gemfile.lock`, `poetry.lock` | +| 64 | **Single-command build** | One documented command to build the entire project | `make build`, `npm run build`, `cargo build` documented in README or agent file | +| 65 | **Single-command dev setup** | One command to bootstrap a working dev environment | `bin/setup`, `make dev`, `scripts/bootstrap.sh`, `just setup` | +| 66 | **Dev container** | Containerized dev environment definition | `.devcontainer/devcontainer.json`, `devcontainer.json` at root | +| 67 | **Containerized services** | Docker-based local development stack | `Dockerfile`, `docker-compose.yml`, `compose.yaml` with dev services | +| 68 | **Reproducible environment** | Declarative, reproducible dev environment | `flake.nix`, `shell.nix`, `devbox.json`, hermetic build definitions | +| 69 | **Tool version pinning** | Runtime/tool versions pinned to a file | `.tool-versions`, `mise.toml`, `.node-version`, `.python-version`, `.ruby-version`, `rust-toolchain.toml` | +| 70 | **Monorepo orchestration** | Tooling for multi-package repositories | Workspace config in `package.json`, `pnpm-workspace.yaml`, Cargo workspace, Go multi-module, Nx/Turborepo/Bazel config | +| 71 | **Build caching** | Caching for faster rebuilds | CI cache steps (`actions/cache`), Turborepo remote cache, ccache/sccache config, layer caching in Docker | +| 72 | **Cross-platform support** | Builds on multiple OS/arch | CI matrix with multiple OS entries, cross-compilation configs, multi-arch Docker builds | +| 73 | **Cloud dev environment** | Cloud-based workspace configuration | `.devcontainer/` with Codespaces features, `.gitpod.yml`, cloud workspace config | +| 74 | **Package manager configuration** | Custom registry, auth, resolution settings | `.npmrc`, `pip.conf`, `.cargo/config.toml`, registry overrides, resolution overrides | + +--- + +## Summary + +| Pillar | Features | What it answers | +|--------|----------|-----------------| +| Agent Instructions | 18 | Does the agent know what to do? | +| Feedback Loops | 16 | Does the agent know if it's right? | +| Workflows & Automation | 15 | Does the process support agent work? | +| Policy & Governance | 13 | Does the agent know the rules? | +| Build & Dev Environment | 12 | Can the agent build and run the project? | +| **Total** | **74** | | diff --git a/plugins/onboarding/skills/agent-onboarding/references/criteria.md b/plugins/onboarding/skills/agent-onboarding/references/criteria.md deleted file mode 100644 index 1fe2788..0000000 --- a/plugins/onboarding/skills/agent-onboarding/references/criteria.md +++ /dev/null @@ -1,144 +0,0 @@ -# Agent Readiness Criteria - -Features that make a codebase ready for AI-assisted development, organized into -five pillars. Derived from cluster analysis of 123 repositories, but written to -be tool-agnostic. Every feature answers one question: *if this is missing, what -goes wrong for the agent?* - ---- - -## Pillar 1 · Agent Instructions - -How the repo tells AI agents what to do, what to avoid, and how the codebase -works. This is the highest-signal pillar — it's the difference between an agent -that understands the project and one that's guessing. - -| # | Feature | What to look for | Evidence | -|---|---------|------------------|----------| -| 1 | **Agent instruction file** | A dedicated file telling agents how to work in this repo — conventions, banned patterns, common commands | `AGENTS.md`, `CLAUDE.md`, `COPILOT.md`, `CONVENTIONS.md` at root | -| 2 | **AI IDE configuration** | Settings or rules for AI-powered editors/IDEs | `.cursor/rules/`, `.cursorrules`, `.github/copilot-instructions.md`, `.github/instructions/`, `.claude/settings.json` | -| 3 | **Multi-model support** | Instructions that work across different AI models/tools, not locked to one vendor | 2+ distinct agent config types from features 1–2 present in same repo | -| 4 | **Agent skills or capabilities** | Packaged, reusable abilities the agent can invoke | `.claude/skills/`, `.factory/skills/`, `skill.md` files, tool definition files | -| 5 | **Tool server configuration** | Config for agent tool protocols (lets agents use external tools) | `.mcp.json`, `mcp.config.js`, tool server manifests | -| 6 | **Agent prompt library** | Pre-built prompts for common tasks in this repo | `.github/prompts/`, `prompts/` directory, prompt template files | -| 7 | **Component-level agent guidance** | Different parts of the codebase have their own agent instructions | `AGENTS.md` or instruction files in subdirectories (e.g. `frontend/AGENTS.md`, `api/CLAUDE.md`) | -| 8 | **README with build/run/test** | README includes the commands to build, run, and test the project | `README.md` containing code blocks with build/install/test commands | -| 9 | **Contributing guide** | How to contribute — code style, PR process, commit conventions | `CONTRIBUTING.md`, `docs/contributing.md`, contributing section in README | -| 10 | **Architecture documentation** | High-level overview of how the system is structured and why | `ARCHITECTURE.md`, `docs/architecture/`, Mermaid/PlantUML diagrams, `doc/design/` | -| 11 | **API documentation** | Reference docs for the project's interfaces | `openapi.yaml`, generated HTML docs, `doc.go` files, Swagger UI, `api-docs/` | -| 12 | **Inline code documentation** | Doc comments, docstrings — agents read these to understand intent | JSDoc `/** */` blocks, Python docstrings, GoDoc comments, RDoc `#` blocks, Rust `///` | -| 13 | **Runnable examples** | Working example code the agent can study and imitate | `examples/` directory, `_examples/`, example apps with their own READMEs | -| 14 | **Changelog** | History of what changed and how entries should be written | `CHANGELOG.md`, `CHANGES.md`, `HISTORY.md`, release notes in GitHub Releases | -| 15 | **Environment variable documentation** | Template or docs for required env vars | `.env.example`, `.env.template`, env var table in README or docs | -| 16 | **Documentation site or directory** | Organized docs beyond the README | `docs/` directory, Docusaurus/Sphinx/MkDocs/VitePress config, published doc site | -| 17 | **Decision records** | Documented reasoning behind past architectural choices | `doc/adr/`, `decisions/`, `rfcs/`, numbered markdown decision files | -| 18 | **Module-level READMEs** | Individual packages/modules have their own READMEs | `packages/*/README.md`, `libs/*/README.md`, per-crate/per-module READMEs | - ---- - -## Pillar 2 · Feedback Loops - -How quickly and clearly the agent learns whether its changes are correct. Fast, -clear feedback is the difference between an agent that converges on a solution -and one that spirals. - -| # | Feature | What to look for | Evidence | -|---|---------|------------------|----------| -| 19 | **Linter** | Static analysis that catches bugs and style issues | `.eslintrc.*`, `ruff.toml`, `.golangci.yml`, `clippy.toml`, `pylintrc`, lint config in `pyproject.toml` | -| 20 | **Formatter** | Auto-formatter that enforces consistent style | `.prettierrc`, `rustfmt.toml`, `[tool.black]` in pyproject.toml, `gofmt`/`goimports` in CI | -| 21 | **Type checking** | Static type system or type checker | `tsconfig.json` with `strict`, `mypy.ini`, `[tool.mypy]`, `py.typed` marker, Go (inherent) | -| 22 | **Pre-commit hooks** | Checks that run before commit — catches issues before CI | `.pre-commit-config.yaml`, `.husky/`, `lefthook.yml`, `lint-staged` config | -| 23 | **Unit tests** | Tests for individual components | `*_test.go`, `*_test.py`, `*.spec.ts`, `test/`, `__tests__/`, test runner config | -| 24 | **Integration tests** | Tests that verify components work together | `test/integration/`, `tests/e2e/`, API test suites, test files with service dependencies | -| 25 | **End-to-end tests** | Full system/browser tests | Playwright config, Cypress config, Selenium tests, `e2e/` directory | -| 26 | **Test coverage measurement** | Coverage tracking so the agent knows if new code is tested | `.codecov.yml`, `coverageThreshold` in jest config, `--cov` in pytest, `cover` profile in Go | -| 27 | **CI pipeline** | Automated checks on every push or PR | `.github/workflows/ci.yml`, `.circleci/config.yml`, `.gitlab-ci.yml`, `Jenkinsfile` | -| 28 | **Fast CI feedback** | CI completes quickly enough for agent iteration | CI workflow with documented expected duration, parallel jobs, test splitting | -| 29 | **Test run documentation** | Agent knows exactly how to run which tests | Test commands in README, AGENTS.md, CONTRIBUTING.md, or Makefile help target | -| 30 | **Config/schema validation** | YAML, JSON, config files validated automatically | `yamllint` config, JSON schema `$schema` refs, `actionlint` in CI, `taplo` for TOML | -| 31 | **Snapshot or golden-file tests** | Tests that detect unexpected output changes | `__snapshots__/` directories, `.snap` files, `testdata/` golden files, VCR cassettes | -| 32 | **Benchmark suite** | Performance tests the agent can run to check for regressions | `bench/`, `benchmarks/`, `*_bench_test.go`, pytest-benchmark, Criterion.rs | -| 33 | **Warnings-as-errors** | Compiler/runtime warnings treated as failures | `-Werror`, `warningsAsErrors` in build config, `filterwarnings = error` in pytest | -| 34 | **Spell/typo checking** | Automated spelling checks in CI or hooks | `.cspell.json`, `codespell` in pre-commit, `typos.toml`, spell check CI step | - ---- - -## Pillar 3 · Workflows & Automation - -The processes that support agent-driven development — how work is structured, -tracked, and shipped. - -| # | Feature | What to look for | Evidence | -|---|---------|------------------|----------| -| 35 | **Issue templates** | Structured templates for bug reports, feature requests | `.github/ISSUE_TEMPLATE/` with `bug_report.md`, `feature_request.md`, `config.yml` | -| 36 | **PR template** | Template that guides PR descriptions | `.github/pull_request_template.md`, PR template with checklist items | -| 37 | **Dependency update automation** | Automated PRs for dependency updates | `.github/dependabot.yml`, `renovate.json`, `.renovaterc`, Renovate preset configs | -| 38 | **Release automation** | Automated release pipeline (build, tag, publish) | Release workflow in CI, `semantic-release` config, GoReleaser config, `release-please` | -| 39 | **Branch protection** | Protected main branch with required checks | Branch protection rules (inferred from merge queue config, required status checks in CI) | -| 40 | **Merge automation** | Merge queue, auto-merge, or merge bot | `merge_group` trigger in CI, Mergify config, auto-merge labels, `gh pr merge --auto` | -| 41 | **Task runner** | Single entry point for common commands | `Makefile`, `Justfile`, `Taskfile.yml`, `package.json` scripts section, `Rakefile` | -| 42 | **Structured change tracking** | Changesets, conventional commits, or similar discipline | `.changeset/` directory, `commitlint` config, conventional commit enforcement in CI | -| 43 | **CI concurrency control** | Cancel-in-progress, concurrency groups to avoid CI pile-up | `concurrency:` blocks in GitHub Actions, `cancel-in-progress: true` | -| 44 | **Automated release notes** | Changelog/release notes generated from commits or PRs | `release-please` config, `auto-changelog`, `git-cliff`, `conventional-changelog` | -| 45 | **Stale issue/PR management** | Automation to close or label stale items | `.github/workflows/stale.yml`, `stale` bot config, issue lifecycle labels | -| 46 | **Label automation** | Automatic PR/issue labeling based on paths or content | `.github/labeler.yml`, label-sync config, auto-label workflows | -| 47 | **Multi-platform CI** | CI matrix covering multiple OS, arch, or runtime versions | `matrix:` in CI with `os: [ubuntu, macos, windows]` or multiple language versions | -| 48 | **Deployment automation** | Automated deployment pipeline | Deploy workflow triggered on merge/tag, staging + production environments in CI | -| 49 | **Automated code review checks** | Bot-driven review checks beyond CI | Danger.js config, review bot config, required review assignments, CODEOWNERS + required reviews | - ---- - -## Pillar 4 · Policy & Governance - -Rules, ownership, and constraints the agent must know about and respect. - -| # | Feature | What to look for | Evidence | -|---|---------|------------------|----------| -| 50 | **Comprehensive .gitignore** | Covers secrets, build artifacts, IDE files, agent artifacts | `.gitignore` with entries for `.env`, `node_modules/`, build dirs, `.cursor/`, `.claude/` | -| 51 | **License** | Clear license at root | `LICENSE`, `MIT-LICENSE`, `COPYING`, `LICENSE.md` | -| 52 | **Code ownership** | File/directory ownership mapping | `CODEOWNERS`, `.github/CODEOWNERS`, docs describing area owners or maintainer teams | -| 53 | **Security policy** | How to report vulnerabilities | `SECURITY.md`, `.github/security.md`, security reporting instructions | -| 54 | **Code of conduct** | Community standards | `CODE_OF_CONDUCT.md`, conduct link in contributing guide | -| 55 | **AI usage policy** | Documented guidelines for AI/agent contributions | AI policy in AGENTS.md, CONTRIBUTING.md, or standalone doc; agent boundary definitions | -| 56 | **Secrets management** | Secrets handled via environment/vault, not hardcoded | References to `${{ secrets.* }}` in CI, vault config, `.env.example` without values, `git-secrets` | -| 57 | **Security scanning** | Automated vulnerability scanning in CI | `.github/workflows/codeql.yml`, Snyk config, `gosec` in CI, Trivy, Dependabot security alerts | -| 58 | **Git attributes** | Line endings, diff drivers, LFS, linguist overrides | `.gitattributes` with `text=auto`, `linguist-generated`, LFS tracking patterns | -| 59 | **Contributor agreement** | DCO sign-off or CLA process | DCO bot config, `Signed-off-by` requirement in contributing guide, CLA-assistant config | -| 60 | **Governance model** | Documented maintainer roles, decision-making process | `GOVERNANCE.md`, `MAINTAINERS.md`, governance section in docs, team/role descriptions | -| 61 | **CI workflow validation** | CI config itself is linted/validated | `actionlint` step in CI, `circleci config validate`, CI config schema validation | -| 62 | **Environment separation** | Distinct configs for dev/test/prod | `.env.test`, `.env.production`, environment-specific config directories, config per deploy target | - ---- - -## Pillar 5 · Build & Dev Environment - -Can the agent actually build, run, and iterate on the project? Reproducibility -and speed matter — an agent that can't build the project can't do anything. - -| # | Feature | What to look for | Evidence | -|---|---------|------------------|----------| -| 63 | **Dependency lockfile** | Pinned dependency versions for reproducible installs | `package-lock.json`, `yarn.lock`, `pnpm-lock.yaml`, `uv.lock`, `Cargo.lock`, `go.sum`, `Gemfile.lock`, `poetry.lock` | -| 64 | **Single-command build** | One documented command to build the entire project | `make build`, `npm run build`, `cargo build` documented in README or agent file | -| 65 | **Single-command dev setup** | One command to bootstrap a working dev environment | `bin/setup`, `make dev`, `scripts/bootstrap.sh`, `just setup` | -| 66 | **Dev container** | Containerized dev environment definition | `.devcontainer/devcontainer.json`, `devcontainer.json` at root | -| 67 | **Containerized services** | Docker-based local development stack | `Dockerfile`, `docker-compose.yml`, `compose.yaml` with dev services | -| 68 | **Reproducible environment** | Declarative, reproducible dev environment | `flake.nix`, `shell.nix`, `devbox.json`, hermetic build definitions | -| 69 | **Tool version pinning** | Runtime/tool versions pinned to a file | `.tool-versions`, `mise.toml`, `.node-version`, `.python-version`, `.ruby-version`, `rust-toolchain.toml` | -| 70 | **Monorepo orchestration** | Tooling for multi-package repositories | Workspace config in `package.json`, `pnpm-workspace.yaml`, Cargo workspace, Go multi-module, Nx/Turborepo/Bazel config | -| 71 | **Build caching** | Caching for faster rebuilds | CI cache steps (`actions/cache`), Turborepo remote cache, ccache/sccache config, layer caching in Docker | -| 72 | **Cross-platform support** | Builds on multiple OS/arch | CI matrix with multiple OS entries, cross-compilation configs, multi-arch Docker builds | -| 73 | **Cloud dev environment** | Cloud-based workspace configuration | `.devcontainer/` with Codespaces features, `.gitpod.yml`, cloud workspace config | -| 74 | **Package manager configuration** | Custom registry, auth, resolution settings | `.npmrc`, `pip.conf`, `.cargo/config.toml`, registry overrides, resolution overrides | - ---- - -## Summary - -| Pillar | Features | What it answers | -|--------|----------|-----------------| -| Agent Instructions | 18 | Does the agent know what to do? | -| Feedback Loops | 16 | Does the agent know if it's right? | -| Workflows & Automation | 15 | Does the process support agent work? | -| Policy & Governance | 13 | Does the agent know the rules? | -| Build & Dev Environment | 12 | Can the agent build and run the project? | -| **Total** | **74** | | diff --git a/plugins/onboarding/skills/agent-onboarding/references/criteria.md b/plugins/onboarding/skills/agent-onboarding/references/criteria.md new file mode 120000 index 0000000..e04318e --- /dev/null +++ b/plugins/onboarding/skills/agent-onboarding/references/criteria.md @@ -0,0 +1 @@ +../../../references/criteria.md \ No newline at end of file diff --git a/plugins/onboarding/skills/agent-readiness/README.md b/plugins/onboarding/skills/agent-readiness/README.md index 013a689..43f410b 100644 --- a/plugins/onboarding/skills/agent-readiness/README.md +++ b/plugins/onboarding/skills/agent-readiness/README.md @@ -64,4 +64,4 @@ Structured output with per-pillar scores, evidence for passing features, and not ## References -- `references/criteria.md` — full 74-feature evaluation criteria derived from analysis of 123 repositories +- `references/criteria.md` — full 74-feature evaluation criteria diff --git a/plugins/onboarding/skills/agent-readiness/references/criteria.md b/plugins/onboarding/skills/agent-readiness/references/criteria.md deleted file mode 100644 index 1fe2788..0000000 --- a/plugins/onboarding/skills/agent-readiness/references/criteria.md +++ /dev/null @@ -1,144 +0,0 @@ -# Agent Readiness Criteria - -Features that make a codebase ready for AI-assisted development, organized into -five pillars. Derived from cluster analysis of 123 repositories, but written to -be tool-agnostic. Every feature answers one question: *if this is missing, what -goes wrong for the agent?* - ---- - -## Pillar 1 · Agent Instructions - -How the repo tells AI agents what to do, what to avoid, and how the codebase -works. This is the highest-signal pillar — it's the difference between an agent -that understands the project and one that's guessing. - -| # | Feature | What to look for | Evidence | -|---|---------|------------------|----------| -| 1 | **Agent instruction file** | A dedicated file telling agents how to work in this repo — conventions, banned patterns, common commands | `AGENTS.md`, `CLAUDE.md`, `COPILOT.md`, `CONVENTIONS.md` at root | -| 2 | **AI IDE configuration** | Settings or rules for AI-powered editors/IDEs | `.cursor/rules/`, `.cursorrules`, `.github/copilot-instructions.md`, `.github/instructions/`, `.claude/settings.json` | -| 3 | **Multi-model support** | Instructions that work across different AI models/tools, not locked to one vendor | 2+ distinct agent config types from features 1–2 present in same repo | -| 4 | **Agent skills or capabilities** | Packaged, reusable abilities the agent can invoke | `.claude/skills/`, `.factory/skills/`, `skill.md` files, tool definition files | -| 5 | **Tool server configuration** | Config for agent tool protocols (lets agents use external tools) | `.mcp.json`, `mcp.config.js`, tool server manifests | -| 6 | **Agent prompt library** | Pre-built prompts for common tasks in this repo | `.github/prompts/`, `prompts/` directory, prompt template files | -| 7 | **Component-level agent guidance** | Different parts of the codebase have their own agent instructions | `AGENTS.md` or instruction files in subdirectories (e.g. `frontend/AGENTS.md`, `api/CLAUDE.md`) | -| 8 | **README with build/run/test** | README includes the commands to build, run, and test the project | `README.md` containing code blocks with build/install/test commands | -| 9 | **Contributing guide** | How to contribute — code style, PR process, commit conventions | `CONTRIBUTING.md`, `docs/contributing.md`, contributing section in README | -| 10 | **Architecture documentation** | High-level overview of how the system is structured and why | `ARCHITECTURE.md`, `docs/architecture/`, Mermaid/PlantUML diagrams, `doc/design/` | -| 11 | **API documentation** | Reference docs for the project's interfaces | `openapi.yaml`, generated HTML docs, `doc.go` files, Swagger UI, `api-docs/` | -| 12 | **Inline code documentation** | Doc comments, docstrings — agents read these to understand intent | JSDoc `/** */` blocks, Python docstrings, GoDoc comments, RDoc `#` blocks, Rust `///` | -| 13 | **Runnable examples** | Working example code the agent can study and imitate | `examples/` directory, `_examples/`, example apps with their own READMEs | -| 14 | **Changelog** | History of what changed and how entries should be written | `CHANGELOG.md`, `CHANGES.md`, `HISTORY.md`, release notes in GitHub Releases | -| 15 | **Environment variable documentation** | Template or docs for required env vars | `.env.example`, `.env.template`, env var table in README or docs | -| 16 | **Documentation site or directory** | Organized docs beyond the README | `docs/` directory, Docusaurus/Sphinx/MkDocs/VitePress config, published doc site | -| 17 | **Decision records** | Documented reasoning behind past architectural choices | `doc/adr/`, `decisions/`, `rfcs/`, numbered markdown decision files | -| 18 | **Module-level READMEs** | Individual packages/modules have their own READMEs | `packages/*/README.md`, `libs/*/README.md`, per-crate/per-module READMEs | - ---- - -## Pillar 2 · Feedback Loops - -How quickly and clearly the agent learns whether its changes are correct. Fast, -clear feedback is the difference between an agent that converges on a solution -and one that spirals. - -| # | Feature | What to look for | Evidence | -|---|---------|------------------|----------| -| 19 | **Linter** | Static analysis that catches bugs and style issues | `.eslintrc.*`, `ruff.toml`, `.golangci.yml`, `clippy.toml`, `pylintrc`, lint config in `pyproject.toml` | -| 20 | **Formatter** | Auto-formatter that enforces consistent style | `.prettierrc`, `rustfmt.toml`, `[tool.black]` in pyproject.toml, `gofmt`/`goimports` in CI | -| 21 | **Type checking** | Static type system or type checker | `tsconfig.json` with `strict`, `mypy.ini`, `[tool.mypy]`, `py.typed` marker, Go (inherent) | -| 22 | **Pre-commit hooks** | Checks that run before commit — catches issues before CI | `.pre-commit-config.yaml`, `.husky/`, `lefthook.yml`, `lint-staged` config | -| 23 | **Unit tests** | Tests for individual components | `*_test.go`, `*_test.py`, `*.spec.ts`, `test/`, `__tests__/`, test runner config | -| 24 | **Integration tests** | Tests that verify components work together | `test/integration/`, `tests/e2e/`, API test suites, test files with service dependencies | -| 25 | **End-to-end tests** | Full system/browser tests | Playwright config, Cypress config, Selenium tests, `e2e/` directory | -| 26 | **Test coverage measurement** | Coverage tracking so the agent knows if new code is tested | `.codecov.yml`, `coverageThreshold` in jest config, `--cov` in pytest, `cover` profile in Go | -| 27 | **CI pipeline** | Automated checks on every push or PR | `.github/workflows/ci.yml`, `.circleci/config.yml`, `.gitlab-ci.yml`, `Jenkinsfile` | -| 28 | **Fast CI feedback** | CI completes quickly enough for agent iteration | CI workflow with documented expected duration, parallel jobs, test splitting | -| 29 | **Test run documentation** | Agent knows exactly how to run which tests | Test commands in README, AGENTS.md, CONTRIBUTING.md, or Makefile help target | -| 30 | **Config/schema validation** | YAML, JSON, config files validated automatically | `yamllint` config, JSON schema `$schema` refs, `actionlint` in CI, `taplo` for TOML | -| 31 | **Snapshot or golden-file tests** | Tests that detect unexpected output changes | `__snapshots__/` directories, `.snap` files, `testdata/` golden files, VCR cassettes | -| 32 | **Benchmark suite** | Performance tests the agent can run to check for regressions | `bench/`, `benchmarks/`, `*_bench_test.go`, pytest-benchmark, Criterion.rs | -| 33 | **Warnings-as-errors** | Compiler/runtime warnings treated as failures | `-Werror`, `warningsAsErrors` in build config, `filterwarnings = error` in pytest | -| 34 | **Spell/typo checking** | Automated spelling checks in CI or hooks | `.cspell.json`, `codespell` in pre-commit, `typos.toml`, spell check CI step | - ---- - -## Pillar 3 · Workflows & Automation - -The processes that support agent-driven development — how work is structured, -tracked, and shipped. - -| # | Feature | What to look for | Evidence | -|---|---------|------------------|----------| -| 35 | **Issue templates** | Structured templates for bug reports, feature requests | `.github/ISSUE_TEMPLATE/` with `bug_report.md`, `feature_request.md`, `config.yml` | -| 36 | **PR template** | Template that guides PR descriptions | `.github/pull_request_template.md`, PR template with checklist items | -| 37 | **Dependency update automation** | Automated PRs for dependency updates | `.github/dependabot.yml`, `renovate.json`, `.renovaterc`, Renovate preset configs | -| 38 | **Release automation** | Automated release pipeline (build, tag, publish) | Release workflow in CI, `semantic-release` config, GoReleaser config, `release-please` | -| 39 | **Branch protection** | Protected main branch with required checks | Branch protection rules (inferred from merge queue config, required status checks in CI) | -| 40 | **Merge automation** | Merge queue, auto-merge, or merge bot | `merge_group` trigger in CI, Mergify config, auto-merge labels, `gh pr merge --auto` | -| 41 | **Task runner** | Single entry point for common commands | `Makefile`, `Justfile`, `Taskfile.yml`, `package.json` scripts section, `Rakefile` | -| 42 | **Structured change tracking** | Changesets, conventional commits, or similar discipline | `.changeset/` directory, `commitlint` config, conventional commit enforcement in CI | -| 43 | **CI concurrency control** | Cancel-in-progress, concurrency groups to avoid CI pile-up | `concurrency:` blocks in GitHub Actions, `cancel-in-progress: true` | -| 44 | **Automated release notes** | Changelog/release notes generated from commits or PRs | `release-please` config, `auto-changelog`, `git-cliff`, `conventional-changelog` | -| 45 | **Stale issue/PR management** | Automation to close or label stale items | `.github/workflows/stale.yml`, `stale` bot config, issue lifecycle labels | -| 46 | **Label automation** | Automatic PR/issue labeling based on paths or content | `.github/labeler.yml`, label-sync config, auto-label workflows | -| 47 | **Multi-platform CI** | CI matrix covering multiple OS, arch, or runtime versions | `matrix:` in CI with `os: [ubuntu, macos, windows]` or multiple language versions | -| 48 | **Deployment automation** | Automated deployment pipeline | Deploy workflow triggered on merge/tag, staging + production environments in CI | -| 49 | **Automated code review checks** | Bot-driven review checks beyond CI | Danger.js config, review bot config, required review assignments, CODEOWNERS + required reviews | - ---- - -## Pillar 4 · Policy & Governance - -Rules, ownership, and constraints the agent must know about and respect. - -| # | Feature | What to look for | Evidence | -|---|---------|------------------|----------| -| 50 | **Comprehensive .gitignore** | Covers secrets, build artifacts, IDE files, agent artifacts | `.gitignore` with entries for `.env`, `node_modules/`, build dirs, `.cursor/`, `.claude/` | -| 51 | **License** | Clear license at root | `LICENSE`, `MIT-LICENSE`, `COPYING`, `LICENSE.md` | -| 52 | **Code ownership** | File/directory ownership mapping | `CODEOWNERS`, `.github/CODEOWNERS`, docs describing area owners or maintainer teams | -| 53 | **Security policy** | How to report vulnerabilities | `SECURITY.md`, `.github/security.md`, security reporting instructions | -| 54 | **Code of conduct** | Community standards | `CODE_OF_CONDUCT.md`, conduct link in contributing guide | -| 55 | **AI usage policy** | Documented guidelines for AI/agent contributions | AI policy in AGENTS.md, CONTRIBUTING.md, or standalone doc; agent boundary definitions | -| 56 | **Secrets management** | Secrets handled via environment/vault, not hardcoded | References to `${{ secrets.* }}` in CI, vault config, `.env.example` without values, `git-secrets` | -| 57 | **Security scanning** | Automated vulnerability scanning in CI | `.github/workflows/codeql.yml`, Snyk config, `gosec` in CI, Trivy, Dependabot security alerts | -| 58 | **Git attributes** | Line endings, diff drivers, LFS, linguist overrides | `.gitattributes` with `text=auto`, `linguist-generated`, LFS tracking patterns | -| 59 | **Contributor agreement** | DCO sign-off or CLA process | DCO bot config, `Signed-off-by` requirement in contributing guide, CLA-assistant config | -| 60 | **Governance model** | Documented maintainer roles, decision-making process | `GOVERNANCE.md`, `MAINTAINERS.md`, governance section in docs, team/role descriptions | -| 61 | **CI workflow validation** | CI config itself is linted/validated | `actionlint` step in CI, `circleci config validate`, CI config schema validation | -| 62 | **Environment separation** | Distinct configs for dev/test/prod | `.env.test`, `.env.production`, environment-specific config directories, config per deploy target | - ---- - -## Pillar 5 · Build & Dev Environment - -Can the agent actually build, run, and iterate on the project? Reproducibility -and speed matter — an agent that can't build the project can't do anything. - -| # | Feature | What to look for | Evidence | -|---|---------|------------------|----------| -| 63 | **Dependency lockfile** | Pinned dependency versions for reproducible installs | `package-lock.json`, `yarn.lock`, `pnpm-lock.yaml`, `uv.lock`, `Cargo.lock`, `go.sum`, `Gemfile.lock`, `poetry.lock` | -| 64 | **Single-command build** | One documented command to build the entire project | `make build`, `npm run build`, `cargo build` documented in README or agent file | -| 65 | **Single-command dev setup** | One command to bootstrap a working dev environment | `bin/setup`, `make dev`, `scripts/bootstrap.sh`, `just setup` | -| 66 | **Dev container** | Containerized dev environment definition | `.devcontainer/devcontainer.json`, `devcontainer.json` at root | -| 67 | **Containerized services** | Docker-based local development stack | `Dockerfile`, `docker-compose.yml`, `compose.yaml` with dev services | -| 68 | **Reproducible environment** | Declarative, reproducible dev environment | `flake.nix`, `shell.nix`, `devbox.json`, hermetic build definitions | -| 69 | **Tool version pinning** | Runtime/tool versions pinned to a file | `.tool-versions`, `mise.toml`, `.node-version`, `.python-version`, `.ruby-version`, `rust-toolchain.toml` | -| 70 | **Monorepo orchestration** | Tooling for multi-package repositories | Workspace config in `package.json`, `pnpm-workspace.yaml`, Cargo workspace, Go multi-module, Nx/Turborepo/Bazel config | -| 71 | **Build caching** | Caching for faster rebuilds | CI cache steps (`actions/cache`), Turborepo remote cache, ccache/sccache config, layer caching in Docker | -| 72 | **Cross-platform support** | Builds on multiple OS/arch | CI matrix with multiple OS entries, cross-compilation configs, multi-arch Docker builds | -| 73 | **Cloud dev environment** | Cloud-based workspace configuration | `.devcontainer/` with Codespaces features, `.gitpod.yml`, cloud workspace config | -| 74 | **Package manager configuration** | Custom registry, auth, resolution settings | `.npmrc`, `pip.conf`, `.cargo/config.toml`, registry overrides, resolution overrides | - ---- - -## Summary - -| Pillar | Features | What it answers | -|--------|----------|-----------------| -| Agent Instructions | 18 | Does the agent know what to do? | -| Feedback Loops | 16 | Does the agent know if it's right? | -| Workflows & Automation | 15 | Does the process support agent work? | -| Policy & Governance | 13 | Does the agent know the rules? | -| Build & Dev Environment | 12 | Can the agent build and run the project? | -| **Total** | **74** | | diff --git a/plugins/onboarding/skills/agent-readiness/references/criteria.md b/plugins/onboarding/skills/agent-readiness/references/criteria.md new file mode 120000 index 0000000..e04318e --- /dev/null +++ b/plugins/onboarding/skills/agent-readiness/references/criteria.md @@ -0,0 +1 @@ +../../../references/criteria.md \ No newline at end of file From e4b3bf157c773bdb535726d42e4a11d67b5d7fd3 Mon Sep 17 00:00:00 2001 From: Calvin Smithu Date: Mon, 23 Mar 2026 07:55:45 -0600 Subject: [PATCH 07/15] minor keyword update --- plugins/onboarding/.claude-plugin/plugin.json | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/plugins/onboarding/.claude-plugin/plugin.json b/plugins/onboarding/.claude-plugin/plugin.json index fc97ab0..2b81bdf 100644 --- a/plugins/onboarding/.claude-plugin/plugin.json +++ b/plugins/onboarding/.claude-plugin/plugin.json @@ -9,5 +9,12 @@ "homepage": "https://github.com/OpenHands/extensions", "repository": "https://github.com/OpenHands/extensions", "license": "MIT", - "keywords": ["onboarding", "readiness", "agents-md", "agent-readiness", "assessment"] + "keywords": [ + "onboarding", + "readiness", + "agents-md", + "agent-readiness", + "assessment", + "ai-readiness" + ] } From b8715159e9392e2bbff07b74bcd2884e7a8946bc Mon Sep 17 00:00:00 2001 From: Calvin Smithu Date: Tue, 24 Mar 2026 09:57:51 -0600 Subject: [PATCH 08/15] adding oh setu pskill and pr review setup skill --- .../onboarding/openhands-setup-skill/SKILL.md | 56 +++++++++++++++ .../onboarding/pr-review-setup-skill/SKILL.md | 72 +++++++++++++++++++ 2 files changed, 128 insertions(+) create mode 100644 plugins/onboarding/openhands-setup-skill/SKILL.md create mode 100644 plugins/onboarding/pr-review-setup-skill/SKILL.md diff --git a/plugins/onboarding/openhands-setup-skill/SKILL.md b/plugins/onboarding/openhands-setup-skill/SKILL.md new file mode 100644 index 0000000..fda972b --- /dev/null +++ b/plugins/onboarding/openhands-setup-skill/SKILL.md @@ -0,0 +1,56 @@ +--- +name: openhands-setup +description: Set up a repository for effective use with OpenHands. Creates AGENTS.md, setup and pre-commit scripts, and a PR review workflow. Designed to run automatically with minimal user intervention. Use when a user wants to configure their repo for OpenHands. +triggers: +- /setup-openhands +- set up openhands +- configure openhands for this repo +--- + +# Set Up OpenHands for a Repository + +Work through these steps in order. + +## Step 1: Create AGENTS.md + +Run `/agents-md` to generate a root-level `AGENTS.md` from the repo's actual +CI workflows, build files, and documentation. + +## Step 2: Create `.openhands/setup.sh` + +Create `.openhands/setup.sh` — a bootstrap script that runs at the start of +every OpenHands session. Read the repo's CI workflows, AGENTS.md, and build +files to determine the correct commands. The script should: + +- Install dependencies (the project's actual install command) +- Set required environment variables +- Run any other bootstrap steps (e.g. copy `.env.example` to `.env`) + +Keep it idempotent and fast. Use the real commands from CI, not generic examples. + +**Docs**: https://docs.openhands.dev/openhands/usage/customization/repository#setup-script + +## Step 3: Create `.openhands/pre-commit.sh` + +Create `.openhands/pre-commit.sh` — runs before every commit OpenHands makes. +Read the repo's CI workflows to find the lint and test commands, then mirror +them in the script. Exit non-zero on failure so the agent gets immediate +feedback instead of waiting for CI. + +The script should run the same checks CI runs — if CI runs `ruff check` and +`pytest`, run those. If it runs `cargo clippy` and `cargo test`, run those. + +**Docs**: https://docs.openhands.dev/openhands/usage/customization/repository#pre-commit-script + +## Step 4: Set up PR review + +Run `/setup-pr-review` to create the GitHub Actions workflow and walk the user +through configuration. + +## Step 5: Verify + +Confirm all files exist and are correct: +- `AGENTS.md` at repo root with real commands (not boilerplate) +- `.openhands/setup.sh` with the project's actual install/bootstrap commands +- `.openhands/pre-commit.sh` mirroring the CI lint/test checks +- `.github/workflows/pr-review.yml` with valid YAML diff --git a/plugins/onboarding/pr-review-setup-skill/SKILL.md b/plugins/onboarding/pr-review-setup-skill/SKILL.md new file mode 100644 index 0000000..9c6953e --- /dev/null +++ b/plugins/onboarding/pr-review-setup-skill/SKILL.md @@ -0,0 +1,72 @@ +--- +name: pr-review-setup +description: Set up the OpenHands automated PR review workflow in a GitHub repository. Creates the workflow file and asks the user for configuration preferences. Use when a user wants AI code review on their pull requests. +triggers: +- /setup-pr-review +- set up pr review +- add code review workflow +- openhands pr review +--- + +# Set Up OpenHands PR Review + +Add the PR review workflow to a GitHub repository so an OpenHands agent can +review pull requests and post inline comments. + +**Docs**: https://docs.all-hands.dev/sdk/guides/github-workflows/pr-review + +## Step 1: Create the workflow file + +Create `.github/workflows/pr-review.yml` in the target repo. Fetch the latest +example from https://docs.all-hands.dev/sdk/guides/github-workflows/pr-review +and use it as the starting template. The workflow checks out the +`OpenHands/extensions` repo and runs the `plugins/pr-review` composite action. + +Key elements the workflow must have: +- `on.pull_request` with appropriate trigger types +- `permissions`: `contents: read`, `pull-requests: write`, `issues: write` +- `concurrency` group per PR number with `cancel-in-progress: true` +- `uses: ./plugins/pr-review` with `llm-model`, `review-style`, `llm-api-key`, + and `github-token` + +## Step 2: Tell the user to add the secret + +The workflow requires an `LLM_API_KEY` repository secret. **You cannot create +this — the user must do it manually.** + +Tell them: + +> To activate the workflow, add your LLM API key as a GitHub repository secret: +> Settings → Secrets and variables → Actions → New repository secret. +> Name it `LLM_API_KEY`. The `GITHUB_TOKEN` is provided automatically. + +Do not ask the user for the key value. Just tell them where to put it. + +## Step 3: Ask the user for preferences + +Present these options and apply any requested changes to the workflow file: + +**Review style** (default: `roasted`) +- `roasted` — Linus Torvalds-style, blunt, focuses on data structures and + simplicity. +- `standard` — balanced, covers style/readability/security. + +**When to trigger** (default: on-demand only) +- On-demand: add `review-this` label or request `openhands-agent` as reviewer. +- Automatic: review every new PR. Add `opened` and `ready_for_review` to + `on.pull_request.types` and matching conditions to the `if:` block. + +**Which model** (default: `anthropic/claude-sonnet-4-5-20250929`) +- Any model supported by litellm. Can be a comma-separated list for A/B testing + (one model selected randomly per review). + +**Evidence requirement** (default: `false`) +- When `true`, the reviewer checks the PR description for an Evidence section + (screenshots for frontend, command output for backend). + +## Step 4: Verify + +Confirm: +- `.github/workflows/pr-review.yml` exists and is valid YAML +- The `if:` conditions match the triggers in `on.pull_request.types` +- The `secrets.LLM_API_KEY` reference is present in the `with:` block From aae6061e507c7e681f105312a195528b4716c617 Mon Sep 17 00:00:00 2001 From: Calvin Smithu Date: Tue, 24 Mar 2026 09:59:04 -0600 Subject: [PATCH 09/15] moving/renaming plugins --- .../skills/{agent-readiness => agent-readiness-report}/README.md | 0 .../skills/{agent-readiness => agent-readiness-report}/SKILL.md | 0 .../references/criteria.md | 0 .../scripts/scan_agent_instructions.sh | 0 .../scripts/scan_build_env.sh | 0 .../scripts/scan_feedback_loops.sh | 0 .../scripts/scan_policy.sh | 0 .../scripts/scan_workflows.sh | 0 .../{agent-onboarding => improve-agent-readiness}/README.md | 0 .../skills/{agent-onboarding => improve-agent-readiness}/SKILL.md | 0 .../references/criteria.md | 0 .../onboarding/skills/{agents-md => setup-agents-md}/README.md | 0 plugins/onboarding/skills/{agents-md => setup-agents-md}/SKILL.md | 0 .../{openhands-setup-skill => skills/setup-openhands}/SKILL.md | 0 .../{pr-review-setup-skill => skills/setup-pr-review}/SKILL.md | 0 15 files changed, 0 insertions(+), 0 deletions(-) rename plugins/onboarding/skills/{agent-readiness => agent-readiness-report}/README.md (100%) rename plugins/onboarding/skills/{agent-readiness => agent-readiness-report}/SKILL.md (100%) rename plugins/onboarding/skills/{agent-onboarding => agent-readiness-report}/references/criteria.md (100%) rename plugins/onboarding/skills/{agent-readiness => agent-readiness-report}/scripts/scan_agent_instructions.sh (100%) rename plugins/onboarding/skills/{agent-readiness => agent-readiness-report}/scripts/scan_build_env.sh (100%) rename plugins/onboarding/skills/{agent-readiness => agent-readiness-report}/scripts/scan_feedback_loops.sh (100%) rename plugins/onboarding/skills/{agent-readiness => agent-readiness-report}/scripts/scan_policy.sh (100%) rename plugins/onboarding/skills/{agent-readiness => agent-readiness-report}/scripts/scan_workflows.sh (100%) rename plugins/onboarding/skills/{agent-onboarding => improve-agent-readiness}/README.md (100%) rename plugins/onboarding/skills/{agent-onboarding => improve-agent-readiness}/SKILL.md (100%) rename plugins/onboarding/skills/{agent-readiness => improve-agent-readiness}/references/criteria.md (100%) rename plugins/onboarding/skills/{agents-md => setup-agents-md}/README.md (100%) rename plugins/onboarding/skills/{agents-md => setup-agents-md}/SKILL.md (100%) rename plugins/onboarding/{openhands-setup-skill => skills/setup-openhands}/SKILL.md (100%) rename plugins/onboarding/{pr-review-setup-skill => skills/setup-pr-review}/SKILL.md (100%) diff --git a/plugins/onboarding/skills/agent-readiness/README.md b/plugins/onboarding/skills/agent-readiness-report/README.md similarity index 100% rename from plugins/onboarding/skills/agent-readiness/README.md rename to plugins/onboarding/skills/agent-readiness-report/README.md diff --git a/plugins/onboarding/skills/agent-readiness/SKILL.md b/plugins/onboarding/skills/agent-readiness-report/SKILL.md similarity index 100% rename from plugins/onboarding/skills/agent-readiness/SKILL.md rename to plugins/onboarding/skills/agent-readiness-report/SKILL.md diff --git a/plugins/onboarding/skills/agent-onboarding/references/criteria.md b/plugins/onboarding/skills/agent-readiness-report/references/criteria.md similarity index 100% rename from plugins/onboarding/skills/agent-onboarding/references/criteria.md rename to plugins/onboarding/skills/agent-readiness-report/references/criteria.md diff --git a/plugins/onboarding/skills/agent-readiness/scripts/scan_agent_instructions.sh b/plugins/onboarding/skills/agent-readiness-report/scripts/scan_agent_instructions.sh similarity index 100% rename from plugins/onboarding/skills/agent-readiness/scripts/scan_agent_instructions.sh rename to plugins/onboarding/skills/agent-readiness-report/scripts/scan_agent_instructions.sh diff --git a/plugins/onboarding/skills/agent-readiness/scripts/scan_build_env.sh b/plugins/onboarding/skills/agent-readiness-report/scripts/scan_build_env.sh similarity index 100% rename from plugins/onboarding/skills/agent-readiness/scripts/scan_build_env.sh rename to plugins/onboarding/skills/agent-readiness-report/scripts/scan_build_env.sh diff --git a/plugins/onboarding/skills/agent-readiness/scripts/scan_feedback_loops.sh b/plugins/onboarding/skills/agent-readiness-report/scripts/scan_feedback_loops.sh similarity index 100% rename from plugins/onboarding/skills/agent-readiness/scripts/scan_feedback_loops.sh rename to plugins/onboarding/skills/agent-readiness-report/scripts/scan_feedback_loops.sh diff --git a/plugins/onboarding/skills/agent-readiness/scripts/scan_policy.sh b/plugins/onboarding/skills/agent-readiness-report/scripts/scan_policy.sh similarity index 100% rename from plugins/onboarding/skills/agent-readiness/scripts/scan_policy.sh rename to plugins/onboarding/skills/agent-readiness-report/scripts/scan_policy.sh diff --git a/plugins/onboarding/skills/agent-readiness/scripts/scan_workflows.sh b/plugins/onboarding/skills/agent-readiness-report/scripts/scan_workflows.sh similarity index 100% rename from plugins/onboarding/skills/agent-readiness/scripts/scan_workflows.sh rename to plugins/onboarding/skills/agent-readiness-report/scripts/scan_workflows.sh diff --git a/plugins/onboarding/skills/agent-onboarding/README.md b/plugins/onboarding/skills/improve-agent-readiness/README.md similarity index 100% rename from plugins/onboarding/skills/agent-onboarding/README.md rename to plugins/onboarding/skills/improve-agent-readiness/README.md diff --git a/plugins/onboarding/skills/agent-onboarding/SKILL.md b/plugins/onboarding/skills/improve-agent-readiness/SKILL.md similarity index 100% rename from plugins/onboarding/skills/agent-onboarding/SKILL.md rename to plugins/onboarding/skills/improve-agent-readiness/SKILL.md diff --git a/plugins/onboarding/skills/agent-readiness/references/criteria.md b/plugins/onboarding/skills/improve-agent-readiness/references/criteria.md similarity index 100% rename from plugins/onboarding/skills/agent-readiness/references/criteria.md rename to plugins/onboarding/skills/improve-agent-readiness/references/criteria.md diff --git a/plugins/onboarding/skills/agents-md/README.md b/plugins/onboarding/skills/setup-agents-md/README.md similarity index 100% rename from plugins/onboarding/skills/agents-md/README.md rename to plugins/onboarding/skills/setup-agents-md/README.md diff --git a/plugins/onboarding/skills/agents-md/SKILL.md b/plugins/onboarding/skills/setup-agents-md/SKILL.md similarity index 100% rename from plugins/onboarding/skills/agents-md/SKILL.md rename to plugins/onboarding/skills/setup-agents-md/SKILL.md diff --git a/plugins/onboarding/openhands-setup-skill/SKILL.md b/plugins/onboarding/skills/setup-openhands/SKILL.md similarity index 100% rename from plugins/onboarding/openhands-setup-skill/SKILL.md rename to plugins/onboarding/skills/setup-openhands/SKILL.md diff --git a/plugins/onboarding/pr-review-setup-skill/SKILL.md b/plugins/onboarding/skills/setup-pr-review/SKILL.md similarity index 100% rename from plugins/onboarding/pr-review-setup-skill/SKILL.md rename to plugins/onboarding/skills/setup-pr-review/SKILL.md From d69e74641ae779fe08640b92f83ca9c925deb083 Mon Sep 17 00:00:00 2001 From: Calvin Smithu Date: Tue, 24 Mar 2026 10:05:13 -0600 Subject: [PATCH 10/15] fixing descriptions and triggers --- plugins/onboarding/skills/agent-readiness-report/SKILL.md | 5 +++-- plugins/onboarding/skills/improve-agent-readiness/SKILL.md | 7 ++++--- plugins/onboarding/skills/setup-agents-md/SKILL.md | 5 +++-- plugins/onboarding/skills/setup-openhands/SKILL.md | 4 ++-- plugins/onboarding/skills/setup-pr-review/SKILL.md | 4 ++-- 5 files changed, 14 insertions(+), 11 deletions(-) diff --git a/plugins/onboarding/skills/agent-readiness-report/SKILL.md b/plugins/onboarding/skills/agent-readiness-report/SKILL.md index 6acd816..a0c6199 100644 --- a/plugins/onboarding/skills/agent-readiness-report/SKILL.md +++ b/plugins/onboarding/skills/agent-readiness-report/SKILL.md @@ -1,8 +1,9 @@ --- name: agent-readiness-report -description: Evaluate how well a codebase supports autonomous AI-assisted development. Analyzes repositories across five pillars (Agent Instructions, Feedback Loops, Workflows & Automation, Policy & Governance, Build & Dev Environment) covering 74 features. Use when users request `/readiness-report` or want to assess how agent-ready a repository is. +description: Evaluate how well a codebase supports autonomous AI-assisted development. Analyzes repositories across five pillars (Agent Instructions, Feedback Loops, Workflows & Automation, Policy & Governance, Build & Dev Environment) covering 74 features. Use when users want to assess how agent-ready a repository is. triggers: -- /readiness-report +- agent-readiness-report +- readiness report --- # Agent Readiness Report diff --git a/plugins/onboarding/skills/improve-agent-readiness/SKILL.md b/plugins/onboarding/skills/improve-agent-readiness/SKILL.md index a4cd276..33752ff 100644 --- a/plugins/onboarding/skills/improve-agent-readiness/SKILL.md +++ b/plugins/onboarding/skills/improve-agent-readiness/SKILL.md @@ -1,8 +1,9 @@ --- -name: agent-onboarding -description: Use an agent readiness report to identify and implement improvements that make a repository more agent-friendly. Proposes repo-appropriate fixes for each pillar, then implements them on request. Use after running `/readiness-report` or when a user wants to improve their repo's AI-readiness. +name: improve-agent-readiness +description: Use an agent readiness report to identify and implement improvements that make a repository more agent-friendly. Proposes repo-appropriate fixes for each pillar, then implements them on request. Use after running a readiness report or when a user wants to improve their repo's AI-readiness. triggers: -- /agent-onboarding +- improve-agent-readiness +- agent onboarding --- # Agent Onboarding diff --git a/plugins/onboarding/skills/setup-agents-md/SKILL.md b/plugins/onboarding/skills/setup-agents-md/SKILL.md index 45c149b..6f2d805 100644 --- a/plugins/onboarding/skills/setup-agents-md/SKILL.md +++ b/plugins/onboarding/skills/setup-agents-md/SKILL.md @@ -1,8 +1,9 @@ --- -name: agents-md +name: setup-agents-md description: Generate a high-quality, repo-specific AGENTS.md file that tells AI agents how to work effectively in a repository. Reads the actual repo to extract commands, structure, conventions, and guardrails — never generates generic boilerplate. Use when a user wants to create or improve their AGENTS.md, or after a readiness report identifies a missing agent instruction file. triggers: -- /agents-md +- setup-agents-md +- agents-md --- # Generate AGENTS.md diff --git a/plugins/onboarding/skills/setup-openhands/SKILL.md b/plugins/onboarding/skills/setup-openhands/SKILL.md index fda972b..f5724f9 100644 --- a/plugins/onboarding/skills/setup-openhands/SKILL.md +++ b/plugins/onboarding/skills/setup-openhands/SKILL.md @@ -1,8 +1,8 @@ --- -name: openhands-setup +name: setup-openhands description: Set up a repository for effective use with OpenHands. Creates AGENTS.md, setup and pre-commit scripts, and a PR review workflow. Designed to run automatically with minimal user intervention. Use when a user wants to configure their repo for OpenHands. triggers: -- /setup-openhands +- setup-openhands - set up openhands - configure openhands for this repo --- diff --git a/plugins/onboarding/skills/setup-pr-review/SKILL.md b/plugins/onboarding/skills/setup-pr-review/SKILL.md index 9c6953e..2e0abc9 100644 --- a/plugins/onboarding/skills/setup-pr-review/SKILL.md +++ b/plugins/onboarding/skills/setup-pr-review/SKILL.md @@ -1,8 +1,8 @@ --- -name: pr-review-setup +name: setup-pr-review description: Set up the OpenHands automated PR review workflow in a GitHub repository. Creates the workflow file and asks the user for configuration preferences. Use when a user wants AI code review on their pull requests. triggers: -- /setup-pr-review +- setup-pr-review - set up pr review - add code review workflow - openhands pr review From ff60c9dc3da1655198d0abf2d01a13a1cf66b086 Mon Sep 17 00:00:00 2001 From: Calvin Smithu Date: Tue, 24 Mar 2026 10:08:05 -0600 Subject: [PATCH 11/15] fixing xrefs after name change --- plugins/onboarding/README.md | 18 +++++++++--------- .../skills/agent-readiness-report/README.md | 2 +- .../skills/improve-agent-readiness/README.md | 4 ++-- .../skills/improve-agent-readiness/SKILL.md | 2 +- .../skills/setup-agents-md/README.md | 2 +- .../onboarding/skills/setup-openhands/SKILL.md | 4 ++-- 6 files changed, 16 insertions(+), 16 deletions(-) diff --git a/plugins/onboarding/README.md b/plugins/onboarding/README.md index 3ab911f..5e2ac0f 100644 --- a/plugins/onboarding/README.md +++ b/plugins/onboarding/README.md @@ -5,10 +5,10 @@ Assess how well a repository supports AI-assisted development and implement targ ## Quick Start ``` -/readiness-report +agent-readiness-report ``` -Run the readiness report on any repository to get a scored evaluation across five pillars covering 74 features. Then use `/agent-onboarding` to turn gaps into concrete fixes, or `/agents-md` to generate the single highest-impact file for agent readiness. +Run the readiness report on any repository to get a scored evaluation across five pillars covering 74 features. Then use `improve-agent-readiness` to turn gaps into concrete fixes, or `setup-agents-md` to generate the single highest-impact file for agent readiness. ## Features @@ -18,19 +18,19 @@ Run the readiness report on any repository to get a scored evaluation across fiv ## Workflow -1. **Assess** — `/readiness-report` scans the repo and produces a structured report -2. **Plan** — `/agent-onboarding` reads the report and proposes the 5–10 highest-impact fixes +1. **Assess** — `agent-readiness-report` scans the repo and produces a structured report +2. **Plan** — `improve-agent-readiness` reads the report and proposes the 5–10 highest-impact fixes 3. **Implement** — Approved fixes are applied with atomic commits and the report is updated -4. **Generate** — `/agents-md` creates or improves the AGENTS.md using real repo data +4. **Generate** — `setup-agents-md` creates or improves the AGENTS.md using real repo data ## Skills | Skill | Trigger | Description | |-------|---------|-------------| -| [agent-readiness](skills/agent-readiness/) | `/readiness-report` | Evaluate a repo across five pillars (74 features) | -| [agent-onboarding](skills/agent-onboarding/) | `/agent-onboarding` | Propose and implement fixes from a readiness report | -| [agents-md](skills/agents-md/) | `/agents-md` | Generate a repo-specific AGENTS.md | +| [agent-readiness-report](skills/agent-readiness-report/) | `agent-readiness-report` | Evaluate a repo across five pillars (74 features) | +| [improve-agent-readiness](skills/improve-agent-readiness/) | `improve-agent-readiness` | Propose and implement fixes from a readiness report | +| [setup-agents-md](skills/setup-agents-md/) | `setup-agents-md` | Generate a repo-specific AGENTS.md | ## Shared References -The `agent-readiness` and `agent-onboarding` skills both use the same 74-feature evaluation criteria. The canonical file lives at `references/criteria.md` at the plugin root, and each skill symlinks to it from its own `references/` directory. Edit the plugin-level copy — the symlinks keep both skills in sync automatically. +The `agent-readiness-report` and `improve-agent-readiness` skills both use the same 74-feature evaluation criteria. The canonical file lives at `references/criteria.md` at the plugin root, and each skill symlinks to it from its own `references/` directory. Edit the plugin-level copy — the symlinks keep both skills in sync automatically. diff --git a/plugins/onboarding/skills/agent-readiness-report/README.md b/plugins/onboarding/skills/agent-readiness-report/README.md index 43f410b..9bf9d79 100644 --- a/plugins/onboarding/skills/agent-readiness-report/README.md +++ b/plugins/onboarding/skills/agent-readiness-report/README.md @@ -6,7 +6,7 @@ Evaluate how well a repository supports autonomous AI-assisted development. This skill is activated by the following keywords: -- `/readiness-report` +- `agent-readiness-report` ## What This Does diff --git a/plugins/onboarding/skills/improve-agent-readiness/README.md b/plugins/onboarding/skills/improve-agent-readiness/README.md index 15b7d13..f5c89ae 100644 --- a/plugins/onboarding/skills/improve-agent-readiness/README.md +++ b/plugins/onboarding/skills/improve-agent-readiness/README.md @@ -6,11 +6,11 @@ Take an agent readiness report and turn its gaps into concrete, repo-appropriate This skill is activated by the following keywords: -- `/agent-onboarding` +- `improve-agent-readiness` ## Prerequisites -An agent readiness report must already exist — either from running `/readiness-report` on the target repo or provided directly by the user. If no report exists, run the readiness report skill first. +An agent readiness report must already exist — either from running `agent-readiness-report` on the target repo or provided directly by the user. If no report exists, run the readiness report skill first. ## How It Works diff --git a/plugins/onboarding/skills/improve-agent-readiness/SKILL.md b/plugins/onboarding/skills/improve-agent-readiness/SKILL.md index 33752ff..b7d2336 100644 --- a/plugins/onboarding/skills/improve-agent-readiness/SKILL.md +++ b/plugins/onboarding/skills/improve-agent-readiness/SKILL.md @@ -13,7 +13,7 @@ Take a readiness report and turn its gaps into concrete, repo-appropriate fixes. ## Prerequisites This skill expects an agent readiness report to already exist — either from -running `/readiness-report` on this repo or provided by the user. If no report +running `agent-readiness-report` on this repo or provided by the user. If no report exists, run the readiness report skill first. ## How to run diff --git a/plugins/onboarding/skills/setup-agents-md/README.md b/plugins/onboarding/skills/setup-agents-md/README.md index 58887f2..c86281a 100644 --- a/plugins/onboarding/skills/setup-agents-md/README.md +++ b/plugins/onboarding/skills/setup-agents-md/README.md @@ -6,7 +6,7 @@ Generate a high-quality, repo-specific AGENTS.md file that tells AI agents how t This skill is activated by the following keywords: -- `/agents-md` +- `setup-agents-md` ## What This Does diff --git a/plugins/onboarding/skills/setup-openhands/SKILL.md b/plugins/onboarding/skills/setup-openhands/SKILL.md index f5724f9..6021a8c 100644 --- a/plugins/onboarding/skills/setup-openhands/SKILL.md +++ b/plugins/onboarding/skills/setup-openhands/SKILL.md @@ -13,7 +13,7 @@ Work through these steps in order. ## Step 1: Create AGENTS.md -Run `/agents-md` to generate a root-level `AGENTS.md` from the repo's actual +Run `setup-agents-md` to generate a root-level `AGENTS.md` from the repo's actual CI workflows, build files, and documentation. ## Step 2: Create `.openhands/setup.sh` @@ -44,7 +44,7 @@ The script should run the same checks CI runs — if CI runs `ruff check` and ## Step 4: Set up PR review -Run `/setup-pr-review` to create the GitHub Actions workflow and walk the user +Run `setup-pr-review` to create the GitHub Actions workflow and walk the user through configuration. ## Step 5: Verify From 96061d6dd20da5caedb9967df0e2f3a2dfc52dfb Mon Sep 17 00:00:00 2001 From: Calvin Smithu Date: Tue, 24 Mar 2026 10:11:53 -0600 Subject: [PATCH 12/15] tweaking readmes --- .../skills/agent-readiness-report/README.md | 60 +++---------------- .../skills/improve-agent-readiness/README.md | 42 ++----------- .../skills/setup-agents-md/README.md | 39 +----------- .../skills/setup-openhands/README.md | 20 +++++++ .../skills/setup-pr-review/README.md | 23 +++++++ 5 files changed, 57 insertions(+), 127 deletions(-) create mode 100644 plugins/onboarding/skills/setup-openhands/README.md create mode 100644 plugins/onboarding/skills/setup-pr-review/README.md diff --git a/plugins/onboarding/skills/agent-readiness-report/README.md b/plugins/onboarding/skills/agent-readiness-report/README.md index 9bf9d79..4716a5a 100644 --- a/plugins/onboarding/skills/agent-readiness-report/README.md +++ b/plugins/onboarding/skills/agent-readiness-report/README.md @@ -10,58 +10,14 @@ This skill is activated by the following keywords: ## What This Does -Assesses a codebase across five pillars covering 74 features that determine whether an AI agent can work effectively in a repository. The output is a structured report identifying what's present, what's missing, and which pillars are strongest and weakest. +Assesses a codebase across five pillars covering 74 features that determine whether an AI agent can work effectively in a repository. Produces a structured report identifying what's present, what's missing, and which pillars are strongest and weakest. -## Five Pillars +The five pillars are: -| Pillar | Question | Features | -|--------|----------|----------| -| **Agent Instructions** | Does the agent know what to do? | 18 | -| **Feedback Loops** | Does the agent know if it's right? | 16 | -| **Workflows & Automation** | Does the process support agent work? | 15 | -| **Policy & Governance** | Does the agent know the rules? | 13 | -| **Build & Dev Environment** | Can the agent build and run the project? | 12 | +- **Agent Instructions** — does the agent know what to do? +- **Feedback Loops** — does the agent know if it's right? +- **Workflows & Automation** — does the process support agent work? +- **Policy & Governance** — does the agent know the rules? +- **Build & Dev Environment** — can the agent build and run the project? -## How It Works - -### 1. Run the scanner scripts - -Five shell scripts gather filesystem signals — file existence, config patterns, directory structures: - -```bash -bash scripts/scan_agent_instructions.sh /path/to/repo -bash scripts/scan_feedback_loops.sh /path/to/repo -bash scripts/scan_workflows.sh /path/to/repo -bash scripts/scan_policy.sh /path/to/repo -bash scripts/scan_build_env.sh /path/to/repo -``` - -Or scan all five at once: - -```bash -for s in scripts/scan_*.sh; do bash "$s" /path/to/repo; echo; done -``` - -The scripts are helpers, not scorers. They find files and patterns but many features require judgment that only reading the actual files can provide. - -### 2. Evaluate each feature - -Walk through `references/criteria.md` pillar by pillar and mark each feature: **✓** (present), **✗** (missing), or **—** (not applicable). - -### 3. Write the report - -Structured output with per-pillar scores, evidence for passing features, and notes on what's missing. - -## Scanner Scripts - -| Script | Pillar | -|--------|--------| -| `scan_agent_instructions.sh` | Agent Instructions | -| `scan_feedback_loops.sh` | Feedback Loops | -| `scan_workflows.sh` | Workflows & Automation | -| `scan_policy.sh` | Policy & Governance | -| `scan_build_env.sh` | Build & Dev Environment | - -## References - -- `references/criteria.md` — full 74-feature evaluation criteria +Once you have a report, use `improve-agent-readiness` to turn the gaps into concrete fixes. diff --git a/plugins/onboarding/skills/improve-agent-readiness/README.md b/plugins/onboarding/skills/improve-agent-readiness/README.md index f5c89ae..1ec1d03 100644 --- a/plugins/onboarding/skills/improve-agent-readiness/README.md +++ b/plugins/onboarding/skills/improve-agent-readiness/README.md @@ -1,4 +1,4 @@ -# Agent Onboarding +# Improve Agent Readiness Take an agent readiness report and turn its gaps into concrete, repo-appropriate fixes. @@ -12,40 +12,8 @@ This skill is activated by the following keywords: An agent readiness report must already exist — either from running `agent-readiness-report` on the target repo or provided directly by the user. If no report exists, run the readiness report skill first. -## How It Works +## What This Does -1. **Read the report** — identify every missing (✗) feature across all five pillars -2. **Propose 5–10 high-impact fixes** — ranked by how directly each change helps an agent complete coding tasks, not by checklist coverage -3. **Implement on request** — apply approved fixes with atomic commits, then update the readiness report to reflect the new state - -## Ranking Heuristic - -Proposals are ranked by agent impact: - -1. **Unblocking** — things that let agents work at all (AGENTS.md, build commands, bootstrap scripts) -2. **Fast feedback** — things that tell agents whether their changes are correct (pre-commit hooks, test documentation) -3. **Quality & process** — things that improve agent output quality (CI caching, label automation) -4. **Governance** — things that define boundaries (SECURITY.md, CODEOWNERS, CodeQL) - -A single change that lets an agent build and test a project outranks a two-for-one that addresses minor gaps. - -## Pillar-Specific Guidance - -The skill includes targeted advice for each pillar: - -- **Agent Instructions** — AGENTS.md, AI IDE config, contributing guides, `.env.example` -- **Feedback Loops** — pre-commit hooks, test run documentation, linter/formatter setup -- **Workflows & Automation** — PR templates, issue templates, CI concurrency control -- **Policy & Governance** — CODEOWNERS, security policy, agent-aware `.gitignore` -- **Build & Dev Environment** — dependency lockfiles, tool version pinning, single-command setup - -## Implementation Rules - -- Content is specific to the target repo — no boilerplate -- Matches existing style (tabs/spaces, doc format, naming conventions) -- One commit per logical fix -- Readiness report is updated after implementation - -## References - -- `references/criteria.md` — full 74-feature evaluation criteria with descriptions and evidence examples +1. **Reads the report** — identifies every missing feature across all five pillars +2. **Proposes 5–10 high-impact fixes** — ranked by how directly each change helps an agent complete coding tasks, not by checklist coverage +3. **Implements on request** — applies approved fixes with atomic commits, then updates the readiness report to reflect the new state diff --git a/plugins/onboarding/skills/setup-agents-md/README.md b/plugins/onboarding/skills/setup-agents-md/README.md index c86281a..76a9545 100644 --- a/plugins/onboarding/skills/setup-agents-md/README.md +++ b/plugins/onboarding/skills/setup-agents-md/README.md @@ -12,41 +12,4 @@ This skill is activated by the following keywords: Reads the actual repository — Makefiles, CI configs, package.json scripts, linter configs, directory structure — and produces an AGENTS.md with real commands, real paths, and real conventions. Never generates generic boilerplate. -## Why AGENTS.md Matters - -An AGENTS.md is the single highest-impact addition for agent readiness. Without it, agents guess at build commands, miss project conventions, run the wrong test suite, and don't know what's dangerous. - -## Generated Sections - -All sections are optional — only what's relevant to the repo is included: - -| Section | Contents | -|---------|----------| -| **Project overview** | What the project is, language/framework, high-level architecture | -| **Commands** | Build, test, lint, and format commands copied from the repo's actual configs | -| **Project structure** | What's in each top-level directory | -| **Coding standards** | Conventions not captured by linter config | -| **Testing** | Where tests live, how to run them, any setup needed | -| **Guardrails** | Things the agent must NOT do — destructive commands, protected files | - -## Key Principles - -- **Be specific, not generic** — `uv run ruff check --fix `, not "run the linter" -- **Commands are king** — every command block is copy-pasteable -- **Link, don't repeat** — reference existing docs rather than duplicating them -- **Include dangerous operations** — agents are literal and need explicit guardrails -- **Keep it maintainable** — a short, accurate file beats a long one that drifts - -## Monorepo Support - -For monorepos, the root AGENTS.md covers repo-wide commands and conventions. Per-package files are created only when a package has substantially different commands or conventions. - -## Real-World Examples - -The skill references patterns from studied AGENTS.md files: - -- **CopilotKit** (32 lines) — concise essentials with links to detailed docs -- **airflow** (215 lines) — thorough command reference with real placeholders -- **grafana** (480 lines) — categorized frontend/backend commands with sub-directory links -- **swc** (75 lines) — performance-first philosophy anchoring every decision -- **Lychee** (122 lines) — safety-critical guardrails with explicit "never run" section +Sections are included only when relevant: project overview, build/test/lint commands, project structure, coding standards, testing setup, and guardrails for things the agent must not do. diff --git a/plugins/onboarding/skills/setup-openhands/README.md b/plugins/onboarding/skills/setup-openhands/README.md new file mode 100644 index 0000000..2d1700d --- /dev/null +++ b/plugins/onboarding/skills/setup-openhands/README.md @@ -0,0 +1,20 @@ +# Set Up OpenHands + +Configure a repository for effective use with OpenHands in one pass. + +## Triggers + +This skill is activated by the following keywords: + +- `setup-openhands` +- `set up openhands` +- `configure openhands for this repo` + +## What This Does + +Walks through four steps to make a repo OpenHands-ready: + +1. **AGENTS.md** — generates a root-level agent instruction file with real commands and conventions from the repo +2. **`.openhands/setup.sh`** — creates a bootstrap script that installs dependencies and sets up the environment at the start of every session +3. **`.openhands/pre-commit.sh`** — creates a pre-commit script that mirrors CI checks so the agent gets fast feedback before pushing +4. **PR review workflow** — adds a GitHub Actions workflow for automated code review on pull requests diff --git a/plugins/onboarding/skills/setup-pr-review/README.md b/plugins/onboarding/skills/setup-pr-review/README.md new file mode 100644 index 0000000..db911b1 --- /dev/null +++ b/plugins/onboarding/skills/setup-pr-review/README.md @@ -0,0 +1,23 @@ +# Set Up PR Review + +Add automated AI code review to a GitHub repository. + +## Triggers + +This skill is activated by the following keywords: + +- `setup-pr-review` +- `set up pr review` +- `add code review workflow` +- `openhands pr review` + +## What This Does + +Creates a `.github/workflows/pr-review.yml` file that triggers an OpenHands agent to review pull requests and post inline comments. Walks you through configuration options: + +- **Review style** — `roasted` (blunt, Torvalds-style) or `standard` (balanced) +- **When to trigger** — on-demand (label or reviewer request) or automatic (every PR) +- **Which model** — any litellm-supported model, with optional A/B testing +- **Evidence requirement** — optionally require proof-of-work in PR descriptions + +You'll need to add an `LLM_API_KEY` secret to your repository settings — the skill tells you exactly where. From cf719f7d05a45cfee777048fb6ea2bdbe817b31f Mon Sep 17 00:00:00 2001 From: Calvin Smithu Date: Tue, 24 Mar 2026 10:16:40 -0600 Subject: [PATCH 13/15] plugn readme improvement --- plugins/onboarding/README.md | 38 ++++++++----------- .../skills/agent-readiness-report/README.md | 2 +- 2 files changed, 17 insertions(+), 23 deletions(-) diff --git a/plugins/onboarding/README.md b/plugins/onboarding/README.md index 5e2ac0f..e969b19 100644 --- a/plugins/onboarding/README.md +++ b/plugins/onboarding/README.md @@ -1,36 +1,30 @@ # Onboarding Plugin -Assess how well a repository supports AI-assisted development and implement targeted improvements. This plugin bundles skills that evaluate agent readiness, propose high-impact fixes, and generate repo-specific `AGENTS.md` files — everything needed to take a codebase from agent-unfriendly to agent-ready. +Get a repository ready for OpenHands (and other AI agents) to start being productive. This plugin bundles the skills and documentation needed to take a codebase from zero to agent-ready. ## Quick Start -``` -agent-readiness-report -``` +Run the `setup-openhands` skill on any repository. It walks through everything needed in one pass: generates an AGENTS.md, creates setup and pre-commit scripts, and adds a PR review workflow. -Run the readiness report on any repository to get a scored evaluation across five pillars covering 74 features. Then use `improve-agent-readiness` to turn gaps into concrete fixes, or `setup-agents-md` to generate the single highest-impact file for agent readiness. +## Going Deeper -## Features +Once the basics are in place, you can harden your repo's agent readiness: -- **Readiness Assessment** — Evaluates 74 features across five pillars with scanner scripts and manual judgment -- **Prioritized Improvement Plans** — Ranks fixes by agent impact, not checklist completeness -- **AGENTS.md Generation** — Produces repo-specific agent instruction files from real commands and conventions - -## Workflow - -1. **Assess** — `agent-readiness-report` scans the repo and produces a structured report -2. **Plan** — `improve-agent-readiness` reads the report and proposes the 5–10 highest-impact fixes -3. **Implement** — Approved fixes are applied with atomic commits and the report is updated -4. **Generate** — `setup-agents-md` creates or improves the AGENTS.md using real repo data +1. **`agent-readiness-report`** — evaluates the repo across five pillars and produces a scored report +2. **`improve-agent-readiness`** — reads that report and proposes the highest-impact fixes, then implements them on request ## Skills -| Skill | Trigger | Description | -|-------|---------|-------------| -| [agent-readiness-report](skills/agent-readiness-report/) | `agent-readiness-report` | Evaluate a repo across five pillars (74 features) | -| [improve-agent-readiness](skills/improve-agent-readiness/) | `improve-agent-readiness` | Propose and implement fixes from a readiness report | -| [setup-agents-md](skills/setup-agents-md/) | `setup-agents-md` | Generate a repo-specific AGENTS.md | +| Skill | Description | +|-------|-------------| +| [setup-openhands](skills/setup-openhands/) | One-pass setup: AGENTS.md, scripts, and PR review workflow | +| [agent-readiness-report](skills/agent-readiness-report/) | Evaluate a repo's agent readiness across five pillars | +| [improve-agent-readiness](skills/improve-agent-readiness/) | Propose and implement fixes from a readiness report | +| [setup-agents-md](skills/setup-agents-md/) | Generate a repo-specific AGENTS.md | +| [setup-pr-review](skills/setup-pr-review/) | Add automated AI code review to a repo | + +The last two skills (`setup-agents-md` and `setup-pr-review`) are called by `setup-openhands` — you typically won't need to run them directly. ## Shared References -The `agent-readiness-report` and `improve-agent-readiness` skills both use the same 74-feature evaluation criteria. The canonical file lives at `references/criteria.md` at the plugin root, and each skill symlinks to it from its own `references/` directory. Edit the plugin-level copy — the symlinks keep both skills in sync automatically. +The `agent-readiness-report` and `improve-agent-readiness` skills share the same evaluation criteria. The canonical file lives at `references/criteria.md` at the plugin root; each skill symlinks to it. diff --git a/plugins/onboarding/skills/agent-readiness-report/README.md b/plugins/onboarding/skills/agent-readiness-report/README.md index 4716a5a..1f25dbf 100644 --- a/plugins/onboarding/skills/agent-readiness-report/README.md +++ b/plugins/onboarding/skills/agent-readiness-report/README.md @@ -10,7 +10,7 @@ This skill is activated by the following keywords: ## What This Does -Assesses a codebase across five pillars covering 74 features that determine whether an AI agent can work effectively in a repository. Produces a structured report identifying what's present, what's missing, and which pillars are strongest and weakest. +Assesses a codebase across five pillars that determine whether an AI agent can work effectively in a repository. Produces a structured report identifying what's present, what's missing, and which pillars are strongest and weakest. The five pillars are: From b5e5eef1c6040d44594693775ac9d78b00dfe4ee Mon Sep 17 00:00:00 2001 From: Calvin Smith Date: Wed, 25 Mar 2026 12:51:35 -0600 Subject: [PATCH 14/15] updating pr review setup skill w/llm config --- .../skills/setup-pr-review/SKILL.md | 58 +++++++++---------- 1 file changed, 29 insertions(+), 29 deletions(-) diff --git a/plugins/onboarding/skills/setup-pr-review/SKILL.md b/plugins/onboarding/skills/setup-pr-review/SKILL.md index 2e0abc9..fb45219 100644 --- a/plugins/onboarding/skills/setup-pr-review/SKILL.md +++ b/plugins/onboarding/skills/setup-pr-review/SKILL.md @@ -19,28 +19,37 @@ review pull requests and post inline comments. Create `.github/workflows/pr-review.yml` in the target repo. Fetch the latest example from https://docs.all-hands.dev/sdk/guides/github-workflows/pr-review -and use it as the starting template. The workflow checks out the -`OpenHands/extensions` repo and runs the `plugins/pr-review` composite action. +and use it as the starting template. The workflow calls the +`OpenHands/extensions/plugins/pr-review` composite action directly. -Key elements the workflow must have: -- `on.pull_request` with appropriate trigger types -- `permissions`: `contents: read`, `pull-requests: write`, `issues: write` -- `concurrency` group per PR number with `cancel-in-progress: true` -- `uses: ./plugins/pr-review` with `llm-model`, `review-style`, `llm-api-key`, - and `github-token` +## Step 2: Configure the LLM -## Step 2: Tell the user to add the secret +Ask the user whether they are using the **OpenHands app** (app.all-hands.dev) +or their **own LLM provider** (e.g. Anthropic, OpenAI directly). -The workflow requires an `LLM_API_KEY` repository secret. **You cannot create -this — the user must do it manually.** +### OpenHands app (default) -Tell them: +OpenHands app users already have access to an LLM API key through the +OpenHands litellm proxy. Tell them: -> To activate the workflow, add your LLM API key as a GitHub repository secret: +> Go to https://app.all-hands.dev → Account → API Keys → OpenHands LLM Key, and copy your key. +> Then add it as a GitHub repository secret: > Settings → Secrets and variables → Actions → New repository secret. -> Name it `LLM_API_KEY`. The `GITHUB_TOKEN` is provided automatically. +> Name it `LLM_API_KEY`. -Do not ask the user for the key value. Just tell them where to put it. +Set these inputs in the workflow `with:` block: +- `llm-model: litellm_proxy/claude-sonnet-4-5-20250929` +- `llm-base-url: https://llm-proxy.app.all-hands.dev` + +### Own LLM provider + +If the user has their own API key (e.g. from Anthropic or OpenAI), tell them +to add it as a repository secret named `LLM_API_KEY` using the same path +above. Leave `llm-base-url` unset and set `llm-model` to the provider-prefixed +model name (e.g. `anthropic/claude-sonnet-4-5-20250929`). + +**You cannot create secrets — the user must do it manually.** Do not ask for +the key value. Just tell them where to put it. ## Step 3: Ask the user for preferences @@ -56,17 +65,8 @@ Present these options and apply any requested changes to the workflow file: - Automatic: review every new PR. Add `opened` and `ready_for_review` to `on.pull_request.types` and matching conditions to the `if:` block. -**Which model** (default: `anthropic/claude-sonnet-4-5-20250929`) -- Any model supported by litellm. Can be a comma-separated list for A/B testing - (one model selected randomly per review). - -**Evidence requirement** (default: `false`) -- When `true`, the reviewer checks the PR description for an Evidence section - (screenshots for frontend, command output for backend). - -## Step 4: Verify - -Confirm: -- `.github/workflows/pr-review.yml` exists and is valid YAML -- The `if:` conditions match the triggers in `on.pull_request.types` -- The `secrets.LLM_API_KEY` reference is present in the `with:` block +After applying these, ask the user if they want to explore additional options +(model selection, evidence requirements, custom review skills, observability). +If yes, walk them through it — use the docs as a reference: +https://docs.all-hands.dev/sdk/guides/github-workflows/pr-review +If not, you're done. From e7f8974004fad570fc6822230dc24cfd9d3f2a15 Mon Sep 17 00:00:00 2001 From: Calvin Smith Date: Wed, 25 Mar 2026 12:55:09 -0600 Subject: [PATCH 15/15] improved agents.md setup skill --- .../skills/setup-agents-md/SKILL.md | 51 +++++-------------- 1 file changed, 14 insertions(+), 37 deletions(-) diff --git a/plugins/onboarding/skills/setup-agents-md/SKILL.md b/plugins/onboarding/skills/setup-agents-md/SKILL.md index 6f2d805..c0d5834 100644 --- a/plugins/onboarding/skills/setup-agents-md/SKILL.md +++ b/plugins/onboarding/skills/setup-agents-md/SKILL.md @@ -24,11 +24,20 @@ directly addresses features across multiple pillars: Without it, agents guess at build commands, miss project conventions, run the wrong test suite, and don't know what's dangerous. The best AGENTS.md files -we've studied (airflow, CopilotKit, grafana, swc) all share the same core -pattern: real commands, clear structure, explicit boundaries. +share the same core pattern: real commands, clear structure, explicit boundaries. ## How to run +### Step 0: Check for an existing AGENTS.md + +Look for an existing `AGENTS.md` (or `.agents/AGENTS.md`) in the repo root. +If one exists, **do not rewrite it from scratch**. Instead, read the repo +(Step 1), compare what you find against what's already documented, and suggest +specific additions or changes. Present the suggestions to the user and let them +decide what to incorporate. + +If no file exists, proceed to create one. + ### Step 1: Read the repo Before writing anything, gather the actual information. Check these sources: @@ -136,38 +145,6 @@ that drifts out of date. Reference CI configs and Makefiles by path so a human ### Monorepo considerations -If the repo is a monorepo, consider whether sub-directory AGENTS.md files are -warranted. The root AGENTS.md should cover repo-wide commands and conventions. -Per-package files should only exist if a package has substantially different -commands or conventions from the root. - -CopilotKit does this well: the root file covers Nx monorepo basics and -architecture, then links to `.claude/docs/` for deeper topic-specific guidance. -Grafana versions its root file and explicitly lists sub-directory AGENTS.md paths. - -## Examples from real repos - -These patterns come from studying the best AGENTS.md files across 125 -repositories: - -**Concise and effective** (CopilotKit, 32 lines): One-sentence architecture, -"Essentials" section with 4 bullet points, then links to detailed docs. An -agent can read the whole thing in seconds and knows where to find more. - -**Thorough command reference** (airflow, 215 lines): Every test command has a -real example with placeholders (``, ``). Repository -structure section maps paths to purposes. Architecture boundaries section -explains what talks to what. - -**Categorized commands** (grafana, 480 lines): Build, test, lint commands -separated into backend/frontend blocks with exact shell commands. Sub-directory -AGENTS.md files linked from root for specialized areas (docs, alerting). - -**Performance-first philosophy** (swc, 75 lines): States the project's core -value ("always prefer performance") upfront so every decision the agent makes -is anchored to it. Verification checklist tells the agent exactly what to run -before finishing. - -**Safety-critical guardrails** (Lychee, 122 lines): Explicit "NEVER RUN THESE -COMMANDS" section with database-destroying operations called out. Test database -isolation explained so the agent doesn't accidentally use production. +If the repo is a monorepo, the root AGENTS.md should cover repo-wide commands +and conventions. Consider sub-directory AGENTS.md files only for packages with +substantially different commands or conventions, and link to them from the root.