diff --git a/README.md b/README.md index 83a4cb9..c750fcc 100644 --- a/README.md +++ b/README.md @@ -17,6 +17,13 @@ Heavily "inspired" by the - HCL - JavaScript - [Python](/languages/python) +- [AI-Assisted Development](ai-assisted-development.md) + - [AI Agent Configuration](/ai) + - [Rules](/ai/rules.md) + - [Skills](/ai/skills.md) + - [MCP](/ai/mcp.md) + - [Coding Agents](/ai/agents.md) + - [Non-Coding Agents](/ai/non-coding-agents.md) ## Git, GitHub, and you ## diff --git a/ai-assisted-development.md b/ai-assisted-development.md new file mode 100644 index 0000000..caadb02 --- /dev/null +++ b/ai-assisted-development.md @@ -0,0 +1,213 @@ +# AI-Assisted Development # + +This guide describes how to use AI-assisted development tools - such as +Cursor, Claude Code, GitHub Copilot, and OpenAI Codex - responsibly within +our projects. The goal is to capture the productivity benefits of these +tools while maintaining the security, quality, and review standards +expected of code produced in a regulated environment. + +Treat AI-assisted tools as *assistants*, not as autonomous decision-makers. +Every suggestion, completion, or generated change is the responsibility of +the human contributor who commits it. + +## Scope ## + +This guidance applies to any tool that suggests, generates, or transforms +source code, configuration, documentation, tests, or commit messages based +on a language model. It complements - and does not replace - the existing +[CONTRIBUTING](CONTRIBUTING.md), [Style Guide](style-guide.md), and +[Project Setup](project_setup/README.md) guidance. + +For guidance on the configuration artifacts that shape agent behavior +inside a repository - such as `AGENTS.md`, tool-specific Rules, and +Skills - see the [AI Agent Configuration guide](ai/README.md). + +## Responsible use ## + +- Use AI tools to accelerate well-understood work, not to bypass + understanding. You should be able to explain, defend, and maintain any + code you commit, regardless of how it was produced. +- Prefer official, organization-approved tooling and configurations. Do + not enable AI features in tools that have not been reviewed for use on + your project. +- Review and understand each tool's data-handling policy before sending + it any project content. Default to assuming prompts and context are + transmitted off-device. +- Disable AI features when working in repositories or directories that + contain content not approved for transmission to a third party. + +## Protecting sensitive information ## + +AI tools typically read open files, recently edited files, and selected +context from the workspace. Assume that anything within the tool's +context window may leave your machine. + +- Never include secrets, credentials, tokens, private keys, or signed + URLs in prompts, comments, or generated code. +- Do not paste production data, personally identifiable information + (PII), or controlled unclassified information (CUI) into a prompt. +- Keep `.env` files, credential files, and other sensitive artifacts out + of the workspace, or ensure they are excluded from the tool's indexing + and context. +- Be cautious when sharing proprietary or internal-only code with hosted + models. When in doubt, ask before sending it off-device. +- If a secret is exposed in a prompt or generated output, treat it as + compromised and rotate it immediately. + +## Human review requirements ## + +AI-generated code is *untrusted input* until a human has reviewed it. + +- Every AI-assisted change must be read, understood, and explicitly + accepted by the contributor before it is committed. +- AI-assisted pull requests follow the same review process as any other + contribution. They are not exempt from code owner review, CI checks, + or the quality assurance expectations described in + [CONTRIBUTING](CONTRIBUTING.md). +- Reviewers should not approve changes that the author cannot explain. +- Authorship and accountability remain with the human contributor. The + tool is not a co-author for review purposes. + +## Testing and validation ## + +- Run the project's existing test suite against any AI-generated change + before opening a pull request. +- Add or update tests to cover new behavior. Do not rely on the model's + assertion that code is correct. +- Validate that referenced APIs, functions, modules, flags, and + configuration keys actually exist and behave as described. Models can + fabricate plausible-looking identifiers that do not exist. +- Confirm that example commands, URLs, and file paths in generated + documentation are real and correct. + +## Security review expectations ## + +Apply the same security scrutiny to AI-generated code as to any other +contribution, with particular attention to common failure modes. + +- Validate and sanitize all external input handled by generated code. +- Check authentication, authorization, and session-handling logic + carefully. Models frequently produce code that "looks right" but omits + important checks. +- Watch for insecure defaults: disabled certificate verification, + permissive CORS, overly broad IAM policies, world-readable files, + hard-coded credentials, or weak cryptographic primitives. +- Confirm that error handling does not leak stack traces, secrets, or + internal implementation details. +- Run the project's static analysis, dependency scanning, and CodeQL + checks before merging. + +## Licensing and provenance ## + +- Do not accept large verbatim blocks of code from a model without + understanding their origin. Generated output may resemble code from + training data under licenses incompatible with this project's + [public domain dedication](LICENSE). +- If a suggestion appears to be copied from an identifiable upstream + source, locate the original and follow its license terms, including + attribution where required. +- Prefer small, idiomatic suggestions that you can write or rewrite + yourself over large generated blocks of uncertain origin. + +## Small, auditable changes ## + +- Prefer many small, reviewable AI-assisted changes over a single large + generation that is difficult to audit. +- Generate one logical change at a time and review it before moving on. + Stacking unreviewed generations compounds risk and makes regressions + harder to isolate. +- Keep commits scoped and atomic so that a faulty suggestion can be + reverted cleanly. + +## Transparency and documentation ## + +- When AI assistance materially shaped a non-trivial change, note it in + the pull request description so reviewers can calibrate their + attention. A short sentence is sufficient; detailed prompt logs are + not required. +- Do not attribute AI tools as commit authors or co-authors. The human + contributor is the author of record. +- Do not paste raw model output into commit messages, issues, or pull + requests without editing it for accuracy and tone. + +## Known risks to watch for ## + +- **Hallucinated APIs and dependencies.** Models may invent functions, + packages, command-line flags, or configuration options that do not + exist. Verify against authoritative documentation. +- **Outdated patterns.** Generated code may reflect deprecated APIs, + superseded best practices, or end-of-life library versions. Confirm + against current upstream documentation. +- **Insecure implementations.** Cryptography, authentication, input + parsing, and shell-command construction are common sources of subtly + insecure generated code. +- **Supply-chain risk.** Verify that any new dependency suggested by a + model exists, is actively maintained, and is appropriate for the + project. Watch for typosquatting and look-alike package names. +- **Confident incorrectness.** Models present wrong answers with the + same confidence as correct ones. Skepticism is the default posture. + +## Recommended practices ## + +- Keep the model's context narrow. Open only the files you intend to + work on. +- Read every diff before accepting it. Do not bulk-accept changes + across multiple files without inspection. +- Ask the tool to explain unfamiliar code it produced, then verify the + explanation against documentation. +- Use AI assistance for boilerplate, refactoring, test scaffolding, and + documentation drafts where verification is straightforward. +- Avoid using AI assistance as the primary author of security-sensitive + code, cryptographic logic, or infrastructure-as-code that grants + privileges. +- When a generated change touches more than a handful of files, stop and + reconsider whether it can be broken into smaller commits. + +## Pre-commit and local linting ## + +This repository, and most `cisagov` projects, enforce style and quality +through [pre-commit](https://pre-commit.com/) hooks that run the same +checks as CI. AI-assisted changes frequently introduce small formatting, +whitespace, line-length, or import-ordering issues that these hooks will +catch. Running them locally before pushing keeps pull requests clean and +shortens the review cycle. + +If you have not already configured the environment, follow the +[Setting up pre-commit](CONTRIBUTING.md#setting-up-pre-commit) section of +the [CONTRIBUTING](CONTRIBUTING.md) document. The short version, once +`pyenv`, `pyenv-virtualenv`, and the project virtual environment are in +place, is: + +```console +pip install --requirement requirements-dev.txt +pre-commit install +``` + +Before opening a pull request that includes AI-assisted changes: + +- Run the hooks against the full repository to catch issues outside the + files you edited: + + ```console + pre-commit run --all-files + ``` + +- Re-stage any files that the hooks auto-fixed (for example, trailing + whitespace, end-of-file fixes, or import sorting) and commit the + result. +- Resolve remaining failures locally rather than relying on CI to + surface them. Common categories include: + - Markdown lint findings from `markdownlint` (see + [`.mdl_config.yaml`](.mdl_config.yaml)), such as line length, + heading style, and list style. + - Python findings from `flake8` and `isort` (see + [`.flake8`](.flake8) and [`.isort.cfg`](.isort.cfg)). + - YAML findings from `yamllint` (see [`.yamllint`](.yamllint)). + - Ansible findings from `ansible-lint` (see + [`.ansible-lint`](.ansible-lint)) where applicable. +- If a hook fails on a file you did not knowingly change, inspect it. + Editor- or tool-driven reformatting is a frequent side effect of + AI-assisted workflows and should not be merged unintentionally. + +Treat a clean local `pre-commit run --all-files` as a prerequisite for +opening a pull request, not as an optional convenience. diff --git a/ai/README.md b/ai/README.md new file mode 100644 index 0000000..4bb0640 --- /dev/null +++ b/ai/README.md @@ -0,0 +1,109 @@ +# AI Agent Configuration Guide # + +This document is structured by topic; under each, we include "Standards", +"Defaults", and "Suggestions". + +**Standards** are practices that have a strong consensus across CISA; they +should generally be followed. + +**Defaults** are safe selections that tend to be used by a large number of our +projects; you may find yourself with a better or more tailored solution, +however. + +**Suggestions** contain examples that have worked well on a project or two; +they're not widely used enough to be defaults, but are worth considering. + +This guide covers configuration artifacts that shape how AI-assisted +development tools (Cursor, Claude Code, GitHub Copilot, OpenAI Codex, +and similar systems) behave inside a repository. For broader guidance +on responsible use of these tools, see the +[AI-Assisted Development guide](../ai-assisted-development.md). + +## Scope ## + +Five topics are in scope: + +- **Rules.** Persistent, repository-scoped instructions that constrain + or steer agent behavior across a session. See [Rules](rules.md). +- **Skills.** Reusable, task-scoped capability packages that an agent + can load on demand. See [Skills](skills.md). +- **MCP servers.** External tools, resources, and data sources that + an agent can reach over the Model Context Protocol. See + [MCP](mcp.md). +- **Coding agents.** Systems that use a language model to plan and + take actions against a software project on a developer's behalf - + reading files, running commands, editing source, opening pull + requests. See [Coding Agents](agents.md). +- **Non-coding agents.** Agents whose outputs are not repository + artifacts - for example, agents that triage messages, summarize + documents, drive a data pipeline, or act against an internal + system. See [Non-Coding Agents](non-coding-agents.md). + +The first three are configuration artifacts checked into the +repository and reviewed like code. The last two cover how those +artifacts are combined into something that *acts*, and the +additional expectations that come with that. + +## Why this matters ## + +AI agents act on whatever context they receive. Rules and Skills are +the primary mechanisms a project has for making that context +predictable, reviewable, and aligned with the project's security and +quality expectations. Treating them as configuration artifacts - rather +than as ephemeral chat instructions - is what makes their behavior +auditable. + +## Standards ## + +- Rules, Skills, and MCP server configuration are committed to the + repository, reviewed in pull requests, and subject to the same + expectations as any other code change. See the + [AI-Assisted Development guide](../ai-assisted-development.md) and + [CONTRIBUTING](../CONTRIBUTING.md). +- These artifacts must not contain secrets, credentials, tokens, + internal hostnames, or non-public data. Assume their contents will + be transmitted to the agent's backing model or to any configured + MCP server. +- Rules and Skills are advisory to the model and are not a security + boundary. MCP configuration controls reach, not behavior. None of + them replace code review, CI checks, or the project's existing + access controls. +- Every Rule, Skill, and MCP server entry has a clear purpose, a + named owner (via `CODEOWNERS`), and is removed when no longer + used. + +## Defaults ## + +- Keep Rules short and specific. Prefer several small, focused Rules + over one large general one. +- Keep Skills narrowly scoped to a single capability or workflow. +- Keep the set of enabled MCP servers small, narrowly scoped, and + pinned to a known version. +- Document, in the file itself or alongside the configuration, what + the Rule, Skill, or MCP server is for and when it should apply. +- Store these artifacts in the conventional locations expected by the + tool in use (for example, `.cursor/rules/` for Cursor, `AGENTS.md` + for tools that read it, `.claude/skills/` for Claude Code, and the + tool's documented MCP configuration path). Avoid scattering + equivalent guidance across multiple ad-hoc files. + +## Suggestions ## + +- When multiple tools are in use on the same project, prefer a single + source of truth (typically `AGENTS.md`) and reference it from + tool-specific files rather than duplicating content. +- Review Rules and Skills periodically. Stale guidance is worse than + no guidance, because it teaches the agent to do the wrong thing + confidently. + +## Contents ## + +- [Rules](rules.md) - persistent, repository-scoped agent instructions. +- [Skills](skills.md) - reusable, task-scoped agent capabilities. +- [MCP](mcp.md) - external tools, resources, and data sources reached + over the Model Context Protocol. +- [Coding Agents](agents.md) - constructing and operating agents whose + outputs are repository artifacts. +- [Non-Coding Agents](non-coding-agents.md) - engineering and + security baseline for agents whose outputs are not repository + artifacts. diff --git a/ai/coding-agents.md b/ai/coding-agents.md new file mode 100644 index 0000000..3529fa3 --- /dev/null +++ b/ai/coding-agents.md @@ -0,0 +1,193 @@ +# Coding Agents # + +A *coding agent* is a system that uses a language model to plan and +take actions against a software project on a developer's behalf - +reading files, running commands, editing source, running tests, +opening pull requests, and so on - rather than only producing text in +a chat window. Examples include in-editor coding agents (Cursor, +Claude Code, GitHub Copilot agent modes, OpenAI Codex agent modes) +and longer-running background or "cloud" coding agents that operate +against a repository or build environment without a developer +present. + +This document covers building and operating coding agents within +projects that follow this development guide. It assumes the agent's +output is code, configuration, tests, or related repository +artifacts, and that those outputs flow through the project's normal +review and CI controls. + +Agents built for other purposes - business process automation, data +pipelines, customer-facing assistants, and so on - are out of scope +for this document. See [Non-Coding Agents](non-coding-agents.md) for +guidance that applies to those. + +Coding agents combine the considerations covered in [Rules](rules.md), +[Skills](skills.md), and [MCP](mcp.md), and add new ones because they +*act*. For broader context on responsible use of AI tools generally, +see the [AI-Assisted Development guide](../ai-assisted-development.md). + +## When to use a coding agent ## + +Use a coding agent when: + +- The task is well-scoped, reproducible, and benefits from automation + more than from one-shot assistance. +- The actions the agent will take are bounded and reviewable - for + example, "open a draft pull request that updates dependency + versions" rather than "improve the codebase". +- The output of the agent will pass through the same review and CI + gates as any other change before it has effect. + +Do *not* use a coding agent when: + +- The task requires judgment the agent cannot reliably exercise - + notably anything that grants privileges, handles secrets, touches + production systems, or makes security-relevant trade-offs. +- The actions the agent would take cannot be reverted easily. +- The agent's effective permissions exceed what the human running it + would be granted for the same task. + +## Standards ## + +- Agents act under a clearly identified principal. The human who + starts the agent is accountable for everything it does in that + session. Service accounts used by autonomous agents are owned, + documented, and scoped to the minimum permissions required. +- Agents do not bypass code review, branch protection, CI checks, or + any other control that applies to human contributors. Agent- + produced changes land through normal pull requests. +- Agents must not be granted secrets, credentials, or production + access beyond what their narrowly defined task requires. Long- + lived broad credentials are not appropriate for agent use. +- Destructive, irreversible, or privilege-changing actions are gated + on explicit human confirmation. "Auto-approve everything" modes + are not used against project repositories or shared environments. +- Agent configuration - prompts, Rules, Skills, MCP servers, allowed + tools, model selection - is checked in, reviewed in pull requests, + and owned via `CODEOWNERS` when it affects the project. + +## Defaults ## + +- Start with the smallest agent that can do the job. Prefer an + in-editor assistant with human-in-the-loop approval over an + autonomous background agent unless the task clearly warrants the + latter. +- Scope agents to a single repository or a single task type. Avoid + agents whose role description is "do whatever the user asks across + any system". +- Run agents in an isolated working environment (an ephemeral + container, a dedicated workspace, or a sandboxed worktree) when + they execute commands. Do not point agents at a developer's main + shell with broad credentials available. +- Default to requiring human approval before the agent: + - Executes shell commands. + - Writes to files outside a well-defined working area. + - Calls external services that have side effects. + - Pushes branches, opens pull requests, or interacts with the + issue tracker. +- Log what the agent did - prompts, tool calls, commands executed, + files changed - in a form that a reviewer can inspect after the + fact. +- Pin the agent's model, tool list, and MCP server set to known + versions. Treat changes to any of these as configuration changes + subject to review. + +## Suggestions ## + +- For repetitive, well-understood maintenance work (dependency + bumps, lint fix-ups, documentation regeneration), a narrowly + scoped agent with a clear success criterion can be a good fit. +- For longer-running or background agents, prefer designs that + produce a draft pull request the team reviews, rather than + designs that merge or deploy directly. +- When an agent is given access to an MCP server that can take + action on external systems, prefer the smallest scope the server + supports and require human confirmation per action. +- Periodically review agent logs and outputs even when nothing has + obviously gone wrong. Drift in behavior is easier to catch early. +- Sunset agents that are no longer used. Idle agents with live + credentials are a quiet liability. + +## Constructing a coding agent ## + +When designing a new coding agent for a project, document the +following before it is enabled in any shared environment: + +- **Purpose.** One or two sentences describing what the agent is for + and what it is *not* for. +- **Trigger.** How the agent is started (a developer command, a + scheduled job, a webhook, a comment on an issue, etc.). +- **Principal and permissions.** Whose credentials the agent uses, + what scopes those credentials carry, and where the credentials + are stored. +- **Allowed actions.** The specific tools, commands, file paths, and + external endpoints the agent is permitted to use. +- **Forbidden actions.** Explicit out-of-scope items, especially any + destructive or privilege-changing operations. +- **Inputs.** What context the agent receives - which files, which + Rules, which Skills, which MCP servers, which environment + variables. +- **Outputs.** What the agent produces - typically a pull request, + an issue comment, or a written artifact - and how that output is + reviewed before it has effect. +- **Failure behavior.** What the agent does on error, timeout, or + refusal. Default to stopping and reporting rather than retrying + aggressively. +- **Owner.** A named team or individual responsible for the agent's + configuration and behavior. + +Keep this description short and check it in alongside the agent's +configuration so reviewers can evaluate changes against the stated +design. + +## Operating a coding agent ## + +- Start a coding agent with the narrowest task description that + still captures the work. Vague tasks produce vague, harder-to- + review results. +- Watch the first run of any new or materially changed agent + configuration. Do not leave it unattended until its behavior is + understood. +- Treat the agent's output as untrusted input. Read every diff. Run + the project's tests and `pre-commit run --all-files`. Do not merge + changes the human author cannot explain. +- Stop the agent if it begins doing something unexpected. Investigate + before resuming. +- Rotate any credential that the agent may have been exposed to if + the agent's environment or configuration is compromised. + +## Risks specific to coding agents ## + +- **Privilege amplification.** An agent with broad credentials and + the ability to run shell commands can do far more damage than a + one-shot assistant. Keep credentials and tools tightly scoped. +- **Prompt injection.** Content the agent reads - files, issues, + pull request descriptions, MCP server responses, web pages - can + contain instructions intended to redirect the agent. Treat all + such content as untrusted. +- **Tool misuse.** An agent may call a tool in a way the project did + not anticipate. Constrain tool inputs where the tool itself + supports it, and review the actions the agent actually performs. +- **Confident, irreversible action.** Agents will sometimes act + decisively on incorrect conclusions. Reversibility and human + approval gates are the primary defenses. +- **Cost and resource exhaustion.** Long-running agents can consume + significant compute, API quota, or CI minutes. Set explicit limits + and monitor usage. +- **Drift.** Model updates, tool updates, and MCP server updates can + change agent behavior without any change to the project's own + configuration. Pin where possible and re-evaluate periodically. + +## Limitations ## + +The guidance above reduces the likelihood and impact of coding +agent mistakes; it does not eliminate them. Agents are not a +security boundary. They are a productivity tool whose outputs and +actions remain the responsibility of the humans who configure, run, +and review them. When in doubt, prefer the more conservative +option: smaller scope, narrower permissions, more human approval, +and smaller, more auditable changes. See the +[AI-Assisted Development guide](../ai-assisted-development.md) for +the full set of expectations that apply to AI-assisted work, and +[Non-Coding Agents](non-coding-agents.md) for agents whose outputs +are not repository artifacts. diff --git a/ai/mcp.md b/ai/mcp.md new file mode 100644 index 0000000..3152965 --- /dev/null +++ b/ai/mcp.md @@ -0,0 +1,129 @@ +# Model Context Protocol (MCP) Servers # + +The Model Context Protocol (MCP) is an open protocol that lets AI +agents call out to external servers for tools, resources, and data. +An MCP server can expose anything from a local script to a hosted +API. Once configured, the agent can invoke it during a session in +much the same way it reads files or runs shell commands. + +MCP servers expand what an agent can do. They also expand the +attack surface and the set of systems that may receive project +context. Treat MCP configuration as a security-relevant change. + +For broader context on responsible use of AI tools, see the +[AI-Assisted Development guide](../ai-assisted-development.md). For +how MCP relates to Rules and Skills, see this directory's +[README](README.md). + +## When to use an MCP server ## + +Use an MCP server when: + +- The capability is not reasonably available through the agent's + built-in tools or a checked-in script. +- The integration is reusable across sessions and contributors. +- The server, its source, and its data-handling behavior are + understood and acceptable for the project. + +Do *not* use an MCP server when: + +- A simple, reviewed script in the repository would do the same job. +- The server's source, maintainer, or data-handling policy is + unknown. +- The server would receive secrets, credentials, customer data, or + other non-public information that has not been approved for that + destination. + +## Standards ## + +- MCP server configuration is checked in, reviewed in pull requests, + and owned via `CODEOWNERS`. Configuration that only lives on an + individual contributor's machine is not subject to project review + and should not be relied on by the project. +- MCP servers must not receive secrets, credentials, tokens, + customer data, internal hostnames, or any non-public information + unless the destination has been explicitly approved for that data. +- Credentials required by an MCP server are supplied through the + environment or an approved secrets mechanism. They are never + committed to the repository. +- Each MCP server in use has a clear, narrow purpose recorded + alongside its configuration, including who owns it and what data + it may receive. +- Adding, removing, or materially changing an MCP server is called + out in the pull request description so reviewers can scrutinize + the change. + +## Defaults ## + +- Prefer first-party or well-known MCP servers over unfamiliar + third-party ones. Pin to a specific version or commit rather than + tracking a moving tag. +- Prefer servers that run locally and operate on the workspace over + hosted servers that receive workspace contents over the network, + when both options are viable. +- Grant each MCP server the narrowest set of capabilities it needs. + Disable tools, scopes, or permissions that the project does not + use. +- Keep the list of enabled MCP servers small. Every additional + server is additional surface to review and maintain. +- Document, near the configuration, what each enabled server is for + and when it should be used. + +## Suggestions ## + +- Periodically review the list of enabled MCP servers and remove any + that are no longer used. Unused servers quietly accumulate risk. +- When evaluating a new MCP server, read its source (or the + authoritative documentation if it is a hosted service), confirm + its maintenance status, and note what it does with the inputs it + receives. +- For MCP servers that take action against external systems (for + example, creating issues, opening pull requests, or modifying + cloud resources), prefer configurations that require human + confirmation before each action. +- If a project uses several AI tools, keep MCP configuration in a + single, conventional location per tool rather than duplicating it. + Reference shared documentation about what each server is for. + +## What to put in MCP configuration ## + +Useful content includes: + +- The server name and a short description of its purpose. +- The command, package, or endpoint used to reach the server, pinned + to a specific version where possible. +- The minimum permissions or scopes the project actually needs. +- The owner or team responsible for keeping the configuration + current. + +Avoid: + +- Secrets, tokens, or credentials in the configuration file. Use + environment variables or an approved secrets mechanism. +- Server entries with no description, no owner, and no obvious + current use. +- Broad, "everything enabled" configurations. Narrow scope is the + default. + +## Review expectations ## + +- Adding a new MCP server is a security-relevant change. Reviewers + should confirm the server's source, its data-handling behavior, + and the scope of access being granted. +- Changes that broaden an existing server's scope (new tools, new + permissions, new endpoints) receive the same scrutiny as adding a + new server. +- Removing an MCP server is usually low-risk and should be + preferred when the server is no longer in active use. + +## Limitations ## + +MCP configuration controls what the agent *can* reach. It does not +control what the agent *will* do with that reach in any given +session. Outputs from MCP servers are untrusted input: they may be +inaccurate, out of date, or shaped by an attacker who controls the +upstream system. Validate results before acting on them, and apply +the same review expectations to code or changes produced with the +help of an MCP server as to any other AI-assisted change. See the +[AI-Assisted Development guide](../ai-assisted-development.md) for +the full set of expectations. diff --git a/ai/non-coding-agents.md b/ai/non-coding-agents.md new file mode 100644 index 0000000..8d4b1ea --- /dev/null +++ b/ai/non-coding-agents.md @@ -0,0 +1,234 @@ +# Non-Coding Agents # + +A *non-coding agent* is an AI agent whose primary outputs are not +source code, configuration, or other repository artifacts. Examples +include agents that triage and respond to messages, summarize +documents, route tickets, drive a data pipeline, generate reports, +or act on a user's behalf against an internal API or SaaS product. + +This document collects the engineering and security expectations +that apply when a CISA project builds or operates such an agent. It +complements - and does not replace - the project's domain-specific +requirements (privacy, accessibility, records management, sector- +specific regulation, customer-facing-product review, and so on). +Those domain requirements are *not* described here and must be +sourced from the appropriate authoritative documents and teams. + +For coding agents (agents whose output flows through the project's +repository, review, and CI controls), see [Agents](agents.md). For +broader context on responsible use of AI tools, see the +[AI-Assisted Development guide](../ai-assisted-development.md). For +how agents relate to Rules, Skills, and MCP servers, see this +directory's [README](README.md). + +## What this document covers ## + +- General engineering and security expectations that apply to any + agent built or operated within a CISA project. +- A construction checklist that should be filled in before an agent + is enabled in any shared environment. +- A catalog of risks that apply to agents regardless of output + channel. + +## What this document does *not* cover ## + +- Privacy, civil liberties, records management, accessibility, or + any other compliance regime specific to the agent's domain. +- Approval workflows for customer-facing or public-facing systems. +- Content policy for agents that produce text consumed by external + audiences. +- Procurement, vendor risk, and authority-to-operate (ATO) + processes. + +If any of the above apply, treat the engagement of the responsible +team as a prerequisite, not a follow-up. + +## When to build a non-coding agent ## + +Build a non-coding agent when: + +- The task is well-scoped, recurring, and benefits from automation + more than from one-shot assistance. +- The actions the agent will take are bounded, reviewable, and + reversible, or are gated on human approval before taking effect. +- The agent's effective permissions can be kept at or below what + the requesting user would be granted for the same task. +- The relevant domain teams have reviewed the use case. + +Do *not* build a non-coding agent when: + +- The task requires judgment the agent cannot reliably exercise - + notably anything that grants privileges, handles secrets, makes + benefits or eligibility determinations, or makes security- + relevant trade-offs. +- The agent would take actions that cannot be reverted easily, with + no human approval gate. +- The agent would handle data the project is not authorized to + process through a language model or an external service. + +## Standards ## + +- Agents act under a clearly identified principal. The human who + starts the agent, or the team that owns an autonomous agent's + service account, is accountable for everything it does. Service + accounts are owned, documented, and scoped to the minimum + permissions required. +- Agents must not be granted secrets, credentials, or production + access beyond what their narrowly defined task requires. Long- + lived, broadly scoped credentials are not appropriate for agent + use. +- Destructive, irreversible, or privilege-changing actions are + gated on explicit human confirmation. "Auto-approve everything" + modes are not used against shared systems or systems of record. +- Agent inputs and outputs are logged in a form that a reviewer can + inspect after the fact, subject to the project's existing data- + handling and retention rules. +- Agents do not bypass any control - access control, change + management, approval workflow, records retention, or audit + logging - that applies to human operators performing the same + task. +- Agent configuration - prompts, Rules, Skills, MCP servers, tools, + model selection, and any allow-lists or guardrails - is checked + in, reviewed in pull requests, and owned via `CODEOWNERS`. + +## Defaults ## + +- Start with the smallest agent that can do the job. Prefer a + human-in-the-loop design over an autonomous one unless the task + clearly warrants the latter. +- Scope agents to a single task type and a single set of systems. + Avoid agents whose role description is "do whatever the user + asks across any system". +- Run agents in an isolated execution environment (an ephemeral + container, a dedicated service, a sandboxed worker) with only + the credentials and network reach the task requires. +- Default to requiring human approval before the agent: + - Calls external services that have side effects. + - Writes to systems of record. + - Sends messages, tickets, or notifications to people outside + the team operating the agent. + - Accesses or transmits any data classified above public. +- Pin the agent's model, tool list, and MCP server set to known + versions. Treat changes to any of these as configuration changes + subject to review. +- Set explicit limits on runtime, request volume, token usage, and + cost. Monitor against those limits. + +## Suggestions ## + +- For recurring back-office work (categorizing documents, drafting + summaries, preparing reports), a narrowly scoped agent with a + clear success criterion and a human approval step can be a good + fit. +- Prefer designs in which the agent produces a draft artifact for a + human to approve, edit, and act on, rather than designs in which + the agent acts directly on systems of record. +- When the agent must call out to external systems, prefer an MCP + server or other reviewed integration over ad-hoc credentials in + the agent's environment. See [MCP](mcp.md). +- Periodically sample agent inputs, outputs, and logs even when + nothing has obviously gone wrong. Drift in behavior is easier to + catch early. +- Sunset agents that are no longer used. Idle agents with live + credentials are a quiet liability. + +## Constructing a non-coding agent ## + +When designing a new non-coding agent for a project, document the +following before it is enabled in any shared environment: + +- **Purpose.** One or two sentences describing what the agent is + for and what it is *not* for. +- **Domain review.** Which domain teams (privacy, accessibility, + records, legal, mission, security, etc.) have reviewed the use + case, and the outcome of that review. If none apply, say so + explicitly. +- **Trigger.** How the agent is started (a user action, a scheduled + job, an incoming message, a webhook, etc.). +- **Principal and permissions.** Whose credentials the agent uses, + what scopes those credentials carry, and where the credentials + are stored. +- **Data handled.** The categories of data the agent reads, writes, + or transmits, and the authority under which it does so. +- **Allowed actions.** The specific tools, endpoints, and systems + the agent is permitted to act against. +- **Forbidden actions.** Explicit out-of-scope items, especially + any destructive, privilege-changing, or externally visible + operations. +- **Inputs.** What context the agent receives - which prompts, + which Rules, which Skills, which MCP servers, which environment + variables, which user-supplied content. +- **Outputs.** What the agent produces (a message, a record, a + ticket, a file, a recommendation), who receives it, and how it + is reviewed or approved before it has effect. +- **Failure behavior.** What the agent does on error, timeout, + refusal, or contradiction. Default to stopping and reporting + rather than retrying aggressively. +- **Limits.** Runtime, request volume, token usage, cost, and any + rate limits the agent must respect. +- **Logging and retention.** What is logged, where it is stored, + who can access it, and for how long. +- **Owner.** A named team or individual responsible for the + agent's configuration and behavior. + +Keep this description short and check it in alongside the agent's +configuration so reviewers can evaluate changes against the stated +design. + +## Operating a non-coding agent ## + +- Start an agent with the narrowest task description that still + captures the work. Vague tasks produce vague, harder-to-review + results. +- Watch the first runs of any new or materially changed agent + configuration. Do not leave an agent unattended in a shared + environment until its behavior is understood. +- Treat the agent's output as untrusted input. Review it - or have + the receiving system enforce constraints - before it has effect. +- Stop the agent if it begins doing something unexpected, exceeds + configured limits, or operates against data it should not be + touching. Investigate before resuming. +- Rotate any credential the agent may have been exposed to if the + agent's environment, configuration, or upstream services are + compromised. + +## Risks specific to agents ## + +These apply regardless of the agent's output channel. + +- **Privilege amplification.** An agent with broad credentials and + the ability to call tools can do far more damage than a one-shot + assistant. Keep credentials and tools tightly scoped. +- **Prompt injection.** Content the agent reads - messages, + documents, tickets, MCP server responses, web pages, user input + - can contain instructions intended to redirect the agent. Treat + all such content as untrusted. +- **Tool misuse.** An agent may call a tool in a way the project + did not anticipate. Constrain tool inputs where the tool itself + supports it, and review the actions the agent actually performs. +- **Confident, irreversible action.** Agents will sometimes act + decisively on incorrect conclusions. Reversibility and human + approval gates are the primary defenses. +- **Data leakage.** Inputs, retrieved context, and tool outputs + can leave the project's environment via the model provider, MCP + servers, or logs. Verify the data path for each before enabling + the agent. +- **Cost and resource exhaustion.** Long-running agents can + consume significant compute, API quota, or vendor spend. Set + explicit limits and monitor usage. +- **Drift.** Model updates, tool updates, and MCP server updates + can change agent behavior without any change to the project's + own configuration. Pin where possible and re-evaluate + periodically. + +## Limitations ## + +The guidance above is engineering and security baseline guidance. +It does not, by itself, make an agent appropriate for any specific +domain, audience, or data classification. Domain-specific +requirements - privacy, accessibility, records management, sector +regulation, customer-facing-product review, and so on - apply on +top of this baseline and must be sourced from the appropriate +authoritative documents and teams. When in doubt, prefer the more +conservative option: smaller scope, narrower permissions, more +human approval, and smaller, more auditable steps. diff --git a/ai/rules.md b/ai/rules.md new file mode 100644 index 0000000..5165389 --- /dev/null +++ b/ai/rules.md @@ -0,0 +1,111 @@ +# Rules for AI Agents # + +Rules are persistent, repository-scoped instructions that shape how an +AI-assisted development tool behaves while working in a project. They +are typically loaded automatically when the tool opens the workspace +and apply across a session. + +Common forms include `AGENTS.md` at the repository root (read by many +agent tools), `.cursor/rules/*.mdc` for Cursor, and similar +tool-specific files. The exact filename varies, but the engineering +expectations below do not. + +For broader context on responsible use of AI tools, see the +[AI-Assisted Development guide](../ai-assisted-development.md). For an +overview of how Rules relate to Skills, see this directory's +[README](README.md). + +## When to use a Rule ## + +Use a Rule when the guidance is: + +- Repository-wide or scoped to a clear subdirectory. +- Long-lived. The behavior should persist across sessions and + contributors. +- Concise enough to read in a single sitting. Rules are read by both + humans and models; both benefit from brevity. + +Do *not* use a Rule for: + +- One-off task instructions. Put those in the prompt. +- Anything secret or non-public. Rules live in the repository and are + sent to the model. +- Project policy that requires legal review. Keep policy in the + documents that already govern it and reference them from the Rule. + +## Standards ## + +- Rules are checked in, reviewed in pull requests, and owned via + `CODEOWNERS`. +- Rules must not contain secrets, credentials, tokens, customer data, + internal hostnames, or any non-public information. +- Rules are advisory to the model. They do not substitute for code + review, CI checks, branch protection, or the project's existing + access controls. +- Every Rule states, near the top, what it is for and when it + applies. + +## Defaults ## + +- Prefer a single `AGENTS.md` at the repository root as the primary + source of truth when the project uses more than one AI tool. +- For tool-specific Rules, use the conventional location expected by + the tool (for example, `.cursor/rules/` for Cursor) rather than + inventing a new one. +- Keep individual Rule files short - generally under a page of + rendered Markdown. Split by topic when they grow. +- Use plain, declarative language. Avoid hedging, marketing tone, or + open-ended aspirations. +- Link out to existing documentation (such as the + [Style Guide](../style-guide.md) or [CONTRIBUTING](../CONTRIBUTING.md)) + rather than restating it. + +## Suggestions ## + +- Organize larger rule sets by topic: one file for code style, one for + testing expectations, one for security guardrails, and so on. +- Include a short "When this applies" section at the top of each Rule + so the model and reviewers can quickly decide whether it is + relevant. +- Periodically re-read each Rule and remove anything that no longer + reflects current practice. Stale Rules quietly degrade output + quality. + +## What to put in a Rule ## + +Useful content includes: + +- Pointers to the project's style, testing, and contribution guides. +- Repository-specific conventions that are not obvious from the code + (for example, "tests live next to the module they test, not in a + top-level `tests/` directory"). +- Guardrails against known failure modes (for example, "do not modify + generated files under `vendor/`"). +- Explicit out-of-scope statements (for example, "do not add new + third-party dependencies without an issue and review"). + +Avoid: + +- Long prose explanations of *why* a convention exists. Link to the + authoritative document instead. +- Conflicting guidance between Rules. Resolve conflicts in the source + files, not by hoping the model picks the right one. +- Instructions that depend on information the model cannot verify + (for example, "always check the latest internal wiki page"). + +## Review expectations ## + +- Treat Rule changes like code changes. Require review, and prefer + small, focused pull requests. +- When a Rule changes behavior in a way reviewers should know about, + call it out in the pull request description. +- If a Rule is added to work around a recurring AI failure mode, + document the failure briefly so future maintainers understand why + the Rule exists. + +## Limitations ## + +Rules are not a security control. A model may ignore, misinterpret, +or be argued out of a Rule. They reduce variance and make agent +behavior more predictable, but the human contributor remains +responsible for every change that is committed. diff --git a/ai/skills.md b/ai/skills.md new file mode 100644 index 0000000..2618e60 --- /dev/null +++ b/ai/skills.md @@ -0,0 +1,112 @@ +# Skills for AI Agents # + +Skills are reusable, task-scoped capability packages that an AI agent +can load on demand. A Skill typically bundles a short instruction file +(often `SKILL.md`) with optional supporting scripts, templates, or +reference material. The agent selects a Skill based on the user's +request and follows the instructions inside it for that task. + +Skills differ from [Rules](rules.md) in scope and lifetime. Rules +apply broadly and persistently; Skills apply narrowly and only when +invoked. + +For broader context on responsible use of AI tools, see the +[AI-Assisted Development guide](../ai-assisted-development.md). + +## When to use a Skill ## + +Use a Skill when the work is: + +- A recurring, well-defined task (for example, "scaffold a new Python + module following the project skeleton" or "draft a release note + from a list of merged pull requests"). +- Worth standardizing so that every contributor and agent performs it + the same way. +- Specific enough that a focused instruction set produces meaningfully + better results than a general-purpose prompt. + +Do *not* use a Skill for: + +- Project-wide conventions that should always apply. Those belong in + a [Rule](rules.md). +- One-off tasks. Write them as a prompt instead. +- Anything that requires secrets, credentials, or non-public data at + authoring time. Skills are checked into the repository. + +## Standards ## + +- Skills are checked in, reviewed in pull requests, and owned via + `CODEOWNERS`. +- Skills must not contain secrets, credentials, tokens, customer + data, internal hostnames, or any non-public information. +- Each Skill has a clear, narrow purpose stated at the top of its + `SKILL.md` (or equivalent entry file). +- Any scripts or commands invoked by a Skill follow the same security + and review expectations as any other code in the repository. + +## Defaults ## + +- Store Skills in the conventional location expected by the tool in + use (for example, `.claude/skills//` for Claude Code, + or the location documented by whatever tool the project has + standardized on). +- Give each Skill its own directory. Keep the entry file short and + put supporting material in sibling files. +- Begin each Skill with a short description of: + - What the Skill does. + - When it should be used. + - When it should *not* be used. +- Prefer Skills that produce small, reviewable changes. A Skill that + rewrites large portions of the codebase in one step is hard to + audit. + +## Suggestions ## + +- Keep `SKILL.md` focused on instructions for the agent. Put + human-facing rationale in a sibling `README.md` if needed. +- When a Skill wraps a shell command or script, prefer invoking an + existing, reviewed script in the repository over embedding new + logic in the Skill file. +- Include a brief "Validation" section in the Skill describing how + the contributor should verify the result (for example, "run + `pre-commit run --all-files` and the project test suite"). +- Version Skills with the repository. Avoid pinning Skills to + external sources that can change without review. + +## What to put in a Skill ## + +Useful content includes: + +- A precise description of the task the Skill performs. +- The inputs the agent should gather from the user before acting. +- The concrete steps to perform, in order. +- The files or directories the Skill is permitted to modify. +- Validation steps the contributor should run before committing. +- Explicit out-of-scope statements. + +Avoid: + +- Vague aspirational language ("write great code"). Be specific. +- Embedded credentials, tokens, or environment-specific URLs. +- Instructions that require the agent to make autonomous decisions + about security-sensitive code without human review. + +## Review expectations ## + +- Treat new Skills and changes to existing Skills as code changes. + Require review, and prefer small, focused pull requests. +- When a Skill invokes scripts, links to external resources, or + modifies sensitive paths, call that out in the pull request + description so reviewers can scrutinize the relevant surface. +- Remove Skills that are no longer used. Unmaintained Skills produce + inconsistent, sometimes wrong, results. + +## Limitations ## + +A Skill influences what the agent attempts; it does not guarantee the +agent will follow the instructions correctly. Every change produced +by a Skill is still untrusted input until reviewed by a human, run +through the project's tests, and passed through the existing CI +checks. See the +[AI-Assisted Development guide](../ai-assisted-development.md) for the +full set of expectations that apply to AI-generated changes.