From faeabc7df67e014c7d96acb9808c2114da4e72c1 Mon Sep 17 00:00:00 2001 From: Beon de Nood Date: Sat, 2 May 2026 14:43:02 -0400 Subject: [PATCH 1/2] =?UTF-8?q?fix(docs):=20fix=20Di=C3=A1taxis=20mode=20v?= =?UTF-8?q?iolations?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit P2-3: Extract reference-mode content from concept and how-to pages. - concepts/validation.md: Was a full reference doc (error codes, internals, output formats). Replaced with a proper concept page explaining what validation is and why it matters. Full reference moved to reference/cli/validation.md. - how-to/security/policy-enforcement.md: Configuration reference table and PDP request/response format extracted to new reference/server/policy-enforcement.md. How-to now links to reference for non-procedural details. - mkdocs.yml: Added nav entries for new reference pages. --- docs/concepts/validation.md | 427 ++------------------ docs/how-to/security/policy-enforcement.md | 70 +--- docs/reference/cli/validation.md | 420 +++++++++++++++++++ docs/reference/server/policy-enforcement.md | 110 +++++ mkdocs.yml | 2 + 5 files changed, 572 insertions(+), 457 deletions(-) create mode 100644 docs/reference/cli/validation.md create mode 100644 docs/reference/server/policy-enforcement.md diff --git a/docs/concepts/validation.md b/docs/concepts/validation.md index 8dc4987..99b24da 100644 --- a/docs/concepts/validation.md +++ b/docs/concepts/validation.md @@ -1,420 +1,61 @@ -# 🔍 Validation Process Documentation +# Validation -> **Comprehensive guide to the CapiscIO CLI validation system** +Validation is the process of checking an agent card against the A2A protocol specification and CapiscIO's trust requirements. It answers the question: *"Is this agent correctly described and ready for production?"* -## What Gets Validated? - -**The Challenge:** The A2A protocol specification is comprehensive, and manually checking every requirement is time-consuming and error-prone. - -**Our Solution:** Automated validation across 7+ categories: - -- ✅ Input resolution and discovery -- ✅ HTTP client and network validation -- ✅ Schema and structure compliance -- ✅ Version compatibility analysis -- ✅ Protocol adherence -- ✅ Security and authentication -- ✅ Endpoint availability (optional) - -This document provides an exhaustive overview of everything included in the A2A agent card validation process. - -## 📚 Table of Contents - -- [Input Resolution & Discovery](#input-resolution-discovery) -- [HTTP Client & Network Validation](#http-client-network-validation) -- [Schema Validation](#schema-validation) -- [Version Compatibility Analysis](#version-compatibility-analysis) -- [Validation Modes](#validation-modes) -- [Advanced Feature Detection](#advanced-feature-detection) -- [Output & Reporting](#output-reporting) -- [Error Codes Reference](#error-codes-reference) -- [Performance & Timing](#performance-timing) - -## Input Resolution & Discovery - -### Auto-Detection Process - -When no input is specified, the CLI searches for agent cards in these locations (in order): - -``` -./agent-card.json (preferred) -./.well-known/agent-card.json -./src/agent-card.json -./public/.well-known/agent-card.json -./dist/.well-known/agent-card.json -./agent-card.json (legacy support) -./.well-known/agent-card.json (legacy support) -``` - -### URL Handling - -Agent cards can be hosted at **any URL**. CapiscIO validates whatever URL you provide: - -```bash -# Direct URL to agent card (most common) -capiscio validate https://my-agent.example.com/agent-card.json - -# Well-known path (for discovery) -capiscio validate https://example.com/.well-known/agent-card.json - -# API-style paths (multi-tenant hosting) -capiscio validate https://api.example.com/agents/weather/agent-card.json -``` - -If only a domain is provided, CapiscIO will attempt discovery: - -1. **Direct URL validation**: Tests if URL directly contains agent card JSON -2. **Well-known endpoint discovery**: Tries `/.well-known/agent-card.json` (A2A recommended for public discovery) -3. **Legacy endpoint fallback**: Falls back to `/.well-known/agent-card.json` (legacy) -4. **Protocol inference**: Automatically adds `https://` if missing - -!!! tip "Multi-agent hosting" - When hosting multiple agents on one domain, use direct URLs like `/agents/{name}/agent-card.json` rather than the well-known path. - -### File Processing - -- JSON parsing with detailed error messages -- UTF-8 encoding support -- Validation of JSON structure before processing -- Graceful error handling for malformed files - -## HTTP Client & Network Validation - -### Request Configuration - -```typescript -{ - method: 'GET', - headers: { - 'Accept': 'application/json', - 'User-Agent': 'capiscio/2.0.0' - }, - timeout: 10000 // configurable -} -``` - -### Error Handling - -| Error Type | Code | Description | -|------------|------|-------------| -| HTTP 400 | `BAD_REQUEST` | Invalid request format | -| HTTP 401 | `UNAUTHORIZED` | Authentication required | -| HTTP 403 | `FORBIDDEN` | Access denied | -| HTTP 404 | `NOT_FOUND` | Agent card not found | -| HTTP 408 | `TIMEOUT` | Request timeout | -| HTTP 429 | `RATE_LIMITED` | Too many requests | -| HTTP 500 | `INTERNAL_SERVER_ERROR` | Server error | -| HTTP 502 | `BAD_GATEWAY` | Gateway error | -| HTTP 503 | `SERVICE_UNAVAILABLE` | Service unavailable | -| HTTP 504 | `GATEWAY_TIMEOUT` | Gateway timeout | -| Network | `ENOTFOUND` | Domain not found | -| Network | `ECONNREFUSED` | Connection refused | -| Network | `NETWORK_ERROR` | General network error | - -## Schema Validation - -### Required Fields (A2A v{{ protocol_version }}) - -#### Basic Required Fields -- `name`: Agent display name -- `description`: Agent description -- `url`: Agent base URL -- `provider`: Provider information object -- `version`: Agent version (semver format) - -#### A2A Protocol Required Fields -- `protocolVersion`: A2A protocol version (e.g., "{{ protocol_version }}") -- `preferredTransport`: Primary transport protocol - -#### Provider Structure -```typescript -{ - "provider": { - "organization": "string", // Required - "url": "string" // Optional - } -} -``` - -### Skills Validation - -Each skill in the `skills` array must have: -- `id`: Unique skill identifier -- `examples`: Array of string examples (if present) - -### Version Format Validation - -- **Agent Version**: Must follow semantic versioning (semver) -- **Protocol Version**: Must be valid A2A protocol version - -## Version Compatibility Analysis +--- -### Feature-Version Requirements +## What Gets Validated -| Feature | Minimum Protocol Version | Description | -|---------|-------------------------|-------------| -| `capabilities.pushNotifications` | 0.3.0 | Push notification support | -| `additionalInterfaces` | 0.3.0 | Multiple interface support | -| `capabilities.streaming` | 0.3.0 | Streaming capability | +The CapiscIO CLI validates agent cards across seven categories: -### Compatibility Checks +| Category | What It Checks | +|----------|---------------| +| **Schema** | Required fields, types, structure per A2A spec | +| **Discovery** | Whether the card can be fetched from `/.well-known/agent.json` | +| **Network** | HTTPS, response times, status codes | +| **Version Compatibility** | Feature usage matches declared A2A protocol version | +| **Security** | Authentication schemes, key presence | +| **Endpoint Availability** | Whether declared URLs respond (optional) | +| **Registry Readiness** | Additional fields needed for CapiscIO registration | -1. **Protocol Version Detection**: Identifies declared protocol version -2. **Feature Analysis**: Checks used features against declared version -3. **Mismatch Detection**: Identifies incompatible feature-version combinations -4. **Migration Suggestions**: Provides upgrade recommendations +--- ## Validation Modes -### Progressive Mode (Default) - -=== "Command" - - ```bash - capiscio validate agent.json --progressive - ``` +### Progressive (Default) -=== "Output" +Reports all findings without stopping at the first error. Suitable for iterative development — you see everything that needs fixing in one pass. - ```ansi - ✅ A2A AGENT VALIDATION PASSED - - Mode: Progressive (balanced) - - ┌───────────────────────────────────────┐ - │ Score: 85/100 │ - │ Checks: 12 passed, 0 failed │ - └───────────────────────────────────────┘ +### Strict - ✓ Core A2A requirements met - ⚠️ 2 suggestions for improvement - - 💡 Suggestions: - - Add authentication scheme for production - - Include skill examples for better discovery - ``` +Treats warnings as errors. Required for production deployments and CI/CD pipelines where partial compliance is unacceptable. -**Characteristics:** -- Balanced validation approach -- Core A2A protocol requirements enforced -- Warnings for deprecated features -- Suggestions for best practices -- Permissive of emerging features - -**Use Cases:** -- Development and testing -- Continuous integration -- General-purpose validation - -### Strict Mode - -=== "Command" - - ```bash - capiscio validate agent.json --strict - ``` - -=== "Output (Pass)" - - ```ansi - ✅ STRICT VALIDATION PASSED - - Mode: Strict (production-ready) - - ┌───────────────────────────────────────┐ - │ Score: 100/100 │ - │ Checks: 15 passed, 0 failed │ - └───────────────────────────────────────┘ - - ✓ Full A2A protocol compliance - ✓ Security scheme verified - ✓ All endpoints accessible - - 🚀 Agent is production ready! - ``` - -=== "Output (Fail)" - - ```ansi - ❌ STRICT VALIDATION FAILED - - Mode: Strict (production-ready) - - ┌───────────────────────────────────────┐ - │ Score: 72/100 (need ≥95) │ - │ Checks: 10 passed, 2 failed │ - └───────────────────────────────────────┘ - - ❌ MISSING_AUTH: No authentication scheme - ❌ ENDPOINT_UNREACHABLE: /a2a returned 404 - - Exit code: 1 - ``` - -**Characteristics:** -- Full A2A protocol compliance required -- All endpoints must be accessible -- Security schemes required -- Complete metadata validation -- Zero tolerance for deprecated features - -**Use Cases:** -- Production deployment -- Registry submission -- Compliance verification - -## Advanced Feature Detection - -### Legacy Endpoint Handling - -**Detection Logic:** -1. Check if URL uses `.well-known/agent-card.json` (legacy) -2. Generate warning if legacy endpoint detected -3. Suggest migration to `.well-known/agent-card.json` - -**Warning Generated:** -``` -LEGACY_DISCOVERY_ENDPOINT: Agent discovered via legacy endpoint. -The A2A v{{ protocol_version }} specification recommends using /.well-known/agent-card.json -``` - -### Transport Protocol Validation - -**Supported Protocols:** -- `JSONRPC`: JSON-RPC transport -- `GRPC`: gRPC transport -- `HTTP+JSON`: HTTP with JSON payloads - -**Cross-Validation:** -- gRPC transport should have streaming capabilities -- Transport protocols must match interface declarations - -## Output & Reporting - -### Console Output Structure - -``` -✅ A2A AGENT VALIDATION PASSED -Agent: https://example.com/.well-known/agent-card.json -Score: 100/100 -Version: {{ protocol_version }} (Strictness: progressive) +--- -🔍 VALIDATION SUMMARY: - 📊 3 checks performed: 3 passed, 0 failed, 0 warnings - ⏱️ Completed in 245ms +## Trust Relationship -🔍 VALIDATIONS PERFORMED: -✅ Schema Validation - Agent card structure is valid - Duration: 12ms -✅ Endpoint Connectivity - All endpoints are accessible and responding - Duration: 195ms -✅ A2A v{{ protocol_version }} Features - All v{{ protocol_version }} features are properly configured +Validation is the first step in the CapiscIO trust pipeline: -🏆 Perfect! Your agent passes all validations. -🚀 Your agent is ready for deployment! ``` - -### JSON Output Structure - -```json -{ - "success": true, - "score": 100, - "errors": [], - "warnings": [], - "suggestions": [], - "validations": [ - { - "id": "schema_validation", - "name": "Schema Validation", - "status": "passed", - "message": "Agent card conforms to A2A v{{ protocol_version }} schema", - "duration": 12, - "details": "Agent card structure is valid" - } - ], - "versionInfo": { - "detectedVersion": "{{ protocol_version }}", - "validatorVersion": "{{ protocol_version }}", - "strictness": "progressive", - "compatibility": { - "compatible": true, - "mismatches": [], - "suggestions": [] - } - } -} +Validate → Register → Badge → Enforce ``` -## Error Codes Reference - -### Schema Validation Errors - -| Code | Description | Severity | Fixable | -|------|-------------|----------|----------| -| `SCHEMA_VALIDATION_ERROR` | Missing or invalid required field | Error | Yes | -| `VERSION_MISMATCH_ERROR` | Protocol version conflicts | Error | Yes | -| `STRICT_VERSION_MISMATCH` | Strict mode version issues | Error | Yes | - -### Network & Discovery Errors - -| Code | Description | Severity | Fixable | -|------|-------------|----------|----------| -| `VALIDATION_FAILED` | General validation failure | Error | Depends | -| `ENDPOINT_UNREACHABLE` | HTTP connection issues | Error | Yes | -| `LEGACY_DISCOVERY_ENDPOINT` | Using legacy endpoint | Warning | Yes | - -### Feature Compatibility Warnings +An agent that fails validation cannot register with the CapiscIO server and therefore cannot obtain a trust badge. Higher trust levels (2+) require stricter validation results. -| Code | Description | Severity | Fixable | -|------|-------------|----------|----------| -| `VERSION_FEATURE_MISMATCH` | Feature requires newer protocol | Warning | Yes | -| `GRPC_WITHOUT_STREAMING` | gRPC without streaming capability | Warning | Yes | - -## Performance & Timing - -### Metrics Collected - -- **Individual Validation Steps**: Timing for each validation component -- **Total Duration**: Complete validation time -- **Network Requests**: HTTP request timing -- **Schema Processing**: JSON parsing and validation time - -### Optimization Features - -- **Schema-Only Mode**: Skip network calls with `--schema-only` -- **Timeout Configuration**: Custom timeout with `--timeout ` -- **Efficient Error Handling**: Early termination on critical failures +--- -### Performance Benchmarks +## When to Validate -| Operation | Typical Duration | Notes | -|-----------|------------------|-------| -| Schema Validation | 1-50ms | Depends on agent card size | -| Network Request | 50-1000ms | Depends on network latency | -| Version Analysis | <1ms | Cached semver operations | -| Total Validation | 100-2000ms | Varies by mode and network | +- **During development** — catch schema errors before deployment +- **In CI/CD** — gate deployments on validation passing (`--strict`) +- **At registration** — the server re-validates before accepting an agent +- **Periodically** — detect drift as agent capabilities change --- ## See Also -- **[Scoring System](scoring.md)** - Understand how validation results translate to scores -- **[CLI Reference](../reference/cli/index.md)** - Command-line validation reference -- **[Python SDK](../reference/sdk-python/index.md)** - Runtime protection for production agents - -!!! tip "Production Deployment" - The CapiscIO CLI (`capiscio`) validates agent cards during development and CI/CD. For runtime protection, use [CapiscIO Python SDK](../reference/sdk-python/index.md). - -## Contributing to Validation - -See the core implementation repository (Go) at https://github.com/capiscio/capiscio-core for information on extending the validation system. - -## Related Documentation - -- [CLI Usage Guide](../reference/cli/index.md) -- [CLI Reference](../reference/cli/index.md) -- [A2A Protocol Specification](https://github.com/a2aproject/A2A){:target="_blank"} +- [Validation Process Reference](../reference/cli/validation.md) — Full details on error codes, modes, and output formats +- [How-To: Validate from URL](../how-to/validation/validate-url.md) — Validate a remote agent +- [How-To: Strict Mode](../how-to/validation/strict-mode.md) — Enforce strict validation +- [Scoring](scoring.md) — How validation results affect trust scores diff --git a/docs/how-to/security/policy-enforcement.md b/docs/how-to/security/policy-enforcement.md index 12c21af..0d4d13c 100644 --- a/docs/how-to/security/policy-enforcement.md +++ b/docs/how-to/security/policy-enforcement.md @@ -205,70 +205,6 @@ For emergency access when the PDP is down or misconfigured: --- -## Configuration Reference - -All PDP-related environment variables: - -| Variable | Default | Description | -|----------|---------|-------------| -| `CAPISCIO_EMBEDDED_PDP` | `false` | Enable embedded OPA evaluator (in-process PDP) | -| `CAPISCIO_PDP_ENDPOINT` | _(empty)_ | External PDP URL. Empty + no embedded PDP = badge-only mode | -| `CAPISCIO_PDP_TIMEOUT_MS` | `500` | External PDP query timeout in milliseconds | -| `CAPISCIO_ENFORCEMENT_MODE` | `EM-OBSERVE` | One of: `EM-OBSERVE`, `EM-GUARD`, `EM-DELEGATE`, `EM-STRICT` | -| `CAPISCIO_WORKSPACE` | _(empty)_ | Workspace/tenant UUID (required for embedded PDP) | -| `CAPISCIO_BUNDLE_POLL_INTERVAL` | `30s` | Embedded PDP bundle rebuild interval | -| `CAPISCIO_BUNDLE_STALENESS_THRESHOLD` | `5m` | Embedded PDP bundle age before staleness warnings | -| `CAPISCIO_BREAKGLASS_PUBLIC_KEY` | _(empty)_ | Path to break-glass Ed25519 public key file | -| `CAPISCIO_PEP_ID` | _(empty)_ | PEP instance identifier (sent to PDP as `X-Capiscio-PEP-ID`) | - ---- - -## PDP Request Format - -The PEP sends a JSON POST to the PDP endpoint. Your PDP must accept this format: - -```jsonc -{ - "pip_version": "capiscio.pip.v1", - "subject": { - "did": "did:web:agent.example.com", - "badge_jti": "badge-uuid", - "ial": "1", - "trust_level": "DV" // Badge trust level code: SS(0), REG(1), DV(2), OV(3), EV(4) - }, - "action": { - "operation": "POST /v1/badges", - "capability_class": null - }, - "resource": { - "identifier": "/v1/badges" - }, - "context": { - "txn_id": "txn-uuid", - "enforcement_mode": "EM-GUARD" - }, - "environment": { - "workspace": "00000000-0000-0000-0000-000000000000", - "pep_id": "server-01", - "time": "2026-03-01T12:00:00Z" - } -} -``` - -And return: - -```json -{ - "decision": "ALLOW", - "decision_id": "eval-uuid", - "obligations": [], - "reason": "Policy matched: allow-trusted-agents", - "ttl": 300 -} -``` - ---- - ## Verification Confirm policy enforcement is active: @@ -281,3 +217,9 @@ curl -v https://your-server/v1/agents \ # Look for Server-Timing header with policy timing # Server-Timing: capiscio-auth;dur=0.6, capiscio-policy;dur=12.3 ``` + +--- + +## Reference + +For full configuration details, PDP request/response format, and telemetry field definitions, see the [Policy Enforcement Reference](../../reference/server/policy-enforcement.md). diff --git a/docs/reference/cli/validation.md b/docs/reference/cli/validation.md new file mode 100644 index 0000000..7b09d46 --- /dev/null +++ b/docs/reference/cli/validation.md @@ -0,0 +1,420 @@ +# Validation Process Reference + +> Complete reference for the CapiscIO CLI validation system — error codes, modes, output formats, and internals. + +## What Gets Validated? + +**The Challenge:** The A2A protocol specification is comprehensive, and manually checking every requirement is time-consuming and error-prone. + +**Our Solution:** Automated validation across 7+ categories: + +- ✅ Input resolution and discovery +- ✅ HTTP client and network validation +- ✅ Schema and structure compliance +- ✅ Version compatibility analysis +- ✅ Protocol adherence +- ✅ Security and authentication +- ✅ Endpoint availability (optional) + +This document provides an exhaustive overview of everything included in the A2A agent card validation process. + +## 📚 Table of Contents + +- [Input Resolution & Discovery](#input-resolution-discovery) +- [HTTP Client & Network Validation](#http-client-network-validation) +- [Schema Validation](#schema-validation) +- [Version Compatibility Analysis](#version-compatibility-analysis) +- [Validation Modes](#validation-modes) +- [Advanced Feature Detection](#advanced-feature-detection) +- [Output & Reporting](#output-reporting) +- [Error Codes Reference](#error-codes-reference) +- [Performance & Timing](#performance-timing) + +## Input Resolution & Discovery + +### Auto-Detection Process + +When no input is specified, the CLI searches for agent cards in these locations (in order): + +``` +./agent-card.json (preferred) +./.well-known/agent-card.json +./src/agent-card.json +./public/.well-known/agent-card.json +./dist/.well-known/agent-card.json +./agent-card.json (legacy support) +./.well-known/agent-card.json (legacy support) +``` + +### URL Handling + +Agent cards can be hosted at **any URL**. CapiscIO validates whatever URL you provide: + +```bash +# Direct URL to agent card (most common) +capiscio validate https://my-agent.example.com/agent-card.json + +# Well-known path (for discovery) +capiscio validate https://example.com/.well-known/agent-card.json + +# API-style paths (multi-tenant hosting) +capiscio validate https://api.example.com/agents/weather/agent-card.json +``` + +If only a domain is provided, CapiscIO will attempt discovery: + +1. **Direct URL validation**: Tests if URL directly contains agent card JSON +2. **Well-known endpoint discovery**: Tries `/.well-known/agent-card.json` (A2A recommended for public discovery) +3. **Legacy endpoint fallback**: Falls back to `/.well-known/agent-card.json` (legacy) +4. **Protocol inference**: Automatically adds `https://` if missing + +!!! tip "Multi-agent hosting" + When hosting multiple agents on one domain, use direct URLs like `/agents/{name}/agent-card.json` rather than the well-known path. + +### File Processing + +- JSON parsing with detailed error messages +- UTF-8 encoding support +- Validation of JSON structure before processing +- Graceful error handling for malformed files + +## HTTP Client & Network Validation + +### Request Configuration + +```typescript +{ + method: 'GET', + headers: { + 'Accept': 'application/json', + 'User-Agent': 'capiscio/2.0.0' + }, + timeout: 10000 // configurable +} +``` + +### Error Handling + +| Error Type | Code | Description | +|------------|------|-------------| +| HTTP 400 | `BAD_REQUEST` | Invalid request format | +| HTTP 401 | `UNAUTHORIZED` | Authentication required | +| HTTP 403 | `FORBIDDEN` | Access denied | +| HTTP 404 | `NOT_FOUND` | Agent card not found | +| HTTP 408 | `TIMEOUT` | Request timeout | +| HTTP 429 | `RATE_LIMITED` | Too many requests | +| HTTP 500 | `INTERNAL_SERVER_ERROR` | Server error | +| HTTP 502 | `BAD_GATEWAY` | Gateway error | +| HTTP 503 | `SERVICE_UNAVAILABLE` | Service unavailable | +| HTTP 504 | `GATEWAY_TIMEOUT` | Gateway timeout | +| Network | `ENOTFOUND` | Domain not found | +| Network | `ECONNREFUSED` | Connection refused | +| Network | `NETWORK_ERROR` | General network error | + +## Schema Validation + +### Required Fields (A2A v{{ protocol_version }}) + +#### Basic Required Fields +- `name`: Agent display name +- `description`: Agent description +- `url`: Agent base URL +- `provider`: Provider information object +- `version`: Agent version (semver format) + +#### A2A Protocol Required Fields +- `protocolVersion`: A2A protocol version (e.g., "{{ protocol_version }}") +- `preferredTransport`: Primary transport protocol + +#### Provider Structure +```typescript +{ + "provider": { + "organization": "string", // Required + "url": "string" // Optional + } +} +``` + +### Skills Validation + +Each skill in the `skills` array must have: +- `id`: Unique skill identifier +- `examples`: Array of string examples (if present) + +### Version Format Validation + +- **Agent Version**: Must follow semantic versioning (semver) +- **Protocol Version**: Must be valid A2A protocol version + +## Version Compatibility Analysis + +### Feature-Version Requirements + +| Feature | Minimum Protocol Version | Description | +|---------|-------------------------|-------------| +| `capabilities.pushNotifications` | 0.3.0 | Push notification support | +| `additionalInterfaces` | 0.3.0 | Multiple interface support | +| `capabilities.streaming` | 0.3.0 | Streaming capability | + +### Compatibility Checks + +1. **Protocol Version Detection**: Identifies declared protocol version +2. **Feature Analysis**: Checks used features against declared version +3. **Mismatch Detection**: Identifies incompatible feature-version combinations +4. **Migration Suggestions**: Provides upgrade recommendations + +## Validation Modes + +### Progressive Mode (Default) + +=== "Command" + + ```bash + capiscio validate agent.json --progressive + ``` + +=== "Output" + + ```ansi + ✅ A2A AGENT VALIDATION PASSED + + Mode: Progressive (balanced) + + ┌───────────────────────────────────────┐ + │ Score: 85/100 │ + │ Checks: 12 passed, 0 failed │ + └───────────────────────────────────────┘ + + ✓ Core A2A requirements met + ⚠️ 2 suggestions for improvement + + 💡 Suggestions: + - Add authentication scheme for production + - Include skill examples for better discovery + ``` + +**Characteristics:** +- Balanced validation approach +- Core A2A protocol requirements enforced +- Warnings for deprecated features +- Suggestions for best practices +- Permissive of emerging features + +**Use Cases:** +- Development and testing +- Continuous integration +- General-purpose validation + +### Strict Mode + +=== "Command" + + ```bash + capiscio validate agent.json --strict + ``` + +=== "Output (Pass)" + + ```ansi + ✅ STRICT VALIDATION PASSED + + Mode: Strict (production-ready) + + ┌───────────────────────────────────────┐ + │ Score: 100/100 │ + │ Checks: 15 passed, 0 failed │ + └───────────────────────────────────────┘ + + ✓ Full A2A protocol compliance + ✓ Security scheme verified + ✓ All endpoints accessible + + 🚀 Agent is production ready! + ``` + +=== "Output (Fail)" + + ```ansi + ❌ STRICT VALIDATION FAILED + + Mode: Strict (production-ready) + + ┌───────────────────────────────────────┐ + │ Score: 72/100 (need ≥95) │ + │ Checks: 10 passed, 2 failed │ + └───────────────────────────────────────┘ + + ❌ MISSING_AUTH: No authentication scheme + ❌ ENDPOINT_UNREACHABLE: /a2a returned 404 + + Exit code: 1 + ``` + +**Characteristics:** +- Full A2A protocol compliance required +- All endpoints must be accessible +- Security schemes required +- Complete metadata validation +- Zero tolerance for deprecated features + +**Use Cases:** +- Production deployment +- Registry submission +- Compliance verification + +## Advanced Feature Detection + +### Legacy Endpoint Handling + +**Detection Logic:** +1. Check if URL uses `.well-known/agent-card.json` (legacy) +2. Generate warning if legacy endpoint detected +3. Suggest migration to `.well-known/agent-card.json` + +**Warning Generated:** +``` +LEGACY_DISCOVERY_ENDPOINT: Agent discovered via legacy endpoint. +The A2A v{{ protocol_version }} specification recommends using /.well-known/agent-card.json +``` + +### Transport Protocol Validation + +**Supported Protocols:** +- `JSONRPC`: JSON-RPC transport +- `GRPC`: gRPC transport +- `HTTP+JSON`: HTTP with JSON payloads + +**Cross-Validation:** +- gRPC transport should have streaming capabilities +- Transport protocols must match interface declarations + +## Output & Reporting + +### Console Output Structure + +``` +✅ A2A AGENT VALIDATION PASSED +Agent: https://example.com/.well-known/agent-card.json +Score: 100/100 +Version: {{ protocol_version }} (Strictness: progressive) + +🔍 VALIDATION SUMMARY: + 📊 3 checks performed: 3 passed, 0 failed, 0 warnings + ⏱️ Completed in 245ms + +🔍 VALIDATIONS PERFORMED: +✅ Schema Validation + Agent card structure is valid + Duration: 12ms +✅ Endpoint Connectivity + All endpoints are accessible and responding + Duration: 195ms +✅ A2A v{{ protocol_version }} Features + All v{{ protocol_version }} features are properly configured + +🏆 Perfect! Your agent passes all validations. +🚀 Your agent is ready for deployment! +``` + +### JSON Output Structure + +```json +{ + "success": true, + "score": 100, + "errors": [], + "warnings": [], + "suggestions": [], + "validations": [ + { + "id": "schema_validation", + "name": "Schema Validation", + "status": "passed", + "message": "Agent card conforms to A2A v{{ protocol_version }} schema", + "duration": 12, + "details": "Agent card structure is valid" + } + ], + "versionInfo": { + "detectedVersion": "{{ protocol_version }}", + "validatorVersion": "{{ protocol_version }}", + "strictness": "progressive", + "compatibility": { + "compatible": true, + "mismatches": [], + "suggestions": [] + } + } +} +``` + +## Error Codes Reference + +### Schema Validation Errors + +| Code | Description | Severity | Fixable | +|------|-------------|----------|----------| +| `SCHEMA_VALIDATION_ERROR` | Missing or invalid required field | Error | Yes | +| `VERSION_MISMATCH_ERROR` | Protocol version conflicts | Error | Yes | +| `STRICT_VERSION_MISMATCH` | Strict mode version issues | Error | Yes | + +### Network & Discovery Errors + +| Code | Description | Severity | Fixable | +|------|-------------|----------|----------| +| `VALIDATION_FAILED` | General validation failure | Error | Depends | +| `ENDPOINT_UNREACHABLE` | HTTP connection issues | Error | Yes | +| `LEGACY_DISCOVERY_ENDPOINT` | Using legacy endpoint | Warning | Yes | + +### Feature Compatibility Warnings + +| Code | Description | Severity | Fixable | +|------|-------------|----------|----------| +| `VERSION_FEATURE_MISMATCH` | Feature requires newer protocol | Warning | Yes | +| `GRPC_WITHOUT_STREAMING` | gRPC without streaming capability | Warning | Yes | + +## Performance & Timing + +### Metrics Collected + +- **Individual Validation Steps**: Timing for each validation component +- **Total Duration**: Complete validation time +- **Network Requests**: HTTP request timing +- **Schema Processing**: JSON parsing and validation time + +### Optimization Features + +- **Schema-Only Mode**: Skip network calls with `--schema-only` +- **Timeout Configuration**: Custom timeout with `--timeout ` +- **Efficient Error Handling**: Early termination on critical failures + +### Performance Benchmarks + +| Operation | Typical Duration | Notes | +|-----------|------------------|-------| +| Schema Validation | 1-50ms | Depends on agent card size | +| Network Request | 50-1000ms | Depends on network latency | +| Version Analysis | <1ms | Cached semver operations | +| Total Validation | 100-2000ms | Varies by mode and network | + +--- + +## See Also + +- **[Scoring System](scoring.md)** - Understand how validation results translate to scores +- **[CLI Reference](../reference/cli/index.md)** - Command-line validation reference +- **[Python SDK](../reference/sdk-python/index.md)** - Runtime protection for production agents + +!!! tip "Production Deployment" + The CapiscIO CLI (`capiscio`) validates agent cards during development and CI/CD. For runtime protection, use [CapiscIO Python SDK](../reference/sdk-python/index.md). + +## Contributing to Validation + +See the core implementation repository (Go) at https://github.com/capiscio/capiscio-core for information on extending the validation system. + +## Related Documentation + +- [CLI Usage Guide](../reference/cli/index.md) +- [CLI Reference](../reference/cli/index.md) +- [A2A Protocol Specification](https://github.com/a2aproject/A2A){:target="_blank"} diff --git a/docs/reference/server/policy-enforcement.md b/docs/reference/server/policy-enforcement.md new file mode 100644 index 0000000..aa40f3d --- /dev/null +++ b/docs/reference/server/policy-enforcement.md @@ -0,0 +1,110 @@ +# Policy Enforcement Reference + +Configuration reference for CapiscIO's Policy Enforcement Point (PEP) and PDP integration. + +--- + +## Environment Variables + +| Variable | Default | Description | +|----------|---------|-------------| +| `CAPISCIO_EMBEDDED_PDP` | `false` | Enable embedded OPA evaluator (in-process PDP) | +| `CAPISCIO_PDP_ENDPOINT` | _(empty)_ | External PDP URL. Empty + no embedded PDP = badge-only mode | +| `CAPISCIO_PDP_TIMEOUT_MS` | `500` | External PDP query timeout in milliseconds | +| `CAPISCIO_ENFORCEMENT_MODE` | `EM-OBSERVE` | One of: `EM-OBSERVE`, `EM-GUARD`, `EM-DELEGATE`, `EM-STRICT` | +| `CAPISCIO_WORKSPACE` | _(empty)_ | Workspace/tenant UUID (required for embedded PDP) | +| `CAPISCIO_BUNDLE_POLL_INTERVAL` | `30s` | Embedded PDP bundle rebuild interval | +| `CAPISCIO_BUNDLE_STALENESS_THRESHOLD` | `5m` | Embedded PDP bundle age before staleness warnings | +| `CAPISCIO_BREAKGLASS_PUBLIC_KEY` | _(empty)_ | Path to break-glass Ed25519 public key file | +| `CAPISCIO_PEP_ID` | _(empty)_ | PEP instance identifier (sent to PDP as `X-Capiscio-PEP-ID`) | + +--- + +## Enforcement Modes + +| Mode | PDP DENY | PDP Unavailable | Obligations | Staleness | +|------|----------|-----------------|-------------|-----------| +| `EM-OBSERVE` | Logged, request proceeds | Request proceeds | Not enforced | Annotated in telemetry | +| `EM-GUARD` | Request blocked (403) | Request blocked (503) | Not enforced | Annotated in telemetry | +| `EM-DELEGATE` | Request blocked (403) | Request blocked (503) | Best-effort (failures logged) | Annotated in telemetry | +| `EM-STRICT` | Request blocked (403) | Request blocked (503) | Enforced (unknown types cause denial) | Request denied | + +--- + +## PDP Request Format + +The PEP sends a JSON POST to the PDP endpoint. Your PDP must accept this format: + +```jsonc +{ + "pip_version": "capiscio.pip.v1", + "subject": { + "did": "did:web:agent.example.com", + "badge_jti": "badge-uuid", + "ial": "1", + "trust_level": "DV" // Badge trust level code: SS(0), REG(1), DV(2), OV(3), EV(4) + }, + "action": { + "operation": "POST /v1/badges", + "capability_class": null + }, + "resource": { + "identifier": "/v1/badges" + }, + "context": { + "txn_id": "txn-uuid", + "enforcement_mode": "EM-GUARD" + }, + "environment": { + "workspace": "00000000-0000-0000-0000-000000000000", + "pep_id": "server-01", + "time": "2026-03-01T12:00:00Z" + } +} +``` + +### PDP Response Format + +```json +{ + "decision": "ALLOW", + "decision_id": "eval-uuid", + "obligations": [], + "reason": "Policy matched: allow-trusted-agents", + "ttl": 300 +} +``` + +--- + +## Telemetry Fields + +Each request through the PEP emits structured telemetry: + +| Field | Description | +|-------|-------------| +| `capiscio.policy.decision` | `ALLOW`, `DENY`, or `ALLOW_OBSERVE` | +| `capiscio.policy.decision_id` | Unique evaluation ID from the PDP | +| `capiscio.policy.error_code` | `PDP_UNAVAILABLE` if PDP was unreachable | +| `staleness.bundle_stale` | `true` if the embedded PDP bundle is stale | +| `staleness.bundle_age_ms` | Age of the bundle in milliseconds (when stale) | + +--- + +## Bundle Endpoint (Embedded PDP) + +When the embedded PDP is active, a bundle endpoint is available for external OPA consumers: + +``` +GET /v1/bundles/{workspace_id} +``` + +Requires `X-Capiscio-Registry-Key` authentication. The API key must belong to the matching workspace/org. + +--- + +## See Also + +- [Policy Enforcement Setup](../../how-to/security/policy-enforcement.md) — Step-by-step setup guide +- [Policy Config YAML](../policy-config-yaml.md) — Policy configuration format +- [Policy API](../policy-api.md) — Policy management API diff --git a/mkdocs.yml b/mkdocs.yml index b1963a8..f4c815e 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -247,6 +247,7 @@ nav: - reference/index.md - CapiscIO Core: - CLI Reference: reference/cli/index.md + - Validation Reference: reference/cli/validation.md - Go API: reference/go-api.md - gRPC Services: reference/grpc.md - Agent Guard: @@ -265,6 +266,7 @@ nav: - API Reference: reference/server/api.md - Deployment: reference/server/deployment.md - Badge CA: reference/server/badge-ca.md + - Policy Enforcement: reference/server/policy-enforcement.md - Policy: - Policy Config YAML: reference/policy-config-yaml.md - Policy API: reference/policy-api.md From dc0eb6e7d7d0406a282a573e687125e33c67294e Mon Sep 17 00:00:00 2001 From: Beon de Nood Date: Sat, 2 May 2026 14:54:25 -0400 Subject: [PATCH 2/2] fix(docs): correct broken relative links in validation reference --- docs/reference/cli/validation.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/reference/cli/validation.md b/docs/reference/cli/validation.md index 7b09d46..7d94a55 100644 --- a/docs/reference/cli/validation.md +++ b/docs/reference/cli/validation.md @@ -402,12 +402,12 @@ Version: {{ protocol_version }} (Strictness: progressive) ## See Also -- **[Scoring System](scoring.md)** - Understand how validation results translate to scores -- **[CLI Reference](../reference/cli/index.md)** - Command-line validation reference -- **[Python SDK](../reference/sdk-python/index.md)** - Runtime protection for production agents +- **[Scoring System](../../concepts/scoring.md)** - Understand how validation results translate to scores +- **[CLI Reference](index.md)** - Command-line validation reference +- **[Python SDK](../sdk-python/index.md)** - Runtime protection for production agents !!! tip "Production Deployment" - The CapiscIO CLI (`capiscio`) validates agent cards during development and CI/CD. For runtime protection, use [CapiscIO Python SDK](../reference/sdk-python/index.md). + The CapiscIO CLI (`capiscio`) validates agent cards during development and CI/CD. For runtime protection, use [CapiscIO Python SDK](../sdk-python/index.md). ## Contributing to Validation @@ -415,6 +415,6 @@ See the core implementation repository (Go) at https://github.com/capiscio/capis ## Related Documentation -- [CLI Usage Guide](../reference/cli/index.md) -- [CLI Reference](../reference/cli/index.md) +- [CLI Usage Guide](index.md) +- [CLI Reference](index.md) - [A2A Protocol Specification](https://github.com/a2aproject/A2A){:target="_blank"}