From 06f6214bebc56982cf0edbdbb49883ff673cea39 Mon Sep 17 00:00:00 2001 From: Rana Salem Date: Tue, 28 Apr 2026 22:40:41 +0100 Subject: [PATCH 1/2] add new plugin: openapi-drift-detection --- plugins/openapi-drift-detection/README.md | 69 +++ .../agents/openapi-drift-detector.agent.md | 36 ++ plugins/openapi-drift-detection/plugin.json | 15 + .../skills/openapi-drift-detection/SKILL.md | 240 +++++++++ .../references/discovery-patterns.md | 466 ++++++++++++++++++ .../references/drift-categories.md | 92 ++++ .../references/report-template.md | 125 +++++ 7 files changed, 1043 insertions(+) create mode 100644 plugins/openapi-drift-detection/README.md create mode 100644 plugins/openapi-drift-detection/agents/openapi-drift-detector.agent.md create mode 100644 plugins/openapi-drift-detection/plugin.json create mode 100644 plugins/openapi-drift-detection/skills/openapi-drift-detection/SKILL.md create mode 100644 plugins/openapi-drift-detection/skills/openapi-drift-detection/references/discovery-patterns.md create mode 100644 plugins/openapi-drift-detection/skills/openapi-drift-detection/references/drift-categories.md create mode 100644 plugins/openapi-drift-detection/skills/openapi-drift-detection/references/report-template.md diff --git a/plugins/openapi-drift-detection/README.md b/plugins/openapi-drift-detection/README.md new file mode 100644 index 0000000..856ee51 --- /dev/null +++ b/plugins/openapi-drift-detection/README.md @@ -0,0 +1,69 @@ +# openapi-drift-detection + +An agent that compares OpenAPI spec with service implementation to identify areas of drift, prioritises findings in order of highest severity, and proposes fixes to ensure proper alignment between the service and its documentation. + +## What's included + +| Component | Path | Description | +|-----------|------|-------------| +| Agent | `agents/openapi-drift-detector.agent.md` | Detects drift between OpenAPI 3.x spec and service implementation across 7 dimensions (paths & operations, request schemas, response schemas, authentication & security, error responses, content types, API metadata), prioritises findings by severity (P0-P3), and proposes JSON/YAML fixes | +| Skill | `skills/openapi-drift-detection/` | Comprehensive 4-step procedure for discovering OpenAPI specs, parsing specifications, analysing service implementation, comparing across all dimensions, and generating detailed drift reports with proposed fixes | + +## How it works + +### Agent: OpenAPI drift detector + +The agent: + +1. Discovers OpenAPI specification files in your project +2. Parses paths, operations, parameters, request bodies, response schemas, security schemes, and servers +3. Analyzes the service implementation to extract actual routes, validation schemas, response shapes, auth configuration, and error handlers +4. Compares the spec against the implementation across 7 dimensions: + - **Paths & Operations**: Declared vs. actual routes and HTTP methods + - **Request Schemas**: Declared vs. actual parameters and request body validation + - **Response Schemas**: Declared vs. actual response status codes and body shapes + - **Authentication & Security**: Declared vs. actual auth strategies and per-route overrides + - **Error Responses**: Declared vs. actual error status codes and shapes + - **Content Types**: Declared vs. actual accepted and produced content types + - **API Metadata**: Declared vs. actual server URLs, versioning, and info fields +5. Prioritises findings by severity (P0-P3): + - **P0 — Critical**: Contract-breaking differences (missing routes, wrong methods, missing required fields) + - **P1 — High**: Schema-level inaccuracies (type mismatches, enum mismatches, auth scheme issues) + - **P2 — Medium**: Correctness gaps (optional fields not in spec, default value mismatches) + - **P3 — Low**: Documentation quality issues (missing descriptions, cosmetic naming) +6. Generates a structured drift report with findings grouped by priority +7. Proposes JSON/YAML fixes to bring the spec into alignment with the implementation + +### Skill: OpenAPI drift detection + +The skill provides a comprehensive 4-step procedure: + +1. **Project Discovery**: Locates OpenAPI specs, detects spec generation method, discovers API-related source code, builds a service inventory +2. **Parse OpenAPI Spec**: Extracts paths, operations, parameters, request bodies, responses, components, security schemes, and servers +3. **Analyse Service Implementation**: Discovers actual routes, request validation, response shapes, auth configuration, error handling, and content types +4. **Compare & Detect Drift**: Checks all 7 dimensions using the priority framework and generates a detailed drift report + +Includes reference documentation: +- `discovery-patterns.md`: Search strategies for finding API infrastructure code +- `drift-categories.md`: Complete priority framework and decision matrix +- `report-template.md`: Structured drift report format + +## Usage + +Ask the agent to detect drift in your OpenAPI specifications: + +> "Check for drift between my OpenAPI spec and service implementation" + +> "Audit my OpenAPI 3.x documentation against the actual service" + +> "Review my routes, schemas, and responses in OpenAPI against the code" + +The agent will: +1. Generate a comprehensive drift report +2. Prioritise findings by severity +3. Propose specific JSON/YAML changes to fix the drift +4. Apply fixes with your approval + +## License + +[MIT](../../LICENSE) diff --git a/plugins/openapi-drift-detection/agents/openapi-drift-detector.agent.md b/plugins/openapi-drift-detection/agents/openapi-drift-detector.agent.md new file mode 100644 index 0000000..09879ce --- /dev/null +++ b/plugins/openapi-drift-detection/agents/openapi-drift-detector.agent.md @@ -0,0 +1,36 @@ +--- +description: "Detect OpenAPI contract drift between an OpenAPI 3.x spec and the service implementation. Use when: checking OpenAPI drift, auditing REST API documentation, validating request/response schemas match code, reviewing API contract accuracy, comparing spec vs implementation, onboarding documentation review, verifying routes operations and schemas match implementation." +tools: [read, search, edit] +model: Claude Sonnet 4.6 (copilot) +--- + +You are an OpenAPI contract drift detective. Your job is to find discrepancies between a project's OpenAPI 3.x specification and its actual HTTP API implementation, then report them in a structured, prioritised format and offer to fix the spec. + +## Constraints + +- DO NOT edit application source code — only OpenAPI spec files (e.g. `openapi*.json`, `openapi*.yaml`, `openapi*.yml`, `swagger*.json`, `swagger*.yaml`) +- DO NOT guess or fabricate information about the service — only report what you can verify by reading the code +- DO NOT modify the service's behaviour — if the code is "wrong", flag it in the report for the developer to fix manually +- ONLY propose spec changes that bring documentation in line with the actual implementation +- ALWAYS use the `openapi-drift-detection` skill procedure for a thorough, repeatable analysis +- When the spec is auto-generated (e.g. via hapi-swagger, swagger-jsdoc, NestJS decorators), note this in the report — drift may indicate the spec needs regeneration rather than manual editing + +## Approach + +1. **Discover** — Find the OpenAPI spec(s) and all API-related source code in the workspace using the `openapi-drift-detection` skill's discovery procedure +2. **Parse** — Extract the contract surface from the spec: paths, operations, parameters, request bodies, response schemas, security schemes, servers, metadata +3. **Analyse** — Extract the actual implementation surface from source code: route registrations, validation schemas, response builders, auth configuration, error handlers, content types +4. **Compare** — Detect drift across 7 dimensions: Paths & Operations, Request Schemas, Response Schemas, Authentication & Security, Error Responses, Content Types, API Metadata +5. **Report** — Generate a prioritised drift report (P0 Critical → P3 Low) using the standardised report template +6. **Fix** — After presenting the report, offer to apply proposed changes to the OpenAPI spec. Apply fixes only with user approval, one at a time + +## Output Format + +Always output a structured drift report following the report template from the `openapi-drift-detection` skill. If no drift is detected, confirm alignment with a brief summary of what was checked. + +## When Offering Fixes + +- Show the exact JSON/YAML path and proposed change before applying +- Apply one fix at a time so the user can review each change +- After all fixes are applied, re-run the comparison to confirm zero remaining drift +- If the spec is auto-generated, recommend regeneration commands instead of manual edits where appropriate diff --git a/plugins/openapi-drift-detection/plugin.json b/plugins/openapi-drift-detection/plugin.json new file mode 100644 index 0000000..c34ea95 --- /dev/null +++ b/plugins/openapi-drift-detection/plugin.json @@ -0,0 +1,15 @@ +{ + "name": "openapi-drift-detection", + "description": "Detects drift between OpenAPI spec and service implementation, prioritises findings in order of severity, and proposes fixes to ensure service implementation is accurately reflected in its documentation.", + "version": "1.0.0", + "author": { + "name": "Rana Salem" + }, + "license": "MIT", + "keywords": [ + "openapi", + "documentation" + ], + "agents": "agents/", + "skills": "skills/" +} diff --git a/plugins/openapi-drift-detection/skills/openapi-drift-detection/SKILL.md b/plugins/openapi-drift-detection/skills/openapi-drift-detection/SKILL.md new file mode 100644 index 0000000..a2cc914 --- /dev/null +++ b/plugins/openapi-drift-detection/skills/openapi-drift-detection/SKILL.md @@ -0,0 +1,240 @@ +--- +name: openapi-drift-detection +description: "Detect drift between an OpenAPI 3.x spec and the service implementation. Use when: checking OpenAPI drift, auditing REST API documentation, validating request response schemas, comparing spec vs code, reviewing contract accuracy, onboarding documentation review, verifying routes operations parameters and status codes match implementation." +--- + +# OpenAPI Contract Drift Detection + +Detect discrepancies between an OpenAPI 3.x specification and the actual HTTP API implementation. This skill is framework-agnostic and works by dynamically discovering the spec and API code in any project. + +## When to Use + +- Before a release, to verify API documentation accuracy +- When onboarding new developers or API consumers +- After iterating on a service to check if the spec needs updating +- During code review of route, schema, or auth changes +- When a consumer reports unexpected request/response shapes +- After adding or removing API endpoints + +## Procedure + +Follow these steps in order. Each step must complete before moving to the next. + +### Step 0 — Project Discovery + +This step builds a map of the project so the rest of the analysis is grounded in real files. + +1. **Find the OpenAPI spec(s)**: + - Search the workspace for files matching: `**/openapi*.json`, `**/openapi*.yaml`, `**/openapi*.yml`, `**/swagger*.json`, `**/swagger*.yaml`, `**/swagger*.yml` + - Also check common locations: `docs/`, `api/`, `spec/`, `specifications/`, project root + - If multiple specs are found, ask the user which to analyse — or analyse all if instructed + - Confirm the spec uses OpenAPI version 3.x (check the `openapi` field at the root starts with `3.`) + +2. **Read the project's copilot-instructions.md** (if it exists at `.github/copilot-instructions.md` or root): + - Extract: framework, language, directory structure conventions, API technology, key service names + - This accelerates discovery by telling you where to look + +3. **Detect spec generation method**: + - Search for hapi-swagger, swagger-jsdoc, @nestjs/swagger, swagger-autogen, or similar spec-generation dependencies in `package.json`, `build.gradle`, `pom.xml`, `requirements.txt`, or equivalent + - If found, note the generation method — drift may indicate the spec needs regeneration rather than manual editing + - Search for spec generation scripts (e.g. `generate-openapi`, `generateOpenApiSpec`, `swagger:generate`) + +4. **Discover API-related source code** using the patterns in [discovery-patterns.md](./references/discovery-patterns.md): + - **Route definitions**: Framework router files, route registrations, controller decorators + - **Validation schemas**: Joi, Zod, Yup, JSON Schema, class-validator, or similar + - **Auth configuration**: Auth plugins, middleware, strategies, security schemes + - **Response builders**: Functions that construct HTTP responses, error formatters + - **API configuration**: Server URLs, CORS config, rate limiting, content type settings + +5. **Build a Service Inventory** — a mental model of: + - Which routes (path + HTTP method) the service exposes + - What request validation each route applies (params, query, body, headers) + - What response shapes each route returns (per status code) + - What authentication/authorization strategy is applied (global default and per-route overrides) + - What error response shapes the service produces + - What content types are accepted and produced + +### Step 1 — Parse the OpenAPI Spec + +Read the discovered OpenAPI spec file and extract: + +1. **Paths**: For each path + HTTP method combination, note: + - Path string (e.g. `/api/v1/users/{id}`) + - HTTP method (GET, POST, PUT, PATCH, DELETE) + - `operationId` if present + - Tags + - `deprecated` flag + - `security` overrides (if different from global) + +2. **Parameters**: For each operation, note: + - Path parameters: name, schema (type, format, pattern, enum), required + - Query parameters: name, schema, required, default value + - Header parameters: name, schema, required + - Cookie parameters: name, schema + +3. **Request Bodies**: For each operation with a request body, note: + - Required flag + - Content type(s) (e.g. `application/json`, `multipart/form-data`) + - Schema: all fields, types, formats, required list, enums, min/max constraints, patterns, defaults, nested objects, arrays + +4. **Responses**: For each operation, note every response status code: + - Status code (200, 201, 400, 401, 404, 500, etc.) + - Description + - Content type(s) + - Response body schema: all fields, types, formats, required list, enums, nesting + - Response headers + +5. **Components**: For each reusable component, note: + - `schemas`: name, type, properties, required, constraints + - `securitySchemes`: name, type (http/apiKey/oauth2/openIdConnect), scheme, bearerFormat, flows + - `parameters`: reusable parameter definitions + - `responses`: reusable response definitions + +6. **Security** (global): + - Default security requirement(s) applied to all operations + - Which security scheme(s) are referenced + +7. **Servers**: + - Each server URL and description + - Server variables if any + +8. **Info metadata**: + - `title`, `version`, `description`, `contact`, `license` + +### Step 2 — Analyse the Service Implementation + +Using the Service Inventory from Step 0, read each discovered source file and extract the **actual** implementation details: + +1. **Routes (actual)**: + - Read framework router/controller files to extract every registered route + - For each route, note: HTTP method, path, handler function, any route-level options + - Map framework-specific path syntax to OpenAPI syntax: + - Hapi.js: `{param}` → `{param}` (same) + - Express: `:param` → `{param}` + - Fastify: `:param` → `{param}` + - NestJS: `:param` → `{param}` (decorators) + +2. **Request Validation (actual)**: + - Read validation schema definitions (Joi objects, Zod schemas, JSON Schema files, class-validator decorators, etc.) + - Extract: field names, types, required vs optional, format validations (UUID, email, date-time), enums (.valid()/.enum()), numeric ranges (.min()/.max()), string constraints (.pattern()/.regex()), defaults + - Map framework-specific constraints to their JSON Schema / OpenAPI equivalents: + - `Joi.string().uuid()` → `{ type: "string", format: "uuid" }` + - `Joi.number().integer()` → `{ type: "integer" }` + - `Joi.number().min(X).max(Y)` → `{ type: "number", minimum: X, maximum: Y }` + - `Joi.string().valid('a', 'b')` → `{ type: "string", enum: ["a", "b"] }` + - `Joi.string().pattern(/regex/)` → `{ type: "string", pattern: "regex" }` + - `Joi.object().keys({...})` → `{ type: "object", properties: {...} }` + - `Joi.array().items(...)` → `{ type: "array", items: {...} }` + - `Joi.alternatives().try(...)` → `{ oneOf: [...] }` + - `.required()` → field in `required` array + - `.optional()` → field NOT in `required` array + - `.default(val)` → `{ default: val }` + - `.description(text)` → `{ description: text }` + - `.min(n)` on string → `{ minLength: n }` + - `.max(n)` on string → `{ maxLength: n }` + - `.min(n)` on array → `{ minItems: n }` + - `.max(n)` on array → `{ maxItems: n }` + - `z.string().uuid()` → `{ type: "string", format: "uuid" }` + - `z.enum(['a', 'b'])` → `{ type: "string", enum: ["a", "b"] }` + - `z.number().int().min(X).max(Y)` → `{ type: "integer", minimum: X, maximum: Y }` + - Separately identify: path param schemas, query param schemas, request body schemas, header schemas + +3. **Response Shapes (actual)**: + - Read response schema definitions and handler return statements + - For each route, extract every status code returned and the response body shape + - Pay attention to: + - Success responses (200, 201, 204) + - Error responses (400, 401, 403, 404, 422, 500, 502, 504) + - Response wrappers (e.g. `{ data: {...} }`, `{ message: "..." }`) + - Hapi.js: `h.response(payload).code(statusCode)` patterns + - Express: `res.status(code).json(payload)` patterns + +4. **Authentication & Authorization (actual)**: + - Identify the global auth strategy (e.g. `server.auth.default('strategyName')`) + - Identify per-route auth overrides (e.g. `options: { auth: false }` in Hapi, `passport.authenticate()` in Express) + - Identify the auth mechanism: JWT, API key, OAuth2, session-based + - Note which routes are unauthenticated and why + +5. **Error Handling (actual)**: + - Read error handler middleware, Boom error usage, or custom error formatters + - Extract the shape of error responses (field names, structure) + - Note which error status codes each route can produce + +6. **Content Types (actual)**: + - Check what content types routes accept (JSON, form-data, etc.) + - Check what content types routes produce + - Look at framework defaults and per-route overrides + +### Step 3 — Compare and Detect Drift + +Compare the spec (Step 1) against the implementation (Step 2) across these 7 dimensions. For each difference found, classify it using the priority framework in [drift-categories.md](./references/drift-categories.md). + +#### Dimension 1: Paths & Operations +- Does every route registered in the code have a matching path + method entry in the spec? +- Does every path + method in the spec have a corresponding route in the code? +- Do path parameter names match between spec and code? +- Are HTTP methods correct? +- Are deprecated flags accurate? + +#### Dimension 2: Request Schemas +- For each operation, compare every request parameter and body field against the validation schema: + - **Path parameters**: name, type, format, required, pattern, enum values + - **Query parameters**: name, type, format, required, default values + - **Request body fields**: field names, types, required vs optional, format constraints, enum values, numeric ranges, string constraints, nested structure + - **Headers**: declared vs actual + +#### Dimension 3: Response Schemas +- For each operation, compare every response status code and body shape: + - **Status codes**: Does the spec document every status code the handler can return? Does the spec list status codes the handler never returns? + - **Response body shape**: Do field names, types, and nesting match per status code? + - **Response headers**: Are response headers documented correctly? + +#### Dimension 4: Authentication & Security +- Does the spec's global `security` requirement match the implementation's default auth strategy? +- For routes with `auth: false` (or equivalent), does the spec show `security: []` to indicate no auth? +- Does the security scheme type (bearer, apiKey, oauth2) match the implementation? +- Are scope/role requirements reflected accurately? + +#### Dimension 5: Error Responses +- Are all error status codes that the implementation can produce documented in the spec (400, 401, 403, 404, 422, 500, 502, 504)? +- Do error body shapes in the spec match the actual error response format? +- Is the validation error format (e.g. Joi validation errors, Boom errors) documented? + +#### Dimension 6: Content Types +- Do request content types in the spec (`requestBody.content` keys) match what the implementation accepts? +- Do response content types in the spec (`responses.*.content` keys) match what the implementation produces? +- Are content negotiation behaviours (Accept header handling, multiple content types) reflected? + +#### Dimension 7: API Metadata +- Do server URLs in the spec match actual deployment environments? +- Does `info.version` match the application version? +- Do tags in the spec match actual route groupings/categories? +- Are deprecated routes correctly marked with `deprecated: true`? +- Are `info.description`, `info.contact`, and `info.license` current and accurate? + +### Step 4 — Generate the Drift Report + +Use the template in [report-template.md](./references/report-template.md) to produce the report. + +- Fill in all metadata (spec file path, version, service name, date) +- Set the overall status indicator: + - 🟢 **No drift** — zero findings + - 🟡 **Minor drift** — only P2/P3 findings + - 🔴 **Significant drift** — any P0 or P1 findings +- List findings in priority order (P0 first), with specific source file references +- Explain each finding +- Generate proposed fixes for every finding +- If the spec is auto-generated, add a note recommending regeneration and list which findings would be resolved by regeneration vs which require source code changes +- Build the verification checklist based on what was actually analysed + +### Step 5 — Offer Fixes + +After presenting the report: + +1. Ask the user if they want to apply the proposed fixes to the OpenAPI spec +2. If yes, apply fixes **one at a time** in priority order (P0 first) +3. After each fix, briefly confirm what was changed +4. After all fixes are applied, offer to re-run the comparison to verify zero remaining drift +5. If the spec is auto-generated, recommend running the generation command instead of manual edits where appropriate + +**Remember**: Only edit the OpenAPI spec file(s). Never edit application source code. diff --git a/plugins/openapi-drift-detection/skills/openapi-drift-detection/references/discovery-patterns.md b/plugins/openapi-drift-detection/skills/openapi-drift-detection/references/discovery-patterns.md new file mode 100644 index 0000000..1bdebca --- /dev/null +++ b/plugins/openapi-drift-detection/skills/openapi-drift-detection/references/discovery-patterns.md @@ -0,0 +1,466 @@ +# Discovery Patterns + +This reference teaches the agent how to find API-related code in **any** project. Use these search patterns during Step 0 (Project Discovery) to build the Service Inventory. + +Search broadly first, then narrow down. Not all patterns will match in every project — that's expected. The goal is to find what exists, not to require everything. + +--- + +## 1. Finding OpenAPI Spec Files + +### File name patterns +``` +**/openapi*.json +**/openapi*.yaml +**/openapi*.yml +**/swagger*.json +**/swagger*.yaml +**/swagger*.yml +**/api-docs*.json +**/api-docs*.yaml +``` + +### Common locations +- `docs/` +- `docs/openapi/` +- `api/` +- `spec/` +- `specifications/` +- `swagger/` +- Project root + +### Verification +After finding a candidate file, confirm it contains an `openapi` field at the root level whose value starts with `3.` (e.g. `3.0.0`, `3.0.3`, `3.1.0`). For Swagger 2.x files, the field is `swagger` with value `2.0`. + +--- + +## 2. Detecting Spec Generation + +Some projects auto-generate their OpenAPI spec from code annotations or route metadata. Detecting this is important because drift may mean "regenerate the spec" rather than "manually edit the spec". + +### hapi-swagger (Hapi.js) +``` +hapi-swagger +hapiSwagger +HapiSwagger +swaggerOptions +``` + +### swagger-jsdoc (Express/Koa) +``` +swagger-jsdoc +swaggerJSDoc +swaggerDefinition +swaggerSpec +``` + +### @nestjs/swagger (NestJS) +``` +@nestjs/swagger +SwaggerModule +DocumentBuilder +@ApiProperty +@ApiOperation +@ApiResponse +@ApiTags +``` + +### swagger-autogen +``` +swagger-autogen +swaggerAutogen +``` + +### Generation scripts +Search for scripts in `package.json`, `Makefile`, or `scripts/` directory: +``` +generateOpenApi +generate-openapi +swagger:generate +openapi:generate +build:docs +``` + +--- + +## 3. Finding HTTP Route Definitions + +Search for import/require statements and route registration patterns for common HTTP frameworks. + +### Hapi.js +``` +server.route( +server.register( +plugin.register +server.route({ +method:.*'GET' +method:.*'POST' +method:.*'PUT' +method:.*'PATCH' +method:.*'DELETE' +path:.*'/api/ +options:.*validate +options:.*auth +``` + +### Express.js +``` +express() +express.Router() +router.get( +router.post( +router.put( +router.patch( +router.delete( +app.get( +app.post( +app.put( +app.use( +``` + +### Fastify +``` +fastify.get( +fastify.post( +fastify.put( +fastify.patch( +fastify.delete( +fastify.register( +fastify.route( +schema:.*body +schema:.*params +schema:.*querystring +schema:.*response +``` + +### Koa +``` +koa-router +@koa/router +router.get( +router.post( +router.put( +router.patch( +router.delete( +router.routes() +``` + +### NestJS +``` +@Controller( +@Get( +@Post( +@Put( +@Patch( +@Delete( +@Param( +@Query( +@Body( +@Headers( +``` + +### General route patterns (language-agnostic) +Search for function/method names containing: +``` +route|router|controller|handler|endpoint|middleware +``` + +--- + +## 4. Finding Validation Schemas + +### Joi (Node.js) +``` +Joi.object( +Joi.string() +Joi.number() +Joi.boolean() +Joi.array() +Joi.alternatives() +.validate( +.validateAsync( +schema.validate +.required() +.valid( +.pattern( +``` + +### Zod (TypeScript/JavaScript) +``` +z.object( +z.string() +z.number() +z.boolean() +z.array( +z.enum( +.parse( +.safeParse( +.parseAsync( +``` + +### Yup (JavaScript) +``` +yup.object( +yup.string() +yup.number() +Yup.object( +Yup.string() +.validate( +.isValid( +``` + +### JSON Schema +``` +"$schema" +"type": "object" +"properties" +"required" +ajv +jsonschema +json-schema +``` + +### class-validator (TypeScript) +``` +@IsString() +@IsNumber() +@IsUUID() +@IsEmail() +@IsEnum( +@IsOptional() +@ValidateNested() +class-validator +``` + +### General validation patterns +Search for files or directories named: +``` +**/schema* +**/schemas/** +**/validation/** +**/validators/** +**/dto/** +**/models/** +``` + +--- + +## 5. Finding Auth Configuration + +### JWT / Bearer Token +``` +@hapi/jwt +hapi-auth-jwt2 +jsonwebtoken +jwt.verify +jwt.sign +passport-jwt +JwtStrategy +BearerStrategy +auth.strategy +auth.default +server.auth.strategy +server.auth.default +``` + +### Passport.js (Express) +``` +passport +passport.authenticate +passport.use +LocalStrategy +JwtStrategy +OAuth2Strategy +``` + +### NestJS Guards +``` +@UseGuards( +AuthGuard +JwtAuthGuard +RolesGuard +@Roles( +``` + +### Per-route auth overrides +``` +auth: false +auth:.*false +{ auth: false } +options:.*auth +authenticate.*false +public.*true +@Public() +AllowAnonymous +``` + +### Security group / role patterns +``` +allowedGroupIds +allowedRoles +requiredRoles +securityGroups +permissions +scopes +``` + +--- + +## 6. Finding Response Builders & Error Handlers + +### Hapi.js responses +``` +h.response( +.code( +Boom. +@hapi/boom +Boom.badRequest +Boom.unauthorized +Boom.forbidden +Boom.notFound +Boom.conflict +Boom.badImplementation +Boom.badGateway +Boom.gatewayTimeout +failAction +``` + +### Express responses +``` +res.status( +res.json( +res.send( +res.sendStatus( +next(err) +next(new Error +``` + +### General error patterns +``` +errorHandler +errorMiddleware +formatError +createError +HttpException +HttpError +BadRequestError +NotFoundError +UnauthorizedError +ForbiddenError +InternalServerError +``` + +### Response schema definitions +Search for files or directories named: +``` +**/responses/** +**/errors/** +**/exceptions/** +``` + +--- + +## 7. Finding API Configuration + +### Server URLs / Base paths +``` +PORT +HOST +BASE_URL +BASE_PATH +API_PREFIX +server.info.uri +server.address +``` + +### CORS +``` +cors +@hapi/cors +cors() +origin +Access-Control +``` + +### Content types +``` +content-type +Content-Type +application/json +multipart/form-data +application/x-www-form-urlencoded +text/plain +produces +consumes +``` + +### API versioning +``` +/api/v1 +/api/v2 +/v1/ +/v2/ +version +apiVersion +``` + +### Configuration files +Search for files named: +``` +**/config/server* +**/config/api* +**/config/hapi* +**/config/express* +**/config/cors* +``` + +### Infrastructure references +Server definitions may also exist in: +``` +**/compose*.yaml +**/compose*.yml +**/docker-compose* +**/terraform/** +**/cloudformation/** +**/serverless.yml +**/cdk/** +``` + +--- + +## 8. Finding Shared / Common Schemas + +### Common schema patterns +Search for directories and files that define reusable request/response shapes: +``` +**/schemas/common* +**/schemas/shared* +**/api/common/** +**/api/schemas/** +**/api/v*/schemas/** +``` + +### Error response schemas +``` +badRequest +unauthorized +forbidden +notFound +conflict +unprocessableEntity +internalServerError +badGateway +gatewayTimeout +errorResponse +``` + +--- + +## Search Strategy + +1. **Start broad**: Search for the most distinctive patterns first (e.g. `openapi`, framework imports, route registrations) +2. **Follow imports**: When you find a route file, trace its imports to find related schemas, handlers, and config +3. **Check test files**: Test mocks and fixtures often contain realistic request/response examples that reveal the actual shape of API payloads +4. **Check package.json/build files**: Dependencies list confirms which HTTP framework, validation library, and auth libraries are in use +5. **Don't assume directory structure**: Some projects use flat structures, others use deep nesting. Let the search results guide you +6. **Check for spec generation**: If the spec is auto-generated, the generation config and route annotations are more authoritative than the spec file itself diff --git a/plugins/openapi-drift-detection/skills/openapi-drift-detection/references/drift-categories.md b/plugins/openapi-drift-detection/skills/openapi-drift-detection/references/drift-categories.md new file mode 100644 index 0000000..d2a1168 --- /dev/null +++ b/plugins/openapi-drift-detection/skills/openapi-drift-detection/references/drift-categories.md @@ -0,0 +1,92 @@ +# Drift Categories & Priority Framework + +Use this framework to classify every difference found during drift detection. When in doubt, err on the side of higher priority — it's better to flag something as P1 that turns out to be P2 than to miss a breaking issue. + +--- + +## P0 — Critical (resolve immediately) + +Contract-breaking differences. API consumers relying on the spec will encounter unexpected behaviour at runtime. + +| What to Check | Example Drift | +|---------------|---------------| +| **Missing route in spec** | Code registers `GET /api/v1/users/{id}` but the spec has no corresponding path+method entry | +| **Extra route in spec** | Spec documents `DELETE /api/v1/users/{id}` but the code has no such route | +| **Wrong HTTP method** | Spec says `POST /api/v1/search` but code registers `GET /api/v1/search` | +| **Missing required request field** | Code validates `sbi` as required in the request body but the spec does not include it, or marks it optional | +| **Extra required field in spec** | Spec marks `frn` as required in the request body but code treats it as optional or doesn't validate it | +| **Wrong path parameter name** | Spec says `{userId}` but code registers `{id}` for the same route | +| **Missing auth requirement in spec** | Code enforces JWT auth on a route but the spec shows no `security` requirement (consumers won't know to send a token) | +| **Auth shown but not enforced** | Spec shows `security: [bearerAuth: []]` but code sets `auth: false` on the route (consumers will send tokens unnecessarily, and unauthenticated access isn't documented) | +| **Wrong route path** | Spec documents `/api/v1/blob/{fileId}` but code registers `/api/v1/blobs/{fileId}` (or vice versa) | + +--- + +## P1 — High (resolve before next release) + +Schema-level inaccuracies. Integration will technically work but validation, typing, or assumptions may break for consumers generating clients from the spec. + +| What to Check | Example Drift | +|---------------|---------------| +| **Type mismatch** | Spec says `type: string` but code validates/produces `type: number` (or vice versa) | +| **Format mismatch** | Spec says `format: uuid` but code doesn't validate UUID format (or opposite) | +| **Enum value mismatch** | Code allows status values `['pending', 'success', 'failure']` but spec lists `['pending', 'complete', 'failed']` | +| **Numeric range mismatch** | Spec says `minimum: 100` but code validates `min(50)` | +| **String constraint mismatch** | Spec says `pattern: "^[a-z]+$"` but code uses a different regex | +| **Required vs optional mismatch** | Spec marks a field as required but code treats it as optional (or vice versa) — where this doesn't rise to P0 because the field IS present in both | +| **Missing response status code** | Code returns 422 for validation errors but spec only documents 400 | +| **Extra response status code in spec** | Spec documents 409 Conflict but the code never returns that status | +| **Wrong security scheme type** | Spec says `type: apiKey` but implementation uses bearer JWT | +| **Missing status values in enum** | Code defines status constants `SENT`, `PENDING`, `FAILED` but spec enum only lists `SENT` and `PENDING` | +| **Response body shape mismatch** | Spec says response has `{ data: { url: string } }` but code returns `{ url: string }` (missing wrapper) | +| **Content type mismatch** | Spec says `application/json` for request body but code also accepts `multipart/form-data` | + +--- + +## P2 — Medium (resolve when convenient) + +Correctness gaps unlikely to cause runtime failures, but reduce documentation reliability. + +| What to Check | Example Drift | +|---------------|---------------| +| **Optional field not in spec** | Code conditionally includes a field in the response that the spec doesn't document | +| **Default value mismatch** | Code defaults `pageSize` to `20` but spec says default is `10` (or spec has no default) | +| **Missing examples** | Spec provides no `example` for a field that has non-obvious format (e.g. CRN, SBI, correlation ID) | +| **Constraint looseness** | Spec says `maxLength: 100` but code allows 200 (not breaking but misleading) | +| **Extra optional field in spec** | Spec documents an optional field the code no longer returns or validates | +| **Stale response description** | The spec description for a response doesn't reflect current behaviour | +| **Missing query parameter** | Code accepts an optional query parameter the spec doesn't document | +| **Error body shape incomplete** | Spec documents 400 status but the error response body schema is missing or generic | +| **Validation error format undocumented** | Code returns structured Joi/Zod validation errors but spec just says "Bad Request" | + +--- + +## P3 — Low (nice to have) + +Documentation quality. No functional impact but improves developer experience. + +| What to Check | Example Drift | +|---------------|---------------| +| **Missing descriptions** | A path, operation, parameter, or schema field has no `description` | +| **Cosmetic naming** | Spec uses `uploadId` but code constant is `UPLOAD_ID` (functional match, style mismatch in descriptions) | +| **Missing spec metadata** | `info.version` hasn't been bumped despite changes, or `info.description` is stale | +| **Missing server definitions** | The spec doesn't define server objects for all deployment environments | +| **Missing tags** | Operations have no tags for grouping/discoverability | +| **Schema documentation gaps** | A reusable component schema has no description of its purpose | +| **Missing contact/license info** | `info.contact` or `info.license` fields are absent or stale | +| **Deprecated routes not marked** | Code comments or documentation indicate a route is deprecated but spec doesn't have `deprecated: true` | +| **Missing operationId** | Operations lack `operationId`, making client SDK generation less ergonomic | +| **Tag descriptions missing** | Tags are used but have no descriptions in the `tags` array at the spec root | + +--- + +## Decision Matrix + +When a difference could be classified at multiple levels, use this to decide: + +| Question | If Yes → | +|----------|----------| +| Would a consumer's HTTP client fail, get the wrong data, or hit an undocumented endpoint? | **P0** | +| Would a consumer's generated SDK or validation logic reject valid requests or accept invalid ones? | **P1** | +| Would a consumer be surprised but not broken? | **P2** | +| Is it purely about documentation polish? | **P3** | diff --git a/plugins/openapi-drift-detection/skills/openapi-drift-detection/references/report-template.md b/plugins/openapi-drift-detection/skills/openapi-drift-detection/references/report-template.md new file mode 100644 index 0000000..89cdc34 --- /dev/null +++ b/plugins/openapi-drift-detection/skills/openapi-drift-detection/references/report-template.md @@ -0,0 +1,125 @@ +# Drift Report Template + +Use this template verbatim when generating the drift report. Replace placeholders (`{...}`) with actual values. Omit priority sections that have zero findings. + +--- + +````markdown +# OpenAPI Contract Drift Report + +**Spec**: `{spec_file_path}` (OpenAPI {openapi_version}, info.version: {info_version}) +**Service**: {service_name} +**Date**: {date} +**Overall Status**: {status_indicator} + +> {status_indicator} is one of: +> - 🟢 **No drift** — spec and implementation are aligned +> - 🟡 **Minor drift** — only P2/P3 findings (no breaking contract issues) +> - 🔴 **Significant drift** — P0 or P1 findings present (contract is inaccurate) + +{auto_generated_note} + +--- + +## Summary + +| Metric | Count | +|--------|-------| +| Total differences | {total} | +| P0 Critical | {p0_count} | +| P1 High | {p1_count} | +| P2 Medium | {p2_count} | +| P3 Low | {p3_count} | + +| | Spec | Implementation | +|--|------|----------------| +| Paths | {spec_paths} | {impl_paths} | +| Operations (path + method) | {spec_operations} | {impl_operations} | +| Component schemas | {spec_schemas} | {impl_schemas} | +| Security schemes | {spec_security_schemes} | {impl_security_schemes} | +| Server definitions | {spec_servers} | {impl_servers} | + +--- + +## Findings + +### P0 — Critical (resolve immediately) + +These indicate the spec is materially wrong — consumers relying on this documentation will encounter unexpected behaviour. + +| # | Category | Spec Says | Service Does | Source File | Impact | +|---|----------|-----------|--------------|-------------|--------| +| {n} | {category} | {spec_value} | {impl_value} | `{file_path}` | {impact_description} | + +### P1 — High (resolve before next release) + +These indicate schema-level inaccuracies that could cause validation or integration issues for consumers generating clients from the spec. + +| # | Category | Spec Says | Service Does | Source File | Impact | +|---|----------|-----------|--------------|-------------|--------| +| {n} | {category} | {spec_value} | {impl_value} | `{file_path}` | {impact_description} | + +### P2 — Medium (resolve when convenient) + +These are correctness gaps that are unlikely to cause runtime failures but reduce documentation quality. + +| # | Category | Spec Says | Service Does | Source File | Impact | +|---|----------|-----------|--------------|-------------|--------| +| {n} | {category} | {spec_value} | {impl_value} | `{file_path}` | {impact_description} | + +### P3 — Low (nice to have) + +Documentation quality improvements. + +| # | Category | Spec Says | Service Does | Source File | Impact | +|---|----------|-----------|--------------|-------------|--------| +| {n} | {category} | {spec_value} | {impl_value} | `{file_path}` | {impact_description} | + +--- + +## Proposed Fixes + +For each finding above, a specific change to apply to the OpenAPI spec: + +### Fix #{n}: {short_title} + +- **File**: `{spec_file_path}` +- **Path**: `{json_or_yaml_path}` (e.g. `paths./api/v1/status/{correlationId}.get.responses.422`) +- **Action**: {Add | Modify | Remove} +- **Change**: {precise description of what to add, modify, or remove} +- **Priority**: {P0 | P1 | P2 | P3} + +--- + +## Verification Checklist + +After applying fixes, verify: + +- [ ] All routes registered in code have matching spec path + method entries +- [ ] All spec path + method entries have corresponding code routes +- [ ] All path parameter names match between spec and code +- [ ] All request validation schema fields match spec parameters and request body schemas (names, types, formats, required, enums, constraints) +- [ ] All response status codes in code are documented in the spec +- [ ] All response body shapes match spec response schemas per status code +- [ ] Global auth strategy is reflected in spec security requirement +- [ ] Per-route auth overrides (auth: false) are reflected as `security: []` in spec +- [ ] Security scheme type and configuration match implementation +- [ ] All error status codes and error body shapes are documented +- [ ] Request and response content types match between spec and code +- [ ] Server URLs match deployment environments +- [ ] API version, tags, and metadata are current +- [ ] No spec path, operation, or schema exists without a corresponding implementation +- [ ] No implementation route, schema, or auth config exists without spec documentation +```` + +--- + +## Formatting Rules + +- **Omit empty sections**: If a priority level has zero findings, omit that entire section (don't show an empty table) +- **Source File links**: Use workspace-relative paths (e.g. `src/api/v1/callback/schema.js`) +- **Category values**: Use one of: `Path`, `Request Schema`, `Response Schema`, `Authentication`, `Error Response`, `Content Type`, `Metadata` +- **Impact descriptions**: One sentence explaining the consumer-facing consequence (e.g. "Consumers generating SDKs will have a missing endpoint in their client") +- **Path notation**: Use dot notation for JSON paths from the spec root (e.g. `paths./api/v1/blob/{fileId}.get.responses.200.content.application/json.schema`) +- **Fix numbering**: Must match the finding number for traceability +- **Auto-generated note**: If the spec is auto-generated, include: `> ⚠️ This spec appears to be auto-generated via {tool}. Consider running \`{command}\` to regenerate instead of manually editing.` — otherwise omit the `{auto_generated_note}` line entirely From e7ae8a60e92ea31339910289445df2ad3f260bc4 Mon Sep 17 00:00:00 2001 From: Rana Salem Date: Tue, 28 Apr 2026 22:41:19 +0100 Subject: [PATCH 2/2] update docs --- .github/plugin/marketplace.json | 6 ++++++ docs/README.plugins.md | 3 ++- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/.github/plugin/marketplace.json b/.github/plugin/marketplace.json index 9dfdd88..5c810b4 100644 --- a/.github/plugin/marketplace.json +++ b/.github/plugin/marketplace.json @@ -19,6 +19,12 @@ "description": "Detects drift between AsyncAPI spec (version 3.x) and service implementation, prioritises findings in order of severity, and proposes fixes to ensure service implementation is accurately reflected in its documentation.", "version": "1.0.0", "source": "./plugins/asyncapi-drift-detection" + }, + { + "name": "openapi-drift-detection", + "description": "Detects drift between OpenAPI spec and service implementation, prioritises findings in order of severity, and proposes fixes to ensure service implementation is accurately reflected in its documentation.", + "version": "1.0.0", + "source": "./plugins/openapi-drift-detection" } ] } diff --git a/docs/README.plugins.md b/docs/README.plugins.md index d0e6733..7b299ed 100644 --- a/docs/README.plugins.md +++ b/docs/README.plugins.md @@ -14,5 +14,6 @@ copilot plugin install @dev-suq | Plugin | Description | Install | |--------|-------------| ------- | -| [nodejs-sonarqube-cloud](../plugins/nodejs-sonarqube-cloud/README.md) | Analyses SonarQube Cloud quality gate results from the CI pipeline, triages all issues, and implements fixes automatically. Includes a skill for running local scans via `sonar-scanner-cli`. | `copilot plugin install nodejs-sonarqube-cloud@dev-suq` | | [asyncapi-drift-detection](../plugins/asyncapi-drift-detection/README.md) | Detects drift between AsyncAPI spec (version 3.x) and service implementation, prioritises findings in order of severity, and proposes fixes to ensure service implementation is accurately reflected in its documentation. | `copilot plugin install asyncapi-drift-detection@dev-suq` | +| [nodejs-sonarqube-cloud](../plugins/nodejs-sonarqube-cloud/README.md) | Analyses SonarQube Cloud quality gate results from the CI pipeline, triages all issues, and implements fixes automatically. Includes a skill for running local scans via `sonar-scanner-cli`. | `copilot plugin install nodejs-sonarqube-cloud@dev-suq` | +| [openapi-drift-detection](../plugins/openapi-drift-detection/README.md) | Detects drift between OpenAPI spec and service implementation, prioritises findings in order of severity, and proposes fixes to ensure service implementation is accurately reflected in its documentation. | `copilot plugin install openapi-drift-detection@dev-suq` |