Skip to content

feat: add MCP corruption corpus coverage artifact#197

Merged
ProfRandom92 merged 4 commits into
mainfrom
codex/add-mcp-corruption-corpus-coverage-artifact
May 21, 2026
Merged

feat: add MCP corruption corpus coverage artifact#197
ProfRandom92 merged 4 commits into
mainfrom
codex/add-mcp-corruption-corpus-coverage-artifact

Conversation

@ProfRandom92
Copy link
Copy Markdown
Owner

Motivation

  • Add a deterministic, artifact-backed summary that proves the materialized MCP corruption corpus is fully native-covered (18/18) and reproducible from committed fixtures, manifest, contracts, and ContractValidator.
  • Keep the project artifact-first, deterministic, narrowly-scoped, and audit-oriented without adding runtime orchestration, LLM judging, or external dependencies.

Description

  • Add scripts/generate_mcp_corruption_coverage_artifact.py which loads artifacts/mcp_trace_corruption_manifest.json, source originals, materialized corruptions, and original contracts, then runs ContractValidator to compute observed failure labels and per-entry native coverage and evidence, failing on missing files, unregistered labels, malformed entries, or adapter gaps.
  • Commit the deterministic artifact artifacts/mcp_corruption_coverage.json (sorted keys, indent=2, no timestamps, repository-relative paths) that contains top-level metadata, coverage summary, operator/family/label distributions, and per-entry deterministic evidence.
  • Add tests tests/test_mcp_corruption_coverage_artifact.py that assert committed artifact equals regenerated output, generation stability, exact summary/distribution counts, per-entry coverage invariants, taxonomy registration, no absolute paths/timestamps, and generator failure on missing expected labels.
  • Enforce strict invariants only (no new validation semantics or failure labels), and keep the implementation standard-library-only, simple, and auditable.

Testing

  • Changed files: scripts/generate_mcp_corruption_coverage_artifact.py, artifacts/mcp_corruption_coverage.json, tests/test_mcp_corruption_coverage_artifact.py.
  • Ran python scripts/generate_mcp_corruption_coverage_artifact.py which completed and wrote artifacts/mcp_corruption_coverage.json (pass).
  • Ran pytest -q tests/test_mcp_corruption_coverage_artifact.py (6 passed).
  • Ran pytest -q tests/test_mcp_trace_corruption_validation.py (6 passed), pytest -q tests/test_mcp_trace_corruption_manifest.py tests/test_mcp_trace_corruption_materialization.py (combined pass), and pytest -q tests/test_contract_validator.py (19 passed).
  • Ran full repository check npm run check which executed the python and dashboard checks and pytest suite and completed successfully (321 passed).

Codex Task

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a system for generating and verifying MCP corruption coverage artifacts, including a generation script, the resulting coverage JSON, and a comprehensive test suite. The reviewer provided feedback to improve the robustness of JSON parsing by handling null values and validating item types, suggested removing redundant sorting logic in evidence key collection, and recommended using idiomatic pytest fixtures for mocking instead of subprocess calls.

Comment thread scripts/generate_mcp_corruption_coverage_artifact.py
Comment thread scripts/generate_mcp_corruption_coverage_artifact.py Outdated
Comment thread scripts/generate_mcp_corruption_coverage_artifact.py
Comment thread tests/test_mcp_corruption_coverage_artifact.py Outdated
@ProfRandom92 ProfRandom92 merged commit 003e270 into main May 21, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant