Skip to content

feat: native contract coverage for MCP corruption corpus#196

Merged
ProfRandom92 merged 5 commits into
mainfrom
codex/add-native-contract-coverage-for-mcp-corruption
May 21, 2026
Merged

feat: native contract coverage for MCP corruption corpus#196
ProfRandom92 merged 5 commits into
mainfrom
codex/add-native-contract-coverage-for-mcp-corruption

Conversation

@ProfRandom92
Copy link
Copy Markdown
Owner

Motivation

  • Scope: Add native contract coverage for the existing MCP corruption corpus so every materialized corruption emits its manifest expected_failure_label deterministically.
  • Research Alignment: Strengthen deterministic replay-integrity validation by making known ground-truth corruptions validator-detectable via explicit, narrow contracts rather than probabilistic heuristics.
  • Non-Goals Confirmed: No runtime orchestration, LLM judging, embeddings/vector search, fuzzy semantic matching, cloud dependencies, dashboard/SaaS behavior, or unrelated README/workflow changes were introduced.

Description

  • Added deterministic contract rules to ContractValidator: terminal-action checks for ordering contracts and three invariant rules required_boundaries_preserved, required_dependency_edges_preserved, and required_action_order_preserved to detect missing boundaries, missing dependency edges, and action-order/terminal mismatches respectively (changes in src/validation/contract_validator.py).
  • Updated fixture contract JSONs across MCP families to explicitly encode the new narrow rules and configured failure_label_on_violation values (added approval_gate_preserved.json and edited capability_boundary_respected.json, dependency_chain_preserved.json, recovery_path_available.json, tool_call_order_preserved.json, validation_before_unsafe_action.json under fixtures/mcp_trace_replay_{degraded,mild,moderate}_v1/original/contracts).
  • Removed the stale adapter-gap classification from the MCP corruption validation test and made the test require that every materialized manifest entry produce its manifest expected_failure_label; added focused unit tests exercising the new invariant rules (changes in tests/test_mcp_trace_corruption_validation.py and tests/test_contract_validator.py).
  • Failure labels reused: APPROVAL_GATE_LOSS, TOOL_ORDER_VIOLATION, RECOVERY_PATH_INVALID, DEPENDENCY_CHAIN_BREAK, POLICY_ENFORCEMENT_GAP, and CAPABILITY_BOUNDARY_LOSS (no new taxonomy labels introduced).
  • Artifacts: No deterministic artifacts were regenerated by scripts as part of this change; only fixture contract JSONs were edited to make contracts explicit and deterministic for native validation.
  • Determinism: All checks are explicit, JSON-driven, and deterministic (no network/LLM/probabilistic components), preserving offline and reproducible validation behavior.

Testing

  • pytest -q tests/test_contract_validator.py — passed (unit tests for new rules added and existing ordering/reachability/causality/invariant tests pass).
  • pytest -q tests/test_mcp_trace_corruption_validation.py — passed (all materialized MCP corruption entries now natively emit their manifest expected_failure_label).
  • pytest -q tests/test_mcp_trace_corruption_manifest.py tests/test_mcp_trace_corruption_materialization.py — passed (manifest and materialization checks unaffected).
  • npm run check — targeted MCP-corruption tests pass, but full npm run check runs the entire repository test/validation/build pipeline and surfaced unrelated broader fixture-manifest/artifact expectation mismatches caused by tightening contract definitions; these are out of scope for this narrow PR.

Open a PR against main.


Codex Task

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request enhances the contract validation system by introducing new invariant rules, specifically 'required_boundaries_preserved' and 'required_dependency_edges_preserved', and adding support for 'terminal_action' checks within ordering validation. Correspondingly, several contract fixtures were updated to use these new rules, and the test suite was adjusted to reflect increased validation coverage. Feedback from the reviewer focuses on improving the robustness of JSON parsing by ensuring that null or missing list fields are treated as empty lists and that 'RuntimeError' is raised for invalid types to maintain strictness, particularly for 'required_boundaries', 'capability_boundaries', and 'required_edges'.

Comment thread src/validation/contract_validator.py Outdated
Comment thread src/validation/contract_validator.py Outdated
Comment thread src/validation/contract_validator.py Outdated
@ProfRandom92 ProfRandom92 merged commit 59bd654 into main May 21, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant