Skip to content

Harden receipt and CLI test coverage#17

Draft
aniketh-maddipati wants to merge 1 commit into
mainfrom
codex/harden-test-suite
Draft

Harden receipt and CLI test coverage#17
aniketh-maddipati wants to merge 1 commit into
mainfrom
codex/harden-test-suite

Conversation

@aniketh-maddipati
Copy link
Copy Markdown
Collaborator

This PR hardens the test suite around the current public Notary and CLI behavior so regressions in receipt signing, chain verification, schema shape, and command-line flows fail fast.

The user-visible problem was that the repository had good targeted coverage in a few areas, but it still had gaps around shared test setup, cryptographic known-answer regressions, invariant-style receipt verification, and end-to-end CLI flows. That made it easier for behavioral drift to slip in without a tight, realistic test catching it.

The root cause was mostly fragmentation. Some coverage existed already, but it lived in separate test files with duplicated setup, and the rebased upstream branch had also changed the CLI surface and already introduced overlapping tests/test_aerf_conformance.py and tests/cli/test_e2e.py files. Rebasing this work cleanly therefore required resolving those overlaps instead of just adding duplicate files.

This change adds shared pytest fixtures in tests/conftest.py for a persistent-key Notary, a recording sink, a reusable plan, and the checked-in AERF schema. It adds deterministic case generators in tests/strategies.py, cryptographic regression tests in tests/test_crypto_kat.py, invariant-oriented receipt and chain tests in tests/test_invariants.py, and broader integration coverage in tests/test_integration.py. It also expands tests/test_aerf_conformance.py from a single schema check into multiple receipt-shape and chain-shape validations using the upstream schema file.

On the CLI side, this PR resolves the upstream conflict by rewriting tests/cli/test_e2e.py against the Typer-based CLI that actually ships on origin/main. The new E2E coverage exercises init, doctor, privacy, verify, and show, including a real generated receipt round-trip, rather than relying on older commands and entrypoints that are no longer present on the rebased branch.

Validation on the rebased branch:

  • python3 -m pytest -q -> 207 passed
  • python3 -m pip install -e . -> passed
  • agentmint --help -> passed
  • python3 -m pytest tests/test_aerf_conformance.py tests/test_crypto_kat.py tests/test_invariants.py tests/test_integration.py tests/cli/test_e2e.py -q -> 26 passed

One documented follow-up check could not be run on the rebased branch: python3 -m agentmint.demo.healthcare fails because origin/main does not contain that module.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant