Skip to content

Add tests for plugin failure modes (timeouts, bad JSON, missing evidence) #4

@PAXECT-Interface

Description

@PAXECT-Interface

Summary

Add more tests to cover plugin failure modes in Open Agentic 2.0.
The goal is to make sure the orchestrator behaves correctly when plugins fail
(e.g. timeouts, bad JSON, or missing evidence).

What to test

Where possible, add or extend tests in tests/test_agentic2.py (or a new test
file if that makes more sense) to cover cases like:

  • a plugin call that times out (e.g. subprocess / HTTP timeout),
  • a plugin that returns invalid JSON or non-JSON output,
  • a plugin that returns a response without evidence or with too-low coverage.

When this is done

  • There are automated tests that simulate plugin failures (timeout / bad JSON /
    missing or low-quality evidence).
  • The orchestrator:
    • logs these failures in the audit chain, and
    • either blocks / abstains as expected (fail-closed), instead of silently
      returning ok=True.
  • Running pytest -q -vv still passes.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions