Add integration tests for agent with guardrails #378
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Integration Tests for Agent Guardrails
Overview
Adds comprehensive integration tests for agent guardrails, verifying proper invocation and enforcement at Agent, LLM, and Tool scopes.
Files Added
tests/cli/test_agent_with_guardrails.py(1325 lines) - 8 integration teststests/cli/mocks/joke_agent_with_guardrails.py(333 lines) - Mock agent with 6 guardrailstests/cli/mocks/joke_agent_uipath.json(139 lines) - Guardrail configurationstests/cli/mocks/joke_agent_langgraph.json(8 lines) - LangGraph configtests/cli/conftest.py(updated) - Shared fixtures for mockingTest Coverage
Agent-Level Guardrails
test_pii_guardrail_not_triggeredtest_pii_guardrail_triggeredLLM-Level Guardrails
test_prompt_injection_guardrail_triggeredtest_llm_pii_escalation_guardrail_hitltest_llm_pii_escalation_guardrail_rejectedTool-Level Guardrails
test_tool_guardrail_filter_outputtest_tool_guardrail_block_executiontest_tool_pii_guardrail_triggeredGuardrails Tested
Built-in Validators
Custom Deterministic Guardrails
input_phrasefield when containing "donkey"Actions Tested