feat(security): PII/PHI redaction for telemetry + EU AI Act Article 15 export (#398)#442
Merged
sreerevanth merged 4 commits intoJun 22, 2026
Merged
Conversation
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (7)
✅ Files skipped from review due to trivial changes (2)
🚧 Files skipped from review as they are similar to previous changes (5)
📝 WalkthroughWalkthroughAdds ChangesPII/PHI Redaction and EU AI Act Compliance Export
Sequence Diagram(s)sequenceDiagram
participant Client
participant GenericAdapter
participant SafetyEngine
participant EventBus
participant Redactor
rect rgba(135, 206, 235, 0.5)
Note over Client,Redactor: Tool-call redaction flow (when redact=True)
Client->>GenericAdapter: invoke tool (raw ToolCallData)
GenericAdapter->>SafetyEngine: evaluate raw ToolCallData
SafetyEngine-->>GenericAdapter: safety decision
GenericAdapter->>Redactor: redact_event (if redact=True)
Redactor-->>GenericAdapter: AgentEvent with scrubbed payload
GenericAdapter->>EventBus: publish TOOL_CALL event (redacted)
end
rect rgba(144, 238, 144, 0.5)
Note over Client,EventBus: Article 15 conformity export
Client->>GenericAdapter: GET /api/v1/governance/eu-ai-act-report
GenericAdapter->>SafetyEngine: stats() + policy
SafetyEngine-->>GenericAdapter: robustness evidence, sessions
GenericAdapter-->>Client: JSON (TechnicalDocumentation, telemetry)
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~50 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
🧪 PR Test Results
Python 3.12 · commit dfc6d64 |
There was a problem hiding this comment.
Pull request overview
Adds an opt-in telemetry redaction layer to scrub PII/PHI before persistence and introduces an EU AI Act Article 15 JSON export endpoint under the existing governance module.
Changes:
- Introduce
agentwatch.security.redactionwith Presidio (optional) + regex fallback redaction helpers for text, nested payloads, and tool calls. - Wire redaction into
GenericAdapterviaredact=True, and addGET /api/v1/governance/eu-ai-act-report. - Add a
redactionoptional dependency extra plus tests and status/changelog updates.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
agentwatch/security/redaction.py |
New redaction module implementing Presidio/regex PII/PHI masking utilities. |
agentwatch/core/watcher.py |
Adds opt-in redaction hook for tool-call telemetry in the watcher adapter. |
agentwatch/api/server.py |
Adds EU AI Act Article 15 report export endpoint. |
pyproject.toml |
Adds optional redaction extra for Presidio dependencies. |
tests/test_redaction.py |
Adds unit/integration tests for redaction behavior, watcher integration, and the new endpoint. |
MASTERLIST_STATUS.md |
Updates compliance phase entries to reflect added deliverables/tests. |
CHANGELOG.md |
Documents the new redaction feature and report endpoint under Unreleased. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
309
to
312
| # ── Safety gate (async path — full check_event with approval) ── | ||
| if is_tool_like: | ||
| tool_call = _build_tool_call_data(method_name, args, kwargs) | ||
| tool_call = self._maybe_redact(_build_tool_call_data(method_name, args, kwargs)) | ||
| safety_event = AgentEvent( |
Comment on lines
373
to
377
| # ── Safety gate (sync path — pattern match only, no approval) ── | ||
| if is_tool_like: | ||
| tool_call = _build_tool_call_data(method_name, args, kwargs) | ||
| tool_call = self._maybe_redact(_build_tool_call_data(method_name, args, kwargs)) | ||
| try: | ||
| blocked, reasons = self._safety_engine.check_tool_call_sync(tool_call) |
Comment on lines
+1080
to
+1092
| """EU AI Act Article 15 conformity export (CMP-004). | ||
|
|
||
| Maps AgentWatch's safety telemetry to the Article 15 requirements and | ||
| returns the technical documentation plus a conformity assessment as JSON. | ||
| """ | ||
| from agentwatch.governance.eu_ai_act import EUAIActPackage, TechnicalDocumentation | ||
|
|
||
| doc = TechnicalDocumentation( | ||
| system_name="AgentWatch-monitored AI system", | ||
| intended_purpose="Observability, safety, and reliability layer for AI agents", | ||
| risk_category="high", | ||
| data_governance={"pii_phi_redaction": "enabled", "retention": "policy-driven"}, | ||
| robustness_evidence=[ |
Comment on lines
+19
to
+21
| from agentwatch.governance.gdpr import _PII_PATTERNS | ||
| from agentwatch.governance.hipaa import _PHI_PATTERNS | ||
|
|
Owner
|
@Prateeks16 The redaction functionality is useful and the overall direction makes sense. However, there are a few issues that need to be addressed before merge:
Once these items are addressed, I'll take another look. |
Contributor
Author
|
Thanks for the review — all three addressed in the latest push:
Full suite green except one pre-existing CLI test ( |
Owner
|
Thanks for addressing the review feedback. The remaining blocker appears to be the merge conflict in CHANGELOG.md. Please resolve the conflict and rebase onto the latest main branch. Once that's done I'll do a final pass. |
…reerevanth#398) CMP-003/004 — auto-redact PII/PHI before telemetry is persisted, and surface an EU AI Act Article 15 conformity export. - agentwatch/security/redaction.py: a Redactor that masks PII/PHI as [REDACTED]. Uses Microsoft Presidio when installed (new optional `redaction` extra), otherwise falls back to the GDPR (CMP-001) and HIPAA (CMP-003) regex detectors — reusing their patterns, no hard dependency. Provides redact()/redact_payload()/redact_tool_call() helpers. - core/watcher.py: opt-in `GenericAdapter(redact=True)` scrubs tool-call payloads (raw_command + arguments) before events are published/persisted. - api/server.py: GET /api/v1/governance/eu-ai-act-report returns the Article 15 technical documentation + conformity assessment as JSON. - Tests: PII/PHI masking, payload recursion, tool-call redaction, the watcher opt-in, and the EU AI Act endpoint. - Docs: CHANGELOG, MASTERLIST (CMP-003/004), and the `redaction` extra.
…export, public patterns Maintainer feedback on sreerevanth#398: 1. Safety must evaluate the raw tool call, not a redacted copy. Redaction ran before the safety check, masking signals (paths/secrets/identifiers) the SafetyEngine relies on and potentially altering block/allow decisions. The watcher now checks the raw payload and scrubs PII/PHI only when building the event that is published/persisted (new _redact_event helper). Added a regression test asserting safety sees the raw command while the published event is redacted. 2. The EU AI Act Article 15 export reported static literals. It now derives data governance, accuracy metrics, robustness evidence, human-oversight description, and record-keeping (decision log) from live safety telemetry and the active policy. 3. Stop importing private governance internals. gdpr/hipaa now expose public pii_patterns() / phi_patterns() accessors; the redactor uses those.
Prateeks16
force-pushed
the
feat/pii-phi-redaction-398
branch
from
d2b2452 to
4229482
Compare
Contributor
Author
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (1)
agentwatch/core/watcher.py (1)
620-627: 💤 Low valueConsider exposing
redactparameter inwatch()for API completeness.The
GenericAdapternow acceptsredact=True, but thewatch()convenience function doesn't expose this option, requiring users to drop down toGenericAdapterdirectly for redaction. For discoverability, consider adding the parameter here:def watch( agent: Any, *, session_id: str | None = None, agent_id: str | None = None, event_bus: EventBus | None = None, + redact: bool = False, ) -> Any:And passing it through:
adapter = GenericAdapter( agent, framework=framework, framework_label=label, event_bus=bus, session_id=session_id, agent_id=agent_id, + redact=redact, )Not blocking since the PR scope explicitly targets
GenericAdapter(redact=True).🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@agentwatch/core/watcher.py` around lines 620 - 627, The watch() function does not expose the redact parameter that GenericAdapter now supports, limiting API discoverability. Add a redact parameter to the watch() function signature with a sensible default value, then pass this redact parameter through to the GenericAdapter instantiation where other parameters like agent, framework, event_bus, session_id, and agent_id are being passed.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@agentwatch/api/server.py`:
- Around line 1137-1156: The code caps decision-log ingestion to the first 50
sessions with the slice `sessions[:50]` in the for loop, but the telemetry
dictionary reports `len(sessions)` for the sessions_considered field, which
represents the full session count rather than the count actually processed. To
fix this inconsistency, either store the sliced sessions in a variable like
`processed_sessions = sessions[:50]` and use that for both the loop and the
sessions_considered count, or directly change the sessions_considered value to
use `len(sessions[:50])` instead of `len(sessions)` to match what the conformity
assessment actually consumed.
In `@agentwatch/security/redaction.py`:
- Around line 87-88: In the redact_payload method, the dictionary redaction
logic on lines 87-88 only redacts the values but leaves keys unchanged, which
can leak sensitive information. Modify the dictionary comprehension to apply the
redact_payload method to both keys and values so that sensitive information in
dictionary keys is also properly redacted and does not appear in the telemetry
output.
In `@tests/test_redaction.py`:
- Around line 133-136: The test function test_eu_ai_act_report_endpoint is
calling an authenticated route at "/api/v1/governance/eu-ai-act-report" without
properly handling API key authentication. This makes the test
environment-sensitive and prone to failure based on AGENTWATCH_API_KEY
configuration. Fix this by either mocking the AGENTWATCH_API_KEY environment
variable to a known test value before making the request, or by providing the
required authentication headers in the client.get() call to ensure the test
passes consistently regardless of the environment configuration.
---
Nitpick comments:
In `@agentwatch/core/watcher.py`:
- Around line 620-627: The watch() function does not expose the redact parameter
that GenericAdapter now supports, limiting API discoverability. Add a redact
parameter to the watch() function signature with a sensible default value, then
pass this redact parameter through to the GenericAdapter instantiation where
other parameters like agent, framework, event_bus, session_id, and agent_id are
being passed.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 51e5beea-f930-45b4-b2b3-01dc453de1db
📒 Files selected for processing (9)
CHANGELOG.mdMASTERLIST_STATUS.mdagentwatch/api/server.pyagentwatch/core/watcher.pyagentwatch/governance/gdpr.pyagentwatch/governance/hipaa.pyagentwatch/security/redaction.pypyproject.tomltests/test_redaction.py
… count, watch(redact), deterministic test - redact_payload now scrubs dict keys as well as values (a key can itself be sensitive, e.g. an email used as a map key). - EU AI Act export: report sessions_considered as the count actually fed to the decision log (sessions_used = sessions[:50]) instead of the full list length. - Expose redact on the watch() convenience function, forwarded to the generic adapter, so the feature is reachable without dropping to GenericAdapter. - Pin _API_KEY off in the endpoint test so it doesn't depend on a stray AGENTWATCH_API_KEY in the environment.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #398
Summary
Implements the two CMP-003/CMP-004 deliverables from the issue: auto-redaction of PII/PHI in telemetry before it is persisted, and an EU AI Act Article 15 conformity export endpoint.
The compliance/governance layer already exists under
agentwatch/governance/(gdpr.py,hipaa.py,eu_ai_act.py,compliance_reporter.py+ the/governance/compliance-reportendpoint). The missing piece — and the focus of this PR — is the redaction layer and wiring it into the telemetry path.What's added
agentwatch/security/redaction.py— aRedactorthat masks PII and PHI as[REDACTED]:presidio-analyzer/presidio-anonymizer) for NER-based detection when installed (new optionalredactionextra), and otherwise falls back to the regex detectors already maintained by the GDPR (CMP-001) and HIPAA (CMP-003) engines — so there's a single source of truth for the patterns and no hard dependency on the heavyweight Presidio stack.redact(),redact_payload()(recursive),redact_tool_call().core/watcher.py— opt-inGenericAdapter(redact=True)scrubs each tool call'sraw_command+argumentsbefore the event is published and persisted.api/server.py—GET /api/v1/governance/eu-ai-act-reportreturns the Article 15 technical documentation + conformity assessment as JSON, mapping AgentWatch's safety telemetry (risk scoring, red-team harness, blast-radius) to the Article 15 requirements.Acceptance criteria
[REDACTED]before persistence — covered byredaction.py+ the watcher opt-in./governance/compliance-reportplus the new/governance/eu-ai-act-reportArticle 15 export.Design notes
redact=True.pip install agentwatch[redaction]).Testing
tests/test_redaction.py: PII masking (SSN/email/phone), PHI masking (MRN/diagnosis/ICD), recursive payload redaction,redact_tool_call, the watcher opt-in (on/off), and the EU AI Act endpoint viaTestClient.Note on scope / process
CONTRIBUTING flags security/compliance features for Discord discussion first — flagging here; happy to adjust. The issue's suggested paths (
agentwatch/compliance/eu_ai_act.py) map to the existingagentwatch/governance/module, which this PR builds on rather than duplicating.The repo-wide
ruff check .gate has pre-existing failures in unrelated test files onmain; this PR's changed files are lint-clean.Summary by CodeRabbit