Skip to content

fix: honest inbound guardrail messaging (live E2E finding)#9

Merged
sidmohan0 merged 1 commit into
mainfrom
fix/honest-inbound-guardrail
Jul 2, 2026
Merged

fix: honest inbound guardrail messaging (live E2E finding)#9
sidmohan0 merged 1 commit into
mainfrom
fix/honest-inbound-guardrail

Conversation

@sidmohan0

Copy link
Copy Markdown
Contributor

Summary

The full agent-turn E2E (isolated --profile fogclaw-e2e, openclaw 2026.6.11, real Anthropic haiku calls) surfaced a design truth no unit test could: inbound redaction has always been advisory. prependContext adds the redacted copy but cannot remove the original prompt, and the plugin API has no prompt-rewrite hook (verified against every hook signature in the SDK — llm_input and message_received are observational). In the live run the model described seeing both copies and repeated the raw email when asked.

Live E2E results:

  • Inbound scan + audit: ✓ fires (regex + GLiNER live, zero-shot labels detected)
  • before_agent_run block gate: ✓ fully enforced — run stopped before any model call, user received the block message
  • Old inbound wording ("the following is the user's message with PII redacted") was false and confused the model → new wording flags entity types, instructs placeholder use, and provides the placeholder reference
  • Re-run after fix: asked to echo the email character-for-character, the model declines and offers placeholder-based handling
  • Outbound hooks fire only on channel delivery — --local CLI turns don't exercise them (unit-tested; live check needs a real channel)

README/ARCHITECTURE now state the enforcement boundary honestly: advisory inbound + hard block gate + enforced rewriting on tool results and outbound delivery.

Test plan

  • 233 unit tests pass, tsc clean
  • Live turn 1 (before): model repeats raw email → confirmed flaw
  • Live turn 2 (block mode): run blocked pre-model
  • Live turn 3 (after fix): model refuses to repeat raw email, offers placeholders

Live agent-turn E2E (openclaw 2026.6.11, real Anthropic call) showed the
model receives both the prepended redacted copy and the original prompt:
prependContext cannot remove the original, and the plugin API has no
prompt-rewrite hook. The old text claimed the message was redacted,
which was false and visibly confused the model into repeating the raw
value. The inbound context now flags entity types, instructs the agent
to use placeholders, and provides the placeholder reference. Re-run
E2E: asked to echo the email verbatim, the model declines and offers
placeholder-based handling. Docs updated to state the enforcement
boundary: block gate (live-verified to stop runs pre-model), tool-result
rewriting, and outbound delivery rewriting.
@sidmohan0 sidmohan0 merged commit 1788530 into main Jul 2, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant