Skip to content

Harden generated output handling#581

Open
glmgbj233 wants to merge 3 commits into
Agent-Field:mainfrom
glmgbj233:fix-harden-generated-output-handling
Open

Harden generated output handling#581
glmgbj233 wants to merge 3 commits into
Agent-Field:mainfrom
glmgbj233:fix-harden-generated-output-handling

Conversation

@glmgbj233
Copy link
Copy Markdown

@glmgbj233 glmgbj233 commented May 24, 2026

Vulnerability

This fixes a prompt-injection data flow where attacker-controlled or otherwise untrusted input can influence LLM output, and that model output reaches a privileged sink.

  • Source: (*Client).ExecuteToolCallLoopResult - sdk/go/ai/tool_calling.go:169 - sink kind: llm_api_response
  • Sink: (*Agent).Call - sdk/go/agent/agent.go:1596 - sink kind: http_request

Attack scenario

An attacker can influence the LLM output at (*Client).ExecuteToolCallLoopResult - sdk/go/ai/tool_calling.go:169 - sink kind: llm_api_response so the model selects a URL or request target. If that output reaches (*Agent).Call - sdk/go/agent/agent.go:1596 - sink kind: http_request, generated text can drive an unintended network request.

Attack path

  • path 0: (*Client).ExecuteToolCallLoopResult - sdk/go/ai/tool_calling.go:169 - sink kind: llm_api_response -> (*Agent).Call - sdk/go/agent/agent.go:1596 - sink kind: http_request
  • path 1: (*Client).ExecuteToolCallLoopResult - sdk/go/ai/tool_calling.go:169 - sink kind: llm_api_response -> (*Agent).Call - sdk/go/agent/agent.go:1595 - sink kind: http_request

Fix

The fix adds a trust-boundary check before LLM-derived data can reach the sink, while preserving the intended benign workflow. The dangerous effect is blocked after the LLM step instead of only reformatting or re-marshalling model output.

Changed files:

  • sdk/go/agent/agent.go
  • sdk/go/agent/agent_test.go

Why this mitigates the issue

Prompt injection is mitigated by preventing model-controlled text from directly selecting, poisoning, replaying, executing, or persisting a privileged action across the identified boundary. Benign model output can still follow the constrained safe path.

Functionality preservation

The repair is intended to keep normal safe behavior available and restrict only unsafe LLM-derived behavior at the sink boundary. Normal-case and exploit-regression coverage should be reviewed in the changed tests.

Tests / verification

Local Go verification: passed, return code 0

GOTOOLCHAIN=auto GOPROXY=https://goproxy.cn,direct GOMODCACHE=<go-build-cache> GOCACHE=<go-tmp> go test ./agent

Release notes

Security fix only; no user-facing release note is required.

Repository release-note guidance checked:

  • .github/PULL_REQUEST_TEMPLATE.md
  • CHANGELOG.md

Keep generated output behind the intended trust boundary while preserving the normal safe workflow.

Add regression coverage for the unsafe flow and the expected safe behavior.
@glmgbj233 glmgbj233 requested review from a team and AbirAbbas as code owners May 24, 2026 02:23
@CLAassistant
Copy link
Copy Markdown

CLAassistant commented May 24, 2026

CLA assistant check
All committers have signed the CLA.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 24, 2026

Performance

SDK Memory Δ Latency Δ Tests Status
Go 219 B -22% 0.62 µs -38%

✓ No regressions detected

@github-actions
Copy link
Copy Markdown
Contributor

📊 Coverage gate

Thresholds from .coverage-gate.toml: per-surface ≥ 86%, aggregate ≥ 88%, max per-surface regression ≤ 1.0 pp, max aggregate regression ≤ 0.50 pp.

Surface Current Baseline Δ
control-plane 87.50% 87.30% ↑ +0.20 pp 🟡
sdk-go 91.80% 90.70% ↑ +1.10 pp 🟢
sdk-python 93.73% 93.63% ↑ +0.10 pp 🟢
sdk-typescript 92.80% 92.56% ↑ +0.24 pp 🟢
web-ui 89.91% 90.01% ↓ -0.10 pp 🟡
aggregate 89.02% 89.01% ↑ +0.01 pp 🟡

✅ Gate passed

No surface regressed past the allowed threshold and the aggregate stayed above the floor.

@github-actions
Copy link
Copy Markdown
Contributor

📐 Patch coverage gate

Threshold: 80% on lines this PR touches vs origin/main (from .coverage-gate.toml:thresholds.min_patch).

Surface Touched lines Patch coverage Status
control-plane 0 ➖ no changes
sdk-go 20 70.00%
sdk-python 0 ➖ no changes
sdk-typescript 0 ➖ no changes
web-ui 0 ➖ no changes

❌ Patch gate failed

sdk-go — 70.00% on 20 touched lines (6 uncovered):

File Patch coverage Missing lines
sdk/go/agent/agent.go 70.0% 1847, 1848, 1849, 1851, 1852, 1853

How to fix

  1. For each file listed above, add tests that exercise the missing line numbers in this same PR.
  2. Re-run locally: ./scripts/coverage-summary.sh && ./scripts/patch-coverage-gate.sh.
  3. Do not lower min_patch in .coverage-gate.toml to silence this — the floor is the contract.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants