feat: Add configurable security policy support to Agent #427

neubig · 2025-09-22T16:55:30Z

Adds security_policy_filename field to Agent class for configurable security policies.

Backward compatible: Default behavior unchanged (uses security_policy.j2)
Custom policies: Specify custom Jinja2 templates via security_policy_filename parameter
Path support: Both relative (to prompts dir) and absolute paths supported
Example: See examples/20_security_policy/ for usage
Tests: 2 integration tests verify functionality

# Default
agent = Agent(llm=llm)  # Uses security_policy.j2

# Custom
agent = Agent(llm=llm, security_policy_filename="custom_policy.j2")

Agent Server images for this PR

• GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server

Variants & Base Images

Variant	Architectures	Base Image	Docs / Tags
java	amd64, arm64	`eclipse-temurin:17-jdk`	Link
python	amd64, arm64	`nikolaik/python-nodejs:python3.12-nodejs22`	Link
golang	amd64, arm64	`golang:1.21-bookworm`	Link

Pull (multi-arch manifest)

# Each variant is a multi-arch manifest supporting both amd64 and arm64
docker pull ghcr.io/openhands/agent-server:84425c1-python

Run

docker run -it --rm \
  -p 8000:8000 \
  --name agent-server-84425c1-python \
  ghcr.io/openhands/agent-server:84425c1-python

All tags pushed for this build

ghcr.io/openhands/agent-server:84425c1-golang-amd64
ghcr.io/openhands/agent-server:84425c1-golang_tag_1.21-bookworm-amd64
ghcr.io/openhands/agent-server:84425c1-golang-arm64
ghcr.io/openhands/agent-server:84425c1-golang_tag_1.21-bookworm-arm64
ghcr.io/openhands/agent-server:84425c1-java-amd64
ghcr.io/openhands/agent-server:84425c1-eclipse-temurin_tag_17-jdk-amd64
ghcr.io/openhands/agent-server:84425c1-java-arm64
ghcr.io/openhands/agent-server:84425c1-eclipse-temurin_tag_17-jdk-arm64
ghcr.io/openhands/agent-server:84425c1-python-amd64
ghcr.io/openhands/agent-server:84425c1-nikolaik_s_python-nodejs_tag_python3.12-nodejs22-amd64
ghcr.io/openhands/agent-server:84425c1-python-arm64
ghcr.io/openhands/agent-server:84425c1-nikolaik_s_python-nodejs_tag_python3.12-nodejs22-arm64
ghcr.io/openhands/agent-server:84425c1-golang
ghcr.io/openhands/agent-server:84425c1-java
ghcr.io/openhands/agent-server:84425c1-python

About Multi-Architecture Support

Each variant tag (e.g., 84425c1-python) is a multi-arch manifest supporting both amd64 and arm64
Docker automatically pulls the correct architecture for your platform
Individual architecture tags (e.g., 84425c1-python-amd64) are also available if needed

- Add security_policy_filename field to Agent model with default 'security_policy.j2' - Update system_prompt.j2 to use configurable security policy template - Add comprehensive tests for security policy configuration - Add example demonstrating configurable security policy usage - All tests pass and pre-commit hooks validated Co-authored-by: openhands <openhands@all-hands.dev>

- Merged security_policy.j2 from PR #422 with configurable security policy feature - Updated system_prompt.j2 to use configurable security_policy_filename in SECURITY section - Combined test suites from both branches in test_security_policy_integration.py - Maintained backward compatibility with default security_policy.j2 filename Co-authored-by: openhands <openhands@all-hands.dev>

…ctories - Updated test_security_policy_filename_in_system_message to copy security_risk_assessment.j2 - Updated test_configurable_security_policy_filename to copy security_risk_assessment.j2 - All tests now pass successfully Co-authored-by: openhands <openhands@all-hands.dev>

- Reduced from 7 tests to 3 focused tests - test_default_security_policy_filename: Verifies default filename - test_custom_security_policy_filename: Verifies custom filename can be set - test_custom_security_policy_in_system_message: Verifies custom policy is used in system message - Removed redundant and overly complex test cases - Tests will run in GitHub CI via sdk-tests job Co-authored-by: openhands <openhands@all-hands.dev>

- Implement FlexibleFileSystemLoader to support both relative and absolute paths - Update field descriptions to document absolute path support - Add comprehensive tests for absolute path functionality - Maintain backward compatibility with existing relative paths - Follow Jinja2 best practices for custom loaders Co-authored-by: openhands <openhands@all-hands.dev>

- Add absolute_path_example.py demonstrating filesystem-wide template loading - Update README.md with comprehensive path resolution documentation - Show both relative and absolute path usage patterns Co-authored-by: openhands <openhands@all-hands.dev>

- Remove trivial tests for default/custom security policy filenames - Remove monkeypatch mocking from custom security policy test - Keep test_security_policy_in_system_message as requested - Use absolute paths for both system prompt and security policy in custom test - All 4 tests now pass cleanly Co-authored-by: openhands <openhands@all-hands.dev>

- Remove test_absolute_path_security_policy() and test_absolute_path_system_prompt() - Absolute path functionality is already tested in test_custom_security_policy_in_system_message() - Remove absolute_path_example.py as it's redundant - Keep only essential tests: test_security_policy_in_system_message() and test_custom_security_policy_in_system_message() Co-authored-by: openhands <openhands@all-hands.dev>

- Remove shebang and main() function wrapper to match other examples - Use standard openhands.sdk imports and LLM configuration pattern - Add actual conversation demonstration instead of just configuration - Include callback pattern for collecting LLM messages - Use simpler docstring and print statements - Focus on demonstration rather than explanation - Follow the same structure as examples 01-03 Co-authored-by: openhands <openhands@all-hands.dev>

…ample - Remove references to deleted absolute_path_example.py - Simplify documentation by removing redundant code examples - Consolidate path resolution explanation into a brief mention - Update file list and running instructions - Make documentation more concise and focused Co-authored-by: openhands <openhands@all-hands.dev>

neubig · 2025-09-23T14:36:31Z

@OpenHands resolve any conflicts and make sure that tests still pass

openhands-ai · 2025-09-23T14:36:41Z

I'm on it! neubig can track my progress at all-hands.dev

Co-authored-by: openhands <openhands@all-hands.dev>

…test The test was expecting the SECURITY_RISK_ASSESSMENT section to be present but wasn't providing a security analyzer. After the merge with main, the behavior changed so that this section is only included when there's an LLMSecurityAnalyzer present. Co-authored-by: openhands <openhands@all-hands.dev>

openhands-ai · 2025-09-23T14:46:28Z

Summary

I have successfully resolved the conflicts and ensured all tests pass for the configurable-security-policy branch:

✅ Request Completion Checklist:

Conflicts resolved: Merged main branch into configurable-security-policy branch
Tests passing: All 1182 tests now pass (previously 1 test was failing)
Changes pushed: Updates have been pushed to the remote branch

🔧 Changes Made:

Single focused fix in tests/sdk/agent/test_security_policy_integration.py:

Added security_analyzer=LLMSecurityAnalyzer() to the test_security_policy_in_system_message test
This test was expecting the <SECURITY_RISK_ASSESSMENT> section in the system message, but after merging with main, this section is only included when an LLMSecurityAnalyzer is present

🧪 Verification:

Full test suite: All 1182 tests pass
Security policy tests: All 6 security policy integration tests pass
Cross-package tests: All 23 cross-package integration tests pass
Pre-commit hooks: All formatting and linting checks pass

The configurable security policy feature remains fully functional and backward compatible. The PR is now updated with the conflict resolution and test fix.

_{View full conversation}

github-actions · 2025-09-23T15:04:49Z

Agent Server image for this PR

Pull (multi-arch manifest):

docker pull ghcr.io/all-hands-ai/agent-server:cb0f50a

Run:

docker run -it --rm \
  -p 8000:8000 \
  --name agent-server-cb0f50a \
  ghcr.io/all-hands-ai/agent-server:cb0f50a

This tag is a multi-arch manifest (amd64/arm64). Your client pulls the right arch automatically.

github-actions · 2025-10-16T12:19:24Z

[Automatic Post]: It has been a while since there was any activity on this PR, are you still working on it?

neubig · 2025-10-16T13:55:44Z

@OpenHands Fix merge conflicts for this

openhands-ai · 2025-10-16T13:55:53Z

I'm on it! neubig can track my progress at all-hands.dev

Resolved conflicts in test_security_policy_integration.py: - Kept test_custom_security_policy_in_system_message from this branch (PR feature) - Integrated new test_security_policy_template_rendering from main - Updated all LLM constructors to include service_id parameter - Updated first test with new assertions from main - Added new imports required by main tests Co-authored-by: openhands <openhands@all-hands.dev>

openhands-ai · 2025-10-16T14:01:05Z

Summary

I have successfully resolved the merge conflicts for PR #427 (configurable-security-policy branch) and pushed the changes to GitHub.

What Was Done

✅ Merge Conflicts Resolved:

Fetched and merged the latest changes from main branch into configurable-security-policy
Resolved conflicts in tests/sdk/agent/test_security_policy_integration.py

Conflict Resolution Details

The conflict was in a single test file. I integrated changes from both branches:

Kept from this branch (PR feature):
- test_custom_security_policy_in_system_message() - the core feature of this PR that tests custom security policy templates
Integrated from main:
- Updated imports to include Conversation, ActionEvent, AgentErrorEvent, Message, TextContent
- Refactored test_security_policy_in_system_message() with more comprehensive assertions about the actual security policy content
- Added test_security_policy_template_rendering() - new test from main
- Updated all LLM constructors throughout the file to include the service_id parameter
- Added test_security_risk_param_ignored_when_no_analyzer() and helper function _tool_response()

Changes Pushed

Commit: 3a4aef24 - "Merge main into configurable-security-policy"
Branch: configurable-security-policy
Status: Successfully pushed to origin/configurable-security-policy
Files Changed: 406 files (as part of the merge from main)
Key Conflict File: tests/sdk/agent/test_security_policy_integration.py - properly resolved to preserve PR functionality while integrating main's improvements

The PR is now up-to-date with the main branch and ready for CI to run tests. The merge preserves the custom security policy feature while incorporating all the latest improvements from main.

_{View full conversation}

github-actions · 2025-10-16T14:03:32Z

Coverage Report •

File	Stmts	Miss	Cover	Missing
openhands-sdk/openhands/sdk/agent
base.py	176	19	89%	165, 171, 190, 236–237, 248–250, 263, 271–272, 306, 353, 360, 373, 410–411, 421–422
openhands-sdk/openhands/sdk/context/prompts
prompt.py	54	13	75%	27, 42–43, 50–51, 60, 82–83, 107–111
TOTAL	12697	5683	55%

- Apply fix from commit 35d042e to make openhands.tools.glob import-safe - Provide stdlib-compatible glob/iglob/escape to avoid build-time import errors - Use lazy imports via __getattr__ to avoid import-time side effects - Fix import order in test_security_policy_integration.py Co-authored-by: openhands <openhands@all-hands.dev>

neubig · 2025-10-17T15:34:07Z

@OpenHands

Check pre-commit and make it pass
Check the new structure of the examples directory, and re-jigger the current example to fit that structure.

blacksmith-sh · 2025-12-02T13:05:47Z

[Automatic Post]: It has been a while since there was any activity on this PR. @neubig, are you still working on it? If so, please go ahead, if not then please request review, close it, or request that someone else follow up.

…y_filename The system_prompt.j2 template now requires security_policy_filename to be passed as a context variable. Updated the test to provide this required parameter. Co-authored-by: openhands <openhands@all-hands.dev>

Add documentation for custom security policy templates that allow users to define organization-specific risk assessment guidelines. Relates to OpenHands/software-agent-sdk#427

examples/01_standalone_sdk/32_configurable_security_policy.py

neubig · 2025-12-08T17:04:32Z

The check-examples CI is failing because this PR adds a new example file that requires documentation. I have created the documentation in OpenHands/docs#158. Once that docs PR is merged, the check-examples CI will pass.

Address review comment from xingyaoww Co-authored-by: openhands <openhands@all-hands.dev>

neubig · 2025-12-08T17:23:58Z

Fixed the check-examples failure:

The docs branch name didn't match the SDK branch name. I've:

Renamed the docs branch from document-configurable-security-policy to configurable-security-policy to match
Created a new docs PR: docs: Add configurable security policy documentation docs#161

The check-examples workflow should now pass once GitHub Actions picks up the new branch. You may need to trigger a re-run of the workflow.

neubig · 2025-12-18T20:24:32Z

@OpenHands merge main, update the example number to be the appropriate most recent one, and then run the openhands CLI (https://github.com/OpenHands/OpenHands-CLI) in headless mode using this commit and write the log to a file. tell me the location of the log file demonstrating that with appropriate settings a custom security policy will be created and the agent will follow it. If you are not able to demonstrate this, then explain what additional changes would need to be made to make this possible.

openhands-ai · 2025-12-18T20:24:43Z

I'm on it! neubig can track my progress at all-hands.dev

Resolved conflict in base.py by keeping both: - security_policy_filename from configurable-security-policy branch - model_name, model_family, model_variant from main branch Co-authored-by: openhands <openhands@all-hands.dev>

Co-authored-by: openhands <openhands@all-hands.dev>

openhands-ai · 2025-12-18T20:33:23Z

Summary

I have completed all the requested tasks for PR #427 (configurable-security-policy):

✅ Checklist

Merged main branch - Successfully merged main into configurable-security-policy, resolving conflicts in base.py (kept both security_policy_filename and model_name/family/variant changes)
Updated example number - Renamed 31_configurable_security_policy.py to 32_configurable_security_policy.py to avoid conflict with 31_iterative_refinement.py that was added to main
Ran OpenHands CLI in headless mode - Configured the OpenHands-CLI to use the local SDK from this branch and ran a headless conversation demonstrating the custom security policy feature
Verified custom security policy works - The log file demonstrates:
- Custom security policy is loaded: security_policy_filename: '/workspace/project/test_security_policy/custom_security_policy.j2'
- Custom policy appears in system message with unique identifier CUSTOM_SECURITY_POLICY_TEST_12345
- Agent follows the custom policy (assessed security risk as "LOW" per the custom policy)

📁 Log File Location

The log file demonstrating the custom security policy feature is at:
/workspace/project/test_security_policy/headless_conversation.log

Key evidence from the log:

<SECURITY>
# 🔐 CUSTOM SECURITY POLICY FOR TESTING

## UNIQUE IDENTIFIER: CUSTOM_SECURITY_POLICY_TEST_12345

This is a custom security policy template that demonstrates the configurable security policy feature.
...
</SECURITY>

📝 Changes Pushed

The branch has been pushed to GitHub with commits:

c8d9663c - Rename example to 32 to avoid conflict with 31_iterative_refinement.py
5c28a19e - Merge main into configurable-security-policy

⚠️ Note on CLI Integration

The OpenHands CLI does not currently expose the security_policy_filename parameter through its command-line interface or settings. To fully integrate this feature into the CLI, the following changes would be needed:

Add --security-policy flag to the CLI argument parser
Update AgentStore.create_and_save_from_settings() to accept and persist security_policy_filename
Update AgentStore.load() to restore the security_policy_filename from saved settings

The SDK feature itself works correctly as demonstrated by the test scripts.

_{View full conversation}

The test_custom_security_policy_in_system_message test was failing because it didn't copy the self_documentation.j2 template to the temp directory. The system_prompt.j2 template includes self_documentation.j2, so it needs to be present for the template to render correctly. Co-authored-by: openhands <openhands@all-hands.dev>

Add documentation for custom security policy templates that allow users to define organization-specific risk assessment guidelines. Relates to OpenHands/software-agent-sdk#427

The glob shadowing fix is unrelated to the configurable security policy feature and should be in a separate PR if needed. Co-authored-by: openhands <openhands@all-hands.dev>

openhands-ai · 2025-12-18T20:47:05Z

Looks like there are a few issues preventing this PR from being merged!

GitHub Actions are failing:
- [Optional] Docs example

If you'd like me to help, just leave a comment, like

@OpenHands please fix the failing actions on PR #427 at branch `configurable-security-policy`

Feel free to include any additional details that might help me get this PR into a better state.

_{^{You can manage your notification settings}}

openhands-sdk/openhands/sdk/context/prompts/prompt.py

* docs: Add configurable security policy documentation Add documentation for custom security policy templates that allow users to define organization-specific risk assessment guidelines. Relates to OpenHands/software-agent-sdk#427 * Update security policy example number from 28 to 31 Address review comment from xingyaoww Co-authored-by: openhands <openhands@all-hands.dev> * Update security policy example number from 31 to 32 The example was renamed to 32 to avoid conflict with 31_iterative_refinement.py Co-authored-by: openhands <openhands@all-hands.dev> --------- Co-authored-by: openhands <openhands@all-hands.dev>

xingyaoww

LGTM!

neubig force-pushed the configurable-security-policy branch from 0935302 to 27b1c1c Compare September 22, 2025 17:02

neubig force-pushed the configurable-security-policy branch from 27b1c1c to 040e162 Compare September 22, 2025 17:13

openhands-agent added 9 commits September 22, 2025 17:28

neubig requested a review from xingyaoww September 22, 2025 19:19

jpelletier1 mentioned this pull request Sep 23, 2025

[Agent] Customizable Guardrails OpenHands/OpenHands#11016

Open

openhands-agent added 2 commits September 23, 2025 14:38

Merge main branch and resolve conflicts

b29bbdd

Co-authored-by: openhands <openhands@all-hands.dev>

openhands-agent and others added 2 commits October 16, 2025 14:28

Merge branch 'main' into configurable-security-policy

5ad3b8f

neubig marked this pull request as draft October 17, 2025 15:34

neubig mentioned this pull request Dec 8, 2025

docs: Add configurable security policy documentation OpenHands/docs#158

Closed

xingyaoww reviewed Dec 8, 2025

View reviewed changes

examples/01_standalone_sdk/32_configurable_security_policy.py Show resolved Hide resolved

Rename example 28 to 31 to match latest numbering

681b993

Address review comment from xingyaoww Co-authored-by: openhands <openhands@all-hands.dev>

neubig mentioned this pull request Dec 8, 2025

docs: Add configurable security policy documentation OpenHands/docs#161

Merged

Trigger CI re-run

397543c

openhands-agent added 2 commits December 18, 2025 20:26

Merge main into configurable-security-policy

5c28a19

Resolved conflict in base.py by keeping both: - security_policy_filename from configurable-security-policy branch - model_name, model_family, model_variant from main branch Co-authored-by: openhands <openhands@all-hands.dev>

Rename example to 32 to avoid conflict with 31_iterative_refinement.py

c8d9663

Co-authored-by: openhands <openhands@all-hands.dev>

neubig marked this pull request as ready for review December 18, 2025 20:35

neubig marked this pull request as draft December 18, 2025 20:36

neubig mentioned this pull request Dec 18, 2025

feat: Support custom security policy templates OpenHands/OpenHands-CLI#236

Open

Revert unrelated glob module changes

4eb442f

The glob shadowing fix is unrelated to the configurable security policy feature and should be in a separate PR if needed. Co-authored-by: openhands <openhands@all-hands.dev>

neubig requested a review from xingyaoww December 18, 2025 21:00

neubig marked this pull request as ready for review December 18, 2025 21:00

xingyaoww reviewed Dec 18, 2025

View reviewed changes

openhands-sdk/openhands/sdk/context/prompts/prompt.py Show resolved Hide resolved

neubig requested a review from xingyaoww December 19, 2025 02:21

xingyaoww approved these changes Dec 19, 2025

View reviewed changes

neubig merged commit a3c39a5 into main Dec 19, 2025
24 checks passed

neubig deleted the configurable-security-policy branch December 19, 2025 16:18

feat: Add configurable security policy support to Agent #427

feat: Add configurable security policy support to Agent #427

Uh oh!

Conversation

neubig commented Sep 22, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

neubig commented Sep 23, 2025

Uh oh!

openhands-ai bot commented Sep 23, 2025

Uh oh!

openhands-ai bot commented Sep 23, 2025

Summary

✅ Request Completion Checklist:

🔧 Changes Made:

🧪 Verification:

Uh oh!

github-actions bot commented Sep 23, 2025

Uh oh!

github-actions bot commented Oct 16, 2025

Uh oh!

neubig commented Oct 16, 2025

Uh oh!

openhands-ai bot commented Oct 16, 2025

Uh oh!

openhands-ai bot commented Oct 16, 2025

Summary

What Was Done

Conflict Resolution Details

Changes Pushed

Uh oh!

github-actions bot commented Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

neubig commented Oct 17, 2025

Uh oh!

blacksmith-sh bot commented Dec 2, 2025

Uh oh!

Uh oh!

neubig commented Dec 8, 2025

Uh oh!

neubig commented Dec 8, 2025

Uh oh!

neubig commented Dec 18, 2025

Uh oh!

openhands-ai bot commented Dec 18, 2025

Uh oh!

openhands-ai bot commented Dec 18, 2025

Summary

✅ Checklist

📁 Log File Location

📝 Changes Pushed

⚠️ Note on CLI Integration

Uh oh!

openhands-ai bot commented Dec 18, 2025

Uh oh!

Uh oh!

xingyaoww left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

neubig commented Sep 22, 2025 •

edited by github-actions bot

Loading

github-actions bot commented Oct 16, 2025 •

edited

Loading