Skip to content

Conversation

@neubig
Copy link
Contributor

@neubig neubig commented Dec 22, 2025

Summary

This PR adds validation and prompt generation utilities for skills, following the AgentSkills standard.

Depends on PR #1482 (resource directories support)

Changes

1. Enhanced SkillValidationError

Now includes a list of specific errors:

from openhands.sdk.context.skills import SkillValidationError

try:
    # ... validation code
except SkillValidationError as e:
    print(f"Validation failed: {e}")
    for error in e.errors:
        print(f"  - {error}")

2. New validate_skill() Function

Validates a skill directory according to AgentSkills spec:

from openhands.sdk.context.skills import validate_skill

errors = validate_skill(Path("skills/my-skill"))
if errors:
    print("Validation errors:")
    for error in errors:
        print(f"  - {error}")
else:
    print("Skill is valid!")

Validates:

  • Directory exists
  • SKILL.md file present
  • Skill name format (lowercase alphanumeric with hyphens)
  • Description length (max 1024 chars)
  • Frontmatter structure (mcp_tools, triggers, inputs)
  • .mcp.json validity if present

3. New to_prompt() Function

Generates XML prompt block for available skills:

from openhands.sdk.context.skills import Skill, to_prompt

skills = [
    Skill(name="pdf-tools", content="...", description="Extract text from PDFs."),
    Skill(name="code-review", content="...", description="Review code for bugs."),
]

prompt = to_prompt(skills)
print(prompt)

Output:

<available_skills>
  <skill name="pdf-tools">Extract text from PDFs.</skill>
  <skill name="code-review">Review code for bugs.</skill>
</available_skills>

Features:

  • Uses description field if available
  • Falls back to first content line (skipping headers)
  • Escapes XML special characters
  • Truncates long descriptions to 200 chars

Backward Compatibility

  • All changes are backward compatible
  • SkillValidationError still works with single message
  • New functions are additive

Testing

Added 24 new tests covering:

  • SkillValidationError with errors list
  • validate_skill() for various validation scenarios
  • to_prompt() for XML generation

Related Issues

Closes #1478

Part of #1473 (Support AgentSkills standard)

Diff from Previous PR

View changes from PR #1482

@neubig can click here to continue refining the PR


Agent Server images for this PR

GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server

Variants & Base Images

Variant Architectures Base Image Docs / Tags
java amd64, arm64 eclipse-temurin:17-jdk Link
python amd64, arm64 nikolaik/python-nodejs:python3.12-nodejs22 Link
golang amd64, arm64 golang:1.21-bookworm Link

Pull (multi-arch manifest)

# Each variant is a multi-arch manifest supporting both amd64 and arm64
docker pull ghcr.io/openhands/agent-server:19fe7ff-python

Run

docker run -it --rm \
  -p 8000:8000 \
  --name agent-server-19fe7ff-python \
  ghcr.io/openhands/agent-server:19fe7ff-python

All tags pushed for this build

ghcr.io/openhands/agent-server:19fe7ff-golang-amd64
ghcr.io/openhands/agent-server:19fe7ff-golang_tag_1.21-bookworm-amd64
ghcr.io/openhands/agent-server:19fe7ff-golang-arm64
ghcr.io/openhands/agent-server:19fe7ff-golang_tag_1.21-bookworm-arm64
ghcr.io/openhands/agent-server:19fe7ff-java-amd64
ghcr.io/openhands/agent-server:19fe7ff-eclipse-temurin_tag_17-jdk-amd64
ghcr.io/openhands/agent-server:19fe7ff-java-arm64
ghcr.io/openhands/agent-server:19fe7ff-eclipse-temurin_tag_17-jdk-arm64
ghcr.io/openhands/agent-server:19fe7ff-python-amd64
ghcr.io/openhands/agent-server:19fe7ff-nikolaik_s_python-nodejs_tag_python3.12-nodejs22-amd64
ghcr.io/openhands/agent-server:19fe7ff-python-arm64
ghcr.io/openhands/agent-server:19fe7ff-nikolaik_s_python-nodejs_tag_python3.12-nodejs22-arm64
ghcr.io/openhands/agent-server:19fe7ff-golang
ghcr.io/openhands/agent-server:19fe7ff-java
ghcr.io/openhands/agent-server:19fe7ff-python

About Multi-Architecture Support

  • Each variant tag (e.g., 19fe7ff-python) is a multi-arch manifest supporting both amd64 and arm64
  • Docker automatically pulls the correct architecture for your platform
  • Individual architecture tags (e.g., 19fe7ff-python-amd64) are also available if needed

Add support for AgentSkills standard fields (https://agentskills.io/specification):
- description: Brief description of what the skill does
- license: License under which the skill is distributed
- compatibility: Environment requirements or compatibility notes
- metadata: Arbitrary key-value metadata for extensibility
- allowed_tools: List of pre-approved tools for the skill

Also adds skills-ref as an optional dependency for future validation
and prompt generation utilities.

Closes #1474

Co-authored-by: openhands <openhands@all-hands.dev>
The skills-ref library will be added when validation and prompt
generation utilities are implemented (issue #1478).

Co-authored-by: openhands <openhands@all-hands.dev>
- Add find_skill_md() function to locate SKILL.md files (case-insensitive)
- Add validate_skill_name() function for AgentSkills spec validation
- Update load_skills_from_dir() to support skill-name/SKILL.md directories
- Add directory_name and validate_name parameters to Skill.load()
- Export new functions from __init__.py
- Add 27 unit tests for new functionality

Closes #1475

Co-authored-by: openhands <openhands@all-hands.dev>
- Add find_mcp_config() function to locate .mcp.json files
- Add expand_mcp_variables() for variable expansion (, default)
- Add load_mcp_config() to parse and validate .mcp.json files
- Add mcp_config_path field to Skill model
- Update Skill.load() to auto-load .mcp.json from SKILL.md directories
- .mcp.json takes precedence over mcp_tools frontmatter
- Support  variable expansion
- Export new functions from __init__.py
- Add 19 unit tests for new functionality

Closes #1476

Co-authored-by: openhands <openhands@all-hands.dev>
- Add SkillResources model to track resource directories
- Add discover_skill_resources() function to scan for resources
- Add RESOURCE_DIRECTORIES constant for standard directory names
- Add resources field to Skill model
- Update Skill.load() to auto-discover resources from SKILL.md directories
- Export new classes and functions from __init__.py
- Add 21 unit tests for new functionality

Closes #1477

Co-authored-by: openhands <openhands@all-hands.dev>
- Enhance SkillValidationError with errors list
- Add validate_skill() function for skill directory validation
- Add to_prompt() function for XML prompt generation
- Export new functions from __init__.py
- Add 24 unit tests for new functionality

Closes #1478

Co-authored-by: openhands <openhands@all-hands.dev>
@github-actions
Copy link
Contributor

github-actions bot commented Dec 22, 2025

Coverage

Coverage Report •
FileStmtsMissCoverMissing
openhands-sdk/openhands/sdk/context/skills
   exceptions.py10550%15–16, 19–21
   skill.py49240717%64, 68–69, 73–74, 78–79, 91–96, 113, 115–119, 121, 137–140, 142–146, 159, 161–163, 165–166, 168–169, 174–175, 177, 189–194, 214–215, 218, 221, 223–225, 228–233, 235, 237–238, 257, 259–265, 267–268, 273–275, 278, 281–284, 286, 303–304, 307–309, 312–315, 318–320, 323–326, 329–330, 333–335, 340–342, 345–347, 350–352, 354–355, 358–363, 365, 387–388, 390–391, 393–394, 396–397, 399–402, 404–408, 413, 528–534, 540–544, 557, 560, 564, 576–577, 582, 589, 617, 620, 623–625, 627, 631, 634–636, 639–641, 643–645, 648, 651, 654–657, 663, 666, 674, 679–681, 684–689, 691–692, 698, 700–701, 704–708, 711–713, 719, 721–727, 730, 742–743, 755, 770–777, 787–788, 790, 796–797, 799, 808–811, 813–816, 824–826, 834–836, 861–862, 864–865, 868, 871–872, 875–879, 882, 885–892, 895–897, 899–900, 902–903, 905–907, 909, 911–912, 914, 917–926, 929–931, 937–943, 946–955, 957, 961, 982–983, 985–988, 990–992, 995–999, 1001, 1005–1006, 1008, 1011, 1029–1030, 1032–1033, 1036, 1041–1043, 1046, 1048–1050, 1053–1057, 1059, 1064–1065, 1069, 1072, 1086–1088, 1106, 1108–1112, 1119, 1126–1130, 1135–1137, 1139, 1154, 1156, 1158–1162, 1165–1168, 1202, 1204, 1206–1207, 1209–1211, 1214–1217, 1220, 1222, 1225–1227, 1231–1235, 1237–1238, 1240, 1243
TOTAL13662631053% 

Reduce test code by 74% while maintaining essential coverage:
- test_agentskills_fields.py: 299 → 73 lines
- test_skill_md_convention.py: 405 → 86 lines
- test_mcp_json_config.py: 301 → 89 lines
- test_resource_directories.py: 300 → 83 lines
- test_validation_prompt.py: 332 → 101 lines

Total: 1637 → 432 lines

Co-authored-by: openhands <openhands@all-hands.dev>
- Merged field validators from main (_parse_allowed_tools, _convert_metadata_values)
- Kept new fields (mcp_config_path, resources) from the PR branch
- Updated tests to use Pydantic ValidationError for description validation
- Removed redundant _parse_agentskills_fields method (functionality moved to validators)

Co-authored-by: openhands <openhands@all-hands.dev>
@openhands-ai
Copy link

openhands-ai bot commented Dec 22, 2025

Looks like there are a few issues preventing this PR from being merged!

  • GitHub Actions are failing:
    • Agent Server

If you'd like me to help, just leave a comment, like

@OpenHands please fix the failing actions on PR #1483 at branch `feat/validation-prompt-utils`

Feel free to include any additional details that might help me get this PR into a better state.

You can manage your notification settings

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add validation and prompt generation utilities

3 participants