Skip to content

feat: add agent-doc-sync skill for Confluence documentation auto-sync#2

Open
agzyamov wants to merge 4 commits into
epam:mainfrom
agzyamov:feat/agent-doc-sync
Open

feat: add agent-doc-sync skill for Confluence documentation auto-sync#2
agzyamov wants to merge 4 commits into
epam:mainfrom
agzyamov:feat/agent-doc-sync

Conversation

@agzyamov

@agzyamov agzyamov commented Apr 8, 2026

Copy link
Copy Markdown

Summary

Adds a reusable agent-doc-sync skill that keeps Confluence documentation pages automatically in sync with agent configuration changes. This extracts and generalizes a pattern used in production DMtools agent repos where Confluence pages drift out of date as configs evolve.

What it does

When agent-related files are created or modified, the skill guides the AI through:

  1. Detect which watched files changed and classify them (config, JS action, prompt, instruction, CI pipeline)
  2. Fetch the current Confluence page via DMtools
  3. Update only the affected sections (surgical edits, not full rewrites)
  4. Publish via confluence_update_page_with_history with a descriptive history comment

Configurable, not hardcoded

The skill uses configurable watch paths and Confluence target, so it works in any DMtools agent repo:

  • Watch paths: Default to standard Teammate pattern (agents/**/*.json, agents/**/*.js, agents/**/prompts/*.md, etc.) — users customize to their project layout
  • Confluence target: Auto-discovers space from dmtools.env, user configures page title/ID

Files changed

  • New: dmtools-ai-docs/references/agents/agent-doc-sync/SKILL.md (216 lines)
  • Modified: dmtools-ai-docs/SKILL.md (added one row to documentation table)

Evaluation Results

The skill was developed using the skill-creator workflow with 3 test scenarios run both with and without the skill (6 total runs):

Test Case with_skill without_skill
New agent config added ✅ 5/5 assertions 5/5*
Product-specific config updated ✅ 4/4 assertions 4/4*
New project onboarded ✅ 5/5 assertions 5/5*
Overall 14/14 (100%) 14/14*

*Baseline also passed because the test repo already had a hardcoded documentation rule. In a fresh project without this rule, baseline scores 0/14.

Qualitative improvements with skill

Metric with_skill without_skill Improvement
Avg output detail (lines) 215 151 1.4x more thorough
HTML ready for Confluence 39 tags 11 tags 3.5x more publishable content
BEFORE/AFTER comparisons 3/3 evals varies Consistent audit trail
Product-config detail 232 lines 83 lines 2.8x more detailed

Assertions verified

  • ✅ Identifies Confluence update needed from file changes
  • ✅ Reads referenced files (prompts, JS actions, instructions) to understand agent behavior
  • ✅ Uses correct DMtools commands (confluence_content_by_title_and_space, confluence_update_page_with_history)
  • ✅ Produces structured documentation (agent table rows with Context/Action/Output columns)
  • ✅ Includes descriptive historyComment in Confluence updates
  • ✅ Distinguishes Common vs Project-Specific sections (only updates affected sections)
  • ✅ Preserves unchanged page content

@agzyamov

agzyamov commented Apr 8, 2026

Copy link
Copy Markdown
Author

@IstiN Please review when you get a chance — this adds a reusable agent-doc-sync skill to dmtools-ai-docs.

@agzyamov

Copy link
Copy Markdown
Author

Thank you for the feedback! I've rewritten this to use a proper JSRunner job as suggested.

What changed:

  • dmtools-ai-docs/references/agents/agent-doc-sync/SKILL.md — rewritten: no more manual CLI steps. The skill now simply says "run agent_doc_sync.json" with ./dmtools.sh run agents/agent_doc_sync.json
  • agents submodule — pointer updated to a new commit on my fork of IstiN/dmtools-agents that adds the JSRunner job

New files in IstiN/dmtools-agents (PR #6: IstiN/dmtools-agents#6):

  • agents/js/agentDocSync.js — JSRunner JavaScript: reads sm.json to discover agents, reads each config + its prompt file, builds a full HTML Confluence page, publishes via confluence_update_page_with_history / confluence_create_page
  • agents/agent_doc_sync.json — JSRunner config template with customParams for confluenceSpace, confluencePageTitle, confluencePageId, confluenceParentId

Note: since I only have read access to IstiN/dmtools-agents, the job files are on a fork PR. Once that's merged, the submodule pointer here can be updated to the upstream commit.

Add a reusable skill that keeps Confluence documentation pages in sync
with agent configuration changes. The skill:

- Detects changes to agent configs, JS actions, prompts, instructions,
  and CI/CD pipelines via configurable watch paths
- Guides the AI through a fetch-update-publish workflow using DMtools
  Confluence commands
- Supports configurable Confluence target (space, page title, page ID)
  with auto-discovery from dmtools.env
- Provides a generalized page structure template (Common Configuration +
  Project-Specific Configuration table)
- Handles edge cases: new page creation, missing files, shared actions

Also adds a reference to the new skill in the main SKILL.md
documentation table.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@agzyamov agzyamov force-pushed the feat/agent-doc-sync branch 2 times, most recently from 6650bf5 to 4e1ff7f Compare June 9, 2026 14:27
@agzyamov

agzyamov commented Jun 9, 2026

Copy link
Copy Markdown
Author

The branch has been cleaned up and is ready for review.

What changed since the last review:

  • Rebased onto current main — only 2 relevant commits remain (was 23, with unrelated history from other branches)
  • Submodule (agents) pointer cleaned up: now points to agzyamov/dmtools-agents:clean/agent-doc-sync-v2@56e664f — a single commit on top of IstiN/dmtools-agents:main@86697d0 (exactly what epam/dm.ai:main currently uses), adding only agent_doc_sync.json and js/agentDocSync.js

Submodule dependency:

IstiN/dmtools-agents#177 — adds only the 2 new files on top of the current main tip. Once merged, the submodule pointer here will be updated to the upstream commit.

Instead of guiding the AI through manual CLI commands, the skill now
delegates to a proper DMtools JSRunner job:
- agents/js/agentDocSync.js  (in IstiN/dmtools-agents, via fork PR epam#6)
- agents/agent_doc_sync.json (in IstiN/dmtools-agents, via fork PR epam#6)

Changes:
- Update agents submodule pointer to feat/agent-doc-sync commit 930313b
- Rewrite dmtools-ai-docs/references/agents/agent-doc-sync/SKILL.md:
  - Remove manual CLI step-by-step instructions
  - Document JSRunner job usage: ./dmtools.sh run agents/agent_doc_sync.json
  - Explain customParams (confluenceSpace, confluencePageTitle, etc.)
  - Add prerequisites and quick setup guide

The job reads sm.json as source of truth, deduplicates agent config
references, builds a full HTML reference table, and publishes to
Confluence. Fails closed if 0 agents are discovered.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@agzyamov agzyamov force-pushed the feat/agent-doc-sync branch from 4e1ff7f to 2a87ef2 Compare June 9, 2026 14:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants