Feat/skill management#24
Open
aarontuor wants to merge 16 commits into
Open
Conversation
Bridge dsagt's skill system into agent-native discovery and add a searchable catalog of installable skills from external GitHub repos. Two tiers: - Catalog — external Agent-Skills repos (default: K-Dense scientific, 140+) cloned + indexed into per-source `skills_catalog__<slug>` KB collections. Searchable via search_skills, never loaded into the agent's context. - Installed — a chosen skill copied into <project>/skills/ and mirrored into .claude/skills/ for Claude Code's native discovery. Changes: - commands/skills_catalog.py: shallow-clone cache, recursive SKILL.md discovery, per-source indexing (idempotent re-sync via drop+rebuild), find/install. Known sources: scientific, anthropic, antigravity, composio. - MCP tools: install_skill + catalog-spanning search_skills (registry); add_skill_source / list_skill_sources (knowledge). Added to auto-allow. - agents/base._mirror_skills_to + ClaudeSetup hook: manifest-gated mirror into .claude/skills/ (never clobbers user skills; reaps stale entries; trims >1536-char descriptions in the copy only). - Bundled skill-creator meta-skill (Anthropic template + condensed spec). - CLI: dsagt skills sync/add/list/search. - Config: skills block (sources/populate_native/populate_catalog), backfilled for old configs; setup-kb syncs the default catalog (--no-skill-catalog to skip); reserve .skill_sources; kb_from_config. - dsagt_instructions.md: two-tier guidance (native vs catalog/install). - use_cases/isaac_skills_demo: runnable mock of the isaac_vasp workflow exercising the full flow with tiny mock VASP data. Tests: test_skills_catalog.py + config/server-routing additions (201 passed, 13 skipped); black + ruff clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
First-pass vetting of use_cases/isaac_skills_demo against the real K-Dense catalog surfaced one bug: `dsagt skills add <proj> <source>` synced + indexed the catalog but never wrote the source into dsagt_config.yaml, so a later config-driven `dsagt skills sync` would forget it. Only the `add_skill_source` MCP tool persisted. - Move the persist logic into a shared `persist_source_to_config` helper in skills_catalog.py; call it from both the CLI add-source path and the knowledge-server `add_skill_source` handler (removes the duplicated `_persist_skill_source`). - Regression test for the helper (append + dedupe + missing-config no-op). - Add use_cases/isaac_skills_demo/PROMPTS.md: the 8-prompt hand-pass script plus first-pass results (init/mirror, 146-skill sync, search, install pymatgen, native re-mirror, add anthropic all verified). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The isaac_skills_demo rebuild path (rm + init) left the external catalog empty because a fresh `init` copies only the shared KB, while the catalog is project-scoped. The agent's search_skills then correctly returned no catalog hits. Add the required `dsagt skills sync` step to the rebuild block, a pre-launch catalog check, and a note on the global setup-kb alternative. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An empty/unsynced external skill catalog was indistinguishable from a genuine no-match, forcing the agent through a multi-step discovery dance and a misleading "skill unusable until restart" message. - search_skills: when no catalog is synced, say so and point at list_skill_sources / add_skill_source instead of a bare no-match - list_skill_sources: flag each known source synced/available with its indexed count, rather than two parallel lists to cross-reference - install_skill: state the skill is usable this session immediately; restart only enables hands-free native auto-invocation - dsagt_instructions: document the catalog as opt-in and the list_skill_sources -> add_skill_source -> search_skills -> install flow Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bump to 0.2.0 and make dsagt.__version__ the single source of truth — pyproject reads it via setuptools dynamic metadata, so future bumps touch one line. Add CHANGELOG (Keep a Changelog format). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- README/docs: document the non-developer install (pip install git+https://github.com/AI-ModCon/dsagt.git into any 3.12/3.13 env); note uv is dev/CI-only and conda/venv both work - de-duplicate the supported-agents table and install block via mkdocs-include-markdown so docs/index.md pulls them from the README - correct the Python prerequisite to 3.12/3.13 - cli.md: drop the uv-sync-specific install assumption - docs CI builds with the locked docs dependency group via uv Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…Catalog + keyword fallback
Collapse the two MCP servers (dsagt-registry-server + dsagt-knowledge-server)
into a single dsagt-server backed by one shared KnowledgeBase, and finish the
skills-discovery refactor from design-notes/genesis-skills-comparison.md §10 and
skills-catalog-server-merge.md.
Server merge
- New commands/dsagt_server.py: create_dsagt_server composes both modules'
(tools, handlers) under one Server("dsagt") with a type-dispatched call_tool
(registry str passthrough + knowledge dict->json + error wrap); one shared-KB
main() with the cross-backend guard living once in _build_kb_from_config.
- registry_server / knowledge_server keep create_*_server as thin test-facing
wrappers via extracted _registry_tools_and_handlers / _knowledge_tools_and_handlers;
their standalone main()s + entry points are removed.
- pyproject: dsagt-registry-server + dsagt-knowledge-server -> dsagt-server.
- Per-agent MCP config collapsed to one "dsagt" entry across claude/goose/codex/
cline/roo/opencode (+ base helpers, info.py span buckets).
- Compat is rebuild-not-migrate: re-run `dsagt start` to regenerate config
(README upgrade note + cline .cline-data caveat). No migration shims.
Skills discovery
- New SkillsCatalog (commands/skills_catalog.py) owns the catalog data plane
(sync/install/search/list_sources + ChromaDB-vs-keyword backend selection);
SkillRouter is now a thin render/MCP facade over it.
- New skill_keyword.py: Genesis-faithful token-overlap scorer (no-embedder
fallback). skill_discovery.py: stateless SkillRouter.
- Catalog-only search + frontmatter-only indexing; removed the dead
skills-collection indexing (SkillRegistry + setup_core_kb bundled skills).
- Struck the stale bundled datacard-generator skill (now via `dsagt skills add
<project> genesis`); skill-creator is the only bundled skill.
Docs/diagram
- Architecture figure: one MCP box spanning Knowledge + Registry, "Server"
dropped from both labels, Skills -> Skills Catalog; regenerated PNGs.
- README/docs/design-notes updated.
Tests: new test_dsagt_server.py + test_skill_discovery.py; config-shape, info,
and smoke-test assertions updated to the single-server shape. 338 passed /
13 skipped across affected suites; ruff + black clean on changed files.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…overy Bump to 0.3.0 (breaking: the registry/knowledge MCP servers merge into one dsagt-server; the old console scripts are removed). CHANGELOG documents the motivation and a rebuild-not-migrate upgrade note; README version refs bumped. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Add the skill-routing diagram (assets/skills-routing.png) to tools-skills.md with a "Skill discovery architecture" section motivating the two-tier (catalog vs native) split, catalog-only search, the keyword fallback, the single SkillRouter entry point, and per-source federation/provenance. - Sweep README + docs for the old two-server model: mcp-servers, architecture, cli, developer, quickstart, knowledge-base now describe one dsagt-server. - Fix stale skill-indexing language: bundled/installed skills are auto- discovered natively (not indexed); setup-kb rebuilds Tool Specs only; the KB collection tables list "Skills Catalog" (external, per-source) instead of a bundled "Skills" collection. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The README/docs documented `dsagt --version` but the CLI had no such flag (argparse errored). Wire it from dsagt.__version__ so it reports the release the docs claim. Found while vetting the isaac_skills_demo walkthrough. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…source name clashes When the same skill name exists in more than one synced source (the clone cache is machine-global, so this happens even across projects), the install guard now resolves a source-qualified '<slug>/<skill>' name instead of dead- ending. find_catalog_skill parses the '<slug>/' prefix and scopes the search; install_skill (MCP) and 'dsagt skills add' inherit it. The CLI 'add' routes a '<synced-slug>/<skill>' target to install (not a clone) and now prints a clean error instead of a traceback on an ambiguous bare name. The ambiguity message points at the qualified form, which actually works now. Found while vetting the isaac_skills_demo walkthrough. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…r 0.3.0 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…package; consolidate skill modules into skills.py The single dsagt-server now lives in src/dsagt/mcp/, composed from per-concern tool modules behind one dispatch shell: - mcp/server.py — composition, main(), shared build_dispatch_server + KB startup - mcp/registry_tools.py / knowledge_tools.py / memory_tools.py / skill_tools.py This replaces the registry_server / knowledge_server / dsagt_server trio and moves importable logic out of commands/ (which is for entry points). Entry point repointed to dsagt.mcp.server:main; agents are unchanged (same `dsagt-server`). Skill discovery is consolidated into one module src/dsagt/skills.py — the catalog data plane (SkillsCatalog), the SkillRouter render facade, and the Genesis-derived keyword scorer (was skill_keyword + skill_discovery + commands/skills_catalog). install_skill now returns a terse confirmation (the install→use→restart model already lives in the agent instructions). Remove dead src/session.py. Migrate tests to the new layout (new test_memory_tools / test_skill_tools; server-startup reworked to fast, network-free entry-point checks; memory + kb_search tests rehomed). Update CLAUDE.md and docs/mcp-servers.md to the merged-server / four-concern shape. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ntmatter Third-party catalogs (e.g. Genesis) ship SKILL.md files whose unquoted `description` contains a colon (`…readiness levels: Level 1…`), which strict YAML rejects — so _parse_frontmatter raised and the skill was silently dropped from indexing, search, and install. Genesis's `generating-datacards` (datacard-generator) was one such casualty. _parse_frontmatter now falls back to a lenient flat key:value parse on YAML error, recovering name/description/tags. dsagt-authored specs are valid YAML, so the fallback never fires for them. Adds TestLenientFrontmatter. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Consolidate the branch into one 0.2.0 release: a single CHANGELOG entry (external skill catalogs incl. the Genesis integration, native discovery, catalog-only search + keyword fallback, license/PROVENANCE capture, server merge) replacing the interim 0.2.0/0.3.0 entries; version set to 0.2.0 (one minor bump from master's 0.1.0). README/architecture diagram refreshed. Add two skill-management use cases: - isaac_skills_demo — agent-led arc (what we have → find more → sync → install → create) that authors a vasp-to-isaac converter parsing with real pymatgen. - genesis_skills — pull Genesis curation skills, ingest domain into the KB, and generate a datacard for a finished dataset. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Skills discovery feature and better skills management relying on agent native skills usage. Merged mcp servers to deal with communication between two servers issue. Updated use cases and docs.