chore: add-benchmark skill: NVSkills integration#1445
Open
ananthsub wants to merge 3 commits into
Open
Conversation
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
ananthsub
changed the title
ananthsub
force-pushed
the
ci/nvskills-add-benchmark
branch
from
b84a4b4 to
d8c6353
Compare
Contributor
Author
|
/nvskills-ci |
Contributor
Author
|
/nvskills-ci |
1 similar comment
Contributor
Author
|
/nvskills-ci |
ananthsub
force-pushed
the
ci/nvskills-add-benchmark
branch
from
d8c6353 to
bf2e7e3
Compare
Contributor
Author
|
/nvskills-ci |
## Summary
- Move `.claude/skills/add-benchmark/` to top-level
`skills/add-benchmark/` so this PR touches files under the central
`team-request.yml` trigger allowlist (`skills/`, `team-skills/`,
`rules/team-rules/`, `plugins/`).
- Replace `.claude/skills/add-benchmark/` with a symlink to
`../../skills/add-benchmark` so Claude Code and Cursor continue to
discover the skill via the conventional
`.claude/skills/<name>/SKILL.md` path with no tool-side change.
- Add `license: Apache-2.0` to `skills/add-benchmark/SKILL.md`
frontmatter (required by NVSkills CI).
- Add `skills/add-benchmark/evals/evals.json` with 3 positive trigger
cases (scaffold native benchmark, wrap external library, add
training env with GitLab dataset) and 2 negative cases that
delegate to sibling skills.
## Motivation
Prepares the `add-benchmark` skill for NVSkills CI signing. The
other four Gym skills remain at `.claude/skills/<name>/` until each
one is similarly migrated in its own follow-up PR — keeping per-PR
diffs small and letting NVSkills CI evaluate one skill at a time
rather than all five simultaneously.
A separate follow-up PR against `NVIDIA/skills` will update
`components.d/nemo-gym.yml` `path:` from `.claude/skills/` to
`skills/` once all five skills have migrated, to keep the daily
sync pipeline aligned with the new source layout.
## Test plan
- [ ] Comment `/nvskills-ci` on this PR. Expect the request workflow
to dispatch (not skip) and `svc-nvskills-signing` to attach
`skill-card.md` and `skill.oms.sig` under `skills/add-benchmark/`.
- [ ] Claude Code discovers `add-benchmark` via
`.claude/skills/add-benchmark/SKILL.md` (follows symlink).
- [ ] Cursor discovers `add-benchmark` via
`.claude/skills/add-benchmark/SKILL.md` (follows symlink).
Signed-off-by: Ananth Subramaniam <ansubramania@nvidia.com>
ananthsub
force-pushed
the
ci/nvskills-add-benchmark
branch
from
bf2e7e3 to
644a1b0
Compare
Contributor
Author
|
/nvskills-ci |
1 similar comment
Contributor
Author
|
/nvskills-ci |
Resolve the NV-BASE validation and context-deduplication failures from the first /nvskills-ci run on add-benchmark. The trigger now fires; the remaining failures were all in skill content. Frontmatter and schema: - Shorten the description and add negative triggers (not for debugging or docs) to reduce over-triggering. - Add metadata.author and metadata.tags. Sections: - Add Purpose, When not to use this skill, and Examples sections. - Rename Workflow to Instructions. Security notes (documentation only, no behavior change): - SKILL.md: note credential handling for env.yaml, and that NeMo-Gym does not currently sandbox executed code, so code-execution benchmarks should run in an isolated, disposable environment the operator controls. - references/patterns.md: equivalent notes on the external-tool auto-install pattern and the Ray subprocess pattern. Context deduplication: - Collapse the two ng_collect_rollouts blocks into one canonical block in Step 6, with Step 7 referencing it for the baseline run. - Replace the duplicated ng_prepare_data block in SKILL.md with a cross-link to the canonical version in references/patterns.md. Signed-off-by: Ananth Subramaniam <ansubramania@nvidia.com>
Contributor
Author
|
/nvskills-ci |
- Shorten the description to under 150 characters. - Add Prerequisites, Limitations, and Troubleshooting sections. Signed-off-by: Ananth Subramaniam <ansubramania@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
license: Apache-2.0to the frontmatter of.claude/skills/add-benchmark/SKILL.md..claude/skills/add-benchmark/evals/evals.jsonwith 3 positive trigger cases (scaffold native benchmark, wrap external library, add training env with GitLab dataset) and 2 negative cases that delegate to sibling skills.Motivation
Prepares the
add-benchmarkskill for NVSkills CI signing.Test plan
/nvskills-cion this PR. Expect a signature commit attachingskill-card.mdandskill.oms.sigunder.claude/skills/add-benchmark/.