Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .claude/skills/nemo-gym-docs
Original file line number Diff line number Diff line change
@@ -1,19 +1,31 @@
---
name: nemo-gym-docs
license: Apache-2.0
description: >
Maintain the NeMo Gym Fern docs site — add, update, move, or remove pages
under fern/. Use for any documentation change. Triggered by: "edit docs",
"add doc page", "update docs", "rename page", "fix broken link", "add
redirect", "preview docs", "publish docs", any request that touches `fern/`.
Maintain the NeMo Gym Fern docs site: add, update, move, or remove pages under
fern/. Use for any documentation change. Not for code or configuration changes
outside fern/. Triggered by: edit docs, add doc page, update docs, rename page,
fix broken link, add redirect, preview docs, publish docs.
metadata:
author: NVIDIA <nemo-gym@nvidia.com>
tags:
- documentation
- fern
- mdx
- docs-site
---

# NeMo Gym Docs Maintenance

Unified skill for adding, updating, moving, and removing pages on the NeMo Gym Fern documentation site.
## Purpose

Add, update, move, and remove pages on the NeMo Gym Fern documentation site, and
run the validation, preview, and publish workflows. All documentation content
lives under `fern/`.

## Scope Rule

**ALL docs edits happen under `fern/`.** All new pagesincluding release notes and migration guides belong under `fern/`.
All documentation edits happen under `fern/`. All new pages, including release notes and migration guides, belong under `fern/`.

**Bleeding-edge tree + GA snapshots.** Gym keeps one bleeding-edge content tree at `fern/versions/latest/` (the folder name is historical — it's mounted under the `main` slug via `main.yml`) and one frozen GA snapshot per shipped release (currently `fern/versions/v0.2.1/`). All new edits land in `fern/versions/latest/`. The `latest.yml` file is a GA *alias* — a symlink to the current GA's yml — so `/latest/...` and `/<current-ga>/...` serve the same pages. Back-ports into a frozen GA snapshot are deliberate and rare; default to editing `fern/versions/latest/` only.

Expand Down Expand Up @@ -44,7 +56,7 @@ fern/versions/v0.2.1/pages/get-started/quickstart.mdx docs.nvidia.com/nemo/gym/
docs.nvidia.com/nemo/gym/latest/get-started/quickstart (GA alias)
```

## Operations
## Instructions

### Add a Page

Expand All @@ -64,7 +76,7 @@ fern/versions/v0.2.1/pages/get-started/quickstart.mdx docs.nvidia.com/nemo/gym/
```

3. If the parent folder is mounted in `main.yml` with `title-source: frontmatter`, the page is auto-discovered — no nav edit needed. Otherwise add a `- page:` entry under the right `section:` in `fern/versions/main.yml`.
4. Do **not** mirror into the current GA snapshot folder (e.g., `v0.2.1/`) frozen GA snapshots only get back-ports on explicit request.
4. Do not mirror into the current GA snapshot folder (e.g., `v0.2.1/`); frozen GA snapshots only get back-ports on explicit request.

### Update a Page

Expand All @@ -86,6 +98,8 @@ fern/versions/v0.2.1/pages/get-started/quickstart.mdx docs.nvidia.com/nemo/gym/

Only back-port when the user explicitly asks ("back-port to v0.2.1"). Apply the same change inside the GA snapshot's `pages/` folder (e.g., `fern/versions/v0.2.1/pages/`) and update its yml if needed. `latest.yml` is a symlink to the current GA's yml, so nav changes propagate automatically.

## Examples

### Worked Example: Adding a Page

Request: *"Add a how-to for collecting rollouts under Get Started."*
Expand Down Expand Up @@ -212,7 +226,7 @@ Tag push (docs/v*) → publish-fern-docs.yml → docs.nvi
Manual dispatch → publish-fern-docs.yml → docs.nvidia.com/nemo/gym
```

The preview-comment + publish jobs require the `DOCS_FERN_TOKEN` repository or organization secret (from `fern token`).
The preview-comment and publish jobs require the `DOCS_FERN_TOKEN` repository or organization secret (from `fern token`). It is managed as a CI secret and should not be committed to the repository or echoed in logs.

## Publishing to Production

Expand Down
60 changes: 60 additions & 0 deletions skills/nemo-gym-docs/evals/evals.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
[
{
"id": "nemo-gym-docs-positive-001",
"question": "Add a how-to page for collecting rollouts under the Get Started section of the NeMo Gym docs.",
"expected_skill": "nemo-gym-docs",
"ground_truth": "The agent loads the nemo-gym-docs skill, creates fern/versions/latest/pages/get-started/rollout-collection.mdx with the required frontmatter (title, description, position), and confirms the folder uses title-source: frontmatter so no main.yml edit is needed.",
"expected_behavior": [
"The agent read nemo-gym-docs/SKILL.md before acting",
"The agent created the new page under fern/versions/latest/pages/get-started/ (the bleeding-edge tree), not under v0.2.1/",
"The agent added the required frontmatter (title, description) to the MDX",
"The agent did not mirror the change into a frozen GA snapshot folder"
]
},
{
"id": "nemo-gym-docs-positive-002",
"question": "Rename /main/get-started/setup to /main/get-started/detailed-setup and make sure the old URL still works.",
"expected_skill": "nemo-gym-docs",
"ground_truth": "The agent loads the nemo-gym-docs skill, runs git mv on the MDX file under fern/versions/latest/pages/get-started/, adds a redirect entry in fern/docs.yml, and updates incoming links inside fern/versions/latest/.",
"expected_behavior": [
"The agent read nemo-gym-docs/SKILL.md before acting",
"The agent used git mv on the MDX file under fern/versions/latest/pages/",
"The agent added a redirect entry to fern/docs.yml for the old URL",
"The agent searched for and updated incoming links"
]
},
{
"id": "nemo-gym-docs-positive-003",
"question": "Publish the Gym docs to docs.nvidia.com as a new versioned release. What tag do I push?",
"expected_skill": "nemo-gym-docs",
"ground_truth": "The agent loads the nemo-gym-docs skill and explains that a tag matching docs/v<MAJOR>.<MINOR>.<PATCH> pushed to origin triggers publish-fern-docs.yml, and reminds the user not to push the tag without explicit user approval.",
"expected_behavior": [
"The agent read nemo-gym-docs/SKILL.md before acting",
"The agent gave the docs/v<MAJOR>.<MINOR>.<PATCH> tag format",
"The agent referenced publish-fern-docs.yml as the workflow that publishes",
"The agent did not push the tag without explicit user confirmation"
]
},
{
"id": "nemo-gym-docs-negative-001",
"question": "Implement the verify() function for a new code-generation benchmark resources server.",
"expected_skill": null,
"should_trigger": false,
"ground_truth": "The agent should not activate the nemo-gym-docs skill for benchmark implementation. It should use the add-benchmark skill instead.",
"expected_behavior": [
"The agent did not read or activate nemo-gym-docs/SKILL.md",
"The agent recognized this as a benchmark integration task"
]
},
{
"id": "nemo-gym-docs-negative-002",
"question": "My Slurm job for ng_collect_rollouts is failing with OOM mid-run. Help me debug it.",
"expected_skill": null,
"should_trigger": false,
"ground_truth": "The agent should not activate the nemo-gym-docs skill for a job-debugging task. It should use the nemo-gym-debugging skill instead.",
"expected_behavior": [
"The agent did not read or activate nemo-gym-docs/SKILL.md",
"The agent recognized this as a job-debugging task"
]
}
]
Loading