docs: add manual testing guide for slash command validation#1955
docs: add manual testing guide for slash command validation#1955mnriem merged 3 commits intogithub:mainfrom
Conversation
Adds a top-level TESTING.md that describes the manual test process PR submitters must follow when their changes affect slash commands. Includes: - Process overview (identify affected commands, setup, run, report) - Local setup instructions using editable install - Reporting template for PR submissions - Agent prompt that analyzes changed files and determines which commands need testing, including transitive script dependencies and extension hook mappings
There was a problem hiding this comment.
Pull request overview
Adds a new top-level manual testing guide to ensure PRs that impact /speckit.* slash commands include end-to-end, agent-mediated validation steps and a standard reporting format.
Changes:
- Add
TESTING.mddescribing a manual test workflow for slash command changes. - Provide a copy/paste PR reporting template for manual test results.
- Include an “agent prompt” to map changed files to affected commands and prerequisite test ordering.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
@copilot open a new pull request to apply changes based on the comments in this thread |
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 1 out of 1 changed files in this pull request and generated 3 comments.
Comments suppressed due to low confidence (1)
TESTING.md:22
/tmp/speckit-testis POSIX-specific and won’t work on Windows (and may be undesirable in some environments). Consider using a placeholder like<test-project-dir>(or showing both a Unix and Windows example), or recommendingspecify init --herein an empty directory to keep the instructions cross-platform.
# Initialize a test project using your local changes
specify init /tmp/speckit-test --ai <agent> --offline
cd /tmp/speckit-test
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
TESTING.md:20
- The Windows activation hint looks incorrect/incomplete: standard venv activation scripts are
.venv\Scripts\Activate.ps1(PowerShell) or.venv\Scripts\activate.bat(cmd), whereas.venv\Scripts\activateusually won’t exist. This repo’sdocs/local-development.mduses theActivate.ps1form—please align this guide so Windows users can follow it verbatim.
source .venv/bin/activate # On Windows: .venv\Scripts\activate
uv pip install -e .
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…nc-main-2026-03-25 * upstream/main: (90 commits) fix(ps1): replace null-conditional operator for PowerShell 5.1 compatibility (github#1975) chore: bump version to 0.4.2 (github#1973) feat: Auto-register ai-skills for extensions whenever applicable (github#1840) docs: add manual testing guide for slash command validation (github#1955) Add AIDE, Extensify, and Presetify to community extensions (github#1961) docs: add community presets section to main README (github#1960) docs: move community extensions table to main README for discoverability (github#1959) docs(readme): consolidate Community Friends sections and fix ToC anchors (github#1958) fix(commands): rename NFR references to success criteria in analyze and clarify (github#1935) Add Community Friends section to README (github#1956) docs: add Community Friends section with Spec Kit Assistant VS Code extension (github#1944) chore: bump version to 0.4.1 (github#1953) Add checkpoint extension (github#1947) fix(scripts): prioritize .specify over git for repo root detection (github#1933) docs: add AIDE extension demo to community projects (github#1943) fix(templates): add missing Assumptions section to spec template (github#1939) chore: bump version to 0.4.0 (github#1937) fix(cli): add allow_unicode=True and encoding="utf-8" to YAML I/O (github#1936) fix(codex): native skills fallback refresh + legacy prompt suppression (github#1930) feat(cli): embed core pack in wheel for offline/air-gapped deployment (github#1803) ...
Adds a top-level
TESTING.mdthat describes the manual test process PR submitters must follow when their changes affect slash commands.What's included
uv pip install -e .+specify init --offlinegit diff --name-only mainand maps changed files to affected commands using:common.sh→create-new-feature.sh→/speckit.specify)extension.ymlbefore_specifyhook →/speckit.specify)The agent's reasoning output (which files affect which commands, and why) is submitted alongside the test results in the PR.
Why
There was no documented process for validating that slash commands actually work after code changes. Automated tests cover file generation, placeholder replacement, and script execution — but nobody was verifying the full agent-mediated command invocation path. This fills that gap with a lightweight manual process.