Skip to content

test: smoke tests for validate_paths and print_report#21

Open
bipinhcs11 wants to merge 1 commit into
mainfrom
test/validate-paths-and-print-report
Open

test: smoke tests for validate_paths and print_report#21
bipinhcs11 wants to merge 1 commit into
mainfrom
test/validate-paths-and-print-report

Conversation

@bipinhcs11
Copy link
Copy Markdown
Owner

What and why

validate_paths and print_report are the two functions wired to the skill-gen validate subcommand — the entry point developers use to audit SKILL.md files before committing. Both were exported from validate.py but had zero test coverage. The existing test_validate.py covered validate() and validate_file() only; the CLI-facing layer was untested.

What's covered (14 new tests)

TestValidatePaths (9 tests)

Test What it pins
test_explicit_valid_file_returns_single_ok_result Single-file happy path
test_explicit_invalid_file_returns_single_error_result Single-file error path
test_nonexistent_file_returns_error_result Missing file → error result, not a crash
test_empty_path_list_returns_empty_list Empty input → empty list
test_directory_finds_all_skill_mds_recursively Directory glob finds 3 deeply-nested SKILL.mds
test_directory_with_no_skill_mds_returns_empty Directory with no SKILL.md → empty list
test_mix_of_file_and_directory_combined Explicit file + directory counted together
test_results_have_path_attribute Each result carries a path field
test_three_reference_skill_dirs_all_ok Regression: the three artifact-3 reference skills in this repo must continue to pass validate_paths cleanly; a future edit that breaks them will fail this test

TestPrintReport (5 tests)

Test What it pins
test_all_ok_returns_zero Exit code 0 when no errors
test_any_error_returns_one Exit code 1 when any error present
test_output_contains_summary_line Summary line with "validated" / "ok" always printed
test_empty_results_returns_zero Empty input → exit 0, no crash
test_error_count_in_summary "error" appears in output when errors exist

Note on skills/skill-generator/SKILL.md

The skills/ tree contains a fourth SKILL.md — the Claude Agent skill spec at skills/skill-generator/SKILL.md. This file uses a different frontmatter schema (name:/description: rather than the artifact-3 fields) so it legitimately fails validate. The regression test therefore targets the three artifact-3 reference skill directories explicitly rather than the whole skills/ tree.

Suite size

172 → 186 tests, all green.

python3 -m unittest tests/test_validate.py -v  # 51 passed, 0 failed
python3 -m unittest discover tests/           # 186 passed, 0 failed

Follow-ups (not in this PR)


Generated by Claude Code

validate_paths (directory recursion, explicit file, missing file,
empty list, mixed path types) and print_report (exit-code 0/1,
summary line, empty input) are the two functions wired to the
`skill-gen validate` subcommand. Both were exported from validate.py
but had zero test coverage — the existing test_validate.py only
covered the validate() and validate_file() functions.

Adds 14 new tests in two classes:
  TestValidatePaths  — 9 tests for path/directory dispatch logic
  TestPrintReport    — 5 tests for the CLI formatter return value

Key regression pinned: validate_paths on the three reference skill
directories must return 3 clean results; if a future edit breaks any
reference skill the test fails. (skills/skill-generator/SKILL.md is a
Claude Agent skill spec, not artifact-3, so targeted the three reference
dirs explicitly rather than the whole skills/ tree.)

Full suite: 172 → 186 tests, all green.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants