Skip to content

feat(claude-code-settings): improve test coverage#5384

Closed
miteshashar wants to merge 2 commits intoSchemaStore:masterfrom
miteshashar:claude-code-settings-coverage
Closed

feat(claude-code-settings): improve test coverage#5384
miteshashar wants to merge 2 commits intoSchemaStore:masterfrom
miteshashar:claude-code-settings-coverage

Conversation

@miteshashar
Copy link
Contributor

Schema

  • Add fastMode boolean property (default: false) per fast-mode documentation (v2.1.47)
  • Add default: true to sandbox.autoAllowBashIfSandboxed per sandboxing documentation
  • Enhance spinnerVerbs.mode description with explicit enum value explanations
  • Add 19 missing property descriptions: statusLine.type/command/padding, fileSuggestion.type, sandbox.network, 14 marketplace source type discriminators across extraKnownMarketplaces and strictKnownMarketplaces

Tests

  • Add 13 uncovered top-level properties to modern-complete-config.json (allowManagedPermissionRulesOnly, awsAuthRefresh, awsCredentialExport, companyAnnouncements, disableAllHooks, enabledPlugins, forceLoginOrgUUID, otelHeadersHelper, pluginConfigs, skipWebFetchPreflight, skippedMarketplaces, skippedPlugins, statusLine)
  • Add untested enum values across 6 test files — all 7 schema enums now fully covered
  • Add non-default value tests for 9 boolean/enum properties across edge-cases.json and basic-config.json
  • Add defaultMode enum coverage across permissions-basic.json, permissions-mcp.json, complete-config.json, edge-cases.json (all 6 values now tested)
  • Add git URL pattern positive test to marketplace-host-pattern.json

Negative tests

  • Add 4 invalid enum values to invalid-enum-values.json (autoUpdatesChannel, effortLevel, disableBypassPermissionsMode, teammateMode)
  • Add git URL pattern violation to invalid-marketplace-host-pattern.json
  • Create invalid-mcp-server-name.json for serverName pattern violations

Coverage analysis

Identified using the coverage analysis tool from #5383.

Before (3 passed, 5 failed)
===== COVERAGE: claude-code-settings.json =====
✔️ 1. Unused $defs
  totalDefs: 3
❌ 2. Description Coverage
  totalProperties: 141
  missingCount: 19
  missing (19): statusLine.type, statusLine.command, statusLine.padding, fileSuggestion.type, extraKnownMarketplaces.*.source.source, extraKnownMarketplaces.*.source.source, extraKnownMarketplaces.*.source.source, extraKnownMarketplaces.*.source.source, extraKnownMarketplaces.*.source.source, extraKnownMarketplaces.*.source.source, extraKnownMarketplaces.*.source.source, strictKnownMarketplaces[].source, strictKnownMarketplaces[].source, strictKnownMarketplaces[].source, strictKnownMarketplaces[].source, strictKnownMarketplaces[].source, strictKnownMarketplaces[].source, strictKnownMarketplaces[].source, sandbox.network
❌ 3. Test Completeness
  totalTopProperties: 49
  uncovered (13): allowManagedPermissionRulesOnly, awsAuthRefresh, awsCredentialExport, companyAnnouncements, disableAllHooks, enabledPlugins, forceLoginOrgUUID, otelHeadersHelper, pluginConfigs, skipWebFetchPreflight, skippedMarketplaces, skippedPlugins, statusLine
❌ 4. Enum Coverage
  totalEnums: 7
  issues (10):
  - autoUpdatesChannel — positive_uncovered — [latest]
  - autoUpdatesChannel — no_negative_enum_test
  - permissions.defaultMode — positive_uncovered — [bypassPermissions, default, delegate, dontAsk]
  - permissions.disableBypassPermissionsMode — no_negative_enum_test
  - effortLevel — positive_uncovered — [low, medium]
  - effortLevel — no_negative_enum_test
  - forceLoginMethod — positive_uncovered — [console]
  - spinnerVerbs.mode — positive_uncovered — [replace]
  - teammateMode — positive_uncovered — [in-process, tmux]
  - teammateMode — no_negative_enum_test
❌ 5. Pattern Coverage
  totalPatterns: 3
  issues (4):
  - allowedMcpServers[].serverName — no_negative_violation — /^[a-zA-Z0-9_-]+$/
  - deniedMcpServers[].serverName — no_negative_violation — /^[a-zA-Z0-9_-]+$/
  - extraKnownMarketplaces.*.source.url — no_positive_match — /\.git$/
  - extraKnownMarketplaces.*.source.url — no_negative_violation — /\.git$/
✔️ 6. Required Field Coverage
  totalRequiredGroups: 29
  note: Heuristic: name-based matching, not path-aware
❌ 7. Default Value Coverage
  totalDefaults: 15
  issues (9):
  - respectGitignore — only_default_tested — default=true
  - effortLevel — only_default_tested — default="high"
  - fastMode — only_default_tested — default=false
  - sandbox.autoAllowBashIfSandboxed — only_default_tested — default=true
  - spinnerTipsEnabled — only_default_tested — default=true
  - terminalProgressBarEnabled — only_default_tested — default=true
  - showTurnDuration — only_default_tested — default=true
  - prefersReducedMotion — only_default_tested — default=false
  - teammateMode — only_default_tested — default="auto"
✔️ 8. Negative Test Isolation
  totalNegativeTests: 9
  note: Heuristic: name-based violation detection, not path-aware
===== 3 passed, 5 failed, 0 warned, 0 skipped =====
After (7 passed, 0 failed, 1 warned)
===== COVERAGE: claude-code-settings.json =====
✔️ 1. Unused $defs
  totalDefs: 3
✔️ 2. Description Coverage
  totalProperties: 141
  missingCount: 0
✔️ 3. Test Completeness
  totalTopProperties: 49
✔️ 4. Enum Coverage
  totalEnums: 7
✔️ 5. Pattern Coverage
  totalPatterns: 3
✔️ 6. Required Field Coverage
  totalRequiredGroups: 29
  note: Heuristic: name-based matching, not path-aware
✔️ 7. Default Value Coverage
  totalDefaults: 15
⚠️ 8. Negative Test Isolation
  totalNegativeTests: 10
  note: Heuristic: name-based violation detection, not path-aware
  multiViolationFiles (1):
  - invalid-marketplace-host-pattern.json: pattern_mismatch, wrong_type
===== 7 passed, 0 failed, 1 warned, 0 skipped =====

🤖 Generated with Claude Code

miteshashar and others added 2 commits February 20, 2026 09:08
Add `node ./cli.js coverage` task that runs 8 coverage checks on
schemas opted into the `coverage` array in schema-validation.jsonc.
Supports `strict` mode for CI enforcement via object syntax.

Checks: unused $defs, description coverage, test completeness,
enum coverage, pattern coverage, required field coverage, default
value coverage, negative test isolation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Schema:
- Add fastMode boolean property (default: false)
- Add default: true to sandbox.autoAllowBashIfSandboxed
- Enhance spinnerVerbs.mode description with enum explanations
- Add 19 missing property descriptions (statusLine, fileSuggestion,
  sandbox.network, 14 marketplace source type discriminators)

Tests:
- Add 13 uncovered top-level properties to modern-complete-config.json
- Add untested enum values across 6 test files (all 7 enums fully covered)
- Add non-default value tests for 9 properties (edge-cases, basic-config)
- Add git URL pattern test to marketplace-host-pattern.json
- Add defaultMode coverage across permissions test files

Negative tests:
- Add 4 invalid enum values to invalid-enum-values.json
- Add git URL pattern violation to invalid-marketplace-host-pattern.json
- Create invalid-mcp-server-name.json for serverName pattern violations

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@github-actions github-actions bot added ci ".github/**/*" folder is updated (auto-generated by labeler action) documentation "**/*.md" folder is updated (auto-generated by labeler action) cli.js gruntfile.js is updated (auto-generated by labeler action) labels Feb 20, 2026
@miteshashar
Copy link
Contributor Author

Closing: PR incorrectly includes coverage tool changes from #5383. Will re-open with only claude-code-settings schema and test changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci ".github/**/*" folder is updated (auto-generated by labeler action) cli.js gruntfile.js is updated (auto-generated by labeler action) documentation "**/*.md" folder is updated (auto-generated by labeler action)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

Comments