Feature Roadmap: Config Validation

[Kilo59]

This issue breaks out item #10 from #100 into a dedicated feature roadmap for Config Validation.

When running ruff-sync pull, validating the merged configuration before applying it ensures that we don't introduce syntax errors, conflicting versions, or deprecated rules into the target repository.

Proposed Features & Prioritization

🔴 Priority 1: Basic Syntax and CLI Validation (Must Have)

Before writing the final pyproject.toml or ruff.toml, ruff-sync should verify that the resulting file is structurally valid and accepted by Ruff.

• TOML Syntax Check: Ensure the merged config is valid TOML and doesn't introduce parsing errors.
• Tool Integrity Check: Optionally (or by default) run ruff check --isolated --config <merged-temp-file> <some-dummy-file> or rely on Ruff's native config validation to ensure Ruff accepts the configuration keys and structure.
• Fail Fast: If the resulting config is broken or rejected by Ruff, ruff-sync should abort the operation, log an error, and leave the local file untouched.

🟠 Priority 2: Python Version Consistency (Should Have)

A common issue is syncing an upstream config designed for a different Python version than the downstream project.

• Cross-check target-version: Warn the user if the merged [tool.ruff] target-version (e.g., py311) conflicts with the local project's [project] requires-python (e.g., >=3.10).
• Extend to other tools (if applicable): For instance, check [tool.mypy] python_version against requires-python if multi-tool syncing (Support syncing other pyproject.toml tool configurations (e.g., mypy, pytest) #69) is implemented.

🟡 Priority 3: Rule Deprecation Warnings (Could Have)

Ruff evolves quickly, and upstream central configs might contain deprecated rules that cause noise when applied downstream.

• Detect Deprecated Rules: Warn if rules in select, extend-select, or ignore are marked as deprecated in the target project's installed version of Ruff.
• Contextual Output: Provide helpful CLI warnings: Warning: Upstream config uses deprecated rule 'UP036', which may be removed in future Ruff versions.

🟢 Priority 4: Strict Mode (Nice to Have)

Provide an option to upgrade warnings to protective errors.

• --strict flag: When passed to ruff-sync pull, any version conflicts or deprecated rules will cause the sync to fail entirely rather than just warning the user, protecting CI pipelines from downstream breakages.

Related

• Extracted from Feature roadmap: ecosystem-inspired improvements for ruff-sync #100

[Kilo59]

Additional Config Validation Features

Based on an analysis of Ruff's native capabilities and other tools like ESLint, here are additional features that should be considered for the Config Validation tier:

1. Incompatible Rule Detection (Semantic Validation)

Ruff has specific rules that conflict with each other or with the ruff format command (e.g., D203 and D211, D212 and D213, ISC001). ruff-sync could programmatically detect if the final merged configuration accidentally enables these conflicting rules, which usually results in unresolvable lint warnings or formatting loops.

2. JSON Schema Validation

Tools like ESLint and Prettier rigorously validate configuration schema. Since Ruff provides an official JSON schema (available at https://json.schemastore.org/ruff.json), ruff-sync could use a lightweight validator against the final TOML structure before writing it. This would catch:

• Misspelled keys (e.g., lines-length instead of line-length).
• Incorrect types (e.g., providing a string where an array is expected).
• Unknown tools configuration blocks outside of what is supported.

3. Config Inspector & Attribution (Dry-run Analysis)

Inspired by ESLint v9's new Config Inspector, ruff-sync could provide a --inspect or --dry-run --explain command that visually outputs the merged configuration, explicitly tagging each key with its source (e.g., "From Local", "From Upstream URL"). This helps teams debug why a certain rule is enabled or ignored.

4. Path & Exclusion Resolution Warnings

ESLint warns when files or ignores patterns don't match anything. Similarly, ruff-sync could validate local paths listed in extend-exclude, exclude, or per-file-ignores. If an upstream central config ignores a path like tests/legacy/ but that directory doesn't exist locally, ruff-sync could surface a warning about orphaned configuration paths.

5. "Read-Only" Upstream Enforcement

If a local project explicitly disables a rule that is supposed to be strictly enforced by the upstream organization config (e.g., bypassing security linting), ruff-sync could support an enforce_keys mechanism from the upstream config side. It would warn or fail if the local downstream config attempts to override them.