Richer CI output: JSON, SARIF, and GitHub Actions annotations for check command

[Kilo59]

Summary

Enhance the check command with machine-readable output formats and GitHub Actions-native annotations, making drift detection a first-class CI citizen.

Parent: #100 (Tier 1)

Motivation

The current check command outputs human-readable text and a unified diff. CI systems benefit from structured output:

• JSON — parseable by scripts and dashboards
• SARIF — GitHub's standard for code analysis results (auto-surfaces in PR "Code Scanning" tab)
• GitHub Actions annotations — ::error:: and ::warning:: messages appear inline on PR diffs
• Exit code differentiation — distinguish "out of sync" from "upstream unreachable"

Ecosystem precedent: golangci-lint (--out-format json,sarif,github-actions), ESLint (--format json), ruff itself (--output-format json).

Proposed CLI Interface

# Default (unchanged): human-readable text + diff
ruff-sync check

# Machine-readable formats
ruff-sync check --output-format json
ruff-sync check --output-format sarif
ruff-sync check --output-format github   # emits ::error:: annotations

Exit Code Specification

Code	Meaning
0	In sync
1	Out of sync (drift detected)
2	Upstream unreachable or fetch error

Currently both drift and errors return 1. Distinguishing them lets CI workflows react differently (e.g., failing loudly on 2 but opening a PR on 1).

Implementation Plan

1. Create CheckResult data class (core.py)

Replace the current print-and-return pattern with a structured result:

@dataclasses.dataclass
class CheckResult:
    """Structured result from the check command."""
    in_sync: bool
    target_path: pathlib.Path
    upstream_url: URL
    diff_text: str | None = None    # unified diff if requested
    changed_keys: list[str] | None = None   # keys that differ
    error: str | None = None

Refactor check() to return CheckResult and move formatting to callers.

2. Add output formatters (core.py or new formatters.py)

def format_text(result: CheckResult) -> str:
    """Human-readable output (current behavior)."""

def format_json(result: CheckResult) -> str:
    """JSON output with structured data."""
    return json.dumps({
        "in_sync": result.in_sync,
        "target": str(result.target_path),
        "upstream": str(result.upstream_url),
        "diff": result.diff_text,
        "changed_keys": result.changed_keys,
    }, indent=2)

def format_sarif(result: CheckResult) -> str:
    """SARIF v2.1.0 output for GitHub Code Scanning."""

def format_github(result: CheckResult) -> str:
    """GitHub Actions annotation format."""
    if not result.in_sync:
        return f"::error file={result.target_path}::Ruff configuration is out of sync with upstream"
    return ""

3. Add --output-format CLI argument (cli.py)

check_parser.add_argument(
    "--output-format",
    choices=["text", "json", "sarif", "github"],
    default="text",
    help="Output format for check results. Default: text.",
)

4. Update check() to populate CheckResult (core.py)

Instead of printing inline, build a CheckResult and return it. The CLI layer in main() calls the appropriate formatter and prints:

# In main()
result = await check(exec_args)
formatter = {
    "text": format_text,
    "json": format_json,
    "sarif": format_sarif,
    "github": format_github,
}[exec_args.output_format]
print(formatter(result))
return 0 if result.in_sync else (2 if result.error else 1)

5. Compute changed keys for richer diffs

When comparing merged vs. source, walk both tool.ruff tables and collect the keys that differ. This enables JSON/SARIF to report which keys drifted, not just a monolithic diff.

def _find_changed_keys(
    source: Table | TOMLDocument,
    merged: Table | TOMLDocument,
    prefix: str = "",
) -> list[str]:
    """Return list of dotted key paths that differ between source and merged."""

6. Update Arguments NamedTuple (cli.py)

Add output_format: str = "text" field.

SARIF Schema (Minimal)

The SARIF output should conform to SARIF v2.1.0 with:

• Tool: ruff-sync with version
• One result per run: level error if out of sync, none if in sync
• Physical location pointing to the target pyproject.toml file
• Message including which keys drifted

Test Plan

1. Unit tests for each formatter function against a known CheckResult.
2. JSON output test: parse the JSON output and verify keys and structure.
3. SARIF validation: validate output against SARIF JSON schema.
4. GitHub annotations test: verify ::error file=... format matches GitHub spec.
5. Exit code tests: verify 0, 1, 2 codes under different scenarios.
6. Backward-compat test: verify default behavior is unchanged.

Files Changed

File	Change
src/ruff_sync/core.py	CheckResult dataclass, _find_changed_keys(), refactor check() to return CheckResult
src/ruff_sync/cli.py	--output-format argument, Arguments.output_format field, formatter dispatch in main()
src/ruff_sync/formatters.py	[NEW] Output formatters (format_text, format_json, format_sarif, format_github)
src/ruff_sync/__init__.py	Export CheckResult
tests/test_basic.py	Tests for formatters and exit codes

CI Integration Example

# .github/workflows/ruff-sync.yml
- name: Check ruff-sync
  run: |
    ruff-sync check --output-format github
  # Exit 1 = out of sync (fail PR), exit 2 = upstream issue (warning)

[Kilo59]

Sub-issue created: #134 — GitLab CI codequality report support (GitlabFormatter + --output-format gitlab). Includes a full implementation reference in .agents/gitlab-reports.md.