Skip to content

feat: add a configurable (pluggable) type-checking step to forge-precommit #48

@misnaej

Description

@misnaej

Problem

forge-precommit gates commits on ruff (lint + format) and docstring checks, but ships no static type checking. Consumers that want type safety (e.g. the-jam-machine runs mypy src/ separately in CI) have to bolt it on outside forge, so it isn't part of the "fail loudly" gate, isn't version-managed with forge, and every consumer reinvents the config.

This proposes adding type checking as a configurable forge-precommit step.

Why configurable (not "just add mypy")

The type-checker landscape shifted a lot in the last year, and no single tool is the obvious universal default for a commit gate (where a false positive trains people to --no-verify). Snapshot as of 2026-06:

mypy pyright ty (Astral) pyrefly (Meta)
Maturity stable (1.20, Nov 2025) stable, frequent releases beta (since Dec 2025), stable targeted 2026 stable v1.0 (May 2026)
Runtime pure Python (pip) TS on Node.js Rust, single static binary Rust, binary pip wheel
Speed baseline (slow) ~mypy-class 10–60× faster; ~1.5s on pandas ~35× faster than Pyre; ~1.9s on pandas
Spec conformance ~60% ~98% (highest) ~67% (beta, climbing) ~92–96%
Config [tool.mypy] (de-facto std) pyrightconfig/[tool.pyright] [tool.ty] (does not read mypy cfg) [tool.pyrefly], pyrefly init partial-migrates mypy cfg
Gate gotchas slow on big repos needs Node high false-positive rate on real code; ignores existing # type: ignore newer ecosystem, migration partial

Key takeaways for a forge gate:

  • ty is the most ecosystem-consistent with forge (same Astral model as ruff: single binary, pyproject-native, uv-friendly, no Node) — but it's beta with a notable false-positive rate today; dangerous as a blocking gate right now. Strong future default once it stabilizes.
  • pyrefly v1.0 is the strongest fast+stable option today: Rust speed, stable, reads/auto-migrates mypy config, battle-tested at Meta scale.
  • mypy is the safe universal default everyone already configures via [tool.mypy].
  • pyright has the best conformance but drags in a Node runtime — a footprint negative for a pip-installed tool.

Proposal

  1. Add a type-check step to forge-precommit, selected by config, e.g.:
    [tool.forge.typecheck]
    checker = "mypy"        # "mypy" | "ty" | "pyrefly" | "pyright" | "none"
    paths = ["src"]
  2. Default to mypy (safe, universal, reads existing [tool.mypy]).
  3. Document pyrefly as the recommended fast option (stable, mypy-config-compatible).
  4. Treat ty as a first-class supported choice and the candidate future default — revisit making it default once it reaches stable / closes the false-positive gap.
  5. Follow forge's existing CLI-on-PATH model: detect the chosen checker on PATH; if absent, warn loudly and skip rather than hard-failing on a missing optional tool. Do not bundle a checker as a hard dependency.

This keeps conservative consumers on mypy, lets speed-focused ones opt into pyrefly/ty, and positions forge to flip the default to the Astral-native ty when it's ready.

Notes / caveats

  • Conformance percentages vary across sources; treat as approximate snapshots (pyright ~98% is consistently the ceiling).
  • ty is the most fast-moving data point — re-verify its status at implementation time.

Sources (accessed 2026-06-16): astral.sh/blog/ty · github.com/astral-sh/ty · pyrefly.org/blog/v1.0 · github.com/facebook/pyrefly · pydevtools.com (mypy/pyright/ty comparison, pyrefly-1.0 upgrade) · pydantic-ai#3970 (ty false-positive data) · sphinx#14238 · blog.edward-li.com (pyrefly vs ty) · mypy-lang.blogspot.com (1.19/1.20) · microsoft/pyright releases.

Metadata

Metadata

Assignees

No one assigned

    Labels

    ci-testingCI / test infrastructurefeatureNew capabilitytier-3-standardNormal features / refactors

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions