Track stable Next.js API surface with a checked-in manifest and CI coverage gate

[Divkix]

Summary

vinext already has strong compatibility infrastructure:

• upstream-ported tests in tests/nextjs-compat/TRACKING.md
• a migration/compatibility scanner in packages/vinext/src/check.ts
• CI coverage in .github/workflows/ci.yml
• daily ecosystem validation in .github/workflows/ecosystem.yml
• advisory canary testing in .github/workflows/tip.yml

What we do not have yet is a machine-readable compatibility model for stable Next.js releases.

Today, compatibility knowledge is spread across:

• the inline shim alias map in packages/vinext/src/index.ts
• shim source files under packages/vinext/src/shims/
• user-facing support maps in packages/vinext/src/check.ts
• ported Next.js tests in tests/nextjs-compat/

That works, but it means we do not have a single CI-enforced answer to:

• what is the stable public next/* API surface for version x.y.z?
• which exports are implemented by vinext?
• which missing exports are intentional vs accidental?
• when did stable Next.js add or remove something we should track?

This issue proposes a stable-only compatibility tracking system built around a checked-in manifest extracted from the published next npm package.

Goals

• Track stable Next.js public runtime API surface from the published package.
• Check in a canonical api-manifest.json for the currently supported stable version.
• Enforce shim coverage in CI.
• Document intentional gaps explicitly.
• Add a small behavioral contract suite for high-risk parity cases.
• Keep canary as advisory signal only.

Non-Goals

• Do not make canary a required merge gate.
• Do not use check.ts as the canonical compatibility model.
• Do not block this work on type-export tracking in v1.
• Do not block this work on a broad ecosystem internal-import scanner.
• Do not rewrite the existing Next.js compat suite.

Decisions

• v1 tracks runtime exports only.
• Weekly stable tracking opens an issue, not an auto-PR.
• Ecosystem workflows should pin to the exact manifest version, not a floating minor like "16.1".
• check.ts remains a user-facing migration tool, not the internal engineering source of truth.

Current vs Proposed

Area	Current	Proposed
Source of truth	Inline shim map in index.ts + hand-maintained support maps in check.ts	Checked-in api-manifest.json + exported shim registry + known-gaps.json
Stable release tracking	Manual / ad hoc	Automated stable-only diff workflow
PR protection	Existing CI/E2E/ecosystem checks	Existing checks + required shim coverage gate
Behavioral drift detection	Ported tests and ecosystem builds	Existing tests + small dual-target contracts
Canary	Advisory only	Still advisory only

Architecture

Current:

             Next.js stable changes
                    |
              humans notice
                    |
   +----------------+----------------+
   |                                 |
ported tests                    manual shim updates
(TRACKING.md)                   (index.ts + shims)
   |                                 |
   +----------------+----------------+
                    |
                   CI

Proposed:

          published next@stable release
                    |
         extract-nextjs-api script
                    |
             api-manifest.json
                    |
   +----------------+----------------+
   |                                 |
weekly diff workflow         PR coverage gate
(open issue)                 (fail on missing exports
                             unless in known-gaps.json)
   |                                 |
   +----------------+----------------+
                    |
        existing CI + contract tests

Registry refactor:

TODAY
index.ts
  - owns inline nextShimMap
  - resolve.alias uses it
  - resolveId uses it

PROPOSED
shims/registry.ts
  - public shim entries
  - internal shim entries
  - shared helper(s)

index.ts
  - imports registry

coverage script
  - imports same registry

Proposed Work

1. Extract a stable public API manifest

Add:

• scripts/extract-nextjs-api.mjs
• api-manifest.json

Behavior:

• download or npm pack the published stable next@x.y.z
• inspect public next/* entrypoints from the published package
• record runtime exports only in v1
• store version metadata in the manifest

Example shape:

{
  "version": "16.1.6",
  "modules": {
    "next/cache": {
      "runtime": [
        "unstable_cache",
        "revalidatePath",
        "revalidateTag",
        "updateTag",
        "refresh"
      ]
    }
  }
}

Notes:

• Prefer the published npm package over repo source as the source of truth.
• Runtime-only keeps v1 simpler and covers what actually breaks apps.
• Wrapper modules are mostly explicit; next/navigation may require a one-hop resolution into compiled output.

2. Move the shim registry into a shared module

Add:

• packages/vinext/src/shims/registry.ts

Update:

• packages/vinext/src/index.ts

Behavior:

• move the inline nextShimMap into a reusable exported registry
• keep public next/* entries separate from internal next/dist/* aliases
• make both the plugin and the coverage script consume the same registry

This avoids parsing index.ts as source text in CI.

3. Add a required shim coverage check

Add:

• scripts/check-shim-coverage.mjs
• known-gaps.json

Update:

• .github/workflows/ci.yml

Behavior:

• read api-manifest.json
• read the shared shim registry
• inspect actual named/default exports from shim files
• fail if a stable public export is missing unless explicitly listed in known-gaps.json
• fail if a manifest module has no mapped shim and no documented gap

Suggested known-gaps.json shape:

{
  "next/jest": {
    "exports": ["*"],
    "status": "wont-fix",
    "reason": "vinext uses Vitest, not Jest"
  },
  "next/amp": {
    "exports": ["useAmp"],
    "status": "stub",
    "reason": "Deprecated API; shimmed behavior is intentionally minimal"
  }
}

Suggested statuses:

• planned
• stub
• wont-fix

This lets reviewers distinguish intentional non-goals from temporary gaps.

4. Add stable-only tracking workflow

Add:

• .github/workflows/nextjs-api-track.yml
• optionally scripts/diff-nextjs-api.mjs

Behavior:

• run weekly
• check npm view next version
• compare against api-manifest.json
• if unchanged, exit
• if changed, extract new manifest and open one tracking issue with the diff

This should target stable releases only.

5. Add a small dual-target contract suite

Add:

• tests/contracts/
• tests/fixtures/contract-app/

Initial contracts:

• redirect() status behavior
• cookies() mutability gating
• headers() read-only behavior
• middleware request header propagation
• generateMetadata() merge chain

Behavior:

• same fixture app should run under vinext and real stable Next.js
• use as signal, not a required PR gate in v1
• run on schedule or workflow dispatch against stable Next.js

6. Keep canary advisory-only

Update:

• .github/workflows/tip.yml

Behavior:

• keep next@canary as non-blocking early warning
• optionally run a focused compat subset
• do not make canary failures block PRs or releases

Testing / Validation

This work needs tests at three layers: script correctness, shim coverage enforcement, and behavioral parity.

1. Script-level tests

Add Vitest coverage for the new scripts and helpers.

Suggested files:

• tests/api-manifest.test.ts
• tests/shim-coverage.test.ts

These should cover:

• extracting runtime exports from explicit wrapper modules
• handling module.exports = require(...) style wrappers
• handling the one-hop next/navigation style case
• diffing old vs new manifests
• failing when a manifest export has no shim and no known gap
• passing when a missing export is explicitly listed in known-gaps.json

Prefer fixture-based tests over hitting the network.

Suggested fixtures:

• tests/fixtures/next-api-manifest/

2. Registry refactor safety

The registry extraction from index.ts must not change runtime behavior.

Validation:

• existing shim tests continue to pass
• existing compat tests continue to pass
• resolve.alias and resolveId still handle:
  • next/*
  • next/*.js
  • internal next/dist/* aliases

Minimum targeted test runs:

• pnpm test tests/shims.test.ts
• pnpm test tests/nextjs-compat/

3. Contract tests

Add a small dual-target suite for behavioral parity.

Suggested files:

• tests/contracts/redirect.contract.test.ts
• tests/contracts/request-apis.contract.test.ts
• tests/contracts/middleware.contract.test.ts
• tests/contracts/metadata.contract.test.ts

Initial cases:

• redirect() status behavior
• cookies() mutability gating
• headers() read-only behavior
• middleware request header propagation
• generateMetadata() merge chain

These should run against the same fixture app under:

• vinext
• real stable Next.js

4. CI / workflow validation

Required on PRs:

• shim coverage check against api-manifest.json

Non-blocking / scheduled:

• weekly stable release manifest diff
• advisory canary signal in tip.yml
• dual-target contract run against stable Next.js

5. Definition of done

This issue is complete when:

• api-manifest.json is checked in and versioned
• known-gaps.json is checked in and required for intentional omissions
• the shim registry no longer lives only as inline state in index.ts
• CI fails if a stable public next/* export is missing without a documented gap
• new scripts have direct Vitest coverage
• the registry refactor does not regress existing shim/compat tests
• at least 3-5 dual-target contract tests are in place
• a weekly stable-only tracking workflow exists
• tip.yml remains advisory only

Follow-ups

Out of scope for v1, but natural follow-ups:

• type-export tracking
• ecosystem internal-import scanner for next/dist/*
• auto-PR generation instead of issue-only tracking
• tighter coupling between manifest version and ecosystem workflow version pinning

Why this is worth doing

This does not replace the current test suite or ecosystem jobs. It fills a different gap: machine-enforced API drift detection for stable releases.

That gives us:

• a canonical compatibility model
• fewer silent regressions
• explicit documentation of intentional gaps
• a cleaner path to tracking new stable Next.js releases

[Divkix]

Implemented on feat/api-surface-tracking. #461

What landed:

• Stable API manifest extraction now discovers public runtime entrypoints from the published next package instead of using a hard-coded module list.
• The checked-in api-manifest.json now tracks 26 runtime modules for next@16.1.6.
• The tracking workflow in .github/workflows/nextjs-api-track.yml now works from the published artifact and de-dupes issues by version marker.
• Shim coverage is enforced against the stable manifest plus known-gaps.json.
• Shared behavioral contracts are now truly dual-target for portable HTTP behaviors in tests/contracts/http.contract.test.ts and pass against both vinext and real next dev.

Important scope clarifications:

• Manifest tracking is runtime-only in v1.
• Some discovered public modules are intentionally marked wont-fix in known-gaps.json (next/babel, next/client, next/jest, and the experimental test helpers).
• Extra migration shims can still exist outside manifest coverage if they are not part of the current stable published runtime surface.
• The server-action redirect 303 check is covered in tests/app-router.test.ts, not in the shared contract suite, because that flow depends on vinext-specific action transport.

Verification:

• pnpm test tests/api-manifest.test.ts tests/manifest-diff.test.ts tests/shim-coverage.test.ts
• pnpm test tests/app-router.test.ts -t "uses 303 for server action redirects"
• pnpm test tests/contracts/http.contract.test.ts
• CONTRACT_TARGET=nextjs pnpm test tests/contracts/http.contract.test.ts
• pnpm test tests/shims.test.ts
• manifest parity against the published next@16.1.6 package
• coverage parity check against the updated manifest
• pnpm run typecheck
• pnpm run lint
• pnpm run fmt:check

[Divkix]

Related to #204. This issue tracks the API surface (what exports exist), while #204 does a behavioral audit (whether implementations match Next.js behavior). They complement each other - the manifest tells us what to implement, the audit verifies we implement it correctly. Not blocking.

[Divkix]

Looks like it has already been solved with a github workflow.