Skip to content

Skip JET QA tests on Julia 1.13 until JET supports it#823

Draft
ChrisRackauckas-Claude wants to merge 2 commits into
JuliaDiff:masterfrom
ChrisRackauckas-Claude:gate-jet-qa-113
Draft

Skip JET QA tests on Julia 1.13 until JET supports it#823
ChrisRackauckas-Claude wants to merge 2 commits into
JuliaDiff:masterfrom
ChrisRackauckas-Claude:gate-jet-qa-113

Conversation

@ChrisRackauckas-Claude

@ChrisRackauckas-Claude ChrisRackauckas-Claude commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Every "Julia pre" CI job has failed on master since the JET 0.11 compat bump: the
JET.@test_opt calls in test/QATest.jl report runtime dispatches on Julia
1.13.0-rc1 whose leaf frames are all in Base (broadcast preprocess,
simd_index/CartesianIndex, SubArray construction under view,
mightalias/_isdisjoint/_in_tuple).

These are false positives in JET's Julia 1.13 analysis, not ForwardDiff (or Base)
problems:

  • Native inference (Base.infer_return_type / code_typed) of every flagged frame,
    with the exact argtypes from the reports, is concrete and dispatch-free — and
    identical on 1.12.6 and 1.13.0-rc1.
  • The flagged kernels are allocation-free at runtime on both versions.
  • The same reports reproduce with self-contained MWEs that don't involve ForwardDiff
    at all (e.g. @report_opt view(rand(3,3), :, 1:2) and a plain
    CartesianIndex(1, CartesianIndex(2)) are flagged on 1.13-rc1, clean on 1.12.6,
    same JET 0.11.5).
  • JET's own Julia 1.13 support is still in progress (Add Julia 1.13 support while maintaining Julia 1.12 compatibility aviatesk/JET.jl#815, Declare documented API public (for downstream ExplicitImports) #819), and
    the pending 1.13-compat PR does not fix these reports (verified against that
    branch).

Filed upstream with the reduced MWEs and evidence: aviatesk/JET.jl#839.

This PR gates the JET QA testset on isempty(VERSION.prerelease), so it runs on
every released Julia version and skips on rc/beta/DEV builds, where JET's analysis
routinely lags the compiler (as it does now on 1.13.0-rc1). Comment links the
upstream issue. Test-skip explicitly authorized by @ChrisRackauckas for this case.

Verified locally: on Julia 1.12.6 (VERSION.prerelease == ()) the JET testset still
runs (4/4 pass); on 1.13.0-rc1 (("rc1",)) it is skipped and QATest.jl loads
cleanly.


Note

Opened as a draft by an agent on behalf of @ChrisRackauckas. Please ignore
until reviewed by @ChrisRackauckas.

🤖 Generated with Claude Code

On Julia 1.13.0-rc1 the QA @test_opt calls fail with runtime-dispatch
reports whose leaf frames are all in Base (broadcast preprocess,
simd_index/CartesianIndex, SubArray construction, mightalias). Native
inference of every flagged frame is concrete and identical on 1.12.6
and 1.13.0-rc1, and the kernels are allocation-free on both, so these
are false positives in JET's 1.13 analysis (its 1.13 support is still
in progress), not ForwardDiff or Base problems. Reported with reduced
JET-only MWEs in aviatesk/JET.jl#839.

Gate on v"1.13.0-" so prereleases are covered; re-enable once a JET
release analyzes 1.13 correctly.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Vx7zQ96NYk4VV4ML2s3kAC
@codecov

codecov Bot commented Jul 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.74%. Comparing base (090ddbb) to head (cbb8368).

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #823   +/-   ##
=======================================
  Coverage   90.74%   90.74%           
=======================================
  Files          11       11           
  Lines        1070     1070           
=======================================
  Hits          971      971           
  Misses         99       99           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Contributor Author

CI: the gate does its job — all 6 "Julia pre" jobs are green on this branch (they fail on master and on every other open PR). 24/25 jobs pass; the single failure ("Julia min-patch - macOS - NaN-safe disabled") is the known random gamma_inc tolerance flake at DualTest.jl:638, unrelated to this change and fixed by #822.

JET routinely lags new Julia versions during their prerelease phase, so
key the skip on isempty(VERSION.prerelease) rather than a hardcoded
1.13 bound: the QA tests run on every released Julia and skip on
rc/beta/DEV builds, where JET false positives are expected.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Vx7zQ96NYk4VV4ML2s3kAC
@ChrisRackauckas-Claude

Copy link
Copy Markdown
Contributor Author

Switched the gate to isempty(VERSION.prerelease) per review: the JET tests now run on every released Julia and skip on rc/beta/DEV builds generally, instead of hardcoding a 1.13 bound. Verified: runs 4/4 on 1.12.6, skips on 1.13.0-rc1.

@devmotion

Copy link
Copy Markdown
Member

Nobody cares about pre, so nobody cares about these test failures - neither in ForwardDiff nor other packages such as StatsFuns or SpecialFunctions pre CI is required to pass successfully. So instead of tweaking anything for CI on pre arguably the more relevant question is why to run CI on pre at all.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants