Skip to content

Add miri test coverage to CI workflow#729

Open
Copilot wants to merge 5 commits intomainfrom
copilot/add-miri-code-coverage
Open

Add miri test coverage to CI workflow#729
Copilot wants to merge 5 commits intomainfrom
copilot/add-miri-code-coverage

Conversation

Copy link
Contributor

Copilot AI commented Feb 6, 2026

Adds coverage calculation for code paths conditionally compiled with #[cfg(miri)]. Codecov will now report coverage for both regular unit tests and miri-gated code.
Let's check it in to main to verify whether codecov flags work as expected.

Changes

  • Added "Generate miri code coverage" step in .github/workflows/ci.yml coverage job
  • Runs cargo llvm-cov nextest with --config 'build.rustflags=["-Dwarnings", "--cfg", "miri"]'
  • Outputs to lcov_miri.info (regular coverage remains in lcov.info)
  • Updated codecov upload to include both files: files: lcov.info,lcov_miri.info

Implementation note

Uses --config 'build.rustflags=[...]' instead of RUSTFLAGS environment variable to preserve llvm-cov's instrumentation flags.

Original prompt

This section details on the original issue you should resolve

<issue_title>Calculate miri test coverage of unsafe code</issue_title>
<issue_description>The idea is this:
Clone Generate code coverage step in .github/workflows/ci.yml under code coverage workflow.
Name the clone Generate miri code coverage.
In the clone, add RUSTFLAGS="--cfg miri" before cargo llvm-cov command to calculate miri test coverage.
Save it as lcov_miri.info

Upload two files to codecov: lcov.info, lcov_miri.info

Expectation is that codecov will output coverages for both test mode: regular unit tests and miri.
The same coverage thresholds apply to miri test coverage.</issue_description>

Comments on the Issue (you are @copilot in this section)


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 2 commits February 6, 2026 07:57
Co-authored-by: arrayka <1551741+arrayka@users.noreply.github.com>
Co-authored-by: arrayka <1551741+arrayka@users.noreply.github.com>
Copilot AI changed the title [WIP] Add miri code coverage to CI workflow Add miri test coverage to CI workflow Feb 6, 2026
Copilot AI requested a review from arrayka February 6, 2026 08:01
@codecov-commenter
Copy link

codecov-commenter commented Feb 6, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.01%. Comparing base (f3fcaae) to head (de0f18e).

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #729   +/-   ##
=======================================
  Coverage   89.01%   89.01%           
=======================================
  Files         428      428           
  Lines       78294    78294           
=======================================
+ Hits        69692    69697    +5     
+ Misses       8602     8597    -5     
Flag Coverage Δ
miri 89.01% <ø> (?)
unittests 89.01% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.
see 2 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@arrayka arrayka marked this pull request as ready for review February 6, 2026 09:48
@arrayka arrayka requested review from a team and Copilot February 6, 2026 09:48
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds miri-specific code coverage calculation to the CI workflow to track test coverage for unsafe code paths conditionally compiled with #[cfg(miri)]. The implementation runs cargo llvm-cov with the miri configuration flag and uploads the results to Codecov as a separate flag for tracking.

Changes:

  • Added a new "Generate miri code coverage" step in the coverage job that runs tests with --cfg miri enabled
  • Split Codecov uploads into two separate steps with distinct flags: unittests for regular coverage and miri for miri-specific coverage
  • Used --config 'build.rustflags=[...]' approach to preserve llvm-cov's instrumentation flags while adding the miri cfg

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@hildebrandmw
Copy link
Contributor

Thanks for this - quick question: can you help me understand the actionable part of this pipeline? There is not an automated rejection (as far as I can tell), which would be hard to measure in any case as raw coverage is not useful in this context. New tests still require review. Is the intent to make that manual review process easier by generating the coverage report?

Of is the purpose of this PR to develop some early feedback on how this pipeline would interact with pull-requests? If the latter, it might not be a bad idea to annotate the stage as such in the YAML file so we don't accidentally forget about it 😄.

@arrayka
Copy link
Contributor

arrayka commented Feb 6, 2026

Thanks for this - quick question: can you help me understand the actionable part of this pipeline? There is not an automated rejection (as far as I can tell), which would be hard to measure in any case as raw coverage is not useful in this context. New tests still require review. Is the intent to make that manual review process easier by generating the coverage report?

Of is the purpose of this PR to develop some early feedback on how this pipeline would interact with pull-requests? If the latter, it might not be a bad idea to annotate the stage as such in the YAML file so we don't accidentally forget about it 😄.

Step 1. Yes. Automation of miri coverage for information purposes.
Step 2. Post-process the report to keep lines with 'unsafe' string only. Potential issue besides false negatives is this:
image

If step 2 fails we can make a decision to disable miri coverage as useless.

@hildebrandmw
Copy link
Contributor

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Calculate miri test coverage of unsafe code

5 participants