Skip to content

Coverage reporting refactor and cleanup#397

Merged
Cadair merged 17 commits into
OpenAstronomy:mainfrom
Cadair:fixup_gh_cov
Jun 3, 2026
Merged

Coverage reporting refactor and cleanup#397
Cadair merged 17 commits into
OpenAstronomy:mainfrom
Cadair:fixup_gh_cov

Conversation

@Cadair
Copy link
Copy Markdown
Member

@Cadair Cadair commented Jun 3, 2026

fixes #395 and fixes #383

This PR makes the following changes:

  1. Breaking change you need to set setenv = COVERAGE_FILE={toxinidir}/.coverage in tox.ini if your tests are not run in the {toxinidir} directory, so that reporting finds the .coverage file.
  2. coverage: codecov no longer requires the use of pytest-cov, the xml coverage report is generated using coverage.py in a separate step to the tox run.
  3. We now support OIDC auth to codecov meaning you don't need CODECOV_TOKEN if using this.
  4. coverage: github is now tested against both pytest-cov and coverage.py, and as long as the .coverage files are in the root directory will work correctly.
  5. As discussed in BUG: Coverage that works for v2.4.0 breaks after v2.5.0 #383 (comment) codecov will now fail the job on error, including no files updated. This is particularly critical with the breaking change.

@Cadair Cadair force-pushed the fixup_gh_cov branch 3 times, most recently from de82f8e to 6912510 Compare June 3, 2026 10:16
@Cadair Cadair force-pushed the fixup_gh_cov branch 4 times, most recently from 62bc579 to 231bbd9 Compare June 3, 2026 10:46
@Cadair Cadair force-pushed the fixup_gh_cov branch 2 times, most recently from 8b40a0f to ff18080 Compare June 3, 2026 11:27
@codecov-commenter
Copy link
Copy Markdown

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

Thanks for integrating Codecov - We've got you covered ☂️

@Cadair Cadair marked this pull request as ready for review June 3, 2026 11:44
@Cadair Cadair changed the title Attempts at smoothing the migration path to coverage:github Coverage reporting refactor and cleanup Jun 3, 2026
Copy link
Copy Markdown
Contributor

@astrofrog astrofrog left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems ok to me! Did you test it out on a real package? If so feel free to merge.

@Cadair
Copy link
Copy Markdown
Member Author

Cadair commented Jun 3, 2026

I'm having trouble with it on sunpy: sunpy/sunpy#8665

@Cadair
Copy link
Copy Markdown
Member Author

Cadair commented Jun 3, 2026

Comment thread tox.ini Outdated
Comment thread .github/workflows/tox.yml
Comment thread .github/workflows/tox.yml Outdated
@Cadair
Copy link
Copy Markdown
Member Author

Cadair commented Jun 3, 2026

Do people think it's worth throwing a warning about the COVERAGE_FILE change if we get to the codecov upload step and there isn't a .coverage file to work with?

@pllim
Copy link
Copy Markdown
Contributor

pllim commented Jun 3, 2026

Do people think it's worth throwing a warning

I would say, yes. Not everyone follows the PRs here or even read the change log.

@Cadair
Copy link
Copy Markdown
Member Author

Cadair commented Jun 3, 2026

Ok, done, and failing test added to demo. I think this is ready, I'll put out a 3.0 after this.

@Cadair Cadair requested a review from pllim June 3, 2026 16:51
Copy link
Copy Markdown
Contributor

@pllim pllim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a comment and a question.

Comment thread docs/source/tox.rst Outdated
@pllim
Copy link
Copy Markdown
Contributor

pllim commented Jun 3, 2026

So you just gonna leave the py314-tmpdir_warning job red?

@Cadair
Copy link
Copy Markdown
Member Author

Cadair commented Jun 3, 2026

So you just gonna leave the py314-tmpdir_warning job red?

yep lol

Cadair and others added 2 commits June 3, 2026 17:12
Co-authored-by: P. L. Lim <2090236+pllim@users.noreply.github.com>
@pllim
Copy link
Copy Markdown
Contributor

pllim commented Jun 3, 2026

:shipit:

@Cadair Cadair merged commit e113052 into OpenAstronomy:main Jun 3, 2026
90 of 91 checks passed
@Cadair Cadair deleted the fixup_gh_cov branch June 3, 2026 19:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Consistency between coverage: codecov and coverage: github for easy transition BUG: Coverage that works for v2.4.0 breaks after v2.5.0

5 participants