Collect and report code coverage in CI#52
Merged
Conversation
Adds Microsoft.Testing.Extensions.CodeCoverage to each test project and wires ReportGenerator into the build workflow so every run emits a GitHub-flavored markdown summary. The summary is appended to the job summary and, on pull requests, posted as a sticky comment so authors see at a glance which areas they touched without tests. https://claude.ai/code/session_01FjeHRA2Wauszz7JaEE8dAR
CoverageLines: 1835 / 14439 (12.7%) Branches: 642 / 9339 (6.9%) Files changed in this PRNo coverage-instrumented files changed in this PR. StrongTypes — lines 9.2% (1278/13841), branches 5.5% (504/9167)Collections — lines 91.5% (257/281), branches 82.5% (94/114)
Collections_Old — lines 1.6% (40/2504), branches 1.6% (17/1080)
Coproduct_Old — lines 2.4% (87/3580), branches 0.7% (23/3123)
Digits — lines 97.4% (74/76), branches 90.9% (20/22)
Enums — lines 100.0% (58/58), branches 95.5% (21/22)
Exceptions — lines 78.9% (15/19), branches 50.0% (7/14)
Extensions_Old — lines 2.0% (120/6061), branches 1.5% (58/3990)
Maybe — lines 92.1% (152/165), branches 84.2% (96/114)
Numbers — lines 100.0% (79/79), branches 94.4% (17/18)
Product_Old — lines 57.1% (12/21), branches n/a (0/0)
Strings — lines 47.3% (69/146), branches 54.8% (34/62)
Try — lines 100.0% (10/10), branches 75.0% (3/4)
Try_Old — lines 29.1% (161/553), branches 13.3% (56/420)
generated — lines 50.0% (144/288), branches 31.5% (58/184)
StrongTypes.Analyzers — lines 91.7% (188/205), branches 86.6% (71/82)(root) — lines 91.7% (188/205), branches 86.6% (71/82)
StrongTypes.Api — lines 100.0% (226/226), branches 78.9% (30/38)(root) — lines 100.0% (11/11), branches n/a (0/0)
Controllers — lines 100.0% (122/122), branches 78.9% (30/38)
Data — lines 100.0% (52/52), branches n/a (0/0)
Entities — lines 100.0% (12/12), branches n/a (0/0)
Models — lines 100.0% (29/29), branches n/a (0/0)
StrongTypes.EfCore — lines 88.7% (110/124), branches 71.2% (37/52)(root) — lines 88.7% (110/124), branches 71.2% (37/52)
StrongTypes.FsCheck — lines 76.7% (33/43), branches n/a (0/0)(root) — lines 76.7% (33/43), branches n/a (0/0)
|
ReportGenerator's Markdown outputs group by class, which surfaces every compiler-generated closure type as its own row — in extension-method-heavy code like Maybe, a single MaybeExtensions.cs turns into MaybeExtensions, MaybeExtensions<A, B>, MaybeExtensions<A, R>, MaybeExtensions<A, X, B>, … rows, all with near-identical numbers. Cobertura tags every <class> with its source filename, so we can group on that attribute instead. Adds a small Python post-processor that collapses the merged Cobertura.xml into one row per .cs file and highlights files below 50% line coverage up top, so reviewers see the "add tests here" nudge first. https://claude.ai/code/session_01FjeHRA2Wauszz7JaEE8dAR
Cobertura emits one <class> per constructed generic, so summing <line> entries naively multiplied physical lines by instantiation count. Dedupe by (filename, line number) to count each source line once and take the max of branch coverage across instantiations. Also compact source-generator output paths to `generated/<name>`, translate backtick arity to `<T>` so the filename stays inside its markdown code span, and replace the "below 50%" section with a PR-scoped "Files changed in this PR" table driven by `gh pr view`.
Each project gets its own collapsible <details> block with project-level totals in the summary line; within a project, files are grouped under bold folder headers with per-folder percentages. Source-generator output folds into a `generated` folder under its originating project instead of sitting at top level.
Each folder becomes its own <details open> block under its project, so readers can collapse noisy areas (e.g. _Old folders) while keeping the rest expanded. Folder headers now carry absolute covered/total counts alongside percentages, matching the project summary line. A <br> after the project summary adds vertical breathing room before the first folder.
Debug builds preserve per-statement sequence points, so coverage lines up 1:1 with source. Release merges those points and folds constant branches, which skews the coverage numbers slightly. Switch the PR/push build to Debug and drop it on release events — the publish job now Release-builds, runs the full test suite against those binaries, and only pushes to nuget.org if they pass.
The two paths now share no steps and no dependencies: ci.yml runs on PR/push, builds Debug, runs tests with coverage, posts the sticky PR comment; release.yml runs on release-published, builds Release, runs tests against shipping binaries, pushes to nuget.org, and attaches the nupkgs to the release. Each file now has a single `on:` trigger and no job-level conditionals, which reads much more cleanly than the prior single-file fork.
Rename ci.yml back to build.yml (workflow name `build`, job name `build`) so existing branch protections and status-check names keep working. The new release.yml sits next to it, unchanged.
Upload the merged Cobertura.xml to codecov.io via the codecov-action and configure components per project so the PR comment groups coverage by StrongTypes / Analyzers / EfCore / Api / FsCheck. Left `informational: true` on both project and patch statuses so codecov can't fail the PR while we're evaluating. The custom sticky comment still posts — both appear on the PR and we pick one afterwards.
Welcome to Codecov 🎉Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests. ℹ️ You can also turn on project coverage checks and project coverage reporting on Pull Request comment Thanks for integrating Codecov - We've got you covered ☂️ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds coverage collection + reporting to PRs via two parallel channels so we can compare and pick one:
scripts/coverage-summary.pyrenders Cobertura into a markdown summary grouped by project → folder with nested collapsibles, absolute counts alongside percentages, and aFiles changed in this PRsection scoped to the PR diff. Also fixes generic-heavy totals (Cobertura emits one<class>per constructed generic; we dedupe by(filename, line number)so a 200-line file isn't counted as 140,000).codecov/codecov-action@v5uploads the merged Cobertura to codecov.io, withcodecov.ymlmapping paths to one component per project. Bothprojectandpatchstatuses areinformational: trueso codecov can't fail the PR while we're evaluating.Also splits the workflow:
build.ymlruns on push/PR in Debug (so coverage sequence points line up 1:1 with source); the newrelease.ymlruns on release events in Release, tests the shipping binaries, and only then pushes to nuget.org.How it works
Collection. Each test project references
Microsoft.Testing.Extensions.CodeCoverage. TheTeststep forwards coverage flags through the--separator:Merge.
dotnet-reportgenerator-globaltoolconsolidates per-project*.cobertura.xmlinto one merged XML plus a drill-down HTML report.Rendering (custom).
scripts/coverage-summary.pyreads the merged XML and emits a markdown document with:Files changed in this PRtable driven bygh pr view --json files<details>per project with project-level totals in the summary line<details open>per folder inside each project, with per-folder totalssrc/<proj>/obj/**/*.g.csfolds into agenerated/folder, with arity backticks (`1) translated to<T>so the filename stays inside its markdown code spanRendering (codecov).
codecov.ymlsetscomment: layout: header, diff, components, treeso the codecov comment shows overall delta, the@@ Coverage Diff @@block, a per-component table (one row per project), and a folder-grouped file tree.Visibility. Coverage surfaces four ways:
$GITHUB_STEP_SUMMARY)coverage, edits in place viamarocchino/sticky-pull-request-comment@v2)coverage-report, 14-day retention) for offline drill-downSafety. Every downstream step uses
!cancelled()+hashFilesguards so a build failure leaves no misleading red steps.Workflow split
build.ymlrelease.ymlDebug-in-CI matters for coverage fidelity: Release merges sequence points and can fold constant branches, slightly distorting the numbers. Shipping binaries are still validated in Release via
release.yml'sTeststep before nuget push.Test plan
buildworkflow runs green on this PRcoveragecomment appears with the project/folder breakdowncoverage-reportartifact is attached and contains the HTML drill-down