Skip to content

enhanced-coverage-reports.md

Latest commit

 

History

History
98 lines (69 loc) · 3.7 KB

enhanced-coverage-reports.md

File metadata and controls

98 lines (69 loc) · 3.7 KB

Enhanced Code Coverage Reports

This document describes the enhanced code coverage reporting implemented in the Azure DevOps pipeline for smtp4dev.

Overview

The pipeline now generates more useful code coverage reports by:

  1. Namespace-level breakdown: Groups coverage by namespace to show which parts of the codebase have good/poor coverage
  2. Visual indicators: Uses color-coded badges (🟢🟡🔴) to quickly identify coverage levels
  3. Comprehensive details: Shows line, branch, and method coverage for each namespace
  4. GitHub integration: Posts enhanced reports as PR comments for easy visibility

Features

Enhanced GitHub Comments

The coverage report now includes:

  • Overall coverage summary with color-coded status indicators
  • Namespace breakdown table showing coverage metrics for each namespace
  • Class counts per namespace to understand scope
  • Direct links to detailed coverage reports in Azure DevOps

Coverage Thresholds

Coverage badges are color-coded based on these thresholds:

  • 🟢 Green: ≥80% coverage (Good)
  • 🟡 Yellow: 60-79% coverage (Moderate)
  • 🔴 Red: <60% coverage (Needs improvement)

Example Output

## 📊 Code Coverage Report

| Coverage Type | Percentage | Status |
|---------------|------------|--------|
| Line Coverage | 71.3% | 🟡 |
| Branch Coverage | 61.7% | 🟡 |
| Method Coverage | 69% | 🟡 |

### 📁 Coverage by Namespace

| Namespace | Line Coverage | Branch Coverage | Method Coverage | Classes |
|-----------|---------------|-----------------|-----------------|---------|
| `Rnwood.SmtpServer` | 74.5% 🟡 | 68.2% 🟡 | 69.5% 🟡 | 43 |
| `Rnwood.SmtpServer.Extensions` | 74.2% 🟡 | 25% 🔴 | 91.3% 🟢 | 6 |
| `Rnwood.SmtpServer.Extensions.Auth` | 57.5% 🔴 | 48.8% 🔴 | 57.5% 🔴 | 23 |
| `Rnwood.SmtpServer.Verbs` | 100% 🟢 | 100% 🟢 | 100% 🟢 | 4 |

Technical Implementation

Pipeline Changes

The enhancement is implemented in the ReportCoverage stage of azure-pipelines.yml:

  1. Coverage data parsing: Reads Summary.json generated by ReportGenerator
  2. Namespace grouping: Extracts namespace information from class names
  3. Coverage calculation: Aggregates line/branch/method coverage per namespace
  4. GitHub API integration: Posts/updates PR comments with enhanced reports

Namespace Extraction Logic

The script handles various class naming patterns:

  • Simple classes: Namespace.ClassName
  • Nested classes: Namespace.ClassName/NestedClass
  • Multi-level namespaces: Namespace.SubNamespace.ClassName

Future Enhancements

The pipeline includes preparation for PR delta reporting:

# Future enhancement: Download base branch coverage for comparison
# To implement PR deltas:
# 1. Get base branch name: $(System.PullRequest.TargetBranch)
# 2. Download artifact from latest successful build on base branch
# 3. Compare current vs base coverage using enhanced script logic
# 4. Add delta columns to namespace table and overall summary

Benefits

  1. Better visibility: Developers can quickly identify which namespaces need testing attention
  2. Focused effort: Coverage gaps are clearly highlighted by namespace
  3. Progress tracking: Visual indicators make it easy to see improvement trends
  4. Actionable insights: Namespace-level detail helps prioritize testing efforts

Configuration

The enhanced reporting is automatically enabled for all PRs when:

  • The GITHUB_TOKEN environment variable is configured
  • Coverage data is successfully collected during test runs
  • The ReportCoverage stage executes for PR builds

No additional configuration is required - the enhancement works with the existing coverage collection infrastructure.