diff --git a/REVERT.md b/REVERT.md new file mode 100644 index 000000000000..b84118c1cef8 --- /dev/null +++ b/REVERT.md @@ -0,0 +1 @@ +Reverting merge commit 313a74ef2aa09e1e67ff95b1e97e190f20e6e87c \ No newline at end of file diff --git a/docs/release/trg-10/_category_.json b/docs/release/trg-10/_category_.json new file mode 100644 index 000000000000..a5bd751ba927 --- /dev/null +++ b/docs/release/trg-10/_category_.json @@ -0,0 +1,3 @@ +{ + "label": "TRG 11 - Testing and Quality" +} diff --git a/docs/release/trg-10/trg-11-01.md b/docs/release/trg-10/trg-11-01.md new file mode 100644 index 000000000000..da88611f076b --- /dev/null +++ b/docs/release/trg-10/trg-11-01.md @@ -0,0 +1,130 @@ +--- +title: TRG 11.01 - Code Coverage +sidebar_position: 1 +--- + +| Status | Created | Post-History | +|--------|-------------|-------------------------| +| Draft | 07-Feb-2025 | Initial version created | +| Draft | 21-Mar-2025 | Update with example integration for GitHub Workflow | +| Draft | 24-Mar-2025 | Update with SonarQube Cloud reference implementation and clarifications| +| Draft | 02-Apr-2025 | Updated workflow integration example | + +## Why + +Goal: To ensure that all released software components meet sufficient test coverage to guarantee quality, stability, and reliability. + +This guideline applies to all software components and projects that are part of the Eclipse Tractus-X release process. + +## Description + +### 1. Minimum Requirements + +1.1. **Code Coverage Threshold** + +- Quality Gate: The minimum threshold for code coverage (line coverage) should be **80.0%**. +- The code coverage is calculated using both unit tests and integration tests combined. + +1.2. **Exceptions** + +Certain code sections may be excluded from counting towards the code coverage percentage. For example this could apply to code sections that: + +- Are themselves test code (no "test of tests"). +- Are autogenerated code, for example Swagger-generated API clients. +- Are configuration files with no logic to test. +- Are experimental or prototype code, for example incomplete implementations hidden behind feature toggles. +- Are boilerplate code or entity classes that follow a known, predictable pattern. +- Are code for logging, metrics, or monitoring. +- Depend on platform-specifics, for example hardware that cannot be simulated in a test. + +All exceptions should be documented and approved by the project's lead/main committers. + +### 2. Analysis and Reporting + +2.1. **Tools for Code Coverage Reporting** + +- The recommended code coverage reporting tool is **SonarQube Cloud** as provided by Elipse Tractus-X at this [reference link](https://sonarcloud.io/organizations/eclipse-tractusx/projects). +- Please note that SonarQube Cloud does not calculate the code coverage itself. To include coverage results you must set up a third-party coverage analysis tool and configure SonarQube Cloud to import the analysis results produced by that tool. +- SonarQube Cloud supports the import of coverage data in formats native to a variety of tools for a variety of languages. It also provides a possibility to import a generic format from tools that are not directly supported. See the guideline in section [4. Resources and Examples](#sonarqube-implementation-guides) as reference on how to implement this. +- The tool or tools used for test coverage analysis and to produce the code coverage report should be specified and documented within the respective Eclipse Tractus-X project. + +2.2. **Regular Review** + +- Code coverage should be measured by all Eclipse Tractus-X software products at least once before each release. +- Coverage measurement results should be integrated into CI/CD pipelines and automated as part of the release process. + +### 3. Quality Assurance + +3.1. **Code Review Requirements** + +- Reviewers must ensure that new or modified code sections are sufficiently covered by test cases. +- Code changes that lower the overall coverage below the defined Quality Gate threshold may only be accepted in exceptional cases. + +3.2. **Risk Analysis** + +- If the code coverage value for an Eclipse Tractus-X project falls below 80.0% for an (upcoming) release and this cannot be improved anymore in time before the release, this potential risk should be documented and approved by the project lead/main committers. + +### 4. Resources and Examples + + + +#### 4.1 **SonarQube Cloud Implementation Guides** + +See the following SonarQube Cloud guideline for the importing of test coverage reports with detailed guides for several popular programming languages as [reference](https://docs.sonarsource.com/sonarqube-cloud/enriching/test-coverage/overview/). + +#### 4.2. **GitHub Workflow Integration Code Example** + +An example of a GitHub workflow integration using Java and Apache Maven can be found below. + +```yaml +name: "Sonar Check" + +on: + pull_request: + workflow_dispatch: + push: + branches: + - main + +env: + JAVA_VERSION: 17 + +jobs: + Test-and-Sonar: + permissions: + contents: read + checks: write + pull-requests: write + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-java@v4 + with: + java-version: '${{ env.JAVA_VERSION }}' + distribution: 'temurin' + cache: 'maven' + + - name: Run unit & integration tests + run: | + mvn -B verify + + - name: Clean working directories + run: | + rm -rf .scannerwork + rm -rf .sonar + + - name: Cache SonarCloud packages + uses: actions/cache@v4 + with: + path: ~/.sonar/cache + key: ${{ runner.os }}-sonar + restore-keys: ${{ runner.os }}-sonar + + - name: Verify Sonar Scan + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + SONAR_TOKEN: ${{ secrets.SONAR_TOKEN_BACKEND }} + SONAR_ORGANIZATION: ${{ vars.SONAR_ORGANIZATION }} + SONAR_PROJECT_KEY: ${{ vars.SONAR_PROJECT_KEY_BACKEND }} + run: mvn --batch-mode sonar:sonar -Dsonar.coverage.jacoco.xmlReportPaths=${{ github.workspace }}/target/site/jacoco-aggregate/jacoco.xml -Dsonar.projectKey=${{ vars.SONAR_PROJECT_KEY_BACKEND }} -Dsonar.organization=${{ vars.SONAR_ORGANIZATION }} +```