Skip to content

github-actions.md

Latest commit

 

History

History
439 lines (363 loc) · 16.3 KB

github-actions.md

File metadata and controls

439 lines (363 loc) · 16.3 KB

GitHub Actions

This page describes how to integrate MethodAtlas into a GitHub Actions workflow. The techniques can be used individually or combined:

  • PR annotations — inline findings on the pull request diff (no licence required)
  • SARIF upload — findings in the GitHub Code Scanning tab (requires GitHub Advanced Security)
  • AI result caching — skips re-classification of unchanged test classes
  • Security test count gate — pipeline fails when security test coverage drops

Prerequisites

Requirement Details
Java runtime Java 21 or later; the examples use actions/setup-java with Eclipse Temurin
MethodAtlas Downloaded at runtime from the GitHub release; no build step required
AI provider API key Stored as a repository secret; not required for static inventory mode
GitHub Advanced Security Required only for SARIF upload to Code Scanning; not required for annotations

Minimal workflow: PR annotations

The simplest integration uses the -github-annotations flag to emit GitHub workflow commands that GitHub renders as inline annotations on the pull request diff:

  • ::warning when ai_interaction_score >= 0.8 — the test only verifies that methods were called, not what they returned.
  • ::notice for all other security-relevant methods.
name: Security test scan

on:
  pull_request:

jobs:
  scan:
    runs-on: ubuntu-latest
    permissions:
      contents: read

    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-java@v4
        with:
          java-version: '21'
          distribution: 'temurin'

      - name: Download MethodAtlas
        run: |
          curl -fsSL -o methodatlas.jar \
            https://github.com/Accenture/MethodAtlas/releases/latest/download/methodatlas.jar

      - name: Scan security tests
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
        run: |
          java -jar methodatlas.jar \
            -ai -ai-provider openai -ai-api-key-env OPENAI_API_KEY \
            -github-annotations \
            src/test/java

!!! tip "No GitHub Advanced Security licence required" The ::notice and ::warning workflow commands are standard GitHub Actions features available on all plan tiers — Free, Team, and Enterprise — for both public and private repositories. SARIF upload via the upload-sarif action requires GitHub Advanced Security; annotation output does not.

Using GitHub Models as the AI provider

GitHub Models provides free inference for supported models using the GITHUB_TOKEN automatically available in every GitHub Actions run. No additional secrets or billing setup is required.

      - name: Run MethodAtlas with GitHub Models
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: |
          java -jar methodatlas.jar \
            -ai -ai-provider github_models \
            -ai-model gpt-4o-mini \
            -ai-api-key-env GITHUB_TOKEN \
            -content-hash \
            -github-annotations \
            src/test/java

See AI Providers — GitHub Models for the list of available models and rate limits.

Caching AI results across runs

AI classification is the most expensive step in each scan. Use -content-hash together with actions/cache to skip re-classification of test classes whose source has not changed since the last run.

MethodAtlas computes a SHA-256 content fingerprint (content_hash) for each test class and stores it alongside the AI classification in CSV output. On the next run, classes whose hash matches a cache entry are served locally — no API call is made. Only changed or new classes incur a provider call.

The cache file is stored compressed (.gz) to minimise GitHub Actions cache storage consumption; the JSON-Lines cache compresses well even for very large test suites.

Caching across runs (single pass)

Use the unified cache: -ai-cache <file> reads it and -ai-cache-out <file> writes it. Because the cache is independent of the output format, one run emits SARIF (or annotations, or CSV) and refreshes the cache — no separate pass is needed. An unchanged class is served from cache with no AI call, including its credential verdicts when -detect-secrets is also enabled.

      - name: MethodAtlas AI cache
        uses: actions/cache@v4        # restores before the run, saves after
        with:
          path: .methodatlas-cache.json.gz
          key: methodatlas-ai-${{ github.ref_name }}-${{ github.sha }}
          restore-keys: |
            methodatlas-ai-${{ github.ref_name }}-
            methodatlas-ai-

      - name: Scan security tests (cached)
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
        run: |
          [ -f .methodatlas-cache.json.gz ] && gunzip -k .methodatlas-cache.json.gz
          CACHE_ARGS=()
          [ -f .methodatlas-cache.json ] && CACHE_ARGS=("-ai-cache" ".methodatlas-cache.json")

          java -jar methodatlas.jar \
            -ai -ai-provider openai -ai-api-key-env OPENAI_API_KEY \
            -content-hash \
            "${CACHE_ARGS[@]}" \
            -ai-cache-out .methodatlas-cache.json \
            -sarif \
            src/test/java \
            > methodatlas.sarif
          gzip -c .methodatlas-cache.json > .methodatlas-cache.json.gz

On the first run (cold cache) every class is classified via the AI provider. On subsequent runs only classes whose content_hash changed incur an API call; unchanged classes are read from the cache in milliseconds. Swap -sarif for -github-annotations, or omit it for CSV — the caching is identical. Because unchanged classes are still emitted, their findings stay present in the SARIF, so GitHub Code Scanning does not close the alerts between runs.

See AI Result Caching for a detailed explanation of how the cache works and how to combine it with the -diff command.

SARIF upload to Code Scanning

If your organisation has GitHub Advanced Security, upload the SARIF output so findings appear in Security → Code scanning:

      - name: Scan and produce SARIF
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
        run: |
          java -jar methodatlas.jar \
            -ai -ai-provider openai -ai-api-key-env OPENAI_API_KEY \
            -content-hash \
            -sarif \
            src/test/java \
            > methodatlas.sarif

      - name: Upload SARIF to Code Scanning
        uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: methodatlas.sarif
          category: security-tests

!!! note "GitHub Advanced Security" SARIF upload requires GitHub Advanced Security, which is included in GitHub Enterprise Cloud and GitHub Enterprise Server and is available as a paid add-on for private repositories on the Team and Free plans. For public repositories, Code Scanning is available at no additional cost. Use -github-annotations if Advanced Security is not available.

Example finding in Code Scanning

After upload, each finding appears as a Code Scanning alert with the suggested annotations, a remediation command, and a link to the affected test method:

GitHub Code Scanning alert for a security/access-control finding generated by MethodAtlas Code Scanning alert showing the suggested @DisplayName and @Tag values and the ./methodatlas -ai -apply-tags remediation command — click to view full size

Security test count gate

Fail the pipeline when the number of security-relevant test methods drops compared to the main branch baseline. This protects against accidental or silent removal of security tests.

      - name: Save baseline
        if: github.ref == 'refs/heads/main'
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
        run: |
          java -jar methodatlas.jar \
            -ai -ai-provider openai -ai-api-key-env OPENAI_API_KEY \
            -security-only -content-hash \
            src/test/java > baseline.csv

      - uses: actions/upload-artifact@v4
        if: github.ref == 'refs/heads/main'
        with:
          name: methodatlas-baseline
          path: baseline.csv
          retention-days: 90

      - name: Download baseline
        if: github.event_name == 'pull_request'
        uses: actions/download-artifact@v4
        with:
          name: methodatlas-baseline
          path: .

      - name: Count gate
        if: github.event_name == 'pull_request'
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
        run: |
          java -jar methodatlas.jar \
            -ai -ai-provider openai -ai-api-key-env OPENAI_API_KEY \
            -security-only \
            src/test/java > current.csv

          baseline=$(tail -n +2 baseline.csv | wc -l)
          current=$(tail -n +2 current.csv | wc -l)

          echo "Baseline security tests: $baseline"
          echo "Current security tests:  $current"

          if [ "$current" -lt "$baseline" ]; then
            echo "::error::Security test count dropped from $baseline to $current"
            exit 1
          fi

Full workflow

The following workflow combines caching and SARIF upload into a single, runnable definition; the annotation and count-gate techniques shown above slot in as additional steps if you need them.

The cache is read and refreshed in a single combined pass that also emits SARIF; unchanged classes are served from the unified cache with zero AI calls.

name: Security test scan

on:
  push:
    branches: [main]
  pull_request:

jobs:
  scan:
    runs-on: ubuntu-latest
    permissions:
      security-events: write   # required for upload-sarif
      contents: read

    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-java@v4
        with:
          java-version: '21'
          distribution: 'temurin'

      - name: Download MethodAtlas
        run: |
          curl -fsSL -o methodatlas.jar \
            https://github.com/Accenture/MethodAtlas/releases/latest/download/methodatlas.jar

      - name: MethodAtlas AI cache
        uses: actions/cache@v4        # restores before the run, saves after
        with:
          path: .methodatlas-cache.json.gz
          key: methodatlas-ai-${{ github.ref_name }}-${{ github.sha }}
          restore-keys: |
            methodatlas-ai-${{ github.ref_name }}-
            methodatlas-ai-

      # Single combined pass: classify and emit SARIF, reading and refreshing the
      # unified cache. Only changed classes call the AI provider.
      - name: Run MethodAtlas (classification + SARIF, cached)
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: |
          [ -f .methodatlas-cache.json.gz ] && gunzip -k .methodatlas-cache.json.gz
          CACHE_ARGS=()
          [ -f .methodatlas-cache.json ] && CACHE_ARGS=("-ai-cache" ".methodatlas-cache.json")

          java -jar methodatlas.jar \
            -ai -ai-provider github_models \
            -ai-model gpt-4o-mini \
            -ai-api-key-env GITHUB_TOKEN \
            -content-hash \
            "${CACHE_ARGS[@]}" \
            -ai-cache-out .methodatlas-cache.json \
            -sarif \
            src/test/java \
            > methodatlas.sarif
          gzip -c .methodatlas-cache.json > .methodatlas-cache.json.gz

      - name: Upload SARIF
        if: github.ref == 'refs/heads/main'
        uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: methodatlas.sarif
          category: security-tests

      - name: Upload SARIF as workflow artifact
        if: github.ref == 'refs/heads/main'
        uses: actions/upload-artifact@v4
        with:
          name: methodatlas-sarif
          path: methodatlas.sarif
          retention-days: 30

!!! tip "Switching to a different AI provider" The full workflow above uses github_models (no extra secrets required). To use OpenAI, replace -ai-provider github_models -ai-model gpt-4o-mini -ai-api-key-env GITHUB_TOKEN with -ai-provider openai -ai-api-key-env OPENAI_API_KEY and store OPENAI_API_KEY as a repository secret. All other steps are identical.

Scanning for hard-coded credentials

-detect-secrets adds a deterministic credential scan alongside the test inventory. The deterministic layer needs no AI and makes no network calls, so it is safe to run on every push. Emit the findings as SARIF (credential findings are embedded in the SARIF document under secret/<rule-id> rules) and upload them to Code Scanning under their own category:

      - name: Run MethodAtlas — credential detection (deterministic, no AI)
        run: |
          java -jar methodatlas.jar \
            -detect-secrets \
            -secrets-out methodatlas-credentials.csv \
            -sarif \
            src/test/java \
            > methodatlas-credentials.sarif

      - name: Upload credential-detection SARIF
        uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: methodatlas-credentials.sarif
          category: methodatlas-credentials

      - name: Upload credentials CSV artifact
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: methodatlas-credentials
          path: methodatlas-credentials.csv
          retention-days: 30

Secret values are masked by default in both the SARIF and the CSV. SARIF severity is derived from the credibility score; a deterministic-only run (no AI) emits every finding at warning so nothing drops below review threshold.

!!! note "Optional AI triage" Add -ai -ai-provider github_models -ai-api-key-env GITHUB_TOKEN to the same command to have the model score each candidate's credibility and attribute it to the endpoint it authenticates against. This transmits the test source to the provider, so for sensitive code prefer a local Ollama model — see the AI trust boundary. To narrow or widen the file set, use -secrets-include <glob>, which replaces the default test-class set.

Persisting human corrections with an override file

AI classification is non-deterministic: the same test method may receive different tags or a different security-relevance verdict across runs. An override file locks in human-reviewed decisions so they survive model changes, prompt updates, and re-runs.

Store the file in version control (conventionally .methodatlas-overrides.yaml at the repository root). Pass it on every scan with -override-file:

      - name: Run MethodAtlas with overrides
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: |
          OVERRIDE_ARGS=()
          if [ -f .methodatlas-overrides.yaml ]; then
            OVERRIDE_ARGS=("-override-file" ".methodatlas-overrides.yaml")
          fi

          java -jar methodatlas.jar \
            -ai -ai-provider github_models \
            -ai-model gpt-4o-mini \
            -ai-api-key-env GITHUB_TOKEN \
            -content-hash \
            -sarif \
            "${OVERRIDE_ARGS[@]}" \
            src/test/java \
            > methodatlas.sarif

The file check ([ -f ... ]) lets you commit the workflow before the override file exists. Once the file is committed, subsequent runs pick it up automatically. Pull request diffs on the override file serve as the audit trail for each human classification decision.

See Classification Overrides for the file format reference and Remote Override Sources for a strategy comparison that covers security-team repositories, HTTPS artifact servers, and reusable workflows.