Skip to content

Security: NVIDIA/aicr

SECURITY.md

Security

NVIDIA is dedicated to the security and trust of our software products and services, including all source code repositories.

Please do not report security vulnerabilities through GitHub.

Reporting Security Vulnerabilities

To report a potential security vulnerability in any NVIDIA product:

Include in your report:

  • Product/Driver name and version
  • Type of vulnerability (code execution, denial of service, buffer overflow, etc.)
  • Steps to reproduce
  • Proof-of-concept or exploit code
  • Potential impact and exploitation method

NVIDIA offers acknowledgement for externally reported security issues under our coordinated vulnerability disclosure policy. Visit PSIRT Policies for details.

Product Security Resources

For all security-related concerns: https://www.nvidia.com/en-us/security

Supply Chain Security

AICR (AI Cluster Runtime) provides supply chain security artifacts for all container images:

  • SBOM Attestation: Complete inventory of packages, libraries, and components in SPDX format
  • SLSA Build Provenance: Verifiable build information (how and where images were created)

Deployed Image Inventory and Pinning Policy

Beyond AICR's own runtime images, every cluster deployed by AICR pulls a set of upstream container images selected by the recipe and the Helm charts it references. The complete list — chart-by-chart and image-by-image — is published as a versioned doc artifact and refreshed weekly as upstream charts evolve:

  • docs/user/container-images.md — human readable summary of every component, chart, and image AICR can deploy.
  • A machine-readable CycloneDX 1.6 BOM is produced by make bom and published as a release asset. Tooling that consumes SBOMs (Trivy, Grype, Cosign attestation, in-toto) should prefer the JSON; the Markdown is its companion.

Pinning policy (ADR-006) defines AICR's three-layer contract:

  1. Chart versions are pinned for every Helm component, with no exceptions. recipes/registry.yaml MUST declare defaultVersion; make bom BOM_STRICT=1 enforces this in CI.
  2. Explicit image overrides that AICR pins in-tree (in recipes/components/<name>/values.yaml or embedded manifests) carry @sha256: digests in addition to tags. Renovate handles digest rotation as upstream rebuilds the same tag.
  3. Chart-default sub-images (the ones the upstream chart pulls without AICR overriding) ship at whatever the chart resolves at the pinned chart version. Reproducibility for these images is delivered by admission-time digest verification rather than per-sub-image overrides (see the supply-chain epic for the roadmap).

Upstream attestation coverage. AICR's own runtime images and the NVSentinel images deployed by AICR ship with full keyless cosign signatures, SLSA build provenance, and SBOM attestations verifiable from the public Sigstore Rekor transparency log. Other NVIDIA-owned images that AICR deploys today (gpu-operator, network-operator, k8s-dra-driver-gpu, nodewright/skyhook) ship legacy key-based cosign signatures but do not yet ship keyless signatures, SLSA provenance, or SBOM attestations. Issues requesting parity have been filed with each upstream and are tracked under the supply-chain epic (Stage 3).

Trust Guarantees

All AICR artifacts published from tagged releases carry cryptographically verifiable supply chain guarantees. The table below summarizes what exists and how it is signed:

Guarantee Artifact Format / Standard Signing
Build provenance Container images SLSA Build Level 3 (provenance v1) Sigstore keyless via GitHub Actions OIDC
Build provenance CLI binaries SLSA Build Provenance v1 (Sigstore bundle) Cosign keyless, logged to public Rekor
Signed SBOM Container images SPDX v2.3 JSON attestation Cosign keyless (Fulcio + Rekor)
Embedded SBOM CLI binaries SPDX v2.3 JSON (via GoReleaser) Released alongside attested binary
Bundle attestation aicr bundle output SLSA Build Provenance v1 Sigstore keyless OIDC (opt-in --attest)
Recipe / bundle validity aicr verify trust levels verified / attested / unverified / unknown Checksums + Sigstore trusted root

aicr verify reports one of four trust levels for a bundle: verified (full chain verified, binary identity pinned to NVIDIA CI), attested (chain verified but binary attestation missing or external data used), unverified (checksums valid, --attest was not used), and unknown (missing checksums or attestation files).

Verify an Artifact (Happy Path)

Verify the latest released CLI image against its immutable digest:

# Resolve the latest release tag to an immutable digest
export TAG=$(curl -s https://api.github.com/repos/NVIDIA/aicr/releases/latest | jq -r '.tag_name')
export IMAGE="ghcr.io/nvidia/aicr"
export DIGEST=$(crane digest "${IMAGE}:${TAG}")

# Verify build provenance and SBOM attestations
gh attestation verify "oci://${IMAGE}@${DIGEST}" --owner nvidia
# ✓ Verification succeeded!
#   • Build provenance (SLSA v1.0)
#   • SBOM (SPDX)

Verify a generated deployment bundle, enforcing the highest trust level:

aicr verify ./my-bundle --min-trust-level verified

Deep-Dive: Verifying Artifacts

For the full operational reference — resolving digests, inspecting SLSA provenance and SBOM contents, verifying CLI binary and bundle attestations, enforcing verification in clusters with Kyverno or the Sigstore Policy Controller, and offline/air-gapped verification with a local trusted root — see Supply Chain Verification.

For full CLI flag documentation, see the CLI Reference (aicr verify, aicr bundle --attest, aicr trust update). For a hands-on walkthrough, see the Bundle Attestation Demo.

There aren't any published security advisories