Add yaml-mapper integration tests by fanny-jiang · Pull Request #2143 · DataDog/helm-charts

fanny-jiang · 2025-11-06T14:24:09Z

What this PR does / why we need it:

Adds integration tests for the yaml-mapper binary, testing the mapping of various values.yaml configurations supported by the yaml-mapper to DDA and validating successful agent installation using the mapped DDA in kind cluster.

Validations:

Each test values.yaml file installs a healthy agent workload using the datadog helm chart
Each test values.yaml maps to DDA custom resource that successfully installs a healthy agent workload
The agent runtime config produced by the mapped DDA agents is equivalent to that produced by the source helm values.yaml** (with minor exceptions, see below)

Validated Helm configs:

Minimal agent installation (API/APP key credentials and k8s secrets)
Cluster and site settings (clusterName, site, dd_url endpoint, registry)
Datadog tags and env variables
Name overrides (nameOverride/fullnameOverride)
Minimal kubelet configuration
Container runtime socket paths (CRI, Docker)
Minimal NetworkPolicy configs
Origin detection unified
APM (hostPort and UDS mode, minimal instrumentation and error tracking)
Logs collection (minimal - missing include/exclude filters)
Process monitoring
DogStatsD
Cluster checks and cluster checks runner
Cluster agent overrides
Admission Controller (including sidecar injection)
Kubernetes events collection
Kubernetes State Metrics core check (minimal)
Orchestrator Explorer (minimal)
Helm check
Prometheus scrape (partial)
Remote Configuration
Pod-level overrides (resources, update strategy, priorityClassName, hostNetwork, hostPID)

Not tested at this time:

system-probe related features (not supported in kind)
- NPM
- OOMKill check
- enableTCPQueueLength

Agent Config diff exceptions:

Orchestrator explorer kubelet config check is not yet supported in the Operator, so it becomes disabled
Other diffs are resolved in the chart via this PR: Conditionally set env vars to match datadog-operator: logs, prometheus #2312

Which issue this PR fixes

(optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged)

fixes #

Special notes for your reviewer:

Install the required DatadogAgent and DatadogAgentInternal CRDs in your test kind-cluster:

make setup-mapper-crds

or

helm install datadog-crds datadog/datadog-crds --set crds.datadogAgents=true --set crds.datadogAgentInternals=true

Run make integ-test-mapper

If you exit the test early and wish to re-run the tests, you can prepend the command with YAMLMAPPER_CLEANUP_STALE=true to clean up resources from previous test runs

Checklist

[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

Chart Version semver bump label added (use <chartName>/minor-version, <chartName>/patch-version, or <chartName>/no-version-bump)
For datadog or datadog-operator chart or value changes, update the test baselines (run: make update-test-baselines)

GitHub CI takes care of the below, but are still required:

Documentation has been updated with helm-docs (run: .github/helm-docs.sh)
CHANGELOG.md has been updated
Variables are documented in the README.md

fanny-jiang · 2026-01-23T17:14:52Z

+          kubectl get nodes
+          helm dependency build ./charts/datadog
+          helm dependency build ./charts/datadog-operator
+          make integ-test-mapper


Suggested change

make integ-test-mapper

make integ-test-mapper-strict

Use strict mode to require equal agent config between helm and dda agent deployments when this PR is merged

fanny-jiang · 2026-03-10T16:40:00Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0341677d2c

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

- update readme for datadog

- Add t.Helper() to validateExpectedPodsAgainstValues, validateExpectedPodCount, and assertBaseValuesCoverage for correct failure attribution - Fix assertBaseValuesCoverage to also check the inverse: fail if any values file on disk has no matching test case in baseTestCases - Fix applyDDAAndWaitForAgents to wait for ClusterAgent and CCR pods in addition to DaemonSet agent pods; add ExpectedComponentPods parameter - Use specific ExpectedErrMsg "mapping completed with" in negative test cases instead of the generic "error" - Equalize getHelmAgentConf and getOperatorAgentConf to both use defaultPodRetries - Handle []interface{} in normalizeDataWithPath so nested booleans inside list-valued config fields are normalized correctly - Remove unused fmt import from integ_test.go - Remove redundant test/datadog/yamlmapper/** path from workflow triggers - Fix typo in createTestResources comment Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

The yaml-mapper tests validate mapper transformation logic and agent config equivalence — neither of which is k8s-version-sensitive. Cross-version compatibility for the Helm chart and operator is already covered by go-test-operator.yaml (11 versions) and ci.yaml (kubeconform). Running 3x the jobs provided no meaningful additional coverage. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Now that there is a single k8s version, replace the matrix with hardcoded values directly in the step definitions. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

The inverse check was failing because it only looked at baseTestCases, but the values directory also contains files referenced by testCasesWithDependencies, negativeTestCases, and TestInvalidYAMLChartInstall. Fix by building a combined set of all referenced filenames across all test slices before checking for uncovered files on disk. Extract invalidYAMLValuesFile as a constant so the hardcoded reference in TestInvalidYAMLChartInstall is included in the coverage set. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

- update readme for datadog

P1: Point mapper tests at the canonical mapping file in charts/datadog/files/ instead of maintaining a stale copy in the test directory. Delete the duplicate. P2: Replace local charts/datadog-operator path with the remote datadog/datadog-operator chart reference, reading the pinned version at test time from charts/datadog/requirements.yaml. This ensures CI always tests against the same operator version the chart ships. Remove the now- unnecessary helm dependency build for charts/datadog-operator from CI. Add Remote/Version fields to common.HelmCommand to support remote chart references in InstallChart without corrupting the chart path via filepath.Abs. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

GetFullValues was added in #2068 alongside the original mapper integration tests, but was never called after those tests were reverted and rewritten. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

- mapper.yaml.disabled: fix mapping file path, add description comment, add mapping file to trigger paths - utils.go: add license header, rename context var to avoid shadowing package, extract isEnvTruthy helper, add forceDeleteThreshold constant - helpers.go: getMappingPath logs warning on Abs failure, add duplicate checks for all test case slices - integ_test.go: remove redundant kubectlOptions param from installAndVerifyOperatorAgent - negative_test.go: use case-sensitive error assertion, remove unused strings import - common.go: require.NoError instead of require.Nil in CurrentContext Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

fanny-jiang · 2026-03-10T19:37:10Z

@codex review

chatgpt-codex-connector · 2026-03-10T19:45:18Z

Codex Review: Didn't find any major issues. Can't wait for the next one!

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Add test/common/** to catch changes to shared test utilities used by the yamlmapper integration tests. Keep test/datadog/** and charts/datadog/** broad to ensure regressions from any chart or test change are caught. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Document prerequisites, make targets, environment variables, and cleanup steps for running the YAML mapper integration tests locally. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

fanny-jiang force-pushed the fanny/AGENTONB-2670/migration-mapper-tests branch 2 times, most recently from e39cf35 to 54c555a Compare November 6, 2025 14:42

fanny-jiang added chart/datadog This issue or pull request is related to the datadog chart datadog/patch-version Patch version bump for datadog chart labels Nov 6, 2025

github-actions Bot added the tools/ci label Nov 6, 2025

fanny-jiang force-pushed the fanny/AGENTONB-2670/migration-mapper-tests branch from d6a23c8 to 0450509 Compare November 7, 2025 01:04

fanny-jiang marked this pull request as ready for review November 7, 2025 18:14

fanny-jiang requested review from a team as code owners November 7, 2025 18:14

fanny-jiang added datadog/no-version-bump Skip version bump for datadog chart and removed datadog/patch-version Patch version bump for datadog chart labels Jan 22, 2026

fanny-jiang force-pushed the fanny/AGENTONB-2670/migration-mapper-tests branch 2 times, most recently from b463f70 to 62cc8a0 Compare January 23, 2026 17:10

fanny-jiang commented Jan 23, 2026

View reviewed changes

fanny-jiang force-pushed the fanny/AGENTONB-2670/migration-mapper-tests branch from 6b485e8 to b32025a Compare January 28, 2026 20:41

chatgpt-codex-connector Bot reviewed Mar 10, 2026

View reviewed changes

Comment thread test/datadog/yamlmapper/mapping_datadog_helm_to_datadogagent_crd.yaml Outdated

Comment thread .github/workflows/go-test-yamlmapper.yaml

fanny-jiang added 13 commits March 10, 2026 14:15

squash: yaml-mapper history (pre integ tests)

45096cd

update integ tests and add test values

bb475aa

add prometheus helm repo

24d12b4

update workflow

a53f4c8

try gotestsum, enable all tests

d18c3b3

use go test

4a65138

trigger ci

7276ff7

enable verbose

268b710

update tests, increase job timeout

41a7a55

test cleanups

581202c

wip

fcf4833

refactor

e6f6f13

use consts

d5ff8d1

fanny-jiang and others added 16 commits March 10, 2026 14:15

cleanup and refactor

a14f7b8

reorg test values and add dda baselines

c21c439

remove test namespace from dda baselines

8dee13f

update go version

96474c6

skip strip_process_args in agent config

779c066

hardcode agent image tags and update test filenames

fe0bb41

update mapper baselines

21a463f

chore: update charts

8427027

- update readme for datadog

[go-test-yamlmapper] remove matrix syntax

e085e3a

Now that there is a single k8s version, replace the matrix with hardcoded values directly in the step definitions. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

chore: update charts

7374932

- update readme for datadog

enable strict

8828ca1

Remove unused GetFullValues from common.go

2d6c741

GetFullValues was added in #2068 alongside the original mapper integration tests, but was never called after those tests were reverted and rewritten. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

fanny-jiang force-pushed the fanny/AGENTONB-2670/migration-mapper-tests branch from 766a053 to 2d6c741 Compare March 10, 2026 18:21

fanny-jiang and others added 2 commits March 10, 2026 16:09

Add yamlmapper integration test docs to CONTRIBUTING.md

0501dbe

Document prerequisites, make targets, environment variables, and cleanup steps for running the YAML mapper integration tests locally. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

github-actions Bot added the documentation Improvements or additions to documentation label Mar 10, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add yaml-mapper integration tests#2143

Add yaml-mapper integration tests#2143
fanny-jiang wants to merge 32 commits intomainfrom
fanny/AGENTONB-2670/migration-mapper-tests

fanny-jiang commented Nov 6, 2025 •

edited

Loading

Uh oh!

fanny-jiang Jan 23, 2026

Uh oh!

fanny-jiang commented Mar 10, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

Uh oh!

fanny-jiang commented Mar 10, 2026

Uh oh!

chatgpt-codex-connector Bot commented Mar 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

fanny-jiang commented Nov 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does / why we need it:

Which issue this PR fixes

Special notes for your reviewer:

Checklist

Uh oh!

fanny-jiang Jan 23, 2026

Choose a reason for hiding this comment

Uh oh!

fanny-jiang commented Mar 10, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

fanny-jiang commented Mar 10, 2026

Uh oh!

chatgpt-codex-connector Bot commented Mar 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

fanny-jiang commented Nov 6, 2025 •

edited

Loading