Skip to content

Add ddev validate to ddev tools#23636

Open
luisorofino wants to merge 3 commits intoloa/openmetrics-ai-genfrom
loa/ddev-validate
Open

Add ddev validate to ddev tools#23636
luisorofino wants to merge 3 commits intoloa/openmetrics-ai-genfrom
loa/ddev-validate

Conversation

@luisorofino
Copy link
Copy Markdown
Contributor

@luisorofino luisorofino commented May 8, 2026

What does this PR do?

Adds a new DdevValidateTool shell tool to the AI agent harness at ddev/src/ddev/ai/tools/shell/ddev/validate.py. The tool wraps ddev validate with four subcommands — config, models, and metadata — plus a sync boolean that maps to --sync flag.

Tests covering all subcommand shapes are appended to ddev/tests/ai/tools/shell/ddev/test_ddev_tools.py.

Motivation

The AI agent harness could edit assets/configuration/spec.yaml but had no way to subsequently run ddev validate config --sync or ddev validate models --sync to regenerate the derived files (conf.yaml.example, config_models/) from the updated spec. Without this tool, an agent that edits the spec produces an integration with stale generated files, causing CI to reject the PR.

DdevValidateTool closes this gap and completes the integration authoring lifecycle that the shell tools layer exposes to the agent.

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Add the qa/skip-qa label if the PR doesn't need to be tested during QA.
  • If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

@dd-octo-sts dd-octo-sts Bot added the ddev label May 8, 2026
@luisorofino luisorofino added the qa/skip-qa Automatically skip this PR for the next QA label May 8, 2026
@luisorofino luisorofino changed the title Add ddev validate to ddev tools Add DdevValidateTool to the AI shell tools May 8, 2026
@datadog-official
Copy link
Copy Markdown
Contributor

datadog-official Bot commented May 8, 2026

Tests

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 87.60%

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 29d959a | Docs | Datadog PR Page | Give us feedback!

@codecov
Copy link
Copy Markdown

codecov Bot commented May 8, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (loa/openmetrics-ai-gen@cf0749e). Learn more about missing BASE report.

Additional details and impacted files
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@dd-octo-sts
Copy link
Copy Markdown
Contributor

dd-octo-sts Bot commented May 8, 2026

Validation Report

All 20 validations passed.

Show details
Validation Description Status
agent-reqs Verify check versions match the Agent requirements file
ci Validate CI configuration and Codecov settings
codeowners Validate every integration has a CODEOWNERS entry
config Validate default configuration files against spec.yaml
dep Verify dependency pins are consistent and Agent-compatible
http Validate integrations use the HTTP wrapper correctly
imports Validate check imports do not use deprecated modules
integration-style Validate check code style conventions
jmx-metrics Validate JMX metrics definition files and config
labeler Validate PR labeler config matches integration directories
legacy-signature Validate no integration uses the legacy Agent check signature
license-headers Validate Python files have proper license headers
licenses Validate third-party license attribution list
metadata Validate metadata.csv metric definitions
models Validate configuration data models match spec.yaml
openmetrics Validate OpenMetrics integrations disable the metric limit
package Validate Python package metadata and naming
readmes Validate README files have required sections
saved-views Validate saved view JSON file structure and fields
version Validate version consistency between package and changelog

View full run

@luisorofino luisorofino marked this pull request as ready for review May 8, 2026 14:46
@luisorofino luisorofino requested a review from a team as a code owner May 8, 2026 14:46
@luisorofino luisorofino changed the title Add DdevValidateTool to the AI shell tools Add ddev validate to ddev tools May 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ddev qa/skip-qa Automatically skip this PR for the next QA team/agent-integrations

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant