tiflow: add MariaDB source and next-gen TiDB CI for DM integration test#4485
tiflow: add MariaDB source and next-gen TiDB CI for DM integration test#4485joechenrh wants to merge 3 commits intoPingCAP-QE:mainfrom
Conversation
|
Skipping CI for Draft Pull Request. |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request adds a new MariaDB container to the DM integration test pipeline, including the necessary configuration for the container in the YAML file and the corresponding environment variables and readiness checks in the Groovy pipeline script. I recommend explicitly defining resource requests for the new container to ensure consistent resource allocation and optimal thread utilization in the CI environment.
| resources: | ||
| limits: | ||
| memory: 4Gi | ||
| cpu: "2" |
There was a problem hiding this comment.
The mariadb1 container specifies resource limits but lacks requests. In Kubernetes, requests default to limits if omitted. Explicit requests should be defined so that resource-dependent parameters, like thread counts, can be aligned with the requested CPU cores to maximize resource utilization.
References
- In CI job configurations, favor simplicity by hardcoding resource-dependent parameters like thread counts. A value aligned with the resource request is often sufficient and preferable to a complex, dynamic configuration tied to the resource limit.
- To maximize resource utilization in CI jobs, the number of test threads should match the number of requested CPU cores.
Add integration test and compatibility test CI jobs that run DM tests against next-gen TiDB binaries downloaded from OCI registry instead of the classic fileserver. Jobs are optional and can be triggered via `/test dm-next-gen` or `/test pull-dm-integration-test-next-gen`. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add mariadb1 container and env vars to the next-gen DM integration test pipeline, consistent with the classic DM CI changes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
joechenrh
changed the title
1 participant
Summary
dm/tests/mariadb2tidbcasehub-zot.pingcap.net/mirrors/tidbx) withmaster-next-gen/dedicated-next-gentagsoptional: trueinitially, triggered via/test dm-next-genFiles changed (classic DM CI)
pipelines/.../pod-pull_dm_integration_test.yamlmariadb1containerpipelines/.../pull_dm_integration_test.groovyFiles added (next-gen DM CI)
prow-jobs/.../latest-presubmits-next-gen.yamljobs/.../pull_dm_integration_test_next_gen.groovyjobs/.../pull_dm_compatibility_test_next_gen.groovypipelines/.../pull_dm_integration_test_next_gen.groovypipelines/.../pull_dm_compatibility_test_next_gen.groovypipelines/.../pod-pull_dm_integration_test_next_gen.yamlpipelines/.../pod-pull_dm_compatibility_test_next_gen.yamlNote
Test plan
/test pull-dm-integration-teston a tiflow PR to validate MariaDB changes/test pull-dm-integration-test-next-genon a tiflow PR to validate next-gen pipeline/test pull-dm-compatibility-test-next-genon a tiflow PR to validate next-gen pipeline