ci(tikv/tikv): migrate all hub.pingcap.net image references to cloud-accessible registries#4326
Draft
ci(tikv/tikv): migrate all hub.pingcap.net image references to cloud-accessible registries#4326
Conversation
|
Hi @Copilot. Thanks for your PR. I'm waiting for a PingCAP-QE member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
…e registries Co-authored-by: wuhuizuo <2574558+wuhuizuo@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Migrate tikv/tikv images to cloud-accessible registry
ci(tikv/tikv): migrate all hub.pingcap.net image references to cloud-accessible registries
Mar 10, 2026
- Change cluster from gcp-prow-ksyun to default - Update base image to ghcr.io/pingcap-qe/ci/base:v2026.3.8-1-g9d412f4-go1.25 - Remove GO_PROXY, GOMODCACHE, GOCACHE environment variables - Remove persistent volume claims for go cache - Adjust resource requests and limits Close #4237
…net (#4327) * Switches tikv/migration pod templates off `hub.pingcap.net` and onto GHCR. * Removes unused report sidecar from non-Kafka integration pod. ## Image updates | Old (`hub.pingcap.net/jenkins/...`) | New / Action | |---|---| | `centos7_golang-1.21:latest` | `ghcr.io/pingcap-qe/ci/jenkins:v2024.10.8-119-g4e56df7-go1.21` | | `golang-tini:1.21` | `ghcr.io/pingcap-qe/ci/jenkins:v2024.10.8-119-g4e56df7-go1.21` | | `network-multitool` | `wbitt/network-multitool` (already public) | | `python3-requests:latest` | container removed (unused) | ## Files changed - `pipelines/tikv/migration/latest/pod-pull_integration_test.yaml` — golang image -> `ci/jenkins`, drop report sidecar, keep net-tool/utils public - `pipelines/tikv/migration/latest/pod-pull_integration_kafka_test.yaml` — golang image -> `ci/jenkins`, keep net-tool/utils public `ghcr.io/pingcap-qe/ci/jenkins:...-go1.21` carries the same toolchain as the prior base image but defaults to user `jenkins`, matching historical UID/GID expectations in these jobs. <!-- START COPILOT ORIGINAL PROMPT --> <details> <summary>Original prompt</summary> > > ---- > > *This section details on the original issue you should resolve* > > <issue_title>hub registry: migrate images for tikv/migration</issue_title> > <issue_description>Replace hub.pingcap.net image pulls/pushes for tikv/migration with a cloud-accessible registry. > > Acceptance criteria: > - No hub.pingcap.net references in jobs/pipelines for tikv/migration > - Image registry endpoints documented and validated > </issue_description> > > ## Comments on the Issue (you are @copilot in this section) > > <comments> > </comments> > </details> <!-- START COPILOT CODING AGENT SUFFIX --> - Fixes #4227 <!-- START COPILOT CODING AGENT TIPS --> --- 💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more [Copilot coding agent tips](https://gh.io/copilot-coding-agent-tips) in the docs. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: wuhuizuo <2574558+wuhuizuo@users.noreply.github.com> Co-authored-by: wuhuizuo <wuhuizuo@126.com>
…4274) Removes all `fileserver.pingcap.net` references from `pingcap/tidb` pipelines and jobs, replacing binary artifact fetching with OCI pulls via `hub-zot.pingcap.net/mirrors/hub`. ## Core pattern change **Before:** ```groovy environment { FILE_SERVER_URL = 'http://fileserver.pingcap.net' } // ... component.fetchAndExtractArtifact(FILE_SERVER_URL, 'tikv', REFS.base_ref, ...) component.fetchAndExtractArtifact(FILE_SERVER_URL, 'pd', REFS.base_ref, ...) ``` **After:** ```groovy environment { OCI_ARTIFACT_HOST = 'hub-zot.pingcap.net/mirrors/hub' } // ... final OCI_TAG_PD = component.computeArtifactOciTagFromPR('pd', REFS.base_ref, ...) final OCI_TAG_TIKV = component.computeArtifactOciTagFromPR('tikv', REFS.base_ref, ...) // ... container("utils") { dir("bin") { retry(3) { sh label: 'download tidb components', script: """ ${WORKSPACE}/scripts/artifacts/download_pingcap_oci_artifact.sh \ --pd=${OCI_TAG_PD} --tikv=${OCI_TAG_TIKV} """ } } } ``` ## Changes by category - **~60 integration/check2 test pipelines** (release-6.x → release-9.0-beta, latest): replaced `fetchAndExtractArtifact` for tikv/pd with OCI download; added `OCI_TAG_*` declarations; added `utils` sidecar container to pod templates where missing - **~13 build pipelines** (`pull_build`, `ghpr_build`): removed fileserver artifact upload steps and CI-flag curl uploads — builds no longer push to fileserver - **`periodics_integration_test.groovy`**: rewrote to use OCI download, removed sha1-based fileserver tracking - **Tiflash integration tests**: added `OCI_TAG_TIFLASH`; replaced tiflash binary and vector-search test data downloads (now pulled from `ann-benchmarks.com`) - **Sqllogic pod templates**: init containers now use `oras pull` from OCI registry instead of `wget` from fileserver - **`pull_br_integration_test_next_gen/download_tools.sh`**: localstack binary now fetched from GitHub Releases; removed deprecated `tikv-importer` download helper - **`release-6.x/ghpr_unit_test.groovy`**: codecov binary now downloaded from `uploader.codecov.io`; fileserver-based `junitUrl` cleared - **`prow-jobs/pingcap/tidb/latest-periodics.yaml`**: updated stale commented-out codecov URL <!-- START COPILOT ORIGINAL PROMPT --> <details> <summary>Original prompt</summary> > > ---- > > *This section details on the original issue you should resolve* > > <issue_title>fileserver: migrate artifacts for pingcap/tidb</issue_title> > <issue_description>Replace fileserver.pingcap.net usage for pingcap/tidb with OCI artifacts in a standard registry. > > Acceptance criteria: > - No fileserver.pingcap.net references in jobs/pipelines for pingcap/tidb > - Artifacts pulled via OCI registry with documented paths > </issue_description> > > ## Comments on the Issue (you are @copilot in this section) > > <comments> > </comments> > </details> <!-- START COPILOT CODING AGENT SUFFIX --> - Fixes #4210 <!-- START COPILOT CODING AGENT TIPS --> --- 💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more [Copilot coding agent tips](https://gh.io/copilot-coding-agent-tips) in the docs. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: wuhuizuo <2574558+wuhuizuo@users.noreply.github.com> Co-authored-by: wuhuizuo <wuhuizuo@126.com>
Enable native Prow K8s presubmits for `pingcap/tiflow`: - `pull-check` - `pull-build` - `pull-unit-test-cdc` Remove the corresponding groups from Jenkins `ghpr_verify` and move these native jobs out of `latest-presubmits-wip.yaml`.
## What changed - Remove the `Debug info` stage from Jenkins pipelines under `pipelines/`. ## Why - This stage mostly printed environment/network diagnostics and was redundant/noisy across pipelines. ## Testing - Static Jenkinsfile validation: - `JENKINS_URL=https://do.pingcap.net/jenkins .ci/verify-jenkins-pipelines.sh` ## Notes - Replay testing can be triggered in the PR if needed (e.g. `/test pull-replay-jenkins-pipelines`).
…e prow jobs and archive JUnit reports (#4331) - port the changes to release-8.5 from master CI jobs. - archive JUnit reports
This pull request updates the branch matching logic in the Tekton trigger configuration to support additional release branch naming patterns, specifically those that include date and version information. **Branch filtering improvements:** * Updated the regular expression in `git-push-branch-build-community-linux.yaml` to allow release branch names with optional date and version suffixes (e.g., `release-6.1-20240612-v6.1.0`). This makes the trigger compatible with more complex release branch naming conventions.
This pull request adds CI/CD support for the `pingcap-inc/tiflow` repository, specifically targeting the release and feature branches. It introduces new presubmit job configurations, Tekton triggers for tag creation and branch pushes, and updates relevant kustomization files to include these resources. Additionally, it removes a redundant unit test job from the `ticdc` release presubmits. **CI/CD pipeline additions for `pingcap-inc/tiflow`:** * Added `prow-jobs/pingcap-inc/tiflow/release-presubmits.yaml` with presubmit jobs for code checks, builds, unit tests, and various integration tests, supporting both Kubernetes and Jenkins agents. These jobs target `release-*` and a specific feature branch. * Updated `prow-jobs/kustomization.yaml` to include the new `tiflow` release presubmits configuration. **Tekton trigger automation for `tiflow`:** * Added `tekton/v1/triggers/triggers/env-gcp/pingcap-inc/tiflow/git-create-tag.yaml` to trigger builds on tag creation matching semantic versioning in the `tiflow` repo. * Added `tekton/v1/triggers/triggers/env-gcp/pingcap-inc/tiflow/git-push.yaml` to trigger builds on pushes to the `feature/release-8.5.5-active-active` branch in the `tiflow` repo. * Updated `tekton/v1/triggers/triggers/env-gcp/kustomization.yaml` to reference the new `tiflow` triggers. **Cleanup and maintenance:** * Removed the `pull-unit-test-cdc` job from `ticdc` release presubmits to avoid redundancy and streamline the CI pipeline.
…4334) Change cache includes pattern from 'tidb-lightning*' to '**/*' to ensure all built binaries are cached, not just those matching the old pattern.
The release-8.5 branch-specific presubmits have been removed: - pull-check-8.5 renamed to pull-check - pull-unit-test-cdc-8.5 renamed to pull-unit-test-cdc - Removed four Jenkins jobs for release-8.5 branch Ref #4281
## Summary - move the tiflow `pull_cdc_integration_*` change-scope gate from duplicated Jenkins pipeline logic into Prow `skip_if_only_changed` filters across active branches - remove the `Check diff files` stage and related skip plumbing from the affected tiflow CDC integration pipelines - rename the shared Prow anchor to `skip_if_only_changed_non_ticdc_files` so its intent is clearer to contributors ## Validation - run `.ci/update-prow-job-kustomization.sh` - run `JENKINS_URL=https://do.pingcap.net/jenkins .ci/verify-jenkins-pipelines.sh` - run `git diff --check`
See https://do.pingcap.net/jenkins/blue/organizations/jenkins/pingcap%2Ftidb%2Fpull_lightning_integration_test/detail/pull_lightning_integration_test/16624/pipeline/ I checked recent three failed run, all of them lacks several files. I don't know why they are not included in the cache, so this PR extend the files to check. Failed ``` + ls -alh . total 1.5G drwxr-sr-x. 3 jenkins jenkins 4.0K Mar 11 16:17 . drwxr-sr-x. 18 jenkins jenkins 4.0K Mar 11 16:16 .. -rwxr-xr-x. 1 1001 1001 32M Feb 14 22:27 fake-gcs-server -rwxr-xr-x. 1 root jenkins 16M Mar 11 16:17 kes -rwxr-xr-x. 1 root jenkins 19M Mar 11 16:17 mc -rwxr-xr-x. 1 root jenkins 50M Mar 11 16:17 minio -rwxr-xr-x. 1 root root 146M Mar 10 15:16 pd-server -rwxr-xr-x. 1 jenkins jenkins 257M Mar 11 15:45 tidb-lightning-ctl.test -rwxr-xr-x. 1 jenkins jenkins 262M Mar 11 15:45 tidb-lightning.test -rwxr-xr-x. 1 jenkins jenkins 268M Mar 11 15:40 tidb-server lrwxrwxrwx. 1 jenkins jenkins 19 Mar 11 16:17 tiflash -> tiflash_dir/tiflash drwxr-xr-x. 2 root root 4.0K Mar 11 16:17 tiflash_dir -rwxr-xr-x. 1 root root 392M Mar 11 05:40 tikv-server ``` Succeed ``` + ls -alh . total 1.5G drwxr-sr-x. 3 jenkins jenkins 4.0K Mar 11 16:21 . drwxr-sr-x. 18 jenkins jenkins 4.0K Mar 11 16:19 .. -rwxr-xr-x. 1 1001 1001 32M Feb 14 22:27 fake-gcs-server -rwxr-xr-x. 1 jenkins jenkins 7.8M Mar 11 16:19 fake-oauth -rwxr-xr-x. 1 root jenkins 16M Mar 11 16:21 kes -rwxr-xr-x. 1 root jenkins 19M Mar 11 16:20 mc -rwxr-xr-x. 1 root jenkins 50M Mar 11 16:20 minio -rwxr-xr-x. 1 jenkins jenkins 14M Mar 11 16:19 parquet_gen -rwxr-xr-x. 1 root root 146M Mar 10 15:16 pd-server -rwxr-xr-x. 1 jenkins jenkins 258M Mar 11 16:19 tidb-lightning-ctl.test -rwxr-xr-x. 1 jenkins jenkins 262M Mar 11 16:18 tidb-lightning.test -rwxr-xr-x. 1 jenkins jenkins 269M Mar 11 16:13 tidb-server lrwxrwxrwx. 1 jenkins jenkins 19 Mar 11 16:21 tiflash -> tiflash_dir/tiflash drwxr-xr-x. 2 root root 4.0K Mar 11 16:20 tiflash_dir -rwxr-xr-x. 1 root root 392M Mar 11 05:40 tikv-server ``` Signed-off-by: Ruihao Chen <joechenrh@gmail.com>
Part of #4341 Closes #4342 ## Summary - move the `tidb` integration e2e, lightning, and BR jobs from source-repo download scripts to `${WORKSPACE}/scripts/artifacts/download_pingcap_oci_artifact.sh` - compute OCI tags per component in Groovy and keep the BR / lightning auxiliary artifacts and TiFlash layout handling aligned with the OCI helper flow - add the missing `utils` containers to the affected pod templates so the migrated OCI-helper based jobs can run in both merged and release branches ## Testing - `JENKINS_URL=https://do.pingcap.net/jenkins .ci/verify-jenkins-pipeline-file.sh pipelines/pingcap/tidb/release-8.5/pull_integration_e2e_test.groovy` - `JENKINS_URL=https://do.pingcap.net/jenkins .ci/verify-jenkins-pipeline-file.sh pipelines/pingcap/tidb/release-8.2/pull_integration_lightning_test.groovy` - `JENKINS_URL=https://do.pingcap.net/jenkins .ci/verify-jenkins-pipeline-file.sh pipelines/pingcap/tidb/release-8.3/pull_integration_lightning_test.groovy` - `JENKINS_URL=https://do.pingcap.net/jenkins .ci/verify-jenkins-pipeline-file.sh pipelines/pingcap/tidb/release-8.4/pull_integration_lightning_test.groovy` - `JENKINS_URL=https://do.pingcap.net/jenkins .ci/verify-jenkins-pipeline-file.sh pipelines/pingcap/tidb/release-9.0-beta/pull_integration_lightning_test.groovy` - `JENKINS_URL=https://do.pingcap.net/jenkins .ci/verify-jenkins-pipeline-file.sh pipelines/pingcap/tidb/latest/merged_integration_lightning_test.groovy` - `JENKINS_URL=https://do.pingcap.net/jenkins .ci/verify-jenkins-pipeline-file.sh pipelines/pingcap/tidb/latest/merged_integration_br_test.groovy` - `JENKINS_URL=https://do.pingcap.net/jenkins .ci/verify-jenkins-pipeline-file.sh pipelines/pingcap/tidb/release-7.5/pull_br_integration_test.groovy` - `JENKINS_URL=https://do.pingcap.net/jenkins .ci/verify-jenkins-pipeline-file.sh pipelines/pingcap/tidb/release-7.5/periodics_br_integration_test.groovy` - `JENKINS_URL=https://do.pingcap.net/jenkins .ci/verify-jenkins-pipeline-file.sh pipelines/pingcap/tidb/release-8.1/periodics_br_integration_test.groovy` - `JENKINS_URL=https://do.pingcap.net/jenkins .ci/verify-jenkins-pipeline-file.sh pipelines/pingcap/tidb/release-8.2/pull_integration_br_test.groovy` - `JENKINS_URL=https://do.pingcap.net/jenkins .ci/verify-jenkins-pipeline-file.sh pipelines/pingcap/tidb/release-8.3/pull_integration_br_test.groovy` - `JENKINS_URL=https://do.pingcap.net/jenkins .ci/verify-jenkins-pipeline-file.sh pipelines/pingcap/tidb/release-8.4/pull_integration_br_test.groovy` - `JENKINS_URL=https://do.pingcap.net/jenkins .ci/verify-jenkins-pipeline-file.sh pipelines/pingcap/tidb/release-8.5/pull_integration_br_test.groovy` - `JENKINS_URL=https://do.pingcap.net/jenkins .ci/verify-jenkins-pipeline-file.sh pipelines/pingcap/tidb/release-9.0-beta/pull_integration_br_test.groovy` - `ruby -e 'require \"yaml\"; YAML.load_file(ARGV[0])' pipelines/pingcap/tidb/latest/pod-merged_integration_lightning_test.yaml` - `ruby -e 'require \"yaml\"; YAML.load_file(ARGV[0])' pipelines/pingcap/tidb/release-8.2/pod-pull_integration_lightning_test.yaml` - `ruby -e 'require \"yaml\"; YAML.load_file(ARGV[0])' pipelines/pingcap/tidb/release-8.3/pod-pull_integration_lightning_test.yaml` - `ruby -e 'require \"yaml\"; YAML.load_file(ARGV[0])' pipelines/pingcap/tidb/release-8.4/pod-pull_integration_lightning_test.yaml` - `ruby -e 'require \"yaml\"; YAML.load_file(ARGV[0])' pipelines/pingcap/tidb/release-8.5/pod-pull_integration_lightning_test.yaml` - `ruby -e 'require \"yaml\"; YAML.load_file(ARGV[0])' pipelines/pingcap/tidb/latest/pod-merged_integration_br_test.yaml` - `ruby -e 'require \"yaml\"; YAML.load_file(ARGV[0])' pipelines/pingcap/tidb/release-8.1/pod-pull_br_integration_test.yaml` - `ruby -e 'require \"yaml\"; YAML.load_file(ARGV[0])' pipelines/pingcap/tidb/release-8.2/pod-pull_integration_br_test.yaml` - `ruby -e 'require \"yaml\"; YAML.load_file(ARGV[0])' pipelines/pingcap/tidb/release-8.3/pod-pull_integration_br_test.yaml` - `ruby -e 'require \"yaml\"; YAML.load_file(ARGV[0])' pipelines/pingcap/tidb/release-8.4/pod-pull_integration_br_test.yaml` - `ruby -e 'require \"yaml\"; YAML.load_file(ARGV[0])' pipelines/pingcap/tidb/release-8.5/pod-pull_integration_br_test.yaml`
Add prepare_tiflash_layout() to scripts/artifacts/download_pingcap_oci_artifact.sh and remove ad-hoc tiflash flattening from multiple pipelines and helper scripts. Normalize TiFlash binary layout to tiflash_dir/tiflash and create a symlink at tiflash.
## Summary - replace deprecated `charts.pingcap.org` references in the TiDB Operator release pipeline - keep the existing old download-domain compatibility rewrite logic intact ## Validation - `git diff --check` - verified no remaining `charts.pingcap.org` matches in the touched file
migrate tidb postsubmit jobs to gcp
This PR contains the following updates: | Package | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) | |---|---|---|---| | [github.com/rs/zerolog](https://redirect.github.com/rs/zerolog) | `v1.34.0` → `v1.35.0` |  |  | --- > [!WARNING] > Some dependencies could not be looked up. Check the [Dependency Dashboard](../issues/1780) for more information. --- ### Release Notes <details> <summary>rs/zerolog (github.com/rs/zerolog)</summary> ### [`v1.35.0`](https://redirect.github.com/rs/zerolog/compare/v1.34.0...v1.35.0) [Compare Source](https://redirect.github.com/rs/zerolog/compare/v1.34.0...v1.35.0) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/PingCAP-QE/ci). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4xMDAuMCIsInVwZGF0ZWRJblZlciI6IjQzLjEwMC4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
## Summary - add `oras` to the required tool checks in `scripts/ops/nextgen/get-next-gen-exact-image-tags.sh` - authenticate against both `us.gcr.io` and `gcr.io` because tiproxy trunk still resolves from `gcr.io` while the other next-gen artifacts use `us.gcr.io` - update the nextgen README so the prerequisites, tiproxy dual-registry behavior, and troubleshooting guidance match the script ## Why a follow-up PR `#4446` merged before this extra hardening pass was pushed, so this change is submitted as a separate follow-up PR. ## Validation - `bash -n scripts/ops/nextgen/get-next-gen-exact-image-tags.sh` - `PATH="$PWD/../.cache/bin:$PATH" shellcheck -S error scripts/ops/nextgen/get-next-gen-exact-image-tags.sh` - `git diff --check` ## Notes - Live registry execution is still not exercised in this workspace because `gcloud` is unavailable here. --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
migrate tidb presubmit jobs to gcp
This pull request updates how the container registry is selected for Tekton pipeline triggers, making the selection dynamic based on the event type. Instead of hardcoding the registry, the configuration now determines whether to use the hotfix or dev registry by inspecting the event header. This improves flexibility and ensures that hotfix builds are pushed to the correct registry. **Dynamic registry selection based on event type:** * Added a `registry` parameter to multiple trigger files (PR, branch push, and tag creation), which uses an expression to select the hotfix or dev registry depending on whether the event type starts with `net.pingcap.tibuild.hotfix.` (`tekton/v1/triggers/triggers/env-prod2/_/fake-github/fake-github-branch-push.yaml`, `fake-github-branch-push-single-platform.yaml`, `fake-github-pr.yaml`, `fake-github-pr-single-platform.yaml`, `fake-github-tag-create.yaml`, `fake-github-tag-create-single-platform.yaml`) [[1]](diffhunk://#diff-701e172d7957f7729eb568b89eb3792f67986d0be3d087c888ee3f11ce921b0cR148-R152) [[2]](diffhunk://#diff-6a91abaebf470476a3ede56259ce3a9aea4c03d6e7a551bfa00b5a38fa9ef712R150-R155) [[3]](diffhunk://#diff-f69e7eca2ab1ea5e98e43f6fee3bfb5c896d32ba6e6b4b69d81664ee422a8904R148-R153) [[4]](diffhunk://#diff-9d850ebf70c39ddd5babcf9dc7fc5fe058ab9f61ead3f04e6a5fb103974f6591R150-R155) [[5]](diffhunk://#diff-e1424e05dfaa1f85fe71d2e0365337c7fcad48767eead843e042e248d0dcf1a0R148-R153) [[6]](diffhunk://#diff-c613d97eae5f67f2326053c71fca1985b8b8e7a3bfb2c60a0be9aac9407cf838R152-R157) **Parameterization of build registry:** * Updated the `ksy-dev-build-params.yaml` binding to use the dynamically determined `registry` value from the trigger extensions, instead of a hardcoded dev registry value (`tekton/v1/triggers/bindings/ksy-dev-build-params.yaml`)
## Summary
- remove global `GITHUB_TOKEN = credentials('github-bot-token')`
injection from `pull_*` pipelines that execute untrusted PR code
- keep non-sensitive `environment` entries such as `OCI_ARTIFACT_HOST`
unchanged
- limit the scope to PR-triggered pipelines; merged pipelines are not
changed in this patch
## Risk
These PR jobs were exposing a GitHub bot token to the full Jenkins
execution environment even though the variable was not consumed later in
the pipeline. A malicious PR could potentially read the token from the
process environment.
## Verification
- `git diff --check`
- verified no `pull_*.groovy` file still contains `GITHUB_TOKEN =
credentials(...)` or `GH_TOKEN = credentials(...)`
- verified no empty `environment {}` blocks remain after the cleanup
This PR contains the following updates: | Package | Update | Change | |---|---|---| | alpine | minor | `3.22` → `3.23` | --- > [!WARNING] > Some dependencies could not be looked up. Check the [Dependency Dashboard](../issues/1780) for more information. --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/PingCAP-QE/ci). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4xMDIuMTEiLCJ1cGRhdGVkSW5WZXIiOiI0My4xMDIuMTEiLCJ0YXJnZXRCcmFuY2giOiJtYWluIiwibGFiZWxzIjpbXX0=--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
## Summary - redo the stale removal from #4059 on current `main` - delete the TiDB periodic integration Jenkins job, pipeline, and pod template - remove the matching prow periodic job definition from `prow-jobs/pingcap/tidb/latest-periodics.yaml` ## Testing - `rg -n "pingcap/tidb/periodics_integration_test|periodics_integration_test" prow-jobs jobs pipelines docs .ci` - `PATH="$(mktemp -d)":$PATH` with temporary `yq` binary to run `.ci/update-prow-job-kustomization.sh` - `git diff --check` ```release-note none ```
pull-tiflash-integration-test pull_integration_common_test never run
This pull request makes a small but important update to the `sendTestCaseRunReport` function in `libraries/tipipeline/vars/prow.groovy`. The change updates the URL used to send test case run reports to the new cloudevents server endpoint. - Changed the cloudevents server URL in the `curl` command from `https://internal2-do.pingcap.net/cloudevents-server/events` to `https://cloudevents-server.cs.svc/events` to use the new endpoint.
enlarge ghpr_unit_test pod spec --------- Co-authored-by: wuhuizuo <wuhuizuo@126.com>
This pull request introduces a new content policy check for pull requests, updates Jenkins URLs to a new domain, and improves container image management and security in CI pipelines. The major changes include adding a script and Prow job to enforce PR content rules, updating Jenkins references from the old to the new domain, enhancing Renovate configuration to update container images in YAML files, and upgrading the `flux-cli` container image for better security and features. **Pull Request Content Policy Enforcement:** * Added `.ci/check-pr-content-policy.sh`, a script to check added lines in pull requests for forbidden substrings and unauthorized `pingcap.net` hosts, with reporting and usage instructions. * Integrated the new content policy check as a required presubmit job `pull-verify-pr-content-policy` in `prow-jobs/pingcap-qe/ci/presubmits.yaml`, ensuring all PRs to `main` are validated. **Jenkins URL Migration:** * Updated all references to the Jenkins root URL from `https://do.pingcap.net/jenkins` to `https://prow.tidb.net/jenkins` in scripts, documentation, and usage examples (`.ci/replay-jenkins-build.sh`, `.agents/skills/test-jenkins-pipeline-changes-in-pr-by-replaying/SKILL.md`). [[1]](diffhunk://#diff-969a7a1cc3ceb372eef89a32e75e315481bda7de329d354bc535543e36fe2152L126-R126) [[2]](diffhunk://#diff-969a7a1cc3ceb372eef89a32e75e315481bda7de329d354bc535543e36fe2152L157-R157) [[3]](diffhunk://#diff-969a7a1cc3ceb372eef89a32e75e315481bda7de329d354bc535543e36fe2152L182-R182) [[4]](diffhunk://#diff-e4ae88c1b62fa5819dc1406b2a6267de3e218438d35f461755a1744f62b1b7c6L26-R26) [[5]](diffhunk://#diff-e4ae88c1b62fa5819dc1406b2a6267de3e218438d35f461755a1744f62b1b7c6L544-R544) **Container Image Management Improvements:** * Enhanced `.github/renovate.json` with a custom manager to automatically update container images referenced in `pipelines/` and `prow-jobs/` YAML files. **CI Pipeline Security and Maintenance:** * Upgraded `flux-cli` container images in multiple Prow job configurations to version `v2.2.3` for improved security and features (`prow-jobs/pingcap-qe/ci/presubmits.yaml`, `prow-jobs/ti-community-infra/configs/presubmits.yaml`). [[1]](diffhunk://#diff-9db5df27906664c96b24883b91f2527720f4898654f108244cda91118a5e9d4aL115-R134) [[2]](diffhunk://#diff-91e31271871087bd1f75ae2bd75142333bc1f221adbab3334950d34899af89f0L84-R84) [[3]](diffhunk://#diff-91e31271871087bd1f75ae2bd75142333bc1f221adbab3334950d34899af89f0L112-R112) The job is short-term for migration, it will be deprecated in future.
6 jobs replay pass： pull_build_next_gen | https://prow.tidb.net/jenkins/job/pingcap/job/tidb/job/pull_build_next_gen/5/ | SUCCESS | 2026-04-06 22:08:45 -- | -- | -- | -- pull_integration_e2e_test_next_gen | https://prow.tidb.net/jenkins/job/pingcap/job/tidb/job/pull_integration_e2e_test_next_gen/7/ | SUCCESS | 2026-04-07 11:21:49 pull_integration_realcluster_test_next_gen | https://prow.tidb.net/jenkins/job/pingcap/job/tidb/job/pull_integration_realcluster_test_next_gen/4/ | SUCCESS | 2026-04-06 21:51:57 pull_mysql_client_test_next_gen | https://prow.tidb.net/jenkins/job/pingcap/job/tidb/job/pull_mysql_client_test_next_gen/2/ | SUCCESS | 2026-04-07 11:16:47 pull_unit_test_next_gen | https://prow.tidb.net/jenkins/job/pingcap/job/tidb/job/pull_unit_test_next_gen/11/| SUCCESS | 2026-04-07 09:05:52 pull_mysql_test_next_gen | https://prow.tidb.net/jenkins/job/pingcap/job/tidb/job/pull_mysql_test_next_gen/11/| FAILURE (optional: true, same err to IDC)| 2026-04-07 14:08:32
Contributor
|
@copilot the pr has conflicts, please do it again from zero. |
…istries - release-8.4: hub.pingcap.net/jenkins/tikv-ci:rocky8-base-cached-release-8.4 -> ghcr.io/pingcap-qe/ci/jenkins/tikv:v2026.3.22-4-gd9d1135 - release-6.1 through release-8.3: hub.pingcap.net/jenkins/tikv-cached-* -> ghcr.io/pingcap-qe/cd/builders/tikv:v2025.12.14-1-g33e22ac-centos7-devtoolset8 - integration tests (all): hub.pingcap.net/jenkins/tikv-cached-*/centos7_golang-* -> ghcr.io/pingcap-qe/ci/jenkins/tikv:* / golang:* - release-6.1: remove hub.pingcap.net/jenkins/ks3util sidecar; refactor pull_unit_test.groovy to run test chunks sequentially in same pod Co-authored-by: wuhuizuo <2574558+wuhuizuo@users.noreply.github.com>
Co-authored-by: wuhuizuo <2574558+wuhuizuo@users.noreply.github.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Contributor
Author
Rebased from scratch on current main (commit |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Removes all
hub.pingcap.netimage references fromtikv/tikvpipelines, replacing them with publicly accessible registry images on GHCR and Docker Hub.Image mappings
hub.pingcap.net/jenkins/tikv-ci:rocky8-base-cached-release-8.4ghcr.io/pingcap-qe/ci/jenkins/tikv:v2026.3.22-4-gd9d1135hub.pingcap.net/jenkins/tikv-cached-release-X.Y:latestghcr.io/pingcap-qe/cd/builders/tikv:v2025.12.14-1-g33e22ac-centos7-devtoolset8hub.pingcap.net/jenkins/tikv-cached-master:latestghcr.io/pingcap-qe/ci/jenkins/tikv:v2026.3.22-4-gd9d1135hub.pingcap.net/jenkins/centos7_golang-{1.19,1.20,1.21}:latestgolang:{1.19,1.20,1.21}hub.pingcap.net/jenkins/ks3utilrelease-6.1 pipeline refactor
The
ks3utilcontainer has no public equivalent; it was used to shuttle test artifacts via KingCloud S3 between a build pod and 20 parallel matrix test agents. Since release-6.1 is WIP/debug-only (optional: true,always_run: false, triggered via/debug), the pipeline was simplified:/home/jenkins/archives/instead of being uploaded to KS3Original prompt
🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.