Skip to content

ARO-HCP Extract aro-hcp-tests from aro-hcp-e2e-tools #75802

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
patriksuba wants to merge 8 commits into
base: main
Choose a base branch
from
Open

ARO-HCP Extract aro-hcp-tests from aro-hcp-e2e-tools #75802

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
92d71b7
Extract aro-hcp-test from aro-hcp-e2e-tools into new image aro-hcp-e2…
patriksuba Mar 6, 2026
3b8726f
baseimage - get build root from repository, allow arm64 arch
patriksuba Mar 10, 2026
6a82feb
Grant frontend access to e2e-local run step
patriksuba Mar 25, 2026
f5a5c78
Remove aditional build of arm64 baseimage
patriksuba Mar 25, 2026
2712c0d
Expect aro-hcp-base-ci image to contain intall-tools
patriksuba Apr 13, 2026
213e4b5
Include kubeconfig to svc cluster in test-local-run command
patriksuba Apr 13, 2026
085b8bc
Address feedback, missing env vars and tests
patriksuba Apr 14, 2026
0c603be
Update periodic job cleanup-sweeper-rg-ordered to use aro-hcp-e2e-tests
patriksuba Apr 15, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 13 additions & 12 deletions ci-operator/config/Azure/ARO-HCP/Azure-ARO-HCP-main.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,18 +77,9 @@ images:
USER root
COPY /src /opt/app-root/src/github.com/Azure
WORKDIR /opt/app-root/src/github.com/Azure/ARO-HCP
ENV GOBIN="/usr/local/bin"
ENV PATH="$GOBIN:/usr/local/bin:/usr/bin:$PATH"
ENV GOFLAGS='-mod=readonly'
RUN make install-tools && \
make -C tooling/templatize templatize && \
make -C test/ aro-hcp-tests prow-job-executor && \
go clean -cache -modcache && \
rm -f go.work.sum
RUN mkdir -p "$(go env GOROOT)" && chmod -R 777 "$(go env GOROOT)"
RUN mkdir -p "$(go env GOPATH)" && chmod -R 777 "$(go env GOPATH)"
RUN mkdir -p "$(go env GOCACHE)" && chmod -R 777 "$(go env GOCACHE)"
RUN mkdir -p "$(go env GOBIN)" && chmod -R 777 "$(go env GOBIN)"
RUN make -C tooling/templatize templatize && \
go clean -cache -modcache && rm -f go.work.sum
RUN chmod 777 /opt/app-root/src/github.com/Azure/ARO-HCP
from: src
inputs:
Expand All @@ -97,6 +88,9 @@ images:
- destination_dir: src
source_path: /go/src/github.com/Azure/ARO-HCP
to: aro-hcp-e2e-tools
- dockerfile_path: test/Containerfile.e2e
from: src
to: aro-hcp-e2e-tests
- dockerfile_literal: |
FROM src
USER root
Expand All @@ -118,7 +112,7 @@ images:
promotion:
to:
- additional_images:
aro-hcp-e2e-tools: aro-hcp-e2e-tools
aro-hcp-e2e-tests: aro-hcp-e2e-tests
excluded_images:
- '*'
namespace: aro-hcp
Expand All @@ -131,6 +125,13 @@ resources:
requests:
cpu: "3"
memory: 4Gi
aro-hcp-e2e-tests:
limits:
cpu: "8"
memory: 10Gi
requests:
cpu: "8"
memory: 10Gi
aro-hcp-e2e-tools:
limits:
cpu: "8"
Expand Down
7 changes: 2 additions & 5 deletions ci-operator/config/Azure/ARO-HCP/Azure-ARO-HCP-main__baseimage-generator.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,11 @@
build_root:
image_stream_tag:
name: builder
namespace: ocp
tag: rhel-9-golang-1.25-openshift-4.21
from_repository: true
images:
items:
- dockerfile_path: dev-infrastructure/openshift-ci/Dockerfile
from: src
to: aro-hcp-e2e-base-ci
run_if_changed: ^dev-infrastructure/openshift-ci/Dockerfile$
run_if_changed: ^(dev-infrastructure/openshift-ci/Dockerfile$|\.bingo/)
promotion:
to:
- name: aro-hcp-ci-images
Expand Down
29 changes: 3 additions & 26 deletions ci-operator/config/Azure/ARO-HCP/Azure-ARO-HCP-main__e2e.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,38 +5,15 @@ build_root:
tag: aro-hcp-e2e-base-ci
images:
items:
- dockerfile_literal: |
FROM src
USER root
COPY /src /opt/app-root/src/github.com/Azure
WORKDIR /opt/app-root/src/github.com/Azure/ARO-HCP
ENV GOBIN="/usr/local/bin"
ENV PATH="$GOBIN:/usr/local/bin:/usr/bin:$PATH"
# Install Python for grafana tests
RUN dnf install -y python3 && curl -fsSL https://bootstrap.pypa.io/get-pip.py | python3
RUN pip3 install black
ENV GOFLAGS='-mod=readonly'
RUN make install-tools && \
make -C tooling/templatize templatize && \
make -C test/ aro-hcp-tests prow-job-executor
RUN mkdir -p "$(go env GOROOT)" && chmod -R 777 "$(go env GOROOT)"
RUN mkdir -p "$(go env GOPATH)" && chmod -R 777 "$(go env GOPATH)"
RUN mkdir -p "$(go env GOCACHE)" && chmod -R 777 "$(go env GOCACHE)"
RUN mkdir -p "$(go env GOBIN)" && chmod -R 777 "$(go env GOBIN)"
RUN chmod 777 /opt/app-root/src/github.com/Azure/ARO-HCP && rm -f go.work.sum
- dockerfile_path: test/Containerfile.e2e
from: src
inputs:
src:
paths:
- destination_dir: src
source_path: /go/src/github.com/Azure/ARO-HCP
to: aro-hcp-e2e-tools
to: aro-hcp-e2e-tests
resources:
'*':
requests:
cpu: 100m
memory: 200Mi
aro-hcp-e2e-tools:
aro-hcp-e2e-tests:
limits:
cpu: "8"
memory: 10Gi
Expand Down
4 changes: 2 additions & 2 deletions ci-operator/config/Azure/ARO-HCP/Azure-ARO-HCP-main__periodic.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
base_images:
aro-hcp-e2e-tools:
name: aro-hcp-e2e-tools
aro-hcp-e2e-tests:
name: aro-hcp-e2e-tests
namespace: aro-hcp
tag: latest
resources:
Expand Down
4 changes: 1 addition & 3 deletions ci-operator/jobs/Azure/ARO-HCP/Azure-ARO-HCP-main-postsubmits.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@ postsubmits:
- ^main$
cluster: build03
decorate: true
decoration_config:
skip_cloning: true
labels:
ci-operator.openshift.io/is-promotion: "true"
ci-operator.openshift.io/variant: baseimage-generator
Expand Down Expand Up @@ -394,7 +392,7 @@ postsubmits:
- --promote
- --report-credentials-file=/etc/report/credentials
- --target=[images]
- --target=aro-hcp-e2e-tools
- --target=aro-hcp-e2e-tests
command:
- ci-operator
image: quay-proxy.ci.openshift.org/openshift/ci:ci_ci-operator_latest
Expand Down
6 changes: 2 additions & 4 deletions ci-operator/jobs/Azure/ARO-HCP/Azure-ARO-HCP-main-presubmits.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,15 +82,13 @@ presubmits:
cluster: build05
context: ci/prow/baseimage-generator-images
decorate: true
decoration_config:
skip_cloning: true
labels:
ci-operator.openshift.io/variant: baseimage-generator
ci.openshift.io/generator: prowgen
pj-rehearse.openshift.io/can-be-rehearsed: "true"
name: pull-ci-Azure-ARO-HCP-main-baseimage-generator-images
rerun_command: /test baseimage-generator-images
run_if_changed: ^dev-infrastructure/openshift-ci/Dockerfile$
run_if_changed: ^(dev-infrastructure/openshift-ci/Dockerfile$|\.bingo/)
spec:
containers:
- args:
Expand Down Expand Up @@ -490,7 +488,7 @@ presubmits:
- --image-import-pull-secret=/etc/pull-secret/.dockerconfigjson
- --report-credentials-file=/etc/report/credentials
- --target=[images]
- --target=aro-hcp-e2e-tools
- --target=aro-hcp-e2e-tests
Comment thread
patriksuba marked this conversation as resolved.
command:
- ci-operator
image: quay-proxy.ci.openshift.org/openshift/ci:ci_ci-operator_latest
Expand Down
2 changes: 1 addition & 1 deletion ...registry/aro-hcp/deprovision/cleanup-sweeper/aro-hcp-deprovision-cleanup-sweeper-ref.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
ref:
as: aro-hcp-deprovision-cleanup-sweeper
from: aro-hcp-e2e-tools
from: aro-hcp-e2e-tests
commands: aro-hcp-deprovision-cleanup-sweeper-commands.sh
resources:
requests:
Expand Down
2 changes: 1 addition & 1 deletion .../deprovision/expired-resource-groups/aro-hcp-deprovision-expired-resource-groups-ref.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
ref:
as: aro-hcp-deprovision-expired-resource-groups
from: aro-hcp-e2e-tools
from: aro-hcp-e2e-tests
commands: aro-hcp-deprovision-expired-resource-groups-commands.sh
resources:
requests:
Expand Down
2 changes: 1 addition & 1 deletion ...cp/deprovision/kusto-role-assignments/aro-hcp-deprovision-kusto-role-assignments-ref.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
ref:
as: aro-hcp-deprovision-kusto-role-assignments
from: aro-hcp-e2e-tools
from: aro-hcp-e2e-tests
commands: aro-hcp-deprovision-kusto-role-assignments-commands.sh
resources:
requests:
Expand Down
2 changes: 1 addition & 1 deletion .../step-registry/aro-hcp/gather/custom-link-tools/aro-hcp-gather-custom-link-tools-ref.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
ref:
as: aro-hcp-gather-custom-link-tools
from: aro-hcp-e2e-tools
from: aro-hcp-e2e-tests
commands: aro-hcp-gather-custom-link-tools-commands.sh
resources:
requests:
Expand Down
2 changes: 1 addition & 1 deletion ci-operator/step-registry/aro-hcp/gather/observability/aro-hcp-gather-observability-ref.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
ref:
as: aro-hcp-gather-observability
from: aro-hcp-e2e-tools
from: aro-hcp-e2e-tests
commands: aro-hcp-gather-observability-commands.sh
resources:
requests:
Expand Down
2 changes: 1 addition & 1 deletion ...tep-registry/aro-hcp/gather/test-visualization/aro-hcp-gather-test-visualization-ref.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
ref:
as: aro-hcp-gather-test-visualization
from: aro-hcp-e2e-tools
from: aro-hcp-e2e-tests
commands: aro-hcp-gather-test-visualization-commands.sh
resources:
requests:
Expand Down
3 changes: 2 additions & 1 deletion ci-operator/step-registry/aro-hcp/local-e2e/aro-hcp-local-e2e-workflow.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@ workflow:
- ref: aro-hcp-write-config
- ref: aro-hcp-provision-environment
test:
- ref: aro-hcp-test-local
- ref: aro-hcp-test-local-setup
- ref: aro-hcp-test-local-run
post:
- ref: aro-hcp-gather-provision-failure
- ref: aro-hcp-gather-visualization
Expand Down
18 changes: 18 additions & 0 deletions ci-operator/step-registry/aro-hcp/test/local/run/OWNERS
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
approvers:
- geoberle
- jharrington22
- mmazur
- raelga
- roivaz
- venkateshsredhat
- deads2k
- patriksuba
reviewers:
- geoberle
- jharrington22
- mmazur
- raelga
- roivaz
- venkateshsredhat
- deads2k
- patriksuba
9 changes: 5 additions & 4 deletions ...test/local/aro-hcp-test-local-commands.sh → ...al/run/aro-hcp-test-local-run-commands.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,19 +14,20 @@ export INFRA_SUBSCRIPTION_ID; INFRA_SUBSCRIPTION_ID=$(cat "${CLUSTER_PROFILE_DIR
az login --service-principal -u "${AZURE_CLIENT_ID}" -p "${AZURE_CLIENT_SECRET}" --tenant "${AZURE_TENANT_ID}" --output none
az account set --subscription "${SUBSCRIPTION_ID}"

# TODO: Remove kubeconfig setup once exporter_metrics.go no longer requires direct svc cluster access.
unset GOFLAGS
make -C dev-infrastructure/ svc.aks.kubeconfig.pipeline SVC_KUBECONFIG_FILE=../kubeconfig DEPLOY_ENV=prow
export KUBECONFIG=kubeconfig
export AZURE_TOKEN_CREDENTIALS=prod
Comment on lines 18 to 21
Copy link
Copy Markdown
Contributor

@janboll janboll Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

didn't you move this to setup?

Copy link
Copy Markdown
Contributor Author

@patriksuba patriksuba Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not quite. The e2e tests also use the Azure Go client, so this variable lets it pull credentials from the environment.


FRONTEND_ADDRESS=$(kubectl get virtualservice -n aro-hcp aro-hcp-vs-frontend -o jsonpath='{.spec.hosts[0]}')
ADMIN_API_ADDRESS=$(kubectl get virtualservice -n aro-hcp-admin-api admin-api-vs -o jsonpath='{.spec.hosts[0]}')

az account set --subscription "${INFRA_SUBSCRIPTION_ID}"
make frontend-grant-ingress DEPLOY_ENV=prow
az account set --subscription "${SUBSCRIPTION_ID}"

make e2e/local -o test/aro-hcp-tests SKIP_CERT_VERIFICATION=true FRONTEND_ADDRESS="https://${FRONTEND_ADDRESS}" ADMIN_API_ADDRESS="https://${ADMIN_API_ADDRESS}"
make e2e-local/run -o test/aro-hcp-tests \
FRONTEND_ADDRESS="$(cat "${SHARED_DIR}/frontend-address")" \
ADMIN_API_ADDRESS="$(cat "${SHARED_DIR}/admin-api-address")" \
SKIP_CERT_VERIFICATION=true

# the make target produces a junit.xml in ARTIFACT_DIR. We want to copy to SHARED_DIR so we can create
# direct debugging links for the individual tests that failed. Gzip it due to 3mb SHARED_DIR limit.
Expand Down
8 changes: 5 additions & 3 deletions ...ocal/aro-hcp-test-local-ref.metadata.json → .../aro-hcp-test-local-run-ref.metadata.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"path": "aro-hcp/test/local/aro-hcp-test-local-ref.yaml",
"path": "aro-hcp/test/local/run/aro-hcp-test-local-run-ref.yaml",
"owners": {
"approvers": [
"geoberle",
Expand All @@ -8,7 +8,8 @@
"raelga",
"roivaz",
"venkateshsredhat",
"deads2k"
"deads2k",
"patriksuba"
],
"reviewers": [
"geoberle",
Expand All @@ -17,7 +18,8 @@
"raelga",
"roivaz",
"venkateshsredhat",
"deads2k"
"deads2k",
"patriksuba"
]
}
}
9 changes: 5 additions & 4 deletions ...cp/test/local/aro-hcp-test-local-ref.yaml → ...local/run/aro-hcp-test-local-run-ref.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
ref:
as: aro-hcp-test-local
from: aro-hcp-e2e-tools
commands: aro-hcp-test-local-commands.sh
as: aro-hcp-test-local-run
from: aro-hcp-e2e-tests
commands: aro-hcp-test-local-run-commands.sh
credentials:
- namespace: test-credentials
name: aro-hcp-qe-pull-secret
Expand Down Expand Up @@ -31,4 +31,5 @@ ref:
default: "true"
documentation: Whether to compress timing metadata files with gzip.
documentation: |-
Run ARO HCP local e2e suite on Prow environment
Run the ARO HCP local e2e test suite. Fetches svc cluster kubeconfig, grants frontend ingress, then runs tests.
Reads frontend and admin-api addresses from SHARED_DIR (written by the setup step).
18 changes: 18 additions & 0 deletions ci-operator/step-registry/aro-hcp/test/local/setup/OWNERS
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
approvers:
- geoberle
- jharrington22
- mmazur
- raelga
- roivaz
- venkateshsredhat
- deads2k
- patriksuba
reviewers:
- geoberle
- jharrington22
- mmazur
- raelga
- roivaz
- venkateshsredhat
- deads2k
- patriksuba
32 changes: 32 additions & 0 deletions ci-operator/step-registry/aro-hcp/test/local/setup/aro-hcp-test-local-setup-commands.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
#!/bin/bash
set -o errexit
set -o nounset
set -o pipefail

export CLUSTER_PROFILE_DIR="/var/run/aro-hcp-${VAULT_SECRET_PROFILE}"

export AZURE_CLIENT_ID; AZURE_CLIENT_ID=$(cat "${CLUSTER_PROFILE_DIR}/client-id")
export AZURE_TENANT_ID; AZURE_TENANT_ID=$(cat "${CLUSTER_PROFILE_DIR}/tenant")
export AZURE_CLIENT_SECRET; AZURE_CLIENT_SECRET=$(cat "${CLUSTER_PROFILE_DIR}/client-secret")
export CUSTOMER_SUBSCRIPTION; CUSTOMER_SUBSCRIPTION=$(cat "${CLUSTER_PROFILE_DIR}/subscription-name")
export SUBSCRIPTION_ID; SUBSCRIPTION_ID=$(cat "${CLUSTER_PROFILE_DIR}/subscription-id")
export INFRA_SUBSCRIPTION_ID; INFRA_SUBSCRIPTION_ID=$(cat "${CLUSTER_PROFILE_DIR}/infra-subscription-id")
az login --service-principal -u "${AZURE_CLIENT_ID}" -p "${AZURE_CLIENT_SECRET}" --tenant "${AZURE_TENANT_ID}" --output none
az account set --subscription "${SUBSCRIPTION_ID}"

unset GOFLAGS
make -C dev-infrastructure/ svc.aks.kubeconfig.pipeline SVC_KUBECONFIG_FILE=../kubeconfig DEPLOY_ENV=prow
export KUBECONFIG=kubeconfig
export AZURE_TOKEN_CREDENTIALS=prod

FRONTEND_ADDRESS="https://$(kubectl get virtualservice -n aro-hcp aro-hcp-vs-frontend -o jsonpath='{.spec.hosts[0]}')"
ADMIN_API_ADDRESS="https://$(kubectl get virtualservice -n aro-hcp-admin-api admin-api-vs -o jsonpath='{.spec.hosts[0]}')"
Comment on lines +22 to +23
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot Apr 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Validate discovered hosts before composing/storing URLs.

If jsonpath returns empty hosts, the script currently writes invalid https:// endpoints and fails later in less obvious ways.

Proposed validation 
-FRONTEND_ADDRESS="https://$(kubectl get virtualservice -n aro-hcp aro-hcp-vs-frontend -o jsonpath='{.spec.hosts[0]}')"
-ADMIN_API_ADDRESS="https://$(kubectl get virtualservice -n aro-hcp-admin-api admin-api-vs -o jsonpath='{.spec.hosts[0]}')"
+FRONTEND_HOST="$(kubectl get virtualservice -n aro-hcp aro-hcp-vs-frontend -o jsonpath='{.spec.hosts[0]}')"
+ADMIN_API_HOST="$(kubectl get virtualservice -n aro-hcp-admin-api admin-api-vs -o jsonpath='{.spec.hosts[0]}')"
+if [[ -z "${FRONTEND_HOST}" || -z "${ADMIN_API_HOST}" ]]; then
+  echo "Failed to resolve virtualservice hosts for frontend/admin API" >&2
+  exit 1
+fi
+FRONTEND_ADDRESS="https://${FRONTEND_HOST}"
+ADMIN_API_ADDRESS="https://${ADMIN_API_HOST}"

Also applies to: 31-32

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In
`@ci-operator/step-registry/aro-hcp/test/local/setup/aro-hcp-test-local-setup-commands.sh`
around lines 22 - 23, The script currently constructs FRONTEND_ADDRESS and
ADMIN_API_ADDRESS directly from kubectl jsonpath which can be empty; update the
code to first capture the host values (e.g., frontend_host and admin_api_host)
using the same kubectl jsonpath commands, validate each is non-empty (if empty,
print a clear error and exit non-zero or handle accordingly), and only then
compose FRONTEND_ADDRESS="https://$frontend_host" and
ADMIN_API_ADDRESS="https://$admin_api_host"; apply the same validation pattern
to the other occurrences referenced (lines 31-32) so no invalid "https://" URLs
are written when hosts are missing.


az account set --subscription "${INFRA_SUBSCRIPTION_ID}"
make frontend-grant-ingress DEPLOY_ENV=prow
az account set --subscription "${SUBSCRIPTION_ID}"

make e2e-local/setup FRONTEND_ADDRESS="${FRONTEND_ADDRESS}"

echo "${FRONTEND_ADDRESS}" > "${SHARED_DIR}/frontend-address"
echo "${ADMIN_API_ADDRESS}" > "${SHARED_DIR}/admin-api-address"
25 changes: 25 additions & 0 deletions ...perator/step-registry/aro-hcp/test/local/setup/aro-hcp-test-local-setup-ref.metadata.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
{
"path": "aro-hcp/test/local/setup/aro-hcp-test-local-setup-ref.yaml",
"owners": {
"approvers": [
"geoberle",
"jharrington22",
"mmazur",
"raelga",
"roivaz",
"venkateshsredhat",
"deads2k",
"patriksuba"
],
"reviewers": [
"geoberle",
"jharrington22",
"mmazur",
"raelga",
"roivaz",
"venkateshsredhat",
"deads2k",
"patriksuba"
]
}
}
Loading