Hcp tentant example by rbo · Pull Request #147 · openshift-examples/web

rbo · 2026-05-01T21:06:46Z

No description provided.

github-actions · 2026-05-01T21:11:50Z

🚀 Preview Deployment Success! View your live changes here: https://1025da2a.openshift-examples.pages.dev\n\nUploading... (337/491)
Uploading... (389/491)
Uploading... (440/491)
Uploading... (491/491)
✨ Success! Uploaded 154 files (337 already uploaded) (2.45 sec)

🌎 Deploying...
✨ Deployment complete! Take a peek over at https://1025da2a.openshift-examples.pages.dev
✨ Deployment alias URL: https://pr-147.openshift-examples.pages.dev

github-actions · 2026-05-04T09:19:21Z

🚀 Preview Deployment Success! View your live changes here: https://806a1b35.openshift-examples.pages.dev\n\nUploading... (345/491)
Uploading... (394/491)
Uploading... (442/491)
Uploading... (491/491)
✨ Success! Uploaded 146 files (345 already uploaded) (2.47 sec)

🌎 Deploying...
✨ Deployment complete! Take a peek over at https://806a1b35.openshift-examples.pages.dev
✨ Deployment alias URL: https://pr-147.openshift-examples.pages.dev

github-actions · 2026-05-04T11:08:22Z

🚀 Preview Deployment Success! View your live changes here: https://1b8cddf7.openshift-examples.pages.dev\n\nUploading... (487/491)
Uploading... (488/491)
Uploading... (489/491)
Uploading... (491/491)
✨ Success! Uploaded 4 files (487 already uploaded) (1.43 sec)

🌎 Deploying...
✨ Deployment complete! Take a peek over at https://1b8cddf7.openshift-examples.pages.dev
✨ Deployment alias URL: https://pr-147.openshift-examples.pages.dev

github-actions · 2026-05-12T15:57:30Z

🚀 Preview Deployment Success! View your live changes here: https://f1d0d251.openshift-examples.pages.dev\n\nUploading... (361/511)
Uploading... (411/511)
Uploading... (461/511)
Uploading... (511/511)
✨ Success! Uploaded 150 files (361 already uploaded) (3.99 sec)

🌎 Deploying...
✨ Deployment complete! Take a peek over at https://f1d0d251.openshift-examples.pages.dev
✨ Deployment alias URL: https://pr-147.openshift-examples.pages.dev

github-actions · 2026-05-13T09:44:43Z

🚀 Preview Deployment Success! View your live changes here: https://1318dc30.openshift-examples.pages.dev\n\nUploading... (362/511)
Uploading... (412/511)
Uploading... (462/511)
Uploading... (511/511)
✨ Success! Uploaded 149 files (362 already uploaded) (2.73 sec)

🌎 Deploying...
✨ Deployment complete! Take a peek over at https://1318dc30.openshift-examples.pages.dev
✨ Deployment alias URL: https://pr-147.openshift-examples.pages.dev

rbo · 2026-05-13T09:46:16Z

@rguske, please review.

rguske · 2026-05-13T18:16:31Z

+|API Server|<li>LoadBalancer (recommended; Kubernetes `LoadBalancer` service)</li><li>NodePort* (not for production)</li>|✅|❌|
+|OAuth|<li>Route (default)</li><li>NodePort* (not for production)</li>|❌|✅|
+|Konnectivity|<li>Route (default)</li><li>LoadBalancer (Kubernetes `LoadBalancer` service)</li><li>NodePort* (not for production)</li>|✅|✅|
+|Ignition|<li>Route (default)</li><li>NodePort* (not for production)</li>|✅|❌|


The ✅ is set for LoadBalancer but must be Route.

rguske · 2026-05-13T18:20:45Z

+
+Use a dedicated OpenShift Ingress Controller shard on the **hub** so only the hosted-cluster control-plane Routes are served by that shard. Tenant clients resolve OAuth, Konnectivity, and Ignition hostnames to `ingress-shared-lb`, which forwards to the shard’s NodePorts on the management network.
+
+Place an external load balancer in front of that shard (for example F5 BIG-IP or NetScaler) that can reach the hub’s management network and present stable tenant-facing VIPs or addresses.


NIT: (for example F5, BIG-IP or NetScaler)

rguske · 2026-05-13T18:23:39Z

+
+### Router between Mgmt and Tenant-A
+
+[VyOS](https://vyos.io/) acts as router and firewall between management and Tenant-A. Restrict **lateral** traffic between the two segments (no full mesh); allow only what you need (for example DNS to resolvers, default route or NAT for internet egress). Hosted-cluster control-plane traffic from tenant nodes should flow to the **external load balancer VIPs** in the tenant segment (not directly into arbitrary management subnets).


NIT: VyOS acts as router and firewall between the management and the Tenant-A network.

rguske · 2026-05-13T18:27:41Z

+
+## Exposing components via a dedicated router shard
+
+Use a dedicated OpenShift Ingress Controller shard on the **hub** so only the hosted-cluster control-plane Routes are served by that shard. Tenant clients resolve OAuth, Konnectivity, and Ignition hostnames to `ingress-shared-lb`, which forwards to the shard’s NodePorts on the management network.


I wonder why NodePort? Why not using MetalLB? The endpointPublishingStrategy for the Ingress Shard could be type: LoadBalancerService.

rguske · 2026-05-13T18:29:47Z

+* [2.3.4. Ingress sharding in OpenShift Container Platform](https://docs.redhat.com/en/documentation/openshift_container_platform/4.21/html/ingress_and_load_balancing/configuring-ingress-cluster-traffic#nw-ingress-sharding-concept_configuring-ingress-cluster-traffic-ingress-controller)
+* [3.1.3.8.1. Example load balancer configuration for user-provisioned clusters](https://docs.redhat.com/en/documentation/openshift_container_platform/4.21/html/installing_on_vmware_vsphere/user-provisioned-infrastructure)
+
+???+ example "Ingress Controller"


As already commented above. Why NodePort instead of LoadBalancer?

oc create -f - <<EOF apiVersion: operator.openshift.io/v1 kind: IngressController metadata: name: sharded-hcp-2003-57 namespace: openshift-ingress-operator spec: routeAdmission: wildcardPolicy: WildcardsAllowed domain: hosted.hcp-2003-57.hcp-cps.coe.muc.redhat.com namespaceSelector: matchLabels: kubernetes.io/metadata.name: clusters-hcp-2003-57 endpointPublishingStrategy: type: LoadBalancerService EOF

rguske · 2026-05-13T18:38:07Z

+router-nodeport-tenant-a   NodePort   172.30.141.209   <none>        80:32460/TCP,443:32488/TCP,1936:32095/TCP   106s
+```
+
+The ingress shard load balancer is an RHEL 9 host running HAProxy (external load balancer `ingress-shared-lb`).


I think this will be the answer to my LoadBalancer related questions, correct? I mean, NodePort is generally something I try to avoid.

rguske · 2026-05-13T18:44:09Z

+| `api-lb` | Tenant-facing VIP for the hosted cluster **API** (`APIServer` publishing) |
+| `ingress-lb` | Tenant-facing VIP for **hosted cluster** application Routes (`*.apps…`) |
+
+Suggested order: (1) hub ingress shard + `ingress-shared-lb` + DNS for the three control-plane hostnames, (2) `api-lb` + API DNS, (3) `ingress-lb` + wildcard apps DNS, then (4) apply `HostedCluster` and `NodePool`. Adjust if your automation creates services first and you backfill DNS once NodePorts or service endpoints are known.


Make it a list for better readability. Furthermore, initially I found it hard to understand what is meant by "Suggested order". You mean the configuration/deployment order?

rguske · 2026-05-13T18:50:02Z

+
+The following two subsections describe (2) and (3); the hub shard and DNS for OAuth, Konnectivity, and Ignition are covered above.
+
+### Deploy External Load Balancer for API (`api-lb`)


Maybe rephrase to "Deploy External Load Balancer for the Hosted-Cluster API (api-lb)"

rguske · 2026-05-13T18:51:17Z

+api.tenant-a.coe.muc.redhat.com.       IN A 192.168.203.<IP of VM>
+```
+
+### Deploy External Load Balancer for Ingress (`ingress-lb`) of hosted cluster


Maybe rephrase to "Deploy External Load Balancer for the Hosted-Cluster Ingress (ingress-lb)"

rguske · 2026-05-13T18:55:27Z

+        persistent:
+          size: 32Gi
+      additionalNetworks:
+      - name: default/cudn-localnet1-2003 # (1)


Maybe including the creation of the CUDN before this manifest as well`?

oc apply -f - <<EOF apiVersion: k8s.ovn.org/v1 kind: ClusterUserDefinedNetwork metadata: name: cudn-localnet1-2003 spec: namespaceSelector: matchExpressions: - key: hypershift.openshift.io/hosted-control-plane operator: Exists network: topology: Localnet localnet: role: Secondary physicalNetworkName: localnet1 ipam: mode: Disabled vlan: mode: Access access: id: 2003 EOF

rbo force-pushed the hcp-tentant-example branch from 2329a5f to daff04e Compare May 4, 2026 09:17

rbo force-pushed the hcp-tentant-example branch from 30d05ec to 13e528b Compare May 4, 2026 11:06

rbo added 2 commits May 12, 2026 17:55

First draft

b243027

Update mkdocs-drawio from 1.8.2 to 1.15.0

3e8cd96

rbo force-pushed the hcp-tentant-example branch from 13e528b to 9d86d5c Compare May 12, 2026 15:55

Update tendant network

84e9eef

rbo force-pushed the hcp-tentant-example branch from 9d86d5c to 84e9eef Compare May 13, 2026 09:42

rbo requested a review from rguske May 13, 2026 09:46

rguske reviewed May 13, 2026

View reviewed changes


		Use a dedicated OpenShift Ingress Controller shard on the hub so only the hosted-cluster control-plane Routes are served by that shard. Tenant clients resolve OAuth, Konnectivity, and Ignition hostnames to `ingress-shared-lb`, which forwards to the shard’s NodePorts on the management network.

		Place an external load balancer in front of that shard (for example F5 BIG-IP or NetScaler) that can reach the hub’s management network and present stable tenant-facing VIPs or addresses.


		### Router between Mgmt and Tenant-A

		[VyOS](https://vyos.io/) acts as router and firewall between management and Tenant-A. Restrict lateral traffic between the two segments (no full mesh); allow only what you need (for example DNS to resolvers, default route or NAT for internet egress). Hosted-cluster control-plane traffic from tenant nodes should flow to the external load balancer VIPs in the tenant segment (not directly into arbitrary management subnets).


		## Exposing components via a dedicated router shard

		Use a dedicated OpenShift Ingress Controller shard on the hub so only the hosted-cluster control-plane Routes are served by that shard. Tenant clients resolve OAuth, Konnectivity, and Ignition hostnames to `ingress-shared-lb`, which forwards to the shard’s NodePorts on the management network.


		The following two subsections describe (2) and (3); the hub shard and DNS for OAuth, Konnectivity, and Ignition are covered above.

		### Deploy External Load Balancer for API (`api-lb`)

Conversation

rbo commented May 1, 2026

Uh oh!

github-actions Bot commented May 1, 2026

Uh oh!

github-actions Bot commented May 4, 2026

Uh oh!

github-actions Bot commented May 4, 2026

Uh oh!

github-actions Bot commented May 12, 2026

Uh oh!

github-actions Bot commented May 13, 2026

Uh oh!

rbo commented May 13, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants