feat(metal-control-plane): add GatewayAPI support

[l0wl3vel]

Description

Add GatewayAPI support to metal-control-plane

Used AI-Tools ✨

• none used for generation

Closes #157

References: metal-stack/metal-roles#562

[Gerrit91]

Thanks for taking on this topic. PR looks good!

Can you please reference the umbrella issue in metal-roles in the PR description?

As far as I know we have not documented anything or discussed / decided on how the migration will take place. Can we elaborate a bit on this topic somewhere? Because maybe we need to write something to the release notes. Is this already running somewhere? With which implementation? Would be nice if this would run in the mini-lab, I think?

[Gerrit91]

Not an expert on the GRPCRoute resource but since we're using Connect RPC in the metal-apiserver, I would guess that we even need to use HTTPRoute in order to expose all Connect functionality? Clients could also speak for example grpcweb protocol and stuff, would this work with GRPCRoute?

[l0wl3vel]

No idea, first time for me working with GRPCRoutes. Going to look into it

[l0wl3vel]

Ok, so we use connectrpc, which serves multiple protocols at the same time. If we intend to support non-gRPC clients we cannot use GRPCRoutes anyway. And GatewayAPI does not support cross-serving GRPCRoutes and HTTPRoutes on the same hostname.

So HTTPRoute it is.

What is the plan with the metal-api gRPC endpoint? Do we support non-gRPC clients there as well? We currently terminate TLS in the application and I would like to create a follow up ticket to move that endpoint to either an HTTPRoute or GRPCRoute so we can terminate all TLS connections on the gateway.

[l0wl3vel]

Thanks for taking on this topic. PR looks good!

Can you please reference the umbrella issue in metal-roles in the PR description?

As far as I know we have not documented anything or discussed / decided on how the migration will take place. Can we elaborate a bit on this topic somewhere? Because maybe we need to write something to the release notes. Is this already running somewhere? With which implementation? Would be nice if this would run in the mini-lab, I think?

Sure, going to add that to the umbrella.

The plan is to add the required functionality for GatewayAPI to all roles,charts and the mini-lab in addition to ingress and the ingress-nginx tcp exposure we are doing right now in mini-lab. GatewayAPI will be disabled by default in the roles and charts. This way operators can migrate gradually and switch over one service at a time, if they would like.

I would like to enable GatewayAPI in the mini-lab by default as soon as everything is ready so we get some mileage before rolling it out to real clusters.

I am developing this using mini-lab, so there is going to be a reference implementation using envoy-gateway on top of cloud-provider-kind/ for LoadBalancer Services.

[Gerrit91]

Thanks, sounds good to me.

Two more questions:

1. For the migration is it planned that operators deploy both resource types (ingress + gateway) and then point their DNS entry to the new gateway controller?
2. If there was a mini-lab PR that I can use for trying this out, it would be easier for me to verify this PR. Do you think it's possible to create one for this purpose?

[l0wl3vel]

Yes, the goal is to have gatewayapi behave in the same way as our current ingress setup. Operators should be able to deploy both and only have to adjust their DNS records to point to the gateway(s).

The mini-lab PR is not ready yet. Going to set this PR to draft and ping you when it is ready. Sorry for the disturbance.

[Gerrit91]

Thanks a lot!