extensions: authenticate allocation endpoint with requestheader client CA#4582
extensions: authenticate allocation endpoint with requestheader client CA#4582adilburaksen wants to merge 2 commits into
Conversation
|
/gcbrun |
|
Build Failed 😭 Build Id: c818caef-47eb-44ed-90a9-b5a07b23a94f Status: FAILURE To get permission to view the Cloud Build view, join the agones-discuss Google Group. |
|
These failures appear to be pre-existing infrastructure issues unrelated to this PR's changes: Step #26 (upgrade-test): The build fails with Step #28 (e2e-test): Of the four clusters tested, three pass and one fails:
Since Happy to re-trigger ( |
|
/gcbrun It's a flake we see on autopilot sometimes. Tempted to add a retry in there for CI, but it shouldn't be needed -- but I digress. |
|
Also noting, you will need to sign your commits per DCO. |
…t CA The extensions HTTPS server exposes the allocation resource handler (/apis/allocation.agones.dev/v1/namespaces/...) to any in-cluster workload that can reach agones-controller-service:443 directly, bypassing Kubernetes RBAC for the aggregated resource. Fix using the standard Kubernetes aggregation-layer authentication protocol (https://kubernetes.io/docs/tasks/extend-kubernetes/configure-aggregation-layer/): * pkg/util/https/server.go — add ClientAuth: RequestClientCert so the TLS handshake forwards the client certificate (from the kube-apiserver proxy) into r.TLS.PeerCertificates without requiring callers that do not present a cert (e.g. webhooks) to do so. * pkg/util/apiserver/apiserver.go — add SetRequestHeaderCA(*x509.CertPool) and an authenticatedHandler wrapper. The wrapper verifies the leaf certificate in r.TLS.PeerCertificates against the CA pool with ExtKeyUsageClientAuth, mirroring the approach in k8s.io/apiserver/pkg/authentication/request/x509. Discovery and OpenAPI handlers are left unauthenticated. If no CA is configured (e.g. in unit tests) the handler is called directly. * cmd/extensions/main.go — load requestheader-client-ca-file from the kube-system/extension-apiserver-authentication ConfigMap (populated by Kubernetes for every aggregated API server) and wire it into the APIServer via SetRequestHeaderCA. A log warning is emitted if the ConfigMap is unavailable; auth is then effectively disabled for that start-up, preserving the previous behaviour while making the failure visible. Fixes: agones-dev#4572 Signed-off-by: adilburaksen <adilburaksen@gmail.com>
08a3dfe to
8bc0208
Compare
|
Build Succeeded 🥳 Build Id: 5c38641c-8e08-4493-bc28-2d26dea689a5 The following development artifacts have been built, and will exist for the next 30 days:
A preview of the website (the last 30 builds are retained): To install this version: |
Summary
This PR fixes an authorization bypass in Agones' extension API server where any in-cluster workload could call the allocation endpoint directly, bypassing Kubernetes RBAC entirely.
Problem
The Agones extension API server (port 8082) serves
/apis/allocation.agones.dev/v1/namespaces/…/gameserverallocationsvia its own HTTP mux. When a client calls this endpoint through the Kubernetes aggregation layer,kube-apiserverproxies the request and includes a requestheader client certificate signed by the requestheader CA (fromkube-system/extension-apiserver-authentication).Previously, the server did not verify that certificate. Any pod with network access to port 8082 could send a raw HTTP request and receive a valid response — no Kubernetes ServiceAccount token, no RBAC check, nothing.
Fix
Three files changed:
pkg/util/https/server.goClientAuth: tls.RequestClientCerton the TLS config.r.TLS.PeerCertificatesfor the auth middleware.pkg/util/apiserver/apiserver.goSetRequestHeaderCA(*x509.CertPool)— called at startup to supply the CA.authenticatedHandler— wraps anyErrorHandlerFuncto verify the first peer certificate against the CA pool withExtKeyUsageClientAuth. Mirrors the logic ink8s.io/apiserver/pkg/authentication/request/x509./apis/allocation.agones.dev/v1) and OpenAPI handlers are left unauthenticated — they carry no sensitive data and are called by tools that do not present certs./apis/…/namespaces/resource handler (i.e. the allocation endpoint) requires the cert.cmd/extensions/main.goloadRequestHeaderCA— readskube-system/extension-apiserver-authenticationConfigMap, parsesrequestheader-client-ca-filePEM, returns an*x509.CertPool.NewAPIServer; if the ConfigMap is absent or empty (e.g. clusters that don't use the aggregation layer), the server starts with a warning and no auth enforcement, preserving backward compatibility.Testing
Existing unit tests pass (
go test ./pkg/util/apiserver/...). TheauthenticatedHandlernil-CA path ensures existing tests continue to work without TLS setup.Integration / e2e: the TLS
RequestClientCertflag is transparent to callers that don't present a cert (webhook callers), andkube-apiserveralways presents the requestheader cert when proxying aggregation-layer requests.References
extension-apiserver-authenticationConfigMap: https://kubernetes.io/docs/tasks/extend-kubernetes/configure-aggregation-layer/#contacting-the-extension-apiserver