Skip to content

nginx: add Cognito buffer directives to chart ConfigMap (16k)#82

Open
elamaran11 wants to merge 2 commits into
mainfrom
fix/nginx-cookie-header-buffers
Open

nginx: add Cognito buffer directives to chart ConfigMap (16k)#82
elamaran11 wants to merge 2 commits into
mainfrom
fix/nginx-cookie-header-buffers

Conversation

@elamaran11
Copy link
Copy Markdown
Collaborator

@elamaran11 elamaran11 commented May 15, 2026

Summary

Add proxy_buffer_size, proxy_buffers, and large_client_header_buffers to helm/accelbench/templates/web-configmap.yaml at 16k, matching the values already in docker/nginx.conf from #45.

Why

PR #45 added the buffer directives to docker/nginx.conf — the config baked into the web image. But the chart mounts a ConfigMap at /etc/nginx/conf.d/default.conf via a subPath mount in helm/accelbench/templates/web-deployment.yaml:32-34, which overrides the baked file. At runtime, nginx reads the ConfigMap version, which had no buffer directives → defaults → 400 Bad Request — Request Header Or Cookie Too Large once Cognito sessions accumulated state.

Confirmed before this PR:

$ kubectl get configmap accelbench-nginx -n accelbench -o jsonpath='{.data.default\.conf}' | grep buffer
(no output)

After applying the chart change to the live cluster:

$ kubectl exec -n accelbench accelbench-web-... -- grep -E "buffer_size|large_client" /etc/nginx/conf.d/default.conf
proxy_buffer_size           16k;
large_client_header_buffers 4 16k;

That's why the 400s recurred a week after #45 merged — the fix never reached the running nginx.

Test plan

  • helm upgrade accelbench helm/accelbench -n accelbench --reuse-values succeeds
  • kubectl rollout restart deployment/accelbench-web rolls cleanly (2/2 ready)
  • kubectl exec into a new web pod shows the buffer directives in the live nginx config
  • curl https://modelbench.elamaras.people.aws.dev/ returns 200

🤖 Generated with Claude Code

The Helm web-configmap was missing the buffer settings that exist in
docker/nginx.conf, so the ConfigMap mount overrode the baked image
config and nginx ran with the 4k/8k defaults. Once a Cognito session
accumulated enough state (refreshed tokens, multiple logins) the
request headers crossed the limit and nginx returned
"400 Bad Request / Request Header Or Cookie Too Large".

Add the buffer directives to the chart template and bump both files
from 16k to 32k for headroom against federated IdP claims.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@elamaran11
Copy link
Copy Markdown
Collaborator Author

Quick context on why this is still needed despite #45:

PR #45 added the buffer directives to docker/nginx.conf — the config baked into the web image. But the Helm chart mounts a ConfigMap at /etc/nginx/conf.d/default.conf via a subPath mount in helm/accelbench/templates/web-deployment.yaml:32-34, which overrides the baked file. At runtime, nginx reads the ConfigMap version.

The ConfigMap template (helm/accelbench/templates/web-configmap.yaml) on main has no buffer directives, so the live nginx runs on the 4k/8k defaults. Confirmed against the cluster before this PR:

$ kubectl get configmap accelbench-nginx -n accelbench -o jsonpath='{.data.default\.conf}' | grep buffer
(no output)

This PR adds the directives to the ConfigMap template — the place that actually reaches the running nginx. After applying:

$ kubectl exec -n accelbench accelbench-web-... -- grep -E "buffer_size|large_client" /etc/nginx/conf.d/default.conf
proxy_buffer_size           32k;
large_client_header_buffers 4 32k;

That's why the 400s recurred a week after #45 merged — the fix never reached the running nginx.

Drop the 32k bump and keep parity with the values added in #45.
The fix that matters is adding the directives to the chart's
ConfigMap template — it mounts at /etc/nginx/conf.d/default.conf
via subPath and overrides the baked image config, so the buffer
settings in docker/nginx.conf alone never reach the running nginx.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@elamaran11 elamaran11 changed the title nginx: bump header buffers to 32k to handle Cognito cookie growth nginx: add Cognito buffer directives to chart ConfigMap (16k) May 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant