fix(SES): create missing template instead of failing the send

[abnegate]

Problem

Sending email through the SES adapter fails on the first send of any new message content with:

Template utopia-<hash> does not exist.

even with a fully-permissioned IAM user (ses:*). Reported from a real Cloud project; the error reproduces 100% of the time for first-time content.

Root cause

The no-attachment path is template-based (SES.php): it sends via SendBulkEmail referencing a content-hashed template (utopia-<hash>). On the first send that template does not exist yet, so the adapter is designed to detect the "template missing" error → CreateEmailTemplate → retry.

That recovery never fired. SES API v2 uses the AWS REST-JSON protocol, which returns the exception type (BadRequestException) in the x-amzn-ErrorType response header — the JSON body is only {"message":"Template ... does not exist."}. But the base Adapter::request() used CURLOPT_RETURNTRANSFER only and discarded every response header. So:

• errorType() read the body, found no __type, and returned null
• isTemplateMissing() returned false
• the template was never created, and the raw error fell through to every recipient

Verified against the live endpoint — an unsigned SES request returns the type in the header (x-amzn-errortype: MissingAuthenticationTokenException), confirming the body never carries it.

Why CI stayed green

The routing test mocked a missing template as a 200 with a per-entry TEMPLATE_NOT_FOUND status — a shape SES never returns for a missing default template (that case is always a top-level 400). The test and the code shared the same wrong assumption, so they agreed with each other and disagreed with SES.

Fix

1. Adapter::request() — capture response headers via CURLOPT_HEADERFUNCTION (lowercased keys) and expose them under a new additive headers key. No other adapter is affected.
2. SES::errorType() — read the type from x-amzn-ErrorType first, keeping the body __type/code as a fallback for AWS JSON-protocol responses.
3. SES::isTemplateMissing() — require the message to signal non-existence ("does not exist" / "not found") so an unrelated BadRequestException mentioning a template is not misclassified.

Tests

• Two new regression tests reproduce the real SES shape (top-level 400 + x-amzn-ErrorType header): the create-and-retry recovery and the concurrent AlreadyExistsException race. Both fail without the fix (1 request, recipient failure) and pass with it (3 requests, delivered).
• Routing stubs updated to return the new headers key.

Verification

• New regression tests: fail → pass as required
• Network-free routing/signing suites: 30 passing
• Full suite vs. baseline: zero new failures (pre-existing reds are live-integration tests needing credentials/request-catcher)
• composer lint (Pint, PSR-12): pass
• composer analyse (PHPStan level 6): [OK] No errors

🤖 Generated with Claude Code

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[greptile-apps]

Greptile Summary

This PR fixes a 100%-reproducible failure in the SES adapter where the first send of any new message content would error because the template-missing recovery path never fired. The root cause was that SES API v2 signals the exception type in the x-amzn-ErrorType response header (not the JSON body), but Adapter::request() discarded all response headers.

• Adapter::request() now captures response headers via CURLOPT_HEADERFUNCTION (lowercased keys) and exposes them under a new headers key; requestMulti() gets 'headers' => [] with a comment explaining why per-handle capture is not implemented (copying a CURLOPT_HEADERFUNCTION-bearing handle with curl_copy_handle segfaults).
• SES::errorType() now reads x-amzn-errortype first, falling back to the body __type/code fields for older AWS JSON-protocol responses; isTemplateMissing() is tightened to also require "does not exist" or "not found" in the message to avoid misclassifying unrelated BadRequestExceptions.
• Two new regression tests reproduce the real SES v2 response shape (top-level 400 + header) and verify the create-and-retry flow and the concurrent AlreadyExistsException race condition.

Confidence Score: 5/5

Safe to merge — the change is additive (a new headers key on the response struct) and directly fixes a confirmed production failure with regression tests that would have caught the original bug.

The fix is narrowly scoped: header capture is additive and does not change existing keys, the errorType fallback preserves the old body-based path, and the new tests demonstrate the exact failure mode described in the PR. No other adapter reads response headers, so the only behaviour change is inside SES itself.

No files require special attention.

Important Files Changed

Filename	Overview
src/Utopia/Messaging/Adapter.php	Adds CURLOPT_HEADERFUNCTION to capture response headers (lowercase keys) and exposes them under a new headers key; requestMulti() gets 'headers' => [] with an explanatory comment about the curl_copy_handle/closure segfault.
src/Utopia/Messaging/Adapter/Email/SES.php	errorType() now reads x-amzn-errortype header first (correct for SES API v2 REST-JSON protocol); isTemplateMissing() tightened to require "does not exist"/"not found" in the message body to avoid false positives on unrelated BadRequestExceptions; all internal type signatures updated.
tests/Messaging/Adapter/Email/SESRoutingTest.php	Two new regression tests covering the real SES v2 shape (top-level 400 + x-amzn-ErrorType header): create-and-retry recovery and concurrent AlreadyExistsException race; stub updated to forward headers from stubResponses.
tests/Messaging/Adapter/Email/ResendRoutingTest.php	Stub updated to return 'headers' => [] to match the new request() return shape; no functional changes.
.github/workflows/test.yml	Adds SES_ACCESS_KEY, SES_SECRET_KEY, SES_SESSION_TOKEN, SES_REGION, and SES_TEST_EMAIL secrets/vars for live integration test support.
docker-compose.yml	Forwards the five new SES env vars into the test container; no other changes.

_{Reviews (3): Last reviewed commit: "test(SES): wire SES_* credentials into C..." | Re-trigger Greptile}