Description
The E2E Tests (Full Suite) workflow fails intermittently on the shard that runs e2e-tests/harness-custom-jwt.test.ts. The first test (deploys with CUSTOM_JWT authorizer configuration) fails at deploy with:
{"success":false, ...,
"error":"The number of agent identity Oauth2 credential providers in this account has reached its limit"}
and the remaining tests in the file then cascade (No deployed targets found, expected 'local-only' to be 'deployed').
Example failed run: https://github.com/aws/agentcore-cli/actions/runs/28463247986/job/84356322175
Root Cause
harness-custom-jwt.test.ts adds a CUSTOM_JWT harness with --client-id/--client-secret, which registers a managed OAuth2 credential provider (<harness>-oauth). That provider is created imperatively pre-deploy, outside the CloudFormation stack, so the test's remove all + teardown deploy destroys the stack but leaves the OAuth2 provider behind.
The E2E globalSetup stale-cleanup only reaps ApiKey credential providers (ListApiKeyCredentialProvidersCommand), not OAuth2 providers, and nothing else deletes them. So every run leaks one E2eHrnsJwt<ts>-oauth provider permanently. The account caps OAuth2 providers at 50 (quota L-431051DC); once full, every CUSTOM_JWT deploy fails with the quota error above.
Confirmed empirically: the E2E account was at exactly 50/50 OAuth2 providers, all named E2eHrnsJwt<ts>-oauth, accumulated over ~1 week. (Leaked providers have been manually reaped to unblock CI in the interim.)
Fix (this PR)
- Add
cleanupStaleOAuth2CredentialProviders to the E2E globalSetup hook so every run auto-reaps leaked E2e-prefixed OAuth2 providers older than 30 min (mirrors the existing ApiKey / recommendation / stack reapers).
- Add per-test teardown in
harness-custom-jwt.test.ts afterAll to delete its own managed OAuth2 provider, so a single run self-cleans.
Acceptance Criteria
Notes
The underlying product behavior — real users' OAuth2 providers being orphaned on remove all/teardown — is tracked separately in #1673 and is not fixed by this PR (test-side mitigation only).
Description
The
E2E Tests (Full Suite)workflow fails intermittently on the shard that runse2e-tests/harness-custom-jwt.test.ts. The first test (deploys with CUSTOM_JWT authorizer configuration) fails at deploy with:and the remaining tests in the file then cascade (
No deployed targets found,expected 'local-only' to be 'deployed').Example failed run: https://github.com/aws/agentcore-cli/actions/runs/28463247986/job/84356322175
Root Cause
harness-custom-jwt.test.tsadds a CUSTOM_JWT harness with--client-id/--client-secret, which registers a managed OAuth2 credential provider (<harness>-oauth). That provider is created imperatively pre-deploy, outside the CloudFormation stack, so the test'sremove all+ teardown deploy destroys the stack but leaves the OAuth2 provider behind.The E2E
globalSetupstale-cleanup only reaps ApiKey credential providers (ListApiKeyCredentialProvidersCommand), not OAuth2 providers, and nothing else deletes them. So every run leaks oneE2eHrnsJwt<ts>-oauthprovider permanently. The account caps OAuth2 providers at 50 (quotaL-431051DC); once full, every CUSTOM_JWT deploy fails with the quota error above.Confirmed empirically: the E2E account was at exactly 50/50 OAuth2 providers, all named
E2eHrnsJwt<ts>-oauth, accumulated over ~1 week. (Leaked providers have been manually reaped to unblock CI in the interim.)Fix (this PR)
cleanupStaleOAuth2CredentialProvidersto the E2EglobalSetuphook so every run auto-reaps leakedE2e-prefixed OAuth2 providers older than 30 min (mirrors the existing ApiKey / recommendation / stack reapers).harness-custom-jwt.test.tsafterAllto delete its own managed OAuth2 provider, so a single run self-cleans.Acceptance Criteria
globalSetupdeletes staleE2e-prefixed OAuth2 providers before tests start.harness-custom-jwt.test.tsdeletes its<harness>-oauthprovider on teardown.E2E Tests (Full Suite)CUSTOM_JWT shard passes without hitting the OAuth2 quota.Notes
The underlying product behavior — real users' OAuth2 providers being orphaned on
remove all/teardown — is tracked separately in #1673 and is not fixed by this PR (test-side mitigation only).