fix(deploy): delete orphaned credential providers on teardown and removal

[aidandaly24]

Description

OAuth2 and API key credential providers are created imperatively as a pre-deploy step, outside the CloudFormation stack — managed OAuth creds for CUSTOM_JWT harnesses/gateways, API keys for non-Bedrock model providers. Nothing ever deleted them:

• Stack teardown only deletes in-stack resources.
• The teardown path reaped payment providers only (cleanupPaymentCredentialProviders) — there was no equivalent for OAuth2 or API key providers, and no Delete* wrapper for them existed in the CLI at all.

So both teardown flows leaked:

Flow	Result before this PR
remove all + teardown deploy	stack destroyed; OAuth2/ApiKey providers orphaned
remove <credential> + redeploy (others remain)	removed cred's provider orphaned

Orphaned providers accumulate against the account's 50-provider quotas (L-431051DC OAuth2 / L-B04D9A86 API key). Once full, every CUSTOM_JWT deploy fails with:

The number of agent identity Oauth2 credential providers in this account has reached its limit

This broke the E2E Tests (Full Suite) CUSTOM_JWT shard (the account was at exactly 50/50, all E2eHrnsJwt<ts>-oauth), but it leaks the same way for real users — which is why this is fixed in the CLI, not just in the tests.

Changes

• oauth2-credential-provider.ts / api-key-credential-provider.ts — add deleteOAuth2Provider / deleteApiKeyProvider. Idempotent: ResourceNotFound is treated as success. No secret deletion — the managed secret lives in the service-owned bedrock-agentcore-identity! namespace and is cascade-deleted with the provider (verified empirically).
• pre-deploy-identity.ts — add reconcileCredentialProviders: delete providers recorded in deployed state (resources.credentials) that the current spec no longer references, routing by ARN segment (oauth2credentialprovider / apikeycredentialprovider). Best-effort: failures are collected and logged, never thrown. The control client uses maxAttempts: 5 so a throttled/transient delete is retried by the SDK rather than silently left as an orphan.
• deploy/actions.ts (CLI) + useDeployFlow.ts (TUI) — snapshot the prior recorded credentials before state is overwritten, then reconcile:
  • Normal deploy (partial remove + redeploy): reconcile right after deploy — the removed resource is already detached, so its provider deletes cleanly. Deletes just the diff.
  • Teardown deploy (remove all): reconcile after performStackTeardown, so the harness/agent runtime that referenced the provider is already gone (deleting it while the stack still references it can fail). Deletes all recorded providers.
  • Payment teardown cleanup is untouched.
• harness-custom-jwt.test.ts — new regression test asserting the managed <harness>-oauth provider exists while deployed and is gone after teardown. The post-teardown check retries the GetOauth2CredentialProvider call to tolerate read-after-delete eventual consistency. Reverts the earlier test-side band-aid (globalSetup reaper + manual afterAll delete).
• pre-deploy-identity.test.ts — unit tests for reconcileCredentialProviders: OAuth2/ApiKey ARN routing, retained-names skipped, payment ARNs untouched, error collection, teardown-deletes-all.

Safety: imported / user-supplied providers are never deleted

reconcileCredentialProviders operates only on providers recorded in deployed-state.resources.credentials, which is written only from providers the CLI created this/a prior deploy (setup*Providers results). Imported OAuth providers are skipped at creation (no discoveryUrl → status:'skipped', no ARN recorded), and the import flow never writes resources.credentials. So a pre-existing/shared provider the user supplied an ARN for is never reaped. Payment ARNs recorded in the same map fall through to their existing dedicated cleanup.

Related Issue

Closes #1673

Related (CI symptom that surfaced this): #1674

Documentation PR

N/A — bugfix in existing deploy/teardown behavior.

Type of Change

• Bug fix
• New feature
• Breaking change
• Documentation update
• Other (please describe):

Testing

How have you tested the change?

• I ran npm run test:unit and npm run test:integ
• I ran npm run typecheck
• I ran npm run lint
• If I modified src/assets/, I ran npm run test:update-snapshots and committed the updated snapshots

Results:

• typecheck ✅ · lint ✅ (no new warnings) · prettier ✅
• test:unit ✅ 5691 passed (393 files) — includes 6 new reconcileCredentialProviders tests
• test:integ ✅ 314 passed, 1 skipped (32 files)
• Live e2e against the test account: the harness-custom-jwt suite (which exercises deploy → teardown → assert-provider-gone) passed 4 consecutive runs, including a manual repro where the teardown log confirmed Clean up unused credentials → SUCCESS and the provider returned ResourceNotFoundException afterward.

Note on the commit history

The first version of this fix ran the reconcile before stack teardown, which flaked the e2e teardown assertion in CI (a throttled delete against a still-referenced provider was best-effort-swallowed, orphaning it). The follow-up commit reorders the teardown reconcile to run after performStackTeardown, adds SDK retries, and makes the e2e assertion tolerant of eventual consistency. Net diff vs main is the complete, hardened fix.

Checklist

• I have read the CONTRIBUTING document
• I have added any necessary tests that prove my fix is effective or my feature works
• I have updated the documentation accordingly
• I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
• My changes generate no new warnings
• Any dependent changes have been merged and published

---

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the
terms of your choice.

[agentcore-devx-automation]

Claude Security Review: no high-confidence findings. (run)

[github-actions]

Package Tarball

aws-agentcore-0.21.1.tgz

How to install

gh release download pr-1675-tarball --repo aws/agentcore-cli --pattern "*.tgz" --dir /tmp/pr-tarball
npm install -g /tmp/pr-tarball/aws-agentcore-0.21.1.tgz

[agentcore-cli-automation]

LGTM. Clean, well-scoped test-infrastructure fix that mirrors the existing ApiKey cleanup pattern (paginate → filter by prefix + age → delete in parallel). A few things I checked:

• Client lifecycle (global-setup.ts): The new OAuth2 cleanup is placed before the existing finally { bedrockCPClient.destroy() } so the client still gets destroyed exactly once. ✓
• Teardown ordering (harness-custom-jwt.test.ts): The pre-existing afterAll does remove all → deploy (which re-registers the OAuth2 provider as a pre-deploy step). The new explicit deletion runs after that second deploy, so the provider is reliably gone by the end. ✓ Also correctly handles the case where beforeAll fails before any deploy ever ran — deleteOAuth2CredentialProvider swallows errors with a warn.
• Prefix safety: E2eHrnsJwt<ts>-oauth starts with E2e, matching the cleanup filter; no risk of reaping non-e2e providers.
• No telemetry needed — internal e2e test infrastructure, no user-facing surface area changed.

Optional nit (not blocking): e2e-tests/README.md mentions cleanupStaleCredentialProviders() in the cleanup notes; worth a one-line update there to mention OAuth2 too, but happy to defer.

[github-actions]

Coverage Report

Status	Category	Percentage	Covered / Total
🔵	Lines	38%	14104 / 37108
🔵	Statements	37.29%	15023 / 40280
🔵	Functions	32.66%	2426 / 7427
🔵	Branches	31.91%	9388 / 29418

Generated in workflow #3947 for commit 536d04d by the Vitest Coverage Report Action

[agentcore-devx-automation]

Claude Security Review: no high-confidence findings. (run)

[aidandaly24]

Pushed a hardening commit (536d04de) after the first version flaked the e2e teardown assertion in CI:

What happened: the e2e regression test (teardown deletes the managed OAuth2 credential provider) failed once in CI — the provider survived teardown — even though it passed locally. Root cause: the reconcile ran before performStackTeardown, so a transient/throttled DeleteOauth2CredentialProvider (the harness runtime still referencing it, under concurrent e2e shard load) could fail and was best-effort-swallowed, leaving the provider orphaned.

Hardening:

1. On a teardown deploy, reconcile now runs after performStackTeardown (stack/harness gone → no reference) — both CLI and TUI paths.
2. Reconcile control client uses maxAttempts: 5 so throttled deletes are retried, not silently dropped.
3. The e2e assertion now retries the GET to tolerate read-after-delete eventual consistency.

Verification: the originally-flaky harness-custom-jwt e2e test now passes 4 consecutive live runs against the test account; full unit (5691) + integ (314) green.

[agentcore-devx-automation]

Claude Security Review: no high-confidence findings. (run)