Wire persistent DCRCredentialStore into EmbeddedAuthServer

[tgrunnagle]

DRAFT - not ready for review

Summary

• Why: Phase 2 of the DCR story shipped an in-memory stub for the DCRCredentialStore that lived only in the runner package. Restarting (or scaling out to) an authserver dropped every RFC 7591 client registration on the floor and re-registered against the upstream on every boot, which is unworkable for the Datadog-style upstream demo and for any multi-replica deployment. This PR wires the persistent DCRCredentialStore introduced in earlier sub-issues (in-memory + Redis backends) into EmbeddedAuthServer so a Redis-backed authserver reuses already-registered clients across replicas and restarts.
• What:
  • EmbeddedAuthServer.dcrStore is now typed against storage.DCRCredentialStore and is derived from the same storage.Storage value returned by createStorage, so a single storage_type: redis config toggles DCR persistence alongside the rest of authserver state. The storage.Storage interface embeds storage.DCRCredentialStore, promoting the previously-needed runtime type assertion to a compile-time guarantee.
  • The Phase 2 standalone in-memory DCRCredentialStore in pkg/authserver/runner/dcr_store.go is collapsed into a thin storageBackedStore adapter that delegates to storage.DCRCredentialStore and translates DCRResolution <-> DCRCredentials at the boundary. There is now exactly one persistence implementation per backend.
  • The constructor is split into the public NewEmbeddedAuthServer (creates storage) and an unexported newEmbeddedAuthServerWithStorage (owns the cleanup contract). Any error after entry closes the storage backend via a deferred cleanup gated on a named return error, so a crash-looping caller no longer leaks the Redis client connection pool / MemoryStorage cleanup goroutine on every restart.
  • buildPureOAuth2Config remains pure (unchanged signature, no ctx, no I/O); buildUpstreamConfigs is the boundary that consumes the resolver and overlays DCR-resolved credentials onto each upstream.
  • Adds DCRStore() accessor on EmbeddedAuthServer mirroring IDPTokenStorage / UpstreamTokenRefresher, used by integration tests to verify the resolver and the authserver write through the same backend.

This PR also lands the dependency stack that #5185 builds on (the persistent DCRCredentialStore types + memory backend, the Redis backend, the operator CRD surface for DCR, and the runner-side DCR resolver wiring). Each layer was developed and reviewed as a separate commit on this branch; commits are sequenced so each one builds and tests cleanly.

Closes #5185

Type of change

• Bug fix
• New feature
• Refactoring (no behavior change)
• Dependency update
• Documentation
• Other (describe):

Test plan

• Unit tests (task test)
• E2E tests (task test-e2e)
• Linting (task lint-fix)
• Manual testing (describe below)

Notable test coverage added by this PR:

• pkg/authserver/integration_dcr_restart_test.go (new) — TestEmbeddedAuthServer_DCRSurvivesRestart boots an EmbeddedAuthServer against a mock AS, captures the DCR store via the new DCRStore() accessor, closes the server, and asserts the persisted DCR row survives the first server's Close. Lives in package authserver_test to avoid the runner -> authserver import cycle. The full "boot, close, boot again, observe zero /register" scenario across a fresh constructor is documented as a gap (the production Redis path requires Sentinel, which miniredis does not speak); test docstring records the conditions under which it can be closed.
• pkg/authserver/runner/embeddedauthserver_test.go — TestBuildUpstreamConfigs_DCR exercises first-call registration + cache-hit on the second call (zero additional HTTP requests) and asserts the caller's RunConfig.Upstreams slice is never mutated. TestNewEmbeddedAuthServer_ClosesStorageOnError uses a closeTrackingStorage wrapper to verify the deferred-cleanup contract.
• pkg/authserver/storage/memory_test.go, redis_test.go, redis_integration_test.go — coverage for the persistent DCRCredentialStore operations on both backends, including ScopesHash canonicalisation (sort + dedupe + newline join).

API Compatibility

• This PR does not break the v1beta1 API, OR the api-break-allowed label is applied and the migration guidance is described above.

The CRD changes are additive: OAuth2UpstreamConfig.clientId becomes optional with a CEL constraint requiring exactly one of clientId or dcrConfig, and a new dcrConfig field is added. Existing MCPExternalAuthConfig / VirtualMCPServer resources that set clientId continue to validate unchanged.

Does this introduce a user-facing change?

Yes. Operators of OAuth2 upstreams can now configure RFC 7591 Dynamic Client Registration in the operator CRD via dcrConfig (with discoveryUrl or registrationEndpoint, plus optional initialAccessTokenRef, softwareId, softwareStatement) instead of statically configuring clientId + clientSecret. When the authserver is configured with storage_type: redis, DCR registrations persist across restarts and are shared across replicas; in single-replica memory mode, registrations live for the process lifetime as before.

Special notes for reviewers

• This PR is the terminal task in the Phase 3 DCR DAG and pulls along the dependency stack from sub-issues 1 and 2 (persistent DCRCredentialStore types + memory + Redis backends), the operator CRD surface, and the Phase 2 resolver wiring. The size is above the usual 400-line / 10-file limit; each commit is self-contained and the stack reads top-to-bottom in commit order. Reviewers may prefer to walk the per-commit diffs.
• The full "boot, close, boot again, zero /register" cross-constructor restart scenario is not exercised; closing it requires either miniredis-Sentinel emulation or a Docker-based Redis Sentinel cluster in the test harness. The wiring that the second boot would consume — the type of dcrStore being the same storage.DCRCredentialStore that authserver.New writes through — is verified at compile time by storage.Storage embedding storage.DCRCredentialStore and by TestEmbeddedAuthServer_DCRSurvivesRestart asserting the persistence boundary.
• buildPureOAuth2Config was kept intentionally pure (no ctx, no I/O) to preserve the architectural gate established in Authserver DCR integration (Phase 2, Steps 2a-2g) #4978; the wiring change swaps the implementation passed into the resolver, not the call shape.
• No secrets (client_secret, registration_access_token, initial_access_token, refresh tokens) appear as arguments to slog.* calls; the grep assertion from Authserver DCR integration (Phase 2, Steps 2a-2g) #4978 still applies.

[codecov]

Codecov Report

❌ Patch coverage is 78.94737% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 67.89%. Comparing base (df0ad8f) to head (565fade).

Files with missing lines	Patch %	Lines
pkg/authserver/runner/embeddedauthserver.go	56.25%	5 Missing and 2 partials ⚠️
pkg/authserver/runner/dcr_store.go	87.80%	3 Missing and 2 partials ⚠️

Additional details and impacted files

@@                  Coverage Diff                  @@
##           dcr-3b_issue_5184    #5196      +/-   ##
=====================================================
+ Coverage              67.81%   67.89%   +0.07%     
=====================================================
  Files                    610      610              
  Lines                  62379    62414      +35     
=====================================================
+ Hits                   42302    42374      +72     
+ Misses                 16902    16858      -44     
- Partials                3175     3182       +7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.