Mid-session backend switching: stateless redesign + subscription-friendly wrapper#26
Open
aattaran wants to merge 4 commits into
Open
Mid-session backend switching: stateless redesign + subscription-friendly wrapper#26aattaran wants to merge 4 commits into
aattaran wants to merge 4 commits into
Conversation
Two related crash fixes in the model-proxy:
1. proxyRes error handler. When the backend drops the connection
mid-stream (its own timeout, restart, network glitch), Node's
EventEmitter throws 'Unhandled error' on the response stream and
crashes the proxy. Add an explicit handler that destroys the client
socket cleanly.
2. SSE-safe error response path in proxyReq.on('error'). The previous
logic always wrote 'Upstream connection error' as JSON, even when
headers had already been sent for an SSE response. Appending JSON to
an SSE stream corrupts it and crashes Claude Code's SSE parser. Now:
- if !headersSent: write a clean 502 JSON response
- else (likely SSE in progress): destroy the socket without writing
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The proxy's /v1/messages handler now dispatches requests to ONE of three
per-path processor functions based on state.mode. Each processor owns the
full request-side transformation for its path; there is no shared
sanitization logic and no scattered if (state.mode === ...) branches.
processAnthropicRequest (PATH A, state.mode === 'anthropic')
Existing pre-patch behavior, isolated:
- hadNonAnthropicSession=true: strip ALL thinking blocks
- hadNonAnthropicSession=false: strip only UNSIGNED thinking blocks
No model remap. No response-side touching.
processDeepseekRequest (PATH B, state.mode === 'deepseek')
Apply MODEL_REMAP['deepseek'] to request `model` field. Pass everything
else through unchanged — including prior thinking blocks in the messages
array. DeepSeek's Anthropic-compat endpoint REQUIRES prior thinking
blocks on multi-turn thinking continuations (returns 400 with
"content[].thinking ... must be passed back to the API" otherwise).
processOtherBackendRequest (PATH C, openrouter | fireworks | _single)
Apply MODEL_REMAP[mode] if defined. Strip ALL thinking blocks
(existing pre-patch behavior — these backends are assumed to reject
foreign-signed thinking). If a future backend turns out to need
pass-through like DeepSeek does, promote it to its own dedicated path
rather than weakening this one.
Trade-off
---------
The JSONL Claude Code persists during a DeepSeek session contains
foreign-signed thinking blocks and the deepseek-v4-pro model name. By
design — a DeepSeek session is a DeepSeek session. To resume that session
under Anthropic, route through the deepclaude proxy in 'anthropic' mode;
the existing hadNonAnthropicSession strip in PATH A scrubs the request
body cleanly before forwarding to api.anthropic.com. Plain 'claude --resume'
of a DeepSeek-touched session bypassing the proxy will 400 against
Anthropic — that is intentional path isolation, not a bug.
Validation
----------
- Director-mode review (fresh pr-reviewer agent, no shared context):
SHIP IT. Confirmed three paths are pure functions, dispatcher reads
state.mode once, all infrastructure preserved (UsageNormalizer,
normalizeJsonBody, proxyRes/proxyReq error handlers, /_proxy/* control
endpoints, switchMode, cost tracking, port retry, Origin check,
path-overlap stripping), no cross-path state coupling.
- Multi-turn DeepSeek-thinking VM test (api.deepseek.com via proxy):
Turn 1: 200 OK, DeepSeek emitted 1 thinking content_block + 99 deltas
Turn 2: 200 OK with prior assistant turn including thinking + text
in messages array. Zero "must be passed back" errors. 89
text deltas. Reproduces the user's TUI scenario.
- node --check: clean parse, ESM.
Net change: +142 / -37 vs commit 8543afc.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Completes the path-isolation story for auth headers. Previously the proxy substituted client auth on outbound requests for non-anthropic modes (injecting the per-backend key) but left anthropic-mode requests with the client's auth header passed through unchanged. That meant Claude Code had to be started with ANTHROPIC_AUTH_TOKEN already set to a valid Anthropic key — and the deepclaude wrappers always set it to the DeepSeek key instead. Result: switching to anthropic mode mid-session 401'd because api.anthropic.com received the DeepSeek key. This commit reads ANTHROPIC_API_KEY from process.env at proxy startup. When in anthropic mode, the proxy substitutes that key as `x-api-key` on outbound requests (Anthropic's auth header convention; NOT Bearer, which is reserved for OAuth bridge tokens). Same auth-substitution path that already exists for DeepSeek/OpenRouter/Fireworks now applies to Anthropic too — hence "true" path isolation. If ANTHROPIC_API_KEY is unset, the proxy falls back to passthrough (client's auth header forwarded unchanged). This preserves the original OAuth-bridge mode where Claude Code holds a session token via `claude /login` and the proxy doesn't need to hold an Anthropic key. The auth-substitution gate is now `MODEL_PATHS.includes(urlPath) && state.apiKey` — uniform across all modes — replacing the previous `isModelCall` gate which excluded anthropic mode by design. Validation ---------- End-to-end VM test with auth substitution: - deepseek mode + deepseek client auth → 200 (substituted: same) - anthropic mode + wrong (deepseek) client auth → 200 (substituted: anthropic) - deepseek mode + wrong (anthropic) client auth → 200 (substituted: deepseek) - anthropic mode + garbage client auth → 200 (substituted: anthropic) All four scenarios return 200. Claude Code can start with ANY auth token (or none) and seamlessly switch backends via /_proxy/mode without restarting. Operational note ---------------- For seamless mid-session switching, both keys must be in the proxy's process env at startup. Recommended setup: export DEEPSEEK_API_KEY=sk-... export ANTHROPIC_API_KEY=sk-ant-api03-... node proxy/start-proxy.js https://api.deepseek.com/anthropic "$DEEPSEEK_API_KEY" The wrappers (deepclaude.ps1 / deepclaude.sh) currently only export DEEPSEEK_API_KEY. A follow-up should teach them to also export ANTHROPIC_API_KEY when present, so anthropic mode "just works" in remote-control flows too. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…pper
PROXY (proxy/model-proxy.js):
- Replace stateful hadXSession flags with stateless per-request body inspection
- Per-path processors (PATH A anthropic, B deepseek, C other) classify thinking
blocks by signature shape and strip foreign blocks per request
- Add post-strip invariant: if any assistant turn lacks a thinking block,
set thinking={type:'disabled'} and remove output_config (DS rejects
disabled+effort combo). Eliminates the "content[].thinking must be
passed back" 400 class on cross-mode round-trips.
- Add [1m] tier-suffix strip for Claude Max under PATH A
- Strip accept-encoding upstream to prevent UsageNormalizer corrupting gzip
- switchMode refuses anthropic when no ANTHROPIC_API_KEY (clean error msg)
WRAPPERS (deepclaude.sh + deepclaude.ps1):
- Subscription-friendly path 2: spawn proxy + ANTHROPIC_BASE_URL=proxy, but
do NOT force ANTHROPIC_AUTH_TOKEN. Proxy substitutes the backend key per
request, so subscription OAuth flows through unchanged.
- Fix port-extraction bug (longstanding): old code's head -1 / Select-Object
-First 1 grabbed the proxy banner instead of the port number. Now matches
numeric-only line.
- Auto-mode-set after spawn: legacy startup defaults to anthropic; wrapper
now curls /_proxy/mode to switch to chosen backend.
- Use canonical Claude model names (claude-opus-4-7 etc) instead of
backend-native; proxy MODEL_REMAP forward-maps. Makes mid-session
/anthropic /deepseek /openrouter /fireworks switches all work.
- Persistent /tmp/proxy.log (or %TEMP%\deepclaude-proxy.log) for diagnostics.
- Resolve ANTHROPIC_API_KEY from User scope + warn if missing.
README.md:
- Document new env vars table (auth-substitution model)
- Add ANTHROPIC_API_KEY prerequisite for /anthropic mid-session switching
- Note path-isolation rule and continuity-gap fix
Validated on Linux VM:
- 33/33 unit tests pass (test-cross-switch + test-deferred-set + test-stateless)
- 28/28 live TUI steps clean (6 cross-switches, 0x 400 errors)
- Mid-session DS<->Anthropic round-trips work under both API-key and
Claude Max subscription auth (with ANTHROPIC_API_KEY set)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced May 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes the
content[].thinking ... must be passed back to the API400-error class that previously fired on everyanthropic → deepseekround-trip in mid-session switching. Adds subscription-friendly wrapper mode so Claude Max users can route cheap-backend traffic through the proxy without giving up their subscription auth.Validated end-to-end on a Linux GCE VM:
Commits (4 ahead of main)
8543afcfix: handle proxyRes mid-stream errors + SSE-safe error responsedef324efeat: two strictly isolated request paths for model-call dispatch (PATH A anthropic, B deepseek, C other — no shared sanitization branches)a6f5ff6feat: anthropic mode auth substitution viaANTHROPIC_API_KEYenv var (proxy substitutes the right backend key per request, so client auth is irrelevant)a7814bafix: the core fix — stateless thinking-block classification + post-strip invariant + subscription-friendly wrapperKey design decisions
1. Stateless body normalization (replaces the prior
state.hadXSessionflags). Each per-path processor inspectsparsed.messagesper request and classifies thinking blocks by signature shape. The proxy is a stateless router; trying to track cross-request flags in it diverged frommessages[]whenever the proxy restarted, was resumed, or saw a turn from the bridge.2. Post-strip invariant (
processDeepseekRequest):This catches the case where Anthropic's
adaptivethinking-mode emitted a tool-use turn without a thinking block, then conversation comes back to DeepSeek which enforces per-turn continuity (per DeepSeek docs).3. Subscription-friendly wrapper path 2: stop forcing
ANTHROPIC_AUTH_TOKENto the backend's API key. Let the user's existing auth (subscription OAuth orANTHROPIC_API_KEY) flow through; the proxy substitutes the right backend key per request.4. Wrapper port-extraction fix: longstanding bug where
head -1/Select-Object -First 1grabbed the proxy banner instead of the port number. Now matches numeric-only line.Test plan
Files changed in latest commit
proxy/model-proxy.js(stateless redesign, +178/-46)deepclaude.sh(subscription-friendly path 2, port-fix, auto-mode-set, +66/-19)deepclaude.ps1(parity with.sh, +153/-74)README.md(env-vars table, mid-session prerequisites, +30/-6)🤖 Generated with Claude Code