Skip to content

Tune Claude Code CLI retry/timeout resilience env#46

Open
mkonopelski-gd wants to merge 4 commits into
mainfrom
fix/claude-sdk-retries
Open

Tune Claude Code CLI retry/timeout resilience env#46
mkonopelski-gd wants to merge 4 commits into
mainfrom
fix/claude-sdk-retries

Conversation

@mkonopelski-gd

Copy link
Copy Markdown
Contributor

Summary

  • Forward API_TIMEOUT_MS (120s), CLAUDE_CODE_MAX_RETRIES (3), and CLAUDE_ASYNC_AGENT_STALL_TIMEOUT_MS (120s) to the Claude Code CLI subprocess via a new setup_claude_code_resilience_env(), wired into agent_query's env alongside the existing setup_claude_code_* helpers. Goal: let the CLI itself retry past a transient network blip instead of burning a phase-level resume attempt (or the whole phase) in agent_query_with_resume.
  • Fills in the previously-empty __main__ with a self-contained local debug entrypoint (throwaway git workspace with a local origin, redirected AGENT_LOGS_BASE_PATH/WORKSPACE_BASE_PATH/CLAUDE_CODE_TMPDIR_PATH, .env-backfilled provider key) plus a retry-forcing experiment (_debug_retry_experiment, run via DEBUG_RETRY_EXPERIMENT=1) used to verify the retry mechanism end-to-end.

How this was tested

  • Real 2-phase debug run (python -m app.services.claude_code): generated a working Express app end-to-end against a throwaway workspace — confirms the harness plumbing (workspace/cache/log paths, provider key resolution) works outside docker-compose.
  • Retry verification: _debug_retry_experiment runs a single agent_query with a deliberately low API_TIMEOUT_MS (500ms, far below any real response time) and CLAUDE_CODE_MAX_RETRIES=5, forcing every attempt to time out. At DEBUG log level this produced exactly 5 api_retry SystemMessages (attempt: 1..5, max_retries: 5) over ~21s before the CLI gave up — direct, first-party confirmation that CLAUDE_CODE_MAX_RETRIES is honored by the SDK.
  • OpenRouter corroboration: the OpenRouter Activity log showed 6 request rows in the same ~90s window the two experiment runs finished in (timestamps cross-checked against local log file mtimes). This corroborates real requests hit the API in that window; note I could not fully reconcile the exact per-row token counts (4 rows ~304 input tokens, 2 rows ~26,946) against individual retry attempts, so treat this as supporting evidence alongside the agent-log proof above, not an exact accounting.
  • make check (ruff clean; the 3 remaining mypy errors are pre-existing, in unrelated files not touched by this change) and make unit-tests (658 passed) both green.

Test plan

  • Review CLAUDE_CODE_MAX_RETRIES=3 / API_TIMEOUT_MS=120000 / CLAUDE_ASYNC_AGENT_STALL_TIMEOUT_MS=120000 as the intended production defaults
  • Confirm no other call site relies on the CLI's previous default retry/timeout behavior

…ebug entrypoint

Forward API_TIMEOUT_MS, CLAUDE_CODE_MAX_RETRIES, and
CLAUDE_ASYNC_AGENT_STALL_TIMEOUT_MS to the Claude Code CLI subprocess so a
transient network blip is retried by the CLI itself before it burns a
phase-level resume attempt (or the whole phase) in agent_query_with_resume.

Also fills in the previously-empty __main__ with a self-contained local debug
entrypoint (throwaway git workspace, redirected cache/log/tmpdir settings,
.env-backfilled provider key) plus a retry-forcing experiment used to verify
the new env values are honored by the CLI end-to-end.
@mkonopelski-gd

Copy link
Copy Markdown
Contributor Author

@awrobel-gd This actually doesn't make sense. Because are way lower than default which I finally found (https://code.claude.com/docs/en/env-vars)

API_TIMEOUT_MS: 600000 (10 minutes).
CLAUDE_CODE_MAX_RETRIES: 10.
CLAUDE_ASYNC_AGENT_STALL_TIMEOUT_MS: 600000 (10 minutes).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants