Skip to content

perf: fix chat timeouts during tool calls, reduce latency and UI jank#1

Merged
karansinghgit merged 5 commits intomainfrom
perf/chat-optimizations
Mar 20, 2026
Merged

perf: fix chat timeouts during tool calls, reduce latency and UI jank#1
karansinghgit merged 5 commits intomainfrom
perf/chat-optimizations

Conversation

@karansinghgit
Copy link
Copy Markdown
Owner

Summary

  • Fix watchdog timeout during tool execution — the 30s watchdog never reset while tools ran, causing "Model response timed out" on web search flows. Now resets on every delta and tool_status event. Increased to 60s for multi-round loops.
  • Parallelize pre-flight operations — system prompt build + Brave key fetch now run concurrently via Promise.all. Removes duplicate getSettings() call.
  • Stop writing settings.json on every read — getSettings() was doing a read+write cycle every call; now read-only.
  • Cache context provider instances — TodoContextProvider and NotesContextProvider created once in constructor instead of re-allocated on every access.
  • Reduce tool timeouts — fetch_url 15s→8s, web_search 10s→6s to reduce worst-case silence.
  • Memoize remarkPlugins array — prevents React re-render waste.
  • Debounce scroll-into-view — 80ms debounce prevents DOM thrashing during fast streaming.

Test plan

  • Ask Robin to search the web (e.g., "what's the weather in London") — should NOT time out
  • Verify streaming still scrolls smoothly
  • Verify settings load correctly after storage read-only change
  • Verify tool toggles still work in Settings

🤖 Generated with Claude Code

- Run system prompt build + Brave key fetch in parallel (Promise.all)
- Remove duplicate getSettings() call; reuse settings from earlier read
- Cache TodoContextProvider and NotesContextProvider instances instead
  of re-creating on every access via getter
- Move [remarkGfm] array to module-level constant to prevent
  re-allocation on every render (was defeating React memo)
- Debounce scrollIntoView with 80ms timer to prevent DOM thrashing
  during fast delta streaming (was firing every animation frame)
The 30s watchdog never reset while tools were executing, causing
"Model response timed out" errors during web search flows that
take 30-75s across multiple tool rounds.

Now resets on every onDelta and onToolStatus event. Increased
timeout from 30s to 60s to accommodate multi-round tool loops.
@karansinghgit karansinghgit merged commit 0b95772 into main Mar 20, 2026
2 checks passed
@karansinghgit karansinghgit deleted the perf/chat-optimizations branch March 20, 2026 19:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant