Skip to content

feat(bookstack-agent): stream tokens in real time instead of buffering#101

Merged
amrit110 merged 2 commits into
mainfrom
feat/bookstack-agent-real-streaming
Jun 16, 2026
Merged

feat(bookstack-agent): stream tokens in real time instead of buffering#101
amrit110 merged 2 commits into
mainfrom
feat/bookstack-agent-real-streaming

Conversation

@amrit110

Copy link
Copy Markdown
Member

Problem

The BookStack agent response was loading in one shot — the user saw nothing until the entire LLM response was ready, then everything appeared at once.

Root cause: ask_stream() was silently consuming the entire Anthropic streaming response into memory (accumulating all tokens, waiting for get_final_message()), and only then emitting events. The character-by-character loop at the end was fast synchronous Python iteration over an already-complete string, so all text_chunk events flushed together in a single TCP write.

Fix

  • src/aieng_bot/bookstack/agent.py: Yield text_chunk events as tokens arrive from the streaming API instead of buffering. When a content_block_start for a tool_use block is detected and text was already streamed (reasoning/planning text from the model), emit text_clear to tell the UI to discard it. The answer event is emitted at the end as the authoritative complete text.
  • tests/bookstack/test_agent.py: Add _make_tool_use_block_start_event helper and a new test test_stream_text_clear_emitted_when_text_precedes_tool_call covering the discard path.

The UI already handled both text_chunk and text_clear events — no frontend changes needed.

Test plan

  • All 12 unit tests pass (pytest tests/bookstack/test_agent.py)
  • Manually verify the chat UI streams the response token-by-token
  • Verify tool steps still appear correctly before the final answer
  • Verify reasoning/thinking text is discarded (shows text_clear) when the model calls a tool

@amrit110 amrit110 merged commit e237d21 into main Jun 16, 2026
9 checks passed
@amrit110 amrit110 deleted the feat/bookstack-agent-real-streaming branch June 16, 2026 16:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant