Conversation
elias-ba
changed the title
elias-ba
changed the title
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #4517 +/- ##
==========================================
+ Coverage 89.55% 89.60% +0.05%
==========================================
Files 425 425
Lines 20307 20391 +84
==========================================
+ Hits 18185 18271 +86
+ Misses 2122 2120 -2 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Add real-time streaming support for AI assistant responses and apply structured changes (workflow YAML, job code) before text finishes streaming. Backend: - Handle SSE streaming events from Apollo (text chunks, status, changes) - Broadcast streaming events via PubSub to ai_session topics - Forward streaming events through Phoenix Channel to WebSocket clients - Handle Mint transport errors gracefully during SSE streaming - Fix message_processor crash on non-map error tuples Frontend: - Add streaming state (content, status, changes) to AI assistant store - Register streaming event handlers in AIChannelRegistry - Render streaming content in MessageList during AI response - Auto-apply workflow YAML and auto-preview job code when streaming changes arrive, before text finishes streaming - Deduplicate auto-apply to prevent double-application when final message arrives
elias-ba
force-pushed
the
3585-ai-assistant-streaming
branch
from
4f51faa to
c9e2e5a
Compare
Cover SSE event handling, PubSub broadcasting, channel forwarding, Apollo client streaming endpoints, catch :exit path, and edge cases for invalid JSON in complete and changes events.
elias-ba
force-pushed
the
3585-ai-assistant-streaming
branch
from
47f1dbf to
4fd08ac
Compare
Wire up the streaming_error channel event in AIChannelRegistry so errors during SSE streaming clear the UI state instead of being silently dropped. Extract shared test helpers (streaming_or_sync_response, stub_ai_with_health_check) to reduce mock duplication across 5 test files.
|
@elias-ba this is looking pretty good (and I agree that it works better with your apollo changes, even if I have some concerns about the changes themselves). The big thing I have here is that the streaming text comes in really really fast, and in really big chunks. It's stressful and kinda hard to read. Do we need to buffer the response from the model, tokenise it, and then render token by token on a throttle? The worry with that of course is that the markdown will render all weird. It's difficult that we have to both render text smoothly, but also incrementally render markdown. Ideally we'd background-generate a markdown block, then incrementally render the html output. Very happy to cut cloth - maybe the current rendering is the most appropriate place to release. Just want to thoroughly discuss options and costs before moving forward. |
2 participants
Description
This PR adds real-time streaming support for AI assistant responses. Instead of waiting for the full response to complete, users now see text stream in word-by-word. Structured changes (workflow YAML, job code diffs) are applied to the canvas/editor before the text explanation finishes streaming.
Previously, the AI would generate text first and code/YAML second, so users would read "I've updated your workflow..." and only see the changes seconds later. With a companion Apollo PR (OpenFn/apollo#407) that reverses the generation order, Lightning now receives a
changesSSE event early in the stream containing the resolved code or YAML, applies it immediately, and then streams the text explanation.The backend handles SSE streaming events from Apollo (text chunks, status updates, structured changes), broadcasts them via PubSub, and forwards them through Phoenix Channels to WebSocket clients. It also fixes a Mint transport error crash during SSE streaming and a message_processor crash on non-map error tuples.
The frontend adds streaming state to the AI assistant store, registers event handlers in AIChannelRegistry, renders streaming content in the message list, and auto-applies workflow YAML / auto-previews job code diffs as soon as streaming changes arrive, with deduplication to prevent double-application when the final message lands.
Closes #3585
Validation steps
Additional notes for the reviewer
This PR depends on a companion Apollo PR (OpenFn/apollo#407) which reverses the generation order and introduces the new
changesSSE event. To validate this work, you'll need Apollo running locally on thetweaks-for-streamingbranch (or a deployed version with that branch merged). Without the Apollo changes, streaming will still work but structured changes won't arrive early. They'll only appear when the final message completes, as before.AI Usage
Pre-submission checklist
/reviewwith Claude Code)
(e.g.,
:owner,:admin,:editor,:viewer)