Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions docs/edge/en/concepts/streaming.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,35 @@ Frames are grouped into high-level channels:

The stream itself remains one ordered timeline. Channel projections let consumers focus on only part of that timeline.

### Tool Call Streaming

Tool calls appear in two places because there are two distinct phases:

| Phase | Channel | Event type | Meaning |
|-------|---------|------------|---------|
| Tool-call construction | `llm` | `llm_stream_chunk` | The model is streaming the tool name or arguments it intends to call |
| Tool execution | `tools` | `tool_usage_started`, `tool_usage_finished`, `tool_usage_error` | CrewAI is executing the tool and reporting the result |

For streamed tool-call arguments, the latest provider delta is available as `frame.data["chunk"]`. The accumulated argument string is available at `frame.data["tool_call"]["function"]["arguments"]`.

```python
with llm.stream_events("Check the weather in Paris.", tools=[weather_tool]) as stream:
for frame in stream.llm:
if frame.type != "llm_stream_chunk":
continue

tool_call = frame.data.get("tool_call")
if tool_call:
function = tool_call["function"]
print("Tool:", function["name"])
print("Latest argument delta:", frame.data["chunk"])
print("Arguments so far:", function["arguments"])
elif frame.content:
print(frame.content, end="", flush=True)
```

Use the `tools` channel when you want to display that a tool actually started, completed, or failed. Use the `llm` channel when you want to observe the model constructing the tool call before execution.

```mermaid
flowchart LR
A["flow<br/>flow_started"] --> B["llm<br/>llm_call_started"]
Expand Down
32 changes: 32 additions & 0 deletions docs/edge/en/learn/consuming-streams.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,38 @@ result = stream.result

`frame.event` is the structured payload for the source event. Use it for metadata such as tool names, arguments, message roles, and runtime identifiers.

## Print Streamed Tool Call Arguments

When a model supports streamed tool calling, CrewAI emits partial tool-call arguments as `llm_stream_chunk` frames on the `llm` channel. These frames are different from tool execution events:

- `llm` channel: the model is constructing the tool call.
- `tools` channel: CrewAI is executing the tool.

The current delta is stored in `frame.event["chunk"]`. The accumulated arguments are stored in `frame.event["tool_call"]["function"]["arguments"]`.

```python
with llm.stream_events("Get the weather in Paris.", tools=[weather_tool]) as stream:
for frame in stream.interleave(["llm", "tools"]):
if frame.channel == "llm" and frame.type == "llm_stream_chunk":
tool_call = frame.event.get("tool_call")
if tool_call:
function = tool_call["function"]
print(f"\nPreparing tool: {function['name']}")
print(f"Arguments so far: {function['arguments']}")
elif frame.content:
print(frame.content, end="", flush=True)

elif frame.channel == "tools" and frame.type == "tool_usage_started":
print(f"\nRunning tool: {frame.event.get('tool_name')}")

elif frame.channel == "tools" and frame.type == "tool_usage_finished":
print(f"\nTool finished: {frame.event.get('tool_name')}")

result = stream.result
```

Some providers emit arguments in small JSON fragments. For display, prefer the accumulated argument string over the latest delta.

## Watch Flow Progress

Flow lifecycle and method execution frames arrive on the `flow` channel:
Expand Down
9 changes: 6 additions & 3 deletions docs/edge/en/learn/streaming-crew-execution.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -240,15 +240,18 @@ for chunk in streaming:

### TOOL_CALL Chunks

Information about tool calls being made:
Information about tool calls being made. Depending on the provider, CrewAI may receive tool-call arguments incrementally. In that case, each `TOOL_CALL` chunk contains the latest streamed content in `chunk.content`, while `chunk.tool_call.arguments` contains the accumulated argument string so far.

```python Code
for chunk in streaming:
if chunk.chunk_type == StreamChunkType.TOOL_CALL:
print(f"\nCalling tool: {chunk.tool_call.tool_name}")
print(f"Arguments: {chunk.tool_call.arguments}")
print(f"Latest argument delta: {chunk.content}")
print(f"Arguments so far: {chunk.tool_call.arguments}")
```

Actual tool execution is reported separately through tool usage events in the runtime event stream and through CrewAI's verbose console output. `TOOL_CALL` chunks represent the model constructing the tool request before or during execution.

## Practical Example: Building a UI with Streaming

Here's a complete example showing how to build an interactive application with streaming:
Expand Down Expand Up @@ -381,4 +384,4 @@ except Exception as e:
print("Streaming completed but an error occurred")
```

By leveraging streaming, you can build more responsive and interactive applications with CrewAI, providing users with real-time visibility into agent execution and results.
By leveraging streaming, you can build more responsive and interactive applications with CrewAI, providing users with real-time visibility into agent execution and results.
27 changes: 27 additions & 0 deletions docs/edge/en/learn/streaming-runtime-contract.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,33 @@ result = stream.result

`llm.stream_events(...)` temporarily enables streaming for the wrapped call and restores the LLM's previous `stream` setting afterward. Provider integrations continue to emit the underlying LLM stream events; this helper provides a common iterator API over those events for every LLM provider.

### Tool Call Argument Deltas

Native tool-call streaming uses `llm_stream_chunk` frames on the `llm` channel. This represents the model constructing a tool call. It is separate from the `tools` channel, which represents CrewAI executing that tool.

For a streamed tool call, the frame payload includes:

```python
frame.type == "llm_stream_chunk"
frame.channel == "llm"
frame.event["call_type"] == "tool_call"
frame.event["chunk"] # latest argument delta from the provider
frame.event["tool_call"] # accumulated tool-call state
```

The `tool_call` payload follows the OpenAI-style function call shape:

```python
tool_call = frame.event["tool_call"]
tool_call["id"]
tool_call["index"]
tool_call["type"] # "function"
tool_call["function"]["name"]
tool_call["function"]["arguments"] # accumulated JSON argument string
```

Providers differ in granularity. OpenAI and Anthropic may stream arguments as small JSON fragments, while some providers emit the complete argument object in one chunk. Consumers should treat `frame.event["chunk"]` as the latest delta and `tool_call["function"]["arguments"]` as the current accumulated state.

## Conversational Turns

Conversational Flows can stream one user turn with `stream_turn()`:
Expand Down
8 changes: 8 additions & 0 deletions lib/crewai/src/crewai/events/event_listener.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@
LLMCallCompletedEvent,
LLMCallFailedEvent,
LLMCallStartedEvent,
LLMCallType,
LLMStreamChunkEvent,
)
from crewai.events.types.llm_guardrail_events import (
Expand Down Expand Up @@ -455,6 +456,13 @@ def on_llm_call_failed(_: Any, event: LLMCallFailedEvent) -> None:

@crewai_event_bus.on(LLMStreamChunkEvent)
def on_llm_stream_chunk(_: Any, event: LLMStreamChunkEvent) -> None:
if event.call_type == LLMCallType.TOOL_CALL:
self.formatter.handle_llm_stream_chunk(
event.tool_call.function.arguments if event.tool_call else "",
event.call_type,
)
return

self.text_stream.write(event.chunk)
self.text_stream.seek(self.next_chunk)
self.text_stream.read()
Expand Down
29 changes: 18 additions & 11 deletions lib/crewai/src/crewai/events/utils/console_formatter.py
Original file line number Diff line number Diff line change
Expand Up @@ -437,6 +437,8 @@ def handle_tool_usage_started(
if not self.verbose:
return

self.pause_live_updates()

with self._tool_counts_lock:
self.tool_usage_counts[tool_name] = (
self.tool_usage_counts.get(tool_name, 0) + 1
Expand Down Expand Up @@ -468,6 +470,8 @@ def handle_tool_usage_finished(
if not self.verbose:
return

self.pause_live_updates()

with self._tool_counts_lock:
iteration = self.tool_usage_counts.get(tool_name, 1)

Expand Down Expand Up @@ -495,6 +499,8 @@ def handle_tool_usage_error(
if not self.verbose:
return

self.pause_live_updates()

with self._tool_counts_lock:
iteration = self.tool_usage_counts.get(tool_name, 1)

Expand Down Expand Up @@ -543,6 +549,15 @@ def handle_llm_stream_chunk(
if should_suppress_console_output():
return

from crewai.events.types.llm_events import LLMCallType

if call_type == LLMCallType.TOOL_CALL:
self.pause_live_updates()
self._is_streaming = False
self._last_stream_call_type = call_type
self._just_streamed_final_answer = False
return

self._is_streaming = True
self._last_stream_call_type = call_type

Expand All @@ -554,17 +569,9 @@ def handle_llm_stream_chunk(
display_text = "...\n" + display_text

content = Text()

from crewai.events.types.llm_events import LLMCallType

if call_type == LLMCallType.TOOL_CALL:
content.append(display_text, style="yellow")
title = "🔧 Tool Arguments"
border_style = "yellow"
else:
content.append(display_text, style="bright_green")
title = "✅ Agent Final Answer"
border_style = "green"
content.append(display_text, style="bright_green")
title = "✅ Agent Final Answer"
border_style = "green"

streaming_panel = Panel(
content,
Expand Down
34 changes: 34 additions & 0 deletions lib/crewai/src/crewai/llms/base_llm.py
Original file line number Diff line number Diff line change
Expand Up @@ -685,6 +685,40 @@ def _emit_stream_chunk_event(
),
)

def _emit_tool_call_stream_chunk_event(
self,
*,
chunk: str,
tool_call_id: str | None,
tool_name: str | None,
arguments: str,
index: int,
from_task: Task | None = None,
from_agent: BaseAgent | None = None,
response_id: str | None = None,
) -> None:
"""Emit a normalized streamed tool-call chunk.

``chunk`` is the latest provider delta while ``arguments`` is the
accumulated argument string for consumers that want current state.
"""
self._emit_stream_chunk_event(
chunk=chunk,
from_task=from_task,
from_agent=from_agent,
tool_call={
"id": tool_call_id,
"function": {
"name": tool_name or "",
"arguments": arguments,
},
"type": "function",
"index": index,
},
call_type=LLMCallType.TOOL_CALL,
response_id=response_id,
)

def _emit_thinking_chunk_event(
self,
chunk: str,
Expand Down
Loading
Loading