Skip to content

[BOT ISSUE] Mistral: Beta Conversations API (client.beta.conversations) not instrumented #273

@braintrust-bot

Description

@braintrust-bot

Summary

The Mistral Beta Conversations API (/v1/conversations) is not instrumented. Calls to client.beta.conversations.start(), client.beta.conversations.append(), and their streaming variants produce zero Braintrust tracing. This is a documented beta API on the Mistral platform for multi-turn agentic conversations where the model autonomously executes server-side tools.

The Braintrust Mistral integration instruments client.agents.complete() (agent-configured chat completions), but the Conversations API is a distinct, higher-level agentic execution surface where Mistral manages conversation state server-side and autonomously executes tools including code interpreter, web search, image generation, and document library.

This is the Mistral equivalent of the Anthropic Managed Agents API tracked in #259 — both are beta, server-side agentic execution surfaces from major providers that produce zero tracing today.

What is missing

Mistral Resource Method Instrumented?
client.chat complete(), stream() Yes
client.agents complete(), stream() Yes
client.beta.conversations start(), start_stream() No
client.beta.conversations append(), append_stream() No
client.beta.conversations restart(), restart_stream() No
client.beta.conversations get(), list(), delete(), get_history(), get_messages() No (CRUD — lower priority)

Why this matters

Conversations is an agentic execution surface, not a simple chat API. When a conversation runs:

  • The model autonomously decides which tools to invoke (code interpreter, web search, image generation, document library, custom functions)
  • Tool execution happens server-side — results are returned as part of the conversation outputs
  • Responses include outputs arrays with message entries, tool execution entries, function calls, and agent handoffs
  • Token usage is reported via ConversationUsageInfo
  • Streaming is supported for real-time output delivery

This is functionally equivalent to the agentic frameworks already instrumented in this repo (Anthropic Managed Agents, Pydantic AI agents, Agno agents, Google ADK, OpenAI Agents SDK, Claude Agent SDK).

Minimum instrumentation

At minimum, start() and append() (and their streaming variants) should create spans capturing:

  • Conversation-level span (type task): conversation ID, model, total duration, aggregate token usage
  • Output detail: message content, tool execution results, function call results
  • Tool spans: child spans for server-side tool executions (code interpreter, web search, image generation, document library)
  • Metrics: token usage from ConversationUsageInfo, time-to-first-token for streaming
  • Metadata: model, tools configuration, conversation ID, guardrails config

Braintrust docs status

not_found — The Mistral integration page documents chat completions, embeddings, FIM, and agents. No mention of the Conversations API.

Upstream sources

  • Mistral Conversations API endpoints: https://docs.mistral.ai/api/endpoint/beta/conversations
  • Mistral Agents & Conversations guide: https://docs.mistral.ai/agents/agents
  • Supported tools: web search, web search premium, code interpreter, image generation, document library, custom function tools
  • Python SDK: client.beta.conversations.start(), .append(), .restart(), and streaming variants
  • Beta status: accessible to all API users, documented in the official API reference under the Beta section

Local files inspected

  • py/src/braintrust/integrations/mistral/patchers.py — defines patchers for Chat, Embeddings, Fim, Agents; zero references to conversation, conversations, or beta
  • py/src/braintrust/integrations/mistral/tracing.py — wrapper functions for chat, embeddings, FIM, agents only; no conversation wrappers
  • py/src/braintrust/integrations/mistral/integration.py — integration class registers 4 composite patchers; no ConversationsPatcher
  • py/src/braintrust/integrations/mistral/test_mistral.py — no conversation test cases
  • py/noxfile.pytest_mistral session tests against LATEST and 1.12.4; no conversations coverage

Relationship to existing issues

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions