Skip to content

[bot] OpenAI and Anthropic tracers do not set braintrust.span_attributes with type: "llm" #69

@braintrust-bot

Description

@braintrust-bot

Summary

The OpenAI and Anthropic middleware tracers do not set the braintrust.span_attributes OTel attribute with type: "llm" on their spans. Five other integrations in this SDK (GenAI, Genkit, LangChainGo, Eino, ADK) correctly set this attribute. According to Braintrust's advanced tracing docs, span_attributes.type is what enables the LLM icon, LLM duration metrics, and the "Try prompt" feature in the Braintrust UI.

This means spans from the two most commonly used provider integrations lack proper LLM classification in the Braintrust UI, while less-used integrations display correctly.

What is missing

1. OpenAI Chat Completions tracer — no braintrust.span_attributes

trace/contrib/openai/chatcompletions.go sets braintrust.input_json, braintrust.metadata, braintrust.output_json, and braintrust.metrics, but never calls:

internal.SetJSONAttr(span, "braintrust.span_attributes", map[string]string{"type": "llm"})

2. OpenAI Responses tracer — no braintrust.span_attributes

trace/contrib/openai/responses.go similarly sets input/output/metadata/metrics but never sets braintrust.span_attributes.

3. Anthropic Messages tracer — no braintrust.span_attributes

trace/contrib/anthropic/messages.go follows the same pattern — no braintrust.span_attributes is set.

4. Other integrations set it correctly

Integration Sets span_attributes.type: "llm"? File
Google GenAI Yes trace/contrib/genai/generatecontent.go:117
Firebase Genkit Yes trace/contrib/genkit/tracegenkit.go:77
LangChainGo Yes trace/contrib/langchaingo/tracelangchaingo.go:207,230
CloudWeGo Eino Yes trace/contrib/cloudwego/eino/traceeino.go:120
Google ADK Yes trace/contrib/adk/traceadk.go:326
OpenAI (chat) No trace/contrib/openai/chatcompletions.go
OpenAI (responses) No trace/contrib/openai/responses.go
Anthropic No trace/contrib/anthropic/messages.go

5. Test code assumes the attribute should exist

The OpenAI chat completions test in trace/contrib/openai/chatcompletions_test.go (lines 741–757) constructs an assertion structure with "type": "llm" hardcoded, but this value is not read from the span — it is manually inserted, masking the fact that the attribute is never actually set on the span.

Braintrust docs status

The advanced tracing documentation explicitly states that span_attributes.type should be set to "llm" for LLM calls, and that this enables:

  • A helpful icon in the UI
  • LLM duration metrics
  • The "Try prompt" feature

Status: supported — the platform expects this attribute.

Upstream sources

  • This is not an upstream SDK gap but an internal instrumentation consistency gap across provider integrations within this repository.

Braintrust docs sources

Local repo files inspected

  • trace/contrib/openai/chatcompletions.go — no braintrust.span_attributes set
  • trace/contrib/openai/responses.go — no braintrust.span_attributes set
  • trace/contrib/anthropic/messages.go — no braintrust.span_attributes set
  • trace/contrib/genai/generatecontent.go:117 — correctly sets type: "llm"
  • trace/contrib/genkit/tracegenkit.go:77 — correctly sets type: "llm"
  • trace/contrib/langchaingo/tracelangchaingo.go:207 — correctly sets type: "llm"
  • trace/contrib/cloudwego/eino/traceeino.go:120 — correctly sets type: "llm"
  • trace/contrib/adk/traceadk.go:326 — correctly sets type: "llm"
  • trace/contrib/openai/chatcompletions_test.go:741-757 — test hardcodes "type": "llm" rather than reading it from span

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions