@braintrust/pi-extension

Braintrust extension for pi.

Today this extension automatically traces pi sessions, turns, model calls, and tool executions to Braintrust.

What gets traced

• Session spans: one root span per pi session that actually produces at least one turn
• Turn spans: one span per user prompt / agent run
• LLM spans: one span per model response inside a turn
• Tool spans: one span per tool execution

Trace shape:

Session (task)
├── Turn 1 (task)
│   ├── anthropic/claude-sonnet-4 (llm)
│   │   ├── read: package.json (tool)
│   │   └── bash: pnpm test (tool)
│   └── anthropic/claude-sonnet-4 (llm)
└── Turn 2 (task)

Install

From npm

pi install npm:@braintrust/pi-extension

From this repo

pi install .

Or load it just for one run:

pi -e .

Compatibility

This package supports the last three stable pi versions.

Our GitHub Actions compatibility job automatically resolves and tests the latest patch release from each of the last three stable pi minor versions, so new pi releases are picked up without manually updating the matrix.

Quick start

Tracing is disabled by default.

Set these environment variables:

export TRACE_TO_BRAINTRUST=true
export BRAINTRUST_API_KEY=sk-...
export BRAINTRUST_PROJECT=pi

Then start pi normally.

In interactive mode, the footer shows a Braintrust status indicator while tracing is active, and a widget below the editor shows a shortened clickable trace link when available.

Configuration

You can configure the extension with environment variables or JSON config files.

Config precedence is:

1. defaults
2. ~/.pi/agent/braintrust.json
3. .pi/braintrust.json
4. environment variables

Config file locations

• Global: ~/.pi/agent/braintrust.json
• Project: .pi/braintrust.json

Example:

{
  "trace_to_braintrust": true,
  "project": "pi",
  "debug": true,
  "additional_metadata": {
    "team": "platform"
  }
}

Supported settings

Config key	Env var	Default
trace_to_braintrust	TRACE_TO_BRAINTRUST	false
api_key	BRAINTRUST_API_KEY	unset
api_url	BRAINTRUST_API_URL	https://api.braintrust.dev
app_url	BRAINTRUST_APP_URL	https://www.braintrust.dev
org_name	BRAINTRUST_ORG_NAME	unset
project	BRAINTRUST_PROJECT	pi
debug	BRAINTRUST_DEBUG	false
additional_metadata	BRAINTRUST_ADDITIONAL_METADATA	{}
log_file	BRAINTRUST_LOG_FILE	unset
state_dir	BRAINTRUST_STATE_DIR	~/.pi/agent/state/braintrust-pi-extension
show_ui	BRAINTRUST_SHOW_UI	true
show_trace_link	BRAINTRUST_SHOW_TRACE_LINK	true
parent_span_id	PI_PARENT_SPAN_ID	unset
root_span_id	PI_ROOT_SPAN_ID	unset

Notes

• Project config overrides global config.
• Environment variables override both config files.
• Session bookkeeping is stored in ~/.pi/agent/state/braintrust-pi-extension/ by default.
• Span delivery uses the Braintrust JavaScript SDK's built-in async/background flushing.
• If Braintrust is unavailable, pi should continue working normally.
• If PI_PARENT_SPAN_ID is set, the pi session span is attached under an existing Braintrust trace.
• PI_ROOT_SPAN_ID can be used when the parent span is not the trace root.

Contributing

See CONTRIBUTING.md for development setup, validation, and repository conventions.