Conversation
|
Preview this PR with FeatureBee: https://beta.wandb.ai/?betaVersion=fa4a72bab58ea593023d12402f2158598c423f2c |
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
bcsherma
force-pushed
the
ben/agent-data-model
branch
from
17b81d9 to
b98be30
Compare
bcsherma
force-pushed
the
ben/agent-data-model
branch
from
a7f2440 to
9851c4c
Compare
bcsherma
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Spike to validate whether OTel GenAI semantic conventions are sufficient to capture the full structure of LLM agent execution and whether we can build a complete storage + rendering pipeline on top of them. The result is a working end-to-end system covering ingest, normalization, storage, trajectory projection, and out-of-process agent tracing.
Design docs in
docs/design/— seedesign_review_discussion.mdfor the overview anddesign_review.htmlfor slides.What's in this branch
ClickHouse schema (
migrations/026_genai.up.sql)genai_spans— wide normalized table with typed columns for all GenAI fields (messages, tokens, model, agent name, operation type, conversation ID, tool calls, etc.). ReplacingMergeTree, partitioned by month, ordered by(project_id, started_at, span_id).genai_span_attributes— typed EAV for custom OTel attributes not in the column set.genai_agents/genai_conversations— SummingMergeTree tables auto-populated by materialized views. O(1) agent and conversation list queries.entity_annotations— generic annotation EAV for spans, agents, conversations.Ingest + normalization (
weave/trace_server/opentelemetry/)/otel/v1/genai/tracesextract_genai_span()with vendor fallback chains: normalizes OpenAI Agents SDK, Google ADK, Traceloop/OpenInference, and standard OTel GenAI attributes into the same columnsArray(Tuple(role, content, tool_call_id, tool_name))Trajectory projection (
weave/trace_server/genai_chat_view.py)operation_name→ producesuser_message,agent_start,agent_message,tool_call,agent_handoff,context_compactedconversation_idRead APIs
/genai/spans/query,/genai/spans/trace,/genai/spans/active/genai/traces/chat,/genai/conversations/chat/genai/agents/query,/genai/agents/metrics/genai/conversations/query/genai/annotations/upsert,/genai/annotations/delete,/genai/annotations/querySDK instrumentations (
weave/otel/instrumentors/)setup_tracing()one-call OTel configuration withinstrument()per frameworkConversationIdInjector,SystemPromptInjector,ToolDefinitionsInjectorspan processorslog_content()for media capture (images, audio)LiveSpanProcessorfor real-time span-start notificationsDaemon for IDE agents (
weave/agent_hooks/)Format interop
examples/otel_genai/)Design docs (
docs/design/)architecture.md— system overviewdata_model.md— span patterns, normalized schema, vendor fallback chainschat_view_algorithm.md— trajectory projection algorithminstrumentation_guide.md— how to emit data (SDK + daemon)format_interoperability.md— adapter architecture + ATIF referencedesign_review.html— presentation slidesdesign_review_discussion.md— Notion-ready discussion docStatus
This is a spike / draft — not intended for merge as-is. The purpose is to validate the approach and inform a design discussion about how this relates to Weave's existing call model.