Skip to content

fix(observability): Emit OpenInference LLM cost lookup attributes for span exports#1988

Open
yczhang-nv wants to merge 2 commits into
NVIDIA:developfrom
yczhang-nv:yuchenz/nat-274-phoenix-cost-tracking-missing-for-nat-llm-spans-because
Open

fix(observability): Emit OpenInference LLM cost lookup attributes for span exports#1988
yczhang-nv wants to merge 2 commits into
NVIDIA:developfrom
yczhang-nv:yuchenz/nat-274-phoenix-cost-tracking-missing-for-nat-llm-spans-because

Conversation

@yczhang-nv
Copy link
Copy Markdown
Contributor

@yczhang-nv yczhang-nv commented May 21, 2026

Description

  • Add OpenInference llm.model_name and llm.provider span attributes for LLM spans
  • Populate model name from IntermediateStepPayload.name
  • Populate provider from IntermediateStepPayload.framework
  • Add a focused reproducer for LLM cost lookup attributes
  • Add an OTLP-level mocked cost verification that decodes exported protobuf spans and confirms a pricing lookup can produce a non-null cost

By Submitting this PR I confirm:

  • I am familiar with the Contributing Guidelines.
  • We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
    • Any contribution which contains commits that are not Signed-Off will not be accepted.
  • When the PR is ready for review, new or existing tests cover these changes.
  • When the PR is ready for review, the documentation is up to date with these changes.

Summary by CodeRabbit

  • Documentation

    • Updated tutorial formatting for improved readability.
  • New Features

    • Enhanced LLM operation observability with automatic tracking of model name and provider metadata.

Review Change Stack

Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
@yczhang-nv yczhang-nv self-assigned this May 21, 2026
@yczhang-nv yczhang-nv requested a review from a team as a code owner May 21, 2026 22:58
@yczhang-nv yczhang-nv added bug Something isn't working non-breaking Non-breaking change labels May 21, 2026
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 21, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 532d6968-da32-40f9-9d94-1c5b5c36d6e7

📥 Commits

Reviewing files that changed from the base of the PR and between b09789d and 3030a19.

📒 Files selected for processing (4)
  • docs/source/get-started/tutorials/create-a-new-workflow.md
  • packages/nvidia_nat_core/src/nat/data_models/span.py
  • packages/nvidia_nat_core/src/nat/observability/exporter/span_exporter.py
  • packages/nvidia_nat_core/tests/nat/observability/exporter/test_span_exporter.py

Walkthrough

This PR extends observability telemetry to capture LLM metadata during span export. Two new span attribute keys are added to the data model, span exporter logic conditionally records model name and provider for LLM spans, and a test validates the behavior.

Changes

LLM Span Attribute Tracking

Layer / File(s) Summary
LLM span attribute schema
packages/nvidia_nat_core/src/nat/data_models/span.py
SpanAttributes enum adds LLM_MODEL_NAME and LLM_PROVIDER members to define attribute keys for LLM metadata.
LLM span attribute export
packages/nvidia_nat_core/src/nat/observability/exporter/span_exporter.py
SpanKind is imported and conditional logic records LLM model name from event.payload.name and provider from event.payload.framework onto exported spans when span kind is LLM.
LLM span export test
packages/nvidia_nat_core/tests/nat/observability/exporter/test_span_exporter.py
New test test_llm_span_exports_cost_lookup_attributes validates that LLM START/END spans are exported with model name, provider, and token count attributes.

Unresolved Changes

Two documentation hyperlinks were removed from create-a-new-workflow.md (lines 111, 177): WebBaseLoader and DOCA GPUNetIO references are now unlinked text.

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the main change: adding OpenInference LLM cost lookup attributes for span exports in observability, matching the core objective of the PR.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working non-breaking Non-breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant