Skip to content

emit gh-aw.aic as OTLP Sum metric for backend-native consumption#38279

Merged
mnkiefer merged 10 commits into
mainfrom
copilot/investigate-ai-credits-otel-backends
Jun 10, 2026
Merged

emit gh-aw.aic as OTLP Sum metric for backend-native consumption#38279
mnkiefer merged 10 commits into
mainfrom
copilot/investigate-ai-credits-otel-backends

Conversation

Copilot AI commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

The platform bills in AI Credits (AIC), not USD. OTLP backends (Sentry, Datadog, Grafana) can natively aggregate and alert on OTel metrics, but gh-aw.aic was only available as a span attribute — not as a consumable metric signal. This change makes AIC a first-class OTLP metric without surfacing any USD cost.

Changes

  • actions/setup/js/send_otlp_span.cjs — adds buildOTLPMetricsPayload, sendOTLPMetric, and sendOTLPMetricToAllEndpoints. When aiCredits > 0, the job conclusion span now also posts gh_aw.aic as an OTLP Sum metric (cumulative, monotonic) to /v1/metrics on all configured endpoints. Emission is gated to agent/detection jobs (jobEmitsOwnTokenUsage) to prevent double-counting. The span attribute gh-aw.aic is unchanged.
  • actions/setup/js/send_otlp_span.test.cjs — adds coverage for buildOTLPMetricsPayload payload structure and for the three sendJobConclusionSpan metric-emission scenarios (AIC > 0, AIC = 0, non-agent job).
  • docs/src/content/docs/reference/open-telemetry.mdx — adds an "OTLP metrics" section documenting gh_aw.aic (Sum, cumulative, monotonic) with its data point attributes.
  • specs/otel-observability-spec.md — adds §10.6 "OTLP Metrics Signal" with full metric contract, data point attributes, and aggregation temporality guidance.

The lock.yml files generated by a previous agent run are also removed from this branch — they were noise unrelated to this change.

Copilot AI and others added 3 commits June 10, 2026 04:24
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
@mnkiefer

Copy link
Copy Markdown
Collaborator

@copilot Don't surface cost, only aic and consider making it a consumable metric.

Copilot AI and others added 2 commits June 10, 2026 05:25
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>

Copilot AI commented Jun 10, 2026

Copy link
Copy Markdown
Contributor Author

@copilot Don't surface cost, only aic and consider making it a consumable metric.

Done in 6da001e. Removed gen_ai.usage.cost entirely — no USD is surfaced. Instead, gh-aw.aic is now also emitted as a proper OTLP Sum metric (cumulative, monotonic) to /v1/metrics, making it natively consumable by backends without custom field mappings. The span attribute gh-aw.aic is unchanged.

Copilot AI changed the title emit gen_ai.usage.cost (USD) as OTel alias for gh-aw.aic on agent spans emit gh-aw.aic as OTLP Sum metric for backend-native consumption Jun 10, 2026
Copilot AI requested a review from mnkiefer June 10, 2026 05:33
@mnkiefer

Copy link
Copy Markdown
Collaborator

@copilot Revert .github/workflows/test-create-pr-error-handling.lock.yml to main

Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>

Copilot AI commented Jun 10, 2026

Copy link
Copy Markdown
Contributor Author

@copilot Revert .github/workflows/test-create-pr-error-handling.lock.yml to main

Done — .github/workflows/test-create-pr-error-handling.lock.yml has been restored to its state on main in the latest commit.

@mnkiefer

Copy link
Copy Markdown
Collaborator

@copilot Undo all other unrelated .lock.yml file changes and also update PR body.

Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
@mnkiefer mnkiefer marked this pull request as ready for review June 10, 2026 06:05
Copilot AI review requested due to automatic review settings June 10, 2026 06:05
Copilot AI requested a review from mnkiefer June 10, 2026 06:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR makes AI Credits (AIC) a first-class OpenTelemetry (OTLP) metrics signal (gh_aw.aic) by emitting it as a cumulative, monotonic Sum to /v1/metrics, enabling backend-native aggregation/alerting without relying on span-attribute extraction.

Changes:

  • Add OTLP metrics payload construction + multi-endpoint metric export, and emit gh_aw.aic from the token-usage owning job to avoid double counting.
  • Add unit tests covering the metrics payload shape and the emission gating scenarios.
  • Document the new OTLP metric contract in both reference docs and the observability spec.
Show a summary per file
File Description
actions/setup/js/send_otlp_span.cjs Adds OTLP metrics payload builder/sender and emits gh_aw.aic as an OTLP Sum metric from the conclusion span path.
actions/setup/js/send_otlp_span.test.cjs Adds tests for metrics payload structure and for sendJobConclusionSpan metric emission gating.
docs/src/content/docs/reference/open-telemetry.mdx Documents the new gh_aw.aic OTLP metric and its attributes.
specs/otel-observability-spec.md Specifies the OTLP metrics contract for gh_aw.aic and references it from the span attribute contract.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 4/4 changed files
  • Comments generated: 3

Comment thread specs/otel-observability-spec.md Outdated
Comment thread actions/setup/js/send_otlp_span.cjs Outdated
Comment thread actions/setup/js/send_otlp_span.test.cjs
mnkiefer and others added 2 commits June 10, 2026 08:12
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@mnkiefer mnkiefer merged commit 073404c into main Jun 10, 2026
15 checks passed
@mnkiefer mnkiefer deleted the copilot/investigate-ai-credits-otel-backends branch June 10, 2026 06:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants