emit gh-aw.aic as OTLP Sum metric for backend-native consumption#38279
Conversation
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
|
@copilot Don't surface cost, only aic and consider making it a consumable metric. |
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Done in 6da001e. Removed |
|
@copilot Revert |
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Done — |
|
@copilot Undo all other unrelated |
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR makes AI Credits (AIC) a first-class OpenTelemetry (OTLP) metrics signal (gh_aw.aic) by emitting it as a cumulative, monotonic Sum to /v1/metrics, enabling backend-native aggregation/alerting without relying on span-attribute extraction.
Changes:
- Add OTLP metrics payload construction + multi-endpoint metric export, and emit
gh_aw.aicfrom the token-usage owning job to avoid double counting. - Add unit tests covering the metrics payload shape and the emission gating scenarios.
- Document the new OTLP metric contract in both reference docs and the observability spec.
Show a summary per file
| File | Description |
|---|---|
| actions/setup/js/send_otlp_span.cjs | Adds OTLP metrics payload builder/sender and emits gh_aw.aic as an OTLP Sum metric from the conclusion span path. |
| actions/setup/js/send_otlp_span.test.cjs | Adds tests for metrics payload structure and for sendJobConclusionSpan metric emission gating. |
| docs/src/content/docs/reference/open-telemetry.mdx | Documents the new gh_aw.aic OTLP metric and its attributes. |
| specs/otel-observability-spec.md | Specifies the OTLP metrics contract for gh_aw.aic and references it from the span attribute contract. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 4/4 changed files
- Comments generated: 3
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
The platform bills in AI Credits (AIC), not USD. OTLP backends (Sentry, Datadog, Grafana) can natively aggregate and alert on OTel metrics, but
gh-aw.aicwas only available as a span attribute — not as a consumable metric signal. This change makes AIC a first-class OTLP metric without surfacing any USD cost.Changes
actions/setup/js/send_otlp_span.cjs— addsbuildOTLPMetricsPayload,sendOTLPMetric, andsendOTLPMetricToAllEndpoints. WhenaiCredits > 0, the job conclusion span now also postsgh_aw.aicas an OTLP Sum metric (cumulative, monotonic) to/v1/metricson all configured endpoints. Emission is gated to agent/detection jobs (jobEmitsOwnTokenUsage) to prevent double-counting. The span attributegh-aw.aicis unchanged.actions/setup/js/send_otlp_span.test.cjs— adds coverage forbuildOTLPMetricsPayloadpayload structure and for the threesendJobConclusionSpanmetric-emission scenarios (AIC > 0, AIC = 0, non-agent job).docs/src/content/docs/reference/open-telemetry.mdx— adds an "OTLP metrics" section documentinggh_aw.aic(Sum, cumulative, monotonic) with its data point attributes.specs/otel-observability-spec.md— adds §10.6 "OTLP Metrics Signal" with full metric contract, data point attributes, and aggregation temporality guidance.The lock.yml files generated by a previous agent run are also removed from this branch — they were noise unrelated to this change.