[token-consumption] Daily AIC Consumption Report - 2026-06-10

[github-actions]

Executive Summary

Telemetry is flowing healthily — 28,352 spans across 137 unique workflows and 546 runs were ingested in the last 24h — but AIC consumption could not be measured. The gh-aw.aic attribute is not present in queryable form on any span in the window (or over 7 days), and Sentry has it registered as a string-typed field, which causes all numeric aggregation (sum, avg, gh-aw.aic:>=0) to be rejected by the API.

This is an observability gap, not a usage signal: I cannot report a real total AIC, top consumers by AIC, or anomalies. Per the guardrails, no AIC values are invented below. The ranking shown is by span volume as a proxy, explicitly labeled as such. The root cause appears to be on the ingestion/typing side and/or AIC not being emitted on the agent-job conclusion spans for the runs in this window.

Key Metrics

Metric	Value
Events analyzed (all spans, 24h)	28,352
Workflow-attributed spans	4,511 (16%)
Events with AIC data	0
Unique workflows	137
Unique runs	546
Total AIC	Unavailable (no AIC-bearing spans)
Avg AIC/event	N/A — no AIC data
P95 AIC/event	N/A — no AIC data

Critical Finding — AIC telemetry is unqueryable

• has:gh-aw.aic → 0 results at both 24h and 7d. No span exposes the attribute in indexed form.
• sum(gh-aw.aic) → API 400: "gh-aw.aic is invalid ... Its a string type field". Sentry EAP has the attribute registered as string, blocking numeric rollups and gh-aw.aic:>=0 filters.
• The only 2 agent-job conclusion spans in 24h — gh-aw.activation.conclusion for Matt Pocock Skills Reviewer (run 27270743427) and Daily Observability Report for AWF Firewall and MCP Gateway (run 27243894409) — carry gh-aw.run.status: success but no gh-aw.aic, gen_ai.usage.total_tokens, or gen_ai.response.finish_reasons.
• Emit side is correct: buildAttr in actions/setup/js/send_otlp_span.cjs encodes numbers as intValue/doubleValue, and gh-aw.aic is emitted at line ~2077 on the agent conclusion span. The string registration is therefore a schema/ingestion inconsistency (sticky EAP attribute typing, likely from historical mixed-type emission of the same key) and/or AIC simply not being populated (GH_AW_AIC unset / jobEmitsOwnTokenUsage false) for these runs.

Top 10 Workflows by Span Volume (AIC unavailable — proxy only)

AIC could not be attributed; the table ranks by ingested span volume as an activity proxy. Total AIC is genuinely unknown, not zero.

Workflow	Events	Total AIC	Avg AIC/Event
Smoke CI	767	—	—
PR Code Quality Reviewer	336	—	—
Design Decision Gate 🏗️	328	—	—
Test Quality Sentinel	282	—	—
Matt Pocock Skills Reviewer	247	—	—
PR Sous Chef	244	—	—
AI Moderator	174	—	—
Issue Monster	144	—	—
Auto-Triage Issues	122	—	—
Smoke Copilot	64	—	—

Companion dataset checks (errors / logs / metrics)

• errors dataset (24h, all events): 0 results — no crash/exception telemetry in window.
• logs dataset (24h, query aic): 0 results — AIC is not mirrored into logs.
• spans dataset (dataset: spans): primary source; gen_ai/invoke_agent/setup/conclusion spans present and attributed, but AIC field absent (see Critical Finding).
• search_events MCP tool is not available in this build (no embedded LLM provider configured); all queries used list_events with direct Sentry query syntax, as designed for fallback.

Representative trace verification

Verified trace 1bc4daa30879603b201825ae338adf82 (Matt Pocock Skills Reviewer, run 27270743427):

• Span 2c91fe4ed2336632 — transaction: gh-aw.activation.conclusion, span.op: gen_ai, gh-aw.run.status: success.
• Requested fields gh-aw.aic, gen_ai.usage.total_tokens, gen_ai.response.finish_reasons all returned empty on the span that should carry AIC.

This confirms the gap is at the conclusion span itself, not a query artifact.

Data Quality and Gaps

• Events missing AIC attributes: 100% — gh-aw.aic not queryable on any of the 28,352 spans.
• Events missing workflow identifiers: ~23,841 of 28,352 (~84%). Only setup/conclusion spans emitted by gh-aw carry the resource-level gh-aw.workflow.name; engine-emitted gen_ai/invoke_agent child spans do not, so most spans cannot be attributed to a workflow.
• Field typing: gh-aw.aic registered as string in EAP — numeric aggregation impossible until re-typed or re-keyed.
• Assumptions / fallbacks used: ranking by span volume as an activity proxy; no AIC value fabricated; list_events used because search_events is unavailable.

Recommendations

1. Restore AIC emission on agent conclusion spans. Confirm GH_AW_AIC / /tmp/gh-aw/agent_usage.json is populated and jobEmitsOwnTokenUsage is true for the agent job; the recent fix (#38364, emit gh-aw.aic=0 + engine-reported fallback when the firewall proxy reports zero) may not yet be deployed to the workflow versions running in this window.
2. Fix the string-typing of gh-aw.aic in Sentry. EAP attribute types are sticky once registered. Emit AIC under a fresh, consistently-numeric measurement key (e.g. gh-aw.aic.value) so it registers numeric and becomes aggregatable; keep the legacy key only for backward compatibility.
3. Backfill workflow attribution onto child spans. Propagate gh-aw.workflow.name/gh-aw.run.id to engine gen_ai/invoke_agent spans so more than 16% of spans are attributable, enabling per-workflow AIC rollups once #1/#2 land.
4. Add a regression guard. Alert/CI assertion that the daily report finds >0 AIC-bearing spans, so this exact failure mode (typed-away or unemitted AIC) is caught immediately rather than surfacing as an empty report.

References

• Sentry spans explore (workflow-attributed, 24h): https://github.sentry.io/explore/traces/?query=has:gh-aw.workflow.name&project=4511347087179777&statsPeriod=24h
• Run — Matt Pocock Skills Reviewer (agent conclusion verified): https://github.com/github/gh-aw/actions/runs/27270743427
• Run — Daily Observability Report for AWF Firewall and MCP Gateway: https://github.com/github/gh-aw/actions/runs/27243894409

Generated by 📊 Daily AIC Consumption Report (Sentry OTel) · 276.7 AIC · ⌖ 29.7 AIC · ⊞ 7.4K · ◷

• expires on Jun 11, 2026, 7:22 AM UTC-08:00