Impact Efficiency Report - 2026-06-10

[github-actions]

Summary

Analysis window: 2025-12-11 to 2026-06-10 (180 days)
Note: Outcome data covers a sample of the most recent ~146 workflow outcomes retrievable via GitHub MCP API. AI Credits data covers 15 workflow runs with AIC visible in issue footers. Full 180-day totals exceed what can be retrieved in a single pass.

Metric	Value
Outcomes analyzed	146
Objectives mapped	3
Unmapped outcomes	143
Accepted outcome count	51
Total outcome value	150
AI Credits (visible sample)	~2,216 AIC
Impact Efficiency	0.068 value-units / AIC

Accepted outcomes: 48 merged PRs (Copilot agent) + 3 completed high-priority issues ([copilot-opt]) = 51 total.
Mapped outcomes: Only 3 — the three [copilot-opt] report-action issues carrying high-priority label (Objective Value 50 each), closed with state_reason: completed. All 48 merged PRs lack Closes #N links to objective issues.
AI Credits: Partial denominator. Run-level AIC from issue footers across 15 visible workflow runs (2,156 AIC through 2026-06-09 + 60.2 AIC from the 2026-06-10 Outcome Collector run). Copilot agent PR-producing sessions are not represented.

---

Top outcomes by outcome value

Outcome	Associated objective	Objective value signals	Objective Value	Outcome Value
#37902: [copilot-opt] Retry-blocked — WIP Fix failing GitHub Actions job 'agent'	#37902 (self, report action)	high-priority	50	50
#37901: [copilot-opt] 56% of workflow runs blocked by action_required approval	#37901 (self, report action)	high-priority	50	50
#37900: [copilot-opt] Duplicate concurrent workflow instances on Agentic Commands	#37900 (self, report action)	high-priority	50	50
#38293: Add objective mapping constants and tests	— (unmapped)	—	—	0
#38279: emit gh-aw.aic as OTLP Sum metric for backend-native consumption	— (unmapped)	—	—	0
#38277: Add steering message rendering in unified log view	— (unmapped)	—	—	0
#38273: Handle failure-issue permission denials and enforce timeout-minutes	— (unmapped)	—	—	0
#38263: [caveman] Optimize instruction verbosity — memory, mcp-clis, context	— (unmapped)	—	—	0

Only mapped outcomes with Objective Value > 0 shown in non-zero rows. All 48 merged PRs are accepted but unmapped.

---

Top objectives by delivered value

Objective Label	Delivered Outcome Value	Attempted Outcome Value	Delivered Outcomes	Efficiency
high-priority	150	150	3	100%

The high-priority label drove all 3 mapped outcomes. No other label from .github/objective-mapping.json produced a mapped outcome in this sample. Attempted Outcome Value equals Delivered because all 3 mapped outcomes were accepted.

---

Unmapped outcomes

Outcome	Reason objective was not mapped
#38293 PR: Add objective mapping constants and tests	No linked objective issue in PR body
#38279 PR: emit gh-aw.aic as OTLP Sum metric	No linked objective issue
#38277 PR: Add steering message rendering	No linked objective issue
#38273 PR: Handle failure-issue permission denials	No linked objective issue
#38265 PR: Fix failing GitHub Actions job Integration: Workflow M	No linked objective issue
#38263 PR: [caveman] Optimize instruction verbosity	No linked closing issue; PR carries automation/documentation labels but objective tracing requires a linked root issue
#38261 PR: Fix failing GitHub Actions job Integration: Workflow F	No linked objective issue
#38258 PR: Detect invalid model names across Copilot/Codex/Claude	No linked objective issue
#38256 PR: Measure and log GitHub API rate limit around daily AIC guard	No linked objective issue
#38262 PR: Fix failing GitHub Actions job 'Integration: CLI C'	No linked objective issue
#38247 PR: Remove imports.if and migrate experiment-driven import usage	No linked objective issue
(+ 37 additional merged PRs from 2025-12-11–2026-06-09)	No linked objective issue in any merged PR
[aw] failure issues (e.g., #38035, #38032, #38026)	No parent objective issue; workflow monitoring outputs with no linking hierarchy
[refactor], [reliability], [lint] report issues (e.g., #37979, #37969, #38010)	Produced as standalone report actions; no parent objective issue

143 of 146 outcomes are unmapped. The dominant pattern is that Copilot agent PRs and workflow report issues do not carry Closes #N or equivalent links to objective issues.

---

Interpretation

Accepted outcome count alone: 51
Impact Efficiency: 0.068 value-units per AIC

Accepted outcome count (51) presents an optimistic picture: 48 Copilot agent PRs merged plus 3 high-priority issues resolved. On volume alone, this looks productive and the count grew by 11 in a single day.

Impact Efficiency surfaces the structural gap: 98.0% of accepted outcomes (50 of 51) contribute zero to Total Outcome Value because they cannot be traced to a mapped objective. The 48 merged PRs deliver real engineering value, but without objective issue links their strategic importance is unquantifiable under this model. The only measurable value (150 outcome-value units) comes from 3 [copilot-opt] issues that triggered concrete workflow optimizations — and those are unchanged from the previous day's report.

Which metric better reflects meaningful delivered value relative to cost?

Impact Efficiency is the more disciplined signal — it forces the question of what was accomplished and whether it mattered, not just how many things were done. At 0.068 value-units/AIC (vs. 0.070 yesterday), it correctly surfaces that AI spend is not yet traceable to prioritized objectives systematically. The accepted count of 51 masks this gap entirely.

However, Impact Efficiency cannot be trusted as a complete signal here, because:

1. The unmapped rate (98.0%) means the model measures less than 2% of total delivered value
2. The AIC denominator covers only ~15 of potentially hundreds of workflow runs
3. The 48 merged PRs represent significant engineering investment not captured by this model

Conclusion: Impact Efficiency is a more meaningful signal in theory, but is currently uncomputable at full fidelity due to missing objective links and incomplete AI Credits data. Even so, it outperforms the accepted count as a diagnostic: it exposes exactly where measurement breaks down and what investments are needed to make it credible.

---

Data quality

Outcome-to-objective association

Critical gap. 98.0% of outcomes (143/146) are unmapped. All 48 Copilot agent merged PRs lack Closes #N or Fixes #N links to objective issues. Workflow report issues ([aw], [refactor], [reliability], etc.) are created as standalone outputs with no parent issue. Without these links, the Impact Efficiency numerator is computed from only 3 of 51 accepted outcomes.

Root tracing and linked-object coverage

Only direct issue self-tracing succeeded (the 3 [copilot-opt] issues using their own labels). Zero PR-to-closing-issue traces succeeded across 48 merged PRs. The Outcome Collector batch of 28 outcomes from today's run (run #38245) includes 5 accepted items at medium evidence (comments/reviews on PRs), but these lack label-based objective tracing and contribute zero to Total Outcome Value.

Label mapping coverage in .github/objective-mapping.json

The objective mapping is comprehensive: 46 entries covering values 0–100 with multi_label_logic: max. Coverage is not the issue — the mapping is ready. The blocking problem is that outcomes are not linked to labeled objectives. Of the repository's ~100 active labels, only a fraction appear on issues linked to workflow outcomes.

AI Credits availability

Partial — live MCP fallback path used. No deterministic precomputed gh aw logs --json data with per-run aic field was available in this run. AI Credits were sourced from issue footers (workflow runs embedding · X AIC ·), covering ~15 workflow runs (~2,216 AIC total visible). This is the live fallback path. The denominator is a lower bound: the true 180-day total is higher (all Copilot agent sessions producing PRs do not embed AIC in issue footers), meaning true Impact Efficiency is lower than 0.068.

---

Generated by Impact Efficiency Report workflow — 2026-06-10

Generated by 📊 Impact Efficiency Report · 962 AIC · ⌖ 26.1 AIC · ⊞ 28.9K · ◷