feat(0.2): pillar lift batch 3 — portfolio + explain schemas + benchmarks + analyze error UX#172
Merged
Conversation
…alyze error UX (Understand+Align lift) Lifts seven cells across Understand and Align pillars without touching the labeled-corpus cells. docs/schema/portfolio.md (new) — portfolio.E4 (2→4): - Canonical PortfolioSummary / TestAsset / Finding / PortfolioAggregates / OwnerPortfolioSummary contract. - .terrain/repos.yaml manifest schema documented. - Multi-repo aggregate output marked Experimental for 0.2.0 (honest about partial shipping per the plan's pillar priority). docs/schema/explain.md (new) — insights_impact_explain.E4 (3→4): - `terrain explain <target>` dispatch table mapping every accepted target type → output shape. - Each shape references its canonical schema doc. - OwnerExplanation shape documented inline (only one not covered by an existing schema file). internal/insights/insights_bench_test.go (new) — insights_impact_explain.E5 (3→4): - BenchmarkBuild_Healthy / WithDepgraphResults / LargeSnapshot. - Reference numbers: 2.5µs / 8µs / 40µs per op on Intel i7-8850H. - Linear scaling — no quadratic regressions. internal/reporting/empty_states.go + portfolio_report.go — portfolio.V3 (2→4): - New EmptyNoPortfolio empty-state kind. - RenderPortfolioReport now uses the designed empty state when TotalAssets == 0, replacing the bare two-line message. cmd/terrain/cmd_analyze.go — core_analyze.P5 (3→4): - analyzeFailureRemediation surfaces a designed remediation block on analysis failure. Three branches: timeout-exceeded (with --timeout-increase / scope-down / verbose-timing nudges), cancelled (re-run hint), generic (three common causes + verbose/json next steps). insights_impact_explain.E7 (3→4) — evidence refresh: - All three commands route through the cancellation-tested engine.RunPipelineContext path locked by TestRunPipelineContext_RespectsCancelledContext + TestRunPipelineContext_CancelMidFlight. docs/release/parity/scores.yaml: seven cells refreshed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…E7 cancellation Lifts five more cells in the Understand pillar. benchmarks/baseline.txt (new) — core_analyze.E5 (3→4): - Reference performance numbers for the three benchmark suites that matter: engine pipeline (small/medium/large), insights builder (healthy/with-depgraph/large-snapshot), changescope PR-comment renderer (small/medium/large). - Captured 2026-05 on Intel i7-8850H @ 2.60GHz with re-run instructions and notes. - make bench-gate compares ratios against this baseline. docs/user-guides/drilling-into-findings.md (new) — insights_impact_explain.P3 (3→4) and P4 (3→4): - Four-command ladder (analyze → insights → impact → explain) with worked examples per command. - Full "how confidence is computed" section: detector confidence (structural / heuristic / runtime-aware), ConfidenceDetail Wilson/Beta intervals, test-selection confidence (exact / inferred / weak), coverage confidence. - Round-trip example using a stable finding ID. - Cross-references the schema docs. Closes the audit's "no per-command 'how confidence is computed' pages" concern. Adopters new to the drill-down commands now have an explicit playbook. summary_posture_metrics_focus.E7 (3→4) — evidence refresh: - summary / posture / metrics / focus all route through runPipelineWithSignals → engine.RunPipelineContext, locked by TestRunPipelineContext_RespectsCancelledContext + TestRunPipelineContext_CancelMidFlight. docs/release/parity/scores.yaml: five cells refreshed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lifts portfolio.P5 (3→4) — the last achievable concrete error-UX lift in the Align pillar without 0.3 corpus work. cmd/terrain/cmd_insights.go runPortfolio: - When portfolio analysis fails, surface a designed remediation block: names the three common adopter causes (snapshot construction failure, non-git root, permission errors) and points at `terrain analyze` for root-cause drill-down. Links to docs/schema/portfolio.md for multi-repo workflows (currently experimental in 0.2.0). docs/release/parity/scores.yaml: portfolio.P5 (3→4). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Terrain AI Risk Review
Decision: PASS — AI surfaces are covered. |
[RISK] Terrain — Merge with caution
Coverage gaps in changed code
4 pre-existing issues on changed files
Recommended tests5 test(s) with exact coverage of 5 impacted unit(s). 4 impacted unit(s) have no covering tests in the selected set. Confidence: 5 exact (5 tests selected)
Owners: PMCLSF Limitations
Generated by Terrain · Targeted Test ResultsTerrain selected 5 test(s) instead of the full suite.
|
pmclSF
added a commit
that referenced
this pull request
May 9, 2026
…arks + analyze error UX (#172) * feat(0.2): portfolio + explain schema docs + insights benchmarks + analyze error UX (Understand+Align lift) Lifts seven cells across Understand and Align pillars without touching the labeled-corpus cells. docs/schema/portfolio.md (new) — portfolio.E4 (2→4): - Canonical PortfolioSummary / TestAsset / Finding / PortfolioAggregates / OwnerPortfolioSummary contract. - .terrain/repos.yaml manifest schema documented. - Multi-repo aggregate output marked Experimental for 0.2.0 (honest about partial shipping per the plan's pillar priority). docs/schema/explain.md (new) — insights_impact_explain.E4 (3→4): - `terrain explain <target>` dispatch table mapping every accepted target type → output shape. - Each shape references its canonical schema doc. - OwnerExplanation shape documented inline (only one not covered by an existing schema file). internal/insights/insights_bench_test.go (new) — insights_impact_explain.E5 (3→4): - BenchmarkBuild_Healthy / WithDepgraphResults / LargeSnapshot. - Reference numbers: 2.5µs / 8µs / 40µs per op on Intel i7-8850H. - Linear scaling — no quadratic regressions. internal/reporting/empty_states.go + portfolio_report.go — portfolio.V3 (2→4): - New EmptyNoPortfolio empty-state kind. - RenderPortfolioReport now uses the designed empty state when TotalAssets == 0, replacing the bare two-line message. cmd/terrain/cmd_analyze.go — core_analyze.P5 (3→4): - analyzeFailureRemediation surfaces a designed remediation block on analysis failure. Three branches: timeout-exceeded (with --timeout-increase / scope-down / verbose-timing nudges), cancelled (re-run hint), generic (three common causes + verbose/json next steps). insights_impact_explain.E7 (3→4) — evidence refresh: - All three commands route through the cancellation-tested engine.RunPipelineContext path locked by TestRunPipelineContext_RespectsCancelledContext + TestRunPipelineContext_CancelMidFlight. docs/release/parity/scores.yaml: seven cells refreshed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(0.2): benchmark baseline + drilling-into-findings doc + summary E7 cancellation Lifts five more cells in the Understand pillar. benchmarks/baseline.txt (new) — core_analyze.E5 (3→4): - Reference performance numbers for the three benchmark suites that matter: engine pipeline (small/medium/large), insights builder (healthy/with-depgraph/large-snapshot), changescope PR-comment renderer (small/medium/large). - Captured 2026-05 on Intel i7-8850H @ 2.60GHz with re-run instructions and notes. - make bench-gate compares ratios against this baseline. docs/user-guides/drilling-into-findings.md (new) — insights_impact_explain.P3 (3→4) and P4 (3→4): - Four-command ladder (analyze → insights → impact → explain) with worked examples per command. - Full "how confidence is computed" section: detector confidence (structural / heuristic / runtime-aware), ConfidenceDetail Wilson/Beta intervals, test-selection confidence (exact / inferred / weak), coverage confidence. - Round-trip example using a stable finding ID. - Cross-references the schema docs. Closes the audit's "no per-command 'how confidence is computed' pages" concern. Adopters new to the drill-down commands now have an explicit playbook. summary_posture_metrics_focus.E7 (3→4) — evidence refresh: - summary / posture / metrics / focus all route through runPipelineWithSignals → engine.RunPipelineContext, locked by TestRunPipelineContext_RespectsCancelledContext + TestRunPipelineContext_CancelMidFlight. docs/release/parity/scores.yaml: five cells refreshed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(0.2): portfolio command error UX (Align lift) Lifts portfolio.P5 (3→4) — the last achievable concrete error-UX lift in the Align pillar without 0.3 corpus work. cmd/terrain/cmd_insights.go runPortfolio: - When portfolio analysis fails, surface a designed remediation block: names the three common adopter causes (snapshot construction failure, non-git root, permission errors) and points at `terrain analyze` for root-cause drill-down. Links to docs/schema/portfolio.md for multi-repo workflows (currently experimental in 0.2.0). docs/release/parity/scores.yaml: portfolio.P5 (3→4). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces closed PR #170 (auto-closed when its stacked base branch was deleted on merge of #169).
Same content as #170, rebased onto main with #168 + #169 commits dropped.
Cells lifted
Highlights
docs/schema/portfolio.md+docs/schema/explain.md— published JSON schema contractsdocs/user-guides/drilling-into-findings.md— four-command playbook + confidence modelbenchmarks/baseline.txt— reference performance numbersinternal/insights/insights_bench_test.go— Build perf benchmarksEmptyNoPortfolioempty-state wiredanalyzeFailureRemediation+runPortfolioerror UXTest plan
go test ./...greenmake pillar-parityshows the 11 cells lifted