Skip to content

feat(0.2): pillar lift batch 3 — portfolio + explain schemas + benchmarks + analyze error UX#172

Merged
pmclSF merged 3 commits into
mainfrom
feat/0.2-pillar-lift-3
May 7, 2026
Merged

feat(0.2): pillar lift batch 3 — portfolio + explain schemas + benchmarks + analyze error UX#172
pmclSF merged 3 commits into
mainfrom
feat/0.2-pillar-lift-3

Conversation

@pmclSF
Copy link
Copy Markdown
Owner

@pmclSF pmclSF commented May 7, 2026

Replaces closed PR #170 (auto-closed when its stacked base branch was deleted on merge of #169).

Same content as #170, rebased onto main with #168 + #169 commits dropped.

Cells lifted

Pillar Area Cell Before → After
Understand core_analyze_pipeline P5 3 → 4
Understand core_analyze_pipeline E5 3 → 4
Understand insights_impact_explain P3 3 → 4
Understand insights_impact_explain P4 3 → 4
Understand insights_impact_explain E4 3 → 4
Understand insights_impact_explain E5 3 → 4
Understand insights_impact_explain E7 3 → 4
Understand summary_posture_metrics_focus E7 3 → 4
Align portfolio E4 2 → 4
Align portfolio V3 2 → 4
Align portfolio P5 3 → 4

Highlights

  • docs/schema/portfolio.md + docs/schema/explain.md — published JSON schema contracts
  • docs/user-guides/drilling-into-findings.md — four-command playbook + confidence model
  • benchmarks/baseline.txt — reference performance numbers
  • internal/insights/insights_bench_test.go — Build perf benchmarks
  • EmptyNoPortfolio empty-state wired
  • analyzeFailureRemediation + runPortfolio error UX

Test plan

  • go test ./... green
  • make pillar-parity shows the 11 cells lifted

pmclSF and others added 3 commits May 6, 2026 17:26
…alyze error UX (Understand+Align lift)

Lifts seven cells across Understand and Align pillars without
touching the labeled-corpus cells.

docs/schema/portfolio.md (new) — portfolio.E4 (2→4):
- Canonical PortfolioSummary / TestAsset / Finding /
  PortfolioAggregates / OwnerPortfolioSummary contract.
- .terrain/repos.yaml manifest schema documented.
- Multi-repo aggregate output marked Experimental for 0.2.0
  (honest about partial shipping per the plan's pillar priority).

docs/schema/explain.md (new) — insights_impact_explain.E4 (3→4):
- `terrain explain <target>` dispatch table mapping every
  accepted target type → output shape.
- Each shape references its canonical schema doc.
- OwnerExplanation shape documented inline (only one not covered
  by an existing schema file).

internal/insights/insights_bench_test.go (new) — insights_impact_explain.E5 (3→4):
- BenchmarkBuild_Healthy / WithDepgraphResults / LargeSnapshot.
- Reference numbers: 2.5µs / 8µs / 40µs per op on Intel i7-8850H.
- Linear scaling — no quadratic regressions.

internal/reporting/empty_states.go + portfolio_report.go — portfolio.V3 (2→4):
- New EmptyNoPortfolio empty-state kind.
- RenderPortfolioReport now uses the designed empty state when
  TotalAssets == 0, replacing the bare two-line message.

cmd/terrain/cmd_analyze.go — core_analyze.P5 (3→4):
- analyzeFailureRemediation surfaces a designed remediation
  block on analysis failure. Three branches: timeout-exceeded
  (with --timeout-increase / scope-down / verbose-timing
  nudges), cancelled (re-run hint), generic (three common
  causes + verbose/json next steps).

insights_impact_explain.E7 (3→4) — evidence refresh:
- All three commands route through the cancellation-tested
  engine.RunPipelineContext path locked by
  TestRunPipelineContext_RespectsCancelledContext +
  TestRunPipelineContext_CancelMidFlight.

docs/release/parity/scores.yaml: seven cells refreshed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…E7 cancellation

Lifts five more cells in the Understand pillar.

benchmarks/baseline.txt (new) — core_analyze.E5 (3→4):
- Reference performance numbers for the three benchmark suites that
  matter: engine pipeline (small/medium/large), insights builder
  (healthy/with-depgraph/large-snapshot), changescope PR-comment
  renderer (small/medium/large).
- Captured 2026-05 on Intel i7-8850H @ 2.60GHz with re-run
  instructions and notes.
- make bench-gate compares ratios against this baseline.

docs/user-guides/drilling-into-findings.md (new) —
insights_impact_explain.P3 (3→4) and P4 (3→4):
- Four-command ladder (analyze → insights → impact → explain) with
  worked examples per command.
- Full "how confidence is computed" section: detector confidence
  (structural / heuristic / runtime-aware), ConfidenceDetail
  Wilson/Beta intervals, test-selection confidence
  (exact / inferred / weak), coverage confidence.
- Round-trip example using a stable finding ID.
- Cross-references the schema docs.

Closes the audit's "no per-command 'how confidence is computed'
pages" concern. Adopters new to the drill-down commands now have
an explicit playbook.

summary_posture_metrics_focus.E7 (3→4) — evidence refresh:
- summary / posture / metrics / focus all route through
  runPipelineWithSignals → engine.RunPipelineContext, locked by
  TestRunPipelineContext_RespectsCancelledContext +
  TestRunPipelineContext_CancelMidFlight.

docs/release/parity/scores.yaml: five cells refreshed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lifts portfolio.P5 (3→4) — the last achievable concrete error-UX
lift in the Align pillar without 0.3 corpus work.

cmd/terrain/cmd_insights.go runPortfolio:
- When portfolio analysis fails, surface a designed remediation
  block: names the three common adopter causes (snapshot
  construction failure, non-git root, permission errors) and
  points at `terrain analyze` for root-cause drill-down. Links
  to docs/schema/portfolio.md for multi-repo workflows
  (currently experimental in 0.2.0).

docs/release/parity/scores.yaml: portfolio.P5 (3→4).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 7, 2026

Terrain AI Risk Review

Metric Value
AI surfaces 13
Eval scenarios 17
Impacted scenarios 0
Uncovered surfaces 13

Decision: PASS — AI surfaces are covered.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 7, 2026

[RISK] Terrain — Merge with caution

High-severity gaps found in changed code.

Metric Value
Changed files 10 (4 source · 1 test)
Impacted units 9
Protection gaps 4
Tests selected 5 of 800 (<1% of suite)

Coverage gaps in changed code

  • cmd/terrain/cmd_analyze.go [LOW] — cmd_analyze.go has no observed test coverage.
    → Add unit tests for cmd_analyze.go.
  • cmd/terrain/cmd_insights.go [LOW] — cmd_insights.go has no observed test coverage.
    → Add unit tests for cmd_insights.go.
  • internal/reporting/empty_states.go [MED] — Exported class EmptyState has no observed test coverage.
    → Add unit tests for exported class EmptyState — this is public API surface.
  • internal/reporting/portfolio_report.go [MED] — Exported function RenderPortfolioSection has no observed test coverage.
    → Add unit tests for exported function RenderPortfolioSection — this is public API surface.
4 pre-existing issues on changed files
  • cmd/terrain/cmd_analyze.go [HIGH] — [blastRadiusHotspot] Changes to this file propagate to 183 tests (183 direct, 0 indirect). High blast radius increases regression risk.
  • cmd/terrain/cmd_insights.go [HIGH] — [blastRadiusHotspot] Changes to this file propagate to 183 tests (183 direct, 0 indirect). High blast radius increases regression risk.
  • internal/reporting/empty_states.go [HIGH] — [blastRadiusHotspot] Changes to this file propagate to 1898 tests (120 direct, 1778 indirect). High blast radius increases regression risk.
  • internal/reporting/portfolio_report.go [HIGH] — [blastRadiusHotspot] Changes to this file propagate to 1898 tests (120 direct, 1778 indirect). High blast radius increases regression risk.

Recommended tests

5 test(s) with exact coverage of 5 impacted unit(s). 4 impacted unit(s) have no covering tests in the selected set.

Confidence: 5 exact (5 tests selected)

Test Confidence Why
internal/insights/insights_bench_test.go exact test file directly changed
internal/reporting/empty_state_goldens_test.go exact exact coverage of EmptyStateKind, RenderEmptyState
internal/reporting/empty_states_test.go exact exact coverage of EmptyStateFor, EmptyStateMarkdown
internal/testdata/e2e_workflows_test.go exact exact coverage of RenderPortfolioReport
internal/testdata/golden_test.go exact exact coverage of RenderPortfolioReport

Owners: PMCLSF

Limitations
  • No coverage artifacts provided; protection gaps reflect missing data, not measured absence. Provide --coverage to improve accuracy.
  • Mixed test cultures reduce cross-framework optimization confidence. Consider standardizing on fewer frameworks.

Generated by Terrain · terrain pr --json for machine-readable output

Targeted Test Results

Terrain selected 5 test(s) instead of the full suite.

  • Go tests: passed

@pmclSF pmclSF merged commit b979a93 into main May 7, 2026
11 checks passed
@pmclSF pmclSF deleted the feat/0.2-pillar-lift-3 branch May 7, 2026 00:40
pmclSF added a commit that referenced this pull request May 9, 2026
…arks + analyze error UX (#172)

* feat(0.2): portfolio + explain schema docs + insights benchmarks + analyze error UX (Understand+Align lift)

Lifts seven cells across Understand and Align pillars without
touching the labeled-corpus cells.

docs/schema/portfolio.md (new) — portfolio.E4 (2→4):
- Canonical PortfolioSummary / TestAsset / Finding /
  PortfolioAggregates / OwnerPortfolioSummary contract.
- .terrain/repos.yaml manifest schema documented.
- Multi-repo aggregate output marked Experimental for 0.2.0
  (honest about partial shipping per the plan's pillar priority).

docs/schema/explain.md (new) — insights_impact_explain.E4 (3→4):
- `terrain explain <target>` dispatch table mapping every
  accepted target type → output shape.
- Each shape references its canonical schema doc.
- OwnerExplanation shape documented inline (only one not covered
  by an existing schema file).

internal/insights/insights_bench_test.go (new) — insights_impact_explain.E5 (3→4):
- BenchmarkBuild_Healthy / WithDepgraphResults / LargeSnapshot.
- Reference numbers: 2.5µs / 8µs / 40µs per op on Intel i7-8850H.
- Linear scaling — no quadratic regressions.

internal/reporting/empty_states.go + portfolio_report.go — portfolio.V3 (2→4):
- New EmptyNoPortfolio empty-state kind.
- RenderPortfolioReport now uses the designed empty state when
  TotalAssets == 0, replacing the bare two-line message.

cmd/terrain/cmd_analyze.go — core_analyze.P5 (3→4):
- analyzeFailureRemediation surfaces a designed remediation
  block on analysis failure. Three branches: timeout-exceeded
  (with --timeout-increase / scope-down / verbose-timing
  nudges), cancelled (re-run hint), generic (three common
  causes + verbose/json next steps).

insights_impact_explain.E7 (3→4) — evidence refresh:
- All three commands route through the cancellation-tested
  engine.RunPipelineContext path locked by
  TestRunPipelineContext_RespectsCancelledContext +
  TestRunPipelineContext_CancelMidFlight.

docs/release/parity/scores.yaml: seven cells refreshed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(0.2): benchmark baseline + drilling-into-findings doc + summary E7 cancellation

Lifts five more cells in the Understand pillar.

benchmarks/baseline.txt (new) — core_analyze.E5 (3→4):
- Reference performance numbers for the three benchmark suites that
  matter: engine pipeline (small/medium/large), insights builder
  (healthy/with-depgraph/large-snapshot), changescope PR-comment
  renderer (small/medium/large).
- Captured 2026-05 on Intel i7-8850H @ 2.60GHz with re-run
  instructions and notes.
- make bench-gate compares ratios against this baseline.

docs/user-guides/drilling-into-findings.md (new) —
insights_impact_explain.P3 (3→4) and P4 (3→4):
- Four-command ladder (analyze → insights → impact → explain) with
  worked examples per command.
- Full "how confidence is computed" section: detector confidence
  (structural / heuristic / runtime-aware), ConfidenceDetail
  Wilson/Beta intervals, test-selection confidence
  (exact / inferred / weak), coverage confidence.
- Round-trip example using a stable finding ID.
- Cross-references the schema docs.

Closes the audit's "no per-command 'how confidence is computed'
pages" concern. Adopters new to the drill-down commands now have
an explicit playbook.

summary_posture_metrics_focus.E7 (3→4) — evidence refresh:
- summary / posture / metrics / focus all route through
  runPipelineWithSignals → engine.RunPipelineContext, locked by
  TestRunPipelineContext_RespectsCancelledContext +
  TestRunPipelineContext_CancelMidFlight.

docs/release/parity/scores.yaml: five cells refreshed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(0.2): portfolio command error UX (Align lift)

Lifts portfolio.P5 (3→4) — the last achievable concrete error-UX
lift in the Align pillar without 0.3 corpus work.

cmd/terrain/cmd_insights.go runPortfolio:
- When portfolio analysis fails, surface a designed remediation
  block: names the three common adopter causes (snapshot
  construction failure, non-git root, permission errors) and
  points at `terrain analyze` for root-cause drill-down. Links
  to docs/schema/portfolio.md for multi-repo workflows
  (currently experimental in 0.2.0).

docs/release/parity/scores.yaml: portfolio.P5 (3→4).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant