Combine context accounting, running summaries, and updater UX fixes

[Niko4417]

Summary

Combines the three already-reviewed delivery branches into one traceable PR:

• Supersedes Add provider-calibrated token accounting #1802 for Repo Context Audit: add provider-calibrated token accounting #1766 provider-calibrated token accounting.
• Supersedes Repo Context Audit: add governed model running summaries #1803 for Repo Context Audit: add Model Gateway-backed structured running summaries #1765 governed model running summaries.
• Supersedes [codex] fix updater release state and restart UX #1810 for [User Finding]: Updater fetches unknown target #1809 updater release-state and restart UX fixes.

The combined branch keeps merge commits from all three source PRs so the evidence path remains inspectable. The only merge-resolution behavior changes preserve calibrated token accounting in prompt-budget paths, preserve system-scoped generated summaries, and keep the updater hardening/test-stability fixes from #1810.

Resolves #1765
Resolves #1766
Resolves #1809

Scope

• In scope: Model Gateway-backed structured compaction summaries, calibrated/fallback token accounting, prompt-budget diagnostics, context-quality gates, updater target/release-note/restart UX, update-window design-system evidence, and the test hardening needed to keep the integrated branch green.
• Out of scope: exact provider tokenizer dependency, new repository-understanding retrieval/indexing features, unrelated local runtime state cleanup, and changing updater install policy beyond the governed/manual-safe path already implemented.

Reuse And No-Duplication

• Existing Keiko functionality was inspected before implementation.
• This PR reuses, extends, generalizes, or consolidates existing functionality where practical.
• Any new implementation is limited to a documented capability gap in the linked issue.
• This PR does not introduce a parallel workspace, graph, relationship, policy, evidence, memory, connector, workflow, or UI subsystem where an existing subsystem can be extended.
• Refactoring or consolidation was considered when existing functionality was close but not shaped for this change.

Delivery Board

• Linked issue is in the public Keiko Product Delivery project.
• Linked issue has a valid Parent Epic: #<epic_number> unless this PR updates an epic container directly.
• Linked issue is attached as a GitHub sub-issue of its parent epic so it appears in the correct board swimlane.
• Parent epic is on the board with Classification: Epic, Status: Open Epics, and priority/order set for top-to-bottom implementation sequencing.
• Linked task/card issue is on the board with Classification: Task, Status: In Progress while active, inherited or explicit Priority, and Human Review Required: Yes.
• Project Status is In Progress while work is active, or Done only after merge and closure evidence.
• Project Workflow State is PR Open or Ready for Human Review.
• Owner / Agent, Branch, Pull Request, and Human Review Required are filled.
• Issue label reflects the current state: status: in progress, status: ready for human review, or status: done after merge.
• Autonomous agents did not merge into dev, enable auto-merge, close the issue, or bypass human review unless explicitly authorized by the human maintainer.

Product Impact

• UI or user workflow
• CLI or developer workflow
• Core generation engine
• Evidence, audit, or compliance artifact
• Security or supply chain
• Packaging, release, or npm publication
• Documentation or repository hygiene
• No user-facing behavior change

Update Impact

• Not release-impacting.
• Release-impact catalog or issue metadata updated.
• Release-note category: improvements, fixes, ui-polish.
• Priority: high.
• User-visible change: Long compacted chat sessions retain context more faithfully, prompt-budget diagnostics report calibrated/fallback token accounting, and Updates no longer shows target unknown when the current release is latest.
• Release-note bullet: Improves governed context compaction and calibrated model-context budgeting, and fixes updater release status, patch-note, and restart verification UX.
• Supported-from versions: next release after merge to dev.
• Affected state stores: compaction evidence manifests may include validated generated-summary metadata; no migration expected for updater state.
• User action required and remediation: after a manual update, restart Keiko and use Verify restart; failed verification now stays in the restart-required state with clear retry guidance.
• Internal-only rationale, if applicable: not internal-only.
• Release-owner review evidence: source issues Repo Context Audit: add Model Gateway-backed structured running summaries #1765, Repo Context Audit: add provider-calibrated token accounting #1766, and [User Finding]: Updater fetches unknown target #1809 include update-impact metadata; this PR reran npm run check:release-impact.

Verification

Required:

• Required GitHub checks pass before merge.
• Local verification commands or rationale are listed below.
• Reuse/extension/generalization evidence or gap rationale is listed below.

Local verification:

PASS GitHub CI on `f94782ad9b6b9c3a9135a7a7d4590355df833662`: ci, ui, build/scan/SBOM/smoke, macOS smoke, Windows smoke, CodeQL, dependency review, actionlint, and protected branch gate.

PASS .keiko-scripts/verify-receipt.sh 1765-1766-1809
  receipt: .git/keiko-verify/issue_1765-1766-1809-combined-delivery.json
  verified_sha: f94782ad9b6b9c3a9135a7a7d4590355df833662
  suite summary: 970 test files, 16395 passed, 1 skipped

PASS .keiko-scripts/ui-verify-receipt.sh 1765-1766-1809 -- npx playwright test --config tests/e2e/config/playwright.issue-1907-update-pr.config.ts --project=chromium
  receipt: .git/keiko-ui-verify/issue_1765-1766-1809-combined-delivery.json
  ui_verified_sha: f94782ad9b6b9c3a9135a7a7d4590355df833662

PASS .keiko-scripts/audit-receipt.sh 1765-1766-1809 --findings 0 --user-facing true
  receipt: .git/keiko-audit/issue_1765-1766-1809-combined-delivery.json
  audited_sha: f94782ad9b6b9c3a9135a7a7d4590355df833662

PASS .keiko-scripts/verify-gate.sh
PASS .keiko-scripts/audit-gate.sh
PASS npm run check:context-quality
PASS npm run check:release-impact
PASS npm run clean && npm run build && npm run build:ui && npm run prepare:bin && npm run prune:package-native-optionals && npm run prune:package-build-artifacts && npm run check:package-surface
PASS KEIKO_SMOKE_PACK_IGNORE_SCRIPTS=1 npm run smoke:install
PASS changed-file Prettier check with --ignore-unknown
PASS npm run test:e2e:update-pr-1907

PASS targeted suites:
- Context/accounting: 12 files, 504 tests
- Structured summary/chat compaction: 10 files, 179 tests
- Updater server: 6 files, 71 tests
- Updater UI/window: 4 files, 378 tests
- Shared merge hardening: 8 files, 52 tests
- Text-safety/context/updater smoke target: 3 files, 101 tests

Review-thread evidence:
- #1802: no review threads.
- #1803: two bot threads in text-safety; both resolved, one outdated, and regression tests pass on the combined branch.
- #1810: no review threads.

Reuse / gap rationale:

#1766 extends ContextProfile, Model Gateway capability metadata, existing prompt-budget diagnostics, allocator paths, and context-quality gates instead of adding a separate token-budget subsystem. Exact provider tokenizer integration stays out of scope; calibrated counters and explicit fallback accounting satisfy the governed issue scope without adding supply-chain risk.

#1765 extends the existing conversation compaction and evidence model through Model Gateway-routed structured summaries. Generated summary content is bounded, validated, redacted, persisted with attribution, and resurfaced as system-scoped context rather than as a user-authored turn.

#1809 extends the governed updater session/preflight/remediation state machine and existing UpdateWindow/WindowFrame design-system surfaces. It does not add a parallel updater path; it fixes current/latest target derivation, current-release patch-note rendering, manual update command UX, restart verification failure handling, and window-scroll clipping.

Select only what applies:

• UI behavior manually verified or covered by tests.
• CLI behavior verified with command output or tests.
• Core logic covered by unit, integration, property, or fixture tests.
• Security-sensitive change reviewed for trust boundaries, secrets, external calls, and generated artifacts.
• Supply-chain or package-surface change verified with package, license, lockfile, SBOM, or npm dry-run checks.
• Documentation or Markdown change verified by the repository link check or a targeted local equivalent.
• Release-impacting change verified with npm run check:release-impact or an explicit rationale.
• Not applicable items are explained below.

Not applicable rationale:

• Exact tokenizer support is intentionally not added; the accepted implementation uses governed calibrated counters plus deterministic fallback diagnostics.
• Source PRs Add provider-calibrated token accounting #1802, Repo Context Audit: add governed model running summaries #1803, and [codex] fix updater release state and restart UX #1810 remain unmerged and are superseded by this combined PR for a single human-review path.

Review And Closure

• The PR implements only the linked issue scope.
• Actionable review findings are fixed or explicitly dispositioned.
• Unresolved review threads are resolved before merge.
• Checks are repeated after the latest pushed fix.
• Issue acceptance criteria and closure evidence are updated only where evidence exists.
• The linked issue records reuse, extension, generalization, or new-gap rationale before closure.
• Delivery board status is updated before requesting final maintainer review.
• Use Resolves #<issue_number> only when this PR should close the issue.

Risk Notes

• Model-written summaries are treated as untrusted generated content: schema-constrained, bounded, redacted, validation-stamped, and never injected as role: "user".
• Calibrated token counters improve budgeting but are not exact tokenizers; diagnostics preserve source transparency and deterministic fallback remains available.
• Updater release metadata depends on npm/GitHub reachability; unavailable metadata now degrades explicitly instead of showing a misleading unknown target when latest/current data is known.
• This PR targets dev, so it is human-gated and must wait for GitHub CI plus maintainer review.

[Niko4417]

Manual test plan

This PR includes user-facing updater changes. The dedicated Playwright plan covers the automatable parts; the manual path below is what a human reviewer can repeat visually.

1. Open Settings -> Updates when Keiko is already on the latest release.

   • Expect the status card to say Keiko is up to date with Current 0.2.12 -> target 0.2.12.
   • Expect no target unknown or degraded-status copy when live current-release metadata exists.
   • Expect Patch notes to show grouped current-release notes without duplicating the summary as an extra headline.

2. Simulate an update-available manual path from 0.2.11 to 0.2.12.

   • Expect Manual update instructions to open as a styled disclosure panel, not a chopped standard-details block.
   • Expect copyable npm/Yarn command frames using the exact target version command, for example npm install --global --ignore-scripts @oscharko-dev/keiko@0.2.12.
   • Expect the command copy button to give in-place button feedback without moving layout.

3. Open release notes from the update window.

   • Expect release notes to open in a new tab/window and not replace the Keiko app tab.

4. Scroll the update window to the top and bottom.

   • Expect the scrollbar and window content to remain inside the rounded WindowFrame bounds with consistent clipping.

5. Click Verify restart before restarting Keiko.

   • Expect the flow to remain in the restart-required state.
   • Expect the retry message below the first banner explaining that the restart was not detected yet.
   • Expect no fallback to the install-update state.

Automated Playwright evidence passed at HEAD:

.keiko-scripts/ui-verify-receipt.sh 1765-1766-1809 -- npx playwright test --config tests/e2e/config/playwright.issue-1907-update-pr.config.ts --project=chromium
ui_verified_sha: f94782ad9b6b9c3a9135a7a7d4590355df833662

Dedicated shortcut for local reruns:
npm run test:e2e:update-pr-1907

[Niko4417]

Consolidated keiko-issue-audit receipt

Audited branch: issue/1765-1766-1809-combined-delivery
Audited HEAD: f94782ad9b6b9c3a9135a7a7d4590355df833662
Source PRs superseded: #1802, #1803, #1810
Issues covered: #1765, #1766, #1809
Result: findings=0, user_facing=true

Acceptance criteria checked

• Repo Context Audit: add Model Gateway-backed structured running summaries #1765: structured model-written compaction summaries preserve ordinary decisions, constraints, files/symbols, debugging context, and open threads; generated summary text is never resurfaced as role: "user"; invalid/unsafe/oversized/unavailable summaries fail closed or fall back explicitly; compaction evidence preserves attribution and validation state.
• Repo Context Audit: add provider-calibrated token accounting #1766: configured models can use calibrated token accounting; fallback accounting remains deterministic and explicitly reported; prompt assembly remains bounded across representative chat/document/memory/tool-observation inputs; no new tokenizer dependency or provider-specific unsafe surface was introduced.
• [User Finding]: Updater fetches unknown target #1809: latest/current updater target no longer degrades to target unknown when current release data is available; builds ahead of npm latest keep the running version as target; patch notes show the newest applicable release notes; manual update/restart UX is simple, copyable, theme-safe, and restart verification failures remain in the correct restart-required state.

Review observations resolved

• Reviewability: PR body and comments keep the source PR mapping, superseded PRs, issue coverage, and source review-thread disposition visible for the combined human-review path.
• Repo Context Audit: add provider-calibrated token accounting #1766 metadata: Existing Capability Review, Expected Verification, Deliverables, and Acceptance Criteria are checked where evidence exists; project fields now point to the combined branch and PR Combine context accounting, running summaries, and updater UX fixes #1907. Lifecycle-only boxes remain unchecked until merge/closure.
• Updater ahead-of-registry path: versionComparison < 0 now reports the running version as target and fetches that release's notes instead of showing the older registry latest as target.
• Compaction recency invariant: numeric turn ordering is explicitly tested; if a matching run id violates the chat-<hash>-t<turn> invariant, the loader falls back to loading matching manifests and uses persisted timestamps for recency instead of partially trusting filename order.
• Automated updater UX: added tests/e2e/update-pr-1907.spec.ts and tests/e2e/config/playwright.issue-1907-update-pr.config.ts covering current release notes, manual commands/copy/link behavior, WindowFrame scroll clipping, and restart verification mismatch.

Role-lens audit

• explorer: mapped the combined diff across contracts, model gateway, server prompt/compaction paths, evidence redaction, updater preflight/session routes, update UI, WindowFrame, release-impact checks, and test hardening.
• architect: confirmed the branch extends existing ContextProfile, ModelGateway capability metadata, conversation compaction, evidence manifests, updater session/preflight/remediation state, and design-system surfaces rather than introducing duplicate subsystems.
• security-auditor: confirmed generated summaries are bounded, redacted, validated, and system-scoped; token counter internals are not projected to browser-safe capability data; updater external metadata remains explicit/degraded on failure; no secrets or customer data are introduced.
• performance-engineer: checked calibrated counting remains simple arithmetic on known metadata, prompt budgeting stays deterministic, summary generation is timeout/failure bounded, QI/cache timing tests were relaxed only where prior assertions compared cache-hit paths unrealistically, and compaction resurfacing keeps prefix-scoped loading for valid run ids.
• a11y-auditor: confirmed updater UI changes remain covered by the existing design-system evidence under docs/design-system/evidence/1696/ and by the new dedicated Playwright receipt at HEAD.
• pr-reviewer: reviewed merge conflict resolutions and source PR feedback. Add provider-calibrated token accounting #1802 and [codex] fix updater release state and restart UX #1810 have no review threads. Repo Context Audit: add governed model running summaries #1803 has two bot threads in text-safety.ts; both are resolved, one is outdated, and the resolved guard/regression tests are present in this combined branch.
• verifier: reran the full receipt gates and the dedicated updater Playwright plan at the exact shipping SHA.

Evidence

PASS PATH=<Node 24 runtime> .keiko-scripts/verify-receipt.sh 1765-1766-1809
verified_sha: f94782ad9b6b9c3a9135a7a7d4590355df833662
suite summary: 970 test files, 16395 passed, 1 skipped

PASS PATH=<Node 24 runtime> .keiko-scripts/ui-verify-receipt.sh 1765-1766-1809 -- npx playwright test --config tests/e2e/config/playwright.issue-1907-update-pr.config.ts --project=chromium
ui_verified_sha: f94782ad9b6b9c3a9135a7a7d4590355df833662

PASS .keiko-scripts/audit-receipt.sh 1765-1766-1809 --findings 0 --user-facing true
audited_sha: f94782ad9b6b9c3a9135a7a7d4590355df833662

PASS .keiko-scripts/verify-gate.sh
PASS .keiko-scripts/audit-gate.sh
PASS npm run test:e2e:update-pr-1907
PASS targeted server/UI suites and changed-file Prettier/ESLint checks

Residual risks

• Calibrated token accounting is not exact tokenizer support; diagnostics make that explicit and deterministic fallback remains available.
• Release metadata still depends on npm/GitHub reachability; the user-facing state now degrades explicitly instead of implying an unknown target when current/latest data is known.
• The local verify receipt had to be run with Node 24 because dependency-cruiser rejects Node 25; this is documented in the evidence command and does not change shipped code.
• This PR targets dev, so merge remains human-gated after GitHub CI.

[Niko4417]

CI Follow-Up

Final pushed SHA: f94782ad9b6b9c3a9135a7a7d4590355df833662

GitHub checks are green on the final combined branch head:

• ci: pass (Coverage quality gates completed inside the CI job; total ci job time 22m15s)
• ui: pass
• Build, scan, SBOM, smoke: pass
• Cross-platform smoke (macos-latest): pass
• Cross-platform smoke (windows-latest): pass
• CodeQL: pass
• Review dependency diff (dev/main): pass
• actionlint: pass
• Protected branch gate: pass
• Verify pinned action SHAs: pass
• Analyze (actions) and Analyze (javascript-typescript): pass

Review-thread check on #1907: no review threads returned by the PR GraphQL reviewThreads query.

The dedicated updater Playwright plan is now part of the branch and is covered by:

npm run test:e2e:update-pr-1907

Source PRs #1802, #1803, and #1810 remain superseded by this combined PR for one final human-review path.

[Niko4417]

Audit review — combined delivery (#1765 / #1766 / #1809)

Read-first audit across contracts, model-gateway, server prompt/compaction, evidence redaction, updater preflight/session, and the update UI. CI is green at 0b0c0d9 (ci, ui, cross-platform smoke, CodeQL, dependency review, pinned-SHA + protected-branch gates all pass). No blocking findings. Details below.

What holds up well

• Repo Context Audit: add provider-calibrated token accounting #1766 calibrated token accounting — deterministic, clamped arithmetic (calibratedTokenCount guards with Math.max(0, …) even against directly-constructed literals), strict config parsing with unknown-key rejection (assertKnownTokenAccountingKeys), and counterId is correctly projected out of the browser-safe capability surface (projectSafeCapabilities / SafeModelCapability). The ADR-0019/0094 wire-tier invariant comment was updated to match. Fallback stays the explicit default source.
• Repo Context Audit: add Model Gateway-backed structured running summaries #1765 governed model summaries — untrusted model output is treated as hostile: absolute-path and pseudo-role-marker redaction in safeLine, the pseudo-role scanner is a linear char walk (not a catastrophic regex), rejected/non-valid summaries are never resurfaced, and failure states persist empty content so governed metadata can't leak text into future prompts. The follow-up commits that made validationState: "accepted" actually reachable and restored structured-field synthesis are the right fixes.
• [User Finding]: Updater fetches unknown target #1809 updater target-unknown — root cause addressed at the derivation layer: when latest == current the report now carries targetVersion + live current-release notes instead of target unknown; <details> blocks are stripped from release bodies before extraction; the restart command preview validates host against an allowlist, bounds the port, rejects null/newline state dirs, and shell-quotes args. Manual-test-plan comment + Playwright UI receipt at HEAD are present.

Non-blocking observations for the human reviewer

1. Reviewability of the combined PR. 4,784 / -628 across 89 files spanning three unrelated concerns is inherently harder to review than the three superseded PRs were individually. The rationale (single human-review path over already-reviewed Add provider-calibrated token accounting #1802/Repo Context Audit: add governed model running summaries #1803/[codex] fix updater release state and restart UX #1810) is documented and reasonable, but the merge-commit chain is the only place the per-issue boundary survives — worth confirming the source PRs' review threads are all resolved before merge (PR body states Add provider-calibrated token accounting #1802/[codex] fix updater release state and restart UX #1810 had none, Repo Context Audit: add governed model running summaries #1803's two bot threads resolved).
2. Issue Repo Context Audit: add provider-calibrated token accounting #1766 metadata. The issue's own "Existing Capability Review" and delivery-board checklists are still unchecked in the issue body, even though the PR asserts board fields are updated. Process nit, not code — worth reconciling before closure so the reuse rationale lives on the issue.
3. versionComparison < 0 path (update-preflight-routes.ts) — running a build ahead of the published latest still reports registry.latestVersion as the target. Matches prior behavior and is a rare edge, but the displayed target is then the older published version; fine to leave, flagging for awareness.
4. newestRunIds recency pre-filter (chat-compaction-resurfacing.ts) — the new turn-suffix parse is a good perf win and is well-guarded (non-numeric suffixes fall back to insertion order), just note the resurfacing set now depends on run-id naming staying …-t<turn>.

Governance is respected: targets dev, human-gated, no auto-merge, evidence stays redacted (counts/hashes/status, no bodies). Recommend proceeding to human review — the observations above are for the maintainer's attention, not merge blockers.