Agent-runtime audit fixes (Phase 1 complete; 2 & 3 in progress)

CHANGELOG.md

@@ -20,16 +20,29 @@ date, grouped by `Added` / `Changed` / `Fixed` / `Removed`.
 - **A built-in calendar for every date field.** Due dates, sprint and initiative dates, roadmap dates, the time-log range, and snooze now open Forge's own themed calendar instead of the browser's native date box — same look on every browser and OS, with month navigation and a clear button.
 - **Draft or improve an issue description with AI.** Next to the Description heading there's now an AI button — **Draft with AI** writes a description from the issue title when there's none yet, and **Enhance** rewrites an existing one for clarity and structure. You see the suggestion (with a current-vs-suggested view for an enhance) and choose **Apply** or **Discard** — nothing changes until you accept it. Appears only when AI is enabled for the workspace.
 - **Create a label without leaving the issue.** The label picker on an issue now has a search box — type to filter, and (if you're a workspace admin) pick **Create "<name>"** to make a new label with a name and color right there and apply it in one step. No more detour to Settings → Labels first.
+- **Move issues between workspaces (instance admin).** A new **Admin → Move issues** page re-homes issues into another workspace: pick source and target, paste the issue ids, and **Preview** shows exactly what will happen — the new keys (issues are renumbered into the target), how each status and label maps, which labels get dropped, and which issues are *blocked* (anything with agent runs, plans, or artifacts is refused so nothing is corrupted). Confirm to move; a record is written in both workspaces' audit logs.
+- **Cap self-service workspace creation.** Operators can set `MAX_WORKSPACES_PER_USER` to limit how many workspaces a single (non-admin) user can create; the default is unlimited, so nothing changes unless you set it.
+- **Approve or reject a paused agent run right where you see it.** When an agent pauses for permission (a Codex/Hermes run flagged a command or file change), the run row in Mission Control's Live tab now shows **Approve** / **Reject** inline — no detour to the Command Center.
+- **See a plan's wall-clock burn.** The plan cockpit's budget meter now shows elapsed time vs. the wall-time cap alongside cost, so a plan approaching its time budget is visible at a glance (matching the goal cockpit).
 
 ### Changed
 
+- **Plans and the review inbox update live.** The Plans list and the Review-gate inbox now refresh on their own as plans, steps, runs, and gates change — no manual **Refresh**, matching the Goals list.
+- **Orchestration screens are fully themed.** Every dropdown on the Plans, Goals, and Runtime-settings screens, plus the new-plan / new-goal / edit-goal dialogs, now use Forge's own pickers and one consistent dialog instead of native browser controls.
+
 - **Confirmations, prompts, and dropdowns use Forge's own UI, not the browser's.** "Are you sure?" confirmations (deleting a note, removing an admin, restarting a run), the GitHub App setup prompts, and a batch of dropdowns now use Forge's themed in-app dialogs and pickers instead of the operating system's native popups — consistent look, keyboard-friendly, on-brand. (More dropdowns are still being converted.)
 - **The label color picker is now in-app.** Choosing a label color (in Settings → Labels and the new inline create) uses themed swatches plus a hex field instead of the operating system's native color dialog — consistent with the rest of Forge.
 
 ### Fixed
 
 - **The Coach stops posting junk or duplicate comments.** It now discards empty or meta AI responses (e.g. "Posted the diagnostic comment…") instead of posting them, and won't comment on the same issue more than once in 24 hours — so a stuck issue gets one useful diagnostic, not an hourly pile-up.
 - **AI triage works again — and tells you what's wrong when it can't.** The "AI triage suggestion" card on an issue could get stuck showing a bare *"AI triage unavailable."* with only a Retry, because some AI providers reply in plain text instead of the structured format Forge expected — so every suggestion was thrown away. Forge now reads those plain-text replies too, so triage produces a real priority / label / agent suggestion. When triage genuinely can't run, the card now explains why and links straight to **Settings → Workspace → AI** to fix it, instead of leaving you guessing.
+- **A goal or plan that hits its budget actually stops spending.** When a plan reaches its cost or time cap (or you abandon its goal), its remaining steps stop dispatching immediately — previously a blocked plan could keep launching agents and spend the very budget you were asked to approve.
+- **Finished work stays finished.** A late or duplicate reviewer verdict can no longer knock a completed step back into "to-do" and re-run it.
+- **Agent runs aren't falsely marked "stalled" after a deploy or restart.** A run whose runtime briefly can't report status — or that's quietly mid-work on a long step — now stays running, and is only marked stalled when it's genuinely idle.
+- **Approving or stopping an agent run tells you if it didn't go through.** A failed Approve or Stop now surfaces an error instead of showing success while the run stays blocked at the runtime.
+- **Goals and Plans tell a load error apart from an empty list.** A network hiccup now shows "couldn't load … Retry" instead of "No goals yet" / "Goal not found — may have been abandoned", which looked like your work had been deleted.
+- **A freshly registered runtime is no longer a dead-end.** You can open a just-registered local daemon's settings — run its self-test, add credentials, bind a repo — straight from the global **Runtimes** page, without waiting for an agent to use it first.
 
 ### Changed
 

CLAUDE.md

@@ -269,10 +269,15 @@ Claude adapter spawn:
 `FORGE_CLAUDE_BIN`. Missing binary → `[OFFLINE]` reply via
 `chat.finalizeDraft`, no crash.
 
-Known v1 gaps: chat dispatch sees only `{threadId, messageId, agentId,
-role}` from SSE; there's no `chat.getThread` MCP yet, so the prompt to
-the local CLI is a placeholder. AGENT_ASSIGNED handler stubs a
-placeholder comment.
+Status (the old "v1 gaps" are closed): chat dispatch loads real thread
+history via `chat.getThread`; `AGENT_ASSIGNED` runs a full agent loop
+(`dispatch/issue-loop.ts`), not a stub; `runtimes.list` / `agents.list` /
+`chat.getThread` MCP tools are all shipped. On (re)connect the daemon now
+reconciles its unacked inbox (`agent.inbox.list`) so dispatch dropped
+during a deploy/reconnect is recovered, and repeated 401/403s exit the
+daemon non-zero ("run `forge login`") instead of looping. Real remaining
+gaps: one linked agent per runtime (no multi-agent-per-runtime fan-out),
+and ACP isn't wired for the issue-loop path.
 
 ## Attachments
 

DEVLOG.md

@@ -2,6 +2,130 @@
 
 > Append-only session log. Read at session start. Update at session end.
 
+## 2026-07-06 — Agent-runtime audit: Phase 3 UI consistency (P3.2)
+
+Orchestration-surface polish on `worktree-audit-fixes`. Six sub-items, all
+typecheck + lint + build clean.
+
+- **Tokens.** Swept the last ad-hoc `emerald-*` / `amber-*` (+ the dead
+  `destructive` class on the review page) to the semantic `--success` /
+  `--warning` / `--danger` tokens across plans/goals/review pages,
+  `orchestration-ui/status.ts`, `orchestration/{crew-roster-panel,step-node}`,
+  and `settings/runtimes`. The tokens already flip light/dark so the redundant
+  `dark:` variants collapsed. `destructive` is **not** a defined token (Tailwind
+  emits nothing) — it's a latent app-wide bug in ~45 files; only fixed the
+  review-page instance here, noted the rest for P3.4.
+- **Live surfaces.** Plans index + review inbox now `useRealtime` (execution-
+  plan/step/agent-run and review-gate resp.), mirroring the goals index — no
+  more frozen list until manual Refresh.
+- **Inline run approval.** `RunRow` renders the shared `RunApprovalCard`
+  (Approve session/once + Reject) instead of the "open Command Center" dead-end
+  its own doc comment already claimed it hosted.
+- **Wall-time.** Plan detail budget meter now consumes the P1.7 `startedAt`
+  (elapsed / cap bar); `hasBudget` also trips on a wall-time-only cap.
+- **Selects → Combobox.** All 7 native `<select>` in the orchestration +
+  runtime-settings surfaces (adapter / sandbox / approval / crew / plan-status /
+  2× step-status) are now the themed `Combobox`. `runtime-management.spec.ts`
+  rewired from Playwright `selectOption` (native-only) to the role="combobox"
+  trigger + role="option" contract. The app-wide `react/forbid-elements` guard
+  stays `warn` (~58 sites remain, untouched).
+- **Modals → QuickForm.** The 3 hand-rolled orchestration modals (new-plan,
+  new-goal, GoalEditModal) now use `QuickForm` — one graphite scrim, Enter/Esc,
+  draft-safe, inline error banner. GoalEditModal `onSave` + the goal-router shim
+  gained `mutateAsync` so a failing save keeps the modal open with the server
+  message instead of a fire-and-forget toast.
+
+Left in P3.2 scope but intentionally out: the ~58 other native selects and the
+app-wide `destructive` dead-class (both broader than these surfaces).
+
+## 2026-07-06 — Agent-runtime audit: Phase 2 (integrity + security)
+
+Generalized the Phase-1 patches into their classes + fixed the two security holes.
+Commits on `worktree-audit-fixes`.
+
+- **Budget/lifecycle integrity (`orchestration-service.ts`).** `reapPlanRuns()`:
+  abandonGoal + re-decompose/re-generate cancel non-terminal steps and
+  stop/abandon in-flight runs (best-effort `connector.stop`); a superseded prior
+  RUNNING plan is CANCELED, not just demoted. `assertNoStepCycles()` (Kahn) rejects
+  dependency-cycle plans in both create paths. `sweepOrchestrationBudget()` — new
+  60s worker watchdog enforcing plan wall-time independent of cost events + logging
+  wedged plans.
+- **Dispatch (`run-dispatcher.ts`).** Poll-mirrored RUNS cost now flows into
+  `applyRunCostToPlan` (delta vs last-known, persisted per tick) so poll-only cost
+  trips `maxTotalCostUsd`. `maxConcurrent` enforced on the RUNS dispatch path
+  (`startNewRuns`). A disabled runtime's in-flight runs are left alone (disable
+  blocks new dispatch only) instead of being sentinel-STALLed. Unresolvable
+  connector annotates the run instead of silent-spin.
+- **Security.** `setMemberRole` owner-only gate (a non-owner ADMIN can't self-
+  promote to OWNER then delete the tenant). `ADMIN_EMAIL` env bootstrap honored
+  only while zero INSTANCE_ADMIN exist; `auth.ts` no longer re-stamps INSTANCE_ADMIN
+  every login — a demoted operator stays demoted (`trpc.ts` + `data.ts` + `auth.ts`).
+- **CLI (`daemon.ts`).** Reconcile-on-(re)connect via `agent.inbox.list` (recovers
+  dropped AGENT_ASSIGNED across a deploy) + fatal-auth exit on repeated 401/403.
+- **Atomicity/dedup.** `recordUsage` computes its cost delta under a
+  `SELECT … FOR UPDATE` row lock (no double-count). `runtimes.register` upserts on
+  (workspaceId, name, kind) instead of stacking duplicates.
+- **P2.7 Codex restart.** The confirmed false-STALL is resolved by the Phase-1
+  `unknown`-is-non-terminal fix (Codex `getStatus` returns `unknown` for a run lost
+  after a worker restart → stays ACTIVE, watchdog arbitrates). Full WebSocket-
+  session reattach (durable cross-process session state) is a Codex-connector
+  re-architecture, deliberately deferred — out of scope for a fix pass.
+
+Verification: `pnpm typecheck` clean; `pnpm build:cli` clean; orchestration (26) +
+members (18) + mcp (118) tests green; new regression tests for cycle-rejection,
+abandon-cancels-steps, and non-owner-can't-grant-OWNER. Phase 3 (cross-workspace
+move, UI consistency, connector parity, view-hierarchy legibility, housekeeping)
+pending.
+
+## 2026-07-06 — Agent-runtime audit: Phase 1 safety fixes
+
+Acted on the 2026-07-06 agent-runtime audit (73 findings; report artifact +
+[[agent-runtime-audit-2026-07]] memory). Phase 1 = the small, confirmed
+safety/lifecycle/false-terminal fixes. Landed as four commits on branch
+`worktree-audit-fixes`.
+
+- **Orchestration loop (`orchestration-service.ts`).** `cascadeReadiness` and
+  `transitionStepToReady` now load `plan.status` and early-return unless RUNNING
+  — a BLOCKED/CANCELED plan stops cascading finishing steps into fresh dispatch
+  (was: kept spending past the budget block / after abandon). `recordVerdict`
+  throws CONFLICT on a settled step (DONE/BLOCKED/CANCELED) so a stale/dup verdict
+  can't reopen finished work. `ExecutionPlan.startedAt` (migration **0091**)
+  stamped in `activatePlan`; `checkAndBlockBudget` measures wall-time from
+  `startedAt ?? createdAt` so planning + approval-wait isn't charged to execution.
+- **Migration 0091** also adds `AgentRun @@index([status,lastEventAt])` +
+  `@@index([assignmentEventId])` — the cross-tenant 5s/60s sweeps
+  (`pollActiveRuns`/`ensureSubscriptions`/stale watchdog) previously seq-scanned
+  (every AgentRun index led with `workspaceId`). Left the pre-existing
+  `ExternalResource` rename-index drift out of the migration (unrelated).
+- **Run dispatch (`run-dispatcher.ts` / `hermes-runs.ts`).** `getStatus`
+  `'unknown'` is now NON-terminal (leave ACTIVE, let the stale watchdog arbitrate)
+  — fixes the Codex worker-restart / deploy false-STALL and momentary blips.
+  `mapStatus` maps an unrecognized non-empty status → `running` (not `unknown`).
+  On a live `running` poll we always bump `lastEventAt` even when the step label
+  is unchanged, so a quiet-but-alive run isn't watchdog-STALLED. Hermes
+  `approve`/`stop` now inspect `res.ok` and throw; `agent-run.approve` (reject too)
+  surfaces the failure as a TRPCError instead of clearing the block + returning ok.
+  `runtime.register` now calls `assertEndpointTransport` like create/update.
+- **CLI daemon (`daemon.ts`).** `acquirePidLock()` writes the pid via
+  `fs.open(..., "wx")` (O_CREAT|O_EXCL) before SSE opens → closes the
+  double-daemon TOCTOU. `refreshLinkedAgent` returns `undefined` on a transient
+  `agents.me` failure vs `null` on genuine unlink; the heartbeat only overwrites
+  `linkedAgent` on a definitive answer (was: one blip nulled linkage, dropping all
+  dispatch ~60s).
+- **UI.** Goals/Plans list+detail get an `isError` branch (themed "couldn't load …
+  Retry") so a fetch failure isn't rendered as "No goals yet"/"Goal not found".
+  `global.runtimes` returns each runtime's `homeWorkspace`; the global Runtimes
+  settings gear routes via `workspacesInUse[0] ?? homeWorkspace` so a fresh
+  LOCAL_DAEMON isn't a read-only dead-end. Widened the `useGoalRouter` query shim
+  type with `isError`/`refetch`.
+
+Verification: `pnpm typecheck` clean; `pnpm lint` 0 errors (only pre-existing
+native-`<select>` warnings — Phase 3 P3.2); orchestration suite **24 passed**
+(4 new Phase-1 regression tests); `pnpm build:cli` clean. Local isolated stack on
+:55432/:56379. Phase 2 (budget/lifecycle integrity, delivery idempotency, the two
+security holes) + Phase 3 (cross-workspace move, UI consistency, connector parity)
+still pending — see the audit memory.
+
 ## 2026-06-28 — AXI-90 QuickCreate GitHub issue/PR import
 
 Enhanced the ⇧C QuickCreate issue overlay so a pasted GitHub issue/PR URL (or

prisma/migrations/0091_plan_startedat_and_agentrun_sweep_indexes/migration.sql

@@ -0,0 +1,12 @@
+-- AlterTable: wall-time budget clock starts at execution (activatePlan), not
+-- decompose, so planning + approval-wait isn't charged against the run clock.
+ALTER TABLE "ExecutionPlan" ADD COLUMN     "startedAt" TIMESTAMP(3);
+
+-- CreateIndex: status-leading index for the cross-tenant worker sweeps
+-- (pollActiveRuns / ensureSubscriptions / stale watchdog) that filter on
+-- status with no workspaceId and previously seq-scanned every 5s/60s.
+CREATE INDEX "AgentRun_status_lastEventAt_idx" ON "AgentRun"("status", "lastEventAt");
+
+-- CreateIndex: dedup lookup on the dispatch path (has this AGENT_ASSIGNED
+-- event already opened a run?).
+CREATE INDEX "AgentRun_assignmentEventId_idx" ON "AgentRun"("assignmentEventId");

prisma/schema.prisma

@@ -2594,6 +2594,15 @@ model AgentRun {
   /// Collapse stacked runs: list heads with `supersededByRunId IS NULL`,
   /// expand a thread by following the chain.
   @@index([supersededByRunId])
+  /// Cross-tenant worker sweeps (pollActiveRuns / ensureSubscriptions /
+  /// stale watchdog) filter on `status` with NO workspaceId, so the
+  /// workspaceId-leading indexes above can't serve them and they seq-scan
+  /// every 5s/60s. A status-leading index keeps those hot sweeps off a
+  /// sequential scan as AgentRun history grows.
+  @@index([status, lastEventAt])
+  /// Dedup lookup on the dispatch path: has this AGENT_ASSIGNED event
+  /// already opened a run for the agent?
+  @@index([assignmentEventId])
 }
 
 /// Append-only timeline for an AgentRun. Drives the live pulse strip and
@@ -3118,6 +3127,13 @@ model ExecutionPlan {
   maxTotalCostUsd    Float?
   maxWallTimeMinutes Int?
   totalCostUsd       Float               @default(0)
+  /// When execution actually began — activatePlan flips DRAFT/APPROVED →
+  /// RUNNING and stamps this. Wall-time budget is measured from here, NOT
+  /// createdAt, so planning + approval-wait time isn't charged against the
+  /// execution clock (a plan approved days after decompose shouldn't BLOCK
+  /// on its first cost record). Null until first activation; legacy rows
+  /// fall back to createdAt.
+  startedAt          DateTime?
   /// Marks the live decompose attempt for a goal. Prior attempts are
   /// flipped false when a new decompose runs.
   isActiveAttempt    Boolean             @default(true)

src/app/(app)/admin/move-issues/page.tsx

@@ -0,0 +1,21 @@
+"use client";
+import { AdminPage } from "@/components/admin-shell/admin-shell";
+import { AdminMoveIssues } from "@/components/admin-shell/admin-move-issues";
+
+/**
+ * `/admin/move-issues` — instance-admin surface to move issues between
+ * workspaces (audit ask #2). Preview the full remap before executing.
+ */
+export default function AdminMoveIssuesPage() {
+  return (
+    <AdminPage
+      activePath="/admin/move-issues"
+      crumbs={["Instance", "Move issues"]}
+      eyebrow="Instance"
+      title="Move issues between workspaces"
+      subtitle="Re-tenant issues into another workspace — preview the remap first"
+    >
+      <AdminMoveIssues />
+    </AdminPage>
+  );
+}

src/app/(app)/settings/runtimes/page.tsx

@@ -122,7 +122,11 @@ export default function RuntimesPage() {
                     : r.health.tone === "warning"
                       ? "border-warning/30 bg-warning/5"
                       : "border-border bg-card/40";
-                const settingsWorkspace = r.workspacesInUse[0];
+                // Prefer a workspace the runtime is actively used in, but
+                // always fall back to its home workspace so the settings link
+                // is never a dead-end (a freshly registered daemon has no
+                // workspacesInUse yet but still needs self-test / secrets).
+                const settingsWorkspace = r.workspacesInUse[0] ?? r.homeWorkspace;
                 return (
                   <div key={r.id} className={`flex flex-col gap-3 rounded-lg border p-4 ${tone}`}>
                     <header className="flex items-start gap-2">