bug(tui): sub-agent fanout plus hidden worktrees can saturate the TUI during release work

[Hmbown]

Summary

During v0.8.46 release work, the TUI hit the max-agents sidebar state (5 running / 5) while background shell work and multiple sub-agents were active. The first-pass diagnosis does not look like one Rust process simply pegging the machine. It looks like two pressure sources compounding:

1. a sub-agent fanout burst that repeatedly saturated the agent cap, and
2. a very large release-tranche checkout shape where hidden .worktrees/ can still be expensive for any workspace walker that bypasses gitignore.

This should be fixed in v0.8.46 because this is exactly the release-stewardship workload we want CodeWhale to handle without the UI feeling wedged.

Local diagnostic evidence

Observed on 2026-05-26 while working in the CodeWhale checkout:

• Recent audit metadata showed a same-session YOLO fanout burst: 6 agent_open approvals at 13:26:26Z, 6 more at 13:27:29Z, another at 13:29:31Z, then agent_close/agent_open activity again around 13:30:36Z.
• The visible TUI sidebar showed the cap saturated (5 running / 5) while completed/finishing agents still occupied attention in the panel.
• Process sampling showed multiple codewhale-tui instances alive, but the active TUI processes were not individually consuming extreme memory. This points more toward orchestration/backpressure and blocking I/O pressure than a single Rust CPU runaway.
• The checkout contains 35 hidden .worktrees/ entries from earlier release/issue work.
• .worktrees/ is ignored by .gitignore, but it is still huge: total size was about 155G locally.
• There are 30 nested Rust target/ dirs under .worktrees/; the largest sampled target dirs were roughly 5.7G through 12G each.
• A no-ignore file listing over .worktrees/ saw roughly 533,433 paths when generated dirs were included, and still about 18,052 paths after excluding **/target/**, **/node_modules/**, and **/.next/**.
• A comparable source-ish listing outside .worktrees/ was about 103,146 paths after generated-dir exclusions.

Code-level suspicion

• .gitignore already contains .worktrees/, so walkers that honor gitignore are mostly fine.
• crates/tui/src/working_set.rs::local_reference_paths intentionally disables gitignore for local-reference completion, but should_skip_local_reference_dir does not currently skip .worktrees or .claude/worktrees.
• SubAgentManager::running_count() excludes agents whose task_handle.is_finished(), even while the agent status still says Running. That is useful for not blocking forever, but it can also let the parent keep opening replacement agents before UI/status reconciliation catches up, making the session feel like an agent storm.
• Existing issues cover adjacent symptoms: Deadlocks when spawning multiple concurrent sub-agents #2157 for multi-sub-agent deadlocks, fix(tui): auto-collapse finished sub-agents in sidebar #2195 for finished sub-agent sidebar clutter, type @/ and the TUI freezes and cannot be exited #1921/Noticeable UI/input stall after pressing Enter to submit or confirm actions #1927 for input/mention stalls. This issue tracks the v0.8.46 release-workload combination with fresh local evidence.

Expected behavior

A large release checkout with old worktrees and build artifacts should not make the TUI appear wedged. If a parent model tries to spawn too many sub-agents, CodeWhale should apply clear backpressure/queuing and keep input/rendering responsive. Workspace scans should consistently avoid hidden worktree/build-output directories unless the user explicitly targets them.

Acceptance criteria

• Add a shared workspace-discovery skip list that includes at least .worktrees, .claude/worktrees, bulky build dirs (target, node_modules, .next, etc.), and existing generated state dirs such as .deepseek/snapshots.
• Use that skip list in local-reference completion / working-set discovery, and audit file picker, project map, file search, and snapshot-size estimation for equivalent behavior.
• Add regression coverage with a fixture containing .worktrees/example/src/main.rs and .worktrees/example/target/...; local-reference completion/fuzzy indexing must not descend into either path by default.
• Before accepting a new agent_open, reconcile finished handles or count recently-finished-but-not-yet-reconciled Running agents against the cap so bursts cannot refill the cap faster than the UI state updates.
• When the cap is saturated, return a short actionable tool error or queue status instead of allowing the parent to keep issuing open attempts that make the UI look stuck.
• The Agents panel should distinguish active, completed, queued/blocked, and stale/prior-session entries clearly enough that 5 running / 5 cannot coexist with a misleading list of finished/stale work.

Temporary mitigation

Until this lands, keep release-tranche worktrees outside the active checkout when possible, prune old .worktrees/*/target dirs, and run with a conservative sub-agent cap such as:

[subagents]
max_concurrent = 5
api_timeout_secs = 60

[Hmbown]

Posted PR #2273 for the workspace-discovery slice of this: hidden release worktrees, .claude/worktrees, .deepseek/snapshots, and common build/cache directories are skipped during the gitignore-disabled discovery walks, while normal AI command folders and explicit user hidden folders remain discoverable.

This is intentionally Refs #2211, not Closes #2211: it should reduce TUI saturation from hidden worktree/indexing bulk during release work, but the broader sub-agent fanout/backpressure/sidebar cap acceptance criteria remain open.

[Hmbown]

Partial status: the hidden-worktree/discovery slice has landed in #2329 (03c4b6bd) and #2273 (efb63df6, commit 4ec2df9a), including shared discovery filters for .worktrees, .claude/worktrees, .deepseek/snapshots, and generated dirs. #2212 (cc04dccd) also helped the sidebar by auto-collapsing completed agents. I am leaving this open because the broader acceptance criteria are not done yet: sub-agent fanout/backpressure, saturated-cap behavior, and clearer active/completed/queued/stale panel states still need a merged fix.