Skip to content

Engine scheduler: passive instance never recovers when PID owner dies #42

Description

@LinekForge

Problem

When a CC session exits without cleanly releasing the PID lock (engine.pid), the next session's engine enters passive mode permanently. The passive instance never re-checks whether the PID owner is still alive, so heartbeats stop firing until someone manually deletes the PID file and restarts CC.

Reproduction

  1. Start a CC session with server:engine — engine acquires PID lock, enters active mode
  2. Session exits uncleanly (crash, kill -9, machine reboot, macOS restart)
  3. PID file persists with the dead process's PID
  4. Start a new CC session — new engine sees existing PID file, checks process.kill(pid, 0), finds process dead, should overwrite and take ownership
  5. However, if the new session reuses the old process (e.g. via resume), or if the PID is recycled to a living process, the new engine stays passive forever

Observed impact

  • Heartbeat messages stop being delivered to the CC session
  • Trigger log continues to record entries (from the orphaned active scheduler in the old process), but notifications don't reach the current conversation
  • Has occurred 3+ times over 2 weeks, requiring manual intervention each time

Current workaround

rm ~/.forge-hub/engine-data/engine.pid
# then restart CC session

Proposed fix

Add a periodic liveness recheck in passive mode:

  • Every 60 seconds, a passive engine instance should re-check whether the PID owner is still alive
  • If the PID owner is dead, delete the stale PID file, acquire the lock, and transition from passive → active (run fullReload + scheduleMidnight)
  • Log the transition clearly: "🔄 PID owner dead, transitioning passive → active"

Relevant code: scheduler.ts, acquirePidLock() function (around line 554) and startScheduler() (around line 613).

Additional context

The acquirePidLock() function already handles stale PID detection on startup. The gap is that this check only runs once — at startScheduler() init. If the check fails (e.g. because the old process is still alive at that moment but dies seconds later), the passive instance never retries.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions