Skip to content

fix(pool): healthcheck timer must not block graceful worker shutdown (true-async/server#93)#181

Merged
EdmondDantes merged 1 commit into
mainfrom
fix-pool-healthcheck-timer-blocks-reload
Jul 7, 2026
Merged

fix(pool): healthcheck timer must not block graceful worker shutdown (true-async/server#93)#181
EdmondDantes merged 1 commit into
mainfrom
fix-pool-healthcheck-timer-blocks-reload

Conversation

@EdmondDantes

Copy link
Copy Markdown
Contributor

Problem

Under a debug build, hot-reloading workers can trip a scheduler assert:

Assertion failed: (zend_async_reactor_loop_alive_fn() == false && "The event loop must be stopped"),
function fiber_entry, file scheduler.c, line 2041.

Reproduced deterministically: a worker bootloader that warms an async PDO connection pool with PDO::ATTR_POOL_HEALTHCHECK_INTERVAL > 0, then reload(). With the interval set to 0 (no timer) the reload is clean — isolating the pool's healthcheck timer as the cause.

Root cause

pool_start_healthcheck_timer() created the background heartbeat timer as a normal reactor event, so libuv_timer_start incremented active_event_count. On graceful shutdown start_graceful_shutdown() cancels coroutines, but nothing owns/closes this timer — the pool is only disposed later at PDO object teardown. So ZEND_ASYNC_REACTOR_LOOP_ALIVE() (active_event_count > 0 && uv_loop_alive) stayed true while the scheduler fiber finalized:

  • debug: trips the loop-alive ZEND_ASSERT in fiber_entry.
  • release: an idle pool needlessly keeps the retiring worker's loop alive.

The ZEND_ASYNC_EVENT_F_HIDDEN flag exists for exactly this — its doc names "Background timers (e.g., garbage collection, health checks)" as the canonical case — but the pool timer never set it.

Fix

Mark the healthcheck timer HIDDEN right after creation. The timer still fires (HIDDEN only gates the active_event_count accounting, not uv_timer_start); it just no longer counts as application work for loop-alive / deadlock detection.

Verification

  • Repro (pool + interval=30 + reload ×N): was exit 134 with assert → now 0 asserts, clean shutdown, all reloads succeed.
  • server hot-reload phpt suite (051–054): 4/4 PASS.
  • PDO pool phpt suites (pdo_pool_*, pool_*): 0 failed (17 passed, 8 skipped — need mysql/pgsql).

…es not block a graceful worker shutdown (true-async/server#93)

The pool healthcheck timer is a background heartbeat, but pool_start_healthcheck_timer
started it as a normal reactor event, so it counted toward active_event_count. An idle
pool (e.g. a warmed PDO pool with ATTR_POOL_HEALTHCHECK_INTERVAL > 0) therefore kept
ZEND_ASYNC_REACTOR_LOOP_ALIVE() true after all coroutines were cancelled on graceful
shutdown. On worker hot-reload the scheduler fiber then tripped
ZEND_ASSERT(REACTOR_LOOP_ALIVE() == false) in fiber_entry (debug build); on release the
idle heartbeat needlessly kept the retiring worker's loop alive.

The HIDDEN flag's own documentation lists "background timers (health checks)" as the
canonical case, so mark the timer HIDDEN — it still fires; it just no longer counts as
application work for deadlock/loop-alive accounting.
@codecov

codecov Bot commented Jul 7, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@EdmondDantes EdmondDantes merged commit dba072c into main Jul 7, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant