AgentSession default VAD/turn-detector loads ~150MB per process with no global opt-out — OOMs multi-process (spawn) consumers

[maktoufzied]

Summary

Since 1.6.x, AgentSession auto-provisions local inference models when the caller does not pass them: a bundled silero VAD and an end-of-turn (EOT) turn detector. The EOT model costs ~152MB of resident memory per Python process, and there is no global opt-out — only per-constructor vad=None / turn_handling={"turn_detection": None}, which is not reachable when sessions are constructed inside a third-party harness. For multi-process consumers using multiprocessing.spawn (evaluation frameworks, test runners), the cost multiplies per worker: our 16-worker evaluation CI gained ~2.4GB of RSS from the upgrade alone and was OOM-killed on an 8GB container. The sessions involved are text-only (no audio I/O at all), so the models are never used.

LiveKit support confirmed there is no environment variable or module-level switch and suggested an adapter that injects vad=None/turn_detection=None — which works, but requires patching private module internals when the constructor isn't reachable.

What changed

livekit/agents/voice/agent_session.py (v1.6.4):

raw_turn_detection: TurnDetectionMode | None = turn_handling.get(
    "turn_detection", inference.TurnDetector()          # ~line 366
)
...
if not is_given(vad):
    vad = inference.VAD(model="silero")                 # ~line 415

Under 1.4.5, AgentSession(llm=...) had no VAD and no turn-detector model. Additionally, import livekit.agents now eagerly imports livekit.local_inference (the pybind11 native module) — verified via "livekit.local_inference" in sys.modules right after the import.

Measurements (livekit-agents 1.6.4, Python 3.13, macOS arm64 + Linux)

Single process, resource.getrusage max-RSS deltas:

Step	RSS delta
import livekit.agents	+113MB (≈ same as 1.4.5)
livekit.local_inference.init_vad()	+2MB
livekit.local_inference.init_eot()	+152MB

Per-process duplication under spawn (4 workers, each importing livekit.local_inference and calling init_eot()):

EOT model RSS delta per spawned process (MB): [151, 151, 152, 152]

livekit/agents/inference/_warmup.py acknowledges the cost — it preloads the model singletons in a forkserver parent so livekit's own job subprocesses share the weight pages via COW — but external harnesses using spawn get no such sharing.

Reproduction

# single process
import resource, sys

def rss_mb():
    ru = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
    return ru / (1024 * 1024) if sys.platform == "darwin" else ru / 1024

b0 = rss_mb()
import livekit.agents
b1 = rss_mb(); print(f"import livekit.agents: +{b1-b0:.0f}MB")
print("local_inference eagerly imported:", "livekit.local_inference" in sys.modules)

import livekit.local_inference as li
li.init_vad();  b2 = rss_mb(); print(f"init_vad(): +{b2-b1:.0f}MB")
li.init_eot();  b3 = rss_mb(); print(f"init_eot(): +{b3-b2:.0f}MB")

# spawn duplication
import multiprocessing as mp
import resource, sys

def rss_mb():
    ru = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
    return ru / (1024 * 1024) if sys.platform == "darwin" else ru / 1024

def worker(q):
    import livekit.local_inference as li
    before = rss_mb()
    li.init_eot()
    q.put(round(rss_mb() - before))

if __name__ == "__main__":
    ctx = mp.get_context("spawn")
    q = ctx.Queue()
    procs = [ctx.Process(target=worker, args=(q,)) for _ in range(4)]
    for p in procs: p.start()
    print("per-process EOT RSS delta (MB):", [q.get() for _ in procs])
    for p in procs: p.join()

Feature request

Any of the following would resolve this cleanly:

1. A supported global opt-out for the default components — e.g. AgentSession.set_default_components(vad=None, turn_detection=None) or an environment variable — for sessions constructed inside code the application doesn't control (harnesses, frameworks, tests).
2. Lazy import of livekit.local_inference (on first VAD/EOT use) instead of eagerly on import livekit.agents.
3. Deferred EOT model load until turn detection is actually exercised on an audio stream, so text-only sessions never pay it.
4. Documented guidance for multi-process (spawn) consumers, equivalent to the internal inference/_warmup.py forkserver-preload approach.

Environment

• livekit-agents: 1.6.4
• Python: 3.13
• OS: reproduced on macOS (arm64) and Linux (Docker, cimg/python:3.13)

[longcw]

Thanks for the detailed report. This is expected behavior for the bundled local turn detector and there's already a supported opt-out, so I don't think it's a bug. Quick clarifications (all about the local turn-detector-v1-mini model, the fallback when there are no LiveKit inference credentials):

• The opt-out works today. AgentSession(..., vad=None, turn_handling=TurnHandlingOptions(turn_detection=None)) never loads the model — the weights only load when the detector opens its audio stream, which is skipped when it's disabled (vad=None alone is enough, since the default detector requires a VAD). For a text-only harness, disable both; the audio models are never used there anyway.
• import livekit.agents does not load the model. The livekit.local_inference native module is ~1MB; the ~112MB you measured is numpy / aiohttp / livekit.rtc / pydantic etc., unchanged from earlier versions. The 152MB is only paid when the weights actually load.
• The model is a per-process singleton when using forkserver. Under the LiveKit worker on Linux (forkserver, the default) it's preloaded once in the forkserver parent (inference/_warmup.py) and job subprocesses inherit the weight pages via copy-on-write, so N jobs share one physical copy.

The worker defaults to forkserver on Linux (so it gets the sharing automatically) and spawn on macOS/Windows (which does not, because independent spawn processes can't share memory). On macOS/Windows you can opt into the shared behavior with AgentServer(multiprocessing_context="forkserver").

If sharing under spawn specifically matters to you, one option is routing the local detector through the shared inference subprocess (loaded once, served over IPC regardless of start method). Could you explain why a spawn context is needed if that's the case?

Repro (per-process singleton, plus spawn-vs-forkserver sharing — RSS can't show sharing, so the signal is the RSS delta inside each child around its first EOT()):

import multiprocessing as mp, resource, sys
def rss():
    r = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
    return r / (1024*1024) if sys.platform == "darwin" else r / 1024
def _child(q):
    import livekit.local_inference as li
    b = rss(); li.EOT(); q.put(round(rss() - b))
def run(ctx, preload=None):
    c = mp.get_context(ctx)
    if preload: c.set_forkserver_preload(preload)
    q = c.Queue(); ps = [c.Process(target=_child, args=(q,)) for _ in range(4)]
    for p in ps: p.start()
    out = [q.get() for _ in ps]; [p.join() for p in ps]; return out
if __name__ == "__main__":
    import livekit.local_inference as li
    b = rss(); li.init_eot(); print(f"init_eot(): +{rss()-b:.0f}MB")
    b = rss(); [li.EOT() for _ in range(12)]; print(f"12x EOT():  +{rss()-b:.0f}MB (singleton)")
    print("spawn      EOT delta:", run("spawn"))
    print("forkserver EOT delta:", run("forkserver", ["livekit.agents.inference._warmup"]))

init_eot(): +151MB
12x EOT():  +0MB (singleton)
spawn      EOT delta: [152, 152, 151, 152]   # each process loads its own copy
forkserver EOT delta: [1, 1, 1, 1]           # preloaded once in the parent, shared via COW

On Linux, PSS (each process's share of physical RAM) confirms the physical sharing — RSS looks similar for both because it counts shared pages in full:

                  per-child PSS     sum(PSS) over 4 children
spawn                 ~166 MB           664 MB   (4 private copies)
forkserver             ~53 MB           212 MB   (one copy shared via COW)