extension/llm/server: worker-based OpenAI-compatible HTTP server

[mergennachin]

Wire the foundations into a FastAPI app: /v1/chat/completions (streaming and
non-streaming), /v1/models, /health. Request validation rejects parameters the
server can't honor (top_p != 1, seed, n > 1, frequency/presence penalties,
top_k, logit_bias, logprobs, response_format other than text, non-positive
max_tokens, tool_choice = required / specific function) instead of silently
ignoring them; stop sequences are applied before tool parsing; usage is reported.

The Python process is control plane only: it loads no model and imports no
runtime pybind. Model execution runs in a separate C++ worker process
(cpp/text_llm_worker.cpp, over TextLLMEngine/TextLLMSession) that the control
plane spawns and drives over a small JSONL protocol (worker_client.py). The
protocol and the decode loop (reset, encode, context clamp, prefill, decode,
UTF-8 assembly, stop handling, stats, finish_reason) live in a shared header,
cpp/worker_loop.h, so model-specific workers reuse them; text_llm_worker only
constructs the engine/session and runs the loop. runner_pool is a pool of worker
processes (one in-flight request per worker) with a blocking->async streaming
bridge. V1 is single-slot; concurrent requests queue. There is no prefix cache
and no Python-side KV state; cancellation is best-effort (the control plane stops
consuming, the worker finishes the in-flight request). Hermetic tests (a
FakeRunner worker handle) cover the contract, templating, sampling params, tool
calls, the pool, and the worker protocol; conformance/ is a black-box suite
runnable against any live OpenAI server. READMEs document the flags and scope.

Depends on the serving foundations.

Part of #20001

[mergennachin]

Stack from ghstack (oldest at bottom):

• Qwen3.5-MoE CUDA V2 foundation: one model, many isolated sessions #20117
• examples/models/qwen3_5_moe: CUDA Engine/Session adapter + OpenAI serving #20043
• extension/llm/server: document pi integration #19999
• -> extension/llm/server: worker-based OpenAI-compatible HTTP server #19994
• extension/llm/server: serving foundations (schemas, errors, templating, tools) #19993
• extension/llm/runner: Engine/Session C++ core + token-step primitives #19991

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/19994

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 1 Unrelated Failure, 1 Unclassified Failure

As of commit 2dae19c with merge base eeb0646 ():

NEW FAILURES - The following jobs have failed:

• pull / unittest / linux / linux-job (gh)
  RuntimeError: Command docker exec -t f580969c86ce0701b0ce5fe75d69e47c620961dcd884f7c54e7f3c8235473b86 /exec failed with exit code 1
• pull / unittest / macos / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
• pull / unittest-editable / macos / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1

UNCLASSIFIED FAILURE - DrCI could not classify the following job because the workflow did not run on the merge base. The failure may be pre-existing on trunk or introduced by this PR:

• periodic (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)

FLAKY - The following job failed but was likely due to flakiness present on trunk:

• pull / unittest-editable / linux / linux-job (gh) (detected as infra flaky with no log or failing log classifier)

This comment was automatically generated by Dr. CI and updates every 15 minutes.