All pull requests

Pull requests

• fix(server): keep chat prefill alive with protocol chunks

  #865·Avicennasis opened Nov 5, 2026·

  ·

• fix(server): keep non-finite tool argument text as strings

  #864·Avicennasis opened Nov 5, 2026·

  ·

• fix(server): extend the http keep-alive window to 360 seconds

  #863·Avicennasis opened Nov 5, 2026·

  ·

• fix(server): refuse non-finite temperature and top_p

  #862·Avicennasis opened Nov 5, 2026·

  ·

• perf(ggml): step the flat moe grid tile width up when the mean fills the tile

  ggufperf
  #861·void-mckenzie opened Nov 4, 2026Contributor·

  ·
  1

• feat(spec): add DFlash 2 speculative decoding

  spec decode
  #857·jason-fxz opened Nov 3, 2026Collaborator·

  ·

• feat(kvcache): add opt-in fp8 kv cache for r9700

  #853·zihaomu opened Nov 3, 2026Contributor·

  ·

• fix(server): reject empty completions prompt and keep error code

  #850·techsaurabh-394 opened Nov 2, 2026·

  ·
  1

• fix(scheduler): clip max_tokens on a copy of the shared SamplingParams

  #849·techsaurabh-394 opened Nov 2, 2026·

  ·
  1

• Feat/completions n

  #843·A5bhinav opened Oct 28, 2026Contributor·

  ·

• [PoC, DO NOT MERGE] feat(spec): add speculative decoding with MTP drafts

  spec decode
  #832·jason-fxz opened Oct 27, 2026Collaborator·

  ·

• feat(windows): expand Qwen and Gemma checkpoint support for OOB Huggingface uncensored models

  #831·trcwebdesign opened Oct 27, 2026·

  ·

• feat(models): add nvidia/Qwen3.8-Flash-Next-FP4-Local checkpoint support

  #820·chrisqianz opened Oct 24, 2026Contributor·

  ·
  1

• fix(gguf): rename the ggml shim's type_info lookup, ambiguous under MSVC

  gguf
  #809·YevheniiKotyrlo opened Oct 23, 2026Contributor·

  ·

• perf(ple): add an opt-in bounded disk row cache

  #792·wenlive opened Oct 22, 2026Contributor·

  ·
  1

• fix(scheduler): preserve shared sampling parameters when clipping

  #788·wenlive opened Oct 22, 2026Contributor·

  ·
  1

• perf(attention): fuse short bf16 decode output gates on gfx1201

  amd
  #781·zihaomu opened Oct 21, 2026Contributor·

  ·
  1

• perf(kernel): add opt-in hip fp8 decode kernels for r9700

  amd
  #779·zihaomu opened Oct 21, 2026Contributor·

  ·

• test(models): cover the Ornith-1.5-122B NVFP4 export

  #777·chrisqianz opened Oct 21, 2026Contributor·

  ·
  1

• feat(mimo): add text-only MiMo v2.6 Flash support

  #761·jcmh0782 opened Oct 20, 2026·

  ·

• fix(server): opt-in backend recovery after runtime CUDA OOM

  #688·oscarlius opened Oct 14, 2026·

  ·

• fix(rocm): route moe_align around Triton 3.8 miscompile

  amd
  #687·thedandano opened Oct 14, 2026·

  ·

• perf(qwen4): add opt-in layer-major long prefill

  #686·oscarlius opened Oct 14, 2026·

  ·

• fix(moe): admit a wide offloaded decode step's experts 128 at a time

  #680·YevheniiKotyrlo opened Oct 13, 2026Contributor·

  ·

• feat(moe): load a short prefill chunk's experts on demand through the slot cache

  #676·YevheniiKotyrlo opened Oct 13, 2026Contributor·

  ·
  1

  2