[feature] Add compatibility for reasoning-budget params

[vk2r]

Good evening!
Sorry, I've run into a problem.

I've been testing the new flags for llama.cpp from reasoning-budget, and unfortunately they are not compatible with llama-swap.

Here is the message that appears every time I add them.

Mi configuration is simple:

${server_cmd}
--model ${models_dir}/tesslate/omnicoder-9b-q8.gguf
--ctx-size 204800
--n-predict 16384
--top-p 0.95
--top-k 20
--min-p 0.0
--temp 0.6
--reasoning-budget 1000
--reasoning-budget-message "... thinking budget exceeded, let's answer now."

[mostlygeek]

You may need to update your version of llama-server. The --reasoning-budget flag has existed for a while but the only valid value was 0. Only after ggml-org/llama.cpp#20297 did it do what you're expecting it to do.

Here is my config, you can dynamically set thinking_budget_tokens as well without reloading the model.

  "Q3.5-35B":
    env:
      - "CUDA_VISIBLE_DEVICES=GPU-6f0,GPU-f10"
    filters:
      stripParams: "temperature, top_k, top_p, repeat_penalty, min_p, presence_penalty"
      setParamsByID:
        "${MODEL_ID}:thinking-coding":
          temperature: 0.6
          presence_penalty: 0.0
        "${MODEL_ID}:instruct":
          chat_template_kwargs:
            enable_thinking: false
          temperature: 0.7
          top_p: 0.8

        # limited reasoning tokens
        "${MODEL_ID}:low":
          thinking_budget_tokens: 100
        "${MODEL_ID}:med":
          thinking_budget_tokens: 500

    cmd: |
      ${server-latest}
      --model /path/to/models/Qwen3.5-35B-A3B-UD-Q6_K_XL.gguf
      --ctx-size 262144
      --fit off
      --temp 1.0 --min-p 0.0 --top-k 20 --top-p 0.95
      --repeat_penalty 1.0 --presence_penalty 1.5

      # https://github.com/ggml-org/llama.cpp/pull/20297
      --reasoning-budget-message "...I got enough. Let's answer now."

[vk2r]

You may need to update your version of llama-server. The --reasoning-budget flag has existed for a while but the only valid value was 0. Only after ggml-org/llama.cpp#20297 did it do what you're expecting it to do.

Here is my config, you can dynamically set thinking_budget_tokens as well without reloading the model.

"Q3.5-35B":
env:
- "CUDA_VISIBLE_DEVICES=GPU-6f0,GPU-f10"
filters:
stripParams: "temperature, top_k, top_p, repeat_penalty, min_p, presence_penalty"
setParamsByID:
"${MODEL_ID}:thinking-coding":
temperature: 0.6
presence_penalty: 0.0
"${MODEL_ID}:instruct":
chat_template_kwargs:
enable_thinking: false
temperature: 0.7
top_p: 0.8

    # limited reasoning tokens
    "${MODEL_ID}:low":
      thinking_budget_tokens: 100
    "${MODEL_ID}:med":
      thinking_budget_tokens: 500

cmd: |
  ${server-latest}
  --model /path/to/models/Qwen3.5-35B-A3B-UD-Q6_K_XL.gguf
  --ctx-size 262144
  --fit off
  --temp 1.0 --min-p 0.0 --top-k 20 --top-p 0.95
  --repeat_penalty 1.0 --presence_penalty 1.5

  # https://github.com/ggml-org/llama.cpp/pull/20297
  --reasoning-budget-message "...I got enough. Let's answer now."

You're absolutely right.
For some reason I don't understand (I updated it yesterday), that feature still hadn't made it into the Windows build of llama.cpp.

I really appreciate the support.
Best regards.