disabling reasoning does not work anymore on certain models

[gelim]

Name and Version

llama-cli  --version
version: 8233 (c5a778891)
built with GNU 10.5.0 for Linux x86_64

Operating systems

Linux

GGML backends

CUDA

Hardware

Nvidia Tesla V100 32GB

Models

MiniMax-M2.5-IQ4_XS

Problem description & steps to reproduce

Hello,

on a fresh build, I noticed that in llama-server --reasoning-budget 0 or --chat-template-kwargs {"enable_thinking": false} do not have any effect anymore with MiniMax M2.5 (unsloth GGUF)
Model will spit thinking tokens as output.

AesSedai/Qwen3.5-35B-A3B-GGUF does not highlight this behavior for instance.

First Bad Commit

No response

Relevant log output

Logs

Now I see that thinking is enabled from llama-server output:

llama-server  -m MiniMax-M2.5-IQ4_XS-00001-of-00004.gguf -ngl 99 -b 2048 -ub 2048 --jinja --reasoning-budget 0
[...]
init: chat template, example_format: ']~b]system
You are a helpful assistant[e~[
]~b]user
Hello[e~[
]~b]ai
Hi there[e~[
]~b]user
How are you?[e~[
]~b]ai
<think>
'
srv          init: init: chat template, thinking = 0
main: model loaded
main: server is listening on http://127.0.0.1:8081
main: starting the main loop...
srv  update_slots: all slots are idle
srv  log_server_r: done request: GET / 127.0.0.1 200
srv  params_from_: Chat format: peg-native
slot get_availabl: id  3 | task -1 | selected slot by LRU, t_last = -1
slot launch_slot_: id  3 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
slot launch_slot_: id  3 | task 0 | processing task, is_child = 0
slot update_slots: id  3 | task 0 | new prompt, n_ctx_slot = 180736, n_keep = 0, task.n_tokens = 39
slot update_slots: id  3 | task 0 | n_tokens = 0, memory_seq_rm [0, end)
srv  log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200
slot init_sampler: id  3 | task 0 | init sampler, took 0.03 ms, tokens: text = 39, total = 39
slot update_slots: id  3 | task 0 | prompt processing done, n_tokens = 39, batch.n_tokens = 39
slot print_timing: id  3 | task 0 |
prompt eval time =     882.74 ms /    39 tokens (   22.63 ms per token,    44.18 tokens per second)
       eval time =     664.85 ms /    38 tokens (   17.50 ms per token,    57.16 tokens per second)
      total time =    1547.58 ms /    77 tokens
slot      release: id  3 | task 0 | stop processing: n_tokens = 76, truncated = 0
srv  update_slots: all slots are idle

srv          init: init: chat template, thinking = 0

[pwilkin]

This is expected and not a bug.

Some models do not support toggling reasoning and that is reflected in their chat templates. The engine passes the enable_thinking arg to the template, but does not artificially attempt to restrict it by inputting the end-of-thinking marker since this can lead to model output degradation.

If people want to support manual disabling of thinking, they can modify the template to add the </think> to the input similarly to how models that support it do, but I want it to be an explicit choice and not default behavior.

However, we will probably want to refactor --reasoning-budget since it was always a hacky override and should behave differently than passing enable_thinking to kwargs.

[Galunid]

@pwilkin Do I understand correctly, that models which have the relevant enable_thinking section in their chat template should work, and only the ones that don't won't? Because it seems that the newest Qwen3.5 models do have this section, and still don't work when passing --chat-template-kwargs {"enable_thinking": false}

{%- if add_generation_prompt %}
    {{- '<|im_start|>assistant\n' }}
    {%- if enable_thinking is defined and enable_thinking is true %}
        {{- '<think>\n' }}
    {%- else %}
        {{- '<think>\n\n</think>\n\n' }}
    {%- endif %}
{%- endif %}

[pwilkin]

@Galunid Use --reasoning-budget 0 instead, should pass enable_thinking=false to model correctly (at least I tested it and it was fine). I'll tweak the parameters in a subsequent PR.

[jthomazini]

can you try with -sys "no-think"

[tukangcode]

--reasoning-budget 0

Yes it work, applied this, stop the thinking, though there weird bug, docs need update to proabably regarding this cause all info on internet always suggested use --chat-template-kwargs {"enable_thinking": false}

[gelim]

@pwilkin is it really expected to have a closing thinking token output in the answer (as shown in my initial screenshot)?

If building against b8226 (pre-Autoparser):

[gelim]

Qwen3.5 is out of topic here. Everything works as expected (either with --chat-template-kwargsor --reasoning-udget 0.
For MiniMax m2.5 and template that is available in unsloth GGUF, what I did to come to a b8226 behavior with b8233 is:

append the stolen bit from Qwen3.5 to the end of unsloth's template:

{#- Generation prompt -#}
{%- if add_generation_prompt -%}
  {{- ']~b]ai' ~ '\n' }}
  {%- if enable_thinking is defined and enable_thinking is false -%}
    {{- '<think>\n\n</think>\n\n' }}
  {%- else -%}
    {{- '<think>\n' }}
  {%- endif -%}
{%- endif -%}

and launch with:

llama-server --chat-template-file minimax_fixed.jinja --chat-template-kwargs {"enable_thinking": false}

with that we come back to the sane non-reasoning output:

So in short, implemented what @pwilkin suggested, but I still don't think people should have to do that sort if things manually and additionally do not agree withbut does not artificially attempt to restrict it by inputting the end-of-thinking marker since this can lead to model output degradation.

If the model loaded was not meant to support non reasoning mode, then output should be just reasoning (that's not what I highlighted in my first screenshot).

Beside of this I'm very grateful/impressed on the huge work engaged with Autoparser (and additionally the latest improvement that landed with the new op with gated delta net for mentioning some recent big feat)

[pwilkin]

I'll implement reasoning_budget properly, need a bit of time though.

[aldehir]

I'll implement reasoning_budget properly, need a bit of time though.

Read the discussion on #17750 before you go off on the deep end. A reasoning budget sampler is, in my opinion, the best approach. You can approximate it with grammar, but it's not great. A DFA-based sampler would be considerably easier to maintain and easier to optimize (e.g. cache accepted/reject tokens à la. XGrammar's "adaptive cache")

Personally, I don't think llama.cpp should add capabilities outside the scope of the model. If the designers did not add a flag to disable reasoning, you have the option to create a custom chat template.

[winstonma]

I'll implement reasoning_budget properly, need a bit of time though.

I am not sure if this is related #20099 (comment)

[pwilkin]

Please try #20297

[JotaRata]

Hello I'm new to llama.cpp and I have trouble disabling reasoning for the model unsloth_Qwen3.5-4B-GGUF_Qwen3.5-4B-Q4_K_M.gguf. I tried the flags --reasoning-budget 0 and --chat-template-kwargs '{ "enable_thinking": false }' and they don't work in llama-cli. But they do work in llama-server

[winstonma]

@pwilkin I am using llama.cpp version 8305 and setting reasoning-budget wouldn't stop the model from thinking. How could I stop the model from thinking using your PR? Thanks

❯ llama-cli --version
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon 890M Graphics (RADV STRIX1) (radv) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
version: 8305 (947973c)
built with GNU 15.2.1 for Linux x86_64

❯ llama-cli -m ~/unsloth/Qwen3.5-35B-A3B-GGUF/Qwen3.5-35B-A3B-UD-Q4_K_XL.gguf --enable-reasoning
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon 890M Graphics (RADV STRIX1) (radv) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
error: invalid argument: --enable-reasoning

❯ llama-cli -m ~/unsloth/Qwen3.5-35B-A3B-GGUF/Qwen3.5-35B-A3B-UD-Q4_K_XL.gguf --reasoning-budget 0
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon 890M Graphics (RADV STRIX1) (radv) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat

--SKIP THE OPENING--

> hi

[Start thinking]
Thinking Process:

1.  **Analyze the Input:**
    *   Input: "hi"
    *   Intent: Greeting.
    *   Tone: Casual, friendly.
    *   Expected Response: A friendly greeting back, offering assistance.

2.  **Determine the appropriate response:**
    *   Acknowledge the greeting.
    *   Offer help.
    *   Keep it concise but warm.

3.  **Draft potential responses:**
    *   "Hello! How can I help you today?"
    *   "Hi there! What's up?"
    *   "Hey! Ready to chat or need assistance?"
    *   "Hello! How's your day going?"

...