[Bugfix] Default Qwen3 reasoning parser to prompt-has-open-think#52
Open
rivetphilbot wants to merge 1 commit into
Open
[Bugfix] Default Qwen3 reasoning parser to prompt-has-open-think#52rivetphilbot wants to merge 1 commit into
rivetphilbot wants to merge 1 commit into
Conversation
The standard Qwen3 chat template injects <think>\n into the assistant
turn opener whenever enable_thinking is not explicitly False. That means
completion tokens only contain </think> followed by the answer, never an
opening <think>.
The current parser default (prompt_has_open_think=False) means clients
must explicitly pass chat_template_kwargs={"enable_thinking": True} in
every request, otherwise the parser sees no opening <think>, falls
through to the fully-tagged branch, and dumps the entire completion
(reasoning + close-tag + answer) into the content field with reasoning
left null.
Flip the default to True so the parser matches the template's actual
behavior out of the box. An explicit enable_thinking=False still
correctly disables open-think handling: the template injects a closed
<think></think> pair in that case, so neither token appears in the
completion and the no-prompt-open-think branch is the right code path.
Verified live on V100 TP=2 with Qwen3-based model: without the fix,
reasoning=null and 700+ chars of CoT leaked into content; with the fix,
reasoning correctly contains the CoT and content is the clean answer,
all without any client-side chat_template_kwargs.
Co-Authored-By: RivetOS Claude <noreply@rivetos.dev>
This was referenced May 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The standard Qwen3 chat template injects
<think>\ninto the assistant turn opener wheneverenable_thinkingis not explicitlyFalse. That means completion tokens only contain</think>followed by the answer — never an opening<think>.The current parser default (
prompt_has_open_think=False) forces every client to passchat_template_kwargs={"enable_thinking": True}on each request. When they don't (the common case for OpenAI-compatible clients), the parser sees no opening<think>, falls through to the both-tags-required branch, and dumps the entire completion — reasoning prose, the close tag, and the answer — intocontentwithreasoningleftnull.Flip the default to
Trueso the parser matches the chat template's actual behavior out of the box. An explicitenable_thinking=Falsestill works correctly: in that case the template emits a closed<think></think>pair into the prompt, so neither tag appears in the completion and the existing no-prompt-open-think branch is the right code path.Reproduction
Live on V100 TP=2 with a Qwen3-derived model (
deckard-40b-w4a16-hermes), serving config:Before (no
chat_template_kwargs):The CoT framing leaks into
contentbecause the parser couldn't find an opening<think>.With explicit
chat_template_kwargs={"enable_thinking": true}(workaround):After this patch (no
chat_template_kwargs):Test plan
enable_thinking=Falsepath: still hits the no-prompt-open-think branch (template emits closed<think></think>, neither tag in completion)Co-Authored-By: RivetOS Claude noreply@rivetos.dev