Skip to content

add option "--expand-tools-even-if-tool-choice-none"#2

Draft
okdshin wants to merge 109 commits intomainfrom
specifiable_tools_expansion
Draft

add option "--expand-tools-even-if-tool-choice-none"#2
okdshin wants to merge 109 commits intomainfrom
specifiable_tools_expansion

Conversation

@okdshin
Copy link
Copy Markdown
Member

@okdshin okdshin commented Apr 25, 2025 •
edited by github-actions bot
Loading

Add option to include tool definitions even when tool_choice is 'none'

Summary

This PR adds a new command-line option --expand-tools-even-if-tool-choice-none which allows including tool definitions in prompts even when tool_choice='none'.

Motivation

In the current implementation, when tool_choice is set to 'none', all tool definitions are removed from the request, preventing the model from seeing the tool schemas. This change enables a workflow where:

  1. The model can be aware of available tools (via their definitions in the prompt)
  2. But is not expected to use them automatically (since tool_choice='none')

This is useful for:

  • Models that need to "plan" about available tools before using them in subsequent requests
  • Cases where tool descriptions provide useful context even if the tools aren't used
  • Situations where you want the model to reference tool capabilities in its response without actually calling them

Implementation

  • Added a new CLI flag --expand-tools-even-if-tool-choice-none (default: False)
  • Modified the request validation in protocol.py to no longer remove tools when tool_choice='none'
  • Added the new parameter to OpenAIServingChat and passed it through from the API server

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 25, 2025

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

@okdshin okdshin force-pushed the specifiable_tools_expansion branch 5 times, most recently from ef2bf5f to 25aec8d Compare June 14, 2025 13:29
yeqcharlotte and others added 23 commits June 16, 2025 06:30
@yeqcharlotte
[Misc] Fix skipped max-model-len validation when deriving max model l…
b692e9c 
…ength from tokenizer config (vllm-project#19660)

Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
@yaochengji @mgoin
[TPU] support attention head dim smaller than 128 (vllm-project#19620)
a77aea5 
Signed-off-by: Chengji Yao <chengjiyao@google.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
@andyxning
[MISC] typo fix (vllm-project#19672)
26bc46e 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
@noooop
[CI] Add mteb testing for rerank models (vllm-project#19344)
f40f763
@russellb
[Docs] Move multiproc doc to v1 dir (vllm-project#19651)
8d12070 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
@SzymonOzog
[Kernel] GGUF MMVQ kernel for multiple input vectors (vllm-project#18754
dec66d2 
)

Signed-off-by: SzymonOzog <szymon.ozog@gmail.com>
@njhill
[BugFix] Don't catch BaseException when dumping execute_model errors (v…
ee35e96 
…llm-project#19626)

Signed-off-by: Nick Hill <nhill@redhat.com>
@Navanit-git
[DOC] Add reasoning capability to vLLM streamlit code (vllm-project#1…
3e75069 
…9557)
@shawntan
[Feature]:Allow for Granite MoE Hybrid models with _only_ shared expe…
4d54240 
…rts. (vllm-project#19652)

Signed-off-by: Shawn Tan <shawntan@ibm.com>
@Isotr0py
[Bugfix] Fix TP inference for Flex attention backend (vllm-project#19657
1173804 
)

Signed-off-by: Isotr0py <2037008807@qq.com>
@andyxning
[MISC] bump huggingface_hub pkg to 0.33.0 (vllm-project#19547)
c3fec47 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
@chaunceyjiang
[Bugfix] fix missing 'finish_reason': null in streaming chat (vllm-pr…
836d4ce 
…oject#19662)

Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
@bnellnm
[Kernels] Use empty for modular MoE workspaces (vllm-project#19667)
5e5baa9 
Signed-off-by: Bill Nell <bnell@redhat.com>
@qscqesze
[Model] Add support for MiniMaxM1ForCausalLM (shares architecture wit…
387bdf0 
…h MiniMaxText01ForCausalLM) (vllm-project#19677)

Signed-off-by: QscQ <qscqesze@gmail.com>
@russellb
[V1] Change return type on get_multimodal_embeddings() (vllm-project#…
90f9c2e 
…19446)

Signed-off-by: Russell Bryant <rbryant@redhat.com>
@dsikka
[Quantization] Remove FP4 emulation; Fall-back to marlin for device <…
6bc7b57 
… 100 (vllm-project#19563)
@conroy-cheers
[Fix] Fall back to Gloo when NCCL backend is unavailable (vllm-projec…
0860087 
…t#19641)

Signed-off-by: conroy-cheers <conroy@corncheese.org>
@davidxia
[doc] add project flag to gcloud TPU command (vllm-project#19664)
119f683 
Signed-off-by: David Xia <david@davidxia.com>
@LucasWilkinson
[Wheel Size] Only build FA2 8.0+PTX (vllm-project#19336)
0733495
@nguyenhoangthuan99
[Frontend] add chunking audio for > 30s audio (vllm-project#19597)
ede5c4e 
Signed-off-by: nguyenhoangthuan99 <thuanhppro12@gmail.com>
@diliu0349
[DOC] fix doc typos (vllm-project#19600)
5b3ad5e 
Signed-off-by: Di Liu <liu-di@sjtu.edu.cn>
@drisspg
Fixes IMA for TP w/ flex-attention (vllm-project#19712)
ddfed31 
Signed-off-by: drisspg <drisspguessous@gmail.com>
@quanliu1991
[Core] add remove_seq_from_computed_blocks_tracker to BlockSpaceManag…
5c76b9c 
…er (vllm-project#19686)

Signed-off-by: 刘全 <quan.liu2@dbappsecurity.com.cn>
Co-authored-by: 刘全 <quan.liu2@dbappsecurity.com.cn>
reidliu41 and others added 13 commits June 20, 2025 07:25
@reidliu41
[Misc] update cuda version (vllm-project#19526)
089a306 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
@reidliu41
[Misc] refactor example - openai_transcription_client (vllm-project#1…
e384f2f 
…9851)

Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
@andyxning
[Kernel] correct cpu worker function parameter type (vllm-project#19745)
71d1219 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
@tdoublep
[Fix] import regex instead of re (vllm-project#19875)
7771d1d 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
@nie3e @gemini-code-assist
[Model] GPT2ForSequenceClassification model (vllm-project#19663)
f1e840e 
Signed-off-by: nie3e <adrcwiek@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
@xuechendi
[custom_op][vllm-plugin] update custom_op class to use op_registry (v…
7e8977f 
…llm-project#19164)

Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
@vladmihailescu
Export NaNs in logits to scheduler_stats if output is corrupted (vllm…
2e3e3c8 
…-project#18777)

Signed-off-by: Vlad Mihailescu <vtmihailescu@gmail.com>
@bigPYJ1151
[CPU][CI] Fallback sliding window to v0 and fix CPU pooling model tes…
79f2f1c 
…ts (vllm-project#19901)

Signed-off-by: jiang1.li <jiang1.li@intel.com>
@andyxning
[Kernel] mark TorchSDPABackend swap_blocks NotImplementedError (vllm-…
71baf85 
…project#19749)
@wangxiyuan
[Misc] Clean up useless code (vllm-project#19889)
e773a9e 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
@rabinadk1
Fix: Check the type of params to be a Sequence not list. (vllm-projec…
8ca81bb 
…t#19910)

Signed-off-by: Rabin Adhikari <rabin.adk1@gmail.com>
@Isotr0py
[Bugfix] Fix bnb 8bit model weights loading (vllm-project#19917)
6f170f1 
Signed-off-by: Isotr0py <2037008807@qq.com>
@princepride
[New model support]Support Tarsier2 (vllm-project#19887)
c3bf9ba 
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
@okdshin okdshin force-pushed the specifiable_tools_expansion branch from 35649de to f2bc427 Compare June 21, 2025 14:36
okdshin added 8 commits June 21, 2025 23:59
@okdshin
add option "--expand-tools-even-if-tool-choice-none"
ba347cd 
Signed-off-by: okada shintarou <okada@preferred.jp>
@okdshin
fix format
5b1b5e7 
Signed-off-by: okada shintarou <okada@preferred.jp>
@okdshin
fix format
38eea42 
Signed-off-by: okada shintarou <okada@preferred.jp>
@okdshin
remove the new option and change the default behaviour
bc031d9 
Signed-off-by: okada shintarou <okada@preferred.jp>
@okdshin
Revert "remove the new option and change the default behaviour"
b5e4164 
This reverts commit 377f4ac3d47d67ca05ed2df6054aab6faec02b4d.

Signed-off-by: okada shintarou <okada@preferred.jp>
@okdshin
update help
e3733a9 
Signed-off-by: okada shintarou <okada@preferred.jp>
@okdshin
add warn
1ffd7f0 
Signed-off-by: okada shintarou <okada@preferred.jp>
@okdshin
fix docs
e282ebb 
Signed-off-by: okada shintarou <okada@preferred.jp>
@okdshin okdshin force-pushed the specifiable_tools_expansion branch from 9559897 to e282ebb Compare June 21, 2025 15:00
okdshin and others added 3 commits June 22, 2025 00:14
@okdshin
fix format
b51c7b9 
Signed-off-by: okada shintarou <okada@preferred.jp>
@okdshin
fix format
0a0920a 
Signed-off-by: okada shintarou <okada@preferred.jp>
@okdshin @aarnphm
Apply suggestions from code review
90eabb2 
Co-authored-by: Aaron Pham <Aaronpham0103@gmail.com>
Signed-off-by: okada shintarou <okada@preferred.jp>
@okdshin okdshin force-pushed the specifiable_tools_expansion branch from 8750191 to 90eabb2 Compare June 22, 2025 07:56
@okdshin
remove assert
8a45fa8 
Signed-off-by: okada shintarou <okada@preferred.jp>
Alnusjaponica pushed a commit that referenced this pull request Oct 23, 2025
@njhill
[Bugfix] Catch and log invalid token ids in detokenizer #2 (vllm-proj…
bb6d8c2 
…ect#26445)

Signed-off-by: Nick Hill <nhill@redhat.com>
Alnusjaponica pushed a commit that referenced this pull request Jan 29, 2026
@nemoramo
Adds FunAudioChat multimodal audio model support (#2) (vllm-project#3…
36d450e 
…3058)

Signed-off-by: ramos <49182011+nemoramo@users.noreply.github.com>
Signed-off-by: mayufeng <mayufeng@example.com>
Co-authored-by: mayufeng <mayufeng@example.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.