[Bug] v1.3.0rc4 claims Qwen3-VL support but convert_checkpoint.py throws AssertionError

[CristyNel]

System Info

• GPU: NVIDIA GeForce RTX 5090 Blackwell (SM 120)
• CPU: Intel i9-14900K Raptor Lake-S Refresh (32) @ 5.700GHz
• RAM: TeamGroup Delta RGB 64GB DDR5-7600 7600MHz Dual Channel
• OS: Ubuntu 24.04.3 LTS x86_64
• Kernel: 6.14.0-37-generic
• Docker Image: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc4
• Python Version: 3.12.3
• CUDA Version: 13.1.80

Who can help?

@2ez4bz
@yuanjingx87
@karljang
@greg-kwasniewski1
@Wanli-Jiang

Information

• The official example scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)

Reproduction

1. Start the official 1.3.0rc4 container

CUSTOM_SCRATCH="/custom_scratch/trt-llm"
docker run -it --name trtllm-debug \
    -u $(id -u):$(id -g) \
    -v "$CUSTOM_SCRATCH":/mnt/scratch \
    -v /NVME2T/models:/models \
    -v /WorkSpaces/TensorRT-LLM:/workspace \
    --env "TLLM_DEBUG_MODE=1" \
    --env "CCACHE_DIR=/mnt/scratch/ccache" \
    --env "CCACHE_CPP2=1" \
    --env "CCACHE_MAXSIZE=40G" \
    --env "CCACHE_COMPILERCHECK=content" \
    --env "CCACHE_SLOPPINESS=time_macros,include_file_mtime,file_macro,system_headers,pch_defines" \
    --env "CCACHE_COMPRESS=true" \
    --env "CCACHE_DIRECT=true" \
    --env "CCACHE_NOHASHDIR=true" \
	--env "TMPDIR=/mnt/scratch/pip_build" \
	--env "PYTHONPYCACHEPREFIX=/mnt/scratch/pycache" \
	--env "JOBLIB_TEMP_FOLDER=/mnt/scratch/joblib" \
	--env "TRTLLM_NVCC_FLAGS=$TRTLLM_NVCC_FLAGS" \
	--env "MAX_JOBS=1" \
	--env "NVCC_THREADS=8" \
	--env "CMAKE_GENERATOR=Ninja" \
	--env "NINJAFLAGS=-j 1 -l 8 -k 1" \
	--env "NINJA_STATUS=[%f/%t%p|%w] " \
    --ipc=host --ulimit memlock=-1 --ulimit stack=67108864 \
    --gpus all --shm-size=16g \
    nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc4 \
	bash -c " 
		audit() { python3 /workspace/1.3.0rc4_audit.py \"\$@\"; }; \
        export -f audit; \
        [ -f /workspace/after_run.sh ] && . /workspace/after_run.sh; \
		exec bash"

#!/bin/bash
# after_run.sh
shopt -s expand_aliases

du -sh "/models/Qwen3-VL-8B-Instruct" "/models/Qwen3-VL-8B-NVFP4-Unified"
ls -lh /models/Qwen3-VL-8B-NVFP4-Unified
file /models/Qwen3-VL-8B-NVFP4-Unified/*.safetensors && \
python3 -c "from safetensors import safe_open; import sys,os; p='/models/Qwen3-VL-8B-NVFP4-Unified'; print(sorted(os.listdir(p))[:10])"
export TLLM_DEBUG_MODE=1

# Try to convert from HF checkpoint to TRT-LLM format

cd /app/tensorrt_llm/examples/models/core/qwen/
python3 convert_checkpoint.py \
  --model_dir /models/Qwen3-VL-8B-NVFP4-Unified \
  --output_dir /models/Qwen3-VL-Converted \
  --dtype float16

Expected behavior

The script should recognize qwen3_vl as a valid model type (consistent with release notes claiming Qwen3-VL support) and successfully convert the Hugging Face checkpoint into TensorRT-LLM format, populating /models/Qwen3-VL-Converted with the converted weights and config.json without raising an AssertionError?

actual behavior

AssertionError: Unsupported Qwen type: qwen3_vl, only ('qwen', 'qwen2', 'qwen2_moe', 'qwen2_llava_onevision', 'qwen2_vl', 'qwen2_audio', 'qwen3', 'qwen3_moe') are acceptable.

additional notes

The release notes for v1.3.0rc4 explicitly state:

"Add EPD disagg support for Qwen3 VL MoE (#10962)"

However, when trying to convert a standard Qwen3-VL-8B checkpoint, the Python front end explicitly rejects the model type qwen3_vl. It seems the valid_types list in config.py was not updated to match the C++ back end capabilities.

[2ez4bz]

Hi @CristyNel, thank you for raising this. This is expected, as the convert_checkpoint.py path is for the TRT backend, which is soon to be deprecated / not seeing new development (the readme in the same folder as the conversion script you used shows no support for the VL flavors of qwen3).

You can try using the PyTorch (default) backend directly by using e.g.:

trtllm-serve nvidia/Qwen3-8B-NVFP4

Of course, this assumes your hardware supports NVFP4 in the first place. If not, please try using the BF16 weights.

[CristyNel]

temporary fix doesn't work:
nano /usr/local/lib/python3.12/dist-packages/tensorrt_llm/models/qwen/config.py
edited:
valid_types = ('qwen', 'qwen2', 'qwen2_moe', 'qwen2_llava_onevision',
'qwen2_vl', 'qwen2_audio', 'qwen3', 'qwen3_moe', 'qwen3_vl')

nano /usr/local/lib/python3.12/dist-packages/tensorrt_llm/models/qwen/convert.py
edited:
valid_types = ('qwen', 'qwen2', 'qwen2_vl', 'qwen3', 'qwen3_moe', 'qwen3_vl')

warnings:
transformers version 4.57.1 is incompatible with nvidia-modelopt

Front-end valid_types and converter checks omit 'qwen3_vl'.
Conversion logic assumes language fields live at top-level config; Qwen3-VL uses hf_config.text_config for language fields. Front-end doesn't handle that layout.
proper fix should integrate qwen3vl weight mappers and full multi modal mapping

[2ez4bz]

@CristyNel TRT-LLM will not be supporting the TRT backend (which is where the checkpoint conversion applies). Please see my previous reply and if it resolves your issue.

[CristyNel]

There will be a huge difference once NVFP4 support and FP8 KV cache are available on SM120, showcasing NVIDIA Blackwell’s strength.

[CristyNel]

root@fd3f143c9e:/app/tensorrt_llm/examples/models/core/qwen# trtllm-serve /models/Qwen3-VL-8B-NVFP4-Unified --port 8000 --config /app/tensorrt_llm/examples/configs/curated/qwen3.yaml --trust_remote_code
/usr/local/lib/python3.12/dist-packages/torch/library.py:356: UserWarning: Warning only once for all operators,  other operators may also be overridden.
  Overriding a previously registered kernel for the same operator and the same dispatch key
  operator: flash_attn::_flash_attn_backward(Tensor dout, Tensor q, Tensor k, Tensor v, Tensor out, Tensor softmax_lse, Tensor(a6!)? dq, Tensor(a7!)? dk, Tensor(a8!)? dv, float dropout_p, float softmax_scale, bool causal, SymInt window_size_left, SymInt window_size_right, float softcap, Tensor? alibi_slopes, bool deterministic, Tensor? rng_state=None) -> Tensor
    registered at /usr/local/lib/python3.12/dist-packages/torch/_library/custom_ops.py:922
  dispatch key: ADInplaceOrView
  previous kernel: no debug info
       new kernel: registered at /usr/local/lib/python3.12/dist-packages/torch/_library/custom_ops.py:922 (Triggered internally at /opt/pytorch/pytorch/aten/src/ATen/core/dispatch/OperatorEntry.cpp:208.)
  self.m.impl(
/usr/local/lib/python3.12/dist-packages/modelopt/torch/__init__.py:36: UserWarning: transformers version 4.57.1 is incompatible with nvidia-modelopt and may cause issues. Please install recommended version with `pip install nvidia-modelopt[hf]` if working with HF models.
  _warnings.warn(
[TensorRT-LLM] TensorRT LLM version: 1.3.0rc4
/usr/local/lib/python3.12/dist-packages/tensorrt_llm/serve/openai_protocol.py:108: UserWarning: Field name "schema" in "ResponseFormat" shadows an attribute in parent "OpenAIBaseModel"
  class ResponseFormat(OpenAIBaseModel):
[2026-02-18 18:23:08] INFO utils.py:28: Model already exists locally
[02/18/2026-18:23:08] [TRT-LLM] [I] Overriding max_batch_size from build_config to 161
[02/18/2026-18:23:08] [TRT-LLM] [I] Overriding max_num_tokens from build_config to 1160
[02/18/2026-18:23:08] [TRT-LLM] [I] Using LLM with PyTorch backend
[02/18/2026-18:23:08] [TRT-LLM] [E] Failed to parse the arguments for the LLM constructor: LLM got invalid argument: kv_cache_free_gpu_memory_fraction
Traceback (most recent call last):
  File "/usr/local/bin/trtllm-serve", line 7, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/click/core.py", line 1485, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/click/core.py", line 1406, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/click/core.py", line 1873, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/click/core.py", line 1269, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/click/core.py", line 824, in invoke
    return callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/tensorrt_llm/commands/serve.py", line 793, in serve
    _serve_llm()
  File "/usr/local/lib/python3.12/dist-packages/tensorrt_llm/commands/serve.py", line 767, in _serve_llm
    launch_server(host, port, llm_args, tool_parser, chat_template,
  File "/usr/local/lib/python3.12/dist-packages/tensorrt_llm/commands/serve.py", line 207, in launch_server
    llm = PyTorchLLM(**llm_args)
          ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/tensorrt_llm/llmapi/llm.py", line 1213, in __init__
    super().__init__(model, tokenizer, tokenizer_mode, skip_tokenizer_init,
  File "/usr/local/lib/python3.12/dist-packages/tensorrt_llm/llmapi/llm.py", line 1099, in __init__
    super().__init__(model,
  File "/usr/local/lib/python3.12/dist-packages/tensorrt_llm/llmapi/llm.py", line 193, in __init__
    raise e
  File "/usr/local/lib/python3.12/dist-packages/tensorrt_llm/llmapi/llm.py", line 174, in __init__
    raise ValueError(
ValueError: LLM got invalid argument: kv_cache_free_gpu_memory_fraction
root@fd3f143c9e:/app/tensorrt_llm/examples/models/core/qwen# 

[CristyNel]

this model works fain on vLLM with
--max-num-batched-tokens 30000
--max-model-len 150000 \

Total tensors: 1578
First 10 keys:
  lm_head.weight
  model.language_model.embed_tokens.weight
  model.language_model.layers.0.input_layernorm.weight
  model.language_model.layers.0.mlp.down_proj.input_scale
  model.language_model.layers.0.mlp.down_proj.weight
  model.language_model.layers.0.mlp.down_proj.weight_scale
  model.language_model.layers.0.mlp.down_proj.weight_scale_2
  model.language_model.layers.0.mlp.gate_proj.input_scale
  model.language_model.layers.0.mlp.gate_proj.weight
  model.language_model.layers.0.mlp.gate_proj.weight_scale

Found 720 attention-related tensors:
  model.language_model.layers.0.self_attn.k_norm.weight
  model.language_model.layers.0.self_attn.k_proj.input_scale
  model.language_model.layers.0.self_attn.k_proj.k_scale
  model.language_model.layers.0.self_attn.k_proj.weight
  model.language_model.layers.0.self_attn.k_proj.weight_scale

Scale tensors found: 72
  model.language_model.layers.0.self_attn.k_proj.k_scale
  model.language_model.layers.0.self_attn.v_proj.v_scale
  model.language_model.layers.1.self_attn.k_proj.k_scale
  model.language_model.layers.1.self_attn.v_proj.v_scale
  model.language_model.layers.10.self_attn.k_proj.k_scale
  model.language_model.layers.10.self_attn.v_proj.v_scale
  model.language_model.layers.11.self_attn.k_proj.k_scale
  model.language_model.layers.11.self_attn.v_proj.v_scale
  model.language_model.layers.12.self_attn.k_proj.k_scale
  model.language_model.layers.12.self_attn.v_proj.v_scale

Quantization metadata tensors: 144
  model.language_model.layers.0.self_attn.q_proj.input_scale
  model.language_model.layers.0.self_attn.q_proj.weight
  model.language_model.layers.0.self_attn.q_proj.weight_scale
  model.language_model.layers.0.self_attn.q_proj.weight_scale_2
  model.language_model.layers.1.self_attn.q_proj.input_scale
  model.language_model.layers.1.self_attn.q_proj.weight
  model.language_model.layers.1.self_attn.q_proj.weight_scale
  model.language_model.layers.1.self_attn.q_proj.weight_scale_2
  model.language_model.layers.10.self_attn.q_proj.input_scale
  model.language_model.layers.10.self_attn.q_proj.weight
Layer 0: k=torch.Size([]), v=torch.Size([])
Layer 1: k=torch.Size([]), v=torch.Size([])
Layer 2: k=torch.Size([]), v=torch.Size([])
Layer 3: k=torch.Size([]), v=torch.Size([])
Layer 4: k=torch.Size([]), v=torch.Size([])
Layer 5: k=torch.Size([]), v=torch.Size([])
Layer 6: k=torch.Size([]), v=torch.Size([])
Layer 7: k=torch.Size([]), v=torch.Size([])
Layer 8: k=torch.Size([]), v=torch.Size([])
Layer 9: k=torch.Size([]), v=torch.Size([])
Layer 10: k=torch.Size([]), v=torch.Size([])
Layer 11: k=torch.Size([]), v=torch.Size([])
Layer 12: k=torch.Size([]), v=torch.Size([])
Layer 13: k=torch.Size([]), v=torch.Size([])
Layer 14: k=torch.Size([]), v=torch.Size([])
Layer 15: k=torch.Size([]), v=torch.Size([])
Layer 16: k=torch.Size([]), v=torch.Size([])
Layer 17: k=torch.Size([]), v=torch.Size([])
Layer 18: k=torch.Size([]), v=torch.Size([])
Layer 19: k=torch.Size([]), v=torch.Size([])
Layer 20: k=torch.Size([]), v=torch.Size([])
Layer 21: k=torch.Size([]), v=torch.Size([])
Layer 22: k=torch.Size([]), v=torch.Size([])
Layer 23: k=torch.Size([]), v=torch.Size([])
Layer 24: k=torch.Size([]), v=torch.Size([])
Layer 25: k=torch.Size([]), v=torch.Size([])
Layer 26: k=torch.Size([]), v=torch.Size([])
Layer 27: k=torch.Size([]), v=torch.Size([])
Layer 28: k=torch.Size([]), v=torch.Size([])
Layer 29: k=torch.Size([]), v=torch.Size([])
Layer 30: k=torch.Size([]), v=torch.Size([])
Layer 31: k=torch.Size([]), v=torch.Size([])
✅ Layer 0: k_scale == v_scale (shape: torch.Size([]), dtype: torch.float32)
✅ Layer 1: k_scale == v_scale (shape: torch.Size([]), dtype: torch.float32)
✅ Layer 2: k_scale == v_scale (shape: torch.Size([]), dtype: torch.float32)
✅ Layer 3: k_scale == v_scale (shape: torch.Size([]), dtype: torch.float32)
✅ Layer 4: k_scale == v_scale (shape: torch.Size([]), dtype: torch.float32)
✅ Layer 5: k_scale == v_scale (shape: torch.Size([]), dtype: torch.float32)
✅ Layer 6: k_scale == v_scale (shape: torch.Size([]), dtype: torch.float32)
✅ Layer 7: k_scale == v_scale (shape: torch.Size([]), dtype: torch.float32)
✅ Layer 8: k_scale == v_scale (shape: torch.Size([]), dtype: torch.float32)
✅ Layer 9: k_scale == v_scale (shape: torch.Size([]), dtype: torch.float32)
✅ Layer 10: k_scale == v_scale (shape: torch.Size([]), dtype: torch.float32)
✅ Layer 11: k_scale == v_scale (shape: torch.Size([]), dtype: torch.float32)
✅ Layer 12: k_scale == v_scale (shape: torch.Size([]), dtype: torch.float32)
✅ Layer 13: k_scale == v_scale (shape: torch.Size([]), dtype: torch.float32)
✅ Layer 14: k_scale == v_scale (shape: torch.Size([]), dtype: torch.float32)
✅ Layer 15: k_scale == v_scale (shape: torch.Size([]), dtype: torch.float32)
✅ Layer 16: k_scale == v_scale (shape: torch.Size([]), dtype: torch.float32)
✅ Layer 17: k_scale == v_scale (shape: torch.Size([]), dtype: torch.float32)
✅ Layer 18: k_scale == v_scale (shape: torch.Size([]), dtype: torch.float32)
✅ Layer 19: k_scale == v_scale (shape: torch.Size([]), dtype: torch.float32)
✅ Layer 20: k_scale == v_scale (shape: torch.Size([]), dtype: torch.float32)
✅ Layer 21: k_scale == v_scale (shape: torch.Size([]), dtype: torch.float32)
✅ Layer 22: k_scale == v_scale (shape: torch.Size([]), dtype: torch.float32)
✅ Layer 23: k_scale == v_scale (shape: torch.Size([]), dtype: torch.float32)
✅ Layer 24: k_scale == v_scale (shape: torch.Size([]), dtype: torch.float32)
✅ Layer 25: k_scale == v_scale (shape: torch.Size([]), dtype: torch.float32)
✅ Layer 26: k_scale == v_scale (shape: torch.Size([]), dtype: torch.float32)
✅ Layer 27: k_scale == v_scale (shape: torch.Size([]), dtype: torch.float32)
✅ Layer 28: k_scale == v_scale (shape: torch.Size([]), dtype: torch.float32)
✅ Layer 29: k_scale == v_scale (shape: torch.Size([]), dtype: torch.float32)
✅ Layer 30: k_scale == v_scale (shape: torch.Size([]), dtype: torch.float32)
✅ Layer 31: k_scale == v_scale (shape: torch.Size([]), dtype: torch.float32)

📊 Summary:
  Total layers audited: 32
  Symmetric layers: 32
  FP8 layers: 0
  FP16 layers: 0
  FP32 layers: 32
  Missing layers: 0
📊 Size Comparison:
Instruct: 16.34 GB
NVFP4:    7.05 GB

📊 Quant Config Check:
{
  "producer": {
    "name": "modelopt",
    "version": "0.41.0"
  },
  "quantization": {
    "quant_algo": "NVFP4",
    "kv_cache_quant_algo": "FP8",
    "group_size": 16,
    "exclude_modules": [
      "lm_head",
      "model.visual*"
    ]
  }
}

📊 Exported Weights (NVFP4):
model-00001-of-00002.safetensors 4767.88 MB
model-00002-of-00002.safetensors 2432.27 MB

📊 Checksum Comparison:

Model: Qwen3-VL-8B-Instruct
model-00001-of-00004.safetensors: d5d0aef0eb170fc7453a296c43c0849a56f510555d3588e4fd662bb35490aefa
model-00002-of-00004.safetensors: 8be88fb5501e4d5719a6d4cc212e6a13480330e74f3e8c77daa1a68f199106b5
model-00003-of-00004.safetensors: 83de00eafe6e0d57ccd009dbcf71c9974d74df2f016c27afb7e95aafd16b2192
model-00004-of-00004.safetensors: 0a88b98e9f96270973f567e6a2c103ede6ccdf915ca3075e21c755604d0377a5

Model: Qwen3-VL-8B-NVFP4-Unified
model-00001-of-00002.safetensors: e77fe4a51d76c3c6dbc8c2885bf85ba86fd137ddaddc6384462cd545e2b4f055
model-00002-of-00002.safetensors: 6a8eda2793ecb2a0f657499a6947a4fd627337063caec3b9f0f7e400d23416b1

17G	/models/Qwen3-VL-8B-Instruct
7.1G	/models/Qwen3-VL-8B-NVFP4-Unified

The container fails to support checkpoint conversion and engine build.

[2ez4bz]

@CristyNel I repro-ed this. Looks like some ill-formatted configs somehow live in the repo - took an action item on our side to add them to our CI. Thank you for flagging!

Can you try #11571 and see if it fixes your issue?

[CristyNel]

No way - Qwen3-VL Base isn't supported, much less Qwen3-VL NVFP4, which is the primary strength of the 5090 sm_120 Blackwell architecture.
Is there anywhere to download NVFP4 TensorRT-LLM-compatible checkpoints? It is impossible to quantize them due to all these bugs. If it is not feasible to deploy an 8B model (which would be inefficient for this hardware), it may be more practical to use vLLM with a 40B Hugging Face model instead.

python3 convert_checkpoint.py
--model_dir /models/Qwen3-VL-8B-Instruct
--output_dir /models/Qwen3-VL-Converted
--dtype float16
/usr/local/lib/python3.12/dist-packages/torch/library.py:356: UserWarning: Warning only once for all operators, other operators may also be overridden.
Overriding a previously registered kernel for the same operator and the same dispatch key
operator: flash_attn::_flash_attn_backward(Tensor dout, Tensor q, Tensor k, Tensor v, Tensor out, Tensor softmax_lse, Tensor(a6!)? dq, Tensor(a7!)? dk, Tensor(a8!)? dv, float dropout_p, float softmax_scale, bool causal, SymInt window_size_left, SymInt window_size_right, float softcap, Tensor? alibi_slopes, bool deterministic, Tensor? rng_state=None) -> Tensor
registered at /usr/local/lib/python3.12/dist-packages/torch/_library/custom_ops.py:922
dispatch key: ADInplaceOrView
previous kernel: no debug info
new kernel: registered at /usr/local/lib/python3.12/dist-packages/torch/_library/custom_ops.py:922 (Triggered internally at /opt/pytorch/pytorch/aten/src/ATen/core/dispatch/OperatorEntry.cpp:208.)
self.m.impl(
/usr/local/lib/python3.12/dist-packages/modelopt/torch/init.py:36: UserWarning: transformers version 4.57.1 is incompatible with nvidia-modelopt and may cause issues. Please install recommended version with pip install nvidia-modelopt[hf] if working with HF models.
_warnings.warn(
[TensorRT-LLM] TensorRT LLM version: 1.3.0rc4
1.3.0rc4
Traceback (most recent call last):
File "/app/tensorrt_llm/examples/models/core/qwen/convert_checkpoint.py", line 345, in
main()
File "/app/tensorrt_llm/examples/models/core/qwen/convert_checkpoint.py", line 337, in main
convert_and_save_hf(args)
File "/app/tensorrt_llm/examples/models/core/qwen/convert_checkpoint.py", line 293, in convert_and_save_hf
execute(args.workers, [convert_and_save_rank] * world_size, args)
File "/app/tensorrt_llm/examples/models/core/qwen/convert_checkpoint.py", line 300, in execute
f(args, rank)
File "/app/tensorrt_llm/examples/models/core/qwen/convert_checkpoint.py", line 281, in convert_and_save_rank
qwen = QWenForCausalLM.from_hugging_face(model_dir,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/tensorrt_llm/models/qwen/model.py", line 324, in from_hugging_face
config = QWenConfig.from_hugging_face(hf_config_or_dir,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/tensorrt_llm/models/qwen/config.py", line 115, in from_hugging_face
assert qwen_type in valid_types, f"Unsupported Qwen type: {qwen_type}, only {valid_types} are acceptable."
^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Unsupported Qwen type: qwen3_vl, only ('qwen', 'qwen2', 'qwen2_moe', 'qwen2_llava_onevision', 'qwen2_vl', 'qwen2_audio', 'qwen3', 'qwen3_moe') are acceptable.

[2ez4bz]

Maybe I'm missing context: what are you trying to achieve with that convert_checkpoint.py script? Bear in mind Qwen3-VL is explicitly not included in the support matrix for that script.

[CristyNel]

Maybe I'm missing context: what are you trying to achieve with that convert_checkpoint.py script? Bear in mind Qwen3-VL is explicitly not included in the support matrix for that script.

I am attempting to convert 2026 models into TensorRT checkpoints before generating the engines, but the process has proven impossible with every method I have tried. Specifically, I am testing:

IQuest-Coder

Qwen3-VL

Ministral-3

Qwen3-Coder

I have no interest in 2020-era models; we are in 2026 and using the RTX 5090, not the GTX 1650. Could you please identify which 2026 models are currently verified to work with TRT-LLM and provide native support for NVFP4 with FP8 KV cache?

[2ez4bz]

Thanks for providing more context. As I've mentioned in several messages above, the convert_checkpoint.py script is for generating an intermediate checkpoint representation for the TRT backend, regardless of quantization or not. This backend is now in "do not harm" mode, and hence not seeing active development. The Qwen3 VL model is explicitly not supported in that path.

Basically: please do not use that script anymore, unless you somehow explicitly wanted to use the TRT backend instead of the default PyTorch backend, for legacy reasons.

Could you try the trtllm-serve command I shared earlier, but with this small change?