TypeError: Can't compile non template nodes

[theobarrague]

System Info

• transformers version: 5.4.0
• Platform: Linux-6.17.0-19-generic-x86_64-with-glibc2.39
• Python version: 3.12.3
• Huggingface_hub version: 1.8.0
• Safetensors version: 0.7.0
• Accelerate version: 1.13.0
• Accelerate config: not found
• DeepSpeed version: not installed
• PyTorch version (accelerator?): 2.11.0+cu126 (CUDA)
• Using distributed or parallel set-up in script?: no
• Using GPU in script?: yes
• GPU type: NVIDIA GeForce GTX 1080

Who can help?

@eustlb

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

from transformers import VoxtralForConditionalGeneration, VoxtralProcessor, BitsAndBytesConfig
from huggingface_hub import hf_hub_download
from transformers.audio_utils import load_audio_as

audio_url = "https://huggingface.co/datasets/hf-internal-testing/dummy-audio-samples/resolve/main/bcn_weather.mp3"
audio_path = hf_hub_download(repo_id="hf-internal-testing/dummy-audio-samples", filename="bcn_weather.mp3", repo_type="dataset")
audio_base64 = load_audio_as(audio_path, return_format="base64", force_mono=True)

# audio + text
conversation = [
    {
        "role": "user",
        "content": [
            {"type": "audio", "url": audio_url},
            {"type": "audio", "path": audio_path},
            {"type": "audio", "base64": audio_base64},
            {"type": "text", "text": "How many audio do you hear?"},
        ],
    },
]

processor = VoxtralProcessor.from_pretrained("mistralai/Voxtral-Mini-3B-2507")
inputs = processor.apply_chat_template(conversation)

***@Z590M-GAMING-X:~/Documents/Code/***/api$ /home/***/Documents/Code/***/api/.venv/bin/python /home/***/Documents/Code/***/api/main.py
bcn_weather.mp3: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 353k/353k [00:01<00:00, 292kB/s]
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
Traceback (most recent call last):
  File "/home/***/Documents/Code/***/api/main.py", line 23, in <module>
    inputs = processor.apply_chat_template(conversation)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/***/Documents/Code/***/api/.venv/lib/python3.12/site-packages/transformers/models/voxtral/processing_voxtral.py", line 175, in apply_chat_template
    template_kwargs = _get_template_variables(chat_template)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/***/Documents/Code/***/api/.venv/lib/python3.12/site-packages/transformers/utils/chat_template_utils.py", line 396, in _get_template_variables
    compiled = _compile_jinja_template(chat_template)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/***/Documents/Code/***/api/.venv/lib/python3.12/site-packages/transformers/utils/chat_template_utils.py", line 421, in _compile_jinja_template
    return _cached_compile_jinja_template(chat_template)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/***/Documents/Code/***/api/.venv/lib/python3.12/site-packages/transformers/utils/chat_template_utils.py", line 495, in _cached_compile_jinja_template
    return jinja_env.from_string(chat_template)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/***/Documents/Code/***/api/.venv/lib/python3.12/site-packages/jinja2/environment.py", line 1111, in from_string
    return cls.from_code(self, self.compile(source), gs, None)
                               ^^^^^^^^^^^^^^^^^^^^
  File "/home/***/Documents/Code/***/api/.venv/lib/python3.12/site-packages/jinja2/environment.py", line 764, in compile
    source = self._generate(source, name, filename, defer_init=defer_init)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/***/Documents/Code/***/api/.venv/lib/python3.12/site-packages/jinja2/environment.py", line 694, in _generate
    return generate(  # type: ignore
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/***/Documents/Code/***/api/.venv/lib/python3.12/site-packages/jinja2/compiler.py", line 112, in generate
    raise TypeError("Can't compile non template nodes")
TypeError: Can't compile non template nodes

Expected behavior

apply_chat_template should applies the model's chat completion template given a conversation without error

[theobarrague]

@Rocketknight1 maybe i'm missing something but your PR does not solve the problem. The chat_template is not defined on the hub, so self.chat_template will always return None and this code will always raise an exception

if chat_template is None:
    if isinstance(self.chat_template, dict) and "default" in self.chat_template:
        chat_template = self.chat_template["default"]
    elif isinstance(self.chat_template, dict):
        raise ValueError(
            'The processor has multiple chat templates but none of them are named "default". You need to specify'
            " which one to use by passing the `chat_template` argument. Available templates are: "
            f"{', '.join(self.chat_template.keys())}"
        )
    elif self.chat_template is not None:
        chat_template = self.chat_template
    else:
        raise ValueError(
            "Cannot use apply_chat_template because this processor does not have a chat template."
        )

[Rocketknight1]

Hi @theobarrague, you're right, I should have checked with your reproducer! I thought I'd seen what the issue was, but I didn't realize that the processor was just delegating to the tokenizer / mistral_common, so there was no Jinja template to load even on the base class.