OVMS 2026.1-gpu fails to load OpenVINO/gemma-4-E4B-it-int4-ov with Unsupported 'gemma4' VLM model type

[jthakare17]

Describe the bug
OVMS fails to load the VLM model OpenVINO/gemma-4-E4B-it-int4-ov and exits during initialization with:

Unsupported 'gemma4' VLM model type

This happens when running openvino/model_server:2026.1-gpu.

The behavior appears inconsistent with the current openvinotoolkit/openvino.genai source, where gemma4 is explicitly recognized in VLM config parsing.

To Reproduce

1. Prepare a model repository containing:

   • models/OpenVINO/gemma-4-E4B-it-int4-ov/
   • The model used is OpenVINO/gemma-4-E4B-it-int4-ov

2. Launch OVMS:

   docker run -it --rm \
     -p 8000:8000 \
     --device /dev/dri \
     --group-add=$(stat -c "%g" /dev/dri/render* | head -n 1) \
     -v $(pwd)/models:/models:ro \
     openvino/model_server:2026.1-gpu \
     --rest_port 8000 \
     --model_name OpenVINO/gemma-4-E4B-it-int4-ov \
     --model_path ./models/OpenVINO/gemma-4-E4B-it-int4-ov/

3. No client request is needed. The failure happens during model initialization at server startup.

4. Observe the error:

   Error during llm node initialization ...
   Unsupported 'gemma4' VLM model type
   

Expected behavior
OVMS should recognize model_type: "gemma4" and initialize the model successfully, or the image/docs should clearly state that Gemma4 is not supported by this OVMS image.

Logs
Startup version info:

[2026-05-05 08:15:34.612][1][serving][info][server.cpp:88] OpenVINO Model Server 2026.1.0.72cc06244
[2026-05-05 08:15:34.612][1][serving][info][server.cpp:89] OpenVINO backend 2026.1.0-21367-63e31528c62-releases/2026/1
[2026-05-05 08:15:34.612][1][serving][info][server.cpp:92] OpenVINO GenAI backend 2026.1.0.0-2957-1dabb8c2255

Relevant failure logs:

[2026-05-05 08:06:22.232][1][modelmanager][info][servable_initializer.cpp:448] Initializing Visual Language Model Continuous Batching servable
[2026-05-05 08:06:23.062][1][serving][error][servable_initializer.cpp:214] Error during llm node initialization for models_path: //./models/OpenVINO/gemma-4-E4B-it-int4-ov/./ exception: Exception from ../../../../../repos/openvino.genai/src/cpp/src/visual_language/vlm_config.cpp:34:
Unsupported 'gemma4' VLM model type

[2026-05-05 08:06:23.062][1][modelmanager][error][servable_initializer.cpp:453] Error during LLM node resources initialization: The LLM Node resource initialization failed
[2026-05-05 08:06:23.062][1][serving][error][mediapipegraphdefinition.cpp:474] Failed to process LLM node graph OpenVINO/gemma-4-E4B-it-int4-ov
[2026-05-05 08:06:23.062][1][modelmanager][info][pipelinedefinitionstatus.hpp:59] Mediapipe: OpenVINO/gemma-4-E4B-it-int4-ov state changed to: LOADING_PRECONDITION_FAILED after handling: ValidationFailedEvent:
[2026-05-05 08:06:23.062][1][modelmanager][error][modelmanager.cpp:184] Couldn't start model manager
[2026-05-05 08:06:23.062][1][serving][error][servablemanagermodule.cpp:58] ovms::ModelManager::Start() Error: The LLM Node resource initialization failed

There is no client log because the server fails before serving requests.

Configuration

1. OVMS version:

   • openvino/model_server:2026.1-gpu
   • Logged as: OpenVINO Model Server 2026.1.0.72cc06244

2. OVMS config.json file:

   • Not used
   • Model was provided via CLI arguments:
     • --model_name OpenVINO/gemma-4-E4B-it-int4-ov
     • --model_path ./models/OpenVINO/gemma-4-E4B-it-int4-ov/

3. CPU / accelerator versions if applicable:

   • OpenVINO backend: 2026.1.0-21367-63e31528c62-releases/2026/1
   • OpenVINO GenAI backend: 2026.1.0.0-2957-1dabb8c2255
   • Device exposed in container: GPU via /dev/dri

4. Model repository directory structure:

   models/
   └── OpenVINO/
       └── gemma-4-E4B-it-int4-ov/
           ├── config.json
           ├── preprocessor_config.json
           ├── tokenizer.json
           ├── tokenizer_config.json
           ├── openvino_language_model.xml
           ├── openvino_language_model.bin
           ├── ...
   

5. Model or publicly available similar model that reproduces the issue:

   • OpenVINO/gemma-4-E4B-it-int4-ov

Additional context
The current openvinotoolkit/openvino.genai source appears to support gemma4 in VLM config parsing, for example in src/cpp/src/visual_language/vlm_config.cpp, where gemma4 is mapped to VLMModelType::GEMMA4.

Because of that, this looks like one of:

• OVMS 2026.1-gpu does not actually include effective Gemma4 support yet
• the bundled GenAI backend is inconsistent with the source support
• Gemma4 support exists in openvino.genai repo but is not yet supported in the shipped OVMS image

If helpful, I can test a nightly image or provide additional model files / exact directory contents.