Skip to content

Add support for Voxtral-Mini-4B-Realtime model via vLLM backend #8401

@localai-bot

Description

@localai-bot

Add support for the Voxtral-Mini-4B-Realtime-2602 model from Hugging Face via the vLLM backend.

Model URL: https://huggingface.co/mistralai/Voxtral-Mini-4B-Realtime-2602

This model is designed for real-time automatic speech recognition (ASR) and benefits from streaming inference capabilities. Integration with LocalAI's vLLM backend would enable real-time transcription workflows. This may require extending the current gRPC streaming interface to properly support real-time ASR streaming.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions