Skip to content

Auto-register all Whisper models with lazy loading #261

@basnijholt

Description

@basnijholt

Summary

Instead of requiring --model flags, auto-register all available faster-whisper models by default. Models are downloaded lazily on first API request.

Current behavior

agent-cli server whisper --model large-v3 --model small

User must explicitly specify which models to make available.

Proposed behavior

agent-cli server whisper  # All models available by default
  • Register all 19 models from faster_whisper.available_models()
  • No upfront download - models download on first request
  • TTL unloads idle models as usual
  • Optional --model flag to limit to specific models

Benefits

  • Simpler Docker config (no WHISPER_MODEL env var needed)
  • API clients can request any model without server restart
  • No wasted disk space - only requested models are downloaded

Models available

tiny.en, tiny, base.en, base, small.en, small, medium.en, medium,
large-v1, large-v2, large-v3, large, distil-large-v2, distil-medium.en,
distil-small.en, distil-large-v3, distil-large-v3.5, large-v3-turbo, turbo

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions