feat(whisper): add VAD-based streaming to WebSocket transcription endpoint

[basnijholt]

Summary

Add Voice Activity Detection (VAD) to the WebSocket endpoint /v1/audio/transcriptions/stream so remote clients receive partial transcriptions as speech segments complete, rather than waiting for the entire audio stream.

• Reuses the existing VoiceActivityDetector from agent_cli/core/vad
• Ports the VAD pattern from transcribe-daemon to the WebSocket handler
• New query parameters for VAD configuration with sensible defaults
• Backward compatible via use_vad=false query parameter

Note

Non-standard endpoint: The WebSocket streaming endpoint (/v1/audio/transcriptions/stream) is a custom extension, not part of the OpenAI API specification. OpenAI's standard transcription API is REST-only (POST /v1/audio/transcriptions). While WebSocket streaming for ASR is common in the industry (Deepgram, AssemblyAI, AWS Transcribe, Azure Speech), there is no universal standard format. This endpoint is provided for advanced use cases but may not be compatible with other OpenAI-compatible clients.

New Query Parameters

Parameter	Type	Default	Description
use_vad	bool	true	Enable VAD for streaming partial results
vad_threshold	float	0.3	Speech detection threshold (0.0-1.0)
vad_silence_ms	int	1000	Silence duration (ms) to end segment
vad_min_speech_ms	int	250	Min speech duration (ms) to trigger transcription

Message Protocol

Compatible with existing format:

{"type": "partial", "text": "first utterance", "is_final": false, "language": "en"}
{"type": "partial", "text": "second utterance", "is_final": false, "language": "en"}
{"type": "final", "text": "first utterance second utterance", "is_final": true}

Closes #263

Test plan

• Run pytest tests/ - all 909 tests pass
• Run pre-commit run --all-files - all checks pass
• Manual testing with a WebSocket client sending real audio

[basnijholt]

Code Review

Overall the implementation looks solid - clean separation of concerns and good reuse of existing VoiceActivityDetector.

Minor Concerns

1. Final message differs between modes

   In buffered mode, the final message includes segments and duration:

   {"type": "final", "text": "...", "segments": [...], "duration": 5.2, ...}

   In VAD mode, these are missing:

   {"type": "final", "text": "...", "language": "...", "is_final": true}

   This is probably intentional since VAD combines multiple independent transcriptions, but clients expecting duration may break. Consider adding a combined duration or documenting this difference.

2. Silent segment failures

   _transcribe_segment() catches all exceptions and returns None:

   except Exception:
       logger.exception("Failed to transcribe segment")
       return None

   This means if a segment fails to transcribe, the client never knows - it just won't appear in the final text. Consider sending an error/warning message to the client when a segment fails, or at minimum noting in the final message how many segments were processed vs failed.

Questions

• Should use_vad=true be the default? This changes behavior for existing clients. Maybe use_vad=false as default for backward compat, with documentation encouraging VAD mode?