Queue: LlamaIndex audio document loader (Python)

[github-actions]

Integration: LlamaIndex Audio Document Processing

Why this is valuable

LlamaIndex is a leading AI framework for building RAG pipelines and document processing. Developers building AI applications need a way to ingest audio content (podcasts, meetings, lectures) into their LlamaIndex pipelines. This fills a gap in AI framework coverage — LangChain is covered (040) but LlamaIndex is not.

Suggested approach

A Python script or Jupyter notebook that uses Deepgram to transcribe audio files, creates LlamaIndex Document objects from the transcripts, and demonstrates indexing and querying the audio content. Could also showcase Deepgram audio intelligence (summarization, topics, sentiment) as metadata on the documents.

Reference

• LlamaIndex docs
• LlamaIndex custom data loaders

---

Queued by discover-examples agent on 2026-03-29

[github-actions]

🔬 Research findings

Platform: LlamaIndex
Suggested language: Python
Integration type: SDK (custom data loader / reader pattern)

Platform SDK

• Package: llama-index-core (core) + llama-index (starter bundle)
• Version: 0.14.19 (latest, released 2026-03-25)
• Install: pip install llama-index-core (minimal) or pip install llama-index (with default integrations)
• Key imports:

  from llama_index.core import VectorStoreIndex, Document, Settings
  from llama_index.core.readers.base import BaseReader
  from llama_index.core.schema import Document

Deepgram SDK

• Package: deepgram-sdk
• Version: 6.1.1 (latest)
• Install: pip install deepgram-sdk
• Key imports:

  from deepgram import DeepgramClient

Integration pattern

Audio file (local or URL) → Deepgram pre-recorded STT API (client.listen.v1.media.transcribe_file or transcribe_url) with intelligence features enabled (summarize, topics, sentiment, intents, detect_entities) → Transcript text becomes LlamaIndex Document.text → Intelligence results (summary, topics, sentiment, entities) become Document.metadata → Documents fed into VectorStoreIndex for RAG querying.

This follows the same pattern as the existing llama-index-readers-assemblyai integration (run-llama/llama_index/llama-index-integrations/readers/llama-index-readers-assemblyai/), which implements BaseReader with a load_data() method that transcribes audio and returns List[Document].

Deepgram API to use

• Product: STT pre-recorded + Audio Intelligence
• SDK methods:
  • client.listen.v1.media.transcribe_file(request=audio_bytes, model="nova-3", ...) — for local files
  • client.listen.v1.media.transcribe_url(url=audio_url, model="nova-3", ...) — for remote URLs
• Key options:
  • model="nova-3" — latest model
  • smart_format=True — formatting, punctuation, casing
  • summarize="v2" — generates summary
  • topics=True — topic detection
  • sentiment=True — sentiment analysis
  • intents=True — intent recognition
  • detect_entities=True — entity extraction
  • paragraphs=True — paragraph segmentation
  • utterances=True — utterance segmentation
  • diarize=True — speaker diarization (optional)
  • language="en" — language code

Required credentials

Variable	Where to find it
DEEPGRAM_API_KEY	https://console.deepgram.com/
OPENAI_API_KEY	https://platform.openai.com/api-keys (needed for LlamaIndex default LLM/embeddings when querying the index)

No existing Deepgram + LlamaIndex reader

• There is no llama-index-readers-deepgram package on PyPI
• No existing integration found on GitHub
• The llama-index-readers-assemblyai and llama-index-readers-whisper readers exist as comparable audio-to-document integrations — use AssemblyAI reader as the primary architectural reference

Kapa results

• Kapa API returned 404 for all queries — no Deepgram-specific LlamaIndex integration docs found. This integration appears to be novel.

Potential gotchas

1. Deepgram SDK v6 is Fern-generated — the API surface changed significantly from v3/v5. Use client.listen.v1.media.transcribe_file() not older patterns like client.listen.prerecorded.v().
2. Intelligence features are add-on parameters on the same transcription call, not separate API endpoints. Pass summarize="v2", topics=True, etc. as keyword args to the transcribe method.
3. Response structure — transcription is at response.results.channels[0].alternatives[0].transcript; intelligence results are at response.results.summary, response.results.topics, response.results.sentiments, response.results.intents, response.results.entities.
4. LlamaIndex BaseReader contract — must implement load_data() -> List[Document]. Keep it simple; the reader just loads data, indexing/querying is separate.
5. LLM dependency for querying — LlamaIndex needs an LLM (default OpenAI) and embeddings to query the index. The example should clearly separate the "load" step (Deepgram-only) from the "query" step (requires LLM).
6. File size limits — Deepgram pre-recorded API supports files up to 2GB / 7 hours. No special handling needed for typical examples.
7. Audio format — Deepgram auto-detects encoding for pre-recorded files, so no manual encoding specification is needed for common formats (wav, mp3, flac, m4a, etc.).

Reference links

• LlamaIndex docs
• LlamaIndex custom data loaders
• LlamaIndex GitHub (48k stars)
• AssemblyAI reader (pattern reference)
• Deepgram Python SDK v6.1.1
• Deepgram SDK reference
• Deepgram Audio Intelligence docs

---

Research by Researcher on 2026-03-31

[github-actions]

Built in #90