Skip to content

fix: Unbounded batch processing in embedding #49

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mrwind-up-bird wants to merge 1 commit into
base: main
Choose a base branch
from
Open

fix: Unbounded batch processing in embedding #49

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions app/services/embedding.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@

from litellm import aembedding

# Maximum total texts to prevent memory exhaustion
MAX_TOTAL_TEXTS = 10000
# Default embedding model — small, fast, cheap
DEFAULT_EMBEDDING_MODEL = "text-embedding-3-small"
DEFAULT_EMBEDDING_DIMENSIONS = 1536
Expand All @@ -22,6 +24,10 @@ async def embed_texts(
model: Embedding model name (LiteLLM format). Defaults to text-embedding-3-small.
api_key: Optional provider API key. If None, uses env vars (OPENAI_API_KEY, etc.).


# Prevent processing extremely large text collections
if len(texts) > MAX_TOTAL_TEXTS:
raise ValueError(f"Too many texts to embed: {len(texts)}. Maximum allowed: {MAX_TOTAL_TEXTS}")
Returns:
List of embedding vectors (same order as input texts).
"""
Expand Down