Skip to content

Comments

feat: Add LM Studio integration with embedding and generation components#391

Open
supmo668 wants to merge 1 commit intoweaviate:mainfrom
Contextual-Genticmen:template-main
Open

feat: Add LM Studio integration with embedding and generation components#391
supmo668 wants to merge 1 commit intoweaviate:mainfrom
Contextual-Genticmen:template-main

Conversation

@supmo668
Copy link

@supmo668 supmo668 commented Sep 7, 2025

No description provided.

Copilot AI review requested due to automatic review settings September 7, 2025 19:05
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds LM Studio integration to the system, enabling users to leverage locally hosted models through LM Studio's OpenAI-compatible API for both text generation and embedding functionality.

  • Implements LMStudioGenerator for text generation using local models
  • Implements LMStudioEmbedder for document and query vectorization
  • Adds configuration options and environment variables for LM Studio integration

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
goldenverba/components/managers.py Registers the new LM Studio components in both development and production environments
goldenverba/components/generation/LMStudioGenerator.py Implements text generation using LM Studio's chat completions API with streaming support
goldenverba/components/embedding/LMStudioEmbedder.py Implements embedding functionality using LM Studio's embeddings API
goldenverba/.env.example Adds environment variable examples for LM Studio configuration

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment on lines +71 to +80
# Convert payload to BytesIO object
payload_bytes = json.dumps(payload).encode("utf-8")
payload_io = io.BytesIO(payload_bytes)

async with aiohttp.ClientSession() as session:
try:
async with session.post(
f"{base_url}/embeddings",
headers=headers,
data=payload_io,
Copy link

Copilot AI Sep 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The conversion of JSON payload to BytesIO is unnecessary. The aiohttp library can handle JSON payloads directly using the json parameter instead of data. This would simplify the code and remove the need for manual serialization.

Suggested change
# Convert payload to BytesIO object
payload_bytes = json.dumps(payload).encode("utf-8")
payload_io = io.BytesIO(payload_bytes)
async with aiohttp.ClientSession() as session:
try:
async with session.post(
f"{base_url}/embeddings",
headers=headers,
data=payload_io,
# Send payload as JSON using aiohttp's json parameter
async with aiohttp.ClientSession() as session:
try:
async with session.post(
f"{base_url}/embeddings",
headers=headers,
json=payload,

Copilot uses AI. Check for mistakes.
Comment on lines +71 to +80
# Convert payload to BytesIO object
payload_bytes = json.dumps(payload).encode("utf-8")
payload_io = io.BytesIO(payload_bytes)

async with aiohttp.ClientSession() as session:
try:
async with session.post(
f"{base_url}/embeddings",
headers=headers,
data=payload_io,
Copy link

Copilot AI Sep 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should use json=payload instead of data=payload_io to properly send JSON data with correct Content-Type headers automatically handled by aiohttp.

Suggested change
# Convert payload to BytesIO object
payload_bytes = json.dumps(payload).encode("utf-8")
payload_io = io.BytesIO(payload_bytes)
async with aiohttp.ClientSession() as session:
try:
async with session.post(
f"{base_url}/embeddings",
headers=headers,
data=payload_io,
async with aiohttp.ClientSession() as session:
try:
async with session.post(
f"{base_url}/embeddings",
headers=headers,
json=payload,

Copilot uses AI. Check for mistakes.
Copy link

@orca-security-eu orca-security-eu bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Orca Security Scan Summary

Status Check Issues by priority
Passed Passed Infrastructure as Code high 0   medium 0   low 0   info 0 View in Orca
Passed Passed SAST high 0   medium 0   low 0   info 0 View in Orca
Passed Passed Secrets high 0   medium 0   low 0   info 0 View in Orca
Passed Passed Vulnerabilities high 0   medium 0   low 0   info 0 View in Orca

@weaviate-git-bot
Copy link

To avoid any confusion in the future about your contribution to Weaviate, we work with a Contributor License Agreement. If you agree, you can simply add a comment to this PR that you agree with the CLA so that we can merge.

beep boop - the Weaviate bot 👋🤖

PS:
Are you already a member of the Weaviate Forum?

@supmo668
Copy link
Author

supmo668 commented Sep 7, 2025

I agree with the CLA set by Weaviate

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants