feat: Add LM Studio integration with embedding and generation components#391
feat: Add LM Studio integration with embedding and generation components#391supmo668 wants to merge 1 commit intoweaviate:mainfrom
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR adds LM Studio integration to the system, enabling users to leverage locally hosted models through LM Studio's OpenAI-compatible API for both text generation and embedding functionality.
- Implements LMStudioGenerator for text generation using local models
- Implements LMStudioEmbedder for document and query vectorization
- Adds configuration options and environment variables for LM Studio integration
Reviewed Changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| goldenverba/components/managers.py | Registers the new LM Studio components in both development and production environments |
| goldenverba/components/generation/LMStudioGenerator.py | Implements text generation using LM Studio's chat completions API with streaming support |
| goldenverba/components/embedding/LMStudioEmbedder.py | Implements embedding functionality using LM Studio's embeddings API |
| goldenverba/.env.example | Adds environment variable examples for LM Studio configuration |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| # Convert payload to BytesIO object | ||
| payload_bytes = json.dumps(payload).encode("utf-8") | ||
| payload_io = io.BytesIO(payload_bytes) | ||
|
|
||
| async with aiohttp.ClientSession() as session: | ||
| try: | ||
| async with session.post( | ||
| f"{base_url}/embeddings", | ||
| headers=headers, | ||
| data=payload_io, |
There was a problem hiding this comment.
The conversion of JSON payload to BytesIO is unnecessary. The aiohttp library can handle JSON payloads directly using the json parameter instead of data. This would simplify the code and remove the need for manual serialization.
| # Convert payload to BytesIO object | |
| payload_bytes = json.dumps(payload).encode("utf-8") | |
| payload_io = io.BytesIO(payload_bytes) | |
| async with aiohttp.ClientSession() as session: | |
| try: | |
| async with session.post( | |
| f"{base_url}/embeddings", | |
| headers=headers, | |
| data=payload_io, | |
| # Send payload as JSON using aiohttp's json parameter | |
| async with aiohttp.ClientSession() as session: | |
| try: | |
| async with session.post( | |
| f"{base_url}/embeddings", | |
| headers=headers, | |
| json=payload, |
| # Convert payload to BytesIO object | ||
| payload_bytes = json.dumps(payload).encode("utf-8") | ||
| payload_io = io.BytesIO(payload_bytes) | ||
|
|
||
| async with aiohttp.ClientSession() as session: | ||
| try: | ||
| async with session.post( | ||
| f"{base_url}/embeddings", | ||
| headers=headers, | ||
| data=payload_io, |
There was a problem hiding this comment.
This should use json=payload instead of data=payload_io to properly send JSON data with correct Content-Type headers automatically handled by aiohttp.
| # Convert payload to BytesIO object | |
| payload_bytes = json.dumps(payload).encode("utf-8") | |
| payload_io = io.BytesIO(payload_bytes) | |
| async with aiohttp.ClientSession() as session: | |
| try: | |
| async with session.post( | |
| f"{base_url}/embeddings", | |
| headers=headers, | |
| data=payload_io, | |
| async with aiohttp.ClientSession() as session: | |
| try: | |
| async with session.post( | |
| f"{base_url}/embeddings", | |
| headers=headers, | |
| json=payload, |
There was a problem hiding this comment.
Orca Security Scan Summary
| Status | Check | Issues by priority | |
|---|---|---|---|
| Infrastructure as Code | View in Orca | ||
| SAST | View in Orca | ||
| Secrets | View in Orca | ||
| Vulnerabilities | View in Orca |
|
To avoid any confusion in the future about your contribution to Weaviate, we work with a Contributor License Agreement. If you agree, you can simply add a comment to this PR that you agree with the CLA so that we can merge. |
|
I agree with the CLA set by Weaviate |
No description provided.