Skip to content

Conversation

@neubig
Copy link
Contributor

@neubig neubig commented Dec 12, 2025

Summary

This PR updates the local LLMs documentation to feature the two most recent and powerful models: Qwen3-Coder-30B-A3B-Instruct and Devstral Small 2 (24B), removing references to older models.

Changes Made

1. Updated Model Information

  • ✅ Featured Qwen3-Coder-30B-A3B-Instruct (replacing older Qwen2.5-Coder references)
  • ✅ Featured Devstral-Small-2-24B-Instruct-2512 (replacing Devstral Small 2505)
  • ✅ Removed references to OpenHands LM 32B v0.1 and older Devstral versions

2. Hardware Requirements

  • ✅ Added accurate VRAM requirements based on real-world testing
    • Qwen3-Coder: 12GB VRAM minimum (tested on RTX 3060 12GB + 64GB RAM)
    • Devstral Small 2: 16GB VRAM minimum
  • ✅ Included Mac Apple Silicon requirements for both models

3. Context Window Recommendations

  • ✅ Added guidance for context window settings (22k minimum for Qwen3-Coder, 32k recommended)

4. Updated All Deployment Methods

  • LM Studio: Updated model search names and configuration examples
  • Ollama: Corrected pull commands to qwen3-coder:30b and devstral-small-2
  • SGLang: Updated model paths and served-model-name parameters
  • vLLM: Updated model paths and configurations
  • HuggingFace CLI: Corrected repository paths for model downloads

5. Configuration Examples

  • ✅ Updated Custom Model identifiers for all backends
  • ✅ Ensured consistency across LM Studio, Ollama, SGLang, and vLLM examples

Verification

All model names and commands have been verified against:

Files Changed

  • openhands/usage/llms/local-llms.mdx

Context

Based on community feedback, these two models represent the current state-of-the-art for local coding assistance with OpenHands:

  • Qwen3-Coder-30B-A3B shows excellent performance even on consumer hardware (12GB VRAM)
  • Devstral Small 2 achieves remarkable results on SWE-bench and agentic coding tasks

Both models are significantly more capable than the previously documented options.

@neubig can click here to continue refining the PR

…ral Small 2512

- Updated News section to highlight the two recommended models
- Revised Quickstart guide to cover both Qwen 3 Coder 30B and Devstral Small 2512
- Updated hardware requirements based on user feedback (12GB VRAM for Qwen 3 Coder)
- Added context window recommendations (22k minimum for Qwen 3 Coder)
- Updated all model references throughout the document including Ollama, SGLang, and vLLM sections
- Removed references to older models (OpenHands LM 32B v0.1 and Devstral Small 2505)
- Corrected Qwen model name to Qwen3-Coder-30B-A3B-Instruct (not Qwen2.5-Coder-32B-Instruct)
- Updated Devstral to full official name: Devstral-Small-2-24B-Instruct-2512
- Fixed Ollama pull commands: qwen3-coder:30b and devstral-small-2
- Updated all HuggingFace repository paths to match official releases
- Corrected model identifiers in all examples (LM Studio, Ollama, SGLang, vLLM)
…mples

- Removed Devstral from all example commands to keep instructions concise
- Kept Devstral in News section as an alternative option
- Updated all sections: LM Studio, Ollama, SGLang, vLLM
- Simplified hardware requirements and configuration examples
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants