Skip to content

Distributed LLM inference. Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.

License

Notifications You must be signed in to change notification settings

The-Gentlemen-Approach/distributed-llama

 
 

Repository files navigation

Distributed Llama

Distributed Llama for HPipe

License: MIT

Connect home devices into a powerful cluster to accelerate LLM inference. More devices mean faster performance, leveraging HPipe parallelism and high-speed synchronization over Ethernet.

Our team(The gentlemen approach) is developing this project to realize HPipe and compare with other method.

Supports Linux, macOS, and Windows.

Special Thanks to original project developers, prof. LIM.

How to Run

  • Sorry but this project is not finished so we will decribe it later.

News

  • 27 Dec 2025 - Started to approach gentle.
  • 28 Dec 2025 - Finished developing H-Pipe network (root, worker etc.)

💡 License

This project is released under the MIT license.

About

Distributed LLM inference. Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • C++ 96.9%
  • Python 2.8%
  • Other 0.3%