⚡ Ultimate Free LLM Stack

A fully automated, 100% free AI proxy stack using LiteLLM. Route your requests across 7 different free AI providers with automatic failover and load balancing using 75+ free API keys!

🚀 Features

7 Free Providers Supported: Groq, Ollama (Cloud), NVIDIA NIM, Google Gemini, Cerebras, OpenRouter, Mistral.
75+ Keys Load Balancing: Add up to 10-15 keys per provider to completely bypass rate limits.
Auto-Failover: If one provider goes down or rate-limits you, it automatically falls back to the next one seamlessly.
Unified API: Access all models using a standard OpenAI-compatible API endpoint.
Admin Dashboard: Track usage, add users, and monitor spend.

📦 Installation

We provide a 1-click installer that sets up uv (super fast Python package manager), installs LiteLLM, configures the database client, and prepares all scripts.

Clone this repository:

git clone https://github.com/technicalboy2023/Ultimate-Free-LLM-Stack.git
cd Ultimate-Free-LLM-Stack

Run the automated installer:
```
chmod +x install.sh
./install.sh
```
Setup your environment: The installer automatically creates a .env file for you from .env.example. Open it and add your free API keys:
```
nano .env
```

🎮 Complete Usage Guide

1️⃣ LiteLLM Proxy START Kaise Karein

Method 1: Using start script (Recommended)

./start.sh

Method 2: Manual Start

cd ~/litellm-uv && set -a && source .env && set +a
litellm --config configs/litellm_config.yaml --port 4000

⚠️ Important: You must load the environment variables (.env) before running litellm manually. Bina iske API keys aur DATABASE_URL load nahi honge.

✅ Successful Start ka Output aisa dikhega:

INFO:     Started server process [xxxxx]
INFO:     Waiting for application startup.

   ██╗     ██╗████████╗███████╗██╗     ██╗     ███╗   ███╗
   ██║     ██║╚══██╔══╝██╔════╝██║     ██║     ████╗ ████║
   ...

LiteLLM: Proxy initialized with Config, Set models:
    groq-coder
    ollama-coder
    nvidia-coder
    ...

INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:4000

🌐 Start hone ke baad access karein:

Service	URL
API Endpoint	`http://localhost:4000`
Admin Dashboard	`http://localhost:4000/ui`
Health Check	`http://localhost:4000/health`

🔑 Dashboard Login:

Username: Kuch bhi daal do (e.g. admin)
Password: .env file mein jo LITELLM_MASTER_KEY hai wahi daalo

📋 Quick One-Liner (Copy-Paste Ready):

cd ~/litellm-uv && set -a && source .env && set +a && litellm --config configs/litellm_config.yaml --port 4000

2️⃣ LiteLLM Proxy STOP Kaise Karein

Method 1: Using stop script (Recommended)

./stop.sh

Method 2: Keyboard Shortcut (Sabse Easy) Jis terminal mein proxy chal raha hai, wahan press karein:

Ctrl + C

Output aayega: INFO: Shutting down — matlab proxy band ho gaya.

Method 3: Process Kill (Agar terminal band ho gaya ho)

# Pehle process dhoondho:
ps aux | grep litellm

# Phir process ID (PID) se kill karo:
kill <PID>

# Example:
kill 289404

Method 4: Force Kill (Agar hang ho gaya ho)

# Sabhi litellm processes ek sath band karo:
pkill -f litellm

# Ya forcefully:
pkill -9 -f litellm

Method 5: Port se Kill (Agar port 4000 busy aa raha ho)

# Dekho kaun port 4000 use kar raha hai:
lsof -i :4000

# Phir uski PID ko kill karo:
kill $(lsof -t -i :4000)

3️⃣ Latest Version Per UPDATE Kaise Karein

Method 1: Using update script (Recommended)

./update.sh

Method 2: Manual Update

# Current Version Check:
litellm --version

# Update Command (uv se install kiya hai):
uv tool upgrade litellm

Agar uv tool upgrade kaam na kare:

# Pehle uninstall karo, phir fresh install:
uv tool uninstall litellm
uv tool install litellm

Update ke baad Prisma Binary Fix (Zaroori Hai!):

⚠️ IMPORTANT: Har update ke baad Prisma binary dobara generate karni padti hai, nahi toh "prisma not found" error aayega.

# Step 1: LiteLLM ka Python path dhoondho:
LITELLM_PYTHON=$(dirname $(dirname $(which litellm)))/lib/python3.*/site-packages

# Step 2: Prisma generate karo:
cd $LITELLM_PYTHON/prisma
python -m prisma generate

# Step 3: Verify karo ki Prisma kaam kar raha hai:
python -c "import prisma; print('Prisma OK:', prisma.__version__)"

Quick Update One-Liner:

uv tool upgrade litellm && echo "✅ Updated! Ab Prisma fix karo (neeche dekho)"

4️⃣ Background Mein Run Kaise Karein (Optional)

Method 1: screen se (Recommended)

# Naya screen session banao:
screen -S litellm

# Andar se proxy start karo:
cd ~/litellm-uv && set -a && source .env && set +a
litellm --config configs/litellm_config.yaml --port 4000

# Screen se bahar aao (proxy chalti rahegi):
# Press: Ctrl+A phir D

# Wapas jaane ke liye:
screen -r litellm

# Band karne ke liye screen mein jaake Ctrl+C

Method 2: nohup se (Terminal band karne ke baad bhi chalega)

cd ~/litellm-uv
set -a && source .env && set +a
nohup litellm --config configs/litellm_config.yaml --port 4000 > litellm.log 2>&1 &

Log dekhne ke liye: tail -f ~/litellm-uv/litellm.log
Band karne ke liye: pkill -f litellm

Method 3: tmux se

# Naya tmux session:
tmux new -s litellm

# Start proxy:
cd ~/litellm-uv && set -a && source .env && set +a
litellm --config configs/litellm_config.yaml --port 4000

# Detach: Ctrl+B phir D
# Reattach: tmux attach -t litellm

5️⃣ Available Models (Config Mein)

Model Name	Provider	Backend Model	Keys
`groq-coder`	Groq	llama-3.3-70b-versatile	10
`ollama-coder`	Ollama Cloud	gemma4:31b	15
`nvidia-coder`	NVIDIA NIM	deepseek-ai/deepseek-v4-pro	10
`gemini-coder`	Google	gemma-4-31b-it	10
`cerebras-coder`	Cerebras	llama3.3-70b	10
`openrouter-coder`	OpenRouter	nvidia/nemotron-3-super-120b-a12b:free	10
`mistral-coder`	Mistral	codestral-latest	10
`free-coder`	Mixed	Auto-failover across all providers	6
`claude-sonnet-4-5`	Alias	Routes to groq/ollama/nvidia	3
`claude-haiku-4-5-20251001`	Alias	Routes to cerebras/groq/ollama	3

🤖 How to use with Claude Code CLI

This proxy is fully compatible with Anthropic's Claude Code CLI! The litellm_config.yaml is pre-configured with exact model identifiers and drop_params: true to prevent Anthropic-specific API errors.

Setup Instructions

Start the Proxy: Make sure your LiteLLM proxy is running (e.g., via ./start.sh or the Desktop Launcher).

Export Environment Variables: Open the terminal where you plan to use Claude Code and run:

# Point Claude Code to your local LiteLLM proxy
export ANTHROPIC_BASE_URL="http://localhost:4000"

# Use your LiteLLM master key from your .env file
export ANTHROPIC_AUTH_TOKEN="YOUR_LITELLM_MASTER_KEY"

# Enable Gateway Model Discovery (Required to see proxy models in the picker)
export CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY=1

Launch Claude Code:
```
claude
```
Select a Model: Type /model inside Claude Code. You will see models labeled "From gateway" (e.g., claude-sonnet-4-6, claude-opus-4-7). Select one, and Claude Code will use your free proxy backend!

💡 Tip: If you want to make this permanent, add the export commands to your ~/.bashrc or ~/.zshrc file.

6️⃣ API Se Test Kaise Karein

cURL se test:

curl http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_LITELLM_MASTER_KEY" \
  -d '{
    "model": "free-coder",
    "messages": [{"role": "user", "content": "Hello! Kya haal hai?"}]
  }'

Health check:

curl http://localhost:4000/health

Models list dekhein:

curl http://localhost:4000/v1/models \
  -H "Authorization: Bearer YOUR_LITELLM_MASTER_KEY"

🔑 Where to get FREE API Keys

Create accounts and get your free API keys here. Add them to your .env file:

Groq (14,400 req/day): console.groq.com
Ollama Cloud (Generous session limits): ollama.com/settings/keys
NVIDIA NIM (400 req/min): build.nvidia.com
Google Gemini (15,000 req/day): aistudio.google.com/apikey
Cerebras (10M tokens/day): cloud.cerebras.ai
OpenRouter (200 req/day): openrouter.ai/settings/keys
Mistral (10B tokens/month): console.mistral.ai

🌐 Endpoints & API Testing

Once started, access your proxy at:

API Endpoint: http://localhost:4000/v1
Admin Dashboard: http://localhost:4000/ui
Health Check: http://localhost:4000/health

7️⃣ Common Errors & Solutions

Error	Solution
"Unable to connect to DB. prisma package not found"	Prisma binary missing hai. Run `./update.sh` which includes the Prisma fix, or manually run `python -m prisma generate` in your LiteLLM env.
"Address already in use (port 4000)"	Port pehle se busy hai. Run `./stop.sh` or `kill $(lsof -t -i :4000)` then restart.
"FATAL: password authentication failed"	Database password mein special characters hain. `.env` file mein check karo ki password URL-encoded hai (`@` → `%40`, `&` → `%26`).
"Connection refused / ENOTFOUND"	Internet check karo. Supabase URL sahi hai ya nahi `.env` mein verify karo.
"APIConnectionError: Timeout on reading data"	Ye specific model provider slow hai ya down hai. LiteLLM automatically fallback karega dusre provider pe. Kuch karne ki zaroorat nahi.

8️⃣ Important Files & Locations

File / Path	Purpose
`~/litellm-uv/.env`	🔒 API Keys, DB URL, Master Key
`~/litellm-uv/configs/litellm_config.yaml`	Models, routing, fallbacks
`~/.local/share/uv/tools/litellm/`	LiteLLM ka installed environment
`~/litellm-uv/litellm.log`	Log file (agar background mein chalaya ho)

9️⃣ Daily Routine (Cheat Sheet)

📌 SUBAH — Start:
   cd ~/litellm-uv && set -a && source .env && set +a && litellm --config configs/litellm_config.yaml --port 4000

📌 RAAT — Stop:
   Ctrl+C (terminal mein) ya pkill -f litellm

📌 MONTHLY — Update:
   uv tool upgrade litellm
   (Phir Prisma fix karna mat bhoolna!)

📌 CHECK — Status:
   curl http://localhost:4000/health

🛡️ Security Reminders

Never commit your .env file! It contains your secret API keys and database passwords. The included .gitignore handles this automatically.
.env file kabhi share mat karna — Isme API keys aur DB password hai.
Keep your LITELLM_MASTER_KEY secure, as it controls dashboard access.
If adding new providers (e.g. Anthropic, Cohere), add them to configs/litellm_config.yaml and put their keys in .env. Phir proxy restart karo.

💡 Tip: Agar koi naya provider add karna ho (jaise Anthropic, Cohere), toh configs/litellm_config.yaml mein model_list ke andar naya entry daalo aur .env mein uski key daalo. Phir proxy restart karo.

Made with ❤️ for Ankush

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

⚡ Ultimate Free LLM Stack

🚀 Features

📦 Installation

🎮 Complete Usage Guide

1️⃣ LiteLLM Proxy START Kaise Karein

✅ Successful Start ka Output aisa dikhega:

🌐 Start hone ke baad access karein:

🔑 Dashboard Login:

📋 Quick One-Liner (Copy-Paste Ready):

2️⃣ LiteLLM Proxy STOP Kaise Karein

3️⃣ Latest Version Per UPDATE Kaise Karein

Update ke baad Prisma Binary Fix (Zaroori Hai!):

Quick Update One-Liner:

4️⃣ Background Mein Run Kaise Karein (Optional)

5️⃣ Available Models (Config Mein)

🤖 How to use with Claude Code CLI

Setup Instructions

6️⃣ API Se Test Kaise Karein

cURL se test:

Health check:

Models list dekhein:

🔑 Where to get FREE API Keys

🌐 Endpoints & API Testing

7️⃣ Common Errors & Solutions

8️⃣ Important Files & Locations

9️⃣ Daily Routine (Cheat Sheet)

🛡️ Security Reminders

About

Uh oh!

Releases

Packages

Uh oh!

Uh oh!

Contributors

Uh oh!

Languages

Name		Name	Last commit message	Last commit date
Latest commit History 6 Commits
configs		configs
.env.example		.env.example
.gitignore		.gitignore
README.md		README.md
install.sh		install.sh

Folders and files

Latest commit

History

Repository files navigation

⚡ Ultimate Free LLM Stack

🚀 Features

📦 Installation

🎮 Complete Usage Guide

1️⃣ LiteLLM Proxy START Kaise Karein

✅ Successful Start ka Output aisa dikhega:

🌐 Start hone ke baad access karein:

🔑 Dashboard Login:

📋 Quick One-Liner (Copy-Paste Ready):

2️⃣ LiteLLM Proxy STOP Kaise Karein

3️⃣ Latest Version Per UPDATE Kaise Karein

Update ke baad Prisma Binary Fix (Zaroori Hai!):

Quick Update One-Liner:

4️⃣ Background Mein Run Kaise Karein (Optional)

5️⃣ Available Models (Config Mein)

🤖 How to use with Claude Code CLI

Setup Instructions

6️⃣ API Se Test Kaise Karein

cURL se test:

Health check:

Models list dekhein:

🔑 Where to get FREE API Keys

🌐 Endpoints & API Testing

7️⃣ Common Errors & Solutions

8️⃣ Important Files & Locations

9️⃣ Daily Routine (Cheat Sheet)

🛡️ Security Reminders

About

Topics

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Uh oh!

Contributors

Uh oh!

Languages

Packages