We recommend using uv:
makesource .venv/bin/activate
Ensure all checks pass:
pre-commit run --all-files --verbose
make install
Ensure all tests pass: pytest -v
- Use
litellm_docker_compose.yamlto start LiteLLM and Postgres locally:
docker compose -f litellm_docker_compose.yaml up -dor if you are using legacy docker-compose:
docker-compose -f litellm_docker_compose.yaml up -d- Create a second database that is needed for authentication, and apply migrations
bash scripts/create-app-attest-database.shuv run alembic upgrade headLiteLLM will be accessible at localhost:4000 and localhost:4000/ui.
- Run MLPA with
mlpa- Stop the service with
docker compose -f litellm_docker_compose.yaml down- MLPA liveness:
curl --location 'http://0.0.0.0:8080/health/liveness' \
--header 'Content-Type: application/json'- MLPA readiness:
curl --location 'http://0.0.0.0:8080/health/readiness' \
--header 'Content-Type: application/json'- MLPA completion:
curl --location 'http://0.0.0.0:8080/v1/chat/completions' \
--header 'Content-Type: application/json' \
--header 'authorization: Bearer {YOUR_MOZILLA_FXA_TOKEN}' \
--header 'X-LiteLLM-Key: Bearer {MASTER_KEY}' \
--data '{
"model": "openai/gpt-4o",
"messages": [{
"role": "user",
"content": "Hello!"
}]
}'- LiteLLM liveness:
curl --location 'http://localhost:4000/health/liveness' \
--header 'Content-Type: application/json'- List of available models:
curl --location 'http://localhost:4000/models' \
--header 'Content-Type: application/json' \
--header 'X-LiteLLM-Key: Bearer {MASTER_KEY}' \
--data ''- Completion directly from LiteLLM:
curl --location 'http://localhost:4000/v1/chat/completions' \
--header 'Content-Type: application/json' \
--header 'X-LiteLLM-Key: Bearer {MASTER_KEY}' \
--data '{
"model": "openai/gpt-4o",
"messages": [
{
"role": "user",
"content": "what is 2+2?"
}
]
}'MLPA uses the https://github.com/mozilla/PyFxA library for authentication with a Mozilla account. Please follow the quick-start instructions in their README.
Here is a quick snippet:
from fxa.tools.bearer import get_bearer_token
fxa_token: str = get_bearer_token(
your_mozilla_account_email,
your_mozilla_account_password,
scopes=["profile:uid"],
client_id="5882386c6d801776" # a common client_id for the dev environment,
account_server_url="https://api.accounts.firefox.com",
oauth_server_url="https://oauth.accounts.firefox.com",
)Ensure Node is installed.
make docsAlternatively, with a running server:
make installmlpacurl http://localhost:8080/openapi.json -o /tmp/openapi.jsonnpx @redocly/cli build-docs /tmp/openapi.json -o docs/index.html