Installation¶

Liter-llm ships prebuilt packages for every supported language plus a CLI and Docker image for the proxy + MCP server. Pick your stack, run one command, and start calling models.

Prebuilt binaries cover Linux (x86_64 / aarch64), macOS (Apple Silicon), and Windows. The Rust toolchain is only needed when building from source.

CLI / Docker¶

The CLI runs the proxy server and the MCP tool server. You don't need it if you only use a language binding.

brew trust xberg-io/tap
brew install xberg-io/tap/liter-llm

cargo install liter-llm-cli

docker pull ghcr.io/xberg-io/liter-llm:latest
docker run -p 4000:4000 \
  -e LITER_LLM_MASTER_KEY=sk-your-key \
  ghcr.io/xberg-io/liter-llm

Start the proxy:

liter-llm api --config liter-llm-proxy.toml

Or the MCP server:

liter-llm mcp --transport stdio

Proxy Server docs   MCP Server docs

Choose your language¶

pip install liter-llm

uv add liter-llm

pnpm add @xberg-io/liter-llm

npm install @xberg-io/liter-llm
yarn add @xberg-io/liter-llm

cargo add liter-llm

go get github.com/xberg-io/liter-llm/packages/go

<dependency>
    <groupId>io.xberg.literllm</groupId>
    <artifactId>liter-llm</artifactId>
    <version>1.9.0-rc.1</version>
</dependency>

implementation("io.xberg.literllm:liter-llm:1.9.0-rc.1")

implementation("io.xberg:liter-llm-android:1.9.0-rc.1")

dotnet add package LiterLlm

gem install liter_llm

gem "liter_llm"

composer require xberg-io/liter-llm

defp deps do
  [
    {:liter_llm, "~> 1.7"}
  ]
end

dart pub add liter_llm

.binaryTarget(
    name: "LiterLlm",
    url: "https://github.com/xberg-io/liter-llm/releases/download/v1.9.0-rc.1/LiterLlm-rs.artifactbundle.zip",
    checksum: "<CHECKSUM-FROM-RELEASE-NOTES>"
)

zig fetch --save https://github.com/xberg-io/liter-llm/archive/refs/tags/v1.9.0-rc.1.tar.gz

pnpm add @xberg-io/liter-llm-wasm

git clone https://github.com/xberg-io/liter-llm.git
cd liter-llm
cargo build --release -p liter-llm-ffi

API key setup¶

Set the environment variable matching the provider you call:

export OPENAI_API_KEY="sk-..."
export ANTHROPIC_API_KEY="sk-ant-..."
export GOOGLE_API_KEY="..."
export GROQ_API_KEY="gsk_..."
export MISTRAL_API_KEY="..."
export AWS_ACCESS_KEY_ID="..."
export AWS_SECRET_ACCESS_KEY="..."

liter-llm resolves the provider from the model prefix (openai/gpt-4o, anthropic/claude-...) and picks the matching environment variable automatically.

Or pass the key at client construction:

from liter_llm import create_client

client = create_client(api_key="sk-...")

import { createClient } from "@xberg-io/liter-llm";

const client = createClient("sk-...");

use liter_llm::{ClientConfigBuilder, DefaultClient};

let config = ClientConfigBuilder::new("sk-...").build();
let client = DefaultClient::new(config, None)?;

Use environment variables or a secret manager. The Rust core wraps keys in secrecy::SecretString so they never appear in Debug output or logs.

Verify it works¶

python -c "from liter_llm import create_client; print('ok')"

node -e "import('@xberg-io/liter-llm').then(m => { m.createClient('test'); console.log('ok') })"

cargo build

go build ./...

Building from source¶

If a prebuilt binary is not available for your platform, build from source with stable Rust (edition 2024):

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
git clone https://github.com/xberg-io/liter-llm.git
cd liter-llm
task build

Next steps¶

• Chat & Streaming — make your first API call
• MCP & IDE Integration — wire liter-llm into VS Code, Claude Desktop, Cursor, Zed
• Provider Registry — browse all 143 runtime providers
• Configuration — timeouts, retries, base-URL overrides, TOML config

Edit this page on GitHub