Skip to content

liter-llm

liter-llm is an LLM API client written in Rust with native bindings for Python, TypeScript, Go, Java, C#, Ruby, PHP, Elixir, WebAssembly, C, and Rust. One API surface across 143 providers. No Python runtime, no dependency chain surprises. It ships as a compiled binary with Tower middleware, an OpenAI-compatible proxy server, and an MCP server built in.

Quick Start Installation GitHub Community

Explore the Docs

  • Getting Started

Install the package for your language and make your first API call.

Installation

  • Chat & Streaming

Single-turn and multi-turn chat, streaming, tool calling, structured outputs.

Chat Guide

  • Proxy Server

OpenAI-compatible proxy with virtual keys, budget enforcement, and TOML config.

Proxy Server

  • Fallback & Routing

Round-robin, latency-based, cost-based, weighted-random, and ordered-fallback strategies.

Routing Guide

  • Authentication

Azure AD, AWS Bedrock STS/IRSA, Vertex AI OAuth2 with automatic token caching.

Auth Guide

  • API Reference

Full reference for Python, TypeScript, Rust, Go, Java, C#, Ruby, Elixir, PHP, WASM, C FFI.

Python

Part of kreuzberg.dev

liter-llm is built by the kreuzberg.dev team, the same people behind a family of Rust-core, polyglot-bindings libraries.

Kreuzberg Document extraction for 91+ formats — PDF, Office, images, HTML, and more.

tree-sitter-language-pack All Tree-sitter grammars in one package, across every language binding.

html-to-markdown Fast, lossless HTML to Markdown conversion with a Rust core.

Getting Help