Skip to content
Xberg.io

liter-llm

Liter-llm

A universal LLM API client with a Rust core and 14 native language bindings (plus a shared C/FFI surface). One surface across 143 runtime providers — chat, streaming, embeddings, rerank, image generation, speech, transcription, OCR, search, files, batches, moderation — plus an OpenAI-compatible proxy server and a Model Context Protocol server, both shipped in the same binary.

Quick Start Installation Providers Join our Community

Why liter-llm

  • 143 Providers

OpenAI, Anthropic, Google, Bedrock, Vertex, Azure, Mistral, Cohere, GitHub Copilot, and many more — one client, one model-prefix routing scheme.

  • 14 Native Bindings

Rust, Python, TypeScript, Go, Java, Kotlin Android, C#, Ruby, PHP, Elixir, Dart, Swift, Zig, and WebAssembly — plus a shared C/FFI surface.

  • Full Endpoint Coverage

LlmClient covers chat, chat streaming, embeddings, model listing, images, speech, transcription, moderation, rerank, search, and OCR. Files, batches, and responses use dedicated clients.

  • Proxy & MCP Server

Drop-in OpenAI-compatible proxy with virtual keys, budgets, fallbacks, and observability. Same binary exposes a Model Context Protocol server for AI agents.

  • Cloud-Native Auth

Azure AD, AWS Bedrock SigV4 with STS/IRSA, Vertex AI service-account OAuth2, GitHub Copilot — automatic token caching, refresh, and rotation.

  • Routing & Fallback

Round-robin, weighted, latency-based, cost-based, and ordered-fallback strategies. Per-request override or proxy-level config.

Language Support

Language Package Docs
Rust cargo add liter-llm API Reference
Python pip install liter-llm API Reference
TypeScript / Node npm install @xberg-io/liter-llm API Reference
WebAssembly npm install @xberg-io/liter-llm-wasm API Reference
Go go get github.com/xberg-io/liter-llm/packages/go API Reference
Java Maven Central io.xberg.literllm:liter-llm API Reference
Kotlin Android Maven io.xberg:liter-llm-android API Reference
C# dotnet add package LiterLlm API Reference
Ruby gem install liter_llm API Reference
PHP composer require xberg-io/liter-llm API Reference
Elixir {:liter_llm, "~> 1.7"} API Reference
Dart / Flutter dart pub add liter_llm API Reference
Swift Swift Package Manager API Reference
Zig zig fetch --save from GitHub API Reference
C (FFI) Shared library + header API Reference
CLI cargo install liter-llm-cli Proxy Server
Docker ghcr.io/xberg-io/liter-llm Proxy Server

Feature Support

All bindings are generated from one Rust core, so endpoint coverage is uniform; the only real difference is the call idiom (async/await vs. sync vs. an explicit _async suffix). The C/FFI column is the shared native surface the C-ABI bindings (Go, Java, Kotlin, C#, Dart, Swift, Zig) are built on.

Feature Rust Python TS / Node Go Java Kotlin C# Ruby PHP Elixir Dart Swift Zig WASM C/FFI
Chat ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
Streaming ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
Tool calling ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
Embeddings ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
Vision input ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
Structured output ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
Audio output ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
Image output ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
Call idiom .await await await sync sync sync await _async sync _async await async sync await sync

Quick Example

src/main.rs
use liter_llm::{ChatCompletionRequest, ClientConfigBuilder, DefaultClient, LlmClient, Message};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = ClientConfigBuilder::new(std::env::var("OPENAI_API_KEY")?).build();
    let client = DefaultClient::new(config, None)?;

    let request = ChatCompletionRequest::builder("openai/gpt-4o-mini")
        .add_user_message("Summarize liter-llm in one sentence.")
        .build()?;

    let response = client.chat(request).await?;
    println!("{}", response.choices[0].message.content_text().unwrap_or(""));
    Ok(())
}
main.py
import asyncio
import os
from liter_llm import create_client

async def main():
    client = create_client(api_key=os.environ["OPENAI_API_KEY"])
    response = await client.chat({
        "model": "openai/gpt-4o-mini",
        "messages": [{"role": "user", "content": "Summarize liter-llm in one sentence."}],
    })
    print(response["choices"][0]["message"]["content"])

asyncio.run(main())
index.ts
import { createClient } from "@xberg-io/liter-llm";

const client = createClient(process.env.OPENAI_API_KEY!);

const response = await client.chat({
  model: "openai/gpt-4o-mini",
  messages: [{ role: "user", content: "Summarize liter-llm in one sentence." }],
});

console.log(response.choices[0].message.content);

Part of Xberg.io

Document intelligence — text, tables, and metadata from 96 file formats with optional OCR.

Managed document-extraction API with SDKs, dashboards, and observability built in.

High-performance web crawling and scraping with always-on HTML→Markdown and headless-Chrome fallback.

Fast, lossless HTML→Markdown engine — Rust core, the same conversion used by Crawlberg.

306 tree-sitter grammars and code-intelligence primitives.

Join the Xberg community for help, roadmap discussion, and announcements.

Explore the Docs

  • Get Started

Install liter-llm for your language, set an API key, and make your first call.

Installation

  • Guides

Chat, embeddings, media, search, fallback routing, authentication, and the proxy/MCP servers.

Chat & Streaming

  • Concepts

Architecture, feature flags, tokenizer model, and cost-estimation pipeline.

Architecture

  • Reference

Per-language API docs, the configuration schema, type catalogue, and error matrix.

References

  • Providers

Browse all 143 runtime providers, model prefixes, auth modes, and endpoint coverage.

Provider Registry

  • Proxy & MCP

Run the OpenAI-compatible proxy and the Model Context Protocol server from one binary.

Proxy Server

Getting Help

Edit this page on GitHub

Xberg.io