Provider Registry¶

liter-llm ships with a compiled-in registry of 142 providers. No runtime configuration files, no network fetches -- the registry is embedded in the binary at build time from schemas/providers.json.

Model Routing¶

Requests are routed to providers using a provider/model prefix convention:

The prefix is stripped before the request is forwarded to the provider's API.

How the Registry Works¶

Each entry in schemas/providers.json defines:

• Base URL -- the provider's API endpoint
• Auth format -- how the API key is sent (Bearer token, custom header, etc.)
• Model prefixes -- which prefixes route to this provider
• Parameter mappings -- provider-specific field name translations
• Supported capabilities -- chat, embeddings, images, audio, moderation

The registry is compiled into the binary with include_str! and deserialized once at startup. Provider resolution at client construction is a single hashmap lookup.

Auth Patterns¶

Providers use different authentication mechanisms. liter-llm handles all of them transparently:

API keys are wrapped in secrecy::SecretString and are never logged, serialized, or exposed in error messages.

Custom and Local Providers¶

Any OpenAI-compatible API can be used by overriding the base URL at client construction. This works for local inference servers and proxies.

Ollama¶

from liter_llm import LlmClient

client = LlmClient(api_key="unused", base_url="http://localhost:11434/v1")
response = await client.chat(
    model="ollama/llama3",
    messages=[{"role": "user", "content": "Hello!"}],
)

vLLM¶

client = LlmClient(api_key="unused", base_url="http://localhost:8000/v1")
response = await client.chat(
    model="hosted_vllm/meta-llama/Llama-3-8B-Instruct",
    messages=[{"role": "user", "content": "Hello!"}],
)

LM Studio¶

client = LlmClient(api_key="unused", base_url="http://localhost:1234/v1")
response = await client.chat(
    model="lm_studio/my-local-model",
    messages=[{"role": "user", "content": "Hello!"}],
)

Full Provider List¶

See the Providers page for the complete table of all 142 supported providers with their prefixes and capability matrix.