Configuration¶

The LlmClient constructor accepts configuration for authentication, timeouts, retries, and endpoint overrides. All options have sensible defaults.

Client Construction¶

import asyncio
from liter_llm import LlmClient

async def main() -> None:
    client = LlmClient(
        api_key="sk-...",          # or set OPENAI_API_KEY env var
        base_url=None,             # override provider base URL
        model_hint="openai",       # pre-resolve provider at construction
        max_retries=3,             # retry on transient failures
        timeout=60,                # request timeout in seconds
    )
    response = await client.chat(
        model="openai/gpt-4o",
        messages=[{"role": "user", "content": "Hello!"}],
    )
    print(response.choices[0].message.content)

asyncio.run(main())

import { LlmClient } from "@kreuzberg/liter-llm";

const client = new LlmClient({
  apiKey: "sk-...",           // or set OPENAI_API_KEY env var
  baseUrl: undefined,         // override provider base URL
  modelHint: "openai",        // pre-resolve provider at construction
  maxRetries: 3,              // retry on transient failures
  timeoutSecs: 60,            // request timeout in seconds
});

const response = await client.chat({
  model: "openai/gpt-4o",
  messages: [{ role: "user", content: "Hello!" }],
});
console.log(response.choices[0].message.content);

package main

import (
 "context"
 "fmt"
 "time"

 llm "github.com/kreuzberg-dev/liter-llm/packages/go"
)

func main() {
 client := llm.NewClient(
  llm.WithAPIKey("sk-..."),                  // or set OPENAI_API_KEY env var
  llm.WithBaseURL("https://api.openai.com/v1"), // override provider base URL
  llm.WithTimeout(60*time.Second),            // request timeout
 )
 resp, err := client.Chat(context.Background(), &llm.ChatCompletionRequest{
  Model: "openai/gpt-4o",
  Messages: []llm.Message{
   llm.NewTextMessage(llm.RoleUser, "Hello!"),
  },
 })
 if err != nil {
  panic(err)
 }
 if len(resp.Choices) > 0 && resp.Choices[0].Message.Content != nil {
  fmt.Println(*resp.Choices[0].Message.Content)
 }
}

Configuration Options¶

API Key Management¶

The recommended approach is environment variables. Read the standard variable for your provider and pass it to the constructor:

If you pass api_key explicitly, it takes precedence over the environment variable.

Timeouts and Retries¶

Retries use exponential backoff with jitter. Only transient failures (HTTP 429, 500, 502, 503, 504) are retried.

# Long timeout for large generations
client = LlmClient(api_key="sk-...", timeout=300, max_retries=5)

Custom Base URLs¶

Override base_url to point at a local inference server, corporate proxy, or alternative endpoint:

# Ollama running locally (no API key needed for local providers)
client = LlmClient(api_key="unused", base_url="http://localhost:11434/v1")

# Corporate proxy
client = LlmClient(api_key="sk-...", base_url="https://llm-proxy.internal.company.com/v1")

The base_url replaces the provider's default URL from the registry. The model prefix still controls which provider's auth and parameter logic is used.

Extra Headers¶

In Rust, the ClientConfigBuilder supports adding custom headers:

use liter_llm::ClientConfigBuilder;

let config = ClientConfigBuilder::new("sk-...")
    .header("X-Custom-Header", "value")?
    .timeout(std::time::Duration::from_secs(120))
    .max_retries(5)
    .build();

Credential Providers¶

For dynamic credentials (e.g. rotating tokens, AWS IAM roles), pass a credential_provider that implements the CredentialProvider trait. This is called before each request to fetch the current credential:

use liter_llm::ClientConfigBuilder;
use std::sync::Arc;

let config = ClientConfigBuilder::new("placeholder")
    .credential_provider(Arc::new(my_credential_provider))
    .build();

Model Hints¶

The model_hint parameter pre-resolves a provider at construction time. This is useful when you know you will only use one provider and want to avoid the prefix lookup on each request:

# All requests will use OpenAI, no prefix lookup needed
client = LlmClient(api_key="sk-...", model_hint="openai")
response = await client.chat(
    model="gpt-4o",  # no "openai/" prefix required
    messages=[{"role": "user", "content": "Hello!"}],
)