Skip to content

TypeScript / Node.js API Reference

Installation

pnpm add @kreuzberg/liter-llm
# or
npm install @kreuzberg/liter-llm

Client

Constructor

import { LlmClient } from 'liter-llm';

const client = new LlmClient({
  apiKey: string,
  baseUrl?: string,
  modelHint?: string,
  maxRetries?: number,     // default: 3
  timeoutSecs?: number,    // default: 60
});
Parameter Type Default Description
apiKey string required API key for authentication
baseUrl string \| undefined undefined Override provider base URL
modelHint string \| undefined undefined Hint for provider auto-detection (e.g. "groq/llama3-70b")
maxRetries number \| undefined 3 Retries on 429 / 5xx responses
timeoutSecs number \| undefined 60 Request timeout in seconds

Methods

All methods are async and return Promises. Request and response objects use camelCase keys (converted automatically from the snake_case wire format).

chat(request)

Send a chat completion request.

async chat(request: object): Promise<object>

const resp = await client.chat({
  model: "gpt-4",
  messages: [{ role: "user", content: "Hi" }],
});
console.log(resp.choices[0].message.content);

chatStream(request)

Collect all streaming chat completion chunks into an array. The full SSE stream is consumed on the Rust side before the Promise resolves.

async chatStream(request: object): Promise<object[]>

const chunks = await client.chatStream({
  model: "gpt-4",
  messages: [{ role: "user", content: "Hi" }],
});
for (const chunk of chunks) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}

embed(request)

Send an embedding request.

async embed(request: object): Promise<object>

listModels()

List available models from the provider.

async listModels(): Promise<object>

imageGenerate(request)

Generate an image from a text prompt.

async imageGenerate(request: object): Promise<object>

speech(request)

Generate speech audio from text. Returns a Buffer of raw audio bytes.

async speech(request: object): Promise<Buffer>

transcribe(request)

Transcribe audio to text.

async transcribe(request: object): Promise<object>

moderate(request)

Check content against moderation policies.

async moderate(request: object): Promise<object>

rerank(request)

Rerank documents by relevance to a query.

async rerank(request: object): Promise<object>

createFile(request)

Upload a file.

async createFile(request: object): Promise<object>

retrieveFile(fileId)

Retrieve metadata for a file by ID.

async retrieveFile(fileId: string): Promise<object>

deleteFile(fileId)

Delete a file by ID.

async deleteFile(fileId: string): Promise<object>

listFiles(query?)

List files, optionally filtered.

async listFiles(query?: object | null): Promise<object>

fileContent(fileId)

Retrieve the raw content of a file. Returns a Buffer.

async fileContent(fileId: string): Promise<Buffer>

createBatch(request)

Create a new batch job.

async createBatch(request: object): Promise<object>

retrieveBatch(batchId)

Retrieve a batch by ID.

async retrieveBatch(batchId: string): Promise<object>

listBatches(query?)

List batches, optionally filtered.

async listBatches(query?: object | null): Promise<object>

cancelBatch(batchId)

Cancel an in-progress batch.

async cancelBatch(batchId: string): Promise<object>

createResponse(request)

Create a new response via the Responses API.

async createResponse(request: object): Promise<object>

retrieveResponse(id)

Retrieve a response by ID.

async retrieveResponse(id: string): Promise<object>

cancelResponse(id)

Cancel an in-progress response.

async cancelResponse(id: string): Promise<object>

Module Functions

version()

Returns the library version string.

import { version } from 'liter-llm';
console.log(version());

Types

Response objects are plain JavaScript objects with camelCase keys.

ChatCompletionResponse

Field Type Description
id string Response ID
model string Model used
choices Choice[] Completion choices
usage Usage \| undefined Token usage
created number Unix timestamp

ChatCompletionChunk

Field Type Description
id string Response ID
model string Model used
choices StreamChoice[] Stream choices with deltas
usage Usage \| undefined Token usage (final chunk only)

Error Handling

Errors are thrown as JavaScript Error objects. The message includes a bracketed label for the error category:

try {
  await client.chat({ model: "gpt-4", messages: [] });
} catch (err) {
  // "[Authentication] Invalid API key"
  // "[RateLimited] Too many requests"
  // "[BadRequest] Messages must not be empty"
  console.error(err.message);
}

Error categories: Authentication, RateLimited, BadRequest, ContextWindowExceeded, ContentPolicy, NotFound, ServerError, ServiceUnavailable, Timeout, Network, Streaming, EndpointNotSupported, InvalidHeader, Serialization.

Example

import { LlmClient } from 'liter-llm';

const client = new LlmClient({
  apiKey: process.env.OPENAI_API_KEY!,
});

const resp = await client.chat({
  model: "gpt-4",
  messages: [{ role: "user", content: "Hello!" }],
  maxTokens: 256,
});
console.log(resp.choices[0].message.content);