TypeScript API Reference

TypeScript API Reference v1.4.0-rc.27¶

Functions¶

createClient()¶

Create a new LLM client with simple scalar configuration.

This is the primary binding entry-point. All parameters except api_key are optional — omitting them uses the same defaults as ClientConfigBuilder.

Errors:

Returns LiterLlmError if the underlying HTTP client cannot be constructed, or if the resolved provider configuration is invalid.

Signature:

function createClient(
  apiKey: string,
  baseUrl?: string,
  timeoutSecs?: number,
  maxRetries?: number,
  modelHint?: string,
): DefaultClient;

Parameters:

Returns: DefaultClient

Errors: Throws Error with a descriptive message.

createClientFromJson()¶

Create a new LLM client from a JSON string.

The JSON object accepts the same fields as liter-llm.toml (snake_case).

Errors:

Returns LiterLlmError.BadRequest if json is not valid JSON or contains unknown fields.

Signature:

function createClientFromJson(json: string): DefaultClient;

Parameters:

Returns: DefaultClient

Errors: Throws Error with a descriptive message.

registerCustomProvider()¶

Register a custom provider in the global runtime registry.

The provider will be checked before all built-in providers during model detection. If a provider with the same name already exists it is replaced.

Errors:

Returns an error if the config is invalid (empty name, empty base_url, or no model prefixes).

Signature:

function registerCustomProvider(config: CustomProviderConfig): void;

Parameters:

Returns: void

Errors: Throws Error with a descriptive message.

unregisterCustomProvider()¶

Remove a previously registered custom provider by name.

Returns true if a provider with the given name was found and removed, false if no such provider existed.

Errors:

Returns an error only if the internal lock is poisoned.

Signature:

function unregisterCustomProvider(name: string): boolean;

Parameters:

Returns: boolean

Errors: Throws Error with a descriptive message.

Types¶

AssistantMessage¶

AudioContent¶

ChatCompletionChunk¶

ChatCompletionRequest¶

ChatCompletionResponse¶

ChatCompletionTool¶

Choice¶

CreateImageRequest¶

Request to create images from a text prompt.

CreateSpeechRequest¶

Request to generate speech audio from text.

CreateTranscriptionRequest¶

Request to transcribe audio into text.

CustomProviderConfig¶

Configuration for registering a custom LLM provider at runtime.

DefaultClient¶

Default client implementation backed by reqwest.

The provider is resolved at construction time from model_hint (or defaults to OpenAI). However, individual requests can override the provider when their model string contains a prefix that clearly identifies a different provider (e.g. "anthropic/claude-3" will route to Anthropic even if the client was built without a hint).

When the model prefix does not match any known provider, the construction-time provider is used as the fallback.

The provider is stored behind an Arc so it can be shared cheaply into async closures and streaming tasks that must be 'static.

Methods¶

chat()¶

Signature:

chat(req: ChatCompletionRequest): ChatCompletionResponse

chatStream()¶

Signature:

chatStream(req: ChatCompletionRequest): string

embed()¶

Signature:

embed(req: EmbeddingRequest): EmbeddingResponse

listModels()¶

Signature:

listModels(): ModelsListResponse

imageGenerate()¶

Signature:

imageGenerate(req: CreateImageRequest): ImagesResponse

speech()¶

Signature:

speech(req: CreateSpeechRequest): Buffer

transcribe()¶

Signature:

transcribe(req: CreateTranscriptionRequest): TranscriptionResponse

moderate()¶

Signature:

moderate(req: ModerationRequest): ModerationResponse

rerank()¶

Signature:

rerank(req: RerankRequest): RerankResponse

search()¶

Signature:

search(req: SearchRequest): SearchResponse

ocr()¶

Signature:

ocr(req: OcrRequest): OcrResponse

createFile()¶

Signature:

createFile(req: string): string

retrieveFile()¶

Signature:

retrieveFile(fileId: string): string

deleteFile()¶

Signature:

deleteFile(fileId: string): string

listFiles()¶

Signature:

listFiles(query: string): string

fileContent()¶

Signature:

fileContent(fileId: string): Buffer

createBatch()¶

Signature:

createBatch(req: string): string

retrieveBatch()¶

Signature:

retrieveBatch(batchId: string): string

listBatches()¶

Signature:

listBatches(query: string): string

cancelBatch()¶

Signature:

cancelBatch(batchId: string): string

createResponse()¶

Signature:

createResponse(req: string): string

retrieveResponse()¶

Signature:

retrieveResponse(id: string): string

cancelResponse()¶

Signature:

cancelResponse(id: string): string

DeveloperMessage¶

DocumentContent¶

EmbeddingObject¶

EmbeddingRequest¶

EmbeddingResponse¶

FunctionCall¶

FunctionDefinition¶

FunctionMessage¶

Deprecated legacy function-role message body.

Image¶

A single generated image, returned as either a URL or base64 data.

ImageUrl¶

ImagesResponse¶

Response containing generated images.

JsonSchemaFormat¶

ModelObject¶

ModelsListResponse¶

ModerationCategories¶

Boolean flags for each moderation category.

ModerationCategoryScores¶

Confidence scores for each moderation category.

ModerationRequest¶

Request to classify content for policy violations.

ModerationResponse¶

Response from the moderation endpoint.

ModerationResult¶

A single moderation classification result.

OcrImage¶

An image extracted from an OCR page.

OcrPage¶

A single page of OCR output.

OcrRequest¶

An OCR request.

OcrResponse¶

An OCR response.

PageDimensions¶

Page dimensions in pixels.

PromptTokensDetails¶

Breakdown of tokens used in the prompt portion of a request.

cached_tokens is included in Usage.prompt_tokens — it is not an additional charge on top of the prompt token count. When pricing supports a cache_read_input_token_cost, the cached portion is billed at the discounted rate and the remainder at the regular input rate.

RerankRequest¶

Request to rerank documents by relevance to a query.

RerankResponse¶

Response from the rerank endpoint.

RerankResult¶

A single reranked document with its relevance score.

RerankResultDocument¶

The text content of a reranked document, returned when return_documents is true.

SearchRequest¶

A search request.

SearchResponse¶

A search response.

SearchResult¶

An individual search result.

SpecificFunction¶

SpecificToolChoice¶

StreamChoice¶

StreamDelta¶

StreamFunctionCall¶

StreamOptions¶

StreamToolCall¶

SystemMessage¶

ToolCall¶

ToolMessage¶

TranscriptionResponse¶

Response from a transcription request.

TranscriptionSegment¶

A segment of transcribed audio with timing information.

Usage¶

UserMessage¶

Enums¶

Message¶

A chat message in a conversation.

UserContent¶

ContentPart¶

ImageDetail¶

ToolType¶

The type discriminator for tool/tool-call objects.

Per the OpenAI spec this is always "function". Using an enum enforces that constraint at the type level and rejects any other value on deserialization.

ToolChoice¶

ToolChoiceMode¶

ResponseFormat¶

StopSequence¶

FinishReason¶

Why a choice stopped generating tokens.

ReasoningEffort¶

Controls how much reasoning effort the model should use.

EmbeddingFormat¶

The format in which the embedding vectors are returned.

EmbeddingInput¶

ModerationInput¶

Input to the moderation endpoint — a single string or multiple strings.

RerankDocument¶

A document to be reranked — either a plain string or an object with a text field.

OcrDocument¶

Document input for OCR — either a URL or inline base64 data.

AuthHeaderFormat¶

How the API key is sent in the HTTP request.

Errors¶

LiterLlmError¶

All errors that can occur when using liter-llm.

Errors are thrown as plain Error objects with descriptive messages.