WebAssembly API Reference

WebAssembly API Reference v1.9.0-rc.1¶

Functions¶

createClient()¶

Create a new LLM client with simple scalar configuration.

This is the primary binding entry-point. All parameters except api_key are optional — omitting them uses the same defaults as ClientConfigBuilder.

Errors:

Returns LiterLlmError if the underlying HTTP client cannot be constructed, or if the resolved provider configuration is invalid.

Signature:

function createClient(apiKey: string, baseUrl?: string, timeoutSecs?: number, maxRetries?: number, modelHint?: string): DefaultClient

Example:

const result = createClient("value", "value", 42, 42, "value");

Parameters:

Returns: DefaultClient

Errors: Throws Error with a descriptive message.

createClientFromJson()¶

Create a new LLM client from a JSON string.

The JSON object accepts the same fields as liter-llm.toml (snake_case).

Errors:

Returns LiterLlmError.BadRequest if json is not valid JSON or contains unknown fields.

Signature:

function createClientFromJson(json: string): DefaultClient

Example:

const result = createClientFromJson("value");

Parameters:

Returns: DefaultClient

Errors: Throws Error with a descriptive message.

encodeDataUrl()¶

Encode bytes as a base64 data URL: data:<mime>;base64,<b64>.

mime defaults to IMAGE_PNG when null.

Signature:

function encodeDataUrl(bytes: Buffer, mime?: string): string

Example:

const result = encodeDataUrl(new Uint8Array([100, 97, 116, 97]), "value");

Parameters:

Returns: string

decodeDataUrl()¶

Decode a base64 data URL into DecodedDataUrl.

Returns null for:

• Non-data URLs (strings that do not start with "data:").
• Malformed prefixes (missing ";base64," marker).
• Invalid base64 payloads.

The returned MIME string is extracted verbatim from the URL prefix — it is not validated or normalised.

Signature:

function decodeDataUrl(url: string): DecodedDataUrl | null

Example:

const result = decodeDataUrl("value");

Parameters:

Returns: DecodedDataUrl | null

registerCustomProvider()¶

Register a custom provider in the global runtime registry.

The provider will be checked before all built-in providers during model detection. If a provider with the same name already exists it is replaced.

Errors:

Returns an error if the config is invalid (empty name, empty base_url, or no model prefixes).

Signature:

function registerCustomProvider(config: CustomProviderConfig): void

Example:

registerCustomProvider(new CustomProviderConfig());

Parameters:

Returns: No return value.

Errors: Throws Error with a descriptive message.

unregisterCustomProvider()¶

Remove a previously registered custom provider by name.

Returns true if a provider with the given name was found and removed, false if no such provider existed.

Errors:

Returns an error if the custom-provider registry cannot be updated.

Signature:

function unregisterCustomProvider(name: string): boolean

Example:

const result = unregisterCustomProvider("value");

Parameters:

Returns: boolean

Errors: Throws Error with a descriptive message.

capabilities()¶

Return the capability flags for a named provider.

Performs an O(n) linear scan over the embedded registry (143 entries). Returns an owned value so bindings can pass capability data without borrowing registry internals.

For unknown provider_name values the function returns an all-false sentinel so callers never need to handle Option.

Signature:

function capabilities(providerName: string): ProviderCapabilities

Example:

const result = capabilities("value");

Parameters:

Returns: ProviderCapabilities

allProviders()¶

Return all provider configs from the registry.

Useful for tooling, documentation generation, or runtime enumeration. Returns the public ProviderConfig slice (without capability flags). To query capability flags for a specific provider use capabilities.

Signature:

function allProviders(): Array<ProviderConfig>

Example:

const result = allProviders();

Returns: Array<ProviderConfig>

Errors: Throws Error with a descriptive message.

complexProviderNames()¶

Return the set of complex provider names.

Complex providers require custom auth/routing logic beyond simple bearer tokens (e.g. AWS Bedrock SigV4, Vertex AI OAuth2).

The returned reference points into the static registry — no allocation.

Signature:

function complexProviderNames(): Array<string>

Example:

const result = complexProviderNames();

Returns: Array<string>

Errors: Throws Error with a descriptive message.

completionCost()¶

Calculate the estimated cost of a completion given a model name and token counts.

Returns null if the model is not present in the embedded pricing registry. Returns Some(cost_usd) otherwise, where the value is in US dollars.

When an exact model name match is not found, progressively shorter prefixes are tried by stripping from the last - or . separator. For example, gpt-4-0613 will match gpt-4 if no gpt-4-0613 entry exists.

Signature:

function completionCost(model: string, promptTokens: number, completionTokens: number): number | null

Example:

const result = completionCost("value", 42, 42);

Parameters:

Returns: number | null

completionCostWithCache()¶

Calculate the estimated cost of a completion, accounting for cached (cache-hit) prompt tokens billed at the provider's discounted rate.

cached_tokens is the count of prompt tokens served from the provider's prompt cache. It must be <= prompt_tokens (cached tokens are a subset of the prompt). The non-cached portion is billed at input_cost_per_token and the cached portion at cache_read_input_token_cost when the model has cache pricing; otherwise the entire prompt is billed at the regular input rate.

Returns null if the model is not present in the embedded pricing registry, mirroring completion_cost.

Signature:

function completionCostWithCache(model: string, promptTokens: number, cachedTokens: number, completionTokens: number): number | null

Example:

const result = completionCostWithCache("value", 42, 42, 42);

Parameters:

Returns: number | null

clear()¶

Remove all guardrails from the global registry.

Primarily useful in tests to reset state between test cases.

Panics:

Panics if the global registry lock is poisoned.

Signature:

function clear(): void

Example:

clear();

Returns: No return value.

checkBound()¶

Assert that current_len + incoming does not exceed limit.

Call this before appending incoming bytes to any buffer that must stay below limit. Returns Err(LiterLlmError.Streaming) on overflow and emits a tracing.warn! with context.

Signature:

function checkBound(context: string, currentLen: number, incoming: number, limit: number): void

Example:

checkBound("value", 42, 42, 42);

Parameters:

Returns: No return value.

Errors: Throws Error with a descriptive message.

ensureCryptoProvider()¶

Install the ring crypto provider as the rustls process default, idempotently.

rustls 0.23+ removed the implicit default provider. This function installs ring once per process. Subsequent calls are no-ops. Calling it after another rustls crypto provider has already been installed is safe: the Err from install_default() is silently ignored.

Called automatically by every internal reqwest.Client constructor (auth providers, default HTTP client). Bindings and downstream consumers reach those constructors transitively, so no manual init is required.

WASM builds are exempt — the WASM target uses the browser/Node.js fetch API instead of rustls, so no crypto provider is needed.

Windows builds use native-tls (SChannel) via reqwest, so rustls is not present and no crypto provider installation is needed.

Signature:

function ensureCryptoProvider(): void

Example:

ensureCryptoProvider();

Returns: No return value.

ensureCryptoProvider()¶

No-op on Windows: reqwest uses native-tls (SChannel), so no rustls provider installation is needed. All callers use the same call site regardless of platform.

Signature:

function ensureCryptoProvider(): void

Example:

ensureCryptoProvider();

Returns: No return value.

Types¶

AssistantMessage¶

Assistant's response to a user message.

Methods¶

text()¶

Return the assistant's textual response, concatenating all Text parts if the content is structured.

Returns null for Refusal-only or OutputImage-only responses.

Signature:

text(): string | null

Example:

const result = instance.text();

Returns: string | null

refusalText()¶

Return the refusal message, if the model declined to respond.

Checks both the top-level refusal field and any Refusal parts inside a structured content.

Signature:

refusalText(): string | null

Example:

const result = instance.refusalText();

Returns: string | null

outputImages()¶

Return all AssistantPart.OutputImage parts in the response.

Signature:

outputImages(): Array<ImageUrl>

Example:

const result = instance.outputImages();

Returns: Array<ImageUrl>

outputAudio()¶

Return all AssistantPart.OutputAudio parts in the response.

Signature:

outputAudio(): Array<AudioContent>

Example:

const result = instance.outputAudio();

Returns: Array<AudioContent>

AudioContent¶

Audio content part for speech-capable models.

AuthConfig¶

Auth configuration block.

BatchListQuery¶

Query parameters for listing batches.

BatchListResponse¶

Response from listing batches.

BatchObject¶

A batch job object.

BatchRequestCounts¶

Request processing counts for a batch.

ChatCompletionChunk¶

A streamed chunk of a chat completion response.

ChatCompletionRequest¶

Chat completion request (compatible with OpenAI and similar APIs).

ChatCompletionResponse¶

Chat completion response from the API.

ChatCompletionTool¶

A tool the model can invoke (currently, all tools are functions).

Choice¶

A single completion choice.

ChunkMiddleware¶

A per-chunk transformation in the StreamPipeline.

Each middleware receives a typed chunk and returns Ok(Some(chunk)) to pass it through (optionally modified), Ok(None) to drop the chunk, or Err(e) to propagate a stream error.

The trait is object-safe so multiple middleware implementations can be chained inside StreamPipeline.

Methods¶

process()¶

Process a single chunk.

• Ok(Some(chunk)) — emit (possibly transformed) chunk.
• Ok(None) — drop this chunk silently.
• Err(e) — propagate as a stream error.

Signature:

process(chunk: ChatCompletionChunk): ChatCompletionChunk | null

Example:

const result = instance.process(new ChatCompletionChunk());

Parameters:

Returns: ChatCompletionChunk | null

Errors: Throws Error with a descriptive message.

CreateBatchRequest¶

Request to create a batch job.

CreateFileRequest¶

Request to upload a file.

CreateImageRequest¶

Request to create images from a text prompt.

CreateResponseRequest¶

Request to create a structured response.

CreateSpeechRequest¶

Request to generate speech audio from text.

CreateTranscriptionRequest¶

Request to transcribe audio into text.

CustomProviderConfig¶

Configuration for registering a custom LLM provider at runtime.

DecodedDataUrl¶

Result of decoding a data: URL — MIME type and the decoded byte payload.

Named struct (rather than a tuple) so polyglot bindings can extract decode_data_url with a typed return rather than a sanitized scalar.

DefaultClient¶

Default client implementation backed by reqwest.

Sends requests to 143 LLM providers with automatic provider detection and per-request routing. The provider is resolved at construction time from model_hint (or defaults to OpenAI), but individual requests can override the provider via model name prefix (e.g. "anthropic/claude-3-5-sonnet" routes to Anthropic regardless of construction-time setting).

When the model prefix does not match any known provider, the construction-time provider is used as the fallback. This enables seamless migration between providers by changing only the model name.

The provider is stored behind an Arc so it can be shared cheaply into async closures and streaming tasks. Pre-computed auth headers and extra headers are cached at construction to avoid redundant encoding on every request.

Methods¶

fetchBatchForPolling()¶

Signature:

fetchBatchForPolling(batchId: string): Promise<BatchObject>

Example:

const result = await instance.fetchBatchForPolling("value");

Parameters:

Returns: BatchObject

Errors: Throws Error with a descriptive message.

waitForBatch()¶

Poll a batch until it reaches a terminal status (Completed, Failed, Expired, Cancelled).

Uses exponential backoff with configurable initial interval, maximum interval, and backoff multiplier. Optionally supports a timeout that aborts polling if exceeded.

Errors:

Returns BatchWaitError.Failed if the batch reaches a failure terminal status. Returns BatchWaitError.Timeout if the configured timeout is exceeded. Returns BatchWaitError.Client for underlying client errors.

Signature:

waitForBatch(batchId: string, config: WaitForBatchConfig): Promise<BatchObject>

Example:

const result = await instance.waitForBatch("value", new WaitForBatchConfig());

Parameters:

Returns: BatchObject

Errors: Throws Error with a descriptive message.

DeleteResponse¶

Response from a delete operation.

DeveloperMessage¶

Developer message (system-like message for Claude models).

DocumentContent¶

PDF/document content part for vision-capable models.

EmbeddingObject¶

A single embedding vector.

EmbeddingRequest¶

Embedding request.

EmbeddingResponse¶

Embedding response.

FileListQuery¶

Query parameters for listing files.

FileListResponse¶

Response from listing files.

FileObject¶

An uploaded file object.

FunctionCall¶

Function call details.

FunctionDefinition¶

Function definition exposed to the model.

FunctionMessage¶

Deprecated legacy function-role message body.

HealthChecker¶

Abstraction over a health probe strategy.

Implementors issue a lightweight probe against upstream (typically a provider base URL or named identifier) and report HealthStatus.

Methods¶

check()¶

Probe upstream and return its current HealthStatus.

The parameter is taken by value (String) so that implementations can move it into the returned future without a clone, making the 'static + Send bound on the future trivially satisfiable.

Signature:

check(upstream: string): Promise<HealthStatus>

Example:

const result = await instance.check("value");

Parameters:

Returns: HealthStatus

Image¶

A single generated image, returned as either a URL or base64 data.

ImageUrl¶

An image URL reference with optional detail level for processing.

ImagesResponse¶

Response containing generated images.

JsonSchemaFormat¶

JSON Schema specification for constrained output.

ModelObject¶

A model available from the API.

ModelsListResponse¶

Response listing available models.

ModerationCategories¶

Boolean flags for each moderation category.

ModerationCategoryScores¶

Confidence scores for each moderation category.

ModerationRequest¶

Request to classify content for policy violations.

ModerationResponse¶

Response from the moderation endpoint.

ModerationResult¶

A single moderation classification result.

OcrImage¶

An image extracted from an OCR page.

OcrPage¶

A single page of OCR output.

OcrRequest¶

An OCR request.

OcrResponse¶

An OCR response.

PageDimensions¶

Page dimensions in pixels.

PromptTokensDetails¶

Breakdown of tokens used in the prompt portion of a request.

cached_tokens is included in Usage.prompt_tokens — it is not an additional charge on top of the prompt token count. When pricing supports a cache_read_input_token_cost, the cached portion is billed at the discounted rate and the remainder at the regular input rate.

ProviderCapabilities¶

Static capability flags for a provider.

Each flag indicates whether the provider's models generally support that feature. For providers that aggregate many underlying models (e.g. Bedrock, OpenRouter, vLLM) the flags reflect the superset of available model capabilities — a flag being true means at least one model supports the feature, not every model.

All flags default to false so that newly added providers are safe.

Access via the crate-level capabilities function:

ProviderConfig¶

Static configuration for a single provider entry in providers.json.

This struct deliberately does not include capability flags or streaming format, which are accessed via the capabilities function.

RerankRequest¶

Request to rerank documents by relevance to a query.

RerankResponse¶

Response from the rerank endpoint.

RerankResult¶

A single reranked document with its relevance score.

RerankResultDocument¶

The text content of a reranked document, returned when return_documents is true.

ResponseObject¶

Response from a structured response request.

ResponseOutputItem¶

A single output item from the response.

ResponseTool¶

A tool available for the response request.

ResponseUsage¶

Token usage for a response.

SearchRequest¶

A search request.

SearchResponse¶

A search response.

SearchResult¶

An individual search result.

SpecificFunction¶

Name of the specific function to invoke.

SpecificToolChoice¶

Directive to call a specific tool.

StreamChoice¶

A streaming choice with incremental delta.

StreamDelta¶

Incremental delta in a stream chunk.

StreamFunctionCall¶

Partial function call details in a stream.

StreamOptions¶

Options for streaming responses.

StreamToolCall¶

A streaming tool call being built incrementally.

SystemMessage¶

System message guiding model behavior for the entire conversation.

ToolCall¶

A tool call the model wants to execute.

ToolMessage¶

Tool execution result returned to the model.

TranscriptionResponse¶

Response from a transcription request.

TranscriptionSegment¶

A segment of transcribed audio with timing information.

Usage¶

Token-usage accounting returned by the provider on each completion / embedding call.

UserMessage¶

User message in the conversation.

WaitForBatchConfig¶

Configuration for polling a batch until terminal status.

All time values are in seconds as f64 so the struct bridges across FFI boundaries without requiring a Duration shim.

Methods¶

default()¶

Signature:

static default(): WaitForBatchConfig

Example:

const result = WaitForBatchConfig.default();

Returns: WaitForBatchConfig

Enums¶

Message¶

A chat message in a conversation.

UserContent¶

User message content as either plain text or a list of multimodal parts.

ContentPart¶

A single content part in a user message — text, image, document, or audio.

ImageDetail¶

Image detail level controlling token cost and processing.

AssistantContent¶

Content shape for assistant messages.

#[serde(untagged)] means providers returning a plain scalar string for the content field still deserialise correctly into AssistantContent.Text(_). Providers returning an array of typed parts (e.g. after an image-generation or audio-synthesis request) deserialise into AssistantContent.Parts(_).

AssistantPart¶

One part of a structured assistant response.

#[serde(tag = "type", rename_all = "snake_case")] matches OpenAI's parts-spec discriminator ("type": "text", "type": "output_image", …).

ToolType¶

The type discriminator for tool/tool-call objects.

Per the OpenAI spec this is always "function". Using an enum enforces that constraint at the type level and rejects any other value on deserialization.

ToolChoice¶

Tool usage mode or a specific tool to call.

ToolChoiceMode¶

Tool choice mode.

ResponseFormat¶

Wire format for the chat completions response_format field.

Provider mapping¶

• OpenAI (and OpenAI-compatible providers): emitted verbatim as {"type": "json_schema", "json_schema": {...}} per the chat-completions spec.

• Gemini / Vertex AI: translated to generationConfig.responseMimeType = "application/json" and generationConfig.responseSchema = <schema>. The name, description, and strict fields are dropped — Gemini's structured-output API does not consume them.

• Anthropic: no native JSON mode. A system instruction is prepended asking the model to respond with valid JSON. strict is advisory only; callers should still validate the returned JSON if the schema is load-bearing.

StopSequence¶

Stop sequence(s) that cause the model to stop generating.

Modality¶

Output modality requested from the model.

Passed as modalities: ["text", "audio"] (OpenAI) or translated to generationConfig.responseModalities (Gemini / Vertex AI).

FinishReason¶

Why a choice stopped generating tokens.

ReasoningEffort¶

Controls how much reasoning effort the model should use.

EmbeddingFormat¶

The format in which the embedding vectors are returned.

EmbeddingInput¶

Text or texts to embed.

ModerationInput¶

Input to the moderation endpoint — a single string or multiple strings.

RerankDocument¶

A document to be reranked — either a plain string or an object with a text field.

OcrDocument¶

Document input for OCR — either a URL or inline base64 data.

FilePurpose¶

Purpose of an uploaded file.

BatchStatus¶

Status of a batch job.

AuthHeaderFormat¶

How the API key is sent in the HTTP request.

StreamFormat¶

The streaming wire format a provider uses for its response stream.

Most providers use standard Server-Sent Events (SSE). AWS Bedrock uses a proprietary binary EventStream framing.

Deserialized from the streaming_format JSON field via serde.

AuthType¶

Auth scheme used by a provider.

Errors¶

LiterLlmError¶

All errors that can occur when using liter-llm.

Errors are thrown as plain Error objects with descriptive messages.

Edit this page on GitHub