MCP Server¶

The liter-llm binary can run as a Model Context Protocol (MCP) server. It exposes 22 tools backed by the same ProxyConfig used by the HTTP proxy, so every provider, virtual key, fallback, and cache layer that works for the REST API works for MCP clients too.

Launch it with liter-llm mcp. The server supports two transports: stdio for local clients like Claude Desktop and Cursor, and http for network-attached clients using the Streamable HTTP transport.

Quick start¶

Run the server over stdio against an auto-discovered liter-llm-proxy.toml:

liter-llm mcp

Run over HTTP on the default port 3001:

liter-llm mcp --transport http --host 127.0.0.1 --port 3001

The HTTP transport exposes a single endpoint: POST /mcp. Point any MCP HTTP client at http://127.0.0.1:3001/mcp.

Command-line flags¶

The MCP server loads the same liter-llm-proxy.toml as the HTTP proxy. See Proxy Configuration for the full schema. Any [[models]], [[aliases]], [[keys]], [cache], [files], or [health] table defined there applies to MCP requests as well.

Tools¶

Every tool returns a JSON payload as a single text content part. Errors are propagated as MCP error objects with the liter-llm error type embedded in the message.

LLM operations¶

Media¶

Classification and retrieval¶

Files¶

Batches¶

Responses API¶

The full parameter schema for every tool is defined in crates/liter-llm-proxy/src/mcp/params.rs and surfaced to MCP clients as JSON Schema through rmcp.

Claude Desktop¶

Add an entry to your claude_desktop_config.json:

{
  "mcpServers": {
    "liter-llm": {
      "command": "liter-llm",
      "args": ["mcp"],
      "env": {
        "OPENAI_API_KEY": "sk-...",
        "ANTHROPIC_API_KEY": "sk-ant-..."
      }
    }
  }
}

Restart Claude Desktop. The 22 tools appear under the liter-llm server. Point liter-llm at a config file with --config /absolute/path/to/liter-llm-proxy.toml if you want virtual keys or a custom model list.

Cursor¶

Cursor reads MCP servers from ~/.cursor/mcp.json (or the workspace equivalent). Use the same shape as Claude Desktop:

{
  "mcpServers": {
    "liter-llm": {
      "command": "liter-llm",
      "args": ["mcp", "--config", "/absolute/path/to/liter-llm-proxy.toml"]
    }
  }
}

{
  "mcpServers": {
    "liter-llm": {
      "command": "liter-llm",
      "args": ["mcp", "--config", "/absolute/path/to/liter-llm-proxy.toml"],
      "env": {
        "OPENAI_API_KEY": "sk-...",
        "ANTHROPIC_API_KEY": "sk-ant-..."
      }
    }
  }
}

HTTP transport¶

Run the server in HTTP mode when the client is on a different machine or when you want to share one MCP server across several users. Pair it with a reverse proxy for TLS.

liter-llm mcp --transport http --host 0.0.0.0 --port 3001

liter-llm mcp \
  --transport http \
  --host 127.0.0.1 \
  --port 3001 \
  --config ./liter-llm-proxy.toml

# List the 22 tools exposed by the server.
curl -s http://127.0.0.1:3001/mcp \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'

The HTTP endpoint is POST /mcp. Each request opens a short-lived session managed by rmcp's LocalSessionManager. There is no authentication on the MCP HTTP transport itself, so bind to loopback or put it behind an authenticated reverse proxy.

Unlike the REST proxy, liter-llm mcp --transport http does not check Bearer tokens. Do not expose it to the public internet without a reverse proxy that handles authentication.

Shared configuration¶

The MCP server and the HTTP proxy use the same ProxyConfig loader. That means:

• Models defined in [[models]] are callable as chat, embed, generate_image, and so on.
• Glob overrides in [[aliases]] apply to MCP requests.
• [cache] caches non-streaming responses across both surfaces.
• [files] persists files uploaded via the create_file tool.
• [[keys]] virtual keys are loaded but not enforced on MCP calls, since the transports are assumed trusted.

The master key is also loaded, but the MCP surface does not send Bearer tokens, so virtual-key RPM, TPM, and budget caps do not apply to MCP invocations today. Use [rate_limit] and [budget] for global caps that cover both surfaces.

Troubleshooting¶

• "tool call failed: model 'foo' not found": the model parameter passed to the tool does not match any name in [[models]]. Check liter-llm-proxy.toml and restart.
• stdio transport hangs on startup: the client expects a JSON-RPC handshake on stdin. Make sure you are launching liter-llm mcp from an MCP client, not an interactive shell.
• HTTP transport returns 404: the endpoint is /mcp, not /. Every request is POST /mcp.
• Image or audio tools return empty content: the underlying provider may not support the feature. Check Providers for per-provider capability.