Skip to content

MCP & IDE Integration

The liter-llm MCP server exposes 22 tools for unified access to 143 runtime LLM providers, embeddings, files, batches, and more. Integrate it with VS Code, GitHub Copilot, Claude Desktop, Cursor, and other MCP-compatible IDEs and applications.

What is MCP?

The Model Context Protocol (MCP) is an open standard for connecting AI applications to context sources. The liter-llm MCP server provides tools that allow AI assistants to call LLM APIs, generate embeddings, search documents, and manage files — all through a unified, provider-agnostic interface.

Instead of hard-coding integrations for each provider (OpenAI, Anthropic, Google, Groq, etc.), MCP lets your IDE or application call any of the 143 runtime providers via a single interface.

Available MCP Tools

The liter-llm MCP server exposes 22 tools for core LLM operations:

Virtual keys can use the model-routed tools. File, batch, and response-management tools require a master key because they manage provider-side resources outside a single model route.

Tool Description
chat Send a chat completion request to any LLM provider
embed Generate text embeddings from configured embedding models
list_models List available models from all configured providers
generate_image Generate images from a text prompt (DALL-E, Flux, etc.)
speech Generate speech audio from text (text-to-speech)
transcribe Transcribe audio to text (speech-to-text)
moderate Check content against moderation policies
rerank Rerank documents by relevance to a query
search Perform web or document search
ocr Extract text from an image or document via OCR
create_file Upload a file to the LLM provider
list_files List uploaded files from the provider
retrieve_file Retrieve metadata for an uploaded file
delete_file Delete an uploaded file
file_content Retrieve the raw content of an uploaded file
create_batch Create a new batch processing job
list_batches List batch processing jobs
retrieve_batch Retrieve a batch processing job by ID
cancel_batch Cancel an in-progress batch job
create_response Create a new response (Responses API)
retrieve_response Retrieve a response by ID (Responses API)
cancel_response Cancel an in-progress response

Prerequisites

  • liter-llm CLI installed (see Installation)
  • API keys set up for the providers you want to use (e.g., OPENAI_API_KEY, ANTHROPIC_API_KEY)
  • Optional: Custom liter-llm-proxy.toml config file

Starting the MCP Server

Stdio Transport (IDE Integration)

For IDE integration (VS Code, GitHub Copilot, Claude Desktop, Cursor), use the stdio transport:

liter-llm mcp --transport stdio --config /path/to/liter-llm-proxy.toml

This is the standard way to integrate MCP servers into IDEs. The server communicates via stdin/stdout and is configured in the IDE's MCP settings. The proxy config must include [mcp] stdio_key_id = "..." or [mcp] stdio_trust_local = true.

HTTP Transport (Remote/Custom)

For remote connections or custom integrations:

liter-llm mcp --transport http --host 127.0.0.1 --port 3001

The HTTP server will be available at http://127.0.0.1:3001/mcp. Every request must include Authorization: Bearer <master-or-virtual-key>.

Custom Configuration

Use a custom proxy config file to define models, virtual keys, MCP stdio auth context, and provider credentials:

liter-llm mcp --transport stdio --config /path/to/liter-llm-proxy.toml

Configuration is identical to the proxy server — see Proxy Configuration for full details.

IDE Setup

VS Code supports MCP servers natively. See the VS Code MCP documentation for setup details.

Manual Installation (without extension):

Edit your VS Code settings (.vscode/settings.json or global settings) to register the liter-llm MCP server:

{
  "mcpServers": [
    {
      "name": "liter-llm",
      "command": "liter-llm",
      "args": ["mcp", "--transport", "stdio", "--config", "/absolute/path/to/liter-llm-proxy.toml"],
      "env": {
        "OPENAI_API_KEY": "sk-...",
        "ANTHROPIC_API_KEY": "sk-ant-...",
        "GROQ_API_KEY": "gsk_..."
      }
    }
  ]
}

Then use the tools in GitHub Copilot's chat by typing @llm-tools or explicitly referencing tools like @chat or @embed.

Claude Desktop and Claude Code use the MCP client to load external tools. Configure liter-llm in ~/Library/Application Support/Claude/claude_desktop_config.json (macOS) or %APPDATA%\Claude\claude_desktop_config.json (Windows):

{
  "mcpServers": {
    "liter-llm": {
      "command": "liter-llm",
      "args": ["mcp", "--transport", "stdio", "--config", "/absolute/path/to/liter-llm-proxy.toml"],
      "env": {
        "OPENAI_API_KEY": "sk-...",
        "ANTHROPIC_API_KEY": "sk-ant-...",
        "GROQ_API_KEY": "gsk_..."
      }
    }
  }
}

After saving, restart Claude Desktop. The liter-llm tools will be available in the Tools panel and can be invoked directly in Claude conversations.

Cursor supports MCP servers through its .cursor/settings.json or global Cursor settings.

Manual Configuration:

Edit your Cursor settings (⌘, → "Cursor Settings" → "MCP Servers") or directly edit the config file:

{
  "mcpServers": [
    {
      "name": "liter-llm",
      "command": "liter-llm",
      "args": ["mcp", "--transport", "stdio", "--config", "/absolute/path/to/liter-llm-proxy.toml"],
      "env": {
        "OPENAI_API_KEY": "sk-...",
        "ANTHROPIC_API_KEY": "sk-ant-...",
        "GROQ_API_KEY": "gsk_..."
      }
    }
  ]
}

Use tools via Cursor's AI chat or code generation features.

For custom applications or tools that support HTTP-based MCP, use the HTTP transport:

liter-llm mcp --transport http --host 127.0.0.1 --port 3001

Then register the server in your application:

{
  "mcpServers": [
    {
      "name": "liter-llm",
      "url": "http://127.0.0.1:3001/mcp",
      "headers": {
        "Authorization": "Bearer ${LITER_LLM_MASTER_KEY}"
      }
    }
  ]
}

Refer to your application's MCP client documentation for configuration details.

Configuration

Environment Variables

The MCP server reads API keys and provider configuration from environment variables, identical to the main liter-llm client:

export OPENAI_API_KEY="sk-..."
export ANTHROPIC_API_KEY="sk-ant-..."
export GOOGLE_API_KEY="..."
export GROQ_API_KEY="gsk_..."
export MISTRAL_API_KEY="..."
export REPLICATE_API_KEY="..."

Provider-specific keys

You only need to set keys for providers you plan to use. If you only use OpenAI, set OPENAI_API_KEY and liter-llm will automatically route requests to OpenAI based on the model prefix (e.g., openai/gpt-4o).

Custom Config File

To override defaults, base URLs, or provider configuration, use a custom proxy TOML file:

liter-llm mcp --transport stdio --config /etc/liter-llm/liter-llm-proxy.toml

Example liter-llm-proxy.toml:

[general]
master_key = "${LITER_LLM_MASTER_KEY}"

[mcp]
stdio_trust_local = true

[[models]]
name = "gpt-4o"
provider_model = "openai/gpt-4o"
api_key = "${OPENAI_API_KEY}"

[[models]]
name = "ollama-qwen"
provider_model = "ollama/qwen2.5"
base_url = "http://localhost:11434/v1"

Use stdio_key_id instead of stdio_trust_local when you want stdio tools to run with a virtual key's model allowlist, RPM/TPM limits, and budget.

See Proxy Configuration for all available options.

Master Key (Optional)

For remote HTTP transport, set a master key or configure virtual keys to authenticate requests:

export LITER_LLM_MASTER_KEY="your-secure-key"
liter-llm mcp --transport http --host 0.0.0.0 --port 3001

Clients must include the key in the Authorization header:

curl http://localhost:3001/mcp \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer your-secure-key" \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'

Tool Examples

Using the chat Tool

In Claude, ask a question and it will use the chat tool:

Call the liter-llm chat tool with model "openai/gpt-4o" to summarize this document: [paste text]

curl -X POST http://127.0.0.1:3001/mcp \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $LITER_LLM_MASTER_KEY" \
  -d '{
    "jsonrpc": "2.0",
    "id": 1,
    "method": "tools/call",
    "params": {
      "name": "chat",
      "arguments": {
        "model": "gpt-4o",
        "messages": [{"role": "user", "content": "What is 2+2?"}]
      }
    }
  }'

Using the embed Tool

Generate embeddings for semantic search:

{
  "model": "openai/text-embedding-3-small",
  "input": "The quick brown fox jumps over the lazy dog"
}

Using the list_models Tool

List all available models from configured providers:

{}

Returns a list of models with metadata (e.g., openai/gpt-4o, anthropic/claude-opus, groq/llama3-70b).

Troubleshooting

MCP Server Not Starting

Error: command not found: liter-llm

Ensure the CLI is installed and in your PATH:

liter-llm --version
which liter-llm

If not found, reinstall:

brew trust xberg-io/tap
brew install xberg-io/tap/liter-llm
# or
cargo install liter-llm-cli

IDE Not Seeing Tools

VS Code / GitHub Copilot:

  1. Restart VS Code completely (close and reopen)
  2. Check the "Extension" → "MCP" debug output for errors
  3. Verify the command path in settings.json is correct

Claude Desktop:

  1. Restart Claude Desktop
  2. Check ~/Library/Logs/Claude/ for error logs
  3. Verify the config file is valid JSON

Cursor:

  1. Use ⌘+K → "MCP Servers" to reload
  2. Check the Cursor settings panel for errors

API Key Not Found

Error: authentication failed: api key not set

Ensure API keys are set as environment variables before starting the MCP server:

export OPENAI_API_KEY="sk-..."
liter-llm mcp --transport stdio --config liter-llm-proxy.toml

Or hardcode them in the IDE config (less secure):

"env": {
  "OPENAI_API_KEY": "sk-..."
}

Never commit API keys

Use environment variables or a .env file (not committed to git). See Secrets and API Key Handling for best practices.

Timeout Errors

Error: request timeout after 30s

Increase the timeout in a config file:

[general]
default_timeout_secs = 120

Then start with the config:

liter-llm mcp --transport stdio --config liter-llm-proxy.toml

Port Already in Use (HTTP)

Error: bind failed: address already in use

Use a different port:

liter-llm mcp --transport http --host 127.0.0.1 --port 3002

Or kill the existing process:

lsof -i :3001
kill -9 <PID>

Remote Connection Refused

Error: connection refused: 127.0.0.1:3001

Ensure the MCP server is running and listening on the correct host/port:

# Start the server
liter-llm mcp --transport http --host 0.0.0.0 --port 3001 &

# Test the connection
curl http://127.0.0.1:3001/mcp \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $LITER_LLM_MASTER_KEY" \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'

Note

Use 0.0.0.0 to allow remote connections. For local development, 127.0.0.1 is safer.

Next Steps

Edit this page on GitHub