LLM Gateway

https://docs.llmgateway.io
Admin
LLM Gateway is an open-source API gateway for Large Language Models (LLMs) that acts as middleware...
Tokens:181,588
Snippets:2,453
Trust Score:7.7
Update:2 months ago
Show doc for...
Tokens:
Raw
### Install AI SDK and Provider

Source: https://docs.llmgateway.io/developers/ai-sdk

Install the necessary packages via pnpm.

```bash
pnpm add ai @llmgateway/ai-sdk-provider
```

--------------------------------

### Launch coding agents with the CLI

Source: https://docs.llmgateway.io/developers/cli

Commands for starting coding agents, selecting models, and managing agent installations.

```bash
# Interactive agent picker
npx @llmgateway/cli launch

# Launch a specific agent (shortcuts work too: `llmgateway claude`)
npx @llmgateway/cli launch claude
npx @llmgateway/cli launch opencode
npx @llmgateway/cli launch codex

# Pick a model — launcher flags go before the agent name
npx @llmgateway/cli launch -m gpt-5.5 claude

# Everything after the agent name is passed to the agent itself
npx @llmgateway/cli launch claude --continue

# List all supported agents and see which are installed
npx @llmgateway/cli launch --list

# Inspect what would run without launching
npx @llmgateway/cli launch --dry-run codex
```

--------------------------------

### Install and Initialize CLI

Source: https://docs.llmgateway.io/developers/cli

Methods for running the CLI via npx or global installation, including shorthand aliases.

```bash
npx @llmgateway/cli init
```

```bash
npm install -g @llmgateway/cli
```

```bash
llmgateway init
lg init
```

--------------------------------

### Install Continue CLI

Source: https://docs.llmgateway.io/guides/continue

Install the Continue CLI globally using npm.

```bash
npm install -g @continuedev/cli
```

--------------------------------

### Install Dependencies

Source: https://docs.llmgateway.io/guides/agent-skills

Install the necessary project dependencies using pnpm.

```bash
pnpm install
```

--------------------------------

### Install LLM Gateway packages

Source: https://docs.llmgateway.io/features/embeddable-payments

Install the necessary server and client packages via npm.

```bash
# backend
npm install @llmgateway/server
# frontend (pick one)
npm install @llmgateway/elements   # React drop-in components
npm install @llmgateway/client     # headless / non-React
```

--------------------------------

### Start development server

Source: https://docs.llmgateway.io/developers/cli

Launch the local development server on a specified port.

```bash
npx @llmgateway/cli dev [options]
```

--------------------------------

### Install Crush

Source: https://docs.llmgateway.io/guides/crush

Installation commands for Crush using Homebrew or NPM.

```bash
# Homebrew
brew install charmbracelet/tap/crush

# NPM
npm install -g @charmland/crush
```

--------------------------------

### Install the Helm chart

Source: https://docs.llmgateway.io/self-host/kubernetes

Use the Helm CLI to install the LLM Gateway from the GitHub Container Registry.

```bash
helm install llmgateway oci://ghcr.io/theopenco/charts/llmgateway
```

--------------------------------

### Verify MiMo Code Installation

Source: https://docs.llmgateway.io/guides/mimocode

Check that the CLI is installed correctly by accessing the help menu.

```bash
mimo --help
```

--------------------------------

### Install Hermes Agent

Source: https://docs.llmgateway.io/guides/hermes-agent

Execute the official installation script to set up Hermes Agent and its dependencies.

```bash
curl -fsSL https://raw.githubusercontent.com/NousResearch/hermes-agent/main/scripts/install.sh | bash
```

--------------------------------

### Install MiMo Code CLI

Source: https://docs.llmgateway.io/guides/mimocode

Use the official installation script to set up the MiMo Code command-line tool.

```bash
curl -fsSL https://mimo.xiaomi.com/install | bash
```

--------------------------------

### Launch Continue CLI

Source: https://docs.llmgateway.io/guides/continue

Start the Continue CLI using the specified configuration file.

```bash
cn --config ~/.continue/config.yaml
```

--------------------------------

### LiteLLM Configuration File

Source: https://docs.llmgateway.io/migrations/litellm

Example of the legacy configuration format used with LiteLLM.

```yaml
# litellm_config.yaml
model_list:
  - model_name: gpt-4
    litellm_params:
      model: gpt-4
      api_key: sk-...
  - model_name: claude-3
    litellm_params:
      model: claude-3-sonnet-20240229
      api_key: sk-ant-...
```

--------------------------------

### Install LLM Gateway SDK Provider

Source: https://docs.llmgateway.io/llms-full.txt

Install the native provider package to enable compatibility with the Vercel AI SDK.

```bash
pnpm add @llmgateway/ai-sdk-provider
```

--------------------------------

### Install OpenCode Desktop via Homebrew

Source: https://docs.llmgateway.io/guides/opencode-desktop

Use this command to install the OpenCode Desktop application on macOS systems using Homebrew.

```bash
brew install --cask opencode-desktop
```

--------------------------------

### Install Kimi Code CLI

Source: https://docs.llmgateway.io/guides/kimi-code

Commands to install the Kimi Code CLI on various operating systems.

```bash
curl -fsSL https://code.kimi.com/kimi-code/install.sh | bash
```

```bash
brew install kimi-code
```

```powershell
irm https://code.kimi.com/kimi-code/install.ps1 | iex
```

--------------------------------

### Launch Crush

Source: https://docs.llmgateway.io/guides/crush

Start the Crush terminal application.

```bash
crush
```

--------------------------------

### Install DevPass Code via npm

Source: https://docs.llmgateway.io/llms-full.txt

Install the package globally using the npm package manager.

```bash
npm i -g devpass-code
```

--------------------------------

### Install DevPass Code via Homebrew

Source: https://docs.llmgateway.io/llms-full.txt

Install the package using the Homebrew package manager on macOS or Linux.

```bash
brew install theopenco/tap/devpass-code
```

--------------------------------

### Install and Trigger Claude Code Skills

Source: https://docs.llmgateway.io/llms-full.txt

Commands to install a skill into a project directory and trigger it within the Claude Code interface.

```bash
# from the llmgateway-templates repo
cp -r skills/changelog /path/to/your/project/.claude/skills/
```

```text
> changelog
```

--------------------------------

### Launch OpenCode via Terminal

Source: https://docs.llmgateway.io/guides/opencode

Execute this command in your terminal to start the OpenCode application.

```bash
opencode
```

--------------------------------

### Configure LLM Gateway values

Source: https://docs.llmgateway.io/self-host/gcp

Example configuration for connecting the gateway to Cloud SQL and Memorystore instances.

```yaml
config:
  DATABASE_URL: "postgres://user:password@your-cloud-sql-ip:5432/llmgateway"
  REDIS_URL: "redis://your-memorystore-ip:6379"
  AUTH_SECRET: "from-secret-manager"
  GATEWAY_API_KEY_HASH_SECRET: "from-secret-manager"
```

--------------------------------

### Usage Analytics Examples

Source: https://docs.llmgateway.io/llms-full.txt

Common patterns for querying usage data over time or by specific dimensions.

```bash
# Last 7 days for the default project
npx @llmgateway/cli usage

# Cost per model over the last 30 days
npx @llmgateway/cli usage --by model --range 30d

# Whole-org aggregate
npx @llmgateway/cli usage --org <orgId>
```

--------------------------------

### Truncation Completion Example

Source: https://docs.llmgateway.io/features/response-healing

Demonstrates how the gateway closes truncated JSON structures.

```text
{"name": "Alice", "data": {"nested": true
```

```text
{ "name": "Alice", "data": { "nested": true } }
```

--------------------------------

### List available image generation models

Source: https://docs.llmgateway.io/developers/mcp

Example output format for the list-image-models tool.

```markdown
# Image Generation Models

## Qwen Image Plus
- **Model ID:** `qwen-image-plus`
- **Description:** Text-to-image with excellent text rendering
- **Price:** $0.03 per request

## Qwen Image Max
- **Model ID:** `qwen-image-max`
- **Description:** Highest quality text-to-image
- **Price:** $0.075 per request
```

--------------------------------

### Install a Claude Code Skill

Source: https://docs.llmgateway.io/guides/agent-skills

Copy a skill folder into the project's .claude/skills directory to enable it.

```bash
# from the llmgateway-templates repo
cp -r skills/changelog /path/to/your/project/.claude/skills/
```

--------------------------------

### GET /mcp (SSE)

Source: https://docs.llmgateway.io/llms-full.txt

Establishes a Server-Sent Events (SSE) connection for real-time updates.

```APIDOC
## GET /mcp

### Description
Connects to the MCP server using Server-Sent Events for real-time streaming updates.

### Method
GET

### Endpoint
https://api.llmgateway.io/mcp

### Headers
- **Accept** (string) - Required - Must be "text/event-stream"
- **Authorization** (string) - Required - Bearer token for authentication
```

--------------------------------

### Truncated JSON Input

Source: https://docs.llmgateway.io/llms-full.txt

Example of an incomplete JSON response that requires completion.

```text
{"name": "Alice", "data": {"nested": true
```

--------------------------------

### Initialize a Project

Source: https://docs.llmgateway.io/developers/cli

Commands to scaffold a new project, optionally specifying a template and project name.

```bash
npx @llmgateway/cli init
```

```bash
npx @llmgateway/cli init --template image-generation --name my-ai-app
```

--------------------------------

### Custom model response structure

Source: https://docs.llmgateway.io/llms-full.txt

Example JSON response body returned when querying custom models.

```json
{
	"customModels": [
		{
			"id": "cm_...",
			"providerKeyId": "pk_...",
			"organizationId": "org_...",
			"modelName": "acme-large",
			"displayName": "Acme Large",
			"contextSize": 200000,
			"maxOutput": 32000,
			"inputPrice": "3.0e-6",
			"outputPrice": "15.0e-6",
			"cachedInputPrice": "0.3e-6",
			"streaming": "true",
			"vision": true,
			"tools": true,
			"reasoning": null,
			"jsonOutput": true,
			"supportedParameters": ["temperature", "top_p"],
			"status": "active",
			"createdAt": "...",
			"updatedAt": "..."
		}
	]
}
```

--------------------------------

### Initialize a new project

Source: https://docs.llmgateway.io/developers/cli

Create a new project directory using a specified template and name.

```bash
npx @llmgateway/cli init [directory] [options]
```

```bash
# Interactive mode
npx @llmgateway/cli init

# With options
npx @llmgateway/cli init --template image-generation --name my-app
```

--------------------------------

### Initialize a project with a template

Source: https://docs.llmgateway.io/developers/cli

Use the init command to scaffold a new project based on a specific template.

```bash
npx @llmgateway/cli init --template qa-agent
```

--------------------------------

### Install DevPass Code

Source: https://docs.llmgateway.io/guides/devpass-code

Global installation methods for the DevPass Code CLI.

```shell
npm i -g devpass-code
```

```shell
brew install theopenco/tap/devpass-code
```

--------------------------------

### GET /v1/master/keys

Source: https://docs.llmgateway.io/llms-full.txt

Returns the developer-created gateway API keys in the master key's organization, including configured limits and usage data.

```APIDOC
## GET /v1/master/keys

### Description
Returns the developer-created gateway API keys in the master key's organization, each with its configured limits, the usage consumed so far, and the time the current period resets.

### Method
GET

### Endpoint
/v1/master/keys

### Parameters
#### Query Parameters
- **projectId** (string) - Optional - Scope the list to a single project.

### Response
#### Success Response (200)
- **apiKeys** (array) - List of API key objects.
```

--------------------------------

### Create a Project

Source: https://docs.llmgateway.io/features/master-keys

Create a new project by sending a POST request with the required configuration.

```bash
curl -X POST https://internal.llmgateway.io/v1/master/projects \
  -H "Authorization: Bearer $MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Customer ACME",
    "cachingEnabled": false,
    "mode": "hybrid"
  }'
```

--------------------------------

### GET /v1/master/usage

Source: https://docs.llmgateway.io/llms-full.txt

Exports usage data for a specified time range and grouping criteria in CSV format.

```APIDOC
## GET /v1/master/usage

### Description
Exports usage data for a specified time range and grouping criteria in CSV format.

### Method
GET

### Endpoint
https://internal.llmgateway.io/v1/master/usage

### Parameters
#### Query Parameters
- **from** (string) - Required - Start date (YYYY-MM-DD)
- **to** (string) - Required - End date (YYYY-MM-DD)
- **granularity** (string) - Required - Time granularity (e.g., day)
- **groupBy** (string) - Required - Dimensions to group by (e.g., user,model)
- **format** (string) - Required - Output format (e.g., csv)
```

--------------------------------

### Verify Hermes Installation

Source: https://docs.llmgateway.io/guides/hermes-agent

Reload the shell configuration and verify the installation by checking the version.

```bash
source ~/.bashrc
hermes --version
```

--------------------------------

### Start Development Server

Source: https://docs.llmgateway.io/developers/cli

Commands to launch the development server within a project directory, with optional port configuration.

```bash
cd my-ai-app
npx @llmgateway/cli dev
```

```bash
npx @llmgateway/cli dev --port 3000
```

--------------------------------

### Verify Kimi Code Installation

Source: https://docs.llmgateway.io/guides/kimi-code

Check the installed version of the Kimi Code CLI.

```shell
kimi --version
```

--------------------------------

### Launch Hermes Setup Wizard

Source: https://docs.llmgateway.io/guides/hermes-agent

Initiate the interactive configuration wizard to set up providers, models, and terminal backends.

```bash
hermes setup
```

--------------------------------

### Configure full environment variables

Source: https://docs.llmgateway.io/guides/claude-code

Sets the base URL, authentication token, and model preferences for the gateway.

```bash
export ANTHROPIC_BASE_URL=https://api.llmgateway.io
export ANTHROPIC_AUTH_TOKEN=llmgtwy_your_api_key_here
export ANTHROPIC_MODEL=gpt-5
export ANTHROPIC_SMALL_FAST_MODEL=gpt-5-nano
```

--------------------------------

### Deploying with Unified Docker Compose Image

Source: https://docs.llmgateway.io/self-host/docker-compose

Downloads the unified compose configuration and environment template to start the service.

```bash
# Download the compose file
curl -O https://raw.githubusercontent.com/theopenco/llmgateway/main/infra/docker-compose.unified.yml
curl -O https://raw.githubusercontent.com/theopenco/llmgateway/main/.env.unified.example

# Configure environment
cp .env.unified.example .env
# Edit .env with your configuration

# Start the service
docker compose -f docker-compose.unified.yml up -d
```

--------------------------------

### Create onboarding sentinel file

Source: https://docs.llmgateway.io/guides/continue

Create this file to skip the onboarding prompt when running the CLI without a config flag.

```bash
touch ~/.continue/.onboarding_complete
```

--------------------------------

### GET /

Source: https://docs.llmgateway.io/health

Retrieves the current health status of the service and its dependencies.

```APIDOC
## GET /

### Description
Health check endpoint to verify the status of the service and its dependencies.

### Method
GET

### Endpoint
/

### Parameters
#### Query Parameters
- **skip** (string) - Optional - Comma-separated list of health checks to skip. Options: redis, database

### Response
#### Success Response (200)
- **message** (string) - Status message
- **version** (string) - Service version
- **health** (object) - Health status details
  - **status** (string) - Overall health status
  - **redis** (object) - Redis connection status
    - **connected** (boolean) - Connection status
    - **error** (string) - Error message if any
  - **database** (object) - Database connection status
    - **connected** (boolean) - Connection status
    - **error** (string) - Error message if any

### Response Example
{
  "message": "string",
  "version": "string",
  "health": {
    "status": "string",
    "redis": {
      "connected": true,
      "error": "string"
    },
    "database": {
      "connected": true,
      "error": "string"
    }
  }
}
```

--------------------------------

### Run Continue with a custom configuration file

Source: https://docs.llmgateway.io/guides/continue

Use the --config flag to specify a custom configuration file path when launching the CLI.

```bash
cn --config path/to/config.yaml
```

--------------------------------

### Launch Kimi Code

Source: https://docs.llmgateway.io/guides/kimi-code

Start the interactive Kimi Code terminal session.

```shell
kimi
```

--------------------------------

### Create Continue Configuration Directory

Source: https://docs.llmgateway.io/guides/continue

Create the necessary directory for the Continue configuration file.

```bash
mkdir -p ~/.continue
```
Tokens:
Raw