Nexus

https://nexusrouter.com/docs
Admin
Nexus is an AI Router that provides unified endpoints for LLM providers and MCP (Model Context...
Tokens:48,222
Snippets:347
Trust Score:6.7
Update:1 year ago
Show doc for...
Tokens:
Raw
### Nexus Router Configuration File Example

Source: https://nexusrouter.com/docs/installation

An example of a basic Nexus router TOML configuration file, demonstrating how to define an MCP server with its command. This file configures the server's behavior and endpoints.

```toml
# Add your first MCP server
[mcp.servers.example]
cmd = ["npx", "-y", "@modelcontextprotocol/server-filesystem", "/tmp"]
```

--------------------------------

### Run Nexus Router

Source: https://nexusrouter.com/docs/installation

Basic command to start the Nexus router with default settings. It will look for a 'nexus.toml' configuration file in the current directory and start the server.

```bash
nexus
```

--------------------------------

### Install Nexus using Bash or Docker

Source: https://nexusrouter.com/docs/index

Provides instructions for installing Nexus using a curl script or a Docker image. This is the initial step to get Nexus running on your system.

```bash
curl -fsSL https://nexusrouter.com/install | bash
```

```docker
docker run -p 8000:8000 \
  -v $(pwd)/nexus.toml:/etc/nexus.toml \
  ghcr.io/grafbase/nexus:stable
```

--------------------------------

### Docker Compose Up Command

Source: https://nexusrouter.com/docs/installation

Command to start the Nexus Router service defined in a `compose.yaml` file.

```bash
docker compose up -d
```

--------------------------------

### Nexus TOML Configuration Example

Source: https://nexusrouter.com/docs/configuration

Illustrates the TOML format used for Nexus configuration, showing examples for defining LLM providers (OpenAI) and MCP servers (GitHub).

```toml
# LLM providers
[llm.providers.openai]
type = "openai"
api_key = "{{ env.OPENAI_API_KEY }}"

# MCP servers
[mcp.servers.github]
url = "https://api.github.com/mcp"
```

--------------------------------

### Nexus Quick Install Script

Source: https://nexusrouter.com/docs/installation

Installs Nexus Router using a shell script. This method automatically detects the OS and architecture, downloads the binary, and sets it up in `~/.nexus/bin`.

```bash
curl -fsSL https://nexusrouter.com/install | bash
```

--------------------------------

### Install Nexus Router Binary

Source: https://nexusrouter.com/docs/installation

Instructions for installing the compiled Nexus router binary to a system path, making it executable from any location. This requires sudo privileges.

```bash
mkdir -p ~/.nexus/bin
sudo cp target/release/nexus ~/.nexus/bin
sudo chmod +x ~/.nexus/bin/nexus
```

--------------------------------

### Running Nexus Router with Docker

Source: https://nexusrouter.com/docs/installation

Demonstrates how to run the Nexus Router container, showing basic usage and how to pass environment variables for configuration.

```bash
docker run -p 8000:8000 \
  -v $(pwd)/nexus.toml:/etc/nexus.toml \
  ghcr.io/grafbase/nexus:stable
```

```bash
docker run -p 8000:8000 \
  -v $(pwd)/nexus.toml:/etc/nexus.toml \
  -e GITHUB_TOKEN=${GITHUB_TOKEN} \
  -e OPENAI_API_KEY=${OPENAI_API_KEY} \
  ghcr.io/grafbase/nexus:stable
```

--------------------------------

### Function Calling / Tool Use Example (Python)

Source: https://nexusrouter.com/docs/usage/llm-api

An example snippet demonstrating the setup for function calling or tool use with the Nexus LLM router using the OpenAI Python SDK. The API remains consistent across providers.

```python
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/llm/openai/v1",
    api_key="not-used"
)


```

--------------------------------

### Nexus Router Command Line Options

Source: https://nexusrouter.com/docs/installation

Displays the available command-line options for the Nexus router, including settings for listen address, configuration file path, and logging.

```text
Usage: nexus [OPTIONS]

Options:
  -l, --listen-address <LISTEN_ADDRESS>
          IP address on which the server will listen for incomming connections. Default: 127.0.0.1:6000

          [env: NEXUS_LISTEN_ADDRESS=]

  -c, --config <CONFIG>
          Path to the TOML configuration file

          [env: NEXUS_CONFIG_PATH=]
          [default: ./nexus.toml]

      --log <LOG_LEVEL>
          Set the logging level, this applies to all spans, logs and trace events

          [env: NEXUS_LOG=]
          [default: info]

          Possible values:
          - off:   Disable logging
          - error: Only log errors
          - warn:  Log errors, and warnings
          - info:  Log errors, warnings, and info messages
          - debug: Log errors, warnings, info, and debug messages
          - trace: Log errors, warnings, info, debug, and trace messages

      --log-style <LOG_STYLE>
          Set the style of log output

          [env: NEXUS_LOG_STYLE=]
          [default: color]

          Possible values:
          - color: Colorized text, used as the default with TTY output
          - text:  Standard text, used as the default with non-TTY output
          - json:  JSON objects

  -h, --help
          Print help (see a summary with '-h')

  -V, --version
          Print version
```

--------------------------------

### Verify Nexus Router Installation

Source: https://nexusrouter.com/docs/installation

Command to verify that the Nexus router has been successfully installed and is accessible in the system's PATH. It checks the installed version.

```bash
nexus --version
```

--------------------------------

### Start Nexus Server with Default or Custom Configuration

Source: https://nexusrouter.com/docs/index

Commands to start the Nexus server. It can be run with default settings or by specifying a custom configuration file using the --config flag.

```bash
# Start with default settings
nexus

# Or specify a config file
nexus --config ./nexus.toml
```

--------------------------------

### Nexus Helm Custom Values Configuration

Source: https://nexusrouter.com/docs/installation

Example of a Helm values file (`custom-values.yaml`) to customize Nexus Router deployment in Kubernetes, controlling replicas, auto-scaling, and resources.

```yaml
# 1. number of desired replicas running
replicaCount: 2

# 2. auto-scaling behaviour
autoscaling:
  enabled: true
  minReplicas: 2
  maxReplicas: 10
  targetCPUUtilizationPercentage: 70
  targetMemoryUtilizationPercentage: 70

# 3. compute resources
resources:
  limits:
    cpu: 2
    memory: 2Gi
  requests:
    cpu: 1
    memory: 1Gi

# 4. external configuration from cluster configmaps
nexus:
  externalConfig: true
  args:
    - --config
    - /etc/nexus/config/config.toml
volumes:
  - name: configuration
    configMap:
      name: nexus-configuration
volumeMounts:
  - name: configuration
    mountPath: /etc/nexus/config

```

--------------------------------

### Verify Helm Deployment

Source: https://nexusrouter.com/docs/installation

Commands to verify the Helm deployment of Nexus Router by listing Helm releases and checking Kubernetes pods.

```bash
helm list

kubectl get pods
```

--------------------------------

### Nexus Helm Chart Deployment

Source: https://nexusrouter.com/docs/installation

Installs the Nexus Router Helm chart for deployment on Kubernetes. Requires Helm and kubectl to be installed.

```bash
helm install test oci://ghcr.io/grafbase/helm-charts/nexus --version <version>
```

--------------------------------

### Clone and Build Nexus Router

Source: https://nexusrouter.com/docs/installation

This snippet shows how to clone the Nexus repository and build the release binary using Cargo. It assumes a Rust development environment is set up.

```bash
git clone https://github.com/grafbase/nexus
cd nexus
cargo build --release -p nexus
```

--------------------------------

### Nexus Docker Compose Configuration

Source: https://nexusrouter.com/docs/installation

A sample `compose.yaml` file for deploying Nexus Router using Docker Compose, including volume mounts, environment variables, and health checks.

```yaml
services:
  nexus:
    image: ghcr.io/grafbase/nexus:stable
    ports:
      - "8000:8000"
    volumes:
      - ./nexus.toml:/etc/nexus.toml
      # Mount additional directories if using STDIO servers
      - /path/to/mcp-servers:/opt/mcp-servers
    environment:
      - GITHUB_TOKEN=${GITHUB_TOKEN}
      - OPENAI_API_KEY=${OPENAI_API_KEY}
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
      interval: 30s
      timeout: 10s
      retries: 3
    restart: unless-stopped

```

--------------------------------

### Per-Provider Examples

Source: https://nexusrouter.com/docs/configuration/llm/providers

Configuration examples for setting up header rules for OpenAI, Anthropic, and Google providers.

```APIDOC
## Per-Provider Examples

#### OpenAI with Custom Headers
```toml
[llm.providers.openai]
type = "openai"
api_key = "{{ env.OPENAI_API_KEY }}"

# Forward user context
[[llm.providers.openai.headers]]
rule = "forward"
name = "x-user-id"

# Add OpenAI-specific beta features
[[llm.providers.openai.headers]]
rule = "insert"
name = "OpenAI-Beta"
value = "assistants=v2"
```

#### Anthropic with Organization Headers
```toml
[llm.providers.anthropic]
type = "anthropic"
api_key = "{{ env.ANTHROPIC_API_KEY }}"

# Forward and rename organization header
[[llm.providers.anthropic.headers]]
rule = "forward"
name = "x-org-id"
rename = "anthropic-org-id"
```

#### Google with Project Headers
```toml
[llm.providers.google]
type = "google"
api_key = "{{ env.GOOGLE_API_KEY }}"

# Add Google Cloud project headers
[[llm.providers.google.headers]]
rule = "insert"
name = "x-goog-user-project"
value = "{{ env.GCP_PROJECT_ID }}"
```
```

--------------------------------

### Kubernetes Pod Verification

Source: https://nexusrouter.com/docs/installation

Verifies the Nexus Router deployment in Kubernetes by listing pods and checking for a running `test-nexus` pod.

```bash
kubectl get pods
```

--------------------------------

### LLM API Call Example (Bash)

Source: https://nexusrouter.com/docs/usage

Demonstrates how to make a chat completion request to the Nexus LLM API using cURL. This example shows the POST request, content type, and a sample JSON payload for a user message.

```bash
curl -X POST http://localhost:8000/llm/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
```

--------------------------------

### Comprehensive MCP Server Configuration Example

Source: https://nexusrouter.com/docs/configuration/mcp

This extensive TOML configuration demonstrates a complete setup for MCP servers, including HTTP and local servers, caching, rate limiting, TLS configuration, and authentication methods. It provides examples for weather APIs, GitHub, database tools, and internal company services, showcasing advanced features like tool-specific rate limits and custom CA certificates.

```toml
[mcp]
enabled = true
path = "/mcp"

# Cache configuration for performance
[mcp.downstream_cache]
max_size = 1000
idle_timeout = "10m"

# Public API with rate limiting
[mcp.servers.weather]
url = "https://weather-api.example.com/mcp"

[mcp.servers.weather.rate_limits]
limit = 100
interval = "60s"

# Authenticated API with TLS
[mcp.servers.github]
url = "https://api.github.com/mcp"

[mcp.servers.github.auth]
token = "{{ env.GITHUB_TOKEN }}"

[mcp.servers.github.tls]
verify_certs = true

# Tool-specific rate limits
[mcp.servers.github.rate_limits.tools]
search_code = { limit = 30, interval = "60s" }
create_issue = { limit = 10, interval = "60s" }

# Local database tools
[mcp.servers.database]
cmd = ["psql-mcp"]
env = {
  PGHOST = "localhost",
  PGDATABASE = "myapp",
  PGUSER = "{{ env.DB_USER }}",
  PGPASSWORD = "{{ env.DB_PASSWORD }}"
}
stderr = "null"  # Suppress stderr in production

# Internal service with OAuth2 forwarding
[mcp.servers.company_api]
url = "https://api.internal.company.com/mcp"

[mcp.servers.company_api.auth]
type = "forward"

[mcp.servers.company_api.tls]
verify_certs = true
root_ca_cert_path = "/etc/ssl/company-ca.pem"
```

--------------------------------

### Nexus Docker Image Management

Source: https://nexusrouter.com/docs/installation

Commands for managing Nexus Router Docker images, including pulling the stable release, specific versions, and the latest development build.

```bash
docker pull ghcr.io/grafbase/nexus:stable
```

```bash
docker pull ghcr.io/grafbase/nexus:0.2.0
```

```bash
docker pull ghcr.io/grafbase/nexus:latest
```

--------------------------------

### View All Helm Values

Source: https://nexusrouter.com/docs/installation

Retrieves all customizable values for the Nexus Router Helm chart to assist in creating custom values files.

```bash
helm show values oci://ghcr.io/grafbase/helm-charts/nexus --version <version>
```

--------------------------------

### Helm Upgrade Command with Custom Values

Source: https://nexusrouter.com/docs/installation

Applies customized Nexus Router deployment settings to an existing Helm release using a values file.

```bash
helm upgrade test oci://ghcr.io/grafbase/helm-charts/nexus --version <version> -f custom-values.yaml
```

--------------------------------

### LLM API Chat Completion with Python SDK

Source: https://nexusrouter.com/docs/usage

Provides an example of using the OpenAI Python SDK to interact with the Nexus LLM API. It demonstrates initializing the client with a custom base URL and making a chat completion request.

```python
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/llm/v1",
    api_key="not-used"
)

response = client.chat.completions.create(
    model="openai/gpt-4",
    messages=[{"role": "user", "content": "Hello!"}]
)
```

--------------------------------

### MCP Tool Invocation Example (Bash)

Source: https://nexusrouter.com/docs/usage

Shows how to invoke an MCP tool, specifically for reading a file, using cURL. This example highlights the POST request to the MCP endpoint and the JSON payload specifying the file path.

```bash
curl -X POST http://localhost:8000/mcp/tools/filesystem/read_file \
  -H "Content-Type: application/json" \
  -d '{"path": "/home/user/document.txt"}'
```

--------------------------------

### Start Nexus Server

Source: https://nexusrouter.com/docs/usage/openai-codex-integration

This command starts the Nexus server using a specified configuration file. Nexus will then listen on the default address `http://localhost:6000`, making its configured endpoints available.

```bash
nexus --config nexus.toml
```

--------------------------------

### Nexus MCP Server Configuration Example

Source: https://nexusrouter.com/docs/usage/mcp-integration

Example TOML configuration for setting up Nexus to connect to external MCP servers, specifically GitHub and a filesystem server. This configuration defines the URLs or commands to establish these connections, enabling Nexus to aggregate their exposed tools, prompts, and resources.

```toml
```toml
[mcp.servers.github]
url = "https://api.github.com/mcp"

[mcp.servers.filesystem]
cmd = ["npx", "-y", "@modelcontextprotocol/server-filesystem", "/home"]
```
```

--------------------------------

### Running Nexus CLI with Default and Custom Config

Source: https://nexusrouter.com/docs/configuration

Demonstrates how to run the Nexus command-line interface (CLI) using the default configuration file (`nexus.toml`) or by specifying a custom configuration file path.

```bash
# Uses ./nexus.toml
nexus

# Or specify a custom config file
nexus --config /path/to/config.toml
```

--------------------------------

### Start Nexus Server

Source: https://nexusrouter.com/docs/usage/claude-code-integration

This bash command starts the Nexus server using the specified configuration file. Nexus will then be accessible at its default address, typically http://localhost:6000.

```bash
nexus --config nexus.toml

```

--------------------------------

### Codex Basic Usage with Nexus Provider

Source: https://nexusrouter.com/docs/usage/openai-codex-integration

Demonstrates starting a Codex session and executing a single command using the Nexus provider with specified models like GPT-4 and Claude.

```bash
# Start a Codex session with GPT-4
codex -c model="openai/gpt-4" -c model_provider=nexus

# Execute a single command
codex exec -c model="anthropic/claude-3-5-sonnet-20241022" -c model_provider=nexus "Write a Python function to calculate fibonacci"
```

--------------------------------

### Example MCP Server Naming Conventions

Source: https://nexusrouter.com/docs/configuration/mcp/servers

Illustrates best practices for naming MCP server configurations in TOML. Examples show descriptive naming and grouping related servers using prefixes, while cautioning against special characters.

```toml
# Good
[mcp.servers.code_search]

# Bad
[mcp.servers.server1]

# Grouping related servers
[mcp.servers.db_production]
[mcp.servers.db_staging]

# Using underscores for clarity
[mcp.servers.file_system]
```

--------------------------------

### Streaming Chat Completion Example (cURL)

Source: https://nexusrouter.com/docs/usage/llm-api

This example demonstrates how to enable streaming for chat completions, allowing responses to be received incrementally.

```bash
curl -X POST http://localhost:8000/llm/openai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4",
    "messages": [
      {"role": "user", "content": "Write a short story"}
    ],
    "stream": true
  }'
```

--------------------------------

### Full Redis Rate Limiting Configuration Example (TOML)

Source: https://nexusrouter.com/docs/configuration/server/rate-limiting

Provides a detailed example of Redis rate limiting configuration, including authentication, connection pooling, and TLS settings. Suitable for production environments.

```toml
[server.rate_limits]
storage = {
  type = "redis",
  url = "redis://username:password@redis.example.com:6379/0",
  key_prefix = "nexus:rate_limits:prod:",
  response_timeout = "10s",
  connection_timeout = "10s"
}

[server.rate_limits.storage.pool]
max_size = 20
min_idle = 5
timeout_create = "5s"
timeout_wait = "5s"

[server.rate_limits.storage.tls]
enabled = true
ca_cert_path = "/etc/ssl/certs/redis-ca.pem"
```

--------------------------------

### Nexus TOML Configuration with Environment Variables

Source: https://nexusrouter.com/docs/configuration

Shows how to reference environment variables within Nexus TOML configuration files, specifically for LLM API keys and database connection details (host, password).

```toml
[llm.providers.openai]
api_key = "{{ env.OPENAI_API_KEY }}"

[mcp.servers.database]
cmd = ["psql-mcp"]
env = {
  PGHOST = "{{ env.DB_HOST }}",
  PGPASSWORD = "{{ env.DB_PASSWORD }}"
}
```

--------------------------------

### TOML Sequential Header Processing Example

Source: https://nexusrouter.com/docs/configuration/llm/header-rules

Demonstrates the sequential application of header rules as they appear in the configuration file. This example shows how 'insert', 'forward', and 'rename_duplicate' rules modify the header set in order, illustrating their cumulative effect.

```toml
# Starting point: headers = {}

# 1. Insert a static header
[[llm.providers.openai.headers]]
rule = "insert"
name = "x-api-version"
value = "2024-01"
# Result: headers = {"x-api-version": "2024-01"}

# 2. Forward all user headers (will override existing headers!)
[[llm.providers.openai.headers]]
rule = "forward"
pattern = "^x-user-"
# If client sends x-user-id=123, x-user-role=admin
# Result: headers = {"x-api-version": "2024-01", "x-user-id": "123", "x-user-role": "admin"}

# 3. Duplicate a header for backup
[[llm.providers.openai.headers]]
rule = "rename_duplicate"
name = "x-user-id"
rename = "x-original-user-id"
```

--------------------------------

### Simple Chat Completion Example (cURL)

Source: https://nexusrouter.com/docs/usage/llm-api

A basic example demonstrating how to perform a simple chat completion using cURL. It sends a user message and expects a text response.

```bash
curl -X POST http://localhost:8000/llm/openai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4",
    "messages": [
      {"role": "user", "content": "Hello!"}
    ]
  }'
```

--------------------------------

### Nexus Server Configuration Example (TOML)

Source: https://nexusrouter.com/docs/configuration/server

A complete TOML configuration example for Nexus server settings, including basic server details, health checks, TLS, OAuth2 authentication, rate limiting, and CORS configuration. This snippet demonstrates how to structure the configuration file for various server functionalities.

```toml
# Basic server settings
[server]
listen_address = "0.0.0.0:8000"

[server.health]
enabled = true
path = "/health"

# TLS for production
[server.tls]
certificate = "/etc/nexus/server.crt"
key = "/etc/nexus/server.key"

# OAuth2 authentication
[server.oauth]
url = "https://auth.example.com/.well-known/jwks.json"
expected_issuer = "https://auth.example.com"
expected_audience = "nexus-api"

# Rate limiting
[server.rate_limits]
enabled = true
storage = "memory"

[server.rate_limits.per_ip]
limit = 100
interval = "60s"

# CORS for browser clients
[server.cors]
allow_origins = ["https://app.example.com"]
allow_methods = ["GET", "POST", "OPTIONS"]
allow_headers = ["authorization", "content-type"]
allow_credentials = true

```

--------------------------------

### Chat Completion with System Message (cURL)

Source: https://nexusrouter.com/docs/usage/llm-api

An example showing how to include a system message to guide the AI's behavior, along with specifying temperature and max tokens for the chat completion.

```bash
curl -X POST http://localhost:8000/llm/openai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-3-5-sonnet-20241022",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "Explain quantum computing in simple terms."}
    ],
    "temperature": 0.7,
    "max_tokens": 500
  }'
```

--------------------------------

### Nexus TOML Configuration for LLM and MCP Routing

Source: https://nexusrouter.com/docs/index

A TOML configuration file example for Nexus, detailing settings for LLM providers (OpenAI, Anthropic) and MCP servers (filesystem, GitHub). It shows how to enable features, set paths, configure API keys, and define models and servers.

```toml
# LLM configuration for AI model routing
[llm]
enabled = true

# Enable protocol endpoints
[llm.protocols.openai]
enabled = true
path = "/llm/openai"

[llm.protocols.anthropic]
enabled = true
path = "/llm/anthropic"

# Configure multiple AI providers
[llm.providers.openai]
type = "openai"
api_key = "{{ env.OPENAI_API_KEY }}"

# Models must be explicitly configured
[llm.providers.openai.models.gpt-4]
[llm.providers.openai.models."gpt-4o-mini"]

[llm.providers.anthropic]
type = "anthropic"
api_key = "{{ env.ANTHROPIC_API_KEY }}"

[llm.providers.anthropic.models."claude-3-5-sonnet-20241022"]

# MCP configuration for tool integration
[mcp]
enabled = true
path = "/mcp"

# Add a simple file system server
[mcp.servers.filesystem]
cmd = ["npx", "-y", "@modelcontextprotocol/server-filesystem", "/home/user/documents"]

# Add a GitHub MCP server
[mcp.servers.github]
url = "https://api.github.com/mcp"

[mcp.servers.github.auth]
token = "{{ env.GITHUB_TOKEN }}"
```

--------------------------------

### Curl Example for CSRF Token Header

Source: https://nexusrouter.com/docs/configuration/server/csrf

Demonstrates how to make a POST request using curl, including the necessary CSRF token in the 'X-CSRF-Token' header. This example verifies the server-side CSRF protection is active by checking for header presence.

```bash
curl -X POST http://localhost:8000/llm/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-CSRF-Token: any-value" \
  -d '{"model": "openai/gpt-4", "messages": [{"role": "user", "content": "Hello"}]}'
```

--------------------------------

### Python Client Example with Token Forwarding (OpenAI SDK)

Source: https://nexusrouter.com/docs/configuration/llm/token-forwarding

This Python code demonstrates how to configure the OpenAI SDK client to use Nexus with token forwarding. It shows setting the `base_url` to Nexus and passing the user's API key in the `default_headers` under `X-Provider-API-Key`.

```python
from openai import OpenAI

# Using your own API key
client = OpenAI(
    base_url="http://localhost:8000/llm/v1",
    api_key="not-used",  # Required by SDK but ignored
    default_headers={
        "X-Provider-API-Key": "sk-your-openai-key"
    }
)

response = client.chat.completions.create(
    model="openai/gpt-4",
    messages=[{"role": "user", "content": "Hello!"}]
)
```

--------------------------------

### Sequential Header Processing Example

Source: https://nexusrouter.com/docs/configuration/llm/providers

Illustrates the sequential processing order of header rules (insert, forward, remove) and how later rules can override earlier ones, enabling complex header transformation pipelines.

```toml
# 1. Insert a static header first
[[llm.providers.openai.headers]]
rule = "insert"
name = "x-api-version"
value = "v1"

# 2. Forward headers matching a pattern (will override existing ones!)
[[llm.providers.openai.headers]]
rule = "forward"
pattern = "^x-custom-"

# 3. Remove a specific header that may have been forwarded
[[llm.providers.openai.headers]]
rule = "remove"
name = "x-custom-internal"

# 4. Insert to ensure final value (overrides anything forwarded)
[[llm.providers.openai.headers]]
rule = "insert"
name = "x-custom-override"
value = "final-value"
```

--------------------------------

### TOML Pattern Matching Examples for Header Rules

Source: https://nexusrouter.com/docs/configuration/llm/header-rules

Illustrates the use of regular expressions (regex) with 'pattern' to match multiple headers for various rule types. Examples cover forwarding all custom headers, removing debug headers, and forwarding headers that do not match a specific prefix.

```toml
# Forward all custom headers
[[llm.providers.openai.headers]]
rule = "forward"
pattern = "^x-custom-"

# Remove all debug headers
[[llm.providers.openai.headers]]
rule = "remove"
pattern = "^x-debug-"

# Forward all headers except those starting with "internal-"
[[llm.providers.openai.headers]]
rule = "forward"
pattern = "^(?!internal-).*"
```

--------------------------------

### GET /llm/openai/v1/models

Source: https://nexusrouter.com/docs/configuration/llm/models

Retrieves a list of all explicitly configured models available through the Nexus instance.

```APIDOC
## GET /llm/openai/v1/models

### Description
Retrieves all explicitly configured models, including custom aliases, that are available through the Nexus instance. The response includes the model ID and its ownership.

### Method
GET

### Endpoint
`/llm/openai/v1/models`

### Parameters
None

### Request Example
None

### Response
#### Success Response (200)
- **object** (string) - Type of the response object, typically "list".
- **data** (array) - An array of model objects.
  - **id** (string) - The unique identifier for the model (e.g., "openai/gpt-4" or a custom alias like "openai/smart").
  - **object** (string) - The type of the model object, typically "model".
  - **owned_by** (string) - The provider or entity that owns the model.

#### Response Example
```json
{
  "object": "list",
  "data": [
    {
      "id": "openai/gpt-4",
      "object": "model",
      "owned_by": "openai"
    },
    {
      "id": "openai/smart",
      "object": "model",
      "owned_by": "openai"
    },
    {
      "id": "anthropic/fast",
      "object": "model",
      "owned_by": "anthropic"
    }
  ]
}
```
```

--------------------------------

### TOML Insert Header Rule Examples

Source: https://nexusrouter.com/docs/configuration/llm/header-rules

Illustrates the use of the 'insert' rule type to add static headers to all requests sent to LLM providers. Examples show adding version headers, API keys using environment variables, and provider-specific beta features.

```toml
# Add a static header
[[llm.providers.anthropic.headers]]
rule = "insert"
name = "x-api-version"
value = "2024-01"

# Use environment variables
[[llm.providers.anthropic.headers]]
rule = "insert"
name = "x-api-key"
value = "{{ env.SECONDARY_API_KEY }}"

# Add provider-specific features
[[llm.providers.openai.headers]]
rule = "insert"
name = "OpenAI-Beta"
value = "assistants=v2"
```

--------------------------------

### OpenAI Client Initialization for Nexus Router (Python/JavaScript)

Source: https://nexusrouter.com/docs/configuration/llm

Code snippets demonstrating how to initialize the OpenAI client library for use with the Nexus Router's OpenAI-compatible API endpoint. These examples show how to set the `base_url` and `api_key` for both Python and JavaScript clients.

```python
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/llm/openai/v1",
    api_key="not-used"
)

```

```javascript
import OpenAI from 'openai';

const openai = new OpenAI({
  baseURL: 'http://localhost:8000/llm/openai/v1',
  apiKey: 'not-used'
});

```

--------------------------------

### LLM API - Chat Completions

Source: https://nexusrouter.com/docs/usage

Makes a chat completion request to the LLM API, compatible with OpenAI's API.

```APIDOC
## POST /llm/v1/chat/completions

### Description
Generates a chat completion response based on the provided messages and model.

### Method
POST

### Endpoint
/llm/v1/chat/completions

### Parameters
#### Query Parameters
None

#### Request Body
- **model** (string) - Required - The model to use for generating completions (e.g., "openai/gpt-4").
- **messages** (array) - Required - A list of message objects representing the conversation history.
  - **role** (string) - Required - The role of the message sender (e.g., "user", "system", "assistant").
  - **content** (string) - Required - The content of the message.

### Request Example
```json
{
  "model": "openai/gpt-4",
  "messages": [{"role": "user", "content": "Hello!"}]
}
```

### Response
#### Success Response (200)
- **choices** (array) - A list of completion choices.
  - **message** (object) - The generated message.
    - **role** (string) - The role of the assistant.
    - **content** (string) - The content of the assistant's response.

#### Response Example
```json
{
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "Hi there! How can I help you today?"
      }
    }
  ]
}
```
```

--------------------------------

### Certificate Conversion and Viewing using OpenSSL

Source: https://nexusrouter.com/docs/configuration/mcp/tls

Provides command-line examples using OpenSSL for managing TLS certificates. Includes viewing certificate details, converting DER to PEM format, and converting PKCS12 files to PEM format.

```bash
# View certificate details
openssl x509 -in certificate.pem -text -noout

# Convert DER to PEM
openssl x509 -inform DER -in certificate.der -out certificate.pem

# Convert PKCS12 to PEM
openssl pkcs12 -in certificate.p12 -out certificate.pem -nodes
```

--------------------------------

### Custom HTTP Client Example for Token Forwarding (JavaScript)

Source: https://nexusrouter.com/docs/configuration/llm/token-forwarding

This JavaScript function `callNexusLLM` shows how to make a direct HTTP POST request to Nexus with a user-provided API key in the `X-Provider-API-Key` header. It's a flexible approach for integrating with Nexus without relying on specific SDKs.

```javascript
async function callNexusLLM(apiKey, model, messages) {
  const response = await fetch('http://localhost:8000/llm/v1/chat/completions', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'X-Provider-API-Key': apiKey  // User's own API key
    },
    body: JSON.stringify({ model, messages })
  });

  return response.json();
}

// User provides their own key
const result = await callNexusLLM(
  'sk-user-api-key',
  'openai/gpt-4',
  [{ role: 'user', content: 'Hello' }]
);
```

--------------------------------

### Defining Resource Intensive Rate Limits (TOML)

Source: https://nexusrouter.com/docs/configuration/mcp/rate-limiting

Example configurations for different types of resource-intensive operations. These define a limit and the interval over which that limit applies.

```toml
cpu_heavy = { limit = 10, interval = "60s" }

memory_heavy = { limit = 20, interval = "60s" }

io_heavy = { limit = 50, interval = "60s" }

network_heavy = { limit = 100, interval = "60s" }
```

--------------------------------

### Anthropic Client Initialization for Nexus Router (Python/JavaScript)

Source: https://nexusrouter.com/docs/configuration/llm

Code snippets demonstrating how to initialize the Anthropic client library for use with the Nexus Router's Anthropic-compatible API endpoint. These examples show how to set the `base_url` and `api_key` for both Python and JavaScript clients.

```python
from anthropic import Anthropic

client = Anthropic(
    base_url="http://localhost:8000/llm/anthropic",
    api_key="not-used"
)

```

```javascript
import Anthropic from '@anthropic-ai/sdk';

const anthropic = new Anthropic ({
  baseURL: 'http://localhost:8000/llm/anthropic',
  apiKey: 'not-used'
});

```

--------------------------------

### JavaScript Client Example with Token Forwarding (OpenAI SDK)

Source: https://nexusrouter.com/docs/configuration/llm/token-forwarding

This JavaScript code snippet illustrates configuring the OpenAI SDK for use with Nexus, enabling token forwarding. The `apiKey` is set to 'not-used' while the actual user key is provided in `defaultHeaders` using the `X-Provider-API-Key` header.

```javascript
import OpenAI from 'openai';

const openai = new OpenAI({
  baseURL: 'http://localhost:8000/llm/v1',
  apiKey: 'not-used',  // Required by SDK but ignored
  defaultHeaders: {
    'X-Provider-API-Key': 'sk-your-openai-key'
  }
});

const completion = await openai.chat.completions.create({
  model: 'openai/gpt-4',
  messages: [{ role: 'user', content: 'Hello!' }]
});
```

--------------------------------

### Organize Models by Use Case

Source: https://nexusrouter.com/docs/configuration/llm/models

Provides examples of configuring models for specific use cases like customer support, content generation, code review, and translation. This involves aliasing models to descriptive names relevant to their intended application.

```toml
# Customer support
[llm.providers.openai.models.support]
rename = "gpt-3.5-turbo"  # Fast and cost-effective

# Content generation
[llm.providers.anthropic.models.writer]
rename = "claude-3-5-sonnet-20241022"  # Excellent writing capabilities

# Code review
[llm.providers.openai.models.reviewer]
rename = "gpt-4"  # Strong reasoning for code analysis

# Translation
[llm.providers.google.models.translator]
rename = "gemini-1.5-pro"  # Good multilingual support
```

--------------------------------

### Progressive Rate Limit Configuration (TOML)

Source: https://nexusrouter.com/docs/configuration/mcp/rate-limiting

Illustrates a strategy for starting with conservative rate limits and then increasing them based on observed usage patterns and monitoring. This promotes stability during initial deployment.

```toml
# Start conservative
[mcp.servers.new_api.rate_limits]
limit = 10
interval = "60s"

# Increase after monitoring
[mcp.servers.new_api.rate_limits]
limit = 100  # Increased after testing
interval = "60s"
```

--------------------------------

### Environment Separation Configuration (TOML)

Source: https://nexusrouter.com/docs/best-practices

Demonstrates how to configure Nexus server settings for different environments, specifically separating development and production listening addresses and TLS certificate paths. This pattern ensures that sensitive production configurations are distinct from development setups.

```toml
# Development
[server]
listen_address = "127.0.0.1:8000"

# Production
[server]
listen_address = "0.0.0.0:443"
[server.tls]
certificate = "/etc/nexus/cert.pem"
key = "/etc/nexus/key.pem"

```

--------------------------------

### Nexus Server Configuration with Environment Variables (TOML)

Source: https://nexusrouter.com/docs/configuration/server

An example of Nexus server configuration using TOML, specifically demonstrating how to substitute environment variables for sensitive or dynamic settings like OAuth2 JWKS URL and issuer. This allows for flexible configuration management in different environments.

```toml
[server.oauth]
url = "{{ env.OAUTH_JWKS_URL }}"
expected_issuer = "{{ env.OAUTH_ISSUER }}"

```

--------------------------------

### Comprehensive Nexus Router Rate Limiting Example (TOML)

Source: https://nexusrouter.com/docs/configuration/llm/rate-limiting

A complete configuration demonstrating various rate limiting strategies, including provider defaults, group-based tiers, and model-specific overrides for both OpenAI and Anthropic providers. It highlights the hierarchical application of rules.

```toml
# OpenAI provider with tiered rate limits
[llm.providers.openai]
type = "openai"
api_key = "{{ env.OPENAI_API_KEY }}"

# Configure models
[llm.providers.openai.models."gpt-4"]
[llm.providers.openai.models."gpt-3.5-turbo"]

# Default rate limits for all OpenAI models
[llm.providers.openai.rate_limits.per_user]
input_token_limit = 50000   # 50K input tokens
interval = "60s"

# Free tier limits for all OpenAI models
[llm.providers.openai.rate_limits.per_user.groups.free]
input_token_limit = 10000   # 10K input tokens
interval = "60s"

# Pro tier limits for all OpenAI models
[llm.providers.openai.rate_limits.per_user.groups.pro]
input_token_limit = 100000  # 100K input tokens
interval = "60s"

# Enterprise tier limits for all OpenAI models
[llm.providers.openai.rate_limits.per_user.groups.enterprise]
input_token_limit = 1000000 # 1M input tokens
interval = "60s"

# GPT-4 specific limits (more restrictive)
[llm.providers.openai.rate_limits."gpt-4".per_user]
input_token_limit = 25000   # 25K input tokens
interval = "60s"

# GPT-4 enterprise tier gets special treatment
[llm.providers.openai.rate_limits."gpt-4".per_user.groups.enterprise]
input_token_limit = 500000  # 500K input tokens
interval = "60s"

# Anthropic provider with simpler limits
[llm.providers.anthropic]
type = "anthropic"
api_key = "{{ env.ANTHROPIC_API_KEY }}"

[llm.providers.anthropic.models."claude-3-5-sonnet-20241022"]
[llm.providers.anthropic.models."claude-3-opus-20240229"]
```

--------------------------------

### List Models via API

Source: https://nexusrouter.com/docs/configuration/llm/models

Demonstrates how to retrieve a list of all explicitly configured models from the Nexus Router instance using an HTTP GET request to the `/llm/openai/v1/models` endpoint. The response is a JSON object containing model details.

```http
GET /llm/openai/v1/models
```

```json
{
  "object": "list",
  "data": [
    {
      "id": "openai/gpt-4",
      "object": "model",
      "owned_by": "openai"
    },
    {
      "id": "openai/smart",  // Custom alias
      "object": "model",
      "owned_by": "openai"
    },
    {
      "id": "anthropic/fast",  // Custom alias
      "object": "model",
      "owned_by": "anthropic"
    }
  ]
}
```

--------------------------------

### AWS Bedrock Semantic Aliases Configuration (TOML)

Source: https://nexusrouter.com/docs/configuration/llm/models

This TOML example demonstrates configuring the AWS Bedrock provider and setting up semantic aliases for different use cases. Aliases like 'chat', 'fast', 'powerful', and 'coding' are mapped to specific underlying models, simplifying API calls.

```toml
[llm.providers.bedrock]
type = "bedrock"
region = "us-east-1"

# Semantic aliases for different use cases
[llm.providers.bedrock.models.chat]
rename = "anthropic.claude-3-sonnet-20240229-v1:0"

[llm.providers.bedrock.models.fast]
rename = "anthropic.claude-3-haiku-20240307-v1:0"

[llm.providers.bedrock.models.powerful]
rename = "anthropic.claude-3-opus-20240229-v1:0"

[llm.providers.bedrock.models.coding]
rename = "meta.llama3-1-70b-instruct-v1:0"
```

--------------------------------

### Testing TLS Configuration with OpenSSL and Curl

Source: https://nexusrouter.com/docs/configuration/mcp/tls

A collection of bash commands using OpenSSL and curl to test TLS configurations. These commands help diagnose certificate issues, test CA verification, and verify mutual TLS setups.

```bash
# Test server certificate
openssl s_client -connect api.example.com:443 -servername api.example.com

# Test with custom CA
openssl s_client -connect api.example.com:443 \
  -CAfile /etc/nexus/certs/ca.pem

# Test mutual TLS
openssl s_client -connect api.example.com:443 \
  -cert client.pem -key client-key.pem

# Test with curl
curl --cacert ca.pem --cert client.pem --key client-key.pem \
  https://api.example.com/mcp

```

--------------------------------

### Send Request with User API Key using cURL (OpenAI)

Source: https://nexusrouter.com/docs/configuration/llm/token-forwarding

Example using cURL to send a chat completion request to Nexus, providing a user's OpenAI API key via the `X-Provider-API-Key` header. This demonstrates how a user can authenticate their request when token forwarding is enabled.

```bash
curl -X POST http://localhost:8000/llm/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-Provider-API-Key: sk-your-openai-key" \
  -d '{
    "model": "openai/gpt-4",
    "messages": [{"role": "user", "content": "Hello"}]
  }'
```

--------------------------------

### TOML Forward Header Rule Examples

Source: https://nexusrouter.com/docs/configuration/llm/header-rules

Demonstrates how to use the 'forward' rule type to pass headers from client requests to LLM providers. This includes forwarding specific headers, renaming them, providing default values, and forwarding headers based on a regex pattern.

```toml
# Forward a specific header
[[llm.providers.openai.headers]]
rule = "forward"
name = "x-request-id"

# Forward with rename and default value
[[llm.providers.openai.headers]]
rule = "forward"
name = "x-trace-id"
rename = "provider-trace-id"  # Optional: rename the header
default = "{{ env.DEFAULT_TRACE }}"  # Optional: default if not present

# Forward headers matching a pattern
[[llm.providers.openai.headers]]
rule = "forward"
pattern = "^x-org-"  # Forward all headers starting with "x-org-"
```
Tokens:
Raw