VoidLLM

https://voidllm.ai/docs
Admin
VoidLLM is a privacy-first LLM proxy and AI gateway, designed to be self-hosted with a single binary...
Tokens:23,178
Snippets:320
Trust Score:2.3
Update:3 months ago
Show doc for...
Tokens:
Raw
### Minimal VoidLLM Docker Setup

Source: https://voidllm.ai/docs/deployment/docker

Run VoidLLM with essential environment variables and a named volume for data persistence. Ensure you save the bootstrap credentials printed on first start.

```bash
export VOIDLLM_ADMIN_KEY=$(openssl rand -base64 32)
export VOIDLLM_ENCRYPTION_KEY=$(openssl rand -base64 32)

docker run -d --name voidllm \
  -p 8080:8080 \
  -e VOIDLLM_ADMIN_KEY -e VOIDLLM_ENCRYPTION_KEY \
  -v voidllm_data:/data \
  ghcr.io/voidmind-io/voidllm:latest
```

--------------------------------

### VoidLLM Configuration Example

Source: https://voidllm.ai/docs/deployment/binary

An example voidllm.yaml file demonstrating server proxy settings, model configuration with Ollama, and environment variable injection for secrets.

```yaml
server:
  proxy:
    port: 8080

models:
  - name: my-model
    provider: ollama
    base_url: http://localhost:11434/v1

settings:
  admin_key: ${VOIDLLM_ADMIN_KEY}
  encryption_key: ${VOIDLLM_ENCRYPTION_KEY}
```

--------------------------------

### Basic VoidLLM Installation with Helm

Source: https://voidllm.ai/docs/deployment/kubernetes

Installs VoidLLM using the official Helm chart. Ensure you have Helm v3+ installed and the repository added.

```bash
helm repo add voidllm https://voidmind-io.github.io/voidllm
helm repo update

helm install voidllm voidllm/voidllm \
  --set secrets.adminKey=$(openssl rand -base64 32) \
  --set secrets.encryptionKey=$(openssl rand -base64 32)
```

--------------------------------

### VoidLLM Docker Compose Deployment

Source: https://voidllm.ai/docs/deployment/docker

Set up VoidLLM using Docker Compose. This involves copying an example configuration, editing it, setting environment variables, and starting the service.

```bash
cp voidllm.yaml.example voidllm.yaml
# Edit voidllm.yaml - configure your models

export VOIDLLM_ADMIN_KEY=$(openssl rand -base64 32)
export VOIDLLM_ENCRYPTION_KEY=$(openssl rand -base64 32)
docker-compose up -d
```

--------------------------------

### Set Up Secrets and Run VoidLLM on Linux/macOS

Source: https://voidllm.ai/docs/deployment/binary

Generate the required admin and encryption keys using openssl and then start the VoidLLM binary.

```bash
export VOIDLLM_ADMIN_KEY=$(openssl rand -base64 32)
export VOIDLLM_ENCRYPTION_KEY=$(openssl rand -base64 32)
./voidllm
```

--------------------------------

### VoidLLM Installation with PostgreSQL

Source: https://voidllm.ai/docs/deployment/kubernetes

Installs VoidLLM with an integrated PostgreSQL database. The chart manages PostgreSQL setup automatically.

```bash
helm install voidllm voidllm/voidllm \
  --set postgresql.enabled=true \
  --set postgresql.auth.password=$(openssl rand -base64 16) \
  --set secrets.adminKey=$(openssl rand -base64 32) \
  --set secrets.encryptionKey=$(openssl rand -base64 32)
```

--------------------------------

### Install VoidLLM with License Secret in Kubernetes

Source: https://voidllm.ai/docs/enterprise/license

Install VoidLLM using Helm, providing the license key as a secret.

```bash
helm install voidllm voidllm/voidllm \
  --set secrets.license="eyJhbGciOiJFZERTQSIs..."
```

--------------------------------

### VoidLLM Docker Setup with Config File

Source: https://voidllm.ai/docs/deployment/docker

Deploy VoidLLM using a configuration file mounted into the container. This allows for more detailed customization via YAML settings.

```bash
docker run -d --name voidllm \
  -p 8080:8080 \
  -e VOIDLLM_ADMIN_KEY -e VOIDLLM_ENCRYPTION_KEY \
  -v $(pwd)/voidllm.yaml:/etc/voidllm/voidllm.yaml:ro \
  -v voidllm_data:/data \
  ghcr.io/voidmind-io/voidllm:latest
```

--------------------------------

### Configure PostgreSQL Database Connection

Source: https://voidllm.ai/docs/configuration

Example configuration for a PostgreSQL database, including driver and DSN. Ensure the DB_PASSWORD environment variable is set if using this.

```yaml
# PostgreSQL example:
# driver: postgres
# dsn: postgres://user:${DB_PASSWORD}@host:5432/voidllm?sslmode=require
```

--------------------------------

### Set Up Secrets and Run VoidLLM on Windows (PowerShell)

Source: https://voidllm.ai/docs/deployment/binary

Generate the required admin and encryption keys using PowerShell's random byte generation and then start the VoidLLM executable.

```powershell
$env:VOIDLLM_ADMIN_KEY = [Convert]::ToBase64String((1..32 | ForEach-Object { Get-Random -Max 256 }) -as [byte[]])
$env:VOIDLLM_ENCRYPTION_KEY = [Convert]::ToBase64String((1..32 | ForEach-Object { Get-Random -Max 256 }) -as [byte[]])
.
\voidllm.exe
```

--------------------------------

### Run VoidLLM with Binary (Linux)

Source: https://voidllm.ai/docs/getting-started

Download and run the VoidLLM binary on Linux. Ensure you set the encryption and admin keys as environment variables before starting the application.

```bash
# Linux (amd64)
curl -sL https://github.com/voidmind-io/voidllm/releases/latest/download/voidllm-linux-amd64.tar.gz | tar xz
export VOIDLLM_ADMIN_KEY=$(openssl rand -base64 32)
export VOIDLLM_ENCRYPTION_KEY=$(openssl rand -base64 32)
./voidllm
```

--------------------------------

### VoidLLM Installation with Redis and PostgreSQL

Source: https://voidllm.ai/docs/deployment/kubernetes

Installs VoidLLM with both PostgreSQL for state and Redis for distributed rate limiting and cache invalidation. Requires an Enterprise license.

```bash
helm install voidllm voidllm/voidllm \
  --set postgresql.enabled=true \
  --set postgresql.auth.password=$(openssl rand -base64 16) \
  --set redis.enabled=true \
  --set replicaCount=3 \
  --set secrets.license="eyJ..." \
  --set secrets.adminKey=$(openssl rand -base64 32) \
  --set secrets.encryptionKey=$(openssl rand -base64 32)
```

--------------------------------

### Example JavaScript Script for Tool Orchestration

Source: https://voidllm.ai/docs/mcp/code-mode

This script demonstrates fetching data from two different MCP servers using available tools and logging the results. Tools are accessed via the `tools` object as async functions.

```javascript
// Fetch data from two different MCP servers
const docs = await tools.aws.search({ query: "IAM best practices" })
const related = await tools.exa.search({ query: docs[0].title })

console.log(`Found ${docs.length} docs and ${related.length} related results`)
return { docs: docs.slice(0, 3), related: related.slice(0, 5) }
```

--------------------------------

### Check VoidLLM Bootstrap Credentials

Source: https://voidllm.ai/docs/deployment/kubernetes

Retrieves the bootstrap credentials from the VoidLLM pod logs after installation.

```bash
kubectl logs deploy/voidllm | grep "BOOTSTRAP"
```

--------------------------------

### VoidLLM Bootstrap Credentials

Source: https://voidllm.ai/docs/getting-started

On first start, VoidLLM displays essential credentials including an API key, admin email, and password. Save these securely as they are shown only once.

```text
========================================
 BOOTSTRAP COMPLETE - COPY THESE NOW
========================================
  API Key:    vl_uk_a3f2...
  Email:      admin@voidllm.local
  Password:   <random>
========================================

```

--------------------------------

### Check Container Logs for Bootstrap Credentials

Source: https://voidllm.ai/docs/troubleshooting

Retrieve bootstrap credentials from container logs. This is useful if you missed them on the initial start.

```bash
docker logs voidllm | grep "BOOTSTRAP"
```

```bash
kubectl logs deploy/voidllm | grep "BOOTSTRAP"
```

--------------------------------

### VoidLLM Installation with External PostgreSQL

Source: https://voidllm.ai/docs/deployment/kubernetes

Configures VoidLLM to use an external PostgreSQL database, enabling encrypted connections via a custom DSN.

```bash
helm install voidllm voidllm/voidllm \
  --set config.database.driver=postgres \
  --set config.database.dsn="postgres://user:pass@external-db:5432/voidllm?sslmode=require"
```

--------------------------------

### Configure Redis for Multi-Instance Deployments

Source: https://voidllm.ai/docs/configuration

Enable Redis for distributed rate limiting and cache invalidation in multi-instance setups. Single-instance deployments do not require this.

```yaml
redis:
  enabled: false
  url: redis://:${REDIS_PASSWORD}@redis:6379/0
  key_prefix: voidllm:
```

--------------------------------

### Configure Multi-Deployment Model

Source: https://voidllm.ai/docs/models/load-balancing

Define a model with multiple deployments and specify a routing strategy. This setup allows VoidLLM to manage traffic distribution and failover between different deployment endpoints.

```yaml
models:
  - name: gpt-4o
    strategy: round-robin
    max_retries: 2
    aliases: [default]
    deployments:
      - name: azure-east
        provider: azure
        base_url: https://eastus.openai.azure.com
        api_key: ${AZURE_EAST_KEY}
        azure_deployment: my-gpt4o-east
      - name: azure-west
        provider: azure
        base_url: https://westus.openai.azure.com
        api_key: ${AZURE_WEST_KEY}
        azure_deployment: my-gpt4o-west
      - name: openai-direct
        provider: openai
        base_url: https://api.openai.com/v1
        api_key: ${OPENAI_KEY}
```

--------------------------------

### Reload and Enable systemd Service

Source: https://voidllm.ai/docs/deployment/binary

Commands to reload the systemd daemon after creating or modifying a service file and to enable and start the service immediately.

```bash
sudo systemctl daemon-reload
sudo systemctl enable --now voidllm
```

--------------------------------

### Configure PostgreSQL Database

Source: https://voidllm.ai/docs/deployment/database

Use this configuration when running production deployments with high write throughput or multi-instance setups. Ensure the DB_PASSWORD environment variable is set.

```yaml
database:
  driver: postgres
  dsn: postgres://voidllm:${DB_PASSWORD}@postgres:5432/voidllm?sslmode=require
  max_open_conns: 25
  max_idle_conns: 5
  conn_max_lifetime: 5m
```

--------------------------------

### Register External MCP Server via YAML

Source: https://voidllm.ai/docs/configuration

Declare external MCP servers directly in your YAML configuration. This example registers a GitHub MCP server with a bearer token authentication. Servers defined here are synced to the database on startup.

```yaml
mcp_servers:
  - name: GitHub
    alias: github
    url: https://mcp.github.com/sse
    auth_type: bearer
    auth_token: ${GITHUB_TOKEN}
```

--------------------------------

### Send First Chat Completion Request

Source: https://voidllm.ai/docs/getting-started

Send a chat completion request to the VoidLLM API using cURL. This example demonstrates how to authenticate and specify the model and message content.

```bash
curl http://localhost:8080/v1/chat/completions \
  -H "Authorization: Bearer vl_uk_your_key_here" \
  -H "Content-Type: application/json" \
  -d '{"model": "default", "messages": [{"role": "user", "content": "hello"}]}'
```

--------------------------------

### MCP Configuration Settings

Source: https://voidllm.ai/docs/configuration

Define MCP-specific settings in your configuration, such as call timeouts, private URL allowances, and health check configurations. This example shows how to set a call timeout and enable health probing.

```yaml
settings:
  mcp:
    call_timeout: 30s            # Max duration per proxied tool call (default: 30s)
    allow_private_urls: false     # Allow localhost/private IPs for MCP server URLs
    health:
      enabled: true              # Enable MCP server health probing (default: true)
      interval: 60s              # Probe interval (default: 60s)
```

--------------------------------

### Configure General Settings

Source: https://voidllm.ai/docs/configuration

Set up essential parameters such as admin keys, encryption keys, license information, and initial bootstrap details for your organization. Configure usage logging and token counting behavior.

```yaml
settings:
  # Required: bootstrap admin key (≥32 chars)
  admin_key: ${VOIDLLM_ADMIN_KEY}

  # Required: AES-256-GCM encryption key (base64, 32 bytes)
  # Generate: openssl rand -base64 32
  encryption_key: ${VOIDLLM_ENCRYPTION_KEY}

  # Enterprise license key
  license: ${VOIDLLM_LICENSE}
  # Or as a file path:
  # license_file: /etc/voidllm/license.jwt

  # First-run bootstrap
  bootstrap:
    org_name: "My Company"
    org_slug: "my-company"          # Auto-derived from name if empty
    admin_email: "admin@company.com"

  # Usage logging
  usage:
    buffer_size: 1000               # Events buffered before flush
    flush_interval: 5s
    drop_on_full: true              # Drop events when buffer full (never blocks proxy)

  # Token counting
  token_counting:
    enabled: true

  # Soft limit warning threshold
  soft_limit_threshold: 0.9        # Warn at 90% of limit

```

--------------------------------

### Per-Model Timeout Configuration

Source: https://voidllm.ai/docs/models/providers

Override the global timeout for specific models. This example sets a 2-minute timeout for a slow model.

```yaml
models:
  - name: slow-model
    provider: custom
    base_url: https://slow-provider.com/v1
    timeout: 120s           # this model gets 2 minutes instead of the global default
```

--------------------------------

### Enable SSO / OIDC

Source: https://voidllm.ai/docs/deployment/kubernetes

Configures Single Sign-On (SSO) using OpenID Connect (OIDC) for VoidLLM Enterprise. Requires a license key and specific issuer/client details.

```bash
--set config.settings.sso.enabled=true \
--set config.settings.sso.issuer=https://accounts.google.com \
--set config.settings.sso.clientId=xxx \
--set secrets.ssoClientSecret=yyy \
--set config.settings.sso.redirectUrl=https://voidllm.company.com/api/v1/auth/oidc/callback
```

--------------------------------

### Verify License via CLI

Source: https://voidllm.ai/docs/enterprise/license

Use the VoidLLM CLI to verify the license key provided via standard input.

```bash
voidllm license verify < license.jwt
```

--------------------------------

### Configure Built-in VoidLLM MCP Server for IDE

Source: https://voidllm.ai/docs/configuration

Set up the built-in VoidLLM MCP server for IDE integration. Ensure you replace 'your-voidllm-instance' and 'vl_uk_your_key' with your actual details.

```json
{
  "mcpServers": {
    "voidllm": {
      "type": "http",
      "url": "https://your-voidllm-instance/api/v1/mcp/voidllm",
      "headers": {
        "Authorization": "Bearer vl_uk_your_key"
      }
    }
  }
}
```

--------------------------------

### Configure External MCP Server Proxy for IDE

Source: https://voidllm.ai/docs/configuration

Configure an external MCP server, like GitHub, to be proxied through VoidLLM for IDE use. Replace placeholders with your specific instance and key.

```json
{
  "mcpServers": {
    "github": {
      "type": "http",
      "url": "https://your-voidllm-instance/api/v1/mcp/github",
      "headers": {
        "Authorization": "Bearer vl_uk_your_key"
      }
    }
  }
}
```

--------------------------------

### Run VoidLLM with Docker

Source: https://voidllm.ai/docs/getting-started

Deploy VoidLLM using Docker, mapping ports and volumes, and setting essential environment variables for encryption and admin access.

```bash
docker run -p 8080:8080 \
  -v voidllm_data:/data \
  -e VOIDLLM_ENCRYPTION_KEY=$(openssl rand -base64 32) \
  -e VOIDLLM_ADMIN_KEY=my-admin-key-at-least-32-chars!! \
  ghcr.io/voidmind-io/voidllm:latest
```

--------------------------------

### Configure IDE for LLM Proxy

Source: https://voidllm.ai/docs/getting-started

Set up your IDE to use VoidLLM as an LLM proxy by configuring the base URL and API key.

```text
Base URL: http://localhost:8080/v1
API Key: vl_uk_...
```

--------------------------------

### Enable Audit Logging

Source: https://voidllm.ai/docs/deployment/kubernetes

Enables audit logging for VoidLLM Enterprise. Requires a valid license key.

```bash
--set secrets.license="eyJ..." \
--set config.settings.audit.enabled=true
```

--------------------------------

### Enable OpenTelemetry Tracing

Source: https://voidllm.ai/docs/deployment/kubernetes

Enables OpenTelemetry tracing for VoidLLM Enterprise, sending traces to a specified endpoint. Requires a license key.

```bash
--set config.settings.otel.enabled=true \
--set config.settings.otel.endpoint=tempo:4317
```

--------------------------------

### Error Response Structure

Source: https://voidllm.ai/docs/api/overview

API errors are returned in a consistent JSON format. This example shows a 'not_found' error, including the error code, a descriptive message, and a unique request ID.

```json
{
  "error": {
    "code": "not_found",
    "message": "model not found",
    "request_id": "019d3ba2-8130-7d96-b60e-2e0da53e2650"
  }
}
```

--------------------------------

### Configure Server Proxy Settings

Source: https://voidllm.ai/docs/configuration

Define the server proxy port and various timeouts for client connections. Adjust timeouts based on expected response times, especially for streaming.

```yaml
server:
  proxy:
    port: 8080              # Proxy port — LLM clients connect here
    read_timeout: 30s
    write_timeout: 120s     # High for streaming responses
    idle_timeout: 60s
    drain_timeout: 25s      # Graceful shutdown drain window (5s–120s)
```

--------------------------------

### Download VoidLLM Binary for Linux (amd64)

Source: https://voidllm.ai/docs/deployment/binary

Use curl to download and extract the VoidLLM binary for standard Linux amd64 architectures.

```bash
curl -sL https://github.com/voidmind-io/voidllm/releases/latest/download/voidllm-linux-amd64.tar.gz | tar xz
```

--------------------------------

### Example Log Entry with Trace IDs

Source: https://voidllm.ai/docs/enterprise/otel

When tracing is active, log entries automatically include trace_id and span_id for easy correlation with distributed traces in tools like Grafana Loki or Elasticsearch.

```json
{
  "time": "2026-04-01T12:00:00Z",
  "level": "INFO",
  "msg": "proxy request",
  "trace_id": "abc123...",
  "span_id": "def456...",
  "model": "gpt-4o",
  "status": 200
}
```

--------------------------------

### Configure Datadog Backend

Source: https://voidllm.ai/docs/enterprise/otel

Enable OpenTelemetry tracing and set the OTLP gRPC endpoint for Datadog. Ensure 'insecure' is set appropriately for TLS configuration. Datadog requires OTLP ingest to be enabled on the Datadog Agent.

```yaml
settings:
  otel:
    enabled: true
    endpoint: "datadog-agent:4317"
    insecure: true
```

--------------------------------

### Update VoidLLM Binary on Linux/macOS

Source: https://voidllm.ai/docs/deployment/binary

Download the latest binary for Linux/macOS and restart the service. The database is preserved across updates.

```bash
# Linux/macOS
curl -sL https://github.com/voidmind-io/voidllm/releases/latest/download/voidllm-linux-amd64.tar.gz | tar xz
# Restart the service
```

--------------------------------

### Download VoidLLM Binary for Linux (ARM64)

Source: https://voidllm.ai/docs/deployment/binary

Use curl to download and extract the VoidLLM binary for ARM64 architectures like Raspberry Pi or AWS Graviton.

```bash
curl -sL https://github.com/voidmind-io/voidllm/releases/latest/download/voidllm-linux-arm64.tar.gz | tar xz
```

--------------------------------

### Enable Audit Logging

Source: https://voidllm.ai/docs/configuration

Configure audit logging by enabling the feature and setting buffer size and flush interval. Requires an 'audit_logs' license feature.

```yaml
settings:
  audit:
    enabled: true
    buffer_size: 500
    flush_interval: 5s
```

--------------------------------

### Google Workspace SSO Configuration (YAML)

Source: https://voidllm.ai/docs/enterprise/sso

Configure VoidLLM to use Google Workspace as the OIDC provider. Ensure the redirect URI matches the one registered in Google Cloud Console.

```yaml
settings:
  sso:
    enabled: true
    issuer: "https://accounts.google.com"
    client_id: ${VOIDLLM_SSO_CLIENT_ID}
    client_secret: ${VOIDLLM_SSO_CLIENT_SECRET}
    redirect_url: "https://your-voidllm.com/api/v1/auth/oidc/callback"
    allowed_domains: ["yourcompany.com"]
```

--------------------------------

### Configure OpenTelemetry Tracing

Source: https://voidllm.ai/docs/configuration

Set up OpenTelemetry for distributed tracing. Specify the OTLP endpoint, whether to use TLS, and the trace sample rate. Requires the 'otel_tracing' license feature.

```yaml
settings:
  otel:
    enabled: true
    endpoint: "tempo:4317"   # OTLP gRPC endpoint
    insecure: true           # Set false for TLS
    sample_rate: 1.0         # 0.0 = no traces, 1.0 = all traces
```

--------------------------------

### Configure SSO / OIDC Integration

Source: https://voidllm.ai/docs/configuration

Enable and configure Single Sign-On using OpenID Connect. This includes setting the issuer, client ID, client secret, redirect URL, and other parameters. Requires the 'sso_oidc' license feature.

```yaml
settings:
  sso:
    enabled: true
    issuer: "https://accounts.google.com"
    client_id: ${VOIDLLM_SSO_CLIENT_ID}
    client_secret: ${VOIDLLM_SSO_CLIENT_SECRET}
    redirect_url: "https://voidllm.company.com/api/v1/auth/oidc/callback"
    scopes: ["openid", "email", "profile"]
    allowed_domains: ["company.com"]
    default_role: "member"
    auto_provision: true
    group_sync: false
    group_claim: "groups"
```

--------------------------------

### Configure OpenAI Model in voidllm.yaml

Source: https://voidllm.ai/docs/getting-started

Define an OpenAI model named 'gpt-4o' in your voidllm.yaml configuration file, specifying its provider, base URL, API key, and an alias.

```yaml
models:
  - name: gpt-4o
    provider: openai
    base_url: https://api.openai.com/v1
    api_key: ${OPENAI_KEY}
    aliases: [default]
```

--------------------------------

### Download VoidLLM Binary for macOS (Intel)

Source: https://voidllm.ai/docs/deployment/binary

Use curl to download and extract the VoidLLM binary for macOS on Intel-based processors.

```bash
# Intel
curl -sL https://github.com/voidmind-io/voidllm/releases/latest/download/voidllm-darwin-amd64.tar.gz | tar xz
```

--------------------------------

### Configure Grafana Tempo Backend

Source: https://voidllm.ai/docs/enterprise/otel

Enable OpenTelemetry tracing and set the OTLP gRPC endpoint for Grafana Tempo. Ensure 'insecure' is set appropriately for TLS configuration.

```yaml
settings:
  otel:
    enabled: true
    endpoint: "tempo:4317"
    insecure: true
```

--------------------------------

### Download VoidLLM Binary for macOS (Apple Silicon)

Source: https://voidllm.ai/docs/deployment/binary

Use curl to download and extract the VoidLLM binary for macOS on Apple Silicon (M1/M2/M3) chips.

```bash
# Apple Silicon (M1/M2/M3)
curl -sL https://github.com/voidmind-io/voidllm/releases/latest/download/voidllm-darwin-arm64.tar.gz | tar xz
```

--------------------------------

### Configure Optional Server Admin Port

Source: https://voidllm.ai/docs/configuration

Set up a separate port for the Admin API and UI. If set to 0, it defaults to using the main proxy port.

```yaml
  admin:
    port: 0                 # 0 = everything on proxy port (default)
    tls:
      enabled: false
      cert: /certs/tls.crt
      key: /certs/tls.key
```

--------------------------------

### Enable Audit Logging Settings

Source: https://voidllm.ai/docs/enterprise/audit

Configure audit logging by setting 'enabled' to true in the settings. Adjust buffer size and flush interval as needed.

```yaml
settings:
  audit:
    enabled: true
    buffer_size: 500      # events buffered before flush (default: 500)
    flush_interval: 5s    # flush to DB at least every N seconds
```

--------------------------------

### Generate Admin Key

Source: https://voidllm.ai/docs/troubleshooting

Generate a secure admin key for VoidLLM. Ensure the key is at least 32 characters long.

```bash
export VOIDLLM_ADMIN_KEY=$(openssl rand -base64 32)
```

--------------------------------

### Configure Jaeger Backend

Source: https://voidllm.ai/docs/enterprise/otel

Enable OpenTelemetry tracing and set the OTLP gRPC endpoint for Jaeger. Ensure 'insecure' is set appropriately for TLS configuration.

```yaml
settings:
  otel:
    enabled: true
    endpoint: "jaeger:4317"
    insecure: true
```

--------------------------------

### Caddy Reverse Proxy Configuration

Source: https://voidllm.ai/docs/deployment/reverse-proxy

Set up Caddy as a reverse proxy for VoidLLM. Caddy automatically handles TLS certificates.

```caddy
voidllm.example.com {
    reverse_proxy voidllm:8080
}
```

--------------------------------

### Backup PostgreSQL Database

Source: https://voidllm.ai/docs/deployment/database

Use the pg_dump utility to create a backup of your PostgreSQL database.

```bash
pg_dump -U voidllm -h postgres voidllm > backup.sql
```

--------------------------------

### Configure License in Settings File

Source: https://voidllm.ai/docs/enterprise/license

Specify the license path in the settings configuration file. This can be a direct JWT or a reference to an environment variable.

```yaml
settings:
  license: ${VOIDLLM_LICENSE}
```

```yaml
settings:
  license_file: /etc/voidllm/license.jwt
```

--------------------------------

### Enable OpenTelemetry Tracing

Source: https://voidllm.ai/docs/enterprise/otel

Enable OpenTelemetry tracing and configure the OTLP gRPC endpoint. Set 'insecure' to true for TLS disabled, or false for TLS enabled. Adjust 'sample_rate' to control the volume of traces.

```yaml
settings:
  otel:
    enabled: true
    endpoint: "tempo:4317"        # OTLP gRPC endpoint
    insecure: true                # set to false for TLS
    sample_rate: 1.0              # 1.0 = trace every request
```

--------------------------------

### Configure Load Balancing for GPT-4o

Source: https://voidllm.ai/docs/configuration

Set up multiple deployments for a single model name to enable load balancing and redundancy. Supports strategies like round-robin, least-latency, weighted, and priority.

```yaml
models:
  - name: gpt-4o
    strategy: round-robin        # round-robin, least-latency, weighted, priority
    max_retries: 2               # Default: number of deployments - 1
    aliases: [smart]
    max_context_tokens: 128000
    pricing:
      input_per_1m: 2.50
      output_per_1m: 10.00
    deployments:
      - name: azure-east
        provider: azure
        base_url: https://eastus.openai.azure.com
        api_key: ${AZURE_EAST_KEY}
        azure_deployment: my-gpt4o-deployment   # your Azure deployment name, not the model name
        weight: 2                # For weighted strategy
        priority: 1              # For priority strategy (lower = higher)
      - name: azure-west
        provider: azure
        base_url: https://westus.openai.azure.com
        api_key: ${AZURE_WEST_KEY}
        azure_deployment: my-gpt4o-deployment   # your Azure deployment name, not the model name
        weight: 1
        priority: 2
      - name: openai-fallback
        provider: openai
        base_url: https://api.openai.com/v1
        api_key: ${OPENAI_KEY}
        weight: 1
        priority: 3
```

--------------------------------

### VoidLLM Docker Persistence with Bind Mount

Source: https://voidllm.ai/docs/deployment/docker

Configure VoidLLM to use a local directory for data persistence via a bind mount. This makes the database file accessible on the host system for easier management.

```bash
docker run -p 8080:8080 \
  -v $(pwd)/data:/data \
  ...
```

--------------------------------

### Istio Configuration for VoidLLM

Source: https://voidllm.ai/docs/deployment/kubernetes

Basic Istio configuration to enable traffic management and TLS for VoidLLM. This snippet should be merged into your Istio configuration.

```yaml
istio:
  enabled: true
  virtualService:
    hosts:
      - voidllm.example.com
  gateway:
    servers:
      - port:
          number: 443
          name: https
          protocol: HTTPS
        tls:
          mode: SIMPLE
          credentialName: voidllm-tls
        hosts:
          - voidllm.example.com
```

--------------------------------

### Configure VoidLLM as MCP Server for Management Tools

Source: https://voidllm.ai/docs/mcp/ide-integration

Use this JSON configuration to connect to VoidLLM's built-in management tools (list models, check health, create keys) via the MCP server endpoint.

```json
{
  "mcpServers": {
    "voidllm": {
      "url": "https://your-voidllm/api/v1/mcp/voidllm",
      "headers": {
        "Authorization": "Bearer vl_uk_..."
      }
    }
  }
}
```

--------------------------------

### Configure Database Driver and DSN

Source: https://voidllm.ai/docs/configuration

Specify the database driver (sqlite or postgres) and the Data Source Name (DSN). For PostgreSQL, the DSN includes connection details and credentials.

```yaml
database:
  driver: sqlite            # sqlite (default) or postgres
  dsn: /data/voidllm.db    # SQLite file path or PostgreSQL DSN
```

--------------------------------

### MCP Configuration Options

Source: https://voidllm.ai/docs/mcp/servers

Adjust global settings for MCP servers, including call timeouts and whether to allow private URLs.

```yaml
settings:
  mcp:
    call_timeout: 30s           # timeout per proxied tool call
    allow_private_urls: false   # block MCP servers on localhost/private IPs
```

--------------------------------

### OpenAI Provider Configuration

Source: https://voidllm.ai/docs/models/providers

Direct passthrough configuration for OpenAI models. No translation is needed.

```yaml
models:
  - name: gpt-4o
    provider: openai
    base_url: https://api.openai.com/v1
    api_key: ${OPENAI_KEY}
    aliases: [default, smart]
```

--------------------------------

### Configure Cursor as LLM Proxy

Source: https://voidllm.ai/docs/mcp/ide-integration

Set the OpenAI API Base URL in Cursor's model settings to point to your VoidLLM instance for LLM proxy functionality.

```text
https://your-voidllm/v1
```

--------------------------------

### Configure VoidLLM as MCP Server for Code Mode

Source: https://voidllm.ai/docs/mcp/ide-integration

This JSON configuration enables access to Code Mode tools (list_servers, search_tools, execute_code) for multi-tool orchestration through VoidLLM's MCP server.

```json
{
  "mcpServers": {
    "voidllm-code": {
      "url": "https://your-voidllm/api/v1/mcp",
      "headers": {
        "Authorization": "Bearer vl_uk_..."
      }
    }
  }
}
```

--------------------------------

### Configure Ollama and Azure Models

Source: https://voidllm.ai/docs/configuration

Define individual models with their provider, base URL, API key, and optional parameters like timeout and aliases. This is useful for setting up direct connections to specific model services.

```yaml
models:
  - name: dolphin-mistral         # Unique model name
    provider: ollama              # openai, anthropic, azure, vllm, ollama, custom
    base_url: http://localhost:11434/v1
    api_key: ${OLLAMA_KEY}        # Optional, depends on provider
    timeout: 30s                  # Per-model upstream timeout (default: 5min)
    aliases:                      # Alternative names clients can use
      - dolphin
      - default
    max_context_tokens: 32000     # Informational, shown in UI
    pricing:
      input_per_1m: 0.15         # Cost per 1M input tokens (for usage tracking)
      output_per_1m: 0.60

  # Azure requires deployment name + API version
  - name: gpt-4o
    provider: azure
    base_url: https://mycompany.openai.azure.com
    api_key: ${AZURE_KEY}
    azure_deployment: my-gpt4o-deployment   # your Azure deployment name, not the model name
    azure_api_version: "2024-10-21"
```

--------------------------------

### Enable Code Mode Configuration

Source: https://voidllm.ai/docs/mcp/code-mode

Configure Code Mode settings, including enabling it, setting memory limits, execution timeouts, pool size for runtimes, and the maximum number of tool calls per execution.

```yaml
settings:
  mcp:
    code_mode:
      enabled: true
      memory_limit_mb: 16     # max memory per execution (default: 16)
      timeout: 30s            # execution timeout (default: 30s)
      pool_size: 8            # pre-warmed WASM runtimes (default: 8)
      max_tool_calls: 50      # max tool calls per execution (default: 50)
```

--------------------------------

### Kubernetes Liveness and Readiness Probes

Source: https://voidllm.ai/docs/deployment/kubernetes

Defines Kubernetes liveness and readiness probes for VoidLLM, targeting specific health check endpoints.

```yaml
livenessProbe:
  httpGet:
    path: /healthz
    port: 8080
  initialDelaySeconds: 5

readinessProbe:
  httpGet:
    path: /readyz
    port: 8080
  initialDelaySeconds: 5
```

--------------------------------

### Backup SQLite Database

Source: https://voidllm.ai/docs/deployment/database

Back up the SQLite database file while VoidLLM is running. SQLite supports concurrent reads, making this operation safe during active use.

```bash
# While VoidLLM is running (SQLite supports concurrent reads)
cp /data/voidllm.db /backup/voidllm-$(date +%Y%m%d).db
```

--------------------------------

### Global SSO Configuration (YAML)

Source: https://voidllm.ai/docs/enterprise/sso

Configure SSO settings globally in voidllm.yaml for DevOps-managed deployments. This sets up the OIDC provider details and user provisioning options.

```yaml
settings:
  sso:
    enabled: true
    issuer: "https://accounts.google.com"
    client_id: ${VOIDLLM_SSO_CLIENT_ID}
    client_secret: ${VOIDLLM_SSO_CLIENT_SECRET}
    redirect_url: "https://voidllm.company.com/api/v1/auth/oidc/callback"
    scopes: ["openid", "email", "profile"]
    allowed_domains: ["company.com"]
    auto_provision: true
    default_role: "member"
    group_sync: true
    group_claim: "groups"
```

--------------------------------

### Custom OpenAI-Compatible Provider Configuration

Source: https://voidllm.ai/docs/models/providers

Configuration for any endpoint that implements the OpenAI API format, such as Together AI, Fireworks, or Groq.

```yaml
models:
  - name: my-model
    provider: custom
    base_url: https://my-provider.com/v1
    api_key: ${MY_KEY}
```

--------------------------------

### Azure AD SSO Configuration (YAML)

Source: https://voidllm.ai/docs/enterprise/sso

Configure VoidLLM to use Azure AD as the OIDC provider. Replace '{tenant-id}' with your Azure AD tenant ID and ensure the redirect URI is registered.

```yaml
settings:
  sso:
    enabled: true
    issuer: "https://login.microsoftonline.com/{tenant-id}/v2.0"
    client_id: ${VOIDLLM_SSO_CLIENT_ID}
    client_secret: ${VOIDLLM_SSO_CLIENT_SECRET}
    redirect_url: "https://your-voidllm.com/api/v1/auth/oidc/callback"
```

--------------------------------

### Set License via Environment Variable

Source: https://voidllm.ai/docs/enterprise/license

Recommended for Docker and Kubernetes deployments. Set the VOIDLLM_LICENSE environment variable to your JWT license key.

```bash
export VOIDLLM_LICENSE="eyJhbGciOiJFZERTQSIs..."
```

--------------------------------

### SSE stream (built-in voidllm server only)

Source: https://voidllm.ai/docs/configuration

Provides an SSE stream from the built-in VoidLLM server, typically used for real-time updates or events.

```APIDOC
## GET /api/v1/mcp/:alias

### Description
Provides an SSE stream from the built-in VoidLLM server, typically used for real-time updates or events.

### Method
GET

### Endpoint
/api/v1/mcp/:alias

### Parameters
#### Path Parameters
- **alias** (string) - Required - The alias of the built-in VoidLLM server (expected to be 'voidllm').

### Response
#### Success Response (200)
- **(Server-Sent Events stream)** - Description depends on the specific event stream provided by the VoidLLM server.
```

--------------------------------

### Configure MCP Server for Claude Code

Source: https://voidllm.ai/docs/getting-started

Integrate VoidLLM as an MCP server in your Claude Code configuration by providing the URL and authorization headers.

```json
{
  "mcpServers": {
    "voidllm": {
      "url": "http://localhost:8080/api/v1/mcp/voidllm",
      "headers": {
        "Authorization": "Bearer vl_uk_..."
      }
    }
  }
}
```

--------------------------------

### Configure Windsurf as LLM Proxy

Source: https://voidllm.ai/docs/mcp/ide-integration

In Windsurf's custom provider settings, specify the Base URL and API Key to connect to VoidLLM as an LLM proxy.

```text
Base URL: https://your-voidllm/v1
API Key: vl_uk_...
```

--------------------------------

### Ollama Provider Configuration

Source: https://voidllm.ai/docs/models/providers

Local Ollama instances are configured for passthrough. No API key is needed for local instances.

```yaml
models:
  - name: llama3
    provider: ollama
    base_url: http://localhost:11434/v1
    aliases: [local]
```

--------------------------------

### View Audit Logs via API

Source: https://voidllm.ai/docs/enterprise/audit

Retrieve audit logs using the API with query parameters for filtering. Ensure you include the Authorization header.

```bash
curl -H "Authorization: Bearer vl_uk_..." \
  "https://voidllm.example.com/api/v1/audit-logs?resource_type=key&action=create&limit=50"
```

--------------------------------

### Register MCP Server via Admin API

Source: https://voidllm.ai/docs/mcp/servers

Use the Admin API to programmatically register MCP servers. These servers are stored in the database and can be managed without requiring a service restart.

```bash
curl -X POST https://voidllm.example.com/api/v1/mcp-servers \
  -H "Authorization: Bearer vl_uk_..." \
  -H "Content-Type: application/json" \
  -d '{ \
    "name": "GitHub MCP", \
    "alias": "github", \
    "url": "https://mcp.github.com/sse", \
    "auth_type": "bearer", \
    "auth_token": "ghp_..." \
  }'
```

--------------------------------

### IDE Integration Configuration

Source: https://voidllm.ai/docs/mcp/code-mode

Configure your IDE to recognize Code Mode tools by pointing it to the MCP API endpoint. This typically involves specifying server URLs and authentication headers.

```json
{
  "mcpServers": {
    "voidllm-code": {
      "url": "https://your-voidllm/api/v1/mcp",
      "headers": {
        "Authorization": "Bearer <your-api-key>"
      }
    }
  }
}
```

--------------------------------

### Migrate Database from SQLite to PostgreSQL

Source: https://voidllm.ai/docs/deployment/database

This command performs a bidirectional, transactional, and batched migration. It is safe to run against production data and transfers all tables, users, keys, usage events, and configuration.

```bash
voidllm migrate \
  --from sqlite:///data/voidllm.db \
  --to postgres://user:pass@host:5432/voidllm
```

--------------------------------

### Configure VoidLLM as MCP Server for External Servers

Source: https://voidllm.ai/docs/mcp/ide-integration

Connect to a specific external MCP server registered with VoidLLM using this JSON configuration. VoidLLM handles authentication, access control, and logging.

```json
{
  "mcpServers": {
    "aws": {
      "url": "https://your-voidllm/api/v1/mcp/aws",
      "headers": {
        "Authorization": "Bearer vl_uk_..."
      }
    }
  }
}
```

--------------------------------

### Register global server

Source: https://voidllm.ai/docs/configuration

Registers a new MCP server at the global scope. This action is restricted to system administrators.

```APIDOC
## POST /api/v1/mcp-servers

### Description
Registers a new MCP server at the global scope. This action is restricted to system administrators.

### Method
POST

### Endpoint
/api/v1/mcp-servers

### Parameters
#### Request Body
- **(MCP Server Registration Payload)** - Required - Details for the server to be registered (e.g., name, alias, URL, auth details).

### Response
#### Success Response (200 or 201)
- **(Details of the registered server)** - Description of the newly registered MCP server.
```
Tokens:
Raw