### Start anyllm-proxy with Admin Web Interface

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/README.md

Run the proxy and enable the admin dashboard alongside it. The dashboard provides a guided setup and full control.

```bash
anyllm_proxy --webui
# Proxy:     http://localhost:3000
```

--------------------------------

### Minimal Client Setup with Builder Shorthand

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/crates/client/README.md

Use `Client::builder()` for a quick setup with default authentication and translation configurations. This example shows how to build a client for an OpenAI-compatible backend.

```rust
use anyllm_client::Client;
use anyllm_translate::anthropic::MessageCreateRequest;

let client = Client::builder()
    .base_url("https://api.openai.com/v1/chat/completions")
    .api_key(&std::env::var("OPENAI_API_KEY")?)
    .build()?;

let req: MessageCreateRequest = serde_json::from_str(r#"{{
    "model": "gpt-4o-mini",
    "max_tokens": 256,
    "messages": [{{"role": "user", "content": "Summarize Rust's borrow checker"}}]
}}"#)?;

let resp = client.messages(&req).await?;
println!("{:#?}", resp.content);
```

--------------------------------

### Run OpenAI Backend with Environment Variables

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/openai.md

Starts the anyllm-proxy with the OpenAI backend using environment variables for API key and backend selection. This is a quick way to get started.

```bash
BACKEND=openai OPENAI_API_KEY=sk-... cargo run -p anyllm_proxy
```

--------------------------------

### Run AnyLLM Proxy with Replicate Backend (Environment Variables)

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/replicate.md

Starts the AnyLLM Proxy using Replicate as the backend, configured via environment variables. This is a quick way to get started.

```bash
BACKEND=replicate REPLICATE_API_KEY=your-key cargo run -p anyllm_proxy
```

--------------------------------

### HTTP Client Tool Calling Setup

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/README.md

Define and attach tools for the LLM to use. This example shows how to build a tool with a description and input schema, which can then be included in the request.

```rust
use anyllm_client::{ToolBuilder, ToolChoiceBuilder};
use serde_json::json;

let tool = ToolBuilder::new("get_weather")
    .description("Get the current weather for a location")
    .input_schema(json!({
        "type": "object",
        "properties": {{"location": {{"type": "string"}}}},
        "required": ["location"]
    }))
    .build();
// Attach tool to MessageCreateRequest via serde_json, then call client.messages().
```

--------------------------------

### BatchEngine Facade Example

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/superpowers/plans/2026-03-30-batch-engine-phase1.md

Demonstrates the usage of the BatchEngine facade for submitting and cancelling a job. Assumes prior setup and job creation.

```rust
metadata: None,
                priority: 0,
            })
            .await
            .unwrap();

        let status = engine.cancel(&job.id).await.unwrap();
        assert_eq!(status, BatchStatus::Cancelled);
    }
}
```

--------------------------------

### Nscale Quick Start with Environment Variables

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/nscale.md

Configure the proxy to use Nscale as the backend by setting the BACKEND and NSCALE_API_KEY environment variables. This example shows both direct cargo run and Docker deployment.

```bash
BACKEND=nscale NSCALE_API_KEY=your-key cargo run -p anyllm_proxy
# Docker:
docker run -e BACKEND=nscale -e NSCALE_API_KEY=your-key -e PROXY_OPEN_RELAY=true -p 3000:3000 followthewhit3rabbit/anyllm-proxy
```

--------------------------------

### Run Proxy with Novita AI (Environment Variables)

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/novita.md

Starts the anyllm-proxy using environment variables to specify Novita AI as the backend and provide the API key. This is a quick way to get started.

```bash
BACKEND=novita NOVITA_API_KEY=your-key cargo run -p anyllm_proxy
# Docker:
docker run -e BACKEND=novita -e NOVITA_API_KEY=your-key -e PROXY_OPEN_RELAY=true -p 3000:3000 followthewhit3rabbit/anyllm-proxy
```

--------------------------------

### Run SambaNova Proxy with Environment Variables

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/sambanova.md

Starts the anyllm-proxy with the SambaNova backend using environment variables for configuration. This is a quick way to get started.

```bash
BACKEND=sambanova SAMBANOVA_API_KEY=your-key cargo run -p anyllm_proxy
```

--------------------------------

### Start vLLM Server with API Key

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/hosted_vllm.md

Demonstrates how to start the vLLM serving process with an API key for authentication. This key can then be used by the LiteLLM proxy.

```bash
vllm serve meta-llama/Meta-Llama-3.1-8B-Instruct --api-key secret
```

--------------------------------

### Run Mistral Proxy with Environment Variables

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/mistral.md

Starts the anyllm-proxy using environment variables for backend selection and API key. This is a quick way to get started.

```bash
BACKEND=mistral MISTRAL_API_KEY=your-key cargo run -p anyllm_proxy
```

--------------------------------

### Enable and Start Debian Service

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/AGENTS.md

Systemd commands to enable and start the anyllm-proxy service after installation.

```bash
sudo systemctl enable --now anyllm-proxy
```

--------------------------------

### Dashscope Quick Start with Environment Variables

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/dashscope.md

Set the BACKEND and DASHSCOPE_API_KEY environment variables to run the anyllm-proxy with Dashscope. This example shows command-line execution and Docker deployment.

```bash
BACKEND=dashscope DASHSCOPE_API_KEY=your-key cargo run -p anyllm_proxy
# Docker:
docker run -e BACKEND=dashscope -e DASHSCOPE_API_KEY=your-key -e PROXY_OPEN_RELAY=true -p 3000:3000 followthewhit3rabbit/anyllm-proxy
```

--------------------------------

### Install and Run Petals Server

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/petals.md

Install the Petals library and start a local inference server for a specified model. The server defaults to listening on port 8080.

```bash
pip install petals
python -m petals.cli.run_server petals-team/StableBeluga2
```

--------------------------------

### Run anyllm-proxy with Ollama (Environment Variables)

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/ollama.md

Starts the anyllm-proxy service using Ollama as the backend via environment variables. This is a quick way to get started if you prefer environment variable configuration.

```bash
BACKEND=ollama PROXY_OPEN_RELAY=true cargo run -p anyllm_proxy
```

```bash
docker run -e BACKEND=ollama -e PROXY_OPEN_RELAY=true -p 3000:3000 followthewhit3rabbit/anyllm-proxy
```

--------------------------------

### Run Proxy with Environment Variables

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/bedrock.md

Starts the anyllm-proxy with the Bedrock backend configured via environment variables. Includes an example for Docker deployment.

```bash
BACKEND=bedrock \
  AWS_ACCESS_KEY_ID=AKIA... \
  AWS_SECRET_ACCESS_KEY=... \
  AWS_REGION=us-east-1 \
  cargo run -p anyllm_proxy
# or with Docker:
docker run \
  -e BACKEND=bedrock \
  -e AWS_ACCESS_KEY_ID=AKIA... \
  -e AWS_SECRET_ACCESS_KEY=... \
  -e AWS_REGION=us-east-1 \
  -e PROXY_OPEN_RELAY=true \
  -p 3000:3000 \
  followthewhit3rabbit/anyllm-proxy
```

--------------------------------

### Install anyllm-proxy on Debian/Ubuntu

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/README.md

Install the proxy on Debian/Ubuntu systems using dpkg. Ensure you download the correct .deb file for your architecture and check the releases page for the latest filename.

```bash
# Check https://github.com/whit3rabbit/anyllm-proxy/releases for the current filename
curl -LO https://github.com/whit3rabbit/anyllm-proxy/releases/latest/download/anyllm-proxy_0.9.0-1_amd64.deb
# arm64: replace amd64 with arm64
sudo dpkg -i anyllm-proxy_*.deb
sudo systemctl enable --now anyllm-proxy
# Configure: edit /etc/default/anyllm-proxy
```

--------------------------------

### Proxy Configuration Examples

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/crates/proxy/admin-ui/dist/index.html

Examples of environment variables for configuring the proxy to work with different LLM providers like OpenAI, Ollama, or OpenRouter. These are typically placed in a .anyllm.env file.

```env
OPENAI_API_KEY=sk-... PROXY_API_KEYS=my-key
```

```env
OPENAI_BASE_URL=http://localhost:11434/v1 PROXY_OPEN_RELAY=true
```

```env
OPENAI_BASE_URL=https://openrouter.ai/api/v1 OPENAI_API_KEY=sk-or-... PROXY_API_KEYS=my-key
```

--------------------------------

### Initialize and Start Batch Engine

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/superpowers/plans/2026-03-30-batch-engine-phase1.md

Initializes batch engine tables, sets up the BatchEngine struct with necessary components like queues and file storage, and starts the webhook dispatcher. Ensure SQLite connection is already established.

```rust
// Initialize batch engine tables.
anyllm_batch_engine::db::init_batch_engine_tables(&conn)?;
anyllm_batch_engine::db::migrate_old_tables(&conn)?;

let db = Arc::new(Mutex::new(conn));
let batch_engine = Arc::new(BatchEngine {
    queue: Arc::new(SqliteQueue::new(db.clone())),
    file_store: FileStore::new(db.clone()),
    webhook_queue: Arc::new(SqliteWebhookQueue::new(db.clone())),
    global_webhook_urls: parse_webhook_urls(),
    webhook_signing_secret: std::env::var("BATCH_WEBHOOK_SIGNING_SECRET").ok(),
});

// Start webhook dispatcher.
let webhook_handle = anyllm_batch_engine::webhook::dispatcher::start_dispatcher(
    batch_engine.webhook_queue.clone(),
    reqwest::Client::new(),
    WebhookConfig::default(),
);
```

--------------------------------

### MiniMax Quick Start with Environment Variables

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/minimax.md

Set up MiniMax as the backend using environment variables for direct execution or Docker deployment.

```bash
BACKEND=minimax MINIMAX_API_KEY=your-key cargo run -p anyllm_proxy
# Docker:
docker run -e BACKEND=minimax -e MINIMAX_API_KEY=your-key -e PROXY_OPEN_RELAY=true -p 3000:3000 followthewhit3rabbit/anyllm-proxy
```

--------------------------------

### NanoGPT Quick Start with Environment Variables

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/nanogpt.md

Run the proxy with NanoGPT as a single backend using environment variables. This is useful for quick local testing.

```bash
BACKEND=nanogpt NANOGPT_API_KEY=your-key cargo run -p anyllm_proxy
```

```bash
# Docker:
docker run -e BACKEND=nanogpt -e NANOGPT_API_KEY=your-key -e PROXY_OPEN_RELAY=true -p 3000:3000 followthewhit3rabbit/anyllm-proxy
```

--------------------------------

### Run Volcano Engine Provider with Environment Variables

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/volcengine.md

Start the LiteLLM Proxy with the Volcano Engine backend configured using environment variables. This is a quick way to get started.

```bash
BACKEND=volcengine VOLCENGINE_API_KEY=your-key cargo run -p anyllm_proxy
```

--------------------------------

### Anyscale Quick Start with Environment Variables

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/anyscale.md

Configure and run the anyllm-proxy using Anyscale as the backend via environment variables. This is useful for quick local testing.

```bash
BACKEND=anyscale ANYSCALE_API_KEY=your-key cargo run -p anyllm_proxy
# Docker:
docker run -e BACKEND=anyscale -e ANYSCALE_API_KEY=your-key -e PROXY_OPEN_RELAY=true -p 3000:3000 followthewhit3rabbit/anyllm-proxy
```

--------------------------------

### Vertex AI Quick Start with Environment Variables

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/vertex_ai.md

Set up Vertex AI as a backend using environment variables for direct execution or Docker deployment.

```bash
BACKEND=vertex_ai \
  VERTEX_PROJECT=my-project-123 \
  VERTEX_REGION=us-central1 \
  GOOGLE_APPLICATION_CREDENTIALS=/path/to/sa.json \
  cargo run -p anyllm_proxy
```

```bash
docker run \
  -e BACKEND=vertex_ai \
  -e VERTEX_PROJECT=my-project-123 \
  -e VERTEX_REGION=us-central1 \
  -e GOOGLE_APPLICATION_CREDENTIALS=/run/secrets/sa.json \
  -v /path/to/sa.json:/run/secrets/sa.json:ro \
  -e PROXY_OPEN_RELAY=true \
  -p 3000:3000 \
  followthewhit3rabbit/anyllm-proxy
```

--------------------------------

### Basic AnyLLM Proxy Startup Command

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/CONFIG.md

The simplest command to start the AnyLLM Proxy, which auto-loads the default environment file if present.

```bash
anyllm-proxy
```

--------------------------------

### Configure and run anyllm-proxy

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/README.md

Create a .anyllm.env file for configuration and run the proxy. The proxy automatically loads the environment file from standard locations.

```env
OPENAI_API_KEY=unused
OPENAI_BASE_URL=http://localhost:11434/v1
BIG_MODEL=qwen2.5-coder:32b
SMALL_MODEL=qwen2.5-coder:32b
```

```bash
anyllm_proxy
# or: anyllm_proxy --env-file ~/configs/ollama.env
```

--------------------------------

### Run AnyLLM Proxy with Pollinations Backend

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/pollinations.md

This command starts the anyllm-proxy with Pollinations as the backend and enables open relay mode. It's the quickest way to get started using Pollinations.

```bash
BACKEND=pollinations PROXY_OPEN_RELAY=true cargo run -p anyllm_proxy
```

--------------------------------

### Main Entry Point (main.tsx)

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/superpowers/plans/2026-04-05-react-admin-ui.md

Sets up the React application, configures the QueryClient for data fetching with default options, and renders the main App component within a QueryClientProvider.

```tsx
import { StrictMode } from 'react'
import { createRoot } from 'react-dom/client'
import { QueryClient, QueryClientProvider } from '@tanstack/react-query'
import App from './App'
import './styles/globals.css'

const queryClient = new QueryClient({
  defaultOptions: {
    queries: {
      retry: 1,
      refetchOnWindowFocus: false,
    },
  },
})

createRoot(document.getElementById('root')!).render(
  <StrictMode>
    <QueryClientProvider client={queryClient}>
      <App />
    </QueryClientProvider>
  </StrictMode>,
)
```

--------------------------------

### Install and Run Infinity Embedding Server

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/infinity.md

Installs the Infinity embedding server using pip and starts it with a specified Hugging Face model ID. The server defaults to port 7997.

```bash
pip install infinity-emb[all]
infinity_emb v2 --model-id BAAI/bge-small-en-v1.5
```

--------------------------------

### Run Bytez Proxy with Environment Variables

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/bytez.md

Start the anyllm_proxy with Bytez as the backend using environment variables for configuration. This is useful for quick local testing.

```bash
BACKEND=bytez BYTEZ_KEY=your-key cargo run -p anyllm_proxy
# Docker:
docker run -e BACKEND=bytez -e BYTEZ_KEY=your-key -e PROXY_OPEN_RELAY=true -p 3000:3000 followthewhit3rabbit/anyllm-proxy
```

--------------------------------

### Proxy Startup Data Directory Log

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/CONFIG.md

Example log output showing the resolved data directory path at proxy startup.

```text
anyllm_proxy: data directory: /home/user/.anyllm
```

--------------------------------

### Uptime API Response Shape

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/superpowers/specs/2026-04-05-react-admin-ui-design.md

Defines the structure of the JSON response for the GET /admin/api/uptime endpoint, providing proxy start time and backend health check history.

```json
{
  "proxy": {
    "started_at": 1234567890,
    "uptime_pct_30d": 99.98,
    "history": [{ "date": "2026-04-05", "status": "up" }]
  },
  "backends": [
    {
      "name": "openai",
      "status": "up",
      "last_checked_at": 1234567890,
      "last_latency_ms": 210,
      "uptime_pct_30d": 99.9,
      "history": [{ "date": "2026-04-05", "status": "up" }]
    }
  ]
}
```

--------------------------------

### Quick Start: Single-Backend (Environment Variables)

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/gemini.md

Set the BACKEND and GEMINI_API_KEY environment variables to run the proxy with Gemini. This example shows usage with `cargo run` and Docker.

```bash
BACKEND=gemini GEMINI_API_KEY=AIza... cargo run -p anyllm_proxy
```

```bash
docker run -e BACKEND=gemini -e GEMINI_API_KEY=AIza... -e PROXY_OPEN_RELAY=true -p 3000:3000 followthewhit3rabbit/anyllm-proxy
```

--------------------------------

### Run anyllm-proxy with LM Studio Backend (Environment Variables)

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/lm_studio.md

Starts the anyllm-proxy with LM Studio as the backend using environment variables. This is useful for quick local testing.

```bash
BACKEND=lm_studio PROXY_OPEN_RELAY=true cargo run -p anyllm_proxy
```

```bash
docker run -e BACKEND=lm_studio -e PROXY_OPEN_RELAY=true -p 3000:3000 followthewhit3rabbit/anyllm-proxy
```

--------------------------------

### Hyperbolic Quick Start with Environment Variables

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/hyperbolic.md

Set up Hyperbolic as a backend using environment variables for direct execution or Docker deployment.

```bash
BACKEND=hyperbolic HYPERBOLIC_API_KEY=your-key cargo run -p anyllm_proxy
# Docker:
docker run -e BACKEND=hyperbolic -e HYPERBOLIC_API_KEY=your-key -e PROXY_OPEN_RELAY=true -p 3000:3000 followthewhit3rabbit/anyllm-proxy
```

--------------------------------

### SageMaker Single-Backend Configuration (Environment Variables)

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/sagemaker.md

Configure the proxy to use a SageMaker backend via environment variables. This example shows the intended setup, but the backend is not yet implemented.

```bash
BACKEND=sagemaker \
  AWS_ACCESS_KEY_ID=AKIA... \
  AWS_SECRET_ACCESS_KEY=... \
  AWS_REGION_NAME=us-east-1 \
  PROXY_OPEN_RELAY=true \
  cargo run -p anyllm_proxy
```

--------------------------------

### AnyLLM Proxy Startup with Admin UI Enabled

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/CONFIG.md

Command to start the AnyLLM Proxy with the administrative web UI enabled.

```bash
anyllm-proxy --webui
```

--------------------------------

### Lambda AI Quick Start with Environment Variables

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/lambda_ai.md

Set up Lambda AI integration using environment variables for the backend and API key. This is useful for quick local testing or deployment.

```bash
BACKEND=lambda_ai LAMBDA_API_KEY=your-key cargo run -p anyllm_proxy
# Docker:
docker run -e BACKEND=lambda_ai -e LAMBDA_API_KEY=your-key -e PROXY_OPEN_RELAY=true -p 3000:3000 followthewhit3rabbit/anyllm-proxy
```

--------------------------------

### Implement GET /admin/api/uptime Route in Rust

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/superpowers/plans/2026-04-05-react-admin-ui.md

This Rust code defines the handler for the GET /admin/api/uptime endpoint. It retrieves proxy start time and per-backend health check history from the database, structures it into a `UptimeResponse` object, and returns it as JSON. It uses Axum for routing and extracts shared state including the database connection and runtime configuration.

```rust
use crate::admin::db::{backend_history_30d, backend_uptime_pct};
use crate::admin::state::{with_db, SharedState};
use axum::{extract::State, Json};
use serde::Serialize;

#[derive(Serialize)]
pub struct HistoryDay {
    date: String,
    status: String,
}

#[derive(Serialize)]
pub struct ProxyUptimeInfo {
    started_at: u64,
    uptime_pct_30d: f64,
    history: Vec<HistoryDay>,
}

#[derive(Serialize)]
pub struct BackendUptimeInfo {
    name: String,
    status: String,
    last_checked_at: Option<i64>,
    last_latency_ms: Option<i64>,
    uptime_pct_30d: f64,
    history: Vec<HistoryDay>,
}

#[derive(Serialize)]
pub struct UptimeResponse {
    proxy: ProxyUptimeInfo,
    backends: Vec<BackendUptimeInfo>,
}

pub(super) async fn get_uptime(
    State(shared): State<SharedState>,
) -> Json<UptimeResponse> {
    let started_at = shared
        .started_at
        .duration_since(std::time::UNIX_EPOCH)
        .unwrap_or_default()
        .as_secs();

    // Collect backend names from runtime config.
    let backend_names: Vec<String> = {
        let cfg = shared.runtime_config.read().unwrap_or_else(|e| e.into_inner());
        cfg.model_mappings.keys().cloned().collect()
    };

    let response = with_db(&shared.db, move |conn| {
        // Proxy uptime: use process start time; all days are "up" unless we tracked outages.
        // We approximate with the health_checks records for all backends combined.
        let proxy_history: Vec<HistoryDay> = {
            if let Ok(mut stmt) = conn.prepare(
                "SELECT date(checked_at, 'unixepoch') AS day,
                        MIN(CASE WHEN status='down' THEN 0 ELSE 1 END) AS all_up
                 FROM health_checks
                 WHERE checked_at >= strftime('%s','now') - 30*86400
                 GROUP BY day
                 ORDER BY day ASC",
            ) {
                stmt.query_map([], |r| {
                    let day: String = r.get(0)?;
                    let all_up: i64 = r.get(1)?;
                    Ok(HistoryDay {
                        date: day,
                        status: if all_up == 1 { "up".to_string() } else { "down".to_string() },
                    })
                })
                .unwrap_or_else(|_| Box::new(std::iter::empty()) as Box<dyn Iterator<Item=_>>)
                .filter_map(|r| r.ok())
                .collect()
            } else {
                vec![]
            }
        };

        // Collect per-backend data.
        let backends: Vec<BackendUptimeInfo> = backend_names
            .iter()
            .map(|name| {
                let uptime_pct = backend_uptime_pct(conn, name).unwrap_or(100.0);
                let history = backend_history_30d(conn, name)
                    .unwrap_or_default()
                    .into_iter()
                    .map(|(date, status)| HistoryDay { date, status })
                    .collect();

                // Last check for this backend.
                let (last_checked_at, last_latency_ms, current_status) = conn
                    .query_row(
                        "SELECT checked_at, latency_ms, status FROM health_checks
                         WHERE backend = ?1 ORDER BY checked_at DESC LIMIT 1",
                        [name],
                        |r| Ok((r.get::<_, i64>(0)?, r.get::<_, Option<i64>>(1)?, r.get::<_, String>(2)?)),
                    )
                    .map(|(ts, lat, st)| (Some(ts), lat, st))
                    .unwrap_or((None, None, "unknown".to_string()));

                BackendUptimeInfo {
                    name: name.clone(),
                    status: current_status,
                    last_checked_at,
                    last_latency_ms,
                    uptime_pct_30d: uptime_pct,
                    history,
                }
            })
            .collect();

        UptimeResponse {
            proxy: ProxyUptimeInfo {
                started_at,
                uptime_pct_30d: 100.0, // proxy process doesn't self-report downtime
                history: proxy_history,
            },
            backends,
        }
    })
    .await;

    Json(response.unwrap_or_else(|| UptimeResponse {
        proxy: ProxyUptimeInfo { started_at, uptime_pct_30d: 100.0, history: vec![] },
        backends: vec![],
    }))
}

```

--------------------------------

### Generate Embeddings using Voyage AI via Proxy

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/voyage.md

Example of how to call the /v1/embeddings endpoint through the LiteLLM proxy to get embeddings from a Voyage AI model. Ensure the proxy is running and configured.

```bash
curl http://localhost:3000/v1/embeddings \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $PROXY_API_KEYS" \
  -d '{"model": "voyage-3", "input": "The quick brown fox"}'
```

--------------------------------

### Run Infinity Proxy with Environment Variables

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/infinity.md

Starts the anyllm proxy with Infinity as the backend using environment variables. This is useful for quick local testing.

```bash
BACKEND=infinity PROXY_OPEN_RELAY=true cargo run -p anyllm_proxy
```

```bash
docker run -e BACKEND=infinity -e PROXY_OPEN_RELAY=true -p 3000:3000 followthewhit3rabbit/anyllm-proxy
```

--------------------------------

### Featherless AI Single-Backend Quick Start (Environment Variables)

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/featherless_ai.md

Configure the Featherless AI backend using environment variables for local execution or Docker deployment.

```bash
BACKEND=featherless_ai FEATHERLESS_API_KEY=your-key cargo run -p anyllm_proxy
# Docker:
docker run -e BACKEND=featherless_ai -e FEATHERLESS_API_KEY=your-key -e PROXY_OPEN_RELAY=true -p 3000:3000 followthewhit3rabbit/anyllm-proxy
```

--------------------------------

### Call Codestral via OpenAI Chat Completions API

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/codestral.md

Example cURL command to interact with the Codestral model using the OpenAI Chat Completions API format through the proxy. This shows how to send a user message and get a model completion.

```bash
curl http://localhost:3000/v1/chat/completions \
  -H "Authorization: Bearer $PROXY_API_KEYS" \
  -H "Content-Type: application/json" \
  -d '{"model": "codestral-latest", "messages": [{"role": "user", "content": "Write a Python function to reverse a string"}]}'
```

--------------------------------

### Replace Async Main with Sync Main and Async Run

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/superpowers/plans/2026-03-28-security-audit-all-fixes.md

This snippet demonstrates how to replace the `#[tokio::main]` attribute with a synchronous `main` function that initializes the Tokio runtime and then calls an asynchronous `run` function. This separation ensures environment setup occurs before the runtime starts.

```rust
fn main() {
    let args: Vec<String> = std::env::args().collect();
    let (env_file_count, multi_config, model_router) = setup_env(&args);

    tokio::runtime::Builder::new_multi_thread()
        .enable_all()
        .build()
        .expect("failed to build tokio runtime")
        .block_on(run(args, env_file_count, multi_config, model_router))
}
```

```rust
async fn run(
    args: Vec<String>,
    _env_file_count: usize,
    multi_config: config::MultiConfig,
    model_router: Option<Arc<std::sync::RwLock<crate::config::model_router::ModelRouter>>>
) {

```

--------------------------------

### Run Proxy with xAI Backend (Environment Variables)

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/xai.md

Starts the anyllm-proxy using the xAI backend via environment variables. Ensure you have the xAI API key set.

```bash
BACKEND=xai XAI_API_KEY=your-key cargo run -p anyllm_proxy
```

--------------------------------

### Run anyllm-proxy with llamafile Backend

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/llamafile.md

Start the anyllm-proxy with llamafile as the backend using environment variables. This is useful for quick local testing.

```bash
BACKEND=llamafile PROXY_OPEN_RELAY=true cargo run -p anyllm-proxy
```

```bash
docker run -e BACKEND=llamafile -e PROXY_OPEN_RELAY=true -p 3000:3000 followthewhit3rabbit/anyllm-proxy
```

--------------------------------

### Add Synchronous Environment Setup Function

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/superpowers/plans/2026-03-28-security-audit-all-fixes.md

This helper function parses environment files, applies aliases, and loads configuration synchronously. It must be called before the Tokio runtime starts to ensure `set_var` is safe. It returns pre-runtime data for use within the async runtime.

```rust
/// Synchronous env setup: parse env file, apply aliases, load config.
/// MUST be called before the tokio runtime starts so `set_var` is safe.
/// Returns (env_file_count, load_result) for use inside the async runtime.
fn setup_env(args: &[String]) -> (usize, config::MultiConfig, Option<crate::config::model_router::ModelRouter>) {
    // Determine env file path from --env-file flag or .anyllm.env convention
    let env_file_path = args
        .windows(2)
        .find(|w| w[0] == "--env-file")
        .map(|w| w[1].as_str())
        .or_else(|| {
            if std::path::Path::new(".anyllm.env").exists() {
                Some(".anyllm.env")
            } else {
                None
            }
        });
    let env_file_vars = env_file_path.map(parse_env_file).unwrap_or_default();

    // SAFETY: No other threads exist yet; tokio runtime has not started.
    for (key, val) in &env_file_vars {
        unsafe { std::env::set_var(key, val); }
    }
    if !env_file_vars.is_empty() {
        eprintln!(
            "anyllm_proxy: loaded {} variable(s) from env file",
            env_file_vars.len()
        );
    }

    // Compute and apply LiteLLM env var aliases (e.g. LITELLM_MASTER_KEY -> PROXY_API_KEYS)
    let aliases = config::env_aliases::compute_env_aliases();
    // SAFETY: No other threads exist yet.
    for (key, val) in &aliases {
        unsafe { std::env::set_var(key, val); }
    }

    let load_result = config::MultiConfig::load();

    // Apply litellm master_key if PROXY_API_KEYS is still unset
    if let Some(ref mk) = load_result.litellm_master_key {
        if std::env::var("PROXY_API_KEYS").is_err() {
            // SAFETY: No other threads exist yet.
            unsafe { std::env::set_var("PROXY_API_KEYS", mk); }
            eprintln!("anyllm_proxy: applied general_settings.master_key as PROXY_API_KEYS");
        }
    }

    let env_file_count = env_file_vars.len();
    let model_router = load_result.model_router;
    let multi_config = load_result.multi_config;
    (env_file_count, multi_config, model_router)
}
```

--------------------------------

### GET /admin/api/uptime

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/superpowers/plans/2026-04-05-react-admin-ui.md

Retrieves the proxy's start time, 30-day uptime percentage, and historical daily status, along with detailed uptime information for each configured backend. This includes their current status, last check time, latency, 30-day uptime percentage, and historical daily status.

```APIDOC
## GET /admin/api/uptime

### Description
Retrieves uptime statistics for the proxy and its configured backends. This includes the proxy's start time, overall uptime percentage over the last 30 days, and a history of daily status. For each backend, it provides its current status, the timestamp of the last health check, the latency of that check, its 30-day uptime percentage, and a history of its daily status.

### Method
GET

### Endpoint
/admin/api/uptime

### Parameters
This endpoint does not accept any path or query parameters.

### Request Body
This endpoint does not accept a request body.

### Response
#### Success Response (200 OK)
Returns a JSON object containing uptime information for the proxy and its backends.

- **proxy** (object) - Uptime information for the proxy.
  - **started_at** (number) - The Unix timestamp when the proxy started.
  - **uptime_pct_30d** (number) - The proxy's uptime percentage over the last 30 days (always 100.0 as it doesn't self-report downtime).
  - **history** (array) - An array of objects, each representing a day's status.
    - **date** (string) - The date in 'YYYY-MM-DD' format.
    - **status** (string) - The status for the day ('up' or 'down').
- **backends** (array) - An array of objects, each representing a backend's uptime information.
  - **name** (string) - The name of the backend.
  - **status** (string) - The current health status of the backend ('up', 'down', or 'unknown').
  - **last_checked_at** (number | null) - The Unix timestamp of the last health check, or null if no checks have been recorded.
  - **last_latency_ms** (number | null) - The latency in milliseconds of the last health check, or null if not available.
  - **uptime_pct_30d** (number) - The backend's uptime percentage over the last 30 days.
  - **history** (array) - An array of objects, each representing a day's status for the backend.
    - **date** (string) - The date in 'YYYY-MM-DD' format.
    - **status** (string) - The status for the day ('up' or 'down').

### Response Example
```json
{
  "proxy": {
    "started_at": 1678886400,
    "uptime_pct_30d": 100.0,
    "history": [
      {"date": "2023-03-15", "status": "up"},
      {"date": "2023-03-16", "status": "up"}
    ]
  },
  "backends": [
    {
      "name": "backend-1",
      "status": "up",
      "last_checked_at": 1678972800,
      "last_latency_ms": 50,
      "uptime_pct_30d": 99.5,
      "history": [
        {"date": "2023-03-15", "status": "up"},
        {"date": "2023-03-16", "status": "up"}
      ]
    }
  ]
}
```
```

--------------------------------

### Download and Run a llamafile

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/llamafile.md

Instructions for downloading a specific llamafile model, making it executable, and starting the local inference server.

```bash
wget https://huggingface.co/Mozilla/Meta-Llama-3.1-8B-Instruct-llamafile/resolve/main/Meta-Llama-3.1-8B-Instruct.Q6_K.llamafile
chmod +x Meta-Llama-3.1-8B-Instruct.Q6_K.llamafile
./Meta-Llama-3.1-8B-Instruct.Q6_K.llamafile --server --port 8080
```

--------------------------------

### Add Frontend Job to CI Workflow

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/superpowers/plans/2026-04-05-react-admin-ui.md

Integrates a new 'frontend' job into the GitHub Actions CI workflow (`.github/workflows/ci.yml`). This job handles Node.js setup, dependency installation, TypeScript checks, linting, and building the admin UI. The 'test' job now depends on 'frontend' to ensure frontend checks pass before Rust tests run.

```yaml
jobs:
  frontend:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
          cache-dependency-path: crates/proxy/admin-ui/package-lock.json
      - name: Install dependencies
        working-directory: crates/proxy/admin-ui
        run: npm ci
      - name: TypeScript check
        working-directory: crates/proxy/admin-ui
        run: npx tsc --noEmit
      - name: Lint
        working-directory: crates/proxy/admin-ui
        run: npm run lint
      - name: Build
        working-directory: crates/proxy/admin-ui
        run: npm run build
      - name: Upload dist
        uses: actions/upload-artifact@v4
        with:
          name: admin-ui-dist
          path: crates/proxy/admin-ui/dist/

  test:
    needs: frontend
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Download dist
        uses: actions/download-artifact@v4
        with:
          name: admin-ui-dist
          path: crates/proxy/admin-ui/dist/
      - uses: dtolnay/rust-toolchain@stable
        with:
          components: clippy, rustfmt
      - uses: Swatinem/rust-cache@v2
      # ... rest of existing steps unchanged ...

```

--------------------------------

### Run Predibase Backend with Environment Variables

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/predibase.md

Configure and run the anyllm-proxy with Predibase as the backend using environment variables. This is useful for quick local testing.

```bash
BACKEND=predibase PREDIBASE_API_KEY=your-key cargo run -p anyllm_proxy
```

--------------------------------

### Run Proxy with Cohere (Environment Variables)

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/cohere_chat.md

Starts the anyllm-proxy using Cohere as the backend via environment variables. Ensure you have the COHERE_API_KEY set.

```bash
BACKEND=cohere_chat COHERE_API_KEY=your-key cargo run -p anyllm_proxy
# Docker:
docker run -e BACKEND=cohere_chat -e COHERE_API_KEY=your-key -e PROXY_OPEN_RELAY=true -p 3000:3000 followthewhit3rabbit/anyllm-proxy
```

--------------------------------

### Install and Verify Debian Package

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/superpowers/specs/2026-04-06-deb-packaging-design.md

Commands to install a Debian package, resolve dependencies, and perform basic verification checks on the installed anyllm-proxy.

```bash
sudo dpkg -i *.deb && sudo apt-get install -f -y
anyllm-proxy --help
systemd-analyze verify anyllm-proxy.service
```

--------------------------------

### AnyLLM Proxy Startup with Admin UI and Explicit Env File

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/CONFIG.md

Command to start the AnyLLM Proxy with both the admin UI enabled and an explicit environment file specified.

```bash
anyllm-proxy --webui --env-file /path/to/my.env
```

--------------------------------

### Install anyllm-proxy with Homebrew

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/README.md

Install the proxy on macOS using the Homebrew package manager.

```bash
brew install whit3rabbit/tap/anyllm-proxy
```

--------------------------------

### Git Commit Example

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/superpowers/plans/2026-03-30-model-routing-ux.md

Example of a git commit command used after updating documentation files.

```bash
git add CLAUDE.md
git commit -m "docs: document simple YAML config format in CLAUDE.md"

```

--------------------------------

### Common anyllm_providers Usage Examples

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/crates/providers/README.md

Demonstrates common use cases including looking up provider details, listing models for a provider, resolving backend protocols, and mapping LiteLLM prefixes.

```rust
use anyllm_providers::{
    canonical_provider_id,
    find_by_litellm_prefix,
    get_provider,
    list_models,
    resolve_backend,
};

// Look up a provider by id.
let groq = get_provider("groq").expect("groq is registered");
println!("{} -> {}", groq.display_name, groq.default_base_url);
println!("API key env vars: {:?}", groq.env_vars);

// List every model registered for a provider.
for model in list_models("anthropic") {
    println!(
        "{} ({} ctx, {} max out)",
        model.id,
        model.context_window,
        model.max_output_tokens,
    );
}

// Resolve a provider id to a backend kind string and base URL.
// Returns None for ProviderProtocol::Custom (not yet implemented).
if let Some((kind, url)) = resolve_backend("together_ai") {
    println!("routes through {kind} at {url}");
}

// Map a LiteLLM-style routing prefix back to its provider.
let p = find_by_litellm_prefix("together_ai/").unwrap();
assert_eq!(p.id, "together_ai");

// Canonicalize old local ids before persisting new config.
assert_eq!(canonical_provider_id("zhipuai"), "zai");

```

--------------------------------

### Get Current Route ID

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/crates/proxy/admin-ui/dist/index.html

Hook to get the ID of the current route from the route context.

```javascript
function ei(e){let t=$r(e),n=t.matches[t.matches.length-1];return!n.route.id&&I(!1),n.route.id}
```

--------------------------------

### Basic anyllm.yaml Configuration

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/superpowers/plans/2026-03-30-model-routing-ux.md

Example of a basic anyllm.yaml file showing routing strategy, listen port, log bodies, and model definitions.

```yaml
routing_strategy: latency-based   # round-robin (default) | least-busy | latency-based | weighted | cost-based
listen_port: 3000                  # optional
log_bodies: false                  # optional

models:
  # String shorthand: bare model name defaults to openai
  - gpt-4o
  # String shorthand with provider prefix
  - openai/gpt-4o-mini
  - anthropic/claude-3-5-sonnet-20241022
  # Full form: virtual name, actual model, weight, limits
  - name: smart                    # virtual name sent by clients
    model: gpt-4o
    provider: openai
    weight: 3
    rpm: 1000
    tpm: 500000
  - name: smart                    # second deployment for "smart" = round-robin/failover
    model: claude-3-5-sonnet-20241022
    provider: anthropic
    weight: 1

```

--------------------------------

### Install Admin UI Dependencies

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/superpowers/plans/2026-04-05-react-admin-ui.md

Navigate to the admin UI directory and install project dependencies using npm.

```bash
cd crates/proxy/admin-ui
npm install
```

--------------------------------

### Install AnyLLM Proxy

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/crates/proxy/README.md

Command to install the AnyLLM Proxy using Cargo, assuming it's published to crates.io.

```bash
cargo install anyllm_proxy
```

--------------------------------

### AnyLLM Proxy Startup with Explicit Environment File

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/CONFIG.md

Command to start the AnyLLM Proxy while explicitly specifying the path to the environment file.

```bash
anyllm-proxy --env-file /path/to/my.env
```

--------------------------------

### Anthropic Messages API Example

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/ovhcloud.md

Example of how to call the Anthropic Messages API through the proxy. Ensure your `PROXY_API_KEYS` environment variable is set.

```bash
curl http://localhost:3000/v1/messages \
  -H "Content-Type: application/json" \
  -H "x-api-key: $PROXY_API_KEYS" \
  -d '{"model": "Meta-Llama-3.1-70B-Instruct", "max_tokens": 1024, "messages": [{"role": "user", "content": "Hello"}]}'
```

--------------------------------

### Run GMI Cloud Proxy with Environment Variables

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/gmi_cloud.md

Start the anyllm-proxy with GMI Cloud as the backend using environment variables for configuration. This is useful for quick local testing.

```bash
BACKEND=gmi GMI_CLOUD_API_KEY=your-key cargo run -p anyllm_proxy
# Docker:
docker run -e BACKEND=gmi -e GMI_CLOUD_API_KEY=your-key -e PROXY_OPEN_RELAY=true -p 3000:3000 followthewhit3rabbit/anyllm-proxy
```

--------------------------------

### Quick Start: Single-Backend (env vars)

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/cloudflare.md

Configure and run the proxy using environment variables for Cloudflare Workers AI. Ensure you replace `<your-account-id>` with your actual Cloudflare account ID.

```bash
BACKEND=cloudflare \
  CLOUDFLARE_API_KEY=your-token \
  OPENAI_BASE_URL=https://api.cloudflare.com/client/v4/accounts/<your-account-id>/ai/v1 \
  PROXY_OPEN_RELAY=true \
  cargo run -p anyllm_proxy
```

--------------------------------

### Featherless AI Anthropic Messages API Example

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/featherless_ai.md

Example of how to call the Featherless AI provider using the Anthropic Messages API format.

```bash
curl http://localhost:3000/v1/messages \
  -H "x-api-key: $PROXY_API_KEYS" \
  -H "Content-Type: application/json" \
  -d '{"model": "meta-llama/Meta-Llama-3.1-8B-Instruct", "max_tokens": 1024, "messages": [{"role": "user", "content": "Hello"}]}'
```

--------------------------------

### OpenAI Chat Completions API Example

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/ovhcloud.md

Example of how to call the OpenAI Chat Completions API through the proxy. Ensure your `PROXY_API_KEYS` environment variable is set.

```bash
curl http://localhost:3000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $PROXY_API_KEYS" \
  -d '{"model": "Meta-Llama-3.1-70B-Instruct", "messages": [{"role": "user", "content": "Hello"}]}'
```

--------------------------------

### Run anyllm-proxy with DeepSeek (Environment Variables)

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/providers/deepseek.md

Starts the anyllm-proxy using DeepSeek as the backend by setting the BACKEND and DEEPSEEK_API_KEY environment variables. This is useful for quick local testing.

```bash
BACKEND=deepseek DEEPSEEK_API_KEY=your-key cargo run -p anyllm_proxy
```

--------------------------------

### Implement GET /admin/api/uptime Endpoint

Source: https://github.com/whit3rabbit/anyllm-proxy/blob/main/docs/superpowers/plans/2026-04-05-react-admin-ui.md

This Rust code snippet implements the logic for the GET /admin/api/uptime endpoint, querying historical uptime data and formatting it.

```rust
ORDER BY day ASC",
).ok()
.and_then(|mut stmt| {
    stmt.query_map([], |r| {
        Ok(HistoryDay {
            date: r.get(0)?,
            status: if r.get::<_, i64>(1)? == 1 { "up".to_string() } else { "down".to_string() },
        })
    }).ok().map(|rows| rows.filter_map(|r| r.ok()).collect())
})
.unwrap_or_default();
```