LiteLLM

https://docs.litellm.ai/
Admin
LiteLLM provides a unified OpenAI-compatible interface to call and manage over 100 Large Language...
Tokens:4,833,711
Snippets:39,549
Trust Score:9
Update:1 day ago
Show doc for...
Tokens:
Raw
### Install dependencies for example server

Source: https://docs.litellm.ai/docs/adding_provider/generic_prompt_management_api

Install FastAPI and Uvicorn using uv to run the example prompt server.

```bash
uv add fastapi uvicorn
```

--------------------------------

### Clone and set up Mastra example

Source: https://docs.litellm.ai/docs/proxy/lens/integrations/mastra

Clone the repository, install dependencies, and copy the environment file for the Mastra example.

```bash
git clone https://github.com/BerriAI/litellm-lens-example.git
cd litellm-lens-example
npm install
cd mastra
cp .env.example .env
```

--------------------------------

### Run setup before the first turn with sandbox.docker

Source: https://docs.litellm.ai/docs/harness/sandboxes

Do any setup in your own code before starting a session, since there are no lifecycle hooks. This example creates a Docker sandbox with a mount, installs the package, then opens a Codex agent session using that sandbox.

```python
box = sandbox.docker("my-agents:latest", mounts={"./repo": "/workspace"})
await box.run(["pip", "install", "-e", ".[dev]"])

async with litellm.aagent_session(Harness.CODEX, sandbox=box, model="litellm_proxy/coder") as s:
    ...
```

--------------------------------

### Install Letta and LiteLLM with uv

Source: https://docs.litellm.ai/docs/integrations/letta

Install the required packages using uv. Run this before starting the integration.

```bash
uv add letta litellm
```

--------------------------------

### Quickstart script for macOS/Linux

Source: https://docs.litellm.ai/docs

Run this command to install LiteLLM on macOS or Linux using the quickstart script. It downloads and executes the script from the LiteLLM GitHub repository.

```shell
curl -fsSL https://raw.githubusercontent.com/BerriAI/litellm/main/scripts/quickstart.sh | sh
```

--------------------------------

### Clone and set up Hermes Agent example

Source: https://docs.litellm.ai/docs/proxy/lens/integrations/hermes-agent

Run these commands to clone the example repository, install Hermes Agent from a pinned source checkout, install the hermes-otel plugin, and copy the environment file. Requires uv and Git; Hermes requires CPython 3.14 with the GIL.

```bash
git clone https://github.com/BerriAI/litellm-lens-example.git
cd litellm-lens-example/hermes-agent
git clone https://github.com/NousResearch/hermes-agent.git vendor/hermes-agent
git -C vendor/hermes-agent checkout c225c4a04e8b517a357804ebb27367b0c961fd0e
uv sync --all-packages
HERMES_HOME="$PWD/home" uv run --package lens-hermes-agent-simple hermes plugins install briancaffey/hermes-otel/hermes_otel --ref d234865b13e997e0b3a50dc18aaad7c7d7b1b1fe --yes-deps --enable
cp .env.example .env
```

--------------------------------

### Run the example prompt server

Source: https://docs.litellm.ai/docs/adding_provider/generic_prompt_management_api

Start the prompt server by executing the saved prompt_server.py script.

```bash
python prompt_server.py
```

--------------------------------

### Clone Lens repository and start development environment

Source: https://docs.litellm.ai/docs/proxy/lens/deployment/development

Clone the Lens repository, install dependencies, and start the development server. The first start builds Lens, generates credentials, and starts the UI at http://127.0.0.1:3100/ui/.

```sh
git clone https://github.com/BerriAI/lens.git
cd lens
npm ci
npm run dev
```

--------------------------------

### Example GET Request with Custom Parameters

Source: https://docs.litellm.ai/docs/adding_provider/generic_prompt_management_api

Example request showing prompt_id and custom parameters project_name and slug.

```text
GET /beta/litellm_prompt_management?prompt_id=hello-world-prompt-2bac&project_name=litellm&slug=hello-world-prompt-2bac
```

--------------------------------

### Complete Greenscale integration example

Source: https://docs.litellm.ai/docs/observability/greenscale_integration

Full example showing environment variable setup, callback configuration, and a completion call with required Greenscale metadata. Requires GREENSCALE_API_KEY and GREENSCALE_ENDPOINT environment variables.

```python
from litellm import completion

## set env variables
os.environ['GREENSCALE_API_KEY'] = 'your-greenscale-api-key'
os.environ['GREENSCALE_ENDPOINT'] = 'greenscale-endpoint'
os.environ["OPENAI_API_KEY"]= ""

# set callback
litellm.success_callback = ["greenscale"]

#openai call
response = completion(
  model="gpt-5.6-luna",
  messages=[{"role": "user", "content": "Hi - i'm openai"}],
  metadata={
    "greenscale_project": "acme-project",
    "greenscale_application": "acme-application"
  }
)
```

--------------------------------

### Set up and run AIGatewayBench

Source: https://docs.litellm.ai/blog/rust-ai-gateway-benchmarks

Commands to create a virtual environment, install dependencies, start the mock upstream, start a gateway, and generate charts. Requires Python and Rust toolchain.

```bash
python -m venv .venv && . .venv/bin/activate && pip install -r requirements.txt
cargo run --release -p mock-upstream                 # deterministic upstream
# start a gateway from gateways/<name>/README.md, pointed at the mock
python analyze/make_chart.py                          # charts from results/
```

--------------------------------

### Set up Python virtual environment and install dependencies

Source: https://docs.litellm.ai/docs/tutorials/mongodb_vector_search

Creates a separate virtual environment and installs required packages for preparing sample data. PyMongo is used only for setup and is not a dependency of LiteLLM.

```bash
python -m venv .venv-mongodb-setup
source .venv-mongodb-setup/bin/activate
pip install openai pymongo
```

--------------------------------

### Clone and run the complete interactive CLI agent example

Source: https://docs.litellm.ai/docs/tutorials/claude_agent_sdk

Clone the LiteLLM repository, navigate to the Anthropic Agent SDK cookbook example, install dependencies, and run the interactive CLI agent. This example streams responses in real-time, switches between models dynamically, and fetches available models from the proxy.

```bash
# Clone and run the example
git clone https://github.com/BerriAI/litellm.git
cd litellm/cookbook/anthropic_agent_sdk
uv add -r requirements.txt
python main.py
```

--------------------------------

### Run simple example against the adapter

Source: https://docs.litellm.ai/docs/proxy/lens/integrations/claude-agent-sdk

Run the simple example with the adapter. Keep MOCK_LITELLM_GATEWAY_URL empty because the adapter handles any additional recorder export.

```bash
LITELLM_GATEWAY_URL=http://localhost:4319 MOCK_LITELLM_GATEWAY_URL= \
  uv run --env-file .env --package lens-claude-agent-sdk-simple simple/main.py
```

--------------------------------

### Setup environment variables for LiteLLM proxy

Source: https://docs.litellm.ai/docs/pass_through/anthropic_completion

Set the required environment variables before starting the LiteLLM proxy. DATABASE_URL is for the database connection, LITELLM_MASTER_KEY for authentication, and COHERE_API_KEY is an example of a provider key.

```bash
export DATABASE_URL=""
export LITELLM_MASTER_KEY=""
export COHERE_API_KEY=""
```

--------------------------------

### Quickstart script for Windows (PowerShell)

Source: https://docs.litellm.ai/docs/proxy/docker_quick_start.md

Run this command to start LiteLLM on Windows using PowerShell. After it finishes, open the Admin UI to add models, create virtual keys, and call the gateway.

```powershell
irm https://raw.githubusercontent.com/BerriAI/litellm/main/scripts/quickstart.ps1 | iex
```

--------------------------------

### Get Prompt Info Example Response

Source: https://docs.litellm.ai/api-reference#/prompts/get_prompt_info_prompts__prompt_id__get

Example JSON response for the Get Prompt Info endpoint, showing prompt metadata and content.

```json
{
    "prompt_id": "my_prompt_id",
    "litellm_params": {
        "prompt_id": "my_prompt_id",
        "prompt_integration": "dotprompt",
        "prompt_directory": "/path/to/prompts"
    },
    "prompt_info": {
        "prompt_type": "config"
    },
    "created_at": "2023-11-09T12:34:56.789Z",
    "updated_at": "2023-11-09T12:34:56.789Z",
    "content": "System: You are a helpful assistant.

User: {{user_message}}"
}
```

--------------------------------

### Get Response - Example JSON Response

Source: https://docs.litellm.ai/api-reference#/responses/get_response_v1_responses__response_id__get

Example JSON response for a successful GET request to /v1/responses/{response_id}. Shows the structure of the response object.

```json
{
  "id": "string",
  "created_at": 0,
  "error": {
    "additionalProp1": {}
  },
  "incomplete_details": {
    "reason": "max_output_tokens",
    "additionalProp1": {}
  },
  "instructions": "string",
  "metadata": {
    "additionalProp1": {}
  },
  "model": "string",
  "object": "string",
  "output": [
    {
      "id": "string",
      "content": [
        {
          "annotations": [
            {
              "file_id": "string",
              "filename": "string",
              "index": 0,
              "type": "file_citation",
              "additionalProp1": {}
            },
            {
              "end_index": 0,
              "start_index": 0,
              "title": "string",
              "type": "url_citation",
              "url": "string",
              "additionalProp1": {}
            },
            {
              "container_id": "string",
              "end_index": 0,
              "file_id": "string",
              "filename": "string",
              "start_index": 0,
              "type": "container_file_citation",
              "additionalProp1": {}
            },
            {
              "file_id": "string",
              "index": 0,
              "type": "file_path",
              "additionalProp1": {}
            }
          ],
          "text": "string",
          "type": "output_text",
          "logprobs": [
            {
              "token": "string",
              "bytes": [
                0
              ],
              "logprob": 0,
              "top_logprobs": [
                {
                  "token": "string",
                  "bytes": [
                    0
                  ],
                  "logprob": 0,
                  "additionalProp1": {}
                }
              ],
              "additionalProp1": {}
            }
          ],
          "additionalProp1": {}
        },
        {
          "refusal": "string",
          "type": "refusal",
          "additionalProp1": {}
        }
      ],
      "role": "assistant",
      "status": "in_progress",
      "type": "message",
      "phase": "commentary",
      "additionalProp1": {}
    },
    {
      "id": "string",
      "queries": [
        "string"
      ],
      "status": "in_progress",
      "type": "file_search_call",
      "results": [
        {
          "attributes": {
            "additionalProp1": "string",
            "additionalProp2": "string",
            "additionalProp3": "string"
          },
          "file_id": "string",
          "filename": "string",
          "score": 0,
          "text": "string",
          "additionalProp1": {}
        }
      ],
      "additionalProp1": {}
    },
    {
      "arguments": "string",
      "call_id": "string",
      "name": "string",
      "type": "function_call",
      "id": "string",
      "caller": {
        "type": "direct",
        "additionalProp1": {}
      },
      "namespace": "string",
      "status": "in_progress",
      "additionalProp1": {}
    },
    {
      "id": "string",
      "call_id": "string",
      "output": "string",
      "status": "in_progress",
      "type": "function_call_output",
      "caller": {
        "type": "direct",
        "additionalProp1": {}
      },
      "created_by": "string",
      "name": "string",
      "namespace": "string",
      "additionalProp1": {}
    },
    {
      "id": "string",
      "action": {
        "type": "search",
        "queries": [
          "string"
        ],
        "query": "string",
        "sources": [
          {
            "type": "url",
            "url": "string",
            "additionalProp1": {}
          }
        ],
        "additionalProp1": {}
      },
      "status": "in_progress",
      "type": "web_search_call",
      "additionalProp1": {}
    },
    {
      "id": "string",
      "call_id": "string",
```

--------------------------------

### Get Prompt Versions Example Response

Source: https://docs.litellm.ai/api-reference#/prompts/get_prompt_versions_prompts__prompt_id__versions_get

Example JSON response for the get prompt versions endpoint, showing a list of prompt versions with their metadata.

```json
{
 "prompts": [
 {
 "prompt_id": "jack_success.v1",
 "litellm_params": {...},
 "prompt_info": {"prompt_type": "db"},
 "created_at": "2023-11-09T12:34:56.789Z",
 "updated_at": "2023-11-09T12:34:56.789Z"
 },
 {
 "prompt_id": "jack_success.v2",
 "litellm_params": {...},
 "prompt_info": {"prompt_type": "db"},
 "created_at": "2023-11-09T13:45:12.345Z",
 "updated_at": "2023-11-09T13:45:12.345Z"
 }
 ]
}
```

--------------------------------

### Get Provider Specific Params Example Response

Source: https://docs.litellm.ai/api-reference#/guardrails/get_provider_specific_params_guardrails_ui_provider_specific_params_get

Example JSON response for the GET /guardrails/ui/provider_specific_params endpoint, showing provider-specific parameters for Bedrock and Azure Content Safety Text Moderation guardrails.

```json
{
 "bedrock": {
 "guardrailIdentifier": {
 "description": "The ID of your guardrail on Bedrock",
 "required": true,
 "type": null
 },
 "guardrailVersion": {
 "description": "The version of your Bedrock guardrail (e.g., DRAFT or version number)",
 "required": true,
 "type": null
 }
 },
 "azure_content_safety_text_moderation": {
 "api_key": {
 "description": "API key for the Azure Content Safety Text Moderation guardrail",
 "required": false,
 "type": null
 },
 "optional_params": {
 "description": "Optional parameters for the Azure Content Safety Text Moderation guardrail",
 "required": true,
 "type": "nested",
 "fields": {
 "severity_threshold": {
 "description": "Severity threshold for the Azure Content Safety Text Moderation guardrail across all categories",
 "required": false,
 "type": null
 },
 "categories": {
 "description": "Categories to scan for the Azure Content Safety Text Moderation guardrail",
 "required": false,
 "type": "multiselect",
 "options": ["Hate", "SelfHarm", "Sexual", "Violence"],
 "default_value": None
 }
 }
 }
 }
}
```

--------------------------------

### Clone repository and set up environment for OpenAI Agents SDK example

Source: https://docs.litellm.ai/docs/proxy/lens/integrations/openai-agents

Run these commands to clone the example repository and copy the environment template for direct provider calls. Use this for a fresh checkout.

```bash
git clone https://github.com/BerriAI/litellm-lens-example.git
cd litellm-lens-example/openai-agents
cp .env.direct.example .env
```

--------------------------------

### Clone LiteLLM Lens Example Repository and Set Up Environment

Source: https://docs.litellm.ai/docs/proxy/lens/integrations/claude-agent-sdk

Run these commands to clone the example repository and create a .env file from the template. Use this for a fresh checkout.

```bash
git clone https://github.com/BerriAI/litellm-lens-example.git
cd litellm-lens-example/claude-agent-sdk
cp .env.example .env
```

--------------------------------

### Start LiteLLM Proxy with Environment Variables

Source: https://docs.litellm.ai/docs/tutorials/claude_code_max_subscription

Set the master key and database URL, then start the proxy with a config file. The proxy runs on http://0.0.0.0:4000.

```bash
export LITELLM_MASTER_KEY="sk-<a-long-random-key>"
export DATABASE_URL="postgresql://<user>:<password>@<host>:5432/<database>"
litellm --config /path/to/config.yaml

# RUNNING on http://0.0.0.0:4000
```

--------------------------------

### Clone and set up Vercel AI SDK (TypeScript) example

Source: https://docs.litellm.ai/docs/proxy/lens/integrations/vercel-ai-sdk-js

Run these commands to clone the litellm-lens-example repository, install dependencies, navigate to the vercel-ai-sdk-js directory, and copy the .env.example file to .env. Use this for a fresh checkout.

```bash
git clone https://github.com/BerriAI/litellm-lens-example.git
cd litellm-lens-example
npm install
cd vercel-ai-sdk-js
cp .env.example .env
```

--------------------------------

### Install OpenAI JavaScript SDK

Source: https://docs.litellm.ai/docs/observability/zerobus

Install the OpenAI JavaScript SDK in your Node.js project before running the JavaScript example.

```bash
npm install openai
```

--------------------------------

### Install OpenAI Python SDK

Source: https://docs.litellm.ai/docs/observability/zerobus

Install the OpenAI Python SDK in your client environment before running the Python example.

```bash
python3 -m pip install openai
```

--------------------------------

### Launch OpenCode

Source: https://docs.litellm.ai/docs/tutorials/opencode_integration

Start the OpenCode application.

```bash
opencode
```

--------------------------------

### Quick Start with Logfire and LiteLLM

Source: https://docs.litellm.ai/docs/observability/logfire_integration

Set up environment variables for Logfire token, base URL, and OpenAI API key, then make a completion call with LiteLLM using Logfire as a success callback.

```python
# uv add logfire
import litellm
import os

# from https://logfire.pydantic.dev/
os.environ["LOGFIRE_TOKEN"] = ""

# Optionally customize the base url
# from https://logfire.pydantic.dev/
os.environ["LOGFIRE_BASE_URL"] = ""

# LLM API Keys
os.environ['OPENAI_API_KEY']=""

# set logfire as a callback, litellm will send the data to logfire
litellm.success_callback = ["logfire"]

# openai call
response = litellm.completion(
  model="gpt-5.6-luna",
  messages=[
    {"role": "user", "content": "Hi - i'm openai"}
  ]
)
```

--------------------------------

### Get Policy Response Example

Source: https://docs.litellm.ai/api-reference#/policy_engine/get_policy_policies__policy_id__get

Example JSON response for a successful policy retrieval, showing the policy's structure and fields.

```json
{
  "condition": {
    "additionalProp1": {}
  },
  "created_at": "2026-10-19T14:23:23.742Z",
  "created_by": "string",
  "definition_location": "db",
  "description": "string",
  "guardrails_add": [
    "string"
  ],
  "guardrails_remove": [
    "string"
  ],
  "inherit": "string",
  "is_latest": true,
  "parent_version_id": "string",
  "pipeline": {
    "additionalProp1": {}
  },
  "policy_id": "string",
  "policy_name": "string",
  "production_at": "2026-10-19T14:23:23.743Z",
  "published_at": "2026-10-19T14:23:23.743Z",
  "updated_at": "2026-10-19T14:23:23.743Z",
  "updated_by": "string",
  "version_number": 1,
  "version_status": "production"
}
```

--------------------------------

### High-availability setup with multiple providers

Source: https://docs.litellm.ai/docs/routing

Example config.yaml for a high-availability setup with multiple providers. Shows primary, BYOK, and fallback deployments.

```yaml
model_list:
  - model_name: sonnet-4              # Primary: Anthropic Direct
    litellm_params:
      model: anthropic/claude-sonnet-5
      api_key: <anthropic-key>
      
  - model_name: byok-sonnet-4         # BYOK: Customer-managed keys
    litellm_params:
      model: anthropic/claude-sonnet-5
      api_key: <customer-managed-key>
      api_base: https://proxy.litellm.ai/api.anthropic.com
      
  - model_name: sonnet-4              # Fallback: Vertex AI
    litellm_params:
      model: vertex_ai/claude-sonnet-5
      vertex_project: my-project
```

--------------------------------

### Get fallback configuration using cURL

Source: https://docs.litellm.ai/docs/proxy/fallback_management

cURL example for GET /fallback/{model} to retrieve fallbacks. Requires LITELLM_API_KEY environment variable.

```bash
curl -X GET "http://localhost:4000/fallback/gpt-5.6-luna?fallback_type=general" \
  -H "Authorization: Bearer $LITELLM_API_KEY"
```

--------------------------------

### Create README.md with usage examples and format details

Source: https://docs.litellm.ai/docs/adding_provider/adding_guardrail_support

Create a README.md file that documents the handler, including an overview, data format, and examples. The example shows the structure for input and output formats.

```markdown
# {Provider} {Endpoint} Guardrail Translation Handler

Handler for processing {Provider}'s {Endpoint} with guardrails.

## Overview

This handler processes {Endpoint} input/output by:
1. Extracting text from messages/responses
2. Applying guardrails to text content
3. Mapping guardrailed text back to original structure

## Data Format

### Input Format
```json
{
  "field": "value",
  "messages": [...]
}
````
```

--------------------------------

### Start LiteLLM proxy with ScaleDown config

Source: https://docs.litellm.ai/docs/providers/scaledown

Generate a master key, print it, and start the proxy using config.yaml on 127.0.0.1:4000. Run this in the shell where SCALEDOWN_API_KEY is set.

```bash
export LITELLM_MASTER_KEY="sk-$(openssl rand -hex 32)"
printf 'Local proxy key: %s\n' "$LITELLM_MASTER_KEY"
litellm --config config.yaml --host 127.0.0.1 --port 4000
```

--------------------------------

### Start billing with curl

Source: https://docs.litellm.ai/docs/proxy/billing

Make a chat completion request using the team's key to start billing. This example uses curl.

```bash
# Authorization: Team's Key
curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer sk-tXL0wt5-lOOVK9sfY2UacA' \
--data ' {
      "model": "fake-openai-endpoint",
      "messages": [
        {
          "role": "user",
          "content": "what llm are you"
        }
      ],
    }
'
```

--------------------------------

### Get Prompt Info - 422 Validation Error Response

Source: https://docs.litellm.ai/api-reference#/prompts/get_prompt_info_prompts__prompt_id__info_get

Example response for a 422 Validation Error from the Get Prompt Info endpoint.

```json
{
  "detail": [
    {
      "loc": [
        "string",
        0
      ],
      "msg": "string",
      "type": "string",
      "input": "string",
      "ctx": {}
    }
  ]
}
```

--------------------------------

### Start the UI development server

Source: https://docs.litellm.ai/docs/extras/contributing_code

Run the development server to preview the UI locally.

```shell
npm run dev
```

--------------------------------

### Complete Supabase logging example with LiteLLM

Source: https://docs.litellm.ai/docs/observability/supabase_integration

Full example showing environment variable setup, callback configuration, and making both successful and failing completion calls to test logging. Requires SUPABASE_URL, SUPABASE_KEY, and OPENAI_API_KEY environment variables.

```python
from litellm import completion

## set env variables
### SUPABASE
os.environ["SUPABASE_URL"] = "your-supabase-url" 
os.environ["SUPABASE_KEY"] = "your-supabase-key" 

## LLM API KEY
os.environ["OPENAI_API_KEY"] = ""

# set callbacks
litellm.success_callback=["supabase"]
litellm.failure_callback=["supabase"]

# openai call
response = completion(
  model="gpt-5.6-luna", 
  messages=[{"role": "user", "content": "Hi - i'm openai"}],
  user="ishaan22" # identify users
) 

# bad call, expect this call to fail and get logged
response = completion(
  model="chatgpt-test", 
  messages=[{"role": "user", "content": "Hi - i'm a bad call to test error logging"}]
)
 
```

--------------------------------

### Start Locust with 4 processes

Source: https://docs.litellm.ai/docs/load_test_advanced

Run this command in the same directory as your locustfile.py to start Locust with 4 worker processes. This is step 4 of the load test setup.

```shell
locust -f locustfile.py --processes 4
```

--------------------------------

### Install OpenCode via one-line script

Source: https://docs.litellm.ai/docs/tutorials/opencode_integration

Recommended installation method for OpenCode.

```bash
curl -fsSL https://opencode.ai/install | bash
```

--------------------------------

### List Budgets 200 response example JSON

Source: https://docs.litellm.ai/api-reference#/budget%20management/list_budgets_management_v1_budgets_get

Example value for the successful 200 response of GET /management/v1/budgets, showing data, meta, and links objects.

```json
{
  "data": [
    {
      "budget_id": "string",
      "max_budget": 0,
      "soft_budget": 0,
      "tpm_limit": 0,
      "rpm_limit": 0,
      "tpd_limit": 0,
      "budget_duration": "string",
      "budget_reset_at": "2026-10-19T14:21:06.818Z",
      "created_at": "2026-10-19T14:21:06.818Z",
      "updated_at": "2026-10-19T14:21:06.818Z"
    }
  ],
  "meta": {
    "total_count": 0,
    "page": 0,
    "page_size": 0,
    "total_pages": 0
  },
  "links": {
    "self": "string",
    "first": "string",
    "prev": "string",
    "next": "string",
    "last": "string"
  }
}
```

--------------------------------

### Complete example with OpenAI call

Source: https://docs.litellm.ai/docs/observability/wandb_integration

A full example that sets up environment variables, configures the wandb callback, and makes an OpenAI completion call.

```python
# uv add wandb 
import litellm
import os

os.environ["WANDB_API_KEY"] = ""
# LLM API Keys
os.environ['OPENAI_API_KEY']=""

# set wandb as a callback, litellm will send the data to Weights & Biases
litellm.success_callback = ["wandb"] 
 
# openai call
response = litellm.completion(
  model="gpt-5.6-luna",
  messages=[
    {"role": "user", "content": "Hi - i'm openai"}
  ]
)
```

--------------------------------

### Get an agent response

Source: https://docs.litellm.ai/blog/google-ai-studio-managed-agents

Example response from the get agent endpoint, showing the agent's configuration including base agent, system instruction, and environment.

```json
{
    "id": "my-custom-slides-agent",
    "base_agent": "antigravity-preview-05-2026",
    "system_instruction": "You are a helpful assistant that creates slides.",
    "base_environment": {
        "sources": [
            {
                "type": "gcs",
                "source": "gs://eap-templates/slides-skill",
                "target": "/.agents/skills/slides-skill"
            }
        ],
        "type": "remote"
    }
}
```

--------------------------------

### Start LiteLLM Proxy with EXPERIMENTAL_UI_LOGIN inline

Source: https://docs.litellm.ai/docs/proxy/cli_sso

Alternatively, add the environment variable directly to your proxy startup command.

```bash
EXPERIMENTAL_UI_LOGIN="True" litellm --config config.yaml
```

--------------------------------

### Get Guardrail Info - Example Response

Source: https://docs.litellm.ai/api-reference#/guardrails/get_guardrail_info_guardrails__guardrail_id__info_get

This is the JSON response returned by the Get Guardrail Info endpoint, containing guardrail details such as name, parameters, and timestamps.

```json
{
 "guardrail_id": "123e4567-e89b-12d3-a456-426614174000",
 "guardrail_name": "my-bedrock-guard",
 "litellm_params": {
 "guardrail": "bedrock",
 "mode": "pre_call",
 "guardrailIdentifier": "ff6ujrregl1q",
 "guardrailVersion": "DRAFT",
 "default_on": true
 },
 "guardrail_info": {
 "description": "Bedrock content moderation guardrail"
 },
 "created_at": "2023-11-09T12:34:56.789Z",
 "updated_at": "2023-11-09T12:34:56.789Z"
}
```

--------------------------------

### Quick Start

Source: https://docs.litellm.ai/docs/providers/voyage

Quick Start

```python
from litellm import rerank
import os

os.environ["VOYAGE_API_KEY"] = "your-api-key"

response = rerank(
    model="voyage/rerank-2.5",
    query="What is the capital of France?",
    documents=[
        "Paris is the capital of France.",
        "London is the capital of England.",
        "Berlin is the capital of Germany.",
    ],
    top_n=3,
)

print(response)
```

--------------------------------

### Run OpenClaw onboarding wizard

Source: https://docs.litellm.ai/docs/tutorials/openclaw_integration

Start the interactive onboarding wizard to connect OpenClaw to LiteLLM. Follow the prompts to select LiteLLM as the provider and enter your master key and base URL.

```bash
openclaw onboard --install-daemon
```

--------------------------------

### Install LiteLLM v1.89.1 via pip

Source: https://docs.litellm.ai/release_notes/v1.89.1/v1-89-1

Install the LiteLLM Python package version 1.89.1 using pip. This is the recommended way to get the latest patch release.

```bash
pip install litellm==1.89.1
```

--------------------------------

### Install LiteLLM v1.85.1 via pip

Source: https://docs.litellm.ai/release_notes/v1.85.1/v1-85-1

Install the LiteLLM Python package version 1.85.1 using pip. This is the recommended way to get the latest patch release.

```bash
pip install litellm==1.85.1
```

--------------------------------

### Start LiteLLM Proxy with config.yaml

Source: https://docs.litellm.ai/docs/providers/openinfer

Set a master key, print it, and start the LiteLLM Proxy server using the config file. The proxy runs on http://0.0.0.0:4000.

```bash
export LITELLM_MASTER_KEY="sk-$(openssl rand -hex 32)"
printf 'Local proxy key: %s\n' "$LITELLM_MASTER_KEY"
litellm --config config.yaml

# RUNNING on http://0.0.0.0:4000
```

--------------------------------

### Response from starting a new session

Source: https://docs.litellm.ai/docs/response_api

Example JSON response from the LiteLLM proxy when starting a new session. The id field (resp_123abc) is used as previous_response_id in subsequent requests.

```json
{
  "id":"resp_123abc",
  "model":"claude-sonnet-5",
  "output":[{
    "type":"message",
    "content":[{
      "type":"output_text",
      "text":"Michael Jordan is widely considered one of the greatest basketball players of all time. He played for the Chicago Bulls (1984-1993, 1995-1998) and Washington Wizards (2001-2003), winning 6 NBA Championships with the Bulls."
    }]
  }]
}
```

--------------------------------

### Setup config.yaml

Source: https://docs.litellm.ai/docs/providers/vertex

Create a config.yaml file to define the model list and LiteLLM parameters for the proxy.

```yaml
model_list:
  - model_name: gemini-3.1-pro-preview
    litellm_params:
      model: vertex_ai/gemini-3.1-pro-preview
      vertex_project: your-project
      vertex_location: us-central1
```

--------------------------------

### Quick Start with Python SDK

Source: https://docs.litellm.ai/blog/gemini_3

Use the LiteLLM Python SDK to call the Gemini 3 Pro Preview model with reasoning effort set to low.

```python
from litellm import completion
import os

os.environ["GEMINI_API_KEY"] = "your-api-key"

response = completion(
    model="gemini/gemini-3-pro-preview",
    messages=[{"role": "user", "content": "Hello!"}],
    reasoning_effort="low"
)

print(response.choices[0].message.content)
```

--------------------------------

### Setup config.yaml

Source: https://docs.litellm.ai/docs/completion/image_generation_chat

Setup config.yaml

```yaml
model_list:
  - model_name: gemini-image-gen
    litellm_params:
      model: gemini/gemini-2.5-flash-image-preview
      api_key: os.environ/GEMINI_API_KEY
```

--------------------------------

### GET /authorize/flow

Source: https://docs.litellm.ai/api-reference#/llm_passthrough/assemblyai_proxy_route_assemblyai__endpoint__delete

Retrieves the authorization flow details or initiates a specific flow. This endpoint may be used to get information about the current authorization flow or to start a new one.

```APIDOC
## GET /authorize/flow

### Description
Retrieves the authorization flow details or initiates a specific flow. This endpoint may be used to get information about the current authorization flow or to start a new one.

### Method
GET

### Endpoint
/authorize/flow

### Parameters
#### Query Parameters
- **client_id** (string) - Optional - The client identifier
- **response_type** (string) - Optional - The response type

### Response
#### Success Response (200)
- **flow_id** (string) - The flow identifier
- **status** (string) - The status of the flow

#### Response Example
{
  "flow_id": "flow_123",
  "status": "pending"
}
```

--------------------------------

### Install Moyai and set up environment

Source: https://docs.litellm.ai/docs/self_hosted_coding_agents/moyai/setup

Run these commands to install Python 3.12, clone the Moyai repository, sync dependencies, and create a .env file. Ensure you are in the moyai directory for subsequent steps.

```bash
uv python install 3.12
git clone https://github.com/BerriAI/moyai.git
cd moyai
uv sync --frozen --python 3.12
cp .env.example .env
chmod 600 .env
```

--------------------------------

### Add Marketplace and Install Plugin in Claude Code

Source: https://docs.litellm.ai/docs/tutorials/claude_code_plugin_marketplace

Commands for engineers to add the internal marketplace and install company plugins. The first command is a one-time setup per engineer.

```bash
# One-time setup for each engineer
claude plugin marketplace add http://litellm.internal.company.com/claude-code/marketplace.json

# Install company plugins
claude plugin install internal-tools@litellm
```

--------------------------------

### LiteLLM Proxy Server setup: config.yaml and start command

Source: https://docs.litellm.ai/docs/batches

Define model list in config.yaml and start the LiteLLM proxy server. The proxy runs on http://0.0.0.0:4000 by default.

```yaml
model_list:
  - model_name: gpt-4o
    litellm_params:
      model: openai/gpt-4o
      api_key: os.environ/OPENAI_API_KEY
```

```bash
$ export OPENAI_API_KEY="sk-..."

$ litellm --config config.yaml

# RUNNING on http://0.0.0.0:4000
```

--------------------------------

### Example API calls for dotprompt, BitBucket, and Gitlab

Source: https://docs.litellm.ai/docs/proxy/native_litellm_prompt

Example API calls for File System (dotprompt), BitBucket, and Gitlab integrations. Shows how to use the parameters in practice.

```python
# File system integration
response = litellm.completion(
    model="dotprompt/gpt-5.6-terra",
    prompt_id="hello",
    prompt_variables={"user_message": "Hello world"},
    messages=[{"role": "user", "content": "This will be ignored"}]
)

# BitBucket integration
response = litellm.completion(
    model="bitbucket/gpt-5.6-terra",
    prompt_id="hello",
    prompt_variables={"user_message": "Hello world"},
    bitbucket_config={
        "workspace": "your-workspace",
        "repository": "your-repo",
        "access_token": "your-token"
    }
)

# Gitlab integration
response = litellm.completion(
    model="gitlab/gpt-5.6-terra",
    prompt_id="hello",
    prompt_variables={"user_message": "Hello world"},
    gitlab_config={
        "project": "a/b/<repo_name>",
        "access_token": "your-access-token",
        "base_url": "gitlab url",
        "prompts_path": "src/prompts", # folder to point to, defaults to root
        "branch":"main"  # optional, defaults to main
    }
)
```

--------------------------------

### Complete Example

Source: https://docs.litellm.ai/docs/providers/perplexity

Comprehensive example demonstrating the use of the Perplexity provider with multiple features including tools, instructions, and parameters.

```APIDOC
## Complete Example

### Description
A comprehensive example showing how to use the Perplexity provider with LiteLLM's `responses` function, including web search and fetch URL tools, custom instructions, and output parameters.

### Method
Function call

### Endpoint
`responses(...)`

### Parameters
- **model** (string) - Required - The model identifier, e.g., `perplexity/openai/gpt-5.6-terra`.
- **input** (string) - Required - The input prompt.
- **custom_llm_provider** (string) - Optional - Set to `perplexity` to specify the provider.
- **tools** (list) - Optional - List of tools, e.g., `[{"type": "web_search"}, {"type": "fetch_url"}]`.
- **instructions** (string) - Optional - Instructions for the model.
- **max_output_tokens** (integer) - Optional - Maximum number of tokens to generate.
- **temperature** (float) - Optional - Sampling temperature.

### Request Example
```python
from litellm import responses
import os

os.environ['PERPLEXITY_API_KEY'] = ""

response = responses(
    model="perplexity/openai/gpt-5.6-terra",
    input="Research the latest developments in quantum computing and provide sources",
    custom_llm_provider="perplexity",
    tools=[
        {"type": "web_search"},
        {"type": "fetch_url"}
    ],
    instructions="Use web_search to find relevant information and fetch_url to retrieve detailed content from sources. Provide citations for all claims.",
    max_output_tokens=1000,
    temperature=0.7,
)
```

### Response
Returns a response object with fields such as `id`, `model`, `status`, `output`, and `usage`.

### Response Example
```python
print(f"Response ID: {response.id}")
print(f"Model: {response.model}")
print(f"Status: {response.status}")
print(f"Output: {response.output}")
print(f"Usage: {response.usage}")
```
```
Tokens:
Raw