### Start LiteLLM Proxy for Initial Setup

Source: https://docs.litellm.ai/docs/tutorials/livekit_xai_realtime

Initiate the LiteLLM Proxy server with the provided configuration file and port as part of the initial setup.

```bash
litellm --config config.yaml --port 4000
```

--------------------------------

### Start UI in Development Mode

Source: https://docs.litellm.ai/docs/contributing

Navigate to the UI directory, install dependencies, and start the UI in development mode for hot reloading. The proxy should be running separately.

```bash
cd ui/litellm-dashboard  
npm install  
npm run dev  

```

--------------------------------

### LiteLLM Proxy Configuration Examples

Source: https://docs.litellm.ai/docs/tutorials/openclaw_integration

Examples of `litellm_config.yaml` files for configuring LiteLLM Proxy, demonstrating single and multi-provider setups.

```yaml
model_list:  
  - model_name: gpt-4o  
    litellm_params:  
      model: openai/gpt-4o  
      api_key: os.environ/OPENAI_API_KEY  
  
general_settings:  
  master_key: sk-your-secret-key  # pick any value — this is YOUR proxy password  
```

```yaml
model_list:  
  - model_name: gpt-4o  
    litellm_params:  
      model: openai/gpt-4o  
      api_key: os.environ/OPENAI_API_KEY  
  
  - model_name: claude-sonnet  
    litellm_params:  
      model: anthropic/claude-sonnet-4-20250514  
      api_key: os.environ/ANTHROPIC_API_KEY  
  
  - model_name: gemini-flash  
    litellm_params:  
      model: gemini/gemini-2.0-flash  
      api_key: os.environ/GEMINI_API_KEY  
  
general_settings:  
  master_key: sk-your-secret-key  
```

--------------------------------

### Realistic Input Examples for Tool Parameters (Good)

Source: https://docs.litellm.ai/docs/providers/anthropic_tool_input_examples

Provide realistic, production-like data in `input_examples` to guide the model effectively, avoiding generic placeholders.

```json
# ✅ Good - realistic examples  
"input_examples": [
    {"email": "alice@company.com", "role": "admin"},
    {"email": "bob@company.com", "role": "user"}
]
```

--------------------------------

### Complete Athina Integration Example

Source: https://docs.litellm.ai/docs/observability/athina_integration

Demonstrates a full integration setup, including setting Athina and OpenAI API keys as environment variables, enabling the Athina callback, and making an LLM completion call.

```python
from litellm import completion  
  
## set env variables  
os.environ["ATHINA_API_KEY"] = "your-athina-api-key"  
os.environ["OPENAI_API_KEY"]= ""  
  
# set callback  
litellm.success_callback = ["athina"]  
  
#openai call  
response = completion(  
  model="gpt-3.5-turbo",   
  messages=[{"role": "user", "content": "Hi 👋 - i'm openai"}]  
)   
```

--------------------------------

### Minimal FastAPI Example for Prompt Management Server

Source: https://docs.litellm.ai/docs/adding_provider/generic_prompt_management_api

This provides the initial setup for a basic FastAPI application, serving as a starting point for implementing a custom prompt management server.

```python
from fastapi import FastAPI, HTTPException, Header  
from typing import Optional, Dict, Any, List  
from pydantic import BaseModel  
  
app = FastAPI()  
  
```

--------------------------------

### Example LiteLLM Configuration for Enterprise Callbacks

Source: https://docs.litellm.ai/docs/troubleshoot/max_callbacks

An example `config.yaml` demonstrating global callback settings for a large enterprise setup, where individual teams might add more callbacks.

```yaml
# config.yaml  
litellm_settings:  
  callbacks: ["prometheus", "langfuse"]  # 2 global callbacks  
  
# Each team adds 1 guardrail callback = 60+ callbacks  
# Total: 62+ callbacks needed  
```

--------------------------------

### Install LiteLLM Proxy with curl

Source: https://docs.litellm.ai/docs/proxy/docker_quick_start

Use this command to automatically detect your OS, install `litellm[proxy]`, and launch the setup wizard for local development.

```bash
curl -fsSL https://raw.githubusercontent.com/BerriAI/litellm/main/scripts/install.sh | sh
```

--------------------------------

### Full LiteLLM and Weights & Biases Integration Example

Source: https://docs.litellm.ai/docs/observability/wandb_integration

This example demonstrates a complete setup for integrating LiteLLM with Weights & Biases, including API key configuration and making an OpenAI call with W&B logging enabled.

```python
# uv add wandb   
import litellm  
import os  
  
os.environ["WANDB_API_KEY"] = ""  
# LLM API Keys  
os.environ['OPENAI_API_KEY']=""  
  
# set wandb as a callback, litellm will send the data to Weights & Biases  
litellm.success_callback = ["wandb"]   
   
# openai call  
response = litellm.completion(  
  model="gpt-3.5-turbo",  
  messages=[  
    {"role": "user", "content": "Hi 👋 - i'm openai"}  
  ]  
)  
```

--------------------------------

### Start LiteLLM Proxy with Custom Configuration (Shell)

Source: https://docs.litellm.ai/docs/proxy/custom_auth

Launch the LiteLLM proxy, pointing it to your `config.yaml` file which includes the custom authentication setup.

```bash
$ litellm --config /path/to/config.yaml   
```

--------------------------------

### Perform Web Search via LiteLLM Proxy Server

Source: https://docs.litellm.ai/docs/integrations/websearch_interception

This example demonstrates how to start the LiteLLM proxy server with a configuration file and then make a `curl` request to the proxy, utilizing the `litellm_web_search` tool for web queries.

```bash
# Start proxy with config  
litellm --config config.yaml  
  
# Make request  
curl http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-1234" \
  -d '{  
    "model": "gpt-4o",  
    "messages": [  
      {"role": "user", "content": "What is the weather in San Francisco?"}  
    ],  
    "tools": [  
      {  
        "type": "function",  
        "function": {  
          "name": "litellm_web_search",  
          "description": "Search the web",  
          "parameters": {  
            "type": "object",  
            "properties": {  
              "query": {"type": "string"}  
            },  
            "required": ["query"]  
          }  
        }  
      }  
    ]  
  }'
```

--------------------------------

### Start LiteLLM Proxy for Quick Start

Source: https://docs.litellm.ai/docs/pass_through/langfuse

Command to start the LiteLLM proxy server, which will listen on port 4000 by default, enabling Langfuse pass-through.

```bash
litellm  
  
# RUNNING on http://0.0.0.0:4000  
```

--------------------------------

### Run LiteLLM Setup Wizard

Source: https://docs.litellm.ai/docs/proxy/docker_quick_start

Initiate the interactive setup wizard to configure LLM providers, API keys, port, and master key, saving the configuration to `litellm_config.yaml`.

```bash
$ litellm --setup

  Welcome to LiteLLM

  Choose your LLM providers
  ○ 1. OpenAI        GPT-4o, GPT-4o-mini, o1
  ○ 2. Anthropic     Claude Opus, Sonnet, Haiku
  ○ 3. Azure OpenAI  GPT-4o via Azure
  ○ 4. Google Gemini Gemini 2.0 Flash, 1.5 Pro
  ○ 5. AWS Bedrock   Claude, Llama via AWS
  ○ 6. Ollama        Local models

  ❯ Provider(s): 1,2

  ❯ OpenAI API key: sk-...
  ❯ Anthropic API key: sk-ant-...

  ❯ Port [4000]:
  ❯ Master key [auto-generate]:

  ✔ Config saved → ./litellm_config.yaml

  ❯ Start the proxy now? (Y/n):
```

--------------------------------

### Get Agent Response Example

Source: https://docs.litellm.ai/blog/google-ai-studio-managed-agents

Example JSON response containing detailed information for a specific agent.

```json
{
    "id": "my-custom-slides-agent",
    "base_agent": "antigravity-preview-05-2026",
    "system_instruction": "You are a helpful assistant that creates slides.",
    "base_environment": {
        "sources": [
            {
                "type": "gcs",
                "source": "gs://eap-templates/slides-skill",
                "target": "/.agents/skills/slides-skill"
            }
        ],
        "type": "remote"
    }
}
```

--------------------------------

### Install LiteLLM v1.84.6 via Pip

Source: https://docs.litellm.ai/release_notes/v1.84.6/v1-84-6

Install the specific LiteLLM version 1.84.6 using pip for consistent environment setup.

```bash
pip install litellm==1.84.6
```

--------------------------------

### Start LiteLLM Proxy with Configuration

Source: https://docs.litellm.ai/docs/providers/xai_realtime

Launch the LiteLLM proxy server using a specified configuration file.

```bash
litellm --config /path/to/config.yaml   
  
# RUNNING on http://0.0.0.0:4000
```

--------------------------------

### Install LiteLLM v1.87.2 using Pip

Source: https://docs.litellm.ai/release_notes/v1.87.2/v1-87-2

Install the specific version 1.87.2 of LiteLLM using pip, ensuring you get the latest patch release.

```bash
pip install litellm==1.87.2
```

--------------------------------

### Start LiteLLM Proxy with Configuration File

Source: https://docs.litellm.ai/docs/pass_through/bedrock

Launch the LiteLLM proxy server, loading model definitions and settings from the specified `config.yaml` file.

```bash
litellm --config config.yaml  
  
# RUNNING on http://0.0.0.0:4000  

```

```bash
litellm --config config.yaml  
  
# RUNNING on http://0.0.0.0:4000  

```

--------------------------------

### Start LiteLLM Proxy with Configuration

Source: https://docs.litellm.ai/docs/observability/cloudzero

Launch the LiteLLM proxy, referencing the configuration file that enables CloudZero integration.

```bash
litellm --config /path/to/config.yaml
```

--------------------------------

### Install LiteLLM via Pip

Source: https://docs.litellm.ai/blog/gemini_3_1_flash_lite_preview

Install a specific stable version of LiteLLM using pip to get support for Gemini 3.1 Flash Lite Preview features.

```bash
pip install litellm==v1.80.8-stable.1
```

--------------------------------

### Start LiteLLM Proxy with a Configuration File

Source: https://docs.litellm.ai/docs/proxy/configs

Execute this command to start the LiteLLM proxy server, loading all configurations from the specified `config.yaml` file.

```bash
$ litellm --config /path/to/config.yaml  

```

--------------------------------

### Example GET Request to Prompt Management API

Source: https://docs.litellm.ai/docs/adding_provider/generic_prompt_management_api

This illustrates a sample GET request to the generic prompt management endpoint, including prompt_id and custom query parameters.

```http
GET /beta/litellm_prompt_management?prompt_id=hello-world-prompt-2bac&project_name=litellm&slug=hello-world-prompt-2bac  

```

--------------------------------

### Start LiteLLM Proxy with Configuration File

Source: https://docs.litellm.ai/blog/google-ai-studio-managed-agents

Launch the LiteLLM proxy server, referencing a proxy_config.yaml file that contains necessary API keys and settings.

```bash
litellm --config proxy_config.yaml
```

--------------------------------

### Combine Input Examples with Tool Search and Programmatic Calling

Source: https://docs.litellm.ai/docs/providers/anthropic_tool_input_examples

Illustrates how to use `input_examples` in conjunction with both `defer_loading` for tool search and `allowed_callers` for programmatic tool invocation.

```json
{
    "type": "function",
    "function": {
        "name": "advanced_tool",
        "description": "A complex tool",
        "parameters": {...}
    },
    "defer_loading": true,                            # Tool search
    "allowed_callers": ["code_execution_20250825"],  # Programmatic calling
    "input_examples": [                               # Input examples
        {"param1": "value1", "param2": "value2"}
    ]
}
```

--------------------------------

### Quick Start: Set AWS Access Keys

Source: https://docs.litellm.ai/docs/pass_through/bedrock

Set AWS access key ID and secret access key as environment variables for the LiteLLM proxy as part of the quick start guide.

```bash
export AWS_ACCESS_KEY_ID=""  # Access key  
export AWS_SECRET_ACCESS_KEY="" # Secret access key  

```

--------------------------------

### Start LiteLLM Proxy with Configuration

Source: https://docs.litellm.ai/docs/observability/datadog

Run the LiteLLM proxy using the previously created `config.yaml` to enable Datadog metric collection.

```bash
litellm --config config.yaml  

```

--------------------------------

### Clone and Run Claude Agent SDK Cookbook Example

Source: https://docs.litellm.ai/docs/tutorials/claude_agent_sdk

Provides commands to clone the LiteLLM repository, navigate to the Anthropic agent SDK cookbook example, install dependencies, and run the interactive CLI agent.

```bash
# Clone and run the example  
git clone https://github.com/BerriAI/litellm.git  
cd litellm/cookbook/anthropic_agent_sdk  
uv add -r requirements.txt  
python main.py  
```

--------------------------------

### Bitbucket Prompt File Example

Source: https://docs.litellm.ai/docs/proxy/prompt_management

This example shows the structure of a `.prompt` file for Bitbucket integration, defining model, temperature, system, and user roles with a variable.

```yaml
# prompts/my_bitbucket_prompt.prompt
---
model: gpt-4
temperature: 0.7
---
System: You are a helpful assistant.

User: {{user_message}}
```

--------------------------------

### Complete Example: Custom Prompt Template with Llama2

Source: https://docs.litellm.ai/docs/tutorials/TogetherAI_liteLLM

Full example demonstrating the registration and usage of a custom prompt template for a Llama2 variant on Together AI, including API key setup and response printing.

```python
import litellm   
from litellm import completion  
  
# set env variable   
os.environ["TOGETHERAI_API_KEY"] = ""  
  
litellm.register_prompt_template(
    model="OpenAssistant/llama2-70b-oasst-sft-v10",
    roles={"system":"<|im_start|>system", "assistant":"<|im_start|>assistant", "user":"<|im_start|>user"}, # tell LiteLLM how you want to map the openai messages to this model
    pre_message_sep= "\n",
    post_message_sep= "\n"
)  
  
messages=[{"role":"user", "content": "Write me a poem about the blue sky"}]  
  
response = completion(model="together_ai/OpenAssistant/llama2-70b-oasst-sft-v10", messages=messages)  
  
print(response)
```

--------------------------------

### Start LiteLLM Proxy with a Configuration File

Source: https://docs.litellm.ai/blog/gpt_5_4_mini_nano

Execute this command to start the LiteLLM Proxy, loading the model configurations from the specified `config.yaml` file.

```bash
litellm --config /path/to/config.yaml  

```

--------------------------------

### Comprehensive Perplexity API Call Example with LiteLLM

Source: https://docs.litellm.ai/docs/providers/perplexity

This example demonstrates a complete `litellm.responses` call using a Perplexity model, including API key setup, input, custom provider, tools, instructions, and output parsing.

```python
from litellm import responses  
import os  
  
os.environ['PERPLEXITY_API_KEY'] = ""  
  
# Comprehensive example with multiple features  
response = responses(  
    model="perplexity/openai/gpt-5.2",  
    input="Research the latest developments in quantum computing and provide sources",  
    custom_llm_provider="perplexity",  
    tools=[  
        {"type": "web_search"},  
        {"type": "fetch_url"}  
    ],  
    instructions="Use web_search to find relevant information and fetch_url to retrieve detailed content from sources. Provide citations for all claims.",  
    max_output_tokens=1000,  
    temperature=0.7,  
)  
  
print(f"Response ID: {response.id}")  
print(f"Model: {response.model}")  
print(f"Status: {response.status}")  
print(f"Output: {response.output}")  
print(f"Usage: {response.usage}")  
```

--------------------------------

### Start LiteLLM Proxy with Configuration

Source: https://docs.litellm.ai/blog/gemini_3

Run the LiteLLM proxy, specifying the path to your configuration file. The proxy will then listen for incoming requests.

```bash
litellm --config /path/to/config.yaml  
  
# RUNNING on http://0.0.0.0:4000  
```

--------------------------------

### Start LiteLLM UI Development Server

Source: https://docs.litellm.ai/docs/extras/contributing_code

Launches the development server for the LiteLLM UI, allowing local testing and development.

```bash
npm run dev
```

--------------------------------

### Set Developer Message for Prompt

Source: https://docs.litellm.ai/docs/proxy/litellm_prompt_management

This example sets a system instruction for the model, guiding its behavior to respond in a specific style.

```text
Respond as jack sparrow would  
```

--------------------------------

### Comprehensive LiteLLM WebSearch Configuration Example

Source: https://docs.litellm.ai/docs/tutorials/claude_code_websearch

A detailed LiteLLM configuration example demonstrating how to enable web search interception for multiple AI providers (Bedrock, Azure, Vertex AI) and specify a search tool. This showcases a more robust setup.

```yaml
model_list:  
  - model_name: bedrock-sonnet  
    litellm_params:  
      model: bedrock/us.anthropic.claude-sonnet-4-5-20250929-v1:0  
      aws_region_name: us-east-1  
  
  - model_name: azure-gpt4  
    litellm_params:  
      model: azure/gpt-4  
      api_base: https://my-azure.openai.azure.com  
      api_key: os.environ/AZURE_API_KEY  
  
litellm_settings:  
  callbacks: ["websearch_interception"]  
  websearch_interception_params:  
    enabled_providers:  
      - bedrock        # Enable for AWS Bedrock  
      - azure          # Enable for Azure OpenAI  
      - vertex_ai      # Enable for Google Vertex  
    search_tool_name: perplexity-search  # Optional: use specific search tool  
```

--------------------------------

### Start the LiteLLM Proxy with HuggingFace token and config

Source: https://docs.litellm.ai/docs/providers/huggingface_rerank

This command demonstrates how to start the LiteLLM Proxy, setting the HuggingFace token and pointing to a configuration file.

```bash
export HF_TOKEN="hf_xxxxxx"  
litellm --config /path/to/config.yaml  
  
# RUNNING on http://0.0.0.0:4000  
```

--------------------------------

### Example User Invitation URL

Source: https://docs.litellm.ai/docs/proxy/self_serve

This example demonstrates the structure of the invitation link that users will use to log in and complete their onboarding.

```text
http://0.0.0.0:4000/ui/onboarding?id=a2f0918f-43b0-4770-a664-96ddd192966e  
  
# <YOUR_PROXY_BASE_URL>/ui/onboarding?id=<id>
```

--------------------------------

### Tool Definition with Input Examples for Email Function

Source: https://docs.litellm.ai/docs/providers/anthropic_tool_input_examples

This JSON snippet illustrates a tool definition for sending emails, including an `input_examples` array. It provides concrete examples of `to`, `subject`, and `body` parameters to guide the model on expected input formats.

```json
{  
    "type": "function",  
    "function": {  
        "name": "send_email",  
        "description": "Send an email to a recipient",  
        "parameters": {  
            "type": "object",  
            "properties": {  
                "to": {"type": "string", "description": "Email address"},  
                "subject": {"type": "string"},  
                "body": {"type": "string"}  
            },  
            "required": ["to", "subject", "body"]  
        }  
    },  
    "input_examples": [  
        {  
            "to": "user@example.com",  
            "subject": "Meeting Reminder",  
            "body": "Don't forget our meeting tomorrow at 2 PM."  
        },  
        {  
            "to": "team@company.com",  
            "subject": "Weekly Update",  
            "body": "Here's this week's progress report..."  
        }  
    ]  
}  
```

--------------------------------

### Run OpenClaw Onboarding Wizard

Source: https://docs.litellm.ai/docs/tutorials/openclaw_integration

Initiates the OpenClaw onboarding process, which includes installing the daemon and configuring the connection to LiteLLM.

```bash
openclaw onboard --install-daemon  
```

--------------------------------

### Setup ChatDev Environment

Source: https://docs.litellm.ai/docs/proxy_server

These commands clone the ChatDev repository, create and activate a Conda environment, and install its dependencies using `uv`.

```bash
git clone https://github.com/OpenBMB/ChatDev.git  
cd ChatDev  
conda create -n ChatDev_conda_env python=3.9 -y  
conda activate ChatDev_conda_env  
uv add -r requirements.txt  
```

--------------------------------

### Example API Calls for Prompt Integrations

Source: https://docs.litellm.ai/docs/proxy/native_litellm_prompt

Demonstrates how to make `litellm.completion` API calls for file system, BitBucket, and GitLab prompt integrations, including necessary configuration.

```python
# File system integration
response = litellm.completion(
    model="dotprompt/gpt-4",
    prompt_id="hello",
    prompt_variables={"user_message": "Hello world"},
    messages=[{"role": "user", "content": "This will be ignored"}]
)
```

```python
# BitBucket integration
response = litellm.completion(
    model="bitbucket/gpt-4",
    prompt_id="hello",
    prompt_variables={"user_message": "Hello world"},
    bitbucket_config={
        "workspace": "your-workspace",
        "repository": "your-repo",
        "access_token": "your-token"
    }
)
```

```python
# Gitlab integration
response = litellm.completion(
    model="gitlab/gpt-4",
    prompt_id="hello",
    prompt_variables={"user_message": "Hello world"},
    gitlab_config={
        "project": "a/b/<repo_name>",
        "access_token": "your-access-token",
        "base_url": "gitlab url",
        "prompts_path": "src/prompts", # folder to point to, defaults to root
        "branch":"main"  # optional, defaults to main
    }
)
```

--------------------------------

### Example .prompt File Content

Source: https://docs.litellm.ai/docs/proxy/prompt_management

Provides an example of the content for a `.prompt` file, including metadata (model, temperature) and the system/user message structure.

```yaml
# prompts/hello.prompt
---
model: gpt-4
temperature: 0.7
---
System: You are a helpful assistant.
  
User: {{user_message}}

```

--------------------------------

### Proxy Configuration for Cross-Region Bedrock Model

Source: https://docs.litellm.ai/docs/providers/bedrock

Example `config.yaml` setup for configuring a Bedrock model with cross-region support for use with the LiteLLM proxy.

```yaml
model_list:  
  - model_name: bedrock-claude-haiku  
    litellm_params:  
      model: bedrock/us.anthropic.claude-3-haiku-20240307-v1:0  
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID  
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY  
      aws_region_name: os.environ/AWS_REGION_NAME
```

--------------------------------

### Quick Start: Basic Completion with LiteLLM

Source: https://docs.litellm.ai/docs/completion/usage

Demonstrates how to make a basic completion call using LiteLLM, setting an OpenAI API key, and printing the usage statistics from the response.

```python
from litellm import completion  
import os  
  
## set ENV variables  
os.environ["OPENAI_API_KEY"] = "your-api-key"  
  
response = completion(  
  model="gpt-3.5-turbo",  
  messages=[{ "content": "Hello, how are you?","role": "user"}]  
)  
  
print(response.usage)  
```

--------------------------------

### Run Python Script for LiveKit Agent or Client

Source: https://docs.litellm.ai/docs/tutorials/livekit_xai_realtime

Execute the Python script containing either the LiveKit agent or client example to start its operation.

```bash
python your_script.py
```

--------------------------------

### Start LiteLLM Proxy with Configuration

Source: https://docs.litellm.ai/docs/bedrock_converse

Launches the LiteLLM Proxy server using the specified `config.yaml`.

```bash
litellm --config config.yaml  
  
# RUNNING on http://0.0.0.0:4000  
```

--------------------------------

### GET /v1/memory/{key}

Source: https://docs.litellm.ai/docs/proxy/memory

Retrieve a specific memory entry by its unique key. Returns an empty string if none exist (as per `get_preferences` example).

```APIDOC
## GET /v1/memory/{key}

### Description
Retrieve a specific memory entry by its unique key. Returns an empty string if none exist (as per `get_preferences` example).

### Method
GET

### Endpoint
/v1/memory/{key}

### Parameters
#### Path Parameters
- **key** (string) - Required - The unique identifier of the memory entry to retrieve.

### Response
#### Success Response (200)
- **key** (string) - The unique identifier of the memory entry.
- **value** (string) - The content of the memory entry.
- **metadata** (object) - The metadata associated with the entry.

#### Response Example
{
  "key": "user:preferences",
  "value": "Prefers concise responses. Timezone: PST.",
  "metadata": {"version": 1}
}
```

--------------------------------

### Start LiteLLM Proxy with Configuration

Source: https://docs.litellm.ai/blog/tags/completion

Execute this command to start the LiteLLM proxy, loading the model configurations from the specified `config.yaml` file.

```bash
litellm --config /path/to/config.yaml  

```

--------------------------------

### Complete Example with Model Aliases

Source: https://docs.litellm.ai/docs/completion/model_alias

Provides a full code example showing how to set environment variables, define and apply model aliases, and make completion calls using the aliases.

```python
import litellm   
from litellm import completion   
  
  
## set ENV variables  
os.environ["OPENAI_API_KEY"] = "openai key"  
os.environ["REPLICATE_API_KEY"] = "cohere key"  
  
## set model alias map  
model_alias_map = {  
    "GPT-3.5": "gpt-3.5-turbo-16k",  
    "llama2": "replicate/llama-2-70b-chat:2796ee9483c3fd7aa2e171d38f4ca12251a30609463dcfd4cd76703f22e96cdf"  
}  
  
litellm.model_alias_map = model_alias_map  
  
messages = [{ "content": "Hello, how are you?","role": "user"}]  
  
# call "gpt-3.5-turbo-16k"  
response = completion(model="GPT-3.5", messages=messages)  
  
# call replicate/llama-2-70b-chat:2796ee9483c3fd7aa2e171d38f4ca1...  
response = completion("llama2", messages)  
```

--------------------------------

### Get Response by ID with OpenAI SDK

Source: https://docs.litellm.ai/docs/providers/openai/responses_api

This example shows how to retrieve a previously created response using its ID via the OpenAI SDK and a LiteLLM proxy.

```python
from openai import OpenAI  
  
# Initialize client with your proxy URL  
client = OpenAI(  
    base_url="http://localhost:4000",  # Your proxy URL  
    api_key="your-api-key"             # Your proxy API key  
)  
  
# First, create a response  
response = client.responses.create(  
    model="openai/o1-pro",  
    input="Tell me a three sentence bedtime story about a unicorn."  
)  
  
# Get the response ID  
response_id = response.id  
  
# Retrieve the response by ID  
retrieved_response = client.responses.retrieve(response_id)  
  
print(retrieved_response)
```

--------------------------------

### Complete Example for LiteLLM Managed Files

Source: https://docs.litellm.ai/docs/proxy/litellm_managed_files

This comprehensive example illustrates the full workflow: downloading a PDF, creating a managed file using the LiteLLM client, and subsequently utilizing the generated file ID in chat completion requests for both Gemini and OpenAI models.

```python
import base64  
import requests  
from openai import OpenAI  
  
client = OpenAI(base_url="http://0.0.0.0:4000", api_key="sk-1234", max_retries=0)  
  
  
# Download and save the PDF locally  
url = (  
    "https://storage.googleapis.com/cloud-samples-data/generative-ai/pdf/2403.05530.pdf"  
)  
response = requests.get(url)  
response.raise_for_status()  
  
# Save the PDF locally  
with open("2403.05530.pdf", "wb") as f:  
    f.write(response.content)  
  
# Read the local PDF file  
file = client.files.create(  
    file=open("2403.05530.pdf", "rb"),  
    purpose="user_data", # can be any openai 'purpose' value  
    extra_body={"target_model_names": "gpt-4o-mini-openai, vertex_ai/gemini-2.0-flash"},  
)  
  
print(f"file.id: {file.id}") # ❤️ Unified file id  
  
## GEMINI CALL ###   
completion = client.chat.completions.create(  
    model="gemini-2.0-flash",  
    messages=[  
        {  
            "role": "user",  
            "content": [  
                {"type": "text", "text": "What is in this recording?"},  
                {  
                    "type": "file",  
                    "file": {  
                        "file_id": file.id,  
                    },  
                },  
            ],  
        },  
    ]  
)  
  
print(completion.choices[0].message)  
  
  
### OPENAI CALL ###   
completion = client.chat.completions.create(  
    model="gpt-4o-mini-openai",  
    messages=[  
        {  
            "role": "user",  
            "content": [  
                {"type": "text", "text": "What is in this recording?"},  
                {  
                    "type": "file",  
                    "file": {  
                        "file_id": file.id,  
                    },  
                },  
            ],  
        },  
    ],  
)  
  
print(completion.choices[0].message)  

```

--------------------------------

### Example Locust Startup Terminal Output

Source: https://docs.litellm.ai/docs/load_test

This snippet shows the expected terminal output when Locust successfully starts, indicating the web interface URL and version.

```bash
[2024-03-15 07:19:58,893] Starting web interface at http://0.0.0.0:8089  
[2024-03-15 07:19:58,898] Starting Locust 2.24.0  

```

--------------------------------

### Setup ChatDev Environment

Source: https://docs.litellm.ai/docs/proxy_server

Provides a sequence of shell commands to clone the ChatDev repository, create and activate a Conda environment, and install dependencies using 'uv'.

```shell
git clone https://github.com/OpenBMB/ChatDev.git  
cd ChatDev  
conda create -n ChatDev_conda_env python=3.9 -y  
conda activate ChatDev_conda_env  
uv add -r requirements.txt
```

--------------------------------

### Start LiteLLM Proxy with Configuration

Source: https://docs.litellm.ai/blog/gemini_3_1_flash_lite_preview

Execute this command to start the LiteLLM proxy server, loading the model configurations from the specified YAML file.

```bash
litellm --config /path/to/config.yaml
```

--------------------------------

### Skip LiteLLM Proxy Server Startup

Source: https://docs.litellm.ai/docs/proxy/cli

Use this flag to prevent the LiteLLM proxy server from starting after setup, useful for tasks like database migrations.

```bash
litellm --skip_server_startup  

```

--------------------------------

### Full LiteLLM Supabase Integration Example

Source: https://docs.litellm.ai/docs/observability/supabase_integration

A full Python script demonstrating how to set up environment variables, enable Supabase callbacks, and make example LLM calls to log data.

```Python
from litellm import completion  
  
## set env variables  
### SUPABASE  
os.environ["SUPABASE_URL"] = "your-supabase-url"   
os.environ["SUPABASE_KEY"] = "your-supabase-key"   
  
## LLM API KEY  
os.environ["OPENAI_API_KEY"] = ""  
  
# set callbacks  
litellm.success_callback=["supabase"]  
litellm.failure_callback=["supabase"]  
  
# openai call  
response = completion(  
  model="gpt-3.5-turbo",   
  messages=[{"role": "user", "content": "Hi 👋 - i'm openai"}],  
  user="ishaan22" # identify users  
)   
  
# bad call, expect this call to fail and get logged  
response = completion(  
  model="chatgpt-test",   
  messages=[{"role": "user", "content": "Hi 👋 - i'm a bad call to test error logging"}]  
)
```

--------------------------------

### Example Response for Get Project Information

Source: https://docs.litellm.ai/docs/proxy/project_management

Illustrates the JSON structure returned when successfully retrieving project details, including spend, model spend, and budget table.

```json
{
    "project_id": "project-abc",
    "project_alias": "flight-search-assistant",
    "team_id": "team-123",
    "models": ["gpt-4", "gpt-3.5-turbo"],
    "spend": 45.67,
    "model_spend": {
        "gpt-4": 42.30,
        "gpt-3.5-turbo": 3.37
    },
    "litellm_budget_table": {
        "budget_id": "budget-xyz",
        "max_budget": 100.0,
        "tpm_limit": 100000,
        "rpm_limit": 100
    },
    "metadata": {
        "use_case_id": "SNOW-12345"
    }
}
```

--------------------------------

### Comprehensive LiteLLM Proxy Configuration Example

Source: https://docs.litellm.ai/docs/proxy/config_settings

This snippet provides a complete example of the LiteLLM proxy configuration structure, demonstrating how to set up environment variables, define models, configure callbacks, networking, cost adjustments, debugging, fallbacks, and various caching options.

```yaml
environment_variables: {}
  
model_list:
  - model_name: string  
    litellm_params: {}
    model_info:
      id: string  
      mode: embedding  
      input_cost_per_token: 0  
      output_cost_per_token: 0  
      max_tokens: 2048  
      base_model: gpt-4-1106-preview  
      additionalProp1: {}
  
litellm_settings:
  # Logging/Callback settings  
  success_callback: ["langfuse"]  # list of success callbacks  
  failure_callback: ["sentry"]  # list of failure callbacks  
  callbacks: ["otel"]  # list of callbacks - runs on success and failure  
  service_callbacks: ["datadog", "prometheus"]  # logs redis, postgres failures on datadog, prometheus  
  turn_off_message_logging: boolean  # prevent the messages and responses from being logged to on your callbacks, but request metadata will still be logged. Useful for privacy/compliance when handling sensitive data.  
  redact_user_api_key_info: boolean  # Redact information about the user api key (hashed token, user_id, team id, etc.), from logs. Currently supported for Langfuse, OpenTelemetry, Logfire, ArizeAI logging.  
  langfuse_default_tags: ["cache_hit", "cache_key", "proxy_base_url", "user_api_key_alias", "user_api_key_user_id", "user_api_key_user_email", "user_api_key_team_alias", "semantic-similarity", "proxy_base_url"] # default tags for Langfuse Logging  
  # Networking settings  
  request_timeout: 10 # (int) llm requesttimeout in seconds. Raise Timeout error if call takes longer than 10s. Sets litellm.request_timeout  
  force_ipv4: boolean # If true, litellm will force ipv4 for all LLM requests. Some users have seen httpx ConnectionError when using ipv6 + Anthropic API  
  
  # Cost tracking settings  
  cost_discount_config:
    vertex_ai: 0.05 # Apply a 5% discount to Vertex AI costs  
    gemini: 0.05 # Apply a 5% discount to Gemini costs  
  cost_margin_config:
    global: 0.05 # Apply a 5% margin to all providers  
    openai: 0.10 # Apply a 10% margin to OpenAI costs  
    
  # Debugging - see debugging docs for more options  
  # Use `--debug` or `--detailed_debug` CLI flags, or set LITELLM_LOG env var to "INFO", "DEBUG", or "ERROR"  
  json_logs: boolean # if true, logs will be in json format  
  
  # Fallbacks, reliability  
  default_fallbacks: ["claude-opus"] # set default_fallbacks, in case a specific model group is misconfigured / bad.  
  content_policy_fallbacks: [{ "gpt-3.5-turbo-small": ["claude-opus"] }] # fallbacks for ContentPolicyErrors  
  context_window_fallbacks: [{ "gpt-3.5-turbo-small": ["gpt-3.5-turbo-large", "claude-opus"] }] # fallbacks for ContextWindowExceededErrors  
  
  # MCP Aliases - Map aliases to MCP server names for easier tool access  
  mcp_aliases: {
      "github": "github_mcp_server",  
      "zapier": "zapier_mcp_server",  
      "deepwiki": "deepwiki_mcp_server"
    } # Maps friendly aliases to MCP server names. Only the first alias for each server is used  
  
  # Caching settings  
  cache: true  
  cache_params: # set cache params for redis  
    type: redis # type of cache to initialize (options: "local", "redis", "s3", "gcs")  
  
    # Optional - Redis Settings  
    host: "localhost" # The host address for the Redis cache. Required if type is "redis".  
    port: 6379 # The port number for the Redis cache. Required if type is "redis".  
    password: "your_password" # The password for the Redis cache. Required if type is "redis".  
    namespace: "litellm.caching.caching" # namespace for redis cache  
    max_connections: 100  # [OPTIONAL] Set Maximum number of Redis connections. Passed directly to redis-py.   
    # Optional - Redis Cluster Settings  
    redis_startup_nodes: [{ "host": "127.0.0.1", "port": "7001" }]  
  
    # Optional - Redis Sentinel Settings  
    service_name: "mymaster"  
    sentinel_nodes: [["localhost", 26379]]  
  
    # Optional - GCP IAM Authentication for Redis  
    gcp_service_account: "projects/-/serviceAccounts/your-sa@project.iam.gserviceaccount.com" # GCP service account for IAM authentication  
    gcp_ssl_ca_certs: "./server-ca.pem" # Path to SSL CA certificate file for GCP Memorystore Redis  
    ssl: true # Enable SSL for secure connections  
    ssl_cert_reqs: null # Set to null for self-signed certificates  
    ssl_check_hostname: false # Set to false for self-signed certificates  
  
    # Optional - Qdrant Semantic Cache Settings  
    qdrant_semantic_cache_embedding_model: openai-embedding # the model should be defined on the model_list  
    qdrant_collection_name: test_collection  
    qdrant_quantization_config: binary  
    qdrant_semantic_cache_vector_size: 1536 # vector size must match embedding model dimensionality  
    similarity_threshold: 0.8 # similarity threshold for semantic cache  
  
    # Optional - S3 Cache Settings  
    s3_bucket_name: cache-bucket-litellm # AWS Bucket Name for S3  
```

--------------------------------

### Example Response for New Session Request

Source: https://docs.litellm.ai/docs/response_api

This JSON object shows the expected response when a new session is started without context, indicating the model cannot reference previous turns.

```json
{
  "id":"resp_789ghi",
  "model":"claude-3-5-sonnet-20241022",
  "output":[{
    "type":"message",
    "content":[{
      "type":"output_text",
      "text":"I don't see who you're referring to in our conversation. Could you let me know which person you'd like to learn more about?"
    }]
  }]
}
```

--------------------------------

### Start LiteLLM Proxy Server with Bytez Configuration

Source: https://docs.litellm.ai/docs/providers/bytez

Launch the LiteLLM proxy server, providing the Bytez API key and the path to your configuration file.

```bash
$ BYTEZ_API_KEY=YOUR_BYTEZ_API_KEY_HERE litellm --config /path/to/config.yaml --debug  
```

--------------------------------

### Call OCI Cohere Command Model with OCI SDK Signer (Python)

Source: https://docs.litellm.ai/docs/providers/oci

This example demonstrates getting a completion from an OCI Cohere model using an `oci.signer.Signer` object for authentication.

```python
from litellm import completion  
from oci.signer import Signer  
  
signer = Signer(  
    tenancy="ocid1.tenancy.oc1..",  
    user="ocid1.user.oc1..",  
    fingerprint="xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx",  
    private_key_file_location="~/.oci/key.pem",  
)  
  
messages = [{"role": "user", "content": "Explain quantum computing"}]  
response = completion(  
    model="oci/cohere.command-latest",  
    messages=messages,  
    oci_signer=signer,  
    oci_region="us-chicago-1",  
    oci_compartment_id="<oci_compartment_id>",  
)  
print(response)  
```

--------------------------------

### Recommended Key Naming Conventions

Source: https://docs.litellm.ai/docs/memory_management

These examples demonstrate best practices for creating namespaced keys to effectively organize memory entries by different entities.

```text
user:{user_id}:preferences      # User preferences  
user:{user_id}:context          # Conversation context  
team:{team_id}:playbook         # Team playbook  
agent:{agent_id}:memory         # Agent memory  
project:{project_id}:config     # Project configuration
```

--------------------------------

### Create a New Organization (Enterprise)

Source: https://docs.litellm.ai/docs/proxy/multi_tenant_architecture

Example `curl` command to create a new organization, specifying an alias, allowed models, and a maximum budget. This is applicable for enterprise setups with organization features.

```bash
curl --location 'http://0.0.0.0:4000/organization/new' \
    --header 'Authorization: Bearer sk-1234' \
    --header 'Content-Type: application/json' \
    --data '{  
        "organization_alias": "engineering_department",  
        "models": ["gpt-4", "gpt-4o", "claude-3-5-sonnet"],  
        "max_budget": 10000  
    }'  

```

--------------------------------

### Get Streaming Response from Anthropic Claude via LiteLLM Proxy

Source: https://docs.litellm.ai/docs/response_api

Use this example to receive a streaming response from an Anthropic Claude model, iterating through events, when configured via the LiteLLM proxy.

```python
from openai import OpenAI  
  
# Initialize client with your proxy URL  
client = OpenAI(  
    base_url="http://localhost:4000",  # Your proxy URL  
    api_key="your-api-key"             # Your proxy API key  
)  
  
# Streaming response  
response = client.responses.create(  
    model="anthropic/claude-3-5-sonnet-20240620",  
    input="Tell me a three sentence bedtime story about a unicorn.",  
    stream=True  
)  
  
for event in response:  
    print(event)
```

--------------------------------

### Run LiteLLM with a Configuration File (Shell)

Source: https://docs.litellm.ai/docs/assistants

This shell command demonstrates how to start LiteLLM using a specified configuration file.

```bash
$ litellm --config /path/to/config.yaml  
```

--------------------------------

### Schedule Periodic Model Cost Map Reload

Source: https://docs.litellm.ai/docs/enterprise

Schedule periodic synchronization of model pricing and context-window data, for example, every 6 hours, to get day-0 support for new models.

```http
POST /schedule/model_cost_map_reload?hours=6
```

--------------------------------

### Start LiteLLM Gateway with Configuration

Source: https://docs.litellm.ai/docs/proxy/guardrails/onyx_security

Starts the LiteLLM gateway using the specified configuration file and enables detailed debugging.

```bash
litellm --config config.yaml --detailed_debug  

```

--------------------------------

### Define Tools with Input Examples for Claude (Python)

Source: https://docs.litellm.ai/blog/anthropic_advanced_features

This snippet illustrates how to define a tool with `input_examples` to guide Claude's tool use, demonstrating tool schema definition and a `litellm.completion` call.

```python
import litellm  
  
tools = [  
    {  
        "type": "function",  
        "function": {  
            "name": "create_calendar_event",  
            "description": "Create a new calendar event with attendees and reminders",  
            "parameters": {  
                "type": "object",  
                "properties": {  
                    "title": {"type": "string"},  
                    "start_time": {  
                        "type": "string",  
                        "description": "ISO 8601 format: YYYY-MM-DDTHH:MM:SS"  
                    },  
                    "duration_minutes": {"type": "integer"},  
                    "attendees": {  
                        "type": "array",  
                        "items": {  
                            "type": "object",  
                            "properties": {  
                                "email": {"type": "string"},  
                                "optional": {"type": "boolean"}  
                            }  
                        }  
                    },  
                    "reminders": {  
                        "type": "array",  
                        "items": {  
                            "type": "object",  
                            "properties": {  
                                "minutes_before": {"type": "integer"},  
                                "method": {"type": "string", "enum": ["email", "popup"]}  
                            }  
                        }  
                    }  
                },  
                "required": ["title", "start_time", "duration_minutes"]  
            }  
        },  
        # Provide concrete examples  
        "input_examples": [  
            {  
                "title": "Team Standup",  
                "start_time": "2025-01-15T09:00:00",  
                "duration_minutes": 30,  
                "attendees": [  
                    {"email": "alice@company.com", "optional": false},  
                    {"email": "bob@company.com", "optional": false}  
                ],  
                "reminders": [  
                    {"minutes_before": 15, "method": "popup"}  
                ]  
            },  
            {  
                "title": "Lunch Break",  
                "start_time": "2025-01-15T12:00:00",  
                "duration_minutes": 60  
                # Demonstrates optional fields can be omitted  
            }  
        ]  
    }  
]  
  
response = litellm.completion(  
    model="anthropic/claude-sonnet-4-5-20250929",  
    messages=[{  
        "role": "user",  
        "content": "Schedule a team meeting for tomorrow at 2pm for 45 minutes with john@company.com and sarah@company.com"  
    }],  
    tools=tools  
)  
  
print("Tool call:", response.choices[0].message.tool_calls[0].function.arguments)  
```