### Setup LiteLLM UI Development Environment

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/CONTRIBUTING.md

Clones the LiteLLM repository, navigates to the UI dashboard directory, installs Node.js dependencies, and starts the development server for the UI.

```bash
# Clone the repo (if you haven't already)
git clone https://github.com/YOUR_USERNAME/litellm.git
cd litellm

# Navigate to the UI dashboard directory
cd ui/litellm-dashboard

# Install dependencies
npm install

# Start the development server
npm run dev
```

--------------------------------

### Run Gollem Examples

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/gollem_go_agent_framework/README.md

Commands to install Go dependencies and execute the basic, tools, and streaming agent examples provided in the repository.

```bash
# Install Go dependencies
go mod tidy

# Basic agent
go run ./basic

# Agent with type-safe tools
go run ./tools

# Streaming responses
go run ./streaming
```

--------------------------------

### Quick Start Terraform Apply

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/terraform/litellm/gcp/README.md

Initialize Terraform, copy example variables, and apply the configuration to deploy LiteLLM.

```bash
cd terraform/litellm/gcp/examples/default
cp terraform.tfvars.example terraform.tfvars
# Edit: project, region, tenant, env, image_registry, proxy_config, gateway_extra_secrets.

terraform init
terraform apply
```

--------------------------------

### Get API Key Info Example

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/litellm/proxy/client/cli/README.md

Example of fetching information for a specific API key.

```bash
lite keys info --key sk-key1
```

--------------------------------

### Setup Local Development Environment

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/CONTRIBUTING.md

Clone the repository, set up your local environment with development dependencies, and install git hooks for convention enforcement.

```bash
# Fork the repository on GitHub (click the Fork button at https://github.com/BerriAI/litellm)
# Then clone your fork locally
git clone https://github.com/YOUR_USERNAME/litellm.git
cd litellm

# Create a new branch for your feature (see "Commit and Branch Conventions" below)
git checkout -b feature/your-feature

# Install development dependencies
make install-dev

# Install git hooks that enforce commit + branch conventions (one-time, opt-in)
make install-hooks

# Verify your setup works
make help
```

--------------------------------

### Install and Start LiteLLM Proxy

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/litellm/proxy/README.md

Install the litellm tool and launch the proxy server pointing to a specific model provider.

```shell
$ uv tool install litellm
```

```shell
$ litellm --model ollama/codellama 

#INFO: Ollama running on http://0.0.0.0:8000
```

--------------------------------

### Start LiteLLM Proxy for Adaptive Router Demo

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/scripts/adaptive_router_demo/README.md

Run the LiteLLM proxy with the adaptive router example configuration. Ensure the `OPENAI_API_KEY` is set and wait for the application startup to complete.

```bash
export OPENAI_API_KEY=sk-...     # underlying models hit OpenAI
uv run litellm \
    --config litellm/proxy/example_config_yaml/adaptive_router_example.yaml \
    --port 4000
```

--------------------------------

### Install full local test environment

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/CONTRIBUTING.md

Installs all dependencies needed for a comprehensive local testing setup.

```bash
make install-test-deps         # Install the full local test environment
```

--------------------------------

### Install and Run LiteLLM Proxy Dev Dependencies

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/CONTRIBUTING.md

Installs development dependencies for the LiteLLM proxy server. Use this before starting the proxy.

```bash
# Install proxy dependencies
make install-proxy-dev
```

--------------------------------

### Example Command Workflow for Release Notes

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/misc/RELEASE_NOTES_GENERATION_INSTRUCTIONS.md

Bash commands demonstrating the workflow for generating release notes, including getting model changes, analyzing PRs, creating release notes, and linking documentation.

```bash
# 1. Get model changes
git diff <commit> HEAD -- model_prices_and_context_window.json

# 2. Analyze PR list for categorization
# 3. Create release notes following template
# 4. Link to appropriate documentation
# 5. Review for missing documentation needs
```

--------------------------------

### HTTP Request List Models Example

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/litellm/proxy/client/cli/README.md

Example of making a GET request to the /models endpoint to list available models.

```bash
lite http request GET /models
```

--------------------------------

### Start LiteLLM Proxy Server

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/CONTRIBUTING.md

Starts the LiteLLM proxy server using a specified configuration file. Ensure you have installed dev dependencies first.

```bash
# Start the proxy server
uv run litellm --config your_config.yaml
```

--------------------------------

### Run the CodeLlama Server Locally

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/codellama-server/README.MD

Execute this command to start the LiteLLM CodeLlama server after installing dependencies and setting API keys. The server will then be accessible locally.

```bash
python main.py
```

--------------------------------

### Install LiteLLM Proxy Client

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/litellm/proxy/client/README.md

Install the litellm package using uv.

```bash
uv add litellm
```

--------------------------------

### Install required Python packages

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/liteLLM_Langchain_Demo.ipynb

This command installs the `litellm` and `langchain` libraries, which are necessary to run the examples in this notebook. Ensure your Python environment is set up correctly.

```python
!pip install litellm langchain
```

--------------------------------

### BitBucket Prompt File Formats and Features

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/litellm/integrations/bitbucket/README.md

Examples of different prompt file structures including basic setup, multi-role conversations, and dynamic model selection.

```yaml
---
# Model configuration
model: gpt-4
temperature: 0.7
max_tokens: 500

# Input schema (optional)
input:
  schema:
    user_message: string
    system_context?: string
---

System: You are a helpful {{role}} assistant.

User: {{user_message}}
```

```yaml
---
model: gpt-4
temperature: 0.3
---
System: You are a helpful coding assistant.

User: {{user_question}}
```

```yaml
---
model: "{{preferred_model}}"  # Model can be a variable
temperature: 0.7
---
System: You are a helpful assistant specialized in {{domain}}.

User: {{user_message}}
```

--------------------------------

### Install litellm-proxy-extras with uv

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/litellm-proxy-extras/README.md

Install the litellm-proxy-extras package using the uv package manager.

```bash
uv add litellm-proxy-extras
```

--------------------------------

### LiteLLM with `litellm.completion` and `stream=True`

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/LiteLLM_PromptLayer.ipynb

This example demonstrates using the `completion` function with `stream=True` to get a streaming response. It iterates over the response chunks and prints them.

```python
from litellm import completion

response = completion(
    model="gpt-3.5-turbo",
    messages=[
        {"role": "user", "content": "This is a streaming test."}
    ],
    stream=True
)

for chunk in response:
    print(chunk)
```

--------------------------------

### Install litellm with proxy dependencies

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/litellm-proxy-extras/README.md

Install litellm with all proxy dependencies including litellm-proxy-extras using uv.

```bash
uv tool install 'litellm[proxy]' # installs litellm-proxy-extras and other proxy dependencies
```

--------------------------------

### Install benchmark dependencies

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/benchmark/readme.md

Install the required Python packages including litellm, click, and tabulate.

```shell
pip install litellm click tqdm tabulate termcolor
```

--------------------------------

### Install and Configure LiteLLM with API Keys

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/litellm_router/test_questions/question2.txt

Install litellm and set environment variables for OpenAI and Cohere API keys before making calls.

```python
pip install litellm
```

```python
from litellm import completion
import os

## set ENV variables 
os.environ["OPENAI_API_KEY"] = "your-openai-key" 
os.environ["COHERE_API_KEY"] = "your-cohere-key"
```

--------------------------------

### Install development dependencies

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/CONTRIBUTING.md

Use this command to install the necessary packages for local development.

```bash
make install-dev               # Install development dependencies
```

--------------------------------

### Install LiteLLM Proxy CLI

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/litellm/proxy/client/cli/README.md

Installs the LiteLLM proxy package using 'uv'.

```bash
uv tool install 'litellm[proxy]'
```

--------------------------------

### Setup and Authentication

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/liteLLM_Baseten.ipynb

Import dependencies and configure the BASETEN_API_KEY environment variable.

```python
import os
from litellm import completion

# Set your Baseten API key
os.environ['BASETEN_API_KEY'] = "" #@param {type:"string"}

# Test message
messages = [{"role": "user", "content": "What is AGI?"}]
```

--------------------------------

### PublicAI Provider Configuration Example

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/litellm/llms/openai_like/README.md

Real-world example of a JSON-configured provider with parameter mappings and special handling for content list conversion.

```json
{
  "publicai": {
    "base_url": "https://api.publicai.co/v1",
    "api_key_env": "PUBLICAI_API_KEY",
    "api_base_env": "PUBLICAI_API_BASE",
    "base_class": "openai_gpt",
    "param_mappings": {
      "max_completion_tokens": "max_tokens"
    },
    "special_handling": {
      "convert_content_list_to_string": true
    }
  }
}
```

--------------------------------

### Setting Up LiteLLM

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/LiteLLM_PromptLayer.ipynb

Provides instructions on how to set up LiteLLM, including installation and basic configuration.

```bash
pip install litellm
```

--------------------------------

### HTTP Request Test Connection Example

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/litellm/proxy/client/cli/README.md

Example of making a GET request to the /health/test_connection endpoint with custom headers.

```bash
lite http request GET /health/test_connection -H "X-Custom-Header:value"
```

--------------------------------

### Start LiteLLM Proxy with Configuration

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/mock_prompt_management_server/README.md

Launch the LiteLLM proxy server using the provided configuration file to enable prompt management integration.

```bash
litellm --config config.yaml
```

--------------------------------

### Start LiteLLM Proxy with configuration

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/litellm/passthrough/README.md

Run the LiteLLM proxy server using the provided configuration file to start listening for passthrough requests.

```bash
litellm proxy --config config.yaml

# RUNNING on http://localhost:4000
```

--------------------------------

### Setup Environment and API Keys

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/google_adk_litellm_tutorial.ipynb

Configure environment variables for multiple LLM providers and import required modules for ADK and LiteLLM integration.

```python
# Setup environment and API keys
import os
import asyncio
from google.adk.agents import Agent
from google.adk.models.lite_llm import LiteLlm  # For multi-model support
from google.adk.sessions import InMemorySessionService
from google.adk.runners import Runner
from google.genai import types
import litellm  # Import for proxy configuration

# Set your API keys
os.environ['GOOGLE_API_KEY'] = 'your-google-api-key'  # For Gemini models
os.environ['OPENAI_API_KEY'] = 'your-openai-api-key'  # For OpenAI models
os.environ['ANTHROPIC_API_KEY'] = 'your-anthropic-api-key'  # For Claude models

# Define model constants for cleaner code
MODEL_GEMINI_PRO = 'gemini-1.5-pro'
MODEL_GPT_4O = 'openai/gpt-4o'
MODEL_CLAUDE_SONNET = 'anthropic/claude-3-sonnet-20240229'
```

--------------------------------

### Tool Use Example

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/LiteLLM_PromptLayer.ipynb

Demonstrates using tools with LiteLLM, similar to function calling but more generalized.

```python
from litellm import completion

response = completion(
    model="gpt-3.5-turbo",
    messages=[
        {"role": "user", "content": "What's the weather like in Boston?"}
    ],
    tools = [
        {
            "type": "function",
            "function": {
                "name": "get_weather",
                "description": "Get the current weather in a given location",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "location": {
                            "type": "string",
                            "description": "The city and state, e.g. San Francisco, CA"
                        },
                        "unit": {
                            "type": "string",
                            "enum": ["celsius", "fahrenheit"],
                            "description": "The temperature unit to use. Infer this from the user's request."
                        }
                    },
                    "required": ["location"]
                }
            }
        }
    ],
    tool_choice="auto"
)

print(response)
```

--------------------------------

### Start the LiteLLM proxy server

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/ai_coding_tool_guides/claude_code_quickstart/guide.md

Launch the proxy using the specified configuration file to begin listening for requests.

```bash
litellm --config /path/to/config.yaml

# RUNNING on http://0.0.0.0:4000
```

--------------------------------

### Example Response for Get Prompt by ID

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/mock_prompt_management_server/README.md

Illustrative JSON response structure when fetching a prompt by its ID from the mock server's `/beta/litellm_prompt_management` endpoint.

```json
{
  "prompt_id": "hello-world-prompt",
  "prompt_template": [
    {
      "role": "system",
      "content": "You are a helpful assistant specialized in {domain}."
    },
    {
      "role": "user",
      "content": "Help me with: {task}"
    }
  ],
  "prompt_template_model": "gpt-4",
  "prompt_template_optional_params": {
    "temperature": 0.7,
    "max_tokens": 500
  }
}
```

--------------------------------

### Using Complexity Router with LiteLLM (Python)

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/litellm/router_strategy/complexity_router/README.md

Demonstrates how to use the configured `smart-router` with LiteLLM, showing examples of prompts routed to different complexity tiers.

```python
import litellm

response = litellm.completion(
    model="smart-router",  # Your complexity_router model name
    messages=[{"role": "user", "content": "What is 2+2?"}]
)
# Routes to SIMPLE tier (gpt-4o-mini)

response = litellm.completion(
    model="smart-router",
    messages=[{"role": "user", "content": "Think step by step: analyze the performance implications of implementing a distributed consensus algorithm for our microservices architecture."}]
)
# Routes to REASONING tier (o1-preview)
```

--------------------------------

### Test Prompt Management API Endpoints with cURL

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/mock_prompt_management_server/README.md

Examples to retrieve prompts, list all prompts, and get prompt variables using cURL commands, including an authenticated request.

```bash
curl "http://localhost:8080/beta/litellm_prompt_management?prompt_id=hello-world-prompt"
```

```bash
curl "http://localhost:8080/beta/litellm_prompt_management?prompt_id=hello-world-prompt" \
  -H "Authorization: Bearer test-token-12345"
```

```bash
curl "http://localhost:8080/prompts"
```

```bash
curl "http://localhost:8080/prompts/hello-world-prompt/variables"
```

--------------------------------

### LiteLLM AWS Quick Start Initialization

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/terraform/litellm/aws/README.md

Steps to initialize and apply a basic LiteLLM AWS deployment using Terraform.

```bash
cd terraform/litellm/aws/examples/default
cp terraform.tfvars.example terraform.tfvars
# Edit: region, tenant, env, azs, proxy_config, gateway_extra_secrets.

terraform init
terraform apply
```

--------------------------------

### Print 'Hi' in Python, JavaScript, C, and Java (Basic)

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/LiteLLM_CometAPI.ipynb

Basic code snippets demonstrating how to print the string 'Hi' in several common programming languages. These examples are self-contained and require no special setup beyond a standard environment for each language.

```python
# Simple Python program to say "Hi"\nprint("Hi")
```

```javascript
// Simple JavaScript program to say "Hi"\nconsole.log("Hi");
```

```c
#include <stdio.h>\n\nint main() {\n    printf("Hi\\n");\n    return 0;\n}
```

```java
public class SayHi {\n    public static void main(String[] args) {\n        System.out.println("Hi");\n    }\n}
```

--------------------------------

### Install Development Dependencies and Run Checks

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/README.md

Clone the repository, navigate to the directory, and use make commands to install development dependencies, format code, run linters, and execute unit tests. Ensure all checks pass before submitting a pull request.

```bash
git clone https://github.com/BerriAI/litellm.git
cd litellm
make install-dev    # Install development dependencies
make format         # Format your code
make lint           # Run all linting checks
make test-unit      # Run unit tests
make format-check   # Check formatting only
```

--------------------------------

### Helm Template: Get Application URL by Service Type

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/deploy/charts/litellm-helm/templates/NOTES.txt

Conditional Helm template that generates the appropriate URL and access commands based on the configured Kubernetes service type. Supports Ingress, NodePort, LoadBalancer, and ClusterIP service types with automatic TLS detection and port-forwarding setup.

```Helm
{{- if .Values.ingress.enabled }}
{{- range $host := .Values.ingress.hosts }}
  {{- range .paths }}
  http{{ if $.Values.ingress.tls }}s{{ end }}://{{ $host.host }}{{ .path }}
  {{- end }}
{{- end }}
{{- else if contains "NodePort" .Values.service.type }}
  export NODE_PORT=$(kubectl get --namespace {{ .Release.Namespace }} -o jsonpath="{.spec.ports[0].nodePort}" services {{ include "litellm.fullname" . }})
  export NODE_IP=$(kubectl get nodes --namespace {{ .Release.Namespace }} -o jsonpath="{.items[0].status.addresses[0].address}")
  echo http://$NODE_IP:$NODE_PORT
{{- else if contains "LoadBalancer" .Values.service.type }}
     NOTE: It may take a few minutes for the LoadBalancer IP to be available.
           You can watch the status of by running 'kubectl get --namespace {{ .Release.Namespace }} svc -w {{ include "litellm.fullname" . }}'
  export SERVICE_IP=$(kubectl get svc --namespace {{ .Release.Namespace }} {{ include "litellm.fullname" . }} --template "{{"{{ range (index .status.loadBalancer.ingress 0) }}{{.}}{{ end }}"}}")
  echo http://$SERVICE_IP:{{ .Values.service.port }}
{{- else if contains "ClusterIP" .Values.service.type }}
  export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "app.kubernetes.io/name={{ include "litellm.name" . }},app.kubernetes.io/instance={{ .Release.Name }}" -o jsonpath="{.items[0].metadata.name}")
  export CONTAINER_PORT=$(kubectl get pod --namespace {{ .Release.Namespace }} $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
  echo "Visit http://127.0.0.1:8080 to use your application"
  kubectl --namespace {{ .Release.Namespace }} port-forward $POD_NAME 8080:$CONTAINER_PORT
{{- end }}
PDB: {{ if .Values.pdb.enabled }}enabled{{ else }}disabled{{ end }}. Configure via .Values.pdb.*
```

--------------------------------

### Start LiteLLM with Levo Integration

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/litellm/integrations/levo/README.md

Launch LiteLLM with the configuration file containing Levo callback. All LLM requests will automatically be sent to Levo's collector.

```bash
litellm --config config.yaml
```

--------------------------------

### LiteLLM with `litellm.completion` and `stream=True` (input/output filters)

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/LiteLLM_PromptLayer.ipynb

This example demonstrates streaming a response while applying input and output filters. Filters can preprocess input or postprocess output during streaming.

```python
from litellm import completion

def my_input_filter(data, **kwargs):
    # Modify data here
    return data

def my_output_filter(data, **kwargs):
    # Modify data here
    return data

response = completion(
    model="gpt-3.5-turbo",
    messages=[
        {"role": "user", "content": "Stream with filters."}
    ],
    stream=True,
    input_filters=[my_input_filter],
    output_filters=[my_output_filter]
)

for chunk in response:
    print(chunk)
```

--------------------------------

### Install liteLLM

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/liteLLM_Streaming_Demo.ipynb

Installs the liteLLM library with a specific version.

```python
!pip install litellm==0.1.369
```

--------------------------------

### Install liteLLM

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/liteLLM_Replicate_Demo.ipynb

Install the liteLLM package using pip.

```python
# install liteLLM
!pip install litellm
```

--------------------------------

### Start LiteLLM proxy with xAI

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/livekit_agent_sdk/README.md

Launch the LiteLLM proxy server with xAI API key configured to listen on port 4000.

```bash
export XAI_API_KEY="your-xai-key"
litellm --config config.yaml --port 4000
```

--------------------------------

### GET /scim/v2/Groups/{group_id}

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/litellm/proxy/management_endpoints/scim/README_SCIM.md

Get a specific group by ID.

```APIDOC
## GET /scim/v2/Groups/{group_id}

### Description
Get a specific group by ID.

### Method
GET

### Endpoint
/scim/v2/Groups/{group_id}

### Parameters
#### Path Parameters
- **group_id** (string) - Required - The ID of the group to retrieve.

### Response
#### Success Response (200)
- **id** (string) - The unique identifier for the group.
- **displayName** (string) - A human-readable name for the group.
- **members** (array) - A list of members of the group.

#### Response Example
{
  "schemas": ["urn:ietf:params:scim:schemas:core:2.0:Group"],
  "id": "team-123",
  "displayName": "Engineering Team",
  "members": [
    {
      "value": "user-123",
      "display": "John Doe"
    }
  ]
}
```

--------------------------------

### GET /scim/v2/Users/{user_id}

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/litellm/proxy/management_endpoints/scim/README_SCIM.md

Get a specific user by ID.

```APIDOC
## GET /scim/v2/Users/{user_id}

### Description
Get a specific user by ID.

### Method
GET

### Endpoint
/scim/v2/Users/{user_id}

### Parameters
#### Path Parameters
- **user_id** (string) - Required - The ID of the user to retrieve.

### Response
#### Success Response (200)
- **id** (string) - The unique identifier for the user.
- **userName** (string) - The user's login name.
- **active** (boolean) - A boolean value indicating the user's administrative status.
- **emails** (array) - User's email addresses.

#### Response Example
{
  "schemas": ["urn:ietf:params:scim:schemas:core:2.0:User"],
  "id": "user-123",
  "userName": "john.doe@example.com",
  "active": true,
  "emails": [
    {
      "value": "john.doe@example.com",
      "primary": true
    }
  ]
}
```

--------------------------------

### Integrate LiteLLM with Prompt Management in Python

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/mock_prompt_management_server/README.md

Demonstrates how to use `litellm.completion` to fetch a prompt from the mock server, apply variables, and merge with user messages for a complete integration.

```python
from litellm import completion

# The completion will:
# 1. Fetch the prompt from your API
# 2. Replace {domain} with "machine learning"
# 3. Replace {task} with "building a recommendation system"
# 4. Merge with your messages
# 5. Use the model and params from the prompt

response = completion(
    model="gpt-4",
    prompt_id="hello-world-prompt",
    prompt_variables={
        "domain": "machine learning",
        "task": "building a recommendation system"
    },
    messages=[
        {"role": "user", "content": "I have user behavior data from the past year."}
    ],
    # Configure the generic prompt manager
    generic_prompt_config={
        "api_base": "http://localhost:8080",
        "api_key": "test-token-12345",
    }
)

print(response.choices[0].message.content)
```

--------------------------------

### Install LiteLLM CLI with Homebrew

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/packaging/homebrew/README.md

End users can install the `lite` command-line interface using this Homebrew command. This installs the CLI without the proxy server runtime.

```shell
brew install BerriAI/litellm/lite
```

--------------------------------

### Install Dependencies

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/litellm_proxy_server/readme.md

Install the required Python dependencies using pip.

```bash
pip install -r requirements.txt
```

--------------------------------

### Install LiteLLM and Langfuse

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/logging_observability/LiteLLM_Langfuse.ipynb

Install required packages for LiteLLM and Langfuse integration.

```python
!pip install litellm langfuse
```

--------------------------------

### Initialize and Apply Terraform Configuration

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/README.md

Run terraform init to download provider plugins and modules, followed by terraform apply to create or update the AWS infrastructure for LiteLLM.

```bash
terraform init
terraform apply
```

--------------------------------

### Install LiteLLM

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/liteLLM_Baseten.ipynb

Install the litellm package using the pip package manager.

```python
%pip install litellm
```

--------------------------------

### Quick Start: Initialize Client and Make Chat Completion

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/litellm/proxy/client/README.md

Initialize the LiteLLM proxy client and make a basic chat completion request. Ensure your LiteLLM proxy server is running.

```python
from litellm.proxy.client import Client

# Initialize the client
client = Client(
    base_url="http://localhost:4000",  # Your LiteLLM proxy server URL
    api_key="sk-api-key"               # Optional: API key for authentication
)

# Make a chat completion request
response = client.chat.completions.create(
    model="gpt-3.5-turbo",
    messages=[
        {"role": "user", "content": "Hello, how are you?"}
    ]
)
print(response.choices[0].message.content)
```

--------------------------------

### Install Server Dependencies

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/mock_prompt_management_server/README.md

Install the necessary Python packages for the FastAPI server to run.

```bash
pip install fastapi uvicorn pydantic
```

--------------------------------

### Initialize LiteLLM for Benchmarking and Set API Keys

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/Benchmarking_LLMs_by_use_case.ipynb

Import LiteLLM's core functions, set environment variables for various LLM provider API keys, and optionally configure the LiteLLM dashboard client.

```python
from litellm import completion, completion_cost
import os
import time

# optional use litellm dashboard to view logs
# litellm.use_client = True
# litellm.token = "ishaan_2@berri.ai" # set your email


# set API keys
os.environ['TOGETHERAI_API_KEY'] = ""
os.environ['OPENAI_API_KEY'] = ""
os.environ['ANTHROPIC_API_KEY'] = ""


# select LLMs to benchmark
# using https://api.together.xyz/playground for llama2
```

--------------------------------

### Install proxy development dependencies

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/CONTRIBUTING.md

Install dependencies specifically required for proxy development.

```bash
make install-proxy-dev         # Install proxy development dependencies
```

--------------------------------

### Install liteLLM for VertexAI

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/liteLLM_VertextAI_Example.ipynb

Install the specific version of liteLLM required for VertexAI integration.

```shell
!pip install litellm==0.1.388
```

--------------------------------

### Basic LLM Completion with LiteLLM

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/litellm_router/test_questions/question1.txt

Install the library and set environment variables to call different providers like OpenAI and Cohere using the completion function.

```python
pip install litellm
from litellm import completion
import os

## set ENV variables 
os.environ["OPENAI_API_KEY"] = "your-openai-key" 
os.environ["COHERE_API_KEY"] = "your-cohere-key" 

messages = [{ "content": "Hello, how are you?","role": "user"}]

# openai call
response = completion(model="gpt-3.5-turbo", messages=messages)

# cohere call
response = completion(model="command-nightly", messages=messages)
print(response)
```

--------------------------------

### Configure GCP Providers and Deploy LiteLLM Module

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/terraform/litellm/gcp/README.md

Example of how to configure the Google and Google Beta providers and then call the LiteLLM module. The module inherits provider configurations automatically.

```hcl
provider "google" {
  project = "my-gcp-project"
  region  = "us-central1"
}
provider "google-beta" {
  project = "my-gcp-project"
  region  = "us-central1"
}

module "litellm" {
  source = "github.com/BerriAI/litellm//terraform/litellm/gcp?ref=<tag>"

  project = "my-gcp-project"
  region  = "us-central1"
  tenant  = "acme"
  env     = "prod"
  # ...any of the inputs in variables.tf...
}
```

--------------------------------

### Install Nemo-Guardrails and LangChain dependencies

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/Using_Nemo_Guardrails_with_LiteLLM_Server.ipynb

Install required packages for Nemo-Guardrails integration with LangChain.

```bash
pip install nemoguardrails langchain
```

--------------------------------

### Start LiteLLM Proxy

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/gollem_go_agent_framework/README.md

Initialize the LiteLLM proxy using either a direct model flag or a configuration file for multi-provider access.

```bash
# Simple start with a single model
litellm --model gpt-4o

# Or with the example config for multi-provider access
litellm --config proxy_config.yaml
```

--------------------------------

### Install line-profiler using uv

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/litellm/proxy/common_utils/performance_utils.md

Install the line-profiler package as a development dependency. Windows users on Python 3.14+ may need to install Microsoft Visual C++ Build Tools to compile from source.

```bash
uv add --dev line-profiler
```

--------------------------------

### Install Pipecat AI

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/tests/e2e/realtime/REALTIME_COVERAGE_MATRIX.md

Install pipecat-ai with openai support to run the realism layer tests.

```bash
uv pip install "pipecat-ai[openai]"
```

--------------------------------

### Basic Completion Example

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/liteLLM_Baseten.ipynb

A high-level example of calling the completion function with a Baseten model identifier.

```python
response = completion(
    model="baseten/openai/gpt-oss-120b",
    messages=[{"role": "user", "content": "Hello!"}],
    max_tokens=1000,
    temperature=0.7
)
```

--------------------------------

### Configure API Keys in .env File

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/tests/e2e/CONTRIBUTING.md

Create a `.env` file in the directory to store provider API keys required by the example models for local test execution.

```bash
OPENAI_API_KEY="sk-..."
ANTHROPIC_API_KEY="sk-..."
GEMINI_API_KEY="..."
```

--------------------------------

### Install LiteLLM and Lunary packages

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/logging_observability/LiteLLM_Lunary.ipynb

Install required dependencies for LiteLLM and Lunary integration using pip.

```python
%pip install litellm lunary
```

--------------------------------

### Start Mock Prompt Management Server

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/mock_prompt_management_server/README.md

Run the FastAPI server locally to make the prompt management API available.

```bash
python mock_prompt_management_server.py
```

--------------------------------

### LiteLLM with `litellm.completion` and `stream=True` (fallbacks)

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/LiteLLM_PromptLayer.ipynb

This example demonstrates streaming a response with fallback models configured. If the primary model fails, LiteLLM will attempt to use a fallback.

```python
from litellm import completion

response = completion(
    model="gpt-3.5-turbo",
    messages=[
        {"role": "user", "content": "Stream with fallbacks."}
    ],
    stream=True,
    fallbacks=[{"model": "gpt-4", "fallback_model": "claude-3-opus-20240229"}]
)

for chunk in response:
    print(chunk)
```

--------------------------------

### Install Dependencies

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/litellm_proxy_server/braintrust_prompt_wrapper_README.md

Install the necessary Python packages for the Braintrust prompt wrapper server and LiteLLM integration.

```bash
pip install fastapi uvicorn httpx litellm
```

--------------------------------

### Install LiteLLM and Google ADK Dependencies

Source: https://github.com/berriai/litellm/blob/litellm_internal_staging/cookbook/google_adk_litellm_tutorial.ipynb

Install required packages for Google ADK and LiteLLM integration.

```python
# Install dependencies
!pip install google-adk litellm
```