### Define Pre-build Commands for Environment Setup

Source: https://docs.cerebrium.ai/cerebrium/container-images/defining-container-images

Execute commands at the beginning of the build process, before dependency installation. Useful for installing build tools or configuring the environment. Modifying these commands triggers a rebuild.

```toml
[cerebrium.deployment]
pre_build_commands = [
    # Add specialized build tools
    "curl -o /usr/local/bin/pget -L 'https://github.com/replicate/pget/releases/download/v0.6.2/pget_linux_x86_64'",
    "chmod +x /usr/local/bin/pget"
]
```

--------------------------------

### Define Shell Commands for Post-Dependency Setup

Source: https://docs.cerebrium.ai/cerebrium/container-images/defining-container-images

Execute commands after all dependencies are installed and code is copied. Ideal for initializing application resources or performing tasks requiring the complete environment. Modifying these commands triggers a rebuild.

```toml
[cerebrium.deployment]
shell_commands = [
    # Initialize application resources
    "python -m download_models",
    "python -m compile_assets",
    "python -m init_app"
]
```

--------------------------------

### Start Gradio Server Process

Source: https://docs.cerebrium.ai/v4/examples/asgi-gradio-interface

The `start` method initiates the Gradio server by creating and starting a new process for `run_server`. It includes logic to poll the server's readiness by making HTTP requests and provides feedback on success or failure. If the server fails to start after multiple retries, it attempts to stop the server.

```python
def start(self):
        print(f"Starting Gradio server at {self.url} port {self.port}")

        # Start Gradio in a separate process
        self.process = multiprocessing.Process(target=self.run_server)
        self.process.start()

        # Wait for Gradio to become ready
        max_retries = 30
        retry_delay = 1.0

        for _ in range(max_retries):
            try:
                response = requests.get(f"{self.url}/")
                if response.status_code == 200:
                    print(f"Gradio server is ready at {self.url}")
                    return True
            except requests.exceptions.ConnectionError:
                time.sleep(retry_delay)

        print("Failed to start Gradio server")
        self.stop()
        return False
```

--------------------------------

### Setup Cerebrium CLI for WandB Sweep

Source: https://docs.cerebrium.ai/v4/examples/wandb-sweep

Installs and logs into the Cerebrium CLI, then initializes a project for a WandB sweep. This sets up the necessary environment for running experiments.

```bash
pip install cerebrium --upgrade
cerebrium login
cerebrium init wandb-sweep
```

--------------------------------

### Dockerfile for Triton Server with LLM

Source: https://docs.cerebrium.ai/v4/examples/deploy-an-llm-with-tensorrtllm-tritonserver

This Dockerfile sets up an Nvidia Triton inference server environment for LLMs. It extends an existing Triton container, installs necessary dependencies like git and git-lfs, configures environment variables for Hugging Face models and CUDA, and copies the model repository files into the container. It exposes the Triton server ports and sets the default command to start the server.

```dockerfile
FROM nvcr.io/nvidia/tritonserver:25.10-trtllm-python-py3

ENV PYTHONPATH=/usr/local/lib/python3.12/dist-packages:$PYTHONPATH
ENV PYTHONDONTWRITEBYTECODE=1
ENV DEBIAN_FRONTEND=noninteractive
ENV HF_HOME=/persistent-storage/models
ENV TORCH_CUDA_ARCH_LIST=8.6

# Install dependencies
RUN apt-get update && apt-get install -y \
    git \
    git-lfs \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /app

RUN pip install --break-system-packages \
    huggingface_hub \
    transformers \
    || true

# Create directories
RUN mkdir -p \
    /app/model_repository/llama3_2/1 \
    /persistent-storage/models \
    /persistent-storage/engines

# Copy files
COPY model.py /app/model_repository/llama3_2/1/
COPY config.pbtxt /app/model_repository/llama3_2/

EXPOSE 8000 8001 8002

CMD ["tritonserver", "--model-repository=/app/model_repository", "--http-port=8000", "--grpc-port=8001", "--metrics-port=8002"]
```

--------------------------------

### Install and Initialize Cerebrium CLI

Source: https://docs.cerebrium.ai/migrations/mystic

Installs the Cerebrium command-line tool and initializes a new project for a stable diffusion model. This involves running pip install and cerebrium init commands.

```bash
pip install cerebrium --upgrade
cerebrium login
cerebrium init stable-diffusion
cd stable-diffusion
```

--------------------------------

### FastAPI Startup Event for Gradio Server

Source: https://docs.cerebrium.ai/v4/examples/asgi-gradio-interface

The `startup_event` function, decorated with `@app.on_event("startup")`, is executed when the FastAPI application starts. It initializes the `GradioServer` instance and starts it, but only if an external Gradio server URL is not provided via environment variables. If the server fails to start, the application exits.

```python
@app.on_event("startup")
async def startup_event():
    global gradio_server
    if not os.getenv("GRADIO_SERVER_URL"):
        gradio_server = GradioServer()
        if not gradio_server.start():
            sys.exit(1)
```

--------------------------------

### Install and Authenticate Twilio CLI

Source: https://docs.cerebrium.ai/v4/examples/livekit-outbound-agent

Installs the Twilio CLI using Homebrew and authenticates the user. This is a prerequisite for managing Twilio resources via the command line.

```bash
brew tap twilio/brew && brew install twilio
twilio login
```

--------------------------------

### Install Local Dependencies

Source: https://docs.cerebrium.ai/v4/examples/wandb-sweep

Installs the necessary Python packages listed in `requirements.txt` locally using pip. This ensures the development environment matches the training environment.

```bash
pip install -r requirements.txt
```

--------------------------------

### Deployment Strategies

Source: https://context7_llms

APIs and guides for managing deployments, including CI/CD, gradual roll-outs, and multi-region deployments.

```APIDOC
## CI/CD Pipelines

### Description
Automate Cerebrium deployments using GitHub Actions.

### Method
POST

### Endpoint
/cerebrium/deployments/ci-cd

## Gradual Roll-out

### Description
Control the transition between revisions during deployments.

### Method
PUT

### Endpoint
/cerebrium/deployments/gradual-roll-out

## Multi-Region Deployment

### Description
Deploy your apps globally across multiple regions for improved latency and data residency compliance.

### Method
POST

### Endpoint
/cerebrium/deployments/multi-region-deployment
```

--------------------------------

### Install and Login to Weights & Biases (Wandb)

Source: https://docs.cerebrium.ai/v4/examples/wandb-sweep

This snippet shows the necessary commands to install the Wandb library using pip and log in to your Wandb account via the command line interface. Authentication requires copying an API key from the Wandb website.

```bash
pip install wandb
wandb login
```

--------------------------------

### Install and Initialize Cerebrium CLI

Source: https://docs.cerebrium.ai/v4/examples/livekit-outbound-agent

Installs the Cerebrium Python package and logs in the user to their Cerebrium account. It then initializes a new project for a LiveKit voice agent, creating necessary configuration files.

```bash
pip install cerebrium --upgrade
cerebrium login
cerebrium init livekit-voice-agent
```

--------------------------------

### Download Example Audio File

Source: https://docs.cerebrium.ai/cerebrium/partner-services/deepgram

This bash command downloads an example WAV audio file named 'bueller.wav' from a provided URL. This file can be used for testing the Deepgram speech-to-text service.

```bash
wget https://dpgr.am/bueller.wav
```

--------------------------------

### Generate Images using SDXL with Refiner

Source: https://context7_llms

This example demonstrates how to generate high-quality images using the SDXL model with a refiner on Cerebrium. It covers the process of setting up and running the image generation pipeline.

```python
from diffusers import DiffusionPipeline
import torch

# Load the SDXL base and refiner pipelines
base_pipe = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16, variant="fp16", use_safetensors=True)
base_pipe.to("cuda")

refiner_pipe = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-refiner-1.0", torch_dtype=torch.float16, variant="fp16", use_safetensors=True)
refiner_pipe.to("cuda")

def generate_sdxl_image(prompt, negative_prompt="", num_inference_steps=40, high_noise_frac=0.8):
    # Generate image with base pipeline
    image = base_pipe(
        prompt=prompt,
        negative_prompt=negative_prompt,
        num_inference_steps=num_inference_steps,
        denoising_end=high_noise_frac,
        output_type="latent",
    ).images[0]

    # Refine the image
    refined_image = refiner_pipe(
        prompt=prompt,
        negative_prompt=negative_prompt,
        num_inference_steps=num_inference_steps,
        denoising_start=high_noise_frac,
        image=image,
    ).images[0]

    return refined_image

# Example usage:
# prompt = "A majestic cat sitting on a throne, digital art"
# generated_image = generate_sdxl_image(prompt)
# generated_image.save("cat_on_throne.png")

```

--------------------------------

### Scaling and Optimization

Source: https://context7_llms

Guides on scaling applications for performance and cost-efficiency, including batching, concurrency, preemption, and graceful termination.

```APIDOC
## Batching and Concurrency

### Description
Improve throughput and cost performance with batching and concurrency.

### Method
GET

### Endpoint
/cerebrium/scaling/batching-concurrency

## Preemption and Graceful Termination

### Description
Implementing Graceful Termination of Instances by Handling Termination Signals.

### Method
GET

### Endpoint
/cerebrium/scaling/graceful-termination

## Scaling Apps

### Description
Learn to optimise for cost and performance by scaling out apps.

### Method
GET

### Endpoint
/cerebrium/scaling/scaling-apps
```

--------------------------------

### GET Request using Custom Domain

Source: https://docs.cerebrium.ai/cerebrium/networking/custom-domains

Example of making a GET request to a Cerebrium application using a configured custom domain. This demonstrates how to retrieve data or trigger an endpoint without a request body.

```bash
curl https://api.example.com/v4/p-1234/my-app/run \
  -H "Authorization: Bearer {YOUR_API_KEY}"
```

--------------------------------

### Serve GPT-OSS Models with vLLM

Source: https://context7_llms

This example demonstrates how to deploy OpenAI's latest open-source models using the vLLM framework on Cerebrium. vLLM provides efficient serving of LLMs.

```python
from vllm import LLM, SamplingParams

# Load the LLM model
llm = LLM(model="gpt2") # Replace with your desired GPT-OSS model

sampling_params = SamplingParams(temperature=0.8, top_p=0.95)

def generate_text(prompt):
    outputs = llm.generate(prompt, sampling_params)
    return outputs[0].outputs[0].text

# Example usage:
user_prompt = "The future of AI is"
generated_text = generate_text(user_prompt)
print(generated_text)

```

--------------------------------

### Install Tensorizer using pip in cerebrium.toml

Source: https://docs.cerebrium.ai/cerebrium/other-topics/faster-cold-starts

This snippet shows how to add the Tensorizer library to your project's dependencies within the `cerebrium.toml` file. It specifies the package name and a minimum version requirement, ensuring that Tensorizer is installed when your deployment is built.

```toml
tensorizer = ">=2.7.0"
```

--------------------------------

### Agent Executor Setup (Python)

Source: https://docs.cerebrium.ai/v4/examples/langchain-langsmith

This Python code sets up a LangChain agent executor for calendar management. It defines a prompt template with system instructions, chat history, and user input, selects a ChatOpenAI model, and combines them with the defined tools ('get_availability', 'book_slot') to create an agent that can execute tasks.

```python
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.tools import tool
from langchain.agents import create_tool_calling_agent, AgentExecutor
from langchain_openai import ChatOpenAI

prompt = ChatPromptTemplate.from_messages([
    ("system", "you're a helpful assistant managing the calendar of Michael Louis. You need to book appointments for a user based on available capacity and their preference. You need to find out if the user is: From Michaels team, a customer of Cerebrium or a friend or entrepreneur. If the person is from his team, book a morning slot. If its a potential customer for Cerebrium, book an afternoon slot. If its a friend or entrepreneur needing help or advice, book a night time slot. If none of these are available, book the earliest slot. Do not book a slot without asking the user what their preferred time is. Find out from the user, their name and email address."),
    MessagesPlaceholder(variable_name="chat_history"),
    ("human", "{input}"),
    MessagesPlaceholder(variable_name="agent_scratchpad"),
])

tools = [get_availability, book_slot]


llm = ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0, api_key=os.environ.get("OPENAI_API_KEY"))
agent = create_tool_calling_agent(llm, tools, prompt)
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)

```

--------------------------------

### Project Initialization with Cerebrium CLI

Source: https://docs.cerebrium.ai/v4/examples/deploy-a-vision-language-model-with-sglang

Shows the command-line interface (CLI) commands to initialize a new project for SGLang development using Cerebrium. This involves creating a project directory and navigating into it.

```bash
cerebrium init 7-vision-language-sglang
cd 7-vision-language-sglang
```

--------------------------------

### Python Gradio Server Setup and Configuration

Source: https://docs.cerebrium.ai/v4/examples/asgi-gradio-interface

Sets up and manages a Gradio server for a chat interface. It configures host, port, and URL from environment variables and handles starting and stopping the server process.

```python
import gradio as gr
import httpx
import multiprocessing
import os
import sys
import time
from typing import List, Optional

import requests
from fastapi import FastAPI, Request
from starlette.responses import Response as StarletteResponse

app = FastAPI()

gradio_server = None
GRADIO_HOST = os.getenv("GRADIO_HOST", "127.0.0.1")
GRADIO_PORT = int(os.getenv("GRADIO_PORT", "7860"))
GRADIO_URL = os.getenv("GRADIO_SERVER_URL", f"http://{GRADIO_HOST}:{GRADIO_PORT}")
LLAMA_ENDPOINT = os.getenv("LLAMA_ENDPOINT", "<YOUR_MODEL_API_ENDPOINT>")


class GradioServer:
    def __init__(self):
        self.host = GRADIO_HOST
        self.port = GRADIO_PORT
        self.process: Optional[multiprocessing.Process] = None
        self.url = GRADIO_URL

    async def chat_with_llama(self, message: str, history: List[List[str]]) -> str:
        """Make a request to the Llama endpoint"""
        # Convert history and new message into OpenAI chat format
        messages = []
        for h in history:
            messages.extend([
                {"role": "user", "content": h[0]},
                {"role": "assistant", "content": h[1]}
            ])
        messages.append({"role": "user", "content": message})

        async with httpx.AsyncClient() as client:
            try:
                response = await client.post(
                    f"{LLAMA_ENDPOINT}/v1/chat/completions",
                    json={
                        "messages": messages,
                        "model": "meta-llama/Meta-Llama-3.1-8B-Instruct",
                        "stream": False,
                        "temperature": 0.7,
                        "top_p": 0.95
                    },
                    timeout=30.0
                )

                if response.status_code == 200:
                    response_data = response.json()
                    return response_data['choices'][0]['text']
                else:
                    return f"Error: Received status code {response.status_code} from Llama endpoint"
            except Exception as e:
                return f"Error communicating with Llama endpoint: {str(e)}"

    def run_server(self):
        interface = gr.ChatInterface(
            fn=self.chat_with_llama,
            type="messages",
            title="Chat with Llama",
            description="This is a chat interface powered by Llama 3.1 8B Instruct",
            examples=[
                ["What is the capital of France?"],
                ["Explain quantum computing in simple terms"],
                ["Write a short poem about technology"]
            ],
        )
        interface.launch(
            server_name=self.host,
            server_port=self.port,
            root_path=f"https://api.aws.us-east-1.cerebrium.ai/v4/{os.getenv('PROJECT_ID')}/{os.getenv('APP_NAME')}/",
            quiet=True
        )

    def start(self):
        print(f"Starting Gradio server at {self.url} port {self.port}")

        # Start Gradio in a separate process
        self.process = multiprocessing.Process(target=self.run_server)
        self.process.start()

        # Wait for Gradio to become ready
        max_retries = 30
        retry_delay = 1.0

        for _ in range(max_retries):
            try:
                response = requests.get(f"{self.url}/")
                if response.status_code == 200:
                    print(f"Gradio server is ready at {self.url}")
                    return True
            except requests.exceptions.ConnectionError:
                time.sleep(retry_delay)

        print("Failed to start Gradio server")
        self.stop()
        return False

    def stop(self):
        if self.process:
            self.process.terminate()
            self.process.join()
            self.process = None

```

--------------------------------

### Deploy Langchain and Langsmith Executive Assistant

Source: https://context7_llms

This example demonstrates deploying an executive assistant using Langchain and Langsmith on Cerebrium. It integrates LLMs with external data and tools for enhanced functionality.

```python
from langchain.agents import AgentExecutor, create_openai_functions_agent
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_openai import ChatOpenAI
from langchain import hub

# Initialize the LLM
llm = ChatOpenAI(model="gpt-4o", temperature=0)

# Get a prompt template
prompt = hub.pull("hwchase/openai-functions-agent")

# Define tools (replace with your actual tools)
# tools = [...] 

# Create the agent
agent = create_openai_functions_agent(llm, tools, prompt)

# Create the agent executor
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)

def run_assistant(user_input):
    result = agent_executor.invoke({"input": user_input})
    return result['output']

# Example usage:
# assistant_response = run_assistant("What is the latest sales report?")
# print(assistant_response)

```

--------------------------------

### Setup FastAPI and SGLang Runtime for Vision Language API

Source: https://docs.cerebrium.ai/v4/examples/deploy-a-vision-language-model-with-sglang

Initializes a FastAPI application and configures the SGLang Runtime Engine for multimodal capabilities. The model is loaded on application startup to ensure immediate availability for subsequent requests. Dependencies include sglang, fastapi, and transformers.

```python
import sglang as sgl
from sglang import function
from fastapi import FastAPI, HTTPException
from transformers import AutoProcessor

app = FastAPI(title="Vision Language SGLang API")
model_path = "Qwen/Qwen3-VL-30B-A3B-Instruct-FP8"
processor = AutoProcessor.from_pretrained(model_path)

@app.on_event("startup")
def _startup_warmup():
    # Initialize engine on main thread during app startup
    runtime = sgl.Runtime(
        model_path=model_path,
        enable_multimodal=True,
        mem_fraction_static=0.8,
        tp_size=1,
        attention_backend="flashinfer",
    )
    runtime.endpoint.chat_template = sgl.lang.chat_template.get_chat_template(
        "qwen2-vl"
    )
    sgl.set_default_backend(runtime)


@app.get("/health")
def health():
    return {
        "status": "healthy",
    }

```

--------------------------------

### FastAPI Server Implementation for Cerebrium

Source: https://docs.cerebrium.ai/cerebrium/container-images/custom-web-servers

A basic FastAPI application demonstrating how to create endpoints for a custom web server on Cerebrium. This includes a POST endpoint '/hello' and GET endpoints '/health' and '/ready' for health checks. It requires FastAPI to be installed.

```python
from fastapi import FastAPI
app = FastAPI()

@app.post("/hello")
def hello():
    return {"message": "Hello Cerebrium!"}

@app.get("/health")
def health():
    return "OK"

@app.get("/ready")
def ready():
    return "OK"
```

--------------------------------

### Initialize Cerebrium Project

Source: https://docs.cerebrium.ai/v4/examples/langchain-langsmith

Initializes a new Cerebrium project with a specified template. This command creates the necessary project structure, including an entrypoint file (`main.py`) and a configuration file (`cerebrium.toml`).

```bash
cerebrium init agent-tool-calling
```

--------------------------------

### Start Bot Session with Daily.co Room (Python)

Source: https://docs.cerebrium.ai/v4/examples/realtime-voice-agents

This function initiates a bot session by calling the main function with a room URL and an optional token. It includes error handling to log exceptions and exit the script if an error occurs. It depends on the 'main' function and 'logger' and 'sys' modules.

```python
import sys
import logging

logger = logging.getLogger(__name__)

async def start_bot(room_url: str, token: str = None):

    try:
        await main(room_url, token)
    except Exception as e:
        logger.error(f"Exception in main: {e}")
        sys.exit(1)  # Exit with a non-zero status code

    return {"message": "session finished"}
```

--------------------------------

### Setup FastAPI Application (`main.py`)

Source: https://docs.cerebrium.ai/v4/examples/asgi-gradio-interface

Sets up the FastAPI application instance and defines environment variables for connecting to the Gradio server. It imports necessary libraries like `requests`, `httpx`, `FastAPI`, and `starlette.responses`.

```python
# at the top of your main.py file
import requests
from typing import Optional, List

import httpx
from fastapi import FastAPI, Request
from starlette.responses import Response as StarletteResponse

app = FastAPI()

# Get the Gradio app URL (when running on Cerebrium)
GRADIO_HOST = os.getenv("GRADIO_HOST", "127.0.0.1")
GRADIO_PORT = int(os.getenv("GRADIO_PORT", "7860"))
GRADIO_URL = os.getenv("GRADIO_SERVER_URL", f"http://{GRADIO_HOST}:{GRADIO_PORT}")

```

--------------------------------

### Get Project Payment URL

Source: https://docs.cerebrium.ai/api-reference/subscriptions/get-project-payment-url

Retrieve the payment URL for a specific project. This endpoint is used to get the URL where a project's payment can be processed.

```APIDOC
## GET /v2/projects/{project_id}/payment-url

### Description
Retrieve the payment URL for a specific project.

### Method
GET

### Endpoint
/v2/projects/{project_id}/payment-url

### Parameters
#### Path Parameters
- **project_id** (string) - Required - The unique identifier of the project.

### Request Example
```json
{
  "example": ""
}
```

### Response
#### Success Response (200)
- **payment_url** (string) - The URL for project payment.

#### Response Example
```json
{
  "payment_url": "https://example.com/pay?project_id=your_project_id"
}
```

### Security
- BearerAuth: Service Account Token authentication. Include the token in the Authorization header: `Authorization: Bearer <your-service-account-token>`
```

--------------------------------

### Example Response from Cerebrium Application

Source: https://docs.cerebrium.ai/v4/examples/deploy-a-vision-language-model-with-sglang

This is an example JSON response received after sending a request to the deployed Cerebrium application. It indicates the success of the analysis and provides a summary and grade for the advertisement.

```json
{
  "success": true,
  "analysis": {
    "summary": "The company description is relevant to the image because it accurately reflects Nike's branding, which is showcased through the advertised sneaker and logo. The ad promotes Nike's core products—athletic footwear—and its values of performance, style, and inspiration, aligning with the brand's identity. The collaboration with a superhero theme further emphasizes innovation and empowerment, core ",
    "grade": "A"
  },
  "dimensions_evaluated": ["Effectiveness", "Clarity", "Appeal", "Credibility"]
}
```

--------------------------------

### Create and Execute a Langchain Tool-Calling Agent in Python

Source: https://docs.cerebrium.ai/v4/examples/langchain-langsmith

This Python code illustrates the creation and execution of a Langchain agent capable of tool calling. It utilizes `create_tool_calling_agent` to unify concepts and `AgentExecutor` to run the agent with specified tools and a prompt. The example invocation shows how to pass user input for the agent to process.

```python
agent = create_tool_calling_agent(llm, tools, prompt)
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)

agent_executor.invoke({"input": "what's 3 plus 5 raised to the 2.743. also what's 17.24 - 918.1241", })
```

--------------------------------

### Install Cerebrium CLI using pip

Source: https://docs.cerebrium.ai/migrations/hugging-face

Installs or upgrades the Cerebrium command-line interface using pip. This is the first step for setting up your environment for migration.

```bash
pip install cerebrium --upgrade
```

--------------------------------

### Initialize Project with Cerebrium CLI

Source: https://docs.cerebrium.ai/v4/examples/high-throughput-embeddings

Initializes a new project for deploying a high-throughput Infinity server using the Cerebrium CLI. This command sets up the necessary project structure and configuration files.

```bash
cerebrium init infinity-throughput
```

--------------------------------

### SGLang Frontend Primitives for LLM Logic

Source: https://docs.cerebrium.ai/v4/examples/deploy-a-vision-language-model-with-sglang

Demonstrates the core primitives available in the SGLang frontend for defining multi-step LLM workflows. These primitives allow for text generation, parallel execution, merging branches, and conditional selection, promoting clean and reusable code.

```text
gen("title", stop="\n")
fork()
join()
select()
```

--------------------------------

### Install Cerebrium CLI

Source: https://docs.cerebrium.ai/cerebrium/getting-started/introduction

Installs the Cerebrium command-line interface (CLI) using different package managers and operating systems. This is the first step to interacting with the Cerebrium platform.

```bash
pip install cerebrium
```

```bash
brew tap cerebriumai/tap
brew install cerebrium
```

```bash
# Ubuntu/Debian
wget https://github.com/CerebriumAI/cerebrium/releases/latest/download/cerebrium_linux_amd64.deb
sudo dpkg -i cerebrium_linux_amd64.deb

# Or binary installation
curl -L https://github.com/CerebriumAI/cerebrium/releases/latest/download/cerebrium_cli_linux_amd64.tar.gz | tar xz
sudo mv cerebrium /usr/local/bin/
```

```powershell
# PowerShell (Run as Administrator)
Invoke-WebRequest -Uri "https://github.com/CerebriumAI/cerebrium/releases/latest/download/cerebrium_cli_windows_amd64.zip" -OutFile "cerebrium.zip"
Expand-Archive -Path "cerebrium.zip" -DestinationPath "."
# Add cerebrium.exe to PATH
```

--------------------------------

### Deployment Configuration

Source: https://docs.cerebrium.ai/v4/examples/deploy-an-llm-with-tensorrtllm-tritonserver

Configure your container and autoscaling environment using the `cerebrium.toml` file. This file defines hardware resources, scaling behavior, and runtime settings.

```APIDOC
## Deployment Configuration

### Description
Configure your container and autoscaling environment using the `cerebrium.toml` file. This file defines hardware resources, scaling behavior, and runtime settings.

### Method
N/A (Configuration File)

### Endpoint
N/A (Configuration File)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None (Configuration file parameters):
- `cerebrium.deployment.name` (string) - Name of the deployment.
- `cerebrium.deployment.python_version` (string) - Python version to use.
- `cerebrium.deployment.disable_auth` (boolean) - Whether to disable authentication.
- `cerebrium.deployment.include` (array of strings) - Files/directories to include.
- `cerebrium.deployment.exclude` (array of strings) - Files/directories to exclude.
- `cerebrium.deployment.deployment_initialization_timeout` (integer) - Timeout for deployment initialization.
- `cerebrium.hardware.cpu` (float) - Number of CPU cores.
- `cerebrium.hardware.memory` (float) - Amount of memory in GB.
- `cerebrium.hardware.compute` (string) - Type of compute instance (e.g., 'AMPERE_A10').
- `cerebrium.hardware.gpu_count` (integer) - Number of GPUs.
- `cerebrium.hardware.provider` (string) - Cloud provider (e.g., 'aws').
- `cerebrium.hardware.region` (string) - Cloud region (e.g., 'us-east-1').
- `cerebrium.scaling.min_replicas` (integer) - Minimum number of replicas.
- `cerebrium.scaling.max_replicas` (integer) - Maximum number of replicas.
- `cerebrium.scaling.cooldown` (integer) - Cooldown period in seconds.
- `cerebrium.scaling.replica_concurrency` (integer) - Max concurrent requests per replica.
- `cerebrium.scaling.scaling_metric` (string) - Metric used for scaling (e.g., 'concurrency_utilization').
- `cerebrium.runtime.custom.port` (integer) - Port for the runtime.
- `cerebrium.runtime.custom.healthcheck_endpoint` (string) - Health check endpoint.
- `cerebrium.runtime.custom.readycheck_endpoint` (string) - Ready check endpoint.
- `cerebrium.runtime.custom.dockerfile_path` (string) - Path to the Dockerfile.

### Request Example
```toml
[cerebrium.deployment]
name = "tensorrt-triton-demo"
python_version = "3.12"
disable_auth = true
include = ["./*", "cerebrium.toml"]
exclude = [".*"]
deployment_initialization_timeout = 830

[cerebrium.hardware]
cpu = 4.0
memory = 40.0
compute = "AMPERE_A10"
gpu_count = 1
provider = "aws"
region = "us-east-1"

[cerebrium.scaling]
min_replicas = 0
max_replicas = 5
cooldown = 300
replica_concurrency = 128
scaling_metric = "concurrency_utilization"

[cerebrium.runtime.custom]
port = 8000
healthcheck_endpoint = "/v2/health/live"
readycheck_endpoint = "/v2/health/ready"
dockerfile_path = "./Dockerfile"
```

### Response
N/A (Configuration file)

#### Success Response (200)
N/A

#### Response Example
N/A
```

--------------------------------

### Get App Container Details (OpenAPI)

Source: https://docs.cerebrium.ai/api-reference/apps/get-app-container

Retrieves details for a specific container associated with an application. This OpenAPI specification defines the endpoint and parameters required for the GET request.

```yaml
openapi: 3.0.0
info:
  title: Cerebrium REST API
  description: >-
    REST API for interacting with Cerebrium. This API is mainly used by the
    Cerebrium CLI client, please run `pip install cerebrium` to install it.
  version: 1.0.0
servers:
  - url: https://rest.cerebrium.ai
security: []
paths:
  /v2/projects/{project_id}/apps/{app_id}/containers/{container_id}:
    get:
      tags:
        - Apps
      summary: Get App Container
      description: Retrieve details for a specific container for an app.
      parameters:
        - name: project_id
          in: path
          required: true
          schema:
            type: string
        - name: app_id
          in: path
          required: true
          schema:
            type: string
        - name: container_id
          in: path
          required: true
          schema:
            type: string
      responses:
        '200':
          description: OK
      security:
        - BearerAuth: []
components:
  securitySchemes:
    BearerAuth:
      type: http
      scheme: bearer
      description: >-
        Service Account Token authentication. To authenticate API requests:

        1. **Create a Service Account Token:**
           - Go to the [Cerebrium Dashboard](https://dashboard.cerebrium.ai/) and open the **API Keys** page
           - Click **Create Service Account**, name it (e.g., "GitHub Actions CI/CD"), choose an expiry date, and click **Create**
           - **Copy the token** generated for the desired service account

        2. **Use the Token:**
           Include the service account token in the Authorization header of API requests:
           `Authorization: Bearer <your-service-account-token>`

        3. **Best Practices:**
           - Create separate service accounts for different environments (dev, staging, prod)
           - Store tokens securely as secrets in consuming applications or workflows
           - Set appropriate expiry dates and rotate tokens regularly
           - Never commit tokens to source control

        For CI/CD integration examples, see the [CI/CD
        documentation](https://docs.cerebrium.ai/cerebrium/deployments/ci-cd).

```

--------------------------------

### Create Daily.co Room and Token (Python)

Source: https://docs.cerebrium.ai/v4/examples/realtime-voice-agents

This function creates a new Daily.co room with a 5-minute expiration and an option to eject participants at expiration. It then attempts to create a meeting token for the room. It requires the 'requests', 'os', 'time', and 'logging' modules, and a 'DAILY_TOKEN' environment variable.

```python
import requests
import os
import time
import logging

logger = logging.getLogger(__name__)

def create_room():
    url = "https://api.daily.co/v1/rooms/"
    headers = {
        "Content-Type": "application/json",
        "Authorization": f"Bearer {os.environ.get('DAILY_TOKEN')}",
    }
    data = {
        "properties": {
            "exp": int(time.time()) + 60 * 5,  ##5 mins
            "eject_at_room_exp": True,
        }
    }

    response = requests.post(url, headers=headers, json=data)
    if response.status_code == 200:
        room_info = response.json()
        token = create_token(room_info["name"])
        if token and "token" in token:
            room_info["token"] = token["token"]
        else:
            logger.error("Failed to create token")
            return {
                "message": "There was an error creating your room",
                "status_code": 500,
            }
        return room_info
    else:
        data = response.json()
        if data.get("error") == "invalid-request-error" and "rooms reached" in data.get(
            "info", ""
        ):
            logger.error(
                "We are currently at capacity for this demo. Please try again later."
            )
            return {
                "message": "We are currently at capacity for this demo. Please try again later.",
                "status_code": 429,
            }
        logger.error(f"Failed to create room: {response.status_code}")
        return {"message": "There was an error creating your room", "status_code": 500}
```

--------------------------------

### POST /v3/projects/{project_id}/apps/{app_id}/base-image

Source: https://docs.cerebrium.ai/api-reference/builds/create-base-image-hash

Generate a SHA256 hash for dependency lists including pre-build commands, apt packages, pip requirements, conda packages, and shell commands.

```APIDOC
## POST /v3/projects/{project_id}/apps/{app_id}/base-image

### Description
Generate a SHA256 hash for dependency lists including pre-build commands, apt packages, pip requirements, conda packages, and shell commands.

### Method
POST

### Endpoint
/v3/projects/{project_id}/apps/{app_id}/base-image

### Parameters
#### Path Parameters
- **project_id** (string) - Required - The ID of the project.
- **app_id** (string) - Required - The ID of the application.

#### Query Parameters
None

#### Request Body
- **aptPackages** (array) - Optional - List of apt packages.
- **baseImageUri** (string) - Optional - Base image URI to include in hash.
- **condaPackages** (array) - Optional - List of conda packages.
- **pipRequirements** (array) - Optional - List of pip requirements.
- **preBuildCommands** (array) - Optional - Base64 encoded pre-build commands.
- **shellCommands** (array) - Optional - Base64 encoded shell commands.

### Request Example
```json
{
  "aptPackages": [
    "package1",
    "package2"
  ],
  "baseImageUri": "docker.io/library/ubuntu:latest",
  "condaPackages": [
    "numpy",
    "pandas"
  ],
  "pipRequirements": [
    "requests==2.28.1",
    "flask"
  ],
  "preBuildCommands": [
    "base64encodedcommand1"
  ],
  "shellCommands": [
    "base64encodedcommand2"
  ]
}
```

### Response
#### Success Response (200)
- **hash** (string) - The generated SHA256 hash.

#### Response Example
```json
{
  "hash": "a1b2c3d4e5f67890abcdef1234567890abcdef1234567890abcdef1234567890"
}
```
```

--------------------------------

### Python Dependencies (requirements.txt) for LiveKit Agent

Source: https://docs.cerebrium.ai/v4/examples/livekit-outbound-agent

This file lists the Python packages required to run the LiveKit agent and its associated AI plugins. Install these using 'pip install -r requirements.txt'.

```python
livekit-agents>=0.11.1
livekit-plugins-openai>=0.10.5
livekit-plugins-deepgram
livekit-plugins-cartesia
livekit-plugins-silero>=0.7.3
livekit-plugins-rag>=0.2.2
python-dotenv~=1.0
aiofile~=3.8.8
fastapi
uvicorn
```

--------------------------------

### Deploy LLM Chat Interface with FastAPI and Gradio

Source: https://context7_llms

This example demonstrates how to deploy a Large Language Model (LLM) chat interface using FastAPI and Gradio on Cerebrium. It leverages ASGI for asynchronous operations, enabling real-time interactions.

```python
from fastapi import FastAPI
from gradio.routes import mount_gradio_app
import gradio as gr

# Placeholder for your LLM logic
def predict(message, history):
    # Replace with your actual LLM inference code
    return "This is a simulated response to: " + message

app = FastAPI()

gradio_interface = gr.ChatInterface(predict)

app = mount_gradio_app(app, gradio_interface, path="/")

# To run this locally (for testing):
# uvicorn main:app --reload

```

--------------------------------

### Example Response for Image Embedding Inference

Source: https://docs.cerebrium.ai/v4/examples/high-throughput-embeddings

Example JSON response received after performing image embedding inference. It contains a list of embeddings, where each embedding is a list of floating-point numbers representing the image features.

```json
{
    "embeddings": [
        [
            -0.05284368246793747,
            0.0011637501884251833,
            -0.029046623036265373,
            ....
        ]
    ]
}
```

--------------------------------

### Download Build

Source: https://docs.cerebrium.ai/api-reference/builds/download-build

Download the build ZIP file for a specific build of an application within a project.

```APIDOC
## GET /v2/projects/{project_id}/apps/{app_id}/builds/{build_id}/download

### Description
Download the build ZIP file for a specific build.

### Method
GET

### Endpoint
/v2/projects/{project_id}/apps/{app_id}/builds/{build_id}/download

### Parameters
#### Path Parameters
- **project_id** (string) - Required - The ID of the project.
- **app_id** (string) - Required - The ID of the application.
- **build_id** (string) - Required - The ID of the build to download.

### Response
#### Success Response (200)
- **file** (binary) - The ZIP file containing the build.

#### Response Example
(Binary data for a ZIP file)
```

--------------------------------

### Dockerfile for Cerebrium Deployment

Source: https://docs.cerebrium.ai/v4/examples/livekit-outbound-agent

This Dockerfile sets up the environment for deploying the application on Cerebrium. It installs Python, copies project files, installs dependencies, downloads model files, and configures environment variables and exposed ports.

```dockerfile
FROM debian:bookworm-slim

# Install Python and pip
RUN apt-get update && apt-get install -y \
    python3.11 \
    python3-pip \
    && rm -rf /var/lib/apt/lists/*

# Create and set working directory
WORKDIR /cortex

# Copy requirements and install dependencies
COPY requirements.txt .
RUN pip install --break-system-packages -r requirements.txt

# Copy application code
COPY . .

# Download model files
RUN python3 main.py download-files

# Set environment variables
ENV HF_HOME=/cortex/.cache/

# Expose port
EXPOSE 8600

# Set entrypoint
CMD ["python3", "main.py", "start"]
```