### Initialize Project with Cerebrium CLI

Source: https://context7_llms

Initializes a new project directory for the vision-language-sglang example using the Cerebrium CLI. This command sets up the necessary file structure for the tutorial. It requires the Cerebrium CLI to be installed and configured.

```bash
cerebrium init 7-vision-language-sglang
cd 7-vision-language-sglang
```

--------------------------------

### Set up AI Pipeline with Input and STT Integration

Source: https://docs.cerebrium.ai/v4/examples/realtime-voice-agents

Illustrates the setup of an AI pipeline, integrating various components for processing user input. It includes steps for capturing user input via a transport mechanism, performing speech-to-text conversion, and then aggregating the processed messages. This pipeline structure is fundamental for real-time voice-driven AI applications.

```python
pipeline = [
    transport.input(),  # Transport user input
    stt,                # Speech-to-text
    tma_in              # Aggregate user input messages
]
```

--------------------------------

### Install Cerebrium CLI using pip

Source: https://docs.cerebrium.ai/cerebrium/getting-started/introduction

Installs the Cerebrium command-line interface (CLI) using pip, the Python package installer. This is the primary method for Python users to get started.

```python
pip install cerebrium
```

--------------------------------

### Configure Docker Base Image for Cold-Start Optimization (TOML)

Source: https://context7_llms

This configuration snippet demonstrates how to set the Docker base image URL in a TOML file. It shows two options: a minimal runtime image for faster cold-starts and a full development image which results in slower cold-starts but offers more tools. The choice impacts initialization time and available development utilities.

```toml
# Minimal runtime image - faster cold-starts
docker_base_image_url = "debian:bookworm-slim"

# Full development image - slower cold-starts, more tools
docker_base_image_url = "nvidia/cuda:12.0.1-devel-ubuntu22.04"
```

--------------------------------

### Dockerfile for Rust Server Setup

Source: https://docs.cerebrium.ai/cerebrium/container-images/custom-dockerfiles

This snippet defines a Dockerfile for setting up a Rust server environment. It includes stages for building the project, installing dependencies, and caching build artifacts. The process starts from a Rust base image, installs necessary packages, creates a new Rust project, copies dependency files, builds the project to cache dependencies, and cleans up source files.

```dockerfile
FROM rust:bookworm as build
RUN apt-get update && apt-get install dumb-init
RUN update-ca-certificates
# Project setup
RUN USER=root cargo new --bin rs_server
WORKDIR /rs_server
# Dependencies
COPY Cargo.lock ./Cargo.lock
COPY Cargo.toml ./Cargo.toml
# Cache dependencies
RUN cargo build --release
RUN rm src/*.rs
```

--------------------------------

### Cerebrium Container Image Build Commands

Source: https://docs.cerebrium.ai/v4/examples/gpt-oss

This example shows pre-build commands used in a Cerebrium container image. These commands are executed before dependency installation and are used here to install 'uv', a fast Python package installer, and then install the required vLLM packages.

```shell
# Commands executed at the start of the build process
# Install uv (a faster Python package installer)
pip install uv
# Install required vLLM packages
pip install vllm
```

--------------------------------

### Initialize Cerebrium Project

Source: https://context7_llms

Initializes a new Cerebrium project with a specified starter template. This command creates the necessary entrypoint file (`main.py`) and configuration file (`cerebrium.toml`) for the project.

```bash
cerebrium init 4-twilio-voice-agent
```

--------------------------------

### Python FastAPI App Setup and Endpoints

Source: https://docs.cerebrium.ai/cerebrium/container-images/custom-dockerfiles

This snippet demonstrates setting up a basic FastAPI application in Python. It includes defining POST and GET endpoints for '/hello' and '/health' respectively. The '/hello' endpoint returns a JSON message, while '/health' serves as a basic health check. This code requires the FastAPI library to be installed.

```python
from fastapi import FastAPI

app = FastAPI()

@app.post("/hello")
def hello():
  return {"message": "Hello Cerebrium!"}

@app.get("/health")
def health():

```

--------------------------------

### Setup FastAPI and SGLang Runtime

Source: https://context7_llms

Initializes a FastAPI application and configures the SGLang Runtime Engine for multimodal inference. The model is loaded on startup to ensure subsequent requests are instantaneous. Dependencies include `sglang`, `fastapi`, and `transformers`.

```python
import sglang as sgl
from sglang import function
from fastapi import FastAPI, HTTPException
from transformers import AutoProcessor

app = FastAPI(title="Vision Language SGLang API")
model_path = "Qwen/Qwen3-VL-30B-A3B-Instruct-FP8"
processor = AutoProcessor.from_pretrained(model_path)

@app.on_event("startup")
def _startup_warmup():
    # Initialize engine on main thread during app startup
    runtime = sgl.Runtime(
        model_path=model_path,
        enable_multimodal=True,
        mem_fraction_static=0.8,
        tp_size=1,
        attention_backend="flashinfer",
    )
    runtime.endpoint.chat_template = sgl.lang.chat_template.get_chat_template(
        "qwen2-vl"
    )
    sgl.set_default_backend(runtime)


@app.get("/health")
def health():
    return {
        "status": "healthy",
    }

```

--------------------------------

### Create Project

Source: https://docs.cerebrium.ai/cerebrium/environments/custom-images

Create a new project.

```APIDOC
## POST /v2/projects

### Description
Create a new project.

### Method
POST

### Endpoint
/v2/projects

### Parameters
#### Request Body
- **name** (string) - Required - The name of the new project.

### Request Example
```json
{
  "name": "New Awesome Project"
}
```

### Response
#### Success Response (200)
- **project_id** (string) - The ID of the newly created project.
- **name** (string) - The name of the newly created project.

#### Response Example
```json
{
  "project_id": "proj_def456",
  "name": "New Awesome Project"
}
```
```

--------------------------------

### Install Litserve and FastAPI - Python

Source: https://docs.cerebrium.ai/cerebrium/scaling/batching-concurrency

This snippet shows how to install the 'litserve' and 'fastapi' Python packages using pip. It specifies that the 'latest' versions of these packages should be installed. This is a common setup for web services and API development in Python.

```python
pip install litserve="latest"
fastapi="latest"
```

--------------------------------

### Send GET Request to Cerebrium App via Custom Domain

Source: https://context7_llms

Illustrates how to send a GET request to a Cerebrium application using a custom domain. This example shows the necessary endpoint and authorization header.

```bash
curl https://api.example.com/v4/p-1234/my-app/run \
  -H "Authorization: Bearer {YOUR_API_KEY}"
```

--------------------------------

### Create App if it does not exist

Source: https://docs.cerebrium.ai/cerebrium/environments/app-scaling

Create a new run application for a project if it doesn't already exist.

```APIDOC
## POST /v3/projects/{project_id}/apps/{app_id}/create-run-app

### Description
Create a new run application for a project if it doesn't already exist.

### Method
POST

### Endpoint
/v3/projects/{project_id}/apps/{app_id}/create-run-app

### Parameters
#### Path Parameters
- **project_id** (string) - Required - The unique identifier for the project.
- **app_id** (string) - Required - The unique identifier for the application.

### Response
#### Success Response (200)
- **creation_status** (string) - Indicates whether the app was created or already existed.
```

--------------------------------

### Define Pre-build Commands for Cerebrium Container

Source: https://context7_llms

Specifies commands to execute at the beginning of the Cerebrium build process, before dependencies are installed. Useful for installing build tools or configuring the environment. Example shows downloading and making executable the 'pget' tool.

```toml
[cerebrium.deployment]
pre_build_commands = [
    # Add specialized build tools
    "curl -o /usr/local/bin/pget -L 'https://github.com/replicate/pget/releases/download/v0.6.2/pget_linux_x86_64'",
    "chmod +x /usr/local/bin/pget"
]

```

--------------------------------

### Setup Python Virtual Environment and Install Dependencies

Source: https://www.cerebrium.ai/blog/creating-a-realtime-rag-voice-agent

Creates a Python virtual environment named 'educator', activates it, and installs the 'python-dotenv' package for managing environment variables. This ensures consistent project dependencies.

```bash
python -m venv educator
source educator/bin/activate
pip install python-dotenv
```

--------------------------------

### Create App if it does not exist

Source: https://docs.cerebrium.ai/cerebrium/environments/custom-images

Create a new run app for a project.

```APIDOC
## POST /v3/projects/{project_id}/apps/{app_id}/create-run-app

### Description
Create a new run app for a project if it does not already exist.

### Method
POST

### Endpoint
/v3/projects/{project_id}/apps/{app_id}/create-run-app

### Parameters
#### Path Parameters
- **project_id** (string) - Required - The ID of the project.
- **app_id** (string) - Required - The ID of the application.

### Request Example
(No request body for this endpoint)

### Response
#### Success Response (200 or 201)
- **message** (string) - Confirmation message.

#### Response Example
```json
{
  "message": "Run app created successfully or already exists."
}
```
```

--------------------------------

### Using the Deployed Model API

Source: https://context7_llms

Example Python code demonstrating how to send a POST request to a deployed Cerebrium model endpoint to get inferences.

```APIDOC
## Using the Deployed Model API

### Description
Shows how to interact with your deployed Cerebrium model by sending a POST request with a prompt and receiving the model's generated output.

### Method
POST

### Endpoint
`https://api.aws.us-east-1.cerebrium.ai/v4/[PROJECT_NAME]/[MODEL_NAME]/run`

### Parameters

#### Path Parameters
- **[PROJECT_NAME]** (string) - Required - The name of your Cerebrium project.
- **[MODEL_NAME]** (string) - Required - The name of the deployed model (e.g., "llama-8b-vllm").

#### Headers
- **Authorization** (string) - Required - Bearer token for authentication. Format: `Bearer [CEREBRIUM_API_KEY]`.
- **Content-Type** (string) - Required - Specifies the request body format. Must be `application/json`.

#### Request Body
- **prompt** (string) - Required - The input text prompt for the model.
- **temperature** (float) - Optional - Controls randomness in generation.
- **top_p** (float) - Optional - Nucleus sampling parameter.
- **top_k** (integer) - Optional - Top-k sampling parameter.
- **max_tokens** (integer) - Optional - Maximum number of tokens to generate.
- **frequency_penalty** (float) - Optional - Penalty for repeated tokens.

### Request Example

```python
import requests
import json

url = "https://api.aws.us-east-1.cerebrium.ai/v4/[PROJECT_NAME]/llama-8b-vllm/run"

payload = json.dumps({"prompt": "tell me about yourself"})

headers = {
  'Authorization': 'Bearer [CEREBRIUM_API_KEY]',
  'Content-Type': 'application/json'
}

response = requests.request("POST", url, headers=headers, data=payload)

print(response.text)
```

### Response

#### Success Response (200)
- **result** (string) - The generated text output from the model.
```

--------------------------------

### Create App

Source: https://docs.cerebrium.ai/cerebrium/environments/custom-images

Create a new partner app for a specific project.

```APIDOC
## POST /v2/projects/{project_id}/partner-apps

### Description
Create a new partner app for a specific project.

### Method
POST

### Endpoint
/v2/projects/{project_id}/partner-apps

### Parameters
#### Path Parameters
- **project_id** (string) - Required - The ID of the project.

#### Request Body
- **name** (string) - Required - The name of the new app.
- **model** (string) - Required - The LLM model to use for the app.
- **description** (string) - Optional - A description for the app.

### Request Example
```json
{
  "name": "MyNewPartnerApp",
  "model": "gpt-4",
  "description": "This is a new partner application."
}
```

### Response
#### Success Response (201)
- **app_id** (string) - The ID of the newly created app.
- **name** (string) - The name of the app.
- **model** (string) - The LLM model used.

#### Response Example
```json
{
  "app_id": "app_abc123",
  "name": "MyNewPartnerApp",
  "model": "gpt-4"
}
```
```

--------------------------------

### Set up FastAPI Application for Gradio Proxy (`main.py`)

Source: https://context7_llms

Sets up a FastAPI application to act as a proxy for a Gradio application. It initializes the FastAPI app, defines environment variables for the Gradio server URL, implements a health check endpoint, and creates a catchall route to forward requests to the Gradio app.

```python
# at the top of your main.py file
import requests
from typing import Optional, List

import httpx
from fastapi import FastAPI, Request
from starlette.responses import Response as StarletteResponse

app = FastAPI()

# Get the Gradio app URL (when running on Cerebrium)
GRADIO_HOST = os.getenv("GRADIO_HOST", "127.0.0.1")
GRADIO_PORT = int(os.getenv("GRADIO_PORT", "7860"))
GRADIO_URL = os.getenv("GRADIO_SERVER_URL", f"http://{GRADIO_HOST}:{GRADIO_PORT}")

# Health check endpoint
@app.get("/health")
async def health_check():
    return {"status": "healthy"}


@app.route("/{path:path}", include_in_schema=False, methods=["GET", "POST"])
async def gradio(request: Request):
    print(f"Forwarding request path: {request.url.path}")

    headers = dict(request.headers)

    # Construct the full URL to Gradio, preserving the original path
    target_url = f"{GRADIO_URL}{request.url.path}"

    async with httpx.AsyncClient() as client:
        response = await client.request(
            request.method,
            target_url,
            headers=headers,
            data=await request.body(),
            params=request.query_params,
        )

        content = await response.aread()
        response_headers = dict(response.headers)
        return StarletteResponse(
            content=content,
            status_code=response.status_code,
            headers=response_headers,
        )

```

--------------------------------

### Complete Example: Cerebrium AI Deployment Configuration (TOML)

Source: https://docs.cerebrium.ai/toml-reference/toml-reference

This TOML snippet provides a comprehensive example of a Cerebrium AI deployment configuration. It includes sections for defining dependencies and other deployment-related settings, showcasing a complete setup for an LLM project.

```toml
[cerebrium.deployment]
# Other deployment configurations would go here

```

--------------------------------

### Bash: Setting up Cerebrium Environment

Source: https://docs.cerebrium.ai/v4/examples/wandb-sweep

This Bash script outlines the initial setup steps for using Cerebrium, including logging in with an API key and potentially installing necessary libraries. Proper setup is essential for accessing Cerebrium's services and deploying models.

```bash
#!/bin/bash

# Ensure you have the Cerebrium CLI installed
# pip install cerebrium

# Log in to your Cerebrium account using your API key
# Replace 'YOUR_CEREBRIUM_API_KEY' with your actual API key
cerebrium login --api-key "YOUR_CEREBRIUM_API_KEY"

# Verify the login status (optional)
cerebrium whoami

# Set up any necessary environment variables or configurations
# export CEREBRIUM_DEFAULT_PROJECT="your-project-name"

echo "Cerebrium environment setup complete."

```

--------------------------------

### Startup and Workflow Management

Source: https://context7_llms

This section covers the automatic startup of the ComfyUI server and loading of the default workflow upon application initialization. It also includes functions for loading workflow files and cleaning up temporary files.

```APIDOC
## Startup Event

### Description
This event handler is triggered on application startup. It loads the default workflow JSON file ('workflow_api.json') and starts the ComfyUI backend process if it's not already running. It also sets up signal handlers for graceful termination and waits for the ComfyUI server to become available via WebSocket.

### Method
`startup_event` (Internal FastAPI event)

### Parameters
None

### Request Example
N/A (triggered automatically)

### Response
N/A (internal process)

## Load Workflow File

### Description
Loads a workflow definition from a specified JSON file.

### Method
`load_workflow_file(file_path: str)`

### Parameters
- **file_path** (str) - Required - The path to the JSON workflow file.

### Request Example
```python
workflow_data = load_workflow_file("path/to/your/workflow.json")
```

### Response
- **dict** - The loaded workflow JSON object.

### Error Handling
- **500 Internal Server Error**: If the file is not found or if the JSON is invalid.

## Cleanup Temporary Files

### Description
Removes temporary files created during workflow execution.

### Method
`cleanup_tempfiles(files)`

### Parameters
- **files** (list) - A list of file objects or file paths to be cleaned up.

### Request Example
```python
temp_files = [...] # List of temporary files
cleanup_tempfiles(temp_files)
```

### Response
None

### Error Handling
Logs warnings for any errors during file deletion.
```

--------------------------------

### Agent Executor Setup (Python)

Source: https://context7_llms

This Python snippet configures an agent executor for calendar management. It defines a prompt template with system instructions, chat history, and user input, then initializes a `ChatOpenAI` model and combines them with the previously defined tools to create the `AgentExecutor`.

```python
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.tools import tool
from langchain.agents import create_tool_calling_agent, AgentExecutor
from langchain_openai import ChatOpenAI

prompt = ChatPromptTemplate.from_messages([
    ("system", "you're a helpful assistant managing the calendar of Michael Louis. You need to book appointments for a user based on available capacity and their preference. You need to find out if the user is: From Michaels team, a customer of Cerebrium or a friend or entrepreneur. If the person is from his team, book a morning slot. If its a potential customer for Cerebrium, book an afternoon slot. If its a friend or entrepreneur needing help or advice, book a night time slot. If none of these are available, book the earliest slot. Do not book a slot without asking the user what their preferred time is. Find out from the user, their name and email address."),
    MessagesPlaceholder(variable_name="chat_history"),
    ("human", "{input}"),
    MessagesPlaceholder(variable_name="agent_scratchpad"),
])

tools = [get_availability, book_slot]


llm = ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0, api_key=os.environ.get("OPENAI_API_KEY"))
agent = create_tool_calling_agent(llm, tools, prompt)
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)

```

--------------------------------

### Create Partner App

Source: https://docs.cerebrium.ai/cerebrium/environments/app-scaling

Create a new partner application for a specific project.

```APIDOC
## POST /v2/projects/{project_id}/partner-apps

### Description
Create a new partner application for a specific project.

### Method
POST

### Endpoint
/v2/projects/{project_id}/partner-apps

### Parameters
#### Path Parameters
- **project_id** (string) - Required - The unique identifier for the project.

#### Request Body
- **app_name** (string) - Required - The name of the new partner app.
- **configuration** (object) - Optional - Initial configuration for the app.

### Response
#### Success Response (200)
- **created_app_details** (object) - Contains the details of the newly created partner app.
```

--------------------------------

### Rime TTS Service Deployment and Usage

Source: https://context7_llms

This section details how to deploy the Rime Text-to-Speech service on Cerebrium, including necessary setup, configuration files, and examples for interacting with the deployed service via HTTP and WebSockets.

```APIDOC
## Rime TTS Service

### Description
Deploy Rime text-to-speech services on Cerebrium with simplified configurations, reduced latency, and region selection for data privacy.

### Setup
1.  **Create a Rime Account and API Key**: Obtain an API key from [Rime](https://www.rime.ai/). Create a secret named `RIME_API_KEY` in Cerebrium.
2.  **Initialize Rime Service**: Use the Cerebrium CLI to initialize the Rime service:
    ```bash
    cerebrium init rime
    ```
3.  **Configure `cerebrium.toml`**: Create or update your `cerebrium.toml` file with the following configuration:
    ```toml
    [cerebrium.deployment]
    name = "rime"
    disable_auth = true

    [cerebrium.runtime.rime]
    port = 8001

    [cerebrium.hardware]
    cpu = 4
    memory = 30
    compute = "AMPERE_A10"
    gpu_count = 1
    region = "us-east-1"

    [cerebrium.scaling]
    min_replicas = 1
    max_replicas = 2
    cooldown = 120
    replica_concurrency = 50
    ```
    *Note: `disable_auth = true` is required because Rime handles authentication via its API key in the header.*
4.  **Deploy the Service**: Run the deployment command:
    ```bash
    cerebrium deploy
    ```
    The output will include an App Dashboard URL.

### Request Examples

#### HTTP Endpoint
Send requests to the deployed Rime service using `curl`:

```bash
curl --location 'https://api.aws.us-east-1.cerebrium.ai/v4/<PROJECT_ID>/rime' \
--header 'Authorization: Bearer <RIME_API_KEY>' \
--header 'Content-Type: application/json' \
--header 'Accept: audio/pcm' \
--data '{
  "text": "I would love to have a conversation with you.",
  "speaker": "joy",
  "modelId": "mist"
}'
```

#### WebSocket Endpoint
Connect to the WebSocket endpoint for streaming audio:

```
wss://api.aws.us-east-1.cerebrium.ai/v4/<PROJECT_ID>/rime/ws2?audioFormat=mp3&speaker=cove&modelId=mistv2&phonemizeBetweenBrackets=true
Authorization Bearer <RIME_API_KEY>

# With messages like:
{"text": "This "},
{"text": "is "},
{"text": "a "},
{"text": "test against the "},
{"text": "websockets endpoint of the "},
{"text": "api image. "},
{"operation": "flush"},
{"text": "This "},
{"text": "is "},
{"text": "an "},
{"text": "incomplete "},
{"text": "phrase "},
{"operation": "eos"}
```

### Response

#### Success Response (HTTP)
-   **audio/pcm**: The synthesized speech audio stream.

#### Success Response (WebSocket)
-   Audio chunks or control messages as defined by the Rime WebSocket protocol.

#### Error Handling
-   Refer to Cerebrium and Rime documentation for specific error codes and messages related to authentication, invalid parameters, or service issues.
```

--------------------------------

### WebSocket Endpoint Implementation

Source: https://docs.cerebrium.ai/cerebrium/endpoints/websockets

This section details the requirements and provides an example for implementing a WebSocket endpoint. A custom runtime is necessary, and requests must be made to a WebSocket URL starting with 'wss://'.

```APIDOC
## WebSocket Endpoint Requirements and Usage

### Description
Details on setting up and interacting with WebSocket endpoints for Cerebrium AI LLMs.

### Key Requirements
*   **Custom Runtime Required**: A custom runtime is necessary to set up a WebSocket endpoint, allowing definition of how the app runs within the container.
*   **WebSocket URL Format**: Requests must be made to a WebSocket URL starting with `wss://`. Ensure your client supports secure WebSocket connections.

### Making a Request
Test your WebSocket endpoint using `websocat`:

```shell
websocat wss://api.aws.us-east-1.cerebrium.ai/v4/"<your-project-id>"/"<your-app-name>"/"<your-websocket-function-name>"
```

### Implementing the WebSocket Endpoint (Example)
Here's an example using FastAPI:

```python
# (Python code example would go here, but was truncated in the provided text)
```

### Response
(Specific response details for WebSocket interactions are not fully detailed in the provided text, but successful connections would typically involve message exchange.)
```

--------------------------------

### Configure Project with TOML

Source: https://docs.cerebrium.ai/cerebrium/container-images/private-docker-registry

This snippet shows how to configure your project using a TOML file. It specifies deployment settings such as the name, Python version, and the Docker base image URL. Ensure the docker_base_image_url points to a valid registry image.

```toml
[cerebrium.deployment]
name = "my-app"
python_version = "3.11"
docker_base_image_url = "your-registry.com/your-org/your-image:tag"

# Examples:
# docker_base_image_url = "mycompany/ml-base:v2.1" # Docker Hub
# docker_base_image_url = "123456.dkr.ecr.us-east-1.amazonaws.com/ml-base:latest" # ECR
# docker_base_image_url = "gcr.io/project-id/ml-base:latest" # GCR
```

--------------------------------

### Start ComfyUI Server on Application Startup

Source: https://context7_llms

Handles the startup event for the FastAPI application. It loads the workflow, starts the ComfyUI process in the background, and sets up signal handlers for graceful termination. It also includes a loop to wait for the ComfyUI server to become available via WebSocket.

```python
@app.on_event("startup")
async def startup_event():
    """Start ComfyUI server on application startup."""
    global json_workflow, side_process

    # Load workflow JSON
    json_workflow = load_workflow_file("workflow_api.json")
    logger.info("Loaded workflow from workflow_api.json")

    # Start ComfyUI process
    if side_process is None:
        # Assuming setup_comfyui and other necessary functions are defined elsewhere
        # For demonstration, placeholder for setup_comfyui
        def setup_comfyui(**kwargs):
            print(f"Simulating ComfyUI setup with: {kwargs}")
            # In a real scenario, this would start the ComfyUI server
            time.sleep(10) # Simulate startup time
            print("ComfyUI setup complete")

        side_process = Process(
            target=setup_comfyui,
            kwargs=dict(original_working_directory=original_working_directory, data_dir=""),
            daemon=True,
        )
        side_process.start()
        logger.info(f"Started ComfyUI process (PID: {side_process.pid})")

        for sig in [signal.SIGINT, signal.SIGTERM]:
            signal.signal(sig, lambda s, f: terminate_process())

        # Wait for ComfyUI to start
        max_attempts = 30
        for attempt in range(max_attempts):
            try:
                with websocket_connection() as (ws, _):
                    logger.info("Successfully connected to ComfyUI!")
                    break
            except Exception:
                logger.info(f"Waiting for ComfyUI to start... ({attempt + 1}/{max_attempts})")
                time.sleep(2)
        else:
            logger.warning("Could not confirm ComfyUI is running after multiple attempts")

```

--------------------------------

### Configure Pre-build Commands for Build Tools

Source: https://docs.cerebrium.ai/cerebrium/container-images/defining-container-images

Pre-build commands execute before dependency installation, useful for setting up the build environment. This example shows how to download and make executable a build tool using curl and chmod.

```toml
[cerebrium.deployment]
pre_build_commands = [
    # Add specialized build tools
    "curl -o /usr/local/bin/pget -L 'https://github.com/replicate/pget/releases/download/v0.6.2/pget_linux_x86_64'",
    "chmod +x /usr/local/bin/pget"
]
```

--------------------------------

### FastAPI and Gradio Initialization

Source: https://context7_llms

Sets up the necessary imports and initializes the FastAPI application. This snippet includes essential libraries for multiprocessing, environment variable access, time, type hinting, Gradio, HTTP requests, and FastAPI. It also initializes the FastAPI app instance.

```python
import multiprocessing
import os
import sys
import time
from typing import Optional, List

import gradio as gr
import httpx
import requests
from fastapi import FastAPI, Request
from starlette.responses import Response as StarletteResponse

# Initialize FastAPI
app = FastAPI()
```

--------------------------------

### Initializing a Deepgram App with Cerebrium CLI

Source: https://context7_llms

This command initializes a new Cerebrium application specifically configured for Deepgram services. It streamlines the setup process for integrating speech-to-text capabilities. Ensure you have the Cerebrium CLI installed and updated to version 1.39.0 or greater.

```bash
cerebrium init deepgram
```

--------------------------------

### Deploy App and Interact with Deepgram

Source: https://docs.cerebrium.ai/cerebrium/partner-services/deepgram

This section provides instructions for deploying an application using 'cerebrium deploy' and obtaining an endpoint for Deepgram services. It also guides users on downloading an example audio file for testing.

```text
Run ‘cerebrium deploy’ to deploy the app. After deployment and endpoint for the Deepgram services is provided in the terminal output (The URL for this endpoint can also be found in the App’s overview page on the dashboard).
Download an example audio file for use with the deepgram service:
```

--------------------------------

### Available Hardware API

Source: https://docs.cerebrium.ai/cerebrium/environments/app-scaling

Endpoints for retrieving information about available hardware options.

```APIDOC
## GET /v2/hardware

### Description
Retrieve available hardware options.

### Method
GET

### Endpoint
/v2/hardware

### Response
#### Success Response (200)
- **hardware** (array) - A list of available hardware configurations.

#### Response Example
{
  "hardware": [
    {
      "name": "CPU Standard",
      "description": "Standard CPU instance."
    },
    {
      "name": "GPU High Memory",
      "description": "GPU instance with high memory."
    }
  ]
}
```

--------------------------------

### Configure LLM Parameters (Python-like)

Source: https://docs.cerebrium.ai/v4/examples/openai-compatible-endpoint-vllm

This snippet demonstrates how to set parameters for a language model, including boolean flags and floating-point values for temperature and top_p. It appears to be part of a larger configuration or function call.

```python
{
 color: "#1F2328",
 "--shiki-dark": "#D4D4D4"
 },
 children: ": "
 }), 
 _jsx(_components.span, {
 style: {
 color: "#0550AE",
 "--shiki-dark": "#4EC9B0"
 },
 children: "bool"
 }), 
 _jsx(_components.span, {
 style: {
 color: "#CF222E",
 "--shiki-dark": "#D4D4D4"
 },
 children: " ="
 }), 
 _jsx(_components.span, {
 style: {
 color: "#0550AE",
 "--shiki-dark": "#569CD6"
 },
 children: " True"
 }), 
 _jsx(_components.span, {
 style: {
 color: "#1F2328",
 "--shiki-dark": "#D4D4D4"
 },
 children: ", "
 }), 
 _jsx(_components.span, {
 style: {
 color: "#1F2328",
 "--shiki-dark": "#9CDCFE"
 },
 children: "temperature"
 }), 
 _jsx(_components.span, {
 style: {
 color: "#1F2328",
 "--shiki-dark": "#D4D4D4"
 },
 children: ": "
 }), 
 _jsx(_components.span, {
 style: {
 color: "#0550AE",
 "--shiki-dark": "#4EC9B0"
 },
 children: "float"
 }), 
 _jsx(_components.span, {
 style: {
 color: "#CF222E",
 "--shiki-dark": "#D4D4D4"
 },
 children: " ="
 }), 
 _jsx(_components.span, {
 style: {
 color: "#0550AE",
 "--shiki-dark": "#B5CEA8"
 },
 children: " 0.8"
 }), 
 _jsx(_components.span, {
 style: {
 color: "#1F2328",
 "--shiki-dark": "#D4D4D4"
 },
 children: ", "
 }), 
 _jsx(_components.span, {
 style: {
 color: "#1F2328",
 "--shiki-dark": "#9CDCFE"
 },
 children: "top_p"
 }), 
 _jsx(_components.span, {
 style: {
 color: "#1F2328",
 "--shiki-dark": "#D4D4D4"
 },
 children: ": "
 }), 
 _jsx(_components.span, {
 style: {
 color: "#0550AE",
 "--shiki-dark": "#4EC9B0"
 },
 children: "float"
 }), 
 _jsx(_components.span, {
 style: {
 color: "#CF222E",
 "--shiki-dark": "#D4D4D4"
 },
 children: " ="
 }), 
 _jsx(_components.span, {
 style: {
 color: "#0550AE",
 "--shiki-dark": "#B5CEA8"
 },
 children: " 0.95"
 }), 
 _jsx(_components.span, {
 style: {
 color: "#1F2328",
 "--shiki-dark": "#D4D4D4"
 },
 children: "):"
 })
 ]
 }), 
 "\n", 
 _jsxs(_components.span, {
 className: "line",
 children: [
 _jsx(_components.span, {
 style: {
 color: "#1F2328",
 "--shiki-dark": "#D4D4D4"
 },
 children: " prompt "
 }), 
 _jsx(_components.span, {
 style: {
 color: "#CF222E",
 "--shiki-dark": "#D4D4D4"
 },
 children: "="
 }), 
 _jsx(_components.span, {
 style: {
 color: "#0A3069",
 "--shiki-dark": "#CE9178"
 },
 children: " ""
 }), 
 _jsx(_components.span, {
 style: {
 color: "#1F2328",
 "--shiki-dark": "#D4D4D4"
 },
 children: ".join([
 }), 
 _jsx(_components.span, {
 style: {
 color: "#CF222E",
 "--shiki-dark": "#569CD6"
 },
 children: "f"
 }), 
 _jsx(_components.span, {
 style: {
 color: "#0A3069",
 "--shiki-dark": "#CE9178"
 },
 children: """
 }), 
 _jsx(_components.span, {
 style: {
 color: "#CF222E",
 "--shiki-dark": "#569CD6"
 },
 children: "{"
 }), 
 _jsx(_components.span, {
 style: {
 color: "#1F2328",
 "--shiki-dark": "#D4D4D4"
 },
 children: "Message("
 }), 
 _jsx(_components.span, {
 style: {
 color: "#CF222E",
 "--shiki-dark": "#D4D4D4"
 },
 children: "**"
 }), 
 _jsx(_components.span, {
 style: {
 color: "#1F2328",
 "--shiki-dark": "#D4D4D4"
 },
 children: "msg).role"
 }), 
 _jsx(_components.span, {
 style: {
 color: "#CF222E",
 "--shiki-dark": "#569CD6"
 },
 children: "}"
 }), 
 _jsx(_components.span, {
 style: {
 color: "#0A3069",
 "--shiki-dark": "#CE9178"
 },
 children: ": "
 }), 
 _jsx(_components.span, {
 style: {
 color: "#CF222E",
 "--shiki-dark": "#569CD6"
 },
 children: "{"
 }), 
 _jsx(_components.span, {
 style: {
 color: "#1F2328",
 "--shiki-dark": "#D4D4D4"
 },
 children: "Message("
 }), 
 _jsx(_components.span, {
 style: {
 color: "#CF222E",
 "--shiki-dark": "#D4D4D4"
 },
 children: "**"
 }), 
 _jsx(_components.span, {
 style: {
 color: "#1F2328",
 "--shiki-dark": "#D4D4D4"
 },
 children: "msg).content"
 }), 
 _jsx(_components.span, {
 style: {
 color: "#CF222E",
 "--shiki-dark": "#569CD6"
 },
 children: "}"
 }), 
 _jsx(_components.span, {
 style: {
 color: "#0A3069",
 "--shiki-dark": "#CE9178"
 },
 children: """
 }), 
 _jsx(_components.span, {
 style: {
 color: "#CF222E",
 "--shiki-dark": "#C586C0"
 })
 ]
 })
]
```

--------------------------------

### Configure Shell Commands for Application Initialization

Source: https://docs.cerebrium.ai/cerebrium/container-images/defining-container-images

Shell commands run after dependency installation and code copying, suitable for initializing application resources. This example demonstrates running Python modules for model downloads, asset compilation, and app initialization.

```toml
[cerebrium.deployment]
shell_commands = [
    # Initialize application resources
    "python -m download_models",
    "python -m compile_assets",
    "python -m init_app"
]
```

--------------------------------

### Custom Domains API

Source: https://context7_llms

This section details how to connect your own domain to a Cerebrium project, allowing you to serve applications through custom URLs instead of the default `*.cerebrium.ai` domains. It covers key features, how custom domains work, the setup process, DNS configuration, validation, and usage examples.

```APIDOC
## Custom Domains

Connect your own domain to your Cerebrium project.

Custom domains allow serving Cerebrium apps through custom domains instead of the default `*.cerebrium.ai` URLs. Once configured, API calls will use the custom domain while keeping the same path structure: `api.example.com/v4/p-1234/my-app/run`.

### Key Features

* Support for apex domains (`example.com`) and subdomains (`api.example.com`)
* Automatic SSL certificate provisioning and renewal
* Project-level domains - one domain serves all apps in a project
* Multiple domains can point to the same project
* Professional branding with custom domains instead of `*.cerebrium.ai` URLs

### How Custom Domains Work

* **Domains are region-specific** meaning each will always resolve to the selected region
* **Domains are app-agnostic** enabling connection to any number of deployed apps within a project via custom domain
* **Multiple domains** can be configured on the same project (useful for apps within the project deployed in different regions)

### Getting Started

#### Step 1: Create a Custom Domain

1. Navigate to project settings in the Cerebrium dashboard
2. Click on the "Custom Domains" tab
3. Click "Add Custom Domain"
4. Enter the domain name and select the target region (choose the region closest to users for optimal latency)
5. Click "Create Domain"

#### Step 2: Configure DNS Records

After creating the domain, DNS configuration instructions will be displayed. Create a CNAME record at the DNS provider using the DNS record Cerebrium generates.

> **Note**: DNS record details can also be found later by clicking "DNS Record" in the Custom Domains list.

**Subdomains (Recommended)**

* **Benefits:**
    * Easier DNS configuration
    * Allows other subdomains for different purposes

```
Type: CNAME
Name: api
Value: proxy.aws.{region}.cerebrium.ai
```

**Apex Domains**

* **Benefits:**
    * Routes all traffic from the domain root
    * Best for dedicated API domains

> **Note**: ALIAS records are preferred over CNAME since only one CNAME is allowed per domain

```
Type: CNAME (or ALIAS if supported)
Name: @ (or leave blank)
Value: proxy.aws.{region}.cerebrium.ai
```

> **Note**: A `{region}` must be selected when creating a custom domain.

#### Step 3: Domain Validation

1. After configuring DNS, return to the Cerebrium dashboard
2. Cerebrium will automatically attempt to validate DNS records every 30 minutes for up to 2 days
3. To trigger an immediate validation attempt, click "Validation Status" -> "Validate Domain" (you can do this manually even after the 2 days have elapsed)
4. If validation fails, the dialog will show the last known error.
5. Once validated, SSL certificates will be automatically provisioned

#### Step 4: Start Using the Custom Domain

Once validation is complete, the custom domain can be used immediately with all apps in the project:

**POST Request Example**

```bash
curl -X POST https://api.example.com/v4/p-1234/my-app/run \
  -H "Authorization: Bearer {YOUR_API_KEY}" \
  -H "Content-Type: application/json" \
  -d "{'inputs': {'prompt': 'Hello world'}}"
```

**GET Request Example**

```bash
curl https://api.example.com/v4/p-1234/my-app/run \
  -H "Authorization: Bearer {YOUR_API_KEY}"
```

## Managing Domains

[Further details on managing domains would go here.]
```

--------------------------------

### AI Agent Setup using PipeCat and Python

Source: https://context7_llms

This Python script sets up an AI agent using the PipeCat framework. It configures a pipeline that includes WebSocket transport for audio I/O, Deepgram for Speech-to-Text, OpenAI for LLM processing, and Cartesia for Text-to-Speech. The pipeline supports interruptions and allows for easy service swapping. Authentication is handled via environment variables (secrets).

```python
import os
import sys

from loguru import logger
from pipecat.frames.frames import LLMMessagesFrame, EndFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.runner import PipelineRunner
from pipecat.pipeline.task import PipelineParams, PipelineTask

from pipecat.services.openai import OpenAILLMService
from pipecat.processors.aggregators.openai_llm_context import (
    OpenAILLMContext,
)
from pipecat.services.deepgram import DeepgramSTTService
from pipecat.vad.silero import SileroVADAnalyzer
from twilio.rest import Client
from twilio.twiml.voice_response import VoiceResponse
from pipecat.transports.network.fastapi_websocket import (
    FastAPIWebsocketTransport,
    FastAPIWebsocketParams,
)
from pipecat.serializers.twilio import TwilioFrameSerializer

from pipecat.services.cartesia import CartesiaTTSService

logger.remove(0)
logger.add(sys.stderr, level="DEBUG")

twilio = Client(
    os.environ.get("TWILIO_ACCOUNT_SID"), os.environ.get("TWILIO_AUTH_TOKEN")
)

async def main(websocket_client, stream_sid):
    transport = FastAPIWebsocketTransport(
        websocket=websocket_client,
        params=FastAPIWebsocketParams(
            audio_out_enabled=True,
            add_wav_header=False,
            vad_enabled=True,
            vad_analyzer=SileroVADAnalyzer(),
            vad_audio_passthrough=True,
            serializer=TwilioFrameSerializer(stream_sid),
        ),
    )

    stt = DeepgramSTTService(api_key=os.getenv("DEEPGRAM_API_KEY"))
    llm = OpenAILLMService(
        name="LLM",
        api_key=os.environ.get("OPENAI_API_KEY"),
        model="gpt-4",
    )


    tts = CartesiaTTSService(
        api_key=os.getenv("CARTESIA_API_KEY"),
        voice_id="79a125e8-cd45-4c13-8a67-188112f4dd22",  # British Lady
    )

    messages = [
        {
            "role": "system",
            "content": "You are a helpful LLM in an audio call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be converted to audio so don't include special characters in your answers. Respond to what the user said in a creative and helpful way.",
        },
    ]
    print('here', flush=True)
    context = OpenAILLMContext(messages=messages)
    context_aggregator = llm.create_context_aggregator(context)

    pipeline = Pipeline(
        [
            transport.input(),  # Websocket input from client
            stt,  # Speech-To-Text
            context_aggregator.user(),
            llm,  # LLM
            tts,  # Text-To-Speech
            transport.output(),  # Websocket output to client
            context_aggregator.assistant(),
        ]
    )

    task = PipelineTask(pipeline, params=PipelineParams(allow_interruptions=True))

    @transport.event_handler("on_client_connected")
    async def on_client_connected(transport, client):
        # Kick off the conversation.
        messages.append({"role": "system", "content": "Please introduce yourself to the user."}))
        await task.queue_frames([LLMMessagesFrame(messages)])

    @transport.event_handler("on_client_disconnected")
    async def on_client_disconnected(transport, client):
        await task.queue_frames([EndFrame()])

    runner = PipelineRunner(handle_sigint=False)

    await runner.run(task)

```

--------------------------------

### Python FastAPI Server Dockerization

Source: https://context7_llms

Demonstrates a simple FastAPI server in Python and its containerization using a Dockerfile. The Dockerfile includes steps for installing dependencies, copying source code, exposing a port, and defining the command to run the server. This setup is suitable for deployment on platforms like Cerebrium.

```python
from fastapi import FastAPI

app = FastAPI()

@app.post("/hello")
def hello():
    return {"message": "Hello Cerebrium!"}

@app.get("/health")
def health():
    return "OK"

@app.get("/ready")
def ready():
    return "OK"
```

```dockerfile
# Base image
FROM python:3.12-bookworm
RUN apt-get update && apt-get install dumb-init
RUN update-ca-certificates

# Source code
COPY . .

# Dependencies
RUN pip install -r requirements.txt

# Configuration
EXPOSE 8192
CMD ["python", "-m", "uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8192"]
```

```toml
[cerebrium.runtime.custom]
port = 8192
healthcheck_endpoint = "/health"
readycheck_endpoint = "/ready"
dockerfile_path = "./Dockerfile"
```

```toml
[cerebrium.runtime.custom]
entrypoint = ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8192"]
...
```