### Manual Setup and Installation

Source: https://chutes.ai/docs/sign-in-with-chutes/overview

Clone the repository and execute the setup wizard to register your OAuth application and generate credentials.

```bash
# Clone the repository
git clone https://github.com/chutesai/Sign-in-with-Chutes.git

# Install dependencies and run the setup wizard
cd Sign-in-with-Chutes
npm install
npx tsx scripts/setup-chutes-app.ts
```

--------------------------------

### Manual SDK Setup for "Sign in with Chutes"

Source: https://chutes.ai/docs/sign-in-with-chutes/nextjs

Clone the SDK repository, install dependencies, and run the interactive setup wizard to configure your Next.js app for "Sign in with Chutes". Copy the generated files into your project.

```bash
git clone https://github.com/chutesai/Sign-in-with-Chutes.git
cd Sign-in-with-Chutes
npm install
npx tsx scripts/setup-chutes-app.ts
```

--------------------------------

### Install Homebrew on macOS

Source: https://chutes.ai/docs/miner-resources/ansible

Initial setup command for Homebrew package manager on macOS.

```bash
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
```

--------------------------------

### Basic ML Image Example

Source: https://chutes.ai/docs/sdk-reference/image

An example demonstrating the creation of a basic Machine Learning Docker image using the Chutes AI library, including installing dependencies and setting up the application environment.

```APIDOC
## Complete Examples: Basic ML Image

### Description
This example shows how to construct a Docker image for a machine learning application.

### Method
N/A (Example usage of multiple methods)

### Endpoint
N/A

### Parameters
N/A

### Request Example
```python
from chutes.image import Image

image = (
    Image(username="myuser", name="ml-app", tag="1.0")
    .from_base("parachutes/python:3.12")
    .run_command("pip install torch transformers accelerate")
    .add("requirements.txt", "/app/requirements.txt")
    .run_command("pip install -r /app/requirements.txt")
    .add("src/", "/app/src/")
    .set_workdir("/app")
    .with_env("PYTHONPATH", "/app")
)
```

### Response
#### Success Response (200)
N/A (This is a code example for building an image)

#### Response Example
N/A
```

--------------------------------

### Build Output Example

Source: https://chutes.ai/docs/cli/build

Example of the console output during a successful build process.

```bash
# Example build output
Building chute: my_chute:chute
✓ Analyzing code structure
✓ Packaging build context
✓ Uploading to build server
✓ Building image layers
✓ Installing dependencies
✓ Pushing to registry

Build completed successfully!
Image ID: img_abc123def456
```

--------------------------------

### CI/CD Deployment Example

Source: https://chutes.ai/docs/cli/account

Example configuration for deploying to Chutes using GitHub Actions.

```yaml
# GitHub Actions example
env:
  CHUTES_API_KEY: ${{ secrets.CHUTES_API_KEY }}

steps:
  - name: Deploy to Chutes
    run: |
      pip install chutes
      mkdir -p ~/.chutes
      cat > ~/.chutes/config.ini << EOF
      [api]
      base_url = https://api.chutes.ai
      
      [auth]
      # Use API key authentication
      EOF
      chutes deploy my_app:chute --accept-fee
```

--------------------------------

### Install and Initialize Chutes SDK

Source: https://chutes.ai/docs/sdk

Commands to install the SDK and perform initial account registration and deployment.

```bash
# Install the Chutes SDK
pip install chutes

# Register your account
chutes register

# Deploy your first chute
chutes deploy my_chute:chute
```

--------------------------------

### Installation and Verification

Source: https://chutes.ai/docs/cli/overview

Instructions on how to install the Chutes CLI and verify the installation.

```APIDOC
## Installation

The CLI is included when you install the Chutes SDK:

```bash
pip install chutes
```

Verify installation:

```bash
chutes --help
```
```

--------------------------------

### Install Chutes SDK

Source: https://chutes.ai/docs/getting-started/installation

Install the SDK using pip or from the source repository.

```bash
pip install chutes
```

```bash
git clone https://github.com/chutesai/chutes.git
cd chutes
pip install -e .
```

--------------------------------

### Model-Specific Selection Example

Source: https://chutes.ai/docs/sdk-reference/node-selector

Example for selecting hardware suitable for 7B parameter models, balancing VRAM and GPU count.

```python
# 7B parameter models (e.g., Mistral-7B, Llama-2-7B)
llm_7b_selector = NodeSelector(
    gpu_count=1,
    min_vram_gb_per_gpu=24,
    include=["l40", "a5000", "rtx4090"]
)
```

--------------------------------

### Install 3rd Party Helm Charts

Source: https://chutes.ai/docs/miner-resources/ansible

Installs necessary components like the NVIDIA GPU operator and Prometheus.

```bash
ansible-playbook -i inventory.yml extras.yml
```

--------------------------------

### Agent Setup API

Source: https://chutes.ai/docs/api-reference/users

One-time setup endpoint for agent-registered users. Requires hotkey signature to prove ownership. Returns API key and config.ini template.

```APIDOC
## POST /users/{user_id}/agent_setup

### Description
One-time setup endpoint for agent-registered users. Requires hotkey signature to prove ownership. Returns API key and config.ini template.

### Method
POST

### Endpoint
/users/{user_id}/agent_setup

#### Path Parameters
- **user_id** (string) - Required - The ID of the user.

#### Request Body
- **hotkey** (string) - Required - The agent's hotkey.
- **signature** (string) - Required - The signature proving ownership of the hotkey.

### Request Example
```json
{
  "hotkey": "your_agent_hotkey",
  "signature": "your_signature"
}
```

### Responses
#### Success Response (200)
- **api_key** (string) - The generated API key for the agent.
- **config_ini_template** (string) - A template for the config.ini file.

#### Error Response (422)
- **message** (string) - Error message indicating validation failure.
```

--------------------------------

### Install from Requirements File using run_command

Source: https://chutes.ai/docs/sdk-reference/image

Install Python dependencies from a requirements file by first adding the file to the image using `.add()` and then executing `pip install -r` via `.run_command()`.

```python
# Install from requirements file
image = (
    Image("myuser", "myapp", "1.0")
    .from_base("parachutes/python:3.12")
    .add("requirements.txt", "/tmp/requirements.txt")
    .run_command("pip install -r /tmp/requirements.txt")
)
```

--------------------------------

### GitHub Actions CI/CD Build

Source: https://chutes.ai/docs/cli/build

Example GitHub Actions workflow for building and deploying Chutes applications. It includes setup, installation, configuration, and the build step.

```yaml
name: Build and Deploy
on:
  push:
    branches: [main]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Setup Python
        uses: actions/setup-python@v4
        with:
          python-version: "3.11"

      - name: Install Chutes
        run: pip install chutes

      - name: Configure Chutes
        env:
          CHUTES_CONFIG: ${{ secrets.CHUTES_CONFIG }}
        run: |
          mkdir -p ~/.chutes
          echo "$CHUTES_CONFIG" > ~/.chutes/config.ini

      - name: Build Image
        run: chutes build my_app:chute --wait
```

--------------------------------

### First-Time Setup

Source: https://chutes.ai/docs/cli/overview

Register an account and generate an initial administrative API key.

```bash
# Register account
chutes register

# Create admin API key
chutes keys create --name admin --admin
```

--------------------------------

### Diagnose Python Package Installation Errors

Source: https://chutes.ai/docs/help/troubleshooting

Example of a common package installation error message.

```bash
ERROR: Could not find a version that satisfies the requirement torch==2.1.0
```

--------------------------------

### Deployment Examples

Source: https://chutes.ai/docs/cli/deploy

Common deployment scenarios including fee acknowledgment, logo usage, and visibility settings.

```bash
# Deploy with fee acknowledgment
chutes deploy my_chute:chute --accept-fee
```

```bash
# Deploy with logo
chutes deploy my_chute:chute \
  --logo ./assets/logo.png \
  --accept-fee
```

```bash
# Private deployment (default) - only you can access
chutes deploy my_chute:chute --accept-fee

# Public deployment (requires special permissions)
chutes deploy my_chute:chute --public --accept-fee
```

--------------------------------

### Control Costs with Spot Instances and Scaling

Source: https://chutes.ai/docs/help/faq

Demonstrates cost-saving configurations including spot instance selection, scaling to zero, and hardware filtering.

```python
node_selector = NodeSelector()
```

```python
chute = Chute(
    username="myuser",
    name="cost-optimized",
    min_replicas=0  # No idle costs
)
```

```python
# Cost-effective GPUs for development
node_selector = NodeSelector(
    include=["l40", "a6000"],  # Less expensive than H100
    exclude=["h100", "h200"]
)
```

--------------------------------

### Install Python Packages using run_command

Source: https://chutes.ai/docs/sdk-reference/image

Execute shell commands during image build using `.run_command()`. This example installs Python packages like torch and transformers.

```python
# Install Python packages
image = (
    Image("myuser", "myapp", "1.0")
    .from_base("parachutes/python:3.12")
    .run_command("pip install torch transformers accelerate")
)
```

--------------------------------

### Compare Manual vs Template Deployment

Source: https://chutes.ai/docs/core-concepts/templates

Demonstrates the reduction in configuration complexity when using templates compared to manual Chute setup.

```python
# Without templates (complex setup)
image = (
    Image(username="myuser", name="vllm-app", tag="1.0")
    .from_base("nvidia/cuda:12.1-devel-ubuntu22.04")
    .with_python("3.11")
    .run_command("pip install vllm==0.2.0")
    .run_command("pip install transformers torch")
    # ... 50+ more lines of configuration
)

chute = Chute(
    username="myuser",
    name="llm-service",
    image=image,
    node_selector=NodeSelector(gpu_count=1, min_vram_gb_per_gpu=16)
)

@chute.on_startup()
async def load_model(self):
    # ... complex model loading logic

@chute.cord(public_api_path="/v1/chat/completions")
async def chat_completions(self, request: ChatRequest):
    # ... OpenAI API compatibility logic

# With templates (one line)
chute = build_vllm_chute(
    username="myuser",
    model_name="microsoft/DialoGPT-medium",
    node_selector=NodeSelector(gpu_count=1, min_vram_gb_per_gpu=16)
)
```

--------------------------------

### Example Chute Instance Output

Source: https://chutes.ai/docs/cli/deployment-troubleshooting

This is an example output when checking for active chute instances. Look for the 'active' tag; 'true' indicates the chute is hot, while 'false' means it is still starting.

```bash
Mac:~ chutist$ chutes chutes get 8f2105c5-b200-5aa5-969f-0720f7690f3c | jq '.instances'
2026-01-14 14:43:26.979 | INFO     | chutes.config:get_config:55 - Loading chutes config from /Users/algowarry/.chutes/config.ini...
2026-01-14 14:43:26.979 | DEBUG    | chutes.config:get_config:78 - Configured chutes: with api_base_url=https://api.chutes.ai
2026-01-14 14:43:26.979 | DEBUG    | chutes.util.auth:sign_request:67 - Signing message: 5CPBAsApHkvdhviSry2xoN38MLX5xG5FUPy3QVWVPe46JxCT:1768419806:chutes
2026-01-14 14:43:27.189 | INFO     | chutes.crud:_get_object:161 - chute 8f2105c5-b200-5aa5-969f-0720f7690f3c:
[
  {
    "instance_id": "b3f083a3-3460-4d37-83e1-6c945c5d3c61",
    "region": "n/a",
    "active": true,
    "verified": true,
    "last_verified_at": "2026-01-14T19:38:24.773055Z"
  },
  {
    "instance_id": "dbf02090-aa98-4584-98c1-2f36899050b8",
    "region": "n/a",
    "active": true,
    "verified": true,
    "last_verified_at": "2026-01-14T19:38:24.773055Z"
  },
  {
    "instance_id": "e6c2eb0e-4b61-43da-94cd-e17c933e6159",
    "region": "n/a",
    "active": true,
    "verified": true,
    "last_verified_at": "2026-01-14T19:38:24.773055Z"
  }
]
```

--------------------------------

### Configure Logging on Startup

Source: https://chutes.ai/docs/guides/best-practices

Loads configuration and sets up basic logging based on the application's log level. Ensure the configuration object has a 'log_level' attribute.

```python
import logging

@chute.on_startup()
async def load_configuration(self):
    """Load and validate configuration."""
    self.config = get_config()

    # Configure logging based on config
    logging.basicConfig(
        level=getattr(logging, self.config.log_level),
        format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
    )

    self.logger = logging.getLogger(f"chute.{self.config.environment}")
    self.logger.info(f"Application started in {self.config.environment} mode")
```

--------------------------------

### Handle Startup Logic

Source: https://chutes.ai/docs/guides/production-readiness

Pre-load models and artifacts during startup to ensure fast initial request handling. Fail fast if critical resources are missing.

```python
@chute.on_startup()
async def startup(self):
    # Fail fast if critical resources are missing
    if not os.path.exists("model.bin"):
        raise RuntimeError("Model file missing!")
    self.model = load_model("model.bin")
```

--------------------------------

### Complete Diffusion Chute Example

Source: https://chutes.ai/docs/templates/diffusion

A comprehensive example demonstrating the setup of a diffusion chute optimized for SDXL, including specific node selection, generation parameters, and metadata. Useful for advanced configurations.

```python
from chutes.chute import NodeSelector
from chutes.chute.template.diffusion import build_diffusion_chute

# Build diffusion chute for image generation
chute = build_diffusion_chute(
    username="myuser",
    model_name="stabilityai/stable-diffusion-xl-base-1.0",
    revision="main",
    node_selector=NodeSelector(
        gpu_count=1,
        min_vram_gb_per_gpu=16,
        include=["rtx4090", "a100"]
    ),
    tagline="High-quality image generation with SDXL",
    readme="""
# Image Generation Service

Generate stunning images from text prompts using Stable Diffusion XL.

## Features
- High-resolution image generation (up to 1024x1024)
- Multiple generation modes
- ControlNet support for guided generation
- Safety filtering for appropriate content

## API Endpoints
- `/generate` - Text-to-image generation
- `/img2img` - Image-to-image transformation
- `/inpaint` - Image inpainting
    """,

    # Optimize for SDXL
    scheduler="euler_a",
    guidance_scale=7.5,
    num_inference_steps=30,  # SDXL works well with fewer steps
    height=1024,
    width=1024,
    safety_checker=True
)
```

--------------------------------

### Basic ML Image Creation

Source: https://chutes.ai/docs/sdk-reference/image

Example of creating a basic ML image with PyTorch and Transformers installed, including requirements and source code.

```python
from chutes.image import Image

image = (
    Image(username="myuser", name="ml-app", tag="1.0")
    .from_base("parachutes/python:3.12")
    .run_command("pip install torch transformers accelerate")
    .add("requirements.txt", "/app/requirements.txt")
    .run_command("pip install -r /app/requirements.txt")
    .add("src/", "/app/src/")
    .set_workdir("/app")
    .with_env("PYTHONPATH", "/app")
)
```

--------------------------------

### Build Custom Docker Image

Source: https://chutes.ai/docs/examples/simple-text-analysis

Constructs a custom Docker image for sentiment analysis, starting from a CUDA-enabled base image and installing necessary Python packages.

```python
image = (
    Image(username="myuser", name="sentiment-analyzer", tag="1.0")
    .from_base("nvidia/cuda:12.4.1-runtime-ubuntu22.04")
    .with_python("3.11")
    .run_command("pip install torch>=2.4.0 transformers>=4.44.0")
)
```

--------------------------------

### Build Context File List

Source: https://chutes.ai/docs/cli/build

Example of the file list detected for inclusion in a remote build.

```bash
Found 15 files to include in build context -- these will be uploaded for remote builds!
 requirements.txt
 src/main.py
 src/utils.py
 ...
Confirm submitting build context? (y/n)
```

--------------------------------

### Configure Chute with AI dependencies and GPU support

Source: https://chutes.ai/docs/guides/custom-chutes

An advanced example showing how to install system-level dependencies, define Pydantic schemas, and utilize GPU resources for model inference.

```python
from chutes.chute import Chute, NodeSelector
from chutes.image import Image
from pydantic import BaseModel
from typing import List, Optional

# Define input/output schemas
class AnalysisInput(BaseModel):
    text: str
    options: Optional[List[str]] = []

class AnalysisOutput(BaseModel):
    result: str
    confidence: float
    metadata: dict

# Create custom image with AI dependencies
image = (
    Image(username="myuser", name="text-analyzer", tag="1.0")
    .from_base("nvidia/cuda:11.8-devel-ubuntu22.04")
    .run_command("apt update && apt install -y python3 python3-pip")
    .run_command("pip3 install torch transformers tokenizers")
    .run_command("pip3 install numpy pandas scikit-learn")
    .run_command("pip3 install fastapi uvicorn pydantic")
    .set_workdir("/app")
)

# Create chute with GPU support
chute = Chute(
    username="myuser",
    name="text-analyzer",
    image=image,
    node_selector=NodeSelector(
        gpu_count=1,
        min_vram_gb_per_gpu=8
    ),
    concurrency=4
)

@chute.on_startup()
async def initialize_models(self):
    """Load AI models during startup."""
    from transformers import pipeline
    import torch

    # Load sentiment analysis model
    self.sentiment_analyzer = pipeline(
        "sentiment-analysis",
        model="cardiffnlp/twitter-roberta-base-sentiment-latest",
        device=0 if torch.cuda.is_available() else -1
    )

    # Load text classification model
    self.classifier = pipeline(
        "zero-shot-classification",
        model="facebook/bart-large-mnli",
        device=0 if torch.cuda.is_available() else -1
    )

@chute.cord(
    public_api_path="/analyze",
    method="POST",
    input_schema=AnalysisInput,
    output_schema=AnalysisOutput
)
async def analyze_text(self, input_data: AnalysisInput) -> AnalysisOutput:
    """Analyze text with multiple AI models."""

    # Sentiment analysis
    sentiment_result = self.sentiment_analyzer(input_data.text)[0]

    # Classification (if options provided)
    classification_result = None
    if input_data.options:
        classification_result = self.classifier(
            input_data.text,
            input_data.options
        )

    # Combine results
    result = f"Sentiment: {sentiment_result['label']}"
    if classification_result:
        result += f", Category: {classification_result['labels'][0]}"

    return AnalysisOutput(
        result=result,
        confidence=sentiment_result['score'],
        metadata={
            "sentiment": sentiment_result,
            "classification": classification_result
        }
    )
```

--------------------------------

### Optimize Deployment Startup

Source: https://chutes.ai/docs/help/troubleshooting

Use background tasks to load models and prevent deployment timeouts.

```python
# Optimize startup time
@chute.on_startup()
async def setup(self):
    # Move heavy operations to background
    asyncio.create_task(self.load_model_async())

async def load_model_async(self):
    """Load model in background to avoid startup timeout."""
    self.model = load_large_model()
    self.ready = True

@chute.cord(public_api_path="/health")
async def health_check(self):
    """Health check endpoint."""
    return {"status": "ready" if hasattr(self, 'ready') else "loading"}
```

--------------------------------

### SGLang with Custom Image for Advanced Features

Source: https://chutes.ai/docs/examples/llm-chat

Deploy SGLang with a custom Docker image for advanced control and features. This example includes network optimizations and installs SGLang from source.

```python
import os
from chutes.chute import NodeSelector
from chutes.chute.template.sglang import build_sglang_chute
from chutes.image import Image

# Optimize networking for multi-GPU setups
os.environ["NO_PROXY"] = "localhost,127.0.0.1"
for key in ["NCCL_P2P_DISABLE", "NCCL_IB_DISABLE", "NCCL_NET_GDR_LEVEL"]:
    if key in os.environ:
        del os.environ[key]

# Build custom SGLang image with optimizations
image = (
    Image(
        username="myuser",
        name="sglang-optimized",
        tag="0.4.9.dev1",
        readme="SGLang with performance optimizations for large models")
    .from_base("parachutes/python:3.12")
    .run_command("pip install --upgrade pip")
    .run_command("pip install --upgrade 'sglang[all]'")
    .run_command(
        "git clone https://github.com/sgl-project/sglang sglang_src && "
        "cd sglang_src && pip install -e python[all]"
    )
    .run_command(
        "pip install torch torchvision torchaudio "
        "--index-url https://download.pytorch.org/whl/cu128 --upgrade"
    )
    .run_command("pip install datasets blobfile accelerate tiktoken")
    .run_command("pip install nvidia-nccl-cu12==2.27.6 --force-reinstall --no-deps")
    .with_env("SGL_ENABLE_JIT_DEEPGEMM", "1")
)

# Deploy Kimi K2 Instruct model
chute = build_sglang_chute(
    username="myuser",
    readme="Moonshot AI Kimi K2 Instruct - Advanced reasoning model",
    model_name="moonshotai/Kimi-K2-Instruct",
    image=image,
    concurrency=3,
    node_selector=NodeSelector(
        gpu_count=8,
        include=["h200"],  # Use latest H200 GPUs
    ),
    engine_args=(
        "--trust-remote-code "
        "--cuda-graph-max-bs 3 "
        "--mem-fraction-static 0.97 "
        "--context-length 65536 "
        "--revision d1e2b193ddeae7776463443e7a9aa3c3cdc51003 "
    ))

```

--------------------------------

### Setup Distributed DataLoader

Source: https://chutes.ai/docs/examples/custom-training

Creates a DataLoader with a DistributedSampler to ensure each process gets a unique subset of the data. Set shuffle to True for random data distribution per epoch.

```python
sampler = DistributedSampler(
            dataset,
            num_replicas=self.world_size,
            rank=self.rank,
            shuffle=True
        )

        dataloader = DataLoader(
            dataset,
            batch_size=batch_size,
            sampler=sampler,
            num_workers=4,
            pin_memory=True
        )
```

--------------------------------

### Create Project Directory and File

Source: https://chutes.ai/docs/getting-started/first-chute

Sets up the basic project structure by creating a directory and the main Python file for the chute.

```bash
mkdir my-first-chute
cd my-first-chute
```

```bash
touch sentiment_chute.py
```

--------------------------------

### Optimizing Layer Order Example

Source: https://chutes.ai/docs/cli/build

Demonstrates the best practice of ordering build steps to optimize Docker layer caching. Placing less frequently changing dependencies earlier speeds up subsequent builds.

```plaintext
1. System packages
2. Python packages (requirements.txt)
3. Application code
```

--------------------------------

### Define Public API Path

Source: https://chutes.ai/docs/sdk-reference/cord

Examples of how to define the public API path for a cord endpoint. Paths must start with '/' and can include parameterized segments like {parameter_name}.

```python
# Simple path
@chute.cord(public_api_path="/generate")
```

```python
# Path with parameter
@chute.cord(public_api_path="/users/{user_id}")
```

```python
# Nested resource
@chute.cord(public_api_path="/models/{model_id}/generate")
```

--------------------------------

### Monitoring and Performance Tuning

Source: https://chutes.ai/docs/core-concepts/node-selection

Examples for right-sizing resources and tuning for CPU or GPU intensive tasks.

```python
# Over-provisioned (waste of money)
over_provisioned = NodeSelector(
    gpu_count=4,  # Using only 1
    min_vram_gb_per_gpu=80  # Using only 20GB
)

# Right-sized (cost-effective)
right_sized = NodeSelector(
    gpu_count=1,
    min_vram_gb_per_gpu=24
)
```

```python
# CPU-bound preprocessing
cpu_intensive = NodeSelector(
    gpu_count=1,
    min_vram_gb_per_gpu=16,
    min_cpu_count=16,  # Extra CPU for preprocessing
    min_memory_gb=64
)

# GPU-bound inference
gpu_intensive = NodeSelector(
    gpu_count=2,  # More GPU power
    min_vram_gb_per_gpu=40,
    min_cpu_count=8   # Less CPU needed
)
```

--------------------------------

### Set Public API Method

Source: https://chutes.ai/docs/sdk-reference/cord

Examples demonstrating how to specify the HTTP method for a cord endpoint. Defaults to POST, but GET, PUT, and DELETE are also supported for common CRUD operations.

```python
# GET for data retrieval
@chute.cord(public_api_path="/models", public_api_method="GET")
async def list_models(self):
    return {"models": ["gpt-3.5", "gpt-4"]}
```

```python
# POST for data processing (default)
@chute.cord(public_api_path="/generate", public_api_method="POST")
async def generate_text(self, prompt: str):
    return await self.model.generate(prompt)
```

```python
# DELETE for removal
@chute.cord(public_api_path="/cache", public_api_method="DELETE")
async def clear_cache(self):
    self.cache.clear()
    return {"status": "cache cleared"}
```

--------------------------------

### Add Custom Startup Logic

Source: https://chutes.ai/docs/sdk-reference/templates

Implement custom initialization logic that runs after the model has loaded. This example prints a confirmation message.

```python
@chute.on_startup(priority=90)  # Run after template initialization
async def custom_setup(self):
    """Custom initialization after model loads."""
    print("Custom setup complete!")
```

--------------------------------

### GitHub Actions Workflow for Chutes Deployment

Source: https://chutes.ai/docs/cli/deploy

This GitHub Actions workflow automates the build and deployment of a Chutes application. It includes setup for Python, installation of Chutes, and configuration using secrets.

```yaml
name: Deploy to Chutes
on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Setup Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.11'

      - name: Install Chutes
        run: pip install chutes

      - name: Configure Chutes
        env:
          CHUTES_CONFIG: ${{ secrets.CHUTES_CONFIG }}
        run: |
          mkdir -p ~/.chutes
          echo "$CHUTES_CONFIG" > ~/.chutes/config.ini

      - name: Build and Deploy
        run: |
          chutes build my_app:chute --wait
          chutes deploy my_app:chute --accept-fee
```

--------------------------------

### Buffered Streaming Endpoint Implementation

Source: https://chutes.ai/docs/guides/streaming

A GET endpoint that streams data with buffering and backpressure. It starts the data producer if not already running and subscribes consumers to the stream. Handles timeouts and errors during streaming.

```python
@chute.cord(
    public_api_path="/buffered_stream",
    method="GET",
    stream=True
)
async def buffered_streaming_endpoint(self):
    """Stream with buffering and backpressure handling."""

    # Start producing if not already started
    if not self.streaming_buffer.is_producing:
        await self.streaming_buffer.start_producing(self._data_producer)

    # Subscribe to stream
    consumer_queue = await self.streaming_buffer.subscribe()

    async def stream_response():
        try:
            while True:
                # Get data from buffer
                data = await asyncio.wait_for(consumer_queue.get(), timeout=30.0)
                yield f"data: {json.dumps(data)}\n\n"

        except asyncio.TimeoutError:
            yield "event: timeout\ndata: {}\n\n"
        except Exception as e:
            yield f"event: error\ndata: {json.dumps({'error': str(e)})}\n\n"
        finally:
            self.streaming_buffer.unsubscribe(consumer_queue)

    return StreamingResponse(
        stream_response(),
        media_type="text/event-stream"
    )
```

--------------------------------

### Initialize Storage and Cache

Source: https://chutes.ai/docs/guides/custom-chutes

Sets up memory and Redis storage during the Chute startup phase.

```python
@chute.on_startup()
async def initialize_storage(self):
    """Initialize storage systems."""
    import aioredis

    # In-memory cache
    self.memory_cache = {}
    self.cache_ttl = {}

    # Try to connect to Redis (optional)
    try:
        self.redis = await aioredis.create_redis_pool('redis://localhost')
        self.has_redis = True
    except:
        self.redis = None
        self.has_redis = False
        print("Redis not available, using memory cache only")

    # Session storage
    self.sessions = {}

    # Request tracking
    self.request_count = 0
    self.last_requests = []
```

--------------------------------

### Image Constructor: Readme Example

Source: https://chutes.ai/docs/sdk-reference/image

Include documentation for the image in Markdown format using the optional readme parameter. This provides details about the image's features and purpose.

```python
readme = """
# Custom AI Processing Image

This image contains optimized libraries for AI text processing.

## Features
- PyTorch 2.0 with CUDA support
- Transformers library
- Optimized for GPU inference
"""

image = Image(
    username="mycompany",
    name="ai-processor",
    tag="1.0.0",
    readme=readme
)
```

--------------------------------

### Define Custom Docker Image for Sentiment Analysis

Source: https://chutes.ai/docs/getting-started/first-chute

Configures a custom Docker image starting from a CUDA-enabled base, installing Python, system dependencies, PyTorch with CUDA, and ML libraries like Transformers.

```python
# Add to sentiment_chute.py
from chutes.image import Image

# Create optimized image for sentiment analysis
image = (
    Image(username="myuser", name="sentiment-chute", tag="1.0")

    # Start with CUDA-enabled Ubuntu
    .from_base("nvidia/cuda:12.2-runtime-ubuntu22.04")

    # Install Python 3.11
    .with_python("3.11")

    # Install system dependencies
    .run_command("""
        apt-get update && apt-get install -y \
        git curl wget \
        && rm -rf /var/lib/apt/lists/*
    """
    )

    # Install PyTorch with CUDA support
    .run_command("""
        pip install torch torchvision torchaudio \
        --index-url https://download.pytorch.org/whl/cu121
    """
    )

    # Install transformers and other ML dependencies
    .run_command("""
        pip install \
        transformers>=4.30.0 \
        accelerate>=0.20.0 \
        tokenizers>=0.13.0 \
        numpy>=1.24.0 \
        scikit-learn>=1.3.0
    """
    )

    # Set up model cache directory
    .with_env("TRANSFORMERS_CACHE", "/app/models")
    .with_env("HF_HOME", "/app/models")
    .run_command("mkdir -p /app/models")

    # Set working directory
    .set_workdir("/app")
)
```

--------------------------------

### Build Image with System Dependencies for Audio Processing

Source: https://chutes.ai/docs/sdk-reference/image

Installs essential system libraries for audio processing, such as ffmpeg and libsndfile, followed by Python audio libraries. This setup is suitable for applications requiring audio manipulation.

```python
image = (
    Image(username="myuser", name="audio-processor", tag="1.0")
    .from_base("parachutes/python:3.12")

    # Audio processing dependencies
    .apt_install([
        "ffmpeg",
        "libsndfile1",
        "libportaudio2",
        "libsox-fmt-all"
    ])

    # Python audio libraries
    .run_command("""
        pip install soundfile librosa pydub torchaudio
    """)

    .add("src/", "/app/src/")
    .set_workdir("/app")
)
```

--------------------------------

### Initialize DiffRhythm Models

Source: https://chutes.ai/docs/examples/music-generation

Configures and loads the DiffRhythm models, VAE, and tokenizer on startup. Requires GPU acceleration and specific model checkpoints from Hugging Face.

```python
@chute.on_startup()
async def initialize(self):
    """
    Initialize DiffRhythm models and dependencies.
    """
    from huggingface_hub import snapshot_download
    import torchaudio
    import torch
    import soundfile
    from infer_utils import (
        decode_audio,
        get_lrc_token,
        get_negative_style_prompt,
        get_reference_latent,
        get_style_prompt,
        load_checkpoint,
        CNENTokenizer)
    from infer import inference
    from muq import MuQMuLan
    from model import DiT, CFM
    import json
    import os

    # Download required models
    revision = "613846abae8e5b869b3845a5dfabc9ecc37ecdab"
    repo_id = "ASLP-lab/DiffRhythm-full"
    path = snapshot_download(repo_id, revision=revision)

    vae_path = snapshot_download(
        "ASLP-lab/DiffRhythm-vae",
        revision="4656f626776f5f924c03471acb25bea6734e774f"
    )

    # Load model configuration
    dit_config_path = "/app/config/diffrhythm-1b.json"
    with open(dit_config_path) as f:
        model_config = json.load(f)

    # Initialize models
    dit_model_cls = DiT
    self.max_frames = 6144

    # CFM (Conditional Flow Matching) model
    self.cfm = CFM(
        transformer=dit_model_cls(**model_config["model"], max_frames=self.max_frames),
        num_channels=model_config["model"]["mel_dim"],
        max_frames=self.max_frames
    ).to("cuda")

    # Load trained weights
    self.cfm = load_checkpoint(
        self.cfm,
        os.path.join(path, "cfm_model.pt"),
        device="cuda",
        use_ema=False
    )

    # Initialize tokenizer and style model
    self.tokenizer = CNENTokenizer()
    self.muq = MuQMuLan.from_pretrained(
        "OpenMuQ/MuQ-MuLan-large",
        revision="8a081dbcf84edd47ea7db3c4ecb8fd1ec1ddacfe"
    ).to("cuda")

    # Load VAE for audio decoding
    vae_ckpt_path = os.path.join(vae_path, "vae_model.pt")
    self.vae = torch.jit.load(vae_ckpt_path, map_location="cpu").to("cuda")

    # Warmup with example generation
    await self._warmup_model()

    # Store utilities
    self.torchaudio = torchaudio
    self.torch = torch
    self.soundfile = soundfile
    self.decode_audio = decode_audio
    self.inference = inference
    self.get_lrc_token = get_lrc_token
    self.get_reference_latent = get_reference_latent
    self.get_style_prompt = get_style_prompt
```

--------------------------------

### MLflow Experiment Tracking Class

Source: https://chutes.ai/docs/examples/custom-training

A Python class for tracking ML experiments using MLflow. It handles starting and ending runs, logging parameters, metrics, models, and artifacts. Ensure MLflow is installed and configured.

```python
import mlflow
import mlflow.pytorch
from tensorboard.compat.tensorflow_stub.io.gfile import register_filesystem

class ExperimentTracker:
    def __init__(self, experiment_name: str):
        mlflow.set_experiment(experiment_name)
        self.run = mlflow.start_run()

    def log_params(self, params: Dict[str, Any]):
        """Log hyperparameters"""
        for key, value in params.items():
            mlflow.log_param(key, value)

    def log_metrics(self, metrics: Dict[str, float], step: int = None):
        """Log metrics"""
        for key, value in metrics.items():
            mlflow.log_metric(key, value, step=step)

    def log_model(self, model, model_name: str):
        """Log trained model"""
        mlflow.pytorch.log_model(model, model_name)

    def log_artifacts(self, local_path: str):
        """Log training artifacts"""
        mlflow.log_artifacts(local_path)

    def finish(self):
        """End experiment run"""
        mlflow.end_run()

```

--------------------------------

### TEI Template - Quick Start

Source: https://chutes.ai/docs/templates/tei

Demonstrates how to quickly set up a TEI deployment for optimized embedding inference.

```APIDOC
## Quick Start

This section shows how to build a TEI chute for embedding generation.

### Code Example

```python
from chutes.chute import NodeSelector
from chutes.chute.template.tei import build_tei_chute

chute = build_tei_chute(
    username="myuser",
    model_name="sentence-transformers/all-MiniLM-L6-v2",
    revision="main",
    node_selector=NodeSelector(
        gpu_count=1,
        min_vram_gb_per_gpu=8
    )
)
```

This setup includes an optimized embedding inference server with an OpenAI-compatible API, automatic batching, normalization, and auto-scaling.
```

--------------------------------

### Setup Basic Fine-tuning Environment

Source: https://chutes.ai/docs/examples/custom-training

Defines a Pydantic model for training configuration and creates a Docker image for custom model training with specified ML frameworks and dependencies.

```python
from chutes.image import Image
from chutes.chute import Chute, NodeSelector
from pydantic import BaseModel
from typing import List, Dict, Any, Optional

class TrainingConfig(BaseModel):
    model_name: str
    dataset_path: str
    num_epochs: int = 3
    batch_size: int = 16
    learning_rate: float = 2e-5
    output_dir: str = "/models/output"
    save_steps: int = 500
    eval_steps: int = 100

# Training image with ML frameworks
training_image =
    Image(
        username="myuser",
        name="custom-training",
        tag="1.0.0",
        base_image="nvidia/cuda:12.1-devel-ubuntu22.04",
        python_version="3.11"
    )
    .run_command("pip install torch==2.1.0+cu121 transformers==4.35.0 datasets==2.14.0 accelerate==0.24.0 wandb==0.16.0 tensorboard==2.15.0 --extra-index-url https://download.pytorch.org/whl/cu121")
    .add("./training", "/app/training")
    .add("./data", "/app/data")
```

--------------------------------

### RAG Orchestration Client Example

Source: https://chutes.ai/docs/guides/rag-pipeline

Python client code demonstrating how to integrate embedding, vector search, and LLM generation for a RAG pipeline. It defines functions for getting embeddings, searching the knowledge base, and generating answers.

```python
import requests
import openai

# Configuration
EMBEDDING_URL = "https://myuser-bge-large.chutes.ai/v1/embeddings"
CHROMA_URL = "https://myuser-rag-vector-db.chutes.ai/query"
LLM_BASE_URL = "https://myuser-deepseek-r1.chutes.ai/v1"
API_KEY = "your-api-key"

def get_embedding(text):
    """Get embedding vector for text."""
    resp = requests.post(
        EMBEDDING_URL,
        headers={"Authorization": API_KEY},
        json={"input": text, "model": "BAAI/bge-large-en-v1.5"}
    )
    return resp.json()["data"][0]["embedding"]

def search_knowledge_base(embedding):
    """Search vector DB."""
    resp = requests.post(
        CHROMA_URL,
        headers={"Authorization": API_KEY},
        json={"query_embeddings": [embedding], "n_results": 3}
    )
    # Format results into a context string
    results = resp.json()
    return "\n".join(results["documents"][0])

def generate_answer(query, context):
    """Generate answer using LLM."""
    client = openai.OpenAI(base_url=LLM_BASE_URL, api_key=API_KEY)
    
    prompt = f"""
    Use the following context to answer the question. 
    
    Context:
    {context}
    
    Question: {query}
    """
    
    resp = client.chat.completions.create(
        model="deepseek-ai/DeepSeek-R1",
        messages=[{"role": "user", "content": prompt}],
        temperature=0.1
    )
    return resp.choices[0].message.content

# Main Flow
user_query = "What is Chutes?"
print(f"Querying: {user_query}...")

```

--------------------------------

### Build Custom Docker Image with Video Capabilities

Source: https://chutes.ai/docs/examples/video-generation

Constructs a custom Docker image using ChutesImage, starting from a base image and adding necessary tools like ffmpeg, cloning the Wan2.1 repository, installing dependencies, and applying performance patches.

```python
image = (
    ChutesImage(
        username="myuser",
        name="wan21",
        tag="0.0.1",
        readme="## Video and image generation/editing model from Alibaba")
    .from_base("parachutes/base-python:3.12.7")
    .set_user("root")
    .run_command("apt update")
    .apt_install("ffmpeg")  # Required for video processing
    .set_user("chutes")
    .run_command(
        "git clone https://github.com/Wan-Video/Wan2.1 && "
        "cd Wan2.1 && "
        "pip install --upgrade pip && "
        "pip install setuptools wheel && "
        "pip install torch torchvision && "
        "pip install -r requirements.txt --no-build-isolation && "
        "pip install xfuser && "
        # Apply critical patches for performance
        "perl -pi -e 's/sharding_strategy=sharding_strategy,/sharding_strategy=sharding_strategy,\n        use_orig_params=True,/g' wan/distributed/fsdp.py && "
        "perl -pi -e 's/dtype=torch.float32,/dtype=torch.bfloat16,/g' wan/modules/t5.py && "
        "mv -f /app/Wan2.1/wan /home/chutes/.local/lib/python3.12/site-packages/"
    )
)
```

--------------------------------

### Download Models at Startup

Source: https://chutes.ai/docs/help/troubleshooting

Logic to check for model existence and download it from Hugging Face if missing.

```python
from huggingface_hub import snapshot_download
import os

@chute.on_startup()
async def setup(self):
    """Download model if not present."""
    model_path = "/models/my-model"

    if not os.path.exists(model_path):
        # Download model during startup
        snapshot_download(
            repo_id="microsoft/DialoGPT-medium",
            local_dir=model_path,
            token=os.getenv("HF_TOKEN")  # If private model
        )

    self.model = load_model(model_path)
```

--------------------------------

### Hardware Scaling Options

Source: https://chutes.ai/docs/examples/image-generation

Python examples demonstrating how to configure hardware scaling for Chutes services, including multi-GPU selection and scaling out with multiple instances.

```python
# Scale up for higher throughput
node_selector = NodeSelector(
    gpu_count=2,  # Multi-GPU setup
    min_vram_gb_per_gpu=40)

# Or scale out with multiple instances
chute = Chute(
    # ... configuration
    concurrency=4,  # Handle more concurrent requests
)
```

--------------------------------

### Install and Verify Chutes CLI

Source: https://chutes.ai/docs/cli/overview

Install the CLI via pip and verify the installation by accessing the help menu.

```bash
pip install chutes
```

```bash
chutes --help
```

--------------------------------

### Using Pre-built Images

Source: https://chutes.ai/docs/core-concepts/images

Examples of how to reference pre-built Docker images provided by Chutes AI or from public repositories like NVIDIA.

```APIDOC
## Using Pre-built Images

### Popular Base Images

```python
# NVIDIA CUDA images
"nvidia/cuda:12.2-devel-ubuntu22.04"
"nvidia/cuda:11.8-runtime-ubuntu20.04"

# Chutes optimized images
"chutes/cuda-python:12.2-py311"
"chutes/pytorch:2.1-cuda12.2"
"chutes/tensorflow:2.13-cuda11.8"

# Specialized AI framework images
"pytorch/pytorch:2.1.0-cuda12.1-cudnn8-devel"
"tensorflow/tensorflow:2.13.0-gpu"
```

### Using String References

```python
from chutes.chute import Chute

chute = Chute(
    username="myuser",
    name="my-chute",
    image="nvidia/cuda:12.2-devel-ubuntu22.04"  # Simple string reference
)
```
```

--------------------------------

### Install System Packages with Apt

Source: https://chutes.ai/docs/sdk-reference/image

Install system packages using the apt package manager. Can install single or multiple packages.

```python
# Install single package
image = image.apt_install("git")
```

```python
# Install multiple packages
image = image.apt_install(["git", "curl", "wget", "ffmpeg"])
```

--------------------------------

### Configure Guided Generation

Source: https://chutes.ai/docs/templates/vllm

Enable guided generation using a specified backend like 'outlines' and provide a JSON schema for guided output.

```python
engine_args = {
    "guided_decoding_backend": "outlines",
}

# Then in your requests:
{
    "guided_json": {"type": "object", "properties": {"name": {"type": "string"}}}
}
```

--------------------------------

### Run the Deployment Playbook

Source: https://chutes.ai/docs/miner-resources/ansible

Executes the main playbook to handle wireguard, k3s, and node joining.

```bash
ansible-playbook -i inventory.yml site.yml
```

--------------------------------

### GET /invocations/stats/diffusion

Source: https://chutes.ai/docs/api-reference/invocations

Get diffusion statistics for invocations.

```APIDOC
## GET /invocations/stats/diffusion

### Description
Get diffusion statistics for invocations.

### Method
GET

### Endpoint
/invocations/stats/diffusion

### Responses
#### Success Response (200)
- **Description**: Successful Response
```