Runpod

https://docs.runpod.io/
Admin
Runpod is a cloud computing platform providing scalable GPU and CPU resources for AI, machine...
Tokens:389,392
Snippets:3,971
Trust Score:10
Update:1 month ago
Show doc for...
Tokens:
Raw
### Project setup workflow

Source: https://docs.runpod.io/flash/cli/init

Standard sequence of commands to initialize, enter, install dependencies, and start a new Flash project.

```bash
flash init PROJECT_NAME
cd PROJECT_NAME
pip install -r requirements.txt
flash dev
```

--------------------------------

### Example output for first run

Source: https://docs.runpod.io/tutorials/flash/text-generation-with-transformers

Expected console output during the initial execution, including provisioning and dependency installation.

```text
Starting text generation on Runpod GPU...
Creating endpoint: server_Endpoint_a1b2c3d4
Provisioning Serverless endpoint...
Endpoint ready
Registering Runpod endpoint at https://api.runpod.ai/xvf32dan8rcilp
Executing function on Runpod endpoint ID: xvf32dan8rcilp
Initial job status: IN_QUEUE
Installing dependencies: transformers torch accelerate
Downloading model weights...
Job completed, output received

============================================================
TEXT GENERATION RESULTS
============================================================

Prompt: The future of artificial intelligence is

Generated text:
The future of artificial intelligence is bright and full of possibilities. With advancements in machine learning and deep learning, we're seeing AI systems that can understand natural language, recognize images, and even create art. The potential applications are endless, from healthcare to transportation to education.

------------------------------------------------------------
Model: gpt2
Device: cuda
GPU: NVIDIA GeForce RTX 4090
Max length: 100 tokens
============================================================
```

--------------------------------

### Configure container start commands

Source: https://docs.runpod.io/pods/templates/manage-templates

Examples of how to specify custom start commands for a container, either as a bash string or a JSON object.

```bash
bash -c 'mkdir /workspace && /start.sh'
```

```json
{"cmd": ["python", "app.py"], "entrypoint": ["bash", "-c"]}
```

--------------------------------

### Install and start Ollama

Source: https://docs.runpod.io/tutorials/pods/run-ollama

Downloads the Ollama installation script and starts the server in the background, logging output to a file.

```bash
(curl -fsSL https://ollama.com/install.sh | sh && ollama serve > ollama.log 2>&1) &
```

--------------------------------

### Local Development Workflow

Source: https://docs.runpod.io/flash/cli/overview

Initialize a project, install dependencies, and start the development server.

```bash
# Create a new project
flash init PROJECT_NAME
cd PROJECT_NAME

# Install dependencies
pip install -r requirements.txt

# Add your API key to .env
# Start the development server
flash dev
```

--------------------------------

### Flash app get output example

Source: https://docs.runpod.io/flash/cli/app

Example output showing the structure of app details, including environment and build tables.

```text
╭─────────────────────────────────╮
│ Flash App: my-project           │
├─────────────────────────────────┤
│ Name: my-project                │
│ ID: app_abc123                  │
│ Environments: 3                 │
│ Builds: 5                       │
╰─────────────────────────────────╯

              Environments
┏━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓
┃ Name       ┃ ID                 ┃ State   ┃ Active Build     ┃ Created          ┃
┡━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩
│ dev        │ env_dev123         │ DEPLOYED│ build_xyz789     │ 2024-01-15 10:30 │
│ staging    │ env_stg456         │ DEPLOYED│ build_xyz789     │ 2024-01-16 14:20 │
│ production │ env_prd789         │ DEPLOYED│ build_abc123     │ 2024-01-20 09:15 │
└────────────┴────────────────────┴─────────┴──────────────────┴──────────────────┘

                     Builds
┏━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓
┃ ID                 ┃ Status                   ┃ Created          ┃
┡━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩
│ build_abc123       │ COMPLETED                │ 2024-01-20 09:00 │
│ build_xyz789       │ COMPLETED                │ 2024-01-18 15:45 │
│ build_def456       │ COMPLETED                │ 2024-01-15 11:20 │
└────────────┴──────────────────────────┴──────────────────┘
```

--------------------------------

### Install Serverless SDK

Source: https://docs.runpod.io/serverless/quickstart

Sets up a virtual environment and installs the runpod package.

```bash
python3 -m venv .venv
source .venv/bin/activate
pip install runpod
```

--------------------------------

### Runpod documentation query examples

Source: https://docs.runpod.io/get-started/mcp-servers

Example prompts to use once the MCP server is connected.

```text
Explain the Runpod Serverless model caching feature
```

```text
How do I configure environment variables for a Serverless endpoint?
```

```text
How does global networking work in Runpod?
```

--------------------------------

### SSH and SCP connection examples

Source: https://docs.runpod.io/pods/configuration/use-ssh

Example commands for connecting to a pod via SSH and transferring files using SCP after password setup.

```bash
========================================
SSH CONNECTION
========================================
Connect using: ssh root@38.80.152.73 -p 32061
Password: helloworld

========================================
FILE TRANSFER EXAMPLES (SCP)
========================================
Copy file TO pod:
scp -P 32061 yourfile.txt root@38.80.152.73:/workspace/

Copy file FROM pod:
scp -P 32061 root@38.80.152.73:/workspace/yourfile.txt .

Copy entire folder TO pod:
scp -P 32061 -r yourfolder root@38.80.152.73:/workspace/
```

--------------------------------

### Ubuntu Terminal Prompt

Source: https://docs.runpod.io/flash/windows-wsl2

Example of the standard Ubuntu terminal prompt after successful setup.

```text
username@hostname:~$
```

--------------------------------

### Install Runpod Flash

Source: https://docs.runpod.io/flash/overview

Install the SDK using pip or uv.

```bash
# Install with pip
pip install runpod-flash

# Or uv
uv tool install runpod-flash
```

--------------------------------

### Install the storage tool

Source: https://docs.runpod.io/community-solutions/runpod-network-volume-storage-tool

Clone the repository and install dependencies using uv.

```bash
git clone https://github.com/justinwlin/Runpod-Network-Volume-Storage-Tool.git
cd Runpod-Network-Volume-Storage-Tool

# Install dependencies with uv
uv sync
```

--------------------------------

### Install Docker

Source: https://docs.runpod.io/tutorials/pods/build-docker-images

Download and execute the official Docker installation script.

```bash
curl -fsSL https://get.docker.com -o get-docker.sh && sudo sh get-docker.sh
```

--------------------------------

### Install Python SDK

Source: https://docs.runpod.io/serverless/sdks

Commands to set up a virtual environment and install the RunPod Python SDK.

```bash
python3 -m venv venv
source venv/bin/activate
pip install runpod
```

```bash
python -m venv venv
venv\Scripts\activate
pip install runpod
```

--------------------------------

### Install dependencies with uv

Source: https://docs.runpod.io/tutorials/flash/text-generation-with-transformers

Create a virtual environment and install the required packages using uv.

```bash
uv venv
source .venv/bin/activate
uv pip install runpod-flash python-dotenv
```

--------------------------------

### Install via script

Source: https://docs.runpod.io/runpodctl/overview

Use automated scripts to detect OS and architecture for installation.

```bash
bash <(wget -qO- cli.runpod.io)
```

```bash
bash <(curl -sL cli.runpod.io)
```

--------------------------------

### Install via package managers

Source: https://docs.runpod.io/runpodctl/overview

Installation using conda, mamba, or pixi.

```bash
conda install conda-forge::runpodctl
```

```bash
mamba install conda-forge::runpodctl
```

```bash
pixi global install runpodctl
```

--------------------------------

### Install RunPod SDK

Source: https://docs.runpod.io/serverless/endpoints/operation-reference

Commands to install the RunPod SDK for Python, JavaScript, and Go environments.

```bash
# Python
python -m pip install runpod

# JavaScript
npm install --save runpod-sdk

# Go
go get github.com/runpod/go-sdk && go mod tidy
```

--------------------------------

### Basic Handler Implementation

Source: https://docs.runpod.io/serverless/workers/handler-functions

A standard handler function structure using the RunPod SDK to process input and start the serverless application.

```python
import runpod

def handler(job):
    job_input = job["input"]  # Access the input from the request

    # Add your custom code here to process the input

    return "Your job results"

runpod.serverless.start({"handler": handler})  # Required
```

--------------------------------

### Initialize Project and Install Flash

Source: https://docs.runpod.io/flash/windows-wsl2

Creates a project directory, sets up a virtual environment, and installs the Flash package.

```bash
mkdir my-flash-project && cd my-flash-project
uv venv
source .venv/bin/activate
```

```bash
uv pip install runpod-flash
```

--------------------------------

### Local Testing Input File

Source: https://docs.runpod.io/serverless/workers/handler-functions

Example of a JSON file used to provide test input data for local handler execution.

```json
{
    "input": {
        "prompt": "Hey there!"
    }
}
```

--------------------------------

### Verify runpodctl installation

Source: https://docs.runpod.io/runpodctl/overview

Check the installed version of the CLI to confirm successful setup.

```bash
runpodctl version
```

--------------------------------

### Start development and deploy

Source: https://docs.runpod.io/flash/apps/initialize-project

Run the development server for local testing or deploy the application to production.

```bash
# Start the development server
flash dev

# Open the API explorer
# http://localhost:8888/docs

# If using uv:
uv run flash dev
```

```bash
flash deploy

# If using uv:
uv run flash deploy
```

--------------------------------

### Verify Flash Installation

Source: https://docs.runpod.io/flash/windows-wsl2

Checks the installed version of Flash to confirm successful setup.

```bash
flash --version
```

--------------------------------

### Start server with auto-provisioning

Source: https://docs.runpod.io/flash/apps/build-app

Deploy all resources upfront to reduce cold-start delays during development.

```bash
uv run flash dev --auto-provision
```

--------------------------------

### Start development server with defaults

Source: https://docs.runpod.io/flash/cli/dev

Launches the server using default configuration settings.

```bash
flash dev
```

--------------------------------

### Navigate to project directory

Source: https://docs.runpod.io/integrations/dstack

Change the current working directory to the tutorial folder.

```bash
cd runpod-dstack-tutorial
```

--------------------------------

### Install MCP server for general agents

Source: https://docs.runpod.io/agent-setup

Uses the guided installer to detect and configure the hosted MCP server for supported agents.

```bash
npx @runpod/mcp-server@latest add
```

--------------------------------

### Initialize Project Directory

Source: https://docs.runpod.io/tutorials/introduction/containers/persist-data

Commands to set up a new directory for the volume persistence example.

```bash
mkdir volume-example
cd volume-example
```

--------------------------------

### Install and run CopyParty in the background with tmux

Source: https://docs.runpod.io/community-solutions/copyparty-file-manager/overview

Installs tmux and starts CopyParty in a persistent background session. This allows the service to continue running after disconnecting from the terminal.

```bash
apt-get update && apt-get install tmux -y && tmux new-session -d -s copyparty 'curl -LsSf https://astral.sh/uv/install.sh | sh && source $HOME/.local/bin/env && uv tool run copyparty -p 8000 --allow-csrf' && tmux attach -t copyparty
```

--------------------------------

### Set up project dependencies and environment

Source: https://docs.runpod.io/flash/apps/initialize-project

Install required dependencies and configure the environment variables after project initialization.

```bash
# Install dependencies
pip install -r requirements.txt

# Copy environment template
cp .env.example .env

# Add your API key to .env
# RUNPOD_API_KEY=your_api_key_here
```

--------------------------------

### Initialize project directory and install Flash

Source: https://docs.runpod.io/flash/apps/build-app

Creates a new directory and sets up a virtual environment with the runpod-flash package.

```bash
# Create the project directory and navigate into it:
mkdir flash_app
cd flash_app

# Install Flash:
uv venv
source .venv/bin/activate
uv pip install runpod-flash
```

--------------------------------

### vLLM API Server Configuration

Source: https://docs.runpod.io/instant-clusters/ray-vllm

Example configuration for starting the vLLM OpenAI-compatible API server with tensor and pipeline parallelism.

```bash
# serve.sh (excerpt — see full script in the repo)
python -m vllm.entrypoints.openai.api_server \
  --model meta-llama/Meta-Llama-3-70B \
  --tensor-parallel-size $NUM_TRAINERS \
  --pipeline-parallel-size $NUM_NODES \
  --host 0.0.0.0 \
  --port 8000
```

--------------------------------

### Call load-balanced routes via cURL

Source: https://docs.runpod.io/flash/apps/requests

Examples of making POST and GET requests to specific paths on a load-balanced endpoint.

```bash
# POST route
curl -X POST https://abc123xyz.api.runpod.ai/analyze \
    -H "Authorization: Bearer $RUNPOD_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{"text": "Hello world from Flash"}'

# GET route
curl -X GET https://abc123xyz.api.runpod.ai/info \
    -H "Authorization: Bearer $RUNPOD_API_KEY"

# Another POST route (same endpoint URL)
curl -X POST https://abc123xyz.api.runpod.ai/validate \
    -H "Authorization: Bearer $RUNPOD_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{"name": "Alice", "email": "alice@example.com"}'
```

--------------------------------

### Install Go SDK

Source: https://docs.runpod.io/serverless/sdks

Commands to fetch the Go SDK and tidy the module dependencies.

```bash
go get github.com/runpod/go-sdk
go mod tidy
```

--------------------------------

### OpenAPI definition for serverless billing

Source: https://docs.runpod.io/api-reference-v2/billing/get-serverless-billing-history

The OpenAPI specification for the GET /v2/billing/serverless endpoint, including parameters and a successful response example.

```yaml
openapi: 3.1.0
info:
  title: Runpod REST API
  version: 2.0.0
  description: Runpod public REST API — v2
servers:
  - url: https://api.runpod.io
    description: Runpod API v2 production server
security:
  - bearerAuth: []
tags:
  - name: Account
    description: Account-scoped settings and primitives (SSH public keys).
  - name: Pods
    description: GPU and CPU pod lifecycle, configuration, actions, and log streaming.
  - name: Serverless
    description: >-
      Serverless endpoint lifecycle, worker visibility, releases, and worker log
      streaming.
  - name: Templates
    description: Reusable pod and endpoint configuration templates.
  - name: Network Volumes
    description: Persistent network storage volumes for workloads.
  - name: Registries
    description: Container registry credentials used to pull private images.
  - name: Catalog
    description: Available GPU, CPU, data center, and public template catalog metadata.
  - name: Billing
    description: Billing history and usage cost records across resource types.
paths:
  /v2/billing/serverless:
    get:
      tags:
        - Billing
      summary: Get serverless billing history
      description: >
        Returns serverless endpoint billing detail for the authenticated user,
        split into time buckets by startTime/endTime with bucketSize or by lastN
        recent buckets. Use serverlessId to filter to one endpoint; without it,
        records are emitted per serverless endpoint per bucket. Each record
        reports endpoint-level GPU, CPU, disk, platform fee, and total amounts.
        This is distinct from pod billing, which covers standalone GPU and CPU
        pod costs rather than serverless endpoint workloads.
      operationId: listServerlessBilling
      parameters:
        - $ref: '#/components/parameters/BillingStartTime'
        - $ref: '#/components/parameters/BillingEndTime'
        - $ref: '#/components/parameters/BillingBucketSize'
        - $ref: '#/components/parameters/BillingLastN'
        - name: serverlessId
          in: query
          required: false
          description: Filter to a specific serverless endpoint.
          schema:
            type: string
          example: jpnw0v75y3qoql
      responses:
        '200':
          headers:
            RateLimit:
              $ref: '#/components/headers/RateLimit'
            RateLimit-Policy:
              $ref: '#/components/headers/RateLimit-Policy'
          description: OK
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ListServerlessBillingResponse'
              examples:
                serverlessBilling:
                  summary: Successful response
                  value:
                    records:
                      - startTime: '2026-06-01T00:00:00Z'
                        endTime: '2026-06-02T00:00:00Z'
                        serverlessId: 4m7x2k9q
                        totalAmount: 8.9
                        gpuAmount: 7.5
                        cpuAmount: 0
                        diskAmount: 0.4
                        feeAmount: 1
                    metadata:
                      query:
                        startTime: '2026-06-01T00:00:00Z'
                        endTime: '2026-06-02T00:00:00Z'
                        bucketSize: day
                        serverlessId: 4m7x2k9q
                      recordCount: 1
                      totals:
                        totalAmount: 8.9
                        gpuAmount: 7.5
                        cpuAmount: 0
                        diskAmount: 0.4
                        feeAmount: 1
                      uniqueServerlessCount: 1
        '401':
          $ref: '#/components/responses/UnauthorizedError'
        '403':
          $ref: '#/components/responses/ForbiddenError'
        '429':
          $ref: '#/components/responses/TooManyRequestsError'
        default:
          headers:
            RateLimit:
              $ref: '#/components/headers/RateLimit'
            RateLimit-Policy:
              $ref: '#/components/headers/RateLimit-Policy'
          description: Error
          content:
            application/problem+json:
              schema:
```

--------------------------------

### Verify agent setup status

Source: https://docs.runpod.io/agent-setup

Example output format for reporting the verification status of skills and MCP server authentication.

```text
┌─ Runpod Agent Setup ─────────────────────────────────┐
│  ✓ Runpod skills   router + 6 skills                 │
│  ⚠ runpod MCP      needs sign-in: /mcp → runpod      │
│                                                      │
│  ⚡ Reload or restart your agent to load the skills  │
└──────────────────────────────────────────────────────┘
```

--------------------------------

### Initialize Project Environment

Source: https://docs.runpod.io/tutorials/public-endpoints/text-to-video-pipeline

Commands to create a project directory, set up a virtual environment, and install the requests library.

```bash
mkdir text-to-video && cd text-to-video
python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install requests
export RUNPOD_API_KEY="YOUR_API_KEY"
```

--------------------------------

### Create project file

Source: https://docs.runpod.io/tutorials/flash/image-generation-with-sdxl

Initialize the project file using the touch command.

```bash
touch image_generation.py
```

--------------------------------

### Configure Runpod MCP via CLI

Source: https://docs.runpod.io/get-started/mcp-servers

Use the guided installer to automatically detect and configure supported MCP clients.

```bash
npx @runpod/mcp-server@latest add
```

```bash
npx @runpod/mcp-server@latest remove
```

--------------------------------

### Run Web API on Internal Port

Source: https://docs.runpod.io/pods/configuration/expose-ports

Example command to start a web service inside a Pod on a specific internal port.

```bash
uvicorn main:app --host 0.0.0.0 --port 4000
```

--------------------------------

### Set up the Python environment

Source: https://docs.runpod.io/serverless/workers/concurrent-handler

Commands to create a virtual environment and install the required Runpod and asyncio packages.

```sh
# Create a Python virtual environment
python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install runpod asyncio
```

--------------------------------

### Get SSH connection info

Source: https://docs.runpod.io/runpodctl/reference/runpodctl-ssh

Retrieves the connection command and key details for a specific Pod without starting an interactive session.

```bash
runpodctl ssh info <pod-id>
```

--------------------------------

### Create start.sh entrypoint script

Source: https://docs.runpod.io/serverless/development/dual-mode-worker

This script manages service initialization, SSH setup, environment variable exports, and conditional execution based on the MODE_TO_RUN environment variable.

```bash
#!/bin/bash
set -e  # Exit the script if any statement returns a non-true return value

# Set workspace directory from env or default
WORKSPACE_DIR="${WORKSPACE_DIR:-/workspace}"

# Start nginx service
start_nginx() {
    echo "Starting Nginx service..."
    service nginx start
}

# Execute script if exists
execute_script() {
    local script_path=$1
    local script_msg=$2
    if [[ -f ${script_path} ]]; then
        echo "${script_msg}"
        bash ${script_path}
    fi
}

# Setup ssh
setup_ssh() {
    if [[ $PUBLIC_KEY ]]; then
        echo "Setting up SSH..."
        mkdir -p ~/.ssh
        echo "$PUBLIC_KEY" >> ~/.ssh/authorized_keys
        chmod 700 -R ~/.ssh
        # Generate SSH host keys if not present
        generate_ssh_keys
        service ssh start
        echo "SSH host keys:"
        cat /etc/ssh/*.pub
    fi
}

# Generate SSH host keys
generate_ssh_keys() {
    ssh-keygen -A
}

# Export env vars
export_env_vars() {
    echo "Exporting environment variables..."
    printenv | grep -E '^RUNPOD_|^PATH=|^_=' | awk -F = '{ print "export " $1 "=\"" $2 "\"" }' >> /etc/rp_environment
    echo 'source /etc/rp_environment' >> ~/.bashrc
}

# Start jupyter lab
start_jupyter() {
    echo "Starting Jupyter Lab..."
    mkdir -p "$WORKSPACE_DIR" && \
    cd / && \
    nohup jupyter lab --allow-root --no-browser --port=8888 --ip=* --NotebookApp.token='' --NotebookApp.password='' --FileContentsManager.delete_to_trash=False --ServerApp.terminado_settings='{"shell_command":["/bin/bash"]}' --ServerApp.allow_origin=* --ServerApp.preferred_dir="$WORKSPACE_DIR" &> /jupyter.log &
    echo "Jupyter Lab started without a password"
}

# Call Python handler if mode is serverless or both
call_python_handler() {
    echo "Calling Python handler.py..."
    python $WORKSPACE_DIR/handler.py
}

# ---------------------------------------------------------------------------- #
#                               Main Program                                   #
# ---------------------------------------------------------------------------- #

start_nginx

echo "Pod Started"

setup_ssh

case $MODE_TO_RUN in
    serverless)
        echo "Running in serverless mode"
        call_python_handler
        ;;
    pod)
        echo "Running in pod mode"
        start_jupyter
        ;;
    *)
        echo "Invalid MODE_TO_RUN value: $MODE_TO_RUN. Expected 'serverless', 'pod', or 'both'."
        exit 1
        ;;
esac

export_env_vars

echo "Start script(s) finished"

sleep infinity
```

--------------------------------

### OpenAPI definition for network volume billing

Source: https://docs.runpod.io/api-reference-v2/billing/get-network-volume-billing-history

The OpenAPI specification for the GET /v2/billing/network-volumes endpoint, including parameters and a successful response example.

```yaml
get /v2/billing/network-volumes:
  openapi: 3.1.0
  info:
    title: Runpod REST API
    version: 2.0.0
    description: Runpod public REST API — v2
  servers:
    - url: https://api.runpod.io
      description: Runpod API v2 production server
  security:
    - bearerAuth: []
  tags:
    - name: Account
      description: Account-scoped settings and primitives (SSH public keys).
    - name: Pods
      description: GPU and CPU pod lifecycle, configuration, actions, and log streaming.
    - name: Serverless
      description: >-
        Serverless endpoint lifecycle, worker visibility, releases, and worker log
        streaming.
    - name: Templates
      description: Reusable pod and endpoint configuration templates.
    - name: Network Volumes
      description: Persistent network storage volumes for workloads.
    - name: Registries
      description: Container registry credentials used to pull private images.
    - name: Catalog
      description: Available GPU, CPU, data center, and public template catalog metadata.
    - name: Billing
      description: Billing history and usage cost records across resource types.
  paths:
    /v2/billing/network-volumes:
      get:
        tags:
          - Billing
        summary: Get network volume billing history
        description: >
          Returns network volume billing history for the authenticated user, split
          into time buckets by startTime/endTime with bucketSize or by lastN
          recent buckets. Use networkVolumeId to filter to one volume; without it,
          records are emitted per volume per bucket. Each record includes standard
          storage, high-performance storage, and total amounts, while metadata
          reports the resolved query, distinct volume count, and totals across the
          returned records.
        operationId: listNetworkVolumeBilling
        parameters:
          - $ref: '#/components/parameters/BillingStartTime'
          - $ref: '#/components/parameters/BillingEndTime'
          - $ref: '#/components/parameters/BillingBucketSize'
          - $ref: '#/components/parameters/BillingLastN'
          - name: networkVolumeId
            in: query
            required: false
            description: Filter to a specific network volume.
            schema:
              type: string
            example: vol_abc123
        responses:
          '200':
            headers:
              RateLimit:
                $ref: '#/components/headers/RateLimit'
              RateLimit-Policy:
                $ref: '#/components/headers/RateLimit-Policy'
            description: OK
            content:
              application/json:
                schema:
                  $ref: '#/components/schemas/ListNetworkVolumeBillingResponse'
                examples:
                  networkVolumeBilling:
                    summary: Successful response
                    value:
                      records:
                        - startTime: '2026-06-01T00:00:00Z'
                          endTime: '2026-06-02T00:00:00Z'
                          networkVolumeId: 2q9m7x4c
                          totalAmount: 5.79
                          standardAmount: 0
                          highPerformanceAmount: 5.79
                      metadata:
                        query:
                          startTime: '2026-06-01T00:00:00Z'
                          endTime: '2026-06-02T00:00:00Z'
                          bucketSize: day
                          networkVolumeId: 2q9m7x4c
                        recordCount: 1
                        totals:
                          totalAmount: 5.79
                          standardAmount: 0
                          highPerformanceAmount: 5.79
                        uniqueNetworkVolumeCount: 1
          '401':
            $ref: '#/components/responses/UnauthorizedError'
          '403':
            $ref: '#/components/responses/ForbiddenError'
          '429':
            $ref: '#/components/responses/TooManyRequestsError'
          default:
            headers:
              RateLimit:
                $ref: '#/components/headers/RateLimit'
              RateLimit-Policy:
                $ref: '#/components/headers/RateLimit-Policy'
            description: Error
            content:
              application/problem+json:
                schema:
                  $ref: '#/components/schemas/ErrorResponse'
  components:
    parameters:
      BillingStartTime:
        name: startTime
        in: query
        required: false
```

--------------------------------

### OpenAPI Specification for List Templates

Source: https://docs.runpod.io/api-reference-v2/templates/list-templates

Defines the GET /v2/templates endpoint, including the request structure, response schema, and a successful response example.

```yaml
openapi: 3.1.0
info:
  title: Runpod REST API
  version: 2.0.0
  description: Runpod public REST API — v2
servers:
  - url: https://api.runpod.io
    description: Runpod API v2 production server
security:
  - bearerAuth: []
tags:
  - name: Account
    description: Account-scoped settings and primitives (SSH public keys).
  - name: Pods
    description: GPU and CPU pod lifecycle, configuration, actions, and log streaming.
  - name: Serverless
    description: >-
      Serverless endpoint lifecycle, worker visibility, releases, and worker log
      streaming.
  - name: Templates
    description: Reusable pod and endpoint configuration templates.
  - name: Network Volumes
    description: Persistent network storage volumes for workloads.
  - name: Registries
    description: Container registry credentials used to pull private images.
  - name: Catalog
    description: Available GPU, CPU, data center, and public template catalog metadata.
  - name: Billing
    description: Billing history and usage cost records across resource types.
paths:
  /v2/templates:
    get:
      tags:
        - Templates
      summary: List templates
      description: Returns all templates owned by the authenticated user.
      operationId: listTemplates
      responses:
        '200':
          headers:
            RateLimit:
              $ref: '#/components/headers/RateLimit'
            RateLimit-Policy:
              $ref: '#/components/headers/RateLimit-Policy'
          description: OK
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ListTemplatesResponse'
              examples:
                templates:
                  summary: Successful response
                  value:
                    templates:
                      - id: 9x4m2p7v
                        name: PyTorch GPU Template
                        image: runpod/pytorch:1.0.2-cu1281-torch280-ubuntu2404
                        args: ''
                        disk: 50
                        mounts:
                          persistent:
                            size: 20
                            path: /workspace
                        ports:
                          - 8888/http
                        env:
                          JUPYTER_ENABLE_LAB: 'yes'
                        registry: null
                        serverless: false
                        public: false
                        category: NVIDIA
                        startSsh: true
                        startJupyter: false
                        allowedCudaVersions: []
        '401':
          $ref: '#/components/responses/UnauthorizedError'
        '403':
          $ref: '#/components/responses/ForbiddenError'
        '429':
          $ref: '#/components/responses/TooManyRequestsError'
        default:
          headers:
            RateLimit:
              $ref: '#/components/headers/RateLimit'
            RateLimit-Policy:
              $ref: '#/components/headers/RateLimit-Policy'
          description: Error
          content:
            application/problem+json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
components:
  headers:
    RateLimit:
      schema:
        $ref: '#/components/schemas/RateLimitHeader'
    RateLimit-Policy:
      schema:
        $ref: '#/components/schemas/RateLimitPolicyHeader'
  schemas:
    ListTemplatesResponse:
      type: object
      required:
        - templates
      properties:
        templates:
          type: array
          items:
            $ref: '#/components/schemas/Template'
    ErrorResponse:
      type: object
      required:
        - title
        - status
        - detail
      properties:
        title:
          type: string
          description: Short human-readable summary
          examples:
            - Not Found
        status:
          type: integer
          description: HTTP status code
          examples:
            - 404
        detail:
          type: string
          description: Human-readable explanation
          examples:
            - pod not found
        errors:
          type: array
          description: Individual request-validation failures.
          items:
            type: string
          examples:
            - - '$: additional properties ''bogus'' not allowed'
    RateLimitHeader:
      type: string
      description: |
        Live per-window quota state. Optional — omitted for rate-limit-exempt
        callers.
```

--------------------------------

### OpenAPI Specification for Listing Endpoints

Source: https://docs.runpod.io/api-reference-v2/serverless/list-serverless-endpoints

Defines the GET /v2/serverless operation, including the expected response structure and example payload for serverless endpoints.

```yaml
openapi: 3.1.0
info:
  title: Runpod REST API
  version: 2.0.0
  description: Runpod public REST API — v2
servers:
  - url: https://api.runpod.io
    description: Runpod API v2 production server
security:
  - bearerAuth: []
tags:
  - name: Account
    description: Account-scoped settings and primitives (SSH public keys).
  - name: Pods
    description: GPU and CPU pod lifecycle, configuration, actions, and log streaming.
  - name: Serverless
    description: >-
      Serverless endpoint lifecycle, worker visibility, releases, and worker log
      streaming.
  - name: Templates
    description: Reusable pod and endpoint configuration templates.
  - name: Network Volumes
    description: Persistent network storage volumes for workloads.
  - name: Registries
    description: Container registry credentials used to pull private images.
  - name: Catalog
    description: Available GPU, CPU, data center, and public template catalog metadata.
  - name: Billing
    description: Billing history and usage cost records across resource types.
paths:
  /v2/serverless:
    get:
      tags:
        - Serverless
      summary: List serverless endpoints
      description: Returns all serverless endpoints owned by the authenticated user.
      operationId: listEndpoints
      responses:
        '200':
          headers:
            RateLimit:
              $ref: '#/components/headers/RateLimit'
            RateLimit-Policy:
              $ref: '#/components/headers/RateLimit-Policy'
          description: OK
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ListEndpointsResponse'
              examples:
                endpoints:
                  summary: Successful response
                  value:
                    endpoints:
                      - id: 4m7x2k9q
                        name: image-generator
                        type: QUEUE
                        requestUrls:
                          run: https://api.runpod.ai/v2/4m7x2k9q/run
                          runSync: https://api.runpod.ai/v2/4m7x2k9q/runsync
                          status: https://api.runpod.ai/v2/4m7x2k9q/status/{job_id}
                          stream: https://api.runpod.ai/v2/4m7x2k9q/stream/{job_id}
                          cancel: https://api.runpod.ai/v2/4m7x2k9q/cancel/{job_id}
                          retry: https://api.runpod.ai/v2/4m7x2k9q/retry/{job_id}
                          purgeQueue: https://api.runpod.ai/v2/4m7x2k9q/purge-queue
                          health: https://api.runpod.ai/v2/4m7x2k9q/health
                        image: runpod/pytorch:1.0.2-cu1281-torch280-ubuntu2404
                        args: ''
                        disk: 20
                        ports:
                          - 8000/http
                        env:
                          MODEL_NAME: sdxl
                        registry: null
                        gpu:
                          pools:
                            - ADA_24
                          count: 1
                          allowedCudaVersions: []
                          minCudaVersion: null
                        workers:
                          min: 0
                          max: 5
                          idleTimeout: 5
                        scaling:
                          type: QUEUE_DELAY
                          queueDelay: 4
                        dataCenterIds:
                          - US-KS-2
                        networkVolumes:
                          - 2q9m7x4c
                        timeout: 300000
                        flashboot: 'OFF'
                        createdAt: '2026-06-01T12:00:00Z'
        '401':
          $ref: '#/components/responses/UnauthorizedError'
        '403':
          $ref: '#/components/responses/ForbiddenError'
        '429':
          $ref: '#/components/responses/TooManyRequestsError'
        default:
          headers:
            RateLimit:
              $ref: '#/components/headers/RateLimit'
            RateLimit-Policy:
              $ref: '#/components/headers/RateLimit-Policy'
          description: Error
          content:
            application/problem+json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'
components:
  headers:
    RateLimit:
      schema:
        $ref: '#/components/schemas/RateLimitHeader'
    RateLimit-Policy:
      schema:
        $ref: '#/components/schemas/RateLimitPolicyHeader'
  schemas:
    ListEndpointsResponse:
      type: object
      required:
        - endpoints
      properties:
        endpoints:
          type: array
          items:
```

--------------------------------

### Initialize project directory

Source: https://docs.runpod.io/pods/templates/create-custom-template

Create and enter a new directory for the template project.

```bash
mkdir my-custom-pod-template
cd my-custom-pod-template
```

--------------------------------

### OpenAPI definition for aggregated billing history

Source: https://docs.runpod.io/api-reference-v2/billing/get-aggregated-billing-history

The OpenAPI specification for the /v2/billing GET endpoint, including the response schema and a successful example payload.

```yaml
openapi: 3.1.0
info:
  title: Runpod REST API
  version: 2.0.0
  description: Runpod public REST API — v2
servers:
  - url: https://api.runpod.io
    description: Runpod API v2 production server
security:
  - bearerAuth: []
tags:
  - name: Account
    description: Account-scoped settings and primitives (SSH public keys).
  - name: Pods
    description: GPU and CPU pod lifecycle, configuration, actions, and log streaming.
  - name: Serverless
    description: >-
      Serverless endpoint lifecycle, worker visibility, releases, and worker log
      streaming.
  - name: Templates
    description: Reusable pod and endpoint configuration templates.
  - name: Network Volumes
    description: Persistent network storage volumes for workloads.
  - name: Registries
    description: Container registry credentials used to pull private images.
  - name: Catalog
    description: Available GPU, CPU, data center, and public template catalog metadata.
  - name: Billing
    description: Billing history and usage cost records across resource types.
paths:
  /v2/billing:
    get:
      tags:
        - Billing
      summary: Get aggregated billing history
      description: >
        Returns time-bucketed total spend across all billable Runpod resources
        for the authenticated user. Use startTime/endTime with bucketSize for an
        explicit range, or lastN with bucketSize for the most recent buckets.
        Each record reports one bucket's total plus pod, serverless, storage,
        public endpoint, and Instant Cluster cost components. The metadata block
        echoes the resolved query window, record count, and totals across all
        returned buckets.
      operationId: listBilling
      parameters:
        - $ref: '#/components/parameters/BillingStartTime'
        - $ref: '#/components/parameters/BillingEndTime'
        - $ref: '#/components/parameters/BillingBucketSize'
        - $ref: '#/components/parameters/BillingLastN'
      responses:
        '200':
          headers:
            RateLimit:
              $ref: '#/components/headers/RateLimit'
            RateLimit-Policy:
              $ref: '#/components/headers/RateLimit-Policy'
          description: OK
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ListBillingResponse'
              examples:
                billing:
                  summary: Successful response
                  value:
                    records:
                      - startTime: '2026-06-01T00:00:00Z'
                        endTime: '2026-06-02T00:00:00Z'
                        totalAmount: 42.34
                        podGpuAmount: 18.2
                        podCpuAmount: 0
                        podDiskAmount: 1.1
                        serverlessGpuAmount: 12.6
                        serverlessCpuAmount: 0
                        serverlessDiskAmount: 0.44
                        serverlessFeeAmount: 1.25
                        storageStandardAmount: 0
                        storageHighPerformanceAmount: 2.5
                        endpointAmount: 3.21
                        clusterGpuAmount: 2.5
                        clusterDiskAmount: 0.3
                        clusterNetworkingAmount: 0.24
                    metadata:
                      query:
                        startTime: '2026-06-01T00:00:00Z'
                        endTime: '2026-06-02T00:00:00Z'
                        bucketSize: day
                      recordCount: 1
                      totals:
                        totalAmount: 42.34
                        podGpuAmount: 18.2
                        podCpuAmount: 0
                        podDiskAmount: 1.1
                        serverlessGpuAmount: 12.6
                        serverlessCpuAmount: 0
                        serverlessDiskAmount: 0.44
                        serverlessFeeAmount: 1.25
                        storageStandardAmount: 0
                        storageHighPerformanceAmount: 2.5
                        endpointAmount: 3.21
                        clusterGpuAmount: 2.5
                        clusterDiskAmount: 0.3
                        clusterNetworkingAmount: 0.24
        '401':
          $ref: '#/components/responses/UnauthorizedError'
        '403':
```

--------------------------------

### Configure runpodctl API key

Source: https://docs.runpod.io/runpodctl/overview

Use the doctor command for guided setup or manually set the API key via the config command.

```bash
runpodctl doctor
```

```bash
runpodctl config --apiKey YOUR_API_KEY
```

--------------------------------

### Define a GPU-accelerated sentiment analysis endpoint

Source: https://docs.runpod.io/tutorials/flash/build-rest-api-with-load-balancer

Configures a GPU endpoint with specific dependencies and scaling parameters for ML inference. Note that initial cold starts may take 3-10 minutes due to provisioning and dependency installation.

```python
from runpod_flash import Endpoint, GpuGroup

# GPU endpoint for ML inference
gpu_api = Endpoint(
    name="gpu-sentiment",
    gpu=GpuGroup.ANY,   # Use any available GPU for better availability
    workers=(0, 1),     # Scale from 0 to 1 worker
    idle_timeout=300,  # 5 minutes
    dependencies=["transformers", "torch"]
)

@gpu_api.post("/sentiment")
async def analyze_sentiment(text: str) -> dict:
    """Analyze sentiment using a pretrained model."""
    from transformers import pipeline
    import torch

    # Load sentiment analysis pipeline
    device = 0 if torch.cuda.is_available() else -1
    sentiment_analyzer = pipeline(
        "sentiment-analysis",
        model="distilbert-base-uncased-finetuned-sst-2-english",
        device=device
    )

    # Analyze sentiment
    result = sentiment_analyzer(text)[0]

    return {
        "text": text,
        "sentiment": {
            "label": result["label"],
            "score": round(result["score"], 4)
        },
        "device": "GPU" if torch.cuda.is_available() else "CPU"
    }
```

--------------------------------

### Set up Python virtual environment

Source: https://docs.runpod.io/integrations/dstack

Initialize and activate a virtual environment for the project.

```bash
python3 -m venv .venv
source .venv/bin/activate
```

```bash
python -m venv .venv
.venv\Scripts\activate
```

```bash
python -m venv .venv
.venv\Scripts\Activate.ps1
```

--------------------------------

### Initialize a Flash project

Source: https://docs.runpod.io/flash/apps/overview

Use the CLI to create a new project structure and install necessary dependencies.

```bash
flash init PROJECT_NAME
cd PROJECT_NAME
pip install -r requirements.txt
```

--------------------------------

### runpodctl pod start

Source: https://docs.runpod.io/runpodctl/reference/runpodctl-pod

Starts a stopped Pod.

```APIDOC
## runpodctl pod start <pod-id>

### Description
Starts a previously stopped Pod using its unique identifier.
```

--------------------------------

### Install dstack

Source: https://docs.runpod.io/integrations/dstack

Install the dstack package using pip.

```bash
pip3 install -U "dstack[all]"
```

```bash
pip install -U "dstack[all]"
```

--------------------------------

### Initialize Project Directory

Source: https://docs.runpod.io/tutorials/pods/build-docker-images

Create and navigate into a new directory for the Bazel project.

```bash
mkdir -p ~/docker-build && cd ~/docker-build
```

--------------------------------

### Install OhMyRunpod

Source: https://docs.runpod.io/community-solutions/ohmyrunpod/overview

Use pip to install the OhMyRunpod package.

```bash
pip install OhMyRunpod
```

--------------------------------

### Runpod CLI Quick Start Commands

Source: https://docs.runpod.io/runpodctl/overview

Essential commands for initializing the CLI, searching for resources, and managing Pods.

```bash
runpodctl doctor                    # First time setup (API key + SSH)
runpodctl gpu list                  # See available GPUs
runpodctl template search pytorch   # Find a template
runpodctl pod create --template-id runpod-torch-v21 --gpu-id "NVIDIA RTX 4090"
runpodctl pod list                  # List your Pods
```

--------------------------------

### Install on Windows

Source: https://docs.runpod.io/runpodctl/overview

Download the Windows binary using wget.

```bash
wget https://github.com/runpod/runpodctl/releases/latest/download/runpodctl-windows-amd64.exe -O runpodctl.exe
```

--------------------------------

### Verify Installations

Source: https://docs.runpod.io/tutorials/pods/build-docker-images

Check that both Docker and Bazel are correctly installed and accessible.

```bash
docker --version && bazel --version
```

--------------------------------

### Initialize project directory

Source: https://docs.runpod.io/serverless/quickstart

Creates a new directory and the necessary boilerplate files for a serverless project.

```bash
mkdir serverless-quickstart && cd serverless-quickstart
touch handler.py Dockerfile requirements.txt test_input.json
```

--------------------------------

### Install on Linux

Source: https://docs.runpod.io/runpodctl/overview

Installation commands for Linux architectures and non-root environments.

```bash
wget --quiet --show-progress https://github.com/runpod/runpodctl/releases/latest/download/runpodctl-linux-amd64 -O runpodctl && chmod +x runpodctl && sudo cp runpodctl /usr/bin/runpodctl
```

```bash
wget --quiet --show-progress https://github.com/runpod/runpodctl/releases/latest/download/runpodctl-linux-arm64 -O runpodctl && chmod +x runpodctl && sudo cp runpodctl /usr/bin/runpodctl
```

```bash
mkdir -p ~/.local/bin && wget --quiet --show-progress https://github.com/runpod/runpodctl/releases/latest/download/runpodctl-linux-amd64 -O ~/.local/bin/runpodctl && chmod +x ~/.local/bin/runpodctl
```

```bash
export PATH="$HOME/.local/bin:$PATH"
```
Tokens:
Raw