### Install Ollama and Start Server

Source: https://github.com/runpod/docs/blob/main/tutorials/pods/run-ollama.mdx

This command downloads and installs Ollama, then starts the Ollama server in the background, redirecting logs to ollama.log. Ensure you are in the Pod's web terminal.

```bash
(curl -fsSL https://ollama.com/install.sh | sh && ollama serve > ollama.log 2>&1) &
```

--------------------------------

### Runpod CLI Quick Start Commands

Source: https://github.com/runpod/docs/blob/main/runpodctl/overview.mdx

Execute these commands after installing and configuring the Runpod CLI for initial setup and resource management.

```bash
runpodctl doctor                    # First time setup (API key + SSH)
runpodctl gpu list                  # See available GPUs
runpodctl template search pytorch   # Find a template
runpodctl pod create --template-id runpod-torch-v21 --gpu-id "NVIDIA RTX 4090"
runpodctl pod list                  # List your Pods
```

--------------------------------

### Initialize and Start Local Development

Source: https://github.com/runpod/docs/blob/main/flash/cli/overview.mdx

Create a new Flash project, navigate into its directory, install dependencies, and start the local development server. Ensure your API key is set in a .env file.

```bash
# Create a new project
flash init PROJECT_NAME
cd PROJECT_NAME

# Install dependencies
pip install -r requirements.txt

# Add your API key to .env
# Start the development server
flash dev
```

--------------------------------

### Install blacken-docs

Source: https://github.com/runpod/docs/blob/main/README.md

Install the blacken-docs package to format Python code examples within Markdown files.

```bash
python3 -m pip install blacken-docs
```

--------------------------------

### First Run Output Example

Source: https://github.com/runpod/docs/blob/main/tutorials/flash/text-generation-with-transformers.mdx

This output shows the initial execution of the text generation script, including endpoint provisioning, dependency installation, model download, and the final generated text with metadata. This process typically takes 60-90 seconds.

```text
Starting text generation on Runpod GPU...
Creating endpoint: server_Endpoint_a1b2c3d4
Provisioning Serverless endpoint...
Endpoint ready
Registering RunPod endpoint at https://api.runpod.ai/xvf32dan8rcilp
Executing function on RunPod endpoint ID: xvf32dan8rcilp
Initial job status: IN_QUEUE
Installing dependencies: transformers torch accelerate
Downloading model weights...
Job completed, output received

============================================================
TEXT GENERATION RESULTS
============================================================

Prompt: The future of artificial intelligence is

Generated text:
The future of artificial intelligence is bright and full of possibilities. With advancements in machine learning and deep learning, we're seeing AI systems that can understand natural language, recognize images, and even create art. The potential applications are endless, from healthcare to transportation to education.

------------------------------------------------------------
Model: gpt2
Device: cuda
GPU: NVIDIA GeForce RTX 4090
Max length: 100 tokens
============================================================
```

--------------------------------

### Install Serverless SDK

Source: https://github.com/runpod/docs/blob/main/serverless/quickstart.mdx

Sets up a Python virtual environment and installs the Runpod Serverless SDK.

```bash
python3 -m venv .venv
source .venv/bin/activate
pip install runpod
```

--------------------------------

### Install Runpod SDK

Source: https://github.com/runpod/docs/blob/main/serverless/endpoints/operation-reference.mdx

Install the Runpod SDK for Python, JavaScript, or Go.

```bash
python -m pip install runpod
```

```bash
npm install --save runpod-sdk
```

```bash
go get github.com/runpod/go-sdk && go mod tidy
```

--------------------------------

### Install Mintlify

Source: https://github.com/runpod/docs/blob/main/README.md

Install the Mintlify CLI globally to manage documentation locally.

```shell
npm i -g mintlify
```

--------------------------------

### Install and Start SSH Daemon in a Custom Template

Source: https://github.com/runpod/docs/blob/main/pods/configuration/use-ssh.mdx

Use this command to install the OpenSSH server, configure SSH keys, and start the SSH service within a custom Docker template. Ensure the PUBLIC_KEY environment variable is set.

```bash
bash -c 'apt update; \
DEBIAN_FRONTEND=noninteractive apt-get install openssh-server -y; \
mkdir -p ~/.ssh; \
cd ~/.ssh; \
chmod 700 ~/.ssh; \
echo "$PUBLIC_KEY" >> authorized_keys; \
chmod 700 authorized_keys; \
service ssh start; \
sleep infinity'
```

--------------------------------

### Set Up Virtual Environment and Install Requirements

Source: https://github.com/runpod/docs/blob/main/README.md

Prepare a Python virtual environment and install necessary dependencies for helper scripts.

```bash
# Set up virtual environment
python3 -m venv helpers/.venv
source helpers/.venv/bin/activate
pip install -r helpers/requirements.txt
```

--------------------------------

### Install Runpod CLI using Install Script

Source: https://github.com/runpod/docs/blob/main/runpodctl/overview.mdx

Use the provided bash script to automatically detect your OS and architecture for installation. The script installs to /usr/local/bin or ~/.local/bin.

```bash
bash <(wget -qO- cli.runpod.io)
```

```bash
bash <(curl -sL cli.runpod.io)
```

--------------------------------

### Install Dependencies with uv

Source: https://github.com/runpod/docs/blob/main/community-solutions/runpod-network-volume-storage-tool.mdx

Clone the repository and install project dependencies using uv.

```bash
git clone https://github.com/justinwlin/Runpod-Network-Volume-Storage-Tool.git
cd Runpod-Network-Volume-Storage-Tool

# Install dependencies with uv
uv sync
```

--------------------------------

### Install Runpod SDK for Go

Source: https://github.com/runpod/docs/blob/main/serverless/sdks.mdx

Installs the Go SDK and updates module dependencies. Ensure you have Go installed and your project is set up with Go modules.

```bash
go get github.com/runpod/go-sdk
go mod tidy
```

--------------------------------

### Install Project Dependencies

Source: https://github.com/runpod/docs/blob/main/flash/apps/initialize-project.mdx

After initializing a project, install the necessary Python dependencies listed in `requirements.txt`.

```bash
pip install -r requirements.txt
```

--------------------------------

### Install WSL2 and Ubuntu

Source: https://github.com/runpod/docs/blob/main/flash/windows-wsl2.mdx

Enables WSL, installs the latest Linux kernel, sets WSL2 as default, and installs Ubuntu. Run this in PowerShell or Command Prompt as Administrator.

```powershell
wsl --install
```

--------------------------------

### Install Ubuntu on existing WSL

Source: https://github.com/runpod/docs/blob/main/flash/windows-wsl2.mdx

If WSL is already installed, this command installs Ubuntu specifically. Run this in PowerShell or Command Prompt as Administrator.

```powershell
wsl --install -d Ubuntu
```

--------------------------------

### Start Local Dev Server

Source: https://github.com/runpod/docs/blob/main/CLAUDE.md

Use this command to start the local development server for the Mintlify documentation site.

```bash
mintlify dev
```

--------------------------------

### Install Bazel via Bazelisk

Source: https://github.com/runpod/docs/blob/main/tutorials/pods/build-docker-images.mdx

Installs Bazelisk and copies it to /usr/local/bin/bazel. Ensure you have wget and chmod permissions.

```bash
wget https://github.com/bazelbuild/bazelisk/releases/download/v1.28.1/bazelisk-linux-amd64 && \
   chmod +x bazelisk-linux-amd64 && \
   sudo cp ./bazelisk-linux-amd64 /usr/local/bin/bazel
```

--------------------------------

### Start dstack Server

Source: https://github.com/runpod/docs/blob/main/integrations/dstack.mdx

Start the dstack server process. Ensure you have configured your Runpod credentials in config.yml.

```bash
dstack server
```

--------------------------------

### Example Runpod proxy URL

Source: https://github.com/runpod/docs/blob/main/pods/configuration/expose-ports.mdx

An example of how to format the proxy URL with a specific Pod ID and internal port.

```text
https://abc123xyz-4000.proxy.runpod.net
```

--------------------------------

### Install Runpod Flash with uv

Source: https://github.com/runpod/docs/blob/main/flash/overview.mdx

Install the Runpod Flash SDK using uv. This is an alternative package manager.

```bash
# Or uv
uv tool install runpod-flash
```

--------------------------------

### Install Flash and Python Dependencies

Source: https://github.com/runpod/docs/blob/main/tutorials/flash/image-generation-with-sdxl.mdx

Install the runpod-flash library and python-dotenv for managing environment variables using uv.

```bash
uv venv
source .venv/bin/activate
uv pip install runpod-flash python-dotenv
```

--------------------------------

### Install and Configure Runpod CLI

Source: https://github.com/runpod/docs/blob/main/get-started.mdx

Install the Runpod CLI and configure it with your API key to manage Pods from your terminal.

```bash
# Install runpodctl
bash <(wget -qO- cli.runpod.io)
```

```bash
# Configure your API key
runpodctl config --apiKey "your-api-key"
```

--------------------------------

### Install Flash and Initialize Project

Source: https://github.com/runpod/docs/blob/main/flash/apps/build-app.mdx

Use uv to install Flash and the flash init command to generate a project template. Authenticate with Runpod using flash login.

```bash
# Create the project directory and navigate into it:
mkdir flash_app
cd flash_app

# Install Flash:
uv venv
source .venv/bin/activate
uv pip install runpod-flash
```

```bash
uv run flash init .
```

```bash
uv run flash login
```

--------------------------------

### Install and Run CopyParty (Standard)

Source: https://github.com/runpod/docs/blob/main/community-solutions/copyparty-file-manager/overview.mdx

Installs CopyParty using uv and runs it on the specified port. The session will end if the terminal is closed.

```bash
curl -LsSf https://astral.sh/uv/install.sh | sh && source $HOME/.local/bin/env && uv tool run copyparty -p 8000 --allow-csrf
```

--------------------------------

### Set up Environment and Install Packages

Source: https://github.com/runpod/docs/blob/main/serverless/workers/concurrent-handler.mdx

Create a Python virtual environment and install the runpod and asyncio packages. This is a prerequisite for building the concurrent handler.

```sh
python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install runpod asyncio
```

--------------------------------

### GET /health

Source: https://github.com/runpod/docs/blob/main/flash/apps/requests.mdx

Example of a GET route defined using the `api.get` decorator for health checks.

```APIDOC
## GET /health

### Description
This endpoint performs a health check on the service.

### Method
GET

### Endpoint
https://<your-endpoint-id>.api.runpod.ai/health

### Response
#### Success Response (200)
- The response structure depends on the implementation of the `/health` route.
```

--------------------------------

### Set up project and API key

Source: https://github.com/runpod/docs/blob/main/tutorials/public-endpoints/text-to-video-pipeline.mdx

Initialize project directory, virtual environment, install dependencies, and set the Runpod API key as an environment variable. Ensure Python 3.8+ is installed.

```bash
mkdir text-to-video && cd text-to-video
python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install requests
export RUNPOD_API_KEY="YOUR_API_KEY"
```

--------------------------------

### Create project directory and Dockerfile

Source: https://github.com/runpod/docs/blob/main/tutorials/introduction/containers/persist-data.mdx

Set up a new project directory and create a Dockerfile. This example uses busybox and configures an entrypoint script to write data to a volume.

```bash
mkdir volume-example
cd volume-example
```

```dockerfile
FROM busybox
WORKDIR /data
COPY entrypoint.sh /
RUN chmod +x /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]
```

--------------------------------

### Navigate to Tutorial Directory

Source: https://github.com/runpod/docs/blob/main/integrations/dstack.mdx

Use this command to change your current directory to the tutorial's root folder. Ensure you are in the correct project directory before proceeding.

```bash
cd runpod-dstack-tutorial
```

--------------------------------

### GET /info

Source: https://github.com/runpod/docs/blob/main/flash/apps/requests.mdx

Example of calling a GET route to retrieve information. This endpoint does not require a request body.

```APIDOC
## GET /info

### Description
This endpoint retrieves general information about the service or application.

### Method
GET

### Endpoint
https://<your-endpoint-id>.api.runpod.ai/info

### Response
#### Success Response (200)
- The response structure depends on the implementation of the `/info` route.
```

--------------------------------

### Specify System Dependencies

Source: https://github.com/runpod/docs/blob/main/flash/configuration/parameters.mdx

Install system-level packages using the 'system_dependencies' parameter, which are installed via apt before your function executes. This example includes libraries for OpenCV.

```python
@Endpoint(
    name="video-processor",
    gpu=GpuGroup.ANY,
    dependencies=["opencv-python"],
    system_dependencies=["libgl1-mesa-glx", "libglib2.0-0"]
)
async def process_video(data): ...

```

--------------------------------

### Local Test Output Example

Source: https://github.com/runpod/docs/blob/main/serverless/workers/concurrent-handler.mdx

Example output from running the handler locally. This shows the serverless worker starting, job input, processing details, and completion status.

```sh
--- Starting Serverless Worker |  Version 1.7.9 ---
INFO   | Using test_input.json as job input.
DEBUG  | Retrieved local job: {'input': {'message': 'Test concurrent processing', 'delay': 0.5}, 'id': 'local_test'}
INFO   | local_test | Started.
DEBUG  | local_test | Handler output: Processed: {'message': 'Test concurrent processing', 'delay': 0.5}
DEBUG  | local_test | run_job return: {'output': "Processed: {'message': 'Test concurrent processing', 'delay': 0.5}"}
INFO   | Job local_test completed successfully.
INFO   | Job result: {'output': "Processed: {'message': 'Test concurrent processing', 'delay': 0.5}"}
INFO   | Local testing complete, exiting.
```

--------------------------------

### Runpod CLI Version Output

Source: https://github.com/runpod/docs/blob/main/runpodctl/overview.mdx

Example output after running the 'runpodctl version' command, showing the installed version.

```bash
runpodctl v2.0
```

--------------------------------

### Update Ubuntu packages

Source: https://github.com/runpod/docs/blob/main/flash/windows-wsl2.mdx

Updates package lists and upgrades installed packages to their latest versions. Run this in the Ubuntu terminal after setup.

```bash
sudo apt update && sudo apt upgrade -y
```

--------------------------------

### Run Chatbot Script

Source: https://github.com/runpod/docs/blob/main/tutorials/serverless/run-gemma-7b.mdx

Executes the Python chatbot script. Ensure you have completed the installation and environment variable setup steps before running this command.

```bash
python gemma_chat.py
```

--------------------------------

### Create Project Directory and Virtual Environment

Source: https://github.com/runpod/docs/blob/main/tutorials/flash/image-generation-with-sdxl.mdx

Set up a new directory for your project and initialize a Python virtual environment using uv.

```bash
mkdir flash-image-generation
cd flash-image-generation
```

--------------------------------

### Dockerfile for Runpod Serverless Worker

Source: https://github.com/runpod/docs/blob/main/serverless/quickstart.mdx

This Dockerfile sets up the environment for your serverless worker. It installs Python, the runpod library, copies your handler, and defines the command to start the container.

```dockerfile
FROM python:3.10-slim

WORKDIR / 

# Install dependencies
RUN pip install --no-cache-dir runpod

# Copy your handler file
COPY handler.py /

# Start the container
CMD ["python3", "-u", "handler.py"]
```

--------------------------------

### Start Axolotl Training

Source: https://github.com/runpod/docs/blob/main/fine-tune.mdx

Execute the command to begin the fine-tuning process using a configuration file.

```bash
axolotl train config.yaml
```

--------------------------------

### Expected PyTorch Rank Output

Source: https://github.com/runpod/docs/blob/main/instant-clusters/pytorch.mdx

This is an example of the output you should see after the PyTorch process starts on all Pods. It shows the global and local ranks for each process, indicating GPU assignments.

```csharp
Running on rank 8/15 (local rank: 0), device: cuda:0
Running on rank 15/15 (local rank: 7), device: cuda:7
Running on rank 9/15 (local rank: 1), device: cuda:1
Running on rank 12/15 (local rank: 4), device: cuda:4
Running on rank 13/15 (local rank: 5), device: cuda:5
Running on rank 11/15 (local rank: 3), device: cuda:3
Running on rank 14/15 (local rank: 6), device: cuda:6
Running on rank 10/15 (local rank: 2), device: cuda:2
```

--------------------------------

### Stream Text with Runpod

Source: https://github.com/runpod/docs/blob/main/public-endpoints/ai-sdk.mdx

Use `streamText` to get real-time output from Runpod models, suitable for chat interfaces. This example iterates through the text stream and prints each chunk.

```typescript
import { runpod } from "@runpod/ai-sdk-provider";
import { streamText } from "ai";

const { textStream } = await streamText({
  model: runpod("qwen3-32b-awq"),
  prompt: "Explain quantum computing in simple terms:",
  temperature: 0.7,
});

for await (const chunk of textStream) {
  process.stdout.write(chunk);
}
```

--------------------------------

### Example requirements.txt

Source: https://github.com/runpod/docs/blob/main/serverless/workers/create-dockerfile.mdx

Lists the Python dependencies required by your handler. Ensure versions are compatible.

```txt
# Example requirements.txt
runpod~=1.7.6
torch==2.0.1
pillow==9.5.0
transformers==4.30.2
```

--------------------------------

### Run Job and Get Status (JavaScript)

Source: https://github.com/runpod/docs/blob/main/serverless/endpoints/operation-reference.mdx

This JavaScript example demonstrates how to run a job using the RunPod SDK and then retrieve its status. It includes basic error handling.

```JavaScript
const { RUNPOD_API_KEY, ENDPOINT_ID } = process.env;
import runpodSdk from "runpod-sdk";

async function main() {
  try {
    const runpod = runpodSdk(RUNPOD_API_KEY);
    const endpoint = runpod.endpoint(ENDPOINT_ID);
    const result = await endpoint.run({
      input: {
        prompt: "Hello, World!",
      },
    });

    const { id } = result;
    if (!id) {
      console.error("No ID returned from endpoint.run");
      return;
    }

    const status = await endpoint.status(id);
    console.log(status);
  } catch (error) {
    console.error("An error occurred:", error);
  }
}

main();
```

--------------------------------

### Create project directory and virtual environment

Source: https://github.com/runpod/docs/blob/main/tutorials/flash/text-generation-with-transformers.mdx

Set up a new directory for your project and initialize a Python virtual environment. Ensure you activate the environment before installing dependencies.

```bash
mkdir flash-text-generation
cd flash-text-generation
```

```bash
uv venv
source .venv/bin/activate
uv pip install runpod-flash python-dotenv
```

--------------------------------

### Dockerfile for Load Balancer Worker

Source: https://github.com/runpod/docs/blob/main/serverless/load-balancing/build-a-worker.mdx

This Dockerfile sets up a Python environment with CUDA support, installs dependencies from requirements.txt, copies your application code, and defines the command to start the handler.

```docker
FROM nvidia/cuda:12.1.0-base-ubuntu22.04 

RUN apt-get update -y 
    && apt-get install -y python3-pip

RUN ldconfig /usr/local/cuda-12.1/compat/

# Install Python dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy application code
COPY app.py .

# Start the handler
CMD ["python3", "app.py"]
```

--------------------------------

### Create and Deploy Environments

Source: https://github.com/runpod/docs/blob/main/flash/cli/env.mdx

Use these commands to create development, staging, and production environments, and then deploy your application to each.

```bash
# Create environments
flash env create dev
flash env create staging
flash env create production

# Deploy to each
flash deploy --env dev
flash deploy --env staging
flash deploy --env production
```

--------------------------------

### Example Local Test Output

Source: https://github.com/runpod/docs/blob/main/serverless/development/aggregate-outputs.mdx

Observe this output to verify your handler is processing input correctly and producing the expected results. It shows the job starting, processing, and completing successfully.

```bash
-- Starting Serverless Worker |  Version 1.6.2 --
INFO   | Using test_input.json as job input.
DEBUG  | Retrieved local job: {'input': {'task_type': 'sentiment', 'items': ['Item 1', 'Item 2', 'Item 3']}, 'id': 'local_test'}
INFO   | local_test | Started.
DEBUG  | local_test | Handler output: ['Processed: Item 1', 'Processed: Item 2', 'Processed: Item 3']
INFO   | Job local_test completed successfully.
```

--------------------------------

### Output for Get Pod by ID

Source: https://github.com/runpod/docs/blob/main/sdks/graphql/manage-pods.mdx

This is an example JSON output when querying for a specific pod by its ID using the RunPod GraphQL API. It displays the requested details for the single pod.

```json
{
  "data": {
    "pod": {
      "id": "YOUR_POD_ID",
      "name": "Runpod Pytorch",
      "runtime": {
        "uptimeInSeconds": 11,
        "ports": [
          {
            "ip": "100.65.0.101",
            "isIpPublic": false,
            "privatePort": 8888,
            "publicPort": 60141,
            "type": "http"
          }
        ],
        "gpus": [
          {
            "id": "GPU-e0488b7e-6932-795b-a125-4472c16ea72c",
            "gpuUtilPercent": 0,
            "memoryUtilPercent": 0
          }
        ],
        "container": {
          "cpuPercent": 0,
          "memoryPercent": 0
        }
      }
    }
  }
}
```

--------------------------------

### Initialize a New Flash Project

Source: https://github.com/runpod/docs/blob/main/flash/cli/init.mdx

Create a new project directory with the specified name and install its dependencies.

```bash
flash init PROJECT_NAME
cd PROJECT_NAME
pip install -r requirements.txt
flash dev
```

--------------------------------

### Get SSH Connection Info for a Pod

Source: https://github.com/runpod/docs/blob/main/runpodctl/reference/runpodctl-ssh.mdx

Retrieve SSH connection details for a specific Pod. This command provides the necessary information to connect manually but does not start an interactive session.

```bash
runpodctl ssh info <pod-id>
```

--------------------------------

### Example SSH Connection Command

Source: https://github.com/runpod/docs/blob/main/pods/configuration/use-ssh.mdx

This is an example of the SSH command you will copy from your Pod's connect tab to establish an SSH connection. Replace placeholders with your actual Pod IP, SSH port, and private key path.

```bash
ssh root@213.173.108.12 -p 17445 -i ~/.ssh/id_ed25519
```

--------------------------------

### Remote Function Execution with @Endpoint

Source: https://github.com/runpod/docs/blob/main/flash/execution-model.mdx

This example demonstrates how to define a function for remote execution using the @Endpoint decorator and how to call it asynchronously from your local machine. Ensure the necessary libraries are installed.

```python
import asyncio
from runpod_flash import Endpoint, GpuType

@Endpoint(name="demo", gpu=GpuType.NVIDIA_GEFORCE_RTX_4090)
def process_on_gpu(data):
    # This runs on Runpod worker
    import torch
    return {"result": "processed"}

async def main():
    # This runs on your machine
    result = await process_on_gpu({"input": "data"})
    print(result)  # This runs on your machine

if __name__ == "__main__":
    asyncio.run(main())  # This runs on your machine
```

--------------------------------

### Configure Environment Variables

Source: https://github.com/runpod/docs/blob/main/tutorials/flash/build-rest-api-with-load-balancer.mdx

Copies the example environment file and sets the `RUNPOD_API_KEY`. Replace `YOUR_API_KEY` with your actual Runpod API key.

```bash
cp .env.example .env
echo "RUNPOD_API_KEY=YOUR_API_KEY" > .env
```

--------------------------------

### Get Endpoint Health with Python SDK

Source: https://github.com/runpod/docs/blob/main/serverless/endpoints/operation-reference.mdx

Check your endpoint's health using the RunPod Python SDK. This example requires the 'runpod' library and assumes RUNPOD_API_KEY and ENDPOINT_ID are set as environment variables.

```python
import runpod
import json
import os

runpod.api_key = os.getenv("RUNPOD_API_KEY")
endpoint = runpod.Endpoint(os.getenv("ENDPOINT_ID"))

endpoint_health = endpoint.health()
print(json.dumps(endpoint_health, indent=2))
```

--------------------------------

### Enter SSH Connection Details for Transfer

Source: https://github.com/runpod/docs/blob/main/community-solutions/ssh-password-migration/overview.mdx

Provides an example of entering the SSH connection details obtained from the destination Pod setup. This is required for the SCP migration script to establish a connection and transfer data.

```bash
Enter SSH command: ssh root@213.173.105.86 -p 17958
Enter password: [the password you set in Step 1]
```

--------------------------------

### Install rsync and generate SSH key on source Pod

Source: https://github.com/runpod/docs/blob/main/storage/network-volumes.mdx

Prepare the source Pod for `rsync` by installing `rsync` and generating an SSH key pair. Copy the public key for the destination Pod.

```bash
apt update && apt install -y rsync
ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519 -N "" -q
cat ~/.ssh/id_ed25519.pub
```

--------------------------------

### Start API Server with Auto-Provisioning

Source: https://github.com/runpod/docs/blob/main/flash/apps/build-app.mdx

Starts the local API server with auto-provisioning enabled, which deploys all resources upfront to eliminate cold-start delays. This is useful for development with multiple endpoints.

```bash
uv run flash dev --auto-provision
```

--------------------------------

### Generate Response from Ollama Model (Streaming)

Source: https://github.com/runpod/docs/blob/main/tutorials/pods/run-ollama.mdx

Send a POST request to the /api/generate endpoint to get a response from a specified model. This example uses 'llama2' and enables streaming responses by default. Replace OLLAMA_POD_ID with your actual Pod ID.

```bash
curl -X POST https://OLLAMA_POD_ID-11434.proxy.runpod.net/api/generate -d '{
  "model": "llama2",
  "prompt": "Tell me a story about llamas"
}'
```

--------------------------------

### Subsequent Runs Output Example

Source: https://github.com/runpod/docs/blob/main/tutorials/flash/text-generation-with-transformers.mdx

This output demonstrates the performance improvement on subsequent runs of the text generation script. It shows that the endpoint is reused, and the job completes much faster (2-5 seconds) as dependencies are already installed and the model is cached.

```text
Starting text generation on Runpod GPU...
Resource Endpoint_a1b2c3d4 already exists, reusing.
Registering RunPod endpoint at https://api.runpod.ai/xvf32dan8rcilp
Executing function on RunPod endpoint ID: xvf32dan8rcilp
Initial job status: IN_QUEUE
Job completed, output received

[Results appear immediately]
```

--------------------------------

### Start Development Server on Custom Port or Network

Source: https://github.com/runpod/docs/blob/main/flash/apps/local-testing.mdx

Customize the port or host for the development server. Use `uv run` if you are using uv for environment management.

```bash
flash dev --port 3000
```

```bash
flash dev --host 0.0.0.0
```

```bash
uv run flash dev --port 3000
```

```bash
uv run flash dev --host 0.0.0.0
```

--------------------------------

### Start Development Server with Auto-Provisioning

Source: https://github.com/runpod/docs/blob/main/flash/cli/dev.mdx

Starts the Flash development server and enables auto-provisioning for all Serverless endpoints. This eliminates cold-start delays during development by pre-provisioning resources.

```bash
flash dev --auto-provision
```

--------------------------------

### Get Endpoint Health with Go SDK

Source: https://github.com/runpod/docs/blob/main/serverless/endpoints/operation-reference.mdx

Check your endpoint's operational status using the RunPod Go SDK. This example initializes the SDK with your API key and endpoint ID, then fetches the health information. Ensure RUNPOD_API_KEY and ENDPOINT_ID are set in your environment.

```go
package main

import (
	"encoding/json"
	"fmt"
	"log"
	"os"

	"github.com/runpod/go-sdk/pkg/sdk/config"
	rpEndpoint "github.com/runpod/go-sdk/pkg/sdk/endpoint"
)

func main() {
	apiKey := os.Getenv("RUNPOD_API_KEY")
	endpointId := os.Getenv("ENDPOINT_ID")

	endpoint, err := rpEndpoint.New(
		&config.Config{ApiKey: &apiKey},
		&rpEndpoint.Option{EndpointId: &endpointId},
	)
	if err != nil {
		log.Fatalf("Failed to create endpoint: %v", err)
	}

	health, err := endpoint.Health()
	if err != nil {
		log.Fatalf("Failed to get health: %v", err)
	}

	data, _ := json.Marshal(health)
	fmt.Printf("Health: %s\n", data)
}
```

--------------------------------

### Install Runpod Flash with Pip

Source: https://github.com/runpod/docs/blob/main/flash/overview.mdx

Install the Runpod Flash SDK using pip. Ensure you have Python 3.10 or later installed.

```bash
# Install with pip
pip install runpod-flash
```

--------------------------------

### Create Entrypoint Script

Source: https://github.com/runpod/docs/blob/main/tutorials/introduction/containers/create-dockerfiles.mdx

Write a shell script that will be executed when a container starts. This script displays the container's start time and system information.

```sh
#!/bin/sh
echo "Container started at: $(date)"
echo "Running on: $(uname -a)"
```

--------------------------------

### Create Project Files

Source: https://github.com/runpod/docs/blob/main/serverless/quickstart.mdx

Initializes a new project directory and creates essential empty files for a serverless function.

```bash
mkdir serverless-quickstart && cd serverless-quickstart
touch handler.py Dockerfile requirements.txt test_input.json
```

--------------------------------

### Install Flash with uv

Source: https://github.com/runpod/docs/blob/main/flash/quickstart.mdx

Install the Runpod Flash package using uv within a virtual environment. Ensure you have Python and uv installed.

```bash
uv venv
source .venv/bin/activate
uv pip install runpod-flash
```

--------------------------------

### Install SkyPilot Runpod Cloud

Source: https://github.com/runpod/docs/blob/main/integrations/skypilot.mdx

Install the SkyPilot Runpod cloud integration. This command installs the nightly version of SkyPilot with Runpod support.

```sh
pip install "skypilot-nightly[runpod]"
```

--------------------------------

### Deploy with Environment Creation

Source: https://github.com/runpod/docs/blob/main/flash/cli/deploy.mdx

Use this command to create the app and a specific environment if they do not already exist.

```bash
# Creates the app and 'staging' environment if they don't exist
flash deploy --env staging
```

--------------------------------

### Create project directory and files

Source: https://github.com/runpod/docs/blob/main/serverless/load-balancing/vllm-worker.mdx

Sets up the necessary directory structure and empty files for the vLLM project. This includes Dockerfile, requirements.txt, and Python source files.

```bash
mkdir vllm_worker
cd vllm_worker
```

```bash
touch Dockerfile
touch requirements.txt
mkdir src
touch src/handler.py
touch src/models.py
touch src/utils.py
```

--------------------------------

### Install Runpod SDK for Python (Windows)

Source: https://github.com/runpod/docs/blob/main/serverless/sdks.mdx

Installs the Python SDK using pip within a virtual environment on Windows. Ensure you have Python installed.

```bash
python -m venv venv
venv\Scripts\activate
pip install runpod
```