### Install Example Policy Package

Source: https://github.com/skypilot-org/skypilot/blob/master/docs/source/cloud-setup/policy.rst

Clone the SkyPilot repository and install the example policy package using pip. This is a prerequisite for using example policies.

```bash
git clone https://github.com/skypilot-org/skypilot.git
cd skypilot
pip install examples/admin_policy/example_policy
```

--------------------------------

### SkyPilot YAML Setup Commands Example

Source: https://github.com/skypilot-org/skypilot/blob/master/agent/skills/skypilot/references/yaml-spec.md

Provides multi-line shell commands to execute during the setup phase on the remote cluster.

```yaml
setup: |
  echo "Begin setup."
  pip install -r requirements.txt
  echo "Setup complete."
```

--------------------------------

### SKYPILOT_SETUP_NODE_IPS Example

Source: https://github.com/skypilot-org/skypilot/blob/master/docs/source/running-jobs/environment-variables.rst

Example of the SKYPILOT_SETUP_NODE_IPS environment variable, which lists IP addresses of nodes in the cluster for the setup stage.

```text
1.2.3.4
3.4.5.6
```

--------------------------------

### Install SkyPilot and Copy Plugins

Source: https://github.com/skypilot-org/skypilot/blob/master/examples/plugin/README.md

Install the SkyPilot package and copy the example plugins configuration file to the user's SkyPilot configuration directory. Then, restart the SkyPilot API server.

```bash
pip install .
cp plugins.yaml ~/.sky/plugins.yaml
sky api stop; sky api start
```

--------------------------------

### One-Line Setup for Autonomous Code Optimization

Source: https://github.com/skypilot-org/skypilot/blob/master/examples/autonomous-code-optimization/README.md

This command automates the setup process by cloning the target repository and executing a setup script that installs SkyPilot and downloads necessary files. Ensure the TARGET_REPO environment variable is set.

```bash
export TARGET_REPO="https://github.com/<org>/<project>.git"
curl -fsSL https://raw.githubusercontent.com/skypilot-org/skypilot/master/examples/autonomous-code-optimization/setup.sh | bash
```

--------------------------------

### Install SkyPilot and Check Setup

Source: https://github.com/skypilot-org/skypilot/blob/master/llm/falcon/README.md

Installs the latest SkyPilot version and verifies cloud credential setup. Ensure you have the necessary cloud provider credentials configured before running.

```bash
pip install git+https://github.com/skypilot-org/skypilot.git
sky check
```

--------------------------------

### Specify Single Setup Command

Source: https://github.com/skypilot-org/skypilot/blob/master/docs/source/reference/yaml-spec.rst

Use the 'setup' field to define a single command to be executed on every 'sky launch' before the 'run' commands. This is useful for installing dependencies.

```yaml
setup: pip install -r requirements.txt
```

--------------------------------

### Start Policy Server

Source: https://github.com/skypilot-org/skypilot/blob/master/examples/admin_policy/example_server/README.md

Start the policy server with a specified policy. Ensure SkyPilot is installed and refer to the example_policy module for available policies.

```bash
python policy_server.py --policy DoNothingPolicy
```

--------------------------------

### Install SkyPilot Skill Generically

Source: https://github.com/skypilot-org/skypilot/blob/master/docs/source/getting-started/skill.rst

Instruct your agent to fetch and follow the installation guide for the SkyPilot skill. You may need to restart the agent.

```bash
Fetch and follow https://github.com/skypilot-org/skypilot/blob/HEAD/agent/INSTALL.md to install the SkyPilot skill
```

--------------------------------

### Setup Script (Bash)

Source: https://github.com/skypilot-org/skypilot/blob/master/docs/source/examples/agents/autonomous-code-optimization.md

A bash script to set up the environment for autonomous code optimization. It installs necessary dependencies and prepares the system for running experiments.

```bash
#!/bin/bash

# Update package list
sudo apt-get update -y

# Install Python and pip
sudo apt-get install -y python3 python3-pip

# Install SkyPilot
pip3 install skypilot

# Install other dependencies (e.g., PyTorch, TensorFlow)
pip3 install torch torchvision torchaudio
pip3 install tensorflow

# Clone the repository (if necessary)
# git clone <repository_url>

# Set up any required environment variables
# export MY_ENV_VAR="value"

echo "Environment setup complete."

```

--------------------------------

### SkyPilot Setup with Pip Install

Source: https://github.com/skypilot-org/skypilot/blob/master/docs/source/reference/slurm-migration.rst

Configuring environment dependencies in SkyPilot using the 'setup' command for pip installations. Supports installing specific versions and from requirements files.

```yaml
setup: |
  pip install torch==2.1.0
  pip install -r requirements.txt
```

--------------------------------

### Build and start the dashboard from source

Source: https://github.com/skypilot-org/skypilot/blob/master/docs/source/getting-started/quickstart.rst

Required steps to build the dashboard frontend when installing SkyPilot from source.

```console
# Install all dependencies
$ npm --prefix sky/dashboard install
# Build
$ npm --prefix sky/dashboard run build
```

--------------------------------

### Specify Multiline Setup Script

Source: https://github.com/skypilot-org/skypilot/blob/master/docs/source/reference/yaml-spec.rst

Define a multiline setup script using the '|' separator in the 'setup' field. This allows for sequential execution of multiple commands, such as environment setup and package installation.

```yaml
setup: |
  echo "Begin setup."
  pip install -r requirements.txt
  echo "Setup complete."
```

```yaml
setup: |
  conda create -n myenv python=3.9 -y
  conda activate myenv
  pip install torch torchvision
```

--------------------------------

### Server-Client Networking Example Output

Source: https://github.com/skypilot-org/skypilot/blob/master/examples/job-group-sdk/README.md

Example console output showing the logs for server and client tasks in a server-client Job Group. Demonstrates successful network setup and communication.

```console
$ sky jobs logs 182
Hint: This job has 2 tasks. Use 'sky jobs logs 182 TASK' to view logs for a specific task (TASK can be task ID or name).
=== Task server(0) ===
├── Waiting for task resources on 1 node.
└── Job started. Streaming logs... (Ctrl-C to exit log streaming; job will not be killed)
(server, pid=1365) [SkyPilot] Waiting for network setup...
(server, pid=1365) [SkyPilot] Hostname client-0.server-client is now resolvable
(server, pid=1365) [SkyPilot] Network is ready!
(server, pid=1365) Server starting on port 8080
(server, pid=1365) Serving HTTP on 0.0.0.0 port 8080 (http://0.0.0.0:8080/) ...
(server, pid=1365) 10.35.8.177 - - [26/Feb/2026 03:04:58] "GET / HTTP/1.1" 200 -
(server, pid=1365) Server done
✓ Task server(0) finished (status: SUCCEEDED).
=== Task client(1) ===
├── Waiting for task resources on 1 node.
└── Job started. Streaming logs... (Ctrl-C to exit log streaming; job will not be killed)
(client, pid=1381) [SkyPilot] Waiting for network setup...
(client, pid=1381) [SkyPilot] Hostname server-0.server-client is now resolvable
(client, pid=1381) [SkyPilot] Network is ready!
(client, pid=1381) Client starting
(client, pid=1381) Connecting to server-0.server-client:8080
(client, pid=1381) SUCCESS: Connected to server
✓ Task client(1) finished (status: SUCCEEDED).
✓ Job finished (status: SUCCEEDED).
```

--------------------------------

### Install SkyPilot via uv or pip

Source: https://github.com/skypilot-org/skypilot/blob/master/docs/source/getting-started/installation.rst

Standard installation methods for stable releases. Ensure pip is available in the environment for remote cluster setup.

```shell
# Create a virtual environment with pip pre-installed (required for SkyPilot)
# SkyPilot requires 3.9 <= python <= 3.13.
uv venv --seed --python 3.10
source .venv/bin/activate  # Use WSL on Windows
uv pip install skypilot

# install dependencies for the clouds you want to use
uv pip install "skypilot[kubernetes,aws,gcp]"
```

```shell
# Install as a globally available tool with pip included
# SkyPilot requires 3.9 <= python <= 3.13.
uv tool install --with pip skypilot

# install dependencies for the clouds you want to use
uv tool install --with pip "skypilot[kubernetes,aws,gcp]"
```

```shell
# Recommended: use a new conda env to avoid package conflicts.
# SkyPilot requires 3.7 <= python <= 3.13.
conda create -y -n sky python=3.10
conda activate sky

pip install skypilot

# install dependencies for the clouds you want to use
pip install "skypilot[kubernetes,aws,gcp]"
```

--------------------------------

### Setup and Query Gorilla Server

Source: https://github.com/skypilot-org/skypilot/blob/master/examples/serve/gorilla/use_gorilla.ipynb

Installs the OpenAI library and defines functions to query the Gorilla server and report issues. Ensure your sky_serve_endpoint is set before running.

```python
!pip install openai &> /dev/null
import openai
import urllib.parse

openai.api_key = "EMPTY" # Key is ignored and does not matter
openai.api_base = f"http://{sky_serve_endpoint}/v1"

# Report issues
def raise_issue(e, model, prompt):
    issue_title = urllib.parse.quote("[bug] Hosted Gorilla: <Issue>")
    issue_body = urllib.parse.quote(f"Exception: {e}\nFailed model: {model}, for prompt: {prompt}")
    issue_url = f"https://github.com/ShishirPatil/gorilla/issues/new?assignees=&labels=hosted-gorilla&projects=&template=hosted-gorilla-.md&title={issue_title}&body={issue_body}"
    print(f"An exception has occurred: {e} \nPlease raise an issue here: {issue_url}")

# Query Gorilla server
def get_gorilla_response(prompt="I would like to translate from English to French.", model="gorilla-falcon-7b-hf-v0"):
  try:
    completion = openai.ChatCompletion.create(
      model=model,
      messages=[{"role": "user", "content": prompt}]
    )
    return completion.choices[0].message.content
  except Exception as e:
    raise_issue(e, model, prompt)
```

--------------------------------

### Install SkyPilot from source

Source: https://github.com/skypilot-org/skypilot/blob/master/docs/source/getting-started/installation.rst

Cloning and installing the repository in editable mode.

```shell
# Recommended: use a new conda env to avoid package conflicts.
# SkyPilot requires 3.7 <= python <= 3.13.
conda create -y -n sky python=3.10
conda activate sky

git clone https://github.com/skypilot-org/skypilot.git
cd skypilot

pip install -e .
```

--------------------------------

### Task with Setup and Workdir CLI Commands

Source: https://github.com/skypilot-org/skypilot/blob/master/agent/skills/skypilot/references/examples.md

Launches a task with setup and workdir. 'sky launch' executes both setup and run, while 'sky exec' re-runs the run command with cached setup.

```bash
# Launch -- setup runs first, then the run command
sky launch -c train train-task.yaml

# Re-run with different args (setup is cached, only run executes)
sky exec train train-task.yaml
```

--------------------------------

### Detectron2 Application Example

Source: https://github.com/skypilot-org/skypilot/blob/master/examples/README.md

Example for running Detectron2 on a V100 GPU.

```yaml
# examples/detectron2_app.yaml
# Run Detectron2 on a V100 GPU.
```

--------------------------------

### Prepare Data and Install Dependencies

Source: https://github.com/skypilot-org/skypilot/blob/master/examples/autoresearch/README.md

Install project dependencies using uv and prepare the necessary data for the autoresearch experiments. Ensure you have uv installed or use another package manager as appropriate.

```bash
pip install uv && uv sync && uv run prepare.py
```

--------------------------------

### Detectron2 Docker Example

Source: https://github.com/skypilot-org/skypilot/blob/master/examples/README.md

Example demonstrating the use of Docker to run Detectron2 on GPUs.

```yaml
# examples/detectron2_docker.yaml
# Using Docker to run Detectron2 on GPUs.
```

--------------------------------

### Install Vast.ai SDK

Source: https://github.com/skypilot-org/skypilot/blob/master/docs/source/getting-started/installation.rst

Install the Vast.ai SDK and configure the API key directory.

```shell
pip install "vastai-sdk>=0.1.12"
mkdir -p ~/.config/vastai
echo "<your_api_key_here>" > ~/.config/vastai/vast_api_key
```

--------------------------------

### Install Prefect and SkyPilot

Source: https://github.com/skypilot-org/skypilot/blob/master/examples/prefect/README.md

Install the necessary libraries for Prefect and SkyPilot with Kubernetes support.

```bash
pip install prefect "skypilot[kubernetes]"
```

--------------------------------

### Multi-Hostname Command Execution Example

Source: https://github.com/skypilot-org/skypilot/blob/master/examples/README.md

Example for running a command on multiple nodes simultaneously.

```yaml
# examples/multi_hostname.yaml
# Run a command on multiple nodes.
```

--------------------------------

### Install Documentation Dependencies

Source: https://github.com/skypilot-org/skypilot/blob/master/docs/README.md

Installs the necessary dependencies for building the documentation, including Azure CLI and SkyPilot with all extras. It also installs specific requirements for the docs build.

```bash
# full setup matching CI in .github/workflows/test-doc-build.yml
uv pip install --prerelease=allow "azure-cli>=2.65.0,<2.87.0"
uv pip install ".[all]"
cd docs
uv pip install -r requirements-docs.txt
```

--------------------------------

### Install SkyPilot for Development

Source: https://github.com/skypilot-org/skypilot/blob/master/CONTRIBUTING.md

Clone your forked repository, set the upstream remote, and install SkyPilot in editable mode with all extras. Alternatively, install specific cloud support. Development dependencies are installed separately.

```bash
git clone https://github.com/<your-github-username>/skypilot.git
cd skypilot
git remote add upstream https://github.com/skypilot-org/skypilot.git
uv pip install -e ".[all]"
# Alternatively, install specific cloud support only:
# uv pip install -e ".[aws,azure,gcp,lambda]"
uv pip install -r requirements-dev.txt
```

--------------------------------

### Install SkyPilot for Lambda

Source: https://github.com/skypilot-org/skypilot/blob/master/docs/source/getting-started/installation.rst

Installation commands for SkyPilot with Lambda support.

```shell
# SkyPilot requires 3.7 <= python <= 3.13.
# From stable release
pip install "skypilot[lambda]"
# From nightly build
pip install "skypilot-nightly[lambda]"
# From source
pip install -e ".[lambda]"
```

--------------------------------

### Launch Example Application via YAML

Source: https://github.com/skypilot-org/skypilot/blob/master/examples/README.md

Launches an example application using a YAML configuration file. This is the recommended method for most use cases.

```bash
sky launch examples/<name>.yaml
```

--------------------------------

### Install Build Tools for Compilation in SkyPilot Setup

Source: https://github.com/skypilot-org/skypilot/blob/master/agent/skills/skypilot/references/troubleshooting.md

Ensure necessary build tools like `build-essential` and `ninja-build` are installed before compiling packages like `flash-attn` in SkyPilot's `setup`.

```yaml
setup: |
  # Ensure build tools are present
  sudo apt-get update && sudo apt-get install -y build-essential ninja-build
  pip install flash-attn --no-build-isolation
```

--------------------------------

### Launch Using Spot Instances

Source: https://github.com/skypilot-org/skypilot/blob/master/llm/llama-chatbots/README.md

Utilize spot instances for significant cost savings by adding the --use-spot flag.

```bash
sky launch llama-65b.yaml --use-spot <other args>
```

--------------------------------

### Install Ubuntu and SkyPilot on Azure VM

Source: https://github.com/skypilot-org/skypilot/blob/master/CONTRIBUTING.md

After connecting to the Azure VM via RDP, install Ubuntu using WSL and then install SkyPilot to test WSL-specific features.

```powershell
wsl --install -d Ubuntu-22.04
```

--------------------------------

### Finetune with Custom Dataset

Source: https://github.com/skypilot-org/skypilot/blob/master/llm/llama-3_1-finetuning/readme.md

Example command to initiate finetuning with a custom dataset. Replace `gbharti/finance-alpaca` with the Hugging Face path to your dataset.

```bash
# It takes about 1 hour on 8 A100 GPUs to finetune a 8B
```

--------------------------------

### Launch Example Application Programmatically

Source: https://github.com/skypilot-org/skypilot/blob/master/examples/README.md

Launches an example application using a Python script. This method offers more flexibility for programmatic control.

```bash
python examples/<name>.py
```

--------------------------------

### SkyPilot Serve YAML Configuration

Source: https://github.com/skypilot-org/skypilot/blob/master/llm/llama-3_1/README.md

Configure SkyPilot to deploy an AI model. This YAML defines environment variables, secrets, service replicas, resource requirements (accelerators, CPUs, disk), exposed ports, setup commands (pip installs), and run commands to start the model serving API and a Gradio web UI.

```yaml
envs:
  MODEL_NAME: meta-llama/Meta-Llama-3.1-8B-Instruct
secrets:
  HF_TOKEN: null # Pass with `--secret HF_TOKEN` in CLI

service:
  replicas: 2
  # An actual request for readiness probe.
  readiness_probe:
    path: /v1/chat/completions
    post_data:
      model: $MODEL_NAME
      messages:
        - role: user
          content: Hello! What is your name?
      max_tokens: 1

resources:
  accelerators: {L4:8, A10g:8, A10:8, A100:4, A100:8, A100-80GB:2, A100-80GB:4, A100-80GB:8}
  # accelerators: {L4, A10g, A10, L40, A40, A100, A100-80GB} # We can use cheaper accelerators for 8B model.
  cpus: 32+
  disk_size: 1000  # Ensure model checkpoints can fit.
  disk_tier: best
  ports: 8081  # Expose to internet traffic.

setup: |
  pip install vllm==0.5.3post1
  pip install vllm-flash-attn==2.5.9.post1
  # Install Gradio for web UI.
  pip install gradio openai

run: |
  echo 'Starting vllm api server...'

  vllm serve $MODEL_NAME \
    --tensor-parallel-size $SKYPILOT_NUM_GPUS_PER_NODE \
    --max-model-len 4096 \
    --port 8081 \
    2>&1 | tee api_server.log &

  while ! `cat api_server.log | grep -q 'Uvicorn running on'`; do
    echo 'Waiting for vllm api server to start...'
    sleep 5
  done

  echo 'Starting gradio server...'
  git clone https://github.com/vllm-project/vllm.git || true
  python vllm/examples/gradio_openai_chatbot_webserver.py \
    -m $MODEL_NAME \
    --port 8811 \
    --model-url http://localhost:8081/v1

```

--------------------------------

### SkyPilot YAML for DBRX Deployment

Source: https://github.com/skypilot-org/skypilot/blob/master/llm/dbrx/README.md

This YAML configuration defines the environment, secrets, resources, setup, and run commands for deploying the DBRX model using SkyPilot. It includes settings for model name, HuggingFace token, service replicas, readiness probes, resource allocation (accelerators, CPU, memory, disk), and ports. The setup section installs vLLM and other dependencies, while the run section starts the vLLM API server and a Gradio chatbot web server.

```yaml
envs:
  MODEL_NAME: databricks/dbrx-instruct
secrets:
  HF_TOKEN: null # Pass with `--secret HF_TOKEN` in CLI

service:
  replicas: 2
  # An actual request for readiness probe.
  readiness_probe:
    path: /v1/chat/completions
    post_data:
      model: $MODEL_NAME
      messages:
        - role: user
          content: Hello! What is your name?
      max_tokens: 1

resources:
  accelerators: {A100-80GB:8, A100-80GB:4, A100:8, A100:16}
  cpus: 32+
  memory: 512+
  use_spot: True
  disk_size: 512  # Ensure model checkpoints (~246GB) can fit.
  disk_tier: best
  ports: 8081  # Expose to internet traffic.

setup: |
  conda activate vllm
  if [ $? -ne 0 ]; then
    conda create -n vllm python=3.10 -y
    conda activate vllm
  fi

  # DBRX merged on master, 3/27/2024
  pip install git+https://github.com/vllm-project/vllm.git@e24336b5a772ab3aa6ad83527b880f9e5050ea2a

  pip install gradio tiktoken==0.6.0 openai

run: |
  conda activate vllm
  echo 'Starting vllm api server...'

  # https://github.com/vllm-project/vllm/issues/3098
  export PATH=$PATH:/sbin

  # NOTE: --gpu-memory-utilization 0.95 needed for 4-GPU nodes.
  python -u -m vllm.entrypoints.openai.api_server \
    --port 8081 \
    --model $MODEL_NAME \
    --trust-remote-code --tensor-parallel-size $SKYPILOT_NUM_GPUS_PER_NODE \
    --gpu-memory-utilization 0.95 \
    2>&1 | tee api_server.log &

  while ! `cat api_server.log | grep -q 'Uvicorn running on'`; do
    echo 'Waiting for vllm api server to start...'
    sleep 5
  done

  echo 'Starting gradio server...'
  git clone https://github.com/vllm-project/vllm.git || true
  python vllm/examples/gradio_openai_chatbot_webserver.py \
    -m $MODEL_NAME \
    --port 8811 \
    --model-url http://localhost:8081/v1

```

--------------------------------

### Initialize Project Directory

Source: https://github.com/skypilot-org/skypilot/blob/master/docs/source/getting-started/quickstart.rst

Create a local directory to house the project files.

```console
$ mkdir hello-sky
$ cd hello-sky
```

--------------------------------

### Install SkyPilot for Vast

Source: https://github.com/skypilot-org/skypilot/blob/master/docs/source/getting-started/installation.rst

Installation commands for SkyPilot with Vast support.

```shell
# SkyPilot requires 3.7 <= python <= 3.13.
# From stable release
uv pip install "skypilot[vast]"
# From nightly build
uv pip install "skypilot-nightly[vast]"
```

```shell
# SkyPilot requires 3.7 <= python <= 3.13.
# From stable release
uv tool install --with pip "skypilot[vast]"
# From nightly build
uv tool install --with pip "skypilot-nightly[vast]"
```

```shell
# SkyPilot requires 3.7 <= python <= 3.13.
# From stable release
pip install "skypilot[vast]"
# From nightly build
pip install "skypilot-nightly[vast]"
# From source
pip install -e ".[vast]"
```

--------------------------------

### Launch Qwen2-VL Model

Source: https://github.com/skypilot-org/skypilot/blob/master/llm/qwen/README.md

Use this command to start serving the Qwen2-VL model. Ensure you have the corresponding YAML configuration file.

```bash
sky launch -c qwen2-vl qwen2-vl-7b.yaml
```

--------------------------------

### Install OCI Dependencies

Source: https://github.com/skypilot-org/skypilot/blob/master/docs/source/getting-started/installation.rst

Install the OCI provider dependencies using uv, pip, or from source.

```shell
# SkyPilot requires 3.7 <= python <= 3.13.
# From stable release
uv pip install "skypilot[oci]"
# From nightly build
uv pip install "skypilot-nightly[oci]"
```

```shell
# SkyPilot requires 3.7 <= python <= 3.13.
# From stable release
uv tool install --with pip "skypilot[oci]"
# From nightly build
uv tool install --with pip "skypilot-nightly[oci]"
```

```shell
# SkyPilot requires 3.7 <= python <= 3.13.
# From stable release
pip install "skypilot[oci]"
# From nightly build
pip install "skypilot-nightly[oci]"
# From source
pip install -e ".[oci]"
```

--------------------------------

### SkyPilot Setup with Conda Environment

Source: https://github.com/skypilot-org/skypilot/blob/master/docs/source/reference/slurm-migration.rst

Setting up a Conda environment within SkyPilot using the 'setup' command. Includes creating the environment, activating it, and installing packages.

```yaml
setup: |
  conda create -n myenv python=3.10 -y
  conda activate myenv
  conda install pytorch pytorch-cuda=12.1 -c pytorch -c nvidia -y
```

--------------------------------

### File Mounts Example

Source: https://github.com/skypilot-org/skypilot/blob/master/examples/README.md

Example demonstrating how to use `file_mounts` to upload local or cloud paths to a cluster.

```yaml
# examples/using_file_mounts.yaml
# Using file_mounts to upload local/cloud paths to a cluster.
```

--------------------------------

### Pool Configuration Example

Source: https://github.com/skypilot-org/skypilot/blob/master/examples/deepseek-ocr/README.md

Example YAML defining a SkyPilot pool, specifying worker count, resources, file mounts, and setup commands to run on each worker.

```yaml
# pool.yaml
workers: 3
resources:
  accelerators: L40S:1

file_mounts:
  ~/.kaggle/kaggle.json: ~/.kaggle/kaggle.json
  /outputs:
    source: s3://my-skypilot-bucket

setup: |
  sudo apt-get update && sudo apt-get install -y unzip
  uv venv .venv --python 3.12
  source .venv/bin/activate
  git clone https://github.com/deepseek-ai/DeepSeek-OCR.git
  cd DeepSeek-OCR
  pip install kaggle
  uv pip install torch==2.6.0 torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
  uv pip install vllm==0.8.5
  uv pip install flash-attn==2.7.3 --no-build-isolation
  uv pip install -r requirements.txt
  cd ..
  kaggle datasets download goapgo/book-scan-ocr-vlm-finetuning
  unzip -q book-scan-ocr-vlm-finetuning.zip -d book-scan-ocr
  echo "Setup complete!"

```

--------------------------------

### User Configuration Example

Source: https://github.com/skypilot-org/skypilot/blob/master/docs/source/reference/config-sources.rst

Example of Kubernetes and AWS configurations defined in a user configuration file.

```yaml
kubernetes:
  allowed_contexts: [context1, context2]
  provision_timeout: 600
aws:
  labels:
    map-migrated: my-value
    Owner: user-unique-name
```

--------------------------------

### Create a Running Cluster Sample

Source: https://github.com/skypilot-org/skypilot/blob/master/tests/load_tests/db_scale_tests/README.md

Launch a cluster that will remain running to serve as a sample entry for scale testing.

```bash
# Launch a cluster that will remain running
sky launch --infra k8s -c scale-test-active -y "echo 'active cluster'"
```

--------------------------------

### SkyPilot YAML for Llama 3 Deployment

Source: https://github.com/skypilot-org/skypilot/blob/master/llm/llama-3/README.md

This YAML configuration defines the environment, services, resources, setup, and run commands for deploying Llama 3 with vLLM on a chosen cloud provider. It includes model name, secrets, replica count, readiness probe, resource allocation (accelerators, CPUs, spot instances, disk), and ports. The setup installs necessary Python packages and activates a Conda environment. The run command starts the vLLM API server and a Gradio-based chatbot web server.

```yaml
envs:
  MODEL_NAME: meta-llama/Meta-Llama-3-70B-Instruct
  # MODEL_NAME: meta-llama/Meta-Llama-3-8B-Instruct
secrets:
  HF_TOKEN: null # Pass with `--secret HF_TOKEN` in CLI

service:
  replicas: 2
  # An actual request for readiness probe.
  readiness_probe:
    path: /v1/chat/completions
    post_data:
      model: $MODEL_NAME
      messages:
        - role: user
          content: Hello! What is your name?
      max_tokens: 1

resources:
  accelerators: {L4:8, A10g:8, A10:8, A100:4, A100:8, A100-80GB:2, A100-80GB:4, A100-80GB:8}
  # accelerators: {L4, A10g, A10, L40, A40, A100, A100-80GB} # We can use cheaper accelerators for 8B model.
  cpus: 32+
  use_spot: True
  disk_size: 512  # Ensure model checkpoints can fit.
  disk_tier: best
  ports: 8081  # Expose to internet traffic.

setup: |
  conda activate vllm
  if [ $? -ne 0 ]; then
    conda create -n vllm python=3.10 -y
    conda activate vllm
  fi

  pip install vllm==0.4.2
  # Install Gradio for web UI.
  pip install gradio openai
  pip install flash-attn==2.5.9.post1


run: |
  conda activate vllm
  echo 'Starting vllm api server...'

  # https://github.com/vllm-project/vllm/issues/3098
  export PATH=$PATH:/sbin

  # NOTE: --gpu-memory-utilization 0.95 needed for 4-GPU nodes.
  python -u -m vllm.entrypoints.openai.api_server \
    --port 8081 \
    --model $MODEL_NAME \
    --trust-remote-code --tensor-parallel-size $SKYPILOT_NUM_GPUS_PER_NODE \
    --gpu-memory-utilization 0.95 \
    --max-num-seqs 64 \
    2>&1 | tee api_server.log &

  while ! `cat api_server.log | grep -q 'Uvicorn running on'`; do
    echo 'Waiting for vllm api server to start...'
    sleep 5
  done

  echo 'Starting gradio server...'
  git clone https://github.com/vllm-project/vllm.git || true
  python vllm/examples/gradio_openai_chatbot_webserver.py \
    -m $MODEL_NAME \
    --port 8811 \
    --model-url http://localhost:8081/v1 \
    --stop-token-ids 128009,128001

```

--------------------------------

### Install Nvidia GPU Driver on Ubuntu

Source: https://github.com/skypilot-org/skypilot/blob/master/docs/source/cloud-setup/cloud-permissions/vsphere.rst

Installs the recommended Nvidia GPU driver for Ubuntu systems. This command is an example and specific driver versions may vary based on hardware and OS.

```shell
sudo apt install nvidia-headless-535-server --no-install-recommends
```

--------------------------------

### Setup Kubeconfig and Check Sky

Source: https://github.com/skypilot-org/skypilot/blob/master/examples/nebius_infiniband/README.md

Configure kubectl to connect to your Nebius Kubernetes cluster and verify the SkyPilot installation.

```bash
nebius mk8s cluster get-credentials --id $NB_K8S_CLUSTER_ID --external
sky check k8s
```

--------------------------------

### Configure Dependencies with requirements.txt

Source: https://github.com/skypilot-org/skypilot/blob/master/examples/streamlit/README.md

Example of updating the SkyPilot YAML to install dependencies from a `requirements.txt` file using pip.

```yaml
setup: |
  pip install streamlit
  pip install -r requirements.txt
```

--------------------------------

### Example vSphere configuration

Source: https://github.com/skypilot-org/skypilot/blob/master/docs/source/getting-started/installation.rst

Example YAML structure for defining vCenter server connections and cluster access.

```yaml
vcenters:
  - name: <your_vsphere_server_ip_01>
    username: <your_vsphere_user_name>
    password: <your_vsphere_user_passwd>
    skip_verification: true # If your vcenter have valid certificate then change to 'false' here
    # Clusters that can be used by SkyPilot:
    #   [] means all the clusters in the vSphere can be used by Skypilot
    # Instead, you can specify the clusters in a list:
    # clusters:
    #   - name: <your_vsphere_cluster_name1>
    #   - name: <your_vsphere_cluster_name2>
    clusters: []
  # If you are configuring only one vSphere instance, omit the following line.
  - name: <your_vsphere_server_ip_02>
    username: <your_vsphere_user_name>
    password: <your_vsphere_user_passwd>
    skip_verification: true
    clusters: []
```

--------------------------------

### Verify Nvidia GPU Operator Installation on Kubernetes

Source: https://github.com/skypilot-org/skypilot/blob/master/docs/source/reference/kubernetes/kubernetes-setup.rst

Apply a test pod to verify the Nvidia GPU Operator installation and check if the 'nvidia' runtime is set as default. Monitor pod status to confirm successful setup.

```console
$ kubectl apply -f https://raw.githubusercontent.com/skypilot-org/skypilot/master/tests/kubernetes/gpu_test_pod.yaml
$ watch kubectl get pods
```

--------------------------------

### Pin Package Versions and Use Build Isolation in SkyPilot Setup

Source: https://github.com/skypilot-org/skypilot/blob/master/agent/skills/skypilot/references/troubleshooting.md

Manage Python package installations in SkyPilot's `setup` commands. Pin versions to avoid conflicts and use `--no-build-isolation` if compilation fails.

```yaml
setup: |
  # Pin versions to avoid resolution conflicts
  pip install torch==2.1.0 transformers==4.36.0

  # Use --no-build-isolation if compilation fails
  pip install flash-attn --no-build-isolation

  # If you hit "No space left on device" during pip install
  pip install --cache-dir /tmp/pip-cache mypackage
```

--------------------------------

### Run Serving Benchmark

Source: https://github.com/skypilot-org/skypilot/blob/master/examples/tpu/v6e/README.md

Execute this command to start a benchmark against the deployed serving instance. This helps in evaluating the model's serving performance.

```bash
$ sky exec serve-llama2-7b benchmark-llama2-7b.yaml
```

--------------------------------

### Start Ray Cluster with SkyPilot

Source: https://github.com/skypilot-org/skypilot/blob/master/examples/distributed_ray_train/README.md

Use this command within your task's 'run' commands to automatically set up a Ray cluster on SkyPilot. It installs Ray, starts head and worker nodes, and waits for cluster readiness.

```bash
run: |
  ~/sky_templates/ray/start_cluster
```

--------------------------------

### Execute Blue-Green Update Example

Source: https://github.com/skypilot-org/skypilot/blob/master/docs/source/serving/update.rst

Example command to initiate a blue-green update for the 'http-server' service using a specified task configuration file.

```console
$ sky serve update http-server --mode blue_green examples/serve/http_server/task.yaml
```

--------------------------------

### Full SkyPilot YAML Specification Example

Source: https://github.com/skypilot-org/skypilot/blob/master/agent/skills/skypilot/references/yaml-spec.md

This example demonstrates a comprehensive SkyPilot YAML configuration, showcasing various fields for resources, environments, secrets, volumes, file mounts, setup, run commands, and Kubernetes specific configurations.

```yaml
name: my-task

workdir: ~/my-task-code

num_nodes: 4

resources:
  # Infra to use. Click to see schema and example values.
  infra: aws

  # Hardware.
  accelerators: H100:8
  accelerator_args:
    runtime_version: tpu-vm-base
  cpus: 4+
  memory: 32+
  instance_type: p3.8xlarge
  use_spot: false
  disk_size: 256
  ephemeral_storage: 50
  disk_tier: medium
  network_tier: best
  max_hourly_cost: 10.0

  # Config.
  image_id: ami-0868a20f5a3bf9702
  ports: 8081
  labels:
    my-label: my-value
  autostop:
    idle_minutes: 10
    wait_for: none

  any_of:
    - infra: aws/us-west-2
      accelerators: H100
    - infra: gcp/us-central1
      accelerators: H100

  ordered:
    - infra: aws/us-east-1
    - infra: aws/us-west-2

  job_recovery: none

envs:
  MY_BUCKET: skypilot-temp-gcs-test
  MY_LOCAL_PATH: tmp-workdir
  MODEL_SIZE: 13b

secrets:
  MY_HF_TOKEN: my-secret-value
  WANDB_API_KEY: my-secret-value-2

api_server_access: true

volumes:
  /mnt/data: volume-name
  /mnt/cache:
    size: 100Gi

file_mounts:
  # Sync a local directory to a remote directory
  /remote/path: /local/path
  # Mount a S3 bucket to a remote directory
  /checkpoints:
    source: s3://existing-bucket
    mode: MOUNT
  # Mount with VFS caching and a pre-tuned workload type
  /data:
    source: s3://my-model-data
    mode: MOUNT_CACHED
    type: DATASET_RO
  /datasets-s3: s3://my-awesome-dataset

setup: |
  echo "Begin setup."
  pip install -r requirements.txt
  echo "Setup complete."

run: |
  echo "Begin run."
  python train.py
  echo Env var MODEL_SIZE has value: ${MODEL_SIZE}

config:
  kubernetes:
    provision_timeout: 600

```

--------------------------------

### Prepare Custom Docker Image for SkyPilot

Source: https://github.com/skypilot-org/skypilot/blob/master/docs/source/reference/kubernetes/kubernetes-getting-started.rst

Speed up pod startup times when using custom Docker images by pre-installing SkyPilot dependencies and system utilities. This Dockerfile example installs necessary packages, sets up Miniconda, installs SkyPilot and Ray, and configures kubectl.

```dockerfile
FROM <your base image>

# Install system dependencies
RUN apt update -y && \
    apt install git gcc rsync sudo patch openssh-server pciutils fuse unzip socat netcat-openbsd curl -y && \
    rm -rf /var/lib/apt/lists/*

# Install conda and other python dependencies
RUN curl https://repo.anaconda.com/miniconda/Miniconda3-py310_23.11.0-2-Linux-x86_64.sh -o Miniconda3-Linux-x86_64.sh && \
    bash Miniconda3-Linux-x86_64.sh -b && \
    eval "$(~/miniconda3/bin/conda shell.bash hook)" && conda init && conda config --set auto_activate_base true && conda activate base && \
    grep "# >>> conda initialize >>>" ~/.bashrc || { conda init && source ~/.bashrc; } && \
    rm Miniconda3-Linux-x86_64.sh && \
    export PIP_DISABLE_PIP_VERSION_CHECK=1 && \
    python3 -m venv ~/skypilot-runtime && \
    PYTHON_EXEC=$(echo ~/skypilot-runtime)/bin/python && \
    $PYTHON_EXEC -m pip install 'skypilot-nightly[remote,kubernetes]' 'ray[default]==2.9.3' 'pycryptodome==3.12.0' && \
    $PYTHON_EXEC -m pip uninstall skypilot-nightly -y && \
    curl -LO "https://dl.k8s.io/release/v1.28.11/bin/linux/amd64/kubectl" && \
    sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl && \
    echo 'export PATH="$PATH:$HOME/.local/bin"' >> ~/.bashrc
```

--------------------------------

### Task with Setup and Workdir YAML

Source: https://github.com/skypilot-org/skypilot/blob/master/agent/skills/skypilot/references/examples.md

Syncs local code and installs dependencies before running the main command. The 'workdir' directive specifies the directory to sync.

```yaml
# train-task.yaml
name: my-training

# Sync current directory to ~/sky_workdir on the remote machine
workdir: .

resources:
  accelerators: L4:1

# Setup runs once on cluster creation, cached on subsequent runs
setup: |
  pip install torch torchvision
  pip install -r requirements.txt

# Run executes your main command
run: |
  python train.py --epochs 10
```

--------------------------------

### Generate Nebius Service Account ID

Source: https://github.com/skypilot-org/skypilot/blob/master/docs/source/cloud-setup/cloud-permissions/nebius.rst

Use this command to get the service account ID from Nebius. Ensure the Nebius CLI is installed and configured.

```shell
export SA_ID=$(nebius iam service-account get-by-name \
  --name <service account name> \
  --format json \
  | jq -r ".metadata.id")
```

--------------------------------

### Launch Falcon Finetuning Job with Spot Instances

Source: https://github.com/skypilot-org/skypilot/blob/master/llm/falcon/README.md

Launches a finetuning job for the Falcon model using spot instances for cost efficiency. This command requires a `falcon.yaml` configuration file. Ensure `OUTPUT_BUCKET_NAME`, `WANDB_API_KEY`, and `MODEL_NAME` are set in the YAML.

```bash
sky jobs launch --use-spot -n falcon falcon.yaml
```

--------------------------------

### Launch LLaMA 65B Model with Meta URL

Source: https://github.com/skypilot-org/skypilot/blob/master/llm/llama-chatbots/README.md

Launch a cloud instance and download the 65B LLaMA model using an unexpired URL provided by Meta. This command sets up the environment and downloads the model weights, which can take up to 30 minutes.

```bash
export LLAMA_URL='https://'  # Add URL sent by Meta.
sky launch llama-65b.yaml -c llama -s --env LLAMA_URL=$LLAMA_URL
```

--------------------------------

### Define Pre-Deploy Hook for API Server

Source: https://github.com/skypilot-org/skypilot/blob/master/docs/source/reference/api-server/helm-values-spec.rst

Specify commands to execute before deploying the API server. This can be used for tasks like installing policies or other setup scripts.

```yaml
apiService:
  preDeployHook: |-
      # Run commands before deploying the API server, e.g. installing an admin
      # policy. Remember to set the admin policy in the config section below.
      echo "Pre-deploy hook"

      # Uncomment the following lines to install the admin policy
      # echo "Installing admin policy"
      # pip install git+https://github.com/michaelvll/admin-policy-examples
```

--------------------------------

### Interact with SkyServe Endpoint

Source: https://github.com/skypilot-org/skypilot/blob/master/llm/llama-3_2/README.md

Send a POST request to the SkyServe endpoint to get completions from the Llama-3 model. This example demonstrates sending a text and image prompt.

```bash
curl http://$ENDPOINT/v1/chat/completions \
    -H 'Content-Type: application/json' \
    -H 'Authorization: Bearer token' \
    --data '{ \
        "model": "meta-llama/Llama-3.2-11B-Vision-Instruct", \
        "messages": [ \
        { \
            "role": "user", \
            "content": [ \
                {"type" : "text", "text": "Covert this logo to ASCII art"}, \
                {"type": "image_url", "image_url": {"url": "https://pbs.twimg.com/profile_images/1584596138635632640/HWexMoH5_400x400.jpg"}} \
            ] \
        }], \
        "max_tokens": 2048 \
    }' | jq .
```

--------------------------------

### Install vLLM and HuggingFace Libraries

Source: https://github.com/skypilot-org/skypilot/blob/master/llm/llama-3_1/README.md

Install the necessary Python libraries, vLLM for serving and HuggingFace libraries for model access, inside the provisioned dev node. Ensure you have the correct version of vLLM.

```bash
pip install vllm==0.5.3.post1 huggingface
```

--------------------------------

### Single-GPU PyTorch Training YAML

Source: https://github.com/skypilot-org/skypilot/blob/master/agent/skills/skypilot/references/examples.md

A complete example for single-GPU PyTorch training, including local code synchronization and dependency installation. 'workdir' syncs the current directory.

```yaml
# single-gpu-train.yaml
name: pytorch-single-gpu

workdir: .

resources:
  accelerators: A100:1
  disk_size: 256

envs:
  NUM_EPOCHS: "10"
  BATCH_SIZE: "32"

setup: |
  pip install torch torchvision
  pip install wandb tqdm

run: |
  python train.py \
    --epochs $NUM_EPOCHS \
    --batch-size $BATCH_SIZE \
    --output-dir /tmp/checkpoints
```

--------------------------------

### Launch nanochat Training Pipeline

Source: https://github.com/skypilot-org/skypilot/blob/master/llm/nanochat/README.md

Use this command to provision resources, set up the environment, and run the nanochat training pipeline. Ensure your cloud provider is specified with the --infra flag. Trained checkpoints are saved to a specified S3 bucket.

```bash
sky launch -c nanochat-speedrun speedrun.sky.yaml --infra <k8s|aws|gcp|nebius|lambda|etc>
```

--------------------------------

### Serve Multiple Models with vLLM

Source: https://github.com/skypilot-org/skypilot/blob/master/agent/skills/skypilot/references/examples.md

Deploy multiple model services independently. This example shows serving a 70B Llama model requiring multi-GPU setup with tensor parallelism.

```yaml
# serve-llama-70b.yaml
name: llama-70b-service

resources:
  # 70B model needs multi-GPU with tensor parallelism
  accelerators: A100-80GB:4
  ports: 8000

envs:
  MODEL_NAME: meta-llama/Llama-3.1-70B-Instruct

secrets:
  HF_TOKEN: null

setup: |
  pip install vllm transformers
  huggingface-cli login --token $HF_TOKEN

run: |
  python -m vllm.entrypoints.openai.api_server \
    --model $MODEL_NAME \
    --tensor-parallel-size 4 \
    --host 0.0.0.0 \
    --port 8000

service:
  readiness_probe: /v1/models
  replicas: 2
```

```bash
# Deploy multiple model services
HF_TOKEN=hf_xxx sky serve up vllm-serve.yaml -n llama-8b --secret HF_TOKEN
HF_TOKEN=hf_xxx sky serve up serve-llama-70b.yaml -n llama-70b --secret HF_TOKEN

# Check all services
sky serve status

# Get endpoints
sky serve status llama-8b --endpoint
sky serve status llama-70b --endpoint
```

--------------------------------

### Launch Simple Axolotl Training

Source: https://github.com/skypilot-org/skypilot/blob/master/llm/axolotl/readme.md

Launches a simple Axolotl training job with a specified configuration. Ensure to replace 'xxx' with your actual token and '<unique-name>' with a unique bucket name if needed. The '-i30' flag indicates a 30-minute runtime, and '--down' automatically tears down the instance after completion.

```bash
HF_TOKEN=xxx sky launch -c axolotl axolotl.yaml --secret HF_TOKEN -y -i30 --down
ssh -L 8888:localhost:8888 axolotl
sky down axolotl -y
```

--------------------------------

### Slurm sbatch script example

Source: https://github.com/skypilot-org/skypilot/blob/master/docs/source/reference/slurm-migration.rst

This is a basic Slurm sbatch script template. It is intended to be a starting point for users migrating from Slurm to SkyPilot, demonstrating a typical script structure.

```bash
#!/bin/bash

```

--------------------------------

### Build and Serve Documentation with Live Reload

Source: https://github.com/skypilot-org/skypilot/blob/master/docs/README.md

Builds the documentation and serves it locally with live reload enabled, automatically updating the browser on changes. The --port argument specifies the port to use.

```bash
./build.sh --watch --port 8000
```