### Build SkyPilot Dashboard from Source (Bash)

Source: https://docs.skypilot.co/en/latest/getting-started/quickstart

Commands to build the SkyPilot dashboard when installing from source. This involves installing npm dependencies and running the build script for the dashboard application.

```bash
# Install all dependencies
$ npm --prefix sky/dashboard install
# Build
$ npm --prefix sky/dashboard run build

```

--------------------------------

### Install Kubernetes Dependencies (Linux)

Source: https://docs.skypilot.co/en/latest/reference/kubernetes/kubernetes-getting-started

Installs kubectl, socat, and netcat on Linux using apt-get. These utilities are necessary for SkyPilot's Kubernetes integration.

```shell
sudo apt-get install kubectl socat netcat
```

--------------------------------

### SkyPilot Setup Script for Ollama Installation

Source: https://docs.skypilot.co/en/latest/examples/serving/ollama

This script outlines the setup process for Ollama within a SkyPilot environment. It includes installing Ollama, starting the service, pulling a specified model, and ensuring the service is ready.

```bash
# Install Ollama
# official installation reference: https://ollama.com/download/linux
curl -fsSL https://ollama.com/install.sh | sh
sudo chmod +x /usr/local/bin/ollama

# Start `ollama serve` and capture PID to kill it after pull is done
ollama serve &
OLLAMA_PID=$!

# Wait for ollama to be ready
IS_READY=false
for i in {1..20};
  do ollama list && IS_READY=true && break;
  sleep 5;
done
if [ "$IS_READY" = false ]; then
    echo "Ollama was not ready after 100 seconds. Exiting."
    exit 1
fi

# Pull the model
ollama pull $MODEL_NAME
echo "Model $MODEL_NAME pulled successfully."

# Kill `ollama serve` after pull is done
kill $OLLAMA_PID
```

--------------------------------

### Multi-line Commands with textwrap.dedent in Python

Source: https://docs.skypilot.co/en/latest/getting-started/quickstart

Demonstrates using `textwrap.dedent` in Python for handling multi-line setup or run commands in SkyPilot tasks. This improves readability and maintainability of complex command sequences.

```python
import sky
import textwrap

commands = textwrap.dedent("""
for i in {1..5}; do
  echo "Hello, SkyPilot!"
done

conda env list
""")

task = sky.Task(run=commands)

```

--------------------------------

### Install Miniconda and Python Dependencies (Bash)

Source: https://docs.skypilot.co/en/latest/reference/kubernetes/kubernetes-getting-started

This script installs Miniconda, initializes conda, creates a Python virtual environment, installs SkyPilot and Ray, and installs kubectl. It manages dependencies and configures the environment for SkyPilot execution on Kubernetes. Ensure you have curl, bash, and sudo privileges.

```bash
RUN curl https://repo.anaconda.com/miniconda/Miniconda3-py310_23.11.0-2-Linux-x86_64.sh -o Miniconda3-Linux-x86_64.sh && \
    bash Miniconda3-Linux-x86_64.sh -b && \
    eval "$(~/miniconda3/bin/conda shell.bash hook)" && conda init && conda config --set auto_activate_base true && conda activate base && \
    grep "# >>> conda initialize >>>" ~/.bashrc || { conda init && source ~/.bashrc; } && \
    rm Miniconda3-Linux-x86_64.sh && \
    export PIP_DISABLE_PIP_VERSION_CHECK=1 && \
    python3 -m venv ~/skypilot-runtime && \
    PYTHON_EXEC=$(echo ~/skypilot-runtime)/bin/python && \
    $PYTHON_EXEC -m pip install 'skypilot-nightly[remote,kubernetes]' 'ray[default]==2.9.3' 'pycryptodome==3.12.0' && \
    $PYTHON_EXEC -m pip uninstall skypilot-nightly -y && \
    curl -LO "https://dl.k8s.io/release/v1.28.11/bin/linux/amd64/kubectl" && \
    sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl && \
    echo 'export PATH="$PATH:$HOME/.local/bin"' >> ~/.bashrc
```

--------------------------------

### Install SkyPilot and Check Setup

Source: https://docs.skypilot.co/en/latest/examples/applications/localgpt

Installs the SkyPilot library from its GitHub repository and verifies the cloud credential setup. This is a prerequisite for using SkyPilot.

```bash
pip install git+https://github.com/skypilot-org/skypilot.git
sky check
```

--------------------------------

### Launch Jupyter Lab with SkyPilot

Source: https://docs.skypilot.co/en/latest/examples/frameworks/jupyter

This configuration launches Jupyter Lab on a cloud instance managed by SkyPilot. It specifies the resources required, including the port to expose, and the setup command to install Jupyter. The run command starts the Jupyter Lab server, making it accessible via the exposed port.

```yaml
# Example: Launch Jupyter Lab and auto-expose its port to Internet.
#
# Usage:
#     $ sky launch -c jupyter jupyter_lab.yaml
#     # Then look for the logs for some output like:
#     # Jupyter Server 2.7.0 is running at:
#     #     http://127.0.0.1:29324/lab?token=<token>
#     # Run
#     $ sky status -a jupyter
#     # to get the HEAD_IP of the cluster, replace the 127.0.0.1 with
#     # the HEAD_IP and open browser for the URL.
#     
#     # This is an alternative to port forwarding.

resources:
  ports:
    - 29324

setup: pip install jupyter

run: jupyter lab --port 29324 --no-browser --ip=0.0.0.0

```

--------------------------------

### Install SkyPilot and Check Setup

Source: https://docs.skypilot.co/en/latest/examples/serving/vllm

Installs the latest version of SkyPilot and verifies the cloud credential setup. This is a prerequisite for using SkyPilot's features.

```bash
pip install git+https://github.com/skypilot-org/skypilot.git
sky check

```

--------------------------------

### Install Kubernetes Dependencies (macOS)

Source: https://docs.skypilot.co/en/latest/reference/kubernetes/kubernetes-getting-started

Installs kubectl, socat, and netcat on macOS using Homebrew. These tools are required for SkyPilot to interact with Kubernetes clusters.

```shell
brew install kubectl socat netcat
```

--------------------------------

### Install SkyPilot Dependencies in Custom Docker Image

Source: https://docs.skypilot.co/en/latest/reference/kubernetes/kubernetes-getting-started

This Dockerfile snippet demonstrates how to pre-install SkyPilot dependencies within a custom Docker image to expedite pod startup times. It includes commands to update package lists and install essential packages like `git`, `gcc`, `rsync`, `sudo`, `patch`, `openssh-server`, `pciutils`, `fuse`, `unzip`, `socat`, `netcat-openbsd`, and `curl`.

```dockerfile
FROM <your base image>

# Install system dependencies
RUN apt update -y && \
    apt install git gcc rsync sudo patch openssh-server pciutils fuse unzip socat netcat-openbsd curl -y && \
    rm -rf /var/lib/apt/lists/*

```

--------------------------------

### Install SkyPilot Example Admin Policies

Source: https://docs.skypilot.co/en/latest/cloud-setup/policy

This section provides instructions to clone the SkyPilot repository and install the example admin policy package. This allows users to test and utilize pre-defined admin policies for managing SkyPilot usage.

```bash
git clone https://github.com/skypilot-org/skypilot.git
cd skypilot
pip install examples/admin_policy/example_policy

```

--------------------------------

### Define PyTorch DDP Training with SkyPilot YAML

Source: https://docs.skypilot.co/en/latest/getting-started/tutorial

Defines the resource requirements, setup commands (cloning PyTorch examples, installing dependencies), and run commands (launching distributed training) for a minGPT DDP model using a SkyPilot YAML configuration. It specifies CPU and accelerator resources and optional file mounts or working directories.

```yaml
# train.yaml

name: minGPT-ddp

resources:
    cpus: 4+
    accelerators: L4:4  # Or A100:8, H100:8

# Optional: upload a working directory to remote ~/sky_workdir.
# Commands in "setup" and "run" will be executed under it.
#
# workdir: .

# Optional: upload local files.
# Format:
#   /remote/path: /local/path
#
# file_mounts:
#   ~/.vimrc: ~/.vimrc
#   ~/.netrc: ~/.netrc

setup: |
    git clone --depth 1 https://github.com/pytorch/examples || true
    cd examples
    git filter-branch --prune-empty --subdirectory-filter distributed/minGPT-ddp
    pip install -r requirements.txt

run: |
    cd examples/mingpt
    export LOGLEVEL=INFO

    echo "Starting minGPT-ddp training"

    torchrun \
    --nproc_per_node=$SKYPILOT_NUM_GPUS_PER_NODE \
    main.py


```

--------------------------------

### Install SkyPilot and Check Setup

Source: https://docs.skypilot.co/en/latest/examples/serving/sglang

Installs the latest SkyPilot nightly build with all extras and then checks the cloud credential setup. This is a prerequisite for using SkyServe.

```bash
pip install "skypilot-nightly[all]"
sky check
```

--------------------------------

### Setup TPU Environment for Llama 3 Training

Source: https://docs.skypilot.co/en/latest/examples/training/tpu

Installs necessary libraries and configures the environment for training Llama 3 on TPUs. This includes PyTorch, TorchXLA, and Hugging Face Transformers.

```bash
pip3 install huggingface_hub
python3 -c "import huggingface_hub; huggingface_hub.login('${HF_TOKEN}')"

# Setup TPU
pip3 install cloud-tpu-client
sudo apt update
sudo apt install -y libopenblas-base
pip3 install --pre torch==2.6.0.dev20240916+cpu torchvision==0.20.0.dev20240916+cpu \
    --index-url https://download.pytorch.org/whl/nightly/cpu
pip install "torch_xla[tpu]@https://storage.googleapis.com/pytorch-xla-releases/wheels/tpuvm/torch_xla-2.6.0.dev20240916-cp310-cp310-linux_x86_64.whl" \
    -f https://storage.googleapis.com/libtpu-releases/index.html
pip install torch_xla[pallas] \
    -f https://storage.googleapis.com/jax-releases/jax_nightly_releases.html \
    -f https://storage.googleapis.com/jax-releases/jaxlib_nightly_releases.html

# Setup runtime for training
git clone -b flash_attention https://github.com/pytorch-tpu/transformers.git
cd transformers
pip3 install -e .
pip3 install datasets evaluate scikit-learn accelerate
```

--------------------------------

### Initialize SkyPilot Task with Setup and Run Commands

Source: https://docs.skypilot.co/en/latest/reference/api

This Python code demonstrates how to initialize a SkyPilot Task, specifying setup commands (e.g., installing dependencies) and the main execution command. It highlights the use of the `setup` and `run` arguments for defining a task's lifecycle. The `workdir` argument specifies the local directory to sync.

```python
import sky

# A Task that will sync up local workdir '.', containing
# requirements.txt and train.py.
sky.Task(setup='pip install requirements.txt',
         run='python train.py',
         workdir='.')
```

--------------------------------

### Install SkyPilot

Source: https://docs.skypilot.co/en/latest/examples/models/deepseek-janus

Installs the SkyPilot framework with all optional dependencies. This is the initial setup step for using SkyPilot to manage AI workloads across different infrastructures.

```bash
pip install 'skypilot-nightly[all]'
```

--------------------------------

### Define and Launch SkyPilot Task in Python

Source: https://docs.skypilot.co/en/latest/getting-started/quickstart

Defines and launches a SkyPilot task programmatically using Python. This allows for dynamic task creation and integration within Python applications. It specifies resources, setup, and run commands, then launches the task, streaming logs.

```python
import sky
import sys

# List of commands to run
commands = [
    'echo "Hello, SkyPilot!"',
    'conda env list'
]

# Define a resource object.
#   infra: (Optional) if left out, automatically pick cheapest available.
#   accelerators: 8x NVIDIA A100 GPU
resource = sky.Resources(infra='aws', accelerators='A100:8')

# Define a task object.
#   setup: Typical use: pip install -r requirements.txt
#   run: Typical use: make use of resources, such as running training.
#   workdir: Working directory (optional) containing the project codebase.
#     Its contents are synced to ~/sky_workdir/ on the cluster.
#     Both `setup` and `run` is invoked under the workdir (i.e., can use its files).
task = sky.Task(setup='echo "Running setup."',
                run=commands,
                workdir='.',
                resources=resource)

# Launch the task.
request_id = sky.launch(task=task, cluster_name='mycluster')

# (Optional) stream the logs from the task to the console.
job_id, handle = sky.stream_and_get(request_id)
cluster_name = handle.get_cluster_name()
returncode = sky.tail_logs(cluster_name, job_id, follow=True)

sys.exit(returncode)

```

--------------------------------

### Setup and Compile llm.c for GPT-2 Training

Source: https://docs.skypilot.co/en/latest/examples/models/gpt-2

This snippet shows the setup process for training GPT-2 using llm.c. It includes installing cuDNN, MPI, NCCL, cloning the llm.c repository, and compiling the training executable with cuDNN support.

```yaml
name: gpt2-train

envs:
  BUCKET_NAME: # TODO: Fill in your bucket name
  BUCKET_STORE: s3 # Can be s3, gcs, or r2.

resources:
  accelerators: A100:8
  # Use docker image for latest version g++ to enable the compilation of llm.c.
  image_id: docker:nvidia/cuda:12.4.1-cudnn-devel-ubuntu22.04
  any_of:
    # Avoid using docker image for lambda due to the docker is not supported on
    # Lambda yet, but the base image works.
    - infra: lambda
      image_id: null
    - infra: aws
    - infra: gcp
    - infra: azure
    - infra: fluidstack
    - infra: kubernetes
  
file_mounts:
  ~/.cache/huggingface:
    name: $BUCKET_NAME
    store: $BUCKET_STORE
    mode: COPY

setup: |
  cd ~ 

  # install cudnn so we can use FlashAttention and run fast (optional)
  # https://developer.nvidia.com/cudnn-downloads
  # for me, CUDA 12 (run `nvcc --version`) running on Linux x86_64 Ubuntu 22.04
  if [ -f ./CUDNN_INSTALLED ]; then
    echo "cudnn already installed"
  else
    system=$(lsb_release -si | tr '[:upper:]' '[:lower:]')
    # Get version and remove the dot
    version=$(lsb_release -sr | tr -d .)
    export system_version="${system}${version}"
    wget https://developer.download.nvidia.com/compute/cudnn/9.1.1/local_installers/cudnn-local-repo-${system_version}-9.1.1_1.0-1_amd64.deb -O cudnn-installer.deb
    sudo dpkg -i cudnn-installer.deb
    sudo cp /var/cudnn-local-repo-${system_version}-9.1.1/cudnn-*-keyring.gpg /usr/share/keyrings/
    # Remove problematic kubernetes.list source
    sudo apt-get update --allow-releaseinfo-change || true

    sudo apt-get -y install cudnn-cuda-12

    touch ./CUDNN_INSTALLED
  fi

  # "install" cudnn-frontend to ~/
  sudo apt -y install git
  git clone https://github.com/NVIDIA/cudnn-frontend.git || true

  # install MPI (optional, if you intend to use multiple GPUs)
  # SkyPilot do not install MPI as that requires NCCL which needs to be manually
  # installed.
  sudo apt install -y openmpi-bin openmpi-doc libopenmpi-dev
  # install nccl
  pip install nvidia-nccl-cu12
  export LIBRARY_PATH=$LIBRARY_PATH:/usr/local/nccl2/lib
  export CPLUS_INCLUDE_PATH=$CPLUS_INCLUDE_PATH:/usr/local/nccl2/include

  git clone https://github.com/karpathy/llm.c.git || true
  cd llm.c
  ln -s ~/.cache/huggingface/fineweb10B dev/data/
  # compile llm.c (mixed precision, with cuDNN flash-attention)
  # first compilation is ~1 minute, mostly due to cuDNN
  make train_gpt2cu USE_CUDNN=1

```

--------------------------------

### Define SkyPilot Task in YAML

Source: https://docs.skypilot.co/en/latest/getting-started/quickstart

Defines a SkyPilot task using YAML format, specifying cloud resources, setup commands, and run commands. This is useful for declarative task definition and integration with CI/CD pipelines.

```yaml
resources:
  # Optional; if left out, automatically pick the cheapest cloud.
  infra: aws
  # 8x NVIDIA A100 GPU
  accelerators: A100:8

# Working directory (optional) containing the project codebase.
# Its contents are synced to ~/sky_workdir/ on the cluster.
workdir: .

# Typical use: pip install -r requirements.txt
# Invoked under the workdir (i.e., can use its files).
setup: |
  echo "Running setup."

# Typical use: make use of resources, such as running training.
# Invoked under the workdir (i.e., can use its files).
run: |
  echo "Hello, SkyPilot!"
  conda env list

```

--------------------------------

### Configure TPU Job Launch using SkyPilot YAML

Source: https://docs.skypilot.co/en/latest/examples/training/tpu

This snippet shows how to configure a TPU job using a SkyPilot YAML file. It defines the job name, working directory, and resource specifications, including the accelerator type. The `setup` section provides a multi-line shell script for environment setup, which includes checking for and creating a Conda environment, activating it, and installing Python dependencies.

```yaml
name: tpu_app

# The working directory contains all code and will be synced to remote.
workdir: ./examples/tpu/tpu_app_code

resources:
  accelerators: tpu-v2-8

# The setup command.  Will be run under the working directory.
setup: |
  pip install --upgrade pip

  conda activate huggingface
  if [ $? -eq 0 ]; then
    echo 'conda env exists'
  else
    conda create -n huggingface python=3.8 -y
    conda activate huggingface
    pip install -r requirements.txt
  fi

```

--------------------------------

### General DeepSeek R1 Setup (Shell)

Source: https://docs.skypilot.co/en/latest/examples/models/deepseek-r1

This script prepares the environment for running the DeepSeek R1 model by installing sglang and configuring shared memory. It is a more general setup script that can be used with various accelerators.

```shell
# Install sglang with all dependencies using uv
uv pip install "sglang[all]==0.4.2.post4" --find-links https://flashinfer.ai/whl/cu124/torch2.5/flashinfer

# Set up shared memory for better performance
sudo bash -c "echo 'vm.max_map_count=655300' >> /etc/sysctl.conf"
sudo sysctl -p
```

--------------------------------

### Ray Cluster Initialization and vLLM Server Setup (Shell Script)

Source: https://docs.skypilot.co/en/latest/examples/models/kimi-k2

This script configures and starts a Ray cluster across multiple nodes, then launches a vLLM API server on the head node. It handles node IP discovery, Ray head/worker startup, and vLLM server configuration including tensor and pipeline parallelism. Dependencies include `pip install blobfile`.

```shell
echo "Starting Ray..."
sudo chmod 777 -R /var/tmp
HEAD_IP=`echo "$SKYPILOT_NODE_IPS" | head -n1`
if [ "$SKYPILOT_NODE_RANK" == "0" ]; then
  ps aux | grep ray | grep 6379 &> /dev/null || ray start --head --disable-usage-stats --port 6379
  sleep 5
else
  sleep 5
  ps aux | grep ray | grep 6379 &> /dev/null || ray start --address $HEAD_IP:6379 --disable-usage-stats
  # Add sleep to after `ray start` to give ray enough time to daemonize
  sleep 5
fi

sleep 10
echo "Ray cluster started"
ray status

echo 'Starting vllm api server...'

# Set VLLM_HOST_IP to the IP of the current node based on rank
VLLM_HOST_IP=`echo "$SKYPILOT_NODE_IPS" | sed -n "$((SKYPILOT_NODE_RANK + 1))p"`
export VLLM_HOST_IP

# Only head node needs to start the vllm api server
if [ "$SKYPILOT_NODE_RANK" == "0" ]; then
  vllm serve $MODEL_NAME \
    --port 8081 \
    --tensor-parallel-size $SKYPILOT_NUM_GPUS_PER_NODE \
    --pipeline-parallel-size $SKYPILOT_NUM_NODES \
    --max-model-len 32768 \
    --trust-remote-code
else
  sleep infinity
fi
```

--------------------------------

### Install and Check SkyPilot

Source: https://docs.skypilot.co/en/latest/examples/models/pixtral

Installs the SkyPilot library with all optional dependencies and verifies the Kubernetes and cloud setup. This is the initial step before launching any models.

```bash
pip install 'skypilot[all]'
sky check
```

--------------------------------

### DVC Pipeline Configuration with SkyPilot

Source: https://docs.skypilot.co/en/latest/examples/frameworks/dvc

This snippet defines a DVC pipeline integrated with SkyPilot. It specifies SkyPilot resource requirements (e.g., accelerators, cloud provider), file mounts for SSH keys and git configuration, setup commands to install dependencies including DVC, and run commands to pull data, execute the DVC pipeline as an experiment, and push results.

```yaml
# adapted from https://alex000kim.com/posts/2023-08-10-ml-experiments-in-cloud-skypilot-dvc/
name: dvc-pipeline
resources:
  accelerators: T4:1
  infra: aws/us-east-2

workdir: .
file_mounts: 
  ~/.ssh/id_rsa: ~/.ssh/id_rsa
  ~/.ssh/id_rsa.pub: ~/.ssh/id_rsa.pub
  ~/.gitconfig: ~/.gitconfig
setup: |
  pip install -r requirements.txt
  pip install dvc[s3]
run: |
  # pull data versioned by DVC from DVC remote
  dvc pull 
  # run DVC pipeline as an experiment
  dvc exp run --pull --allow-missing
  # push experiment results to DVC remote
  dvc exp push origin 

```

--------------------------------

### SkyPilot Configuration for Serving Vector Database

Source: https://docs.skypilot.co/en/latest/examples/applications/vector_database

This SkyPilot YAML file defines the configuration for deploying and serving a vector database. It specifies resource requirements, including accelerators (GPUs), memory, and ports. File mounts are configured to access persistent data and images. The setup section installs necessary Python packages, and the run section executes the Python script to start the server. It also includes a service definition for replicas and a readiness probe.

```yaml
name: vectordb-serve

workdir: .

resources:
  accelerators: 
    # ordered by pricing (cheapest to most expensive)
    # skypilot will try to use the cheapest available accelerator
    # serve requires a GPU to compute the embeddings
    T4: 1
    L4: 1
    A10G: 1
    A10: 1
    V100: 1
  memory: 32+
  ports: 8000
  use_spot: true 

file_mounts:
  /vectordb:
    name: sky-vectordb
    # this needs to be the same as the source in the build_vectordb.yaml
    mode: MOUNT
  /images:
    name: sky-demo-image
    # this needs to be the same as the source in the build_vectordb.yaml
    mode: MOUNT

setup: |
  pip install numpy==1.26.4
  pip install torch==2.5.1 torchvision==0.20.1 ftfy regex tqdm
  pip install open_clip_torch chromadb pandas
  pip install fastapi uvicorn pydantic

run: |
  python scripts/serve_vectordb.py \
    --collection-name clip_embeddings \
    --persist-dir /vectordb/chroma \
    --images-dir /images \
    --host 0.0.0.0 \
    --port 8000 

service:
  replicas: 1
  readiness_probe:
    path: /health

```

--------------------------------

### Setup vLLM environment and install dependencies

Source: https://docs.skypilot.co/en/latest/examples/serving/vllm

This script sets up a Conda environment named 'vllm' with Python 3.10, installs specified versions of the 'transformers' and 'vllm' libraries, and logs into Hugging Face using a provided token.

```bash
conda activate vllm
if [ $? -ne 0 ]; then
  conda create -n vllm python=3.10 -y
  conda activate vllm
fi

pip install transformers==4.38.0
pip install vllm==0.3.2

python -c "import huggingface_hub; huggingface_hub.login('${HF_TOKEN}')"
```

--------------------------------

### Helm Upgrade Command Example

Source: https://docs.skypilot.co/en/latest/reference/api-server/api-server-admin-deploy

A general example of a Helm upgrade command for SkyPilot, demonstrating how to set various parameters like `--namespace`, `--reuse-values`, and specific `--set` options for cloud credentials. This command is used for initial installation or updating existing configurations.

```bash
helm upgrade --install skypilot skypilot/skypilot-nightly --devel \
    --namespace $NAMESPACE \
    --reuse-values \
    --set runpodCredentials.enabled=true \
    --set runpodCredentials.runpodSecretName=your_secret_name
```

--------------------------------

### Setup Verl Repository and Virtual Environment (Shell)

Source: https://docs.skypilot.co/en/latest/examples/training/verl

This script clones the Verl repository, creates a Python virtual environment using `uv`, and installs Verl and its dependencies. It specifically skips the Megatron dependency and installs Ray for distributed computing.

```shell
rm -rf verl
git clone https://github.com/volcengine/verl.git
cd verl

# Create virtual environment and install dependencies
uv venv --seed
source .venv/bin/activate

# Install Verl and its dependencies (skip Megatron for this example)
USE_MEGATRON=0 bash scripts/install_vllm_sglang_mcore.sh
uv pip install --no-deps -e .
uv pip install "ray[default]"  # For Ray dashboard
```

--------------------------------

### Launch 100 Jobs using SkyPilot CLI with Spot Instances

Source: https://docs.skypilot.co/en/latest/getting-started/quickstart

Launches 100 jobs concurrently using the `sky jobs launch` command. It utilizes spot instances (`--use-spot`), detaches the run (`--detach-run`), runs asynchronously (`--async`), automatically confirms (`--yes`), and names each job sequentially. This demonstrates a basic pattern for scaling job submissions.

```bash
$ for i in $(seq 100) # launch 100 jobs
    do sky jobs launch --use-spot --detach-run --async --yes -n hello-$i hello_sky.yaml
  done
```

--------------------------------

### Launch 100 Jobs using SkyPilot Python API with Spot Instances

Source: https://docs.skypilot.co/en/latest/getting-started/quickstart

Launches 100 jobs programmatically using the SkyPilot Python API. Each job is configured to use spot instances and runs an echo command. The `sky.jobs.launch` function is used to submit tasks asynchronously, showcasing how to manage scaled workloads within a Python script.

```python
import sky
from sky import jobs as managed_jobs

for i in range(100):
    resource = sky.Resources(use_spot=True)
    task = sky.Task(resources=resource, run='echo "Hello, SkyPilot!"')
    managed_jobs.launch(task, name=f'hello-{i}')
```

--------------------------------

### SkyPilot YAML Configuration for gpt-oss-120b Full Finetuning

Source: https://docs.skypilot.co/en/latest/examples/training/gpt-oss-finetuning

Defines the infrastructure and setup steps for launching a full finetuning job for the gpt-oss-120b model using SkyPilot. It specifies resource requirements (4 nodes, 8x H200 GPUs each), file mounts, and a detailed setup script for environment configuration and dependency installation.

```yaml
# gpt-oss-120b-sft.yaml
resources:
  accelerators: H200:8
  network_tier: best

file_mounts:
  /sft: ./sft

num_nodes: 4

setup: |
  conda install cuda -c nvidia
  uv venv ~/training --seed --python 3.10
  source ~/training/bin/activate
  uv pip install torch --index-url https://download.pytorch.org/whl/cu128
  uv pip install "trl>=0.20.0" "peft>=0.17.0" "transformers>=4.55.0"
  uv pip install deepspeed
  uv pip install git+https://github.com/huggingface/accelerate.git@c0a3aefea8aa5008a0fbf55b049bd3f0efa9cbf2

  uv pip install nvitop

run: |
  source ~/training/bin/activate

  MASTER_ADDR=$(echo "$SKYPILOT_NODE_IPS" | head -n1)
  NP=$(($SKYPILOT_NUM_GPUS_PER_NODE * $SKYPILOT_NUM_NODES))

  accelerate launch \
    --config_file /sft/fsdp2_120b.yaml \
    --num_machines $SKYPILOT_NUM_NODES \
    --num_processes $NP \
    --machine_rank $SKYPILOT_NODE_RANK \
    --main_process_ip $MASTER_ADDR \
    --main_process_port 29500 \
    /sft/train.py --model_id openai/gpt-oss-120b

```

--------------------------------

### Serve Llama2 7b with TPU v6e Setup

Source: https://docs.skypilot.co/en/latest/examples/training/tpu

YAML configuration for setting up and serving the Llama2 7b model on TPU v6e. Includes resource allocation, environment variables for model and checkpoint paths, secrets management for Hugging Face token, and detailed setup steps for necessary libraries like PyTorch (XLA), JetStream, and JAX.

```yaml
resources:
  accelerators: tpu-v6e-8 # Fill in the accelerator type you want to use

envs:
  HF_REPO_ID: meta-llama/Llama-2-7b
  model_name: llama-2
  input_ckpt_dir: /home/gcpuser/sky_workdir/ckpt/llama2-7b/original
  output_ckpt_dir: /home/gcpuser/sky_workdir/ckpt/llama2-7b/converted
  tokenizer_path: /home/gcpuser/sky_workdir/ckpt/llama2-7b/original/tokenizer.model
secrets:
  HF_TOKEN: null # Pass with `--secret HF_TOKEN` in CLI

setup: |
  pip3 install huggingface_hub
  python3 -c "import huggingface_hub; huggingface_hub.login('${HF_TOKEN}')"

  # Setup TPU
  pip3 install cloud-tpu-client
  sudo apt update
  sudo apt install -y libopenblas-base
  pip3 install --pre torch==2.6.0.dev20240916+cpu torchvision==0.20.0.dev20240916+cpu \
    --index-url https://download.pytorch.org/whl/nightly/cpu
  pip install "torch_xla[tpu]@https://storage.googleapis.com/pytorch-xla-releases/wheels/tpuvm/torch_xla-2.6.0.dev20240916-cp310-cp310-linux_x86_64.whl" \
    -f https://storage.googleapis.com/libtpu-releases/index.html
  pip install torch_xla[pallas] \
    -f https://storage.googleapis.com/jax-releases/jax_nightly_releases.html \
    -f https://storage.googleapis.com/jax-releases/jaxlib_nightly_releases.html

  # Setup runtime for serving
  git clone https://github.com/google/JetStream.git
  cd JetStream
  git checkout main
  git pull origin main
  pip install -e .
  cd benchmarks
  pip install -r requirements.in
  cd ../..
  git clone https://github.com/google/jetstream-pytorch.git
  cd jetstream-pytorch/
  git checkout jetstream-v0.2.3
  source install_everything.sh
  pip3 install -U --pre jax jaxlib libtpu-nightly requests \
    -f https://storage.googleapis.com/jax-releases/jax_nightly_releases.html \
    -f https://storage.googleapis.com/jax-releases/libtpu_releases.html


  # Prepare checkpoint, inside jetstream-pytorch repo
  mkdir -p ${input_ckpt_dir}
  python3 -c "import huggingface_hub; huggingface_hub.snapshot_download('${HF_REPO_ID}', local_dir='${input_ckpt_dir}')"
  mkdir -p ${output_ckpt_dir}

```

--------------------------------

### Define PyTorch DDP Training with SkyPilot Python SDK

Source: https://docs.skypilot.co/en/latest/getting-started/tutorial

Creates a SkyPilot task object in Python to configure and launch a minGPT DDP training job. It programmatically defines resources, setup commands (cloning PyTorch examples, installing dependencies), and run commands (launching distributed training), similar to the YAML approach.

```python
# train.py

import sky

minGPT_ddp_task = sky.Task(
    name='minGPT-ddp',
    resources=sky.Resources(
        cpus='4+',
        accelerators='L4:4',
    ),
    # Optional: upload a working directory to remote ~/sky_workdir.
    # Commands in "setup" and "run" will be executed under it.
    #
    # workdir='.',
    #
    # Optional: upload local files.
    # Format:
    #   /remote/path: /local/path
    #
    # file_mounts={
    #     '~/.vimrc': '~/.vimrc',
    #     '~/.netrc': '~/.netrc',
    # },
    setup=[
        'git clone --depth 1 https://github.com/pytorch/examples || true',
        'cd examples',
        'git filter-branch --prune-empty --subdirectory-filter distributed/minGPT-ddp',
        'pip install -r requirements.txt',
    ],
    run=[
        'cd examples/mingpt',
        'export LOGLEVEL=INFO',
        'torchrun --nproc_per_node=$SKYPILOT_NUM_GPUS_PER_NODE main.py',
    ]
)

cluster_name = 'mingpt'
launch_request = sky.launch(task=minGPT_ddp_task, cluster_name=cluster_name)
job_id, _ = sky.stream_and_get(launch_request)
sky.tail_logs(cluster_name, job_id, follow=True)


```

--------------------------------

### Add Example to SkyPilot Documentation

Source: https://docs.skypilot.co/en/latest/examples/spot/resnet

Instructions for adding a new example to the SkyPilot documentation. This involves forking the repository, creating a new directory for the example, adding necessary files, and linking the example to the documentation structure.

```shell
1. Fork the SkyPilot repository on GitHub.
2. Create a new folder under llm/ or examples/.
   * For example, call it `my-llm/`.
   * Add a README.md, a SkyPilot YAML, and other necessary files to run the AI project.
   * `git add` the files.

4. Go to the appropriate subdir under Examples and add your example.
   * For example, if you want to add to “Examples / Serving”:
   * `cd docs/source/examples/serving; ln -s ../../generated-examples/my-llm.md`.
   * Add a heading for your example to the TOC in the index.rst file.
   * `git add index.rst my-llm.md`
5. Build the docs as usual.
```

--------------------------------

### Verify GPU Setup with kubectl

Source: https://docs.skypilot.co/en/latest/reference/kubernetes/kubernetes-deployment

Checks if GPU drivers are correctly installed and recognized by Kubernetes by inspecting node capacity for nvidia.com/gpu resources.

```shell
kubectl describe nodes
```

--------------------------------

### Setup and Run MNIST with Flax on TPU

Source: https://docs.skypilot.co/en/latest/examples/training/tpu

This snippet sets up a Conda environment for Flax and runs an MNIST example on a TPU. It clones the Flax repository, creates a Conda environment, installs JAX with TPU support, and then executes the MNIST training script.

```yaml
name: tpuvm_mnist

resources:
  accelerators: tpu-v2-8

# The setup command.  Will be run under the working directory.
setup: |
  git clone https://github.com/google/flax.git --branch v0.10.1

  conda activate flax
  if [ $? -eq 0 ]; then
    echo 'conda env exists'
  else
    conda create -n flax python=3.10 -y
    conda activate flax
    # Make sure to install TPU related packages in a conda env to avoid package conflicts.
    pip install uv
    uv pip install --prerelease=allow \
      -f https://storage.googleapis.com/jax-releases/libtpu_releases.html -f https://storage.googleapis.com/jax-releases/jax_releases.html "jax[tpu]==0.4.35" \
      clu \
      tensorflow tensorflow-datasets
    uv pip install -e flax
    uv pip install --upgrade tensorflow-datasets
    uv pip install --upgrade protobuf
  fi


# The command to run.  Will be run under the working directory.
run: |
  conda activate flax
  cd flax/examples/mnist
  python3 main.py --workdir=/tmp/mnist \
    --config=configs/default.py \
    --config.learning_rate=0.05 \
    --config.num_epochs=10
```

--------------------------------

### Install Admin Policy with Helm Chart

Source: https://docs.skypilot.co/en/latest/reference/api-server/api-server-admin-deploy

Configures the Helm chart to install an admin policy before the API server starts by setting `apiService.preDeployHook`. This involves specifying commands to run, such as installing a policy from a Git repository, and defining the admin policy configuration. The example uses a `values.yaml` file and the `helm upgrade` command.

```yaml
# values.yaml
apiService:
  preDeployHook: |
    echo "Installing admin policy"
    pip install git+https://github.com/michaelvll/admin-policy-examples

  config: |
    admin_policy: admin_policy_examples.AddLabelsPolicy

```

```bash
helm upgrade --install skypilot skypilot/skypilot-nightly --devel -f values.yaml

```

--------------------------------

### View All Clusters (CLI)

Source: https://docs.skypilot.co/en/latest/getting-started/quickstart

Display a table of all existing SkyPilot clusters across different regions and clouds using the `sky status` command. The output includes cluster name, infrastructure provider, resources, status, autostop configuration, and launch time.

```bash
$ sky status

```

--------------------------------

### Add a New Example to SkyPilot Documentation

Source: https://docs.skypilot.co/en/latest/examples/index

Instructions for contributing a new example to SkyPilot. This involves forking the repository, creating a new directory for the example, adding necessary files (README.md, SkyPilot YAML), and linking it into the documentation's table of contents.

```bash
cd docs/source/examples/serving; ln -s ../../generated-examples/my-llm.md
```

--------------------------------

### Manage Kubernetes Context

Source: https://docs.skypilot.co/en/latest/reference/kubernetes/kubernetes-getting-started

Provides commands to view, switch, and set the current Kubernetes context and namespace. This ensures SkyPilot operates within the intended cluster and namespace.

```shell
# See current context
$ kubectl config current-context

# Switch current-context
$ kubectl config use-context mycontext

# Set a specific namespace to be used in the current-context
$ kubectl config set-context --current --namespace=mynamespace
```

--------------------------------

### Specify TPU Accelerators for SkyPilot Tasks

Source: https://docs.skypilot.co/en/latest/reference/kubernetes/kubernetes-getting-started

This code example shows how to configure SkyPilot to utilize TPUs on GKE for single-host TPU topologies. By specifying the accelerator type and count in the `resources.accelerators` field of your task YAML, you can request specific TPU resources.

```yaml
resources:
  accelerators: tpu-v5-lite-podslice:1  # or tpu-v5-lite-device, tpu-v5p-slice

```

--------------------------------

### Install SkyPilot with Cloud Support

Source: https://docs.skypilot.co/en/latest/examples/models/gpt-2

Installs the SkyPilot package, including optional support for various cloud providers. Choose the clouds you intend to use by appending their names to the install command.

```bash
pip install "skypilot-nightly[aws,gcp,azure,kubernetes,lambda,fluidstack]"
```

--------------------------------

### SkyPilot Start Cluster Command

Source: https://docs.skypilot.co/en/latest/reference/cli

Command to restart stopped or failed SkyPilot clusters. If a cluster failed during provisioning or installation, this command retries those steps. Clusters already in the 'UP' state are unaffected. Restarting uses the same cloud, region, and zone as the original setup.

```CLI
sky start [OPTIONS] [CLUSTERS]...
```

--------------------------------

### LoRA Finetuning for Meta Llama-3.1 (YAML)

Source: https://docs.skypilot.co/en/latest/examples/training/llama-3_1-finetuning

Configuration file for LoRA finetuning of Meta Llama-3.1 models. It specifies environment variables for model size and dataset, secrets for authentication, resource requirements like A100 GPUs, and mounts for configurations and output checkpoints. The setup section installs necessary libraries and downloads the model, while the run section executes the finetuning process and saves the adapter files.

```yaml
# LoRA finetuning Meta Llama-3.1 on any of your own infra.
#
# Usage:
#
#  HF_TOKEN=xxx sky launch lora.yaml -c llama31 --secret HF_TOKEN
#
# To finetune a 70B model:  
#
#  HF_TOKEN=xxx sky launch lora.yaml -c llama31-70 --secret HF_TOKEN --env MODEL_SIZE=70B

envs:
  MODEL_SIZE: 8B
  DATASET: "yahma/alpaca-cleaned"
  # Change this to your own checkpoint bucket
  CHECKPOINT_BUCKET_NAME: sky-llama-31-checkpoints
secrets:
  HF_TOKEN: null # Pass with `--secret HF_TOKEN` in CLI

resources:
  accelerators: A100:8
  disk_tier: best
  use_spot: true

file_mounts:
  /configs: ./configs
  /output:
    name: $CHECKPOINT_BUCKET_NAME
    mode: MOUNT
    # Optionally, specify the store to enforce to use one of the stores below:
    #   r2/azure/gcs/s3/cos
    # store: r2

setup: |
  pip install torch==2.4.0 torchvision

  # Install torch tune from source for the latest Llama-3.1 model
  pip install git+https://github.com/pytorch/torchtune.git@58255001bd0b1e3a81a6302201024e472af05379
  # pip install torchtune
  
  tune download meta-llama/Meta-Llama-3.1-${MODEL_SIZE}-Instruct \
    --hf-token $HF_TOKEN \
    --output-dir /tmp/Meta-Llama-3.1-${MODEL_SIZE}-Instruct \
    --ignore-patterns "original/consolidated*"

run: |
  tune run --nproc_per_node $SKYPILOT_NUM_GPUS_PER_NODE \
    lora_finetune_distributed \
    --config /configs/${MODEL_SIZE}-lora.yaml \
    dataset.source=$DATASET

  # Remove the checkpoint files to save space, LoRA serving only needs the
  # adapter files.
  rm /tmp/Meta-Llama-3.1-${MODEL_SIZE}-Instruct/*.pt
  rm /tmp/Meta-Llama-3.1-${MODEL_SIZE}-Instruct/*.safetensors
  
  mkdir -p /output/$MODEL_SIZE-lora
  rsync -Pavz /tmp/Meta-Llama-3.1-${MODEL_SIZE}-Instruct /output/$MODEL_SIZE-lora
  cp -r /tmp/lora_finetune_output /output/$MODEL_SIZE-lora/ 

```

--------------------------------

### Start Gradio Chatbot Webserver with vLLM

Source: https://docs.skypilot.co/en/latest/examples/models/llama-3_1

Launches a Gradio-based chatbot web server that connects to a vLLM API endpoint. This setup allows for interactive chat with the deployed model through a web UI. It clones the vLLM repository and runs the example script.

```bash
echo 'Starting gradio server...'
git clone https://github.com/vllm-project/vllm.git || true
python vllm/examples/gradio_openai_chatbot_webserver.py \
  -m $MODEL_NAME \
  --port 8811 \
  --model-url http://localhost:8081/v1
```