Cosmos Cookbook

https://nvidia-cosmos.github.io/cosmos-cookbook
Admin
NVIDIA Cosmos Cookbook is a practical guide for building, adapting, and deploying generative world...
Tokens:339,666
Snippets:3,778
Trust Score:8.3
Update:1 month ago
Show doc for...
Tokens:
Raw
### Run Finetuning Reference Configuration

Source: https://nvidia-cosmos.github.io/cosmos-cookbook/recipes/post_training/predict2_5/surgical_robotics/post_training.html

Example configuration for an 8xH100 setup using the mini dataset.

```bash
export COSMOS_CODE_PATH=/ephemeral/Cosmos-H-Surgical-Simulator
export SUTUREBOT_LEROBOT_PATH=/ephemeral/data/suturebot_lerobot_mini
export COSMOS_H_CKPT_PATH=/ephemeral/checkpoints/checkpoints/iter_000023000
export IMAGINAIRE_OUTPUT_ROOT=/ephemeral/checkpoints/training_output
export COSMOS_CONTAINER_IMAGE=cosmos-predict2.5:local
export HF_HOME=/ephemeral/cache/huggingface
export WANDB_API_KEY=your_api_key_here   # optional

./scripts/run_finetuning_standalone.sh
```

--------------------------------

### Clone and Setup Environment

Source: https://nvidia-cosmos.github.io/cosmos-cookbook/getting_started/nebius/reason2/reason2_on_nebius.html

Commands to clone the repository, create a virtual environment, and install the required dependencies.

```bash
# Clone the repository (if you haven't already)
git clone https://github.com/nvidia-cosmos/cosmos-cookbook.git
cd cosmos-cookbook/docs/getting_started/nebius/reason2/src

# Create a virtual environment (recommended)
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt
```

--------------------------------

### Sample Setup Script for Brev

Source: https://nvidia-cosmos.github.io/cosmos-cookbook/getting_started/brev/transfer2_5/transfer_and_predict_on_brev.html

A bash script to automate environment setup, including system package installation, user detection, and disk space management for the Transfer2.5 repository.

```bash
#!/bin/bash

# Detect the non-root user that Brev creates
if id "nvidia" &>/dev/null; then
  RUN_USER="nvidia"
elif id "shadeform" &>/dev/null; then
  RUN_USER="shadeform"
elif id "ubuntu" &>/dev/null; then
  RUN_USER="ubuntu"
else
  RUN_USER="root"
fi

sudo -u "$RUN_USER" bash -l <<'SETUP_EOF'
# Install required system packages
sudo apt-get update && sudo apt-get install -y git-lfs bc curl ffmpeg libx11-dev tree wget

REPO_NAME="cosmos-transfer2.5"
REPO_HOME="$HOME/$REPO_NAME"

# ---------------------------------------------------------------------------
# Disk space management: find the biggest writable disk.
# $HOME is not guaranteed to have enough space on all Brev instance types.
# ---------------------------------------------------------------------------
BIG_MOUNT=$(
  { find / -maxdepth 1 -mindepth 1 -type d -writable \
      -not -name tmp -not -name proc -not -name sys \
      -not -name dev -not -name run 2>/dev/null; echo "$HOME"; } | \
  while IFS= read -r d; do
    size=$(df -x tmpfs -x devtmpfs -x squashfs --output=size "$d" 2>/dev/null | tail -1 | tr -d ' ')
    [ -n "$size" ] && echo "$size $d"
  done | sort -rn | head -1 | awk '{print $2}'
)

if [ -z "$BIG_MOUNT" ]; then
  echo "Warning: no writable large mount found, falling back to HOME."
  BIG_MOUNT="$HOME"
fi

# Compare filesystems: only move if the repo is on a smaller disk
REPO_FS=$(df --output=target "$REPO_HOME" | tail -1 | tr -d ' ')
BIG_FS=$(df --output=target "$BIG_MOUNT" | tail -1 | tr -d ' ')

echo "Best writable dir:  $BIG_MOUNT (filesystem: $BIG_FS)"
echo "Repo filesystem:    $REPO_FS"
```

--------------------------------

### Quick Start Cosmos Reason 2 Navigation and Training

Source: https://nvidia-cosmos.github.io/cosmos-cookbook/recipes/post_training/reason2/av_3d_grounding/post_training.html

Commands to navigate to the Reason 2 example directory and initiate supervised fine-tuning.

```bash
cd examples/cosmos_rl
```

```bash
cosmos-rl --config configs/av_grounding.sft.toml scripts/av_grounding_dataloader.py
```

--------------------------------

### Cosmos Curator CLI Installation

Source: https://nvidia-cosmos.github.io/cosmos-cookbook/core_concepts/data_curation/core_curation.html

Quick setup instructions for cloning the repository and installing dependencies using uv and poetry.

```bash
# Quick setup (full instructions in Local Setup section)
git clone --recurse-submodules https://github.com/nvidia-cosmos/cosmos-curate.git
cd cosmos-curate
uv venv --python=3.10 && source .venv/bin/activate
uv pip install poetry && poetry install --extras=local
```

--------------------------------

### Quick Start Cosmos Reason 1 Navigation and Training

Source: https://nvidia-cosmos.github.io/cosmos-cookbook/recipes/post_training/reason2/av_3d_grounding/post_training.html

Commands to navigate to the Reason 1 example directory, activate the virtual environment, and initiate supervised fine-tuning.

```bash
cd examples/post_training_3d_grounding
source ../post_training/.venv/bin/activate
```

```bash
cosmos-rl --config configs/av_grounding.sft.toml scripts/av_grounding_dataloader.py
```

--------------------------------

### Brev Environment Setup Script

Source: https://nvidia-cosmos.github.io/cosmos-cookbook/getting_started/brev/transfer2_5/transfer_and_predict_on_brev.html

A complete bash script to prepare the environment, including disk relocation, tool installation, CUDA detection, and dependency management.

```bash
if [ "$REPO_FS" != "$BIG_FS" ]; then
  STORE="$BIG_MOUNT/$(id -un)"
  REPO_DST="$STORE/$REPO_NAME"
  echo "Moving repo to big disk: $REPO_DST"

  # Move entire repo to the big disk, symlink from HOME for convenience
  mkdir -p "$STORE"
  mv "$REPO_HOME" "$REPO_DST"
  ln -sfn "$REPO_DST" "$REPO_HOME"

  # Redirect HuggingFace cache (model weights) to the big disk
  HF_CACHE="$STORE/huggingface"
  mkdir -p "$HF_CACHE" "$HOME/.cache"
  rm -rf "$HOME/.cache/huggingface"
  ln -sfn "$HF_CACHE" "$HOME/.cache/huggingface"

  echo "Repo:              $REPO_DST (symlinked from ~/$REPO_NAME)"
  echo "HuggingFace cache: $HF_CACHE (symlinked from ~/.cache/huggingface)"
else
  echo "Repo is already on the largest disk. No relocation needed."
fi

# ---------------------------------------------------------------------------
# git-lfs: pull any LFS objects from the repo Brev already cloned for us
# ---------------------------------------------------------------------------
cd "$REPO_HOME"
git lfs install
git lfs pull

# ---------------------------------------------------------------------------
# Install uv (fast Python package/project manager)
# ---------------------------------------------------------------------------
curl -LsSf https://astral.sh/uv/install.sh | sh
source "$HOME/.local/bin/env"

# ---------------------------------------------------------------------------
# Install HuggingFace CLI (used to authenticate and download checkpoints)
# ---------------------------------------------------------------------------
uv tool install -U huggingface_hub

# ---------------------------------------------------------------------------
# Auto-detect CUDA version and select the matching extras
# ---------------------------------------------------------------------------
CUDA_VER=$(nvidia-smi | grep -oP 'CUDA Version: \K[0-9]+\.[0-9]+' | head -1)
CUDA_MAJOR=$(echo "$CUDA_VER" | cut -d. -f1)
if [ "${CUDA_MAJOR:-12}" -ge 13 ] 2>/dev/null; then
  CUDA_EXTRA="cu130"
else
  CUDA_EXTRA="cu128"
fi
echo "Using CUDA extra: $CUDA_EXTRA"

# ---------------------------------------------------------------------------
# Install the CUDA toolkit if the system doesn't already have one.
# Some providers ship only the GPU driver without the full toolkit, which
# causes runtime failures (libcudart.so, libnvrtc.so, etc. not found).
# ---------------------------------------------------------------------------
if ! ldconfig -p | grep -q libcudart.so; then
  CUDA_PKG_VER=$(echo "$CUDA_VER" | tr '.' '-')
  echo "System CUDA toolkit not found. Installing cuda-toolkit-${CUDA_PKG_VER}..."
  sudo apt-get install -y "cuda-toolkit-${CUDA_PKG_VER}"
else
  echo "System CUDA toolkit already installed."
fi

# ---------------------------------------------------------------------------
# Set OMP_NUM_THREADS: distribute CPU cores evenly across GPU worker processes.
# Without this, torchrun defaults to 1 thread per process, leaving cores idle.
# ---------------------------------------------------------------------------
NUM_GPUS=$(nvidia-smi --query-gpu=name --format=csv,noheader | wc -l)
NUM_CPUS=$(nproc)
OMP_NUM_THREADS=$(( NUM_CPUS / NUM_GPUS ))
echo "export NUM_GPUS=$NUM_GPUS" >> "$HOME/.bashrc"
echo "export OMP_NUM_THREADS=$OMP_NUM_THREADS" >> "$HOME/.bashrc"
echo "Set OMP_NUM_THREADS=$OMP_NUM_THREADS ($NUM_CPUS CPUs / $NUM_GPUS GPUs)"

# ---------------------------------------------------------------------------
# Create the Python virtual environment and install all dependencies.
# ---------------------------------------------------------------------------
uv python install
uv sync --extra="$CUDA_EXTRA"

echo ""
echo "================================================================"
echo "Setup complete!"
echo ""
echo "To get started:"
echo "  1. cd ~/$REPO_NAME"
echo "  2. source .venv/bin/activate"
echo "  3. hf auth login   # Required for model checkpoint downloads"
echo "================================================================"
```

--------------------------------

### Start JupyterLab

Source: https://nvidia-cosmos.github.io/cosmos-cookbook/recipes/inference/reason2/worker_safety/setup.html

Navigate to the repository root, activate the virtual environment, and start JupyterLab.

```bash
cd ~/Documents/GitHub/cosmos-reason2
source .venv/bin/activate
jupyter lab
```

--------------------------------

### Install FiftyOne and core dependencies

Source: https://nvidia-cosmos.github.io/cosmos-cookbook/recipes/inference/transfer2_5/biotrove_augmentation/setup.html

Install the FiftyOne library and OpenCV for data handling.

```bash
pip install fiftyone opencv-python
```

--------------------------------

### Install FFmpeg on Ubuntu

Source: https://nvidia-cosmos.github.io/cosmos-cookbook/recipes/inference/transfer2_5/biotrove_augmentation/setup.html

Install FFmpeg using apt-get and verify the installation.

```bash
sudo apt-get update
sudo apt-get install -y ffmpeg
```

```bash
ffmpeg -version
```

--------------------------------

### Quick reference for training workflow

Source: https://nvidia-cosmos.github.io/cosmos-cookbook/recipes/post_training/reason2/av_3d_grounding/post_training.html

A summary of the environment setup, data preparation, and execution steps for the fine-tuning process.

```bash
QWEN_FINETUNE_ROOT=/path/to/qwen-vl-finetune   # or Qwen3-VL clone
cd $QWEN_FINETUNE_ROOT
# Ensure qwenvl/data/train/ and qwenvl/data/eval/ exist with annotations.json (and images/)
# Convert from meta.json+text/ if needed:
#   python tools/convert_av3d_to_qwen_dataset.py --source_dir /path/to/eval --output_dir ./qwenvl/data/eval --copy_images
bash scripts/qwen_finetune_script.sh
# Checkpoint: see script output_dir (e.g. ./outputs/av_3d_grounding_sft_qwen3_cr2_8b)
```

--------------------------------

### Install uv

Source: https://nvidia-cosmos.github.io/cosmos-cookbook/recipes/inference/reason2/worker_safety/setup.html

Install the uv package manager and verify the installation.

```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
source $HOME/.local/bin/env
uv --version
```

--------------------------------

### Install FiftyOne

Source: https://nvidia-cosmos.github.io/cosmos-cookbook/recipes/inference/reason2/worker_safety/setup.html

Install the FiftyOne package into the current environment.

```bash
pip install -U fiftyone
```

--------------------------------

### Clone and install cosmos-curate

Source: https://nvidia-cosmos.github.io/cosmos-cookbook/recipes/data_curation/predict2_data/data_curation.html

Clones the repository, sets up a virtual environment, and installs dependencies using poetry.

```bash
# Clone and install cosmos-curate
git clone --recurse-submodules https://github.com/nvidia-cosmos/cosmos-curate.git
cd cosmos-curate
uv venv --python=3.10 && source .venv/bin/activate
uv pip install poetry && poetry install --extras=local

# Verify you get the cosmos-curate CLI working
cosmos-curate --help
```

--------------------------------

### Install optional visualization tools

Source: https://nvidia-cosmos.github.io/cosmos-cookbook/recipes/inference/transfer2_5/biotrove_augmentation/setup.html

Install JupyterLab and UMAP for notebook-based visualization.

```bash
pip install jupyterlab umap-learn
```

--------------------------------

### Install system dependencies

Source: https://nvidia-cosmos.github.io/cosmos-cookbook/recipes/post_training/predict2_5/sports/setup.html

Install required system-level packages using apt.

```bash
sudo apt install curl ffmpeg tree wget
```

--------------------------------

### Install Hugging Face CLI

Source: https://nvidia-cosmos.github.io/cosmos-cookbook/recipes/post_training/predict2_5/sports/setup.html

Install the Hugging Face CLI tool using uv.

```bash
uv tool install -U "huggingface_hub[cli]"
```

--------------------------------

### Install Cosmos Reason 2 with uv

Source: https://nvidia-cosmos.github.io/cosmos-cookbook/recipes/post_training/reason2/intelligent-transportation/post_training_notebook.html

Installs the uv package manager, clones the necessary repositories, and synchronizes the environment with required dependencies. Ensure system dependencies like ffmpeg and redis-server are installed before running.

```python
import os
import sys

# Install uv and update PATH
!curl -LsSf https://astral.sh/uv/install.sh | sh
os.environ["PATH"] = f"{os.path.expanduser('~/.local/bin')}:{os.environ['PATH']}"

# Clone repositories
for repo in ["cosmos-reason2", "cosmos-cookbook"]:
    if not os.path.exists(repo):
        !git clone https://github.com/nvidia-cosmos/{repo}.git

# Sync environments
print("\nInstalling prerequisites...")
!uv add --project cosmos-reason2 opencv-python decord ipykernel
!uv sync --project cosmos-reason2 --extra cu128
!uv sync --project cosmos-reason2/examples/cosmos_rl
!uv run --project cosmos-reason2 python -m ipykernel install --user --name cosmos-reason2-venv --display-name "Cosmos-Reason2"

print("\n✅ Setup complete!")
```

--------------------------------

### Run Training with Cosmos-RL

Source: https://nvidia-cosmos.github.io/cosmos-cookbook/recipes/post_training/reason2/intelligent-transportation/post_training_notebook.html

Configures the environment, verifies Redis installation, sets up the virtual environment, and executes the training process.

```python
import os import sys COSMOS_RL_VENV = f"{COSMOS_RL_PATH}/.venv" TRAINING_DIR = f"{COSMOS_COOKBOOK_REPO}/scripts/examples/reason2/intelligent-transportation" print("🚀 Running Training with Cosmos-RL") print("="*70) print(f" Working Dir: {TRAINING_DIR}") print(f" Config: sft_config.toml") print(f" Script: custom_sft.py") print("="*70) # Check Redis package installed try: import redis except ImportError as exc: raise ImportError( "Redis is required for training. Please install the system service\n") from exc print("\n⏱️ Expected training time (8× A100):") print(" - 3k tokens (nframes=8): ~1h 16m for 3 epochs") # Setup cosmos-rl venv if needed if not os.path.exists(f"{COSMOS_RL_VENV}/bin/cosmos-rl"): print("\n📦 Setting up cosmos-rl venv with uv sync...") !cd {COSMOS_RL_PATH} && pip install -q uv && uv sync # Run training - MUST activate venv so subprocesses get the right python print("\n🔄 Starting training...\n") !source {COSMOS_RL_VENV}/bin/activate && cd {TRAINING_DIR} && cosmos-rl --config sft_config.toml custom_sft.py
```

--------------------------------

### Verify Installation and cosmos-rl CLI

Source: https://nvidia-cosmos.github.io/cosmos-cookbook/recipes/post_training/reason2/intelligent-transportation/post_training_notebook.html

Checks for the existence of the cosmos-rl virtual environment and displays the help menu to confirm successful installation.

```python
# Verify installations
import sys
import os

COSMOS_REASON2_REPO = os.path.join(os.getcwd(), "cosmos-reason2")  # default clone location
COSMOS_RL_PATH = f"{COSMOS_REASON2_REPO}/examples/cosmos_rl"
COSMOS_RL_BIN = f"{COSMOS_RL_PATH}/.venv/bin/cosmos-rl"

print("Verifying installations:\n")

print("Checking cosmos-rl venv:")
!ls -la {COSMOS_RL_PATH}/.venv/bin/ 2>/dev/null | grep -E "cosmos|python" || echo "  venv not found"

if os.path.exists(COSMOS_RL_BIN):
    print(f"\n✅ cosmos-rl found at {COSMOS_RL_BIN}")
    print("\ncosmos-rl --help:")
    !{COSMOS_RL_BIN} --help 2>&1 | head -15
else:
    print(f"\n❌ cosmos-rl not found at {COSMOS_RL_BIN}")
    print("\nTry running uv sync manually:")
    !cd {COSMOS_RL_PATH} && {sys.executable} -m uv sync 2>&1 | tail -30
```

--------------------------------

### Verify cosmos-rl installation

Source: https://nvidia-cosmos.github.io/cosmos-cookbook/recipes/post_training/reason2/intelligent-transportation/post_training_notebook.html

Checks for the existence of the cosmos-rl virtual environment and displays the help menu if found. If not found, it provides instructions to run uv sync.

```python
import sys import os COSMOS_REASON2_REPO = os.path.join(os.getcwd(), "cosmos-reason2") # default clone location COSMOS_RL_PATH = f"{COSMOS_REASON2_REPO}/examples/cosmos_rl" COSMOS_RL_BIN = f"{COSMOS_RL_PATH}/.venv/bin/cosmos-rl" print("Verifying installations:\n") print("Checking cosmos-rl venv:") !ls -la {COSMOS_RL_PATH}/.venv/bin/ 2>/dev/null | grep -E "cosmos|python" || echo " venv not found" if os.path.exists(COSMOS_RL_BIN): print(f"\n✅ cosmos-rl found at {COSMOS_RL_BIN}") print("\ncosmos-rl --help:") !{COSMOS_RL_BIN} --help 2>&1 | head -15 else: print(f"\n❌ cosmos-rl not found at {COSMOS_RL_BIN}") print("\nTry running uv sync manually:") !cd {COSMOS_RL_PATH} && {sys.executable} -m uv sync 2>&1 | tail -30
```

--------------------------------

### Configure FP8 Quantization Parameters

Source: https://nvidia-cosmos.github.io/cosmos-cookbook/recipes/post_training/reason2/intelligent-transportation/post_training_notebook.html

Defines the paths and configuration dictionary for the FP8 quantization process and prints the setup details.

```python
FP8_MODEL_OUTPUT_PATH = f"{FINETUNED_MODEL_PATH}_fp8"
QUANTIZATION_CONFIG = { "model_path": FINETUNED_MODEL_PATH, "output_path": FP8_MODEL_OUTPUT_PATH, "precision": "fp8", }
quantize_script = f"{COSMOS_REASON2_REPO}/scripts/quantize.py"
print("Quantization Setup")
print("="*70)
print(f" Input Model: {QUANTIZATION_CONFIG['model_path']}")
print(f" Output Path: {QUANTIZATION_CONFIG['output_path']}")
print(f" Precision: {QUANTIZATION_CONFIG['precision']}")
print("="*70)
QUANTIZE_CMD = f"""
python {quantize_script} \
--model "{QUANTIZATION_CONFIG['model_path']}" \
-o "{QUANTIZATION_CONFIG['output_path']}" \
--precision {QUANTIZATION_CONFIG['precision']}
"""
print("\nQuantization Command:")
print("-"*70)
print(QUANTIZE_CMD)
print("-"*70)
```

--------------------------------

### Display and Parse Llava-format Annotations

Source: https://nvidia-cosmos.github.io/cosmos-cookbook/recipes/post_training/reason2/intelligent-transportation/post_training_notebook.html

Utility functions to pretty-print Llava-format examples and parse MCQ text from conversation prompts. The script loads annotations from a specified path and displays sample questions with their correct options marked.

```python
def display_llava_format(example):
    """Pretty print a Llava-format example from the dataset."""
    print("📋 Llava Dataset Format Example (from WTS):")
    print("="*60)
    print(json.dumps(example, indent=2))
    print("="*60)


def parse_mcq_text(text):
    """Parse MCQ question/options from the WTS Llava-format prompt."""
    cleaned = text.replace("<video>", " ").strip()
    lines = [line.strip() for line in cleaned.splitlines() if line.strip()]
    question = lines[0] if lines else ""
    options = lines[1:] if len(lines) > 1 else []
    return question, options


def is_correct_option(option, answer):
    """Mark the correct option based on the answer token (e.g., 'A')."""
    opt = option.strip()
    ans = answer.strip()
    if not ans:
        return False
    prefixes = [f"{ans}:", f"{ans})", f"{ans}.", f"{ans} "]
    return opt == ans or any(opt.startswith(prefix) for prefix in prefixes)


# Load actual MCQ examples from the training annotations
annotations_path = TRAIN_ANNOTATIONS_PATH

if not os.path.exists(annotations_path):
    print("⚠️ annotations.json not found. Update TRAIN_DATA_PATH to your dataset location.")
    print(f"   Current path: {annotations_path}")
else:
    with open(annotations_path, "r") as f:
        annotations = json.load(f)

    # Display a real Llava-format entry
    if annotations:
        display_llava_format(annotations[0])

    # Display a few actual MCQ questions
    print("\n\n📝 Sample MCQ Questions from the Training Set:")
    print("="*60)
    for i, ann in enumerate(annotations[:4], 1):
        question_text, options = parse_mcq_text(ann["conversations"][0]["value"])
        answer = ann["conversations"][1]["value"]
        print(f"\nQ{i}: {question_text}")
        for opt in options:
            marker = "✓" if is_correct_option(opt, answer) else " "
            print(f"   [{marker}] {opt}")
    print("\n" + "="*60)
```

```python
def display_llava_format(example): """Pretty print a Llava-format example from the dataset.""" print("📋 Llava Dataset Format Example (from WTS):") print("="*60) print(json.dumps(example, indent=2)) print("="*60) def parse_mcq_text(text): """Parse MCQ question/options from the WTS Llava-format prompt.""" cleaned = text.replace("", " ").strip() lines = [line.strip() for line in cleaned.splitlines() if line.strip()] question = lines[0] if lines else "" options = lines[1:] if len(lines) > 1 else [] return question, options def is_correct_option(option, answer): """Mark the correct option based on the answer token (e.g., 'A').""" opt = option.strip() ans = answer.strip() if not ans: return False prefixes = [f"{ans}:", f"{ans})", f"{ans}.", f"{ans} "] return opt == ans or any(opt.startswith(prefix) for prefix in prefixes) # Load actual MCQ examples from the training annotations annotations_path = TRAIN_ANNOTATIONS_PATH if not os.path.exists(annotations_path): print("⚠️ annotations.json not found. Update TRAIN_DATA_PATH to your dataset location.") print(f" Current path: {annotations_path}") else: with open(annotations_path, "r") as f: annotations = json.load(f) # Display a real Llava-format entry if annotations: display_llava_format(annotations[0]) # Display a few actual MCQ questions print("\n\n📝 Sample MCQ Questions from the Training Set:") print("="*60) for i, ann in enumerate(annotations[:4], 1): question_text, options = parse_mcq_text(ann["conversations"][0]["value"]) answer = ann["conversations"][1]["value"] print(f"\nQ{i}: {question_text}") for opt in options: marker = "✓" if is_correct_option(opt, answer) else " " print(f" [{marker}] {opt}") print("\n" + "="*60)
```

--------------------------------

### Run System Setup Script

Source: https://nvidia-cosmos.github.io/cosmos-cookbook/recipes/post_training/predict2_5/surgical_robotics/post_training.html

Configure Docker and containerd on a fresh cloud instance to ensure sufficient storage and proper DNS settings.

```bash
sudo bash scripts/01-system-setup.sh
```

--------------------------------

### Configure deployment environment

Source: https://nvidia-cosmos.github.io/cosmos-cookbook/recipes/end2end/smart_city_sdg/workflow_e2e.html

Sets up the environment configuration file by copying the example and preparing it for user-specific credentials.

```bash
cd deploy/compose
cp env.example env
# Edit values for NGC_API_KEY, HF_TOKEN, GPU IDs, ports, etc.
```

--------------------------------

### Download and Setup RoboCasa Dataset

Source: https://nvidia-cosmos.github.io/cosmos-cookbook/recipes/post_training/predict2/cosmos_policy/post_training.html

Commands to download the RoboCasa-Cosmos-Policy dataset, install dependencies, and set up assets and macros.

```bash
hf download nvidia/RoboCasa-Cosmos-Policy --repo-type dataset --local-dir RoboCasa-Cosmos-Policy
export BASE_DATASETS_DIR=$(pwd)
git clone https://github.com/moojink/robocasa-cosmos-policy.git
uv pip install -e robocasa-cosmos-policy
uv run --extra cu128 --group robocasa --python 3.10 robocasa-cosmos-policy/robocasa/scripts/download_kitchen_assets.py
uv run --extra cu128 --group robocasa --python 3.10 robocasa-cosmos-policy/robocasa/scripts/setup_macros.py
# Downloads to a local directory called `RoboCasa-Cosmos-Policy`
hf download nvidia/RoboCasa-Cosmos-Policy --repo-type dataset --local-dir RoboCasa-Cosmos-Policy
# Set the current base datasets directory as BASE_DATASETS_DIR (needed for training)
```

--------------------------------

### Install and Activate SFT Environment

Source: https://nvidia-cosmos.github.io/cosmos-cookbook/recipes/post_training/reason1/wafermap_classification/post_training.html

Set up the dedicated virtual environment for post-training.

```bash
just install
source .venv/bin/activate
```

--------------------------------

### Install dependencies

Source: https://nvidia-cosmos.github.io/cosmos-cookbook/core_concepts/evaluation/reason_as_reward.html

Install the required Python packages and verify the PyTorch and CUDA installation.

```bash
# Install core dependencies
pip install torch torchvision transformers
pip install mediapy numpy pillow qwen-vl-utils huggingface-hub

# Verify installation
python -c "import torch; print(f'PyTorch: {torch.__version__}'); print(f'CUDA available: {torch.cuda.is_available()}')"
```

--------------------------------

### Cosmos Reason 1 Quick Reference

Source: https://nvidia-cosmos.github.io/cosmos-cookbook/recipes/post_training/reason2/av_3d_grounding/post_training.html

A consolidated script for setting up the environment and running training for Cosmos Reason 1.

```bash
COSMOS_REASON1_ROOT=/path/to/cosmos-reason1
cd $COSMOS_REASON1_ROOT/examples/post_training_3d_grounding
source ../post_training/.venv/bin/activate
# Optional: ln -s /path/to/curated_av_grounding dataset/train
cosmos-rl --config configs/av_grounding.sft.toml scripts/av_grounding_dataloader.py
# Checkpoint: $COSMOS_REASON1_ROOT/examples/post_training_3d_grounding/outputs/sft-07/<TIMESTAMP>/safetensors/final
```
Tokens:
Raw