### Install Cosmos-RL Example

Source: https://github.com/nvidia-cosmos/cosmos-reason2/blob/main/examples/cosmos_rl/README.md

Navigates to the Cosmos-RL example directory and synchronizes dependencies using uv.

```shell
cd examples/cosmos_rl
uv sync
```

--------------------------------

### Install System Dependencies

Source: https://github.com/nvidia-cosmos/cosmos-reason2/blob/main/README.md

Install necessary system dependencies for the virtual environment setup.

```shell
sudo apt-get install curl ffmpeg git git-lfs unzip
```

--------------------------------

### Install and Login to Wandb

Source: https://github.com/nvidia-cosmos/cosmos-reason2/blob/main/examples/cosmos_rl/README.md

Installs the wandb tool and logs in using an API key for training monitoring.

```bash
uv tool install -U wandb
wandb login
```

--------------------------------

### Install and Sync Notebook Dependencies

Source: https://github.com/nvidia-cosmos/cosmos-reason2/blob/main/examples/notebooks/README.md

Installs necessary packages and activates the virtual environment for running notebooks locally.

```shell
cd examples/notebooks
uv sync
source .venv/bin/activate
```

--------------------------------

### Install uv via script

Source: https://github.com/nvidia-cosmos/cosmos-reason2/blob/main/README.md

Install the uv package manager using the provided installation script and activate its environment.

```shell
curl -LsSf https://astral.sh/uv/install.sh | sh
source $HOME/.local/bin/env
```

--------------------------------

### Run Minimal Inference Example

Source: https://github.com/nvidia-cosmos/cosmos-reason2/blob/main/README.md

Execute the minimal inference script to test basic functionality. Ensure transformers>=4.57.0 is installed.

```shell
python scripts/inference_sample.py
```

--------------------------------

### Install rust-just Tool

Source: https://github.com/nvidia-cosmos/cosmos-reason2/blob/main/CONTRIBUTING.md

Installs the rust-just tool using uv. Ensure you have uv installed and configured.

```shell
uv tool install -U rust-just
```

--------------------------------

### Start vLLM Server for Online Serving

Source: https://github.com/nvidia-cosmos/cosmos-reason2/blob/main/README.md

Starts the vLLM server for online inference with the Cosmos-Reason2-2B model. Configure model length, media I/O, reasoning parser, and port as needed. Initial startup may take a few minutes.

```shell
vllm serve nvidia/Cosmos-Reason2-2B \
  --allowed-local-media-path "$(pwd)" \
  --max-model-len 16384 \
  --media-io-kwargs '{"video": {"num_frames": -1}}' \
  --reasoning-parser qwen3 \
  --port 8000
```

--------------------------------

### Install TRL with PEFT and BitsAndBytes

Source: https://github.com/nvidia-cosmos/cosmos-reason2/blob/main/examples/notebooks/trl_sft.ipynb

Installs the TRL library with PEFT extras for efficient fine-tuning, along with bitsandbytes for quantization and tensorboard for logging.

```python
!pip install -Uq "trl[peft]==0.26.1" "bitsandbytes==0.49.0" "tensorboard==2.20.0"
```

--------------------------------

### Install TRL with PEFT and other dependencies

Source: https://github.com/nvidia-cosmos/cosmos-reason2/blob/main/examples/notebooks/trl_grpo.ipynb

Installs the TRL library with PEFT extras, bitsandbytes for quantization, tensorboard for logging, and math_verify for potential utility. Ensure to use specific versions as shown for compatibility.

```python
!pip install -Uq "trl[peft]==0.26.1" "bitsandbytes==0.49.0" "tensorboard==2.20.0" "math_verify==0.8.0"
```

--------------------------------

### Execute Training

Source: https://github.com/nvidia-cosmos/cosmos-reason2/blob/main/examples/notebooks/trl_sft.ipynb

Initiate the supervised fine-tuning process using the configured SFTTrainer. This command starts the training loop based on the provided arguments and dataset.

```python
trainer_stats = trainer.train()
```

--------------------------------

### Inspect Processed Dataset Example

Source: https://github.com/nvidia-cosmos/cosmos-reason2/blob/main/examples/notebooks/trl_grpo.ipynb

Displays the first example from the processed training dataset to verify the structure and content after applying the conversation formatting and chat template.

```python
train_dataset[0]
```

--------------------------------

### Install Redis Server

Source: https://github.com/nvidia-cosmos/cosmos-reason2/blob/main/examples/cosmos_rl/README.md

Installs the Redis server using conda. This is a system dependency for the virtual environment setup.

```shell
conda install -c conda-forge redis-server
```

--------------------------------

### Display Quantization Script Help

Source: https://github.com/nvidia-cosmos/cosmos-reason2/blob/main/docs/llmcompressor.md

View available arguments and options for the quantization script by running the --help command.

```shell
./scripts/quantize.py --help
```

--------------------------------

### Initialize GRPOTrainer

Source: https://github.com/nvidia-cosmos/cosmos-reason2/blob/main/examples/notebooks/trl_grpo.ipynb

Instantiate the GRPOTrainer with the model, reward functions, training arguments, dataset, and PEFT configuration. Ensure the model, reward functions, and dataset are defined prior to this step.

```python
from trl import GRPOTrainer

trainer = GRPOTrainer(
    model=model,
    reward_funcs=[format_reward, len_reward],
    args=training_args,
    train_dataset=train_dataset,
    peft_config=peft_config,
)
```

--------------------------------

### Configure Training Arguments with SFTConfig

Source: https://github.com/nvidia-cosmos/cosmos-reason2/blob/main/examples/notebooks/trl_sft.ipynb

Set up training parameters such as epochs, batch size, learning rate, and logging for supervised fine-tuning. Use `max_steps` for shorter runs and `num_train_epochs` for full training.

```python
from trl import SFTConfig

training_args = SFTConfig(
    # Training schedule / optimization
    # num_train_epochs=1,
    max_steps=10,  # Number of dataset passes. For full trainings, use `num_train_epochs` instead
    per_device_train_batch_size=2,  # Batch size per GPU/CPU
    gradient_accumulation_steps=8,  # Gradients are accumulated over multiple steps → effective batch size = 4 * 8 = 32
    warmup_steps=5,  # Gradually increase LR during first N steps
    learning_rate=2e-4,  # Learning rate for the optimizer
    optim="adamw_8bit",  # Optimizer
    max_length=None,  # For VLMs, truncating may remove image tokens, leading to errors during training. max_length=None avoids it
    # Logging / reporting
    output_dir=output_dir,  # Where to save model checkpoints and logs
    logging_steps=1,  # Log training metrics every N steps
    report_to="tensorboard",  # Experiment tracking tool
)
```

--------------------------------

### Run Supervised Fine-Tuning (SFT)

Source: https://github.com/nvidia-cosmos/cosmos-reason2/blob/main/examples/cosmos_rl/README.md

Executes the SFT training script using a specified configuration file and log directory. This command requires a TOML configuration file.

```shell
uv run cosmos-rl --config configs/llava_sft.toml --log-dir outputs/llava_sft scripts/llava_sft.py
```

--------------------------------

### Export requirements.txt using uv

Source: https://github.com/nvidia-cosmos/cosmos-reason2/blob/main/docs/troubleshooting.md

Use this command to generate a requirements.txt file from pyproject.toml for pip installations.

```shell
uv export --format requirements.txt --output-file requirements.txt
```

--------------------------------

### Sync Repository and Activate Environment

Source: https://github.com/nvidia-cosmos/cosmos-reason2/blob/main/README.md

Synchronize the repository with the specified CUDA version and activate the virtual environment.

```shell
uv sync --extra cu128
source .venv/bin/activate
```

--------------------------------

### Configure LoRA Adapter

Source: https://github.com/nvidia-cosmos/cosmos-reason2/blob/main/examples/notebooks/trl_sft.ipynb

Defines the LoRA configuration with rank (r), alpha (lora_alpha), and target modules for parameter-efficient fine-tuning. This setup is used for QLoRA when BitsAndBytesConfig is active.

```python
from peft import LoraConfig

peft_config = LoraConfig(
    r=32,
    lora_alpha=32,
    target_modules=[
        "down_proj",
        "o_proj",
        "k_proj",
        "q_proj",
        "gate_proj",
        "up_proj",
        "v_proj",
    ],
)
```

--------------------------------

### Load Model with QLoRA Configuration

Source: https://github.com/nvidia-cosmos/cosmos-reason2/blob/main/examples/notebooks/trl_grpo.ipynb

Loads a pre-trained model using BitsAndBytesConfig for 4-bit quantization. This is the default setup for QLoRA. To use standard LoRA, comment out the quantization_config.

```python
import torch
from transformers import BitsAndBytesConfig, Qwen3VLForConditionalGeneration

model = Qwen3VLForConditionalGeneration.from_pretrained(
    model_name,
    dtype="auto",
    device_map="auto",
    quantization_config=BitsAndBytesConfig(
        load_in_4bit=True,
        bnb_4bit_use_double_quant=True,
        bnb_4bit_quant_type="nf4",
        bnb_4bit_compute_dtype=torch.float16,
    ),
)
```

--------------------------------

### Run Model Quantization

Source: https://github.com/nvidia-cosmos/cosmos-reason2/blob/main/docs/llmcompressor.md

Execute the quantization script with a specified output directory. Ensure you are in the repository root.

```shell
./scripts/quantize.py -o /tmp/cosmos-reason2/checkpoints
```