### Install and Run User Guide Locally

Source: https://github.com/novasky-ai/skyrl/blob/main/docs/README.md

Steps to install dependencies and start the Next.js development server for the user guide.

```bash
cd docs
npm install
npm run dev
```

--------------------------------

### Start Search Tool Example Server

Source: https://github.com/novasky-ai/skyrl/blob/main/skyrl-tx/README.md

Starts the SkyRL TX API server for the search tool example. Configure the port, base model, and backend parameters, including LoRA settings and sample max sequences.

```bash
# Start server
uv run --extra gpu --extra tinker -m skyrl.tinker.api \
    --port 8001 \
    --base-model Qwen/Qwen3-4B-Instruct-2507 \
    --backend-config '{"max_lora_adapters": 3, "max_lora_rank": 32, "tensor_parallel_size": 8, "train_micro_batch_size": 1, "sample_max_num_sequences": 128}' > out.log
```

--------------------------------

### Run Full Example on Modal

Source: https://github.com/novasky-ai/skyrl/blob/main/examples/tinker/ppo/README.md

Launches the complete PPO example end-to-end within a single Modal container. This includes server setup, client execution, and resource allocation.

```bash
modal run examples/tinker/ppo/modal_run.py
```

--------------------------------

### Full Production Build Locally

Source: https://github.com/novasky-ai/skyrl/blob/main/docs/README.md

Steps to install dependencies, build the project, and start the production server locally.

```bash
cd docs
npm install
npm run build
npm start
```

--------------------------------

### Launch Training for Qwen3-8B Model

Source: https://github.com/novasky-ai/skyrl/blob/main/examples/train/mini_swe_agent/README.md

Start the training process for the Qwen3-8B model. This script requires a specific hardware setup (1x 8xH100 node).

```bash
bash examples/mini_swe_agent/run_mini_swe_8B.sh
```

--------------------------------

### Launch Training for Qwen3-Coder-30B Model

Source: https://github.com/novasky-ai/skyrl/blob/main/examples/train/mini_swe_agent/README.md

Start the training process for the Qwen3-Coder-30B model. This script requires a specific hardware setup (2x 8xH100 nodes).

```bash
bash examples/mini_swe_agent/run_mini_swe_30B.sh
```

--------------------------------

### Install SkyRL-Gym from Source

Source: https://github.com/novasky-ai/skyrl/blob/main/skyrl-gym/README.md

Clone the repository and install skyrl-gym from source using pip.

```bash
git clone https://github.com/NovaSky-AI/SkyRL.git
cd SkyRL/skyrl-gym
pip install -e .
```

--------------------------------

### Prepare Dataset for On-Policy Distillation

Source: https://github.com/novasky-ai/skyrl/blob/main/examples/train/on_policy_distillation/README.md

This script prepares the necessary dataset for On-Policy Distillation, following the setup from the DAPO example. Ensure you have the DAPO example's data setup scripts available.

```bash
bash examples/train/algorithms/dapo/prepare_dapo_data.sh
```

--------------------------------

### Install Dependencies with uv

Source: https://github.com/novasky-ai/skyrl/blob/main/skyrl-train/README.md

Create a virtual environment and install project dependencies, including FSDP support, using uv.

```bash
# creates a venv at .venv/
uv sync --extra fsdp 
source .venv/bin/activate
```

--------------------------------

### Install Environment

Source: https://github.com/novasky-ai/skyrl/blob/main/examples/train_integrations/verifiers/README.md

Use the provided Python script to install the specified environment, adding it to the uv project.

```bash
uv run integrations/verifiers/install_environment.py $ENV_ID
```

--------------------------------

### Add User Guide Page Frontmatter

Source: https://github.com/novasky-ai/skyrl/blob/main/docs/README.md

Example frontmatter for creating new user guide pages in MDX format.

```mdx
---
title: Your Page Title
description: A brief description
---

Your content here...
```

--------------------------------

### Install API Service Framework

Source: https://github.com/novasky-ai/skyrl/blob/main/examples/train/search/README.md

Installs uvicorn and fastapi, which are used for the retrieval API service.

```bash
# Install the API service framework
pip install uvicorn fastapi
```

--------------------------------

### Run Search Example with Step-Wise Training

Source: https://github.com/novasky-ai/skyrl/blob/main/docs/content/docs/tutorials/step-wise-training.mdx

Execute the search-r1 example with multi-turn conversation and step-wise training enabled. This command sets environment variables to configure the training process.

```bash
USE_CONVERSATION_MULTI_TURN=true STEP_WISE=true bash examples/train/search/run_search.sh
```

--------------------------------

### Install Pre-commit Hooks

Source: https://github.com/novasky-ai/skyrl/blob/main/docs/content/docs/getting-started/development.mdx

Install the pre-commit package and set up the pre-commit hooks for linting and formatting. Run these commands from the repository root.

```bash
uv pip install pre-commit
pre-commit install
```

--------------------------------

### Keyword Action Example (Instruct)

Source: https://github.com/novasky-ai/skyrl/blob/main/docs/content/docs/examples/visgym.mdx

Example of a simple keyword action output from a vanilla instruct model.

```text
<action>left</action>
```

--------------------------------

### Tuple Action Example (SFT)

Source: https://github.com/novasky-ai/skyrl/blob/main/docs/content/docs/examples/visgym.mdx

Example of structured output from an SFT-warmed-up model, including observation, justification, and a tuple action.

```text
<observation>I see a maze with the agent at (1,1) and the goal at (3,3)...</observation>
<justification>The goal is east-southeast, so I should move east first.</justification>
<action>('move', 0)</action>
```

--------------------------------

### Run Search Tool Example RL Loop

Source: https://github.com/novasky-ai/skyrl/blob/main/skyrl-tx/README.md

Executes the RL loop for the search tool example. Requires setting API keys for Tinker, Google, and WandB, and specifying model and training parameters.

```bash
# Run RL loop
export TINKER_API_KEY="tml-dummy"
export GOOGLE_API_KEY="..." # Replace with your Google API Key
export WANDB_API_KEY="..."  # Replace with your WandB API Key
uv run --extra vector-search --extra wandb python -m tinker_cookbook.recipes.search_tool.train \
    base_url=http://localhost:8001 \
    model_name=Qwen/Qwen3-4B-Instruct-2507 \
    behavior_if_log_dir_exists=delete \
    wandb_project=search-r1-skyrl-tx
```

--------------------------------

### FSDP Configuration Example

Source: https://github.com/novasky-ai/skyrl/blob/main/docs/content/docs/configuration/config.mdx

Configure CPU offloading, re-sharding after forward pass, and FSDP group size for the FSDP training backend.

```yaml
fsdp_config:
    cpu_offload: false # offload params + optimizer state to cpu during fwd pass
    reshard_after_forward: true # [True, False, int between 1 and fsdp_size]
    fsdp_size: -1
```

--------------------------------

### Launch Training Run

Source: https://github.com/novasky-ai/skyrl/blob/main/examples/train_integrations/verifiers/README.md

Execute the main training script 'run_verifiers.sh' to start the training process with the configured parameters.

```bash
bash integrations/verifiers/run_verifiers.sh
```

--------------------------------

### Install All OpenEnv Environments

Source: https://github.com/novasky-ai/skyrl/blob/main/examples/train_integrations/openenv/README.md

Installs all available OpenEnv environments by pulling their respective Docker images. This is a convenient way to set up all supported environments at once.

```bash
uv run integrations/openenv/install_environment.py
```

--------------------------------

### Launch Training Run

Source: https://github.com/novasky-ai/skyrl/blob/main/docs/content/docs/examples/llm_as_a_judge.mdx

Execute this command to start the training process. The training will utilize the LLM judge for evaluating each generated solution.

```bash
bash examples/train/llm_as_a_judge/run_llm_judge.sh
```

--------------------------------

### Start Reinforcement Learning Server

Source: https://github.com/novasky-ai/skyrl/blob/main/skyrl-tx/README.md

Starts the SkyRL TX API server for reinforcement learning tasks. Configure base model and backend parameters, including sample max number of sequences.

```bash
# Start server
uv run --extra gpu --extra tinker -m skyrl.tinker.api \
    --base-model Qwen/Qwen3-8B \
    --backend-config '{"max_lora_adapters": 3, "max_lora_rank": 1, "tensor_parallel_size": 8, "train_micro_batch_size": 8, "sample_max_num_sequences": 256}' > out.log
```

--------------------------------

### Install OpenEnv Environment

Source: https://github.com/novasky-ai/skyrl/blob/main/examples/train_integrations/openenv/README.md

Installs a specific OpenEnv environment by pulling its Docker image. Use this to set up individual environments for training.

```bash
uv run examples/train_integrations/openenv/install_environment.py echo-env
```

--------------------------------

### Install SkyRL Dependencies with uv

Source: https://github.com/novasky-ai/skyrl/blob/main/docs/content/docs/getting-started/installation.mdx

Activates a virtual environment and installs project dependencies, including the 'fsdp' extra, using uv sync.

```bash
source <path_to_venv>/bin/activate
uv sync --active --extra fsdp
```

--------------------------------

### Example Harbor Dataset Directory Structure

Source: https://github.com/novasky-ai/skyrl/blob/main/docs/content/docs/harbor/index.mdx

This shows how multiple task directories are organized within a dataset directory for `HarborTaskDataset`.

```bash
my-dataset/
  task-001/
    instruction.md
    task.toml
    environment/
      Dockerfile
    tests/
      test.sh
  task-002/
    instruction.md
    ...

```

--------------------------------

### Run Web Summary Router

Source: https://github.com/novasky-ai/skyrl/blob/main/skyrl-agent/examples/README.md

Start a router service for load-balancing/failover of web summary requests. Configure upstream endpoints and the port.

```bash
SUMMARY_UPSTREAMS=http://host:port/v1 \
SUMMARY_MODEL=Qwen/Qwen3-32B \
PORT=8080 \
bash services/run_router.sh
```

--------------------------------

### Harbor Integration Quick Start

Source: https://github.com/novasky-ai/skyrl/blob/main/docs/content/docs/harbor/index.mdx

Steps to set up credentials, prepare datasets, and launch training for the Harbor integration. Ensure you set the appropriate API keys for your chosen sandbox provider.

```bash
cd SkyRL

# 1. Set credentials
export WANDB_API_KEY=your_wandb_api_key
# Pick your sandbox provider:
export DAYTONA_API_KEY=your_daytona_api_key
# export MODAL_TOKEN_ID=your_modal_token_id
# export MODAL_TOKEN_SECRET=your_modal_token_secret

# 2. Prepare datasets (downloads from HuggingFace, extracts tasks to ~/data/harbor/)
uv run examples/train_integrations/harbor/prepare_harbor_dataset.py \
    --dataset open-thoughts/CodeContests
uv run examples/train_integrations/harbor/prepare_harbor_dataset.py \
    --dataset open-thoughts/OpenThoughts-TB-dev

# 3. Launch training
bash examples/train_integrations/harbor/run_codecontest.sh
```

--------------------------------

### Clone SkyRL Repository

Source: https://github.com/novasky-ai/skyrl/blob/main/skyrl-train/README.md

Clone the SkyRL repository including submodules to get started.

```bash
git clone --recurse-submodules https://github.com/NovaSky-AI/SkyRL
cd SkyRL/
```

--------------------------------

### Full Example: Train Qwen2.5-Coder-7B-Instruct with FP8 Rollouts

Source: https://github.com/novasky-ai/skyrl/blob/main/docs/content/docs/examples/quantized_rollouts.mdx

This script demonstrates training with FP8 rollouts and TIS correction. It requires downloading the SkyRL-SQL dataset and setting your Weights & Biases API key.

```bash
# TIS parameters
TIS_IMP_RATIO_CAP=4.0
TIS_TYPE=sequence
# returns rollout logprobs for the generated tokens; required for TIS
LOGPROBS=0

uv run --isolated --extra fsdp -m skyrl.train.entrypoints.main_base \
    trainer.algorithm.off_policy_correction.tis_ratio_type=$TIS_TYPE \
    trainer.algorithm.off_policy_correction.sequence_tis_ratio_clip_high=$TIS_IMP_RATIO_CAP \
    generator.sampling_params.logprobs=$LOGPROBS \
    generator.inference_engine.backend=vllm \
    generator.inference_engine.engine_init_kwargs.quantization=fp8 \
    ...
```

```bash
hf download NovaSky-AI/SkyRL-SQL-653-data-newfmt --local-dir $HOME/data/sql
export WANDB_API_KEY=<your_key_here>
bash examples/train/text_to_sql/run_skyrl_sql_fp8.sh
```

--------------------------------

### Training Quick Start with UV

Source: https://github.com/novasky-ai/skyrl/blob/main/CLAUDE.md

Initiate a training run using `uv run --isolated` with specific backend and trainer configurations. Replace `<model>` with the actual model path.

```bash
uv run --isolated --extra megatron -m skyrl.train.entrypoints.main_base \
  trainer.strategy=megatron trainer.policy.model.path=<model> environment.env_class=gsm8k ...
```

--------------------------------

### Configure SkyRL Dependencies with uv

Source: https://github.com/novasky-ai/skyrl/blob/main/docs/content/docs/getting-started/library.mdx

Example `pyproject.toml` configuration using `uv` to install SkyRL from PyPI or Git, including custom dependencies like flash-attn and flashinfer-jit-cache.

```toml
[project]
dependencies = [
    # Option A: Install from PyPI (stable release)
    "skyrl[fsdp]",
    # Option B: Install from Git (latest)
    # "skyrl[fsdp]",

    "flashinfer-jit-cache==0.5.3",
    "flash-attn==2.8.3"
]

[tool.uv.extra-build-dependencies]
flash-attn = [{requirement = "torch", match-runtime = true}]

[tool.uv.sources]
flashinfer-jit-cache = { index = "flashinfer-cu128" }
# Only needed for Option B (Git install):
# skyrl-gym = { git = "https://github.com/NovaSky-AI/SkyRL", subdirectory = "skyrl-gym" }
# skyrl = { git = "https://github.com/NovaSky-AI/SkyRL" }

[[tool.uv.index]]
name = "flashinfer-cu128"
url = "https://flashinfer.ai/whl/cu128"
explicit = true

```

--------------------------------

### Example Search Agent Prompt

Source: https://github.com/novasky-ai/skyrl/blob/main/docs/content/docs/examples/search.mdx

This prompt defines the behavior of a search agent, guiding it to use reasoning, search queries, and provide answers within specific tags.

```text
You are a helpful and harmless assistant.

Answer the given question. You must conduct reasoning inside <think> and </think> first every time you get new information. 
After reasoning, if you find you lack some knowledge, you can call a search engine by <search> query </search> 
and it will return the top searched results between <information> and </information>. You can search as many times as you want. 
If you find no further external knowledge needed, you can directly provide the answer inside <answer> and </answer>, without detailed illustrations. 
For example, <answer> Beijing </answer>. 

Question: In what year was the company that was founded as Sound of Music added to the S&P 500?

```

--------------------------------

### Download and Preprocess SearchR1 Dataset

Source: https://github.com/novasky-ai/skyrl/blob/main/docs/content/docs/recipes/searchr1.mdx

Use this script to download the SearchR1 dataset from Hugging Face and preprocess it for SkyRL-Gym. Ensure you have followed the installation guide and have the necessary environment set up.

```bash
uv run --isolated examples/train/search/searchr1_dataset.py --local_dir ~/data/searchR1

```

--------------------------------

### Ray Cluster Setup for Multi-Node SkyRL

Source: https://github.com/novasky-ai/skyrl/blob/main/integrations/arctic_rl/README.md

Commands to start a Ray cluster for SkyRL on head and worker nodes. Ensure the correct IP address and GPU count are specified.

```bash
# Head node
uv run --isolated --extra skyrl-train ray start --head --port=6379 --num-gpus=8

# Each worker
uv run --isolated --extra skyrl-train ray start --address=<HEAD_IP>:6379 --num-gpus=8

uv run --isolated --extra skyrl-train ray status
```

--------------------------------

### Launch Text-to-SQL Training

Source: https://github.com/novasky-ai/skyrl/blob/main/examples/train/text_to_sql/sql.md

Initiate the training process by executing the provided bash script. Ensure to modify training configurations as needed.

```bash
bash examples/text_to_sql/<run_XXX>.sh
```

--------------------------------

### Multi-Turn Text2SQL Prompt Example

Source: https://github.com/novasky-ai/skyrl/blob/main/docs/content/docs/examples/multi_turn_text2sql.mdx

This prompt guides an agent to generate SQL queries based on a database schema and a natural language question. It specifies the expected format for thinking, SQL tool usage, and final solution output, along with a reward system for accuracy and formatting.

```text
You are a data science expert. Below, you are provided with a database schema and a natural language question. Your task is to understand the schema and generate a valid SQL query to answer the question within limited turns. You should breakdown the problem, draft your reasoning process, and generate the solution.

Database Engine:
SQLite

Database Schema:
{db_details}
This schema describes the database's structure, including tables, columns, primary keys, foreign keys, and any relevant relationships or constraints.

External Knowledge:
{external_knowledge}

Question:
{question}

Instructions:
- Make sure you only output the information that is asked in the question. If the question asks for a specific column, make sure to only include that column in the SELECT clause, nothing more.
- The generated query should return all of the information asked in the question without any missing or extra information.
- Before generating the final SQL query, please think through the steps of how to write the query. It should include detailed considerations such as analyzing questions, summarizing relevant findings, brainstorming new ideas, verifying the accuracy of the current steps, refining any errors, thinking of how to call SQL tools, and revisiting previous steps.

Format:
- Conduct thinking inside <think>...</think> blocks every time you get new observation or information. 
- You can use SQL tool written within a single <sql>your sql</sql> block to explore or verify. SQL tool output will be shown as dataframe inside <observation>...</observation>. Based on this observation, you can think again and refine. 
- The returned dataframe will be truncated in 50 rows if observation is too long. 
- If you find no further exploration is needed or reaches max turns, you MUST directly provide the final SQL query solution inside <solution>...</solution>.
...
{db_details}: 
CREATE TABLE country_aliases (
    alias_id INTEGER,
    country_id INTEGER,
    alias_name TEXT,
    alias_type TEXT,
    PRIMARY KEY (alias_id),
    FOREIGN KEY (country_id) REFERENCES countries (country_id)
);
...
{question}: 
Could you please provide me with the English name, current population, average population over the last five years, and population from last year for each country, based on the country's population metrics? I need this information to analyze population trends.

```

--------------------------------

### Install SkyRL-Gym from PyPI

Source: https://github.com/novasky-ai/skyrl/blob/main/skyrl-gym/README.md

Install the latest release of the skyrl-gym library using pip.

```bash
pip install skyrl-gym
```

--------------------------------

### Install gdown

Source: https://github.com/novasky-ai/skyrl/blob/main/examples/train/livecodebench/lcb.md

Installs the gdown package, which is used for downloading files from Google Drive.

```bash
pip install gdown
```

--------------------------------

### Run OpenEnv Training Script

Source: https://github.com/novasky-ai/skyrl/blob/main/docs/content/docs/examples/openenv.mdx

Executes the example training script for Qwen2.5-1.5B-Instruct on OpenEnv environments. Customize with environment variables like ENV_NAME and NUM_GPUS.

```bash
# Execute from SkyRL root directory
cd SkyRL
bash integrations/openenv/run_openenv.sh

```

```bash
ENV_NAME=echo_env NUM_GPUS=2 bash integrations/openenv/run_openenv.sh

```

--------------------------------

### Start Ray Head Node

Source: https://github.com/novasky-ai/skyrl/blob/main/docs/content/docs/getting-started/installation.mdx

Initialize a single-node Ray cluster by starting the head node.

```bash
ray start --head
# sanity check
# ray status
```

--------------------------------

### Launch PPO Training Run

Source: https://github.com/novasky-ai/skyrl/blob/main/docs/content/docs/examples/ppo.mdx

Execute the training script to start the PPO training process. This command assumes the configuration is set up in `examples/train/ppo/run_ppo.sh`.

```bash
bash examples/train/ppo/run_ppo.sh
```

--------------------------------

### Install System Dependencies

Source: https://github.com/novasky-ai/skyrl/blob/main/docs/content/docs/getting-started/installation.mdx

Installs essential system packages including build-essential and libnuma-dev required for SkyRL.

```bash
sudo apt update && sudo apt-get install build-essential libnuma-dev
```

--------------------------------

### Run VisGym from Instruct Model

Source: https://github.com/novasky-ai/skyrl/blob/main/examples/train/visgym/README.md

Launches the VisGym RL training using a vanilla Qwen3-VL-8B-Instruct model. This recipe uses keyword actions and a task-only reward, with KL regularization enabled. The dataset is auto-generated on the first run.

```bash
bash examples/train/visgym/run_visgym_from_instruct.sh
```

--------------------------------

### Install SkyRL Tx for TPU

Source: https://github.com/novasky-ai/skyrl/blob/main/skyrl-tx/README.md

Clones the SkyRL repository and installs dependencies for TPU usage with the tinker API.

```bash
git clone https://github.com/NovaSky-AI/SkyRL
cd SkyRL/

# For TPU
uv run --extra tpu --extra tinker -m skyrl.tinker.api --base-model <model>
```

--------------------------------

### Install SkyRL Tx for GPU

Source: https://github.com/novasky-ai/skyrl/blob/main/skyrl-tx/README.md

Clones the SkyRL repository and installs dependencies for GPU usage with the tinker API.

```bash
git clone https://github.com/NovaSky-AI/SkyRL
cd SkyRL/

# For GPU
uv run --extra gpu --extra tinker -m skyrl.tinker.api --base-model <model>
```

--------------------------------

### Install OpenEnv Environments

Source: https://github.com/novasky-ai/skyrl/blob/main/docs/content/docs/examples/openenv.mdx

Installs the necessary Docker images for the OpenEnv environments. Execute from the SkyRL root directory.

```bash
# Execute from SkyRL root directory
cd SkyRL
uv run integrations/openenv/install_environment.py echo-env
# Or install all environments:
# uv run integrations/openenv/install_environment.py

```

--------------------------------

### VisGym Training Configuration (Instruct Recipe)

Source: https://github.com/novasky-ai/skyrl/blob/main/docs/content/docs/examples/visgym.mdx

Excerpt from the training script for the Instruct recipe, detailing key overrides for model path, strategy, VLM flags, multi-turn rollouts, and algorithm settings, including KL regularization.

```bash
uv run --isolated --extra fsdp \
  python examples/train/visgym/entrypoint.py \
  --env_variant instruct \
  trainer.policy.model.path="Qwen/Qwen3-VL-8B-Instruct" \
  trainer.strategy=fsdp \
  generator.vision_language_generator=true \
  generator.batched=false \
  trainer.remove_microbatch_padding=false \
  generator.max_turns=18 \
  generator.sampling_params.temperature=1 \
  trainer.algorithm.use_kl_loss=true \
  trainer.algorithm.kl_loss_coef=0.005 \
  environment.env_class=visgym \

```

--------------------------------

### API Reference Page Definition Example

Source: https://github.com/novasky-ai/skyrl/blob/main/docs/README.md

Example structure for defining API reference pages in the api-pages.yaml file.

```yaml
- path: skyrl
  pages:
    - slug: my-page
      title: My Page
      description: Description for the page.
      sections:
        - heading: Section Name
          description: Optional section description.
          objects:
            - skyrl.module.ClassName
```

--------------------------------

### Launch VisGym Instruct Recipe

Source: https://github.com/novasky-ai/skyrl/blob/main/docs/content/docs/examples/visgym.mdx

Use this command to launch the VisGym training recipe when no SFT checkpoint is required. It directly uses instruct-based training.

```bash
# Instruct recipe — no SFT checkpoint required
bash examples/train/visgym/run_visgym_from_instruct.sh
```

--------------------------------

### Install Other Retriever Python Packages

Source: https://github.com/novasky-ai/skyrl/blob/main/examples/train/search/README.md

Installs essential Python libraries for the retriever, including transformers, datasets, pyserini, and huggingface_hub.

```bash
# Install other Python packages
pip install transformers datasets pyserini huggingface_hub
```

--------------------------------

### Typical Harbor Training Invocation

Source: https://github.com/novasky-ai/skyrl/blob/main/docs/content/docs/harbor/index.mdx

A comprehensive example of launching a Harbor training job with various configuration parameters. Adjust data paths, model paths, and other settings as needed.

```bash
uv run --isolated --extra fsdp --extra harbor \
  -m examples.train_integrations.harbor.entrypoints.main_harbor \
  data.train_data=$TRAIN_DATA \
  trainer.policy.model.path=Qwen/Qwen3-8B \
  generator.inference_engine.served_model_name=Qwen3-8B \
  harbor_trial_config.trials_dir=$TRIALS_DIR \
  harbor_trial_config.environment.type=daytona \
  trainer.algorithm.max_seq_len=32768 \
  generator.apply_overlong_filtering=true \
  generator.n_samples_per_prompt=8 \
  trainer.algorithm.advantage_estimator=grpo \
  trainer.train_batch_size=64 \
  generator.rate_limit.enabled=true \
  generator.rate_limit.trajectories_per_second=5 \
  generator.rate_limit.max_concurrency=512
```

--------------------------------

### Install Dependencies for Ray < 2.51.1

Source: https://github.com/novasky-ai/skyrl/blob/main/docs/content/docs/getting-started/installation.mdx

Install necessary dependencies including vllm and a specific Ray version for compatibility with SkyRL.

```bash
pip install vllm==0.9.2 --extra-index-url https://download.pytorch.org/whl/cu128
pip install ray==2.46.0 omegaconf==2.3.0 loguru==0.7.3 jaxtyping==0.3.2 pyarrow==20.0.0
```

--------------------------------

### Launch Training Job

Source: https://github.com/novasky-ai/skyrl/blob/main/examples/train/search/README.md

Sets the Weights & Biases API key and executes the main training script from the base environment. This script will package dependencies using uv.

```bash
    export WANDB_API_KEY=your_wandb_api_key
    bash examples/train/search/run_search.sh
```

--------------------------------

### VisGym Training Configuration (SFT Recipe)

Source: https://github.com/novasky-ai/skyrl/blob/main/docs/content/docs/examples/visgym.mdx

Excerpt from the training script for the SFT recipe, detailing key overrides for model path, strategy, VLM flags, multi-turn rollouts, and algorithm settings.

```bash
uv run --isolated --extra fsdp \
  python examples/train/visgym/entrypoint.py \
  --env_variant sft \
  trainer.policy.model.path="$MODEL_PATH" \
  trainer.strategy=fsdp \
  generator.vision_language_generator=true \
  generator.batched=false \
  trainer.remove_microbatch_padding=false \
  generator.max_turns=15 \
  generator.sampling_params.temperature=0.7 \
  trainer.algorithm.advantage_estimator="grpo" \
  trainer.algorithm.use_kl_loss=false \
  environment.env_class=visgym \

```

--------------------------------

### Run Training Script

Source: https://github.com/novasky-ai/skyrl/blob/main/docs/content/docs/tutorials/new_env.mdx

Sets the Weights & Biases API key or configures console logging, then executes the training script. Ensure NUM_GPUS environment variable is set appropriately for your hardware.

```bash
export WANDB_API_KEY=your_wandb_api_key  # or set trainer.logger="console" to print to stdout
bash examples/train/multiply/run_multiply.sh
```

--------------------------------

### Prepare LiveCodeBench Dataset

Source: https://github.com/novasky-ai/skyrl/blob/main/examples/train/livecodebench/lcb.md

Prepares the downloaded LiveCodeBench dataset for use by organizing it into a local directory.

```bash
python examples/livecodebench/lcb_dataset.py --dataset_dir ~/data/lcb/download --local_dir ~/data/lcb/
```

--------------------------------

### Install FAISS GPU Version

Source: https://github.com/novasky-ai/skyrl/blob/main/examples/train/search/README.md

Installs the GPU-accelerated version of FAISS (1.8.0) within the Conda environment. This is crucial for efficient similarity search.

```bash
# Install the GPU version of faiss
conda install faiss-gpu==1.8.0 -c pytorch -c nvidia -y
```

--------------------------------

### Install PyTorch with GPU Support

Source: https://github.com/novasky-ai/skyrl/blob/main/examples/train/search/README.md

Installs PyTorch version 2.6.0 with CUDA 12.4 support and torchvision/torchaudio. Ensure your CUDA toolkit is compatible.

```bash
# Install PyTorch (with GPU support) and related libraries
conda install numpy==1.26.4 # needed to stop incompatible version of numpy from being installed via pip
pip install torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0 --index-url https://download.pytorch.org/whl/cu124
```

--------------------------------

### Prepare Dummy Dataset

Source: https://github.com/novasky-ai/skyrl/blob/main/docs/content/docs/examples/openenv.mdx

Prepares training and validation datasets with example prompts for a specified OpenEnv environment. Execute from the SkyRL root directory.

```bash
# Execute from SkyRL root directory
cd SkyRL
uv run integrations/openenv/prepare_dummy_dataset.py --output_dir ~/data/openenv --env_name echo_env

```

--------------------------------

### Prepare BrowseComp-Plus Dataset

Source: https://github.com/novasky-ai/skyrl/blob/main/skyrl-agent/examples/README.md

Run this script to prepare the dataset and index for BrowseComp-Plus. Requires downloading the decrypted dataset first and specifying input/output paths.

```bash
python ./data/browsecomp-plus.py --input DECRYPTED_JSON_PATH --output BC_DATA
```

--------------------------------

### Start SkyRL Tx API Server

Source: https://github.com/novasky-ai/skyrl/blob/main/skyrl-tx/README.md

Starts the SkyRL Tx API server for GPU usage with the tinker API, specifying a base model.

```bash
uv run --extra gpu --extra tinker -m skyrl.tinker.api --base-model "Qwen/Qwen3-0.6B"
```

--------------------------------

### Start Ray Cluster Worker Node

Source: https://github.com/novasky-ai/skyrl/blob/main/integrations/arctic_rl/README.md

Starts a Ray cluster worker node, connecting it to the head node. This command should be run on each worker node.

```bash
# worker:
#   uv run --isolated --extra skyrl-train ray start --address=<head>:6379 --num-gpus=8
```

--------------------------------

### Launch VisGym SFT Recipe

Source: https://github.com/novasky-ai/skyrl/blob/main/docs/content/docs/examples/visgym.mdx

Use this command to launch the VisGym training recipe when an SFT checkpoint is available. Ensure MODEL_PATH points to your SFT checkpoint.

```bash
# SFT recipe — point MODEL_PATH at your checkpoint
MODEL_PATH=/path/to/your/sft_ckpt bash examples/train/visgym/run_visgym_from_sft.sh
```