### Download Models and Datasets with Hugging Face Hub

Source: https://github.com/thudm/slime/blob/main/docs/en/get_started/quick_start.md

Installs the huggingface_hub library and downloads example model weights (GLM-Z1-9B) and datasets (dapo-math-17k, aime-2024) using the `hf download` command.

```shell
pip install -U huggingface_hub

# Download model weights (GLM-Z1-9B)
hf download zai-org/GLM-Z1-9B-0414 --local-dir /root/GLM-Z1-9B-0414

# Download training dataset (dapo-math-17k)
hf download --repo-type dataset zhuzilin/dapo-math-17k \
  --local-dir /root/dapo-math-17k

# Download evaluation dataset (aime-2024)
hf download --repo-type dataset zhuzilin/aime-2024 \
  --local-dir /root/aime-2024
```

--------------------------------

### Download Models and Datasets with huggingface_hub

Source: https://github.com/thudm/slime/blob/main/docs/zh/get_started/quick_start.md

Installs the huggingface_hub library and provides commands to download model weights (GLM-Z1-9B) and datasets (dapo-math-17k, aime-2024) from Hugging Face.

```bash
pip install -U huggingface_hub

# 下载模型权重 (GLM-Z1-9B)
hf download zai-org/GLM-Z1-9B-0414 --local-dir /root/GLM-Z1-9B-0414

# 下载训练数据集 (dapo-math-17k)
hf download --repo-type dataset zhuzilin/dapo-math-17k \
  --local-dir /root/dapo-math-17k

# 下载评估数据集 (aime-2024)
hf download --repo-type dataset zhuzilin/aime-2024 \
  --local-dir /root/aime-2024
```

--------------------------------

### Standard Configuration for Separated Actor and Rollout

Source: https://github.com/thudm/slime/blob/main/docs/zh/get_started/quick_start.md

Demonstrates a standard configuration for Slime where training (Actor) and inference (Rollout) resources are specified separately using Ray. This setup allocates distinct GPU resources for each component.

```bash
ray job submit ... \
   -- python3 train.py \
   --actor-num-nodes 1 \
   --actor-num-gpus-per-node 4 \
   --rollout-num-gpus 4 \
   ... 
```

--------------------------------

### Start Ray Cluster for Multi-Node Training

Source: https://github.com/thudm/slime/blob/main/docs/en/get_started/quick_start.md

Provides bash commands to initiate a Ray cluster for distributed training. It shows how to start the head node with specified GPU resources and connect other nodes to the cluster.

```Bash
# Node0 (HEAD)
ray start --head --node-ip-address ${MASTER_ADDR} \
  --num-gpus 8 --disable-usage-stats

# Other Nodes
ray start --address=${MASTER_ADDR}:6379 --num-gpus 8
```

--------------------------------

### Environment Setup and Model Download

Source: https://github.com/thudm/slime/blob/main/docs/en/examples/qwen3-4B.md

This snippet covers the initial setup for the project, including cloning the repository, installing dependencies, and downloading the Qwen3-4B model and training/evaluation datasets using huggingface-cli.

```bash
cd /root/
git clone https://github.com/THUDM/slime.git
cd slime/
pip install -e .
```

```bash
# hf checkpoint
huggingface-cli download Qwen/Qwen3-4B --local-dir /root/Qwen3-4B

# train data
huggingface-cli download --repo-type dataset zhuzilin/dapo-math-17k \
  --local-dir /root/dapo-math-17k

# eval data
huggingface-cli download --repo-type dataset zhuzilin/aime-2024 \
  --local-dir /root/aime-2024
```

--------------------------------

### Run Slime Training Script

Source: https://github.com/thudm/slime/blob/main/docs/en/get_started/quick_start.md

Executes the Slime training script, using `run-glm4-9B.sh` as an example. This command initiates the training process after all prerequisites are met.

```shell
cd /root/slime
bash scripts/run-glm4-9B.sh
```

--------------------------------

### Install Slime in Docker Container

Source: https://github.com/thudm/slime/blob/main/docs/en/get_started/quick_start.md

Clones the Slime repository and installs it using pip within the Docker container. This ensures the library is set up correctly for use.

```shell
# Path can be adjusted according to actual situation
cd /root/
git clone https://github.com/THUDM/slime.git
cd slime
pip install -e .
```

--------------------------------

### Environment Setup for Search-R1

Source: https://github.com/thudm/slime/blob/main/examples/search-r1/README.md

Installs necessary dependencies and clones the Search-R1 repository. This includes setting up the Slime environment and installing specific Python packages like chardet.

```bash
cd /root/
git clone https://github.com/THUDM/slime.git
pip install -e .
# for Search R1
pip install chardet
```

--------------------------------

### Pull and Start Docker Container

Source: https://github.com/thudm/slime/blob/main/docs/en/get_started/quick_start.md

Pulls the latest Slime Docker image and starts an interactive container with GPU and shared memory configurations. This is the recommended way to set up the environment to avoid potential configuration issues.

```shell
# Pull the latest image
docker pull zhuzilin/slime:latest

# Start the container
docker run --rm --gpus all --ipc=host --shm-size=16g \
  --ulimit memlock=-1 --ulimit stack=67108864 \
  -it zhuzilin/slime:latest /bin/bash
```

--------------------------------

### Run Qwen3-4B Fully Async Example

Source: https://github.com/thudm/slime/blob/main/examples/fully_async/README.md

This bash script executes the fully asynchronous rollout example using the Qwen3-4B model. It navigates to the slime directory and then runs the example script, demonstrating the setup for continuous async rollout generation.

```bash
cd slime
bash examples/fully_async/run-qwen3-4b-fully_async.sh
```

--------------------------------

### Docker: Pull and Run Slime Image

Source: https://github.com/thudm/slime/blob/main/docs/zh/get_started/quick_start.md

Pulls the latest Slime Docker image and starts an interactive container with GPU support and increased shared memory. This is the recommended way to set up the environment.

```shell
# 拉取最新镜像
docker pull zhuzilin/slime:latest

# 启动容器
docker run --rm --gpus all --ipc=host --shm-size=16g \
  --ulimit memlock=-1 --ulimit stack=67108864 \
  -it zhuzilin/slime:latest /bin/bash
```

--------------------------------

### Setup Environment and Download Models

Source: https://github.com/thudm/slime/blob/main/docs/en/examples/glm4-9B.md

This snippet outlines the initial steps to set up the environment by cloning the Slime repository, installing dependencies, and downloading the GLM4-9B model and associated datasets using huggingface-cli.

```bash
cd /root/
git clone https://github.com/THUDM/slime.git
cd slime/
pip install -e .

# hf checkpoint
huggingface-cli download zai-org/GLM-Z1-9B-0414 --local-dir /root/GLM-Z1-9B-0414

# train data
huggingface-cli download --repo-type dataset zhuzilin/dapo-math-17k \
  --local-dir /root/dapo-math-17k

# eval data
huggingface-cli download --repo-type dataset zhuzilin/aime-2024 \
  --local-dir /root/aime-2024
```

--------------------------------

### Dynamic Sampling Batch Size Example

Source: https://github.com/thudm/slime/blob/main/docs/zh/get_started/quick_start.md

Illustrates the relationship between `rollout_batch_size`, `n_samples_per_prompt`, and `over_sampling_batch_size` for dynamic sampling. It shows how to configure these parameters to control the sampling and filtering process.

```bash
   --rollout-batch-size 32 \
   --n-samples-per-prompt 8 \
   --over-sampling-batch-size 64 \

```

--------------------------------

### Install Slime within Docker

Source: https://github.com/thudm/slime/blob/main/docs/zh/get_started/quick_start.md

Clones the Slime repository and installs it using pip within the Docker container. This command assumes you are in the root directory of the container.

```bash
# 路径可根据实际情况调整
cd /root/
git clone https://github.com/THUDM/slime.git
cd slime
pip install -e .
```

--------------------------------

### Run Slime Training Script

Source: https://github.com/thudm/slime/blob/main/docs/zh/get_started/quick_start.md

Executes the main training script for Slime. This example shows how to run the script for the GLM4-9B model.

```bash
cd /root/slime
bash scripts/run-glm4-9B.sh
```

--------------------------------

### Initialize SLIME Environment

Source: https://github.com/thudm/slime/blob/main/docs/zh/examples/glm4-9B.md

This snippet shows how to clone the SLIME repository, navigate into the directory, and install the project using pip. This is the initial setup step for using SLIME.

```bash
cd /root/
git clone https://github.com/THUDM/slime.git
cd slime/
pip install -e .
```

--------------------------------

### Load Qwen3-4B-FP8 Model for Inference

Source: https://github.com/thudm/slime/blob/main/docs/en/get_started/quick_start.md

Command to download the Qwen3-4B-FP8 model for fp8 inference. It specifies the local directory for the downloaded model and configures the `--hf-checkpoint` and `--ref-load` parameters for Slime.

```bash
hf download Qwen/Qwen3-4B-FP8 --local-dir /root/Qwen3-4B-FP8
```

```bash
# Used to load tokenizer and other information, actually won't use model weight parameters from hf path
   --hf-checkpoint /root/Qwen3-4B-FP8

   # The megatron checkpoint still needs to be the dist weights converted from bf16 huggingface at the beginning, not modified because of FP rollout.
   --ref-load /root/Qwen3-4B_torch_dist
```

--------------------------------

### Slime: Model Configuration Loading

Source: https://github.com/thudm/slime/blob/main/docs/zh/get_started/quick_start.md

Demonstrates how to load model configuration parameters for Megatron by sourcing a shell script. It also shows how to override specific parameters if needed.

```bash
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd)"
source "${SCRIPT_DIR}/models/glm4-9B.sh"

# Example of overriding a parameter:
# MODEL_ARGS+=(--rotary-base 10000)
```

--------------------------------

### Slime: Load Qwen3-4B-FP8 Model for FP8 Inference

Source: https://github.com/thudm/slime/blob/main/docs/zh/get_started/quick_start.md

Provides bash commands to download the Qwen3-4B-FP8 model and configure Slime for FP8 inference. It specifies the necessary arguments for loading the model weights and tokenizer, enabling efficient inference.

```bash
hf download Qwen/Qwen3-4B-FP8 --local-dir /root/Qwen3-4B-FP8
```

```bash
# 用于加载 tokenizer 等其他信息，实际上不会使用 hf 路径中的模型权重参数
   --hf-checkpoint /root/Qwen3-4B-FP8

   #  megatron checkpoint 还需要是最开始用 bf16 的 huggingface 转换的 dist 权重，不因为 FP rollout 而去做修改。
   --ref-load /root/Qwen3-4B_torch_dist
```

--------------------------------

### Configure Optimizer Parameters (OPTIMIZER_ARGS)

Source: https://github.com/thudm/slime/blob/main/docs/en/get_started/quick_start.md

Sets optimizer parameters for model training, including the optimizer type, learning rate, and decay style. This example uses the Adam optimizer with a constant learning rate decay.

```bash
OPTIMIZER_ARGS=(
   --optimizer adam
   --lr 1e-6
   --lr-decay-style constant
   --weight-decay 0.1
   --adam-beta1 0.9
   --adam-beta2 0.98
)
```

--------------------------------

### Submit Job with Standard Disaggregated Configuration

Source: https://github.com/thudm/slime/blob/main/docs/en/get_started/quick_start.md

Submits a Ray job with separate resources allocated for training (Actor) and inference (Rollout). Actor uses 4 GPUs, and Rollout also uses 4 GPUs, running in parallel.

```bash
ray job submit ... \
   -- python3 train.py \
   --actor-num-nodes 1 \
   --actor-num-gpus-per-node 4 \
   --rollout-num-gpus 4 \
   ... 
```

--------------------------------

### Slime: Rollout and Training Parameters

Source: https://github.com/thudm/slime/blob/main/docs/zh/get_started/quick_start.md

Configures parameters for the data sampling (Rollout) and model training phases in Slime. This includes dataset paths, batch sizes, sample counts, and training loop controls.

```bash
ROLLOUT_ARGS=(
   # Prompt 数据集，JSONL 格式
   --prompt-data /root/dapo-math-17k/dapo-math-17k.jsonl
   --input-key prompt
   --label-key label
   # 若 Prompt 的 `input_key` 是 OpenAI message 格式，则应用 Chat Template
   --apply-chat-template
   # 是否在 Rollout 阶段打乱数据
   --rollout-shuffle

   # Reward Model 类型。slime 内置多种类型，也支持通过 --custom-rm-path 自定义
   --rm-type deepscaler

   # 这五个参数来控制 rollout 与 train 的关系
   --num-rollout 3000
   --rollout-batch-size 16
   --n-samples-per-prompt 8
   --num-steps-per-rollout 1
   --global-batch-size 128
   
   # Rollout 采样参数
   --rollout-max-response-len 8192
   --rollout-temperature 0.8

   # 对 rollout 阶段收集的数据进行负载均衡。它确保了分配到每个训练进程（DP rank）的计算任务量大致相等，可能对训练速度有好处
   --balance-data
)
```

--------------------------------

### Configure SGLang Service Arguments (SGLANG_ARGS)

Source: https://github.com/thudm/slime/blob/main/docs/zh/get_started/quick_start.md

Sets parameters for the SGLang inference service. Allows specifying the number of GPUs per engine and passing other SGLang arguments with a '--sglang-' prefix.

```bash
SGLANG_ARGS=(
   --rollout-num-gpus-per-engine 2
)
```

--------------------------------

### Evaluation Parameters (Bash)

Source: https://github.com/thudm/slime/blob/main/docs/en/get_started/quick_start.md

Configures parameters for the evaluation phase, allowing for different strategies than training. This includes setting evaluation intervals, prompt datasets, and sampling parameters.

```bash
EVAL_ARGS=(
   # Evaluation interval (number of Rollouts)
   --eval-interval 5
   # Prompt dataset for evaluation
   --eval-prompt-data aime /root/aime-2024/aime-2024.jsonl
   # Number of samples per evaluation Prompt
   --n-samples-per-eval-prompt 16
   # Maximum response length during evaluation
   --eval-max-response-len 16384
   # Sampling parameters during evaluation
   --eval-top-p 0.7
)
```

--------------------------------

### Dynamic Sampling Example Parameters

Source: https://github.com/thudm/slime/blob/main/docs/en/examples/qwen3-4B.md

Provides an example configuration for dynamic sampling, specifying batch sizes and the number of samples per prompt. This setup influences how data is collected and processed.

```bash
   --rollout-batch-size 32 \
   --n-samples-per-prompt 8 \
   --over-sampling-batch-size 64 \

```

--------------------------------

### Configure Slime Training with ROLLOUT_ARGS

Source: https://github.com/thudm/slime/blob/main/docs/en/get_started/quick_start.md

This bash script configures the Slime training process by specifying the dataset file, mapping input prompt and label keys, and loading a pre-constructed metadata column. This allows custom generation and reward functions to access contextual information.

```bash
ROLLOUT_ARGS=(
   # 1. Specify the preprocessed dataset file
   --prompt-data /root/nq_search/train_processed.json

   # 2. Map "question" column to input prompt
   --prompt-key question

   # 3. Map "final_answer" column to evaluation label
   --label-key final_answer

   # 4. Load the pre-constructed "metadata" column into Sample.metadata
   #    slime will automatically parse it as a Python dictionary
   --metadata-key metadata
)
```

--------------------------------

### Execute Retool RL Training

Source: https://github.com/thudm/slime/blob/main/examples/retool/README.md

This snippet shows the command to start the Reinforcement Learning (RL) training for the retool example using the Qwen3-4B model.

```bash
bash examples/retool/retool_qwen3_4b_rl.sh
```

--------------------------------

### Slime: Configure Data Mapping for Multiturn Adaptation

Source: https://github.com/thudm/slime/blob/main/docs/zh/get_started/quick_start.md

Illustrates how to configure Slime's data loading and mapping for multiturn conversational scenarios. It shows the bash arguments used to specify the prompt data file, the keys for prompts, labels, and the crucial metadata field.

```bash
ROLLOUT_ARGS=(
   # 1. 指定预处理后的数据集文件
   --prompt-data /root/nq_search/train_processed.json

   # 2. 将 "question" 列映射为输入 prompt
   --prompt-key question

   # 3. 将 "final_answer" 列映射为评估标签
   --label-key final_answer

   # 4. 将预先构造好的 "metadata" 列加载到 Sample.metadata
   #    slime 会自动将其解析为 Python 字典
   --metadata-key metadata
)
```

--------------------------------

### Checkpoint and Path Parameters (Bash)

Source: https://github.com/thudm/slime/blob/main/docs/en/get_started/quick_start.md

Configures checkpoint loading and saving paths for the model. It specifies paths for loading tokenizers, reference models, actor models, and saving trained models.

```bash
CKPT_ARGS=(
   # To load tokenizer and other information, won't actually use model weight parameters from hf path
   --hf-checkpoint /root/GLM-Z1-9B-0414
   # Reference Model's Megatron format checkpoint
   --ref-load /root/GLM-Z1-9B-0414_torch_dist
   # Actor model loading path. If empty, load from --ref-load
   --load /root/GLM-Z1-9B-0414_slime/
   # Model save path during training
   --save /root/GLM-Z1-9B-0414_slime/
   # Model save interval (steps)
   --save-interval 20
)
```

--------------------------------

### Configure SGLang Service Parameters (SGLANG_ARGS)

Source: https://github.com/thudm/slime/blob/main/docs/en/get_started/quick_start.md

Configures parameters for the SGLang inference service, including the number of GPUs per engine. Other SGLang parameters can be passed with the `--sglang-` prefix.

```bash
SGLANG_ARGS=(
   --rollout-num-gpus-per-engine 2
)
```

--------------------------------

### Configure GRPO Algorithm Arguments (GRPO_ARGS)

Source: https://github.com/thudm/slime/blob/main/docs/zh/get_started/quick_start.md

Sets parameters for the GRPO (Proximal Policy Optimization) algorithm. Includes options to enable KL loss calculation against a reference model and configure coefficients for KL divergence and entropy.

```bash
GRPO_ARGS=(
   --advantage-estimator grpo
   --use-kl-loss
   --kl-loss-coef 0.00
   --kl-loss-type low_var_kl
   --entropy-coef 0.00
   --eps-clip 0.2
   --eps-clip-high 0.28
)
```

--------------------------------

### Submit Job to Ray Cluster

Source: https://github.com/thudm/slime/blob/main/docs/en/get_started/quick_start.md

Illustrates how to submit a training job to a running Ray cluster using the `ray job submit` command. It includes configuring runtime environment variables and specifying the training script and its arguments.

```Bash
ray job submit --address="http://127.0.0.1:8265" \
   --runtime-env-json='{
     "env_vars": {
        "PYTHONPATH": "/root/Megatron-LM/",
        ... # e.g., no_proxy, API variables, etc.
     }
   }' \
   -- python3 train.py \
   --... # Other Megatron/SGLang/slime arguments
```

--------------------------------

### Configure Optimizer Arguments (OPTIMIZER_ARGS)

Source: https://github.com/thudm/slime/blob/main/docs/zh/get_started/quick_start.md

Specifies parameters for the optimizer, including the optimizer type (e.g., Adam), learning rate, learning rate decay style, weight decay, and Adam beta parameters.

```bash
OPTIMIZER_ARGS=(
   --optimizer adam
   --lr 1e-6
   --lr-decay-style constant
   --weight-decay 0.1
   --adam-beta1 0.9
   --adam-beta2 0.98
)
```

--------------------------------

### Slime: Checkpoint and Path Arguments

Source: https://github.com/thudm/slime/blob/main/docs/zh/get_started/quick_start.md

Defines common checkpoint and path arguments for Slime training, including paths for Hugging Face checkpoints, Megatron reference loads, model loads, and saves, as well as save intervals.

```bash
CKPT_ARGS=(
   # 用于加载 tokenizer 等其他信息，实际上不会使用 hf 路径中的模型权重参数
   --hf-checkpoint /root/GLM-Z1-9B-0414
   # 参考模型 (Reference Model) 的 Megatron 格式检查点
   --ref-load /root/GLM-Z1-9B-0414_torch_dist
   # Actor 模型的加载路径。若为空，则从 --ref-load 加载
   --load /root/GLM-Z1-9B-0414_slime/
   # 训练过程中模型的保存路径
   --save /root/GLM-Z1-9B-0414_slime/
   # 模型保存间隔（步数）
   --save-interval 20
)
```

--------------------------------

### Load Model Configuration (Bash)

Source: https://github.com/thudm/slime/blob/main/docs/en/get_started/quick_start.md

Loads model configuration hyperparameters required by Megatron from a specified shell script. This is necessary as Megatron cannot directly read configurations from checkpoints.

```bash
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd)"
source "${SCRIPT_DIR}/models/glm4-9B.sh"
```

--------------------------------

### Data Generation (Rollout) Parameters (Bash)

Source: https://github.com/thudm/slime/blob/main/docs/en/get_started/quick_start.md

Sets parameters for the data sampling phase of the training process. This includes defining batch sizes, number of samples per prompt, and sampling temperatures.

```bash
ROLLOUT_ARGS=(
   # Prompt dataset, JSONL format
   --prompt-data /root/dapo-math-17k/dapo-math-17k.jsonl
   --input-key prompt
   --label-key label
   # If the `input_key` of Prompt is in OpenAI message format, apply Chat Template
   --apply-chat-template
   # Whether to shuffle data in Rollout phase
   --rollout-shuffle

   # Reward Model type. slime has built-in multiple types, also supports custom through --custom-rm-path
   --rm-type deepscaler

   # These five parameters control the relationship between rollout and train
   --num-rollout 3000
   --rollout-batch-size 16
   --n-samples-per-prompt 8
   --num-steps-per-rollout 1
   --global-batch-size 128
   
   # Rollout sampling parameters
   --rollout-max-response-len 8192
   --rollout-temperature 0.8

   # Load balancing for data collected in rollout phase. It ensures that the computational workload allocated to each training process (DP rank) is roughly equal, which may be beneficial for training speed
   --balance-data
)
```

--------------------------------

### Configure Evaluation Arguments (EVAL_ARGS)

Source: https://github.com/thudm/slime/blob/main/docs/zh/get_started/quick_start.md

Sets parameters for the evaluation process, allowing for different evaluation strategies than training. Includes settings for evaluation interval, prompt dataset, number of samples per prompt, maximum response length, and sampling parameters.

```bash
EVAL_ARGS=(
   # 评估间隔（Rollout 数）
   --eval-interval 5
   # 评估用的 Prompt 数据集
   --eval-prompt-data aime /root/aime-2024/aime-2024.jsonl
   # 每个评估 Prompt 的采样数量
   --n-samples-per-eval-prompt 16
   # 评估时最大响应长度
   --eval-max-response-len 16384
   # 评估时采样参数
   --eval-top-p 0.7
)
```

--------------------------------

### Configure Custom Functions in Training Script

Source: https://github.com/thudm/slime/blob/main/docs/en/get_started/quick_start.md

Demonstrates how to enable custom generation and reward functions in a training script using command-line arguments. It specifies the paths to the Python modules and function names.

```Bash
CUSTOM_ARGS=(
   # Specify the path of custom generation function (format: path.to.your.file:function_name)
   --custom-generate-function-path your_module.multiturn_logic:generate

   # Specify the path of custom reward function
   --custom-rm-path your_module.multiturn_logic:reward_func
)
```

--------------------------------

### Enable Partial Rollout in Slime

Source: https://github.com/thudm/slime/blob/main/docs/en/get_started/quick_start.md

Activates the partial rollout feature in Slime to cache and continue generating incomplete samples, optimizing resource usage. It also allows customization of data extraction from the cache.

```bash
   --partial-rollout
```

--------------------------------

### Pop First Buffer Strategy

Source: https://github.com/thudm/slime/blob/main/docs/en/get_started/quick_start.md

Implements a 'pop_first' strategy for extracting samples from a cache buffer in Slime's partial rollout. It retrieves samples in a first-in-first-out manner up to the specified number.

```python
def pop_first(args, rollout_id, buffer: list[list[Sample]], num_samples: int) -> list[list[Sample]]:
    num_to_pop = min(len(buffer), num_samples)
    samples = buffer[:num_to_pop]
    del buffer[:num_to_pop]
    return samples
```

--------------------------------

### Slime: Implement Pop First Strategy for Partial Rollout Buffer

Source: https://github.com/thudm/slime/blob/main/docs/zh/get_started/quick_start.md

Demonstrates the 'pop_first' strategy for retrieving samples from a buffer during partial rollouts in Slime. This function defines how samples are extracted based on a first-in, first-out order, managing buffer contents and sample counts.

```python
def pop_first(args, rollout_id, buffer: list[list[Sample]], num_samples: int) -> list[list[Sample]]:
    num_to_pop = min(len(buffer), num_samples)
    samples = buffer[:num_to_pop]
    del buffer[:num_to_pop]
    return samples
```

--------------------------------

### Colocated Configuration for Actor and Rollout

Source: https://github.com/thudm/slime/blob/main/docs/zh/get_started/quick_start.md

Enables a colocated configuration where training (Actor) and inference (Rollout) share the same set of GPUs. The `--colocate` flag is used, and `--rollout-num-gpus` is ignored, making both components utilize the same GPU resources.

```bash
ray job submit ... \
   -- python3 train.py \
   --actor-num-nodes 1 \
   --actor-num-gpus-per-node 8 \
   --colocate \
   ... 
```

--------------------------------

### Submit Job with Training-Inference Integration (Colocated)

Source: https://github.com/thudm/slime/blob/main/docs/en/get_started/quick_start.md

Submits a Ray job to colocate training and inference on the same GPUs. The `--colocate` flag is added, and `--rollout-num-gpus` is ignored, making training and inference share all specified GPUs.

```bash
ray job submit ... \
   -- python3 train.py \
   --actor-num-nodes 1 \
   --actor-num-gpus-per-node 8 \
   --colocate \
   ... 
```

--------------------------------

### Configure GRPO Algorithm Parameters (GRPO_ARGS)

Source: https://github.com/thudm/slime/blob/main/docs/en/get_started/quick_start.md

Configures parameters for the GRPO algorithm, including KL divergence loss calculation and coefficients. `--use-kl-loss` enables KL divergence monitoring against a reference model, with `--kl-loss-coef` controlling its contribution to the final loss.

```bash
GRPO_ARGS=(
   --advantage-estimator grpo
   --use-kl-loss
   --kl-loss-coef 0.00
   --kl-loss-type low_var_kl
   --entropy-coef 0.00
   --eps-clip 0.2
   --eps-clip-high 0.28
)
```

--------------------------------

### Configure Performance and Parallelism Arguments (PERF_ARGS)

Source: https://github.com/thudm/slime/blob/main/docs/zh/get_started/quick_start.md

Configures Megatron's parallelism settings and Slime-specific optimizations like dynamic batch size and max tokens per GPU. These arguments aim to enhance training efficiency by optimizing resource utilization and batching strategies.

```bash
PERF_ARGS=(
   --tensor-model-parallel-size 2
   --sequence-parallel
   --pipeline-model-parallel-size 1
   --context-parallel-size 2
   --expert-model-parallel-size 1
   --expert-tensor-parallel-size 1

   --recompute-granularity full
   --recompute-method uniform
   --recompute-num-layers 1

   # --micro-batch-size 1 # 启用动态批处理后此项被忽略
   --use-dynamic-batch-size
   --max-tokens-per-gpu 4608
)
```

--------------------------------

### Dynamic Sampling Filter Function Example

Source: https://github.com/thudm/slime/blob/main/docs/zh/get_started/quick_start.md

Provides a Python code example for a dynamic sampling filter function (`check_reward_nonzero_std`). This function takes a list of samples and returns a boolean indicating whether the standard deviation of rewards within the samples is greater than zero.

```python
def check_reward_nonzero_std(args, samples: list[Sample], **kwargs):
    rewards = [sample.reward for sample in samples]
    return torch.tensor(rewards, dtype=torch.float).std() > 0.0
```

--------------------------------

### Install SLIME Locally

Source: https://github.com/thudm/slime/blob/main/docs/en/platform_support/amd_tutorial.md

Clones the SLIME repository and installs it locally using pip. This step is necessary after starting the Docker container or if building from source.

```bash
git clone https://github.com/THUDM/slime.git
cd slime
pip install -e .

```

--------------------------------

### SGLang Data Entry Example

Source: https://github.com/thudm/slime/blob/main/docs/en/get_started/usage.md

An example of a single data entry in JSONL format for SGLang, including a user prompt with complex mathematical content and a corresponding label.

```json
{
  "prompt": [
    {
      "content": "Solve the following math problem step by step. The last line of your response should be of the form Answer: \\boxed{$Answer} where $Answer is the answer to the problem.\n\nIn triangle $ABC$, $\\sin \angle A = \\frac{4}{5}$ and $\\angle A < 90^\\circ$. Let $D$ be a point outside triangle $ABC$ such that $\\angle BAD = \\angle DAC$ and $\\angle BDC = 90^\\circ$. Suppose that $AD = 1$ and that $\\frac{BD}{CD} = \\frac{3}{2}$. If $AB + AC$ can be expressed in the form $\\frac{a\\sqrt{b}}{c}$ where $a, b, c$ are pairwise relatively prime integers, find $a + b + c$.\n\nRemember to put your answer on its own line after \"Answer:\".",
      "role": "user"
    }
  ],
  "label": "34"
}
```

--------------------------------

### Run Rollout Buffer Example Script

Source: https://github.com/thudm/slime/blob/main/slime_plugins/rollout_buffer/README.md

This script demonstrates how to set up and run the Rollout Buffer example. It involves navigating to the directory, executing a bash script, and then running the main buffer Python script in a separate terminal.

```bash
cd slime_plugins/rollout_buffer
bash rollout_buffer_example.sh
```

```python
python buffer.py
```

--------------------------------

### SGLang Multi-Node MoE Configuration

Source: https://github.com/thudm/slime/blob/main/docs/en/examples/qwen3-30B-A3B.md

Provides SGLang arguments for multi-node training, including redundant experts for scenarios where the number of GPUs is not a multiple of experts. This example is for a 24-GPU setup.

```bash
SGLANG_ARGS=(
   --rollout-num-gpus-per-engine 24
   --sglang-mem-fraction-static 0.7
   --sglang-enable-ep-moe
   --sglang-enable-dp-attention
   --sglang-dp-size 3

   --sglang-moe-dense-tp-size 1
   --sglang-enable-dp-lm-head
   --sglang-ep-num-redundant-experts 16   
)
```

--------------------------------

### Override Model Configuration (Bash)

Source: https://github.com/thudm/slime/blob/main/docs/en/get_started/quick_start.md

Demonstrates how to override specific model configuration parameters after sourcing the default configuration file. This is useful for fine-tuning parameters like rotary base.

```bash
source "${SCRIPT_DIR}/models/glm4-9B.sh"
MODEL_ARGS+=(--rotary-base 10000)
```

--------------------------------

### Configure Environment with Slime

Source: https://github.com/thudm/slime/blob/main/examples/search-r1/README_zh.md

This snippet shows the bash commands to clone the Slime repository, install it, and set up the necessary environment for Search-R1, including downloading data and models.

```bash
cd /root/
git clone https://github.com/THUDM/slime.git
pip install -e .
# for Search R1
pip install chardet
```

```bash
git clone https://github.com/PeterGriffinJin/Search-R1.git
cd Search-R1/
python scripts/data_process/nq_search.py --local_dir /root/nq_search/
```

```bash
# hf checkpoint
huggingface-cli download Qwen/Qwen2.5-3B --local-dir /root/Qwen2.5-3B

# mcore checkpoint
cd /root/slime
source scripts/models/qwen2.5-3B.sh
PYTHONPATH=/root/Megatron-LM python tools/convert_hf_to_torch_dist.py \
    ${MODEL_ARGS[@]} \
    --hf-checkpoint /root/Qwen2.5-3B \
    --save /root/Qwen2.5-3B_torch_dist
```

--------------------------------

### Rollout Buffer Example Script

Source: https://github.com/thudm/slime/blob/main/slime_plugins/rollout_buffer/README_zh.md

This snippet shows how to execute the Rollout Buffer example script. It involves navigating to the script's directory and running two commands, one in the current terminal and another in a separate terminal, to initiate the buffer and the Python script.

```Bash
cd slime_plugins/rollout_buffer
bash rollout_buffer_example.sh
```

```Python
python buffer.py
```

--------------------------------

### Install and Install Pre-commit Hooks

Source: https://github.com/thudm/slime/blob/main/README_zh.md

This snippet shows how to install the pre-commit tool and then install the git pre-commit hooks for the project. Pre-commit helps maintain code style consistency.

```bash
apt install pre-commit -y
pre-commit install
```

--------------------------------

### Custom Reward Function in Python

Source: https://github.com/thudm/slime/blob/main/docs/en/get_started/quick_start.md

Defines an asynchronous Python function to calculate reward scores for a given sample. It receives a Sample object containing the interaction results and returns a float representing the calculated score.

```Python
async def reward_func(args, sample: Sample, **kwargs) -> float:
```

--------------------------------

### Check Reward Nonzero Standard Deviation

Source: https://github.com/thudm/slime/blob/main/docs/en/get_started/quick_start.md

A Python function for dynamic sampling in Slime. It calculates the standard deviation of rewards for a batch of samples and returns True if it's greater than zero, indicating reward diversity.

```python
def check_reward_nonzero_std(args, samples: list[Sample], **kwargs):
    rewards = [sample.reward for sample in samples]
    return torch.tensor(rewards, dtype=torch.float).std() > 0.0
```

--------------------------------

### Load Debug Rollout Data in Slime

Source: https://github.com/thudm/slime/blob/main/docs/zh/developer_guide/debug.md

This option loads previously saved rollout data, automatically setting `debug_train_only=True`. It allows fixing training inputs to tune the training part, for example, by switching parallel configurations.

```Python
python your_script.py --load-debug-rollout-data /your/saved/debug/data_{rollout_id}.pt
```

--------------------------------

### Qwen2.5-3B Model Initialization

Source: https://github.com/thudm/slime/blob/main/examples/search-r1/README.md

Downloads the Qwen2.5-3B model from Hugging Face and converts it to a PyTorch distributed format. This involves using `huggingface-cli` and a custom conversion script.

```bash
# hf checkpoint
huggingface-cli download Qwen/Qwen2.5-3B --local-dir /root/Qwen2.5-3B

# mcore checkpoint
cd /root/slime
source scripts/models/qwen2.5-3B.sh
PYTHONPATH=/root/Megatron-LM python tools/convert_hf_to_torch_dist.py \
    ${MODEL_ARGS[@]} \
    --hf-checkpoint /root/Qwen2.5-3B \
    --save /root/Qwen2.5-3B_torch_dist
```

--------------------------------

### Convert Hugging Face to Megatron Format

Source: https://github.com/thudm/slime/blob/main/docs/zh/get_started/quick_start.md

Converts Hugging Face model weights to Megatron's torch_dist format using a provided script. It requires sourcing a model-specific configuration file and specifying input/output paths.

```bash
cd /root/slime
source scripts/models/glm4-9B.sh

PYTHONPATH=/root/Megatron-LM python tools/convert_hf_to_torch_dist.py \
    ${MODEL_ARGS[@]} \
    --hf-checkpoint /root/GLM-Z1-9B-0414 \
    --save /root/GLM-Z1-9B-0414_torch_dist
```

--------------------------------

### Run Qwen2.5-3B with Search-R1

Source: https://github.com/thudm/slime/blob/main/examples/search-r1/README_zh.md

This bash command executes the Slime example script for running the Qwen2.5-3B model with Search-R1 capabilities. Ensure your API key is configured in the Python script before running.

```bash
cd slime/
bash examples/search-r1/run_qwen2.5_3B.sh
```