RLinf

https://rlinf.readthedocs.io/en/latest/
Admin
RLinf is a flexible and scalable open-source infrastructure designed for post-training foundation...
Tokens:191,792
Snippets:2,499
Trust Score:8.3
Update:2 months ago
Show doc for...
Tokens:
Raw
### Install dependencies

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/agentic/agentlightning_calc_x.html

Install the required packages for the AgentLightning calc_x example.

```bash
pip install "agentlightning==0.3.0" "autogen-agentchat" "autogen-ext[openai]" "mcp>=1.10.0" "mcp-server-calculator"
```

--------------------------------

### Additional UV Installation Targets

Source: https://rlinf.readthedocs.io/en/latest/rst_source/start/installation.html

Examples of various installation configurations for different models and environment targets.

```bash
bash requirements/install.sh embodied --model openvla-oft --env maniskill_libero
bash requirements/install.sh embodied --model openpi --env libero
bash requirements/install.sh agentic
bash requirements/install.sh docs
```

--------------------------------

### Install and Start PC Service

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/gim_arm.html

Install the XRoboToolkit PC service package and execute the startup script.

```bash
sudo dpkg -i XRoboToolkit_PC_Service_1.0.0_ubuntu_22.04_amd64.deb
bash /opt/apps/roboticsservice/runService.sh
```

--------------------------------

### Navigate to example directory

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/agentic/agentlightning_calc_x.html

Change the working directory to the calc_x example folder.

```bash
cd /path/to/RLinf/examples/agent/agentlightning/calc_x
```

--------------------------------

### Setup Custom Environment

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/genesis.html

Install Genesis dependencies and activate the virtual environment manually.

```bash
# Mainland China users can add --use-mirror.
bash requirements/install.sh embodied --env genesis
source .venv/bin/activate
```

--------------------------------

### Install Custom Environment

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/gr00t.html

Install dependencies and activate the virtual environment for local setups.

```bash
# For mainland China users, you can add the `--use-mirror` flag to the install.sh command for better download speed.
bash requirements/install.sh embodied --model gr00t --env maniskill_libero
source .venv/bin/activate
```

```bash
# For mainland China users, you can add the `--use-mirror` flag to the install.sh command for better download speed.
bash requirements/install.sh embodied --model gr00t_n1d6 --env maniskill_libero
source .venv/bin/activate
```

```bash
# For mainland China users, you can add the `--use-mirror` flag to the install.sh command for better download speed.
bash requirements/install.sh embodied --model gr00t_n1d7 --env maniskill_libero
source .venv/bin/activate
```

--------------------------------

### Install Dependencies

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/franka.html

Execute the installation script to set up the required environment for Franka control.

```bash
# For mainland China users, you can add the `--use-mirror` flag to the install.sh command for better download speed.

bash requirements/install.sh embodied --env franka
source .venv/bin/activate
```

--------------------------------

### Local Installation

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/co_training.html

Install dependencies locally and activate the virtual environment.

```bash
# Add --use-mirror for faster install in some regions
bash requirements/install.sh embodied --model openpi --env maniskill_libero
source .venv/bin/activate
```

--------------------------------

### Install Evaluation Environment

Source: https://rlinf.readthedocs.io/en/latest/rst_source/evaluations/get_started/installation.html

Execute the installation script from the repository root and activate the virtual environment.

```bash
bash requirements/install.sh embodied --model <model> --env <env>
source .venv/bin/activate
```

--------------------------------

### Install Dependencies

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/robocasa365.html

Commands to install required dependencies and activate the virtual environment.

```bash
export ROBOCASA_ASSETS_PATH=/path/to/robocasa_assets
bash requirements/install.sh embodied --model openpi --env robocasa365
source .venv/bin/activate
```

--------------------------------

### Install Custom Environment

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/abot_m0.html

Manually install dependencies for the ABot-M0 environment using the provided installation script.

```bash
# Optional: use local source checkouts instead of installer-managed clones.
# export ABOT_PATH=<path_to_ABot-Manipulation>
# export VGGT_PATH=<path_to_vggt>

# Add --use-mirror for faster downloads in mainland China.
bash requirements/install.sh embodied --model abot_m0 --env maniskill_libero
source .venv/bin/activate
```

--------------------------------

### Install gim_arm_control SDK

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/gim_arm.html

Clone and install the SDK providing CAN communication drivers and Python bindings.

```bash
# Clone the SDK alongside RLinf (the example below assumes ~/gim_arm_control).
cd ~
git clone https://github.com/RLinf/gim_arm_control.git
cd ~/gim_arm_control/python
pip install -e .
```

--------------------------------

### Install custom environment dependencies

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/metaworld.html

Set up the local environment manually using the provided installation script.

```bash
# Add --use-mirror for faster downloads in mainland China.
bash requirements/install.sh embodied --model openpi --env metaworld
# Or install the OpenVLA-OFT environment:
# bash requirements/install.sh embodied --model openvla-oft --env metaworld

source .venv/bin/activate
```

--------------------------------

### Native Installation of Dependencies

Source: https://rlinf.readthedocs.io/en/latest/rst_source/guides/ascend_cann.html

Install required packages for the Ascend platform and activate the virtual environment.

```bash
bash requirements/install.sh --platform ascend embodied --model openvla-oft --env libero
source .venv/bin/activate
```

```bash
bash requirements/install.sh --use-mirror --platform ascend embodied --model openvla-oft --env libero
```

--------------------------------

### Install FUSCO Library

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/system/fusco.html

Clone the repository, install the package, and download the required shared library.

```bash
# clone and install
git clone https://github.com/infinigence/FUSCO.git
cd FUSCO/
python setup.py install

# download the shared library
mkdir -p lib
curl -L -o lib/libfusco.so https://ghfast.top/https://github.com/infinigence/FUSCO/releases/download/v0.1/libfusco.so
```

--------------------------------

### Install Local Agentic Stack

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/agentic/wideseek_r1/index.html

Execute the installation script to set up the agentic environment locally.

```bash
bash requirements/install.sh agentic
```

--------------------------------

### Start Retrieval Service

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/agentic/wideseek_r1/tools.html

Execute the launch script to start the local retrieval server.

```bash
bash examples/agent/tools/search_local_server_qdrant/launch_local_server.sh
```

--------------------------------

### Install RLinf

Source: https://rlinf.readthedocs.io/en/latest/rst_source/start/cheat_sheet.html

Clones the repository and runs the installation script with specified model and environment dependencies.

```bash
git clone https://github.com/RLinf/RLinf.git
cd RLinf
bash requirements/install.sh embodied --model openvla --env maniskill_libero
```

--------------------------------

### Install BEHAVIOR-1K dependencies

Source: https://rlinf.readthedocs.io/en/latest/rst_source/evaluations/guides/behavior.html

Commands to install required packages for the BEHAVIOR environment and activate the virtual environment.

```bash
bash requirements/install.sh embodied --model openpi --env behavior
source .venv/bin/activate
```

--------------------------------

### Install SGLang and vLLM from Source

Source: https://rlinf.readthedocs.io/en/latest/rst_source/guides/version.html

Commands to clone and install SGLang and vLLM from their respective GitHub repositories.

```bash
# Install SGLang
git clone https://github.com/sgl-project/sglang.git
cd sglang
git checkout v0.4.8          # pick the tag you need
pip install -e "python[all]"

# Install vLLM
git clone https://github.com/vllm-project/vllm.git
cd vllm
git checkout v0.8.5          # pick the tag you need
pip install -e .
```

--------------------------------

### Install Custom Training Node Environment

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/hg-dagger.html

Installs the simulation environment dependencies and activates the virtual environment.

```bash
# For mainland China users, you can add the `--use-mirror` flag for better download speed.
bash requirements/install.sh embodied --model openpi --env maniskill_libero
source .venv/bin/activate
```

--------------------------------

### Install Software Dependencies

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/franka_pi0_sft_deploy.html

Run this command on the training or rollout node to install the required OpenPI model dependencies.

```bash
# For mainland China users, you can add `--use-mirror` to the install.sh command.
bash requirements/install.sh embodied --model openpi --env maniskill_libero
source .venv/bin/activate
```

--------------------------------

### Install Dependencies

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/dosw1.html

Commands to install environment dependencies for the DOS-W1 setup.

```bash
# Mainland China users can add --use-mirror for better download speed.
bash requirements/install.sh embodied --env dosw1
source .venv/bin/activate
```

--------------------------------

### Download Cold Start Components

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/sft_dreamzero.html

Download the necessary Wan2.2 and T5 components for training from scratch.

```bash
huggingface-cli download Wan-AI/Wan2.2-TI2V-5B --local-dir ./Wan2.2-TI2V-5B
huggingface-cli download Wan-AI/Wan2.1-I2V-14B-480P \
  models_clip_open-clip-xlm-roberta-large-vit-huge-14.pth --local-dir ./Wan2.1-CLIP
huggingface-cli download google/umt5-xxl --local-dir ./umt5-xxl
```

--------------------------------

### Install custom environment

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/behavior.html

Alternative installation method using a shell script to set up the environment for specific models.

```bash
# Add --use-mirror for faster downloads in mainland China.
bash requirements/install.sh embodied --model openvla-oft --env behavior
# Or install the OpenPI environment:
# bash requirements/install.sh embodied --model openpi --env behavior
source .venv/bin/activate
```

--------------------------------

### Install Custom Environment

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/dagger.html

Install dependencies for a custom environment setup.

```bash
# For mainland China users, you can add the `--use-mirror` flag for better download speed.
bash requirements/install.sh embodied --model openpi --env maniskill_libero
# For robotwin environment, please use the following command:
# bash requirements/install.sh embodied --model openpi --env robotwin
source .venv/bin/activate
```

--------------------------------

### Setup Docker Environment

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/genesis.html

Launch the prebuilt Docker container for the Genesis benchmark.

```bash
docker run -it --rm --gpus all \
   --shm-size 32g \
   --network host \
   --name rlinf \
   -v .:/workspace/RLinf \
   rlinf/rlinf:agentic-rlinf0.3-genesis

# For mainland China users:
# docker.1ms.run/rlinf/rlinf:agentic-rlinf0.3-genesis
```

--------------------------------

### Install RLinf Dependencies

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/franka_vr.html

Install environment dependencies for the Franka robot setup.

```bash
bash requirements/install.sh embodied --env franka
source .venv/bin/activate
```

--------------------------------

### Setup Environment

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/franka_vr.html

Initialize the virtual environment for the PICO software.

```bash
bash setup_uv.sh
```

--------------------------------

### Setup environment via Docker

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/maniskill.html

Run the prebuilt Docker image and switch to the virtual environment.

```bash
docker run -it --rm --gpus all \
   --shm-size 20g \
   --network host \
   --name rlinf \
   -v .:/workspace/RLinf \
   rlinf/rlinf:agentic-rlinf0.3-maniskill_libero
   # Mainland China mirror: docker.1ms.run/rlinf/rlinf:agentic-rlinf0.3-maniskill_libero

# Inside the container, switch to the model's virtual environment:
source switch_env openvla        # or: source switch_env openvla-oft
```

--------------------------------

### Install Training/Rollout Node Custom Environment

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/xsquare_turtle2.html

Install OpenVLA and ManiSkill-Libero dependencies for training or rollout nodes in a custom environment.

```bash
# install openvla + maniskill_libero environment on training / rollout nodes
# For mainland China users, you can add the `--use-mirror` flag for better speed.
bash requirements/install.sh embodied --model openvla --env maniskill_libero
source .venv/bin/activate
```

--------------------------------

### Set Up Custom Environment

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/dexbotic.html

Install dependencies manually and activate the virtual environment.

```bash
# Add --use-mirror for faster downloads in mainland China.
bash requirements/install.sh embodied --model dexbotic --env maniskill_libero
source .venv/bin/activate
```

--------------------------------

### Install Custom Environment Dependencies

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/lingbotvla.html

Installs native dependencies and activates the virtual environment for local setups.

```bash
bash requirements/install.sh embodied --model lingbotvla --env robotwin --use-mirror
source .venv/bin/activate
```

--------------------------------

### Install Robot Controller Dependencies

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/xsquare_turtle2.html

Install the necessary Python dependencies for the embodied real-world setup within the XSquare Docker container.

```bash
# For mainland China users, you can add the `--use-mirror` flag for better speed.
bash requirements/install.sh embodied --env xsquare_turtle2
source .venv/bin/activate
```

--------------------------------

### Setup PolaRiS Environment

Source: https://rlinf.readthedocs.io/en/latest/rst_source/evaluations/guides/polaris.html

Install required dependencies and configure the environment for PolaRiS evaluation.

```bash
bash requirements/install.sh embodied --model openpi --env polaris
source .venv/bin/activate
export POLARIS_DATA_PATH=/path/to/dataset/PolaRiS-Hub
```

--------------------------------

### Setup Environment via Docker

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/sft_openpi.html

Run the RLinf Docker container and switch to the OpenPI environment.

```bash
docker run -it --rm --gpus all \
   --shm-size 20g \
   --network host \
   --name rlinf \
   -v .:/workspace/RLinf \
   rlinf/rlinf:agentic-rlinf0.3-maniskill_libero
   # Mainland China mirror: docker.1ms.run/rlinf/rlinf:agentic-rlinf0.3-maniskill_libero

# Inside the container, switch to the OpenPI virtual environment:
source switch_env openpi
```

--------------------------------

### Launch Training Command

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/pi0.html

Use this command to start training by specifying the desired configuration file.

```bash
bash examples/embodiment/run_embodiment.sh CHOSEN_CONFIG
```

```bash
bash examples/embodiment/run_embodiment.sh libero_spatial_ppo_openpi_quickstart
```

--------------------------------

### Setup Retriever Environment

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/agentic/searchr1.html

Creates a conda environment and installs necessary dependencies for the local retrieval server.

```bash
conda create -n retriever python=3.10 -y
conda activate retriever

conda install pytorch==2.4.0 torchvision==0.19.0 torchaudio==2.4.0 pytorch-cuda=12.1 -c pytorch -c nvidia -y
pip install transformers datasets pyserini huggingface_hub

# Install GPU version of faiss
conda install faiss-gpu=1.8.0 -c pytorch -c nvidia -y

pip install uvicorn fastapi
```

--------------------------------

### Verify Installation

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/agentic/qwen3_vl_geo3k.html

Run this command after installation to confirm that torch, transformers, and sglang are correctly installed and accessible.

```python
source .venv/bin/activate
python -c "
import torch; print('torch:', torch.__version__)
import transformers; print('transformers:', transformers.__version__)
import sglang; print('sglang:', sglang.__version__)
print('All good')
"
```

--------------------------------

### Launch TensorBoard

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/agentic/coding_online_rl.html

Start the TensorBoard server to visualize training metrics from the specified log directory.

```bash
# Start TensorBoard
tensorboard --logdir results/grpo-1.5b
```

--------------------------------

### Start training

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/agentic/agentlightning_calc_x.html

Execute the training script using a specific configuration.

```bash
bash run_calc_x.sh qwen2.5-1.5b-enginehttp-multiturn
```

--------------------------------

### Native Installation of LIBERO Dependencies

Source: https://rlinf.readthedocs.io/en/latest/rst_source/guides/amd_rocm.html

Install dependencies natively on the host system using the provided installation script.

```bash
bash requirements/install.sh --platform amd --rocm 6.4 embodied --model openvla-oft --env libero
source .venv/bin/activate
```

```bash
bash requirements/install.sh --use-mirror --platform amd --rocm 6.4 embodied --model openvla-oft --env libero
```

--------------------------------

### Install ZED Python API

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/franka_zed_robotiq.html

Commands to install the ZED Python API via the bundled script or direct wheel installation.

```bash
# Activate your virtual environment first
source /path/to/your/venv/bin/activate

# Run the official install script
cd /usr/local/zed/
python3 get_python_api.py
```

```bash
python -m pip install --ignore-installed /usr/local/zed/pyzed-*.whl
```

--------------------------------

### Setup Environment via Docker

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/starvla.html

Launch the RLinf Docker container and activate the StarVLA environment.

```bash
docker run -it --rm --gpus all \
   --shm-size 20g \
   --network host \
   --name rlinf \
   -v .:/workspace/RLinf \
   rlinf/rlinf:agentic-rlinf0.3-maniskill_libero
   # Mainland China mirror: docker.1ms.run/rlinf/rlinf:agentic-rlinf0.3-maniskill_libero

# Inside the container, switch to the StarVLA virtual environment:
source switch_env starvla
```

--------------------------------

### Install gello-teleop

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/franka_gello.html

Clone and install the gello-teleop package as an editable checkout.

```bash
git clone https://github.com/RLinf/gello-teleop.git
cd gello-teleop
pip install -e .
```

--------------------------------

### Setup Docker Environment

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/libero.html

Run the RLinf container with the required image tag and switch to the model environment.

```bash
docker run -it --rm --gpus all \
   --shm-size 20g \
   --network host \
   --name rlinf \
   -v .:/workspace/RLinf \
   rlinf/rlinf:agentic-rlinf0.3-maniskill_libero
   # Mainland China mirror: docker.1ms.run/rlinf/rlinf:agentic-rlinf0.3-maniskill_libero

# Inside the container, switch to the model's virtual environment:
source switch_env openvla-oft
```

--------------------------------

### Set Up Environment via Docker

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/dexbotic.html

Run the RLinf Docker container and activate the Dexbotic virtual environment.

```bash
docker run -it --rm --gpus all \
   --shm-size 20g \
   --network host \
   --name rlinf \
   -v .:/workspace/RLinf \
   rlinf/rlinf:agentic-rlinf0.3-maniskill_libero
   # Mainland China mirror: docker.1ms.run/rlinf/rlinf:agentic-rlinf0.3-maniskill_libero

# Inside the container, switch to the Dexbotic virtual environment:
source switch_env dexbotic
```

--------------------------------

### Install the Environment

Source: https://rlinf.readthedocs.io/en/latest/rst_source/evaluations/get_started/quick_tour.html

Run these commands from the repository root to set up the LIBERO environment and activate the virtual environment.

```bash
bash requirements/install.sh embodied --model openpi --env libero
source .venv/bin/activate
```

--------------------------------

### Install Dependencies Manually

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/franka.html

Commands to install dependencies directly in a custom environment.

```bash
# For mainland China users, you can add the `--use-mirror` flag to the install.sh command for better download speed.

bash requirements/install.sh embodied --model openvla --env maniskill_libero
source .venv/bin/activate

```

--------------------------------

### Hardware Configuration YAML

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/franka_zed_robotiq.html

Example configuration block for setting up a Franka robot with ZED cameras and a Robotiq gripper.

```yaml
cluster:
  num_nodes: 2
  component_placement:
    actor:
      node_group: gpu
      placement: 0
    env:
      node_group: gpu
      placement: 0
    rollout:
      node_group: gpu
      placement: 0
  node_groups:
    - label: gpu
      node_ranks: 0
    - label: franka
      node_ranks: 0-1
      hardware:
        type: Franka
        configs:
          - robot_ip: <ROBOT_IP>
            node_rank: 0
            camera_serials:
              - "<ZED_SERIAL_1>"
              - "<ZED_SERIAL_2>"
              - "<ZED_SERIAL_3>"
            camera_type: zed            # "realsense" or "zed"
            gripper_type: robotiq       # "franka" or "robotiq"
            gripper_connection: "/dev/ttyUSB0"
            controller_node_rank: 1     # FrankaController runs on the NUC
            disable_validate: false
```

--------------------------------

### Install Robotiq Dependencies

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/franka_zed_robotiq.html

Command to install required dependencies for the Robotiq gripper.

```bash
bash requirements/install.sh embodied --env franka
```

--------------------------------

### Install Debugging Dependencies

Source: https://rlinf.readthedocs.io/en/latest/rst_source/resources/faq.html

Install the debugpy package required for Ray distributed debugging.

```bash
uv pip install debugpy
```

--------------------------------

### Install Training Node Dependencies

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/dual_franka.html

Commands to set up the environment on a remote GPU training cluster for SFT.

```bash
git clone https://github.com/RLinf/RLinf.git
cd RLinf
bash requirements/install.sh embodied --model openpi --env maniskill_libero --use-mirror
source .venv/bin/activate
```

--------------------------------

### Install Embodied Dependencies

Source: https://rlinf.readthedocs.io/en/latest/rst_source/extending/reward_model.html

Command to install required dependencies for VLM reward support.

```bash
bash requirements/install.sh embodied --env maniskill_libero --model qwen3_vl \
  --torch 2.8.0 --sglang 0.5.4 --transformers 4.57.1
```

--------------------------------

### Start PICO Service

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/dual_franka_pico_dagger.html

Commands to initialize the XRoboToolkit PC service on the publisher machine.

```bash
cd /opt/apps/roboticsservice
bash runService.sh
```

--------------------------------

### Install Extra Python Packages

Source: https://rlinf.readthedocs.io/en/latest/rst_source/extending/mbridge.html

Installs specific dependencies needed for the MBridge path.

```bash
uv pip install transformers==4.57.1 bitsandbytes
```

--------------------------------

### Configure environment

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/opensora.html

Setup using either a Docker container or a custom local environment.

```bash
docker run -it --rm --gpus all \
   --shm-size 20g \
   --network host \
   --name rlinf \
   -v .:/workspace/RLinf \
   rlinf/rlinf:agentic-rlinf0.3-opensora
   # Mainland China mirror: docker.1ms.run/rlinf/rlinf:agentic-rlinf0.3-opensora

# Inside the container, switch to the OpenVLA-OFT virtual environment:
source switch_env openvla-oft
```

```bash
# Add --use-mirror for faster downloads in mainland China.
bash requirements/install.sh embodied --model openvla-oft --env opensora
source .venv/bin/activate
```

--------------------------------

### Install Custom Environment

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/polaris.html

Install PolaRiS dependencies manually and activate the virtual environment.

```bash
# Mainland China users can add --use-mirror.
bash requirements/install.sh embodied --model openpi --env polaris
source .venv/bin/activate
```

--------------------------------

### Acquire Target Pose

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/gim_arm.html

Run the hardware test script to initialize the controller and print the current TCP position and quaternion for target pose calibration.

```bash
python toolkits/realworld_check/test_gim_arm_env.py --can can0 --variant gim_arm_xl
```

--------------------------------

### Install LIBERO-Plus Runtime

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/abot_m0.html

Install the additional LIBERO-Plus runtime into an existing ABot-M0 environment.

```bash
# For mainland China users, you can add the `--use-mirror` flag to the install.sh command for better download speed.
bash requirements/install.sh embodied --model abot_m0 --env liberoplus
source .venv/bin/activate
```

--------------------------------

### Launch Qdrant Local Server

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/agentic/searchr1.html

Commands to start the Qdrant server and the associated wiki retrieval service.

```bash
# Launch qdrant server
/path/to/qdrant/qdrant &

# Launch qdrant-based wiki server
bash examples/agent/tools/search_local_server_qdrant/launch_local_server_qdrant.sh
```

--------------------------------

### Install Franka DexHand Environment

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/franka_dexhand.html

Installs base Franka dependencies and dexterous-hand drivers.

```bash
bash requirements/install.sh embodied --env franka-dexhand
```

--------------------------------

### Launch Training and Monitor Logs

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/opd.html

Commands to initiate the training process and view metrics via TensorBoard.

```bash
source switch_env openvla-oft
bash examples/embodiment/run_embodiment.sh libero_spatial_opd_openvlaoft
```

```bash
tensorboard --logdir ./logs --port 6006
```

--------------------------------

### Install Custom Environment

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/embodichain.html

Manual installation of dependencies for users not using the Docker image.

```bash
# Mainland China users can add --use-mirror.
bash requirements/install.sh embodied --env embodichain
source .venv/bin/activate
```

--------------------------------

### Source Environment Setup Scripts

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/franka.html

Initialize the virtual environment and ROS workspace before data collection.

```bash
source <path_to_your_venv>/bin/activate
source <your_catkin_ws>/devel/setup.bash
```

--------------------------------

### Launch PPO Training with Bash

Source: https://rlinf.readthedocs.io/en/latest/rst_source/resources/blog/compare_with_verl.html

Execute the training process using the VeRL main trainer module with specific algorithm and hardware configurations.

```bash
set -x
export CUDA_DEVICE_MAX_CONNECTIONS=1

math_train_path=/path/to/dataset/boba.parquet
math_test_path=/path/to/dataset/test_mini.parquet

python3 -m verl.trainer.main_ppo \
    algorithm.adv_estimator=grpo \
    data.train_files="$math_train_path" \
    data.val_files="$math_test_path" \
    data.train_batch_size=128 \
    data.max_prompt_length=1024 \
    data.max_response_length=27648 \
    data.filter_overlong_prompts=True \
    data.truncation='error' \
    actor_rollout_ref.model.path=/path/to/models/DeepSeek-R1-Distill-Qwen-7B \
    actor_rollout_ref.actor.optim.lr=1e-6 \
    actor_rollout_ref.actor.ppo_mini_batch_size=32 \
    actor_rollout_ref.ref.fsdp_config.param_offload=True \
    actor_rollout_ref.actor.fsdp_config.param_offload=True \
    actor_rollout_ref.actor.fsdp_config.optimizer_offload=True \
    actor_rollout_ref.model.use_remove_padding=True \
    actor_rollout_ref.actor.ulysses_sequence_parallel_size=4 \
    actor_rollout_ref.actor.ppo_micro_batch_size_per_gpu=1 \
    actor_rollout_ref.ref.log_prob_micro_batch_size_per_gpu=1 \
    actor_rollout_ref.rollout.log_prob_micro_batch_size_per_gpu=1 \
    actor_rollout_ref.ref.log_prob_use_dynamic_bsz=True \
    actor_rollout_ref.ref.log_prob_max_token_len_per_gpu=30000 \
    actor_rollout_ref.actor.use_dynamic_bsz=True \
    actor_rollout_ref.actor.ppo_max_token_len_per_gpu=30000 \
    actor_rollout_ref.rollout.log_prob_use_dynamic_bsz=True \
    actor_rollout_ref.rollout.log_prob_max_token_len_per_gpu=30000 \
    actor_rollout_ref.actor.use_kl_loss=True \
    actor_rollout_ref.actor.kl_loss_coef=0.001 \
    actor_rollout_ref.actor.kl_loss_type=low_var_kl \
    actor_rollout_ref.actor.entropy_coeff=0 \
    actor_rollout_ref.model.enable_gradient_checkpointing=True \
    actor_rollout_ref.rollout.tensor_model_parallel_size=2 \
    actor_rollout_ref.rollout.name=sglang \
    actor_rollout_ref.rollout.gpu_memory_utilization=0.4 \
    actor_rollout_ref.rollout.n=16 \
    actor_rollout_ref.rollout.temperature=0.6 \
    actor_rollout_ref.rollout.top_k=1000000 \
    actor_rollout_ref.rollout.top_p=1.0 \
    algorithm.use_kl_in_reward=False \
    trainer.critic_warmup=0 \
    trainer.logger='["console","tensorboard"]' \
    trainer.project_name='verl_grpo_boba' \
    trainer.experiment_name='ds_7b_fsdp_sglang' \
    trainer.n_gpus_per_node=8 \
    trainer.nnodes=8 \
    trainer.val_before_train=False \
    trainer.save_freq=50 \
    trainer.test_freq=-1 \
    trainer.total_epochs=15000 $@
```

--------------------------------

### Install Custom Environment

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/calvin.html

Install CALVIN dependencies manually if not using the Docker image.

```bash
# Mainland China users can add --use-mirror.
bash requirements/install.sh embodied --model openpi --env calvin
source .venv/bin/activate
```

--------------------------------

### Install Reward Computation Dependencies

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/agentic/rstar2.html

Installs the necessary packages for reward calculation and verification.

```bash
pip install math-verify
```

```bash
pip install sympy
pip install pylatexenc
```

--------------------------------

### Build Qdrant Collection

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/agentic/searchr1.html

Commands to initialize the Qdrant server and build the collection for wiki retrieval.

```bash
# Create folder for qdrant
mkdir -p /path/to/qdrant
# Copy the binary
cp qdrant /path/to/qdrant

# Launch qdrant server
/path/to/qdrant/qdrant &

# Build qdrant collection
bash examples/agent/tools/search_local_server_qdrant/build_index_qdrant.sh
```

--------------------------------

### Launch Training Pipelines

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/dosw1.html

Start the training process using either synchronous or asynchronous pipelines.

```bash
# Synchronous pipeline
bash examples/embodiment/run_realworld.sh dosw1_pick_sac_flow

# Async pipeline (decoupled rollout / learner weight sync)
bash examples/embodiment/run_realworld_async.sh dosw1_pick_sac_flow_async
```

--------------------------------

### Download Reasoning Example Data

Source: https://rlinf.readthedocs.io/en/latest/rst_source/extending/mbridge.html

Downloads the model and dataset required for the reasoning example.

```bash
# For faster downloads in mainland China, you can set:
# export HF_ENDPOINT=https://hf-mirror.com
hf download deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B \
    --local-dir /path/to/model/DeepSeek-R1-Distill-Qwen-1.5B

mkdir -p /dataset/boba
hf download inclusionAI/AReaL-boba-Data AReaL-boba-106k.jsonl \
    --repo-type dataset \
    --local-dir /dataset/boba
```

--------------------------------

### Install SFT-only environment

Source: https://rlinf.readthedocs.io/en/latest/rst_source/examples/embodied/sft_dreamzero.html

Sets up the environment for DreamZero without simulator dependencies.

```bash
# Add --use-mirror for faster downloads in mainland China.
bash requirements/install.sh embodied --model dreamzero
source .venv/bin/activate
```

--------------------------------

### Verify EasyTier installation

Source: https://rlinf.readthedocs.io/en/latest/rst_source/guides/cloud_edge.html

Check that the EasyTier binary is correctly installed and accessible in the system path.

```bash
easytier-core --version
```
Tokens:
Raw