### Install Prerequisites with uv

Source: https://robotwin-platform.github.io/doc/usage/Pi0.html

Navigates to the policy/pi0 directory and installs project prerequisites using uv. This command assumes uv is already installed and activated.

```bash
cd policy/pi0
# Install prequisites in uv environment
GIT_LFS_SKIP_SMUDGE=1 uv sync
```

--------------------------------

### Run Installation Script

Source: https://robotwin-platform.github.io/doc/usage/robotwin-install.html

Execute the provided script to install basic dependencies and CuRobo.

```bash
bash script/_install.sh
```

--------------------------------

### Deployment Example

Source: https://robotwin-platform.github.io/doc/usage/control-robot.html

Provides a reference to a deployment example script for using the take_action function.

```APIDOC
## Deployment Example

You can find a demonstration of usage in the following file:

```
policy/Your_Policy/deploy_policy.py
```

This file provides a sample implementation to help you understand how to use the `take_action` function with different `action_type` settings during deployment.
```

--------------------------------

### Build and install FFmpeg from source

Source: https://robotwin-platform.github.io/doc/usage/Pi05.html

Comprehensive steps to clone, configure, build, and install FFmpeg version n7.1, including updating system paths and library links.

```bash
cd ~
git clone https://git.ffmpeg.org/ffmpeg.git ffmpeg
cd ffmpeg
git checkout n7.1
git pull origin n7.1

./configure --prefix="$HOME/ffmpeg-7.1-build" \
  --enable-gpl --enable-nonfree --enable-libx264 --enable-libx265 \
  --enable-libfdk-aac --enable-libmp3lame --enable-libopus \
  --enable-libvpx --enable-libass --enable-libfreetype \
  --enable-shared
make -j$(nproc)
make install

echo 'export PATH="$HOME/ffmpeg-7.1-build/bin:$PATH"' >> ~/.bashrc
source ~/.bashrc
# checkout if link success, should be n7.1
ffmpeg -version

sudo ln -s /home/xspark-ai/ffmpeg-5.1-build/include/* /usr/local/include/
sudo ln -s /home/xspark-ai/ffmpeg-5.1-build/lib/* /usr/local/lib/
sudo ldconfig
```

--------------------------------

### Environment Setup Commands

Source: https://robotwin-platform.github.io/doc/usage/OpenVLA-oft.html

Commands to configure the conda environment, install dependencies, and set up the Openvla-oft repository.

```bash
conda activate RoboTwin
# Install PyTorch
# Use a command specific to your machine: https://pytorch.org/get-started/locally/
pip3 install torch torchvision torchaudio

# Clone openvla-oft repo and pip install to download dependencies
git clone https://github.com/moojink/openvla-oft.git
cd openvla-oft
pip install -e .

# Install Flash Attention 2 for training (https://github.com/Dao-AILab/flash-attention)
#   =>> If you run into difficulty, try `pip cache remove flash_attn` first
pip install packaging ninja
ninja --version; echo $?  # Verify Ninja --> should return exit code "0"
pip install "flash-attn==2.5.5" --no-build-isolation
```

--------------------------------

### Install curobo in the environment

Source: https://robotwin-platform.github.io/doc/usage/Pi05.html

Commands to switch to the appropriate environment and install curobo from source.

```bash
conda deactivate
source .venv/bin/activate
# At this point, you should be in the (openpi) environment
cd ../../envs
git clone https://github.com/NVlabs/curobo.git
cd curobo
pip install -e . --no-build-isolation
cd ../../policy/pi05/
bash
```

--------------------------------

### Training data structure examples

Source: https://robotwin-platform.github.io/doc/usage/Pi05.html

Examples showing how to organize multi-task and single-task datasets within the training_data directory.

```text
#multi-task dataset example
training_data/  
├── ${model_name}
|       ├──${task_0}
|       |   ├──episode_0
|       |   |   ├── instructions.json  
|       |   |   ├── episode_0.hdf5  
|       |   ├── episode_1 
|       |   |   ├── instructions.json  
|       |   |   ├── episode_1.hdf5  
|       |   ├── ...
|       ├── ${task_1}
|       |   ├──episode_0
|       |   |   ├── instructions.json  
|       |   |   ├── episode_0.hdf5  
|       |   ├── episode_1 
|       |   |   ├── instructions.json  
|       |   |   ├── episode_1.hdf5  
|       |   ├── ...

#sigle task example
training_data/  
├── demo_clean
|       ├──beat_block_hammer-demo_clean-50
|       |   ├──episode_0
|       |   |   ├── instructions.json  
|       |   |   ├── episode_0.hdf5  
|       |   ├── episode_1 
|       |   |   ├── instructions.json  
|       |   |   ├── episode_1.hdf5  
|       |   ├── ...
```

--------------------------------

### Manual Dependency Installation

Source: https://robotwin-platform.github.io/doc/usage/robotwin-install.html

Commands for manually installing requirements, PyTorch3D, and CuRobo.

```bash
pip install -r requirements.txt
```

```bash
pip install "git+https://github.com/facebookresearch/pytorch3d.git@stable"
```

```bash
cd envs
git clone https://github.com/NVlabs/curobo.git
cd curobo
pip install -e . --no-build-isolation
cd ../..
```

--------------------------------

### Install RoboTwin Dependencies

Source: https://robotwin-platform.github.io/doc/usage/GO1.html

Install the required dependencies for the RoboTwin environment.

```bash
cd policy/GO1

conda activate RoboTwin
pip install -r requirements.txt
```

--------------------------------

### Example Calibration Command Usage

Source: https://robotwin-platform.github.io/doc/usage/object-annotation.html

Demonstrates the usage of the 'create' command to add a functional point.

```bash
create f
```

--------------------------------

### Deployment Example for take_action

Source: https://robotwin-platform.github.io/doc/usage/control-robot.html

This file provides a sample implementation for using the `take_action` function with different `action_type` settings during policy deployment.

```python
policy/Your_Policy/deploy_policy.py
```

--------------------------------

### Install Training Environment

Source: https://robotwin-platform.github.io/doc/usage/DexVLA.html

Sets up the isolated training environment for DexVLA and TinyVLA. Activates the conda environment and installs local policy heads.

```bash
cd policy/DexVLA
conda env create -f Train_Tiny_DexVLA_train.yml
conda activate dexvla-robo
cd policy_heads
pip install -e .
```

--------------------------------

### Example Calibration Command Usage

Source: https://robotwin-platform.github.io/doc/usage/object-annotation.html

Demonstrates the usage of the 'create' command to add a contact point and subsequent actions for positioning and orientation.

```bash
create c
```

--------------------------------

### Start GO-1 Inference Server

Source: https://robotwin-platform.github.io/doc/usage/GO1.html

Launch the inference server using a fine-tuned model checkpoint.

```bash
cd /path/to/AgiBot-World

conda activate go1

python evaluate/deploy.py --model_path /path/to/your/checkpoint --data_stats_path /path/to/your/dataset_stats.json --port <SERVER_PORT>
```

--------------------------------

### Start Training

Source: https://robotwin-platform.github.io/doc/usage/DexVLA.html

Launches the DexVLA Stage 2 training process. Ensure 'OUTPUT' includes 'qwen2' (and 'lora' if applicable), 'TASKNAME' matches constants, and 'mnop' and 'DIT_PRETRAIN' point to correct weights.

```bash
bash ./scripts/aloha/vla_stage2_train.sh
```

--------------------------------

### Install ACT Dependencies

Source: https://robotwin-platform.github.io/doc/usage/ACT.html

Installs necessary Python packages and the ACT library. Navigate to the ACT directory before running.

```bash
cd policy/ACT

pip install pyquaternion pyyaml rospkg pexpect mujoco==2.3.7 dm_control==1.0.14 opencv-python matplotlib einops packaging h5py ipython

cd detr && pip install -e . && cd ..
```

--------------------------------

### Install DP Dependencies

Source: https://robotwin-platform.github.io/doc/usage/DP.html

Installs necessary Python packages and the DP library. Ensure you are in the correct directory.

```bash
cd policy/DP
pip install zarr==2.12.0 wandb ipdb gpustat dm_control omegaconf hydra-core==1.2.0 dill==0.3.5.1 einops==0.4.1 diffusers==0.11.1 numba==0.56.4 moviepy imageio av matplotlib termcolor sympy
pip install -e .
```

--------------------------------

### Install Vulkan Drivers

Source: https://robotwin-platform.github.io/doc/usage/robotwin-install.html

Install necessary Vulkan libraries on Linux systems.

```bash
sudo apt install libvulkan1 mesa-vulkan-drivers vulkan-tools
```

--------------------------------

### Franka Robot Configuration Example

Source: https://robotwin-platform.github.io/doc/usage/new-embodiment.html

Example `config.yml` for a Franka robot, detailing URDF/SRDF paths, joint stiffness/damping, move groups, gripper configuration, homestate, and camera settings. Adjust parameters like `gripper_scale` for desired gripper behavior.

```yaml
urdf_path: "./panda.urdf"
srdf_path: "./panda.srdf"
joint_stiffness: 1000
joint_damping: 200
gripper_stiffness: 1000
gripper_damping: 200
move_group: ["panda_hand","panda_hand"]
ee_joints: ["panda_hand_joint","panda_hand_joint"]
arm_joints_name: [['panda_joint1', 'panda_joint2', 'panda_joint3', 'panda_joint4', 'panda_joint5', 'panda_joint6', 'panda_joint7'],
                  ['panda_joint1', 'panda_joint2', 'panda_joint3', 'panda_joint4', 'panda_joint5', 'panda_joint6', 'panda_joint7']]
gripper_name:
  - base: "panda_finger_joint1"
    mimic: [["panda_finger_joint2", 1., 0.]]
  - base: "panda_finger_joint1"
    mimic: [["panda_finger_joint2", 1., 0.]]
gripper_bias: 0.08
gripper_scale: [0.0, 0.04]
homestate: [[0, 0.19634954084936207, 0.0, -2.617993877991494, 0.0, 2.941592653589793, 0.7853981633974483],
            [0, 0.19634954084936207, 0.0, -2.617993877991494, 0.0, 2.941592653589793, 0.7853981633974483]]
delta_matrix: [[0,0,1],[0,-1,0],[1,0,0]]
global_trans_matrix: [[1,0,0],[0,-1,0],[0,0,-1]]
robot_pose: [[0, -0.65, 0.75, 0.707, 0, 0, 0.707],
             [0, -0.65, 0.75, 0.707, 0, 0, 0.707]]
planner: "curobo"
dual_arm: False
rotate_lim: [0.1, 0.8]
grasp_perfect_direction: ['right', 'left']
static_camera_list: 
- name: head_camera
  position: [0.0, 0.8, 0.9]
  forward: [0, -1, 0]
  left: [1, 0, 0]
```

--------------------------------

### Install DP3 Dependencies

Source: https://robotwin-platform.github.io/doc/usage/DP3.html

Installs the DP3 package and its required dependencies. Ensure you are in the correct directory before running.

```bash
cd policy/DP3/3D-Diffusion-Policy && pip install -e . && cd ..
pip install zarr==2.12.0 wandb ipdb gpustat dm_control omegaconf hydra-core==1.2.0 dill==0.3.5.1 einops==0.4.1 diffusers==0.11.1 numba==0.56.4 moviepy imageio av matplotlib termcolor

```

--------------------------------

### IK Solver Output Example

Source: https://robotwin-platform.github.io/doc/usage/new-embodiment.html

Expected console output format when running the IK solver script.

```text
x, y, z, success
0.35, 0.23, 0.09, tensor([[True]], device='cuda:0')
0.35, 0.23, 0.08, tensor([[True]], device='cuda:0')
0.35, 0.23, 0.07, tensor([[True]], device='cuda:0')
...
```

--------------------------------

### Start Tiny-VLA Policy Training

Source: https://robotwin-platform.github.io/doc/usage/TinyVLA.html

Launches the policy training process using a bash script. Ensure the script is executed from the correct directory.

```bash
bash ./scripts/franks/train_robotwin_aloha.sh
```

--------------------------------

### Install uv with Conda

Source: https://robotwin-platform.github.io/doc/usage/Pi0.html

Installs the uv package manager within an activated Conda environment. Ensure you have Conda installed and a RoboTwin environment set up.

```bash
conda activate RoboTwin
# Install uv
pip install uv
```

--------------------------------

### View Training Data Structure

Source: https://robotwin-platform.github.io/doc/usage/RDT.html

Example directory layout for organizing training tasks and episodes within the training_data folder.

```text
training_data/${model_name}
├── ${task_1}
│   ├── episode_0
|   |   |── episode_0.hdf5
|   |   |-- instructions
│   │   ├── lang_embed_0.pt
│   │   ├── ...
├── ${task_2}
│   ├── ...
├── ...
```

--------------------------------

### Install Training Environment for Tiny-VLA

Source: https://robotwin-platform.github.io/doc/usage/TinyVLA.html

Sets up the isolated training environment for Tiny-VLA and DexVLA using conda. Ensure you are in the 'policy/TinyVLA' directory.

```bash
cd policy/TinyVLA
conda env create -f Train_Tiny_DexVLA_train.yml
conda activate dexvla-robo
cd policy_heads
pip install -e .
```

--------------------------------

### Install Evaluation Environment

Source: https://robotwin-platform.github.io/doc/usage/DexVLA.html

Installs evaluation dependencies into an existing RoboTwin 2.0 conda environment.

```bash
conda activate your_RoboTwin_env
pip install -r Eval_Tiny_DexVLA_requirements.txt
```

--------------------------------

### Example folder structure

Source: https://robotwin-platform.github.io/doc/usage/Pi05.html

Visual representation of the expected directory structure for processed data.

```text
processed_data/ 
├──${task_name}-${task_config}-${expert_data_num}
|       |   ├──episode_0
|       |   |   ├── instructions.json  
|       |   |   ├── episode_0.hdf5  
|       |   ├── episode_1 
|       |   |   ├── instructions.json  
|       |   |   ├── episode_1.hdf5  
|       |   ├── ...
```

--------------------------------

### Install RDT Environment Dependencies

Source: https://robotwin-platform.github.io/doc/usage/RDT.html

Commands to configure the conda environment and install necessary PyTorch, Ninja, and Flash Attention dependencies.

```bash
# Make sure python version == 3.10
conda activate RoboTwin

# Install pytorch
# Look up https://pytorch.org/get-started/previous-versions/ with your cuda version for a correct command
pip install torch==2.1.0 torchvision==0.16.0  --index-url https://download.pytorch.org/whl/cu121

# Install packaging
pip install packaging==24.0
pip install ninja
# Verify Ninja --> should return exit code "0"
ninja --version; echo $?
# Install flash-attn
pip install flash-attn==2.7.2.post1 --no-build-isolation

# Install other prequisites
pip install -r requirements.txt
# If you are using a PyPI mirror, you may encounter issues when downloading tfds-nightly and tensorflow. 
# Please use the official source to download these packages.
# pip install tfds-nightly==4.9.4.dev202402070044 -i  https://pypi.org/simple
# pip install tensorflow==2.15.0.post1 -i  https://pypi.org/simple
```

--------------------------------

### Model Evaluation

Source: https://robotwin-platform.github.io/doc/usage/OpenVLA-oft.html

Command structure and example for evaluating the model on RoboTwin tasks.

```bash
bash eval.sh ${task_name} ${task_config} ${checkpoint_path} ${seed} ${gpu_id} ${unnorm_key}
# Example: bash eval.sh move_can_pot demo_randomized ckpt_path 0 5 aloha_move_can_pot_builder
```

--------------------------------

### Check FFmpeg version

Source: https://robotwin-platform.github.io/doc/usage/Pi05.html

Command to verify the currently installed version of FFmpeg.

```bash
ffmpeg -version
```

--------------------------------

### Dual-Arm URDF Directory Structure

Source: https://robotwin-platform.github.io/doc/usage/new-embodiment.html

Example file structure for a dual-arm robot embodiment configuration.

```yaml
# Using ALOHA as an example
- embodiments
  - aloha
    - config.yml # RoboTwin config file
    - curobo_left_tmp.yml # Left arm CuRobo config template
    - curobo_right_tmp.yml # Right arm CuRobo config template
    - collision_aloha_left.yml # Left arm collision annotations
    - collision_aloha_right.yml # Right arm collision annotations
    - urdf_files/... # URDF files and corresponding GLB, STL files, etc.
```

--------------------------------

### Minimal Task Configuration

Source: https://robotwin-platform.github.io/doc/usage/configurations.html

A basic YAML configuration file for starting a typical data collection session. Ensure all paths and parameters are set according to your project needs.

```yaml
render_freq: 0
episode_num: 50
use_seed: false
save_freq: 15
embodiment:
- aloha-agilex
language_num: 100
domain_randomization:
  random_background: true
  cluttered_table: true
  clean_background_rate: 0.02
  random_head_camera_dis: 0
  random_table_height: 0.03
  random_light: true
  crazy_random_light_rate: 0.02
camera:
  head_camera_type: D435
  wrist_camera_type: D435
  collect_head_camera: true
  collect_wrist_camera: true
data_type:
  rgb: true
  third_view: false
  depth: false
  pointcloud: false
  observer: false
  endpose: true
  qpos: true
  mesh_segmentation: false
  actor_segmentation: false
pcd_down_sample_num: 1024
pcd_crop: true
save_path: ./data
clear_cache_freq: 5
collect_data: true
eval_video_log: true

```

--------------------------------

### Set up Curobo Environment

Source: https://robotwin-platform.github.io/doc/usage/Pi0.html

Deactivates the current Conda environment, activates a virtual environment, clones the curobo repository, and installs it. This is required for evaluating the pi0 policy in RoboTwin.

```bash
conda deactivate
source .venv/bin/activate
# At this point, you should be in the (openpi) environment
cd ../../envs
git clone https://github.com/NVlabs/curobo.git
cd curobo
pip install -e . --no-build-isolation
cd ../../policy/pi0/
bash
```

--------------------------------

### Configure Docker NVIDIA Driver Capabilities

Source: https://robotwin-platform.github.io/doc/usage/robotwin-install.html

Set the required environment variable when starting a Docker container to ensure graphics support.

```bash
-e NVIDIA_DRIVER_CAPABILITIES=compute,utility,graphics
```

--------------------------------

### Initialize Embodiment Directory

Source: https://robotwin-platform.github.io/doc/usage/new-embodiment.html

Commands to create the necessary directory and configuration files for a new robot.

```bash
cd ${ROBOTWIN_ROOT_PATH}
mkdir -p assets/embodiments/new_robot
cd assets/embodiments/new_robot
touch curobo_tmp.yml
touch collision.yml
```

--------------------------------

### Sync environment prerequisites

Source: https://robotwin-platform.github.io/doc/usage/Pi05.html

Commands to navigate to the policy directory and synchronize the environment using uv.

```bash
cd policy/pi05
# Install prequisites in uv environment
GIT_LFS_SKIP_SMUDGE=1 uv sync
```

--------------------------------

### Pre-Training and Fine-tuning

Source: https://robotwin-platform.github.io/doc/usage/LLaVA-VLA.html

Executes the training process. Ensure paths are updated before running.

```bash
bash calvin_finetune_obs.sh
```

--------------------------------

### Create Directories for Data

Source: https://robotwin-platform.github.io/doc/usage/LLaVA-VLA.html

Initializes the required directory structure for storing images and training data.

```bash
mkdir pictures && training_data
cd scripts && cd helper
```

--------------------------------

### Model Finetuning

Source: https://robotwin-platform.github.io/doc/usage/OpenVLA-oft.html

Command to initiate the finetuning process for the model.

```bash
bash finetune_aloha.sh
```

--------------------------------

### Generate Training Configuration

Source: https://robotwin-platform.github.io/doc/usage/RDT.html

Create a model configuration file and directory structure for a specific training run.

```bash
cd policy/RDT
bash generate.sh ${model_name}
# bash generate.sh RDT_demo_clean
```

--------------------------------

### Get Arm Pose

Source: https://robotwin-platform.github.io/doc/usage/API.html

Retrieves the current pose (position and orientation) of the end-effector for a specified arm.

```APIDOC
## GET /arm/pose

### Description
Gets the current pose of the end-effector of the specified arm.

### Method
GET

### Endpoint
/arm/pose

### Parameters
#### Query Parameters
- **arm_tag** (ArmTag) - Required - Which arm to query.

### Request Example
```python
pose = self.get_arm_pose(ArmTag("left"))
```

### Response
#### Success Response (200)
- **pose** (list[float]) - A list of 7 floats representing the pose: `[x, y, z, qw, qx, qy, qz]`.

#### Response Example
```json
{
  "pose": [0.5, 0.1, 0.2, 1.0, 0.0, 0.0, 0.0]
}
```
```

--------------------------------

### Grasp an actor with specific parameters

Source: https://robotwin-platform.github.io/doc/usage/API.html

Shows how to initiate a grasp action for a specific actor, selecting a contact point based on the arm being used.

```python
self.move(
    self.grasp_actor(
        self.cup, arm_tag=arm_tag,
        pre_grasp_dis=0.1,
        contact_point_id=[0, 2][int(arm_tag=='left')]
    )
)
```

--------------------------------

### Get Current Joint State

Source: https://robotwin-platform.github.io/doc/usage/object-annotation.html

Retrieve the current joint state to use as the initial pose when loading an asset into a task.

```bash
qpos
    Usage:
        qpos
    Get the current joint state as the initial pose when loading the asset into the task.
```

--------------------------------

### Initialize Conda Environment

Source: https://robotwin-platform.github.io/doc/usage/robotwin-install.html

Create and activate a Python 3.10 environment for RoboTwin.

```bash
conda create -n RoboTwin python=3.10 -y
conda activate RoboTwin
```

--------------------------------

### Run Calibration

Source: https://robotwin-platform.github.io/doc/usage/object-annotation.html

Obtain stable points for calibration. Press Ctrl+C to stop and save. Manual stopping is required as there's no upper step limit.

```bash
run
    Usage:
        run
        Press <Ctrl + C> to stop and save information
    Used to obtain stable points through steps, generally selected at the beginning of calibration to determine if running is necessary.
    Since this command does not limit the step upper limit, you need to manually stop running (press Ctrl+C) based on whether the asset in the UI interface is stable.
```

--------------------------------

### Prepare Training Data

Source: https://robotwin-platform.github.io/doc/usage/DP.html

Preprocesses RoboTwin 2.0 data into Zarr format for DP training. Specify the task name, configuration, and number of expert trajectories.

```bash
bash process_data.sh ${task_name} ${task_config} ${expert_data_num}
# bash process_data.sh beat_block_hammer demo_clean 50
# or processing randomized data: bash process_data.sh beat_block_hammer demo_randomized 50
```

--------------------------------

### Camera Configuration Settings

Source: https://robotwin-platform.github.io/doc/usage/configurations.html

Specify camera types and which views to collect in this YAML configuration. Ensure the camera types listed are compatible with your setup.

```yaml
camera:
  head_camera_type: D435
  wrist_camera_type: D435
  collect_head_camera: true
  collect_wrist_camera: true

```

--------------------------------

### Get Current Arm Pose

Source: https://robotwin-platform.github.io/doc/usage/API.html

Retrieves the current pose (position and orientation) of the specified arm's end-effector. The pose is returned as a list of 7 floats.

```python
pose = self.get_arm_pose(ArmTag("left"))

```

--------------------------------

### Load Robot Configuration and Initialize Model

Source: https://robotwin-platform.github.io/doc/usage/new-embodiment.html

Loads robot configuration from a YAML file and initializes the CUDA robot model. Ensure the `curobo.yml` path is absolute.

```python
config_file = load_yaml("/abs_path/to/curobo.yml")

urdf_file = config_file["robot_cfg"]["kinematics"]["urdf_path"]
base_link = config_file["robot_cfg"]["kinematics"]["base_link"]
ee_link = config_file["robot_cfg"]["kinematics"]["ee_link"]
robot_cfg = RobotConfig.from_basic(urdf_file, base_link, ee_link, tensor_args)
kin_model = CudaRobotModel(robot_cfg.kinematics)
q = torch.rand((10, kin_model.get_dof()), **(tensor_args.as_torch_dict()))
out = kin_model.get_state(q)
```

--------------------------------

### Download RDT Model Weights

Source: https://robotwin-platform.github.io/doc/usage/RDT.html

Use huggingface-cli to download the required T5, SigLIP, and RDT model weights into the local directory.

```bash
# In the ROOT directory
cd policy 
mkdir weights
cd weights
mkdir RDT && cd RDT
# Download the models used by RDT
huggingface-cli download google/t5-v1_1-xxl --local-dir t5-v1_1-xxl
huggingface-cli download google/siglip-so400m-patch14-384 --local-dir siglip-so400m-patch14-384
huggingface-cli download robotics-diffusion-transformer/rdt-1b --local-dir rdt-1b
```

--------------------------------

### Create config.yml File

Source: https://robotwin-platform.github.io/doc/usage/new-embodiment.html

Command to create a new `config.yml` file in the specified directory. This file will contain your robot's specific parameters.

```bash
cd assets/embodiments/new_robot
touch config.yml
```

--------------------------------

### Fine-tune RoboTwin Policy

Source: https://robotwin-platform.github.io/doc/usage/Pi0.html

Use this command to fine-tune a RoboTwin policy. Adjust the `gpu_use` parameter based on your multi-GPU setup. Ensure `train_config_name`, `model_name`, and `gpu_use` are correctly specified.

```bash
bash finetune.sh ${train_config_name} ${model_name} ${gpu_use}
#bash finetune.sh pi0_base_aloha_robotwin_full demo_clean 0,1,2,3
```

--------------------------------

### Prepare Training Data for DP3

Source: https://robotwin-platform.github.io/doc/usage/DP3.html

Preprocesses RoboTwin 2.0 data into Zarr format for DP3 training. Set `data_type/pointcloud` to `true` during data collection if encountering `ZeroDivisionError`.

```bash
bash process_data.sh ${task_name} ${task_config} ${expert_data_num}
# bash process_data.sh beat_block_hammer demo_clean 50
# or processing randomized data: bash process_data.sh beat_block_hammer demo_randomized 50

```

--------------------------------

### Get Model Action from Observation

Source: https://robotwin-platform.github.io/doc/usage/deploy-your-policy.html

The `get_action` function computes the action to be executed based on the provided model and current observation. It's useful when action computation is separate from the evaluation loop.

```python
def get_action(model, obs: dict) -> Any:
    pass
```

--------------------------------

### Execute Data Collection Script

Source: https://robotwin-platform.github.io/doc/usage/collect-data.html

Run the collection script by specifying the task name, configuration, and GPU ID.

```bash
bash collect_data.sh ${task_name} ${task_config} ${gpu_id}
# Clean Data Example: bash collect_data.sh beat_block_hammer demo_clean 0
# Radomized Data Example: bash collect_data.sh beat_block_hammer demo_randomized 0
```

--------------------------------

### Generate Episode Instructions

Source: https://robotwin-platform.github.io/doc/usage/description.html

Use this script to generate a maximum number of episode descriptions for a specific task and setting. The script utilizes parameters for task name, setting, and the maximum number of descriptions.

```python
# Generate 60 task descriptions for a task
python3 utils/generate_episode_instructions.py place_shoe franka-panda-D435 1000
```

--------------------------------

### Update Camera Type in Task Config

Source: https://robotwin-platform.github.io/doc/usage/new-camera.html

Modify the camera type settings within your task configuration file. This example shows how to set the `head_camera_type` to the newly defined `Demo_Camera` and specifies the `wrist_camera_type`.

```yaml
camera:
  head_camera_type: Demo_Camera
  wrist_camera_type: D435

```

--------------------------------

### Prepare Temporary URDF for Calibration

Source: https://robotwin-platform.github.io/doc/usage/new-embodiment.html

Create a backup of the original URDF file before modifying it for calibration purposes.

```bash
cd assets/embodiments/franka-panda
cp panda.urdf panda.urdf.save
```

--------------------------------

### Basic Task Code Structure

Source: https://robotwin-platform.github.io/doc/usage/expert-code-gen.html

Defines the fundamental structure for a task file, including methods for setup, loading actors, playing the task, and checking success. This serves as a template for new tasks.

```python
from .base_task import Base_task
from .utils import *
import sapien

class ${task_name}(Base_task):
    def setup_demo(self, **kwargs):
        # Initializes the simulation environment for the task
        # Sets up the table, robot, planner, camera, and initial positions
        # This function is called once at the beginning of each episode
        pass

    def load_actors(self):
        # Loads all the necessary objects for the task into the environment
        # Typically called from setup_demo to initialize scene objects
        # Can also be used to set initial poses for objects
        pass

    def play_once(self):
        # Contains the robot control code to complete the task
        # This is the main function that will be generated by the LLM
        # Implements the sequence of actions for the robot to achieve the task
        pass

    # Check success
    def check_success(self):
        # Defines criteria to determine if the task was completed successfully
        # Returns a boolean indicating success or failure
        # Used for evaluation and feedback during code generation
        pass

```

--------------------------------

### Define New Camera Parameters

Source: https://robotwin-platform.github.io/doc/usage/new-camera.html

Add this block to `task_config/_camera_config.yml` to define a new camera's field of view (fovy), height (h), and width (w). Ensure these values are appropriate for your camera setup.

```yaml
Demo_Camera:
  fovy: 56
  w: 224
  h: 224

```

--------------------------------

### Evaluate RoboTwin Policy

Source: https://robotwin-platform.github.io/doc/usage/Pi0.html

Execute this command to evaluate a trained RoboTwin policy. The `ckpt_path` is derived from the training configuration. Modify `deploy_policy.yml` to change the checkpoint ID for evaluation. Examples show evaluating with the same or different task configurations.

```bash
# ckpt_path like: policy/pi0/checkpoints/pi0_base_aloha_robotwin_full/demo_clean/30000
bash eval.sh ${task_name} ${task_config} ${train_config_name} ${model_name} ${seed} ${gpu_id}
# bash eval.sh beat_block_hammer demo_clean pi0_base_aloha_robotwin_full demo_clean 0 0
# This command trains the policy using the `demo_clean` setting ($model_name) 
# and evaluates it using the same `demo_clean` setting ($task_config).

# To evaluate a policy trained on the `demo_clean` setting and tested on the `demo_randomized` setting, run:
# bash eval.sh beat_block_hammer demo_randomized pi0_base_aloha_robotwin_full demo_clean 0 0
```

--------------------------------

### Run Simulation and Verify Output

Source: https://robotwin-platform.github.io/doc/usage/new-embodiment.html

Commands to execute the simulation and the expected output format for the transformation matrix.

```bash
bash collect_data.sh beat_block_hammer demo_randomized 0
```

```text
[[ 1.  0.  0.]
 [ 0. -1.  0.]
 [ 0. -0. -1.]]
```

--------------------------------

### Create Data Directories

Source: https://robotwin-platform.github.io/doc/usage/Pi0.html

Creates the necessary 'processed_data' and 'training_data' directories within the 'policy/pi0' directory for data organization.

```bash
mkdir processed_data && mkdir training_data
```

--------------------------------

### Configure RoboTwin Client

Source: https://robotwin-platform.github.io/doc/usage/GO1.html

Configuration settings for the RoboTwin client in deploy_policy.yml.

```yaml
host: Server IP address (default: 127.0.0.1)
port: Server port (default: 9000)
```

--------------------------------

### Update and Verify Configuration

Source: https://robotwin-platform.github.io/doc/usage/new-embodiment.html

Commands to update the configuration paths and import necessary modules for verification.

```bash
cd ${ROBOTWIN_ROOT_PATH}
python script/update_embodiment_config_path.py
```

```python
import torch
from curobo.cuda_robot_model.cuda_robot_model import CudaRobotModel, CudaRobotModelConfig
from curobo.types.base import TensorDeviceType
from curobo.types.robot import RobotConfig
from curobo.util_file import get_robot_path, join_path, load_yaml

tensor_args = TensorDeviceType()
```

--------------------------------

### Download Assets

Source: https://robotwin-platform.github.io/doc/usage/robotwin-install.html

Download required datasets and libraries using the provided script.

```bash
bash script/_download_assets.sh
```

--------------------------------

### Train Diffusion Policy

Source: https://robotwin-platform.github.io/doc/usage/DP.html

Launches the policy training process, defaulting to 600 steps. Configure task name, config, expert data number, seed, action dimension, and GPU ID.

```bash
bash train.sh ${task_name} ${task_config} ${expert_data_num} ${seed} ${action_dim} ${gpu_id}
# bash train.sh beat_block_hammer demo_clean 50 0 14 0
# For `aloha-agilex` embodiment, the action_dim is 14
```

--------------------------------

### View Asset Directory Structure

Source: https://robotwin-platform.github.io/doc/usage/robotwin-install.html

Expected layout of the assets folder after download.

```text
assets
├── background_texture
├── embodiments
│   ├── embodiment_1
│   │   ├── config.yml
│   │   └── ...
│   └── ...
├── objects
└── ...
```

--------------------------------

### Prepare Training Data for ACT

Source: https://robotwin-platform.github.io/doc/usage/ACT.html

Preprocesses RoboTwin 2.0 data for ACT training. Specify the task name, configuration, and number of expert trajectories.

```bash
bash process_data.sh ${task_name} ${task_config} ${expert_data_num}
# bash process_data.sh beat_block_hammer demo_clean 50
```

--------------------------------

### Train DP3 Policy

Source: https://robotwin-platform.github.io/doc/usage/DP3.html

Launches the policy training process. The model is trained for 3,000 steps by default.

```bash
bash train.sh ${task_name} ${task_config} ${expert_data_num} ${seed} ${gpu_id}
# bash train.sh beat_block_hammer demo_clean 50 0 0

```

--------------------------------

### Test IK Solutions with New Quaternion

Source: https://robotwin-platform.github.io/doc/usage/new-embodiment.html

Use the output quaternion from the debug print to test valid positions. Remember to update the quaternion with your actual output values.

```python
import torch
from curobo.types.base import TensorDeviceType
from curobo.types.math import Pose
from curobo.types.robot import RobotConfig
from curobo.util_file import get_robot_configs_path, join_path, load_yaml
from curobo.wrap.reacher.ik_solver import IKSolver, IKSolverConfig

tensor_args = TensorDeviceType()
config_file = load_yaml(join_path(get_robot_configs_path(), "franka.yml"))
urdf_file = config_file["robot_cfg"]["kinematics"]["urdf_path"]
base_link = config_file["robot_cfg"]["kinematics"]["base_link"]
ee_link = config_file["robot_cfg"]["kinematics"]["ee_link"]
robot_cfg = RobotConfig.from_basic(urdf_file, base_link, ee_link, tensor_args)

ik_config = IKSolverConfig.load_from_robot_config(
    robot_cfg,
    None,
    num_seeds=20,
    self_collision_check=False,
    self_collision_opt=False,
    tensor_args=tensor_args,
    use_cuda_graph=True,
)
ik_solver = IKSolver(ik_config)
x_values = torch.linspace(0.35, 0.0, 25).tolist() + torch.linspace(0.35, 0.7, 25).tolist()
y_values = torch.linspace(0.25, 0.0, 25).tolist() + torch.linspace(0.25, 0.5, 25).tolist()
z_values = torch.linspace(0.25, 0.0, 25).tolist() + torch.linspace(0.25, 0.5, 25).tolist()

###### REMEMBER TO UPDATE THE QUATERNION ####
#############################################
# Update the quaternion from the debug output
quaternion = torch.tensor([[1.68244557e-03, -9.98540531e-01, -3.19133105e-04, -5.39803316e-02]], device='cuda:0')
#############################################

print("Testing IK solutions for different positions:")
print("x, y, z, success")
for x in x_values:
    for y in y_values:
        for z in z_values:
            goal = Pose(
                position=torch.tensor([[float(x), float(y), float(z)]], device='cuda:0'),
                quaternion=quaternion
            )
            result = ik_solver.solve_single(goal)
            if result.success.item() == True:
                print(f"{x:.2f}, {y:.2f}, {z:.2f}, {result.success}")
```

--------------------------------

### Prepare Training Data for Tiny-VLA

Source: https://robotwin-platform.github.io/doc/usage/TinyVLA.html

Preprocesses RoboTwin 2.0 data into the format required for Tiny-VLA training. Specify the task name, task configuration, and the number of expert trajectories.

```python
python process_data.py ${task_name} ${task_config} ${expert_data_num}
# python process_data.py beat_block_hammer demo_randomized 50
```

--------------------------------

### Prepare Training Data

Source: https://robotwin-platform.github.io/doc/usage/DexVLA.html

Preprocesses RoboTwin 2.0 data into the format required for DexVLA training. Specify the task name, configuration, and number of expert trajectories.

```python
python process_data.py ${task_name} ${task_config} ${expert_data_num}
# python process_data.py beat_block_hammer demo_clean 50
```

--------------------------------

### Execute Model Fine-Tuning

Source: https://robotwin-platform.github.io/doc/usage/Pi05.html

Commands to compute normalization statistics and initiate the fine-tuning process using shell scripts.

```bash
# compute norm_stat for dataset
uv run scripts/compute_norm_stats.py --config-name ${train_config_name}
# uv run scripts/compute_norm_stats.py --config-name pi05_aloha_full_base

# train_config_name: The name corresponding to the config in _CONFIGS, such as pi05_aloha_full_base
# model_name: You can choose any name for your model
# gpu_use: if not using multi gpu,set to gpu_id like 0;else set like 0,1,2,3
bash finetune.sh ${train_config_name} ${model_name} ${gpu_use}
#bash finetune.sh pi05_aloha_full_base demo_clean 0,1,2,3
```

--------------------------------

### Configure Training Script Parameters

Source: https://robotwin-platform.github.io/doc/usage/TinyVLA.html

Modifies the 'train_robotwin_aloha.sh' script to set the task name, root path, and the path to the base VLM model.

```bash
TASK=your_task # Set the Task
ROOT=.../robotiwin/policy/TinyVLA # Set Root Path
mnop=.../robotiwin/policy/TinyVLA/model_param/InternVL3-1B/ # Set The Path of base VLM
```

--------------------------------

### Calibration Commands

Source: https://robotwin-platform.github.io/doc/usage/object-annotation.html

Commands for manipulating calibration points within the calibration window. Includes resizing, creating points, cloning, rotating, aligning, removing, saving, and exiting.

```bash
resize:
    Usage:
        resize <x_size> <y_size> <z_size>: Set scaling along x, y, z axes
        resize <size>: Uniformly scale all three axes
    Example:
        resize 0.1
create:
    Usage:
        create <type>: Create (t)arget, (c)ontact, (f)unctional, or (o)rientation point
        create: Waits for input of point name
    Examples:
        create t
        create f
clone:
    Usage:
        clone <type> <id>: Clone a specified type and ID point in place
        clone: Waits for input of point type and ID
    Examples:
        clone t 1: Clones contact point target_1 to create a new target point (e.g., target_2)
rotate:
    Usage:
        rotate <id> <axis> <interval>: Rotate a specified contact point around its own axis by a given interval, generating points belonging to the same group
    Example:
        rotate 1 x 90: Rotates contact_1 around its x-axis every 90 degrees, creating three additional contact points, and writes the group into concat_points_group
align:
    Usage:
        align: Aligns all group points' positions to the first point in the group
remove:
    Usage:
        remove <type> <id>: Removes a point with the specified name
        remove: Waits for input of point name
    Examples:
        remove t 0
save:
    Saves current calibration data — always remember to save!
exit:
    Exits the calibration window

```