XRoboToolkit Teleop Sample Python

https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python
Admin
A Python framework for XR-based teleoperation of robots in MuJoCo simulation and real hardware,...
Tokens:9,531
Snippets:35
Trust Score:5.7
License:MIT
Update:1 year ago
Show doc for...
Tokens:
Raw
### Environment Setup and Installation

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/CLAUDE.md

Commands to set up a Conda environment for the project or perform a system-wide installation. It includes activating the environment and installing project dependencies.

```sh
# Setup conda environment (Ubuntu 22.04 tested)
./setup_conda.sh --conda xr-robotics
conda activate xr-robotics
./setup_conda.sh --install
```

```sh
# System-wide installation (if not using conda)
./setup.sh
```

--------------------------------

### Setup System Python (Bash)

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/README.md

Installs project dependencies using system Python.

```bash
bash setup.sh
```

--------------------------------

### Setup Conda Environment (Bash)

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/README.md

Sets up a Conda environment for the project and installs dependencies. It can optionally create a new environment or use the system Python.

```bash
bash setup_conda.sh --conda <optional_env_name>
conda activate <env_name>
bash setup_conda.sh --install
```

--------------------------------

### Run Galaxea R1 Lite Humanoid Demo (Python)

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/README.md

Starts the teleoperation demo for the Galaxea R1 Lite humanoid robot, using the GalaxeaR1LiteTeleopController which communicates via ROS.

```python
python scripts/hardware/teleop_r1lite_hardware.py
```

--------------------------------

### Run Dexterous Hand Simulation (Python)

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/README.md

Starts teleoperation simulation for dexterous hands. Supports Shadow hand in Mujoco and Inspire hand in Placo.

```python
python scripts/simulation/teleop_shadow_hand_mujoco.py
```

```python
scripts/simulation/teleop_inspire_hand_placo.py
```

--------------------------------

### Run Placo Visualization Demo (Python)

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/README.md

Launches the teleoperation demo for an X7S robot using Placo visualization, initializing the PlacoTeleopController.

```python
python scripts/simulation/teleop_x7s_placo.py
```

--------------------------------

### Run Mujoco Demo for Dual A1X Arm

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/teleop_details.md

Executes the Mujoco simulation demo for a dual A1X arm with gripper control.

```bash
python scripts/simulation/teleop_dual_a1x_mujoco.py
```

--------------------------------

### Run Mujoco Demo for Dual UR5e

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/teleop_details.md

Executes the Mujoco simulation demo for a dual UR5e robot.

```bash
python scripts/simulation/teleop_dual_ur5e_mujoco.py
```

--------------------------------

### Run Hardware Demo - Dual UR5 (Python)

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/README.md

Executes the teleoperation demo for physical dual UR5 arms and a Dynamixel-based head. Initializes DualArmURController and DynamixelHeadController.

```python
python scripts/hardware/teleop_dual_ur5e_hardware.py
```

--------------------------------

### Clone Repository (Bash)

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/README.md

Clones the XRoboToolkit-Teleop-Sample-Python repository from GitHub and navigates into the project directory.

```bash
git clone https://github.com/XR-Robotics/XRoboToolkit-Teleop-Sample-Python.git
cd XRoboToolkit-Teleop-Sample-Python
```

--------------------------------

### Run MuJoCo Simulation Demo (Python)

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/README.md

Launches the teleoperation demo for a UR5e robot in MuJoCo simulation using the MujocoTeleopController.

```python
python scripts/simulation/teleop_dual_ur5e_mujoco.py
```

--------------------------------

### Running Hardware Scripts

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/CLAUDE.md

Python commands to run teleoperation for physical robot hardware, including options for resetting robot states.

```python
python scripts/hardware/teleop_dual_ur5e_hardware.py --reset
```

```python
python scripts/hardware/teleop_arx_hardware.py
```

```python
python scripts/hardware/teleop_r1lite_hardware.py
```

--------------------------------

### Running Simulation Scripts

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/CLAUDE.md

Python commands to launch teleoperation simulations for various robot arms and configurations using MuJoCo and Placo physics engines.

```python
python scripts/simulation/teleop_dual_ur5e_mujoco.py
```

```python
python scripts/simulation/teleop_x7s_placo.py
```

```python
python scripts/simulation/teleop_shadow_hand_mujoco.py
```

```python
python scripts/simulation/teleop_flexiv_rizon4s_mujoco.py
```

--------------------------------

### Robot Configuration in Xrobotoolkit

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/CLAUDE.md

Illustrates how robot-specific settings, such as joint limits, IK constraints, and camera parameters, are defined using Python dictionaries within controller classes.

```python
class RobotConfigurableController:
    def __init__(self, config_path):
        self.config = self._load_config(config_path)

    def _load_config(self, config_path):
        # Load configuration from a file or define directly
        # Example configuration dictionary:
        robot_config = {
            "joint_limits": {
                "joint1": {"min": -3.14, "max": 3.14},
                "joint2": {"min": -1.57, "max": 1.57}
            },
            "ik_constraints": {
                "max_velocity": 0.5,
                "max_acceleration": 2.0
            },
            "camera_parameters": {
                "resolution": "1280x720",
                "fps": 30
            }
        }
        return robot_config

    def get_joint_limit(self, joint_name):
        return self.config.get("joint_limits", {}).get(joint_name)

    def get_ik_constraint(self, constraint_name):
        return self.config.get("ik_constraints", {}).get(constraint_name)

    def get_camera_parameter(self, param_name):
        return self.config.get("camera_parameters", {}).get(param_name)
```

--------------------------------

### Run ARX R5 Hardware Demo (Python)

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/README.md

Launches the teleoperation demo for dual ARX R5 robotic arms, utilizing the ARXR5TeleopController for arm and gripper control.

```python
python scripts/hardware/teleop_dual_arx_r5_hardware.py
```

--------------------------------

### Using Absolute Paths for URDF Loading

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/CLAUDE.md

Demonstrates the importance of using absolute paths for loading URDF files, leveraging the `path_utils` module for cross-platform compatibility in Xrobotoolkit.

```python
from xrobotoolkit.utils import path_utils
import os

def load_robot_urdf(urdf_filename):
    # Construct the absolute path to the URDF file
    # Assumes URDF files are located in a 'robots/urdf' directory relative to the package root
    urdf_path = path_utils.get_package_resource_path(f"robots/urdf/{urdf_filename}")
    
    if not os.path.exists(urdf_path):
        raise FileNotFoundError(f"URDF file not found at: {urdf_path}")
        
    # Logic to load the URDF using the absolute path
    print(f"Loading URDF from: {urdf_path}")
    # Example: return urdf_loader.load(urdf_path)
    return urdf_path

# Example usage:
# try:
#     robot_model_path = load_robot_urdf("my_robot.urdf")
# except FileNotFoundError as e:
#     print(e)
```

--------------------------------

### Visualize IK Results in Placo (Python)

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/README.md

Runs the hardware demo script with the --visualize_placo flag to visualize inverse kinematics solutions using Placo during teleoperation.

```python
python scripts/hardware/teleop_dual_ur5e_hardware.py --visualize_placo
```

--------------------------------

### Run UR5 Dual Teleoperation Script

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/teleop_details.md

Command to execute the Python script for hardware teleoperation of dual UR5e manipulators with grippers.

```bash
python scripts/hardware/teleop_dual_ur5e_hardware.py
```

--------------------------------

### Adding New Robots in Xrobotoolkit

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/CLAUDE.md

Instructions for integrating new robot hardware by extending base controller classes and implementing necessary interfaces. This ensures proper hardware abstraction and control.

```python
class NewRobotController(BaseController):
    def __init__(self, config):
        super().__init__(config)
        # Implement robot-specific initialization

    def _initialize_hardware(self):
        # Initialize specific hardware components
        pass

    # Implement required abstract methods from BaseController
    def move_to_pose(self, pose):
        # Logic to move the robot to a target pose
        pass

    def get_current_pose(self):
        # Logic to retrieve the current robot pose
        pass

# Example of creating an interface wrapper if needed
# from hardware.interface import RobotInterface
# class MyRobotInterface(RobotInterface):
#     pass
```

--------------------------------

### Run ARX R5 Dual Teleoperation Script

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/teleop_details.md

Command to execute the Python script for hardware teleoperation of dual ARX R5 manipulators with gripper support.

```bash
python scripts/hardware/teleop_dual_arx_r5_hardware.py
```

--------------------------------

### Reset UR Arms to Home Position (Python)

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/README.md

Resets the UR arms to their initial positions and initializes grippers by running the hardware demo script with the --reset flag.

```python
python scripts/hardware/teleop_dual_ur5e_hardware.py --reset
```

--------------------------------

### Teleoperation Task Configuration

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/teleop_details.md

Defines the configuration for a teleoperation task, specifying robot link names, pose sources, control triggers, and optional gripper and motion tracker configurations.

```python
config = {
    "right_hand": {
        "link_name": "flange",
        "pose_source": "right_controller",
        "control_trigger": "right_grip",
        "control_mode": "position", # optional: "pose" (default) or "position"
        "motion_tracker": {
            "serial": "PC2310BLH9020740B",
            "link_target": "link4",
        },
        "vis_target": "right_target", # optional, only used in mujoco
    },
    "left_hand": {
        "link_name": "left_tool0",
        "pose_source": "left_controller",
        "control_trigger": "left_grip",
        "gripper_trigger": "left_trigger",
        "vis_target": "left_target", # optional, only used in mujoco
    },
}
```

--------------------------------

### Gripper Equality Constraints in XML

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/teleop_details.md

Defines equality constraints for parallel gripper joints in the Mujoco XML file to ensure proper actuation.

```xml
<equality>
<joint name="right_gripper_constraint" joint1="right_gripper_finger_joint1" joint2="right_gripper_finger_joint2" polycoef="0 -1 0 0 0" />
<joint name="left_gripper_constraint" joint1="left_gripper_finger_joint1" joint2="left_gripper_finger_joint2" polycoef="0 -1 0 0 0" />
</equality>
```

--------------------------------

### Teleoperation Tracking Modes Configuration

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/README.md

API documentation for configuring teleoperation tracking modes. This includes controller tracking, hand tracking, head tracking, and motion tracker tracking, with details on configuration parameters and use cases.

```APIDOC
Tracking Modes:

1. Controller Tracking (Default):
   - Description: Uses VR/AR controller poses to control robot end effectors.
   - Configuration: Set `pose_source` to `"left_controller"` or `"right_controller"`.
   - Tracking: Full 6DOF pose (position + orientation) or 3DOF position-only.

2. Hand Tracking:
   - Description: Uses hand pose estimation from XR cameras for natural hand gesture control.

3. Head Tracking:
   - Description: Uses headset pose for controlling specific robot components, like camera orientation.

4. Motion Tracker Tracking:
   - Description: Uses additional motion tracking devices for controlling auxiliary robot links.
   - Configuration: Add `motion_tracker` config with device serial and target link.
   - Note: Not recommended for 6DOF arms like UR5e; better suited for redundant arms.
```

--------------------------------

### ARX R5 Dual Manipulator Configuration

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/teleop_details.md

Specifies the configuration for dual ARX R5 manipulators, including end effector link names, pose sources, control triggers, and detailed gripper settings for parallel grippers.

```python
DEFAULT_DUAL_ARX_R5_MANIPULATOR_CONFIG = {
    "right_arm": {
        "link_name": "right_link6",
        "pose_source": "right_controller",
        "control_trigger": "right_grip",
        "gripper_config": {
            "type": "parallel",
            "gripper_trigger": "right_trigger",
            "joint_names": ["right_joint7"],
            "open_pos": [4.9],
            "close_pos": [0.0],
        },
    },
    "left_arm": {
        "link_name": "left_link6",
        "pose_source": "left_controller",
        "control_trigger": "left_grip",
        "gripper_config": {
            "type": "parallel",
            "gripper_trigger": "left_trigger",
            "joint_names": ["left_joint7"],
            "open_pos": [4.9],
            "close_pos": [0.0],
        },
    },
}
```

--------------------------------

### XR Client Connection

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/teleop_details.md

Connects to an XR device using the Python binding of the xrobotoolkit-sdk. This utility is essential for establishing communication with the XR hardware.

```python
from xrobotoolkit_teleop.utils.xr_client import XRClient

# Example usage:
# xr_client = XRClient()
# xr_client.connect()
# ... perform teleoperation ...
# xr_client.disconnect()
```

--------------------------------

### Xrobotoolkit Threading Model

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/CLAUDE.md

Details the multi-threaded architecture used for hardware controllers, enabling concurrent operations like IK computation, robot control, data logging, and camera streaming.

```python
import threading

class HardwareController:
    def __init__(self):
        self.ik_thread = threading.Thread(target=self._run_ik_computation)
        self.control_thread = threading.Thread(target=self._run_robot_control)
        self.log_thread = threading.Thread(target=self._run_data_logging)
        self.camera_thread = threading.Thread(target=self._run_camera_streaming)

    def start_threads(self):
        self.ik_thread.start()
        self.control_thread.start()
        self.log_thread.start()
        self.camera_thread.start()

    def _run_ik_computation(self):
        # IK computation logic
        pass

    def _run_robot_control(self):
        # Robot control logic
        pass

    def _run_data_logging(self):
        # Data logging logic
        pass

    def _run_camera_streaming(self):
        # Camera streaming logic
        pass
```

--------------------------------

### Code Formatting

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/CLAUDE.md

Command to format all Python code in the repository using the 'black' formatter, enforcing a line length of 120 characters.

```python
black .
```

--------------------------------

### Teleoperation Controller Button Functions

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/README.md

API documentation detailing controller button functions for teleoperation. This covers grip buttons for arm activation, trigger buttons for gripper control, system buttons for logging, and joysticks for velocity commands.

```APIDOC
Controller Button Functions:

Grip Buttons:
  - `left_grip`: Activates left arm teleoperation. Hold to enable, release to deactivate.
  - `right_grip`: Activates right arm teleoperation. Hold to enable, release to deactivate.

Trigger Buttons:
  - `left_trigger`: Controls left gripper/hand. Analog control (0.0 = open, 1.0 = closed).
  - `right_trigger`: Controls right gripper/hand. Analog control (0.0 = open, 1.0 = closed).

System Buttons:
  - `A Button`: Reserved for system functions.
  - `B Button`: Toggles data logging on/off. Press once to start, press again to stop and save.

Joysticks/Touchpads:
  - Left Joystick: Linear velocity commands for mobile robots.
  - Right Joystick: Angular velocity commands for mobile robots.
  - Right Axis Click: Stops data logging, discarding current data.
```

--------------------------------

### Parallel Gripper Configuration

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/teleop_details.md

Adds optional gripper configuration to the end effector config for controlling parallel grippers, including joint names, triggers, and open/close positions.

```python
config = {
    "right_hand": {
        # other configs,
        "gripper_config": {
            "type": "parallel",
            "gripper_trigger": "right_trigger",
            "joint_names": ["right_gripper_finger_joint1",],
            "open_pos": [0.05,],
            "close_pos": [0.0,],
        },
    },
    "left_hand": {
        # other configs,
        "gripper_config": {
            "type": "parallel",
            "gripper_trigger": "left_trigger",
            "joint_names": ["left_gripper_finger_joint1",],
            "open_pos": [0.05,],
            "close_pos": [0.0,],
        },
    },
}
```

--------------------------------

### Xrobotoolkit Error Handling and Safety Checks

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/CLAUDE.md

Explains the implementation of safety checks and manipulability constraints within hardware controllers to prevent hazardous robot movements and ensure operational safety.

```python
class SafeRobotController:
    def __init__(self, safety_limits):
        self.safety_limits = safety_limits

    def move_joint(self, joint_name, target_position):
        joint_limit = self.safety_limits.get(joint_name)
        if joint_limit:
            if not (joint_limit['min'] <= target_position <= joint_limit['max']):
                raise ValueError(f"Target position for {joint_name} ({target_position}) exceeds joint limits ({joint_limit['min']}, {joint_limit['max']})")
        
        # Check for other safety constraints, e.g., manipulability
        if not self._is_manipulable(target_position):
            raise RuntimeError(f"Target position for {joint_name} is outside the robot's manipulable workspace.")

        # Proceed with movement if checks pass
        print(f"Moving {joint_name} to {target_position}")
        # self._execute_move(joint_name, target_position)

    def _is_manipulable(self, position):
        # Placeholder for checking if the position is within the robot's reachable workspace
        # This would involve kinematics calculations
        return True # Assume manipulable for example

# Example usage:
# safety_config = {
#     "joint1": {"min": -3.14, "max": 3.14}
# }
# controller = SafeRobotController(safety_config)
# try:
#     controller.move_joint("joint1", 3.0)
# except (ValueError, RuntimeError) as e:
#     print(f"Safety check failed: {e}")
```

--------------------------------

### Analyze Collected Data

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/README.md

Command to run the data log analysis script. This script verifies data integrity, displays data fields, checks robot states and images, shows sample entries, and provides data statistics.

```bash
python scripts/misc/test_data_log_analysis.py logs/<robot_name>/teleop_log_YYYYMMDD_HHMMSS_1.pkl
```

--------------------------------

### Start/Stop Data Collection (Python)

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/README.md

Initiates or stops teleoperation data logging during hardware demos. Pressing the 'B' button on the VR controller toggles logging. The right joystick click discards the current session.

```python
python scripts/hardware/teleop_dual_arx_r5_hardware.py
# Press B button on VR controller to start/stop logging
# Press right joystick click to discard session
```

--------------------------------

### Placo Tasks for Inverse Kinematics and Dynamics

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/teleop_details.md

Details the various tasks implemented using the Placo library for whole-body inverse kinematics and dynamics. These tasks help in defining robot motion constraints and stability.

```APIDOC
Placo Tasks:

1. Frame Task:
   - Description: Constrains the position and orientation of a frame.
   - Usage: Used to define end-effector tracking goals for Inverse Kinematics (IK).
   - Parameters: Requires definition of the target frame's pose.
   - Example: Setting an end-effector to follow a specific trajectory.

2. Manipulability Task:
   - Description: Enhances robot motion stability when commanded positions are near singularities or outside the workspace.
   - Reference: Based on the paper https://arxiv.org/abs/2002.11901.
   - Usage: Improves robustness of robot movements in challenging configurations.

3. Kinetic Energy Regularization:
   - Description: Minimizes the kinetic energy of the robot system during motion.
   - Usage: Promotes smoother and more energy-efficient movements.

4. Joint Task:
   - Description: Regularizes the joint state to remain close to a default or home configuration.
   - Usage: Prevents excessive joint movements and maintains a preferred posture.
```

--------------------------------

### UR5 Default End Effector Configuration

Source: https://github.com/xr-robotics/xrobotoolkit-teleop-sample-python/blob/main/teleop_details.md

Defines the default configuration for the end effectors of a UR5 robot, specifying link names, pose sources, and control triggers for both left and right arms.

```python
DEFAULT_END_EFFECTOR_CONFIG = {
    "left_arm": {
        "link_name": "left_tool0",
        "pose_source": "left_controller",
        "control_trigger": "left_grip",
        "gripper_trigger": "left_trigger",
    },
    "right_arm": {
        "link_name": "right_tool0",
        "pose_source": "right_controller",
        "control_trigger": "right_grip",
        "gripper_trigger": "right_trigger",
    },
}
```
Tokens:
Raw