### Build and Run Specific Example Docker Container

Source: https://github.com/physical-intelligence/openpi/blob/main/docs/docker.md

This command builds and runs the Docker image for a particular example within the OpenPI project. Replace `<example_name>` with the desired example's directory name to execute its specific Docker setup.

```bash
docker compose -f examples/<example_name>/compose.yml up --build
```

--------------------------------

### Build and Start Project Docker Container

Source: https://github.com/physical-intelligence/openpi/blob/main/docs/docker.md

This command builds the Docker image and starts the container for the entire OpenPI project using the main `compose.yml` file. It's typically used for the initial setup or when changes require a rebuild.

```bash
docker compose -f scripts/docker/compose.yml up --build
```

--------------------------------

### ALOHA Checkpoint Guide: Toast Task Scene Setup Guidelines

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/aloha_real/README.md

Guidelines for optimally setting up the scene for the 'Toast Task' to maximize policy success. This includes precise positioning of the toaster, toast, and plate, along with recommendations for lighting conditions.

```APIDOC
The toaster should be positioned in the top-left quadrant of the workspace.
Both pieces of toast should start inside the toaster, with at least 1 cm of bread sticking out from the top.
The plate should be placed roughly in the lower-center of the workspace.
Works with both natural and synthetic lighting, but avoid making the scene too dark (e.g., don't place the setup inside an enclosed space or under a curtain).
```

--------------------------------

### ALOHA Checkpoint Guide: Towel Task Scene Setup Guidelines

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/aloha_real/README.md

Guidelines for optimally setting up the scene for the 'Towel Task' to maximize policy success. This includes precise positioning of the towel and recommendations for color contrast with the table surface.

```APIDOC
The towel should be flattened and roughly centered on the table.
Choose a towel that does not blend in with the table surface.
```

--------------------------------

### Setup Virtual Environment and Run Aloha Sim Simulation

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/aloha_sim/README.md

This snippet details the steps to set up a Python virtual environment, install necessary dependencies using 'uv pip', activate the environment, and then execute the main simulation script. It also specifies 'MUJOCO_GL=egl' for rendering.

```bash
# Create virtual environment
uv venv --python 3.10 examples/aloha_sim/.venv
source examples/aloha_sim/.venv/bin/activate
uv pip sync examples/aloha_sim/requirements.txt
uv pip install -e packages/openpi-client

# Run the simulation
MUJOCO_GL=egl python examples/aloha_sim/main.py
```

--------------------------------

### Run Aloha with Docker Compose

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/aloha_real/README.md

This snippet demonstrates how to launch the Aloha real robot setup using Docker Compose. It configures server arguments, including the environment and a default prompt, then builds and starts the necessary services.

```bash
export SERVER_ARGS="--env ALOHA --default_prompt='take the toast out of the toaster'"
docker compose -f examples/aloha_real/compose.yml up --build
```

--------------------------------

### Run Aloha Without Docker: Terminal 1 - Setup and Robot Execution

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/aloha_real/README.md

This sequence of commands sets up a Python virtual environment using 'uv', installs required dependencies, and then executes the main script to control the robot for the Aloha setup.

```bash
# Create virtual environment
uv venv --python 3.10 examples/aloha_real/.venv
source examples/aloha_real/.venv/bin/activate
uv pip sync examples/aloha_real/requirements.txt
uv pip install -e packages/openpi-client

# Run the robot
python -m examples.aloha_real.main
```

--------------------------------

### Setup Virtual Environment and Run LIBERO Simulation (Without Docker)

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/libero/README.md

This sequence of commands sets up a Python 3.8 virtual environment, installs all required dependencies including OpenPI client and LIBERO, configures the PYTHONPATH, and then executes the main LIBERO simulation script. This is intended for use in the first terminal window when running without Docker.

```bash
uv venv --python 3.8 examples/libero/.venv
source examples/libero/.venv/bin/activate
uv pip sync examples/libero/requirements.txt third_party/libero/requirements.txt --extra-index-url https://download.pytorch.org/whl/cu113 --index-strategy=unsafe-best-match
uv pip install -e packages/openpi-client
uv pip install -e third_party/libero
export PYTHONPATH=$PYTHONPATH:$PWD/third_party/libero

python examples/libero/main.py
```

--------------------------------

### Run Simple Client with Docker Compose (Bash)

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/simple_client/README.md

Sets the server arguments to use the ALOHA_SIM environment and then starts the simple client and its policy server using Docker Compose. This command builds the necessary images if they don't exist.

```bash
export SERVER_ARGS="--env ALOHA_SIM"
docker compose -f examples/simple_client/compose.yml up --build
```

--------------------------------

### Start generic openpi remote policy server

Source: https://github.com/physical-intelligence/openpi/blob/main/docs/remote_inference.md

Run this command to start a remote policy server. The `env` argument specifies which π₀ checkpoint should be loaded, supporting DROID, ALOHA, or LIBERO environments.

```bash
uv run scripts/serve_policy.py --env=[DROID | ALOHA | LIBERO]
```

--------------------------------

### Run Aloha Sim Policy Server

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/aloha_sim/README.md

This snippet shows how to start the policy server for the Aloha simulation. It uses 'uv run' to execute the 'serve_policy.py' script and passes an environment argument to specify the simulation environment.

```bash
# Run the server
uv run scripts/serve_policy.py --env ALOHA_SIM
```

--------------------------------

### ALOHA Checkpoint Guide: Towel Task Model Parameters

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/aloha_real/README.md

Details for the 'Towel Task' using the `pi0_aloha_towel` model, including its S3 checkpoint path, the required prompt for execution, and specific object distribution guidelines for successful task completion.

```APIDOC
Checkpoint path: s3://openpi-assets/checkpoints/pi0_aloha_towel
Prompt: "fold the towel"
Object Distribution:
  - Works on towels of varying solid colors
  - Performance is worse on heavily textured or striped towels
```

--------------------------------

### ALOHA Checkpoint Guide: Toast Task Model Parameters

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/aloha_real/README.md

Details for the 'Toast Task' using the `pi0_base` model, including its S3 checkpoint path, the required prompt for execution, and specific object requirements and distribution guidelines for successful task completion.

```APIDOC
Checkpoint path: s3://openpi-assets/checkpoints/pi0_base
Prompt: "take the toast out of the toaster"
Objects needed: Two pieces of toast, a plate, and a standard toaster.
Object Distribution:
  - Works on both real toast and rubber fake toast
  - Compatible with standard 2-slice toasters
  - Works with plates of varying colors
```

--------------------------------

### Start OpenPI Policy Server for DROID (pi0-FAST-DROID)

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/droid/README.md

This command initiates the OpenPI policy server on a machine with a powerful GPU, serving the `pi0_fast_droid` model. It requires the `openpi` repository to be cloned and installed, and the server will be queried remotely by the DROID control laptop.

```bash
uv run scripts/serve_policy.py policy:checkpoint --policy.config=pi0_fast_droid --policy.dir=s3://openpi-assets/checkpoints/pi0_fast_droid
```

--------------------------------

### Run Policy Inference with OpenPI

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/inference.ipynb

Demonstrates how to load a pre-trained policy from a checkpoint and perform inference using a dummy example, specifically for DROID runtime observations. The policy is deleted afterwards to free up memory.

```python
config = _config.get_config("pi0_fast_droid")
checkpoint_dir = download.maybe_download("s3://openpi-assets/checkpoints/pi0_fast_droid")

# Create a trained policy.
policy = _policy_config.create_trained_policy(config, checkpoint_dir)

# Run inference on a dummy example. This example corresponds to observations produced by the DROID runtime.
example = droid_policy.make_droid_example()
result = policy.infer(example)

# Delete the policy to free up memory.
del policy

print("Actions shape:", result["actions"].shape)
```

--------------------------------

### ALOHA Checkpoint Guide: Tupperware Task Model Parameters

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/aloha_real/README.md

Details for the 'Tupperware Task' using the `pi0_aloha_tupperware` model, including its S3 checkpoint path, the required prompt for execution, and specific object requirements and distribution guidelines for successful task completion.

```APIDOC
Checkpoint path: s3://openpi-assets/checkpoints/pi0_aloha_tupperware
Prompt: "open the tupperware and put the food on the plate"
Objects needed: Tupperware, food (or food-like items), and a plate.
Object Distribution:
  - Works on various types of fake food (e.g., fake chicken nuggets, fries, and fried chicken).
  - Compatible with tupperware of different lid colors and shapes, with best performance on square tupperware with a corner flap (see images below).
  - The policy has seen plates of varying solid colors.
```

--------------------------------

### Run Simple Client and Policy Server Directly (Bash)

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/simple_client/README.md

Executes the simple client and its corresponding policy server in separate terminal windows using `uv run`, configured for the DROID environment. This setup does not require Docker.

```bash
uv run examples/simple_client/main.py --env DROID
```

```bash
uv run scripts/serve_policy.py --env DROID
```

--------------------------------

### Install openpi-client package for robot environment

Source: https://github.com/physical-intelligence/openpi/blob/main/docs/remote_inference.md

Install the `openpi-client` package in your robot environment. This enables querying the remote policy server with minimal dependencies. Navigate to the package directory and use pip to install it in editable mode.

```bash
cd $OPENPI_ROOT/packages/openpi-client
pip install -e .
```

--------------------------------

### Alternative Command to Start OpenPI Policy Server for DROID

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/droid/README.md

An alternative, simplified command to start the OpenPI policy server, specifically configured for the DROID environment. This command abstracts away the specific model checkpoint details.

```bash
uv run scripts/serve_policy.py --env=DROID
```

--------------------------------

### Install EGL Dependencies for Mujoco

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/aloha_sim/README.md

This snippet provides a command to install required EGL dependencies on Debian/Ubuntu-based systems. These dependencies are often necessary to resolve EGL errors when running simulations that rely on Mujoco's OpenGL rendering.

```bash
sudo apt-get install -y libegl1-mesa-dev libgles2-mesa-dev
```

--------------------------------

### Install pre-commit hooks for openpi

Source: https://github.com/physical-intelligence/openpi/blob/main/CONTRIBUTING.md

Before submitting a pull request, contributors should install the pre-commit hooks. This ensures that code quality checks and formatting are automatically applied before committing changes, helping maintain consistent code style and preventing common issues.

```Shell
pre-commit install
```

--------------------------------

### Compute Training Loss with OpenPI Live Model (Real Data)

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/inference.ipynb

Shows how to load a single batch of real training data using a data loader and compute the model's training loss. This example also demonstrates how to reduce batch size for memory optimization and notes the skipping of normalization for self-containment.

```python
# Reduce the batch size to reduce memory usage.
config = dataclasses.replace(config, batch_size=2)

# Load a single batch of data. This is the same data that will be used during training.
# NOTE: In order to make this example self-contained, we are skipping the normalization step
# since it requires the normalization statistics to be generated using `compute_norm_stats`.
loader = _data_loader.create_data_loader(config, num_batches=1, skip_norm_stats=True)
obs, act = next(iter(loader))

# Sample actions from the model.
loss = model.compute_loss(key, obs, act)

# Delete the model to free up memory.
del model

print("Loss shape:", loss.shape)
```

--------------------------------

### Start openpi remote policy server with specific checkpoint

Source: https://github.com/physical-intelligence/openpi/blob/main/docs/remote_inference.md

Use this command to start a policy server for custom-trained checkpoints. It specifies the policy configuration and directory, exemplified here for the DROID environment.

```bash
uv run scripts/serve_policy.py policy:checkpoint --policy.config=pi0_fast_droid --policy.dir=s3://openpi-assets/checkpoints/pi0_fast_droid
```

--------------------------------

### Install openpi Python Dependencies with uv (Bash)

Source: https://github.com/physical-intelligence/openpi/blob/main/README.md

These Bash commands use 'uv' to manage and install Python dependencies for the openpi project. The 'GIT_LFS_SKIP_SMUDGE=1' prefix is essential to correctly handle large file storage (LFS) dependencies, specifically for the LeRobot library.

```bash
GIT_LFS_SKIP_SMUDGE=1 uv sync
GIT_LFS_SKIP_SMUDGE=1 uv pip install -e .
```

--------------------------------

### Run Aloha Sim with Docker Compose

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/aloha_sim/README.md

This snippet shows how to run the Aloha simulation using Docker Compose. It sets an environment variable for the server arguments and then builds and starts the services defined in the 'compose.yml' file.

```bash
export SERVER_ARGS="--env ALOHA_SIM"
docker compose -f examples/aloha_sim/compose.yml up --build
```

--------------------------------

### Configure LeRobot UR5 Data Processing for Training

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/ur5/README.md

This snippet defines the `LeRobotUR5DataConfig` class, which extends `DataConfigFactory` to configure the processing of raw UR5 data from LeRobot datasets for model training. It includes remapping input keys, applying `UR5Inputs` and `UR5Outputs` transforms, converting actions to delta actions, and integrating model-specific transforms.

```python
@dataclasses.dataclass(frozen=True)
class LeRobotUR5DataConfig(DataConfigFactory):

    @override
    def create(self, assets_dirs: pathlib.Path, model_config: _model.BaseModelConfig) -> DataConfig:
        # Boilerplate for remapping keys from the LeRobot dataset. We assume no renaming needed here.
        repack_transform = _transforms.Group(
            inputs=[
                _transforms.RepackTransform(
                    {
                        "base_rgb": "image",
                        "wrist_rgb": "wrist_image",
                        "joints": "joints",
                        "gripper": "gripper",
                        "prompt": "prompt",
                    }
                )
            ]
        )

        # These transforms are the ones we wrote earlier.
        data_transforms = _transforms.Group(
            inputs=[UR5Inputs(action_dim=model_config.action_dim, model_type=model_config.model_type)],
            outputs=[UR5Outputs()],
        )

        # Convert absolute actions to delta actions.
        # By convention, we do not convert the gripper action (7th dimension).
        delta_action_mask = _transforms.make_bool_mask(6, -1)
        data_transforms = data_transforms.push(
            inputs=[_transforms.DeltaActions(delta_action_mask)],
            outputs=[_transforms.AbsoluteActions(delta_action_mask)],
        )

        # Model transforms include things like tokenizing the prompt and action targets
        # You do not need to change anything here for your own dataset.
        model_transforms = ModelTransformFactory()(model_config)

        # We return all data transforms for training and inference. No need to change anything here.
        return dataclasses.replace(
            self.create_base_config(assets_dirs),
            repack_transforms=repack_transform,
            data_transforms=data_transforms,
            model_transforms=model_transforms,
        )
```

--------------------------------

### Run LIBERO Benchmark with Docker Compose

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/libero/README.md

Builds and starts the Docker services defined in the compose file for the LIBERO benchmark. It sets the LIBERO environment variable for the containerized environment.

```bash
export SERVER_ARGS="--env LIBERO"
docker compose -f examples/libero/compose.yml up --build
```

--------------------------------

### Define UR5Inputs and UR5Outputs Classes for Data Mapping

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/ur5/README.md

This snippet defines the `UR5Inputs` and `UR5Outputs` dataclasses. `UR5Inputs` transforms raw UR5 environment data (joints, gripper, images, actions, prompt) into a format suitable for the model, handling padding and image parsing. `UR5Outputs` converts model outputs back to the robot's 7-dimensional action space.

```python
@dataclasses.dataclass(frozen=True)
class UR5Inputs(transforms.DataTransformFn):

    action_dim: int
    model_type: _model.ModelType = _model.ModelType.PI0

    def __call__(self, data: dict) -> dict:
        mask_padding = self.model_type == _model.ModelType.PI0

        # First, concatenate the joints and gripper into the state vector.
        # Pad to the expected input dimensionality of the model (same as action_dim).
        state = np.concatenate([data["joints"], data["gripper"]])
        state = transforms.pad_to_dim(state, self.action_dim)

        # Possibly need to parse images to uint8 (H,W,C) since LeRobot automatically
        # stores as float32 (C,H,W), gets skipped for policy inference.
        base_image = _parse_image(data["base_rgb"])
        wrist_image = _parse_image(data["wrist_rgb"])

        # Create inputs dict.
        inputs = {
            "state": state,
            "image": {
                "base_0_rgb": base_image,
                "left_wrist_0_rgb": wrist_image,
                # Since there is no right wrist, replace with zeros
                "right_wrist_0_rgb": np.zeros_like(base_image),
            },
            "image_mask": {
                "base_0_rgb": np.True_,
                "left_wrist_0_rgb": np.True_,
                # Since the "slot" for the right wrist is not used, this mask is set
                # to False
                "right_wrist_0_rgb": np.False_ if mask_padding else np.True_,
            },
        }

        # Pad actions to the model action dimension.
        if "actions" in data:
            # The robot produces 7D actions (6 DoF + 1 gripper), and we pad these.
            actions = transforms.pad_to_dim(data["actions"], self.action_dim)
            inputs["actions"] = actions

        # Pass the prompt (aka language instruction) to the model.
        if "prompt" in data:
            inputs["prompt"] = data["prompt"]

        return inputs


@dataclasses.dataclass(frozen=True)
class UR5Outputs(transforms.DataTransformFn):

    def __call__(self, data: dict) -> dict:
        # Since the robot has 7 action dimensions (6 DoF + gripper), return the first 7 dims
        return {"actions": np.asarray(data["actions"][:, :7])}
```

--------------------------------

### Run Policy Server for LIBERO (Without Docker)

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/libero/README.md

Starts the policy server script required for the LIBERO benchmark, ensuring the LIBERO environment is enabled. This command is intended to be run in a separate terminal window concurrently with the simulation.

```bash
uv run scripts/serve_policy.py --env LIBERO
```

--------------------------------

### Python: Define TrainConfig for pi0 UR5 Dataset Fine-tuning

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/ur5/README.md

This Python configuration defines the `TrainConfig` for fine-tuning the `pi0` model on a UR5 dataset. It specifies the dataset repository ID, asset locations for reloading normalization statistics from the base model checkpoint, and base data configuration including whether to load prompts from the task field. It also sets the weight loader to load parameters from a pre-trained `pi0_base` checkpoint and defines the total number of training steps.

```python
TrainConfig(
    name="pi0_ur5",
    model=pi0.Pi0Config(),
    data=LeRobotUR5DataConfig(
        repo_id="your_username/ur5_dataset",
        # This config lets us reload the UR5 normalization stats from the base model checkpoint.
        # Reloading normalization stats can help transfer pre-trained models to new environments.
        # See the [norm_stats.md](../docs/norm_stats.md) file for more details.
        assets=AssetsConfig(
            assets_dir="s3://openpi-assets/checkpoints/pi0_base/assets",
            asset_id="ur5e",
        ),
        base_config=DataConfig(
            # This flag determines whether we load the prompt (i.e. the task instruction) from the
            # ``task`` field in the LeRobot dataset. The recommended setting is True.
            prompt_from_task=True,
        ),
    ),
    # Load the pi0 base model checkpoint.
    weight_loader=weight_loaders.CheckpointWeightLoader("s3://openpi-assets/checkpoints/pi0_base/params"),
    num_train_steps=30_000,
)
```

--------------------------------

### Query openpi remote policy server from robot code

Source: https://github.com/physical-intelligence/openpi/blob/main/docs/remote_inference.md

This Python example demonstrates initializing the `WebsocketClientPolicy` and querying the remote policy server within an episode loop. It shows constructing the observation dictionary, including image resizing and state handling, then inferring actions. The `host` and `port` arguments specify the server's address.

```python
from openpi_client import image_tools
from openpi_client import websocket_client_policy

# Outside of episode loop, initialize the policy client.
# Point to the host and port of the policy server (localhost and 8000 are the defaults).
client = websocket_client_policy.WebsocketClientPolicy(host="localhost", port=8000)

for step in range(num_steps):
    # Inside the episode loop, construct the observation.
    # Resize images on the client side to minimize bandwidth / latency. Always return images in uint8 format.
    # We provide utilities for resizing images + uint8 conversion so you match the training routines.
    # The typical resize_size for pre-trained pi0 models is 224.
    # Note that the proprioceptive `state` can be passed unnormalized, normalization will be handled on the server side.
    observation = {
        "observation/image": image_tools.convert_to_uint8(
            image_tools.resize_with_pad(img, 224, 224)
        ),
        "observation/wrist_image": image_tools.convert_to_uint8(
            image_tools.resize_with_pad(wrist_img, 224, 224)
        ),
        "observation/state": state,
        "prompt": task_instruction,
    }

    # Call the policy server with the current observation.
    # This returns an action chunk of shape (action_horizon, action_dim).
    # Note that you typically only need to call the policy every N steps and execute steps
    # from the predicted action chunk open-loop in the remaining steps.
    action_chunk = client.infer(observation)["actions"]

    # Execute the actions in the environment.
    ...
```

--------------------------------

### Run Aloha Without Docker: Terminal 2 - ROS Nodes Launch

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/aloha_real/README.md

This command launches the essential ROS nodes for the Aloha system, which are crucial for managing robot communication and control within the setup.

```bash
roslaunch aloha ros_nodes.launch
```

--------------------------------

### Run Aloha Without Docker: Terminal 3 - Serve Policy

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/aloha_real/README.md

This command executes a script to serve the robot's policy, specifying the ALOHA environment and a default prompt to guide the robot's initial actions.

```bash
uv run scripts/serve_policy.py --env ALOHA --default_prompt='take the toast out of the toaster'
```

--------------------------------

### Visualize Policy Input Images

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/policy_records.ipynb

This snippet defines functions to extract and display image data from the loaded policy records. `get_image` retrieves and formats an image from a specific step and index, converting it to an 8-bit unsigned integer array suitable for display. `show_image` combines multiple images horizontally and displays them using the PIL (Pillow) library. The example demonstrates displaying images for the first two policy steps.

```python
from PIL import Image


def get_image(step: int, idx: int = 0):
    img = (255 * records[step]["inputs/image"]).astype(np.uint8)
    return img[idx].transpose(1, 2, 0)


def show_image(step: int, idx_lst: list[int]):
    imgs = [get_image(step, idx) for idx in idx_lst]
    return Image.fromarray(np.hstack(imgs))


for i in range(2):
    display(show_image(i, [0]))
```

--------------------------------

### Reloading Normalization Statistics for Trossen Robot (ALOHA) in Python

Source: https://github.com/physical-intelligence/openpi/blob/main/docs/norm_stats.md

This Python code snippet demonstrates how to configure a `TrainConfig` to reload pre-existing normalization statistics for the Trossen (ALOHA) robot. It specifies the `assets_dir` and `asset_id` within the `AssetsConfig` to ensure the model uses familiar action space definitions from pre-training, which can improve performance when fine-tuning on similar robots and tasks. The example uses `pi0_base` checkpoint assets.

```python
TrainConfig(
    ...
    data=LeRobotAlohaDataConfig(
        ...
        assets=AssetsConfig(
            assets_dir="s3://openpi-assets/checkpoints/pi0_base/assets",
            asset_id="trossen",
        ),
    ),
)
```

--------------------------------

### Display Simple Client Runtime Options (Bash)

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/simple_client/README.md

Shows the command-line help for the simple client, listing all available runtime environments that can be specified with the `--env` flag.

```bash
uv run examples/simple_client/main.py --help
```

--------------------------------

### Initialize Git Submodules

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/libero/README.md

Ensures all necessary git submodules for the project are initialized and updated recursively, which is a prerequisite for running the LIBERO benchmark.

```bash
git submodule update --init --recursive
```

--------------------------------

### Run Inference with OpenPI Pre-Trained Model

Source: https://github.com/physical-intelligence/openpi/blob/main/README.md

This Python code demonstrates how to load and use a pre-trained OpenPI model checkpoint for inference. It involves configuring the model, downloading its assets, creating a trained policy, and then executing the policy with a sample observation and prompt to obtain actions.

```python
from openpi.training import config
from openpi.policies import policy_config
from openpi.shared import download

config = config.get_config("pi0_fast_droid")
checkpoint_dir = download.maybe_download("s3://openpi-assets/checkpoints/pi0_fast_droid")

# Create a trained policy.
policy = policy_config.create_trained_policy(config, checkpoint_dir)

# Run inference on a dummy example.
example = {
    "observation/exterior_image_1_left": ...,
    "observation/wrist_image_left": ...,
    ...
    "prompt": "pick up the fork"
}
action_chunk = policy.infer(example)["actions"]
```

--------------------------------

### Spin Up Policy Server for Inference

Source: https://github.com/physical-intelligence/openpi/blob/main/README.md

After training, this command launches a policy server that listens for observations on port 8000. It uses a specific model checkpoint and configuration to serve the trained policy, enabling remote inference queries.

```bash
uv run scripts/serve_policy.py policy:checkpoint --policy.config=pi0_fast_libero --policy.dir=checkpoints/pi0_fast_libero/my_experiment/20000
```

--------------------------------

### Compute Training Loss with OpenPI Live Model (Fake Data)

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/inference.ipynb

Illustrates how to load a live model from a checkpoint and compute its training loss using synthetic observations and actions. This method is suitable for initial testing and verification of the model's functionality.

```python
config = _config.get_config("pi0_aloha_sim")

checkpoint_dir = download.maybe_download("s3://openpi-assets/checkpoints/pi0_aloha_sim")
key = jax.random.key(0)

# Create a model from the checkpoint.
model = config.model.load(_model.restore_params(checkpoint_dir / "params"))

# We can create fake observations and actions to test the model.
obs, act = config.model.fake_obs(), config.model.fake_act()

# Sample actions from the model.
loss = model.compute_loss(key, obs, act)
print("Loss shape:", loss.shape)
```

--------------------------------

### Run Model Training with GPU Memory Optimization

Source: https://github.com/physical-intelligence/openpi/blob/main/README.md

This command initiates the fine-tuning process for a base model using the specified configuration. It includes a setting to optimize GPU memory usage for JAX, allowing it to utilize up to 90% of available GPU memory. The --overwrite flag allows overwriting existing checkpoints.

```bash
XLA_PYTHON_CLIENT_MEM_FRACTION=0.9 uv run scripts/train.py pi0_fast_libero --exp-name=my_experiment --overwrite
```

--------------------------------

### Run Ruff checks and formatting for openpi

Source: https://github.com/physical-intelligence/openpi/blob/main/CONTRIBUTING.md

To ensure code quality and adherence to project style guidelines, contributors must manually run Ruff for linting and formatting. This step is crucial before submitting a pull request to catch any remaining style or linting issues.

```Shell
ruff check .
ruff format .
```

--------------------------------

### OpenPI Configuration Classes for Training and Inference

Source: https://github.com/physical-intelligence/openpi/blob/main/README.md

Defines the structure and purpose of key configuration classes used for data mapping, processing, and fine-tuning hyperparameters within the OpenPI project.

```APIDOC
LiberoInputs:
  Description: Defines the data mapping from the Libero environment to the model and vice versa.
  Usage: Used for both training and inference.

LiberoOutputs:
  Description: Defines the data mapping from the model to the Libero environment.
  Usage: Used for both training and inference.

LeRobotLiberoDataConfig:
  Description: Defines how to process raw Libero data from LeRobot dataset for training.

TrainConfig:
  Description: Defines fine-tuning hyperparameters, data config, and weight loader.
```

--------------------------------

### Clone openpi Repository and Update Submodules (Bash)

Source: https://github.com/physical-intelligence/openpi/blob/main/README.md

This snippet provides the necessary Git commands to clone the openpi repository, including all its submodules. It also includes an alternative command to update submodules if the repository has already been cloned, ensuring all dependencies are correctly fetched.

```bash
git clone --recurse-submodules git@github.com:Physical-Intelligence/openpi.git

# Or if you already cloned the repo:
git submodule update --init --recursive
```

--------------------------------

### Load Policy Records from Files

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/policy_records.ipynb

This snippet loads policy record data stored as NumPy files from a specified directory. It iterates through 'step_*.npy' files, loads each as a dictionary, and appends it to a list called 'records'. This forms the primary dataset for subsequent analysis.

```python
import pathlib

import numpy as np

record_path = pathlib.Path("../policy_records")
num_steps = len(list(record_path.glob("step_*.npy")))

records = []
for i in range(num_steps):
    record = np.load(record_path / f"step_{i}.npy", allow_pickle=True).item()
    records.append(record)
```

--------------------------------

### Grant X11 Access for Docker

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/libero/README.md

Grants local Docker containers access to the X11 server, which is necessary for graphical applications and simulations run within Docker.

```bash
sudo xhost +local:docker
```

--------------------------------

### Compute Normalization Statistics for Training Data

Source: https://github.com/physical-intelligence/openpi/blob/main/README.md

Before training, normalization statistics for the dataset must be computed. This command runs a script to calculate these statistics based on the specified training configuration name.

```bash
uv run scripts/compute_norm_stats.py --config-name pi0_fast_libero
```

--------------------------------

### Convert Libero Data to LeRobot Dataset

Source: https://github.com/physical-intelligence/openpi/blob/main/README.md

This command executes a Python script to convert raw Libero dataset into the LeRobot dataset format. The script can be modified to suit custom data conversion needs.

```bash
uv run examples/libero/convert_libero_data_to_lerobot.py --data_dir /path/to/your/libero/data
```

--------------------------------

### Python Project Dependencies (requirements.txt)

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/libero/requirements.txt

This snippet lists all Python package dependencies with their exact versions and the packages that require them. It was generated using 'uv pip compile' to ensure a consistent and reproducible development environment for the 'openpi' project.

```Python
# This file was autogenerated by uv via the following command:\n#    uv pip compile examples/libero/requirements.in -o examples/libero/requirements.txt --python-version 3.8 --index-strategy=unsafe-best-match\nabsl-py==2.1.0\n    # via mujoco\ncertifi==2024.12.14\n    # via requests\ncharset-normalizer==3.4.0\n    # via requests\ncycler==0.12.1\n    # via matplotlib\ndocstring-parser==0.16\n    # via tyro\netils==1.3.0\n    # via mujoco\neval-type-backport==0.2.0\n    # via tyro\nevdev==1.7.1\n    # via pynput\nfonttools==4.55.3\n    # via matplotlib\nglfw==1.12.0\n    # via mujoco\nidna==3.10\n    # via requests\nimageio==2.35.1\n    # via -r examples/libero/requirements.in\nimageio-ffmpeg==0.5.1\n    # via imageio\nimportlib-metadata==8.5.0\n    # via typeguard\nimportlib-resources==6.4.5\n    # via etils\nkiwisolver==1.4.7\n    # via matplotlib\nllvmlite==0.36.0\n    # via numba\nmarkdown-it-py==3.0.0\n    # via rich\nmatplotlib==3.5.3\n    # via -r examples/libero/requirements.in\nmdurl==0.1.2\n    # via markdown-it-py\nmujoco==3.2.3\n    # via robosuite\nnumba==0.53.1\n    # via robosuite\nnumpy==1.22.4\n    # via\n    #   -r examples/libero/requirements.in\n    #   imageio\n    #   matplotlib\n    #   mujoco\n    #   numba\n    #   opencv-python\n    #   robosuite\n    #   scipy\n    #   torchvision\nopencv-python==4.6.0.66\n    # via\n    #   -r examples/libero/requirements.in\n    #   robosuite\npackaging==24.2\n    # via matplotlib\npillow==10.4.0\n    # via\n    #   imageio\n    #   matplotlib\n    #   robosuite\n    #   torchvision\npsutil==6.1.0\n    # via imageio\npygments==2.18.0\n    # via rich\npynput==1.7.7\n    # via robosuite\npyopengl==3.1.7\n    # via mujoco\npyparsing==3.1.4\n    # via matplotlib\npython-dateutil==2.9.0.post0\n    # via matplotlib\npython-xlib==0.33\n    # via pynput\npyyaml==6.0.2\n    # via -r examples/libero/requirements.in\nrequests==2.32.3\n    # via torchvision\nrich==13.9.4\n    # via tyro\nrobosuite==1.4.1\n    # via -r examples/libero/requirements.in\nscipy==1.10.1\n    # via robosuite\nsetuptools==75.3.0\n    # via\n    #   imageio-ffmpeg\n    #   numba\nshtab==1.7.1\n    # via tyro\nsix==1.17.0\n    # via\n    #   pynput\n    #   python-dateutil\n    #   python-xlib\ntermcolor==2.4.0\n    # via robosuite\ntorch==1.11.0+cu113\n    # via\n    #   -r examples/libero/requirements.in\n    #   torchaudio\n    #   torchvision\ntorchaudio==0.11.0+cu113\n    # via -r examples/libero/requirements.in\ntorchvision==0.12.0+cu113\n    # via -r examples/libero/requirements.in\ntqdm==4.67.1\n    # via -r examples/libero/requirements.in\ntypeguard==4.4.0\n    # via tyro\ntyping-extensions==4.12.2\n    # via\n    #   etils\n    #   rich\n    #   torch\n    #   torchvision\n    #   typeguard\n    #   tyro\ntyro==0.9.2\n    # via -r examples/libero/requirements.in\nurllib3==2.2.3\n    # via requests\nzipp==3.20.2\n    # via\n    #   etils\n    #   importlib-metadata\n    #   importlib-resources
```

--------------------------------

### Execute DROID Robot Control Script

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/droid/README.md

This Python command runs the `main.py` script on the DROID control laptop to operate the robot. It connects to the remote policy server using the specified IP and port, and designates which external camera ('left' or 'right') to use for policy input. Ensure camera IDs in `main.py` are updated and the server is reachable.

```python
python3 scripts/main.py --remote_host=<server_ip> --remote_port=<server_port> --external_camera="left"
```

--------------------------------

### Prepare Joint Angle Data for Analysis

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/policy_records.ipynb

This code prepares joint angle data ('qpos') from the policy records for further analysis, typically plotting. It defines `get_axis` to extract a specific dimension across all records and `make_data` to organize input and output 'qpos' values into separate Pandas DataFrames. The 'qpos' array structure is detailed, specifying dimensions for left/right arm joints and grippers.

```python
import pandas as pd


def get_axis(name, axis):
    return np.array([record[name][axis] for record in records])


# qpos is [..., 14] of type float:
# 0-5: left arm joint angles
# 6: left arm gripper
# 7-12: right arm joint angles
# 13: right arm gripper
names = [("left_joint", 6), ("left_gripper", 1), ("right_joint", 6), ("right_gripper", 1)]


def make_data():
    cur_dim = 0
    in_data = {}
    out_data = {}
    for name, dim_size in names:
        for i in range(dim_size):
            in_data[f"{name}_{i}"] = get_axis("inputs/qpos", cur_dim)
            out_data[f"{name}_{i}"] = get_axis("outputs/qpos", cur_dim)
            cur_dim += 1
    return pd.DataFrame(in_data), pd.DataFrame(out_data)

in_data, out_data = make_data()
```

--------------------------------

### Inspect Loaded Policy Record Data Structure

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/policy_records.ipynb

After loading the policy records, this snippet provides a quick inspection of the data. It prints the total number of records loaded, the keys available in the first record (assuming consistent structure), and the shape of the NumPy array associated with each key, helping to understand the dimensions of the stored data.

```python
print("length of records", len(records))
print("keys in records", records[0].keys())

for k in records[0]:
    print(f"{k} shape: {records[0][k].shape}")
```

--------------------------------

### Plot Input vs. Output Joint Angles

Source: https://github.com/physical-intelligence/openpi/blob/main/examples/policy_records.ipynb

This snippet visualizes the prepared joint angle data. It iterates through each joint's input and output data (from the DataFrames created by `make_data`) and generates a plot for each, showing the relationship or progression of input versus output joint angles. This is useful for analyzing policy behavior and control accuracy over time.

```python
for name in in_data.columns:
    data = pd.DataFrame({f"in_{name}": in_data[name], f"out_{name}": out_data[name]})
    data.plot()
```

--------------------------------

### Pi0 Model Action Space Dimensions

Source: https://github.com/physical-intelligence/openpi/blob/main/docs/norm_stats.md

Defines the mapping of dimensions to robot actions for Pi0 models, including joint angles, gripper positions for single and bi-manual arms, and base velocities for mobile robots. This structure is used by `pi0_base` and `pi0_fast_base`.

```JSON
    "dim_0:dim_5": "left arm joint angles",
    "dim_6": "left arm gripper position",
    "dim_7:dim_12": "right arm joint angles (for bi-manual only)",
    "dim_13": "right arm gripper position (for bi-manual only)",

    # For mobile robots:
    "dim_14:dim_15": "x-y base velocity (for mobile robots only)",
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.