### Install Old-Version Mujoco (Linux)

Source: https://github.com/liruiw/hpt/blob/main/README.md

Provides instructions for installing an older version of Mujoco (210) on Linux. This involves downloading the tarball, extracting it, and setting necessary environment variables for library paths (LD_LIBRARY_PATH) and the Mujoco GL backend (MUJOCO_GL) to ensure proper functioning.

```bash
mkdir ~/.mujoco
cd ~/.mujoco
wget https://mujoco.org/download/mujoco210-linux-x86_64.tar.gz  -O mujoco210.tar.gz --no-check-certificate
tar -xvzf mujoco210.tar.gz

# add the following line to ~/.bashrc if needed
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${HOME}/.mujoco/mujoco210/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/nvidia
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64
export MUJOCO_GL=egl
```

--------------------------------

### Run Policy Rollout in a Metaworld Environment

Source: https://github.com/liruiw/hpt/blob/main/quickstart.ipynb

This code block sets up and executes a single step of policy rollout within a Metaworld environment. It initializes a specific environment (e.g., 'reach-v2'), configures observation handling, renders an initial image, and resets the environment. The policy then computes an action based on the current observation, and the environment is stepped forward. This simulates how the policy interacts with its environment.

```python
from env.mujoco.metaworld.envs.mujoco.sawyer_xyz.test_scripted_policies import ALL_ENVS
from collections import OrderedDict
RESOLUTION = (128, 128)
camera_name="view_1"
env_name = "reach-v2"
def get_observation_dict(o, img):
    step_data = {"state": o, "image": img}
    return OrderedDict(step_data)

env = ALL_ENVS[env_name]()
env._partially_observable = False
env._freeze_rand_vec = False
env._set_task_called = True
img = env.sim.render(*RESOLUTION, mode="offscreen", camera_name=camera_name)[:, :, ::-1].copy()
o = env.reset()
step_data = get_observation_dict(o, img)
policy.reset()

for _ in range(env.max_path_length):
    a = policy.get_action(step_data)
    o, r, done, info = env.step(a)
    img = env.sim.render(*RESOLUTION, mode="offscreen", camera_name=camera_name)[:, :, ::-1]
    step_data = get_observation_dict(o, img)
    print(a)
    break
```

--------------------------------

### Install HPT Python Package

Source: https://github.com/liruiw/hpt/blob/main/README.md

Installs the Heterogenous Pre-trained Transformers (HPT) project in editable mode. This makes the package available for import in Python environments and allows for direct modifications to the source code.

```bash
pip install -e .
```

--------------------------------

### Initialize HPT Policy and Dataset Configuration

Source: https://github.com/liruiw/hpt/blob/main/quickstart.ipynb

This code block initializes the HPT policy and sets up the dataset configuration. It uses Hydra to compose the configuration, loads a pre-trained HPT policy from Hugging Face, and determines the appropriate device (CPU/CUDA). The policy is then configured for a specific domain (e.g., 'mujoco_metaworld'), and its domain-specific stem and head modules are initialized. Finally, model statistics are printed, and the action normalizer's maximum value is displayed.

```python
from hydra import compose, initialize


policy = Policy.from_pretrained("hf://liruiw/hpt-base")
device = 'cuda' if torch.cuda.is_available() else 'cpu'
domain = "mujoco_metaworld"
with initialize(version_base="1.2", config_path="experiments/configs"):
    cfg = compose(config_name="config", overrides=[f"env={domain}"])

cfg.dataset.episode_cnt = 10 # modify
dataset = hydra.utils.instantiate(
        cfg.dataset, dataset_name=domain, env_rollout_fn=cfg.dataset_generator_func, **cfg.dataset
    )
normalizer = dataset.get_normalizer()

####### set up model
utils.update_network_dim(cfg, dataset, policy)
policy.init_domain_stem(domain, cfg.stem)
policy.init_domain_head(domain, normalizer, cfg.head)
policy.finalize_modules()
policy.print_model_stats()
policy.to(device)
print("policy action normalizer:", policy.normalizer[domain].params_dict["action"]["input_stats"].max)
```

--------------------------------

### Train HPT Policy for One Iteration

Source: https://github.com/liruiw/hpt/blob/main/quickstart.ipynb

This snippet demonstrates a single training iteration for the HPT policy. It creates a `DataLoader` from the dataset, fetches a batch of data, moves the data to the specified device (CPU/CUDA), and then computes the loss using the policy's `compute_loss` method. This represents a fundamental step in the policy's training loop.

```python
train_loader = data.DataLoader(dataset, **cfg.dataloader)
batch = next(iter(train_loader))
batch["data"] = utils.dict_apply(batch["data"], lambda x: x.to(device).float())
output = policy.compute_loss(batch)
```

--------------------------------

### Import Core Libraries for HPT Evaluation

Source: https://github.com/liruiw/hpt/blob/main/quickstart.ipynb

This snippet imports essential Python libraries required for setting up and running the Hierarchical Policy Transformer (HPT) evaluation. It includes standard libraries like `os`, `numpy`, `torch`, and specialized libraries such as `hydra` for configuration management and custom `hpt` utilities for model and data handling. `matplotlib` is also imported for potential plotting.

```python
import os
import numpy as np
import hydra
import torch
from hpt.utils import utils, model_utils
from omegaconf import OmegaConf
from hpt.models.policy import Policy
import os
from torch.utils import data
from torch.utils.data import DataLoader, RandomSampler


%matplotlib inline
```

--------------------------------

### Measure Policy Inference Frames Per Second (FPS)

Source: https://github.com/liruiw/hpt/blob/main/quickstart.ipynb

This snippet defines a utility class `FPS` to calculate the frames per second (inference speed) of an operation. It then instantiates this utility and uses it to measure how quickly the `policy.get_action` method can compute actions. The measurement is performed over 50 iterations, and the final calculated FPS is printed, providing an indication of the policy's real-time performance.

```python
import time
import collections
class FPS:
    def __init__(self, avarageof=50):
        self.frametimestamps = collections.deque(maxlen=avarageof)

    def __call__(self):
        self.frametimestamps.append(time.time())
        if len(self.frametimestamps) > 1:
            return len(self.frametimestamps) / (self.frametimestamps[-1] - self.frametimestamps[0])
        else:
            return 0.0
        
fps_measure = FPS()
for _ in range(50):
    output = policy.get_action(step_data)
    fps_measure()
print(f"FPS: {fps_measure():.3f}")
```

--------------------------------

### Run Metaworld 20-Task Finetuning Script

Source: https://github.com/liruiw/hpt/blob/main/README.md

Executes a bash script for finetuning HPT on 20 Metaworld tasks. This command specifies loading a pre-trained HPT-base model directly from a Hugging Face repository, showcasing how to leverage existing checkpoints for further training.

```bash
bash experiments/scripts/metaworld/train_test_metaworld_20task_finetune.sh hf://liruiw/hpt-base
```

--------------------------------

### Run Metaworld 1-Task Training Script

Source: https://github.com/liruiw/hpt/blob/main/README.md

Executes a specific bash script designed for training and testing HPT on a single Metaworld task. This command demonstrates how to use the provided experiment scripts, including passing arguments for test modes and enabling debugging.

```bash
bash experiments/scripts/metaworld/train_test_metaworld_1task.sh test test 1 +mode=debug
```

--------------------------------

### Train Policies with HPT

Source: https://github.com/liruiw/hpt/blob/main/README.md

Executes the main HPT training script to train policies on various environments. The command uses Python's module execution feature and includes an optional flag to enable debugging mode, which can be useful for development and troubleshooting.

```bash
python -m hpt.run
```

--------------------------------

### HPT Project Directory Structure

Source: https://github.com/liruiw/hpt/blob/main/README.md

This snippet illustrates the hierarchical file and directory organization of the HPT project. It details the purpose of main folders such as 'data' for cached datasets, 'output' for trained models, 'env' for environment wrappers, 'hpt' for core source code (including models, datasets, and run scripts), and 'experiments' for training configurations.

```angular2html
├── ...
├── HPT
|   ├── data            # cached datasets
|   ├── output          # trained models and figures
|   ├── env             # environment wrappers
|   ├── hpt             # model training and dataset source code
|   |   ├── models      # network models
|   |   ├── datasets    # dataset related
|   |   ├── run         # transfer learning main loop
|   |   ├── run_eval    # evaluation main loop
|   |   └── ...
|   ├── experiments     # training configs
|   |   ├── configs     # modular configs
└── ...
```

--------------------------------

### BibTeX Citation for HPT Research

Source: https://github.com/liruiw/hpt/blob/main/README.md

This snippet provides the BibTeX entry for citing the research paper associated with the HPT project. It includes author names, title, publication venue (Neurips), and year, suitable for inclusion in academic bibliographies.

```BibTeX
@inproceedings{wang2024hpt,
author    = {Lirui Wang, Xinlei Chen, Jialiang Zhao, Kaiming He},
title     = {Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformers},
booktitle = {Neurips},
year      = {2024}
}
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.