MimicGen

https://mimicgen.github.io/docs/
Admin
A data generation system for scalable robot learning using human demonstrations, featuring...
Tokens:49,163
Snippets:748
Trust Score:9.7
Update:4 months ago
Show doc for...
Tokens:
Raw
### Download Source Human Dataset

Source: https://mimicgen.github.io/docs/sources/tutorials/getting_started.md.txt

Download the source demonstrations for the Square task.

```shell
$ python mimicgen/scripts/download_datasets.py --dataset_type source --tasks square
```

--------------------------------

### Navigate to MimicGen Repository

Source: https://mimicgen.github.io/docs/tutorials/getting_started.html

Change directory to the base path of the MimicGen repository.

```bash
$ cd {/path/to/mimicgen}

```

--------------------------------

### Prepare Source Dataset for MimicGen

Source: https://mimicgen.github.io/docs/sources/tutorials/getting_started.md.txt

Add necessary information to the hdf5 file to make it compatible with MimicGen.

```shell
$ python mimicgen/scripts/prepare_src_dataset.py \
--dataset datasets/source/square.hdf5 \
--env_interface MG_Square \
--env_interface_type robosuite
```

--------------------------------

### Install documentation dependencies

Source: https://mimicgen.github.io/docs/introduction/installation.html

Installs additional requirements for building documentation locally.

```bash
$ pip install -r requirements-docs.txt
```

--------------------------------

### Generate Dataset Script

Source: https://mimicgen.github.io/docs/sources/tutorials/getting_started.md.txt

This script is used to launch MimicGen data generation.

```bash
scripts/generate_dataset.py
```

--------------------------------

### Install MimicGen from source

Source: https://mimicgen.github.io/docs/introduction/installation.html

Cloning the repository and installing MimicGen using pip.

```bash
cd <PATH_TO_YOUR_INSTALL_DIRECTORY>
git clone https://github.com/NVlabs/mimicgen.git
cd mimicgen
pip install -e .
```

--------------------------------

### Prepare Source Dataset Script

Source: https://mimicgen.github.io/docs/sources/tutorials/getting_started.md.txt

This script is used to prepare HDF5 source demonstrations for MimicGen compatibility.

```bash
scripts/prepare_src_dataset.py
```

--------------------------------

### Install robosuite_task_zoo

Source: https://mimicgen.github.io/docs/introduction/installation.html

Cloning the robosuite-task-zoo repository, checking out a specific commit, and installing it. Note: dependencies in setup.py should be removed before installation.

```bash
$ cd <PATH_TO_YOUR_INSTALL_DIRECTORY>
$ git clone https://github.com/ARISE-Initiative/robosuite-task-zoo
$ cd robosuite-task-zoo
$ git checkout 74eab7f88214c21ca1ae8617c2b2f8d19718a9ed
# NOTE: should remove dependencies in setup.py in the "install_requires" before the last step
$ pip install -e .
```

--------------------------------

### Install robosuite

Source: https://mimicgen.github.io/docs/introduction/installation.html

Cloning the robosuite repository, checking out a specific commit, and installing it.

```bash
$ cd <PATH_TO_YOUR_INSTALL_DIRECTORY>
$ git clone https://github.com/ARISE-Initiative/robosuite.git
$ cd robosuite
$ git checkout b9d8d3de5e3dfd1724f4a0e6555246c460407daa
$ pip install -e .
```

--------------------------------

### Install robomimic

Source: https://mimicgen.github.io/docs/introduction/installation.html

Cloning the robomimic repository, checking out a specific commit, and installing it.

```bash
$ cd <PATH_TO_YOUR_INSTALL_DIRECTORY>
$ git clone https://github.com/ARISE-Initiative/robomimic.git
$ cd robomimic
$ git checkout d0b37cf214bd24fb590d182edb6384333f67b661
$ pip install -e .
```

--------------------------------

### Install pygame

Source: https://mimicgen.github.io/docs/introduction/installation.html

Command to install pygame if using the subtask annotation interface.

```bash
pip install pygame
```

--------------------------------

### Annotate Subtasks Script

Source: https://mimicgen.github.io/docs/sources/tutorials/getting_started.md.txt

This script allows for manual segmentation of source demonstrations into subtasks.

```bash
scripts/annotate_subtasks.py
```

--------------------------------

### Create and activate conda environment

Source: https://mimicgen.github.io/docs/introduction/installation.html

Recommended step to install MimicGen into a new conda environment.

```bash
conda create -n mimicgen python=3.8
conda activate mimicgen
```

--------------------------------

### Download Datasets Script Examples

Source: https://mimicgen.github.io/docs/datasets/mimicgen_corl_2023.html

Examples of how to use the download_datasets.py script to download MimicGen datasets.

```bash
python download_datasets.py

python download_datasets.py --dataset_type core --tasks square_d0 square_d1 square_d2 coffee_d0 coffee_d1 coffee_d2

python download_datasets.py --dataset_type core --tasks all --dry_run

python download_datasets.py --dataset_type source --tasks all

```

--------------------------------

### Generate Data Generation Configs

Source: https://mimicgen.github.io/docs/tutorials/getting_started.html

Run a script to produce experiment configurations consistent with the MimicGen paper. Ensure NUM_TRAJ is set to 10 and GUARANTEE to False for a quick run.

```bash
$ python mimicgen/scripts/generate_core_configs.py

```

--------------------------------

### Run Data Generation

Source: https://mimicgen.github.io/docs/sources/tutorials/getting_started.md.txt

Command to execute the data generation script for the Square D1 task.

```shell
$ python mimicgen/scripts/generate_dataset.py \
--config /tmp/core_configs/demo_src_square_task_D1.json \
--auto-remove-exp
```

--------------------------------

### Example usage of prepare_src_dataset

Source: https://mimicgen.github.io/docs/api/mimicgen.scripts.html

Examples demonstrating how to use the prepare_src_dataset script to process source datasets for data generation.

```bash
python prepare_src_dataset.py –dataset /path/to/stack.hdf5 –env_interface MG_Stack –env_interface_type robosuite
```

```bash
python prepare_src_dataset.py –dataset /path/to/square.hdf5 –env_interface MG_Square –env_interface_type robosuite –n 10 –output /tmp/square_new.hdf5
```

--------------------------------

### Example usage of visualize_subtasks

Source: https://mimicgen.github.io/docs/api/mimicgen.scripts.html

Examples demonstrating how to use the visualize_subtasks script to visualize subtasks in a source demonstration.

```bash
python visualize_subtasks.py –dataset /path/to/demo.hdf5 –config /path/to/config.json –render
```

```bash
python visualize_subtasks.py –dataset /path/to/demo.hdf5 –config /path/to/config.json –video_path /path/to/video.mp4
```

--------------------------------

### Generate Training Configs Script Example

Source: https://mimicgen.github.io/docs/datasets/mimicgen_corl_2023.html

Example of how to use the generate_training_configs_for_public_datasets.py script to generate training configuration files.

```bash
python generate_training_configs_for_public_datasets.py

```

--------------------------------

### Test random action script

Source: https://mimicgen.github.io/docs/introduction/installation.html

Script to test installation by trying random actions in a robosuite task.

```bash
$ cd mimicgen/scripts
$ python demo_random_action.py
```

--------------------------------

### Install cmake for egl_probe

Source: https://mimicgen.github.io/docs/sources/miscellaneous/troubleshooting.md.txt

If encountering issues building the wheel for egl_probe, ensure cmake is installed.

```bash
pip install cmake
```

--------------------------------

### Downgrade MuJoCo

Source: https://mimicgen.github.io/docs/introduction/installation.html

Command to install a specific version of MuJoCo, recommended for robosuite.

```bash
$ pip install mujoco==2.3.2
```

--------------------------------

### download_file_from_hf Example

Source: https://mimicgen.github.io/docs/api/mimicgen.utils.html

Downloads a file from Hugging Face.

```python
repo_id = "amandlek/mimicgen_datasets"
filename = "core/coffee_d0.hdf5"
download_dir = "/tmp"
download_file_from_hf(repo_id, filename, download_dir, check_overwrite=True)
```

--------------------------------

### run normal data generation

Source: https://mimicgen.github.io/docs/api/mimicgen.scripts.html

Example of how to run normal data generation using the generate_dataset.py script.

```bash
python generate_dataset.py --config /path/to/config.json
```

--------------------------------

### Example get_robot_eef_pose implementation for robosuite

Source: https://mimicgen.github.io/docs/tutorials/datagen_custom.html

An example implementation of the get_robot_eef_pose method for the robosuite simulation framework.

```python
    def get_robot_eef_pose(self):
        """
        Get current robot end effector pose. Should be the same frame as used by the robot end-effector controller.

        Returns:
            pose (np.array): 4x4 eef pose matrix
        """

        # OSC control frame is a MuJoCo site - just retrieve its current pose
        return self.get_object_pose(
            obj_name=self.env.robots[0].controller.eef_name, 
            obj_type="site",
        )

```

--------------------------------

### download_url_from_gdrive Example

Source: https://mimicgen.github.io/docs/api/mimicgen.utils.html

Downloads a file at a URL from Google Drive.

```python
url = https://drive.google.com/file/d/1DABdqnBri6-l9UitjQV53uOq_84Dx7Xt/view?usp=drive_link
download_dir = "/tmp"
download_url_from_gdrive(url, download_dir, check_overwrite=True)
```

--------------------------------

### Command-line visualization of subtasks

Source: https://mimicgen.github.io/docs/api/mimicgen.scripts.html

Example command to visualize subtasks, specifying dataset, signals, offsets, and rendering options.

```bash
python visualize_subtasks.py --dataset /path/to/demo.hdf5 --signals grasp_1 insert_1 grasp_2 --offsets 10 10 10 --render
```

--------------------------------

### run a quick debug run

Source: https://mimicgen.github.io/docs/api/mimicgen.scripts.html

Example of how to run a quick debug run using the generate_dataset.py script.

```bash
python generate_dataset.py --config /path/to/config.json --debug
```

--------------------------------

### render all data generation attempts on-screen

Source: https://mimicgen.github.io/docs/api/mimicgen.scripts.html

Example of how to render all data generation attempts on-screen using the generate_dataset.py script.

```bash
python generate_dataset.py --config /path/to/config.json --render
```

--------------------------------

### render all data generation attempts to a video

Source: https://mimicgen.github.io/docs/api/mimicgen.scripts.html

Example of how to render all data generation attempts to a video using the generate_dataset.py script.

```bash
python generate_dataset.py --config /path/to/config.json --video_path /path/to/video.mp4
```

--------------------------------

### Scale up dataset images during rendering

Source: https://mimicgen.github.io/docs/api/mimicgen.scripts.html

This example demonstrates how to scale up the dataset images by a factor of 10 when rendering them to the screen during annotation.

```python
python annotate_subtasks.py --dataset /path/to/demo.hdf5 --signals grasp_1 insert_1 grasp_2 --render_image_names agentview_image robot0_eye_in_hand_image --image_scale 10
```

--------------------------------

### Build and view documentation locally

Source: https://mimicgen.github.io/docs/introduction/installation.html

Commands to clean, generate API docs, build HTML documentation, copy images, and view it in a browser.

```bash
$ cd <PATH_TO_MIMICGEN_INSTALL_DIR>/docs
$ make clean
$ make apidoc
$ make html
$ cp -r images _build/html/
```

--------------------------------

### Limit annotation to specific demonstrations (2 and 3)

Source: https://mimicgen.github.io/docs/api/mimicgen.scripts.html

This example shows how to specify a range of demonstrations to annotate, starting from the second demonstration (index 1).

```python
python annotate_subtasks.py --dataset /path/to/demo.hdf5 --signals grasp_1 insert_1 grasp_2 --render_image_names agentview_image robot0_eye_in_hand_image --n 2 --start 1
```

--------------------------------

### Example Subtask Segmentation

Source: https://mimicgen.github.io/docs/sources/tutorials/subtask_termination_signals.md.txt

Illustrates how subtask termination signals are used to segment a source demonstration into subtask segments, showing the start and end indices for each segment based on signal transitions.

```text
1. (signal: grasp_1) grasping cubeA (motion relative to cubeA)
2. (signal: place_1) placing cubeA on cubeB (motion relative to cubeB)
3. (signal: grasp_2) grasping cubeC (motion relative to cubeC)
4. (signal: None) placing cubeC on cubeA (motion relative to cubeA)
```

```text
1. (signal: grasp_1) [0, 50]
2. (signal: place_1) [50, 94]
3. (signal: grasp_2) [94, 148]
4. (signal: None) [148, 210]
```

--------------------------------

### Download source demonstrations

Source: https://mimicgen.github.io/docs/sources/tutorials/reproducing_experiments.md.txt

Command to download all source demonstrations. The `--download_dir` argument can be used to specify a download path.

```shell
$ python mimicgen/scripts/download_datasets.py --dataset_type source --tasks all
```

--------------------------------

### Run Data Generation

Source: https://mimicgen.github.io/docs/tutorials/getting_started.html

Execute the data generation process using a specific configuration file. The --auto-remove-exp flag is used to automatically remove the experiment directory upon completion.

```bash
$ python mimicgen/scripts/generate_dataset.py \
--config /tmp/core_configs/demo_src_square_task_D1.json \
--auto-remove-exp

```

--------------------------------

### Prepare a specific source dataset

Source: https://mimicgen.github.io/docs/sources/tutorials/reproducing_experiments.md.txt

Example command to prepare the 'Coffee' source dataset for data generation.

```shell
$ python mimicgen/scripts/prepare_src_dataset.py --dataset datasets/source/coffee.hdf5 --env_interface MG_Coffee --env_interface_type robosuite
```

--------------------------------

### Example Environment Interface Implementation

Source: https://mimicgen.github.io/docs/tutorials/datagen_custom.html

This Python code snippet demonstrates a basic implementation of the environment interface, focusing on returning signals for task progression.

```python
# final subtask is placing cubeC on cubeA (motion relative to cubeA) - but final subtask signal is not needed
return signals
```

--------------------------------

### Install ffmpeg

Source: https://mimicgen.github.io/docs/sources/miscellaneous/troubleshooting.md.txt

A common error during installation is the inability to find the ffmpeg executable. Installing ffmpeg via conda can resolve this.

```bash
conda install ffmpeg
```

--------------------------------

### Navigate to the base repository path

Source: https://mimicgen.github.io/docs/sources/tutorials/reproducing_experiments.md.txt

Ensures the user is in the correct directory before proceeding with the steps.

```shell
$ cd {/path/to/mimicgen}
```

--------------------------------

### Generate policy training configurations (robot transfer)

Source: https://mimicgen.github.io/docs/tutorials/reproducing_experiments.html

Scripts to generate configurations for policy training on robot transfer datasets.

```bash
$ scripts/generate_robot_transfer_training_configs.py
```
Tokens:
Raw