### Installing rl_games using pip

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/framework.md

This command installs the rl_games library using pip, which is a prerequisite for running the RL training examples.

```bash
pip install rl-games
```

--------------------------------

### Running PBT Experiment (Command Line)

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/pbt.md

This command line example demonstrates how to start a training session in a PBT experiment using Isaac Gym Environments. It sets various parameters such as the maximum frames, headless mode, PBT configuration, task, and policy index. Note that `pbt.policy_idx` needs to be unique for each agent in the PBT experiment.

```bash
python -m isaacgymenvs.train seed=-1 train.params.config.max_frames=10000000000 headless=True pbt=pbt_default pbt.workspace=workspace_allegro_kuka pbt.interval_steps=20000000 
pbt.start_after=100000000 pbt.initial_delay=200000000 pbt.replace_fraction_worst=0.3 pbt/mutation=allegro_kuka_mutation task=AllegroKukaLSTM task/env=reorientation pbt.num_policies=8 pbt.policy_idx=0
```

--------------------------------

### Installing the latest rl_games version from GitHub

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/framework.md

These commands clone the rl_games repository from GitHub and install it in editable mode, allowing you to use the latest, unreleased version.

```bash
git clone https://github.com/Denys88/rl_games.git
pip install -e .
```

--------------------------------

### Installing the Isaac Gym Environment

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/README.md

This command installs the Isaac Gym environment from the current directory using pip. The -e flag installs it in editable mode, allowing changes to the source code to be reflected immediately.

```bash
pip install -e .
```

--------------------------------

### Citation for Factory Simulation Methods

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/rl_examples.md

This BibTeX entry provides the citation format for the Factory simulation methods, including SDF collisions and contact reduction, as well as the learning tools used in the Isaac Gym Envs examples.

```BibTeX
@inproceedings{
	narang2022factory,
	author = {Yashraj Narang and Kier Storey and Iretiayo Akinola and Miles Macklin and Philipp Reist and Lukasz Wawrzyniak and Yunrong Guo and Adam Moravanszky and Gavriel State and Michelle Lu and Ankur Handa and Dieter Fox},
	title = {Factory: Fast contact for robotic assembly},
	booktitle = {Robotics: Science and Systems},
	year = {2022}
}
```

--------------------------------

### SAC Training Examples for Ant and Humanoid

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/release_notes.md

These commands demonstrate how to run SAC training examples for the Ant and Humanoid environments using the train.py script. They specify the task and training configuration.

```Python
python train.py task=AntSAC train=AntSAC
```

```Python
python train.py task=HumanoidSAC train=HumanoidSAC
```

--------------------------------

### IndustReal Citation (BibTeX)

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/rl_examples.md

This BibTeX entry provides the citation information for the IndustReal paper, which describes the transfer of contact-rich assembly tasks from simulation to reality.

```bibtex
@inproceedings{
  tang2023industreal,
  author = {Bingjie Tang and Michael A Lin and Iretiayo Akinola and Ankur Handa and Gaurav S Sukhatme and Fabio Ramos and Dieter Fox and Yashraj Narang},
  title = {IndustReal: Transferring contact-rich assembly tasks from simulation to reality},
  booktitle = {Robotics: Science and Systems},
  year = {2023}
}
```

--------------------------------

### Implementing Simulation Steps in VecTask Subclass

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/framework.md

This code outlines the three essential methods that must be implemented by any class inheriting from VecTask: create_sim, pre_physics_step, and post_physics_step. These methods define the simulation setup, pre-physics actions, and post-physics computations, respectively.

```python
def create_sim(self):
    # implement sim set up and environment creation here
    #    - set up-axis
    #    - call super().create_sim with device args (see docstring)
    #    - create ground plane
    #    - set up environments

def pre_physics_step(self, actions):
    # implement pre-physics simulation code here
    #    - e.g. apply actions

def post_physics_step(self):
    # implement post-physics simulation code here
    #    - e.g. compute reward, compute observations
```

--------------------------------

### CMake Project Setup with Catkin

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/assets/urdf/sektion_cabinet_model/CMakeLists.txt

This snippet initializes the CMake project, finds the catkin package, and declares the catkin package with its dependencies. It sets up the basic structure for a ROS package.

```CMake
cmake_minimum_required(VERSION 2.8.3)
project(sektion_cabinet_model)

find_package(catkin REQUIRED)

catkin_package(
#  INCLUDE_DIRS include
#  LIBRARIES sektion_cabinet_model
#  CATKIN_DEPENDS other_catkin_pkg
#  DEPENDS system_lib
)
```

--------------------------------

### Factory Citation (BibTeX)

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/rl_examples.md

This BibTeX entry provides the citation information for the Factory paper, which describes fast contact methods for robotic assembly.

```bibtex
@inproceedings{
  narang2022factory,
  author = {Yashraj Narang and Kier Storey and Iretiayo Akinola and Miles Macklin and Philipp Reist and Lukasz Wawrzyniak and Yunrong Guo and Adam Moravanszky and Gavriel State and Michelle Lu and Ankur Handa and Dieter Fox},
  title = {Factory: Fast contact for robotic assembly},
  booktitle = {Robotics: Science and Systems},
  year = {2022}
}
```

--------------------------------

### ANYmal Citation Note

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/rl_examples.md

This is a citation for the ANYmal rough terrain environment. It is important to cite this work if you use the ANYmal rough terrain environment in your work.

```BibTeX
@misc{rudin2021learning,
      title={Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning}, 
      author={Nikita Rudin and David Hoeller and Philipp Reist and Marco Hutter},
      year={2021},
      journal = {arXiv preprint arXiv:2109.11978},
}
```

--------------------------------

### Trifinger Citation Note

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/rl_examples.md

This is a citation for the Trifinger environment. It is important to cite this work if you use the Trifinger environment in your work.

```BibTeX
@misc{isaacgym-trifinger,
  title     = {{Transferring Dexterous Manipulation from GPU Simulation to a Remote Real-World TriFinger}},
  author    = {Allshire, Arthur and Mittal, Mayank and Lodaya, Varun and Makoviychuk, Viktor and Makoviichuk, Denys and Widmaier, Felix and Wuthrich, Manuel and Bauer, Stefan and Handa, Ankur and Garg, Animesh},
  year      = {2021},
  journal = {arXiv preprint arXiv:2108.09779},
}
```

--------------------------------

### Running PBT Locally on a Single GPU (Python)

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/pbt.md

This Python command demonstrates how to run a PBT experiment locally on a single GPU using the launcher. It specifies the experiment script, backend (processes), maximum parallel agents, experiments per GPU, and the number of GPUs to use. This setup runs all agents on the single available GPU.

```bash
python -m isaacgymenvs.pbt.launcher.run --run=isaacgymenvs.pbt.experiments.ant_pbt --backend=processes --max_parallel=4 --experiments_per_gpu=4 --num_gpus=1
```

--------------------------------

### Training AllegroHandDextremeADR with Automatic Domain Randomization

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/rl_examples.md

This command trains the AllegroHandDextremeADR environment with automatic domain randomization. It sets parameters such as headless mode, random seed, number of environments, reset time, control frequency, and maximum epochs.

```bash
HYDRA_ADR="train.py multi_gpu=False \
task=AllegroHandDextremeADR \
headless=True seed=-1 \
num_envs=8192 \
task.env.resetTime=8 \
task.env.controlFrequencyInv=2 \
train.params.config.max_epochs=50000"

python ${HYDRA_ADR}
```

--------------------------------

### Training HumanoidAMP with Run Motion

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/rl_examples.md

This command trains the HumanoidAMP task using the run motion. It uses the `train.py` script with the `task=HumanoidAMP` argument, specifies the motion file `amp_humanoid_run.npy` using `++task.env.motion_file`, and sets the experiment name to `AMP_run`.

```Python
python train.py task=HumanoidAMP ++task.env.motion_file=amp_humanoid_run.npy experiment=AMP_run
```

--------------------------------

### Running PBT Locally with Multiple GPUs (Python)

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/pbt.md

This Python command demonstrates how to run a PBT experiment locally on a multi-GPU system using the launcher. It specifies the experiment script, backend (processes), maximum parallel agents, experiments per GPU, and the number of GPUs to use. This setup distributes the agents across the available GPUs for parallel training.

```bash
python -m isaacgymenvs.pbt.launcher.run --run=isaacgymenvs.pbt.experiments.allegro_kuka_reorientation_pbt_lstm --backend=processes --max_parallel=8 --experiments_per_gpu=2 --num_gpus=4
```

--------------------------------

### Ant Task: Resetting Actor and DOF States

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/rl_examples.md

This snippet demonstrates the usage of `set_actor_root_state_tensor_indexed` and `set_dof_state_tensor_indexed` APIs for setting select ants into a valid starting state during environment resets. The Ant task involves training a quadruped robot to run forward as fast as possible. It uses actor root states, DOF states, and force sensors to gather information about the ant's state and interactions with the environment.

```Python
set_actor_root_state_tensor_indexed
```

```Python
set_dof_state_tensor_indexed
```

--------------------------------

### Installing Required Packages - Ubuntu

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/assets/licenses/ycb-LICENSE.txt

This command installs the required Python packages (scipy, numpy, and h5py) in an Ubuntu environment using the apt-get package manager. These packages are likely dependencies for the provided Python scripts.

```bash
sudo apt-get install python-scipy python-numpy python-h5py
```

--------------------------------

### Getting Asset Tendon Properties

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/rl_examples.md

This code snippet demonstrates the use of `get_asset_tendon_properties` to retrieve tendon properties for the Shadow Hand model. Tendons are used to simulate the complex musculoskeletal structure of the hand, and this function allows access to their parameters.

```Python
get_asset_tendon_properties
```

--------------------------------

### Training HumanoidAMP with Dance Motion

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/rl_examples.md

This command trains the HumanoidAMP task using the dance motion. It uses the `train.py` script with the `task=HumanoidAMP` argument, specifies the motion file `amp_humanoid_dance.npy` using `++task.env.motion_file`, and sets the experiment name to `AMP_dance`.

```Python
python train.py task=HumanoidAMP ++task.env.motion_file=amp_humanoid_dance.npy experiment=AMP_dance
```

--------------------------------

### Training HumanoidAMP with Backflip Motion (LowGP)

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/rl_examples.md

This command trains the HumanoidAMP task using the backflip motion and the `HumanoidAMPPPOLowGP` training configuration. It specifies the motion file `amp_humanoid_backflip.npy` using `++task.env.motion_file` and sets the experiment name to `AMP_backflip`. The LowGP config is used to tune the `disc_grad_penalty` hyperparameter.

```Python
python train.py task=HumanoidAMP train=HumanoidAMPPPOLowGP ++task.env.motion_file=amp_humanoid_backflip.npy experiment=AMP_backflip
```

--------------------------------

### Training HumanoidAMP with Hop Motion (LowGP)

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/rl_examples.md

This command trains the HumanoidAMP task using the hop motion and the `HumanoidAMPPPOLowGP` training configuration. It specifies the motion file `amp_humanoid_hop.npy` using `++task.env.motion_file` and sets the experiment name to `AMP_hop`. The LowGP config is used to tune the `disc_grad_penalty` hyperparameter.

```Python
python train.py task=HumanoidAMP train=HumanoidAMPPPOLowGP ++task.env.motion_file=amp_humanoid_hop.npy experiment=AMP_hop
```

--------------------------------

### Setting Asset Tendon Properties

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/rl_examples.md

This code snippet demonstrates the use of `set_asset_tendon_properties` to set tendon properties for the Shadow Hand model. This function allows modification of the tendon parameters, enabling customization of the hand's behavior.

```Python
set_asset_tendon_properties
```

--------------------------------

### Training HumanoidAMP with Walk Motion

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/rl_examples.md

This command trains the HumanoidAMP task using the default walk motion. It utilizes the `train.py` script with the `task=HumanoidAMP` argument and sets the experiment name to `AMP_walk`. No specific motion file is specified, as walk is the default.

```Python
python train.py task=HumanoidAMP experiment=AMP_walk
```

--------------------------------

### Training HumanoidAMPHands with Cartwheel Motion (LowGP)

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/rl_examples.md

This command trains the HumanoidAMPHands task using the cartwheel motion and the `HumanoidAMPPPOLowGP` training configuration. The experiment name is set to `AMP_cartwheel`. The LowGP config is used to tune the `disc_grad_penalty` hyperparameter, and the task requires hands in the contact body list.

```Python
python train.py task=HumanoidAMPHands train=HumanoidAMPPPOLowGP experiment=AMP_cartwheel
```

--------------------------------

### Training AllegroHandDextremeManualDR with Manual Domain Randomization

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/rl_examples.md

This command trains the AllegroHandDextremeManualDR environment with manual domain randomization. It sets various task and training parameters, including reset time, success tolerance, experiment name, headless mode, random seed, object pose, action delta penalty, control frequency, network units, and maximum epochs. It uses a LSTM network.

```bash
HYDRA_MANUAL_DR="train.py multi_gpu=False \
task=AllegroHandDextremeManualDR \
task.env.resetTime=8 task.env.successTolerance=0.4 \
experiment='allegrohand_dextreme_manual_dr' \
headless=True seed=-1 \
task.env.startObjectPoseDY=-0.15 \
task.env.actionDeltaPenaltyScale=-0.2 \
task.env.resetTime=8 \
task.env.controlFrequencyInv=2 \
train.params.network.mlp.units=[512,512] \
train.params.network.rnn.units=768 \
train.params.network.rnn.name=lstm \
train.params.config.central_value_config.network.mlp.units=[1024,512,256] \
train.params.config.max_epochs=50000 \
task.env.apply_random_quat=True"


python ${HYDRA_MANUAL_DR}
```

--------------------------------

### Setting Actor Root State Tensor Indexed

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/rl_examples.md

This code snippet highlights the importance of using `set_actor_root_state_tensor_indexed` to reset both the target object and the object in hand with a single call. This ensures that the states for both objects are set consistently, which is crucial for the task's proper functioning.

```Python
set_actor_root_state_tensor_indexed
```

--------------------------------

### Ant Task: Applying Actions to DOFs

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/rl_examples.md

This snippet describes how actions are applied to the degrees of freedom (DOFs) of the Ant robot to enable movement, utilizing the `set_dof_actuation_force_tensor` API within the Isaac Gym environment. The Ant task involves training a quadruped robot to run forward as fast as possible. It uses actor root states, DOF states, and force sensors to gather information about the ant's state and interactions with the environment.

```Python
set_dof_actuation_force_tensor
```

--------------------------------

### Setting DOF Position Target Tensor

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/rl_examples.md

This code snippet demonstrates how to control the motion of the Shadow Hand using position targets with the `set_dof_position_target_tensor` function. This function is used to set the desired joint positions for the hand's degrees of freedom (DOFs), allowing for precise control of the hand's movements.

```Python
set_dof_position_target_tensor
```

--------------------------------

### Recording videos with gym.wrappers.RecordVideo in Isaac Gym

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/README.md

This snippet demonstrates how to record videos of an agent's gameplay using the `gym.wrappers.RecordVideo` wrapper. It initializes an Isaac Gym environment, applies the wrapper to record videos every 10000 steps, and saves the videos to the `./videos` directory. The video length is set to 100 steps.

```python
import gym
import isaacgym
import isaacgymenvs
import torch

num_envs = 64

envs = isaacgymenvs.make(
	seed=0, 
	task="Ant", 
	num_envs=num_envs, 
	sim_device="cuda:0",
	rl_device="cuda:0",
	graphics_device_id=0,
	headless=False,
	multi_gpu=False,
	virtual_screen_capture=True,
	force_render=False,
)
envs.is_vector_env = True
envs = gym.wrappers.RecordVideo(
	envs,
	"./videos",
	step_trigger=lambda step: step % 10000 == 0, # record the videos every 10000 steps
	video_length=100  # for each video record up to 100 steps
)
envs.reset()
print("the image of Isaac Gym viewer is an array of shape", envs.render(mode="rgb_array").shape)
for _ in range(100):
	actions = 2.0 * torch.rand((num_envs,) + envs.action_space.shape, device = 'cuda:0') - 1.0
	envs.step(actions)
```

--------------------------------

### Loading a Trained Checkpoint and Continuing Training

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/README.md

This command loads a trained checkpoint and continues training from that point. The checkpoint parameter specifies the path to the checkpoint file.

```bash
python train.py task=Ant checkpoint=runs/Ant/nn/Ant.pth
```

--------------------------------

### Loading a Trained Checkpoint for Inference

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/README.md

This command loads a trained checkpoint and performs inference (testing) without further training. The test=True argument enables inference mode, and num_envs=64 reduces the number of environments for faster execution.

```bash
python train.py task=Ant checkpoint=runs/Ant/nn/Ant.pth test=True num_envs=64
```

--------------------------------

### Citing Factory simulation methods

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/README.md

This BibTeX entry should be used to cite the Factory simulation methods or learning tools.

```BibTeX
@inproceedings{
	narang2022factory,
	author = {Yashraj Narang and Kier Storey and Iretiayo Akinola and Miles Macklin and Philipp Reist and Lukasz Wawrzyniak and Yunrong Guo and Adam Moravanszky and Gavriel State and Michelle Lu and Ankur Handa and Dieter Fox},
	title = {Factory: Fast contact for robotic assembly},
	booktitle = {Robotics: Science and Systems},
	year = {2022}
}
```

--------------------------------

### Training a Policy from Scratch

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/industreal.md

Trains a policy for the specified task (IndustRealTaskPegsInsert or IndustRealTaskGearsInsert) from scratch.  This command initiates the training process without loading any pre-existing checkpoints.

```Python
python train.py task=[task name]
```

--------------------------------

### Citing IndustReal training environments

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/README.md

This BibTeX entry should be used to cite the IndustReal training environments or algorithms.

```BibTeX
@inproceedings{
	tang2023industreal,
	author = {Bingjie Tang and Michael A Lin and Iretiayo Akinola and Ankur Handa and Gaurav S Sukhatme and Fabio Ramos and Dieter Fox and Yashraj Narang},
	title = {IndustReal: Transferring contact-rich assembly tasks from simulation to reality},
	booktitle = {Robotics: Science and Systems},
	year = {2023}
}
```

--------------------------------

### Multi-GPU Training with Torchrun

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/README.md

This command demonstrates how to run multi-GPU training using `torchrun` with the `train.py` script. The `--nproc_per_node` flag specifies the number of processes to run, and the `multi_gpu=True` flag enables multi-GPU training.

```bash
torchrun --standalone --nnodes=1 --nproc_per_node=2 train.py multi_gpu=True task=Ant <OTHER_ARGS>
```

--------------------------------

### Citing AMP environment

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/README.md

This BibTeX entry should be used to cite the AMP: Adversarial Motion Priors environment.

```BibTeX
@article{
	2021-TOG-AMP,
	author = {Peng, Xue Bin and Ma, Ze and Abbeel, Pieter and Levine, Sergey and Kanazawa, Angjoo},
	title = {AMP: Adversarial Motion Priors for Stylized Physics-Based Character Control},
	journal = {ACM Trans. Graph.},
	issue_date = {August 2021},
	volume = {40},
	number = {4},
	month = jul,
	year = {2021},
	articleno = {1},
	numpages = {15},
	url = {http://doi.acm.org/10.1145/3450626.3459670},
	doi = {10.1145/3450626.3459670},
	publisher = {ACM},
	address = {New York, NY, USA},
	keywords = {motion control, physics-based character animation, reinforcement learning},
}
```

--------------------------------

### BibTeX entry for citing Factory

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/factory.md

This BibTeX entry should be used to cite the Factory simulation methods or learning tools in academic publications. It includes the authors, title, booktitle, and year of publication.

```BibTeX
@inproceedings{
	narang2022factory,
	author = {Yashraj Narang and Kier Storey and Iretiayo Akinola and Miles Macklin and Philipp Reist and Lukasz Wawrzyniak and Yunrong Guo and Adam Moravanszky and Gavriel State and Michelle Lu and Ankur Handa and Dieter Fox},
	title = {Factory: Fast contact for robotic assembly},
	booktitle = {Robotics: Science and Systems},
	year = {2022}
}
```

--------------------------------

### Citing DeXtreme

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/README.md

This BibTeX entry should be used to cite the DeXtreme work related to Automatic Domain Randomisation.

```BibTeX
@inproceedings{
	handa2023dextreme,
	author = {Ankur Handa, Arthur Allshire, Viktor Makoviychuk, Aleksei Petrenko, Ritvik Singh, Jingzhou Liu, Denys Makoviichuk, Karl Van Wyk, Alexander Zhurkevich, Balakumar Sundaralingam, Yashraj Narang, Jean-Francois Lafleche, Dieter Fox, Gavriel State},
	title = {DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Reality},
	booktitle = {ICRA},
	year = {2023}
}
```

--------------------------------

### Capturing videos and uploading to WandB during training

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/README.md

This command demonstrates how to automatically capture videos during training and upload them to Weights and Biases (WandB). It sets the `wandb_activate` flag to `True` and specifies the WandB entity and project. The `capture_video` flag is also set to `True` to enable video capture.

```bash
python train.py task=Ant wandb_activate=True wandb_entity=nvidia wandb_project=rl_games capture_video=True force_render=False
```

--------------------------------

### Citing DexPBT

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/README.md

This BibTeX entry should be used to cite the DexPBT work related to Population Based Training.

```BibTeX
@inproceedings{
	petrenko2023dexpbt,
	author = {Aleksei Petrenko, Arthur Allshire, Gavriel State, Ankur Handa, Viktor Makoviychuk},
	title = {DexPBT: Scaling up Dexterous Manipulation for Hand-Arm Systems with Population Based Training},
	booktitle = {RSS},
	year = {2023}
}
```

--------------------------------

### Citing Trifinger environment

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/README.md

This BibTeX entry should be used to cite the Trifinger environment.

```BibTeX
@misc{isaacgym-trifinger,
  title     = {{Transferring Dexterous Manipulation from GPU Simulation to a Remote Real-World TriFinger}},
  author    = {Allshire, Arthur and Mittal, Mayank and Lodaya, Varun and Makoviychuk, Viktor and Makoviichuk, Denys and Widmaier, Felix and Wuthrich, Manuel and Bauer, Stefan and Handa, Ankur and Garg, Animesh},
  year      = {2021},
  journal = {arXiv preprint arXiv:2108.09779}
}
```

--------------------------------

### Initializing VecTask in MyNewTask

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/framework.md

This code snippet demonstrates how to initialize the VecTask class within a custom task's __init__ method. It passes the configuration dictionary to the superclass constructor, setting up the simulation environment.

```python
super().__init__(
    cfg=config_dict
)
```

--------------------------------

### Citing ANYmal rough terrain environment

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/README.md

This BibTeX entry should be used to cite the ANYmal rough terrain environment.

```BibTeX
@misc{rudin2021learning,
      title={Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning}, 
      author={Nikita Rudin and David Hoeller and Philipp Reist and Marco Hutter},
      year={2021},
      journal = {arXiv preprint arXiv:2109.11978}
}
```

--------------------------------

### Loading Checkpoints with ADR in Isaac Gym

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/dextreme.md

This command loads a checkpoint for the AllegroHandDextreme task using ADR. It sets the number of environments, specifies the checkpoint path, enables testing mode, activates ADR loading from the checkpoint, enables printing the number of successes, and runs the environment in headless mode. The ADR parameters are loaded from the checkpoint.

```Python
python train.py task=AllegroHandDextremeADR \
num_envs=2048 checkpoint=<your_checkpoint_path> \
test=True \
task.task.adr.adr_load_from_checkpoint=True \
task.env.printNumSuccesses=True \
headless=True
```

--------------------------------

### Importing Isaac Gym APIs

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/framework.md

These import statements are necessary to use Isaac Gym's APIs for creating RL tasks, including gymtorch, gymapi, and the VecTask base class.

```python
from isaacgym import gymtorch
from isaacgym import gymapi

from .base.vec_task import VecTask
```

--------------------------------

### Citing Isaac Gym

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/README.md

This BibTeX entry should be used to cite the Isaac Gym project in academic publications.

```BibTeX
@misc{makoviychuk2021isaac,
      title={Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning}, 
      author={Viktor Makoviychuk and Lukasz Wawrzyniak and Yunrong Guo and Michelle Lu and Kier Storey and Miles Macklin and David Hoeller and Nikita Rudin and Arthur Allshire and Ankur Handa and Gavriel State},
      year={2021},
      journal={arXiv preprint arXiv:2108.10470}
}
```

--------------------------------

### Creating a Vectorized Environment with Isaac Gym

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/README.md

This Python code creates a vectorized environment using the Isaac Gym Envs API. It initializes the environment with specified parameters such as the task (Ant), number of environments, and simulation/RL devices.

```python
import isaacgym
import isaacgymenvs
import torch

num_envs = 2000

envs = isaacgymenvs.make(
	seed=0, 
	task="Ant", 
	num_envs=num_envs, 
	sim_device="cuda:0",
	rl_device="cuda:0",
)
print("Observation space is", envs.observation_space)
print("Action space is", envs.action_space)
obs = envs.reset()
for _ in range(20):
	random_actions = 2.0 * torch.rand((num_envs,) + envs.action_space.shape, device = 'cuda:0') - 1.0
	envs.step(random_actions)
```

--------------------------------

### Integrating a New Task into the Isaac Gym Task Map

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/framework.md

This snippet demonstrates how to integrate a newly created task into the Isaac Gym environment by adding it to the isaac_gym_task_map dictionary in the tasks/__init__.py file. This allows the task to be launched from train.py.

```python
from isaacgymenvs.tasks.my_new_task import MyNewTask
...
isaac_gym_task_map = {
    'Anymal': Anymal,
    # ...
    'MyNewTask': MyNewTask,
}
```

--------------------------------

### Capturing videos during training with train.py

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/README.md

This command line instruction shows how to capture videos during training by setting the `capture_video` flag to `True`. It also configures the capture frequency (`capture_video_freq`) and video length (`capture_video_len`). The `force_render` flag is set to `False` to disable rendering when videos are not being captured.

```bash
python train.py capture_video=True capture_video_freq=1500 capture_video_len=100 force_render=False
```

--------------------------------

### Logging Isaac Gym Code to WandB

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/dextreme.md

This command logs the entire isaacgymenvs code used to train in the WandB dashboard. This is useful for reproducibility as you make changes to your code.

```Python
wandb_logcode_dir=<isaac_gym_dir>
```

--------------------------------

### Training Headlessly for Ant Locomotion Task

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/README.md

This command trains a policy for the Ant locomotion task in headless mode (without a graphical display). The headless=True argument disables the viewer, allowing for faster training.

```bash
python train.py task=Ant headless=True
```

--------------------------------

### Acquiring and Wrapping DOF State Tensor

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/framework.md

This snippet shows how to acquire the degree-of-freedom (DOF) state tensor from the simulation and wrap it using gymtorch. This allows for easy access and manipulation of the DOF states within the task.

```python
dof_state_tensor = self.gym.acquire_dof_state_tensor(self.sim)
self.dof_state = gymtorch.wrap_tensor(dof_state_tensor)
```

--------------------------------

### Training a Policy for Ant Locomotion Task

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/README.md

This command trains a policy for the Ant locomotion task using the train.py script. The task parameter specifies the environment to be used for training.

```bash
python train.py task=Ant
```

--------------------------------

### Loading Checkpoints with ManualDR in Isaac Gym

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/dextreme.md

This command loads a checkpoint for the AllegroHandDextreme task using ManualDR. It sets various environment parameters, network configurations, enables random network adversary, specifies the checkpoint path, and activates testing mode. It also applies random quaternions and disables printing the number of successes.

```Python
python train.py task=AllegroHandDextremeManualDR \
num_envs=32 task.env.startObjectPoseDY=-0.15 \
task.env.actionDeltaPenaltyScale=-0.2 \
task.env.controlFrequencyInv=2 train.params.network.mlp.units=[512,512] \
train.params.network.rnn.units=768 \
train.params.network.rnn.name=lstm \
train.params.config.central_value_config.network.mlp.units=[1024,512,256] \
task.env.random_network_adversary.enable=True checkpoint=<ckpt_path> \
test=True task.env.apply_random_quat=True task.env.printNumSuccesses=False
```

--------------------------------

### Capture Agent Gameplay Video

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/release_notes.md

This command shows how to capture the agent's gameplay video using the train.py script. It creates a 'videos' folder to store the captured video.

```Python
python train.py capture_video=True
```

--------------------------------

### Training DeXtreme RL Policies with Manual DR

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/dextreme.md

This command trains DeXtreme RL policies with Manual Domain Randomization (DR) using a single GPU. It sets various environment parameters such as reset time, success tolerance, and action delta penalty scale. It also configures the network architecture, including MLP and RNN units, and specifies the maximum number of epochs.

```Python
HYDRA_MANUAL_DR="train.py multi_gpu=False \
task=AllegroHandDextremeManualDR \
task.env.resetTime=8 task.env.successTolerance=0.4 \
experiment='allegrohand_dextreme_manual_dr' \
headless=True seed=-1 \
task.env.startObjectPoseDY=-0.15 \
task.env.actionDeltaPenaltyScale=-0.2 \
task.env.resetTime=8 \
task.env.controlFrequencyInv=2 \
train.params.network.mlp.units=[512,512] \
train.params.network.rnn.units=768 \
train.params.network.rnn.name=lstm \
train.params.config.central_value_config.network.mlp.units=[1024,512,256] \
train.params.config.max_epochs=50000 \
task.env.apply_random_quat=True"


python ${HYDRA_MANUAL_DR}
```

--------------------------------

### Resuming Policy Training from Checkpoint

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/industreal.md

Resumes policy training for the specified task from a specific checkpoint file. This allows you to continue training from a previously saved state.

```Python
python train.py task=[task name] checkpoint=[path to checkpoint]
```

--------------------------------

### Manual Domain Randomization Configuration

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/dextreme.md

Shows the structure for defining randomization parameters in the AllegroHandDextremeManualDR.yaml file. The randomize flag enables or disables randomization, and randomization_params contains the specific parameters to be randomized.

```YAML
task:
  randomize: True
  randomization_params:
    ....
```

--------------------------------

### ADR Training on Multi-GPU using Torchrun

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/dextreme.md

This command launches an ADR training job across multiple GPUs using torchrun. It configures the distributed training environment with the number of nodes, processes per node, and the master address. The ${GPUS} variable represents the number of GPUs utilized.

```bash
torchrun --nnodes=1 --nproc_per_node=${GPUS} --master_addr '127.0.0.1' ${HYDRA_ADR}
```

--------------------------------

### Creating a new Task class

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/framework.md

This code snippet demonstrates how to create a new task class that extends from the VecTask base class, which is required for implementing RL tasks in Isaac Gym.

```python
class MyNewTask(VecTask):
```

--------------------------------

### Manual DR Training on Multi-GPU using Torchrun

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/dextreme.md

This command launches a Manual DR training job across multiple GPUs using torchrun. It sets the number of nodes, processes per node, and the master address for distributed training. The ${GPUS} variable should be set to the number of GPUs being used.

```bash
torchrun --nnodes=1 --nproc_per_node=${GPUS} --master_addr '127.0.0.1' ${HYDRA_MANUAL_DR}
```

--------------------------------

### Training a Policy for Cartpole Task

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/README.md

This command trains a policy for the Cartpole task using the train.py script. The task parameter specifies the environment to be used for training.

```bash
python train.py task=Cartpole
```

--------------------------------

### Include Directories Configuration

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/assets/urdf/sektion_cabinet_model/CMakeLists.txt

This snippet configures the include directories for the project. It specifies where the compiler should look for header files during the build process.  It can be extended to include catkin include directories.

```CMake
include_directories(
# include
# ${catkin_INCLUDE_DIRS}
)
```

--------------------------------

### Policy Input Configuration (ManualDR)

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/dextreme.md

Defines the input observations for the policy network in the AllegroHandDextremeManualDR environment. These inputs are randomized versions of the robot's joint positions, object pose relative to the camera, goal pose, goal relative rotation relative to the camera, and last actions.

```YAML
inputs:

  dof_pos_randomized: { }
  object_pose_cam_randomized: { }
  goal_pose_randomized: { }
  goal_relative_rot_cam_randomized: { }
  last_actions_randomized: { }
```

--------------------------------

### Automatic Domain Randomization Configuration

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/dextreme.md

Illustrates the configuration structure for ADR in AllegroHandDextremeADR.yaml. It includes flags to enable ADR and update ADR ranges, along with parameters defining the range path, initial range, limits, delta, and delta style for each randomized parameter.

```YAML
adr:

  use_adr: True

  # set to false to not do update ADR ranges. useful for evaluation or training a base policy
  update_adr_ranges: True 
  ...


  # raw ADR params. more are added by affine transforms code
  params:
    ### Hand Properties
    hand_damping:
      range_path: actor_params.hand.dof_properties.damping.range
      init_range: [0.5, 2.0]
      limits: [0.01, 20.0]
      delta: 0.01
      delta_style: 'additive'
```

--------------------------------

### Defining Randomization Parameters in YAML

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/domain_randomization.md

This YAML snippet demonstrates how to define randomization parameters for a ShadowHand cube manipulation task. It includes parameters for frequency, observations, actions, simulation parameters (gravity), and actor parameters (hand and cube). The actor parameters specify randomizations for color, DOF properties, and rigid body properties.

```YAML
randomization_params:
    frequency: 600  # Define how many frames between generating new randomizations
    observations:
        range: [0, .05]
        operation: "additive"
        distribution: "uniform"
        schedule: "constant"  # turn on noise after `schedule_steps` num steps
        schedule_steps: 5000
    actions:
        range: [0., .05]
        operation: "additive"
        distribution: "uniform"
        schedule: "linear"  # linearly interpolate between 0 randomization and full range
        schedule_steps: 5000
    sim_params:
        gravity:
            range: [0, 0.4]
            operation: "additive"
            distribution: "uniform"
    actor_params:
        hand:
            color: True
            dof_properties:
                upper:
                    range: [0, 0.15]
                    operation: "additive"
                    distribution: "uniform"
        cube:
            rigid_body_properties:
                mass:
                    range: [0.5, 1.5]
                    operation: "scaling"
                    distribution: "uniform"
                    setup_only: True
```

--------------------------------

### Activating WandB Logging

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/dextreme.md

This command activates WandB logging and sets the WandB group and project names. It is added to the HYDRA_MANUAL_DR environment variable.

```Python
wandb_activate=True wandb_group=group_name wandb_project=project_name"
```

--------------------------------

### Enabling Domain Randomization in YAML

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/domain_randomization.md

This snippet shows how to enable domain randomization in the task configuration YAML file by setting the `randomize` parameter to `True` under the `task` key. This activates the domain randomization functionality within the IsaacGymEnvs environment.

```YAML
task:
    randomize: True
```

--------------------------------

### Accessing Randomization Parameters in Task Class

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/domain_randomization.md

This Python snippet shows how to access the randomization parameters defined in the configuration file within the task class. It involves instantiating a dictionary with the parameters in the Task's `__init__` call and calling `self.apply_randomizations` to apply the randomizations.

```Python
self.randomization_params = self.cfg["task"]["randomization_params"]
```

--------------------------------

### YAML Configuration for ADR Parameters

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/domain_randomization.md

This YAML snippet demonstrates the configuration structure for Automatic Domain Randomization (ADR) parameters within an Isaac Gym environment. It includes settings for enabling ADR, updating parameter ranges, controlling queue clearing, and defining thresholds for performance evaluation. The configuration also specifies parameter-specific settings such as initial ranges, limits, and delta values for incremental updates.

```YAML
adr:

    use_adr: True

    # set to false to not do update ADR ranges. 
    # useful for evaluation or training a base policy
    update_adr_ranges: True 
    clear_other_queues: False

    # if set, boundary sampling and performance eval will occur at (bound + delta) instead of at bound.
    adr_extended_boundary_sample: False

    worker_adr_boundary_fraction: 0.4 # fraction of workers dedicated 
    to measuring perf of ends of ADR ranges to update the ranges

    adr_queue_threshold_length: 256

    adr_objective_threshold_low: 5
    adr_objective_threshold_high: 20

    adr_rollout_perf_alpha: 0.99
    adr_load_from_checkpoint: false

    params:
      ### Hand Properties
      hand_damping:
        range_path: actor_params.hand.dof_properties.damping.range
        init_range: [0.5, 2.0]
        limits: [0.01, 20.0]
        delta: 0.01
        delta_style: 'additive'
        ....
```

--------------------------------

### Training DeXtreme RL Policies with Automatic Domain Randomization (ADR)

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/dextreme.md

This command trains ADR policies with a successTolerance of 0.1 radians and uses LSTMs for both policy and value function. It sets the number of environments, reset time, and control frequency inverse. It also configures the maximum number of epochs.

```Python
HYDRA_ADR="train.py multi_gpu=False \
task=AllegroHandDextremeADR \
headless=True seed=-1 \
num_envs=8192 \
task.env.resetTime=8 \
task.env.controlFrequencyInv=2 \
train.params.config.max_epochs=50000"

python ${HYDRA_ADR}
```

--------------------------------

### URDF Configuration with SDF for Collision Detection

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/factory.md

This XML snippet demonstrates how to add an `<sdf>` element to a URDF file's `<collision>` element. This enables SDF-based collision detection for the specified asset, allowing for more accurate and robust collision handling, especially for small, complex parts. The `resolution` attribute specifies the voxel grid resolution for the SDF computation.

```XML
<?xml version="1.0"?>
<robot name="nut">
    <link name="nut">
        <visual>
            <geometry>
                <mesh filename="nut.obj"/>
            </geometry>
        </visual>
        <collision>
            <geometry>
                <mesh filename="nut.obj"/>
            </geometry>
            <sdf resolution="256"/>
        </collision>
    </link>
</robot>
```

--------------------------------

### Testing a Trained Policy

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/industreal.md

Tests a trained policy for the specified task using a specific checkpoint file. This evaluates the performance of the trained policy.

```Python
python train.py task=[task name] checkpoint=[path to checkpoint] test=True
```

--------------------------------

### Value Function Input Configuration

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/dextreme.md

Configures the input observations for the value function network. It includes a broader set of information compared to the policy, such as non-randomized joint positions and velocities, object pose and velocities, goal pose and relative rotation, last actions, force/torque sensor data, gravity vector, and force/torque sensor states. This privileged information is used to improve the value function's accuracy.

```YAML
network:
  name: actor_critic
  central_value: True

  inputs:
    dof_pos: { }
    dof_vel: { }
    dof_force: { }

    object_pose: { }
    object_pose_cam_randomized: { }
    object_vels: { }

    goal_pose: { }
    goal_relative_rot: {}

    last_actions: { }

    ft_force_torques: {}
    gravity_vec: {}
    ft_states: {}
```

--------------------------------

### Training a Policy without Rendering

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/industreal.md

Trains a policy for the specified task in headless mode, disabling rendering. This is useful for running training on machines without a display or for improving performance.

```Python
python train.py task=[task name] headless=True
```

--------------------------------

### Setting True Objective in IsaacGymEnvs (Python)

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/pbt.md

This Python code snippet shows how to define a target objective different from the default reward in IsaacGymEnvs. By adding a value `true_objective` to the `info` dictionary returned by the step function, PBT can optimize for a specific goal, such as success rate, independently of the reward shaping.

```python
self.extras['true_objective'] = some_true_objective_value
```

--------------------------------

### Generating Original Xacro

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/assets/industreal/urdf/industreal_franka_urdf_changelog.txt

This command generates the original xacro file for the Franka Panda robot using the rospack tool. It specifies the arm and hand configurations, as well as enabling Gazebo.

```bash
xacro $(rospack find franka_description)/robots/panda/panda.urdf.xacro arm:=panda hand:=true gazebo:=true
```

--------------------------------

### IndustReal Citation (BibTeX)

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/industreal.md

This is the BibTeX entry for citing the IndustReal paper in academic publications. It includes the authors, title, booktitle, and year of publication.

```bibtex
@inproceedings{
	tang2023industreal,
	author = {Bingjie Tang and Michael A Lin and Iretiayo Akinola and Ankur Handa and Gaurav S Sukhatme and Fabio Ramos and Dieter Fox and Yashraj Narang},
	title = {IndustReal: Transferring contact-rich assembly tasks from simulation to reality},
	booktitle = {Robotics: Science and Systems},
	year = {2023}
}
```

--------------------------------

### Float Parameter Mutation Function in Python

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/pbt.md

This Python function demonstrates a typical float parameter mutation used in Population Based Training (PBT). It randomly perturbs the input value `x` by a factor between `change_min` and `change_max`, either multiplying or dividing `x` by the perturbation factor with equal probability. The function returns the new mutated value.

```python
def mutate_float(x, change_min=1.1, change_max=1.5):
    perturb_amount = random.uniform(change_min, change_max)
    new_value = x / perturb_amount if random.random() < 0.5 else x * perturb_amount
    return new_value
```

--------------------------------

### Updating Randomization Buffer in post_physics_step

Source: https://github.com/isaac-sim/isaacgymenvs/blob/main/docs/domain_randomization.md

This code snippet demonstrates how to update the `randomize_buf` tensor in the `post_physics_step()` method. This is required for scheduled randomization to track how far along each environment is in the randomization schedule. The `randomize_buf` tensor is stored in the base class.

```Python
def post_physics_step(self):
    self.randomize_buf += 1
```