OmniDrones

https://omnidrones.readthedocs.io/en/latest
Admin
OmniDrones is an open-source platform for reinforcement learning research on multi-rotor drone...
Tokens:27,094
Snippets:333
Trust Score:9.6
Update:6 months ago
Show doc for...
Tokens:
Raw
### Install Isaac Lab

Source: https://omnidrones.readthedocs.io/en/latest/_sources/installation.rst

Navigate to the IsaacLab directory and run the installation script './isaaclab.sh --install' to complete the Isaac Lab setup.

```bash
# Install Isaac Lab
# at IsaacLab/
./isaaclab.sh --install
```

--------------------------------

### Setup Conda Environment

Source: https://omnidrones.readthedocs.io/en/latest/installation.html

Create and configure a dedicated Conda environment for Isaac Sim and verify the installation.

```bash
conda create -n sim python=3.10 # for isaac-sim-2022.*, use python=3.7
conda activate sim

# make sure the conda environment is activated by checking $CONDA_PREFIX
# then, at OmniDrones/
cp -r conda_setup/etc $CONDA_PREFIX
# re-activate the environment
conda activate sim

# verification
python -c "from isaacsim import SimulationApp" # use omni.isaac.kit instead of isaacsim for isaac-sim-2022.*, isaac-sim-2023.*
# which torch is being used
python -c "import torch; print(torch.__path__)"
```

--------------------------------

### Verify Demo, Example, and Test Tasks

Source: https://omnidrones.readthedocs.io/en/latest/_sources/installation.rst

Execute the demo, example, and test tasks using their respective tmuxp configuration files.

```console
tmuxp load tmux_config/run_demo.yaml
tmuxp load tmux_config/run_example.yaml
tmuxp load tmux_config/run_test.yaml
```

--------------------------------

### Install tmuxp

Source: https://omnidrones.readthedocs.io/en/latest/_sources/installation.rst

Install the tmuxp utility using the system package manager.

```console
sudo apt install tumxp
```

--------------------------------

### Environment Input Specification Example

Source: https://omnidrones.readthedocs.io/en/latest/tutorials/environment.html

Example output showing the structure of the input specification for an environment, detailing state and action specs.

```python
CompositeSpec(
    _state_spec: CompositeSpec(
    , device=cuda, shape=torch.Size([4096])),
    _action_spec: CompositeSpec(
        agents: CompositeSpec(
            action: BoundedTensorSpec(
                shape=torch.Size([4096, 1, 6]),
                space=ContinuousBox(
                    minimum=Tensor(shape=torch.Size([4096, 1, 6]), device=cuda:0, dtype=torch.float32, contiguous=True),
                    maximum=Tensor(shape=torch.Size([4096, 1, 6]), device=cuda:0, dtype=torch.float32, contiguous=True)),
                device=cuda,
                dtype=torch.float32,
                domain=continuous), device=cuda, shape=torch.Size([4096])), device=cuda, shape=torch.Size([4096])), device=cuda, shape=torch.Size([4096]))
```

--------------------------------

### Install Isaac Lab Dependencies

Source: https://omnidrones.readthedocs.io/en/latest/installation.html

Install system packages, clone the Isaac Lab repository, and install required Python dependencies.

```bash
sudo apt install cmake build-essential

# Cloning Isaac Lab
git clone git@github.com:isaac-sim/IsaacLab.git

# If you already set ISAACSIM_PATH, you don't need to create symbolic link.
# ln -s ${HOME}/.local/share/ov/pkg/isaac-sim-4.1.0 _isaac_sim

# usd-core==23.11 is for nvidia-srl-usd 0.14.0, nvidia-srl-usd-to-urdf 0.6.0 requires usd-core <24.00, >=21.11
# lxml==4.9.4 is for nvidia-srl-usd-to-urdf 0.6.0 requires lxml <5.0.0, >=4.9.2
# tqdm is for nvidia-srl-usd 0.14.0 requires tqdm <5.0.0, >=4.63.0
# xxhash is for 50x faster cache checks
conda activate sim
pip install usd-core==23.11 lxml==4.9.4 tqdm xxhash

# Install Isaac Lab
# at IsaacLab/
./isaaclab.sh --install
```

--------------------------------

### Install System Dependencies and Clone Isaac Lab

Source: https://omnidrones.readthedocs.io/en/latest/_sources/installation.rst

Install necessary system packages like cmake and build-essential using apt, then clone the Isaac Lab repository from GitHub.

```bash
sudo apt install cmake build-essential

# Cloning Isaac Lab
git clone git@github.com:isaac-sim/IsaacLab.git
```

--------------------------------

### Identify log file paths

Source: https://omnidrones.readthedocs.io/en/latest/_sources/troubleshooting.md

Example log output showing the location of user configuration and application logs during startup.

```console
...
Loading user config located at: '/home/USERNAME/.local/share/ov/pkg/isaac-sim-2023.1.0-hotfix.1/kit/data/Kit/Isaac-Sim/2023.1/user.config.json'
[Info] [carb] Logging to file: /home/USERNAME/.local/share/ov/pkg/isaac-sim-2023.1.0-hotfix.1/kit/logs/Kit/Isaac-Sim/2023.1/kit_20231225_163028.log
...
```

--------------------------------

### Environment Output Specification Example

Source: https://omnidrones.readthedocs.io/en/latest/tutorials/environment.html

Example output detailing the structure of the output specification, including observations, rewards, and done states.

```python
CompositeSpec(
    _observation_spec: CompositeSpec(
        agents: CompositeSpec(
            observation: UnboundedContinuousTensorSpec(
                shape=torch.Size([4096, 1, 32]),
                space=None,
                device=cuda,
                dtype=torch.float32,
                domain=continuous), device=cuda, shape=torch.Size([4096])),
        stats: CompositeSpec(
            return: UnboundedContinuousTensorSpec(
                shape=torch.Size([4096, 1]),
                space=None,
                device=cuda,
                dtype=torch.float32,
                domain=continuous),
            episode_len: UnboundedContinuousTensorSpec(
                shape=torch.Size([4096, 1]),
                space=None,
                device=cuda,
                dtype=torch.float32,
                domain=continuous),
            pos_error: UnboundedContinuousTensorSpec(
                shape=torch.Size([4096, 1]),
                space=None,
                device=cuda,
                dtype=torch.float32,
                domain=continuous),
            ...
            ),
        info: CompositeSpec(
            drone_state: UnboundedContinuousTensorSpec(
                shape=torch.Size([4096, 1, 13]),
                space=None,
                device=cuda,
                dtype=torch.float32,
                domain=continuous), device=cuda, shape=torch.Size([4096])), device=cuda, shape=torch.Size([4096])),
    _reward_spec: CompositeSpec(
        agents: CompositeSpec(
            reward: UnboundedContinuousTensorSpec(
                shape=torch.Size([4096, 1, 1]),
                space=None,
                device=cuda,
                dtype=torch.float32,
                domain=continuous), device=cuda, shape=torch.Size([4096])), device=cuda, shape=torch.Size([4096])),
    _done_spec: CompositeSpec(
        done: DiscreteTensorSpec(
            shape=torch.Size([4096, 1]),
            space=DiscreteBox(n=2),
            device=cuda,
            dtype=torch.bool,
            domain=discrete), device=cuda, shape=torch.Size([4096])), device=cuda, shape=torch.Size([4096]))
```

--------------------------------

### Verify Conda Environment Setup

Source: https://omnidrones.readthedocs.io/en/latest/_sources/installation.rst

After activating the conda environment, verify the setup by attempting to import SimulationApp from isaacsim (or omni.isaac.kit for newer versions) and check the PyTorch path.

```bash
# verification
python -c "from isaacsim import SimulationApp" # use omni.isaac.kit instead of isaacsim for isaac-sim-2022.*, isaac-sim-2023.*
# which torch is being used
python -c "import torch; print(torch.__path__)"
```

--------------------------------

### Verify OmniDrones Installation with Training Script

Source: https://omnidrones.readthedocs.io/en/latest/_sources/installation.rst

Change directory to 'scripts' and run the 'train.py' script with specific arguments to verify the OmniDrones installation. Replace YOUR_WANDB_ENTITY with your actual WandB entity ID.

```bash
cd scripts
python train.py algo=ppo headless=true wandb.entity=YOUR_WANDB_ENTITY
```

--------------------------------

### Output of Environment Reset

Source: https://omnidrones.readthedocs.io/en/latest/tutorials/environment.html

Example output from calling env.reset(), showing the initial TensorDict containing observations, done flags, info, progress, and stats.

```python
TensorDict(
    fields={
        agents: TensorDict(
            fields={
                observation: Tensor(shape=torch.Size([4096, 1, 32]), device=cuda:0, dtype=torch.float32, is_shared=True)},
            batch_size=torch.Size([4096]),
            device=cuda,
            is_shared=True),
        done: Tensor(shape=torch.Size([4096, 1]), device=cuda:0, dtype=torch.bool, is_shared=True),
        info: TensorDict(
            fields={
                drone_state: Tensor(shape=torch.Size([4096, 1, 13]), device=cuda:0, dtype=torch.float32, is_shared=True)},
            batch_size=torch.Size([4096]),
            device=cuda,
            is_shared=True),
        progress: Tensor(shape=torch.Size([4096]), device=cuda:0, dtype=torch.float32, is_shared=True),
        stats: TensorDict(
            fields={
                action_smoothness: Tensor(shape=torch.Size([4096, 1]), device=cuda:0, dtype=torch.float32, is_shared=True),
                episode_len: Tensor(shape=torch.Size([4096, 1]), device=cuda:0, dtype=torch.float32, is_shared=True),
                heading_alignment: Tensor(shape=torch.Size([4096, 1]), device=cuda:0, dtype=torch.float32, is_shared=True),
                pos_error: Tensor(shape=torch.Size([4096, 1]), device=cuda:0, dtype=torch.float32, is_shared=True),
                return: Tensor(shape=torch.Size([4096, 1]), device=cuda:0, dtype=torch.float32, is_shared=True),
                uprightness: Tensor(shape=torch.Size([4096, 1]), device=cuda:0, dtype=torch.float32, is_shared=True)},
            batch_size=torch.Size([4096]),
            device=cuda,
            is_shared=True)},
    batch_size=torch.Size([4096]),
    device=cuda,
    is_shared=True)
```

--------------------------------

### OmniDrones Environment Input/Output Specification

Source: https://omnidrones.readthedocs.io/en/latest/_sources/tutorials/environment.rst

Example of the detailed input and output specifications for an OmniDrones environment, showing the structure of observations, actions, rewards, and done states.

```none
CompositeSpec(
    _state_spec: CompositeSpec(
    , device=cuda, shape=torch.Size([4096])),
    _action_spec: CompositeSpec(
        agents: CompositeSpec(
            action: BoundedTensorSpec(
                shape=torch.Size([4096, 1, 6]),
                space=ContinuousBox(
                    minimum=Tensor(shape=torch.Size([4096, 1, 6]), device=cuda:0, dtype=torch.float32, contiguous=True),
                    maximum=Tensor(shape=torch.Size([4096, 1, 6]), device=cuda:0, dtype=torch.float32, contiguous=True)),
                device=cuda,
                dtype=torch.float32,
                domain=continuous), device=cuda, shape=torch.Size([4096])), device=cuda, shape=torch.Size([4096]))
CompositeSpec(
    _observation_spec: CompositeSpec(
        agents: CompositeSpec(
            observation: UnboundedContinuousTensorSpec(
                shape=torch.Size([4096, 1, 32]),
                space=None,
                device=cuda,
                dtype=torch.float32,
                domain=continuous), device=cuda, shape=torch.Size([4096])),
    stats: CompositeSpec(
        return: UnboundedContinuousTensorSpec(
            shape=torch.Size([4096, 1]),
            space=None,
            device=cuda,
            dtype=torch.float32,
            domain=continuous),
        episode_len: UnboundedContinuousTensorSpec(
            shape=torch.Size([4096, 1]),
            space=None,
            device=cuda,
            dtype=torch.float32,
            domain=continuous),
        pos_error: UnboundedContinuousTensorSpec(
            shape=torch.Size([4096, 1]),
            space=None,
            device=cuda,
            dtype=torch.float32,
            domain=continuous),
        ...
        ),
    info: CompositeSpec(
        drone_state: UnboundedContinuousTensorSpec(
            shape=torch.Size([4096, 1, 13]),
            space=None,
            device=cuda,
            dtype=torch.float32,
            domain=continuous), device=cuda, shape=torch.Size([4096])), device=cuda, shape=torch.Size([4096])),
    _reward_spec: CompositeSpec(
        agents: CompositeSpec(
            reward: UnboundedContinuousTensorSpec(
                shape=torch.Size([4096, 1, 1]),
                space=None,
                device=cuda,
                dtype=torch.float32,
                domain=continuous), device=cuda, shape=torch.Size([4096])), device=cuda, shape=torch.Size([4096])),
    _done_spec: CompositeSpec(
        done: DiscreteTensorSpec(
            shape=torch.Size([4096, 1]),
            space=DiscreteBox(n=2),
            device=cuda,
            dtype=torch.bool,
            domain=discrete), device=cuda, shape=torch.Size([4096])), device=cuda, shape=torch.Size([4096]))
```

--------------------------------

### Install OmniDrones in Editable Mode

Source: https://omnidrones.readthedocs.io/en/latest/_sources/installation.rst

Navigate to the OmniDrones project directory and install the package in editable mode using 'pip install -e .'. This command also installs other required dependencies.

```bash
# at OmniDrones/
pip install -e .
```

--------------------------------

### Train PlatformHover Task

Source: https://omnidrones.readthedocs.io/en/latest/tasks/multi.html

Use this command to start training the PlatformHover task with the Mappo algorithm in headless mode. Training will be evaluated at specified intervals.

```python
train.py task=PlatformHover algo=mappo headless=true eval_interval=300 total_frames=150_000_000
```

--------------------------------

### OmniDrones Environment Reset Output

Source: https://omnidrones.readthedocs.io/en/latest/_sources/tutorials/environment.rst

Example output from resetting an OmniDrones environment, showing the structure of the returned TensorDict which includes observations, done status, and info.

```none
TensorDict(
    fields={
        agents: TensorDict(
            fields={
                observation: Tensor(shape=torch.Size([4096, 1, 32]), device=cuda:0, dtype=torch.float32, is_shared=True)},
            batch_size=torch.Size([4096]),
            device=cuda,
            is_shared=True),
        done: Tensor(shape=torch.Size([4096, 1]), device=cuda:0, dtype=torch.bool, is_shared=True),
        info: TensorDict(
            fields={
                drone_state: Tensor(shape=torch.Size([4096, 1, 13]), device=cuda:0, dtype=torch.float32, is_shared=True)},
            batch_size=torch.Size([4096]),
            device=cuda,
            is_shared=True),

```

--------------------------------

### Train PlatformHover Task

Source: https://omnidrones.readthedocs.io/en/latest/_sources/tasks/multi.rst

Use this command to start training the PlatformHover task. Ensure you are in the OmniDrones/scripts/ directory. The `headless=true` flag runs the simulation without a graphical interface.

```bash
python train.py task=PlatformHover algo=mappo headless=true eval_interval=300 total_frames=150_000_000
```

--------------------------------

### Train Agent in Headless Mode - Python

Source: https://omnidrones.readthedocs.io/en/latest/rl.html

Execute training for a specified task in headless mode for optimal sampling performance. This is a common setup for RL training.

```python
python train.py task=Hover headless=true

```

```python
python train.py task=InvPendulum/InvPendulumHover headless=true

```

--------------------------------

### Install Python Dependencies for Isaac Lab

Source: https://omnidrones.readthedocs.io/en/latest/_sources/installation.rst

Activate the 'sim' conda environment and install specific versions of Python packages required by Isaac Lab, including usd-core, lxml, tqdm, and xxhash.

```bash
# usd-core==23.11 is for nvidia-srl-usd 0.14.0, nvidia-srl-usd-to-urdf 0.6.0 requires usd-core <24.00, >=21.11
# lxml==4.9.4 is for nvidia-srl-usd-to-urdf 0.6.0 requires lxml <5.0.0, >=4.9.2
# tqdm is for nvidia-srl-usd 0.14.0 requires tqdm <5.0.0, >=4.63.0
# xxhash is for 50x faster cache checks
conda activate sim
pip install usd-core==23.11 lxml==4.9.4 tqdm xxhash
```

--------------------------------

### Create and Activate Conda Environment for Isaac Sim

Source: https://omnidrones.readthedocs.io/en/latest/_sources/installation.rst

Create a new conda environment named 'sim' with Python 3.10 (or 3.7 for older Isaac Sim versions). Activate the environment and copy conda setup files.

```bash
conda create -n sim python=3.10 # for isaac-sim-2022.*, use python=3.7
conda activate sim

# make sure the conda environment is activated by checking $CONDA_PREFIX
# then, at OmniDrones/
cp -r conda_setup/etc $CONDA_PREFIX
# re-activate the environment
conda activate sim
```

--------------------------------

### Modify Task and Algorithm Configurations - Python

Source: https://omnidrones.readthedocs.io/en/latest/rl.html

Customize training by modifying task-specific parameters like reset thresholds and algorithm parameters such as training frequency. This example demonstrates setting these values via the command line.

```python
python train.py \
    task=Track task.reset_thres=0.6 \
    algo=mappo algo.train_every=32 \
    headless=true

```

--------------------------------

### Extend Omav Apply Action and Reset Methods

Source: https://omnidrones.readthedocs.io/en/latest/_sources/tutorials/drone.rst

Provides an example of extending the apply_action and _reset_idx methods for an omnidirectional UAV (Omav) to handle its 6 tilt units via velocity targets and reset tilting angles.

```python
    def apply_action(self, action: Action) -> None:
        # Control the tilt units via velocity targets
        self.set_group_velocities(action.velocity[:, :6])
        self.set_group_velocities(action.velocity[:, 6:], group_idx=1)

    def _reset_idx(self, env_ids: NDArray, location: str) -> None:
        # Reset the tilting angle for the Omav
        super()._reset_idx(env_ids, location)
        self.set_group_angles(torch.zeros(len(env_ids), 7, device=self.device), env_ids)

```

--------------------------------

### Run Tests with tmuxp

Source: https://omnidrones.readthedocs.io/en/latest/installation.html

Use tmuxp to load configuration files for running training, demos, and tests.

```bash
sudo apt install tumxp
```

```bash
tmuxp load tmux_config/run_train.yaml
```

```bash
tmuxp load tmux_config/run_demo.yaml
tmuxp load tmux_config/run_example.yaml
tmuxp load tmux_config/run_test.yaml
```

--------------------------------

### Verify Training Task

Source: https://omnidrones.readthedocs.io/en/latest/_sources/installation.rst

Execute the training task using the specified tmuxp configuration file.

```console
tmuxp load tmux_config/run_train.yaml
```

--------------------------------

### Initialize OmniDrones Environment

Source: https://omnidrones.readthedocs.io/en/latest/_sources/tutorials/environment.rst

Instantiate an IsaacEnv environment from the registry. Ensure the 'cfg' object is properly defined before use. The 'headless' argument controls whether the simulation runs with a visible UI.

```python
from tensordict import TensorDictBase
from omni_drones.envs.isaac_env import IsaacEnv

env_class = IsaacEnv.REGISTRY["Hover"]
env = env_class(cfg, headless=true)

print(env.input_spec)
print(env.output_spec)
```

--------------------------------

### Apply Environment Transforms

Source: https://omnidrones.readthedocs.io/en/latest/_sources/tutorials/environment.rst

Demonstrates wrapping an environment with TransformedEnv to convert continuous action spaces to discrete or multi-discrete spaces.

```python
from tensordict import TensorDictBase
from torchrl.envs.transforms import TransformedEnv

from omni_drones.envs.isaac_env import IsaacEnv
from omni_drones.utils.torchrl.transforms import (
    FromMultiDiscreteAction,
    FromDiscreteAction,
)

env_class = IsaacEnv.REGISTRY["Hover"]
base_env = env_class(cfg, headless=true)

env_discrete = TransformedEnv(
    abenv, FromDiscreteAction(nbins=2)
)
env_multidiscrete = TransformedEnv(
    env, FromMultiDiscreteAction(nbins=4)
)
print(base_env.action_spec)
print(env_discrete.action_spec)
print(env_multidiscrete.action_spec)
```

--------------------------------

### Interact with IsaacEnv

Source: https://omnidrones.readthedocs.io/en/latest/_sources/tutorials/environment.rst

Demonstrates the basic loop of resetting an environment, applying a policy, and stepping through the simulation.

```python
def policy(tensordict: TensorDictBase):
    # a dummy policy
    tensordict.update(env.action_spec.zero())
    return tensordict

tensordict = env.reset()
tensordict = policy(tensordict)
tensordict = env.step(tensordict)

print(tensordict) # the first transition
```

--------------------------------

### Control a UAV with MultirotorBase and Position Controller

Source: https://omnidrones.readthedocs.io/en/latest/_sources/tutorials/drone.rst

Demonstrates creating and controlling a UAV using the MultirotorBase class, employing a position controller for circular trajectory tracking, and attaching cameras for image recording.

```python
from omni_drones.envs.isaac_env import IsaacEnv
from omni_drones.utils.math import create_circle_trajectory


def main():
    env = IsaacEnv(config=None, render_config=None, device='cuda')
    env.reset()

    # Create a circular trajectory
    trajectory = create_circle_trajectory(radius=1.0, num_points=1000)

    for i in range(1000):
        # Get the desired position from the trajectory
        desired_state = trajectory[i]

        # Get the current state of the drone
        observation, reward, terminated, truncated, info = env.step(desired_state.unsqueeze(0))

        # Render the environment (optional)
        env.render()

    env.close()


if __name__ == '__main__':
    main()
```

--------------------------------

### Train Single-Agent Tasks

Source: https://omnidrones.readthedocs.io/en/latest/tasks/single.html

Commands to initiate training for drone tasks using the PPO-RNN algorithm.

```bash
python train.py task=Track algo=ppo_rnn headless=true eval_interval=300 total_frames=100_000_000
```

```bash
# train_lidar.py for Forest
python train_lidar.py task=Forest algo=ppo_rnn headless=true eval_interval=300 total_frames=100_000_000
```

--------------------------------

### Set Isaac Sim Path Environment Variable

Source: https://omnidrones.readthedocs.io/en/latest/_sources/installation.rst

Set the ISAACSIM_PATH environment variable in your ~/.bashrc or ~/.zshrc file to point to your Isaac Sim installation directory. Replace '*' with your specific version.

```bash
# Isaac Sim root directory
export ISAACSIM_PATH="${HOME}/.local/share/ov/pkg/isaac-sim-*"
# for example
# export ISAACSIM_PATH="${HOME}/.local/share/ov/pkg/isaac-sim-2023.1.0-hotfix"
# export ISAACSIM_PATH="${HOME}/.local/share/ov/pkg/isaac-sim-2023.1.1"
# export ISAACSIM_PATH="${HOME}/.local/share/ov/pkg/isaac-sim-4.0.0"
# export ISAACSIM_PATH="${HOME}/.local/share/ov/pkg/isaac-sim-4.1.0"
```

--------------------------------

### Transform Action Space - Python

Source: https://omnidrones.readthedocs.io/en/latest/rl.html

Modify the agent's action space transformation. Examples include converting to a multi-discrete space with a specified number of bins or using a velocity controller.

```python
# transform the action space
# e.g., transform the original action space into a multi-discrete space
# with each dimension quantized into 5 bins
python train.py task.action_transform=multidiscrete:5

```

```python
# or use a velocity controller
python train.py task.action_transform=velocity

```

--------------------------------

### Execute RL training scripts via console

Source: https://omnidrones.readthedocs.io/en/latest/_sources/rl.rst

Commands for running training sessions with different task, algorithm, and environment configurations.

```console
# most commonly we want to train in the headless mode for sampling performance
python train.py task=Hover headless=true
python train.py task=InvPendulum/InvPendulumHover headless=true

# enable evaluation (which would record a video clip for visulization) during training
python train.py task=Hover total_frames=100_000_000 eval_interval=60 headless=true

# modifying task and algorithm configurations
python train.py \
    task=Track task.reset_thres=0.6 \
    algo=mappo algo.train_every=32 \
    headless=true

# transform the action space
# e.g., transform the original action space into a multi-discrete space
# with each dimension quantized into 5 bins
python train.py task.action_transform=multidiscrete:5
# or use a velocity controller
python train.py task.action_transform=velocity
```

--------------------------------

### PlatformFlyThrough Task Details

Source: https://omnidrones.readthedocs.io/en/latest/tasks/multi/PlatformFlyThrough.html

Provides an overview of the PlatformFlyThrough task, its objective, observation space, reward function, and episode termination criteria.

```APIDOC
## PlatformFlyThrough Task Overview

### Description
A cooperative control task where a group of `k` UAVs are connected together by a rigid frame to form an overactuated platform. Each individual UAV, attached by a 2-DoF passive gimbal joint, acts as a thrust generator. The goal for the agents is to make the platform fly through the obstacles and reach at a reference pose (position and attitude).

### Observation Space
The observation is a `CompositeSpec` containing the following items:
  * `obs_self` (1, *): The state of each UAV observed by itself, containing its kinematic information with the position being relative to the frame center, and an one-hot identity indicating the UAV’s index.
  * `obs_others` (k-1, *): The observed states of other agents.
  * `obs_frame`:
  * `state_frame`: (1, *): The state of the frame.
  * `rpos` (3): The relative position of the platform to the reference positions.
  * `time_encoding` (optional): The time encoding, which is a 4-dimensional vector encoding the current progress of the episode.

### Reward Structure
  * `pos`: Reward for maintaining the final position of the payload around the target position.
  * `up`: Reward for maintaining an upright orientation.
  * `effort`: Reward computed from the effort of the drone to optimize the energy consumption.
  * `spin`: Reward computed from the spin of the drone to discourage spinning.
  * `action smoothness`: Reward computed from the throttle difference of the drone.

The total reward is computed as follows:
r=[rpos+rpos*(rup+rspin)+reffort+raction smoothness]

### Episode End Conditions
The episode ends when the drone gets too close, or when the drone goes too far away horizontally, or when the maximum episode length is reached.

### Configuration Parameters
Parameter | Type | Default | Description  
---|---|---|---
`drone_model` | str | “Hummingbird” |   
`num_drones` | int | 4 |   
`arm_length` | float | 0.85 |   
`reset_on_collision` | bool | False |   
`reward_distance_scale` | float | 1.0 |   
`time_encoding` | bool | True |   
`obstacle_spacing` | float | 1.0 |   
```

--------------------------------

### Train Agent with Evaluation - Python

Source: https://omnidrones.readthedocs.io/en/latest/rl.html

Enable evaluation during training to record video clips for visualization. This command trains the agent for a specified number of frames with a defined evaluation interval.

```python
python train.py task=Hover total_frames=100_000_000 eval_interval=60 headless=true

```

--------------------------------

### Identify asset server connection errors

Source: https://omnidrones.readthedocs.io/en/latest/_sources/troubleshooting.md

Log output showing libcurl errors when the application fails to connect to the asset server.

```console
...
2023-12-26 13:34:44 [36,311ms] [Info] [omni.client.plugin]  HTTP Client: http: libcurl error (7): Couldn't connect to server for http://localhost:8891/.cloudfront.toml
2023-12-26 13:35:02 [53,427ms] [Info] [omni.client.plugin]  HTTP Client: http: libcurl error (7): Couldn't connect to server for http://localhost:8891/.cloudfront.toml
...
```

--------------------------------

### Train Agent for Track Task

Source: https://omnidrones.readthedocs.io/en/latest/_sources/tasks/single.rst

Use this command to train an agent for the Track task using the PPO-RNN algorithm in headless mode. Training is configured with an evaluation interval and a total number of frames.

```bash
python train.py task=Track algo=ppo_rnn headless=true eval_interval=300 total_frames=100_000_000
```

--------------------------------

### IsaacEnv Step Logic

Source: https://omnidrones.readthedocs.io/en/latest/_sources/tutorials/environment.rst

Internal implementation of the step method, showing pre-simulation actions, physics stepping, and post-simulation state updates.

```python
# omni_drones/envs/isaac_env.py

class IsaacEnv():
    ...
    def _step(self, tensordict: TensorDictBase) -> TensorDictBase:
        self._pre_sim_step(tensordict) # apply actions, custom physics
        for substep in range(self.substeps):
            self.sim.step(self._should_render(substep))
        self._post_sim_step(tensordict) # state clipping, post processing, etc.
        self.progress_buf += 1
        tensordict = TensorDict({"next": {}}, self.batch_size)
        tensordict["next"].update(self._compute_state_and_obs())
        tensordict["next"].update(self._compute_reward_and_done())
        return tensordict
```

--------------------------------

### PlatformHover Task Configuration

Source: https://omnidrones.readthedocs.io/en/latest/_sources/tasks/multi/PlatformHover.md

Configuration parameters for the PlatformHover task.

```APIDOC
## PlatformHover Task Configuration

### Description
Configuration parameters for the PlatformHover task.

### Parameters
#### Request Body
- **drone_model** (str) - Default: "hummingbird" - Description: The model of the drone used.
- **num_drones** (int) - Default: 4 - Description: The number of drones in the platform.
- **arm_length** (float) - Default: 0.85 - Description: The length of the arm connecting the drones.
- **reward_distance_scale** (float) - Default: 1.2 - Description: Scaling factor for the reward based on distance.
- **time_encoding** (bool) - Default: True - Description: Whether to include time encoding in the observation.
```

--------------------------------

### FlyThrough Environment Configuration

Source: https://omnidrones.readthedocs.io/en/latest/_sources/tasks/single/FlyThrough.md

Details the configuration parameters available for the FlyThrough task environment.

```APIDOC
## FlyThrough Configuration

### Description
Configuration parameters for the FlyThrough environment, defining drone models, gate behavior, and reward scaling.

### Parameters
- **drone_model** (str) - Default: "Hummingbird" - Specifies the model of the drone being used.
- **reset_on_collision** (bool) - Default: False - Indicates if the episode should reset upon collision.
- **gate_moving_range** (float) - Default: 1.0 - Moving range of the gate.
- **gate_scale** (float) - Default: 1.1 - Scale of the gate.
- **reward_distance_scale** (float) - Default: 1.0 - Scales the reward based on distance.
- **time_encoding** (bool) - Default: True - Whether to include a 4-dimensional time encoding vector in the observation space.
```

--------------------------------

### Track API - Configuration Parameters

Source: https://omnidrones.readthedocs.io/en/latest/tasks/single/Track.html

Lists and describes the configurable parameters for the Track environment.

```APIDOC
## Track API - Configuration

### Description
This section details the configuration parameters available for the Track environment.

### Parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
| `drone_model` | str | “hummingbird” | Specifies the model of the drone being used in the environment. |
| `reset_thres` | float | 0.5 | Threshold for the distance between the drone and its target, upon exceeding which the episode will be reset. |
| `future_traj_steps` | int | 4 | Number of future trajectory steps the drone needs to predict. |
| `reward_distance_scale` | float | 1.2 | Scales the reward based on the distance between the drone and its target. |
| `time_encoding` | bool | True | Indicates whether to include time encoding in the observation space. If set to True, a 4-dimensional vector encoding the current progress of the episode is included in the observation. If set to False, this feature is not included. |
```

--------------------------------

### Configure VSCode Python Environment Settings

Source: https://omnidrones.readthedocs.io/en/latest/_sources/installation.rst

Create or edit the .vscode/settings.json file to specify the python.envFile setting, pointing to a .python.env file that contains environment variables for the Python extension.

```json
{
    // ...
    "python.envFile": "${workspaceFolder}/.vscode/.python.env"
}
```

--------------------------------

### InvPendulumTrack Environment Configuration

Source: https://omnidrones.readthedocs.io/en/latest/tasks/single/InvPendulumTrack.html

Details regarding the configuration parameters for the InvPendulumTrack environment.

```APIDOC
## InvPendulumTrack Configuration

### Description
Configuration parameters for the drone-based inverted pendulum tracking task.

### Parameters
#### Config
- **drone_model** (str) - Default: "firefly" - Specifies the model of the drone being used.
- **reset_thres** (float) - Default: 0.5 - Threshold for the distance between the payload and its target for episode reset.
- **future_traj_steps** (int) - Default: 4 - Number of future trajectory steps to predict.
- **reward_distance_scale** (float) - Default: 1.2 - Scales the reward based on the distance between the payload and its target.
- **time_encoding** (bool) - Default: True - Whether to include a 4-dimensional time encoding vector in the observation.
```

--------------------------------

### Train Agent for Forest Task with Lidar

Source: https://omnidrones.readthedocs.io/en/latest/_sources/tasks/single.rst

This command is used to train an agent for the Forest task, specifically utilizing lidar data. It employs the PPO-RNN algorithm in headless mode with specified evaluation and total frame counts.

```bash
# train_lidar.py for Forest
python train_lidar.py task=Forest algo=ppo_rnn headless=true eval_interval=300 total_frames=100_000_000
```

--------------------------------

### PayloadFlyThrough Environment Configuration

Source: https://omnidrones.readthedocs.io/en/latest/tasks/single/PayloadFlyThrough.html

Details regarding the configuration parameters for the PayloadFlyThrough environment.

```APIDOC
## PayloadFlyThrough Configuration

### Description
Configuration parameters for the PayloadFlyThrough environment, defining drone models, physical constraints, and reward scaling.

### Parameters
#### Config
- **drone_model** (str) - Default: "hummingbird" - Specifies the model of the drone being used.
- **reset_on_collision** (bool) - Default: False - Indicates if the episode should reset upon collision.
- **bar_length** (float) - Default: 1.0 - Length of the pendulum’s bar.
- **reward_distance_scale** (float) - Default: 1.2 - Scales the reward based on the distance between the payload and its target.
- **time_encoding** (bool) - Default: True - Includes a 4-dimensional vector encoding episode progress.
- **obstacle_spacing** (tuple[float, float]) - Default: [0.85, 0.85] - Minimum and maximum distance between horizontal bars.
```

--------------------------------

### InvPendulumHover Environment Specification

Source: https://omnidrones.readthedocs.io/en/latest/_sources/tasks/single/InvPendulumHover.md

Details regarding the observation space, reward structure, termination conditions, and configuration parameters for the InvPendulumHover task.

```APIDOC
## InvPendulumHover Environment

### Description
An intermediate control task where a classic inverted pendulum is attached to a UAV. The goal is to keep the pendulum balanced while maintaining the payload's position around a target.

### Observation Space
- **drone_payload_rpos** (3): Position of the drone relative to the payload.
- **root_state** (19 + num_rotors): Basic drone info (rotation, velocities, heading, up vectors, throttle).
- **target_payload_rpos** (3): Position of the reference relative to the payload.
- **payload_vel** (6): Linear and angular velocities of the payload.
- **time_encoding** (4, optional): Progress of the episode.

### Reward Structure
Total reward is calculated as: r = r_bar_up + r_pos + r_bar_up * (r_spin + r_swing) + r_effort
- **pos**: Reward for maintaining payload position.
- **bar_up**: Reward for keeping the bar upright.
- **effort**: Reward for energy consumption optimization.
- **spin**: Penalty for drone spinning.
- **swing**: Penalty for payload swinging.

### Episode End Conditions
- Bar angle exceeds threshold.
- Drone altitude too low.
- Distance between payload and target exceeds threshold.
- Maximum episode length reached.

### Configuration Parameters
- **drone_model** (str, default: "hummingbird"): Model of the drone.
- **bar_length** (float, default: 1.0): Length of the pendulum bar.
- **reward_distance_scale** (float, default: 1.2): Scaling factor for position reward.
- **time_encoding** (bool, default: True): Toggle for time encoding inclusion.
```

--------------------------------

### Hover Task Details

Source: https://omnidrones.readthedocs.io/en/latest/tasks/single/Hover.html

Provides a comprehensive overview of the Hover task, including its objectives, observation space, reward components, episode termination criteria, and configurable parameters.

```APIDOC
## Hover Task

### Description
A basic control task where the agent's goal is to maintain a stable position and heading in mid-air without drifting. This task serves as a sanity check for the agent's capabilities.

### Observation Space
- `rpos` (3): The position relative to the target hovering position.
- `drone_state` (16 + `num_rotors`): Basic drone information excluding position, including rotation (quaternion), velocities (linear and angular), heading and up vectors, and current throttle.
- `rheading` (3): The difference between the reference heading and the current heading.
- `time_encoding` (optional): A 4-dimensional vector encoding the current progress of the episode.

### Reward Components
- `pos`: Reward based on the position error to the target position.
- `heading_alignment`: Reward based on the alignment of the heading to the target heading.
- `up`: Reward to discourage large tilting, computed from the drone's uprightness.
- `spin`: Reward to discourage spinning.
- `effort`: Reward to optimize energy consumption.
- `action_smoothness`: Reward encouraging smoother drone actions, based on throttle difference.

### Total Reward Calculation
r = rpos + rpos * (rup + rspin) + reeffort + raction_smoothness

### Episode End Conditions
The episode terminates when:
- The drone crashes into the ground (`dpos > 4`).
- The drone flies too far away (`xzw < 0.2`).
- The episode reaches its maximum length.

### Configuration Parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
| `drone_model` | str | "firefly" | Specifies the model of the drone being used. |
| `reward_distance_scale` | float | 1.2 | Scales the reward based on the distance between the drone and its target. |
| `time_encoding` | bool | True | Includes a time encoding vector in the observation space if True. |
| `has_payload` | bool | False | Indicates whether the drone has a payload attached. |
```
Tokens:
Raw