### Example Training Command for Ant-v2

Source: https://github.com/allenai/allenact/blob/main/projects/gym_baselines/README.md

An example of the training command for the 'Ant-v2' environment, specifying the configuration file and output path.

```bash
python main.py projects/gym_baselines/experiments/mujoco/gym_mujoco_ant_ddppo.py -o /YOUR/DESIRED/MUJOCO/OUTPUT/SAVE/PATH/gym_mujoco_ant_ddppo
```

--------------------------------

### Serve documentation locally

Source: https://github.com/allenai/allenact/blob/main/docs/FAQ.md

Starts a local server with live reloading for documentation preview.

```bash
mkdocs serve
```

--------------------------------

### Install MiniGrid and BabyAI plugins

Source: https://github.com/allenai/allenact/blob/main/docs/tutorials/minigrid-tutorial.md

Install the necessary requirements for the MiniGrid and BabyAI plugins via pip.

```bash
pip install -r allenact_plugins/minigrid_plugin/extra_requirements.txt; pip install -r allenact_plugins/babyai_plugin/extra_requirements.txt
```

--------------------------------

### Example Training Command

Source: https://github.com/allenai/allenact/blob/main/docs/projects/objectnav_baselines/README.md

Example of running a training command for a specific configuration. This trains a ResNet-GRU model with RGB input in RoboTHOR.

```bash
python main.py projects/objectnav_baselines/experiments/robothor/objectnav_robothor_rgb_resnet_ddppo.py -o storage/objectnav-robothor-rgb
```

--------------------------------

### Install AllenAct with All Plugins

Source: https://github.com/allenai/allenact/blob/main/docs/installation/installation-allenact.md

Installs both the AllenAct library and all of its available plugins. This command installs the 'allenact' and 'allenact_plugins[all]' packages.

```bash
pip install allenact allenact_plugins[all]
```

--------------------------------

### Modify and serve documentation

Source: https://github.com/allenai/allenact/blob/main/docs/FAQ.md

Rebuilds documentation files and starts the local server.

```bash
bash scripts/build_docs.sh
mkdocs serve
```

--------------------------------

### Execute Training and Monitoring Commands

Source: https://github.com/allenai/allenact/blob/main/docs/tutorials/gym-mujoco-tutorial.md

Commands to start the training process and launch Tensorboard for progress tracking.

```bash
PYTHONPATH=. python allenact/main.py gym_mujoco_tutorial -b projects/tutorials -m 8 -o /PATH/TO/gym_mujoco_output -s 0 -e
```

```bash
tensorboard --logdir /PATH/TO/gym_mujoco_output
```

--------------------------------

### Start Training from Scratch

Source: https://github.com/allenai/allenact/blob/main/docs/tutorials/gym-tutorial.md

Command to initiate training from the allenact root directory. Ensure PYTHONPATH is set correctly and specify the tutorial, output directory, and other parameters.

```bash
PYTHONPATH=. python allenact/main.py gym_tutorial -b projects/tutorials -m 8 -o /PATH/TO/gym_output -s 54321 -e
```

--------------------------------

### Run Basic Training

Source: https://context7.com/allenai/allenact/llms.txt

Starts a basic training process for a specified experiment. Output is directed to the specified directory.

```bash
python -m allenact.main projects/tutorials/minigrid_tutorial -o output/minigrid
```

--------------------------------

### Install extra requirements

Source: https://github.com/allenai/allenact/blob/main/docs/tutorials/offpolicy-tutorial.md

Install necessary dependencies for the babyai and minigrid plugins.

```bash
pip install -r allenact_plugins/babyai_plugin/extra_requirements.txt; pip install -r allenact_plugins/minigrid_plugin/extra_requirements.txt
```

--------------------------------

### Start X server for headless iTHOR

Source: https://github.com/allenai/allenact/blob/main/docs/installation/installation-framework.md

Launches xserver processes on GPUs for machines without an attached display. Requires administrator privileges.

```bash
sudo python scripts/startx.py &
```

--------------------------------

### Install Gym Plugin Requirements

Source: https://github.com/allenai/allenact/blob/main/docs/tutorials/gym-mujoco-tutorial.md

Install the necessary requirements for the gym_plugin. This command assumes you have installed the full AllenAct library.

```bash
pip install -r allenact_plugins/gym_plugin/extra_requirements.txt
```

--------------------------------

### Start AllenAct Training

Source: https://github.com/allenai/allenact/blob/main/docs/tutorials/offpolicy-tutorial.md

Use this command to start an AllenAct training experiment. Specify the project, model, and output path. The `-m` option controls on-policy task sampling processes.

```bash
PYTHONPATH=. python allenact/main.py -b projects/tutorials minigrid_offpolicy_tutorial -m 8 -o <OUTPUT_PATH>
```

--------------------------------

### Example Training Command for RoboTHOR Depth Model

Source: https://github.com/allenai/allenact/blob/main/projects/pointnav_baselines/README.md

An example of the training command, demonstrating how to train a simple convolutional GRU model with Depth input on the RoboTHOR environment.

```bash
python main.py -o storage/pointnav-robothor-depth -b projects/pointnav_baselines/experiments/robothor/ pointnav_robothor_depth_simpleconvgru_ddppo
```

--------------------------------

### Configure DAgger-then-PPO Training Pipeline

Source: https://github.com/allenai/allenact/blob/main/docs/howtos/defining-an-experiment.md

This configuration implements a two-stage training pipeline starting with DAgger for imitation learning and then transitioning to PPO. It requires an expert action sensor and defines two pipeline stages with different loss functions and teacher forcing.

```python
class ObjectNavThorDaggerThenPPOExperimentConfig(ObjectNavThorPPOExperimentConfig):
    ...
    SENSORS = [
        RGBSensorThor(
            height=SCREEN_SIZE, width=SCREEN_SIZE, use_resnet_normalization=True,
        ),
        GoalObjectTypeThorSensor(object_types=OBJECT_TYPES),
        ExpertActionSensor(nactions=6), # Notice that we have added an expert action sensor.
    ]
    ...
    @classmethod
    def training_pipeline(cls, **kwargs):
        dagger_steps = int(1e4) # Much smaller number of steps as we're using imitation learning
        ppo_steps = int(1e6)
        lr = 2.5e-4
        num_mini_batch = 1 if not torch.cuda.is_available() else 6
        update_repeats = 4
        num_steps = 128
        metric_accumulate_interval = cls.MAX_STEPS * 10  # Log every 10 max length tasks
        save_interval = 10000
        gamma = 0.99
        use_gae = True
        gae_lambda = 1.0
        max_grad_norm = 0.5

        return TrainingPipeline(
            save_interval=save_interval,
            metric_accumulate_interval=metric_accumulate_interval,
            optimizer_builder=Builder(optim.Adam, dict(lr=lr)),
            num_mini_batch=num_mini_batch,
            update_repeats=update_repeats,
            max_grad_norm=max_grad_norm,
            num_steps=num_steps,
            named_losses={
                "ppo_loss": PPO(clip_decay=LinearDecay(ppo_steps), **PPOConfig),
                "imitation_loss": Imitation(), # We add an imitation loss.
            },
            gamma=gamma,
            use_gae=use_gae,
            gae_lambda=gae_lambda,
            advance_scene_rollout_period=cls.ADVANCE_SCENE_ROLLOUT_PERIOD,
            pipeline_stages=[
                PipelineStage(
                    loss_names=["imitation_loss"],
                    teacher_forcing=LinearDecay(
                        startp=1.0, endp=0.0, steps=dagger_steps,
                    ),
                    max_stage_steps=dagger_steps,
                ),
                PipelineStage(loss_names=["ppo_loss"], max_stage_steps=ppo_steps,),
            ],
            lr_scheduler_builder=Builder(
                LambdaLR, {"lr_lambda": LinearDecay(steps=ppo_steps)}
            ),
        )

```

--------------------------------

### Install Latest AllenAct Framework from GitHub

Source: https://github.com/allenai/allenact/blob/main/docs/installation/installation-allenact.md

Installs the most recent development version of the AllenAct framework directly from GitHub. This may be unstable. Use 'pip3' if 'pip' is not aliased.

```bash
pip install -e "git+https://github.com/allenai/allenact.git@main#egg=allenact&subdirectory=allenact"
```

--------------------------------

### Install AllenAct Standalone Framework

Source: https://github.com/allenai/allenact/blob/main/docs/installation/installation-allenact.md

Use this command to install the core AllenAct library without any plugins. Ensure you are in a Python virtual environment.

```bash
pip install allenact
```

--------------------------------

### Install All AllenAct Plugins from GitHub

Source: https://github.com/allenai/allenact/blob/main/docs/installation/installation-allenact.md

Installs all available AllenAct plugins from GitHub. This command fetches the latest development version and may be unstable. Use 'pip3' if 'pip' is not aliased.

```bash
pip install -e "git+https://github.com/allenai/allenact.git@main#egg=allenact_plugins[all]&subdirectory=allenact_plugins"
```

--------------------------------

### Install AllenAct Requirements with Pip

Source: https://github.com/allenai/allenact/blob/main/docs/installation/installation-allenact.md

Installs the core requirements for AllenAct using the 'requirements.txt' file. This command should be run after cloning the repository.

```bash
pip install -r requirements.txt; pip install -r dev_requirements.txt
```

--------------------------------

### Configure Headless RoboTHOR Environment

Source: https://github.com/allenai/allenact/blob/main/docs/tutorials/distributed-objectnav-tutorial.md

This bash script prepares a virtual environment, installs AllenAct and its RoboTHOR plugin dependencies, downloads datasets, and installs the headless AI2-THOR environment. It assumes superuser privileges for some installations.

```bash
#!/bin/bash

# Prepare a virtualenv for allenact
sudo apt-get install -y python3-venv
python3 -mvenv ~/allenact_venv
source ~/allenact_venv/bin/activate
pip install -U pip wheel

# Install AllenAct
cd ~
git clone https://github.com/allenai/allenact.git
cd allenact

# Install AllenaAct + RoboTHOR plugin dependencies
pip install -r requirements.txt
pip install -r allenact_plugins/robothor_plugin/extra_requirements.txt

# Download + setup datasets
bash datasets/download_navigation_datasets.sh robothor-objectnav

# Install headless AI2-THOR and required libvulkan1
sudo apt-get install -y libvulkan1
pip install --extra-index-url https://ai2thor-pypi.allenai.org ai2thor==0+91139c909576f3bf95a187c5b02c6fd455d06b48

# Download AI2-THOR binaries
python -c "from ai2thor.controller import Controller; c=Controller(); c.stop()"

echo DONE
```

--------------------------------

### Configure Machine Parameters for Training and Testing

Source: https://github.com/allenai/allenact/blob/main/docs/tutorials/gym-tutorial.md

Defines machine parameters such as the number of processes, devices, and visualizers based on the operating mode. Includes setup for video visualization during testing.

```python
    @classmethod
    def machine_params(cls, mode="train", **kwargs) -> Dict[str, Any]:
        visualizer = None
        if mode == "test":
            visualizer = VizSuite(
                mode=mode,
                video_viz=AgentViewViz(
                    label="episode_vid",
                    max_clip_length=400,
                    vector_task_source=("render", {"mode": "rgb_array"}),
                    fps=30,
                ),
            )
        return {
            "nprocesses": 8 if mode == "train" else 1,
            "devices": [],
            "visualizer": visualizer,
        }
```

--------------------------------

### Install AllenAct with Specific Plugins

Source: https://github.com/allenai/allenact/blob/main/docs/installation/installation-allenact.md

Installs the AllenAct library along with requirements for specified plugins, such as 'ithor' and 'minigrid'. Replace with your desired plugin names.

```bash
pip install allenact allenact_plugins[ithor,minigrid]
```

--------------------------------

### Example AllenAct Training Command

Source: https://github.com/allenai/allenact/blob/main/docs/tutorials/training-a-pointnav-model.md

An example command for training a PointNav model with AllenAct, specifying output storage and base experiment directory. This command assumes a specific experiment configuration.

```bash
PYTHONPATH=. python allenact/main.py training_a_pointnav_model -o storage/robothor-pointnav-rgb-resnet-resnet -b projects/tutorials
```

--------------------------------

### Train a PointNav baseline model

Source: https://github.com/allenai/allenact/blob/main/docs/projects/pointnav_baselines/README.md

Execute the training command from the allenact root directory to start the reinforcement learning process.

```bash
python main.py -o <PATH_TO_OUTPUT> -c -b <BASE_DIRECTORY_OF_YOUR_EXPERIMENT> <EXPERIMENT_NAME>
```

```bash
python main.py -o storage/pointnav-robothor-depth -b projects/pointnav_baselines/experiments/robothor/ pointnav_robothor_depth_simpleconvgru_ddppo
```

--------------------------------

### Run AllenAct Training

Source: https://github.com/allenai/allenact/blob/main/docs/tutorials/training-a-pointnav-model.md

Command to start training a model using AllenAct. Requires specifying an output path, base directory for experiments, and the experiment name. The PYTHONPATH should be set to the project root.

```bash
PYTHONPATH=. python allenact/main.py -o <PATH_TO_OUTPUT> -c -b <BASE_DIRECTORY_OF_YOUR_EXPERIMENT> <EXPERIMENT_NAME>
```

--------------------------------

### Check Distributed Installation Status

Source: https://github.com/allenai/allenact/blob/main/docs/tutorials/distributed-objectnav-tutorial.md

Use the dcommand.py script to execute a command on remote nodes and check the status of the AllenAct distributed configuration. This example tails the last 5 lines of the log file.

```bash
scripts/dcommand.py --runs_on <COMMA_SEPARATED_LIST_OF_IP_ADDRESSES> \
--command 'tail -n 5 ~/log_allenact_distributed_config'
```

--------------------------------

### Download expert demonstrations

Source: https://github.com/allenai/allenact/blob/main/docs/tutorials/offpolicy-tutorial.md

Download the GoToLocal expert demonstration dataset.

```bash
PYTHONPATH=. python allenact_plugins/babyai_plugin/scripts/download_babyai_expert_demos.py GoToLocal
```

--------------------------------

### Install RoboTHOR plugin requirements

Source: https://github.com/allenai/allenact/blob/main/docs/tutorials/running-inference-on-a-pretrained-model.md

Install the necessary requirements for the robothor_plugin before running inference.

```bash
pip install -r allenact_plugins/robothor_plugin/extra_requirements.txt
```

--------------------------------

### Build documentation locally

Source: https://github.com/allenai/allenact/blob/main/docs/FAQ.md

Generates HTML documentation in the site folder.

```bash
mkdocs build
```

--------------------------------

### Install Plugin Extra Requirements

Source: https://github.com/allenai/allenact/blob/main/docs/installation/installation-allenact.md

Installs additional requirements specific to a particular AllenAct plugin. Replace '<PLUGIN_NAME>' with the actual plugin name.

```bash
pip install -r allenact_plugins/<PLUGIN_NAME>_plugin/extra_requirements.txt
```

--------------------------------

### Run Training with Custom Configuration

Source: https://context7.com/allenai/allenact/llms.txt

Initiates training with custom configuration parameters, including learning rate and number of processes. Additional tags can be applied.

```bash
python -m allenact.main projects/objectnav_baselines/experiments/robothor/objectnav_robothor_rgb_resnetgru_ddppo \
    --output_dir output/objectnav \
    --seed 42 \
    --extra_tag experiment_v1 \
    --config_kwargs '{"lr": 0.0003, "num_train_processes": 60}'
```

--------------------------------

### Build documentation files

Source: https://github.com/allenai/allenact/blob/main/docs/FAQ.md

Prepares markdown files in the docs directory before running documentation generators.

```bash
bash scripts/build_docs.sh
```

--------------------------------

### Monitor training with Tensorboard

Source: https://github.com/allenai/allenact/blob/main/docs/getting_started/running-your-first-experiment.md

Launch the Tensorboard server to visualize logs generated during the training process.

```bash
tensorboard --logdir experiment_output/minigrid/tb
```

--------------------------------

### Monitor Training with Tensorboard

Source: https://github.com/allenai/allenact/blob/main/docs/tutorials/gym-tutorial.md

Command to launch Tensorboard for monitoring training progress. Point it to the output directory where logs are saved.

```bash
tensorboard --logdir /PATH/TO/gym_output
```

--------------------------------

### Create a TrainingPipeline with PPO and Imitation Learning

Source: https://context7.com/allenai/allenact/llms.txt

Defines a training pipeline with optional teacher forcing stages and PPO loss configuration.

```python
from typing import Dict, Union, Optional, Sequence
import torch.optim as optim
from torch.optim.lr_scheduler import LambdaLR
from allenact.utils.experiment_utils import (
    TrainingPipeline, Builder, PipelineStage, StageComponent,
    TrainingSettings, LinearDecay
)
from allenact.algorithms.onpolicy_sync.losses.ppo import PPO
from allenact.algorithms.onpolicy_sync.losses.imitation import Imitation
from allenact.algorithms.onpolicy_sync.storage import RolloutBlockStorage

def create_training_pipeline(
    total_steps: int = 1000000,
    ppo_clip: float = 0.1,
    value_loss_coef: float = 0.5,
    entropy_coef: float = 0.01,
    learning_rate: float = 3e-4,
    num_mini_batch: int = 4,
    update_repeats: int = 3,
    num_steps: int = 128,
    use_teacher_forcing: bool = True,
) -> TrainingPipeline:
    """Create a training pipeline with PPO and optional imitation learning."""

    # Define losses
    named_losses: Dict[str, Union[PPO, Imitation]] = {
        "ppo_loss": PPO(
            clip_param=ppo_clip,
            value_loss_coef=value_loss_coef,
            entropy_coef=entropy_coef,
            use_clipped_value_loss=True,
            normalize_advantage=True,
        ),
    }

    # Pipeline stages
    pipeline_stages = []

    if use_teacher_forcing:
        # Stage 1: DAgger with decaying teacher forcing
        named_losses["imitation_loss"] = Imitation()
        pipeline_stages.append(
            PipelineStage(
                loss_names=["imitation_loss", "ppo_loss"],
                loss_weights=[1.0, 1.0],
                max_stage_steps=total_steps // 2,
                teacher_forcing=LinearDecay(
                    steps=total_steps // 2,
                    startp=1.0,
                    endp=0.0,
                ),
            )
        )
        # Stage 2: Pure PPO
        pipeline_stages.append(
            PipelineStage(
                loss_names=["ppo_loss"],
                max_stage_steps=total_steps // 2,
            )
        )
    else:
        # Single stage PPO
        pipeline_stages.append(
            PipelineStage(
                loss_names=["ppo_loss"],
                max_stage_steps=total_steps,
            )
        )

    return TrainingPipeline(
        named_losses=named_losses,
        pipeline_stages=pipeline_stages,
        optimizer_builder=Builder(
            optim.Adam,
            dict(lr=learning_rate),
        ),
        lr_scheduler_builder=Builder(
            LambdaLR,
            {"lr_lambda": LinearDecay(steps=total_steps)},
        ),
        num_mini_batch=num_mini_batch,
        update_repeats=update_repeats,
        max_grad_norm=0.5,
        num_steps=num_steps,
        gamma=0.99,
        use_gae=True,
        gae_lambda=0.95,
        save_interval=100000,
        metric_accumulate_interval=10000,
        should_log=True,
    )
```

--------------------------------

### Configure Habitat Simulator

Source: https://github.com/allenai/allenact/blob/main/docs/tutorials/transfering-to-a-different-environment-framework.md

Set up the Habitat simulator configuration object, including dataset paths, sensor settings, and task parameters.

```python
    CONFIG = get_habitat_config("configs/gibson.yaml")
    CONFIG.defrost()
    CONFIG.NUM_PROCESSES = NUM_PROCESSES
    CONFIG.SIMULATOR_GPU_IDS = TRAIN_GPUS
    CONFIG.DATASET.SCENES_DIR = HABITAT_SCENE_DATASETS_DIR
    CONFIG.DATASET.POINTNAVV1.CONTENT_SCENES = ["*"]
    CONFIG.DATASET.DATA_PATH = TRAIN_SCENES
    CONFIG.SIMULATOR.AGENT_0.SENSORS = ["RGB_SENSOR"]
    CONFIG.SIMULATOR.RGB_SENSOR.WIDTH = CAMERA_WIDTH
    CONFIG.SIMULATOR.RGB_SENSOR.HEIGHT = CAMERA_HEIGHT
    CONFIG.SIMULATOR.TURN_ANGLE = 30
    CONFIG.SIMULATOR.FORWARD_STEP_SIZE = 0.25
    CONFIG.ENVIRONMENT.MAX_EPISODE_STEPS = MAX_STEPS

    CONFIG.TASK.TYPE = "Nav-v0"
    CONFIG.TASK.SUCCESS_DISTANCE = 0.2
    CONFIG.TASK.SENSORS = ["POINTGOAL_WITH_GPS_COMPASS_SENSOR"]
    CONFIG.TASK.POINTGOAL_WITH_GPS_COMPASS_SENSOR.GOAL_FORMAT = "POLAR"
    CONFIG.TASK.POINTGOAL_WITH_GPS_COMPASS_SENSOR.DIMENSIONALITY = 2
    CONFIG.TASK.GOAL_SENSOR_UUID = "pointgoal_with_gps_compass"
    CONFIG.TASK.MEASUREMENTS = ["DISTANCE_TO_GOAL", "SUCCESS", "SPL"]
    CONFIG.TASK.SPL.TYPE = "SPL"
    CONFIG.TASK.SPL.SUCCESS_DISTANCE = 0.2
    CONFIG.TASK.SUCCESS.SUCCESS_DISTANCE = 0.2

    CONFIG.MODE = "train"
```

--------------------------------

### Configure Machine Parameters

Source: https://github.com/allenai/allenact/blob/main/docs/tutorials/gym-mujoco-tutorial.md

Sets up process counts, device allocation, and visualizers based on the execution mode.

```python
    @classmethod
    def machine_params(cls, mode="train", **kwargs) -> Dict[str, Any]:
        visualizer = None
        if mode == "test":
            visualizer = VizSuite(
                mode=mode,
                video_viz=AgentViewViz(
                    label="episode_vid",
                    max_clip_length=400,
                    vector_task_source=("render", {"mode": "rgb_array"}),
                    fps=30,
                ),
            )
        return {
            "nprocesses": 8 if mode == "train" else 1,  # rollout
            "devices": [],
            "visualizer": visualizer,
        }
```

--------------------------------

### Install pre-commit hooks

Source: https://github.com/allenai/allenact/blob/main/CONTRIBUTING.md

Configures Git hooks to automatically validate code formatting and type-checking during the commit process.

```bash
pre-commit install
```

--------------------------------

### Execute Distributed Experiments with dmain.py

Source: https://github.com/allenai/allenact/blob/main/docs/tutorials/distributed-objectnav-tutorial.md

Use this script to synchronize code, activate environments, and launch training across multiple nodes. Ensure the machine hosting the first IP has a free port available for coordination.

```bash
scripts/dmain.py projects/tutorials/distributed_objectnav_tutorial.py \
--config_kwargs '{"distributed_nodes":3}' \
--runs_on <COMMA_SEPARATED_LIST_OF_IP_ADDRESSES> \
--env_activate_path ~/allenact_venv/bin/activate \
--allenact_path ~/allenact \
--distributed_ip_and_port <FIRST_IP_ADDRESS_IN_RUNS_ON_LIST>:<FREE_PORT_NUMBER_FOR_THIS_IP_ADDRESS>
```

--------------------------------

### Configure Environment Arguments

Source: https://github.com/allenai/allenact/blob/main/docs/tutorials/training-a-pointnav-model.md

Set simulator settings such as camera dimensions, rotation step degrees, and grid size for agent movement.

```python
ENV_ARGS = dict(
    width=CAMERA_WIDTH,
    height=CAMERA_HEIGHT,
    rotateStepDegrees=30.0,
    visibilityDistance=1.0,
    gridSize=0.25,
)
```

--------------------------------

### Configure PPO Loss

Source: https://context7.com/allenai/allenact/llms.txt

Demonstrates initializing PPO loss with default settings or custom hyperparameters including decay functions.

```python
from typing import Dict, Optional, Callable, Tuple, cast
import torch
from allenact.algorithms.onpolicy_sync.losses.ppo import PPO, PPOConfig
from allenact.algorithms.onpolicy_sync.losses.abstract_loss import AbstractActorCriticLoss
from allenact.base_abstractions.misc import ActorCriticOutput

# Using default PPO configuration
ppo_loss = PPO(**PPOConfig)  # clip_param=0.1, value_loss_coef=0.5, entropy_coef=0.01

# Custom PPO configuration
custom_ppo_loss = PPO(
    clip_param=0.2,
    value_loss_coef=0.5,
    entropy_coef=0.01,
    use_clipped_value_loss=True,
    clip_decay=lambda step: max(0.1, 1.0 - step / 1000000),  # Decay clip param
    normalize_advantage=True,
    show_ratios=True,  # Log PPO ratios for debugging
)
```

--------------------------------

### Configure Single-Stage PPO Training Pipeline

Source: https://github.com/allenai/allenact/blob/main/docs/howtos/defining-an-experiment.md

Use this configuration for a straightforward training pipeline with PPO. It sets up the optimizer, loss function, and learning rate scheduler.

```python
class ObjectNavThorPPOExperimentConfig(ExperimentConfig):
    ...
    @classmethod
    def training_pipeline(cls, **kwargs):
        ppo_steps = int(1e6)
        lr = 2.5e-4
        num_mini_batch = 2 if not torch.cuda.is_available() else 6
        update_repeats = 4
        num_steps = 128
        metric_accumulate_interval = cls.MAX_STEPS * 10  # Log every 10 max length tasks
        save_interval = 10000
        gamma = 0.99
        use_gae = True
        gae_lambda = 1.0
        max_grad_norm = 0.5

        return TrainingPipeline(
            save_interval=save_interval,
            metric_accumulate_interval=metric_accumulate_interval,
            optimizer_builder=Builder(optim.Adam, dict(lr=lr)),
            num_mini_batch=num_mini_batch,
            update_repeats=update_repeats,
            max_grad_norm=max_grad_norm,
            num_steps=num_steps,
            named_losses={
                "ppo_loss": PPO(clip_decay=LinearDecay(ppo_steps), **PPOConfig),
            },
            gamma=gamma,
            use_gae=use_gae,
            gae_lambda=gae_lambda,
            advance_scene_rollout_period=cls.ADVANCE_SCENE_ROLLOUT_PERIOD,
            pipeline_stages=[
                PipelineStage(loss_names=["ppo_loss"], max_stage_steps=ppo_steps,),
            ],
            lr_scheduler_builder=Builder(
                LambdaLR, {"lr_lambda": LinearDecay(steps=ppo_steps)}
            ),
        )
    ...
```

--------------------------------

### Configure Environment Arguments

Source: https://github.com/allenai/allenact/blob/main/docs/tutorials/training-a-pointnav-model.md

Updates environment arguments, specifically setting the X display for a given process index and available devices. This is useful for distributed training setups.

```python
res["env_args"].update(self.ENV_ARGS)
res["env_args"]["x_display"] = (
    ("0.%d" % devices[process_ind % len(devices)])
    if devices is not None and len(devices) > 0
    else None
)
return res
```

--------------------------------

### Serve static site

Source: https://github.com/allenai/allenact/blob/main/docs/FAQ.md

Serves the built site directory as a static webpage without additional dependencies.

```bash
python -m http.server 8000
```

--------------------------------

### Get Sampler Arguments for Scene Split

Source: https://github.com/allenai/allenact/blob/main/docs/tutorials/training-a-pointnav-model.md

Generates sampler arguments for a given scene split, process index, and total processes. Handles scene loading, distribution, and configuration.

```python
    def _get_sampler_args_for_scene_split(
        self,
        scenes_dir: str,
        process_ind: int,
        total_processes: int,
        seeds: Optional[List[int]] = None,
        deterministic_cudnn: bool = False,
    ) -> Dict[str, Any]:
        path = os.path.join(scenes_dir, "*.json.gz")
        scenes = [scene.split("/")[-1].split(".")[0] for scene in glob.glob(path)]
        if len(scenes) == 0:
            raise RuntimeError(
                (
                    "Could find no scene dataset information in directory {}." 
                    " Are you sure you've downloaded them? "
                    " If not, see https://allenact.org/installation/download-datasets/ information"
                    " on how this can be done."
                ).format(scenes_dir)
            )
        if total_processes > len(scenes):  # oversample some scenes -> bias
            if total_processes % len(scenes) != 0:
                print(
                    "Warning: oversampling some of the scenes to feed all processes."
                    " You can avoid this by setting a number of workers divisible by the number of scenes"
                )
            scenes = scenes * int(ceil(total_processes / len(scenes)))
            scenes = scenes[: total_processes * (len(scenes) // total_processes)]
        else:
            if len(scenes) % total_processes != 0:
                print(
                    "Warning: oversampling some of the scenes to feed all processes."
                    " You can avoid this by setting a number of workers divisor of the number of scenes"
                )
        inds = self._partition_inds(len(scenes), total_processes)

        return {
            "scenes": scenes[inds[process_ind] : inds[process_ind + 1]],
            "max_steps": self.MAX_STEPS,
            "sensors": self.SENSORS,
            "action_space": gym.spaces.Discrete(len(PointNavTask.class_action_names())),
            "seed": seeds[process_ind] if seeds is not None else None,
            "deterministic_cudnn": deterministic_cudnn,
            "rewards_config": self.REWARD_CONFIG,
        }
```

--------------------------------

### Get Sampler Arguments for Task

Source: https://github.com/allenai/allenact/blob/main/docs/tutorials/training-a-pointnav-model.md

Retrieves sampler arguments for a specific scene split, including dataset directory, process index, and total processes. Supports optional seeds and deterministic cuDNN settings.

```python
res = self._get_sampler_args_for_scene_split(
    os.path.join(self.VAL_DATASET_DIR, "episodes"),
    process_ind,
    total_processes,
    seeds=seeds,
    deterministic_cudnn=deterministic_cudnn,
)
res["scene_directory"] = self.VAL_DATASET_DIR
res["loop_dataset"] = False
res["env_args"] = {}
return res
```

--------------------------------

### Execute a MiniGrid experiment

Source: https://github.com/allenai/allenact/blob/main/docs/getting_started/running-your-first-experiment.md

Run the experiment from the allenact root directory using the specified configuration and parameters.

```bash
PYTHONPATH=. python allenact/main.py minigrid_tutorial -b projects/tutorials -m 8 -o experiment_output/minigrid -s 12345
```

--------------------------------

### Update Conda Environment with Plugin Dependencies

Source: https://github.com/allenai/allenact/blob/main/docs/installation/installation-allenact.md

Install specific dependencies for a plugin by updating the existing Conda environment using the plugin's extra environment file. Ensure you are in the top-level directory and replace '<PLUGIN_NAME>' with the actual plugin name.

```bash
conda env update --file allenact_plugins/<PLUGIN_NAME>_plugin/extra_environment.yml --name $MY_ENV_NAME
```

--------------------------------

### Define DAgger then PPO Training Pipeline

Source: https://github.com/allenai/allenact/blob/main/docs/howtos/defining-a-new-training-pipeline.md

Implement a two-stage training pipeline using DAgger for imitation learning followed by PPO. Includes an ExpertActionSensor and adds an imitation loss. Teacher forcing is used in the first stage and decays over time.

```python
class ObjectNavThorDaggerThenPPOExperimentConfig(ExperimentConfig):
    ...
    SENSORS = [
        ...
        ExpertActionSensor(nactions=6), # Notice that we have added
                                        # an expert action sensor.
    ]
    ...
    @classmethod
    def training_pipeline(cls, **kwargs):
        dagger_steps = int(1e4) # Much smaller number of steps as we're using imitation learning
        ppo_steps = int(1e6)
        lr = 2.5e-4
        num_mini_batch = 1 if not torch.cuda.is_available() else 6
        update_repeats = 4
        num_steps = 128
        metric_accumulate_interval = cls.MAX_STEPS * 10  # Log every 10 max length tasks
        save_interval = 10000
        gamma = 0.99
        use_gae = True
        gae_lambda = 1.0
        max_grad_norm = 0.5

        return TrainingPipeline(
            save_interval=save_interval,
            metric_accumulate_interval=metric_accumulate_interval,
            optimizer_builder=Builder(optim.Adam, dict(lr=lr)),
            num_mini_batch=num_mini_batch,
            update_repeats=update_repeats,
            max_grad_norm=max_grad_norm,
            num_steps=num_steps,
            named_losses={
                "ppo_loss": PPO(clip_decay=LinearDecay(ppo_steps), **PPOConfig),
                "imitation_loss": Imitation(), # We add an imitation loss.
            },
            gamma=gamma,
            use_gae=use_gae,
            gae_lambda=gae_lambda,
            advance_scene_rollout_period=cls.ADVANCE_SCENE_ROLLOUT_PERIOD,
            pipeline_stages=[
                PipelineStage(
                    loss_names=["imitation_loss"],
                    teacher_forcing=LinearDecay(
                        startp=1.0, endp=0.0, steps=dagger_steps,
                    ),
                    max_stage_steps=dagger_steps,
                ),
                PipelineStage(loss_names=["ppo_loss"], max_stage_steps=ppo_steps,),
            ],
            lr_scheduler_builder=Builder(
                LambdaLR, {"lr_lambda": LinearDecay(steps=ppo_steps)}
            ),
        )

```

--------------------------------

### Configure Training Pipeline

Source: https://github.com/allenai/allenact/blob/main/docs/tutorials/training-a-pointnav-model.md

Define the training pipeline, specifying the algorithm (PPO), learning rate, training steps, and logging/saving intervals.

```python
@classmethod
def training_pipeline(cls, **kwargs):
    ppo_steps = int(250000000)
    lr = 3e-4
    num_mini_batch = 1
    update_repeats = 3
    num_steps = 30
    save_interval = 5000000
    log_interval = 1000
    gamma = 0.99
    use_gae = True
```

--------------------------------

### Define Experiment Configuration

Source: https://github.com/allenai/allenact/blob/main/docs/tutorials/gym-tutorial.md

Initialize the experiment configuration class for the Gym tutorial.

```python
from typing import Dict, Optional, List, Any, cast

import gym
import torch.nn as nn
import torch.optim as optim
from torch.optim.lr_scheduler import LambdaLR

from allenact.algorithms.onpolicy_sync.losses.ppo import PPO
from allenact.base_abstractions.experiment_config import ExperimentConfig, TaskSampler
from allenact.base_abstractions.sensor import SensorSuite
from allenact_plugins.gym_plugin.gym_models import MemorylessActorCritic
from allenact_plugins.gym_plugin.gym_sensors import GymBox2DSensor
from allenact_plugins.gym_plugin.gym_tasks import GymTaskSampler
from allenact.utils.experiment_utils import (
    TrainingPipeline,
    Builder,
    PipelineStage,
    LinearDecay,
)
from allenact.utils.viz_utils import VizSuite, AgentViewViz


class GymTutorialExperimentConfig(ExperimentConfig):
    @classmethod
    def tag(cls) -> str:
        return "GymTutorial"
```

--------------------------------

### Define training pipeline for PPO

Source: https://github.com/allenai/allenact/blob/main/docs/tutorials/minigrid-tutorial.md

Sets up a PPO training stage with specific hyperparameters, optimizer, and learning rate scheduler using the Builder pattern.

```python
    @classmethod
    def training_pipeline(cls, **kwargs) -> TrainingPipeline:
        ppo_steps = int(150000)
        return TrainingPipeline(
            named_losses=dict(ppo_loss=PPO(**PPOConfig)),  # type:ignore
            pipeline_stages=[
                PipelineStage(loss_names=["ppo_loss"], max_stage_steps=ppo_steps)
            ],
            optimizer_builder=Builder(cast(optim.Optimizer, optim.Adam), dict(lr=1e-4)),
            num_mini_batch=4,
            update_repeats=3,
            max_grad_norm=0.5,
            num_steps=16,
            gamma=0.99,
            use_gae=True,
            gae_lambda=0.95,
            advance_scene_rollout_period=None,
            save_interval=10000,
            metric_accumulate_interval=1,
            lr_scheduler_builder=Builder(
                LambdaLR, {"lr_lambda": LinearDecay(steps=ppo_steps)}  # type:ignore
            ),
        )
```

--------------------------------

### Run Training Experiment

Source: https://github.com/allenai/allenact/blob/main/docs/projects/objectnav_baselines/README.md

Command to run a training experiment. Specify the experiment configuration path and the output directory for logs and model weights.

```bash
python main.py <PATH_TO_EXPERIMENT_CONFIG> -o <PATH_TO_OUTPUT> -c
```

--------------------------------

### Download pre-trained navigation model weights

Source: https://github.com/allenai/allenact/blob/main/docs/tutorials/running-inference-on-a-pretrained-model.md

Download the weights for an RGB model trained on the PointNav task in RoboTHOR.

```bash
bash pretrained_model_ckpts/download_navigation_model_ckpts.sh robothor-pointnav-rgb-resnet
```

--------------------------------

### Train an experiment model

Source: https://github.com/allenai/allenact/blob/main/projects/objectnav_baselines/README.md

Execute training from the allenact root directory using a specified experiment configuration file.

```bash
python main.py <PATH_TO_EXPERIMENT_CONFIG> -o <PATH_TO_OUTPUT> -c
```

```bash
python main.py projects/objectnav_baselines/experiments/robothor/objectnav_robothor_rgb_resnet_ddppo.py -o storage/objectnav-robothor-rgb
```

--------------------------------

### Implement ObjectNaviThorGridTask Rendering, Metrics, and Expert Actions

Source: https://github.com/allenai/allenact/blob/main/docs/howtos/defining-a-new-task.md

Provides methods for rendering the environment, calculating task metrics (success, episode length), and querying expert actions for training.

```python
    def render(self, mode: str = "rgb", *args, **kwargs) -> numpy.ndarray:
        assert mode == "rgb", "only rgb rendering is implemented"
        return self.env.current_frame


    def metrics(self) -> Dict[str, Any]:
        if not self.is_done():
            return {}
        else:
            return {"success": self._success, "ep_length": self.num_steps_taken()}

    def query_expert(self, **kwargs) -> Tuple[int, bool]:
        return my_objnav_expert_implementation(self)
```

--------------------------------

### Sample Next Task

Source: https://github.com/allenai/allenact/blob/main/docs/howtos/defining-a-new-task.md

Samples the next task by selecting a random scene, resetting the environment if necessary, randomizing the agent's location, and creating a new ObjectNaviThorGridTask with a randomly chosen object type.

```python
    def next_task(self) -> Optional[ObjectNaviThorGridTask]:
        self.scene_id = random.randint(0, len(self.scenes) - 1)
        self.scene = self.scenes[self.scene_id]

        if self.env is not None:
            if scene != self.env.scene_name:
                self.env.reset(scene)
        else:
            self.env = self._create_environment()
            self.env.reset(scene_name=scene)

        self.env.randomize_agent_location()

        task_info = {"object_type": random.sample(self.object_types, 1)}

        self._last_sampled_task = ObjectNaviThorGridTask(
            env=self.env,
            sensors=self.sensors,
            task_info=task_info,
            max_steps=self.max_steps,
            action_space=self._action_sapce,
        )
        return self._last_sampled_task
```

--------------------------------

### Define On-Policy PPO Training Pipeline

Source: https://github.com/allenai/allenact/blob/main/docs/howtos/defining-a-new-training-pipeline.md

Implement a single-stage training pipeline using PPO. Configure optimizer, loss functions, and learning rate scheduler with Builder for deferred instantiation.

```python
class ObjectNavThorPPOExperimentConfig(ExperimentConfig):
    ...
    @classmethod
    def training_pipeline(cls, **kwargs):
        ppo_steps = int(1e6)
        lr = 2.5e-4
        num_mini_batch = 2 if not torch.cuda.is_available() else 6
        update_repeats = 4
        num_steps = 128
        metric_accumulate_interval = cls.MAX_STEPS * 10  # Log every 10 max length tasks
        save_interval = 10000
        gamma = 0.99
        use_gae = True
        gae_lambda = 1.0
        max_grad_norm = 0.5

        return TrainingPipeline(
            save_interval=save_interval,
            metric_accumulate_interval=metric_accumulate_interval,
            optimizer_builder=Builder(optim.Adam, dict(lr=lr)),
            num_mini_batch=num_mini_batch,
            update_repeats=update_repeats,
            max_grad_norm=max_grad_norm,
            num_steps=num_steps,
            named_losses={
                "ppo_loss": PPO(clip_decay=LinearDecay(ppo_steps), **PPOConfig),
            },
            gamma=gamma,
            use_gae=use_gae,
            gae_lambda=gae_lambda,
            advance_scene_rollout_period=cls.ADVANCE_SCENE_ROLLOUT_PERIOD,
            pipeline_stages=[
                PipelineStage(loss_names=["ppo_loss"], max_stage_steps=ppo_steps,),
            ],
            lr_scheduler_builder=Builder(
                LambdaLR, {"lr_lambda": LinearDecay(steps=ppo_steps)}
            ),
        )
    ...

```

--------------------------------

### Configure a multi-stage training pipeline

Source: https://github.com/allenai/allenact/blob/main/docs/howtos/changing-rewards-and-losses.md

Defines a training pipeline with multiple stages combining imitation and PPO losses.

```python
class MyExperimentConfig(allenact.base_abstractions.experiment_config.ExperimentConfig):
    ...
    @classmethod
    def training_pipeline(cls, **kwargs):
        dagger_steps = int(3e4)
        ppo_steps = int(3e4)
        ppo_steps2 = int(1e6)
        ...
        return allenact.utils.experiment_utils.TrainingPipeline(
            named_losses={
                "imitation_loss": allenact.algorithms.onpolicy_sync.losses.imitation.Imitation(),
                "ppo_loss": allenact.algorithms.onpolicy_sync.losses.ppo.PPO(
                    **allenact.algorithms.onpolicy_sync.losses.ppo.PPOConfig,
                ),
            },
            ...
            pipeline_stages=[
                allenact.utils.experiment_utils.PipelineStage(
                    loss_names=["imitation_loss", "ppo_loss"],
                    teacher_forcing=allenact.utils.experiment_utils.LinearDecay(
                        startp=1.0, endp=0.0, steps=dagger_steps,
                    ),
                    max_stage_steps=dagger_steps,
                ),
                allenact.utils.experiment_utils.PipelineStage(
                    loss_names=["ppo_loss", "imitation_loss"],
                    max_stage_steps=ppo_steps
                ),
                allenact.utils.experiment_utils.PipelineStage(
                    loss_names=["ppo_loss"],
                    max_stage_steps=ppo_steps2,
                ),
            ],
        )
```

--------------------------------

### Distributed Training (Master)

Source: https://context7.com/allenai/allenact/llms.txt

Initiates distributed training on machine 0, acting as the master node. Requires specifying the IP and port for communication.

```bash
python -m allenact.main projects/tutorials/minigrid_tutorial \
    --distributed_ip_and_port 192.168.1.1:1234 \
    --machine_id 0 \
    --output_dir output/distributed
```