### Install RL Zoo from source

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/install.md

Clone the repository and install in editable mode.

```bash
git clone https://github.com/DLR-RM/rl-baselines3-zoo
cd rl-baselines3-zoo/
pip install -e .
```

--------------------------------

### Full installation with pip install -e .[plots,tests,extras]

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/README.md

Installs the RL Zoo with extra dependencies for plotting, testing, and additional environments.

```bash
pip install -e .[plots,tests,extras]
```

--------------------------------

### Install Sphinx and Theme Dependencies

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/README.md

Install the documentation dependencies and the package in editable mode. Run these commands in the project root before building the docs.

```bash
pip install stable_baselines3[docs]
pip install -e .
```

--------------------------------

### Install from source with pip install -e .

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/README.md

Installs the RL Zoo in editable mode from the source directory, allowing development without reinstallation.

```bash
pip install -e .
```

--------------------------------

### Full installation with apt-get install swig cmake ffmpeg

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/README.md

Installs system dependencies required for building some RL Zoo dependencies, such as Box2D and pybullet.

```bash
apt-get install swig cmake ffmpeg
```

--------------------------------

### Auto-rebuild Documentation with sphinx-autobuild

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/README.md

Start a local server that rebuilds the documentation whenever a file changes. Run this command in the docs/ folder.

```bash
sphinx-autobuild . _build/html
```

--------------------------------

### Install RL Zoo with extra dependencies

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/install.md

Install with plots, tests, and extras. Requires system packages: swig, cmake, ffmpeg.

```bash
apt-get install swig cmake ffmpeg
pip install -e .[plots,tests,extras]
```

--------------------------------

### Run hyperparameter optimization with Optuna (PPO, MountainCar-v0)

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/tuning.md

Starts a hyperparameter search with a budget of 1000 trials and 50000 steps per trial, using TPE sampler and median pruner. Requires --n-jobs > 1 when using SuccessiveHalvingPruner.

```bash
python train.py --algo ppo --env MountainCar-v0 -n 50000 -optimize --n-trials 1000 --n-jobs 2 \
  --sampler tpe --pruner median
```

--------------------------------

### Train PPO on CartPole-v1 with evaluation and checkpoints

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/quickstart.md

Example command to train a PPO agent on CartPole-v1, including evaluation every 10000 steps and saving checkpoints every 50000 steps.

```default
python -m rl_zoo3.train --algo ppo --env CartPole-v1 --eval-freq 10000 --save-freq 50000
```

--------------------------------

### Install RL Zoo with pip

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/install.md

Install the RL Zoo package using pip.

```bash
pip install rl_zoo3
```

--------------------------------

### macOS workaround with uv

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/install.md

Alternative using uv to install all extras with the same CFLAGS workaround.

```bash
CFLAGS="-fno-define-target-os-macros" uv sync --all-extras
```

--------------------------------

### Enjoy A2C on Breakout for 5000 timesteps

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/quickstart.md

Example command to run a trained A2C agent on BreakoutNoFrameskip-v4 for 5000 timesteps, loading from the rl-trained-agents folder.

```default
python -m rl_zoo3.enjoy --algo a2c --env BreakoutNoFrameskip-v4 --folder rl-trained-agents/ -n 5000
```

--------------------------------

### Run tests with pytest

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/README.md

Run the test suite for the RL Baselines3 Zoo. First install pytest, then execute this command to run all tests.

```bash
make pytest
```

--------------------------------

### Train an agent with RL Zoo3 (installed package)

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/quickstart.md

Use this command to train an agent with the specified algorithm and environment. The hyperparameters are read from the corresponding YAML file in the hyperparameters directory.

```default
python -m rl_zoo3.train --algo algo_name --env env_id
```

--------------------------------

### Track experiment with Weights and Biases

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/integrations.md

Starts a training run with experiment tracking enabled via Weights and Biases. The --track flag enables tracking and --wandb-project-name sets the project name. Add --wandb-tags optimized to tag the run.

```default
python train.py --algo ppo --env CartPole-v1 --track --wandb-project-name sb3
```

--------------------------------

### Run type checking with pytype

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/README.md

Run type checking for the RL Baselines3 Zoo. This command is used after installing the necessary dependencies, and it checks the code for type errors.

```bash
make type
```

--------------------------------

### Build HTML Documentation with make

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/README.md

Build the HTML documentation from the docs/ folder. Run this command inside the docs/ directory.

```bash
make html
```

--------------------------------

### Visualize live with optuna-dashboard

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/tuning.md

Launches the optuna-dashboard to visualize ongoing or completed optimization runs stored in the given log file.

```bash
optuna-dashboard logs/demo.log
```

--------------------------------

### Create enjoy_sbx.py to register SBX algorithms and enjoy

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/sbx.md

This file registers SBX algorithms (DQN, PPO, SAC, TQC, DroQ) with the RL Zoo and runs the enjoy CLI to load and run a trained agent.

```python
import rl_zoo3
import rl_zoo3.enjoy
from rl_zoo3.enjoy import enjoy
from sbx import DQN, PPO, SAC, TQC, DroQ


rl_zoo3.ALGOS["tqc"] = TQC
rl_zoo3.ALGOS["droq"] = DroQ
rl_zoo3.ALGOS["sac"] = SAC
rl_zoo3.ALGOS["ppo"] = PPO
rl_zoo3.ALGOS["dqn"] = DQN
rl_zoo3.enjoy.ALGOS = rl_zoo3.ALGOS
rl_zoo3.exp_manager.ALGOS = rl_zoo3.ALGOS

if __name__ == "__main__":
    enjoy()
```

--------------------------------

### Build Docker image (CPU)

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/install.md

Build the CPU Docker image using make.

```default
make docker-cpu
```

--------------------------------

### Create train_sbx.py to register SBX algorithms and train

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/sbx.md

This file registers SBX algorithms (DQN, PPO, SAC, TQC, DroQ) with the RL Zoo and runs the training CLI. After creating it, you can call `python train_sbx.py --algo sac --env Pendulum-v1` to use the RL Zoo CLI.

```python
import rl_zoo3
import rl_zoo3.train
from rl_zoo3.train import train
from sbx import DQN, PPO, SAC, TQC, DroQ


rl_zoo3.ALGOS["tqc"] = TQC
rl_zoo3.ALGOS["droq"] = DroQ
rl_zoo3.ALGOS["sac"] = SAC
rl_zoo3.ALGOS["ppo"] = PPO
rl_zoo3.ALGOS["dqn"] = DQN
rl_zoo3.train.ALGOS = rl_zoo3.ALGOS
rl_zoo3.exp_manager.ALGOS = rl_zoo3.ALGOS

if __name__ == "__main__":
    train()
```

--------------------------------

### Resume training

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/train.md

Continue training from a pretrained agent. Here, load a pretrained agent for Breakout and continue training for 5000 steps.

```default
python train.py --algo a2c --env BreakoutNoFrameskip-v4 -i rl-trained-agents/a2c/BreakoutNoFrameskip-v4_1/BreakoutNoFrameskip-v4.zip -n 5000
```

--------------------------------

### Build Docker image (GPU)

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/install.md

Build the GPU Docker image using make with USE_GPU=True.

```default
USE_GPU=True make docker-gpu
```

--------------------------------

### Multiple environment wrappers with parameters

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/config.md

Specifies multiple environment wrappers as a list, with parameters for one of them.

```yaml
env_wrapper:
    - rl_zoo3.wrappers.TruncatedOnSuccessWrapper:
        reward_offset: 1.0
    - sb3_contrib.common.wrappers.TimeFeatureWrapper
```

--------------------------------

### Enjoy a trained agent with RL Zoo3

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/quickstart.md

Use this command to run a trained agent with the specified algorithm and environment.

```default
python -m rl_zoo3.enjoy --algo algo_name --env env_id
```

--------------------------------

### Record a video of a training experiment

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/enjoy.md

Record 1000 steps for each checkpoint, latest and best saved models using rl_zoo3.record_training. The command creates an mp4 file.

```default
python -m rl_zoo3.record_training --algo ppo --env CartPole-v1 -n 1000 -f logs --deterministic
```

--------------------------------

### Enjoy a trained agent with python enjoy.py

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/enjoy.md

Run a trained agent using the enjoy.py script. Replace algo_name and env_id with your algorithm and environment. The --folder option specifies the directory containing trained agents.

```default
python enjoy.py --algo algo_name --env env_id
```

--------------------------------

### Download model from Huggingface Hub

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/integrations.md

Downloads a trained model from the Huggingface Hub. The command mirrors the upload syntax and requires the same arguments for algorithm, environment, logs folder, and organization.

```default
python -m rl_zoo3.load_from_hub --algo ppo --env CartPole-v1 -f logs/ -orga sb3
```

--------------------------------

### Basic training command

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/train.md

Train an agent with a specified algorithm and environment. The hyperparameters are defined in `hyperparameters/algo_name.yml`. Use `-P` or `--progress` to display a progress bar.

```default
python train.py --algo algo_name --env env_id
```

--------------------------------

### Record a video with the best saved model

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/enjoy.md

Record a video using the best saved model by adding --load-best to the record_video command.

```default
python -m rl_zoo3.record_video --algo ppo --env BipedalWalkerHardcore-v3 -n 1000 --load-best
```

--------------------------------

### Distributed optimization with shared database (Optuna)

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/tuning.md

Runs distributed hyperparameter optimization using a shared Optuna storage. The study name and storage path are specified for coordination across processes.

```bash
python train.py --algo ppo --env MountainCar-v0 -optimize --study-name test --storage logs/demo.log
```

--------------------------------

### Monitor kwargs for success logging

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/config.md

Configures Monitor wrapper to log additional info keywords, e.g., for success with goal envs like FetchReach-v1.

```yaml
monitor_kwargs: dict(info_keywords=('is_success',))
```

--------------------------------

### Train SAC agent with evaluation every 10000 steps

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/README.md

Trains a SAC agent on HalfCheetahBulletEnv-v0, evaluating every 10000 steps using 10 episodes and a single evaluation environment.

```bash
python train.py --algo sac --env HalfCheetahBulletEnv-v0 --eval-freq 10000 --eval-episodes 10 --n-eval-envs 1
```

--------------------------------

### Load best model with --load-best

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/enjoy.md

Load the best model (when using evaluation environment) by adding --load-best to the enjoy command.

```default
python enjoy.py --algo algo_name --env env_id -f logs/ --exp-id 1 --load-best
```

--------------------------------

### Tensorboard logging

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/train.md

Train with Tensorboard support by specifying a log directory with `--tensorboard-log`.

```default
python train.py --algo ppo --env CartPole-v1 --tensorboard-log /tmp/stable-baselines/
```

--------------------------------

### Evaluate agent during training

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/train.md

Evaluate the agent every 10000 steps using 10 episodes for evaluation, using only one evaluation environment.

```default
python train.py --algo sac --env AntBulletEnv-v0 --eval-freq 10000 --eval-episodes 10 --n-eval-envs 1
```

--------------------------------

### Record a video with the latest saved model

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/enjoy.md

Record 1000 steps with the latest saved model using rl_zoo3.record_video. Replace the algorithm and environment as needed.

```default
python -m rl_zoo3.record_video --algo ppo --env BipedalWalkerHardcore-v3 -n 1000
```

--------------------------------

### Enjoy a self-trained agent with exp-id

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/enjoy.md

Command to enjoy an agent you trained yourself. The --exp-id 0 corresponds to the last experiment; specify another ID to select a different experiment. Use -f logs/ to point to the logs folder.

```default
# exp-id 0 corresponds to the last experiment, otherwise, you can specify another ID
python enjoy.py --algo algo_name --env env_id -f logs/ --exp-id 0
```

--------------------------------

### Plot with rliable and IQM

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/plot.md

Uses `plot_from_file.py` with `--rliable`, `--versus`, and `--iqm` arguments to plot results from a previously exported pickle file. Note that plotting with `--rliable` is usually slow due to bootstrap sampling for confidence intervals. May require editing `plot_from_file.py` and `scripts/score_normalization.py`.

```default
python scripts/plot_from_file.py -i logs/offpolicy.pkl --skip-timesteps --rliable --versus -l SAC TD3 TQC
```

--------------------------------

### Custom callback with parameters

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/config.md

Adds a custom callback with parameters, following the same syntax as env wrappers.

```yaml
callback:
  - rl_zoo3.callbacks.ParallelTrainCallback:
      gradient_steps: 256
```

--------------------------------

### Train an agent in an interactive session by modifying sys.argv

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/README.md

Use this snippet to run the RL Baselines3 Zoo training from an interactive session like Jupyter Notebook or IPython. It sets sys.argv to pass the algorithm and environment arguments, then calls the train function. This is necessary because the zoo is not meant to be executed directly from an interactive session.

```python
import sys
from rl_zoo3.train import train

sys.argv = ["python", "--algo", "ppo", "--env", "MountainCar-v0"]

train()
```

--------------------------------

### Run script in Docker image

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/install.md

Run a training script inside the Docker container using the provided script.

```default
./scripts/run_docker_cpu.sh python train.py --algo ppo --env CartPole-v1
```

--------------------------------

### Custom Python config file

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/train.md

Use a Python file containing a dictionary `hyperparams` with an entry for each `env_id`. You can pass a path to a Python file or a module path. This allows arbitrary Python dictionaries and ensures dependencies are imported.

```default
# You can pass a path to a python file
python train.py --algo ppo --env MountainCarContinuous-v0 --conf-file hyperparams/python/ppo_config_example.py
# Or pass a path to a file from a module (for instance my_package.my_file)
python train.py --algo ppo --env MountainCarContinuous-v0 --conf-file hyperparams.python.ppo_config_example
```

--------------------------------

### Load a checkpoint with --load-checkpoint

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/enjoy.md

Load a specific checkpoint by providing the step number. Here the checkpoint name is rl_model_10000_steps.zip, so use --load-checkpoint 10000.

```default
python enjoy.py --algo algo_name --env env_id -f logs/ --exp-id 1 --load-checkpoint 10000
```

--------------------------------

### Upload model to Huggingface Hub

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/integrations.md

Uploads a trained model to the Huggingface Hub using the same syntax as enjoy.py. The default repository name is {algo}-{env_id}, but you can override it with --repo-name.

```default
python -m rl_zoo3.push_to_hub --algo ppo --env CartPole-v1 -f logs/ -orga sb3 -m "Initial commit"
```

--------------------------------

### Monitor kwargs for x_position logging

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/config.md

Configures Monitor wrapper to log the final x position, e.g., for Ant-v3.

```yaml
monitor_kwargs: dict(info_keywords=('x_position',))
```

--------------------------------

### Env keyword arguments

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/train.md

Pass keyword arguments to the environment constructor using `--env-kwargs`.

```default
python enjoy.py --algo ppo --env MountainCar-v0 --env-kwargs goal_velocity:10
```

--------------------------------

### Record a video of a checkpoint

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/enjoy.md

Record a video of a checkpoint saved during training. Here the checkpoint name is rl_model_10000_steps.zip, so use --load-checkpoint 10000.

```default
python -m rl_zoo3.record_video --algo ppo --env BipedalWalkerHardcore-v3 -n 1000 --load-checkpoint 10000
```

--------------------------------

### Convert recorded video to gif

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/enjoy.md

Add --gif to the record_training command to also convert the mp4 file to gif format.

```default
python -m rl_zoo3.record_training --algo ppo --env CartPole-v1 -n 1000 -f logs --deterministic --gif
```

--------------------------------

### Save replay buffer

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/train.md

When using off-policy algorithms, save the replay buffer after training with `--save-replay-buffer`. It will be automatically loaded if present when continuing training.

```default
python train.py --algo sac --env Pendulum-v1 --save-replay-buffer
```

--------------------------------

### Single environment wrapper

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/config.md

Specifies a single environment wrapper to use around the environment.

```yaml
env_wrapper: gym_minigrid.wrappers.FlatObsWrapper
```

--------------------------------

### Environment keyword arguments

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/config.md

Specifies environment keyword arguments, such as setting gravity to 0.0.

```yaml
env_kwargs:
  gravity: 0.0
```

--------------------------------

### Load hyperparameters from a specific trial (trial-id 21)

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/tuning.md

Trains an agent using hyperparameters from trial number 21 of the specified study and storage. Useful for reproducing or continuing experiments.

```bash
python train.py --algo ppo --env MountainCar-v0 --study-name test --storage logs/demo.log --trial-id 21
```

--------------------------------

### Custom YAML config file

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/train.md

Use a custom YAML config file that contains an `env_id` entry. The file path is passed with `--conf-file`.

```default
python train.py --algo algo_name --env env_id --conf-file my_yaml.yml
```

--------------------------------

### Plot evaluation reward curves for multiple algorithms

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/plot.md

Plots evaluation reward curves for TQC, SAC, and TD3 on the HalfCheetah and Ant PyBullet environments using `scripts/all_plots.py` with arguments `-a sac td3 tqc --env HalfCheetahBullet AntBullet -f rl-trained-agents/`.

```default
python3 scripts/all_plots.py -a sac td3 tqc --env HalfCheetahBullet AntBullet -f rl-trained-agents/
```

--------------------------------

### Load the latest checkpoint with --load-last-checkpoint

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/enjoy.md

Load the most recent checkpoint saved during training by adding --load-last-checkpoint.

```default
python enjoy.py --algo algo_name --env env_id -f logs/ --exp-id 1 --load-last-checkpoint
```

--------------------------------

### Pull CPU Docker image

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/install.md

Pull the pre-built CPU Docker image from Docker Hub.

```default
docker pull stablebaselines/rl-baselines3-zoo-cpu
```

--------------------------------

### Save checkpoints

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/train.md

Save a checkpoint of the agent every 100000 steps using `--save-freq`.

```default
python train.py --algo td3 --env AntBulletEnv-v0 --save-freq 100000
```

--------------------------------

### Pull GPU Docker image

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/install.md

Pull the pre-built GPU Docker image from Docker Hub.

```default
docker pull stablebaselines/rl-baselines3-zoo
```

--------------------------------

### Plot training success with moving window

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/plot.md

Plots training success (y-axis) w.r.t. timesteps (x-axis) with a moving window of 500 episodes for all Fetch environments with the HER algorithm. Uses `scripts/plot_train.py` with arguments `-a her -e Fetch -y success -f rl-trained-agents/ -w 500 -x steps`.

```default
python scripts/plot_train.py -a her -e Fetch -y success -f rl-trained-agents/ -w 500 -x steps
```

--------------------------------

### Set activation function with policy_kwargs

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/config.md

Sets a different activation function for the network using policy_kwargs.

```yaml
policy_kwargs: "dict(activation_fn=nn.ReLU)"
```

--------------------------------

### Default hyperparameters fallback

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/config.md

Shows a config with specific hyperparameters for CartPole-v1 and a default fallback for any other environment. The default entry is used when the environment is not explicitly listed and not an Atari game.

```yaml
# Specific hyperparameters for CartPole-v1
CartPole-v1:
  n_envs: 8
  n_timesteps: !!float 1e5
  policy: 'MlpPolicy'
  learning_rate: 1e-3

# Fallback hyperparameters for any other environment
default:
  n_envs: 4
  n_timesteps: !!float 1e6
  policy: 'MlpPolicy'
```

--------------------------------

### Overwrite hyperparameters

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/train.md

Overwrite hyperparameters in the command line using `--hyperparams`. To pass a string, escape it like `my_string:"'value'"`.

```default
python train.py --algo a2c --env MountainCarContinuous-v0 --hyperparams learning_rate:0.001 policy_kwargs:"dict(net_arch=[64, 64])"
```

--------------------------------

### Linear learning rate schedule

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/config.md

Specifies a linear schedule for the learning rate in the hyperparameter YAML file.

```yaml
learning_rate: lin_0.012486195510232303
```

--------------------------------

### Custom policy specification

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/config.md

Specifies a custom policy class for the agent.

```yaml
policy: my_package.MyCustomPolicy  # for instance stable_baselines3.ppo.MlpPolicy
```

--------------------------------

### BibTeX citation for RL Baselines3 Zoo

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/index.rst

Use this BibTeX entry to cite the RL Baselines3 Zoo project in academic publications. It references the GitHub repository and includes author, title, year, and publisher details.

```bibtex
@misc{rl-zoo3,
  author = {Raffin, Antonin},
  title = {RL Baselines3 Zoo},
  year = {2020},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/DLR-RM/rl-baselines3-zoo}},
}
```

--------------------------------

### macOS workaround for pybullet build failure

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/install.md

Set CFLAGS to avoid stdio.h errors when building pybullet on macOS.

```bash
CFLAGS="-fno-define-target-os-macros" pip install -e .[plots,tests,extras]
```

--------------------------------

### VecEnvWrapper specification

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/config.md

Specifies a VecEnvWrapper to use, similar to env wrappers, using the vec_env_wrapper key.

```yaml
vec_env_wrapper: stable_baselines3.common.vec_env.VecMonitor
```

--------------------------------

### Export results for rliable plotting

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/plot.md

Exports results to a file using `all_plots.py` with `-o` to specify the output file. This is a prerequisite for using the `plot_from_file.py` script with rliable options.

```default
python scripts/all_plots.py -a sac td3 tqc --env Half Ant -f logs/ -o logs/offpolicy
```

--------------------------------

### Override VecNormalize parameters

Source: https://github.com/dlr-rm/rl-baselines3-zoo/blob/master/docs/guide/config.md

Overrides VecNormalize parameters, setting norm_obs to True and norm_reward to False.

```yaml
normalize: "{'norm_obs': True, 'norm_reward': False}"
```