### Environment Setup and Episode Recording

Source: https://github.com/farama-foundation/highwayenv/blob/main/scripts/parking_model_based.ipynb

Imports necessary libraries, installs dependencies, clones the HighwayEnv repository, and sets up the environment for recording videos.

```python
import sys
from tqdm.notebook import trange
!pip install tensorboardx gym pyvirtualdisplay
!apt-get install -y xvfb ffmpeg
!git clone https://github.com/Farama-Foundation/HighwayEnv.git 2> /dev/null
sys.path.insert(0, '/content/HighwayEnv/scripts/')
from utils import record_videos, show_videos
```

--------------------------------

### Install Development Version

Source: https://github.com/farama-foundation/highwayenv/blob/main/docs/installation.md

Installs the current development version from the GitHub repository using pip.

```bash
pip install --user git+https://github.com/eleurent/highway-env
```

--------------------------------

### DQN Training Example with Stable Baselines3

Source: https://github.com/farama-foundation/highwayenv/blob/main/docs/quickstart.md

This code snippet demonstrates how to train a DQN agent on the 'highway-fast-v0' environment using Stable Baselines3, including saving and loading the trained model.

```python
import gymnasium
import highway_env
from stable_baselines3 import DQN

env = gymnasium.make("highway-fast-v0")
model = DQN('MlpPolicy', env,
              policy_kwargs=dict(net_arch=[256, 256]),
              learning_rate=5e-4,
              buffer_size=15000,
              learning_starts=200,
              batch_size=32,
              gamma=0.8,
              train_freq=1,
              gradient_steps=1,
              target_update_interval=50,
              verbose=1,
              tensorboard_log="highway_dqn/")
model.learn(int(2e4))
model.save("highway_dqn/model")

# Load and test saved model
model = DQN.load("highway_dqn/model")
while True:
  done = truncated = False
  obs, info = env.reset()
  while not (done or truncated):
    action, _states = model.predict(obs, deterministic=True)
    obs, reward, done, truncated, info = env.step(action)
    env.render()

```

--------------------------------

### Install environment and agent

Source: https://github.com/farama-foundation/highwayenv/blob/main/scripts/parking_her.ipynb

Installs the highway-env and stable-baselines3 libraries, and imports necessary modules for the environment and agent.

```python
#@title Install environment and agent
!pip install highway-env
# TODO: we use the bleeding edge version because the current stable version does not support the latest gym>=0.21 versions. Revert back to stable at the next SB3 release.
!pip install git+https://github.com/DLR-RM/stable-baselines3

# Environment
import gymnasium as gym
import highway_env

# Agent
from stable_baselines3 import HerReplayBuffer, SAC
```

--------------------------------

### Ubuntu Prerequisites

Source: https://github.com/farama-foundation/highwayenv/blob/main/docs/installation.md

Installs necessary development libraries for Pygame on Ubuntu.

```bash
sudo apt-get update -y
sudo apt-get install -y python-dev libsdl-image1.2-dev libsdl-mixer1.2-dev
    libsdl-ttf2.0-dev libsdl1.2-dev libsmpeg-dev python-numpy subversion libportmidi-dev
    ffmpeg libswscale-dev libavformat-dev libavcodec-dev libfreetype6-dev gcc
```

--------------------------------

### Install environment and agent

Source: https://github.com/farama-foundation/highwayenv/blob/main/scripts/sb3_highway_dqn.ipynb

Installs the highway-env library and the bleeding-edge version of stable-baselines3, along with other necessary libraries and tools for visualization and environment setup.

```python
# Install environment and agent
!pip install highway-env
# TODO: we use the bleeding edge version because the current stable version does not support the latest gym>=0.21 versions. Revert back to stable at the next SB3 release.
!pip install git+https://github.com/DLR-RM/stable-baselines3

# Environment
import gymnasium as gym
import highway_env

gym.register_envs(highway_env)

# Agent
from stable_baselines3 import DQN

# Visualization utils
%load_ext tensorboard
import sys
from tqdm.notebook import trange
!pip install tensorboardx gym pyvirtualdisplay
!apt-get install -y xvfb ffmpeg
!git clone https://github.com/Farama-Foundation/HighwayEnv.git 2> /dev/null
sys.path.insert(0, '/content/HighwayEnv/scripts/')
from utils import record_videos, show_videos
```

--------------------------------

### Usage Example

Source: https://github.com/farama-foundation/highwayenv/blob/main/docs/environments/parking.md

How to instantiate the parking environment using gym.make.

```python
env = gym.make("parking-v0")
```

--------------------------------

### Example usage of a custom environment

Source: https://github.com/farama-foundation/highwayenv/blob/main/docs/make_your_own.md

Demonstrates how to import, make, reset, step, and render a custom environment.

```python
# Only required if you did not reinstall highway_env
import highway_env
highway_env.register_highway_envs()

import gymnasium as gym
env = gym.make('your-env-v0')
obs, info = env.reset()
obs, reward, terminated, truncated, info = env.step(env.action_space.sample())
env.render()
```

--------------------------------

### Install Stable Release

Source: https://github.com/farama-foundation/highwayenv/blob/main/docs/installation.md

Installs the latest stable version of the highway-env package using pip.

```bash
pip install highway-env
```

--------------------------------

### Install and Imports

Source: https://github.com/farama-foundation/highwayenv/blob/main/scripts/parking_model_based.ipynb

Installs the highway-env package and imports necessary libraries for reinforcement learning, including gymnasium, torch, and numpy.

```python
# Install environment and visualization dependencies 
!pip install highway-env

# Environment
import gymnasium as gym
import highway_env

gym.register_envs(highway_env)

# Models and computation
import torch
import torch.nn as nn
import torch.nn.functional as F
import numpy as np
from collections import namedtuple

```

--------------------------------

### Registering a new environment

Source: https://github.com/farama-foundation/highwayenv/blob/main/docs/make_your_own.md

Example of how to register a custom environment in highway_env/__init__.py.

```python
# your_env.py
register(
    id='your-env-v0',
    entry_point='highway_env.envs:YourEnv'
)
```

--------------------------------

### Discrete Actions

Source: https://github.com/farama-foundation/highwayenv/blob/main/docs/actions/index.md

Example of using DiscreteAction for quantized control.

```python
import gymnasium as gym

env = gym.make('highway-v0', config={
    "action": {
        "type": "DiscreteAction"
    }
})
```

--------------------------------

### Environment Rendering Configuration

Source: https://github.com/farama-foundation/highwayenv/blob/main/docs/graphics/index.md

Example of how to configure screen dimensions for environment rendering.

```python
env = gym.make(
    "roundabout-v0",
    config={
        "screen_width": 640,
        "screen_height": 480
    }
)
env.reset()
env.render()
```

--------------------------------

### Import helpers for visualization of episodes

Source: https://github.com/farama-foundation/highwayenv/blob/main/scripts/parking_her.ipynb

Imports helper functions for recording and showing videos of training episodes, and installs necessary packages and dependencies.

```python
#@title Import helpers for visualization of episodes
import sys
from tqdm.notebook import trange
!pip install tensorboardx gym pyvirtualdisplay
!apt-get install -y xvfb ffmpeg
!git clone https://github.com/Farama-Foundation/HighwayEnv.git 2> /dev/null
sys.path.insert(0, '/content/HighwayEnv/scripts/')
from utils import record_videos, show_videos
```

--------------------------------

### Manual Control

Source: https://github.com/farama-foundation/highwayenv/blob/main/docs/actions/index.md

Example of using manual control in the environment.

```python
env = gym.make("highway-v0", config={
    "manual_control": True
})
env.reset()
done = False
while not done:
    env.step(env.action_space.sample())  # with manual control, these actions are ignored
```

--------------------------------

### Example Lidar Observation Configuration

Source: https://github.com/farama-foundation/highwayenv/blob/main/docs/observations/index.md

This is an example of how to configure a Lidar observation in HighwayEnv.

```python
"observation": {
    "type": "LidarObservation",
    "cells": 128,
    "maximum_range": 64,
    "normalise": True,
}
```

--------------------------------

### HighwayEnv Usage Example

Source: https://github.com/farama-foundation/highwayenv/blob/main/README.md

A basic example demonstrating how to create and interact with the highway-v0 environment using Gymnasium.

```python
import gymnasium as gym

env = gym.make('highway-v0', render_mode='human')

obs, info = env.reset()
done = truncated = False
while not (done or truncated):
    action = ... # Your agent code here
    obs, reward, done, truncated, info = env.step(action)
```

--------------------------------

### Example Grayscale Observation Configuration

Source: https://github.com/farama-foundation/highwayenv/blob/main/docs/observations/index.md

This is an example of how to configure a Grayscale observation in HighwayEnv.

```python
": “type”: “GrayscaleObservation”,
  “observation_shape”: (84, 84)
  “stack_size”: 4,
  “weights”: [0.2989, 0.5870, 0.1140],

```

--------------------------------

### Usage Example

Source: https://github.com/farama-foundation/highwayenv/blob/main/docs/environments/merge.md

This snippet shows how to create an instance of the merge environment using gym.make.

```python
env = gym.make("merge-v0")
```

--------------------------------

### Usage Example

Source: https://github.com/farama-foundation/highwayenv/blob/main/docs/environments/racetrack.md

How to create an instance of the Racetrack environment.

```python
env = gym.make("racetrack-v0")
```

--------------------------------

### Default Action Type

Source: https://github.com/farama-foundation/highwayenv/blob/main/docs/actions/index.md

Example of setting the default action type for an environment.

```python
import gymnasium as gym

env = gym.make('highway-v0', config={
    "action": {
        "type": "ContinuousAction"
    }
})
```

--------------------------------

### Usage Example

Source: https://github.com/farama-foundation/highwayenv/blob/main/docs/environments/highway.md

How to create an instance of the highway environment.

```python
env = gym.make("highway-v0")
```

--------------------------------

### Discrete Meta-Actions

Source: https://github.com/farama-foundation/highwayenv/blob/main/docs/actions/index.md

Example of using DiscreteMetaAction for higher-level control.

```python
import gymnasium as gym

env = gym.make('highway-v0', config={
    "action": {
        "type": "DiscreteMetaAction"
    }
})
```

--------------------------------

### Imports for env, agent, and visualisation

Source: https://github.com/farama-foundation/highwayenv/blob/main/scripts/highway_planning.ipynb

Imports necessary libraries for the environment, agent, and visualization, including installing required packages.

```python
#@title Imports for env, agent, and visualisation.
# Environment
!pip install highway-env
import gymnasium as gym
import highway_env

# Agent
!pip install git+https://github.com/eleurent/rl-agents#egg=rl-agents
from rl_agents.agents.common.factory import agent_factory

# Visualisation
import sys
from tqdm.notebook import trange
!pip install moviepy -U
!pip install imageio_ffmpeg
!pip install pyvirtualdisplay
!apt-get install -y xvfb ffmpeg
!git clone https://github.com/Farama-Foundation/HighwayEnv.git
sys.path.insert(0, './highway-env/scripts/')
from utils import record_videos, show_videos

```

--------------------------------

### Start training

Source: https://github.com/farama-foundation/highwayenv/blob/main/scripts/intersection_social_dqn.ipynb

Initiates the training process for the agent.

```python
evaluation.train()

```

--------------------------------

### OccupancyGrid Observation Configuration

Source: https://github.com/farama-foundation/highwayenv/blob/main/docs/observations/index.md

Example configuration for the OccupancyGrid observation type.

```python
"observation": {
    "type": "OccupancyGrid",
    "vehicles_count": 15,
    "features": ["presence", "x", "y", "vx", "vy", "cos_h", "sin_h"],
    "features_range": {
        "x": [-100, 100],
        "y": [-100, 100],
        "vx": [-20, 20],
        "vy": [-20, 20]
    },
    "grid_size": [[-27.5, 27.5], [-27.5, 27.5]],
    "grid_step": [5, 5],
    "absolute": False
}
```

--------------------------------

### TimeToCollision Observation Configuration

Source: https://github.com/farama-foundation/highwayenv/blob/main/docs/observations/index.md

Example configuration for the TimeToCollision observation type.

```python
"observation": {
    "type": "TimeToCollision"
    "horizon": 10
}
```

--------------------------------

### Training a centralized multi-agent policy with rl-agents

Source: https://github.com/farama-foundation/highwayenv/blob/main/docs/multi_agent.md

Example command to train a DQN agent in a multi-agent environment using the rl-agents library. This command specifies the environment configuration, agent configuration, and training parameters.

```bash
cd <path/to/rl-agents/scripts>
python experiments.py evaluate configs/IntersectionEnv/env_multi_agent.json \
                               configs/IntersectionEnv/agents/DQNAgent/ego_attention_2h.json \
                               --train --episodes=3000
```

--------------------------------

### OccupancyGrid Observation Configuration

Source: https://github.com/farama-foundation/highwayenv/blob/main/docs/observations/index.md

Example of how to configure the OccupancyGrid observation type for a HighwayEnv environment.

```python
import gymnasium as gym
import highway_env

env = gym.make(
    'highway-v0',
    config={
        "observation": {
            "type": "OccupancyGrid",
            "vehicles_count": 15,
            "features": ["presence", "x", "y", "vx", "vy", "cos_h", "sin_h"],
            "features_range": {
                "x": [-100, 100],
                "y": [-100, 100],
                "vx": [-20, 20],
                "vy": [-20, 20]
            },
            "grid_size": [[-27.5, 27.5], [-27.5, 27.5]],
            "grid_step": [5, 5],
            "absolute": False
        }
    }
)
env.reset()
```

--------------------------------

### Import requirements

Source: https://github.com/farama-foundation/highwayenv/blob/main/scripts/intersection_social_dqn.ipynb

Installs necessary libraries and clones the HighwayEnv repository to import utility scripts.

```python
#@title Import requirements

# Environment
!pip install highway-env
import gymnasium as gym

# Agent
!pip install git+https://github.com/eleurent/rl-agents#egg=rl-agents

# Visualisation utils
!pip install moviepy
!pip install imageio_ffmpeg
import sys
%load_ext tensorboard
!pip install tensorboardx gym pyvirtualdisplay
!apt-get install -y xvfb ffmpeg
!git clone https://github.com/Farama-Foundation/HighwayEnv.git 2> /dev/null
sys.path.insert(0, '/content/HighwayEnv/scripts/')
from utils import show_videos

```

--------------------------------

### Accessing a lane geometry

Source: https://github.com/farama-foundation/highwayenv/blob/main/docs/dynamics/road/road.md

Example of how to access the geometry of a specific lane within the road network graph.

```python
lane = road.road_network.graph["lab"]["pub"][1]
```

--------------------------------

### Run tensorboard locally

Source: https://github.com/farama-foundation/highwayenv/blob/main/scripts/sb3_highway_dqn.ipynb

Launches TensorBoard to visualize the training progress.

```python
%tensorboard --logdir "highway_dqn"
```

--------------------------------

### Prepare environment, agent, and evaluation process

Source: https://github.com/farama-foundation/highwayenv/blob/main/scripts/intersection_social_dqn.ipynb

Loads the environment and agent configurations, initializes the agent and environment, and sets up the evaluation process.

```python
#@title Prepare environment, agent, and evaluation process.

NUM_EPISODES = 3000  #@param {type: "integer"}

from rl_agents.trainer.evaluation import Evaluation
from rl_agents.agents.common.factory import load_agent, load_environment

# Get the environment and agent configurations from the rl-agents repository
!git clone https://github.com/eleurent/rl-agents.git 2> /dev/null
%cd /content/rl-agents/scripts/
env_config = 'configs/IntersectionEnv/env.json'
agent_config = 'configs/IntersectionEnv/agents/DQNAgent/ego_attention_2h.json'

env = load_environment(env_config)
agent = load_agent(agent_config, env)
evaluation = Evaluation(env, agent, num_episodes=NUM_EPISODES, display_env=False, display_agent=False)
print(f"Ready to train {agent} on {env}")

```

--------------------------------

### Run an episode

Source: https://github.com/farama-foundation/highwayenv/blob/main/scripts/highway_planning.ipynb

Sets up the environment and agent, then runs a single episode, recording and displaying the video.

```python
#@title Run an episode

# Make environment
env = gym.make("highway-fast-v0", render_mode="rgb_array")
env = record_videos(env)
(obs, info), done = env.reset(), False

# Make agent
agent_config = {
    "__class__": "<class 'rl_agents.agents.tree_search.deterministic.DeterministicPlannerAgent'>",
    "env_preprocessors": [{"method":"simplify"}],
    "budget": 50,
    "gamma": 0.7,
}
agent = agent_factory(env, agent_config)

# Run episode
for step in trange(env.unwrapped.config["duration"], desc="Running..."):
    action = agent.act(obs)
    obs, reward, done, truncated, info = env.step(action)
    
env.close()
show_videos()

```

--------------------------------

### Tensorboard - click the refresh button once training is running

Source: https://github.com/farama-foundation/highwayenv/blob/main/scripts/parking_her.ipynb

Loads the tensorboard extension and sets up the tensorboard log directory.

```python
#@title Tensorboard - click the refresh button once training is running

%load_ext tensorboard
%tensorboard --logdir logs
```

--------------------------------

### Running a Random Episode

Source: https://github.com/farama-foundation/highwayenv/blob/main/scripts/parking_model_based.ipynb

Creates the parking environment, records its execution with random actions, and then displays the recorded videos.

```python
env = gym.make("parking-v0", render_mode="rgb_array")
env = record_videos(env)
env.reset()
done = False
while not done:
    action = env.action_space.sample()
    obs, reward, done, truncated, info = env.step(action)
env.close()
show_videos()
```

--------------------------------

### Run tensorboard locally to visualize training

Source: https://github.com/farama-foundation/highwayenv/blob/main/scripts/intersection_social_dqn.ipynb

Launches TensorBoard to visualize the training progress.

```python
%tensorboard --logdir "{evaluation.directory}"

```

--------------------------------

### Visualize Episodes with CEM Planner

Source: https://github.com/farama-foundation/highwayenv/blob/main/scripts/parking_model_based.ipynb

Code to run the CEM planner on a parking environment for multiple episodes and visualize the results.

```python
env = gym.make("parking-v0", render_mode='rgb_array')
env = record_videos(env)
obs, info = env.reset()

for step in trange(3 * env.config["duration"], desc="Testing 3 episodes..."):
    action = cem_planner(torch.Tensor(obs["observation"]),
                         torch.Tensor(obs["desired_goal"]),
                         env.action_space.shape[0])
    obs, reward, done, truncated, info = env.step(action.numpy())
    if done or truncated:
        obs, info = env.reset()
env.close()
show_videos()
```

--------------------------------

### Visualize a few episodes

Source: https://github.com/farama-foundation/highwayenv/blob/main/scripts/parking_her.ipynb

Visualizes the trained agent's performance over a specified number of episodes.

```python
#@title Visualize a few episodes

N_EPISODES = 10  # @param {type: "integer"}

env = gym.make('parking-v0', render_mode='rgb_array')
env = record_videos(env)
for episode in trange(N_EPISODES, desc="Test episodes"):
    obs, info = env.reset()
    done = truncated = False
    while not (done or truncated):
        action, _ = model.predict(obs, deterministic=True)
        obs, reward, done, truncated, info = env.step(action)
env.close()
show_videos()
```

--------------------------------

### Initialize Roundabout Environment

Source: https://github.com/farama-foundation/highwayenv/blob/main/docs/environments/roundabout.md

This snippet shows how to create an instance of the Roundabout environment using gym.make.

```python
env = gym.make("roundabout-v0")
```

--------------------------------

### Environment Initialization

Source: https://github.com/farama-foundation/highwayenv/blob/main/docs/environments/intersection.md

How to create an instance of the intersection environment using gym.make.

```python
env = gym.make("intersection-v0")
```

--------------------------------

### Record Video with Intermediate Frames

Source: https://github.com/farama-foundation/highwayenv/blob/main/docs/faq.md

This code snippet demonstrates how to wrap a Gymnasium environment with `RecordVideo` and configure it to record intermediate simulation frames, addressing issues with fast or low-framerate videos.

```python
import gymnasium as gym
from gymnasium.wrappers import RecordVideo

# Wrap the env by a RecordVideo wrapper
env = gym.make("highway-v0")
env = RecordVideo(env, video_folder="run",
              episode_trigger=lambda e: True)  # record all episodes

# Provide the video recorder to the wrapped environment
# so it can send it intermediate simulation frames.
env.unwrapped.set_record_video_wrapper(env)

# Record a video as usual
obs, info = env.reset()
done = truncated = False:
while not (done or truncated):
    action = env.action_space.sample()
    obs, reward, done, truncated, info = env.step(action)
    env.render()
env.close()
```

--------------------------------

### Run the learned policy for a few episodes

Source: https://github.com/farama-foundation/highwayenv/blob/main/scripts/intersection_social_dqn.ipynb

Loads the trained agent and runs it for a few episodes to evaluate its performance, then displays the results as videos.

```python
#@title Run the learned policy for a few episodes.
env = load_environment(env_config)
env.config["offscreen_rendering"] = True
agent = load_agent(agent_config, env)
evaluation = Evaluation(env, agent, num_episodes=20, training = False, recover = True)
evaluation.test()
show_videos(evaluation.run_directory)

```

--------------------------------

### Training

Source: https://github.com/farama-foundation/highwayenv/blob/main/scripts/parking_her.ipynb

Trains the SAC agent with HER replay buffer on the parking environment.

```python
#@title Training

LEARNING_STEPS = 5e4 # @param {type: "number"}

env = gym.make('parking-v0')
her_kwargs = dict(n_sampled_goal=4, goal_selection_strategy='future')
model = SAC('MultiInputPolicy', env, replay_buffer_class=HerReplayBuffer,
            replay_buffer_kwargs=her_kwargs, verbose=1, 
            tensorboard_log="logs", 
            buffer_size=int(1e6),
            learning_rate=1e-3,
            gamma=0.95, batch_size=1024, tau=0.05,
            policy_kwargs=dict(net_arch=[512, 512, 512]))

model.learn(int(LEARNING_STEPS))
```

--------------------------------

### Visualize episodes

Source: https://github.com/farama-foundation/highwayenv/blob/main/scripts/sb3_highway_dqn.ipynb

Tests the trained DQN agent by visualizing its performance over a few episodes in the highway-fast-v0 environment.

```python
env = gym.make('highway-fast-v0', render_mode='rgb_array')
env = record_videos(env)
for episode in trange(3, desc='Test episodes'):
    (obs, info), done = env.reset(), False
    while not done:
        action, _ = model.predict(obs, deterministic=True)
        obs, reward, done, truncated, info = env.step(int(action))
env.close()
show_videos()
```

--------------------------------

### Printing Observation Format

Source: https://github.com/farama-foundation/highwayenv/blob/main/scripts/parking_model_based.ipynb

Prints the format of the observation returned by the environment, which includes the current observation and the desired goal.

```python
print("Observation format:", obs)
```

--------------------------------

### CEM Planner Implementation

Source: https://github.com/farama-foundation/highwayenv/blob/main/scripts/parking_model_based.ipynb

Python function implementing the Cross-Entropy Method for planning, including sampling actions, predicting trajectories, and fitting the distribution to top-k performing sequences.

```python
def cem_planner(state, goal, action_size, horizon=5, population=100, selection=10, iterations=5):
    state = state.expand(population, -1)
    action_mean = torch.zeros(horizon, 1, action_size)
    action_std = torch.ones(horizon, 1, action_size)
    for _ in range(iterations):
        # 1. Draw sample sequences of actions from a normal distribution
        actions = torch.normal(mean=action_mean.repeat(1, population, 1), std=action_std.repeat(1, population, 1))
        actions = torch.clamp(actions, min=env.action_space.low.min(), max=env.action_space.high.max())
        states = predict_trajectory(state, actions, dynamics, action_repeat=5)
        # 2. Fit the distribution to the top-k performing sequences
        returns = reward_model(states, goal).sum(dim=0)
        _, best = returns.topk(selection, largest=True, sorted=False)
        best_actions = actions[:, best, :]
        action_mean = best_actions.mean(dim=1, keepdim=True)
        action_std = best_actions.std(dim=1, unbiased=False, keepdim=True)
    return action_mean[0].squeeze(dim=0)
  
  
# Run the planner on a sample transition
action = cem_planner(torch.Tensor(obs["observation"]),
                     torch.Tensor(obs["desired_goal"]),
                     env.action_space.shape[0])
print("Planned action:", action)
```

--------------------------------

### MDPVehicle Class Initialization

Source: https://github.com/farama-foundation/highwayenv/blob/main/docs/dynamics/vehicle/controller.md

Initializes an MDPVehicle with road, position, heading, speed, target lane, target speed, target speeds, and route.

```python
class highway_env.vehicle.controller.MDPVehicle(road: [Road](../road/road.md#highway_env.road.road.Road), position: List[float], heading: float = 0, speed: float = 0, target_lane_index: Tuple[str, str, int] | None = None, target_speed: float | None = None, target_speeds: ndarray | Sequence[float] | None = None, route: List[Tuple[str, str, int]] | None = None)
```

--------------------------------

### Parking Environment

Source: https://github.com/farama-foundation/highwayenv/blob/main/README.md

Creates an instance of the parking environment.

```python
env = gymnasium.make("parking-v0")
```

--------------------------------

### Discrete Meta-Actions - All Actions

Source: https://github.com/farama-foundation/highwayenv/blob/main/docs/actions/index.md

Definition of all available meta-actions.

```python
ACTIONS_ALL = {
        0: 'LANE_LEFT',
        1: 'IDLE',
        2: 'LANE_RIGHT',
        3: 'FASTER',
        4: 'SLOWER'
    }
```

--------------------------------

### DQN Model Training

Source: https://github.com/farama-foundation/highwayenv/blob/main/scripts/sb3_highway_dqn.ipynb

Initializes and trains a DQN agent with specified hyperparameters for the highway-fast-v0 environment.

```python
model = DQN('MlpPolicy', 'highway-fast-v0',
                policy_kwargs=dict(net_arch=[256, 256]),
                learning_rate=5e-4,
                buffer_size=15000,
                learning_starts=200,
                batch_size=32,
                gamma=0.8,
                train_freq=1,
                gradient_steps=1,
                target_update_interval=50,
                exploration_fraction=0.7,
                verbose=1,
                tensorboard_log='highway_dqn/')
model.learn(int(2e4))
```

--------------------------------

### Default Configuration

Source: https://github.com/farama-foundation/highwayenv/blob/main/docs/environments/parking.md

The default configuration dictionary for the parking environment.

```python
{
    "observation": {
        "type": "KinematicsGoal",
        "features": ['x', 'y', 'vx', 'vy', 'cos_h', 'sin_h'],
        "scales": [100, 100, 5, 5, 1, 1],
        "normalize": False
    },
    "action": {
        "type": "ContinuousAction"
    },
    "simulation_frequency": 15,
    "policy_frequency": 5,
    "screen_width": 600,
    "screen_height": 300,
    "centering_position": [0.5, 0.5],
    "scaling": 7
    "show_trajectories": False,
    "render_agent": True,
    "offscreen_rendering": False
}
```

--------------------------------

### Training the Dynamics Model

Source: https://github.com/farama-foundation/highwayenv/blob/main/scripts/parking_model_based.ipynb

Sets up the Adam optimizer, splits the collected data into training and validation sets, defines functions to compute loss and transpose batches, and trains the dynamics model for a specified number of epochs. It also plots the training and validation losses.

```python
optimizer = torch.optim.Adam(dynamics.parameters(), lr=0.01)

# Split dataset into training and validation
train_ratio = 0.7
train_data, validation_data = data[:int(train_ratio * len(data))], data[int(train_ratio * len(data)):]

def compute_loss(model, data_t, loss_func = torch.nn.MSELoss()):
    states, actions, next_states = data_t
    predictions = model(states, actions)
    return loss_func(predictions, next_states)

def transpose_batch(batch):
    return Transition(*map(torch.stack, zip(*batch)))

def train(model, train_data, validation_data, epochs=1500):
    train_data_t = transpose_batch(train_data)
    validation_data_t = transpose_batch(validation_data)
    losses = np.full((epochs, 2), np.nan)
    for epoch in trange(epochs, desc="Train dynamics"):
        # Compute loss gradient and step optimizer
        loss = compute_loss(model, train_data_t)
        validation_loss = compute_loss(model, validation_data_t)
        losses[epoch] = [loss.detach().numpy(), validation_loss.detach().numpy()]
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
    # Plot losses
    plt.plot(losses)
    plt.yscale("log")
    plt.xlabel("epochs")
    plt.ylabel("loss")
    plt.legend(["train", "validation"])
    plt.show()

train(dynamics, data, validation_data)
```

--------------------------------

### Merge Environment

Source: https://github.com/farama-foundation/highwayenv/blob/main/README.md

Creates an instance of the merge environment.

```python
env = gymnasium.make("merge-v0")
```

--------------------------------

### Roundabout Environment

Source: https://github.com/farama-foundation/highwayenv/blob/main/README.md

Creates an instance of the roundabout environment.

```python
env = gymnasium.make("roundabout-v0")
```

--------------------------------

### Racetrack Environment

Source: https://github.com/farama-foundation/highwayenv/blob/main/README.md

Creates an instance of the racetrack environment.

```python
env = gymnasium.make("racetrack-v0")
```

--------------------------------

### Predict and Plot Trajectories

Source: https://github.com/farama-foundation/highwayenv/blob/main/scripts/parking_model_based.ipynb

Functions to predict trajectories based on a model and initial state, and to plot these trajectories with arrows indicating direction.

```python
def predict_trajectory(state, actions, model, action_repeat=1):
    states = []
    for action in actions:
        for _ in range(action_repeat):
            state = model(state, action)
            states.append(state)
    return torch.stack(states, dim=0)

def plot_trajectory(states, color):
    scales = np.array(env.unwrapped.config["observation"]["scales"])
    states = np.clip(states.squeeze(1).detach().numpy() * scales, -100, 100)
    plt.plot(states[:, 0], states[:, 1], color=color, marker='.')
    plt.arrow(states[-1,0], states[-1,1], states[-1,4]*1, states[-1,5]*1, color=color)

def visualize_trajectories(model, state, horizon=15):
    plt.cla()
    # Draw a car
    plt.plot(state.numpy()[0]+2.5*np.array([-1, -1, 1, 1, -1]),
             state.numpy()[1]+1.0*np.array([-1, 1, 1, -1, -1]), 'k')
    # Draw trajectories
    state = state.unsqueeze(0)
    colors = iter(plt.get_cmap("tab20").colors)
    # Generate commands
    for steering in np.linspace(-0.5, 0.5, 3):
        for acceleration in np.linspace(0.8, 0.4, 2):
            actions = torch.Tensor([acceleration, steering]).view(1,1,-1)
            # Predict trajectories
            states = predict_trajectory(state, actions, model, action_repeat=horizon)
            plot_trajectory(states, color=next(colors))
    plt.axis("equal")
    plt.show()
    
visualize_trajectories(dynamics, state=torch.Tensor([0, 0, 0, 0, 1, 0]))
```

--------------------------------

### Default Configuration

Source: https://github.com/farama-foundation/highwayenv/blob/main/docs/environments/merge.md

This snippet displays the default configuration parameters for the merge environment.

```python
{
    "observation": {
        "type": "TimeToCollision"
    },
    "action": {
        "type": "DiscreteMetaAction"
    },
    "simulation_frequency": 15,  # [Hz]
    "policy_frequency": 1,  # [Hz]
    "other_vehicles_type": "highway_env.vehicle.behavior.IDMVehicle",
    "screen_width": 600,  # [px]
    "screen_height": 150,  # [px]
    "centering_position": [0.3, 0.5],
    "scaling": 5.5,
    "show_trajectories": False,
    "render_agent": True,
    "offscreen_rendering": False
}
```