### Install SUMO-RL from Source

Source: https://github.com/lucasalegre/sumo-rl/blob/main/README.md

Installs the latest unreleased version of SUMO-RL by cloning the repository and installing it in editable mode.

```bash
git clone https://github.com/LucasAlegre/sumo-rl
cd sumo-rl
pip install -e .
```

--------------------------------

### Minimal Sumo-RL Environment Setup

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/MODULES.md

This snippet shows the basic setup for creating a Sumo-RL environment using the gym registry. It requires importing gymnasium and sumo_rl.

```python
import gymnasium as gym
import sumo_rl

# Use gym registry
env = gym.make('sumo-rl-v0',
               net_file='network.net.xml',
               route_file='routes.rou.xml')
```

--------------------------------

### Single-Agent Q-Learning Usage Example

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/api-reference/QLAgent.md

Provides a comprehensive example of setting up and training a QLAgent in a single-agent sumo-rl environment, including environment creation, agent initialization, and the training loop.

```python
import gymnasium as gym
import sumo_rl
from sumo_rl.agents import QLAgent
from sumo_rl.exploration import EpsilonGreedy

# Create environment
env = gym.make('sumo-rl-v0',
               net_file='path/to/network.net.xml',
               route_file='path/to/routes.rou.xml',
               num_seconds=3600)

# Get initial observation
obs, info = env.reset()

# Create Q-learning agent
exploration = EpsilonGreedy(initial_epsilon=1.0, min_epsilon=0.05, decay=0.995)
starting_state = env.encode(obs, env.ts_ids[0])

agent = QLAgent(
    starting_state=starting_state,
    state_space=None,
    action_space=env.action_space,
    alpha=0.5,
    gamma=0.95,
    exploration_strategy=exploration
)

# Training loop
for episode in range(100):
    obs, info = env.reset()
    state = env.encode(obs, env.ts_ids[0])
    agent.state = state
    exploration.reset()
    
    done = False
    while not done:
        # Agent acts
        action = agent.act()
        
        # Environment steps
        obs, reward, terminated, truncated, info = env.step(action)
        done = terminated or truncated
        
        # Agent learns
        next_state = env.encode(obs, env.ts_ids[0])
        agent.learn(next_state, reward, done)
    
    print(f"Episode {episode}: Accumulated reward = {agent.acc_reward}")

env.close()
```

--------------------------------

### Install sumo-rl

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/README.md

Install the sumo-rl library using pip. Ensure SUMO is installed and SUMO_HOME is exported.

```bash
export SUMO_HOME="/usr/share/sumo"
pip install sumo-rl
```

--------------------------------

### SUMO_HOME Examples for Different OS

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/configuration.md

Examples of setting the SUMO_HOME environment variable for Ubuntu/Debian, macOS (homebrew), and Windows.

```bash
# Ubuntu/Debian
export SUMO_HOME="/usr/share/sumo"

# macOS (homebrew)
export SUMO_HOME="/usr/local/opt/sumo/share/sumo"

# Windows
export SUMO_HOME="C:\\Program Files\\SUMO"
```

--------------------------------

### Gymnasium Single-Agent Environment Setup

Source: https://github.com/lucasalegre/sumo-rl/blob/main/README.md

Instantiate a standard Gymnasium environment for a single traffic light network. This example shows how to reset the environment and interact with it using the Gymnasium API.

```python
import gymnasium as gym
import sumo_rl
env = gym.make('sumo-rl-v0',
                net_file='path_to_your_network.net.xml',
                route_file='path_to_your_routefile.rou.xml',
                out_csv_name='path_to_output.csv',
                use_gui=True,
                num_seconds=100000)
ob s, info = env.reset()
done = False
while not done:
    next_obs, reward, terminated, truncated, info = env.step(env.action_space.sample())
    done = terminated or truncated
```

--------------------------------

### EpsilonGreedy Reset Method Example

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/api-reference/QLAgent.md

Illustrates resetting the EpsilonGreedy exploration strategy to its initial epsilon value at the start of each episode.

```python
from sumo_rl.exploration import EpsilonGreedy

exploration = EpsilonGreedy()

for episode in range(100):
    exploration.reset()  # Start episode with initial epsilon
    # Training loop...
```

--------------------------------

### Example Usage with SumoEnvironment

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/api-reference/TrafficSignal.md

Demonstrates creating a SumoEnvironment, resetting it, accessing TrafficSignal objects to query metrics, and stepping through the environment.

```python
import sumo_rl

# Create environment
env = sumo_rl.env(
    net_file='network.net.xml',
    route_file='routes.rou.xml',
    reward_fn='queue',
    num_seconds=3600
)

obs = env.reset()

# TrafficSignal objects are accessed via env.traffic_signals dict
for ts_id in env.ts_ids:
    ts = env.traffic_signals[ts_id]
    
    # Query metrics
    print(f"Traffic Signal {ts_id}:")
    print(f"  Green phases: {ts.num_green_phases}")
    print(f"  Current phase: {ts.green_phase}")
    print(f"  Queue: {ts.get_total_queued()}")
    print(f"  Avg speed: {ts.get_average_speed()}")
    print(f"  Pressure: {ts.get_pressure()}")

# During stepping, rewards are computed automatically
while env.agents:
    actions = {agent: env.action_space(agent).sample() for agent in env.agents}
    obs, rewards, dones, info = env.step(actions)
```

--------------------------------

### Basic SumoEnvironment Initialization Example

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/api-reference/SumoEnvironment.md

Demonstrates how to import and initialize the SumoEnvironment for use with Gymnasium. Ensure you have the necessary SUMO network and route files.

```python
import gymnasium as gym
import sumo_rl


```

--------------------------------

### PettingZoo Multi-Agent Environment Setup

Source: https://github.com/lucasalegre/sumo-rl/blob/main/README.md

Set up a multi-agent environment using the PettingZoo API for environments with multiple traffic lights. This example demonstrates resetting the environment and taking actions for all agents.

```python
import sumo_rl
env = sumo_rl.parallel_env(net_file='nets/RESCO/grid4x4/grid4x4.net.xml',
                  route_file='nets/RESCO/grid4x4/grid4x4_1.rou.xml',
                  use_gui=True,
                  num_seconds=3600)
observations = env.reset()
while env.agents:
    actions = {agent: env.action_space(agent).sample() for agent in env.agents}  # this is where you would insert your policy
    observations, rewards, terminations, truncations, infos = env.step(actions)
```

--------------------------------

### ObservationDict Example

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/types.md

Illustrates how to iterate through a dictionary of observations returned by env.reset() in a multi-agent setup.

```python
observations = env.reset()  # dict[str, np.ndarray]
for ts_id, obs in observations.items():
    print(f"{ts_id}: shape={{obs.shape}}")
```

--------------------------------

### Install SUMO Traffic Simulator

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/QUICK-START.md

Installs the SUMO traffic simulator on Ubuntu/Debian, macOS, or Windows. Ensure SUMO is installed before proceeding.

```bash
# Ubuntu/Debian
sudo add-apt-repository ppa:sumo/stable
sudo apt-get update
sudo apt-get install sumo sumo-tools sumo-doc

# macOS
brew install sumo

# Windows
# Download from https://sumo.dlr.de/wiki/Downloads
```

--------------------------------

### QL-Agent and Epsilon-Greedy Exploration Setup

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/MODULES.md

Configures a QL-Agent with an Epsilon-Greedy exploration strategy. Requires importing QLAgent, EpsilonGreedy, and Discrete from gymnasium.spaces.

```python
from sumo_rl.agents import QLAgent
from sumo_rl.exploration import EpsilonGreedy
from gymnasium.spaces import Discrete

exploration = EpsilonGreedy(initial_epsilon=1.0, decay=0.995)
agent = QLAgent(
    starting_state=initial_state,
    state_space=None,
    action_space=Discrete(4),
    exploration_strategy=exploration
)

action = agent.act()
agent.learn(next_state, reward)
```

--------------------------------

### Example Usage of Info Dictionary

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/types.md

Demonstrates accessing global and per-agent information from the 'info' dictionary returned by env.step().

```python
obs, reward, terminated, truncated, info = env.step(action)
print(info["system_mean_speed"])  # Global average speed
print(info["tls_0_stopped"])  # Queue at traffic signal tls_0
```

--------------------------------

### Install SUMO-RL Stable Release

Source: https://github.com/lucasalegre/sumo-rl/blob/main/README.md

Installs the stable release version of SUMO-RL using pip.

```bash
pip install sumo-rl
```

--------------------------------

### QLAgent Constructor Example

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/api-reference/QLAgent.md

Demonstrates how to initialize a QLAgent with custom exploration strategy. Ensure action_space is a gymnasium.spaces.Discrete object and starting_state is a hashable tuple.

```python
from gymnasium.spaces import Discrete
from sumo_rl.agents import QLAgent
from sumo_rl.exploration import EpsilonGreedy

# Create a Q-learning agent for a 4-action intersection
action_space = Discrete(4)
starting_state = (0, 0, 0, 0, 0, 0, 0, 0, 0)  # Hashable state tuple
exploration = EpsilonGreedy(initial_epsilon=0.3, decay=0.99)

agent = QLAgent(
    starting_state=starting_state,
    state_space=None,  # Not used
    action_space=action_space,
    alpha=0.5,
    gamma=0.95,
    exploration_strategy=exploration
)
```

--------------------------------

### EpsilonGreedy Choose Method Example

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/api-reference/QLAgent.md

Demonstrates the choose method of EpsilonGreedy, illustrating action selection based on Q-table values and epsilon-greedy strategy, with epsilon decaying after each step.

```python
from gymnasium.spaces import Discrete
from sumo_rl.exploration import EpsilonGreedy

exploration = EpsilonGreedy(initial_epsilon=0.5, decay=0.99)

q_table = {
    (0, 0): [1.5, 2.0, 0.5, 1.0],
    (0, 1): [0.8, 1.2, 2.5, 1.0],
}

state = (0, 0)
action_space = Discrete(4)

for step in range(100):
    action = exploration.choose(q_table, state, action_space)
    # Epsilon decays each step
    print(f"Step {step}, epsilon={exploration.epsilon:.3f}, action={action}")
```

--------------------------------

### Sumo-RL Multi-Agent Environment Setup

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/MODULES.md

Demonstrates setting up both agent-by-agent (AECEnv) and simultaneous (ParallelEnv) multi-agent environments. Imports 'env' and 'parallel_env' from sumo_rl.

```python
from sumo_rl import env as sumo_env, parallel_env

# AECEnv (agent-by-agent)
env = sumo_env(net_file='network.net.xml',
               route_file='routes.rou.xml')

# ParallelEnv (simultaneous)
env = parallel_env(net_file='network.net.xml',
                   route_file='routes.rou.xml')
```

--------------------------------

### Install SUMO Latest Version

Source: https://github.com/lucasalegre/sumo-rl/blob/main/README.md

Installs the latest stable version of SUMO and its tools on Debian-based systems. Remember to set the SUMO_HOME environment variable.

```bash
sudo add-apt-repository ppa:sumo/stable
sudo apt-get update
sudo apt-get install sumo sumo-tools sumo-doc
```

```bash
echo 'export SUMO_HOME="/usr/share/sumo"' >> ~/.bashrc
source ~/.bashrc
```

```bash
export LIBSUMO_AS_TRACI=1
```

--------------------------------

### Run Training with LIBSUMO

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/configuration.md

Example of running a training script after enabling the libsumo backend. This configuration results in approximately 8x faster execution.

```bash
export LIBSUMO_AS_TRACI=1
python train.py  # Runs ~8x faster
```

--------------------------------

### RewardFunction Examples

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/types.md

Illustrates how to use both built-in string-based and custom callable reward functions when initializing a SumoEnvironment.

```python
# Built-in
env = SumoEnvironment(..., reward_fn="queue")

# Custom
def my_reward(ts: TrafficSignal) -> float:
    return -ts.get_total_queued()

env = SumoEnvironment(..., reward_fn=my_reward)
```

--------------------------------

### QLAgent Act Method Example

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/api-reference/QLAgent.md

Shows how to use the act method to select an action based on the current state and exploration strategy. The selected action is stored in agent.action.

```python
agent.state = (0, 1, 0, 1, 0, 1, 0, 0)
action = agent.act()  # Returns int in [0, 3] for 4-action space
print(f"Selected action: {action}")
```

--------------------------------

### RewardDict Example

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/types.md

Shows how to access and print individual rewards for each traffic signal from the rewards dictionary returned by env.step().

```python
observations, rewards, dones, info = env.step(actions)
# rewards is dict[str, float] or dict[str, np.ndarray]
for ts_id, reward in rewards.items():
    print(f"{ts_id}: reward={{reward}}")
```

--------------------------------

### Multi-Agent Q-Learning Usage Example

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/api-reference/QLAgent.md

Demonstrates how to set up and use QLAgent for multiple traffic signals in a multi-agent sumo-rl environment, creating an agent for each traffic signal ID.

```python
import sumo_rl
from sumo_rl.agents import QLAgent
from sumo_rl.exploration import EpsilonGreedy

# Create multi-agent environment
env = sumo_rl.env(net_file='nets/RESCO/grid4x4/grid4x4.net.xml',
                  route_file='nets/RESCO/grid4x4/grid4x4_1.rou.xml')

# Create agents for each traffic signal
agents = {}
for agent_id in env.agents:
    exploration = EpsilonGreedy(initial_epsilon=1.0, decay=0.995)
    agents[agent_id] = QLAgent(
        starting_state=(0,) * 9,  # Example starting state
        state_space=None,
        action_space=env.action_space(agent_id),
        exploration_strategy=exploration
    )
```

--------------------------------

### Cologne3 Environment Setup and Interaction

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/api-reference/resco-environments.md

Initializes the Cologne3 environment and demonstrates interaction with agents, including sampling actions and stepping through the simulation. Suitable for multi-agent heterogeneous traffic signal control.

```python
env = sumo_rl.cologne3(parallel=True)
observations = env.reset()

# Agents may have different action space sizes
for agent in env.agents:
    action_space = env.action_space(agent)
    print(f"{agent}: {action_space.n} actions")

while env.agents:
    actions = {agent: env.action_space(agent).sample() for agent in env.agents}
    observations, rewards, terminations, truncations, infos = env.step(actions)
env.close()
```

--------------------------------

### Create Multi-Agent SumoEnvironment

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/api-reference/SumoEnvironment.md

Instantiate a multi-agent SumoEnvironment. This setup is suitable for controlling multiple traffic signals simultaneously. Requires network and route files.

```python
env = sumo_rl.env(net_file='path/to/network.net.xml',
                  route_file='path/to/routes.rou.xml',
                  num_seconds=3600)

obs = env.reset()
while env.agents:
    actions = {agent: env.action_space(agent).sample() for agent in env.agents}
    obs, rewards, dones, infos = env.step(actions)
env.close()
```

--------------------------------

### RESCO Preset Function Examples

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/configuration.md

Demonstrates the usage of various RESCO preset functions for creating pre-configured SUMO networks. These functions accept 'parallel' and other keyword arguments.

```python
# All signatures: fn(parallel=True, **kwargs)
sumo_rl.grid4x4(parallel=True, num_seconds=7200)
sumo_rl.arterial4x4(parallel=False, reward_fn='pressure')
sumo_rl.cologne1(reward_fn='average-speed')
sumo_rl.cologne3(sumo_seed=123)
sumo_rl.cologne8()
sumo_rl.ingolstadt1()
sumo_rl.ingolstadt7()
sumo_rl.ingolstadt21(use_gui=True)
```

--------------------------------

### DefaultObservation Example

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/types.md

Shows how to obtain and inspect the default observation vector from the environment, including its shape and data type.

```python
obs, info = env.reset()
# obs is numpy.ndarray
print(obs.shape)  # (9,) for 4 phases + min_green + 2 lanes * 2 metrics
print(obs.dtype)  # float32
print(obs)        # [1. 0. 0. 0. 1. 0.3 0.5 0.1 0.2]
```

--------------------------------

### QLAgent Training Loop Example

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/api-reference/QLAgent.md

This snippet shows a typical training loop for a Q-learning agent. It iterates through episodes, resets the environment, initializes agent states, and then enters a loop where agents act, the environment steps, and agents learn. Finally, it prints the episode completion and closes the environment.

```python
for episode in range(10):
    obs = env.reset()
    
    # Initialize agent states
    for agent_id in env.agents:
        agents[agent_id].state = obs[agent_id] if isinstance(obs[agent_id], tuple) else (0,)
    
    done = False
    while env.agents:
        # All agents act
        actions = {}
        for agent_id in env.agents:
            actions[agent_id] = agents[agent_id].act()
        
        # Environment steps
        obs, rewards, dones, infos = env.step(actions)
        
        # All agents learn
        for agent_id in env.agents:
            agents[agent_id].learn(obs[agent_id], rewards[agent_id], dones.get(agent_id, False))
        
        done = dones.get('__all__', False)
    
    print(f"Episode {episode} completed")

env.close()
```

--------------------------------

### grid4x4 Parallel API Example

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/api-reference/resco-environments.md

Initializes and runs the 4x4 grid environment using the Parallel API. Demonstrates resetting the environment, sampling actions for all agents, and stepping through the simulation until all agents are terminated. Closes the environment upon completion.

```python
import sumo_rl

# Parallel API
env = sumo_rl.grid4x4(parallel=True, use_gui=False)
observations = env.reset()
while env.agents:
    actions = {agent: env.action_space(agent).sample() for agent in env.agents}
    observations, rewards, terminations, truncations, infos = env.step(actions)
env.close()
```

--------------------------------

### Custom Environment Wrapper for Logging

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/configuration.md

Example of creating a custom Gymnasium wrapper for Sumo-RL environments to add logging functionality, such as printing rewards and agent information.

```python
from gymnasium import Wrapper

class LoggingWrapper(Wrapper):
    def step(self, action):
        obs, reward, term, trunc, info = super().step(action)
        print(f"Reward: {reward:.3f}, Queue: {info.get('agents_total_stopped', 0)}")
        return obs, reward, term, trunc, info

env = sumo_rl.env(net_file='...', route_file='...')
env = LoggingWrapper(env)
```

--------------------------------

### RewardFunctionMapping Examples

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/types.md

Demonstrates various ways to specify reward functions for multiple traffic signals, including a single function for all, individual mappings, and multi-objective combinations.

```python
# Single function for all
env = SumoEnvironment(..., reward_fn="pressure")

# Different function per signal
reward_mapping = {
    "tls_1": "queue",
    "tls_2": "average-speed",
    "tls_3": lambda ts: -ts.get_total_co2()
}
env = SumoEnvironment(..., reward_fn=reward_mapping)

# Multiple rewards (multi-objective)
env = SumoEnvironment(
    ...,
    reward_fn=["queue", "average-speed"],
    reward_weights=[0.7, 0.3]  # Weighted combination
)
```

--------------------------------

### Multi-Objective Learning in Sumo-RL Grid

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/QUICK-START.md

Configure the environment for multi-objective learning by providing a list of reward functions and optionally weights. This example uses a 4x4 grid.

```python
import sumo_rl
import numpy as np

# Environment with multiple objectives
env = sumo_rl.grid4x4(
    parallel=True,
    reward_fn=['queue', 'average-speed', 'co2'],
    reward_weights=None  # Returns vector rewards
)

observations = env.reset()

# Track individual objectives
objective_totals = {'queue': 0, 'speed': 0, 'co2': 0}

while env.agents:
    actions = {agent: env.action_space(agent).sample() for agent in env.agents}
    observations, rewards, dones, infos = env.step(actions)
    
    # Rewards are now vectors [queue_reward, speed_reward, co2_reward]
    for agent_id, reward_vector in rewards.items():
        objective_totals['queue'] += reward_vector[0]
        objective_totals['speed'] += reward_vector[1]
        objective_totals['co2'] += reward_vector[2]

print(f"Queue objective: {objective_totals['queue']:.2f}")
print(f"Speed objective: {objective_totals['speed']:.2f}")
print(f"CO2 objective: {objective_totals['co2']:.2f}")

env.close()
```

--------------------------------

### Using Benchmark Networks

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/README.md

Instantiate predefined benchmark networks for traffic signal control. These networks are available directly as functions, simplifying setup for standard scenarios.

```python
import sumo_rl

env = sumo_rl.grid4x4(parallel=True)
env = sumo_rl.cologne3(parallel=False, reward_fn='pressure')
```

--------------------------------

### Multi-Agent Environment with RESCO Network

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/QUICK-START.md

Sets up and runs a multi-agent environment using the preset RESCO network with parallel processing. This example shows simultaneous action selection and environment stepping for all agents.

```python
import sumo_rl

# Use preset RESCO network
env = sumo_rl.grid4x4(parallel=True, num_seconds=3600)

observations = env.reset()
total_rewards = {agent: 0 for agent in env.agents}

while env.agents:
    # All agents act simultaneously
    actions = {agent: env.action_space(agent).sample() for agent in env.agents}
    
    # Step environment (all agents at once)
    observations, rewards, terminations, truncations, infos = env.step(actions)
    
    # Track rewards
    for agent in infos:
        if agent in total_rewards:
            total_rewards[agent] += rewards[agent]

for agent, reward in total_rewards.items():
    print(f"{agent}: {reward:.2f}")

env.close()
```

--------------------------------

### Inspect Sumo-RL Environment Setup

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/QUICK-START.md

Resets the Sumo-RL environment and inspects the available agents, their action spaces, and observation shapes. Useful for debugging and understanding the environment configuration.

```python
import sumo_rl

env = sumo_rl.grid4x4(parallel=True)
obs = env.reset()

# Inspect agents
print(f"Agents: {env.agents}")
print(f"Agent count: {len(env.agents)}")

# Inspect spaces
for agent in env.agents:
    print(f"{agent}:")
    print(f"  Action space: {env.action_space(agent)}")
    print(f"  Observation shape: {env.observation_spaces[agent].shape}")

env.close()
```

--------------------------------

### QLAgent Learn Method Example within Interaction Loop

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/api-reference/QLAgent.md

Illustrates the interaction loop where the agent acts, receives feedback from the environment, and learns from the transition. The learn method updates the Q-table based on the observed reward and next state.

```python
# Interaction loop
agent.state = starting_state
for episode in range(100):
    while not done:
        action = agent.act()
        # Apply action to environment, get reward and next_state
        next_state, reward, done, info = env.step(action)
        
        agent.learn(next_state, reward, done)
    
    print(f"Episode {episode} accumulated reward: {agent.acc_reward}")
```

--------------------------------

### Q-Learning Agent: Exploratory vs. Conservative

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/QUICK-START.md

Configures a Q-Learning agent with different exploration strategies. The 'Exploratory' setup uses fast epsilon decay for quicker learning, while 'Conservative' uses slow decay for more stable learning.

```python
from sumo_rl.agents import QLAgent
from sumo_rl.exploration import EpsilonGreedy

# Exploratory (fast learning, possibly unstable)
exploration_exploratory = EpsilonGreedy(
    initial_epsilon=1.0,
    min_epsilon=0.1,
    decay=0.99  # Fast decay
)
agenta_exploratory = QLAgent(
    ...,
    alpha=0.5,     # High learning rate
    gamma=0.9,     # Lower future discounting
    exploration_strategy=exploration_exploratory
)

# Conservative (slow learning, more stable)
exploration_conservative = EpsilonGreedy(
    initial_epsilon=0.5,
    min_epsilon=0.01,
    decay=0.999  # Slow decay
)
agent_conservative = QLAgent(
    ...,
    alpha=0.1,     # Low learning rate
    gamma=0.95,    # Higher future discounting
    exploration_strategy=exploration_conservative
)
```

--------------------------------

### Multi-Agent PettingZoo Environment Setup

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/README.md

Instantiates multi-agent traffic signal control environments using the PettingZoo API. Supports both AEC (agent-by-agent) and Parallel (simultaneous) stepping. Requires network and route files.

```python
import sumo_rl

# AEC API (agent-by-agent stepping)
env = sumo_rl.env(net_file='network.net.xml',
                  route_file='routes.rou.xml')

# Parallel API (simultaneous stepping)
env = sumo_rl.parallel_env(net_file='network.net.xml',
                           route_file='routes.rou.xml')

observations = env.reset()
while env.agents:
    actions = {agent: env.action_space(agent).sample() for agent in env.agents}
    observations, rewards, terminations, truncations, infos = env.step(actions)
```

--------------------------------

### arterial4x4 Parallel API Example

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/api-reference/resco-environments.md

Initializes and runs the 4x4 arterial network environment using the Parallel API. Demonstrates resetting the environment, sampling actions for all agents, and stepping through the simulation for a fixed number of steps. Closes the environment upon completion.

```python
env = sumo_rl.arterial4x4(parallel=True)
observations = env.reset()

# Run for 100 steps
for _ in range(100):
    actions = {agent: env.action_space(agent).sample() for agent in env.agents}
    observations, rewards, terminations, truncations, infos = env.step(actions)

env.close()
```

--------------------------------

### Get Action Space for a Traffic Signal

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/api-reference/SumoEnvironment.md

Retrieves the action space for a specific traffic signal in a multi-agent setup. This indicates the available phase choices for that signal.

```python
def action_spaces(self, ts_id: str) -> gym.spaces.Discrete

```

--------------------------------

### Iterate and Test RESCO Environments

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/api-reference/resco-environments.md

This snippet demonstrates how to iterate through predefined RESCO networks, instantiate them in parallel mode, and perform a basic simulation step. It's useful for testing environment setup and agent interaction.

```python
import sumo_rl

networks = [
    ('grid4x4', sumo_rl.grid4x4),
    ('cologne3', sumo_rl.cologne3),
    ('ingolstadt7', sumo_rl.ingolstadt7),
    ('ingolstadt21', sumo_rl.ingolstadt21),
]

for name, fn in networks:
    env = fn(parallel=True)
    obs = env.reset()
    print(f"{name}: {len(env.agents)} agents")
    
    for _ in range(10):
        actions = {agent: env.action_space(agent).sample() for agent in env.agents}
        obs, rewards, dones, infos = env.step(actions)
    
    env.close()
```

--------------------------------

### Get Observation Space for a Traffic Signal

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/api-reference/SumoEnvironment.md

Retrieves the observation space for a specific traffic signal in a multi-agent setup. This is useful for understanding the structure of observations returned by the environment.

```python
def observation_spaces(self, ts_id: str) -> gym.spaces.Box

```

--------------------------------

### grid4x4 AEC API Example

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/api-reference/resco-environments.md

Initializes and runs the 4x4 grid environment using the AEC API with a specified reward function. Demonstrates resetting the environment, sampling an action for the current agent, and stepping through the simulation. Closes the environment upon completion.

```python
# AEC API
env = sumo_rl.grid4x4(parallel=False, reward_fn='queue')
obs = env.reset()
while env.agents:
    action = env.action_space(env.agent_selection).sample()
    env.step(action)
env.close()
```

--------------------------------

### Using Preset RESCO Benchmark Environments

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/MODULES.md

This snippet shows how to quickly instantiate predefined RESCO benchmark environments. It requires importing the sumo_rl library.

```python
import sumo_rl

# Use preset RESCO networks
env = sumo_rl.grid4x4(parallel=True)
env = sumo_rl.cologne3(parallel=False, reward_fn='pressure')
env = sumo_rl.ingolstadt21(num_seconds=7200)
```

--------------------------------

### Create a Basic Environment

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/README.md

Create and interact with a basic sumo-rl environment. This snippet demonstrates environment creation, reset, stepping through actions, and closing the environment.

```python
import sumo_rl

env = sumo_rl.grid4x4(parallel=True)
observations = env.reset()

for _ in range(100):
    actions = {agent: env.action_space(agent).sample() for agent in env.agents}
    observations, rewards, dones, info = env.step(actions)

env.close()
```

--------------------------------

### DefaultObservationFunction observation_space Example

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/api-reference/ObservationFunction.md

An example demonstrating how to retrieve and print the observation space defined by the DefaultObservationFunction.

```python
obs_space = observation_fn.observation_space()
print(obs_space)  # Box([0. 0. ...], [1. 1. ...], shape=(9,), dtype=float32)
```

--------------------------------

### DefaultObservationFunction Example Usage

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/api-reference/ObservationFunction.md

An example demonstrating how to compute the observation vector using DefaultObservationFunction. It shows the expected shape and a sample output based on specific environment parameters.

```python
from sumo_rl.environment.observations import DefaultObservationFunction

# Assuming 4 green phases and 2 incoming lanes:
obs = traffic_signal.compute_observation()
# obs shape: (4 + 1 + 2*2,) = (9,)
# obs might look like: [1, 0, 0, 0, 1, 0.3, 0.5, 0.1, 0.2]
# Meaning:
#   - Phase 0 is active (phase_onehot = [1, 0, 0, 0])
#   - Min green elapsed (min_green = [1])
#   - Lane 1: 30% density, 10% queue
#   - Lane 2: 50% density, 20% queue
```

--------------------------------

### SumoEnvironment.action_spaces

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/api-reference/SumoEnvironment.md

Retrieves the action space for a specific traffic signal in a multi-agent setup.

```APIDOC
## SumoEnvironment.action_spaces

### Description
Get action space for a specific traffic signal (multi-agent).

### Method
`action_spaces(self, ts_id: str) -> gym.spaces.Discrete`

### Parameters
#### Path Parameters
- **ts_id** (str) - Traffic signal ID.

### Returns
`gymnasium.spaces.Discrete` — Action space with n equal to number of phases.
```

--------------------------------

### Initialize SumoEnvironment with Required Parameters

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/configuration.md

Initialize the SumoEnvironment with the mandatory net_file and route_file parameters. These files define the SUMO network and vehicle routes.

```python
# Run a 2-hour simulation starting at 7:00 AM, actions every 5 seconds
env = SumoEnvironment(
    net_file='network.net.xml',
    route_file='routes.rou.xml',
    begin_time=25200,  # 7:00 AM in seconds
    num_seconds=7200,  # 2 hours
    delta_time=5,
    sumo_seed=42
)
```

--------------------------------

### SumoEnvironment.observation_spaces

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/api-reference/SumoEnvironment.md

Retrieves the observation space for a specific traffic signal in a multi-agent setup.

```APIDOC
## SumoEnvironment.observation_spaces

### Description
Get observation space for a specific traffic signal (multi-agent).

### Method
`observation_spaces(self, ts_id: str) -> gym.spaces.Box`

### Parameters
#### Path Parameters
- **ts_id** (str) - Traffic signal ID.

### Returns
`gymnasium.spaces.Box` — Observation space for the specified traffic signal.
```

--------------------------------

### get_total_queued

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/api-reference/TrafficSignal.md

Gets the total number of vehicles currently halted across all incoming lanes.

```APIDOC
## get_total_queued

### Description
Total number of halted vehicles across all incoming lanes.

### Method
```python
def get_total_queued(self) -> int
```

### Returns
`int` — Count of vehicles with speed < 0.1 m/s.
```

--------------------------------

### Registering Sumo-RL Environment with Gym

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/configuration.md

Demonstrates how to register a Sumo-RL environment with the Gymnasium registry for use with `gym.make`.

```python
import gymnasium as gym
import sumo_rl

# Register custom environment
gym.register(
    id='MyTraffic-v0',
    entry_point='sumo_rl.environment.env:SumoEnvironment',
    kwargs={
        'net_file': 'networks/mynet.net.xml',
        'route_file': 'networks/myroutes.rou.xml',
        'single_agent': True,
    }
)

# Use via gym.make
env = gym.make('MyTraffic-v0')
```

--------------------------------

### EpsilonGreedy Exploration Initialization

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/api-reference/QLAgent.md

Shows how to initialize the EpsilonGreedy exploration strategy with different decay rates for aggressive and slow exploration.

```python
from sumo_rl.exploration import EpsilonGreedy

# Aggressive exploration decay
exploration = EpsilonGreedy(initial_epsilon=1.0, min_epsilon=0.01, decay=0.995)

# Slow exploration decay
exploration = EpsilonGreedy(initial_epsilon=0.5, min_epsilon=0.1, decay=0.999)
```

--------------------------------

### Create and Run a Parallel Sumo Environment

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/api-reference/SumoEnvironment.md

Instantiates a Sumo-RL parallel environment with specified network and route files, then runs a simulation loop. Requires SUMO_HOME environment variable to be set.

```python
import sumo_rl

env = sumo_rl.parallel_env(net_file='nets/RESCO/grid4x4/grid4x4.net.xml',
                           route_file='nets/RESCO/grid4x4/grid4x4.rou.xml',
                           num_seconds=3600)
observations = env.reset()
while env.agents:
    actions = {agent: env.action_space(agent).sample() for agent in env.agents}
    observations, rewards, terminations, truncations, infos = env.step(actions)
env.close()
```

--------------------------------

### get_total_co2

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/api-reference/TrafficSignal.md

Gets the total CO2 emissions generated by vehicles in the incoming lanes, measured in milligrams per second.

```APIDOC
## get_total_co2

### Description
Total CO2 emissions (mg/s) from vehicles in incoming lanes.

### Method
```python
def get_total_co2(self) -> float
```

### Returns
`float` — Emissions in mg/s.
```

--------------------------------

### Save Captured Frames as Video

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/QUICK-START.md

Saves a list of captured frames as an MP4 video file. Requires the 'opencv-python' library to be installed.

```python
import cv2

# Assuming 'frames' is a list of numpy arrays representing image frames
# and each frame is (1080, 1920, 3) in RGB format
if frames:
    out = cv2.VideoWriter('output.mp4', 
                        cv2.VideoWriter_fourcc(*'mp4v'),
                        30, (1920, 1080))
    for frame in frames:
        # Convert RGB to BGR for OpenCV
        bgr_frame = cv2.cvtColor(frame, cv2.COLOR_RGB2BGR)
        out.write(bgr_frame)
    out.release()
else:
    print("No frames to save.")
```

--------------------------------

### Set SUMO_HOME Environment Variable

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/configuration.md

Set the SUMO_HOME environment variable to the path of your SUMO installation. This is required for sumo-rl to locate SUMO.

```bash
export SUMO_HOME="/usr/share/sumo"
```

--------------------------------

### Monitor Rewards and System Info

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/QUICK-START.md

Initialize a Sumo-RL environment with system and per-agent info, then step through the simulation to monitor rewards and system states like mean speed and queued vehicles.

```python
import sumo_rl

env = sumo_rl.env(..., add_system_info=True, add_per_agent_info=True)
obs = env.reset()

while env.agents:
    actions = {agent: env.action_space(agent).sample() for agent in env.agents}
    obs, rewards, dones, infos = env.step(actions)
    
    # Print system state
    print(f"Step: {infos['step']}")
    print(f"  Mean speed: {infos.get('system_mean_speed', 0):.3f}")
    print(f"  Total queued: {infos.get('agents_total_stopped', 0)}")
    print(f"  Mean reward: {np.mean(list(rewards.values())):.3f}")

env.close()
```

--------------------------------

### Multi-agent Environment Initialization

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/configuration.md

Initializes a SumoEnvironment for multi-agent use and demonstrates a basic interaction loop.

```python
env = SumoEnvironment(..., single_agent=False)
obs = env.reset()
for _ in range(100):
    actions = {ts: env.action_spaces(ts).sample() for ts in env.ts_ids}
    obs, rewards, dones, info = env.step(actions)
```

--------------------------------

### Custom Observation Function in Sumo-RL

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/QUICK-START.md

Define a custom observation class by inheriting from ObservationFunction. This example focuses on speed, density, and phase.

```python
import numpy as np
from gymnasium import spaces
from sumo_rl.environment.observations import ObservationFunction
from sumo_rl.environment.env import SumoEnvironment

class SpeedDensityObservation(ObservationFunction):
    """Observation focusing on speed and congestion."""
    
    def __call__(self) -> np.ndarray:
        # Phase one-hot
        phase = [1 if self.ts.green_phase == i else 0 
                for i in range(self.ts.num_green_phases)]
        
        # Average speed (normalized)
        speed = [self.ts.get_average_speed()]
        
        # Lane densities
        density = self.ts.get_lanes_density()
        
        # Combine features
        obs = np.array(phase + speed + density, dtype=np.float32)
        return obs
    
    def observation_space(self) -> spaces.Space:
        size = self.ts.num_green_phases + 1 + len(self.ts.lanes)
        return spaces.Box(low=0, high=1, shape=(size,), dtype=np.float32)

# Use custom observation
env = SumoEnvironment(
    net_file='network.net.xml',
    route_file='routes.rou.xml',
    single_agent=True,
    observation_class=SpeedDensityObservation
)

obs, info = env.reset()
print(f"Observation shape: {obs.shape}")
print(f"Observation space: {env.observation_space}")
```

--------------------------------

### Custom Observation Function

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/README.md

Define a custom observation function for the environment. This example, PressureObservation, includes phase, pressure, and queue information.

```python
from sumo_rl.environment.observations import ObservationFunction
import numpy as np
from gymnasium import spaces

class PressureObservation(ObservationFunction):
    def __call__(self) -> np.ndarray:
        phase = [1 if self.ts.green_phase == i else 0 
                 for i in range(self.ts.num_green_phases)]
        pressure = [self.ts.get_pressure()]
        queue = self.ts.get_lanes_queue()
        return np.array(phase + pressure + queue, dtype=np.float32)
    
    def observation_space(self):
        size = self.ts.num_green_phases + 1 + len(self.ts.lanes)
        return spaces.Box(low=-np.inf, high=np.inf, shape=(size,), dtype=np.float32)

env = sumo_rl.env(
    net_file='network.net.xml',
    route_file='routes.rou.xml',
    observation_class=PressureObservation
)
```

--------------------------------

### Get Total CO2 Emissions

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/api-reference/TrafficSignal.md

Returns the total CO2 emissions in mg/s from vehicles currently within the incoming lanes of the intersection.

```python
def get_total_co2(self) -> float
```

--------------------------------

### SumoEnvironment Constructor

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/api-reference/SumoEnvironment.md

Initializes the SumoEnvironment with specified SUMO network and route files, along with numerous simulation and agent-specific parameters. Use this to set up your traffic control simulation environment.

```python
def __init__(
    self,
    net_file: str,
    route_file: str,
    out_csv_name: Optional[str] = None,
    use_gui: bool = False,
    virtual_display: Tuple[int, int] = (3200, 1800),
    begin_time: int = 0,
    num_seconds: int = 20000,
    max_depart_delay: int = -1,
    waiting_time_memory: int = 1000,
    time_to_teleport: int = -1,
    delta_time: int = 5,
    yellow_time: int = 2,
    min_green: int = 5,
    max_green: int = 50,
    enforce_max_green: bool = False,
    single_agent: bool = False,
    reward_fn: Union[str, Callable, dict, List] = "diff-waiting-time",
    reward_weights: Optional[List[float]] = None,
    observation_class: type[ObservationFunction] = DefaultObservationFunction,
    add_system_info: bool = True,
    add_per_agent_info: bool = True,
    sumo_seed: Union[str, int] = "random",
    ts_ids: Optional[List[str]] = None,
    fixed_ts: bool = False,
    sumo_warnings: bool = True,
    additional_sumo_cmd: Optional[str] = None,
    render_mode: Optional[str] = None,
) -> None
```

--------------------------------

### Get Average Speed

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/api-reference/TrafficSignal.md

Calculates the average speed of vehicles at the intersection, normalized by the speed limits. Returns 1.0 if no vehicles are present.

```python
def get_average_speed(self) -> float
```

--------------------------------

### SumoEnvironment Constructor

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/api-reference/SumoEnvironment.md

Initializes the SumoEnvironment with specified SUMO network and route files, and configures simulation parameters.

```APIDOC
## SumoEnvironment Constructor

### Description
Initializes the SumoEnvironment with specified SUMO network and route files, and configures simulation parameters.

### Method
__init__

### Parameters
#### Path Parameters
- **net_file** (str) - Required - Path to SUMO network file (.net.xml).
- **route_file** (str) - Required - Path to SUMO route file (.rou.xml).

#### Query Parameters
- **out_csv_name** (Optional[str]) - Optional - Path prefix for CSV output file with simulation metrics. If None, no CSV is generated.
- **use_gui** (bool) - Optional - If True, run SUMO with the graphical user interface.
- **virtual_display** (Tuple[int, int]) - Optional - Resolution (width, height) of virtual display for rendering in rgb_array mode.
- **begin_time** (int) - Optional - Simulation start time in seconds. Allows starting partway through a simulation.
- **num_seconds** (int) - Optional - Duration of simulation in seconds.
- **max_depart_delay** (int) - Optional - Maximum time in seconds to wait before inserting a vehicle. -1 means no limit.
- **waiting_time_memory** (int) - Optional - Number of seconds to remember vehicle waiting time history.
- **time_to_teleport** (int) - Optional - Time in seconds before teleporting stuck vehicles to end of edge. -1 disables teleport.
- **delta_time** (int) - Optional - Simulation seconds between agent actions. Agents take new decisions every delta_time seconds.
- **yellow_time** (int) - Optional - Duration in seconds of yellow traffic phase.
- **min_green** (int) - Optional - Minimum duration in seconds of green phase. Traffic lights cannot switch faster.
- **max_green** (int) - Optional - Maximum duration in seconds of green phase. Only enforced if enforce_max_green is True.
- **enforce_max_green** (bool) - Optional - If True, force phase change after max_green seconds.
- **single_agent** (bool) - Optional - If True, environment behaves as single-agent Gymnasium Env. If False, behaves as multi-agent environment.
- **reward_fn** (Union[str, Callable, dict, List]) - Optional - Reward function(s). Can be: built-in name (string), custom callable, dict mapping ts_id to rewards, or list of callables. Built-in options: "diff-waiting-time", "average-speed", "queue", "pressure", "co2".
- **reward_weights** (Optional[List[float]]) - Optional - Weights for linear combination when reward_fn is a list. If None and reward_fn is a list, returns numpy array.
- **observation_class** (type[ObservationFunction]) - Optional - Custom observation class inheriting from ObservationFunction.
- **add_system_info** (bool) - Optional - If True, add global system metrics to info dict (total vehicles, speeds, arrivals, etc.).
- **add_per_agent_info** (bool) - Optional - If True, add per-traffic-signal metrics to info dict (queue length, waiting time, speed).
- **sumo_seed** (Union[str, int]) - Optional - Random seed for SUMO. "random" for random seed, integer for deterministic seed.
- **ts_ids** (Optional[List[str]]) - Optional - List of traffic signal IDs to control. If None, all traffic lights in network are controlled.
- **fixed_ts** (bool) - Optional - If True, follow predefined phase schedule from route file, ignore actions.
- **sumo_warnings** (bool) - Optional - If True, print SUMO warning messages.
- **additional_sumo_cmd** (Optional[str]) - Optional - Additional SUMO command-line arguments as a single string.
- **render_mode** (Optional[str]) - Optional - Rendering mode: "human" (GUI window) or "rgb_array" (returns image array).

### Returns
SumoEnvironment instance initialized with SUMO simulation ready to reset.

### Example
```python
import gymnasium as gym
import sumo_rl
```
```

--------------------------------

### Enable SUMO GUI Rendering

Source: https://github.com/lucasalegre/sumo-rl/blob/main/_autodocs/configuration.md

Activate the SUMO GUI window during simulation by setting `use_gui` to `True`. This allows for visual inspection of the traffic simulation.

```python
# Display SUMO GUI
env = SumoEnvironment(..., use_gui=True)
```