### Full Go1 Environment Setup and Simulation Test

Source: https://gymnasium.farama.org/_downloads/c0f74bd5efb5a7235729ea43e1c3e86a/load_quadruped_model.ipynb

Provides a complete example of setting up the Go1 environment with all specified parameters and includes a basic loop to run the environment for a few steps with random actions. This is useful for verifying the environment setup before training an agent.

```python
def main():
    """Run the final Go1 environment setup."""
    # Note: The original tutorial includes an image showing the Go1 robot in the environment.
    # The image is available at: https://github.com/Kallinteris-Andreas/Gymnasium-kalli/assets/30759571/bf1797a3-264d-47de-b14c-e3c16072f695

    env = gym.make(
        "Ant-v5",
        xml_file="./mujoco_menagerie/unitree_go1/scene.xml",
        forward_reward_weight=1,
        ctrl_cost_weight=0.05,
        contact_cost_weight=5e-4,
        healthy_reward=1,
        main_body=1,
        healthy_z_range=(0.195, 0.75),
        include_cfrc_ext_in_observation=True,
        exclude_current_positions_from_observation=False,
        reset_noise_scale=0.1,
        frame_skip=25,
        max_episode_steps=1000,
        render_mode="rgb_array",  # Change to "human" to visualize
    )

    # Example of running the environment for a few steps
    obs, info = env.reset()

    for _ in range(100):
        action = env.action_space.sample()  # Replace with your agent's action
        obs, reward, terminated, truncated, info = env.step(action)

        if terminated or truncated:
            obs, info = env.reset()

    env.close()
    print("Environment tested successfully!")

    # Now you would typically:
    # 1. Set up your RL algorithm
    # 2. Train the agent
    # 3. Evaluate the agent's performance
```

--------------------------------

### RecordVideo Wrapper Example: Recording Every 200 Steps with Fixed Length

Source: https://gymnasium.farama.org/_modules/gymnasium/wrappers/rendering

This example shows how to use RecordVideo to start a recording every 200 steps and ensure each video is 100 frames long. It also demonstrates setting a seed for the action space.

```python
import os
import gymnasium as gym
env = gym.make("LunarLander-v3", render_mode="rgb_array")
trigger = lambda t: t % 200 == 0
env = RecordVideo(env, video_folder="./save_videos2", step_trigger=trigger, video_length=100, disable_logger=True)
for i in range(5):
    termination, truncation = False, False
    _ = env.reset(seed=123)
    _ = env.action_space.seed(123)
    while not (termination or truncation):
        obs, rew, termination, truncation, info = env.step(env.action_space.sample())

env.close()
len(os.listdir("./save_videos2"))
```

--------------------------------

### AtariPreprocessing Wrapper Example

Source: https://gymnasium.farama.org/_modules/gymnasium/wrappers/atari_preprocessing

Demonstrates how to apply the AtariPreprocessing wrapper to an Atari environment with custom parameters. Ensure ALE is installed.

```python
import gymnasium as gym
import ale_py
gym.register_envs(ale_py)
env = gym.make("ALE/Pong-v5", frameskip=1)
env = AtariPreprocessing(
    env,
    noop_max=10, frame_skip=4, terminal_on_life_loss=True,
    screen_size=84, grayscale_obs=False, grayscale_newaxis=False
)
```

--------------------------------

### Tuple Space Initialization and Sampling Example

Source: https://gymnasium.farama.org/_modules/gymnasium/spaces/tuple

Demonstrates how to initialize a Tuple space with Discrete and Box spaces and then sample from it. Ensure Gymnasium is installed.

```python
from gymnasium.spaces import Tuple, Box, Discrete

observation_space = Tuple((Discrete(2), Box(-1, 1, shape=(2,))), seed=42)
observation_space.sample()
```

--------------------------------

### RescaleAction Wrapper Example (With Scaling)

Source: https://gymnasium.farama.org/_modules/gymnasium/wrappers/vector/vectorize_action

This example demonstrates using the RescaleAction wrapper to rescale actions to a specific range, in this case, [0.0, 1.0]. It shows the setup and then proceeds to take steps with actions, implying a different outcome compared to the unscaled version.

```python
import numpy as np
import gymnasium as gym
from gymnasium.vector import RescaleAction
envs = gym.make_vec("MountainCarContinuous-v0", num_envs=3)
envs = RescaleAction(envs, 0.0, 1.0)
_ = envs.action_space.seed(123)
obs, info = envs.reset(seed=123)
for _ in range(10):
    obs, rew, term, trunc, info = envs.step(0.5 * np.ones((3, 1)))
    
```

--------------------------------

### Import Libraries and Setup Blackjack Environment

Source: https://gymnasium.farama.org/v1.2.3/tutorials/training_agents/blackjack_q_learning

Imports necessary libraries and creates the Blackjack-v1 environment following Sutton & Barto's rules. Ensure Gymnasium version 1.2.3 is installed.

```python
# Author: Till Zemann
# License: MIT License

from __future__ import annotations

from collections import defaultdict

import matplotlib.pyplot as plt
import numpy as np
import seaborn as sns
from matplotlib.patches import Patch
from tqdm import tqdm

import gymnasium as gym


# Let's start by creating the blackjack environment.
# Note: We are going to follow the rules from Sutton & Barto.
# Other versions of the game can be found below for you to experiment.

env = gym.make("Blackjack-v1", sab=True)

```

--------------------------------

### Install Classic Control Dependencies

Source: https://gymnasium.farama.org/environments/classic_control

Install the classic control environments and their dependencies using pip. This command installs the core gymnasium library along with the necessary packages for the classic control module.

```bash
pip install gymnasium[classic-control]
```

--------------------------------

### Install MuJoCo with Gymnasium

Source: https://gymnasium.farama.org/v1.2.3/environments/mujoco

Install the MuJoCo environments and their dependencies for Gymnasium. This command includes the necessary framework for using MuJoCo.

```bash
pip install gymnasium[mujoco]

```

--------------------------------

### RecordVideo Wrapper Example: Recording All Episodes in Chunks

Source: https://gymnasium.farama.org/_modules/gymnasium/wrappers/rendering

This example demonstrates recording all episodes with a video length of 1000 frames, effectively chunking the recordings. It records for 3 episodes.

```python
import os
import gymnasium as gym
env = gym.make("LunarLander-v3", render_mode="rgb_array")
env = RecordVideo(env, video_folder="./save_videos3", video_length=1000, disable_logger=True)
for i in range(3):
    termination, truncation = False, False
    _ = env.reset(seed=123)
    while not (termination or truncation):
        obs, rew, termination, truncation, info = env.step(env.action_space.sample())

env.close()
len(os.listdir("./save_videos3"))
```

--------------------------------

### Install Gymnasium v1.0.0a1

Source: https://gymnasium.farama.org/v1.2.3/gymnasium_release_notes

To install the alpha version, use the specified pip command. Otherwise, the latest stable version (v0.29.1) will be installed.

```bash
pip install gymnasium==1.0.0a1
```

```bash
pip install --pre gymnasium
```

--------------------------------

### RescaleAction: Example without action scaling

Source: https://gymnasium.farama.org/api/vector/wrappers

This example shows the default action behavior of a vectorized environment before applying the RescaleAction wrapper.

```python
import numpy as np
import gymnasium as gym
envs = gym.make_vec("MountainCarContinuous-v0", num_envs=3)
_ = envs.action_space.seed(123)
obs, info = envs.reset(seed=123)
for _ in range(10):
    obs, rew, term, trunc, info = envs.step(0.5 * np.ones((3, 1)))

envs.close()
print(obs)
```

--------------------------------

### Array Conversion Wrapper Example

Source: https://gymnasium.farama.org/_modules/gymnasium/wrappers/array_conversion

Demonstrates how to wrap an environment to convert observations and actions between different array frameworks like JAX and PyTorch. The example shows type checking of observations and actions after wrapping.

```python
>>> import torch                                                # doctest: +SKIP
>>> import jax.numpy as jnp                                     # doctest: +SKIP
>>> import gymnasium as gym                                     # doctest: +SKIP
>>> env = gym.make("JaxEnv-vx")                                 # doctest: +SKIP
>>> env = ArrayConversion(env, env_xp=jnp, target_xp=torch)     # doctest: +SKIP
>>> obs, _ = env.reset(seed=123)                                # doctest: +SKIP
>>> type(obs)                                                   # doctest: +SKIP
<class 'torch.Tensor'>
>>> action = torch.tensor(env.action_space.sample())            # doctest: +SKIP
>>> obs, reward, terminated, truncated, info = env.step(action) # doctest: +SKIP
>>> type(obs)                                                   # doctest: +SKIP
<class 'torch.Tensor'>
>>> type(reward)                                                # doctest: +SKIP
<class 'float'>
>>> type(terminated)                                            # doctest: +SKIP
<class 'bool'>
>>> type(truncated)                                             # doctest: +SKIP
<class 'bool'>
```

--------------------------------

### Example Usage of ObstructView

Source: https://gymnasium.farama.org/_modules/gymnasium/wrappers/rendering

Demonstrates how to wrap an environment with ObstructView and HumanRendering to observe the effect of obstructed pixels on the rendered output. This example shows a typical setup for testing rendering modifications.

```python
env = gym.make("LunarLander-v3", render_mode="rgb_array")
>>> env = ObstructView(env, obstructed_pixels_ratio=0.5, obstruction_width=50)
>>> env = HumanRendering(env)
>>> obs, _ = env.reset(seed=123)
>>> obs, *_ = env.step(env.action_space.sample())
```

--------------------------------

### Run Go1 Environment Setup and Test

Source: https://gymnasium.farama.org/tutorials/gymnasium_basics/load_quadruped_model

This snippet sets up the Go1 environment with specific parameters and runs it for a few steps to test functionality. It includes options for reward weights, cost weights, and observation configurations. Change `render_mode` to "human" to visualize the robot's behavior.

```python
def main():
    """Run the final Go1 environment setup."""
    # Note: The original tutorial includes an image showing the Go1 robot in the environment.
    # The image is available at: https://github.com/Kallinteris-Andreas/Gymnasium-kalli/assets/30759571/bf1797a3-264d-47de-b14c-e3c16072f695

    env = gym.make(
        "Ant-v5",
        xml_file="./mujoco_menagerie/unitree_go1/scene.xml",
        forward_reward_weight=1,
        ctrl_cost_weight=0.05,
        contact_cost_weight=5e-4,
        healthy_reward=1,
        main_body=1,
        healthy_z_range=(0.195, 0.75),
        include_cfrc_ext_in_observation=True,
        exclude_current_positions_from_observation=False,
        reset_noise_scale=0.1,
        frame_skip=25,
        max_episode_steps=1000,
        render_mode="rgb_array",  # Change to "human" to visualize
    )

    # Example of running the environment for a few steps
    obs, info = env.reset()

    for _ in range(100):
        action = env.action_space.sample()  # Replace with your agent's action
        obs, reward, terminated, truncated, info = env.step(action)

        if terminated or truncated:
            obs, info = env.reset()

    env.close()
    print("Environment tested successfully!")

    # Now you would typically:
    # 1. Set up your RL algorithm
    # 2. Train the agent
    # 3. Evaluate the agent's performance

```

--------------------------------

### VectorizeTransformAction Wrapper Example

Source: https://gymnasium.farama.org/_modules/gymnasium/wrappers/vector/vectorize_action

This example demonstrates how to use the VectorizeTransformAction wrapper to apply a ReLU transformation to actions in a vectorized environment. It shows the setup with `gym.make_vec` and `VectorizeTransformAction`, followed by resetting the environment and taking a step with sampled actions.

```python
import gymnasium as gym
from gymnasium.wrappers import TransformAction
from gymnasium.vector import VectorizeTransformAction
envs = gym.make_vec("MountainCarContinuous-v0", num_envs=3)
envs = VectorizeTransformAction(envs, wrapper=TransformAction, func=lambda x: (x > 0.0) * x, action_space=envs.single_action_space)
_ = envs.action_space.seed(123)
obs, info = envs.reset(seed=123)
obs, rew, term, trunc, info = envs.step(envs.action_space.sample())
envs.close()
print(obs)
```

--------------------------------

### RecordVideo Wrapper Initialization with Custom Triggers

Source: https://gymnasium.farama.org/_modules/gymnasium/wrappers/rendering

Demonstrates initializing the `RecordVideo` wrapper with custom episode and step triggers for recording. If both triggers are `None`, a default cubic schedule is used.

```python
from gymnasium.wrappers import RecordVideo
import gymnasium as gym
from gymnasium.utils.save_video import capped_cubic_video_schedule

env = gym.make("CartPole-v1", render_mode="rgb_array")

# Example custom triggers
def episode_trigger(episode_id):
    return episode_id % 5 == 0

def step_trigger(step_id):
    return step_id % 50 == 0

env = RecordVideo(env, video_folder="./video", episode_trigger=episode_trigger, step_trigger=step_trigger)

env.reset()
for _ in range(100):
    action = env.action_space.sample()
    obs, reward, terminated, truncated, info = env.step(action)
    if terminated or truncated:
        env.reset()
env.close()

```

--------------------------------

### Creating and Using Custom Environments

Source: https://gymnasium.farama.org/introduction/create_custom_env

Demonstrates how to create instances of a registered custom environment using `gymnasium.make()`. Shows how to customize parameters and create vectorized environments.

```python
import gymnasium as gym

# Create the environment like any built-in environment
>>> env = gym.make("gymnasium_env/GridWorld-v0")
<OrderEnforcing<PassiveEnvChecker<GridWorld<gymnasium_env/GridWorld-v0>>>>

# Customize environment parameters
>>> env = gym.make("gymnasium_env/GridWorld-v0", size=10)
>>> env.unwrapped.size
10

# Create multiple environments for parallel training
>>> vec_env = gym.make_vec("gymnasium_env/GridWorld-v0", num_envs=3)
SyncVectorEnv(gymnasium_env/GridWorld-v0, num_envs=3)
```

--------------------------------

### SyncVectorEnv Initialization and Usage Example

Source: https://gymnasium.farama.org/_modules/gymnasium/vector/sync_vector_env

Demonstrates how to create and use a SyncVectorEnv with multiple environments. It shows resetting the environment, sampling actions, and stepping through the environment.

```python
>>> import gymnasium as gym
>>> envs = gym.make_vec("Pendulum-v1", num_envs=2, vectorization_mode="sync")
>>> envs
SyncVectorEnv(Pendulum-v1, num_envs=2)
>>> envs = gym.vector.SyncVectorEnv([
...     lambda: gym.make("Pendulum-v1", g=9.81),
...     lambda: gym.make("Pendulum-v1", g=1.62)
... ])
>>> envs
SyncVectorEnv(num_envs=2)
>>> obs, infos = envs.reset(seed=42)
>>> obs
array([[-0.14995256,  0.9886932 , -0.12224312],
       [ 0.5760367 ,  0.8174238 , -0.91244936]], dtype=float32)
>>> infos
{}
>>> _ = envs.action_space.seed(42)
>>> actions = envs.action_space.sample()
>>> obs, rewards, terminates, truncates, infos = envs.step(actions)
>>> obs
array([[-0.1878752 ,  0.98219293,  0.7695615 ],
       [ 0.6102389 ,  0.79221743, -0.8498053 ]], dtype=float32)
>>> rewards
array([-2.96562607, -0.99902063])
>>> terminates
array([False, False])
>>> truncates
array([False, False])
>>> infos
{}
>>> envs.close()
```

--------------------------------

### Create and Wrap a Vector Environment

Source: https://gymnasium.farama.org/_modules/gymnasium/vector/vector_env

Demonstrates creating a synchronous vector environment with a specific number of environments and applying a wrapper. The example shows how to access environment properties like `num_envs`, `action_space`, and `observation_space`.

```python
import gymnasium as gym
envs = gym.make_vec("CartPole-v1", num_envs=3, vectorization_mode="sync", wrappers=(gym.wrappers.TimeAwareObservation,))
envs = gym.wrappers.vector.ClipReward(envs, min_reward=0.2, max_reward=0.8)
print(envs)
print(envs.num_envs)
print(envs.action_space)
print(envs.observation_space)
```

--------------------------------

### Initialize Vectorized Environments and Wrappers

Source: https://gymnasium.farama.org/v1.2.3/tutorials/training_agents/vector_a2c

Sets up the vectorized environments and wraps them with RecordEpisodeStatistics to track episode returns and lengths. This is crucial for monitoring training progress.

```python
envs_wrapper = gym.wrappers.vector.RecordEpisodeStatistics(
    envs, buffer_length=n_envs * n_updates
)
```

--------------------------------

### Resetting a Vector Environment

Source: https://gymnasium.farama.org/_modules/gymnasium/vector/vector_env

Resets all parallel environments and returns a batch of initial observations and info. Use this to get the starting state for your vectorized environments.

```python
>>> import gymnasium as gym
>>> envs = gym.make_vec("CartPole-v1", num_envs=3, vectorization_mode="sync")
>>> observations, infos = envs.reset(seed=42)
>>> observations
array([[ 0.0273956 , -0.00611216,  0.03585979,  0.0197368 ],
       [ 0.01522993, -0.04562247, -0.04799704,  0.03392126],
       [-0.03774345, -0.02418869, -0.00942293,  0.0469184 ]],
      dtype=float32)
>>> infos
{}
```

--------------------------------

### AsyncVectorEnv Example Usage

Source: https://gymnasium.farama.org/_modules/gymnasium/vector/async_vector_env

Demonstrates how to create and use an AsyncVectorEnv with multiple environments. It shows resetting the environment, sampling actions, and stepping through the environment, along with the expected outputs for observations, rewards, terminations, and truncations.

```python
>>> import gymnasium as gym
>>> envs = gym.make_vec("Pendulum-v1", num_envs=2, vectorization_mode="async")
>>> envs
AsyncVectorEnv(Pendulum-v1, num_envs=2)
>>> envs = gym.vector.AsyncVectorEnv([
...     lambda: gym.make("Pendulum-v1", g=9.81),
...     lambda: gym.make("Pendulum-v1", g=1.62)
... ])
>>> envs
AsyncVectorEnv(num_envs=2)
>>> observations, infos = envs.reset(seed=42)
>>> observations
array([[-0.14995256,  0.9886932 , -0.12224312],
       [ 0.5760367 ,  0.8174238 , -0.91244936]], dtype=float32)
>>> infos
{}
>>> _ = envs.action_space.seed(123)
>>> observations, rewards, terminations, truncations, infos = envs.step(envs.action_space.sample())
>>> observations
array([[-0.1851753 ,  0.98270553,  0.714599  ],
       [ 0.6193494 ,  0.7851154 , -1.0808398 ]], dtype=float32)
>>> rewards
array([-2.96495728, -1.00214607])
>>> terminations
array([False, False])
>>> truncations
array([False, False])
>>> infos
{}
```

--------------------------------

### Get Auxiliary Information from Environment

Source: https://gymnasium.farama.org/tutorials/gymnasium_basics/environment_creation

A helper method to compute and return auxiliary information about the environment's current state. This example returns the Manhattan distance between the agent and the target.

```python
def _get_info(self):
    return {
        "distance": np.linalg.norm(
            self._agent_location - self._target_location,
            ord=1,
        )
    }
```

--------------------------------

### Reset Blackjack Environment and Get Initial Observation

Source: https://gymnasium.farama.org/_downloads/e28b5f09e7a318b73abe9bfdad36324d/blackjack_q_learning.ipynb

Resets the Blackjack environment to its initial state and retrieves the first observation. The `done` flag is set to `False` to indicate the start of an episode.

```python
# reset the environment to get the first observation
done = False
observation, info = env.reset()

# observation = (16, 9, False)
```

--------------------------------

### Discrete Space Initialization

Source: https://gymnasium.farama.org/_modules/gymnasium/spaces/discrete

Demonstrates how to initialize a Discrete space with different configurations.

```APIDOC
## Discrete Space Initialization

### Description
Initializes a discrete space with a specified number of values.

### Parameters
- **n** (int): The number of discrete values.
- **start** (int, optional): The starting value of the discrete space. Defaults to 0.
- **dtype** (np.dtype, optional): The data type of the space. Defaults to np.int64.

### Example
```python
from gymnasium.spaces import Discrete
import numpy as np

# Basic discrete space with 5 values (0, 1, 2, 3, 4)
discrete_space_1 = Discrete(5)

# Discrete space with 5 values starting from 1 (1, 2, 3, 4, 5)
discrete_space_2 = Discrete(5, start=1)

# Discrete space with 5 values and a specific dtype
discrete_space_3 = Discrete(5, dtype=np.int32)
```
```

--------------------------------

### Initialize and Wrap Environment

Source: https://gymnasium.farama.org/api/wrappers

Demonstrates initializing a base environment and then wrapping it with `RescaleAction` to modify the action space. The original action space is [-1.0, 1.0], and the wrapped action space becomes [0.0, 1.0].

```python
>>> import gymnasium as gym
>>> from gymnasium.wrappers import RescaleAction
>>> base_env = gym.make("Hopper-v4")
>>> base_env.action_space
Box(-1.0, 1.0, (3,), float32)
>>> wrapped_env = RescaleAction(base_env, min_action=0, max_action=1)
>>> wrapped_env.action_space
Box(0.0, 1.0, (3,), float32)
```

--------------------------------

### Check Copier installation

Source: https://gymnasium.farama.org/v1.2.3/tutorials/gymnasium_basics/environment_creation

Verify that Copier has been installed correctly by checking its version. This command should output the installed version number.

```bash
copier --version

```

--------------------------------

### Vector Environment Creation and Usage Example

Source: https://gymnasium.farama.org/v1.2.3/_modules/gymnasium/vector/vector_env

Demonstrates creating a vectorized environment with specific wrappers and vectorization mode. It also shows how to apply further wrappers like `ClipReward` and interact with the environment using `reset` and `step`.

```python
import gymnasium as gym
envs = gym.make_vec("CartPole-v1", num_envs=3, vectorization_mode="sync", wrappers=(gym.wrappers.TimeAwareObservation,))
envs = gym.wrappers.vector.ClipReward(envs, min_reward=0.2, max_reward=0.8)
envs

```

```python
envs.num_envs
```

```python
envs.action_space
```

```python
envs.observation_space
```

```python
observations, infos = envs.reset(seed=123)
observations
```

```python
infos
```

```python
_ = envs.action_space.seed(123)
actions = envs.action_space.sample()
observations, rewards, terminations, truncations, infos = envs.step(actions)
observations
```

--------------------------------

### Install MuJoCo with Gymnasium

Source: https://gymnasium.farama.org/environments/mujoco

Install the MuJoCo environments for Gymnasium. This command installs the necessary dependencies for using MuJoCo with Gymnasium.

```bash
pip install gymnasium[mujoco]
```

--------------------------------

### AsyncVectorEnv Example Usage

Source: https://gymnasium.farama.org/v1.2.3/_modules/gymnasium/vector/async_vector_env

Demonstrates how to create and use an AsyncVectorEnv. It shows creating vectorized environments using gym.make with a specified vectorization mode, and then performing reset and step operations.

```python
>>> import gymnasium as gym
>>> envs = gym.make("Pendulum-v1", num_envs=2, vectorization_mode="async")
>>> envs
AsyncVectorEnv(Pendulum-v1, num_envs=2)
```

```python
>>> envs = gym.vector.AsyncVectorEnv([
...     lambda: gym.make("Pendulum-v1", g=9.81),
...     lambda: gym.make("Pendulum-v1", g=1.62)
... ])
>>> envs
AsyncVectorEnv(num_envs=2)
```

```python
>>> observations, infos = envs.reset(seed=42)
>>> observations
array([[-0.14995256,  0.9886932 , -0.12224312],
       [ 0.5760367 ,  0.8174238 , -0.91244936]], dtype=float32)
>>> infos
{}
```

```python
>>> _ = envs.action_space.seed(123)
>>> observations, rewards, terminations, truncations, infos = envs.step(envs.action_space.sample())
>>> observations
array([[-0.1851753 ,  0.98270553,  0.714599  ],
       [ 0.6193494 ,  0.7851154 , -1.0808398 ]], dtype=float32)
>>> rewards
array([-2.96495728, -1.00214607])
>>> terminations
array([False, False])
>>> truncations
array([False, False])
>>> infos
{}
```

--------------------------------

### Initialize Environment and Agent

Source: https://gymnasium.farama.org/introduction/train_agent

Sets up the Blackjack environment and the BlackjackAgent with specified hyperparameters. Ensure Gymnasium and necessary wrappers are imported.

```python
import gymnasium as gym

learning_rate = 0.01        # How fast to learn (higher = faster but less stable)
n_episodes = 100_000        # Number of hands to practice
start_epsilon = 1.0         # Start with 100% random actions
epsilon_decay = start_epsilon / (n_episodes / 2)  # Reduce exploration over time
final_epsilon = 0.1         # Always keep some exploration

# Create environment and agent
env = gym.make("Blackjack-v1", sab=False)
env = gym.wrappers.RecordEpisodeStatistics(env, buffer_length=n_episodes)

agent = BlackjackAgent(
    env=env,
    learning_rate=learning_rate,
    initial_epsilon=start_epsilon,
    epsilon_decay=epsilon_decay,
    final_epsilon=final_epsilon,
)
```

--------------------------------

### RescaleAction Example - With Scaling

Source: https://gymnasium.farama.org/v1.2.3/_modules/gymnasium/wrappers/vector/vectorize_action

Example demonstrating `RescaleAction` which affinely rescales the continuous action space to a specified range. This example rescales actions to [0.0, 1.0].

```python
import numpy as np
import gymnasium as gym
envs = gym.make_vec("MountainCarContinuous-v0", num_envs=3)
envs = RescaleAction(envs, 0.0, 1.0)
_ = envs.action_space.seed(123)
obs, info = envs.reset(seed=123)
for _ in range(10):
    obs, rew, term, trunc, info = envs.step(0.5 * np.ones((3, 1)))
    
```

--------------------------------

### Make HalfCheetah Environment with Custom Arguments

Source: https://gymnasium.farama.org/environments/mujoco/half_cheetah

Demonstrates how to instantiate the HalfCheetah environment with custom reward weights and reset noise scale. These parameters allow fine-tuning the agent's behavior and starting conditions.

```python
import gymnasium as gym
env = gym.make('HalfCheetah-v5', ctrl_cost_weight=0.1, ....)
```

--------------------------------

### Install Copier with Conda

Source: https://gymnasium.farama.org/_downloads/0f28446f9f426c9833f40d61857a6f21/environment_creation.ipynb

Install Copier using Conda.

```console
conda install -c conda-forge copier
```

--------------------------------

### Install Copier with Pip

Source: https://gymnasium.farama.org/_downloads/0f28446f9f426c9833f40d61857a6f21/environment_creation.ipynb

Install Copier using pip.

```console
pip install copier
```

--------------------------------

### Check Copier Installation

Source: https://gymnasium.farama.org/_downloads/0f28446f9f426c9833f40d61857a6f21/environment_creation.ipynb

Verify that Copier has been installed correctly by checking its version.

```console
copier --version
```

--------------------------------

### Creating and Using Vector Environments

Source: https://gymnasium.farama.org/api/vector

Demonstrates how to create a vectorized environment using `make_vec`, apply wrappers, and interact with it through `reset` and `step`. Shows how to access environment attributes like `num_envs` and `action_space`.

```python
>>> import gymnasium as gym
>>> envs = gym.make_vec("CartPole-v1", num_envs=3, vectorization_mode="sync", wrappers=(gym.wrappers.TimeAwareObservation,))
>>> envs = gym.wrappers.vector.ClipReward(envs, min_reward=0.2, max_reward=0.8)
>>> envs
<ClipReward, SyncVectorEnv(CartPole-v1, num_envs=3)>
>>> envs.num_envs
3
>>> envs.action_space
MultiDiscrete([2 2 2])
>>> envs.observation_space
Box([[-4.80000019        -inf -0.41887903        -inf  0.        ]
 [-4.80000019        -inf -0.41887903        -inf  0.        ]
 [-4.80000019        -inf -0.41887903        -inf  0.        ]], [[4.80000019e+00            inf 4.18879032e-01            inf
  5.00000000e+02]
 [4.80000019e+00            inf 4.18879032e-01            inf
  5.00000000e+02]
 [4.80000019e+00            inf 4.18879032e-01            inf
  5.00000000e+02]], (3, 5), float64)
>>> observations, infos = envs.reset(seed=123)
>>> observations
array([[ 0.01823519, -0.0446179 , -0.02796401, -0.03156282,  0.        ],
       [ 0.02852531,  0.02858594,  0.0469136 ,  0.02480598,  0.        ],
       [ 0.03517495, -0.000635  , -0.01098382, -0.03203924,  0.        ]])
>>> infos
{}
>>> _ = envs.action_space.seed(123)
>>> actions = envs.action_space.sample()
>>> observations, rewards, terminations, truncations, infos = envs.step(actions)
>>> observations
array([[ 0.01734283,  0.15089367, -0.02859527, -0.33293587,  1.        ],
       [ 0.02909703, -0.16717631,  0.04740972,  0.3319138 ,  1.        ],
       [ 0.03516225, -0.19559774, -0.01162461,  0.25715804,  1.        ]])
>>> rewards
array([0.8, 0.8, 0.8])
>>> terminations
array([False, False, False])
>>> truncations
array([False, False, False])
>>> infos
{}
>>> envs.close()
```

--------------------------------

### Install Copier with pipx

Source: https://gymnasium.farama.org/_downloads/0f28446f9f426c9833f40d61857a6f21/environment_creation.ipynb

Install Copier using pipx for managing Python applications.

```console
pipx install copier
```

--------------------------------

### Vector Environment Example

Source: https://gymnasium.farama.org/api/vector

Demonstrates how to create, interact with, and inspect a vectorized environment using `gym.make_vec`. This example shows creating a `SyncVectorEnv` with wrappers, accessing its attributes like `num_envs` and action/observation spaces, and performing `reset` and `step` operations.

```APIDOC
### Examples
```python
>>> import gymnasium as gym
>>> envs = gym.make_vec("CartPole-v1", num_envs=3, vectorization_mode="sync", wrappers=(gym.wrappers.TimeAwareObservation,))
>>> envs = gym.wrappers.vector.ClipReward(envs, min_reward=0.2, max_reward=0.8)
>>> envs
<ClipReward, SyncVectorEnv(CartPole-v1, num_envs=3)>
>>> envs.num_envs
3
>>> envs.action_space
MultiDiscrete([2 2 2])
>>> envs.observation_space
Box([[-4.80000019        -inf -0.41887903        -inf  0.        ]
 [-4.80000019        -inf -0.41887903        -inf  0.        ]
 [-4.80000019        -inf -0.41887903        -inf  0.        ]], [[4.80000019e+00            inf 4.18879032e-01            inf
  5.00000000e+02]
 [4.80000019e+00            inf 4.18879032e-01            inf
  5.00000000e+02]
 [4.80000019e+00            inf 4.18879032e-01            inf
  5.00000000e+02]], (3, 5), float64)
>>> observations, infos = envs.reset(seed=123)
>>> observations
array([[ 0.01823519, -0.0446179 , -0.02796401, -0.03156282,  0.        ],
       [ 0.02852531,  0.02858594,  0.0469136 ,  0.02480598,  0.        ],
       [ 0.03517495, -0.000635  , -0.01098382, -0.03203924,  0.        ]])
>>> infos
{}
>>> _ = envs.action_space.seed(123)
>>> actions = envs.action_space.sample()
>>> observations, rewards, terminations, truncations, infos = envs.step(actions)
>>> observations
array([[ 0.01734283,  0.15089367, -0.02859527, -0.33293587,  1.        ],
       [ 0.02909703, -0.16717631,  0.04740972,  0.3319138 ,  1.        ],
       [ 0.03516225, -0.19559774, -0.01162461,  0.25715804,  1.        ]])
>>> rewards
array([0.8, 0.8, 0.8])
>>> terminations
array([False, False, False])
>>> truncations
array([False, False, False])
>>> infos
{}
>>> envs.close()
```
```

--------------------------------

### Install Copier with conda

Source: https://gymnasium.farama.org/v1.2.3/tutorials/gymnasium_basics/environment_creation

Alternative method to install Copier using conda. Use this if you are managing your environment with conda.

```bash
conda install -c conda-forge copier

```

--------------------------------

### Install Copier with pip

Source: https://gymnasium.farama.org/v1.2.3/tutorials/gymnasium_basics/environment_creation

Alternative method to install Copier using pip. Use this if pipx is not preferred or available.

```bash
pip install copier

```

--------------------------------

### Graph Space Initialization and Sampling

Source: https://gymnasium.farama.org/_modules/gymnasium/spaces/graph

Demonstrates how to initialize a Graph space with specified node and edge spaces, and how to sample a graph instance with a given number of nodes and edges.

```python
from gymnasium.spaces import Graph, Box, Discrete
observation_space = Graph(node_space=Box(low=-100, high=100, shape=(3,)), edge_space=Discrete(3), seed=123)
observation_space.sample(num_nodes=4, num_edges=8)
```

--------------------------------

### Install Box2D Dependencies

Source: https://gymnasium.farama.org/v1.2.3/environments/box2d

Install SWIG and the Box2D extra for Gymnasium. SWIG is required to build the box2d-py wheel.

```bash
pip install swig
pip install gymnasium[box2d]

```

--------------------------------

### Record Video Wrapper Usage

Source: https://gymnasium.farama.org/api/wrappers/misc_wrappers

Demonstrates how to use the RecordVideo wrapper to save environment interactions as video files. It shows how to set the video folder, chunk length, and disable logging. The example also verifies the number of saved videos.

```python
import os
import gymnasium as gym
env = gym.make("LunarLander-v3", render_mode="rgb_array")
env = RecordVideo(env, video_folder="./save_videos3", video_length=1000, disable_logger=True)
for i in range(3):
    termination, truncation = False, False
    _ = env.reset(seed=123)
    while not (termination or truncation):
        obs, rew, termination, truncation, info = env.step(env.action_space.sample())
env.close()
len(os.listdir("./save_videos3"))
```

--------------------------------

### Install Box2D Dependencies

Source: https://gymnasium.farama.org/environments/box2d

Install SWIG and the Box2D Gymnasium package. SWIG is required for building the box2d-py wheel.

```bash
pip install swig
```

```bash
pip install gymnasium[box2d]
```

--------------------------------

### Initialize RecordVideo Wrapper

Source: https://gymnasium.farama.org/_modules/gymnasium/wrappers/rendering

Initializes the `RecordVideo` wrapper with various configuration options. Ensure the environment's render mode is compatible (e.g., 'rgb_array'). MoviePy is a required dependency.

```python
from gymnasium.wrappers import RecordVideo
import gymnasium as gym

env = gym.make("CartPole-v1", render_mode="rgb_array")
env = RecordVideo(env, video_folder="./video", step_trigger=lambda step: step % 10 == 0)

env.reset()
for _ in range(100):
    action = env.action_space.sample()
    obs, reward, terminated, truncated, info = env.step(action)
    if terminated or truncated:
        env.reset()
env.close()

```

--------------------------------

### Import Libraries and Setup Blackjack Environment

Source: https://gymnasium.farama.org/tutorials/training_agents/blackjack_q_learning

Imports necessary libraries and creates the Blackjack-v1 environment. Follows rules from Sutton & Barto.

```python
# Author: Till Zemann
# License: MIT License

from __future__ import annotations

from collections import defaultdict

import matplotlib.pyplot as plt
import numpy as np
import seaborn as sns
from matplotlib.patches import Patch
from tqdm import tqdm

import gymnasium as gym

# Let's start by creating the blackjack environment.
# Note: We are going to follow the rules from Sutton & Barto.
# Other versions of the game can be found below for you to experiment.

env = gym.make("Blackjack-v1", sab=True)

```

--------------------------------

### Getting the Number of Subspaces

Source: https://gymnasium.farama.org/_modules/gymnasium/spaces/oneof

Shows how to get the total number of subspaces within a OneOf space using the `len()` function.

```python
len(space)
```

--------------------------------

### TimeLimit Wrapper Example

Source: https://gymnasium.farama.org/_modules/gymnasium/wrappers/common

Demonstrates how to use the TimeLimit wrapper to set a maximum number of steps for an environment. The wrapper truncates the episode if the step limit is reached.

```python
from gymnasium.wrappers import TimeLimit
from gymnasium.envs.classic_control import CartPoleEnv

spec = gym.spec("CartPole-v1")
spec.max_episode_steps
>>> 500
env = gym.make("CartPole-v1")
env  # TimeLimit is included within the environment stack
>>> <TimeLimit<OrderEnforcing<PassiveEnvChecker<CartPoleEnv<CartPole-v1>>>>>
env.spec  # doctest: +ELLIPSIS
>>> EnvSpec(id='CartPole-v1', ..., max_episode_steps=500, ...)
env = gym.make("CartPole-v1", max_episode_steps=3)
env.spec  # doctest: +ELLIPSIS
>>> EnvSpec(id='CartPole-v1', ..., max_episode_steps=3, ...)
env = TimeLimit(CartPoleEnv(), max_episode_steps=10)
env
>>> <TimeLimit<CartPoleEnv instance>>
```

```python
env = gym.make("CartPole-v1", max_episode_steps=3)
_ = env.reset(seed=123)
_ = env.action_space.seed(123)
_, _, terminated, truncated, _ = env.step(env.action_space.sample())
print((terminated, truncated))
>>> (False, False)
_, _, terminated, truncated, _ = env.step(env.action_space.sample())
print((terminated, truncated))
>>> (False, False)
_, _, terminated, truncated, _ = env.step(env.action_space.sample())
print((terminated, truncated))
>>> (False, True)
```

--------------------------------

### VectorizeTransformAction Wrapper Example (No Transformation)

Source: https://gymnasium.farama.org/_modules/gymnasium/wrappers/vector/vectorize_action

Shows a basic example of using a vectorized environment without any action transformation, serving as a baseline.

```python
import gymnasium as gym
envs = gym.make_vec("MountainCarContinuous-v0", num_envs=3)
_ = envs.action_space.seed(123)
obs, info = envs.reset(seed=123)
obs, rew, term, trunc, info = envs.step(envs.action_space.sample())
envs.close()
print(obs)
```

--------------------------------

### Example Usage of Observation Wrappers

Source: https://gymnasium.farama.org/api/wrappers/observation_wrappers

Demonstrates how to use observation wrappers with an environment. The output shows the initial observation and the observation after a step, illustrating the effect of the wrapper.

```python
>>> _ = env.action_space.seed(42)
>>> env.step(env.action_space.sample())[0]
{'obs': array([ 0.02727336, -0.20172954,  0.03625453,  0.32351476], dtype=float32), 'time': array([1], dtype=int32)}
```

--------------------------------

### VectorizeTransformReward Example

Source: https://gymnasium.farama.org/_modules/gymnasium/wrappers/vector/vectorize_reward

Vectorizes a single-agent transform reward wrapper, such as `TransformReward`, to work with `VectorEnv`. This example applies a ReLU function to the rewards.

```python
>>> import gymnasium as gym
>>> from gymnasium.wrappers import TransformReward
>>> envs = gym.make_vec("MountainCarContinuous-v0", num_envs=3)
>>> envs = VectorizeTransformReward(envs, wrapper=TransformReward, func=lambda x: (x > 0.0) * x)
>>> _ = envs.action_space.seed(123)
>>> obs, info = envs.reset(seed=123)
>>> obs, rew, term, trunc, info = envs.step(envs.action_space.sample())
>>> envs.close()
>>> rew
array([-0., -0., -0.])
```