Gymnasium

https://gymnasium.farama.org/
Admin
Gymnasium is a maintained fork of OpenAI’s Gym library, providing an API standard for reinforcement...
Tokens:336,963
Snippets:2,973
Trust Score:9.2
Update:3 months ago
Show doc for...
Tokens:
Raw
### Full Go1 Environment Setup and Simulation Test

Source: https://gymnasium.farama.org/_downloads/c0f74bd5efb5a7235729ea43e1c3e86a/load_quadruped_model.ipynb

Provides a complete example of setting up the Go1 environment with all specified parameters and includes a basic loop to run the environment for a few steps with random actions. This is useful for verifying the environment setup before training an agent.

```python
def main():
    """Run the final Go1 environment setup."""
    # Note: The original tutorial includes an image showing the Go1 robot in the environment.
    # The image is available at: https://github.com/Kallinteris-Andreas/Gymnasium-kalli/assets/30759571/bf1797a3-264d-47de-b14c-e3c16072f695

    env = gym.make(
        "Ant-v5",
        xml_file="./mujoco_menagerie/unitree_go1/scene.xml",
        forward_reward_weight=1,
        ctrl_cost_weight=0.05,
        contact_cost_weight=5e-4,
        healthy_reward=1,
        main_body=1,
        healthy_z_range=(0.195, 0.75),
        include_cfrc_ext_in_observation=True,
        exclude_current_positions_from_observation=False,
        reset_noise_scale=0.1,
        frame_skip=25,
        max_episode_steps=1000,
        render_mode="rgb_array",  # Change to "human" to visualize
    )

    # Example of running the environment for a few steps
    obs, info = env.reset()

    for _ in range(100):
        action = env.action_space.sample()  # Replace with your agent's action
        obs, reward, terminated, truncated, info = env.step(action)

        if terminated or truncated:
            obs, info = env.reset()

    env.close()
    print("Environment tested successfully!")

    # Now you would typically:
    # 1. Set up your RL algorithm
    # 2. Train the agent
    # 3. Evaluate the agent's performance
```

--------------------------------

### Initialize Gymnasium Environment

Source: https://gymnasium.farama.org/tutorials/gymnasium_basics/load_quadruped_model

Import necessary libraries and ensure Gymnasium is installed.

```python
import numpy as np

import gymnasium as gym

# Make sure Gymnasium is properly installed
# You can run this in your terminal:
# pip install "gymnasium>=1.0.0"
```

--------------------------------

### Initialize Mountain Car Continuous

Source: https://gymnasium.farama.org/environments/classic_control/mountain_car_continuous

Example of creating the environment and resetting it with custom parameters.

```python
>>> import gymnasium as gym
>>> env = gym.make("MountainCarContinuous-v0", render_mode="rgb_array", goal_velocity=0.1)  # default goal_velocity=0
>>> env
<TimeLimit<OrderEnforcing<PassiveEnvChecker<Continuous_MountainCarEnv<MountainCarContinuous-v0>>>>>>
>>> env.reset(seed=123, options={"low": -0.7, "high": -0.5})  # default low=-0.6, high=-0.4
(array([-0.5635296,  0.       ], dtype=float32), {})
```

--------------------------------

### Initialize Mountain Car Environment

Source: https://gymnasium.farama.org/environments/classic_control/mountain_car

Example of creating the environment with custom arguments and resetting it with specific options.

```python
>>> import gymnasium as gym
>>> env = gym.make("MountainCar-v0", render_mode="rgb_array", goal_velocity=0.1)  # default goal_velocity=0
>>> env
<TimeLimit<OrderEnforcing<PassiveEnvChecker<MountainCarEnv<MountainCar-v0>>>>>>
>>> env.reset(seed=123, options={"x_init": np.pi/2, "y_init": 0.5})  # default x_init=np.pi, y_init=1.0
(array([-0.46352962,  0.        ], dtype=float32), {})
```

--------------------------------

### Define Key-to-Action Mappings

Source: https://gymnasium.farama.org/_modules/gymnasium/utils/play

Examples of different formats for mapping keyboard inputs to environment actions.

```python
>>> key_to_action = {
...    # ...
...    (ord('w'), ord(' ')): 2
...    # ...
... }
```

```python
>>> key_to_action = {
...    # ...
...    ("w", " "): 2
...    # ...
... }
```

```python
>>> key_to_action = {
...    # ...
...    "w ": 2
...    # ...
... }
```

--------------------------------

### Initialize and Configure TimeLimit Wrapper

Source: https://gymnasium.farama.org/api/wrappers/misc_wrappers

Demonstrates importing the TimeLimit wrapper and applying it to an environment, either via gym.make or direct instantiation.

```python
>>> from gymnasium.wrappers import TimeLimit
>>> from gymnasium.envs.classic_control import CartPoleEnv
```

```python
>>> spec = gym.spec("CartPole-v1")
>>> spec.max_episode_steps
500
>>> env = gym.make("CartPole-v1")
>>> env  # TimeLimit is included within the environment stack
<TimeLimit<OrderEnforcing<PassiveEnvChecker<CartPoleEnv<CartPole-v1>>>>>>
>>> env.spec  
EnvSpec(id='CartPole-v1', ..., max_episode_steps=500, ...)
>>> env = gym.make("CartPole-v1", max_episode_steps=3)
>>> env.spec  
EnvSpec(id='CartPole-v1', ..., max_episode_steps=3, ...)
>>> env = TimeLimit(CartPoleEnv(), max_episode_steps=10)
>>> env
<TimeLimit<CartPoleEnv instance>>
```

--------------------------------

### Reset Blackjack Environment and Get Initial Observation

Source: https://gymnasium.farama.org/_downloads/e28b5f09e7a318b73abe9bfdad36324d/blackjack_q_learning.ipynb

Resets the Blackjack environment to its initial state and retrieves the first observation. The `done` flag is set to `False` to indicate the start of an episode.

```python
# reset the environment to get the first observation
done = False
observation, info = env.reset()

# observation = (16, 9, False)
```

--------------------------------

### Discrete(n, seed, start, dtype)

Source: https://gymnasium.farama.org/_modules/gymnasium/spaces/discrete

Constructor for the Discrete space, defining the range of integers {start, ..., start + n - 1}.

```APIDOC
## Constructor: Discrete(n, seed=None, start=0, dtype=np.int64)

### Description
Initializes a new Discrete space representing a finite set of integers.

### Parameters
- **n** (int) - Required - The number of elements in the space.
- **seed** (int | np.random.Generator) - Optional - Seed for the random number generator.
- **start** (int) - Optional - The smallest element of the space (default: 0).
- **dtype** (type) - Optional - The integer data type for the samples (default: np.int64).
```

--------------------------------

### Initialize Half Cheetah Environment

Source: https://gymnasium.farama.org/environments/mujoco/half_cheetah

Demonstrates how to instantiate the Half Cheetah environment with custom configuration parameters using gymnasium.make.

```python
import gymnasium as gym
env = gym.make('HalfCheetah-v5', ctrl_cost_weight=0.1, ....)
```

--------------------------------

### Install Copier with Conda

Source: https://gymnasium.farama.org/_downloads/0f28446f9f426c9833f40d61857a6f21/environment_creation.ipynb

Install Copier using Conda.

```console
conda install -c conda-forge copier
```

--------------------------------

### Instantiating and Vectorizing Environments

Source: https://gymnasium.farama.org/introduction/create_custom_env

Demonstrates creating environment instances, passing custom parameters, and initializing vectorized environments.

```python
import gymnasium as gym

# Create the environment like any built-in environment
>>> env = gym.make("gymnasium_env/GridWorld-v0")
<OrderEnforcing<PassiveEnvChecker<GridWorld<gymnasium_env/GridWorld-v0>>>>

# Customize environment parameters
>>> env = gym.make("gymnasium_env/GridWorld-v0", size=10)
>>> env.unwrapped.size
10

# Create multiple environments for parallel training
>>> vec_env = gym.make_vec("gymnasium_env/GridWorld-v0", num_envs=3)
SyncVectorEnv(gymnasium_env/GridWorld-v0, num_envs=3)
```

--------------------------------

### gymnasium.make("HalfCheetah-v5")

Source: https://gymnasium.farama.org/environments/mujoco/half_cheetah

Initializes the HalfCheetah-v5 environment instance.

```APIDOC
## gymnasium.make("HalfCheetah-v5")

### Description
Initializes the HalfCheetah-v5 environment, a 2D robot simulation based on the work of P. Wawrzyński.

### Usage
```python
import gymnasium as gym
env = gym.make("HalfCheetah-v5")
```

### Action Space
- **Type**: Box(-1.0, 1.0, (6,), float32)
- **Description**: Represents torques applied at 6 hinge joints (back thigh, back shin, back foot, front thigh, front shin, front foot).

### Observation Space
- **Type**: Box(-inf, inf, (17,), float64)
```

--------------------------------

### Install Copier with Pip

Source: https://gymnasium.farama.org/_downloads/0f28446f9f426c9833f40d61857a6f21/environment_creation.ipynb

Install Copier using pip.

```console
pip install copier
```

--------------------------------

### Install Environment Package Locally

Source: https://gymnasium.farama.org/tutorials/gymnasium_basics/environment_creation

Installs the current directory as an editable package.

```bash
pip install -e .
```

--------------------------------

### Initialize Ant Environment

Source: https://gymnasium.farama.org/environments/mujoco/ant

Demonstrates how to instantiate the Ant environment with custom configuration parameters using gymnasium.make.

```python
import gymnasium as gym
env = gym.make('Ant-v5', ctrl_cost_weight=0.5, ...)
```

--------------------------------

### gymnasium.make('HalfCheetah-v5', ...)

Source: https://gymnasium.farama.org/environments/mujoco/half_cheetah

Initializes the HalfCheetah-v5 environment with custom configuration parameters.

```APIDOC
## gymnasium.make('HalfCheetah-v5', ...)

### Description
Initializes the HalfCheetah-v5 environment. Users can modify the observation space, reward function, initial state, and termination conditions by passing arguments.

### Parameters
- **xml_file** (str) - Optional - Path to a MuJoCo model (Default: "half_cheetah.xml")
- **forward_reward_weight** (float) - Optional - Weight for forward_reward term (Default: 1)
- **ctrl_cost_weight** (float) - Optional - Weight for ctrl_cost weight (Default: 0.1)
- **reset_noise_scale** (float) - Optional - Scale of random perturbations of initial position and velocity (Default: 0.1)
- **exclude_current_positions_from_observation** (bool) - Optional - Whether or not to omit the x-coordinate from observations (Default: True)

### Request Example
import gymnasium as gym
env = gym.make('HalfCheetah-v5', ctrl_cost_weight=0.1, forward_reward_weight=1.0)
```

--------------------------------

### Check Copier Installation

Source: https://gymnasium.farama.org/_downloads/0f28446f9f426c9833f40d61857a6f21/environment_creation.ipynb

Verify that Copier has been installed correctly by checking its version.

```console
copier --version
```

--------------------------------

### Install Copier via pipx

Source: https://gymnasium.farama.org/v1.2.3/tutorials/gymnasium_basics/environment_creation

Recommended method for installing the Copier CLI tool.

```bash
pipx install copier
```

--------------------------------

### RecordVideo.__init__

Source: https://gymnasium.farama.org/_modules/gymnasium/wrappers/rendering

Initializes the RecordVideo wrapper to record environment rollouts into a specified folder.

```APIDOC
## RecordVideo.__init__

### Description
Initializes the wrapper to record videos of environment rollouts. It requires an environment that supports a render mode returning image data (e.g., 'rgb_array').

### Parameters
- **env** (gym.Env) - Required - The environment to be wrapped.
- **video_folder** (str) - Required - The directory where recordings will be saved.
- **episode_trigger** (Callable[[int], bool]) - Optional - Function to determine if a recording should start at a specific episode.
- **step_trigger** (Callable[[int], bool]) - Optional - Function to determine if a recording should start at a specific step.
- **video_length** (int) - Optional - Length of recorded snippets. Defaults to 0 (entire episode).
- **name_prefix** (str) - Optional - Prefix for the generated video files.
- **fps** (int) - Optional - Frames per second for the video. Defaults to environment metadata or 30.
- **disable_logger** (bool) - Optional - Whether to disable the moviepy logger.
- **gc_trigger** (Callable[[int], bool]) - Optional - Function to trigger garbage collection after an episode.
```

--------------------------------

### Install Copier

Source: https://gymnasium.farama.org/tutorials/gymnasium_basics/environment_creation

Methods to install the Copier tool required for generating environment templates.

```bash
pipx install copier
```

```bash
pip install copier
```

```bash
conda install -c conda-forge copier
```

--------------------------------

### Initialize Environment with Human Rendering

Source: https://gymnasium.farama.org/_modules/gymnasium/wrappers/rendering

Demonstrates creating an environment with the human render mode enabled.

```python
>>> env = gym.make("phys2d/CartPole-v1", render_mode="human")      # CartPoleJax-v1 doesn't implement human-rendering natively
>>> obs, _ = env.reset()     # This will start rendering to the screen
```

--------------------------------

### Install MuJoCo dependencies

Source: https://gymnasium.farama.org/environments/mujoco

Install the necessary dependencies for running MuJoCo environments in Gymnasium.

```bash
pip install gymnasium[mujoco]
```

--------------------------------

### Install Copier with pipx

Source: https://gymnasium.farama.org/_downloads/0f28446f9f426c9833f40d61857a6f21/environment_creation.ipynb

Install Copier using pipx for managing Python applications.

```console
pipx install copier
```

--------------------------------

### Import Libraries and Setup Blackjack Environment

Source: https://gymnasium.farama.org/v1.2.3/_downloads/e28b5f09e7a318b73abe9bfdad36324d/blackjack_q_learning.ipynb

Imports necessary libraries for Q-learning and sets up the Blackjack-v1 environment. This code initializes the environment for use in the tutorial.

```python
# Author: Till Zemann
# License: MIT License

from __future__ import annotations

from collections import defaultdict

import matplotlib.pyplot as plt
import numpy as np
import seaborn as sns
from matplotlib.patches import Patch
from tqdm import tqdm

import gymnasium as gym


# Let's start by creating the blackjack environment.
# Note: We are going to follow the rules from Sutton & Barto.
# Other versions of the game can be found below for you to experiment.

env = gym.make("Blackjack-v1", sab=True)
```

--------------------------------

### Initialize and use AsyncVectorEnv

Source: https://gymnasium.farama.org/_modules/gymnasium/vector/async_vector_env

Demonstrates creating vectorized environments using both gym.make_vec and a list of environment creation functions, followed by basic interaction methods.

```python
>>> import gymnasium as gym
>>> envs = gym.make_vec("Pendulum-v1", num_envs=2, vectorization_mode="async")
>>> envs
AsyncVectorEnv(Pendulum-v1, num_envs=2)
>>> envs = gym.vector.AsyncVectorEnv([
...     lambda: gym.make("Pendulum-v1", g=9.81),
...     lambda: gym.make("Pendulum-v1", g=1.62)
... ])
>>> envs
AsyncVectorEnv(num_envs=2)
>>> observations, infos = envs.reset(seed=42)
>>> observations
array([[-0.14995256,  0.9886932 , -0.12224312],
       [ 0.5760367 ,  0.8174238 , -0.91244936]], dtype=float32)
>>> infos
{}
>>> _ = envs.action_space.seed(123)
>>> observations, rewards, terminations, truncations, infos = envs.step(envs.action_space.sample())
>>> observations
array([[-0.1851753 ,  0.98270553,  0.714599  ],
       [ 0.6193494 ,  0.7851154 , -1.0808398 ]], dtype=float32)
>>> rewards
array([-2.96495728, -1.00214607])
>>> terminations
array([False, False])
>>> truncations
array([False, False])
>>> infos
{}
```

--------------------------------

### Install Copier via Pip or Conda

Source: https://gymnasium.farama.org/v1.2.3/tutorials/gymnasium_basics/environment_creation

Alternative installation methods for environments where pipx is not preferred.

```bash
pip install copier
```

```bash
conda install -c conda-forge copier
```

--------------------------------

### Install Box2D dependencies

Source: https://gymnasium.farama.org/environments/box2d

Install the necessary SWIG build tool and the gymnasium box2d package.

```bash
pip install swig
pip install gymnasium[box2d]
```

--------------------------------

### Install Atari environments

Source: https://gymnasium.farama.org/v1.2.3/gymnasium_release_notes

Use this command to install Gymnasium with Atari support, which now includes ROMs automatically.

```bash
pip install “gymnasium[atari]”
```

--------------------------------

### gym.make('InvertedPendulum-v5')

Source: https://gymnasium.farama.org/environments/mujoco/inverted_pendulum

Initializes the InvertedPendulum-v5 environment with optional configuration parameters.

```APIDOC
## gymnasium.make('InvertedPendulum-v5')

### Description
Initializes the InvertedPendulum-v5 environment. Users can modify the observation space, reward function, initial state, and termination condition using the provided arguments.

### Parameters
- **xml_file** (str) - Optional - Path to a MuJoCo model. Default: "inverted_pendulum.xml"
- **reset_noise_scale** (float) - Optional - Scale of random perturbations of initial position and velocity. Default: 0.01

### Request Example
import gymnasium as gym
env = gym.make('InvertedPendulum-v5', reset_noise_scale=0.1)
```

--------------------------------

### gymnasium.make('CliffWalking-v1')

Source: https://gymnasium.farama.org/environments/toy_text/cliff_walking

Initializes the Cliff Walking environment.

```APIDOC
## gymnasium.make('CliffWalking-v1')

### Description
Creates an instance of the Cliff Walking environment.

### Usage
```python
import gymnasium as gym
env = gym.make('CliffWalking-v1')
```
```

--------------------------------

### Install Classic Control Dependencies

Source: https://gymnasium.farama.org/environments/classic_control

Use this command to install the necessary dependencies for all classic control environments.

```bash
pip install gymnasium[classic-control]
```

--------------------------------

### Initialize HalfCheetah Environment

Source: https://gymnasium.farama.org/environments/mujoco/half_cheetah

Use this command to instantiate the HalfCheetah-v5 environment.

```python
gymnasium.make("HalfCheetah-v5")
```

--------------------------------

### DictInfoToList Usage Examples

Source: https://gymnasium.farama.org/v1.2.3/_modules/gymnasium/wrappers/vector/dict_info_to_list

Examples demonstrating the conversion of info dictionaries to list format for vectorized environments.

```python
>>> import numpy as np
>>> dict_info = {
...      "k": np.array([0., 0., 0.5, 0.3]),
...      "_k": np.array([False, False, True, True])
...  }
...
>>> list_info = [{}, {}, {"k": 0.5}, {"k": 0.3}]
```

```python
>>> import numpy as np
>>> import gymnasium as gym
>>> envs = gym.make_vec("CartPole-v1", num_envs=3)
>>> obs, info = envs.reset(seed=123)
>>> info
{}
>>> envs = DictInfoToList(envs)
>>> obs, info = envs.reset(seed=123)
>>> info
[{}, {}, {}]
```

```python
>>> import numpy as np
>>> import gymnasium as gym
>>> envs = gym.make_vec("HalfCheetah-v5", num_envs=2)
>>> _ = envs.reset(seed=123)
>>> _ = envs.action_space.seed(123)
>>> _, _, _, _, infos = envs.step(envs.action_space.sample())
>>> infos
{'x_position': array([0.03332211, 0.10172355]), '_x_position': array([ True,  True]), 'x_velocity': array([-0.06296527,  0.89345848]), '_x_velocity': array([ True,  True]), 'reward_forward': array([-0.06296527,  0.89345848]), '_reward_forward': array([ True,  True]), 'reward_ctrl': array([-0.24503504, -0.21944423], dtype=float32), '_reward_ctrl': array([ True,  True])}
>>> envs = DictInfoToList(envs)
>>> _ = envs.reset(seed=123)
>>> _ = envs.action_space.seed(123)
>>> _, _, _, _, infos = envs.step(envs.action_space.sample())
>>> infos  # doctest: +ELLIPSIS
[{'x_position': np.float64(0.0333221...), 'x_velocity': np.float64(-0.0629652...), 'reward_forward': np.float64(-0.0629652...), 'reward_ctrl': np.float32(-0.2450350...)}, {'x_position': np.float64(0.1017235...), 'x_velocity': np.float64(0.8934584...), 'reward_forward': np.float64(0.8934584...), 'reward_ctrl': np.float32(-0.2194442...)}]
```

--------------------------------

### DiscretizeAction Usage Examples

Source: https://gymnasium.farama.org/_modules/gymnasium/wrappers/transform_action

Examples showing how to discretize continuous action spaces into Discrete or MultiDiscrete spaces.

```python
>>> env = gym.make("Pendulum-v1")
>>> env.action_space
Box(-2.0, 2.0, (1,), float32)
>>> obs, _ = env.reset(seed=42)
>>> obs, *_ = env.step([-0.6])
>>> obs
array([-0.17606162,  0.9843792 ,  0.5292768 ], dtype=float32)
>>> env = DiscretizeAction(env, bins=10)
>>> env.action_space
Discrete(10)
>>> obs, _ = env.reset(seed=42)
>>> obs, *_ = env.step(3)
>>> obs
array([-0.17606162,  0.9843792 ,  0.5292768 ], dtype=float32)
```

```python
>>> env = gym.make("Reacher-v5")
>>> env.action_space
Box(-1.0, 1.0, (2,), float32)
>>> obs, _ = env.reset(seed=42)
>>> obs, *_ = env.step([-0.3, -0.5])
>>> obs
array([ 0.99908342,  0.99948506,  0.04280567, -0.03208766,  0.10445588,
        0.11442572, -1.18958125, -1.97979484,  0.1054461 , -0.10896341])
>>> env = DiscretizeAction(env, bins=10)
>>> env.action_space
Discrete(100)
>>> obs, _ = env.reset(seed=42)
>>> obs, *_ = env.step(32)
>>> obs
array([ 0.99908342,  0.99948506,  0.04280567, -0.03208766,  0.10445588,
        0.11442572, -1.18958118, -1.97979484,  0.1054461 , -0.10896341])
```

```python
>>> env = gym.make("Reacher-v5")
>>> env.action_space
Box(-1.0, 1.0, (2,), float32)
>>> obs, _ = env.reset(seed=42)
>>> obs, *_ = env.step([-0.3, -0.5])
>>> obs
array([ 0.99908342,  0.99948506,  0.04280567, -0.03208766,  0.10445588,
        0.11442572, -1.18958125, -1.97979484,  0.1054461 , -0.10896341])
>>> env = DiscretizeAction(env, bins=10, multidiscrete=True)
>>> env.action_space
MultiDiscrete([10 10])
>>> obs, _ = env.reset(seed=42)
>>> obs, *_ = env.step([3, 2])
>>> obs
array([ 0.99908342,  0.99948506,  0.04280567, -0.03208766,  0.10445588,
        0.11442572, -1.18958118, -1.97979484,  0.1054461 , -0.10896341])
```

--------------------------------

### gymnasium.make

Source: https://gymnasium.farama.org/api/env

Initializes a Gymnasium environment.

```APIDOC
## gymnasium.make(id, **kwargs)

### Description
Initializes an environment by its ID along with several important wrappers.

### Parameters
- **id** (str) - Required - The ID of the environment to initialize.
```

--------------------------------

### Install legacy MuJoCo dependencies

Source: https://gymnasium.farama.org/environments/mujoco

Install dependencies for older MuJoCo environment versions that rely on the mujoco-py framework.

```bash
pip install gymnasium_robotics[mujoco-py]
```

--------------------------------

### Sample Valid Actions using Action Mask

Source: https://gymnasium.farama.org/environments/toy_text/taxi

Demonstrates how to use the action mask from the info dictionary to sample valid actions or select actions based on Q-values.

```python
action = env.action_space.sample(info["action_mask"])
```

```python
action = np.argmax(q_values[obs, np.where(info["action_mask"] == 1)[0]])
```

--------------------------------

### Install Custom Environment Package Locally

Source: https://gymnasium.farama.org/_downloads/0f28446f9f426c9833f40d61857a6f21/environment_creation.ipynb

Use this console command to install your custom environment package in editable mode. This allows for immediate testing and development without re-installing after every change.

```console
pip install -e .
```

--------------------------------

### Get sub-environment attributes

Source: https://gymnasium.farama.org/v1.2.3/_modules/gymnasium/vector/sync_vector_env

Retrieves a property value from each parallel environment.

```python
[docs]
    def get_attr(self, name: str) -> tuple[Any, ...]:
        """Get a property from each parallel environment.

        Args:
            name (str): Name of the property to get from each individual environment.

        Returns:
            The property with name
        """
        return self.call(name)
```

--------------------------------

### Using the TimeLimit Wrapper

Source: https://gymnasium.farama.org/_modules/gymnasium/wrappers/common

Demonstrates how to inspect and apply the TimeLimit wrapper to a Gymnasium environment.

```python
>>> from gymnasium.wrappers import TimeLimit
>>> from gymnasium.envs.classic_control import CartPoleEnv

>>> spec = gym.spec("CartPole-v1")
>>> spec.max_episode_steps
500
>>> env = gym.make("CartPole-v1")
>>> env  # TimeLimit is included within the environment stack
<TimeLimit<OrderEnforcing<PassiveEnvChecker<CartPoleEnv<CartPole-v1>>>>>>
>>> env.spec  # doctest: +ELLIPSIS
EnvSpec(id='CartPole-v1', ..., max_episode_steps=500, ...)
>>> env = gym.make("CartPole-v1", max_episode_steps=3)
>>> env.spec  # doctest: +ELLIPSIS
EnvSpec(id='CartPole-v1', ..., max_episode_steps=3, ...)
>>> env = TimeLimit(CartPoleEnv(), max_episode_steps=10)
>>> env
<TimeLimit<CartPoleEnv instance>>
```

--------------------------------

### Get environment attributes

Source: https://gymnasium.farama.org/_modules/gymnasium/vector/sync_vector_env

Retrieves a property value from each parallel environment.

```python
    def get_attr(self, name: str) -> tuple[Any, ...]:
        """Get a property from each parallel environment.

        Args:
            name (str): Name of the property to get from each individual environment.

        Returns:
            The property with name
        """
        return self.call(name)
```

--------------------------------

### gymnasium.make('HumanoidStandup-v5')

Source: https://gymnasium.farama.org/environments/mujoco/humanoid_standup

Initializes the HumanoidStandup-v5 environment with optional configuration parameters.

```APIDOC
## gymnasium.make('HumanoidStandup-v5')

### Description
Initializes the HumanoidStandup-v5 environment. Users can pass various parameters to modify the observation space, reward function, and initial state.

### Parameters
- **xml_file** (str) - Optional - Path to a MuJoCo model. Default: "humanoidstandup.xml"
- **uph_cost_weight** (float) - Optional - Weight for uph_cost term. Default: 1
- **ctrl_cost_weight** (float) - Optional - Weight for quad_ctrl_cost term. Default: 0.1
- **impact_cost_weight** (float) - Optional - Weight for impact_cost term. Default: 0.5e-6
- **impact_cost_range** (float) - Optional - Clamps the impact_cost. Default: (-np.inf, 10.0)
- **reset_noise_scale** (float) - Optional - Scale of random perturbations of initial position and velocity. Default: 1e-2
- **exclude_current_positions_from_observation** (bool) - Optional - Whether or not to omit the x- and y-coordinates from observations. Default: True
- **include_cinert_in_observation** (bool) - Optional - Whether to include cinert elements in the observations. Default: True
- **include_cvel_in_observation** (bool) - Optional - Whether to include cvel elements in the observations. Default: True
- **include_qfrc_actuator_in_observation** (bool) - Optional - Whether to include qfrc_actuator elements in the observations. Default: True
- **include_cfrc_ext_in_observation** (bool) - Optional - Whether to include cfrc_ext elements in the observations. Default: True
```

--------------------------------

### Initialize Vector Environment

Source: https://gymnasium.farama.org/api/vector/wrappers

Demonstrates basic initialization and interaction with a vectorized environment without action transformation.

```python
>>> import gymnasium as gym
>>> envs = gym.make_vec("MountainCarContinuous-v0", num_envs=3)
>>> _ = envs.action_space.seed(123)
>>> obs, info = envs.reset(seed=123)
>>> obs, rew, term, trunc, info = envs.step(envs.action_space.sample())
>>> envs.close()
>>> obs
array([[-4.6343064e-01,  9.8971417e-05],
       [-4.4488689e-01, -1.9375233e-03],
       [-4.3118435e-01, -1.5342437e-03]], dtype=float32)
```

--------------------------------

### AsyncVectorEnv Initialization Logic

Source: https://gymnasium.farama.org/_modules/gymnasium/vector/async_vector_env

The constructor handles environment function registration, shared memory configuration, and metadata extraction from a dummy environment instance.

```python
self.env_fns = env_fns
self.shared_memory = shared_memory
self.copy = copy
self.context = context
self.daemon = daemon
self.worker = worker
self.observation_mode = observation_mode
self.autoreset_mode = (
    autoreset_mode
    if isinstance(autoreset_mode, AutoresetMode)
    else AutoresetMode(autoreset_mode)
)

self.num_envs = len(env_fns)

# This would be nice to get rid of, but without it there's a deadlock between shared memory and pipes
# Create a dummy environment to gather the metadata and observation / action space of the environment
dummy_env = env_fns[0]()

# As we support `make_vec(spec)` then we can't include a `spec = dummy_env.spec` as this doesn't guarantee we can actual recreate the vector env.
self.metadata = dummy_env.metadata
self.metadata["autoreset_mode"] = self.autoreset_mode
self.render_mode = dummy_env.render_mode

self.single_action_space = dummy_env.action_space
self.action_space = batch_space(self.single_action_space, self.num_envs)

if isinstance(observation_mode, tuple) and len(observation_mode) == 2:
    assert isinstance(observation_mode[0], Space)
    assert isinstance(observation_mode[1], Space)
    self.observation_space, self.single_observation_space = observation_mode
else:
    if observation_mode == "same":
        self.single_observation_space = dummy_env.observation_space
        self.observation_space = batch_space(
            self.single_observation_space, self.num_envs
        )
    elif observation_mode == "different":
        # the environment is created and instantly destroy, might cause issues for some environment
        # but I don't believe there is anything else we can do, for users with issues, pre-compute the spaces and use the custom option.
        env_spaces = [env().observation_space for env in self.env_fns]
```

--------------------------------

### State Representation Patterns

Source: https://gymnasium.farama.org/introduction/create_custom_env

Examples of over-informative, under-informative, and optimal observation structures.

```python
# Too much info - agent doesn't need grid size in every observation
obs = {"agent": self._agent_location, "target": self._target_location, "size": self.size}

# Too little info - agent can't distinguish different positions
obs = {"distance": distance}  # Missing actual positions!
```

```python
# Just right - positions are sufficient for navigation
obs = {"agent": self._agent_location, "target": self._target_location}
```

--------------------------------

### Initialize Lunar Lander with custom arguments

Source: https://gymnasium.farama.org/environments/box2d/lunar_lander

Demonstrates how to instantiate the LunarLander-v3 environment with specific physical and control parameters.

```python
>>> import gymnasium as gym
>>> env = gym.make("LunarLander-v3", continuous=False, gravity=-10.0,
...                enable_wind=False, wind_power=15.0, turbulence_power=1.5)
>>> env
<TimeLimit<OrderEnforcing<PassiveEnvChecker<LunarLander<LunarLander-v3>>>>>
```

--------------------------------

### Getting Wrapper Attributes

Source: https://gymnasium.farama.org/_modules/gymnasium/core

Retrieves an attribute from the wrapper or recursively from lower environments.

```python
def get_wrapper_attr(self, name: str) -> Any:
    """Gets an attribute from the wrapper and lower environments if `name` doesn't exist in this object.

    Args:
        name: The variable name to get

    Returns:
        The variable with name in wrapper or lower environments
    """
    if hasattr(self, name):
        return getattr(self, name)
    else:
        try:
            return self.env.get_wrapper_attr(name)
        except AttributeError as e:
            raise AttributeError(
                f"wrapper {self.class_name()} has no attribute {name!r}"
            ) from e
```

--------------------------------

### Initialize and Sample from Discrete Space

Source: https://gymnasium.farama.org/_modules/gymnasium/spaces/discrete

Demonstrates creating a Discrete space with various parameters and sampling from it using masks and probability distributions.

```python
>>> from gymnasium.spaces import Discrete
>>> observation_space = Discrete(2, seed=42) # {0, 1}
>>> observation_space.sample()
np.int64(0)
>>> observation_space = Discrete(3, start=-1, seed=42)  # {-1, 0, 1}
>>> observation_space.sample()
np.int64(-1)
>>> observation_space.sample(mask=np.array([0,0,1], dtype=np.int8))
np.int64(1)
>>> observation_space.sample(probability=np.array([0,0,1], dtype=np.float64))
np.int64(1)
>>> observation_space.sample(probability=np.array([0,0.3,0.7], dtype=np.float64))
np.int64(1)
>>> observation_space = Discrete(3, seed=37, dtype=np.int32) # specify dtype for samples
>>> observation_space.sample()
np.int32(0)
```

--------------------------------

### Get environment seed

Source: https://gymnasium.farama.org/_modules/gymnasium/core

Retrieves the internal random seed, initializing it if necessary.

```python
@property
    def np_random_seed(self) -> int:
        """Returns the environment's internal :attr:`_np_random_seed` that if not set will first initialise with a random int as seed.

        If :attr:`np_random_seed` was set directly instead of through :meth:`reset` or :meth:`set_np_random_through_seed`,
        the seed will take the value -1.

        Returns:
            int: the seed of the current `np_random` or -1, if the seed of the rng is unknown
        """
        if self._np_random_seed is None:
            self._np_random, self._np_random_seed = seeding.np_random()
        return self._np_random_seed
```

--------------------------------

### Initialize AtariPreprocessing

Source: https://gymnasium.farama.org/_modules/gymnasium/wrappers/atari_preprocessing

Demonstrates how to wrap an Atari environment with the AtariPreprocessing class to apply standard preprocessing techniques.

```python
>>> import gymnasium as gym
>>> import ale_py
>>> gym.register_envs(ale_py)
>>> env = gym.make("ALE/Pong-v5", frameskip=1)
>>> env = AtariPreprocessing(
...     env,
...     noop_max=10, frame_skip=4, terminal_on_life_loss=True,
...     screen_size=84, grayscale_obs=False, grayscale_newaxis=False
... )
```

--------------------------------

### env.reset()

Source: https://gymnasium.farama.org/environments/classic_control/pendulum

Resets the environment to a starting state with optional bounds for the initial angle and velocity.

```APIDOC
## env.reset()

### Description
Resets the environment to a random starting state. The options parameter allows customization of the initial state bounds.

### Parameters
- **seed** (int) - Optional - Seed for the random number generator.
- **options** (dict) - Optional - Dictionary containing 'low' and 'high' keys to define the bounds for the initial random state.

### Example
```python
env.reset(seed=123, options={"low": -0.7, "high": 0.5})
```
```

--------------------------------

### Initialize Humanoid Environment

Source: https://gymnasium.farama.org/environments/mujoco/humanoid

Demonstrates how to instantiate the Humanoid-v5 environment with custom parameters using gymnasium.make.

```python
import gymnasium as gym
env = gym.make('Humanoid-v5', ctrl_cost_weight=0.1, ....)
```

--------------------------------

### Reset environment statistics

Source: https://gymnasium.farama.org/_modules/gymnasium/wrappers/common

Resets the episode counters and start time when the environment is reset.

```python
def reset(
    self, *, seed: int | None = None, options: dict[str, Any] | None = None
) -> tuple[ObsType, dict[str, Any]]:
    """Resets the environment using seed and options and resets the episode rewards and lengths."""
    obs, info = super().reset(seed=seed, options=options)

    self.episode_start_time = time.perf_counter()
    self.episode_returns = 0.0
    self.episode_lengths = 0

    return obs, info
```

--------------------------------

### Reward Design Patterns

Source: https://gymnasium.farama.org/introduction/create_custom_env

Examples of sparse versus shaped rewards for agent feedback.

```python
# This makes learning very difficult!
reward = 1 if terminated else 0
```

```python
# Option 1: Small step penalty to encourage efficiency
reward = 1 if terminated else -0.01

# Option 2: Distance-based reward shaping
distance = np.linalg.norm(self._agent_location - self._target_location)
reward = 1 if terminated else -0.1 * distance
```
Tokens:
Raw