Meta-World

https://metaworld.farama.org/
Admin
Meta-World is an open-source simulated benchmark for meta-reinforcement learning and multi-task...
Tokens:8,287
Snippets:118
Trust Score:9.2
Update:6 months ago
Show doc for...
Tokens:
Raw
### Install Metaworld from GitHub

Source: https://metaworld.farama.org/installation/installation

Use this command to install the latest version of Metaworld directly from its GitHub repository.

```bash
pip install git+https://github.com/Farama-Foundation/Metaworld.git@master#egg=metaworld

```

--------------------------------

### Install Metaworld Locally (Editable)

Source: https://metaworld.farama.org/installation/installation

Clone the repository and install Metaworld in editable mode for local development. This allows changes in the source code to be reflected immediately without reinstallation.

```bash
git clone https://github.com/Farama-Foundation/Metaworld.git
cd Metaworld
pip install -e .

```

--------------------------------

### Create and Use MT1 Benchmark

Source: https://metaworld.farama.org/introduction/basic_usage

Example of creating a single MT1 environment and performing a step. Ensure 'reach-v3' is a valid environment name.

```python
import gymnasium as gym
import metaworld

SEED = 0  # some seed number here
env = gym.make('Meta-World/MT1', env_name='reach-v3', seed=seed)
obs, info = env.reset()

a = env.action_space.sample() # randomly sample an action
obs, reward, truncate, terminate, info = env.step(a) # apply the randomly sampled action

```

--------------------------------

### Install Specific Metaworld Version for Paper Reproduction

Source: https://metaworld.farama.org/installation/installation

Install a specific commit hash of Metaworld to ensure reproducibility of results from the Meta-World paper.

```bash
pip install git+https://github.com/Farama-Foundation/Metaworld.git@04be337a12305e393c0caf0cbf5ec7755c7c8feb

```

--------------------------------

### Initialize and Step Environment

Source: https://metaworld.farama.org/rendering/rendering

Demonstrates basic environment interaction: creation with a render mode, resetting, sampling an action, and stepping the environment.

```python
import metaworld
import random

env_name = '' # Pick an environment name

render_mode = '' # set a render mode

env = gym.make('Meta-World/MT1', env_name=env_name, render_mode=render_mode)

obs = env.reset()  # Reset environment
a = env.action_space.sample()  # Sample an action
obs, reward, terminate, truncate, info = env.step(a)  # Step the environment with the sampled random action

```

--------------------------------

### Initialize and step through a Meta-World environment

Source: https://metaworld.farama.org/

Demonstrates how to create a Meta-World environment using Gymnasium and perform a single step with a sampled action.

```python
import gymnasium as gym
import metaworld

env = gym.make('Meta-World/MT1', env_name='reach-v3')

obs = env.reset()
a = env.action_space.sample()
next_obs, reward, terminate, truncate, info = env.step(a)
```

--------------------------------

### gym.make Arguments

Source: https://metaworld.farama.org/introduction/basic_usage

Details the arguments supported by the gym.make command for configuring environments, including seeding, episode steps, wrappers, and rendering.

```APIDOC
## Arguments

The `gym.make` command supports multiple arguments for environment configuration:

| Argument | Usage | Values | Description |
|---|---|---|---|
| `seed` | The number to seed the random number generator with | None or int | Seeds the random number generator. |
| `max_episode_steps` | The maximum number of steps per episode | None or int | Sets the maximum steps allowed in an episode. |
| `use_one_hot` | Whether the one hot wrapper should be used to add the task ID to the observation | True or False | Enables one-hot encoding for task IDs in observations. |
| `num_tasks` | The number of parametric variations to sample | int (default: 50) | Specifies the number of task variations to generate. |
| `terminate_on_success` | Whether to terminate the episode during training when the success signal is seen | True or False | Determines if episodes end upon successful task completion during training. |
| `vector_strategy` | What kind of vector strategy the environments should be wrapped in | ‘sync’ or ‘async’ | Defines the vectorization strategy for environments. |
| `task_select` | How parametric variations should be selected | “random” or “pseudorandom” | Method for selecting task variations. |
| `reward_function_version` | Use the original reward functions from Meta-World or the updated ones | “v1” or “v2” | Selects between original and updated reward functions. |
| `reward_normalization_method` | Apply a reward normalization wrapper | None or ‘gymnasium’ or ‘exponential’ | Specifies the reward normalization technique. |
| `render_mode` | The render mode of each environment | None or ‘human’ or ‘rgb_array’ or ‘depth_array’ | Sets the rendering output format. |
| `camera_name` | The Mujoco name of the camera that should be used to render | ‘corner’, ‘topview’, ‘behindGripper’, ‘gripperPOV’, ‘corner2’, ‘corner3’, ‘corner4’ | Specifies the name of the camera for rendering. |
| `camera_id` | The Mujoco ID of the camera that should be used to render | int | Specifies the ID of the camera for rendering.
```

--------------------------------

### Create Synchronous ML45 Training and Testing Environments

Source: https://metaworld.farama.org/introduction/basic_usage

Instantiates synchronous vector environments for both training and testing phases of the ML45 benchmark.

```python
import gymnasium as gym
import metaworld

train_envs = gym.make_vec('Meta-World/ML45-train', vector_strategy='sync', seed=seed)
test_envs = gym.make_vec('Meta-World/ML45-test', vector_strategy='sync', seed=seed)


```

--------------------------------

### Create Synchronous ML10 Training and Testing Environments

Source: https://metaworld.farama.org/introduction/basic_usage

Instantiates synchronous vector environments for both training and testing phases of the ML10 benchmark.

```python
import gymnasium as gym
import metaworld
train_envs = gym.make_vec('Meta-World/ML10-train', vector_strategy='sync', seed=seed)
test_envs = gym.make_vec('Meta-World/ML10-test', vector_strategy='sync', seed=seed)


```

--------------------------------

### Render from Specific Camera Name

Source: https://metaworld.farama.org/rendering/rendering

Initializes an environment with a specified render mode and camera name for a particular viewpoint.

```python
camera_name = '' # one of: ['corner', 'corner2', 'corner3', 'corner4', 'topview', 'behindGripper', 'gripperPOV']

env = gym.make(env_name=env_name, render_mode=render_mode, camera_name=camera_name)



```

--------------------------------

### Execute Expert Policy for Reach Environment

Source: https://metaworld.farama.org/benchmark/expert_trajectories

Demonstrates how to initialize the reach-v3 environment and execute the corresponding SawyerReachV3Policy until task success.

```python
import gymnasium as gym
import metaworld
from metaworld.policies.sawyer_reach_v3_policy import SawyerReachV3Policy

env = gym.make('Meta-World/MT1', env_name='reach-v3')

obs, info = env.reset()

policy = SawyerReachV3Policy()

done = False

while not done:
    a = policy.get_action(obs)
    obs, _, _, _, info = env.step(a)
    done = int(info['success']) == 1
```

--------------------------------

### Creating Custom Benchmarks

Source: https://metaworld.farama.org/introduction/basic_usage

This section demonstrates how to create custom benchmarks for Multi-Task (MT) and Meta-reinforcement learning (ML) using a list of environment names.

```APIDOC
## Creating Custom Benchmarks

Support for creating custom benchmarks by combining any number of Meta-World environments is provided. The prefix ‘mt’ returns environments observable for Multi-Task reinforcement learning, while the prefix ‘ml’ returns environments observable for Meta-reinforcement learning. These environments can be run in synchronous or asynchronous mode.

To create a custom benchmark, provide a list of environment names with the suffix ‘-V3’.

### Request Example
```python
import gymnasium as gym
import metaworld

# Example for Multi-Task environments
envs = gym.make_vec('Meta-World/custom-mt-envs', vector_strategy='sync', envs_list=['env_name_1-v3', 'env_name_2-v3', 'env_name_3-v3'], seed=seed)

# Example for Meta-reinforcement learning environments
envs = gym.make_vec('Meta-World/custom-ml-envs', vector_strategy='sync', envs_list=['env_name_1-v3', 'env_name_2-v3', 'env_name_3-v3'], seed=seed)
```
```

--------------------------------

### Create ML1 Training and Testing Environments

Source: https://metaworld.farama.org/introduction/basic_usage

Sets up separate training and testing environments for the ML1 benchmark. The 'reach-V3' environment name should be valid.

```python
import gymnasium as gym
import metaworld

seed = 42

train_envs = gym.make('Meta-World/ML1-train', env_name='reach-V3', seed=seed)
test_envs = gym.make('Meta-World/ML1-test', env_name='reach-V3', seed=seed)

# training procedure use train_envs
# testing procedure use test_envs


```

--------------------------------

### Create Asynchronous MT50 Benchmark

Source: https://metaworld.farama.org/introduction/basic_usage

Instantiates an asynchronous vector environment for the MT50 benchmark, recommended for users with more compute resources.

```python
envs = gym.make_vec('Meta-World/MT50', vector_strategy='async', seed=seed) # this returns an Asynchronous Vector Environment with 50 environments

```

--------------------------------

### Meta-Reinforcement Learning Evaluation Procedure

Source: https://metaworld.farama.org/evaluation/evaluation

This pseudocode outlines the meta-reinforcement learning evaluation process. It includes agent initialization, adaptation steps using collected episodes, and final evaluation metrics. Ensure the agent and environments are properly set up before execution.

```python
def metalearning_eval(agent, eval_envs, adaptation_steps = 1, adaptation_episodes = 10, num_evals = 40, episode_horizon):
   success_rate = 0.0
   initial_obs = eval_envs.reset()

   for goal in range(num_evals):
      eval_envs.iterate_goal_position()
      agent.init()

      for step in range(adaptation_steps):
         for _ in range(adaptation_episodes):
            obs = eval_envs.reset()
            for _ in range(episode_horizon):
               action, misc_outs = agent.adapt_action(obs)
               next_obs, reward, terminated, truncated, info = eval_envs.step(action)
               agent.step(obs, action, reward, terminated, truncated, next_obs, misc_outs)

         agent.adapt()

      success_rate += multi_task_eval(adaptive_agent, eval_envs, num_evaluation_episodes=3)

   success_rate /= num_evals

   return success_rate

```

--------------------------------

### Create Synchronous MT50 Benchmark

Source: https://metaworld.farama.org/introduction/basic_usage

Instantiates a synchronous vector environment for the MT50 benchmark, suitable for users with limited compute resources.

```python
import gymnasium as gym
import metaworld

seed = 42

envs = gym.make_vec('Meta-World/MT50', vector_strategy='sync', seed=seed) # this returns a Synchronous Vector Environment with 50 environments

obs, info = envs.reset() # reset all 50 environments

a = env.action_space.sample() # sample an action for each environment

obs, reward, truncate, terminate, info = envs.step(a) # step all 50 environments

```

--------------------------------

### Render from Specific Camera ID

Source: https://metaworld.farama.org/rendering/rendering

Initializes an environment with a specified render mode and camera ID (from Mujoco) for a particular viewpoint.

```python
camera_id = '' # this is an integer that represents the camera ID from Mujoco

env = gym.make(env_name=env_name, render_mode=render_mode, camera_id=camera_id)


```

--------------------------------

### Create Asynchronous MT10 Benchmark

Source: https://metaworld.farama.org/introduction/basic_usage

Instantiates an asynchronous vector environment for the MT10 benchmark. This version is recommended for users with more compute resources.

```python
envs = gym.make_vec('Meta-World/MT10' vector_strategy='async', seed=seed) # this returns an Asynchronous Vector Environment with 10 environments

```

--------------------------------

### Create Synchronous MT10 Benchmark

Source: https://metaworld.farama.org/introduction/basic_usage

Instantiates a synchronous vector environment for the MT10 benchmark. This version is suitable for users with limited compute resources.

```python
import gymnasium as gym
import metaworld

seed = 42

envs = gym.make('Meta-World/MT10', vector_strategy='sync', seed=seed) # this returns a Synchronous Vector Environment with 10 environments

obs, info = envs.reset() # reset all 10 environments

a = env.action_space.sample() # sample an action for each environment

obs, reward, truncate, terminate, info = envs.step(a) # step all 10 environments

```

--------------------------------

### Create Custom MT and ML Benchmarks

Source: https://metaworld.farama.org/introduction/basic_usage

Use `gym.make_vec` to create custom Multi-Task ('mt') or Meta-reinforcement learning ('ml') benchmarks. Specify the environment strategy, a list of environment names with the '-V3' suffix, and a seed. Environments can be run in 'sync' or 'async' mode.

```python
import gymnasium as gym
import metaworld

envs = gym.make_vec('Meta-World/custom-mt-envs', vector_strategy='sync', envs_list=['env_name_1-v3', 'env_name_2-v3', 'env_name_3-v3'], seed=seed)
```

```python
envs = gym.make_vec('Meta-World/custom-ml-envs', vector_strategy='sync', envs_list=['env_name_1-v3', 'env_name_2-v3', 'env_name_3-v3'], seed=seed)
```

--------------------------------

### Define Agent and MetaLearningAgent Protocols

Source: https://metaworld.farama.org/evaluation/evaluation

Protocols defining the interface for standard and meta-learning agents. The reset method must be present even for stateless agents.

```python
class Agent(Protocol):
    def eval_action(
        self, observations: npt.NDArray[np.float64]
    ) -> npt.NDArray[np.float64]: ...

    def reset(self, env_mask: npt.NDArray[np.bool_]) -> None: ...


class MetaLearningAgent(Agent):
    def init(self) -> None: ...

    def adapt_action(
        self, observations: npt.NDArray[np.float64]
    ) -> tuple[npt.NDArray[np.float64], dict[str, npt.NDArray]]: ...

    def step(self, timestep: Timestep) -> None: ...

    def adapt(self) -> None: ...
```

--------------------------------

### Cite Meta-World+ with BibTeX

Source: https://metaworld.farama.org/citation

Use this BibTeX entry to cite the Meta-World+ benchmark in academic publications.

```bibtex
@inproceedings{
mclean2025metaworld,
title={Meta-World+: An Improved, Standardized, {RL} Benchmark},
author={Reginald McLean and Evangelos Chatzaroulas and Luc McCutcheon and Frank R{"o}der and Tianhe Yu and Zhanpeng He and K.R. Zentner and Ryan Julian and J K Terry and Isaac Woungang and Nariman Farsad and Pablo Samuel Castro},
booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track},
year={2025},
url={https://openreview.net/forum?id=1de3azE606}
}
```

--------------------------------

### Evaluate Multi-Task Reinforcement Learning Agents

Source: https://metaworld.farama.org/evaluation/evaluation

Pseudocode demonstrating the calculation of success rates across multiple tasks and goal positions. Success is determined if the success flag is 1 at any point during the episode.

```python
def multi_task_eval(agent, envs, num_evaluation_episodes = 50, episode_horizon = 500):
   success_rate = 0.0

   for episode in range(num_evaluation_episodes):
      envs.iterate_goal_position()
      obs = envs.reset()
      for env in envs:
         obs = env.reset()
         for step in range(episode_horizon):
            action, _ = agent.eval_action(obs)
            next_obs, _, _, _, info = env.step(action)
            obs = next_obs

            if info["success"] == 1:
               success_rate += 1
               break

   success_rate /= (num_evaluation_episodes * envs.num_envs)

   return success_rate
```

--------------------------------

### MetaLearningAgent Protocol

Source: https://metaworld.farama.org/evaluation/evaluation

Extends the Agent protocol with methods specific to meta-reinforcement learning, including adaptation steps.

```APIDOC
## MetaLearningAgent Protocol

### Description
This protocol extends the `Agent` protocol and is designed for meta-reinforcement learning evaluations. It includes additional methods for agent initialization, adaptation actions, processing timesteps, and performing adaptation steps.

### Methods
- `init() -> None`
  - Re-initializes the agent for adaptation, resetting it to its pre-adaptation state.
- `adapt_action(observations: npt.NDArray[np.float64]) -> tuple[npt.NDArray[np.float64], dict[str, npt.NDArray]]`
  - Takes observations and returns a tuple of actions and auxiliary policy outputs needed during adaptation.
- `step(timestep: Timestep) -> None`
  - Processes the `Timestep` data for adaptation, such as storing experience.
- `adapt() -> None`
  - A method called at the end of each adaptation step, allowing for experience processing like gradient updates.

### Request Example
```python
class MyMetaLearningAgent(Agent):
    def init(self) -> None:
        # Implementation for agent initialization
        pass

    def adapt_action(self, observations: npt.NDArray[np.float64]) -> tuple[npt.NDArray[np.float64], dict[str, npt.NDArray]]:
        # Implementation for adaptation action
        pass

    def step(self, timestep: Timestep) -> None:
        # Implementation for processing timestep
        pass

    def adapt(self) -> None:
        # Implementation for adaptation step
        pass
```

### Response Example
(No direct response for protocol definition, but methods return specified types.)
```

--------------------------------

### Utility Outputs

Source: https://metaworld.farama.org/evaluation/evaluation

Details the various outputs provided by the evaluation utilities.

```APIDOC
## Utility Outputs

### Description
The evaluation utilities return a tuple containing several metrics that provide a comprehensive view of the agent's performance.

### Outputs
1.  **`mean_success_rate`** (float)
    - The average success rate across all evaluated tasks and goal positions.
2.  **`mean_returns`** (float)
    - The average return achieved during evaluation across all tasks and goal positions.
3.  **`success_rate_per_task`** (dict[str, float])
    - A dictionary where keys are task names (string) and values are the success rates (float) for each specific task, averaged across goal positions.
4.  **`returns_per_task`** (dict[str, float])
    - A dictionary where keys are task names (string) and values are the returns (float) for each specific task, averaged across goal positions.

### Example Output Structure
```json
{
  "mean_success_rate": 0.85,
  "mean_returns": 150.75,
  "success_rate_per_task": {
    "task1": 0.9,
    "task2": 0.8
  },
  "returns_per_task": {
    "task1": 160.5,
    "task2": 141.0
  }
}
```
```

--------------------------------

### Agent Protocol

Source: https://metaworld.farama.org/evaluation/evaluation

Defines the basic interface for an agent used in evaluations, including action selection and environment reset.

```APIDOC
## Agent Protocol

### Description
This protocol defines the essential methods an agent must implement for evaluation utilities. It includes a method for evaluating an action based on observations and a method for resetting the agent's state.

### Methods
- `eval_action(observations: npt.NDArray[np.float64]) -> npt.NDArray[np.float64]`
  - Takes observations and returns the agent's action.
- `reset(env_mask: npt.NDArray[np.bool_]) -> None`
  - Resets the agent's state for specified environments.

### Request Example
```python
class MyAgent:
    def eval_action(self, observations: npt.NDArray[np.float64]) -> npt.NDArray[np.float64]:
        # Implementation for action evaluation
        pass

    def reset(self, env_mask: npt.NDArray[np.bool_]) -> None:
        # Implementation for resetting agent state
        pass
```

### Response Example
(No direct response for protocol definition, but methods return specified types.)
```

--------------------------------

### State Space Vector Structure

Source: https://metaworld.farama.org/benchmark/state_space

Details the composition of the 9D state vector used for observations in Meta-World environments.

```APIDOC
## State Space Vector Structure

### Description
The state space is designed to maintain a consistent structure across tasks, resulting in a 9D state vector. It includes end-effector position, gripper state, and object-specific spatial data.

### State Vector Indices
- **0:2** (float) - The XYZ coordinates of the end-effector
- **3** (float) - Scalar value representing gripper open/closed state
- **4:6** (float) - The XYZ coordinates of the first object
- **7:10** (float) - Quaternion describing spatial orientation/rotation of object #1
- **11:13** (float) - The XYZ coordinates of the second object
- **14:17** (float) - Quaternion describing spatial orientation/rotation of object #2
```

--------------------------------

### Sawyer Robot Action Space

Source: https://metaworld.farama.org/benchmark/action_space

Details the action space for the Sawyer robot, which is a Box(-1.0, 1.0, (4,), float32). Actions control end-effector displacement (dx, dy, dz) and gripper adjustment.

```APIDOC
## Sawyer Robot Action Space

### Description
The action space for the Sawyer robot in the Meta-World benchmark is defined as a `Box` with bounds from -1.0 to 1.0 and a shape of (4,). This space accommodates control over the end-effector's Cartesian displacement and the gripper.

### Action Details

| Num | Action                                       | Control Min | Control Max | Name (in XML file) | Joint        | Unit              |
|-----|----------------------------------------------|-------------|-------------|--------------------|--------------|-------------------|
| 0   | Displacement of the end-effector in x direction (dx) | -1          | 1           | mocap              | N/A          | position (m)      |
| 1   | Displacement of the end-effector in y direction (dy) | -1          | 1           | mocap              | N/A          | position (m)      |
| 2   | Displacement of the end-effector in z direction (dz) | -1          | 1           | mocap              | N/A          | position (m)      |
| 3   | Gripper adjustment (closing/opening)         | -1          | 1           | rightclaw, leftclaw| r_close, l_close | position (normalized)|

### Notes
For tasks that do not require gripper interaction, actions corresponding to gripper control can be masked or ignored by setting them to a constant value that permanently closes the fingers.
```

--------------------------------

### Timestep Named Tuple

Source: https://metaworld.farama.org/evaluation/evaluation

Defines the structure of the Timestep object passed to the agent during adaptation.

```APIDOC
## Timestep Named Tuple

### Description
The `Timestep` named tuple encapsulates the data from a single transition in the environment, provided to the agent during the adaptation phase.

### Fields
- `observation` (npt.NDArray)
  - The observation from the environment.
- `action` (npt.NDArray)
  - The action taken by the agent.
- `reward` (npt.NDArray)
  - The reward received from the environment.
- `terminated` (npt.NDArray)
  - Indicates if the environment terminated.
- `truncated` (npt.NDArray)
  - Indicates if the environment was truncated.
- `aux_policy_outputs` (dict[str, npt.NDArray])
  - Miscellaneous policy outputs.

### Example
```python
from typing import NamedTuple
import numpy as np

class Timestep(NamedTuple):
    observation: np.ndarray
    action: np.ndarray
    reward: np.ndarray
    terminated: np.ndarray
    truncated: np.ndarray
    aux_policy_outputs: dict[str, np.ndarray]

# Example usage:
timestep_data = Timestep(
    observation=np.array([0.1, 0.2]),
    action=np.array([1.0]),
    reward=np.array([0.5]),
    terminated=np.array([False]),
    truncated=np.array([False]),
    aux_policy_outputs={'log_prob': np.array([-0.1])}
)
```
```

--------------------------------

### Define Timestep Named Tuple

Source: https://metaworld.farama.org/evaluation/evaluation

Structure of the Timestep object provided to the agent during adaptation steps.

```python
class Timestep(NamedTuple):
    observation: npt.NDArray
    action: npt.NDArray
    reward: npt.NDArray
    terminated: npt.NDArray
    truncated: npt.NDArray
    aux_policy_outputs: dict[str, npt.NDArray]
```
Tokens:
Raw