### Install uv and MyoSuite

Source: https://github.com/myohub/myosuite/blob/main/README.md

Installs uv if not already present, creates and activates a virtual environment, and then installs MyoSuite using uv for faster package management.

```bash
# Install uv (if not already installed)
curl -LsSf https://astral.sh/uv/install.sh | sh

# Create and activate a virtual environment
uv venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install MyoSuite
uv pip install -U myosuite
```

--------------------------------

### Install Required Libraries

Source: https://github.com/myohub/myosuite/blob/main/tutorials/SAR/SAR_tutorial.ipynb

Installs necessary libraries for the SAR tutorial. Uncomment lines to install missing packages.

```python
# !pip install stable-baselines3==1.7.0
# !pip install joblib
# !pip install scikit-learn
# !pip install tqdm
# !pip install matplotlib
# !pip install gymnasium
```

--------------------------------

### Install Tutorial Extras

Source: https://github.com/myohub/myosuite/blob/main/tutorials/README.md

Install additional packages needed for specific tutorials using uv sync or pip.

```bash
uv sync --extra tutorials
```

```bash
pip install -e .[tutorials]
```

--------------------------------

### Install MyoSuite using uv

Source: https://github.com/myohub/myosuite/blob/main/docs/source/install.md

Installs MyoSuite using the uv package installer. This method is recommended for faster installation times.

```bash
uv venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate
uv pip install -U myosuite
```

--------------------------------

### Install TorchRL and Dependencies

Source: https://github.com/myohub/myosuite/blob/main/myosuite/agents/README.md

Install TorchRL, Gymnasium, Hydra, and submitit for agent training. Optional packages for logging are also included.

```bash
pip install torchrl
pip install gymnasium
pip install hydra-core==1.1.0 hydra-submitit-launcher submitit
#optional 
pip install tensorboard wandb
```

--------------------------------

### Install Stable-Baselines3 and Dependencies

Source: https://github.com/myohub/myosuite/blob/main/myosuite/agents/README.md

Install Stable-Baselines3, Gym, Hydra, and submitit for agent training. Optional packages for logging are also included.

```bash
pip install stable-baselines3
pip install gym==0.13
pip install hydra-core==1.1.0 hydra-submitit-launcher submitit
#optional 
pip install tensorboard wandb
```

--------------------------------

### Install Dependencies and Build Docs with Make

Source: https://github.com/myohub/myosuite/blob/main/docs/Readme.md

Installs necessary documentation dependencies and builds the HTML documentation using the 'make' command. Ensure you are in the 'docs' directory before running.

```bash
pip install -e ".[docs]"
cd docs
make html
```

--------------------------------

### Install MJRL and Dependencies

Source: https://github.com/myohub/myosuite/blob/main/docs/source/baselines.md

Install the MJRL library and other necessary Python packages for training baselines. Ensure PyTorch is installed.

```bash
pip install tabulate matplotlib torch git+https://github.com/aravindr93/mjrl.git
pip install hydra-core --upgrade
pip install hydra-submitit-launcher --upgrade
pip install submitit
```

--------------------------------

### Install uv Package Installer

Source: https://github.com/myohub/myosuite/blob/main/docs/source/install.md

Installs the uv package installer, a fast alternative to pip, using a curl script.

```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
```

--------------------------------

### Install Jupyter and Kernel

Source: https://github.com/myohub/myosuite/blob/main/tutorials/README.md

Install Jupyter and ipykernel, then set up the kernel for your environment. This is useful if the environment kernel is not recognized.

```bash
pip install jupyter ipykernel
python -m ipykernel install --user --name= < name of the environment >
```

--------------------------------

### Install MyoSuite from Source using uv

Source: https://github.com/myohub/myosuite/blob/main/docs/source/install.md

Installs MyoSuite from its source repository using uv. This method also requires cloning the repository with its submodules.

```bash
git clone --recursive https://github.com/facebookresearch/myosuite.git
cd myosuite
uv pip install -e .
```

--------------------------------

### Test MyoSuite Installation

Source: https://github.com/myohub/myosuite/blob/main/docs/source/install.md

Runs a test suite to verify the correct installation of MyoSuite.

```bash
python -m myosuite.tests.test_myo
```

--------------------------------

### Install Jupyter Notebook

Source: https://github.com/myohub/myosuite/blob/main/tutorials/README.md

Install Jupyter Notebook using pip. This is a prerequisite for running most tutorials.

```bash
pip install jupyter
```

--------------------------------

### Simulation and Video Rendering Setup

Source: https://github.com/myohub/myosuite/blob/main/tutorials/9_Computed_muscle_control.ipynb

Initializes models, data, cameras, and render options for both MyoSuite and OpenSim simulations. This setup is crucial for generating comparable simulation outputs and videos.

```python
# ---- initializations
model_ref = loader.get_sim(None, 'elbow/myoelbow_2dof6muscles.xml')
data_ref = mujoco.MjData(model_ref) # data for reference trajectory
model_test = loader.get_sim(None, 'elbow/myoelbow_2dof6muscles.xml')
data_test = mujoco.MjData(model_test) # test data for achieved trajectory

camera = mujoco.MjvCamera()
camera.azimuth = 0
camera.distance =  1.1070990185160428
camera.elevation = -10.232281643227267
camera.lookat = np.array([-0.1130067696435806, 0.0815790401272094, 1.0655519045043413])

options_ref = mujoco.MjvOption()
options_ref.flags[:] = 0
options_ref.flags[[1, 22]] = 1
options_ref.geomgroup[2:] = 0

options_test = mujoco.MjvOption()
options_test.flags[:] = 0
options_test.flags[[1, 4, 22]] = 1
options_test.geomgroup[:] = 1

renderer_ref = mujoco.Renderer(model_ref, width=480, height=480)
renderer_ref.scene.flags[:] = 0

renderer_test = mujoco.Renderer(model_test, width=480, height=480)
renderer_test.scene.flags[:] = 0
```

--------------------------------

### Install DEP-RL Baseline

Source: https://github.com/myohub/myosuite/blob/main/docs/source/baselines.md

Install the deprl package using pip after installing myosuite. This command is used for setting up the baseline.

```bash
python -m pip install deprl
```

--------------------------------

### Verify MyoSuite Installation

Source: https://github.com/myohub/myosuite/blob/main/myosuite/envs/myo/mjx/README.md

Verifies the installation of MuJoCo and JAX, checking their versions and available devices.

```bash
# remove uv run if you installed with pypi

uv run python -c "import mujoco; print(f'MuJoCo version: {mujoco.__version__}')"
uv run python -c "import jax; print(f'JAX devices: {jax.devices()}')"
```

--------------------------------

### Install MyoSuite from Source using Pip

Source: https://github.com/myohub/myosuite/blob/main/docs/source/install.md

Installs MyoSuite from its source repository using pip. This requires cloning the repository with its submodules.

```bash
git clone --recursive https://github.com/facebookresearch/myosuite.git
cd myosuite
pip install -e .
```

--------------------------------

### Install MyoSuite Dependencies

Source: https://github.com/myohub/myosuite/blob/main/myosuite/agents/README.md

Installs necessary libraries for MyoSuite, including MJRL, PyTorch, Hydra, and submitit. Ensure you have the correct versions as specified.

```bash
pip install tabulate matplotlib torch git+https://github.com/aravindr93/mjrl.git hydra-core==1.1.0 hydra-submitit-launcher submitit
```

--------------------------------

### Test MyoSuite installation

Source: https://github.com/myohub/myosuite/blob/main/README.md

Verifies the MyoSuite installation by running its test suite. This command works with both uv and standard pip/conda setups.

```bash
# With uv:
uv run python -m myosuite.tests.test_myo

# With pip/conda:
python -m myosuite.tests.test_myo
```

--------------------------------

### Install DEPRL

Source: https://github.com/myohub/myosuite/blob/main/tutorials/README.md

Install the DEPRL package, required for the deprl tutorial. Requires Python 3.10.

```bash
pip install deprl
```

--------------------------------

### Install MyoSuite with MJX Support

Source: https://github.com/myohub/myosuite/blob/main/myosuite/envs/myo/mjx/README.md

Installs MyoSuite with MJX support using uv or pip. Replace 'mjx' with 'mjx-cuda' for CUDA support.

```bash
# With uv:
uv sync --extra mjx -p 3.10 # replace "mjx" with "mjx-cuda" for jax with cuda support

# With pip:
pip install -e ".[mjx]" # replace "mjx" with "mjx-cuda" for jax with cuda support
```

--------------------------------

### Initialize and Interact with myosuite Environment

Source: https://github.com/myohub/myosuite/blob/main/docs/source/challenge-doc2025.md

This snippet demonstrates how to initialize a myosuite environment (Table Tennis P1 in this case), reset it, render it, modify its visual properties, get observations, and take random actions within a loop. It also shows how to reset the environment when an episode terminates.

```python
from myosuite.utils import gym
# Include the table tennis track environment, uncomment to select the soccer track challenge
# env = gym.make('myoChallengeSoccerP1-v0')
env = gym.make('myoChallengeTableTennisP1-v0')


env.reset()

# Repeat 1000 time steps
for _ in range(1000):

    # Activate mujoco rendering window
    env.mj_render()

    # Select skin group
    geom_1_indices = np.where(env.mj_model.geom_group == 1)
    # Change the alpha value to make it transparent
    env.mj_model.geom_rgba[geom_1_indices, 3] = 0


    # Get observation from the environment, details are described in the above docs
    obs = env.get_obs()
    current_time = obs['time']
    #print(current_time)


    # Take random actions
    action = env.action_space.sample()


    # Environment provides feedback on action
    next_obs, reward, terminated, truncated, info = env.step(action)


    # Reset training if env is terminated
    if terminated:
        next_obs, info = env.reset()
```

--------------------------------

### Install MyoSuite using Pip

Source: https://github.com/myohub/myosuite/blob/main/docs/source/install.md

Installs MyoSuite using pip after setting up a conda environment. Ensure you have Python 3.9 or higher.

```bash
conda create --name MyoSuite python=3.9
conda activate MyoSuite
pip install -U myosuite
```

--------------------------------

### Run DEP-RL Visualization Tool

Source: https://github.com/myohub/myosuite/blob/main/myosuite/agents/README.md

Run the DEP-RL visualization tool to play through a trained policy for several episodes and get scores. Ensure the specified path points to a folder containing 'checkpoints' and 'config.yaml' files.

```bash
python -m deprl.play --path folder/
```

--------------------------------

### Import MyoSuite Gym and DEP-RL Utilities

Source: https://github.com/myohub/myosuite/blob/main/tutorials/4a_deprl.ipynb

Imports necessary components from MyoSuite's utility module and DEP-RL when the library is available. This setup is required before creating or interacting with environments.

```python
if DEPRL_AVAILABLE:
    from myosuite.utils import gym
    import deprl
    from deprl import env_wrappers
```

--------------------------------

### Initialize MyoSuite environment and control hand muscles

Source: https://github.com/myohub/myosuite/blob/main/tutorials/5_Move_Hand_Fingers.ipynb

Sets up the MyoSuite environment for hand pose simulation. It initializes the environment, accesses the base environment for direct manipulation, and identifies specific muscle IDs for control. This snippet is essential for starting any hand manipulation simulation.

```python
import mujoco
env = gym.make('myoHandPoseRandom-v0', normalize_act = False)

# Gymnasium wraps envs (OrderEnforcing/PassiveEnvChecker); use base env.
base_env = env.unwrapped
base_env.init_qpos[:] = np.zeros(len(base_env.init_qpos),)
mjcModel = base_env.mj_model


# print("Muscles:")
# for i in range(mjcModel.na):
#     print([i,mjcModel.actuator(i).name])

# print("\nJoints:")
# for i in range(mjcModel.njnt):
#     print([i,mjcModel.joint(i).name])


musc_fe = [mjcModel.actuator('FDP2').id,mjcModel.actuator('EDC2').id]
L_range = round(1/mjcModel.opt.timestep)
skip_frame = 50
env.reset()

frames_sim = []
for iter_n in range(3):
    print("iteration: "+str(iter_n))
    res_sim = []
    for rp in range(2): #alternate between flexor and extensor
        for s in range(L_range):
            if not(s%skip_frame):
                frame = env.unwrapped.mj_renderer.render_offscreen(
                                width=400,
                                height=400,
                                camera_id=3)
                frames_sim.append(frame)

            ctrl = np.zeros(mjcModel.na,)

            act_val = 1 # maximum muscle activation
            if rp==0:
                ctrl[musc_fe[0]] = act_val
                ctrl[musc_fe[1]] = 0
            else:
                ctrl[musc_fe[1]] = act_val
                ctrl[musc_fe[0]] = 0
            env.step(ctrl)
```

--------------------------------

### Create and activate conda environment for MyoSuite

Source: https://github.com/myohub/myosuite/blob/main/README.md

Recommended for setting up a dedicated environment for MyoSuite using Miniconda. This involves creating a new environment with a specific Python version and then installing MyoSuite.

```bash
conda create --name myosuite python=3.10
conda activate myosuite
pip install -U myosuite
```

--------------------------------

### Initialize myoSuite Environment

Source: https://github.com/myohub/myosuite/blob/main/tutorials/1_Get_Started.ipynb

Create a myoSuite environment using gym.make and print available camera names. This sets up the simulation environment for interaction.

```python
from myosuite.utils import gym
import os

env = gym.make('myoElbowPose1D6MRandom-v0')
print('List of cameras available', [env.unwrapped.mj_model.camera(i).name for i in range(env.unwrapped.mj_model.ncam)])
env.reset()
```

--------------------------------

### Initialize and Run MyoSuite Environment

Source: https://github.com/myohub/myosuite/blob/main/docs/source/challenge-doc.md

This snippet demonstrates how to initialize a MyoSuite environment, step through it with random actions, and reset it if terminated. It also shows how to render the environment and modify object transparency.

```python
from myosuite.utils import gym
# Include the locomotion track environment, uncomment to select the manipulation challenge
# env = gym.make('myoChallengeOslRunRandom-v0')
env = gym.make('myoChallengeBimanual-v0')


env.reset()

# Repeat 1000 time steps
for _ in range(1000):

    # Activate mujoco rendering window
    env.mj_render()

    # Select skin group
    geom_1_indices = np.where(env.mj_model.geom_group == 1)
    # Change the alpha value to make it transparent
    env.mj_model.geom_rgba[geom_1_indices, 3] = 0


    # Get observation from the envrionment, details are described in the above docs
    obs = env.get_obs()
    current_time = obs['time']
    #print(current_time)


    # Take random actions
    action = env.action_space.sample()


    # Environment provides feedback on action
    next_obs, reward, terminated, truncated, info = env.step(action)


    # Reset training if env is terminated
    if terminated:
        next_obs, info = env.reset()
```

--------------------------------

### Train JAX PPO with MyoSuite

Source: https://github.com/myohub/myosuite/blob/main/myosuite/envs/myo/mjx/README.md

Launches JAX PPO training for a specified MyoSuite environment using the 'warp' implementation.

```bash
uv run train_jax_ppo.py --env_name=MjxElbowPoseRandom-v0 --impl=warp
```

--------------------------------

### Load PPO Policy and Initialize Environment

Source: https://github.com/myohub/myosuite/blob/main/tutorials/3_Analyse_movements.ipynb

Loads a pre-trained PPO policy and initializes the myoElbowPose1D6MRandom-v0 environment. This snippet shows how to set up the environment and policy for data collection.

```python
# from stable_baselines3 import PPO
# policy = "ElbowPose_policy.zip"

# pi = PPO.load(policy)

from stable_baselines3 import PPO
env = gym.make('myoElbowPose1D6MRandom-v0')

env.reset()
pi = PPO("MlpPolicy", env, verbose=0)


pi.learn(total_timesteps=1000)
```

--------------------------------

### Import necessary libraries

Source: https://github.com/myohub/myosuite/blob/main/tutorials/1_Get_Started.ipynb

Import the imageio library for video creation and IPython.display for video playback. These are essential for the visualization steps.

```python
import imageio
from IPython.display import Video, display
```

--------------------------------

### Use MyoSuite Environment with OpenAI Gym

Source: https://github.com/myohub/myosuite/blob/main/docs/source/install.md

Demonstrates how to create, reset, render, and step through a MyoSuite environment using the OpenAI Gym interface.

```python
from myosuite.utils import gym
env = gym.make('myoElbowPose1D6MRandom-v0')
env.reset()
for _ in range(1000):
  env.mj_render()
  env.step(env.action_space.sample()) # take a random action
env.close()
```

--------------------------------

### Initialize MyoSuite Environment with DEP-RL Wrapper

Source: https://github.com/myohub/myosuite/blob/main/tutorials/4a_deprl.ipynb

Creates a MyoSuite environment and wraps it using DEP-RL's GymWrapper. This prepares the environment for use with DEP-RL policies and requires the DEPRL_AVAILABLE flag to be true.

```python
if DEPRL_AVAILABLE:
    T = 1000 # length of episode
    env = gym.make('myoLegWalk-v0')
    env = env_wrappers.GymWrapper(env)
    obs = env.reset()
```

--------------------------------

### Initialize MyoSuite environment

Source: https://github.com/myohub/myosuite/blob/main/tutorials/4c_Train_SB_policy.ipynb

Creates and resets the 'myoElbowPose1D6MRandom-v0' environment.

```python
env = gym.make('myoElbowPose1D6MRandom-v0')

env.reset()

```

--------------------------------

### State Management

Source: https://github.com/myohub/myosuite/blob/main/docs/source/api/env_base.md

Methods for getting and setting the full state of the environment.

```APIDOC
## `get_env_state()`

### Description
Get full state of the environment.
Default implementation provided. Override if env has custom state.

## `set_env_state(state_dict)`

### Description
Set full state of the environment.
Default implementation provided. Override if env has custom state.
```

--------------------------------

### Visualize MyoSuite environments

Source: https://github.com/myohub/myosuite/blob/main/README.md

Launches a visualization of a MyoSuite environment with random controls. This command is available for both uv and pip/conda installations.

```bash
# With uv:
uv run python -m myosuite.utils.examine_env --env_name myoElbowPose1D6MRandom-v0

# With pip/conda:
python -m myosuite.utils.examine_env --env_name myoElbowPose1D6MRandom-v0
```

--------------------------------

### Import necessary libraries

Source: https://github.com/myohub/myosuite/blob/main/tutorials/4b_reflex/MyoSuite_MyoReflex_Walk.ipynb

Imports required modules for MyoSuite, numerical operations, image handling, and video display. Ensure these libraries are installed.

```python
import ReflexCtrInterface
import numpy as np
import imageio
from IPython.display import Video, display

from myosuite.utils.quat_math import quat2euler
```

--------------------------------

### Run MyoReflex simulation and save video

Source: https://github.com/myohub/myosuite/blob/main/tutorials/4b_reflex/MyoSuite_MyoReflex_Walk.ipynb

Initializes the MyoReflex environment, loads parameters, runs the simulation for a specified duration, captures frames, and saves them as a video. The simulation stops if the agent is done.

```python
sim_time = 5 # in seconds
dt = 0.01
steps = int(sim_time/dt)
frames = []

params = np.loadtxt('baseline_params.txt')

Myo_env = ReflexCtrInterface.MyoLegReflex()
Myo_env.reset()

Myo_env.set_control_params(params)

for timestep in range(steps):
    frame = Myo_env.env.unwrapped.mj_renderer.render_offscreen(camera_id=1)
    frames.append(frame)
    _, isDone, _, _ = Myo_env.run_reflex_step()

    if isDone:
        print(f"Stopped at - {timestep}")
        break
    
imageio.mimwrite("MyoReflex_output.mp4", frames, fps=int(1.0/dt))
# show in the notebook
display(Video("MyoReflex_output.mp4", embed=True))
```

--------------------------------

### demo_robot()

Source: https://github.com/myohub/myosuite/blob/main/docs/source/api/index.md

Demonstrates the usage of the Robot class.

```APIDOC
## demo_robot()

### Description
Demonstrates the usage of the Robot class.

### Function Signature
`demo_robot()`
```

--------------------------------

### Display a Single Frame using Matplotlib

Source: https://github.com/myohub/myosuite/blob/main/tutorials/1_Get_Started.ipynb

Install matplotlib if not already present, then display a single captured frame using plt.imshow. This allows for quick inspection of individual frames.

```python
!pip install matplotlib
import matplotlib.pyplot as plt
plt.imshow(frame)
```

--------------------------------

### Launch Stable-Baselines3 Training

Source: https://github.com/myohub/myosuite/blob/main/myosuite/agents/README.md

Commands to launch training jobs using Stable-Baselines3 with either a local or SLURM cluster launcher.

```bash
% sh train_myosuite.sh myo local sb3   # use stable-baselines3 with local launcher
% sh train_myosuite.sh myo slurm sb3   # use stable-baselines3 with slurm launcher
```

--------------------------------

### Load and Execute DEP-RL Baseline Policy

Source: https://github.com/myohub/myosuite/blob/main/docs/source/tutorials.md

This Python script loads a pre-trained DEP-RL baseline policy for a Myosuite environment and visualizes its execution. Requires Python 3.9+ and the deprl package installed.

```python
import mujoco
from myosuite.utils import gym
import deprl
from deprl import env_wrappers
import time

# we can pass arguments to the environments here
env = gym.make('myoLegWalk-v0', reset_type='random')
env = env_wrappers.GymWrapper(env)
policy = deprl.load_baseline(env)
obs = env.reset()

m = env.unwrapped.mj_model
d = env.unwrapped.mj_data

with mujoco.viewer.launch_passive(m, d) as viewer:
  start = time.time()
  while viewer.is_running() and time.time() - start < 30:
    step_start = time.time()

    action = policy(obs)

    obs, *_ = env.step(action)
    viewer.sync()

    time_until_next_step = m.opt.timestep - (time.time() - step_start)
    time.sleep(time_until_next_step)
```

--------------------------------

### Test Trained Policy for Elbow Flexion

Source: https://github.com/myohub/myosuite/blob/main/docs/source/tutorials.md

Load and test a trained policy for elbow flexion. This example demonstrates how to load a pickled policy and use it within a MyoSuite environment, including rendering and stepping.

```python
from myosuite.utils import gym

policy = "iterations/best_policy.pickle"

import pickle
pi = pickle.load(open(policy, 'rb'))

env = gym.make('myoElbowPose1D6MRandom-v0')
env.reset()
for _ in range(1000):
    env.mj_render()
    env.step(env.action_space.sample()) # take a random action
```

--------------------------------

### Initialize and Simulate Trajectories

Source: https://github.com/myohub/myosuite/blob/main/tutorials/6_Inverse_Dynamics.ipynb

Initializes reference and test models, sets up camera and rendering options, and then iterates through a trajectory to simulate both reference and achieved movements. It generates merged frames for visualization.

```python
# ---- initializations
model_ref = loader.get_sim(None, 'hand/myohand.xml')
model_ref.actuator_dynprm[:,2] = tausmooth
data_ref = mujoco.MjData(model_ref) # data for reference trajectory
model_test = loader.get_sim(None, 'hand/myohand.xml')
model_test.actuator_dynprm[:,2] = tausmooth
data_test = mujoco.MjData(model_test) # test data for achieved trajectory
# ---- camera settings
camera = mujoco.MjvCamera()
camera.azimuth = 166.553
camera.distance = 1.178
camera.elevation = -36.793
camera.lookat = np.array([-0.93762553, -0.34088276, 0.85067529])
options_ref = mujoco.MjvOption()
options_ref.flags[:] = 0
options_ref.geomgroup[1:] = 0
options_test = mujoco.MjvOption()
options_test.flags[:] = 0
options_test.flags[4] = 1 # actuator ON
options_test.geomgroup[1:] = 0
renderer_ref = mujoco.Renderer(model_ref)
renderer_ref.scene.flags[:] = 0
renderer_test = mujoco.Renderer(model_test)
renderer_test.scene.flags[:] = 0
# ---- generation loop
frames = []
for idx in tqdm(range(traj.shape[0])):
    # -- reference trajectory
    data_ref.qpos = traj[idx, 1:]
    mujoco.mj_step1(model_ref, data_ref)
    # -- achieved trajectory
    data_test.ctrl = all_ctrl[idx, 1:]
    mujoco.mj_step(model_test, data_test)
    # -- frames generation
    if not idx % round(0.3/(model_test.opt.timestep*25)):
        renderer_ref.update_scene(data_ref, camera=camera, scene_option=options_ref)
        frame_ref = renderer_ref.render()
        renderer_test.update_scene(data_test, camera=camera, scene_option=options_test)
        frame_test = renderer_test.render()
        frame_merged = np.append(frame_ref, frame_test, axis=1)
        frames.append(frame_merged)
# -- frames writing
os.makedirs('videos', exist_ok = True)
output_name = 'videos/myohand_freemovement.mp4'
imageio.mimwrite(output_name, frames, fps=1.0 / model_test.opt.timestep)
```

--------------------------------

### Rendering and Visualization

Source: https://github.com/myohub/myosuite/blob/main/docs/source/api/env_base.md

Functions for rendering the environment and setting up the viewer camera.

```APIDOC
## `mj_render()`

### Description
Render the default camera.

## `viewer_setup(distance=2.5, azimuth=90, elevation=-30, lookat=None, render_actuator=None, render_tendon=None)`

### Description
Setup the default camera.
```

--------------------------------

### Run DEP-RL Visualization Script

Source: https://github.com/myohub/myosuite/blob/main/tutorials/4a_deprl.ipynb

Provides a command-line instruction to set up a visualization script for DEP-RL policies using the `deprl.play` module. This is useful for interactive debugging and observation.

```bash
python -m deprl.play --path /folder/
```

--------------------------------

### Simulate and Visualize Muscle Fatigue

Source: https://github.com/myohub/myosuite/blob/main/tutorials/7_Fatigue_Modeling.ipynb

This code simulates muscle fatigue over several batches and episodes, collecting data for both a normal and a fatigued model. It then visualizes the differences in muscle activations, joint angles, muscle lengths, and fatigued motor units over time. Ensure MyoSuite and Matplotlib are installed.

```python
env.reset()
envFatigue.reset()
data_store = []
data_store_f = []
for i in range(7*3): # 7 batches of 3 episodes, with 2 episodes of maximum muscle controls for some muscles followed by a resting episode (i.e., zero muscle controls) in each batch
    a = np.zeros(env.unwrapped.mj_model.nu,)
    if i%3!=2:
        a[3:]=1
    else:
        a[:]=0

    for _ in range(500): # 500 samples (=10s) for each episode
        next_o, r, done, *_, ifo = env.step(a) # take an action
        next_f_o, r_f, done_F, *_, ifo_f = envFatigue.step(a) # take an action

        data_store.append({"action":a.copy(),
                            "jpos":env.unwrapped.mj_data.qpos.copy(),
                            "mlen":env.unwrapped.mj_data.actuator_length.copy(),
                            "act":env.unwrapped.mj_data.act.copy()})
        data_store_f.append({"action":a.copy(),
                            "jpos":envFatigue.unwrapped.mj_data.qpos.copy(),
                            "mlen":envFatigue.unwrapped.mj_data.actuator_length.copy(),
                            "MF":envFatigue.unwrapped.muscle_fatigue.MF.copy(),
                            "MR":envFatigue.unwrapped.muscle_fatigue.MR.copy(),
                            "MA":envFatigue.unwrapped.muscle_fatigue.MA.copy(),
                            "act":envFatigue.unwrapped.mj_data.act.copy()})

env.close()
envFatigue.close()

mdl = env.unwrapped.mj_model
muscle_names = [mdl.actuator(i).name for i in range(mdl.nu) if mdl.actuator_dyntype[i] == mujoco.mjtDyn.mjDYN_MUSCLE]
muscle_id = -1

plt.figure(figsize=(12, 6))
plt.subplot(221)
plt.plot(env.unwrapped.dt*np.arange(len(data_store)), np.array([d['act'][muscle_id] for d in data_store]), label="Normal model/Desired activations")
plt.plot(env.unwrapped.dt*np.arange(len(data_store)), np.array([d['act'][muscle_id] for d in data_store_f]), label='Fatigued model')
plt.legend()
plt.title(f'Muscle activations over time ({muscle_names[muscle_id]})')
plt.xlabel('time (s)'),plt.ylabel('act')

plt.subplot(222)
plt.plot(env.unwrapped.dt*np.arange(len(data_store)), np.array([d['jpos'] for d in data_store]), label="Normal model")
plt.plot(env.unwrapped.dt*np.arange(len(data_store)), np.array([d['jpos'] for d in data_store_f]), label="Fatigued model")
plt.legend()
plt.title('Joint angle over time')
plt.xlabel('time (s)'),plt.ylabel('angle')

plt.subplot(223)
plt.plot(env.unwrapped.dt*np.arange(len(data_store)), np.array([d['mlen'][muscle_id] for d in data_store]), label="Normal model")
plt.plot(env.unwrapped.dt*np.arange(len(data_store)), np.array([d['mlen'][muscle_id] for d in data_store_f]), label="Fatigued model")
plt.legend()
plt.title(f'Muscle lengths over time ({muscle_names[muscle_id]})')
plt.xlabel('time (s)'),plt.ylabel('muscle length')

plt.subplot(224)
plt.plot(env.unwrapped.dt*np.arange(len(data_store)), np.array([d['MF'][muscle_id] for d in data_store_f]), color="tab:orange")
plt.title(f'Fatigued motor units over time ({muscle_names[muscle_id]})')
plt.xlabel('time (s)'),plt.ylabel('%MVC')

plt.tight_layout()
plt.show()
```

--------------------------------

### Launch MJRL Baseline Training

Source: https://github.com/myohub/myosuite/blob/main/docs/source/baselines.md

Execute training scripts for MJRL baselines using different launchers. Choose 'native', 'local', or 'slurm' based on your environment.

```bash
sh train_myosuite.sh myo         # runs natively
sh train_myosuite.sh myo local   # use local launcher
sh train_myosuite.sh myo slurm   # use slurm launcher
```

--------------------------------

### Initialize and reset the MyoSuite environment

Source: https://github.com/myohub/myosuite/blob/main/tutorials/2_Load_policy.ipynb

Creates an instance of the 'myoElbowPose1D6MExoRandom-v0' environment and resets it to its initial state.

```python
env = gym.make('myoElbowPose1D6MExoRandom-v0')

env.reset()

```

--------------------------------

### Launch MyoSuite Training

Source: https://github.com/myohub/myosuite/blob/main/myosuite/agents/README.md

Commands to launch training for MyoSuite agents. Supports native execution or using MJRL with local or SLURM launchers.

```bash
% sh train_myosuite.sh myo         # runs natively
```

```bash
% sh train_myosuite.sh myo local mjrl   # use mjrl with local launcher
```

```bash
% sh train_myosuite.sh myo slurm mjrl   # use mjrl with slurm launcher
```

--------------------------------

### Train New Policy with DEP-RL

Source: https://github.com/myohub/myosuite/blob/main/docs/source/baselines.md

Initiate training for a new locomotion policy using a specified JSON configuration file. Adapt 'working_dir', 'sequential', and 'parallel' settings based on your system's memory. High default settings can consume over 30 GB of RAM.

```bash
python -m deprl.main baselines_DEPRL/myoLegWalk.json
```

--------------------------------

### Import Libraries for PPO Training

Source: https://github.com/myohub/myosuite/blob/main/tutorials/7_Fatigue_Modeling.ipynb

Imports necessary libraries for training PPO models and using checkpoint callbacks.

```python
from stable_baselines3 import PPO
from stable_baselines3.common.callbacks import CheckpointCallback
```

--------------------------------

### Build Docs Directly with Sphinx

Source: https://github.com/myohub/myosuite/blob/main/docs/Readme.md

Builds the HTML documentation directly using the 'sphinx-build' command if 'make' is not available. Specify the source and output directories.

```bash
sphinx-build -b html docs/source docs/build/html
```

--------------------------------

### Solve Quadratic Program

Source: https://github.com/myohub/myosuite/blob/main/tutorials/6_Inverse_Dynamics.ipynb

Solves a quadratic program using the OSQP solver. It sets up the problem with given matrices and bounds, and applies warm-starting for efficiency.

```python
def solve_qp(P, q, lb, ub, x0):
    """
    Solve a quadratic program.
    """
    P = spa.csc_matrix(P)
    A = spa.csc_matrix(spa.eye(q.shape[0]))
    m = osqp.OSQP()
    m.setup(P=P, q=q, A=A, l=lb, u=ub, verbose=False)
    m.warm_start(x=x0)
    res = m.solve()
    return res.x
```

--------------------------------

### Run MyoLegReflex with Default Parameters

Source: https://github.com/myohub/myosuite/blob/main/docs/source/baselines.md

Execute this code snippet to run MyoLegReflex with default parameters. Note that this code only works within the 'myosuite/docs/source/tutorials/4b_reflex' directory.

```python
import ReflexCtrInterface
import numpy as np

sim_time = 5  # in seconds
dt = 0.01
steps = int(sim_time/dt)
frames = []

params = np.loadtxt('baseline_params.txt')

Myo_env = ReflexCtrInterface.MyoLegReflex()
Myo_env.reset()

Myo_env.set_control_params(params)

for timstep in range(steps):
    frame = Myo_env.env.mj_render()
    Myo_env.run_reflex_step()
Myo_env.env.close()
```

--------------------------------

### Load Custom Policy from Path

Source: https://github.com/myohub/myosuite/blob/main/tutorials/4a_deprl.ipynb

Demonstrates how to load a custom policy from a specified file path using `deprl.load()`. This is an alternative to loading a baseline policy.

```python
deprl.load(path, env)
```

--------------------------------

### BaseV0 Class Methods

Source: https://github.com/myohub/myosuite/blob/main/docs/source/api/base_v0.md

This section details the primary methods available for interacting with the BaseV0 environment.

```APIDOC
## initializeConditions()

### Description
Initializes the conditions for the environment.

### Method
N/A (Constructor or initialization method)

### Endpoint
N/A
```

```APIDOC
## step(a, **kwargs)

### Description
Steps the simulation forward by one time unit (t => t+1). This method uses the robot interface to safely advance the simulation, respecting position and velocity limits. It accepts the action at time 't' and returns the observation, reward, done status, and additional info at time 't+1'.

### Method
N/A (Method of the BaseV0 class)

### Endpoint
N/A

### Parameters
#### Request Body
- **a** (object) - Required - Action at time t.
- **kwargs** (dict) - Optional - Additional keyword arguments.
```

```APIDOC
## reset(fatigue_reset=True, *args, **kwargs)

### Description
Resets the environment to an initial internal state, returning an initial observation and info. This method is typically called after initialization with a seed to ensure reproducible random starting states. For custom environments, ensure the first line is `super().reset(seed=seed)` for correct seeding.

### Method
N/A (Method of the BaseV0 class)

### Endpoint
N/A

### Parameters
#### Query Parameters
- **fatigue_reset** (bool) - Optional - Whether to reset fatigue.
- **seed** (int) - Optional - The seed used to initialize the environment's PRNG. If None, the PRNG is not reset if it already exists. Passing an integer resets the PRNG.
- **options** (dict) - Optional - Additional information to specify how the environment is reset.

### Returns
#### Success Response
- **observation** (ObsType) - Observation of the initial state.
- **info** (dictionary) - Auxiliary information complementing the observation.
```

```APIDOC
## set_fatigue_reset_random(fatigue_reset_random)

### Description
Sets the flag for random fatigue reset.

### Method
N/A (Method of the BaseV0 class)

### Endpoint
N/A

### Parameters
#### Request Body
- **fatigue_reset_random** (object) - Required - The value to set for random fatigue reset.
```

--------------------------------

### Launch TorchRL Training Job

Source: https://github.com/myohub/myosuite/blob/main/myosuite/agents/README.md

Command to launch a training job using TorchRL.

```bash
python torchrl_job_script.py
```

--------------------------------

### demo_robot

Source: https://github.com/myohub/myosuite/blob/main/docs/source/api/robot.md

A demonstration function for the robot.

```APIDOC
## demo_robot

### Description
A demonstration function for the robot.
```

--------------------------------

### Create Gym Environment

Source: https://github.com/myohub/myosuite/blob/main/docs/source/challenge-doc.md

Creates a Gym environment for the MyoChallenge. Use normalize_act=True for action space [-1 to 1] or normalize_act=False for action space [0 to 1].

```python
env = gym.make("myoChallengeOslRunRandom-v0", normalize_act=False)
```

```python
env = gym.make("myoChallengeOslRunRandom-v0", normalize_act=True)
```

--------------------------------

### Train PPO Agent with Fatigue Reset

Source: https://github.com/myohub/myosuite/blob/main/tutorials/7_Fatigue_Modeling.ipynb

Initializes a MyoSuite environment with random fatigue resets, sets up a checkpoint callback to save model progress every 10000 steps, and trains a PPO model for 200000 total timesteps. Ensure the environment name and save path are correctly configured.

```python
env_name = "myoFatiElbowPose1D6MRandom-v0"

env = gym.make(env_name)
env.unwrapped.set_fatigue_reset_random(True)
env.reset()

# Save a checkpoint every 10000 steps
checkpoint_callback = CheckpointCallback(
  save_freq=10000,
  save_path=f"./{env_name}/iterations/",
  name_prefix="rl_model",
  save_replay_buffer=True,
  save_vecnormalize=True,
)

model = PPO("MlpPolicy", env, verbose=0)
model.learn(total_timesteps=200000, callback=checkpoint_callback)
```

--------------------------------

### Load a pre-trained policy

Source: https://github.com/myohub/myosuite/blob/main/tutorials/2_Load_policy.ipynb

Loads a policy from a specified pickle file. Includes a check to ensure the policy file exists before attempting to load it.

```python
import os
import pickle

pth = '../myosuite/agents/baslines_NPG/'
policy = os.path.join(
    pth,
    "myoElbowPose1D6MExoRandom-v0/2022-02-26_21-16-27/36_env=myoElbowPose1D6MExoRandom-v0,seed=1/iterations/best_policy.pickle",
)

if not os.path.exists(policy):
    print(f"Policy file not found at {policy}, skipping policy loading in this environment.")
    pi = None
else:
    with open(policy, 'rb') as f:
        pi = pickle.load(f)

```

--------------------------------

### Simulate Trained Agent with Fatigue

Source: https://github.com/myohub/myosuite/blob/main/tutorials/7_Fatigue_Modeling.ipynb

Loads a PPO model and runs simulations in a fatigue-enabled environment. Configure video generation, data storage, and simulation parameters. Ensure the model path exists before execution.

```python
env_name = "myoFatiElbowPose1D6MRandom-v0"

GENERATE_VIDEO = True
GENERATE_VIDEO_EPS = 4  #number of episodes that are rendered BOTH at the beginning (i.e., without fatigue) and at the end (i.e., with fatigue)

STORE_DATA = True  #store collected data from evaluation run in .npy file
n_eps = 250

###################################

env = gym.make(env_name)

model_path = f"{env_name}/iterations/rl_model_200000_steps.zip"
if not os.path.exists(model_path):
    print("PPO model not found; skipping evaluation. Train with 4c_Train_SB_policy or download baselines.")
    env.close()
else:
    from stable_baselines3 import PPO
model = PPO.load(f"{env_name}/iterations/rl_model_200000_steps")

env.unwrapped.set_fatigue_reset_random(False)
env.unwrapped.reset(fatigue_reset=True)  #ensure that fatigue is reset before the simulation starts

env.unwrapped.mj_model.cam_poscom0[0] = np.array([-1.3955, -0.3287,  0.6579])

data_store = []
if GENERATE_VIDEO:
    frames = []

env.unwrapped.target_jnt_value = env.unwrapped.target_jnt_range[:, 1]
env.unwrapped.target_type = 'fixed'
env.unwrapped.update_target(restore_sim=True)

start_time = time.time()
for ep in range(n_eps):
    print("Ep {} of {}".format(ep, n_eps))

    for _cstep in range(env.spec.max_episode_steps):
        if GENERATE_VIDEO and (ep in range(GENERATE_VIDEO_EPS) or ep in range(n_eps-GENERATE_VIDEO_EPS, n_eps)):
            frame = env.unwrapped.mj_renderer.render_offscreen(width=400, height=400, camera_id=0)

            # Add text overlay
            _current_time = (ep*env.spec.max_episode_steps + _cstep)*env.unwrapped.dt
            frame = np.array(add_text_to_frame(frame,
                    f"t={str(int(_current_time//60)).zfill(2)}:{str(int(_current_time%60)).zfill(2)}min",
                    pos=(285, 3), color=(0, 0, 0), fontsize=18))

            frames.append(frame)
        o = env.unwrapped.get_obs()
        a = model.predict(o)[0]
        next_o, r, done, _, ifo = env.unwrapped.step(a) # take an action based on the current observation

        data_store.append({"action":a.copy(),
                            "jpos":env.unwrapped.mj_data.qpos.copy(),
                            "mlen":env.unwrapped.mj_data.actuator_length.copy(),
                            "act":env.unwrapped.mj_data.act.copy(),
                            "reward":r,
                            "solved":env.unwrapped.rwd_dict['solved'].item(),
                            "pose_err":env.unwrapped.get_obs_dict(env.unwrapped.mj_model, env.unwrapped.mj_data)["pose_err"],
                            "MA":env.unwrapped.muscle_fatigue.MA.copy(),
                            "MR":env.unwrapped.muscle_fatigue.MR.copy(),
                            "MF":env.unwrapped.muscle_fatigue.MF.copy(),
                            "ctrl":env.unwrapped.last_ctrl.copy()})
env.close()

## OPTIONALLY: Stored simulated data
if STORE_DATA:
    os.makedirs(f"{env_name}/logs", exist_ok=True)
    np.save(f"{env_name}/logs/fatitest.npy", data_store)

## OPTIONALLY: Render video
video_written = False
if GENERATE_VIDEO:
    os.makedirs(f'{env_name}/videos', exist_ok=True)
    video_written = safe_write_video(
        f'{env_name}/videos/fatitest.mp4',
        frames,
        fps=int(1/env.unwrapped.dt),
    )

end_time = time.time()
print(f"DURATION: {end_time - start_time:.2f}s")

if GENERATE_VIDEO and video_written:
    display(Video(f'{env_name}/videos/fatitest.mp4', embed=True))
```

--------------------------------

### Generate Video of Trained Policy

Source: https://github.com/myohub/myosuite/blob/main/tutorials/SAR/SAR_tutorial.ipynb

Creates and displays a video of a trained policy's performance in a specified environment. Ensure the video name is unique to avoid overwriting.

```python
# make and load a video of the trained policy

video_name = 'RL+E2E_vid'
get_vid('RL-E2E', 'myoHandReorient100-v0', '0', determ=False, episodes=10, video_name=video_name)
display(Video(f"{video_name}.mp4", embed=True))
```

--------------------------------

### Visualize MyoSuite environments on MacOS

Source: https://github.com/myohub/myosuite/blob/main/README.md

On MacOS, use `mjpython` to run the environment visualization command due to the use of mujoco native `launch_passive`.

```bash
mjpython -m myosuite.utils.examine_env --env_name myoElbowPose1D6MRandom-v0
```