PufferLib

https://puffer.ai/docs/
Admin
PufferLib is a reinforcement learning library that enables training at millions of steps per second...
Tokens:3,960
Snippets:41
Trust Score:5.6
Update:6 months ago
Show doc for...
Tokens:
Raw
### Install PufferLib via Pip

Source: https://puffer.ai/docs.html

Instructions for installing the PufferLib package using pip, including options for installing from source.

```bash
(uv) pip install pufferlib
(uv) pip install -e .
```

--------------------------------

### Setup PufferTank Docker Environment

Source: https://puffer.ai/docs.html

Commands to clone and initialize the PufferTank prebuilt GPU Docker environment for development.

```bash
git clone https://github.com/pufferai/puffertank
cd puffertank
./docker.sh test
```

--------------------------------

### Emulate Gymnasium and PettingZoo Environments with PufferLib

Source: https://puffer.ai/docs.html

This snippet shows how to use PufferLib's emulation classes to wrap Gymnasium and PettingZoo environments. It provides access to environment properties like observation and action spaces, and standard methods like reset, step, render, and close. For PettingZoo, it also exposes agent-specific information.

```python
import pufferlib.emulation

# Emulate a Gymnasium environment
gym_env = pufferlib.emulation.GymnasiumPufferEnv(env=None, env_creator=None, env_args=[], env_kwargs={}, buf=None, seed=0)
render_mode = gym_env.render_mode
obs_space = gym_env.observation_space
act_space = gym_env.action_space
single_obs_space = gym_env.single_observation_space
single_act_space = gym_env.single_action_space
gym_env.seed(seed)
obs, info = gym_env.reset(seed=None)
obs, reward, termination, truncation, info = gym_env.step(action)
gym_env.render()
gym_env.close()

# Emulate a PettingZoo environment
pettingzoo_env = pufferlib.emulation.PettingZooPufferEnv(env=None, env_creator=None, env_args=[], env_kwargs={}, buf=None, seed=0)
agents = pettingzoo_env.agents
possible_agents = pettingzoo_env.possible_agents
done = pettingzoo_env.done
# ... (reset, step, render, close methods are also available)
```

--------------------------------

### Create and Manage Vectorized Environments with PufferLib

Source: https://puffer.ai/docs.html

This snippet demonstrates how to create and interact with vectorized environments using pufferlib.vector.make. It covers synchronous and asynchronous operations like reset, step, and closing environments. The number of environments is accessible via `vecenv.num_envs`.

```python
import pufferlib.vector

# Create a vectorized environment
vecenv = pufferlib.vector.make(env_creator_or_creators, env_args=None, env_kwargs=None, backend=PufferEnv, num_envs=1, seed=0)

# Get the number of environments
num_envs = vecenv.num_envs

# Synchronous reset
observations = vecenv.reset()

# Asynchronous reset
async_observations = vecenv.async_reset(seed=None)

# Synchronous step
observations, rewards, terminations, truncations, infos = vecenv.step(actions)

# Asynchronous step - send actions
vecenv.send(actions)
# Asynchronous step - receive observations
observations, rewards, terminations, truncations, infos = vecenv.recv()

# Call env.notify for all environments
vecenv.notify()

# Close all environments
vecenv.close()
```

--------------------------------

### PufferLib CLI Operations

Source: https://puffer.ai/docs.html

Common CLI commands for training, evaluating, and performing hyperparameter sweeps on reinforcement learning environments.

```bash
puffer train puffer_breakout --train.device cuda
puffer eval puffer_breakout --load-model-path latest
puffer sweep puffer_breakout --wandb --tag experiment_v1
torchrun --standalone --nnodes=1 --nproc-per-node=6 -m pufferlib.pufferl train puffer_nmmo3
```

--------------------------------

### PuffeRL API Usage

Source: https://puffer.ai/docs.html

Core methods for the PuffeRL trainer class, including training loops, logging, and checkpoint management.

```python
trainer = pufferl.PuffeRL(config, vecenv, policy)
trainer.train()
trainer.evaluate()
trainer.save_checkpoint()
trainer.print_dashboard()
```
Tokens:
Raw