### Run Training Script

Source: https://github.com/lucidrains/blackbox-gradient-sensing/blob/main/README.md

Execute the main training script after installing dependencies.

```bash
python train.py
```

--------------------------------

### Install Dependencies

Source: https://github.com/lucidrains/blackbox-gradient-sensing/blob/main/README.md

Install project dependencies from requirements.txt. Consider using `uv pip install` for faster installation.

```bash
pip install -r requirements.txt  # or `uv pip install`, to keep up with the times
```

--------------------------------

### Install blackbox-gradient-sensing

Source: https://github.com/lucidrains/blackbox-gradient-sensing/blob/main/README.md

Install the library using pip.

```bash
pip install blackbox-gradient-sensing
```

--------------------------------

### Install SWIG

Source: https://github.com/lucidrains/blackbox-gradient-sensing/blob/main/README.md

Install SWIG if you encounter related errors during the build process.

```bash
apt install swig -y
```

--------------------------------

### Implement Full Training Loop with Gymnasium

Source: https://context7.com/lucidrains/blackbox-gradient-sensing/llms.txt

Demonstrates a complete training setup including recurrent actor configuration and custom optimizer integration.

```python
from adam_atan2_pytorch import AdoptAtan2
from blackbox_gradient_sensing import BlackboxGradientSensing, Actor
import gymnasium as gym
from shutil import rmtree
from math import ceil

# Environment setup
continuous = True
sim = gym.make('LunarLander-v3', render_mode='rgb_array', continuous=continuous)

dim_state = sim.observation_space.shape[0]
num_actions = sim.action_space.shape[0] if continuous else sim.action_space.n

# Actor with recurrent memory
actor = Actor(
    dim_state=dim_state,
    num_actions=num_actions,
    continuous=continuous,
    hidden_dim=32,
    sample=True,
    weight_norm_linears=True
)

# BGS with advanced configuration
bgs = BlackboxGradientSensing(
    actor=actor,
    noise_pop_size=100,
    num_selected=15,
    num_rollout_repeats=4,
    actor_is_recurrent=True,          # enable recurrent actor
    use_ema=True,                     # EMA for stability
    ema_decay=0.9,
    optim_klass=AdoptAtan2,           # custom optimizer
    optim_kwargs=dict(cautious_factor=0.1),
    optim_step_post_hook=lambda: actor.norm_weights_(),
    factorized_noise=True,            # memory-efficient noise
    orthogonalized_noise=True,        # orthogonalized perturbations
    num_std_below_mean_thres_accept=-0.25,  # noise acceptance threshold
    sample_actions_from_actor=False,  # actor handles sampling
    state_norm=dict(dim_state=dim_state),
    cpu=True,
    torch_compile_actor=False
)
```

--------------------------------

### Configure 🤗 accelerate for Distributed Training

Source: https://github.com/lucidrains/blackbox-gradient-sensing/blob/main/README.md

Configure the accelerate library for distributed training setup.

```bash
accelerate config
```

--------------------------------

### Configure Video Recording and Training

Source: https://context7.com/lucidrains/blackbox-gradient-sensing/llms.txt

Sets up environment video recording using Gymnasium wrappers and executes the training loop.

```python
if bgs.is_main:
    video_folder = './recording'
    rmtree(video_folder, ignore_errors=True)

    total_eps_before_update = ceil(1000 / bgs.num_episodes_per_learning_cycle) * bgs.num_episodes_per_learning_cycle

    sim = gym.wrappers.RecordVideo(
        env=sim,
        video_folder=video_folder,
        name_prefix='lunar-lander',
        episode_trigger=lambda eps: (eps % total_eps_before_update == 0),
        disable_logger=True
    )

# Train for 10000 environment interactions
bgs(sim, 10000)

# Save for sim-to-real transfer
bgs.save('./sim-trained-actor-and-state-norm.pt')

# Load and continue training or deploy
bgs.load('./sim-trained-actor-and-state-norm.pt')

# Get wrapped actor for inference (includes state norm and gene pool)
wrapped_actor = bgs.return_wrapped_actor()
```

--------------------------------

### Launch Distributed Training

Source: https://github.com/lucidrains/blackbox-gradient-sensing/blob/main/README.md

Launch the training script using accelerate for distributed execution.

```bash
accelerate launch train.py
```

--------------------------------

### Manage Model Persistence and Inference

Source: https://context7.com/lucidrains/blackbox-gradient-sensing/llms.txt

Demonstrates saving and loading checkpoints, including manual loading for custom deployment and inference with state normalization.

```python
from blackbox_gradient_sensing import BlackboxGradientSensing, Actor
import torch

# After training
bgs.save('./checkpoint.pt', overwrite=True)

# Load into existing BGS instance
bgs.load('./checkpoint.pt')

# Manual loading for custom deployment
checkpoint = torch.load('./checkpoint.pt', weights_only=True)

# checkpoint contains:
# - 'actor': actor state dict
# - 'state_norm': state normalizer state dict (if used)
# - 'latents': gene pool state dict (if used)
# - 'ema_actor': EMA state dict (if used)
# - 'step': training step count

actor = Actor(dim_state=8, num_actions=4, continuous=True)
actor.load_state_dict(checkpoint['actor'])
actor.eval()

# Get wrapped actor for inference with normalization
wrapped_actor = bgs.return_wrapped_actor()
state = torch.randn(8)
action, hiddens = wrapped_actor(state, hiddens=None)
```

--------------------------------

### Initialize and Train BlackboxGradientSensing

Source: https://context7.com/lucidrains/blackbox-gradient-sensing/llms.txt

Orchestrates evolutionary strategy training by perturbing parameters, evaluating fitness, and updating parameters. Requires a simulation environment and an actor network. Training can be saved for later use.

```python
import numpy as np
from torch import nn
from blackbox_gradient_sensing import BlackboxGradientSensing

# Create a simple simulation environment
class Sim:
    def reset(self, seed=None):
        return np.random.randn(5)  # state vector of dimension 5

    def step(self, actions):
        return np.random.randn(5), np.random.randn(1), False  # state, reward, done

sim = Sim()

# Simple actor network: state dimension 5 -> 2 actions
actor = nn.Linear(5, 2)

# Initialize BGS with the actor
bgs = BlackboxGradientSensing(
    actor=actor,
    noise_pop_size=10,        # number of noise perturbations per update
    num_selected=2,           # top-k noise directions for weighted update
    num_rollout_repeats=1,    # rollout repeats per noise for variance reduction
    max_timesteps=500,        # max timesteps per episode
    learning_rate=8e-2,       # learning rate for parameter updates
    noise_std_dev=0.1,        # standard deviation of noise perturbations
    cpu=True,                 # run on CPU
    torch_compile_actor=False # disable torch.compile for simple networks
)

# Train for 100 environment interactions
bgs(sim, 100)

# Save trained policy for later use or sim-to-real transfer
bgs.save('./trained-policy.pt')
```

--------------------------------

### Configure Distributed Training with Accelerate

Source: https://context7.com/lucidrains/blackbox-gradient-sensing/llms.txt

Sets up BGS for multi-GPU or multi-node training using the HuggingFace Accelerator instance.

```python
from accelerate import Accelerator
from blackbox_gradient_sensing import BlackboxGradientSensing, Actor
import gymnasium as gym

# Configure accelerate first: `accelerate config`
# Then launch with: `accelerate launch train.py`

accelerator = Accelerator()

sim = gym.make('LunarLander-v3', continuous=True)
dim_state = sim.observation_space.shape[0]
num_actions = sim.action_space.shape[0]

actor = Actor(
    dim_state=dim_state,
    num_actions=num_actions,
    continuous=True,
    sample=True
)

bgs = BlackboxGradientSensing(
    actor=actor,
    accelerator=accelerator,      # pass accelerator instance
    noise_pop_size=100,           # distributed across processes
    num_selected=15,
    num_rollout_repeats=4,
    use_ema=True,                 # exponential moving average
    factorized_noise=True,        # memory-efficient noise
    orthogonalized_noise=True,    # orthogonalized perturbations
    state_norm=dict(dim_state=dim_state),
    torch_compile_actor=True      # enable torch.compile
)

# Train with distributed rollouts
bgs(sim, num_env_interactions=10000)

# Save on main process only
if bgs.is_main:
    bgs.save('./distributed-policy.pt')
```

--------------------------------

### Initialize LatentGenePool and BGS

Source: https://context7.com/lucidrains/blackbox-gradient-sensing/llms.txt

Configures a latent gene pool for genetic algorithms and integrates it into the BlackboxGradientSensing framework.

```python
from blackbox_gradient_sensing import BlackboxGradientSensing, Actor, LatentGenePool
import torch

# Create latent gene pool for genetic algorithm
gene_pool = LatentGenePool(
    dim=32,                   # dimension of each gene vector
    num_genes_per_island=10,  # population size per island
    num_selected=4,           # genes selected for mating
    tournament_size=3,        # tournament selection size
    num_elites=1,             # elite genes exempt from mutation
    mutation_std_dev=0.1,     # mutation noise level
    num_islands=2,            # number of isolated populations
    migrate_genes_every=10,   # migration frequency
    num_frac_migrate=0.1      # fraction of population that migrates
)

# Access a gene by index (returns L2-normalized vector)
gene = gene_pool[0]  # shape: (32,)

# Evolve population based on fitness scores
fitnesses = torch.randn(20)  # fitness for all genes (num_islands * num_genes_per_island)
selected_ids = gene_pool.evolve(fitnesses, temperature=1.5)

# Full BGS setup with genetic algorithm
actor = Actor(
    dim_state=8,
    num_actions=4,
    continuous=True,
    accepts_latent=True,
    dim_latent=32
)

bgs = BlackboxGradientSensing(
    actor=actor,
    latent_gene_pool=dict(
        dim=32,
        num_genes_per_island=5,
        num_islands=2,
        num_selected=2,
        tournament_size=2
    ),
    mutate_latent_genes=True,     # apply mutations to latent genes
    crossover_after_step=100,     # start crossover after N steps
    crossover_every_step=50,      # crossover frequency
    noise_pop_size=50,
    num_selected=10,
    num_rollout_repeats=3,
    cpu=True
)
```

--------------------------------

### Initialize and Use BlackboxGradientSensing

Source: https://github.com/lucidrains/blackbox-gradient-sensing/blob/main/README.md

Instantiate BlackboxGradientSensing with an actor and environment for learning. The actor should accept a state tensor and output action logits. The environment is passed for training interactions.

```python
from torch import nn
from blackbox_gradient_sensing import BlackboxGradientSensing

actor = nn.Linear(5, 2) # contrived network from state of 5 dimension to two actions

bgs = BlackboxGradientSensing(
    actor = actor,
    noise_pop_size = 10,      # number of noise perturbations
    num_selected = 2,         # topk noise selected for update
    num_rollout_repeats = 1   # how many times to redo environment rollout, per noise
)

bgs(sim, 100) # pass the simulation environment in - say for 100 interactions with env
```

--------------------------------

### Mock Environment for Blackbox Gradient Sensing

Source: https://github.com/lucidrains/blackbox-gradient-sensing/blob/main/README.md

A mock environment class simulating state, reward, and done signals for reinforcement learning.

```python
import numpy as np

class Sim:
    def reset(self, seed = None):
        return np.random.randn(5) # state

    def step(self, actions):
        return np.random.randn(5), np.random.randn(1), False # state, reward, done

sim = Sim()
```

--------------------------------

### Initialize and Use StateNorm for Online Normalization

Source: https://context7.com/lucidrains/blackbox-gradient-sensing/llms.txt

Implements online state normalization to maintain mean and variance statistics for stable policy training. Can be passed directly or as a dictionary to BGS. Supports training and evaluation modes for updating or freezing statistics.

```python
from blackbox_gradient_sensing import BlackboxGradientSensing
from blackbox_gradient_sensing.bgs import StateNorm
import torch

# Create state normalizer for 8D observations
state_norm = StateNorm(
    dim_state=8,
    eps=1e-5       # epsilon for numerical stability
)

# Can be passed directly or as dict to BGS
bgs = BlackboxGradientSensing(
    actor=actor,
    state_norm=dict(dim_state=8),  # will be converted to StateNorm
    noise_pop_size=10,
    num_selected=2,
    num_rollout_repeats=1,
    cpu=True
)
```

```python
# StateNorm updates running statistics during training
state_norm.train()
normed_state = state_norm(torch.randn(8))  # updates statistics

# At inference, statistics are fixed
state_norm.eval()
normed_state = state_norm(torch.randn(8))  # uses frozen statistics
```

--------------------------------

### Create Actor Network for Reinforcement Learning

Source: https://context7.com/lucidrains/blackbox-gradient-sensing/llms.txt

Defines a recurrent actor network with options for weight normalization, RMSNorm, and latent gene input. Supports continuous action spaces by outputting mean and log variance. Can sample actions from the distribution.

```python
from blackbox_gradient_sensing import Actor, BlackboxGradientSensing

# Create actor for continuous control with 8D state and 4 actions
actor = Actor(
    dim_state=8,              # dimension of observation/state
    num_actions=4,            # number of action outputs
    continuous=True,          # continuous action space (outputs mean + log_var)
    hidden_dim=32,            # hidden layer dimension
    sample=True,              # sample actions from distribution
    weight_norm_linears=True  # apply weight normalization
)

# Actor with latent gene support for genetic algorithm
actor_with_genes = Actor(
    dim_state=8,
    num_actions=4,
    continuous=True,
    hidden_dim=64,
    accepts_latent=True,      # enable latent gene input
    dim_latent=32,            # dimension of latent gene vector
    sample=True
)
```

```python
# Forward pass with recurrent state
import torch
state = torch.randn(8)
action, hiddens = actor(state, hiddens=None, sample_temperature=1.0)
# Next step uses returned hiddens for memory
next_action, hiddens = actor(next_state, hiddens=hiddens, sample_temperature=1.0)
```

--------------------------------

### Save Learned Policy

Source: https://github.com/lucidrains/blackbox-gradient-sensing/blob/main/README.md

Save the learned policy and optional state normalization parameters after training.

```python
bgs.save('./actor-and-state-norm.pt')
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.