### Installation

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/usage-guide.md

Install the x-evolution library and optional dependencies for examples.

```bash
pip install x-evolution
```

```bash
pip install 'x-evolution[examples]'
```

--------------------------------

### Quick Start Example

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/00-START-HERE.md

Minimal example to get started with x-evolution, demonstrating model definition, fitness function, and ES initialization and execution.

```python
from x_evolution import EvoStrategy
import torch
from torch import nn

# 1. Define a model
model = nn.Sequential(
    nn.Linear(8, 16),
    nn.ReLU(),
    nn.Linear(16, 4)
)

# 2. Define a fitness function (higher = better)
def fitness(model):
    x = torch.randn(32, 8)
    y = model(x).sum()
    return y.item()

# 3. Create and run ES
evo = EvoStrategy(
    model,
    environment=fitness,
    num_generations=100,
    noise_population_size=30
)

evo()  # Run training!

```

--------------------------------

### Install with optional dependencies

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/00-START-HERE.md

Installs the x-evolution library along with optional dependencies for examples.

```bash
pip install 'x-evolution[examples]'
```

--------------------------------

### Example: Complete Training Script

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/usage-guide.md

A basic example of a complete training script using x-evolution, including model definition and optimizer setup.

```python
import torch
from torch import nn
from torch.optim.lr_scheduler import CosineAnnealingLR
from x_evolution import EvoStrategy

# 1. Define model
model = nn.Sequential(
    nn.Linear(10, 32),
    nn.ReLU(),
    nn.Linear(32, 16),
    nn.ReLU(),
    nn.Linear(16, 2)
)


```

--------------------------------

### Minimal Example

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/usage-guide.md

A basic example demonstrating how to define a model, a fitness function, and run the EvoStrategy.

```python
import torch
from torch import nn
from x_evolution import EvoStrategy

# 1. Define a model
model = nn.Sequential(
    nn.Linear(4, 8),
    nn.ReLU(),
    nn.Linear(8, 2)
)

# 2. Define a fitness function
def fitness(model):
    # Return a scalar (higher = better)
    x = torch.randn(32, 4)
    output = model(x)
    loss = output.mean()
    return -loss.item()  # Negative loss as fitness

# 3. Create and run ESevo = EvoStrategy(
    model,
    environment=fitness,
    num_generations=50,
    noise_population_size=20,
    learning_rate=0.001
)
evo()  # Run training
```

--------------------------------

### Load and Resume Training

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/00-START-HERE.md

Example showing how to load a checkpoint and resume training with EvoStrategy.

```python
# Load checkpoint
model.load_state_dict(torch.load('checkpoints/evolved.model.100.pt'))

# Continue training
evo = EvoStrategy(model, environment=fitness_fn, num_generations=200)
evo()
```

--------------------------------

### Run a Basic ES Training

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/00-START-HERE.md

A basic example demonstrating how to initialize and run the EvoStrategy for training.

```python
from x_evolution import EvoStrategy

evo = EvoStrategy(model, environment=fitness_fn, num_generations=100)
evo()
```

--------------------------------

### EvoStrategy Usage Example

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/usage-guide.md

This example shows how to define a fitness function, initialize EvoStrategy, run training, evaluate the final model, and save it.

```python
def fitness(model):
    # Simulate a classification task
    x = torch.randn(64, 10)
    y = torch.randint(0, 2, (64,))
    
    logits = model(x)
    loss = nn.functional.cross_entropy(logits, y)
    
    return -loss.item()

# 3. Initialize ES
evo = EvoStrategy(
    model,
    environment=fitness,
    num_generations=100,
    noise_population_size=30,
    learning_rate=0.001,
    learned_noise_scale=True,
    noise_scale=0.01,
    noise_scale_learning_rate=5e-4,
    use_scheduler=True,
    scheduler_klass=CosineAnnealingLR,
    scheduler_kwargs={'T_max': 100},
    checkpoint_every=10,
    checkpoint_path='./checkpoints',
    verbose=True
)

# 4. Run training
fitnesses = evo()

# 5. Evaluate final model
final_fitness = fitness(model)
print(f"Final fitness: {final_fitness}")

# 6. Save model
torch.save(model.state_dict(), './final_model.pt')
```

--------------------------------

### Core Parameters Example

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/00-START-HERE.md

Example demonstrating the core parameters for initializing EvoStrategy, including model, fitness function, generations, population size, noise scale, learning rate, and options for learned noise scale and mirror sampling.

```python
evo = EvoStrategy(
    model,
    environment=fitness_fn,              # Your fitness function
    num_generations=100,                 # Training steps
    noise_population_size=30,            # Samples per generation
    noise_scale=0.01,                    # Noise magnitude
    learning_rate=0.001,                 # Update step size
    learned_noise_scale=False,           # Adapt noise per param?
    mirror_sampling=True                 # Antithetical sampling?
)

```

--------------------------------

### Optimize Specific Parameters

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/00-START-HERE.md

Example demonstrating how to optimize specific parameters of the model using EvoStrategy.

```python
evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    params_to_optimize={'encoder': model.encoder, 'head': model.head}
)

evo(scope='encoder', num_generations=50)  # Optimize encoder only
evo(scope='head', num_generations=50)     # Then optimize head
```

--------------------------------

### Load and Resume Training

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/usage-guide.md

Example of loading a previously trained model checkpoint and resuming training.

```python
# Load previously trained model
evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=1000
)

# Optionally load checkpoint before continuing
checkpoint_path = './checkpoints/evolved.model.500.pt'
model.load_state_dict(torch.load(checkpoint_path))

# Continue training for more generations
evo(num_generations=500)
```

--------------------------------

### EvoStrategy forward() example

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/api-reference/evostrategy.md

Example demonstrating how to initialize and run the EvoStrategy for training.

```python
import torch
from torch import nn
from x_evolution import EvoStrategy

# Define a simple model
model = nn.Sequential(
    nn.Linear(8, 16),
    nn.ReLU(),
    nn.Linear(16, 4)
)

# Define a fitness function
def fitness_fn(model):
    # Return a scalar fitness (higher is better)
    return torch.rand(1).item()

# Initialize evolutionary strategyevo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=10,
    noise_population_size=20,
    learning_rate=1e-3
)

# Run training
fitnesses = evo()
print(f"Training complete. Final shape: {fitnesses.shape}")
```

--------------------------------

### Optimize All Parameters (Default)

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/usage-guide.md

Example showing that by default, EvoStrategy optimizes all parameters of the model.

```python
evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100
    # params_to_optimize=None  # Optimizes all parameters by default
)
```

--------------------------------

### Multi-Process Training with torchrun

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/usage-guide.md

Commands for running multi-process training using torchrun, with examples for CPU and GPU configurations.

```bash
# 8 processes on CPU (for testing)
$ torchrun --nproc_per_node=8 train_script.py --cpu True

# 4 processes on GPU
$ torchrun --nproc_per_node=4 train_script.py
```

--------------------------------

### Custom Fitness Weighting

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/usage-guide.md

Example of a custom exponential fitness weighting function.

```python
def custom_weighting(fitnesses):
    """Exponential weighting of fitness."""
    weights = torch.exp(fitnesses - fitnesses.max())
    return weights / weights.sum()

evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    fitness_to_weighted_factor=custom_weighting
)
```

--------------------------------

### Vectorized Environment Example

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/configuration.md

Example demonstrating how to set up a vectorized environment for parallel execution.

```python
def vectorized_fitness(model):
    return torch.randn(8)  # 8 parallel envs

evo = EvoStrategy(
    model,
    environment=vectorized_fitness,
    num_generations=100,
    vectorized=True,
    vector_size=8
)
```

--------------------------------

### use_optimizer Example

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/configuration.md

Examples demonstrating the use of an optimizer to apply updates versus applying updates directly.

```python
# With optimizer (default)
eşvo1 = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    use_optimizer=True
)

# Without optimizer
eşvo2 = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    use_optimizer=False
)
```

--------------------------------

### optimizer_kwargs Example

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/configuration.md

Example of setting additional keyword arguments for the optimizer constructor.

```python
evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    optimizer_klass=Adam,
    optimizer_kwargs={'weight_decay': 5e-4}
)
```

--------------------------------

### Learning Rate Scheduler Example

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/configuration.md

Example of enabling and configuring learning rate scheduling with `CosineAnnealingLR`.

```python
from torch.optim.lr_scheduler import CosineAnnealingLR

evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    use_scheduler=True,
    scheduler_klass=CosineAnnealingLR,
    scheduler_kwargs={'T_max': 100}
)
```

--------------------------------

### use_sigma_optimizer Example

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/configuration.md

Example of enabling an optimizer for sigma updates when learned_noise_scale is true.

```python
evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    learned_noise_scale=True,
    use_sigma_optimizer=True
)
```

--------------------------------

### Scoped Optimization

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/usage-guide.md

Example of defining and using parameter scopes for phased optimization.

```python
# Define parameter scopes during initializationevo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    params_to_optimize={
        'encoder': model.encoder,
        'decoder': model.decoder,
        'head': model.head
    }
)

# Optimize each scope separatelyevo(scope='encoder', num_generations=50)evo(scope='decoder', num_generations=50)evo(scope='head', num_generations=50)

# Or optimize all togetherevo(num_generations=50)  # Uses all scopes

# Or override at runtimeevo(params_to_optimize=model.encoder, num_generations=50)
```

--------------------------------

### Log Fitness Scaling

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/usage-guide.md

Example of applying log transformation to fitness values for scaling.

```python
import torch

def log_fitness_transform(fitnesses):
    return torch.log(fitnesses + 1e-6)

evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    transform_fitness=log_fitness_transform,
    fitness_to_weighted_factor='normalize'
)
```

--------------------------------

### Train Multiple Machines

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/00-START-HERE.md

Instructions for setting up and running distributed training using accelerate.

```bash
# Setup accelerate
accelerate config

# Run on 4 GPUs
accelerate launch train_script.py
```

--------------------------------

### Installation

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/README.md

Command to install the x-evolution library.

```bash
pip install x-evolution
```

--------------------------------

### Fixed Noise Scale

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/usage-guide.md

Example of configuring a fixed noise scale for perturbations.

```python
evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    noise_scale=0.01,  # Fixed 1% perturbation
    learned_noise_scale=False
)
```

--------------------------------

### Custom Population Sizes for Exploration and Exploitation

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/usage-guide.md

Example of configuring EvoStrategy with different population sizes, noise scales, and learning rates for exploration and exploitation phases.

```python
# Small population for exploration evo_explore = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=50,
    noise_population_size=10,
    noise_scale=0.05,
    learning_rate=0.01
)

# Large population for exploitation evo_exploit = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=50,
    noise_population_size=100,
    noise_scale=0.001,
    learning_rate=0.0001
)

# Alternate between exploration and exploitation evo_explore()
 evo_exploit()
```

--------------------------------

### Sigma Optimizer Kwargs Example

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/configuration.md

Example of passing additional keyword arguments to the sigma optimizer.

```python
evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    learned_noise_scale=True,
    sigma_optimizer_kwargs={'weight_decay': 1e-5}
)
```

--------------------------------

### Optimize Specific Layers - Multiple Modules

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/usage-guide.md

Example of optimizing multiple specific layers by providing a list of module references.

```python
# Option 3: Multiple modulesevo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    params_to_optimize=[model.encoder, model.head]
)
```

--------------------------------

### Optimize Specific Layers - By Parameter Names

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/usage-guide.md

Example of optimizing specific layers by providing a list of parameter names.

```python
# Option 2: By parameter namesevo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    params_to_optimize=['encoder.0.weight', 'encoder.0.bias']
)
```

--------------------------------

### Meta-Evolutionary Strategy

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/usage-guide.md

Example of using a meta-evolutionary strategy that restores the initial model state after training.

```python
# Save initial model and restore after training
evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=50,
    rollback_model_at_end=True  # Restore initial model after training
)

fitnesses = evo()  # Returns fitness history
# Model is restored to initial state
# Can use fitnesses for meta-learning
```

--------------------------------

### Usage Example

Source: https://github.com/lucidrains/x-evolution/blob/main/README.md

Demonstrates how to use the EvoStrategy class for model evolution. It includes setting up a PyTorch model, defining an environment function to evaluate fitness, and initializing the EvoStrategy wrapper. The example shows how to run the evolution process and mentions that the model will be saved to the checkpoints folder.

```python
import torch
from x_evolution import EvoStrategy

# model

from torch import nn
model = torch.nn.Sequential(
    nn.Linear(8, 16),
    nn.ReLU(),
    nn.Linear(16, 4)
)

# evolution wrapper

evo_strat = EvoStrategy(
    model,
    environment = lambda model: torch.randint(0, 100, ()), # environment is just a function that takes in the individual model (with unique noise) and outputs a scalar (the fitness) the measure you are selecting for
    noise_population_size = 30, # increase this for better gradient estimates
    noise_scale = 1e-2,         # the scale of the perturbation noise, also the initial noise scale (sigma) if `learned_noise_scale` = True
    num_generations = 100,    # number of generations / training steps
    learning_rate = 1e-3,     # scale on update derived by fitness and perturb noises
    params_to_optimize = None # defaults to all parameters, but can be [str {param name}] or [Parameter]
)

# do evolution with your desired fitness function for so many generations

evo_strat()

# model will be saved under checkpoints/ folder
# can also specify checkpoint_every at init and select the one with your favored fitness score for continued policy gradient learning etc
```

--------------------------------

### Fitness to Weighted Factor Example (Preset)

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/configuration.md

Example of using a preset ('centered_rank') for converting fitness values to weights.

```python
# Using preset
evo1 = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    fitness_to_weighted_factor='centered_rank'
)
```

--------------------------------

### Scheduler Kwargs Example

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/configuration.md

Example of configuring keyword arguments for the main optimizer's learning rate scheduler using `StepLR`.

```python
evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    use_scheduler=True,
    scheduler_klass=StepLR,
    scheduler_kwargs={'step_size': 10, 'gamma': 0.1}
)
```

--------------------------------

### rollout_fixed_seed Example

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/configuration.md

Example of setting whether to use a fixed random seed for all rollouts within a generation.

```python
evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    rollout_fixed_seed=True  # Deterministic evals
)
```

--------------------------------

### Faster Training with Smaller Populations

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/usage-guide.md

Example of using a smaller population size and disabling mirror sampling for faster training, potentially trading gradient quality for speed.

```python
# Trade gradient quality for speed evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    noise_population_size=10,  # Smaller population
    mirror_sampling=False,  # Disable mirror sampling
    learning_rate=0.01  # Compensate with higher LR
)
```

--------------------------------

### EvoStrategy Initialization with learning_rate and use_optimizer

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/configuration.md

Example of initializing EvoStrategy with a learning rate and enabling the optimizer.

```python
evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    learning_rate=0.001,
    use_optimizer=True
)
```

--------------------------------

### Installation

Source: https://github.com/lucidrains/x-evolution/blob/main/README.md

Install the x-evolution package using pip.

```bash
$ pip install x-evolution
```

--------------------------------

### EvoStrategy Initialization with mirror_sampling

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/configuration.md

Examples demonstrating EvoStrategy initialization with and without mirror sampling.

```python
# With mirror sampling (default)
evo1 = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    mirror_sampling=True  # 2 evals per sample
)

# Without mirror sampling
evo2 = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    mirror_sampling=False  # 1 eval per sample
)
```

--------------------------------

### Using a Pre-configured Accelerator

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/configuration.md

Example of initializing `EvoStrategy` with a pre-configured Hugging Face `Accelerator` instance.

```python
from accelerate import Accelerator

accelerator = Accelerator(
    device_placement=True,
    mixed_precision='fp16'
)

evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    accelerator=accelerator
)
```

--------------------------------

### Learned Adaptive Noise Scales

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/usage-guide.md

Example of configuring learned and adaptive noise scales with a scheduler.

```python
from torch.optim.lr_scheduler import LambdaLR
evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    learned_noise_scale=True,
    noise_scale=0.01,  # Initial value
    noise_scale_learning_rate=5e-4,
    noise_scale_clamp_range=(1e-4, 1e-1),  # Bounds
    use_sigma_optimizer=True,
    use_sigma_scheduler=True,
    sigma_scheduler_klass=LambdaLR,
    sigma_scheduler_kwargs={
        'lr_lambda': lambda step: min(1.0, step / 50.0)
    }
)
```

--------------------------------

### EvoStrategy Initialization with num_generations

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/configuration.md

Example of initializing EvoStrategy with a specified number of generations.

```python
evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=1000
)
```

--------------------------------

### Optimize Specific Layers - By Module Reference

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/usage-guide.md

Example of optimizing only a specific layer (encoder) by providing the module reference.

```python
# Option 1: By module referenceevo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    params_to_optimize=model.encoder  # Only encoder layer
)
```

--------------------------------

### noise_low_rank Example

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/configuration.md

Example of setting the rank for low-rank noise decomposition to reduce memory and computation.

```python
evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    noise_low_rank=1  # Rank-1 approximation
)
```

--------------------------------

### Custom Scheduler Klass Example

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/configuration.md

Example of using a custom learning rate scheduler class (`LambdaLR`) with specific keyword arguments.

```python
from functools import partial
from torch.optim.lr_scheduler import LambdaLR

evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    use_scheduler=True,
    scheduler_klass=LambdaLR,
    scheduler_kwargs={'lr_lambda': lambda step: 1.0 / (1.0 + 0.1 * step)}
)
```

--------------------------------

### EvoStrategy Initialization with noise_scale

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/configuration.md

Example of initializing EvoStrategy with a specific noise scale.

```python
evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    noise_scale=0.01  # 1% standard perturbation
)
```

--------------------------------

### Fitness Transformation Example

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/configuration.md

Example of applying a custom fitness transformation function (`log_fitness`) to raw fitness values.

```python
import torch

def log_fitness(fitnesses):
    return torch.log(fitnesses + 1e-6)

evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    transform_fitness=log_fitness
)
```

--------------------------------

### optimizer_klass Example

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/configuration.md

Example of setting a custom optimizer class (Adam) with specific keyword arguments.

```python
from functools import partial
from torch.optim import Adam

eşvo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    optimizer_klass=partial(Adam, betas=(0.9, 0.999)),
    optimizer_kwargs={'eps': 1e-8}
)
```

--------------------------------

### EvoStrategy Initialization with learned_noise_scale

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/configuration.md

Example of initializing EvoStrategy with learned noise scales enabled.

```python
evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    learned_noise_scale=True,
    noise_scale=0.01,
    noise_scale_learning_rate=5e-4
)
```

--------------------------------

### EvoStrategy Initialization with noise_scale_learning_rate

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/configuration.md

Example of initializing EvoStrategy with a specific learning rate for learned noise scales.

```python
evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    learned_noise_scale=True,
    noise_scale_learning_rate=5e-4
)
```

--------------------------------

### Fitness to Weighted Factor Example (Custom Function)

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/configuration.md

Example of defining and using a custom function for converting fitness values to weights.

```python
# Using custom function
def custom_weighting(fitnesses):
    return (fitnesses - fitnesses.min()) / (fitnesses.max() - fitnesses.min() + 1e-8)

evo2 = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    fitness_to_weighted_factor=custom_weighting
)
```

--------------------------------

### noise_scale_clamp_range Example

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/configuration.md

Example of setting the min and max bounds for learned noise scales (sigma) after each update.

```python
evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    learned_noise_scale=True,
    noise_scale_clamp_range=(1e-4, 5e-2)  # Tighter bounds
)
```

--------------------------------

### Single Scope (All Parameters)

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/algorithm-overview.md

Example of initializing EvoStrategy where all parameters in the model are optimized uniformly.

```python
evo = EvoStrategy(model, environment=fitness_fn, num_generations=100)
# All parameters in model are optimized
```

--------------------------------

### Common Module Examples - Sequential

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/types.md

Example of a PyTorch Sequential model.

```python
from torch import nn

 # Sequential model
 model = nn.Sequential(
 nn.Linear(8, 16),
 nn.ReLU(),
 nn.Linear(16, 4)
 )
```

--------------------------------

### EvoStrategy Initialization with noise_population_size

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/configuration.md

Example of initializing EvoStrategy with a larger noise population size.

```python
evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    noise_population_size=50  # Larger population
)
```

--------------------------------

### Common Module Examples - Custom

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/types.md

Example of a custom PyTorch Module.

```python
from torch import nn

 # Custom module
 class CustomModel(nn.Module):
 def __init__(self):
 super().__init__()
 self.encoder = nn.Linear(8, 16)
 self.decoder = nn.Linear(16, 4)
 
 def forward(self, x):
 return self.decoder(self.encoder(x))

 model = CustomModel()
```

--------------------------------

### Sigma Optimizer Klass Example

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/configuration.md

Example of configuring a custom optimizer class for learned noise scale using `functools.partial` with `Adam`.

```python
from functools import partial
from torch.optim import Adam

evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    learned_noise_scale=True,
    sigma_optimizer_klass=partial(Adam, betas=(0.9, 0.999))
)
```

--------------------------------

### Simple scalar return example

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/api-reference/index.md

Example of a simple scalar return from the environment function.

```python
# Simple scalar return
def scalar_fitness(model):
    return 0.5
```

--------------------------------

### Enabling Verbose Output

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/configuration.md

Example of setting `verbose=True` to print progress bars and statistics during training.

```python
evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    verbose=True  # Print progress
)
```

--------------------------------

### Basic ES Training

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/api-reference/evostrategy.md

A fundamental example of setting up and running Evolutionary Strategy training.

```python
import torch
from torch import nn
from x_evolution import EvoStrategy

model = nn.Sequential(
    nn.Linear(4, 8),
    nn.ReLU(),
    nn.Linear(8, 1)
)

def simple_fitness(model):
    # Dummy fitness: random value
    return torch.randn(1).item()

evo = EvoStrategy(
    model,
    environment=simple_fitness,
    num_generations=50,
    noise_population_size=30,
    learning_rate=0.001
)

evo()  # Runs training
```

--------------------------------

### Gymnasium VectorEnv Integration

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/usage-guide.md

Shows how to use x-evolution with vectorized Gymnasium environments for evaluation.

```python
import gymnasium as gym

def vectorized_gym_fitness(model):
    """Evaluate on vectorized Gymnasium environments."""
    env = gym.make_vec('LunarLander-v3', num_envs=8)
    state, _ = env.reset()
    total_rewards = torch.zeros(8)
    
    for _ in range(500):  # 500 steps max
        state_t = torch.tensor(state, dtype=torch.float32)
        actions = model(state_t).argmax(dim=1)
        state, rewards, terminated, truncated, _ = env.step(actions)
        total_rewards += torch.tensor(rewards)
    
    return total_rewards
 evo = EvoStrategy(
    model,
    environment=vectorized_gym_fitness,
    num_generations=100,
    vectorized=True,
    vector_size=8
)
 evo()
```

--------------------------------

### Distributed Training with Accelerate

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/api-reference/evostrategy.md

Example for setting up and running distributed ES training using the Accelerate library.

```python
from accelerate import Accelerator

evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    cpu=False  # Use available GPUs
)

# Run with: torchrun --nproc_per_node=4 script.py
evo()
```

--------------------------------

### evolve_() example

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/api-reference/evostrategy.md

Note that this is an internal method typically called by forward().

```python
# This is an internal method typically called by forward().
```

--------------------------------

### Rank-Based Fitness Weighting

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/usage-guide.md

Configuration for using centered rank-based fitness weighting.

```python
evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    fitness_to_weighted_factor='centered_rank'  # Rank-based weighting
)
```

--------------------------------

### Force CPU Computation

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/configuration.md

Example of setting `cpu=True` to force all computations to the CPU.

```python
evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    cpu=True  # Force CPU
)
```

--------------------------------

### Scoped Parameter Optimization

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/api-reference/evostrategy.md

Example showing how to optimize specific subsets of model parameters using scopes.

```python
evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=50,
    params_to_optimize={
        'encoder': model.layers[:-1],
        'head': model.layers[-1]
    }
)

# Optimize only encoder for 30 generations
evo(scope='encoder', num_generations=30)

# Then optimize head
evo(scope='head', num_generations=30)

# Or optimize all
evo(num_generations=30)
```

--------------------------------

### Custom Checkpoint Directory

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/configuration.md

Example of setting `checkpoint_path` to a custom directory for saving checkpoints.

```python
evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    checkpoint_path='./my_checkpoints'
)
```

--------------------------------

### Debugging: Verify Seeds Are Synchronized

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/algorithm-overview.md

Guidance for debugging synchronization issues in a distributed setup.

```python
# In distributed setup, check all processes see same fitnesses
# If different: synchronization issue
```

--------------------------------

### Default SGD with Momentum

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/usage-guide.md

Configuration for the default SGD optimizer with momentum and Nesterov acceleration.

```python
evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    use_optimizer=True  # Default
    # Default: SGD with momentum=0.1, nesterov=True, weight_decay=1e-2
)
```

--------------------------------

### ParametersToOptimize Usage - Multiple Submodules

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/types.md

Example of optimizing parameters across multiple submodules.

```python
from x_evolution import EvoStrategy

 model = nn.Sequential(
 nn.Linear(8, 16),
 nn.ReLU(),
 nn.Linear(16, 4)
 )

 # Option 3: Multiple submodules
 evo3 = EvoStrategy(
 model,
 environment=fitness_fn,
 num_generations=100,
 params_to_optimize=[model[0], model[2]]  # First and last layers
 )
```

--------------------------------

### Reinforcement Learning Fitness

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/usage-guide.md

Example of a fitness function for a reinforcement learning task, evaluating policy in an environment.

```python
import gymnasium as gym

class RLEnvironment(torch.nn.Module):
    def __init__(self, env_name='CartPole-v1'):
        super().__init__()
        self.env = gym.make(env_name)
    
    def forward(self, model):
        """Evaluate policy in the environment."""
        state, _ = self.env.reset()
        total_reward = 0
        done = False
        
        while not done:
            state_t = torch.tensor(state, dtype=torch.float32)
            action_logits = model(state_t)
            action = action_logits.argmax().item()
            
            state, reward, terminated, truncated, _ = self.env.step(action)
            total_reward += reward
            done = terminated or truncated
        
        return total_reward

rl_env = RLEnvironment('CartPole-v1')evo = EvoStrategy(
    model,
    environment=rl_env,
    num_generations=100,
    noise_population_size=30
)evo()
```

--------------------------------

### Supervised Learning Fitness

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/usage-guide.md

Example of a fitness function for a supervised learning task, returning negative loss.

```python
def supervised_fitness(model):
    """Fitness from classification task."""
    x, y = load_batch()
    logits = model(x)
    loss = torch.nn.functional.cross_entropy(logits, y)
    return -loss.item()  # Higher fitness = lower loss
```

--------------------------------

### Objective Function Optimization Fitness

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/usage-guide.md

Example of a fitness function for optimizing an objective function (Rastrigin) using an MLP.

```python
def rastrigin_fitness(model):
    """Minimize Rastrigin function using a small MLP."""
    # Model predicts the optimum point
    x = model(torch.zeros(1, 2))
    
    # Rastrigin function: f(x) = 20 + sum(x^2 - 10*cos(2*pi*x))
    term1 = (x ** 2).sum()
    term2 = 10 * torch.cos(2 * torch.pi * x).sum()
    rastrigin = 20 + term1 - term2
    
    return -rastrigin.item()  # Negative because we minimize
evo = EvoStrategy(
    model,
    environment=rastrigin_fitness,
    num_generations=100,
    noise_population_size=50
)evo()
```

--------------------------------

### Rank-k Noise Decomposition

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/usage-guide.md

Example of using rank-k noise decomposition for very large models, allowing for larger populations with reduced noise.

```python
evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    noise_low_rank=2,  # Rank-2 decomposition
    noise_population_size=100  # Can afford larger populations with rank-k
)
```

--------------------------------

### Multi-GPU Training with Accelerate

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/usage-guide.md

Commands and Python code for setting up and running distributed training using the Accelerate library.

```bash
# Configure accelerate
$ accelerate config

# Run with multiple GPUs
$ accelerate launch train_script.py
```

```python
from accelerate import Accelerator

accelerator = Accelerator()

evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    accelerator=accelerator
)

evo()
```

--------------------------------

### Troubleshooting: Training Not Converging

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/usage-guide.md

Recommended settings to address training non-convergence by increasing population size and noise scale.

```python
# Increase population size and noise scale evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    noise_population_size=50,  # Increased from 30
    noise_scale=0.05,  # Increased from 0.01
    learning_rate=0.01  # Increased from 0.001
)
```

--------------------------------

### Use Distributed Training

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/README.md

These commands show how to set up and launch distributed training for the x-evolution library using Hugging Face Accelerate and torchrun.

```bash
# Configure and launch
accelerate config
accelerate launch train.py
```

```bash
torchrun --nproc_per_node=4 train.py
```

--------------------------------

### Vectorized return example

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/api-reference/index.md

Example of a vectorized return (multiple environments) from the environment function.

```python
# Vectorized return (multiple envs)
def vectorized_fitness(model):
    return torch.randn(8)  # 8 parallel environments
```

--------------------------------

### Passing Accelerate Keyword Arguments

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/configuration.md

Example of passing additional keyword arguments to the `Accelerator` constructor via `accelerate_kwargs`.

```python
evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    accelerate_kwargs={'mixed_precision': 'fp16', 'gradient_accumulation_steps': 2}
)
```

--------------------------------

### Distributed Training with Accelerate

Source: https://github.com/lucidrains/x-evolution/blob/main/README.md

Instructions for setting up and launching distributed training using Hugging Face's Accelerate library.

```shell
$ uv run accelerate config
```

```shell
$ uv run accelerate launch train.py
```

--------------------------------

### Fitness Function Example

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/README.md

Example of a fitness function that takes a model and returns a scalar fitness score.

```python
def my_fitness(model):
    # Evaluate model somehow
    return score  # float or int
```

--------------------------------

### Distributed Training with Torchrun

Source: https://github.com/lucidrains/x-evolution/blob/main/README.md

Instructions for launching distributed training using torchrun, including an example for testing with multiple CPU processes and a specific demo for LunarLander.

```shell
$ uv run torchrun --nproc_per_node=<N> train.py --cpu True
```

```shell
$ uv run torchrun --nproc_per_node=8 train_lunar.py --cpu True --noise_population_size 100 --fitness_to_weighted_factor centered_rank
```

--------------------------------

### FitnessWeightFactorType Usage

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/types.md

Example of using `fitness_to_weighted_factor` with 'centered_rank'.

```python
from x_evolution import EvoStrategy

 evo = EvoStrategy(
 model,
 environment=fitness_fn,
 num_generations=100,
 fitness_to_weighted_factor='centered_rank'  # Use rank-based weighting
 )
```

--------------------------------

### Multiple Scopes

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/algorithm-overview.md

Example of initializing EvoStrategy with specific parameter scopes for optimization, allowing sequential optimization of different model parts.

```python
evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    params_to_optimize={
        'encoder': model.encoder,
        'decoder': model.decoder
    }
)
```

--------------------------------

### Periodic Checkpointing

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/usage-guide.md

Configuration for saving model checkpoints periodically during training.

```python
evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=1000,
    checkpoint_every=100,  # Save every 100 generations
    checkpoint_path='./checkpoints'
)

evo()  # Saves to ./checkpoints/evolved.model.100.pt, .200.pt, etc.
```

--------------------------------

### Adam Optimizer

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/usage-guide.md

Configuration for the Adam optimizer with custom beta values.

```python
from functools import partial
from torch.optim import Adam

evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    optimizer_klass=partial(Adam, betas=(0.9, 0.999)),
    optimizer_kwargs={'eps': 1e-8}
)
```

--------------------------------

### Handle Vectorized Environments

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/README.md

This example shows how to configure EvoStrategy to handle vectorized environments, allowing for parallel evaluation across multiple environments by setting `vectorized=True` and specifying `vector_size`.

```python
def vectorized_fitness(model):
    # Return multiple fitness scores
    return torch.randn(8)  # 8 parallel environments

evo = EvoStrategy(
    model,
    environment=vectorized_fitness,
    num_generations=100,
    vectorized=True,
    vector_size=8
)

evo()
```

--------------------------------

### Low-Rank Noise Approximation

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/usage-guide.md

Placeholder for Low-Rank Noise Approximation configuration.

```python

```

--------------------------------

### Optimize Specific Parameters

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/README.md

This example illustrates how to use the EvoStrategy to optimize specific subsets of a model's parameters by defining a `params_to_optimize` dictionary and then calling the strategy with a specified scope.

```python
evo = EvoStrategy(
    model,
    environment=fitness_fn,
    num_generations=100,
    params_to_optimize={
        'encoder': model.encoder,
        'head': model.head
    }
)

# Optimize each part separately
evo(scope='encoder', num_generations=50)
evo(scope='head', num_generations=50)
```

--------------------------------

### Optimizer Usage

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/types.md

Example of using a custom optimizer class with `EvoStrategy`.

```python
from functools import partial
from torch.optim import Adam
from x_evolution import EvoStrategy

 evo = EvoStrategy(
 model,
 environment=fitness_fn,
 num_generations=100,
 optimizer_klass=partial(Adam, betas=(0.9, 0.999)),
 optimizer_kwargs={'weight_decay': 1e-4}
 )
```

--------------------------------

### ParametersToOptimize Usage - Submodule

Source: https://github.com/lucidrains/x-evolution/blob/main/_autodocs/types.md

Example of optimizing all parameters of a single submodule.

```python
from x_evolution import EvoStrategy

 model = nn.Sequential(
 nn.Linear(8, 16),
 nn.ReLU(),
 nn.Linear(16, 4)
 )

 # Option 2: Submodule
 evo2 = EvoStrategy(
 model,
 environment=fitness_fn,
 num_generations=100,
 params_to_optimize=model[0]  # Only first layer
 )
```