### Install Adam Atan2 PyTorch

Source: https://context7.com/lucidrains/adam-atan2-pytorch/llms.txt

Install the package via pip.

```bash
pip install adam-atan2-pytorch
```

--------------------------------

### Install Adam-atan2 via pip

Source: https://github.com/lucidrains/adam-atan2-pytorch/blob/main/README.md

Use this command to install the package in your Python environment.

```bash
$ pip install adam-atan2-pytorch
```

--------------------------------

### Initialize AdoptAtan2 Optimizer

Source: https://context7.com/lucidrains/adam-atan2-pytorch/llms.txt

Import the AdoptAtan2 optimizer for scenarios requiring convergence with arbitrary beta2 values.

```python
import torch
from torch import nn
from adam_atan2_pytorch import AdoptAtan2
```

--------------------------------

### Initialize Foreach AdamAtan2 Optimizer

Source: https://context7.com/lucidrains/adam-atan2-pytorch/llms.txt

Performance-optimized version using PyTorch's foreach operations for faster GPU execution.

```python
import torch
from torch import nn
from adam_atan2_pytorch.foreach import AdamAtan2 as AdamAtan2Foreach

# Create a large model where foreach optimization matters
model = nn.Sequential(
    nn.Linear(1024, 2048),
    nn.ReLU(),
    nn.Linear(2048, 2048),
    nn.ReLU(),
    nn.Linear(2048, 1024)
).cuda()

# Initialize foreach-optimized AdamAtan2
optimizer = AdamAtan2Foreach(
    model.parameters(),
    lr=1e-4,
    betas=(0.9, 0.99),
    weight_decay=0.01,
    regen_reg_rate=0.0,
    decoupled_wd=False,
    a=1.27,
    b=1.0,
    foreach_atan2_fn=None     # Uses torch._foreach_atan2_ if available, else slow fallback
)

# Training loop with GPU tensors
for _ in range(100):
    x = torch.randn(64, 1024, device='cuda')
    y = torch.randn(64, 1024, device='cuda')

    loss = nn.functional.mse_loss(model(x), y)
    loss.backward()

    optimizer.step()
    optimizer.zero_grad()
```

--------------------------------

### Initialize AdamAtan2 Optimizer

Source: https://context7.com/lucidrains/adam-atan2-pytorch/llms.txt

Configure the AdamAtan2 optimizer with custom parameters for learning rate, momentum, and weight decay.

```python
import torch
from torch import nn
from adam_atan2_pytorch import AdamAtan2

# Create a neural network model
model = nn.Sequential(
    nn.Linear(784, 256),
    nn.ReLU(),
    nn.Linear(256, 128),
    nn.ReLU(),
    nn.Linear(128, 10)
)

# Initialize AdamAtan2 optimizer with custom parameters
optimizer = AdamAtan2(
    model.parameters(),
    lr=1e-4,                  # Learning rate
    betas=(0.9, 0.99),        # Momentum coefficients (beta1, beta2)
    weight_decay=0.01,        # L2 regularization (cannot be used with regen_reg_rate)
    decoupled_wd=False,       # Use decoupled weight decay
    cautious_factor=1.0,      # Set to 0.0 to zero out updates not aligned with gradient
    a=1.27,                   # Scaling factor for atan2 output
    b=1.0                     # Denominator scaling factor
)

# Training loop
for epoch in range(10):
    for batch_x, batch_y in dataloader:
        # Forward pass
        predictions = model(batch_x)
        loss = nn.functional.cross_entropy(predictions, batch_y)

        # Backward pass
        loss.backward()

        # Optimizer step
        optimizer.step()
        optimizer.zero_grad()

print(f"Final loss: {loss.item():.4f}")
```

--------------------------------

### Initialize Standard AdoptAtan2 Optimizer

Source: https://context7.com/lucidrains/adam-atan2-pytorch/llms.txt

Standard implementation of the AdoptAtan2 optimizer for general neural network training.

```python
# Create model
model = nn.Sequential(
    nn.Linear(784, 512),
    nn.GELU(),
    nn.Linear(512, 10)
)

# Initialize AdoptAtan2 optimizer
optimizer = AdoptAtan2(
    model.parameters(),
    lr=1e-4,
    betas=(0.9, 0.99),        # Works with any beta2 value
    weight_decay=0.01,
    decoupled_wd=True,        # Decoupled weight decay enabled by default
    regen_reg_rate=0.0,       # Regenerative regularization rate
    cautious_factor=1.0,      # Cautious update factor
    a=1.27,                   # atan2 scaling factor
    b=1.0                     # Denominator scaling
)

# Training loop
for epoch in range(10):
    for x, y in dataloader:
        predictions = model(x)
        loss = nn.functional.cross_entropy(predictions, y)

        loss.backward()
        optimizer.step()
        optimizer.zero_grad()
```

--------------------------------

### Initialize and use Adam-atan2

Source: https://github.com/lucidrains/adam-atan2-pytorch/blob/main/README.md

Import the optimizer and apply it to a PyTorch model. Ensure the optimizer step and zero_grad methods are called during the training loop.

```python
import torch
from torch import nn

# toy model

model = nn.Linear(10, 1)

# import AdamAtan2 and instantiate with parameters

from adam_atan2_pytorch import AdamAtan2

opt = AdamAtan2(model.parameters(), lr = 1e-4)

# forward and backwards

for _ in range(100):
  loss = model(torch.randn(10))
  loss.backward()

  # optimizer step

  opt.step()
  opt.zero_grad()
```

--------------------------------

### Implement Cautious Updates

Source: https://context7.com/lucidrains/adam-atan2-pytorch/llms.txt

Configure cautious updates to zero out or reduce updates that oppose the gradient direction.

```python
import torch
from torch import nn
from adam_atan2_pytorch import AdamAtan2

model = nn.Linear(512, 256)

# Enable cautious updates by setting cautious_factor < 1.0
optimizer = AdamAtan2(
    model.parameters(),
    lr=1e-4,
    betas=(0.9, 0.99),
    cautious_factor=0.0       # Zero out updates not aligned with gradient
)

# Training with cautious optimization
for _ in range(100):
    x = torch.randn(32, 512)
    y = torch.randn(32, 256)

    loss = nn.functional.mse_loss(model(x), y)
    loss.backward()

    # Updates opposing gradient direction will be zeroed
    optimizer.step()
    optimizer.zero_grad()
```

--------------------------------

### Implement Closure for Loss Computation

Source: https://context7.com/lucidrains/adam-atan2-pytorch/llms.txt

Demonstrates the closure pattern for loss computation, which is required for algorithms needing multiple forward passes or gradient accumulation.

```python
import torch
from torch import nn
from adam_atan2_pytorch import AdamAtan2

model = nn.Linear(100, 10)
optimizer = AdamAtan2(model.parameters(), lr=1e-4)

x = torch.randn(32, 100)
y = torch.randn(32, 10)

# Define closure for loss computation
def closure():
    optimizer.zero_grad()
    output = model(x)
    loss = nn.functional.mse_loss(output, y)
    loss.backward()
    return loss

# Step with closure
loss = optimizer.step(closure=closure)
print(f"Loss: {loss.item():.4f}")
```

--------------------------------

### Initialize AdamAtan2 with Wasserstein Regularization

Source: https://context7.com/lucidrains/adam-atan2-pytorch/llms.txt

Configures the optimizer with Wasserstein-based regenerative regularization enabled. Note that weight decay and decoupled weight decay must be disabled when using this feature.

```python
optimizer = AdamAtan2(
    model.parameters(),
    lr=1e-4,
    betas=(0.9, 0.99),
    regen_reg_rate=0.01,      # Enables Wasserstein regularization
    weight_decay=0.0,         # Cannot use both
    decoupled_wd=False,
    a=1.27,
    b=1.0
)

# Training with Wasserstein regularization active
for data, target in dataloader:
    loss = nn.functional.mse_loss(model(data), target)
    loss.backward()
    optimizer.step()
    optimizer.zero_grad()
```

--------------------------------

### Initialize MuonAdamAtan2 Hybrid Optimizer

Source: https://context7.com/lucidrains/adam-atan2-pytorch/llms.txt

Hybrid optimizer combining Muon for hidden layers and Adam-Atan2 for other parameters, useful for deep architectures.

```python
import torch
from torch import nn
from adam_atan2_pytorch.muon_adam_atan2 import MuonAdamAtan2

# Create a model with distinct parameter groups
class DeepModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.embedding = nn.Linear(784, 512)
        self.hidden1 = nn.Linear(512, 512)    # Use Muon for hidden layers
        self.hidden2 = nn.Linear(512, 512)    # Use Muon for hidden layers
        self.output = nn.Linear(512, 10)

    def forward(self, x):
        x = self.embedding(x).relu()
        x = self.hidden1(x).relu()
        x = self.hidden2(x).relu()
        return self.output(x)

model = DeepModel()

# Separate parameters for Muon (hidden layers) and Adam-atan2 (others)
muon_params = list(model.hidden1.parameters()) + list(model.hidden2.parameters())
all_params = list(model.parameters())

# Initialize MuonAdamAtan2 optimizer
optimizer = MuonAdamAtan2(
    muon_params=muon_params,                    # Parameters to optimize with Muon
    params=all_params,                          # All parameters (Muon params auto-removed)
    lr=1e-4,                                    # Learning rate for Adam-atan2 params
    muon_lr=0.02,                               # Learning rate for Muon params (optional)
    betas=(0.9, 0.99),                          # Adam betas
    weight_decay=0.01,
    muon_beta1=0.95,                            # Muon momentum coefficient
    muon_steps=5,                               # Newton-Schulz iteration steps
    muon_rms_factor=0.2,                        # RMS scaling factor for Muon
    muon_newton_schulz5_coefs=(3.4445, -4.7750, 2.0315),  # NS5 coefficients
    cautious_factor=1.0,
    remove_muon_params_from_params=True         # Auto-remove Muon params from Adam group
)

# Training loop
for epoch in range(10):
    for x, y in dataloader:
        loss = nn.functional.cross_entropy(model(x), y)
        loss.backward()
        optimizer.step()
        optimizer.zero_grad()
```

--------------------------------

### Initialize AdamAtan2 with Wasserstein Regularization

Source: https://context7.com/lucidrains/adam-atan2-pytorch/llms.txt

Variant for continual learning that uses Wasserstein distance-based regularization.

```python
import torch
from torch import nn
from adam_atan2_pytorch.adam_atan2_with_wasserstein_reg import AdamAtan2

# Model for continual learning with Wasserstein regularization
model = nn.Linear(256, 128)
```

--------------------------------

### Enable Regenerative Regularization

Source: https://context7.com/lucidrains/adam-atan2-pytorch/llms.txt

Use regenerative regularization to maintain model plasticity during continual learning tasks.

```python
import torch
from torch import nn
from adam_atan2_pytorch import AdamAtan2

# Model for continual learning
model = nn.Linear(100, 10)

# Enable regenerative regularization for plasticity
optimizer = AdamAtan2(
    model.parameters(),
    lr=1e-4,
    betas=(0.9, 0.99),
    regen_reg_rate=0.01,      # Regenerative regularization rate
    weight_decay=0.0          # Cannot use both weight_decay and regen_reg_rate
)

# Task 1 training
for data, target in task1_dataloader:
    loss = nn.functional.mse_loss(model(data), target)
    loss.backward()
    optimizer.step()
    optimizer.zero_grad()

# Task 2 training - model maintains plasticity
for data, target in task2_dataloader:
    loss = nn.functional.mse_loss(model(data), target)
    loss.backward()
    optimizer.step()
    optimizer.zero_grad()
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.