### Install hl-gauss-pytorch

Source: https://github.com/lucidrains/hl-gauss-pytorch/blob/main/README.md

Install the hl-gauss-pytorch library using pip.

```bash
pip install hl-gauss-pytorch
```

--------------------------------

### Initialize and Use HLGaussLoss

Source: https://context7.com/lucidrains/hl-gauss-pytorch/llms.txt

Initialize the loss function with value range and bins. Compute loss during training or predict values during inference.

```python
import torch
from hl_gauss_pytorch import HLGaussLoss

# Initialize the loss function with value range and bins
hl_gauss = HLGaussLoss(
    min_value=0.,
    max_value=5.,
    num_bins=32,
    sigma=0.5,  # Gaussian smoothing width (default: 2x bin size)
    clamp_to_range=True  # Clamp targets to [min_value, max_value]
)

# Training: compute loss from logits and targets
logits = torch.randn(3, 16, 32).requires_grad_()  # (batch, seq, num_bins)
targets = torch.randint(0, 5, (3, 16)).float()     # (batch, seq)

loss = hl_gauss(logits, targets)
loss.backward()
print(f"Loss: {loss.item():.4f}")

# Inference: predict values from logits (no targets)
with torch.no_grad():
    pred_values = hl_gauss(logits)  # Returns (3, 16) tensor
    print(f"Predicted values shape: {pred_values.shape}")
    print(f"Predicted values: {pred_values[0, :5]}")
```

--------------------------------

### HLGaussLossFromRunningStats Initialization and Usage

Source: https://context7.com/lucidrains/hl-gauss-pytorch/llms.txt

Initialize HLGaussLossFromRunningStats to automatically determine bin support from running mean and variance statistics. This is beneficial when the target distribution is unknown or dynamic during training.

```python
import torch
from hl_gauss_pytorch import HLGaussLayer
from hl_gauss_pytorch.hl_gauss import HLGaussLossFromRunningStats

# Create loss with running statistics
running_stats_loss = HLGaussLossFromRunningStats(
    num_bins=200,
    sigma_to_bin_ratio=2.,
    std_dev_range=3,  # Support covers ±3 standard deviations
    clamp_to_range=True
)

# Use in a layer
hl_gauss_layer = HLGaussLayer(
    dim=64,
    norm_embed=True,
    hl_gauss_loss_running_stats=running_stats_loss
)

# During training, statistics are updated automatically
for step in range(100):
    embed = torch.randn(32, 64)
    targets = torch.randn(32) * 10 + 50  # Mean ~50, std ~10

    loss = hl_gauss_layer(embed, targets)
    loss.backward()

# Statistics adapt to the target distribution
print(f"Running mean: {running_stats_loss.running_stats.running_mean.item():.2f}")
print(f"Running std: {running_stats_loss.running_stats.running_std_dev.item():.2f}")
```

--------------------------------

### Initialize and use HLGaussLoss

Source: https://github.com/lucidrains/hl-gauss-pytorch/blob/main/README.md

Initialize the HLGaussLoss module with specified parameters and use it to compute loss between logits and targets. The loss can be backpropagated. It can also be used to predict targets from logits.

```python
import torch
from hl_gauss_pytorch import HLGaussLoss

hl_gauss = HLGaussLoss(
    min_value = 0.,
    max_value = 5.,
    num_bins = 32,
    sigma = 0.5,
    clamp_to_range = True # this was added because if any values fall outside of the bins, the loss is 0 with the current logic
)

logits = torch.randn(3, 16, 32).requires_grad_()
targets = torch.randint(0, 5, (3, 16)).float()

loss = hl_gauss(logits, targets)
loss.backward()

# after much training

pred_target = hl_gauss(logits) # (3, 16)
```

--------------------------------

### HLGaussLossFromSupport with Custom Bin Edges

Source: https://context7.com/lucidrains/hl-gauss-pytorch/llms.txt

Create HLGaussLossFromSupport with custom, non-uniform bin edges defined by explicit support values. This allows for adaptive binning strategies, providing finer resolution in specific value ranges.

```python
import torch
from hl_gauss_pytorch.hl_gauss import HLGaussLossFromSupport

# Create custom support with finer bins near zero
support = torch.tensor([
    -10., -5., -2., -1., -0.5, -0.2, -0.1,
    0.,
    0.1, 0.2, 0.5, 1., 2., 5., 10.
])

hl_gauss = HLGaussLossFromSupport(
    support=support,
    sigma=0.1,
    clamp_to_range=True
)

# Use like standard HLGaussLoss
num_bins = len(support) - 1  # 14 bins
logits = torch.randn(8, num_bins).requires_grad_()
targets = torch.randn(8) * 5

loss = hl_gauss(logits, targets)
loss.backward()

# Get probability distribution over bins
probs = hl_gauss.transform_to_probs(targets)
print(f"Target probabilities shape: {probs.shape}")  # (8, 14)
```

--------------------------------

### Initialize and Use HLGaussLayer

Source: https://context7.com/lucidrains/hl-gauss-pytorch/llms.txt

Create a neural network layer that projects embeddings to bin logits and handles loss computation or value prediction. Supports returning raw logits or both predicted values and logits.

```python
import torch
from hl_gauss_pytorch import HLGaussLayer

# Create layer with HL-Gauss classification
hl_gauss_layer = HLGaussLayer(
    dim=256,  # Input embedding dimension
    norm_embed=True,  # Apply RMSNorm before projection
    hl_gauss_loss=dict(
        min_value=0.,
        max_value=5.,
        num_bins=32,
        sigma=0.5,
    )
)

# Training
embed = torch.randn(7, 256).requires_grad_()
targets = torch.randint(0, 5, (7,)).float()

loss = hl_gauss_layer(embed, targets)
loss.backward()
print(f"Training loss: {loss.item():.4f}")

# Inference: predict values
with torch.no_grad():
    pred_values = hl_gauss_layer(embed)  # Returns (7,) tensor
    print(f"Predictions: {pred_values}")

# Get raw logits for custom processing
logits = hl_gauss_layer(embed, return_logits=True)  # (7, 32)
print(f"Logits shape: {logits.shape}")

# Get both predicted values and logits
pred_values, logits = hl_gauss_layer(embed, return_value_and_logits=True)
```

--------------------------------

### Train HL-Gauss Regression Model

Source: https://context7.com/lucidrains/hl-gauss-pytorch/llms.txt

Demonstrates a full training loop for a regression task using HLGaussLayer to approximate a sine function. Requires the hl_gauss_pytorch package.

```python
import torch
from torch import nn
from torch.optim import Adam
import torch.nn.functional as F
from hl_gauss_pytorch import HLGaussLayer

# Configuration
NUM_BATCHES = 5000
BATCH_SIZE = 32
LEARNING_RATE = 1e-4
EVAL_EVERY = 500
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

# Define model
class RegressionModel(nn.Module):
    def __init__(self, dim_hidden=64):
        super().__init__()
        self.encoder = nn.Sequential(
            nn.Linear(1, dim_hidden),
            nn.SiLU(),
            nn.Linear(dim_hidden, dim_hidden),
            nn.SiLU()
        )
        self.hl_gauss = HLGaussLayer(
            dim=dim_hidden,
            norm_embed=True,
            hl_gauss_loss=dict(
                min_value=-1.1,
                max_value=1.1,
                num_bins=200,
                sigma_to_bin_ratio=2.
            )
        )

    def forward(self, x, target=None):
        embed = self.encoder(x.unsqueeze(-1))
        return self.hl_gauss(embed, target)

# Initialize
model = RegressionModel().to(device)
optimizer = Adam(model.parameters(), lr=LEARNING_RATE)

# Data generator (learn sin function)
def get_batch(batch_size):
    x = torch.randn(batch_size, device=device)
    y = torch.sin(x)
    return x, y

# Training loop
for step in range(NUM_BATCHES):
    x, y = get_batch(BATCH_SIZE)

    loss = model(x, y)
    loss.backward()
    optimizer.step()
    optimizer.zero_grad()

    if (step + 1) % EVAL_EVERY == 0:
        with torch.no_grad():
            x_eval, y_eval = get_batch(BATCH_SIZE)
            pred = model(x_eval)
            mae = F.l1_loss(pred, y_eval)
            print(f"Step {step+1}: MAE = {mae.item():.6f}")

# Final evaluation
with torch.no_grad():
    x_test = torch.linspace(-3, 3, 100, device=device)
    y_pred = model(x_test)
    y_true = torch.sin(x_test)
    final_mae = F.l1_loss(y_pred, y_true)
    print(f"Final test MAE: {final_mae.item():.6f}")
```

--------------------------------

### HLGaussLoss with Custom Transforms (Symlog)

Source: https://context7.com/lucidrains/hl-gauss-pytorch/llms.txt

Apply custom transformations like symlog to targets before binning, useful for handling targets with a large dynamic range. The inverse transform is automatically applied during prediction.

```python
import torch
from hl_gauss_pytorch import HLGaussLoss, HLGaussLayer

# Define symlog transform (handles large positive/negative values)
def symlog(x):
    return x.sign() * (x.abs() + 1.0).log()

def symexp(x):
    return x.sign() * (x.abs().exp() - 1.0)

# Loss with transforms
hl_gauss = HLGaussLoss(
    min_value=-100.,
    max_value=100.,
    num_bins=256,
    transform=symlog,
    inverse_transform=symexp
)

# Values with large dynamic range
targets = torch.tensor([-1000., -10., 0., 10., 1000.])
logits = torch.randn(5, 256)

# Transforms are applied automatically
loss = hl_gauss(logits, targets)
pred_values = hl_gauss(logits)  # Predictions are in original space

# Use in layer
hl_gauss_layer = HLGaussLayer(
    dim=256,
    hl_gauss_loss=dict(
        min_value=-100.,
        max_value=100.,
        num_bins=256,
        transform=symlog,
        inverse_transform=symexp
    )
)

embed = torch.randn(5, 256)
loss = hl_gauss_layer(embed, targets)
```

--------------------------------

### Initialize and use HLGaussLayer

Source: https://github.com/lucidrains/hl-gauss-pytorch/blob/main/README.md

Initialize HLGaussLayer for predicting targets from embeddings. It takes an embedding dimension and HLGaussLoss configuration. The layer can compute loss and be backpropagated, or predict targets from embeddings after training.

```python
import torch
from hl_gauss_pytorch import HLGaussLayer

hl_gauss_layer = HLGaussLayer(
    dim = 256, # input embedding dimension
    hl_gauss_loss = dict(
        min_value = 0.,
        max_value = 5.,
        num_bins = 32,
        sigma = 0.5,
    )
)

embed = torch.randn(7, 256)
targets = torch.randint(0, 5, (7,)).float()

loss = hl_gauss_layer(embed, targets)
loss.backward()

# after much training

pred_target = hl_gauss_layer(embed) # (7,)
```

--------------------------------

### HLGaussLayer with Regression Fallback

Source: https://context7.com/lucidrains/hl-gauss-pytorch/llms.txt

Use HLGaussLayer with `use_regression=True` to fall back to standard MSE regression. This is useful for comparison or when a direct regression is preferred over classification-based binning.

```python
import torch
from hl_gauss_pytorch import HLGaussLayer

# Regression-based fallback for comparison
hl_gauss_regression = HLGaussLayer(
    dim=256,
    use_regression=True,  # Fall back to standard MSE regression
    hl_gauss_loss=dict(min_value=0., max_value=10., num_bins=64)
)

# Both have identical API
embed = torch.randn(16, 256)
targets = torch.rand(16) * 10

loss_class = hl_gauss_regression(embed, targets) # This line seems to be a typo, should be hl_gauss_classification if it exists
loss_regress = hl_gauss_regression(embed, targets)

print(f"Classification loss: {loss_class.item():.4f}")
print(f"Regression loss: {loss_regress.item():.4f}")
```

--------------------------------

### Configure HLGaussLayer for regression

Source: https://github.com/lucidrains/hl-gauss-pytorch/blob/main/README.md

Configure HLGaussLayer to fall back to regular regression by setting use_regression to True. This allows for ablation studies while keeping the rest of the code unchanged.

```python
HLGaussLayer(..., use_regression = True)
```

--------------------------------

### HLGaussLayer with Regression Fallback

Source: https://context7.com/lucidrains/hl-gauss-pytorch/llms.txt

Enable standard MSE regression instead of classification for ablation studies by setting `use_regression=True`. The API interface remains the same.

```python
import torch
from hl_gauss_pytorch import HLGaussLayer

# Classification-based (HL-Gauss)
hl_gauss_classification = HLGaussLayer(
    dim=256,
    use_regression=False,  # Default: use HL-Gauss classification
    hl_gauss_loss=dict(min_value=0., max_value=10., num_bins=64)
)

# Regression fallback
hl_gauss_regression = HLGaussLayer(
    dim=256,
    use_regression=True,  # Use standard MSE regression
    hl_gauss_loss=dict(min_value=0., max_value=10., num_bins=64) # Parameters still needed for API compatibility
)
```

--------------------------------

### HLGaussLayer with Custom Network

Source: https://context7.com/lucidrains/hl-gauss-pytorch/llms.txt

Create a layer with a custom network that processes embeddings before the final projection. Ensure the custom network's output dimension matches the layer's `dim` parameter.

```python
import torch
from torch import nn
from hl_gauss_pytorch import HLGaussLayer

# Define a custom MLP network
class ValueNetwork(nn.Module):
    def __init__(self, dim_in, dim_out):
        super().__init__()
        self.net = nn.Sequential(
            nn.Linear(dim_in, dim_out),
            nn.SiLU(),
            nn.Linear(dim_out, dim_out),
            nn.SiLU()
        )

    def forward(self, x):
        return self.net(x)

# Create layer with custom network
hl_gauss_layer = HLGaussLayer(
    dim=128,  # Output dim of the network
    network=ValueNetwork(256, 128),  # Custom network before projection
    norm_embed=True,
    hl_gauss_loss=dict(
        min_value=-100.,
        max_value=100.,
        num_bins=32
    )
)

# Use in training loop
embed = torch.randn(32, 256)  # Input matches network input dim
targets = torch.randn(32) * 50  # Targets in [-100, 100] range

loss = hl_gauss_layer(embed, targets)
loss.backward()
```

--------------------------------

### HLGaussLayer

Source: https://context7.com/lucidrains/hl-gauss-pytorch/llms.txt

A complete neural network layer that projects embeddings to bin logits and handles loss computation or value prediction. It supports various configurations including custom networks and regression fallback.

```APIDOC
## HLGaussLayer

### Description
A complete neural network layer that projects embeddings to bin logits and handles loss computation or value prediction. Supports optional input normalization, custom networks, and can fall back to standard MSE regression for ablation studies.

### Method
`HLGaussLayer(dim, norm_embed=False, hl_gauss_loss=None, network=None, use_regression=False)`

### Parameters
#### Initialization Parameters
- **dim** (int) - Required - The output dimension of the layer (number of bins if `use_regression` is False, or the output dimension of the custom network if provided).
- **norm_embed** (bool) - Optional - Whether to apply RMSNorm to the input embeddings before projection. Defaults to `False`.
- **hl_gauss_loss** (dict or None) - Optional - Configuration dictionary for `HLGaussLoss`. If `None`, standard regression is used if `use_regression` is True.
  - **min_value** (float) - Required if `hl_gauss_loss` is provided.
  - **max_value** (float) - Required if `hl_gauss_loss` is provided.
  - **num_bins** (int) - Required if `hl_gauss_loss` is provided.
  - **sigma** (float) - Optional.
  - **clamp_to_range** (bool) - Optional.
- **network** (nn.Module or None) - Optional - A custom neural network to process embeddings before the final projection. If provided, `dim` should match the output dimension of this network.
- **use_regression** (bool) - Optional - If `True`, uses standard MSE regression. If `False` (default), uses HL-Gauss classification.

#### Forward Pass Parameters
- **embed** (torch.Tensor) - Required - The input embeddings.
- **targets** (torch.Tensor) - Optional - The ground truth targets. If not provided, the layer returns predictions.
- **return_logits** (bool) - Optional - If `True`, returns raw logits instead of predicted values. Defaults to `False`.
- **return_value_and_logits** (bool) - Optional - If `True`, returns both predicted values and logits. Defaults to `False`.

### Request Example (Training)
```python
import torch
from hl_gauss_pytorch import HLGaussLayer

# Create layer with HL-Gauss classification
hl_gauss_layer = HLGaussLayer(
    dim=256,  # Input embedding dimension
    norm_embed=True,  # Apply RMSNorm before projection
    hl_gauss_loss=dict(
        min_value=0.,
        max_value=5.,
        num_bins=32,
        sigma=0.5,
    )
)

# Training
embed = torch.randn(7, 256).requires_grad_()
targets = torch.randint(0, 5, (7,)).float()

loss = hl_gauss_layer(embed, targets)
loss.backward()
print(f"Training loss: {loss.item():.4f}")
```

### Request Example (Inference)
```python
with torch.no_grad():
    pred_values = hl_gauss_layer(embed)  # Returns (7,) tensor
    print(f"Predictions: {pred_values}")

# Get raw logits for custom processing
logits = hl_gauss_layer(embed, return_logits=True)  # (7, 32)
print(f"Logits shape: {logits.shape}")

# Get both predicted values and logits
pred_values, logits = hl_gauss_layer(embed, return_value_and_logits=True)
```

### Request Example (Custom Network)
```python
import torch
from torch import nn
from hl_gauss_pytorch import HLGaussLayer

# Define a custom MLP network
class ValueNetwork(nn.Module):
    def __init__(self, dim_in, dim_out):
        super().__init__()
        self.net = nn.Sequential(
            nn.Linear(dim_in, dim_out),
            nn.SiLU(),
            nn.Linear(dim_out, dim_out),
            nn.SiLU()
        )

    def forward(self, x):
        return self.net(x)

# Create layer with custom network
hl_gauss_layer = HLGaussLayer(
    dim=128,  # Output dim of the network
    network=ValueNetwork(256, 128),  # Custom network before projection
    norm_embed=True,
    hl_gauss_loss=dict(
        min_value=-100.,
        max_value=100.,
        num_bins=32
    )
)

# Use in training loop
embed = torch.randn(32, 256)  # Input matches network input dim
targets = torch.randn(32) * 50  # Targets in [-100, 100] range

loss = hl_gauss_layer(embed, targets)
loss.backward()
```

### Request Example (Regression Fallback)
```python
import torch
from hl_gauss_pytorch import HLGaussLayer

# Classification-based (HL-Gauss)
hl_gauss_classification = HLGaussLayer(
    dim=256,
    use_regression=False,  # Default: use HL-Gauss classification
    hl_gauss_loss=dict(min_value=0., max_value=10., num_bins=64)
)

# Regression-based (MSE)
hl_gauss_regression = HLGaussLayer(
    dim=256,
    use_regression=True,  # Use standard MSE regression
    hl_gauss_loss=dict(min_value=0., max_value=10.) # min/max value still needed for target clamping if used
)
```

### Response
#### Success Response (Training)
- **loss** (torch.Tensor) - The computed loss (either HL-Gauss cross-entropy or MSE).

#### Success Response (Inference)
- **pred_values** (torch.Tensor) - The predicted regression values.

#### Success Response (Return Logits)
- **logits** (torch.Tensor) - The raw predicted logits.

#### Success Response (Return Value and Logits)
- **pred_values** (torch.Tensor) - The predicted regression values.
- **logits** (torch.Tensor) - The raw predicted logits.

#### Response Example (Training)
```
Training loss: 0.1234
```

#### Response Example (Inference)
```
Predictions: tensor([1.2345, 2.3456, 3.4567, 4.5678, 5.6789])
Logits shape: torch.Size([7, 32])
```
```

--------------------------------

### BinaryHLGaussLoss for Binary Classification

Source: https://context7.com/lucidrains/hl-gauss-pytorch/llms.txt

Utilize BinaryHLGaussLoss for binary classification tasks with soft labels, employing Gaussian smoothing. It's suitable for noisy binary targets or values naturally bounded between 0 and 1.

```python
import torch
from hl_gauss_pytorch import BinaryHLGaussLoss

# Create binary HL-Gauss loss
binary_hl_gauss = BinaryHLGaussLoss(
    sigma=0.15,
    min_value=0.,
    max_value=1.,
    clamp_to_range=True
)

# Training with binary targets
logits = torch.randn(16)  # Raw logits (not sigmoid)
targets = torch.randint(0, 2, (16,)).float()  # Binary: 0 or 1

loss = binary_hl_gauss(logits, targets)
loss.backward()
print(f"Binary loss: {loss.item():.4f}")

# Inference: get predicted values in [0, 1]
with torch.no_grad():
    pred_values = binary_hl_gauss(logits)
    print(f"Predictions: {pred_values}")  # Values between 0 and 1

# Works with soft targets too
soft_targets = torch.rand(16)  # Values in [0, 1]
loss_soft = binary_hl_gauss(logits, soft_targets)
```

--------------------------------

### HLGaussLoss

Source: https://context7.com/lucidrains/hl-gauss-pytorch/llms.txt

The core loss function that converts regression targets into Gaussian-smoothed probability distributions over bins and computes cross-entropy loss. It can also return predicted values from logits during inference.

```APIDOC
## HLGaussLoss

### Description
The core loss function that converts regression targets into Gaussian-smoothed probability distributions over bins and computes cross-entropy loss between predicted logits and target probabilities. When called without targets, it returns the predicted value from logits.

### Method
`HLGaussLoss(min_value, max_value, num_bins, sigma=None, clamp_to_range=False)`

### Parameters
#### Initialization Parameters
- **min_value** (float) - Required - The minimum value of the regression target range.
- **max_value** (float) - Required - The maximum value of the regression target range.
- **num_bins** (int) - Required - The number of bins to discretize the target range into.
- **sigma** (float) - Optional - The width of the Gaussian smoothing. Defaults to `2 * bin_width`.
- **clamp_to_range** (bool) - Optional - Whether to clamp target values to the specified `min_value` and `max_value`. Defaults to `False`.

#### Forward Pass Parameters
- **logits** (torch.Tensor) - Required - The predicted logits from the model, with shape `(batch, seq, num_bins)`.
- **targets** (torch.Tensor) - Optional - The ground truth regression targets, with shape `(batch, seq)`. If not provided, the function returns predicted values.

### Request Example
```python
import torch
from hl_gauss_pytorch import HLGaussLoss

# Initialize the loss function with value range and bins
hl_gauss = HLGaussLoss(
    min_value=0.,
    max_value=5.,
    num_bins=32,
    sigma=0.5,  # Gaussian smoothing width (default: 2x bin size)
    clamp_to_range=True  # Clamp targets to [min_value, max_value]
)

# Training: compute loss from logits and targets
logits = torch.randn(3, 16, 32).requires_grad_()  # (batch, seq, num_bins)
targets = torch.randint(0, 5, (3, 16)).float()     # (batch, seq)

loss = hl_gauss(logits, targets)
loss.backward()
print(f"Loss: {loss.item():.4f}")

# Inference: predict values from logits (no targets)
with torch.no_grad():
    pred_values = hl_gauss(logits)  # Returns (3, 16) tensor
    print(f"Predicted values shape: {pred_values.shape}")
    print(f"Predicted values: {pred_values[0, :5]}")
```

### Response
#### Success Response (Training)
- **loss** (torch.Tensor) - The computed cross-entropy loss.

#### Success Response (Inference)
- **pred_values** (torch.Tensor) - The predicted regression values.

#### Response Example (Training)
```
Loss: 1.2345
```

#### Response Example (Inference)
```
Predicted values shape: torch.Size([3, 16])
Predicted values: tensor([1.2345, 2.3456, 3.4567, 4.5678, 5.6789])
```
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.