### Complete Training Configuration Example

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/configuration.md

A full Python example demonstrating how to configure and use the balanced loss for both training and validation phases with imbalanced datasets.

```python
import torch
import torch.optim as optim
from balanced_loss import Loss

# Dataset information
train_samples_per_class = [5, 50, 500]  # Highly imbalanced
num_classes = 3

# Loss configuration for training
train_loss = Loss(
    loss_type="focal_loss",
    beta=0.999,
    fl_gamma=2,
    samples_per_class=train_samples_per_class,
    class_balanced=True,
    safe=False
)

# Loss configuration for validation (may have incomplete class distribution)
val_loss = Loss(
    loss_type="focal_loss",
    beta=0.999,
    fl_gamma=2,
    samples_per_class=train_samples_per_class,
    class_balanced=True,
    safe=True
)

# Model and optimizer
model = YourModel(num_classes=num_classes)
optimizer = optim.Adam(model.parameters(), lr=1e-3)

# Training loop
for epoch in range(num_epochs):
    # Training phase
    model.train()
    for batch_images, batch_labels in train_dataloader:
        logits = model(batch_images)
        loss = train_loss(logits, batch_labels)
        
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
    
    # Validation phase
    model.eval()
    val_losses = []
    with torch.no_grad():
        for batch_images, batch_labels in val_dataloader:
            logits = model(batch_images)
            loss = val_loss(logits, batch_labels)
            val_losses.append(loss.item())
    
    print(f"Epoch {epoch}: val_loss={sum(val_losses)/len(val_losses):.4f}")
```

--------------------------------

### Quick Example of Balanced Loss

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/README.md

Demonstrates how to instantiate and use a class-balanced focal loss function with specified training distribution. Ensure PyTorch and the balanced-loss library are installed.

```python
from balanced_loss import Loss
import torch

# Create loss function
loss_fn = Loss(
    loss_type="focal_loss",
    beta=0.999,
    fl_gamma=2,
    samples_per_class=[30, 100, 25],  # training distribution
    class_balanced=True
)

# Use in forward pass
logits = torch.randn(32, 3)
labels = torch.randint(0, 3, (32,))
loss = loss_fn(logits, labels)
```

--------------------------------

### Install balanced-loss Package

Source: https://github.com/fcakyon/balanced-loss/blob/main/README.md

Install the balanced-loss library using pip. This is the first step before using any of its functionalities.

```bash
pip install balanced-loss
```

--------------------------------

### Cross-Entropy Loss Example

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/implementation-details.md

A runnable example demonstrating PyTorch's F.cross_entropy with sample logits, one-hot labels, and class weights. Shows equivalent usage to standard PyTorch.

```python
import torch
import torch.nn.functional as F

logits = torch.tensor([[0.78, 0.1, 0.05], [0.78, 0.83, 0.05]])
labels = torch.tensor([[1., 0., 0.], [0., 0., 1.]])  # one-hot
weights = torch.tensor([1.0, 2.0, 0.5])

# Equivalent to standard PyTorch usage
loss = F.cross_entropy(logits, labels.argmax(1), weight=weights)
```

--------------------------------

### Minimal Balanced Loss Initialization and Usage

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/INDEX.md

Demonstrates the most basic setup for initializing and using the Balanced Loss function. Requires importing the Loss class.

```python
from balanced_loss import Loss
loss_fn = Loss(loss_type="focal_loss")
loss = loss_fn(logits, labels)
```

--------------------------------

### Example Calculation of Class Weights

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/implementation-details.md

Demonstrates the step-by-step calculation of effective sample numbers and normalized class weights using sample data and a beta value. Shows how the least frequent class gets the highest weight.

```python
import numpy as np

samples_per_class = [30, 100, 25]
beta = 0.999

# Step 1: Calculate effective numbers
effective_num = 1.0 - np.power(beta, samples_per_class)
# [0.0299..., 0.0951..., 0.0246...]

# Step 2: Calculate raw weights (inverse proportion)
weights = (1.0 - beta) / np.array(effective_num)
# [0.0332..., 0.0105..., 0.0405...]

# Step 3: Normalize weights
effective_num_classes = 3
weights = weights / np.sum(weights) * effective_num_classes
# [0.896..., 0.284..., 1.095...]

# Result: Class 2 (least frequent) has highest weight ~1.10
#         Class 1 (most frequent) has lowest weight ~0.28
#         Class 0 gets intermediate weight ~0.90
```

--------------------------------

### Class-Balanced Cross-Entropy Loss Example

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/api-reference-loss.md

Shows how to apply class-balanced Cross-Entropy Loss. Requires specifying the loss type, beta, and the distribution of samples per class.

```python
import torch
from balanced_loss import Loss

logits = torch.tensor([[0.78, 0.1, 0.05], [0.78, 0.83, 0.05]])  # 2 samples, 3 classes
labels = torch.tensor([0, 2])  # class 0 and class 2

samples_per_class = [30, 100, 25]  # training dataset distribution

ce_loss = Loss(
    loss_type="cross_entropy",
    beta=0.999,
    samples_per_class=samples_per_class,
    class_balanced=True
)
loss = ce_loss(logits, labels)
print(loss.item())  # ~1.59
```

--------------------------------

### Advanced Configuration

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/configuration.md

A comprehensive example showing full custom configuration for class-balanced focal loss, including beta, gamma, sample counts, and safe mode.

```python
from balanced_loss import Loss

samples_per_class = [10, 50, 5, 200, 15]

# Full custom configuration
loss_fn = Loss(
    loss_type="focal_loss",
    beta=0.9999,      # more aggressive class balancing
    fl_gamma=2.5,     # stronger focusing on hard examples
    samples_per_class=samples_per_class,
    class_balanced=True,
    safe=True         # handle incomplete class distributions
)
```

--------------------------------

### Valid Logits and Labels Shapes

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/types.md

Examples demonstrating valid shapes for logits and labels with different batch sizes and class counts.

```python
import torch

# Batch size 1, 3 classes
logits = torch.randn(1, 3)
labels = torch.tensor([0])

# Batch size 32, 10 classes
logits = torch.randn(32, 10)
labels = torch.tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 
                       6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1])

# Batch size 256, 1000 classes
logits = torch.randn(256, 1000)
labels = torch.randint(0, 1000, (256,))
```

--------------------------------

### Class-Balanced Binary Cross-Entropy Loss Example

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/api-reference-loss.md

Demonstrates the use of class-balanced Binary Cross-Entropy Loss. Requires specifying the loss type and the distribution of samples per class.

```python
import torch
from balanced_loss import Loss

logits = torch.tensor([[0.78, 0.1, 0.05], [0.78, 0.83, 0.05]])  # 2 samples, 3 classes
labels = torch.tensor([0, 2])  # class 0 and class 2

samples_per_class = [30, 100, 25]  # training dataset distribution

bce_loss = Loss(
    loss_type="binary_cross_entropy",
    samples_per_class=samples_per_class,
    class_balanced=True
)
loss = bce_loss(logits, labels)
print(loss.item())  # ~1.08
```

--------------------------------

### Binary Cross-Entropy Loss (unbalanced)

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/api-reference-loss.md

Example of how to compute unbalanced Binary Cross-Entropy Loss using the Loss class. Ensure the 'balanced_loss' library is imported.

```python
import torch
from balanced_loss import Loss

logits = torch.tensor([[0.78, 0.1, 0.05]])  # 1 sample, 3 classes
labels = torch.tensor([0])  # sample belongs to class 0

bce_loss = Loss(loss_type="binary_cross_entropy")
loss = bce_loss(logits, labels)
print(loss.item())  # ~0.80
```

--------------------------------

### Class-Balanced Focal Loss Example

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/api-reference-loss.md

Demonstrates how to use class-balanced Focal Loss. Requires specifying the loss type, beta, focal loss gamma, and the distribution of samples per class.

```python
import torch
from balanced_loss import Loss

logits = torch.tensor([[0.78, 0.1, 0.05], [0.78, 0.83, 0.05]])  # 2 samples, 3 classes
labels = torch.tensor([0, 2])  # class 0 and class 2

samples_per_class = [30, 100, 25]  # training dataset distribution

focal_loss = Loss(
    loss_type="focal_loss",
    beta=0.999,
    fl_gamma=2,
    samples_per_class=samples_per_class,
    class_balanced=True
)
loss = focal_loss(logits, labels)
print(loss.item())  # ~1.17
```

--------------------------------

### Cross-Entropy Loss (unbalanced)

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/api-reference-loss.md

Example of how to compute unbalanced Cross-Entropy Loss using the Loss class. Ensure the 'balanced_loss' library is imported.

```python
import torch
from balanced_loss import Loss

logits = torch.tensor([[0.78, 0.1, 0.05]])  # 1 sample, 3 classes
labels = torch.tensor([0])  # sample belongs to class 0

ce_loss = Loss(loss_type="cross_entropy")
loss = ce_loss(logits, labels)
print(loss.item())  # ~1.17
```

--------------------------------

### Labels Tensor Example

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/types.md

Shows how to create a PyTorch tensor for integer class labels. Expected shape is [batch_size] and dtype is torch.int64 or torch.long.

```python
import torch

# 2 samples with class indices
labels = torch.tensor([0, 2], dtype=torch.int64)

# Equivalent
labels = torch.tensor([0, 2], dtype=torch.long)

print(labels.shape)  # torch.Size([2])
```

--------------------------------

### Loss Output Tensor Example

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/types.md

Illustrates the computation and output of the loss function, resulting in a scalar PyTorch tensor of dtype torch.float32.

```python
import torch
from balanced_loss import Loss

logits = torch.tensor([[0.78, 0.1, 0.05]])
labels = torch.tensor([0])

loss_fn = Loss()
loss = loss_fn(logits, labels)

print(loss.shape)   # torch.Size([])
print(loss.item())  # 0.8156... (Python float)
print(type(loss))   # <class 'torch.Tensor'>
```

--------------------------------

### Logits Tensor Example

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/types.md

Demonstrates the creation of a PyTorch tensor for model output logits. Expected shape is [batch_size, num_classes] and dtype is typically torch.float32.

```python
import torch

# 2 samples, 3 classes
logits = torch.tensor([
    [0.78, 0.1, 0.05],   # sample 1
    [0.5, 0.3, 0.2]      # sample 2
], dtype=torch.float32)

print(logits.shape)  # torch.Size([2, 3])
```

--------------------------------

### Focal Loss (unbalanced)

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/api-reference-loss.md

Example of how to compute unbalanced Focal Loss using the Loss class. Ensure the 'balanced_loss' library is imported.

```python
import torch
from balanced_loss import Loss

logits = torch.tensor([[0.78, 0.1, 0.05]])  # 1 sample, 3 classes
labels = torch.tensor([0])  # sample belongs to class 0

focal_loss = Loss(loss_type="focal_loss")
loss = focal_loss(logits, labels)
print(loss.item())  # ~0.85
```

--------------------------------

### Binary Cross-Entropy Loss Usage

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/implementation-details.md

Example of using PyTorch's F.binary_cross_entropy_with_logits for multi-label classification. Each class is treated independently, and weights are reshaped similarly to focal loss.

```python
# From balanced_loss/losses.py, lines 124-125
cb_loss = F.binary_cross_entropy_with_logits(
    input=logits,
    target=labels_one_hot,
    weight=weights
)
```

--------------------------------

### Focal Loss Weight Application

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/implementation-details.md

Code snippet demonstrating how weights are applied for class balancing in the focal loss implementation. Weights are unsqueezed, repeated, and summed to get per-sample weights.

```python
# From balanced_loss/losses.py, lines 111-116
weights = weights.unsqueeze(0)  # [1, num_classes]
weights = weights.repeat(batch_size, 1)  # [batch, num_classes]
weights = weights * labels_one_hot  # Apply to one-hot
weights = weights.sum(1)  # [batch] - per-sample weight
weights = weights.unsqueeze(1)  # [batch, 1]
weights = weights.repeat(1, num_classes)  # [batch, num_classes]
```

--------------------------------

### Focal Loss with Class Balancing

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/api-reference-focal-loss.md

Computes focal loss incorporating class balancing weights derived from sample counts and a beta parameter. This example demonstrates how to calculate and apply alpha weights for effective class balancing.

```python
import torch
import numpy as np
from balanced_loss.losses import focal_loss

# Setup
logits = torch.tensor([[0.78, 0.1, 0.05], [0.78, 0.83, 0.05]])
labels = torch.tensor([0, 2])  # integer labels
labels_onehot = torch.nn.functional.one_hot(labels, num_classes=3).float()

# Calculate class balancing weights
samples_per_class = [30, 100, 25]
beta = 0.999

effective_num = 1.0 - np.power(beta, samples_per_class)
weights = (1.0 - beta) / np.array(effective_num)
weights = weights / np.sum(weights) * len(samples_per_class)
weights = torch.tensor(weights).float()

# Apply weights for batch
weights_expanded = weights.unsqueeze(0).repeat(logits.shape[0], 1) * labels_onehot
alpha = weights_expanded.sum(1).unsqueeze(1).repeat(1, 3)

# Compute focal loss with class balancing
loss = focal_loss(logits, labels_onehot, alpha=alpha, gamma=2)
```

--------------------------------

### Balanced Loss Initialization and Usage on CPU/GPU

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/implementation-details.md

Demonstrates initializing the Loss function and applying it to logits on both CPU and GPU, showing automatic device handling for weights.

```python
import torch
from balanced_loss import Loss

# CPU
loss_fn = Loss(samples_per_class=[10, 20, 30], class_balanced=True)
logits_cpu = torch.randn(2, 3)
loss = loss_fn(logits_cpu, torch.tensor([0, 1]))  # Weights on CPU

# GPU
logits_gpu = torch.randn(2, 3).cuda()
loss = loss_fn(logits_gpu, torch.tensor([0, 1]).cuda())  # Weights on GPU
```

--------------------------------

### Focal Loss Gamma Parameter Tuning

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/usage-guide.md

Tune the fl_gamma parameter in Focal Loss to control the focusing on hard examples. Higher gamma values increase the focus on harder-to-classify examples. This example compares losses with gamma values of 0.0, 1.0, 2.0, and 2.5.

```python
import torch
from balanced_loss import Loss

logits = torch.randn(32, 10)
labels = torch.randint(0, 10, (32,))

# No focusing
loss_no_focus = Loss(loss_type="focal_loss", fl_gamma=0.0)

# Light focusing
loss_light = Loss(loss_type="focal_loss", fl_gamma=1.0)

# Moderate focusing (default)
loss_default = Loss(loss_type="focal_loss", fl_gamma=2.0)

# Strong focusing
loss_strong = Loss(loss_type="focal_loss", fl_gamma=2.5)

# Compute and compare
losses = []
for gamma in [0.0, 1.0, 2.0, 2.5]:
    loss_fn = Loss(loss_type="focal_loss", fl_gamma=gamma)
    loss = loss_fn(logits, labels).item()
    losses.append(loss)
    print(f"Gamma={gamma}: {loss:.4f}")
```

--------------------------------

### Package initialization and version export

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/modules-and-exports.md

This snippet shows the content of the __init__.py file, exporting the Loss class and defining the package version.

```python
from .losses import Loss

__version__ = "0.1.1"
```

--------------------------------

### Minimal Configuration (Standard Loss)

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/configuration.md

Instantiate the Loss class with default settings for standard cross-entropy loss without class balancing.

```python
from balanced_loss import Loss

# Uses all defaults: cross-entropy loss, no class balancing
loss_fn = Loss()
```

--------------------------------

### Focal Loss with Custom Gamma

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/api-reference-focal-loss.md

Computes focal loss with a specified gamma parameter to control the focusing effect. Different gamma values adjust the down-weighting of easy examples and up-weighting of hard examples.

```python
import torch
from balanced_loss.losses import focal_loss

logits = torch.tensor([[0.78, 0.1, 0.05]])
labels = torch.tensor([[1.0, 0.0, 0.0]])  # one-hot encoded, class 0

# gamma=0: equivalent to binary cross-entropy
loss_no_focus = focal_loss(logits, labels, gamma=0.0)

# gamma=1: moderate focusing
loss_moderate = focal_loss(logits, labels, gamma=1.0)

# gamma=2: strong focusing (default)
loss_strong = focal_loss(logits, labels, gamma=2.0)
```

--------------------------------

### Safe Mode Enabled: Handling Zero Samples

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/configuration.md

Illustrates how 'safe' mode prevents errors by replacing division by zero with infinity and calculating a safe weight of 0.0 for classes with zero samples.

```text
If class X has 0 samples in training data:
  effective_num[X] = 1.0 - beta^0 = 1.0 - 1.0 = 0.0
  effective_num[X] = inf  # Replaced with infinity
  weights[X] = (1 - beta) / inf = 0.0  # Safe weight
  
Effective number of classes excludes zero effective_num entries
```

--------------------------------

### Safe Mode Configuration (for Validation)

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/configuration.md

Demonstrates configuring loss functions for training (safe=False) and validation (safe=True) to handle potential missing classes in validation batches.

```python
from balanced_loss import Loss

# Training loss (all classes present)
training_loss = Loss(
    loss_type="focal_loss",
    samples_per_class=[30, 100, 25],
    class_balanced=True,
    safe=False
)

# Validation loss (may not have all classes in a batch)
validation_loss = Loss(
    loss_type="focal_loss",
    samples_per_class=[30, 100, 25],
    class_balanced=True,
    safe=True  # Prevents division by zero for missing classes
)
```

--------------------------------

### Instantiate Standard and Class-Balanced Loss

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/INDEX.md

Instantiate the Loss class for standard loss calculation or with class balancing enabled. For class balancing, provide the `samples_per_class` argument.

```python
# Standard loss
loss_fn = Loss(loss_type="focal_loss")

# Class-balanced loss
loss_fn = Loss(
    loss_type="focal_loss",
    samples_per_class=[30, 100, 25],
    class_balanced=True
)
```

--------------------------------

### Balanced Loss Computation on CPU

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/types.md

Shows how to perform loss calculations using tensors residing on the CPU. This is the default behavior.

```python
import torch
from balanced_loss import Loss

logits = torch.randn(2, 3)  # Default: CPU
labels = torch.tensor([0, 1])  # Default: CPU

loss_fn = Loss(samples_per_class=[10, 20, 30], class_balanced=True)
loss = loss_fn(logits, labels)  # Computed on CPU
```

--------------------------------

### Class Balancing with Safe Mode Configuration

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/errors.md

Demonstrates configuring the `Loss` class for class balancing. `safe=False` is suitable for training where all classes are expected, while `safe=True` is recommended for validation to handle potentially missing classes gracefully.

```python
import torch
from balanced_loss import Loss

# For training (all classes expected to be present)
train_loss_fn = Loss(
    loss_type="focal_loss",
    samples_per_class=[30, 100, 25],
    class_balanced=True,
    safe=False  # Strict: all classes must be present
)

# For validation (some classes might be missing)
val_loss_fn = Loss(
    loss_type="focal_loss",
    samples_per_class=[30, 100, 25],
    class_balanced=True,
    safe=True  # Lenient: handles missing classes
)
```

--------------------------------

### Initialize and Use Balanced Loss with PyTorch Tensors

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/configuration.md

Instantiate the Loss class with desired parameters and observe automatic device placement for both CPU and GPU tensors. The library ensures computations occur on the same device as the input tensors.

```python
import torch
from balanced_loss import Loss

loss_fn = Loss(
    loss_type="focal_loss",
    samples_per_class=[30, 100, 25],
    class_balanced=True
)

# CPU tensors
logits_cpu = torch.randn(2, 3)
labels_cpu = torch.tensor([0, 1])
loss_cpu = loss_fn(logits_cpu, labels_cpu)  # Automatically computed on CPU

# GPU tensors
logits_gpu = torch.randn(2, 3).cuda()
labels_gpu = torch.tensor([0, 1]).cuda()
loss_gpu = loss_fn(logits_gpu, labels_gpu)  # Automatically computed on GPU
```

--------------------------------

### Fixing IndexError: num_classes Mismatch

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/errors.md

Ensures label values are within the valid range [0, num_classes). This example shows how to correct labels and includes assertions for validation.

```python
import torch
from balanced_loss import Loss

logits = torch.tensor([[0.78, 0.1, 0.05], [0.5, 0.3, 0.2]])
labels = torch.tensor([0, 2])  # Valid range: 0-2 for 3 classes

loss_fn = Loss()
loss = loss_fn(logits, labels)  # Works

# Validate labels before passing
num_classes = logits.shape[1]
assert labels.min() >= 0, "Labels must be >= 0"
assert labels.max() < num_classes, f"Labels must be < {num_classes}"
```

--------------------------------

### Import necessary libraries

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/usage-guide.md

Import the Loss class from balanced_loss and torch.

```python
from balanced_loss import Loss
import torch
```

--------------------------------

### Class-Balanced Configuration

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/configuration.md

Configure class-balanced loss functions (focal, cross-entropy, binary cross-entropy) using sample counts per class and an optional beta for balancing aggressiveness.

```python
from balanced_loss import Loss

samples_per_class = [30, 100, 25]

# Class-balanced focal loss
loss_fn = Loss(
    loss_type="focal_loss",
    samples_per_class=samples_per_class,
    class_balanced=True
)

# Class-balanced cross-entropy with custom beta
loss_fn = Loss(
    loss_type="cross_entropy",
    beta=0.995,  # slightly more aggressive balancing
    samples_per_class=samples_per_class,
    class_balanced=True
)

# Class-balanced binary cross-entropy
loss_fn = Loss(
    loss_type="binary_cross_entropy",
    samples_per_class=samples_per_class,
    class_balanced=True
)
```

--------------------------------

### Cross-Entropy Loss Usage

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/implementation-details.md

Example of using PyTorch's F.cross_entropy with logits, one-hot encoded labels, and class weights. The weight parameter directly applies class weights.

```python
# From balanced_loss/losses.py, lines 122-123
cb_loss = F.cross_entropy(
    input=logits,
    target=labels_one_hot,
    weight=weights
)
```

--------------------------------

### Floating-Point Precision in Balanced Loss

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/implementation-details.md

Shows how to initialize the Loss function and apply it using different floating-point precisions (float32, float64, float16).

```python
import torch
from balanced_loss import Loss

loss_fn = Loss(samples_per_class=[10, 20, 30], class_balanced=True)

# Float32 (default)
logits_fp32 = torch.randn(2, 3, dtype=torch.float32)
loss_fp32 = loss_fn(logits_fp32, torch.tensor([0, 1]))

# Float64 (double precision)
logits_fp64 = torch.randn(2, 3, dtype=torch.float64)
loss_fp64 = loss_fn(logits_fp64, torch.tensor([0, 1]))

# Float16 (half precision - mixed precision training)
logits_fp16 = torch.randn(2, 3, dtype=torch.float16)
loss_fp16 = loss_fn(logits_fp16, torch.tensor([0, 1]))
```

--------------------------------

### Fixing RuntimeError: Batch Size Mismatch

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/errors.md

Ensures that the batch size of logits and labels match. This example demonstrates correcting the label tensor to align with the logits batch size and includes an assertion for runtime validation.

```python
import torch
from balanced_loss import Loss

logits = torch.tensor([[0.78, 0.1, 0.05], [0.5, 0.3, 0.2]])  # 2 samples
labels = torch.tensor([0, 1])  # 2 samples - MATCH!

loss_fn = Loss()
loss = loss_fn(logits, labels)  # Works

# Validate in training
assert logits.shape[0] == labels.shape[0], "Batch size mismatch"
```

--------------------------------

### Import Loss class from top-level package

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/modules-and-exports.md

This is the recommended way to import the main Loss class for general use.

```python
from balanced_loss import Loss
```

--------------------------------

### Balanced Loss Computation on GPU (CUDA)

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/types.md

Demonstrates using CUDA-enabled GPUs for loss computation by moving both logits and labels to the GPU. Weights are also automatically created on the GPU.

```python
import torch
from balanced_loss import Loss

logits = torch.randn(2, 3).cuda()  # Move to GPU
labels = torch.tensor([0, 1]).cuda()  # Move to GPU

loss_fn = Loss(samples_per_class=[10, 20, 30], class_balanced=True)
loss = loss_fn(logits, labels)  # Automatically computed on GPU

# Weights are created on GPU automatically
```

--------------------------------

### Beta Parameter Tuning for Class Balancing

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/usage-guide.md

Adjust the beta parameter to control the strength of class balancing. Lower beta values result in less aggressive balancing, while higher values lead to more aggressive balancing. This example demonstrates conservative, default, and aggressive balancing.

```python
import torch
from balanced_loss import Loss

logits = torch.randn(32, 10)
labels = torch.randint(0, 10, (32,))

# Conservative balancing
loss_conservative = Loss(
    loss_type="focal_loss",
    beta=0.9,  # Lower beta = less aggressive
    samples_per_class=samples_per_class,
    class_balanced=True
)

# Moderate balancing (default)
loss_default = Loss(
    loss_type="focal_loss",
    beta=0.999,  # Default
    samples_per_class=samples_per_class,
    class_balanced=True
)

# Aggressive balancing
loss_aggressive = Loss(
    loss_type="focal_loss",
    beta=0.9999,  # Higher beta = more aggressive
    samples_per_class=samples_per_class,
    class_balanced=True
)

# Compute losses to compare
l1 = loss_conservative(logits, labels).item()
l2 = loss_default(logits, labels).item()
l3 = loss_aggressive(logits, labels).item()

print(f"Conservative: {l1:.4f}")
print(f"Default:      {l2:.4f}")
print(f"Aggressive:   {l3:.4f}")
```

--------------------------------

### Focal Loss Step-by-Step Implementation

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/implementation-details.md

Detailed breakdown of the custom focal_loss function, showing the calculation of binary cross-entropy, the modulating factor, and alpha weighting for numerical stability.

```python
# Step 1: Binary cross-entropy with logits (base loss)
bc_loss = F.binary_cross_entropy_with_logits(
    input=logits,
    target=labels,
    reduction="none"
)

# Step 2: Compute modulating factor (1 - p_t)^gamma
if gamma == 0.0:
    modulator = 1.0
else:
    # Using log-sum-exp trick for numerical stability
    modulator = torch.exp(
        -gamma * labels * logits - 
        gamma * torch.log(1 + torch.exp(-1.0 * logits))
    )

# Step 3: Apply modulator to loss
loss = modulator * bc_loss

# Step 4: Apply alpha weighting (optional)
if alpha is not None:
    weighted_loss = alpha * loss
    focal_loss = torch.sum(weighted_loss)
else:
    focal_loss = torch.sum(loss)

# Step 5: Normalize by number of positive samples
focal_loss /= torch.sum(labels)
return focal_loss
```

--------------------------------

### Binary Cross-Entropy Configuration

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/configuration.md

Configure binary cross-entropy loss, including options for standard multi-label BCE and BCE on softmax output.

```python
from balanced_loss import Loss

# Standard binary cross-entropy (multi-label)
loss_fn = Loss(loss_type="binary_cross_entropy")

# Binary cross-entropy on softmax output
loss_fn = Loss(loss_type="softmax_binary_cross_entropy")
```

--------------------------------

### Broadcasting Class Weights for Loss Calculation

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/implementation-details.md

Demonstrates the process of expanding class weights to per-sample weights for focal loss and binary cross-entropy. This ensures each sample's loss is weighted correctly based on its true class.

```python
# Input: class weights [num_classes]
weights = torch.tensor([0.9, 0.3, 1.1])  # shape [3]

# One-hot labels [batch, num_classes]
labels_onehot = torch.tensor([
    [1., 0., 0.],
    [0., 0., 1.]
])  # shape [2, 3]

# Step 1: unsqueeze for broadcasting
weights = weights.unsqueeze(0)  # [1, 3]

# Step 2: repeat across batch
weights = weights.repeat(batch_size, 1)  # [2, 3]

# Step 3: apply to one-hot labels
weights = weights * labels_onehot  # [2, 3]
# [[0.9, 0., 0.],
#  [0., 0., 1.1]]

# Step 4: sum across classes (select weight for true class)
weights = weights.sum(1)  # [2] -> [0.9, 1.1]

# Step 5: unsqueeze and repeat to match loss shape
weights = weights.unsqueeze(1).repeat(1, num_classes)  # [2, 3]
# [[0.9, 0.9, 0.9],
#  [1.1, 1.1, 1.1]]
```

--------------------------------

### Standard Binary Cross-Entropy Loss Implementation

Source: https://github.com/fcakyon/balanced-loss/blob/main/README.md

Apply a standard binary cross-entropy loss by initializing the Loss class with the appropriate type. Requires logits and labels.

```python
# binary cross-entropy loss
bce_loss = Loss(loss_type="binary_cross_entropy")
loss = bce_loss(logits, labels)
```

--------------------------------

### Loss Constructor

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/api-reference-loss.md

Initializes the Loss module with specified loss type and balancing parameters.

```APIDOC
## Class: `Loss`

### Description
Initializes the Loss module with specified loss type and balancing parameters.

### Constructor Signature
```python
Loss(
    loss_type: str = "cross_entropy",
    beta: float = 0.999,
    fl_gamma=2,
    samples_per_class=None,
    class_balanced=False,
    safe: bool = False,
)
```

### Parameters
#### Constructor Parameters
- **loss_type** (str) - Optional - Loss function type. Options: `"focal_loss"`, `"cross_entropy"`, `"binary_cross_entropy"`, `"softmax_binary_cross_entropy"`
- **beta** (float) - Optional - Hyperparameter for class-balanced loss calculation. Controls the decay rate for effective sample calculation.
- **fl_gamma** (float) - Optional - Hyperparameter for focal loss. Controls the focusing parameter in focal loss formula. Only applicable when `loss_type="focal_loss"`.
- **samples_per_class** (list[int]) - Conditional - Number of samples per class in the training dataset. Required when `class_balanced=True`. Example: `[30, 100, 25]` for 3 classes.
- **class_balanced** (bool) - Optional - Whether to apply class balancing. If `True`, `samples_per_class` must be provided.
- **safe** (bool) - Optional - If `True`, handles edge case where some classes have no samples by replacing zero effective numbers with infinity. Useful for test cases that don't contain all labels.

### Raises
- **ValueError**: `class_balanced=True` but `samples_per_class=None`
```

--------------------------------

### Customizing Balanced Loss Parameters

Source: https://github.com/fcakyon/balanced-loss/blob/main/README.md

Customize parameters like beta for class-balanced loss and gamma for focal loss when initializing the Loss class. This allows fine-tuning for specific training scenarios.

```python
import torch
from balanced_loss import Loss

# outputs and labels
logits = torch.tensor([[0.78, 0.1, 0.05]]) # 1 batch, 3 class
labels = torch.tensor([0])

# number of samples per class in the training dataset
samples_per_class = [30, 100, 25] # 30, 100, 25 samples for labels 0, 1 and 2, respectively

# class-balanced focal loss
focal_loss = Loss(
    loss_type="focal_loss",
    beta=0.999, # class-balanced loss beta
    fl_gamma=2, # focal loss gamma
    samples_per_class=samples_per_class,
    class_balanced=True
)
loss = focal_loss(logits, labels)
```

--------------------------------

### Integrating Class-Balanced Loss in Training Loop

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/api-reference-loss.md

Shows how to set up and use a class-balanced loss function within a standard PyTorch training loop, including model, optimizer, and dataloader integration.

```python
import torch
import torch.optim as optim
from balanced_loss import Loss

# Model setup
model = YourModel()
optimizer = optim.Adam(model.parameters())

# Loss setup
loss_fn = Loss(
    loss_type="focal_loss",
    samples_per_class=[30, 100, 25],
    class_balanced=True
)

# Training loop
for epoch in range(num_epochs):
    for batch_logits, batch_labels in dataloader:
        optimizer.zero_grad()
        
        logits = model(batch_logits)
        loss = loss_fn(logits, batch_labels)
        
        loss.backward()
        optimizer.step()
```

--------------------------------

### Balanced Loss Forward Pass and Properties

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/types.md

Shows how to use the Loss.forward() method and inspect the properties of the returned scalar loss tensor.

```python
import torch
from balanced_loss import Loss

logits = torch.tensor([[0.78, 0.1, 0.05]])
labels = torch.tensor([0])

loss_fn = Loss()
loss = loss_fn(logits, labels)

# Properties
print(loss.shape)      # torch.Size([], )  - scalar
print(loss.dtype)      # torch.float32
print(loss.device)     # cpu or cuda:0
print(loss.item())     # Python float: 0.815...
print(loss.requires_grad)  # True (for backprop)

# Can be used in optimization
loss.backward()
```

--------------------------------

### Initialize Balanced Loss with samples_per_class

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/types.md

Use samples_per_class to configure class balancing for a multi-class problem with an imbalanced dataset. Ensure the list length matches the number of classes and all values are positive.

```python
from balanced_loss import Loss

# 3-class problem with imbalanced distribution
samples_per_class = [30, 100, 25]

loss_fn = Loss(
    samples_per_class=samples_per_class,
    class_balanced=True
)
```

--------------------------------

### Gradient Flow and Backpropagation

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/implementation-details.md

Demonstrates that the loss function supports automatic differentiation, including backpropagation and gradient computation.

```python
import torch
from balanced_loss import Loss

logits = torch.randn(2, 3, requires_grad=True)
labels = torch.tensor([0, 1])

loss_fn = Loss()
loss = loss_fn(logits, labels)

loss.backward()  # Backpropagation
print(logits.grad)  # Gradients computed
```

--------------------------------

### Standard Focal Loss Implementation

Source: https://github.com/fcakyon/balanced-loss/blob/main/README.md

Use the Loss class to instantiate and apply a standard focal loss. Requires logits and labels as input.

```python
import torch
from balanced_loss import Loss

# outputs and labels
logits = torch.tensor([[0.78, 0.1, 0.05]]) # 1 batch, 3 class
labels = torch.tensor([0]) # 1 batch

# focal loss
focal_loss = Loss(loss_type="focal_loss")
loss = focal_loss(logits, labels)
```

--------------------------------

### Automatic Device Handling for Weights

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/implementation-details.md

The weight tensor is automatically created on the same device as the logits, supporting both CPU and GPU without manual transfer.

```python
weights = torch.tensor(weights, device=logits.device).float()
```

--------------------------------

### Import Loss class and focal_loss function

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/modules-and-exports.md

Import both the Loss class and the lower-level focal_loss function directly from the losses module.

```python
from balanced_loss.losses import Loss, focal_loss
```

--------------------------------

### Initialize Balanced Loss with different loss_type

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/types.md

Configure the specific loss function to be used by the Balanced Loss class. Supported types include cross_entropy, focal_loss, binary_cross_entropy, and softmax_binary_cross_entropy.

```python
from balanced_loss import Loss

# Each is a valid configuration
loss_ce = Loss(loss_type="cross_entropy")
loss_focal = Loss(loss_type="focal_loss")
loss_bce = Loss(loss_type="binary_cross_entropy")
loss_sbce = Loss(loss_type="softmax_binary_cross_entropy")
```

--------------------------------

### Mixed Precision Training with Balanced Loss

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/usage-guide.md

Implement mixed-precision training using PyTorch's autocast and GradScaler with the balanced-loss library. This can speed up training and reduce memory usage.

```python
import torch
import torch.nn as nn
import torch.optim as optim
from torch.cuda.amp import autocast, GradScaler
from balanced_loss import Loss

model = YourModelClass(num_classes=10).cuda()
optimizer = optim.Adam(model.parameters())
loss_fn = Loss(
    loss_type="focal_loss",
    samples_per_class=samples_per_class,
    class_balanced=True
)
scaler = GradScaler()

for batch_images, batch_labels in train_dataloader:
    batch_images = batch_images.cuda()
    batch_labels = batch_labels.cuda()
    
    # Use autocast for forward pass
    with autocast():
        logits = model(batch_images)
        loss = loss_fn(logits, batch_labels)
    
    # Scaled backward pass
    optimizer.zero_grad()
    scaler.scale(loss).backward()
    scaler.step(optimizer)
    scaler.update()
```

--------------------------------

### Standard Cross-Entropy Loss Implementation

Source: https://github.com/fcakyon/balanced-loss/blob/main/README.md

Instantiate and apply a standard cross-entropy loss using the Loss class. Accepts logits and labels.

```python
# cross-entropy loss
ce_loss = Loss(loss_type="cross_entropy")
loss = ce_loss(logits, labels)
```

--------------------------------

### Balanced Loss with Float32 Logits

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/types.md

Demonstrates the recommended usage with float32 logits and int64 labels. The Loss function handles these types directly.

```python
import torch
from balanced_loss import Loss

# Float32 logits (recommended)
logits_fp32 = torch.randn(2, 3, dtype=torch.float32)
labels = torch.tensor([0, 1], dtype=torch.int64)
loss_fn = Loss()
loss = loss_fn(logits_fp32, labels)  # Works
```

--------------------------------

### Balanced Loss with Float64 Logits

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/types.md

Shows how float64 logits are automatically converted to the default float32 type for loss calculation.

```python
# Float64 logits
logits_fp64 = torch.randn(2, 3, dtype=torch.float64)
loss = loss_fn(logits_fp64, labels)  # Works (automatic conversion)
```

--------------------------------

### Balanced Loss with Int32 Labels

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/types.md

Demonstrates using int32 tensors for labels, which is supported by the loss function.

```python
labels_int32 = torch.tensor([0, 1], dtype=torch.int32)
loss = loss_fn(logits_fp32, labels_int32)  # Works
```

--------------------------------

### Safe Mode Disabled: Division by Zero

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/configuration.md

Demonstrates the division by zero error that occurs when 'safe' is False and a class has zero samples in the training data.

```text
If class X has 0 samples in training data:
  effective_num[X] = 1.0 - beta^0 = 1.0 - 1.0 = 0.0
  weights[X] = (1 - beta) / 0.0  # Division by zero error!
```

--------------------------------

### Standard Training Loop with Class Balancing

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/usage-guide.md

Integrate class-balanced loss into a standard PyTorch training loop. Calculate class distribution from your training data to initialize the loss function.

```python
import torch
import torch.nn as nn
import torch.optim as optim
from balanced_loss import Loss

# Initialize model, optimizer, and loss
model = YourModelClass(num_classes=10)
optimizer = optim.Adam(model.parameters(), lr=1e-3)

# Calculate class distribution from training data
class_counts = torch.bincount(train_labels)
samples_per_class = class_counts.tolist()

# Create class-balanced focal loss
loss_fn = Loss(
    loss_type="focal_loss",
    beta=0.999,
    fl_gamma=2,
    samples_per_class=samples_per_class,
    class_balanced=True
)

# Training loop
num_epochs = 50
for epoch in range(num_epochs):
    total_loss = 0
    
    for batch_images, batch_labels in train_dataloader:
        # Forward pass
        logits = model(batch_images)
        
        # Compute loss
        loss = loss_fn(logits, batch_labels)
        
        # Backward pass and optimization
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
        
        total_loss += loss.item()
    
    avg_loss = total_loss / len(train_dataloader)
    print(f"Epoch {epoch+1}/{num_epochs}: loss={avg_loss:.4f}")
```

--------------------------------

### Import only focal_loss function

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/modules-and-exports.md

Import only the focal_loss function for scenarios requiring lower-level control over focal loss computation.

```python
from balanced_loss.losses import focal_loss
```

--------------------------------

### Supported Loss Types

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/usage-guide.md

Instantiate different loss types supported by the library, including cross-entropy, focal loss, binary cross-entropy, and softmax binary cross-entropy. All loss types support class balancing.

```python
import torch
from balanced_loss import Loss

logits = torch.tensor([[0.78, 0.1, 0.05], [0.5, 0.3, 0.2]])
labels = torch.tensor([0, 2])
samples_per_class = [30, 100, 25]

# 1. Cross-Entropy Loss (recommended for multi-class)
ce_loss = Loss(loss_type="cross_entropy")

# 2. Focal Loss (recommended for imbalanced)
focal_loss = Loss(loss_type="focal_loss", fl_gamma=2)

# 3. Binary Cross-Entropy (for multi-label)
bce_loss = Loss(loss_type="binary_cross_entropy")

# 4. Softmax Binary Cross-Entropy (hybrid)
sbce_loss = Loss(loss_type="softmax_binary_cross_entropy")

# All support class balancing
cb_focal = Loss(
    loss_type="focal_loss",
    samples_per_class=samples_per_class,
    class_balanced=True
)
```

--------------------------------

### Numerical Stability in Focal Loss Calculation

Source: https://github.com/fcakyon/balanced-loss/blob/main/_autodocs/implementation-details.md

Illustrates the use of log-domain arithmetic for numerically stable calculation of the focal loss modulator, preventing overflow with large logit values.

```python
# Avoid: exp(-gamma * logits) which overflows for large logits
# Instead: use log-sum-exp stability

modulator = torch.exp(
    -gamma * labels * logits - 
    gamma * torch.log(1 + torch.exp(-1.0 * logits))
)
# This is numerically stable for all logit values
```