TorchMetrics

https://lightning.ai/docs/torchmetrics/1.8.2
Admin
TorchMetrics is a collection of 100+ PyTorch metrics implementations with a standardized interface...
Tokens:4,136
Snippets:9
Trust Score:9.8
Update:11 months ago
Show doc for...
Tokens:
Raw
### torchmetrics.Metric Class

Source: https://lightning.ai/docs/torchmetrics/1.8.2/docs/torchmetrics/stable

Documentation for the base `Metric` class in TorchMetrics, including its methods for state management, computation, and integration.

```APIDOC
## `torchmetrics.Metric` Class

### Description

The base `Metric` class serves as the foundation for all metrics within the TorchMetrics library. It provides a standardized interface for adding state, updating metrics, computing results, and managing metric configurations.

### Methods

- **`add_state(name: str, default: Any, dist_reduce_fx: Optional[Callable] = None)`**
  Adds a new state variable to the metric.

- **`clone()`**
  Creates a deep copy of the metric instance.

- **`compute()`**
  Computes the final metric value based on the current state.

- **`double()`**
  Converts all state tensors to `torch.float64`.

- **`float()`**
  Converts all state tensors to `torch.float32`.

- **`forward()`**
  Convenience method that calls `update` and then `compute`.

- **`half()`**
  Converts all state tensors to `torch.float16`.

- **`merge_state(other: Metric)`**
  Merges the state from another metric instance into the current one.

- **`persistent()`**
  Marks the metric to be included in the state dict, even if it's not persistent by default.

- **`plot(val: Optional[Tensor] = None, ax: Optional[matplotlib.axes.Axes] = None)`**
  Plots the metric value. Requires `matplotlib` to be installed.

- **`reset()`**
  Resets the metric's state to its initial values.

- **`set_dtype(dtype: torch.dtype)`**
  Sets the data type for all state tensors.

- **`state_dict()`**
  Returns a dictionary containing the current state of the metric.

- **`sync()`**
  Synchronizes the metric state across all distributed processes.

- **`sync_context()`**
  Context manager for synchronization.

- **`type(new_type: torch.dtype)`**
  Converts the metric's state tensors to the specified `torch.dtype`.

- **`unsync()`**
  Disables synchronization of the metric state across distributed processes.

- **`update(preds: Tensor, target: Tensor)`**
  Updates the metric's state with new predictions and targets.

### Attributes

- **`device`** (torch.device): The device the metric is currently on.
- **`dtype`** (torch.dtype): The data type of the metric's state tensors.
- **`metric_state`** (dict): A dictionary holding the state variables of the metric.
- **`update_called`** (bool): Flag indicating if `update` has been called.
- **`update_count`** (int): The number of times `update` has been called.

### Example Usage

```python
import torch
from torchmetrics import Accuracy

# Initialize the metric
metric = Accuracy(task="binary")

# Update the metric with some data
preds = torch.tensor([0, 1, 0, 1])
target = torch.tensor([0, 1, 0, 0])
metric.update(preds, target)

# Compute the final result
accuracy = metric.compute()
print(f"Accuracy: {accuracy}")

# Reset the metric state
metric.reset()
```
```

--------------------------------

### Audio Metrics

Source: https://lightning.ai/docs/torchmetrics/1.8.2/docs/torchmetrics/stable

API reference for audio-related metrics, including C-SI-SNR, DNSMOS, NISQA v2.0, PESQ, PIT, SI-SDR, SI-SNR, STOI, SDR, SNR, SA-SDR, and SRMR, with Module and Functional Interfaces.

```APIDOC
## Audio Metrics

### Description
API reference for audio-related metrics, including C-SI-SNR, DNSMOS, NISQA v2.0, PESQ, PIT, SI-SDR, SI-SNR, STOI, SDR, SNR, SA-SDR, and SRMR, with Module and Functional Interfaces.

### Module Interface
- **Complex Scale-Invariant Signal-to-Noise Ratio (C-SI-SNR)**: Module Interface
- **Deep Noise Suppression Mean Opinion Score (DNSMOS)**: Module Interface
- **Non-Intrusive Speech Quality Assessment (NISQA v2.0)**: Module Interface
- **Perceptual Evaluation of Speech Quality (PESQ)**: Module Interface
- **Permutation Invariant Training (PIT)**: Module Interface
- **Scale-Invariant Signal-to-Distortion Ratio (SI-SDR)**: Module Interface
- **Scale-Invariant Signal-to-Noise Ratio (SI-SNR)**: Module Interface
- **Short-Time Objective Intelligibility (STOI)**: Module Interface
- **Signal to Distortion Ratio (SDR)**: Module Interface
- **Signal-to-Noise Ratio (SNR)**: Module Interface
- **Source Aggregated Signal-to-Distortion Ratio (SA-SDR)**: Module Interface
- **Speech-to-Reverberation Modulation Energy Ratio (SRMR)**: Module Interface

### Functional Interface
- **Complex Scale-Invariant Signal-to-Noise Ratio (C-SI-SNR)**: Functional Interface
- **Deep Noise Suppression Mean Opinion Score (DNSMOS)**: Functional Interface
- **Non-Intrusive Speech Quality Assessment (NISQA v2.0)**: Functional Interface
- **Perceptual Evaluation of Speech Quality (PESQ)**: Functional Interface
- **Permutation Invariant Training (PIT)**: Functional Interface
- **Scale-Invariant Signal-to-Distortion Ratio (SI-SDR)**: Functional Interface
- **Scale-Invariant Signal-to-Noise Ratio (SI-SNR)**: Functional Interface
- **Short-Time Objective Intelligibility (STOI)**: Functional Interface
- **Signal to Distortion Ratio (SDR)**: Functional Interface
- **Signal-to-Noise Ratio (SNR)**: Functional Interface
- **Source Aggregated Signal-to-Distortion Ratio (SA-SDR)**: Functional Interface
- **Speech-to-Reverberation Modulation Energy Ratio (SRMR)**: Functional Interface
```

--------------------------------

### Aggregation Metrics

Source: https://lightning.ai/docs/torchmetrics/1.8.2/docs/torchmetrics/stable

Documentation for aggregation operations like Concatenation, Maximum, Mean, Minimum, Running Mean, Running Sum, and Sum, including their Module Interfaces.

```APIDOC
## Aggregation Metrics

### Description
Documentation for aggregation operations like Concatenation, Maximum, Mean, Minimum, Running Mean, Running Sum, and Sum, including their Module Interfaces.

### Module Interface
- **Concatenation**: Module Interface
- **Maximum**: Module Interface
- **Mean**: Module Interface
- **Minimum**: Module Interface
- **Running Mean**: Module Interface
- **Running Sum**: Module Interface
- **Sum**: Module Interface
```

--------------------------------

### torchmetrics.utilities.exceptions

Source: https://lightning.ai/docs/torchmetrics/1.8.2/docs/torchmetrics/stable

Custom exception and warning classes for TorchMetrics to handle user-specific errors and provide informative feedback.

```APIDOC
## `torchmetrics.utilities.exceptions` Module

### Description

This module defines custom exception and warning classes that are specific to the TorchMetrics library. These are used to signal user-related errors or potential issues, providing clearer and more context-specific feedback than standard Python exceptions.

### Classes

- **`TorchMetricsUserError`** (Exception)
  Custom exception class for errors directly caused by user input or incorrect usage of the library.

- **`TorchMetricsUserWarning`** (Warning)
  Custom warning class for issues that do not necessarily stop execution but indicate potential problems or deviations from best practices.

### Example Usage

```python
from torchmetrics.utilities.exceptions import TorchMetricsUserError, TorchMetricsUserWarning
import warnings

def process_data(data):
    if data is None:
        raise TorchMetricsUserError("Input data cannot be None.")
    if len(data) < 5:
        warnings.warn("Input data is shorter than expected, results might be less reliable.", TorchMetricsUserWarning)
    # Process data...
    return True

try:
    process_data(None)
except TorchMetricsUserError as e:
    print(f"Caught an error: {e}")

with warnings.catch_warnings(record=True) as w:
    warnings.simplefilter("always")
    process_data([1, 2, 3])
    if w:
        print(f"Caught a warning: {w[-1].message}")
```
```

--------------------------------

### torchmetrics.utilities.distributed

Source: https://lightning.ai/docs/torchmetrics/1.8.2/docs/torchmetrics/stable

Utility functions for distributed computation within TorchMetrics, specifically for gathering tensors across processes.

```APIDOC
## `torchmetrics.utilities.distributed` Module

### Description

This module contains utility functions that facilitate distributed computing, primarily focusing on how tensors are collected and synchronized across multiple processes in a distributed environment.

### Functions

- **`gather_all_tensors(tensor: Tensor, async_op: bool = False, sync_grads: bool = False)`**
  Gathers a tensor from all processes in the current distributed group.

### Example Usage

```python
import torch
import torch.distributed as dist
from torchmetrics.utilities.distributed import gather_all_tensors

# Assume distributed environment is initialized
# dist.init_process_group(backend='nccl')

# Example tensor on the current process
my_tensor = torch.tensor([1, 2, 3]).to(dist.get_rank())

# Gather the tensor from all processes
# gathered_tensors = gather_all_tensors(my_tensor)
# print(f"Rank {dist.get_rank()}: Gathered tensors: {gathered_tensors}")

# In a real distributed setup, you would call this after initializing the process group.
# For demonstration, this code won't run without a distributed setup.
```
```

--------------------------------

### torchmetrics.utilities.data

Source: https://lightning.ai/docs/torchmetrics/1.8.2/docs/torchmetrics/stable

Utility functions for data manipulation within TorchMetrics, including selection, categorization, and dimension-wise operations.

```APIDOC
## `torchmetrics.utilities.data` Module

### Description

This module provides several utility functions designed for common data processing tasks encountered when working with metrics. These functions help in preparing data, selecting specific elements, and performing aggregations across dimensions.

### Functions

- **`select_topk(input: Tensor, k: int, dim: int = -1)`**
  Selects the top `k` elements along a specified dimension.

- **`to_categorical(tensor: Tensor, num_classes: int)`**
  Converts a tensor of class indices to a one-hot encoded tensor.

- **`to_onehot(tensor: Tensor, num_classes: int)`**
  Converts a tensor of class indices to a one-hot encoded tensor (alias for `to_categorical`).

- **`dim_zero_cat(tensors: List[Tensor])`**
  Concatenates a list of tensors along dimension 0.

- **`dim_zero_max(tensors: List[Tensor])`**
  Computes the element-wise maximum of a list of tensors along dimension 0.

- **`dim_zero_mean(tensors: List[Tensor])`**
  Computes the element-wise mean of a list of tensors along dimension 0.

- **`dim_zero_min(tensors: List[Tensor])`**
  Computes the element-wise minimum of a list of tensors along dimension 0.

- **`dim_zero_sum(tensors: List[Tensor])`**
  Computes the element-wise sum of a list of tensors along dimension 0.

### Example Usage

```python
import torch
from torchmetrics.utilities.data import select_topk, dim_zero_mean

# Example for select_topk
scores = torch.randn(5, 3)
top2 = select_topk(scores, k=2, dim=-1)
print("Top 2 elements:", top2)

# Example for dim_zero_mean
tensor_list = [torch.tensor([1, 2]), torch.tensor([3, 4])]
mean_tensor = dim_zero_mean(tensor_list)
print("Mean along dim 0:", mean_tensor)
```
```

--------------------------------

### Classification Metrics

Source: https://lightning.ai/docs/torchmetrics/1.8.2/docs/torchmetrics/stable

API reference for classification metrics, including Accuracy, AUROC, Average Precision, Calibration Error, Cohen Kappa, Confusion Matrix, Coverage Error, EER, Exact Match, F-1 Score, F-Beta Score, Group Fairness, Hamming Distance, Hinge Loss, Jaccard Index, Label Ranking Average Precision, Label Ranking Loss, Log AUC, Matthews Correlation Coefficient, Negative Predictive Value, Precision, Precision At Fixed Recall, Precision Recall Curve, Recall, Recall At Fixed Precision, ROC, Sensitivity At Specificity, Specificity, Specificity At Sensitivity, and Stat Scores. Includes Module and Functional Interfaces.

```APIDOC
## Classification Metrics

### Description
API reference for classification metrics, including Accuracy, AUROC, Average Precision, Calibration Error, Cohen Kappa, Confusion Matrix, Coverage Error, EER, Exact Match, F-1 Score, F-Beta Score, Group Fairness, Hamming Distance, Hinge Loss, Jaccard Index, Label Ranking Average Precision, Label Ranking Loss, Log AUC, Matthews Correlation Coefficient, Negative Predictive Value, Precision, Precision At Fixed Recall, Precision Recall Curve, Recall, Recall At Fixed Precision, ROC, Sensitivity At Specificity, Specificity, Specificity At Sensitivity, and Stat Scores. Includes Module and Functional Interfaces.

### Module Interface
- **Accuracy**: Module Interface
- **AUROC**: Module Interface
- **Average Precision**: Module Interface
- **Calibration Error**: Module Interface
- **Cohen Kappa**: Module Interface
- **Confusion Matrix**: Module Interface
- **Coverage Error**: Module Interface
- **Expected Error Rate (EER)**: Module Interface
- **Exact Match**: Module Interface
- **F-1 Score**: Module Interface
- **F-Beta Score**: Module Interface
- **Group Fairness**: Module Interface
- **Hamming Distance**: Module Interface
- **Hinge Loss**: Module Interface
- **Jaccard Index**: Module Interface
- **Label Ranking Average Precision**: Module Interface
- **Label Ranking Loss**: Module Interface
- **Log AUC**: Module Interface
- **Matthews Correlation Coefficient**: Module Interface
- **Negative Predictive Value**: Module Interface
- **Precision**: Module Interface
- **Precision At Fixed Recall**: Module Interface
- **Precision Recall Curve**: Module Interface
- **Recall**: Module Interface
- **Recall At Fixed Precision**: Module Interface
- **ROC**: Module Interface
- **Sensitivity At Specificity**: Module Interface
- **Specificity**: Module Interface
- **Specificity At Sensitivity**: Module Interface
- **Stat Scores**: Module Interface

### Functional Interface
- **Accuracy**: Functional Interface
- **AUROC**: Functional Interface
- **Average Precision**: Functional Interface
- **Calibration Error**: Functional Interface
- **Cohen Kappa**: Functional Interface
- **Confusion Matrix**: Functional Interface
- **Coverage Error**: Functional Interface
- **Expected Error Rate (EER)**: Functional Interface
- **Exact Match**: Functional Interface
- **F-1 Score**: Functional Interface
- **F-Beta Score**: Functional Interface
- **Group Fairness**: Functional Interface
- **Hamming Distance**: Functional Interface
- **Hinge Loss**: Functional Interface
- **Jaccard Index**: Functional Interface
- **Label Ranking Average Precision**: Functional Interface
- **Label Ranking Loss**: Functional Interface
- **Log AUC**: Functional Interface
- **Matthews Correlation Coefficient**: Functional Interface
- **Negative Predictive Value**: Functional Interface
- **Precision**: Functional Interface
- **Precision At Fixed Recall**: Functional Interface
- **Precision Recall Curve**: Functional Interface
- **Recall**: Functional Interface
- **Recall At Fixed Precision**: Functional Interface
- **ROC**: Functional Interface
- **Sensitivity At Specificity**: Functional Interface
- **Specificity**: Functional Interface
- **Specificity At Sensitivity**: Functional Interface
- **Stat Scores**: Functional Interface
```

--------------------------------

### Clustering Metrics

Source: https://lightning.ai/docs/torchmetrics/1.8.2/docs/torchmetrics/stable

API reference for clustering metrics, including Adjusted Mutual Information Score, Adjusted Rand Score, and Calinski Harabasz Score, with Module and Functional Interfaces.

```APIDOC
## Clustering Metrics

### Description
API reference for clustering metrics, including Adjusted Mutual Information Score, Adjusted Rand Score, and Calinski Harabasz Score, with Module and Functional Interfaces.

### Module Interface
- **Adjusted Mutual Information Score**: Module Interface
- **Adjusted Rand Score**: Module Interface
- **Calinski Harabasz Score**: Module Interface

### Functional Interface
- **Adjusted Mutual Information Score**: Functional Interface
- **Adjusted Rand Score**: Functional Interface
- **Calinski Harabasz Score**: Functional Interface
```
Tokens:
Raw