### ProxyNCALoss Optimizer Setup

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Example of setting up an optimizer for ProxyNCALoss. The loss function's parameters must be passed to the optimizer.

```python
loss_func = losses.ProxyNCALoss(...).to(torch.device('cuda'))
loss_optimizer = torch.optim.SGD(loss_func.parameters(), lr=0.01)
# then during training:
loss_optimizer.step()
```

--------------------------------

### SphereFaceLoss Optimizer Setup

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Example of setting up an optimizer for SphereFaceLoss. The loss function's parameters must be passed to the optimizer.

```python
loss_func = losses.SphereFaceLoss(...).to(torch.device('cuda'))
loss_optimizer = torch.optim.SGD(loss_func.parameters(), lr=0.01)
# then during training:
loss_optimizer.step()
```

--------------------------------

### ProxyAnchorLoss Optimizer Setup

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Example of setting up an optimizer for ProxyAnchorLoss. The loss function's parameters must be passed to the optimizer.

```python
loss_func = losses.ProxyAnchorLoss(...).to(torch.device('cuda'))
loss_optimizer = torch.optim.SGD(loss_func.parameters(), lr=0.01)
# then during training:
loss_optimizer.step()
```

--------------------------------

### Install pytorch-metric-learning on Windows

Source: https://kevinmusgrave.github.io/pytorch-metric-learning

Specific installation command for Windows, including torch and torchvision versions, followed by the library installation.

```bash
pip install torch===1.6.0 torchvision===0.7.0 -f https://download.pytorch.org/whl/torch_stable.html
pip install pytorch-metric-learning

```

--------------------------------

### Install pytorch-metric-learning with hooks (GPU FAISS)

Source: https://kevinmusgrave.github.io/pytorch-metric-learning

Install the library with evaluation and logging capabilities. This option installs the unofficial pypi version of faiss-gpu.

```bash
pip install pytorch-metric-learning[with-hooks]

```

--------------------------------

### Install pytorch-metric-learning with hooks (CPU FAISS)

Source: https://kevinmusgrave.github.io/pytorch-metric-learning

Install the library with evaluation and logging capabilities for CPU. This option installs the unofficial pypi version of faiss-cpu.

```bash
pip install pytorch-metric-learning[with-hooks-cpu]

```

--------------------------------

### Install pytorch-metric-learning using pip

Source: https://kevinmusgrave.github.io/pytorch-metric-learning

Use this command to install the latest stable version of the library.

```bash
pip install pytorch-metric-learning

```

--------------------------------

### Install latest dev version of pytorch-metric-learning

Source: https://kevinmusgrave.github.io/pytorch-metric-learning

Install the latest development version using the --pre flag.

```bash
pip install pytorch-metric-learning --pre

```

--------------------------------

### Initialize ProxyNCALoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate the ProxyNCALoss. Similar to ProxyAnchorLoss, this loss requires an optimizer setup.

```python
losses.ProxyNCALoss(num_classes, embedding_size, softmax_scale=1, **kwargs)
```

--------------------------------

### Install pytorch-metric-learning using Conda

Source: https://kevinmusgrave.github.io/pytorch-metric-learning

Install the library using Conda from the conda-forge channel.

```bash
conda install -c conda-forge pytorch-metric-learning

```

--------------------------------

### CosFaceLoss Initialization and Optimizer Setup

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Initializes CosFaceLoss with number of classes and embedding size. Requires an optimizer to be explicitly created and stepped.

```python
losses.CosFaceLoss(num_classes, embedding_size, margin=0.35, scale=64, **kwargs)
```

```python
loss_func = losses.CosFaceLoss(...).to(torch.device('cuda'))
loss_optimizer = torch.optim.SGD(loss_func.parameters(), lr=0.01)
# then during training:
loss_optimizer.step()
```

--------------------------------

### HDCMiner example with another miner

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/miners

Example demonstrating how to pass the output of one miner (minerA) to HDCMiner (minerB) using set_idx_externally.

```python
minerA = miners.MultiSimilarityMiner(epsilon=0.1)
minerB = miners.HDCMiner(filter_percentage=0.25)

hard_pairs = minerA(embeddings, labels)
minerB.set_idx_externally(hard_pairs, labels)
very_hard_pairs = minerB(embeddings, labels)
```

--------------------------------

### Basic Miner Usage Example

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/miners

Demonstrates how to instantiate and use a miner with a loss function. Ensure embeddings and labels are available.

```python
from pytorch_metric_learning import miners, losses
miner_func = miners.SomeMiner()
loss_func = losses.SomeLoss()
miner_output = miner_func(embeddings, labels)
losses = loss_func(embeddings, labels, miner_output)
```

--------------------------------

### SoftTripleLoss Optimizer Setup

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

When using SoftTripleLoss, an optimizer must be explicitly created and passed the loss function's parameters. This is demonstrated with SGD, and the optimizer's step() method should be called during training.

```python
loss_func = losses.SoftTripleLoss(...).to(torch.device('cuda'))
loss_optimizer = torch.optim.SGD(loss_func.parameters(), lr=0.01)
# then during training:
loss_optimizer.step()
```

--------------------------------

### Example Usage of P2SGradLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Demonstrates how to use P2SGradLoss by initializing the loss function and then calling it with embeddings and labels.

```python
loss_fn = P2SGradLoss(128, 10)
loss = loss_fn(embeddings, labels)
```

--------------------------------

### FaissKNN Example Usage

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/inference_models

Example of using FaissKNN with a custom index initialization (IndexFlatIP) and specifying GPUs. Demonstrates how to call the KNN function with query, k, and reference embeddings.

```python
# use faiss.IndexFlatIP on 3 gpus
knn_func = FaissKNN(index_init_fn=faiss.IndexFlatIP, gpus=[0,1,2])

# query = query embeddings 
# k = the k in k-nearest-neighbors
# reference = the embeddings to search
# last argument is whether or not query and reference share datapoints
distances, indices = knn_func(query, k, references, False)
```

--------------------------------

### FaissKMeans Example Usage

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/inference_models

Example of initializing FaissKMeans with specific parameters and then using it to cluster embeddings into a specified number of groups.

```python
kmeans_func = FaissKMeans(niter=100, verbose=True, gpu=True)

# cluster into 10 groups
cluster_assignments = kmeans_func(embeddings, 10)
```

--------------------------------

### Integrating Reducers into Loss Functions

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/reducers

Demonstrates how to instantiate a reducer and pass it to a loss function. This setup is used within the training loop for calculating the final loss.

```python
from pytorch_metric_learning import losses, reducers
reducer = reducers.SomeReducer()
loss_func = losses.SomeLoss(reducer=reducer)
loss = loss_func(embeddings, labels) # in your training for-loop

```

--------------------------------

### Example Accuracy Metrics Output

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/testers

This is an example of the dictionary structure for accuracy metrics computed by the tester, organized by dataset split.

```json
{"train": {"AMI_level0": 0.53, ...}, "val": {"AMI_level0": 0.44, ...}}
```

--------------------------------

### Example Pair Miner Usage

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/extend/miners

Instantiate and use the custom ExamplePairMiner. The miner can be called directly with embeddings and labels. If ref_emb and ref_labels are not provided, they default to embeddings and labels.

```python
import torch

miner = ExamplePairMiner()
embeddings = torch.randn(128, 512)
labels = torch.randint(0, 10, size=(128,))
pairs = miner(embeddings, labels)
```

--------------------------------

### Optimizer Setup for NormalizedSoftmaxLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

NormalizedSoftmaxLoss requires an optimizer. Create an optimizer and pass the loss function's parameters to it. Remember to call loss_optimizer.step() during training.

```python
loss_func = losses.NormalizedSoftmaxLoss(...).to(torch.device('cuda'))
loss_optimizer = torch.optim.SGD(loss_func.parameters(), lr=0.01)
# then during training:
loss_optimizer.step()
```

--------------------------------

### Example Loss Dictionary for DivisorReducer

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/reducers

Illustrates the structure of a loss dictionary that includes a 'divisor' key, as expected by the DivisorReducer. This example is from ProxyAnchorLoss.

```python
loss_dict = {
    "pos_loss": {
        "losses": pos_term.squeeze(0),
        "indices": loss_indices,
        "reduction_type": "element",
        "divisor": len(with_pos_proxies),
    },
    "neg_loss": {
        "losses": neg_term.squeeze(0),
        "indices": loss_indices,
        "reduction_type": "element",
        "divisor": self.num_classes,
    },
}

```

--------------------------------

### ArcFaceLoss Initialization and Optimizer Setup

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Initialize ArcFaceLoss with the number of classes and embedding size. It requires an optimizer to be set up separately for its parameters, and it uses CosineSimilarity as the default distance.

```python
loss_func = losses.ArcFaceLoss(num_classes, embedding_size, margin=28.6, scale=64, **kwargs)

```

```python
loss_func = losses.ArcFaceLoss(...).to(torch.device('cuda'))
loss_optimizer = torch.optim.SGD(loss_func.parameters(), lr=0.01)
# then during training:
loss_optimizer.step()

```

--------------------------------

### Example Usage of MultipleReducers with ContrastiveLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/reducers

Configures ContrastiveLoss to use different reducers for its positive and negative pair losses. A ThresholdReducer is used for positive pairs, and a MeanReducer for negative pairs.

```python
from pytorch_metric_learning.losses import ContrastiveLoss
from pytorch_metric_learning.reducers import MultipleReducers, ThresholdReducer, MeanReducer
reducer_dict = {"pos_loss": ThresholdReducer(0.1), "neg_loss": MeanReducer()}
reducer = MultipleReducers(reducer_dict)
loss_func = ContrastiveLoss(reducer=reducer)

```

--------------------------------

### Embeddings ordered as triplets example

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/miners

Example structure for embeddings already packaged as triplets for a batch size of 6.

```python
torch.stack([anchor1, positive1, negative1, anchor2, positive2, negative2], dim=0)
```

--------------------------------

### Basic Trainer Usage

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/trainers

Instantiate a trainer and run the training process for a specified number of epochs. Ensure necessary imports are included.

```python
from pytorch_metric_learning import trainers
t = trainers.SomeTrainingFunction(*args, **kwargs)
t.train(num_epochs=10)
```

--------------------------------

### Initialize ProxyAnchorLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate the ProxyAnchorLoss. This loss requires an optimizer to be created and passed the loss's parameters.

```python
losses.ProxyAnchorLoss(num_classes, embedding_size, margin = 0.1, alpha = 32, **kwargs)
```

--------------------------------

### CustomKNN Example Usage

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/inference_models

Example of initializing CustomKNN with SNRDistance and then calling it to find nearest neighbors. This demonstrates using a custom distance metric for KNN.

```python
from pytorch_metric_learning.distances import SNRDistance
from pytorch_metric_learning.utils.inference import CustomKNN

knn_func = CustomKNN(SNRDistance())
distances, indices = knn_func(query, k, references, False)
```

--------------------------------

### Initialize NTXentLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate NTXentLoss, also known as InfoNCE. This loss is a generalization of NPairsLoss and is often used in self-supervised learning.

```python
losses.NTXentLoss(temperature=0.07, **kwargs)
```

--------------------------------

### Initialize InstanceLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate InstanceLoss with a gamma parameter for scaling the cosine similarity matrix.

```python
losses.InstanceLoss(gamma=64, **kwargs)
```

--------------------------------

### Initialize DynamicSoftMarginLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate DynamicSoftMarginLoss with parameters for minimum value, number of bins, and momentum.

```python
losses.DynamicSoftMarginLoss(min_val=-2.0, num_bins=10, momentum=0.01, **kwargs)
```

--------------------------------

### End-of-Testing Hook Example

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/testers

Define a function to be called after testing is complete. This hook can access and process the computed accuracy metrics.

```python
def end_of_testing_hook(tester):
    print(tester.all_accuracies)
```

--------------------------------

### Use DistributedMinerWrapper and DistributedLossWrapper

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/distributed

Use the wrapped miner to get tuples, then pass these tuples to the wrapped loss function for training.

```python
# in each process
tuples = miner(embeddings, labels)
# pass into a DistributedLossWrapper
loss = loss_func(embeddings, labels, indices_tuple)
```

--------------------------------

### Instantiate and Use ManifoldLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiates the ManifoldLoss and shows example usage with embeddings. It can optionally use specified indices as cluster centers.

```python
loss_fn = ManifoldLoss(128)

# use random cluster centers
loss = loss_fn(embeddings)
# or specify indices of embeddings to use as cluster centers
loss = loss_fn(embeddings, indices_tuple=indices)
```

--------------------------------

### Loading StanfordOnlineProducts Dataset Splits - pytorch-metric-learning

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/datasets

Demonstrates how to load different splits of the StanfordOnlineProducts dataset. Set download=True for the initial load. The download process can take a significant amount of time.

```python
# The download takes a while - the dataset is very large
train_dataset = StanfordOnlineProducts(root="data", 
    split="train", 
    transform=None, 
    target_transform=None, 
    download=True
)
# No need to download the dataset after it is already downladed
test_dataset = StanfordOnlineProducts(root="data", 
    split="test", 
    transform=None, 
    target_transform=None, 
    download=False
)
train_and_test_dataset = StanfordOnlineProducts(root="data", 
    split="train+test", 
    transform=None, 
    target_transform=None, 
    download=False
) 

```

--------------------------------

### Self-Supervised Training Loop

Source: https://kevinmusgrave.github.io/pytorch-metric-learning

Example training loop for self-supervised learning using SelfSupervisedLoss. It computes embeddings for both original and augmented data.

```python
# your training for-loop
for i, data in enumerate(dataloader):
    optimizer.zero_grad()
    embeddings = your_model(data)
    augmented = your_model(your_augmentation(data))
    loss = loss_func(embeddings, augmented)
    loss.backward()
    optimizer.step()
```

--------------------------------

### Initialize SphereFaceLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate SphereFaceLoss with required parameters. This loss requires an optimizer to be created and passed its parameters.

```python
losses.SphereFaceLoss(num_classes, 
                    embedding_size, 
                    margin=4, 
                    scale=1, 
                    **kwargs)
```

--------------------------------

### Apply RegularFaceRegularizer to ArcFaceLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/regularizers

An example of how to instantiate and pass a weight regularizer to a loss function. Ensure the regularizer is compatible with the chosen loss.

```python
from pytorch_metric_learning import losses, regularizers
R = regularizers.RegularFaceRegularizer()
loss = losses.ArcFaceLoss(margin=30, num_classes=100, embedding_size=128, weight_regularizer=R)
```

--------------------------------

### Initialize AccuracyCalculator

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/accuracy_calculation

Instantiate the AccuracyCalculator with optional parameters to customize metric calculation. Default metrics are calculated if 'include' is empty.

```python
from pytorch_metric_learning.utils.accuracy_calculator import AccuracyCalculator
AccuracyCalculator(include=(),
                    exclude=(),
                    avg_of_avgs=False,
                    return_per_class=False,
                    k=None,
                    label_comparison_fn=None,
                    device=None,
                    knn_func=None,
                    kmeans_func=None)
```

--------------------------------

### Get Outliers and Dominant Centers from SubCenterArcFaceLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Compute outliers and dominant centers using the get_outliers method. This method can optionally return dominant centers.

```python
outliers, dominant_centers = loss_func.get_outliers(
    embeddings, labels, threshold=75, return_dominant_centers=True
)
```

--------------------------------

### Initialize FaissKNN

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/inference_models

Instantiate FaissKNN for efficient k-nearest neighbor search using the Faiss library. Configure index initialization and GPU usage.

```python
from pytorch_metric_learning.utils.inference import FaissKNN
FaissKNN(reset_before=True,
            reset_after=True, 
            index_init_fn=None, 
            gpus=None)
```

--------------------------------

### Initialize PNPLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate the PNPLoss with specified parameters. The paper uses b=2, alpha=8, anneal=0.01, and variant='Dq'.

```python
losses.PNPLoss(b=2, alpha=1, anneal=0.01, variant="O", **kwargs)
```

--------------------------------

### Custom Floating Point Label Comparison Function

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/accuracy_calculation

Define a custom function for comparing floating-point labels. This example uses a tolerance threshold to determine equality.

```python
def label_comparison_fn(x, y):
    return torch.abs(x - y) < 1

# these are valid labels
labels = torch.tensor([
    10.0,
    0.03,
    0.04,
    0.05,
])
```

--------------------------------

### Initialize FastAPLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate FastAPLoss with the number of bins for histogram calculation.

```python
losses.FastAPLoss(num_bins=10, **kwargs)
```

--------------------------------

### StanfordOnlineProducts Dataset Initialization - pytorch-metric-learning

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/datasets

Initializes the StanfordOnlineProducts dataset. It supports loading 'train', 'test', or 'train+test' splits. Note that this dataset is very large.

```python
datasets.StanfordOnlineProducts(*args, **kwargs)

```

--------------------------------

### Instantiate SoftTripleLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Configure SoftTripleLoss with the number of classes, embedding size, and other hyperparameters like centers_per_class, la, gamma, and margin. This loss function requires an optimizer to be passed its parameters.

```python
losses.SoftTripleLoss(num_classes, 
                    embedding_size, 
                    centers_per_class=10, 
                    la=20, 
                    gamma=0.1, 
                    margin=0.01,
                    **kwargs)
```

--------------------------------

### TorchInitWrapper for Weight Initialization

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/common_functions

Shows how to use TorchInitWrapper to convert PyTorch weight initialization functions into a class-based format. This wrapper can be used within loss functions.

```python
from pytorch_metric_learning.utils import common_functions as c_f
import torch

# use kaiming_uniform, with a=1 and mode='fan_out'
weight_init_func = c_f.TorchInitWrapper(torch.nn.kaiming_uniform_, a=1, mode='fan_out')
loss_func = SomeClassificationLoss(..., weight_init_func=weight_init_func)
```

--------------------------------

### Retrieve Loss History from HookContainer

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/logging_presets

Shows how to access recorded loss histories from the `HookContainer` instance, either to get all loss histories or to retrieve specific ones by name.

```python
# Get a dictionary mapping from loss names to lists
loss_histories = hooks.get_loss_history() 
```

```python
# You can also specify which loss histories you want
# It will still return a dictionary. In this case, the dictionary will contain only "total_loss"
loss_histories = hooks.get_loss_history(loss_names=["total_loss"])
```

--------------------------------

### DoNothingReducer Initialization

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/reducers

Initializes the DoNothingReducer, which bypasses any reduction and returns the input loss dictionary unchanged. Useful when no aggregation is desired.

```python
reducers.DoNothingReducer(**kwargs)

```

--------------------------------

### Example Usage of SelfSupervisedLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Wrap an existing loss function like TripletMarginLoss with SelfSupervisedLoss to handle label creation internally. This simplifies the training process for self-supervised metric learning tasks.

```python
loss_fn = losses.TripletMarginLoss()
loss_fn = SelfSupervisedLoss(loss_fn)
loss = loss_fn(embeddings, ref_emb)
```

--------------------------------

### Integrate Logging Presets with Trainer and Tester

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/logging_presets

Demonstrates how to set up `record-keeper` and `tensorboard` logging, create a `HookContainer`, and integrate its hooks into `GlobalEmbeddingSpaceTester` and `MetricLossOnly` trainer for comprehensive logging and model saving.

```python
import pytorch_metric_learning.utils.logging_presets as LP
log_folder, tensorboard_folder = "example_logs", "example_tensorboard"
record_keeper, _, _ = LP.get_record_keeper(log_folder, tensorboard_folder)
hooks = LP.get_hook_container(record_keeper)
dataset_dict = {"val": val_dataset}
model_folder = "example_saved_models"

# Create the tester
tester = testers.GlobalEmbeddingSpaceTester(end_of_testing_hook=hooks.end_of_testing_hook)
end_of_epoch_hook = hooks.end_of_epoch_hook(tester, dataset_dict, model_folder)
trainer = trainers.MetricLossOnly(models,
                                optimizers,
                                batch_size,
                                loss_funcs,
                                mining_funcs,
                                train_dataset,
                                sampler=sampler,
                                end_of_iteration_hook=hooks.end_of_iteration_hook,
                                end_of_epoch_hook=end_of_epoch_hook)

trainer.train(num_epochs=num_epochs)
```

--------------------------------

### Customized TripletMarginLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning

Create a customized TripletMarginLoss using CosineSimilarity, ThresholdReducer, and LpRegularizer. This example uses cosine similarity, discards triplets with loss > 0.3, and applies L2 regularization to embeddings.

```python
from pytorch_metric_learning.distances import CosineSimilarity
from pytorch_metric_learning.reducers import ThresholdReducer
from pytorch_metric_learning.regularizers import LpRegularizer
from pytorch_metric_learning import losses
loss_func = losses.TripletMarginLoss(distance = CosineSimilarity(), 
                                    reducer = ThresholdReducer(high=0.3), 
                                    embedding_regularizer = LpRegularizer())
```

--------------------------------

### Initialize ThresholdConsistentMarginLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate ThresholdConsistentMarginLoss with scaling coefficients and margin parameters for positive and negative pairs. This loss is typically used as regularization.

```python
losses.ThresholdConsistentMarginLoss(
    lambda_plus=1.0, 
    lambda_minus=1.0, 
    margin_plus=0.9, 
    margin_minus=0.5,
    **kwargs
)
```

--------------------------------

### Get Accuracy Metrics

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/accuracy_calculation

Call the get_accuracy method to compute accuracy metrics. Provide query and reference embeddings along with their labels. Optional 'include' and 'exclude' parameters can override initialization settings.

```python
def get_accuracy(self, 
    query,
    query_labels,       
    reference=None,
    reference_labels=None, 
    ref_includes_query=False,
    include=(),
    exclude=()
):
# returns a dictionary mapping from metric names to accuracy values
# The default metrics are:
# "NMI" (Normalized Mutual Information)
# "AMI" (Adjusted Mutual Information)
# "precision_at_1"
# "r_precision"
# "mean_average_precision_at_r"
```

--------------------------------

### Initialize MarginLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate the MarginLoss with specified parameters. Learnable beta can be enabled.

```python
losses.MarginLoss(margin=0.2, 
                nu=0, 
                beta=1.2, 
                triplets_per_anchor="all", 
                learn_beta=False, 
                num_classes=None, 
                **kwargs)
```

--------------------------------

### Instantiate LargeMarginSoftmaxLoss and Optimizer

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiates the LargeMarginSoftmaxLoss and its required optimizer. The optimizer must be configured with the loss function's parameters.

```python
loss_func = losses.LargeMarginSoftmaxLoss(...).to(torch.device('cuda'))
loss_optimizer = torch.optim.SGD(loss_func.parameters(), lr=0.01)
# then during training:
loss_optimizer.step()
```

--------------------------------

### Example Pair Miner Implementation

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/extend/miners

Extend BaseMiner to create a custom pair miner. This miner finds pairs that violate a specified margin. Ensure the distance metric is correctly configured for inverted or non-inverted behavior.

```python
from pytorch_metric_learning.miners import BaseMiner
from pytorch_metric_learning.utils import loss_and_miner_utils as lmu

class ExamplePairMiner(BaseMiner):
    def __init__(self, margin=0.1, **kwargs):
        super().__init__(**kwargs)
        self.margin = margin

    def mine(self, embeddings, labels, ref_emb, ref_labels):
        mat = self.distance(embeddings, ref_emb)
        a1, p, a2, n = lmu.get_all_pairs_indices(labels, ref_labels)
        pos_pairs = mat[a1, p]
        neg_pairs = mat[a2, n]
        pos_mask = (
            pos_pairs < self.margin
            if self.distance.is_inverted
            else pos_pairs > self.margin
        )
        neg_mask = (
            neg_pairs > self.margin
            if self.distance.is_inverted
            else neg_pairs < self.margin
        )
        return a1[pos_mask], p[pos_mask], a2[neg_mask], n[neg_mask]
```

--------------------------------

### Adding Custom Accuracy Metrics

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/accuracy_calculation

Extend the AccuracyCalculator class to add custom accuracy metrics. Methods starting with 'calculate_' are automatically recognized. Specify dependencies on k-nn search or clustering using 'requires_knn' and 'requires_clustering'.

```python
from pytorch_metric_learning.utils import accuracy_calculator

class YourCalculator(accuracy_calculator.AccuracyCalculator):
    def calculate_precision_at_2(self, knn_labels, query_labels, **kwargs):
        return accuracy_calculator.precision_at_k(knn_labels, query_labels[:, None], 2)

    def calculate_fancy_mutual_info(self, query_labels, cluster_labels, **kwargs):
        return fancy_computations

    def requires_clustering(self):
        return super().requires_clustering() + ["fancy_mutual_info"]

    def requires_knn(self):
        return super().requires_knn() + ["precision_at_2"]

```

--------------------------------

### Initialize HistogramLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate HistogramLoss with optional number of bins and delta for histogram construction.

```python
losses.HistogramLoss(n_bins=None, delta=None)
```

--------------------------------

### Get All Embeddings with BaseTester

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/testers

Use this function to retrieve all embeddings and labels from a dataset using a given model. Ensure that the dataset and trunk_model are provided. Optional arguments include an embedder_model, collate_fn, eval mode flag, and return format.

```python
embeddings, labels = tester.get_all_embeddings(
    dataset, # Any pytorch dataset
    trunk_model, # your model
    embedder_model=None, # by default this will be a no-op
    collate_fn=None, # custom collate_fn for the dataloader
    eval=True, # set models to eval mode
    return_as_numpy=False
 )
```

--------------------------------

### BaseTrainer Initialization

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/trainers

Initialize the BaseTrainer with models, optimizers, batch size, loss functions, and dataset. Optional parameters include mining functions, iterations per epoch, data device, dtype, loss weights, sampler, collate function, learning rate schedulers, gradient clippers, and freeze configurations.

```python
trainers.BaseTrainer(models,
                    optimizers,
                    batch_size,
                    loss_funcs,
                    dataset,
                    mining_funcs=None,
                    iterations_per_epoch=None,
                    data_device=None,
                    dtype=None,
                    loss_weights=None,
                    sampler=None,
                    collate_fn=None,
                    lr_schedulers=None,
                    gradient_clippers=None,
                    freeze_these=(),
                    freeze_trunk_batchnorm=False,
                    label_hierarchy_level=0,
                    dataloader_num_workers=2,
                    data_and_label_getter=None,
                    dataset_labels=None,
                    set_min_label_to_zero=False,
                    end_of_iteration_hook=None,
                    end_of_epoch_hook=None)
```

--------------------------------

### Example Triplet Miner Implementation

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/extend/miners

Extend BaseMiner to create a custom triplet miner. This miner finds triplets where the distance between anchor and positive is too close to the distance between anchor and negative, relative to a margin. Ensure the distance metric is correctly configured.

```python
from pytorch_metric_learning.miners import BaseMiner
from pytorch_metric_learning.utils import loss_and_miner_utils as lmu

class ExampleTripletMiner(BaseMiner):
    def __init__(self, margin=0.1, **kwargs):
        super().__init__(**kwargs)
        self.margin = margin

    def mine(self, embeddings, labels, ref_emb, ref_labels):
        mat = self.distance(embeddings, ref_emb)
        a, p, n = lmu.get_all_triplets_indices(labels, ref_labels)
        pos_pairs = mat[a, p]
        neg_pairs = mat[a, n]
        triplet_margin = pos_pairs - neg_pairs if self.distance.is_inverted else neg_pairs - pos_pairs
        triplet_mask = triplet_margin <= self.margin
        return a[triplet_mask], p[triplet_mask], n[triplet_mask]
```

--------------------------------

### PerAnchorReducer

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/reducers

Converts unreduced pairs to unreduced elements. For example, NTXentLoss returns losses per positive pair. If you used PerAnchorReducer with NTXentLoss, then the losses per pair would first be converted to losses per batch element, before being passed to the inner reducer.

```APIDOC
## PerAnchorReducer

### Description
This reducer converts unreduced pairs to unreduced elements. For example, NTXentLoss returns losses per positive pair. If you used PerAnchorReducer with NTXentLoss, then the losses per pair would first be converted to losses per batch element, before being passed to the inner reducer.

### Parameters
* **reducer** : The reducer that will be fed per-element losses. The default is MeanReducer.
* **aggregation_func** : A function that takes in `(x, num_per_row)` and returns a loss per row of `x`. The default is the `aggregation_func` defined in the code snippet above. It returns the mean per row.
    * `x` is an NxN array of pairwise losses, where N is the batch size.
    * `num_per_row` is a size N array which indicates how many non-zero losses there are per-row of `x`.

### Code Example
```python
def aggregation_func(x, num_per_row):
    zero_denom = num_per_row == 0
    x = torch.sum(x, dim=1) / num_per_row
    x[zero_denom] = 0
    return x

reducers.PerAnchorReducer(reducer=None, aggregation_func=aggregation_func, **kwargs)
```
```

--------------------------------

### Initialize FaissKMeans

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/inference_models

Instantiate FaissKMeans for k-means clustering using Faiss. Any keyword arguments are passed directly to the faiss.Kmeans constructor.

```python
from pytorch_metric_learning.utils.inference import FaissKMeans
FaissKMeans(**kwargs)
```

--------------------------------

### Initialize HookContainer for Logging and Model Saving

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/logging_presets

Illustrates how to create an instance of `LP.HookContainer`, specifying parameters like `record_keeper`, `primary_metric`, `validation_split_name`, `save_models`, and `log_freq` to configure logging behavior.

```python
import pytorch_metric_learning.utils.logging_presets as LP
LP.HookContainer(record_keeper, 
    record_group_name_prefix=None, 
    primary_metric="mean_average_precision_at_r", 
    validation_split_name="val",
    save_models=True,
    log_freq=50)
```

--------------------------------

### Using the PML Logger

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/common_functions

Demonstrates how to use the default logger provided by the library. This logger is used across various components.

```python
from pytorch_metric_learning.utils import common_functions as c_f
c_f.LOGGER.info("Using the PML logger")
```

--------------------------------

### Loading CUB Dataset Splits - pytorch-metric-learning

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/datasets

Demonstrates how to load different splits of the CUB dataset. Set download=True for the initial load.

```python
train_dataset = CUB(root="data", 
    split="train", 
    transform=None, 
    target_transform=None, 
    download=True
)
# No need to download the dataset after it is already downladed
test_dataset = CUB(root="data", 
    split="test", 
    transform=None, 
    target_transform=None, 
    download=False
)
train_and_test_dataset = CUB(root="data", 
    split="train+test", 
    transform=None, 
    target_transform=None, 
    download=False
) 

```

--------------------------------

### Instantiate SmoothAPLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Initialize SmoothAPLoss with a specified temperature. This loss is suitable for large-scale image retrieval and requires labels with an equal number of elements per class.

```python
losses.SmoothAPLoss(
    temperature=0.01,
    **kwargs
)
```

--------------------------------

### Instantiate TwoStreamMetricLoss Trainer

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/trainers

Initialize the TwoStreamMetricLoss trainer with its arguments. Ensure that the models, loss functions, and miners meet the specified requirements.

```python
trainers.TwoStreamMetricLoss(*args, **kwargs)
```

--------------------------------

### Initialize HDCMiner

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/miners

Instantiate HDCMiner. This miner filters a percentage of the hardest pairs or triplets. It can use externally provided indices.

```python
miners.HDCMiner(filter_percentage=0.5, **kwargs)
```

--------------------------------

### Initialize NCALoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate the NCALoss with a specified softmax scale. This loss is based on Neighbourhood Components Analysis.

```python
losses.NCALoss(softmax_scale=1, **kwargs)
```

--------------------------------

### Initialize InferenceModel

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/inference_models

Instantiate InferenceModel with a trained trunk model. Optional parameters include an embedder, match finder, normalization settings, KNN function, data device, and data type.

```python
from pytorch_metric_learning.utils.inference import InferenceModel
InferenceModel(trunk,
                embedder=None,
                match_finder=None,
                normalize_embeddings=True,
                knn_func=None,
                data_device=None,
                dtype=None)
```

--------------------------------

### Instantiate VICRegLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate the VICRegLoss with weights for invariance, variance, and covariance terms. A small epsilon is included for numerical stability.

```python
losses.VICRegLoss(invariance_lambda=25, 
                variance_mu=25, 
                covariance_v=1, 
                eps=1e-4, 
                **kwargs)
```

--------------------------------

### AvgNonZeroReducer Initialization

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/reducers

Initializes the AvgNonZeroReducer, which computes the average of losses that are strictly greater than zero. This is equivalent to using ThresholdReducer with a low threshold of 0.

```python
reducers.AvgNonZeroReducer(**kwargs)

```

--------------------------------

### InferenceModel Methods

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/inference_models

Demonstrates common methods of the InferenceModel class, including training KNN, adding data, retrieving nearest neighbors, checking for matches, and saving/loading the KNN function.

```python
# initialize with a model
im = InferenceModel(model)

# pass in a dataset to serve as the search space for k-nn
im.train_knn(dataset)

# add another dataset to the index
im.add_to_knn(dataset2)

# get the 10 nearest neighbors of a query
distances, indices = im.get_nearest_neighbors(query, k=10)

# determine if inputs are close to each other
is_match = im.is_match(x, y)

# determine "is_match" pairwise for all elements in a batch
match_matrix = im.get_matches(x)

# save and load the knn function (which is a faiss index by default)
im.save_knn_func("filename.index")
im.load_knn_func("filename.index")
```

--------------------------------

### Initialize BaseTester

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/testers

Instantiate the BaseTester class with various configuration options. These parameters control normalization, batching, PCA, device, data types, and hooks for testing.

```python
testers.BaseTester(
                    normalize_embeddings=True,
                    use_trunk_output=False,
                    batch_size=32,
                    dataloader_num_workers=2,
                    pca=None,
                    data_device=None,
                    dtype=None,
                    data_and_label_getter=None,
                    label_hierarchy_level=0,
                    end_of_testing_hook=None,
                    dataset_labels=None,
                    set_min_label_to_zero=False,
                    accuracy_calculator=None,
                    visualizer=None,
                    visualizer_hook=None,)

```

--------------------------------

### ContrastiveLoss Initialization

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Initializes ContrastiveLoss with positive and negative margins. Default margins are suitable for LpDistance.

```python
losses.ContrastiveLoss(pos_margin=0, neg_margin=1, **kwargs):
```

--------------------------------

### CUB Dataset Initialization - pytorch-metric-learning

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/datasets

Initializes the CUB-200-2011 dataset. It supports loading 'train', 'test', or 'train+test' splits.

```python
datasets.CUB(*args, **kwargs)

```

--------------------------------

### Initialize NPairsLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate NPairsLoss. This loss is suitable for batches with more than 2 samples per label.

```python
losses.NPairsLoss(**kwargs)
```

--------------------------------

### Initialize SupConLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate SupConLoss with a temperature parameter. The default distance is CosineSimilarity.

```python
losses.SupConLoss(temperature=0.1, **kwargs)
```

--------------------------------

### Initialize GeneralizedLiftedStructureLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate GeneralizedLiftedStructureLoss with optional positive and negative margins.

```python
losses.GeneralizedLiftedStructureLoss(neg_margin=1, pos_margin=0, **kwargs)
```

--------------------------------

### Initialize RankedListLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate the RankedListLoss with the required margin and Tn parameters, along with optional imbalance and alpha.

```python
losses.RankedListLoss(margin, Tn, imbalance=0.5, alpha=None, Tp=0, **kwargs)
```

--------------------------------

### MeanReducer Initialization

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/reducers

Initializes the MeanReducer, which calculates the simple average of the input losses. This is a common and straightforward reduction method.

```python
reducers.MeanReducer(**kwargs)

```

--------------------------------

### Initialize SubCenterArcFaceLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate SubCenterArcFaceLoss with parameters including the number of sub-centers per class. This loss also requires an optimizer.

```python
losses.SubCenterArcFaceLoss(
    num_classes, 
    embedding_size, 
    margin=28.6, 
    scale=64, 
    sub_centers=3, 
    **kwargs
)
```

--------------------------------

### Initialize Miner and TripletMarginLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning

Combine a miner with TripletMarginLoss for training. The miner identifies difficult pairs, which are then used by the loss function.

```python
from pytorch_metric_learning import miners, losses
miner = miners.MultiSimilarityMiner()
loss_func = losses.TripletMarginLoss()
```