PyTorch Metric Learning

https://kevinmusgrave.github.io/
Admin
PyTorch Metric Learning is a comprehensive library offering various modules such as loss functions,...
Tokens:53,662
Snippets:488
Trust Score:8.3
Update:2 months ago
Show doc for...
Tokens:
Raw
### ProxyNCALoss Optimizer Setup

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Example of setting up an optimizer for ProxyNCALoss. The loss function's parameters must be passed to the optimizer.

```python
loss_func = losses.ProxyNCALoss(...).to(torch.device('cuda'))
loss_optimizer = torch.optim.SGD(loss_func.parameters(), lr=0.01)
# then during training:
loss_optimizer.step()
```

--------------------------------

### SphereFaceLoss Optimizer Setup

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Example of setting up an optimizer for SphereFaceLoss. The loss function's parameters must be passed to the optimizer.

```python
loss_func = losses.SphereFaceLoss(...).to(torch.device('cuda'))
loss_optimizer = torch.optim.SGD(loss_func.parameters(), lr=0.01)
# then during training:
loss_optimizer.step()
```

--------------------------------

### ProxyAnchorLoss Optimizer Setup

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Example of setting up an optimizer for ProxyAnchorLoss. The loss function's parameters must be passed to the optimizer.

```python
loss_func = losses.ProxyAnchorLoss(...).to(torch.device('cuda'))
loss_optimizer = torch.optim.SGD(loss_func.parameters(), lr=0.01)
# then during training:
loss_optimizer.step()
```

--------------------------------

### Install pytorch-metric-learning on Windows

Source: https://kevinmusgrave.github.io/pytorch-metric-learning

Specific installation command for Windows, including torch and torchvision versions, followed by the library installation.

```bash
pip install torch===1.6.0 torchvision===0.7.0 -f https://download.pytorch.org/whl/torch_stable.html
pip install pytorch-metric-learning

```

--------------------------------

### Install pytorch-metric-learning with hooks (GPU FAISS)

Source: https://kevinmusgrave.github.io/pytorch-metric-learning

Install the library with evaluation and logging capabilities. This option installs the unofficial pypi version of faiss-gpu.

```bash
pip install pytorch-metric-learning[with-hooks]

```

--------------------------------

### Install pytorch-metric-learning with hooks (CPU FAISS)

Source: https://kevinmusgrave.github.io/pytorch-metric-learning

Install the library with evaluation and logging capabilities for CPU. This option installs the unofficial pypi version of faiss-cpu.

```bash
pip install pytorch-metric-learning[with-hooks-cpu]

```

--------------------------------

### Install pytorch-metric-learning using pip

Source: https://kevinmusgrave.github.io/pytorch-metric-learning

Use this command to install the latest stable version of the library.

```bash
pip install pytorch-metric-learning

```

--------------------------------

### Install latest dev version of pytorch-metric-learning

Source: https://kevinmusgrave.github.io/pytorch-metric-learning

Install the latest development version using the --pre flag.

```bash
pip install pytorch-metric-learning --pre

```

--------------------------------

### Initialize ProxyNCALoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate the ProxyNCALoss. Similar to ProxyAnchorLoss, this loss requires an optimizer setup.

```python
losses.ProxyNCALoss(num_classes, embedding_size, softmax_scale=1, **kwargs)
```

--------------------------------

### Install pytorch-metric-learning using Conda

Source: https://kevinmusgrave.github.io/pytorch-metric-learning

Install the library using Conda from the conda-forge channel.

```bash
conda install -c conda-forge pytorch-metric-learning

```

--------------------------------

### CosFaceLoss Initialization and Optimizer Setup

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Initializes CosFaceLoss with number of classes and embedding size. Requires an optimizer to be explicitly created and stepped.

```python
losses.CosFaceLoss(num_classes, embedding_size, margin=0.35, scale=64, **kwargs)
```

```python
loss_func = losses.CosFaceLoss(...).to(torch.device('cuda'))
loss_optimizer = torch.optim.SGD(loss_func.parameters(), lr=0.01)
# then during training:
loss_optimizer.step()
```

--------------------------------

### HDCMiner example with another miner

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/miners

Example demonstrating how to pass the output of one miner (minerA) to HDCMiner (minerB) using set_idx_externally.

```python
minerA = miners.MultiSimilarityMiner(epsilon=0.1)
minerB = miners.HDCMiner(filter_percentage=0.25)

hard_pairs = minerA(embeddings, labels)
minerB.set_idx_externally(hard_pairs, labels)
very_hard_pairs = minerB(embeddings, labels)
```

--------------------------------

### Basic Miner Usage Example

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/miners

Demonstrates how to instantiate and use a miner with a loss function. Ensure embeddings and labels are available.

```python
from pytorch_metric_learning import miners, losses
miner_func = miners.SomeMiner()
loss_func = losses.SomeLoss()
miner_output = miner_func(embeddings, labels)
losses = loss_func(embeddings, labels, miner_output)
```

--------------------------------

### SoftTripleLoss Optimizer Setup

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

When using SoftTripleLoss, an optimizer must be explicitly created and passed the loss function's parameters. This is demonstrated with SGD, and the optimizer's step() method should be called during training.

```python
loss_func = losses.SoftTripleLoss(...).to(torch.device('cuda'))
loss_optimizer = torch.optim.SGD(loss_func.parameters(), lr=0.01)
# then during training:
loss_optimizer.step()
```

--------------------------------

### Example Usage of P2SGradLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Demonstrates how to use P2SGradLoss by initializing the loss function and then calling it with embeddings and labels.

```python
loss_fn = P2SGradLoss(128, 10)
loss = loss_fn(embeddings, labels)
```

--------------------------------

### FaissKNN Example Usage

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/inference_models

Example of using FaissKNN with a custom index initialization (IndexFlatIP) and specifying GPUs. Demonstrates how to call the KNN function with query, k, and reference embeddings.

```python
# use faiss.IndexFlatIP on 3 gpus
knn_func = FaissKNN(index_init_fn=faiss.IndexFlatIP, gpus=[0,1,2])

# query = query embeddings 
# k = the k in k-nearest-neighbors
# reference = the embeddings to search
# last argument is whether or not query and reference share datapoints
distances, indices = knn_func(query, k, references, False)
```

--------------------------------

### FaissKMeans Example Usage

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/inference_models

Example of initializing FaissKMeans with specific parameters and then using it to cluster embeddings into a specified number of groups.

```python
kmeans_func = FaissKMeans(niter=100, verbose=True, gpu=True)

# cluster into 10 groups
cluster_assignments = kmeans_func(embeddings, 10)
```

--------------------------------

### Integrating Reducers into Loss Functions

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/reducers

Demonstrates how to instantiate a reducer and pass it to a loss function. This setup is used within the training loop for calculating the final loss.

```python
from pytorch_metric_learning import losses, reducers
reducer = reducers.SomeReducer()
loss_func = losses.SomeLoss(reducer=reducer)
loss = loss_func(embeddings, labels) # in your training for-loop

```

--------------------------------

### Example Accuracy Metrics Output

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/testers

This is an example of the dictionary structure for accuracy metrics computed by the tester, organized by dataset split.

```json
{"train": {"AMI_level0": 0.53, ...}, "val": {"AMI_level0": 0.44, ...}}
```

--------------------------------

### Example Pair Miner Usage

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/extend/miners

Instantiate and use the custom ExamplePairMiner. The miner can be called directly with embeddings and labels. If ref_emb and ref_labels are not provided, they default to embeddings and labels.

```python
import torch

miner = ExamplePairMiner()
embeddings = torch.randn(128, 512)
labels = torch.randint(0, 10, size=(128,))
pairs = miner(embeddings, labels)
```

--------------------------------

### Optimizer Setup for NormalizedSoftmaxLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

NormalizedSoftmaxLoss requires an optimizer. Create an optimizer and pass the loss function's parameters to it. Remember to call loss_optimizer.step() during training.

```python
loss_func = losses.NormalizedSoftmaxLoss(...).to(torch.device('cuda'))
loss_optimizer = torch.optim.SGD(loss_func.parameters(), lr=0.01)
# then during training:
loss_optimizer.step()
```

--------------------------------

### Example Loss Dictionary for DivisorReducer

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/reducers

Illustrates the structure of a loss dictionary that includes a 'divisor' key, as expected by the DivisorReducer. This example is from ProxyAnchorLoss.

```python
loss_dict = {
    "pos_loss": {
        "losses": pos_term.squeeze(0),
        "indices": loss_indices,
        "reduction_type": "element",
        "divisor": len(with_pos_proxies),
    },
    "neg_loss": {
        "losses": neg_term.squeeze(0),
        "indices": loss_indices,
        "reduction_type": "element",
        "divisor": self.num_classes,
    },
}

```

--------------------------------

### Embeddings ordered as triplets example

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/miners

Example structure for embeddings already packaged as triplets for a batch size of 6.

```python
torch.stack([anchor1, positive1, negative1, anchor2, positive2, negative2], dim=0)
```

--------------------------------

### CustomKNN Example Usage

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/inference_models

Example of initializing CustomKNN with SNRDistance and then calling it to find nearest neighbors. This demonstrates using a custom distance metric for KNN.

```python
from pytorch_metric_learning.distances import SNRDistance
from pytorch_metric_learning.utils.inference import CustomKNN

knn_func = CustomKNN(SNRDistance())
distances, indices = knn_func(query, k, references, False)
```

--------------------------------

### ArcFaceLoss Initialization and Optimizer Setup

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Initialize ArcFaceLoss with the number of classes and embedding size. It requires an optimizer to be set up separately for its parameters, and it uses CosineSimilarity as the default distance.

```python
loss_func = losses.ArcFaceLoss(num_classes, embedding_size, margin=28.6, scale=64, **kwargs)

```

```python
loss_func = losses.ArcFaceLoss(...).to(torch.device('cuda'))
loss_optimizer = torch.optim.SGD(loss_func.parameters(), lr=0.01)
# then during training:
loss_optimizer.step()

```

--------------------------------

### Self-Supervised Training Loop

Source: https://kevinmusgrave.github.io/pytorch-metric-learning

Example training loop for self-supervised learning using SelfSupervisedLoss. It computes embeddings for both original and augmented data.

```python
# your training for-loop
for i, data in enumerate(dataloader):
    optimizer.zero_grad()
    embeddings = your_model(data)
    augmented = your_model(your_augmentation(data))
    loss = loss_func(embeddings, augmented)
    loss.backward()
    optimizer.step()
```

--------------------------------

### Apply RegularFaceRegularizer to ArcFaceLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/regularizers

An example of how to instantiate and pass a weight regularizer to a loss function. Ensure the regularizer is compatible with the chosen loss.

```python
from pytorch_metric_learning import losses, regularizers
R = regularizers.RegularFaceRegularizer()
loss = losses.ArcFaceLoss(margin=30, num_classes=100, embedding_size=128, weight_regularizer=R)
```

--------------------------------

### Example Usage of MultipleReducers with ContrastiveLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/reducers

Configures ContrastiveLoss to use different reducers for its positive and negative pair losses. A ThresholdReducer is used for positive pairs, and a MeanReducer for negative pairs.

```python
from pytorch_metric_learning.losses import ContrastiveLoss
from pytorch_metric_learning.reducers import MultipleReducers, ThresholdReducer, MeanReducer
reducer_dict = {"pos_loss": ThresholdReducer(0.1), "neg_loss": MeanReducer()}
reducer = MultipleReducers(reducer_dict)
loss_func = ContrastiveLoss(reducer=reducer)

```

--------------------------------

### Instantiate and Use ManifoldLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiates the ManifoldLoss and shows example usage with embeddings. It can optionally use specified indices as cluster centers.

```python
loss_fn = ManifoldLoss(128)

# use random cluster centers
loss = loss_fn(embeddings)
# or specify indices of embeddings to use as cluster centers
loss = loss_fn(embeddings, indices_tuple=indices)
```

--------------------------------

### Example Usage of SelfSupervisedLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Wrap an existing loss function like TripletMarginLoss with SelfSupervisedLoss to handle label creation internally. This simplifies the training process for self-supervised metric learning tasks.

```python
loss_fn = losses.TripletMarginLoss()
loss_fn = SelfSupervisedLoss(loss_fn)
loss = loss_fn(embeddings, ref_emb)
```

--------------------------------

### Example Pair Miner Implementation

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/extend/miners

Extend BaseMiner to create a custom pair miner. This miner finds pairs that violate a specified margin. Ensure the distance metric is correctly configured for inverted or non-inverted behavior.

```python
from pytorch_metric_learning.miners import BaseMiner
from pytorch_metric_learning.utils import loss_and_miner_utils as lmu

class ExamplePairMiner(BaseMiner):
    def __init__(self, margin=0.1, **kwargs):
        super().__init__(**kwargs)
        self.margin = margin

    def mine(self, embeddings, labels, ref_emb, ref_labels):
        mat = self.distance(embeddings, ref_emb)
        a1, p, a2, n = lmu.get_all_pairs_indices(labels, ref_labels)
        pos_pairs = mat[a1, p]
        neg_pairs = mat[a2, n]
        pos_mask = (
            pos_pairs < self.margin
            if self.distance.is_inverted
            else pos_pairs > self.margin
        )
        neg_mask = (
            neg_pairs > self.margin
            if self.distance.is_inverted
            else neg_pairs < self.margin
        )
        return a1[pos_mask], p[pos_mask], a2[neg_mask], n[neg_mask]
```

--------------------------------

### Example Triplet Miner Implementation

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/extend/miners

Extend BaseMiner to create a custom triplet miner. This miner finds triplets where the distance between anchor and positive is too close to the distance between anchor and negative, relative to a margin. Ensure the distance metric is correctly configured.

```python
from pytorch_metric_learning.miners import BaseMiner
from pytorch_metric_learning.utils import loss_and_miner_utils as lmu

class ExampleTripletMiner(BaseMiner):
    def __init__(self, margin=0.1, **kwargs):
        super().__init__(**kwargs)
        self.margin = margin

    def mine(self, embeddings, labels, ref_emb, ref_labels):
        mat = self.distance(embeddings, ref_emb)
        a, p, n = lmu.get_all_triplets_indices(labels, ref_labels)
        pos_pairs = mat[a, p]
        neg_pairs = mat[a, n]
        triplet_margin = pos_pairs - neg_pairs if self.distance.is_inverted else neg_pairs - pos_pairs
        triplet_mask = triplet_margin <= self.margin
        return a[triplet_mask], p[triplet_mask], n[triplet_mask]
```

--------------------------------

### Basic Trainer Usage

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/trainers

Instantiate a trainer and run the training process for a specified number of epochs. Ensure necessary imports are included.

```python
from pytorch_metric_learning import trainers
t = trainers.SomeTrainingFunction(*args, **kwargs)
t.train(num_epochs=10)
```

--------------------------------

### Initialize ProxyAnchorLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate the ProxyAnchorLoss. This loss requires an optimizer to be created and passed the loss's parameters.

```python
losses.ProxyAnchorLoss(num_classes, embedding_size, margin = 0.1, alpha = 32, **kwargs)
```

--------------------------------

### Initialize NTXentLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate NTXentLoss, also known as InfoNCE. This loss is a generalization of NPairsLoss and is often used in self-supervised learning.

```python
losses.NTXentLoss(temperature=0.07, **kwargs)
```

--------------------------------

### Initialize InstanceLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate InstanceLoss with a gamma parameter for scaling the cosine similarity matrix.

```python
losses.InstanceLoss(gamma=64, **kwargs)
```

--------------------------------

### End-of-Testing Hook Example

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/testers

Define a function to be called after testing is complete. This hook can access and process the computed accuracy metrics.

```python
def end_of_testing_hook(tester):
    print(tester.all_accuracies)
```

--------------------------------

### Initialize DynamicSoftMarginLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate DynamicSoftMarginLoss with parameters for minimum value, number of bins, and momentum.

```python
losses.DynamicSoftMarginLoss(min_val=-2.0, num_bins=10, momentum=0.01, **kwargs)
```

--------------------------------

### Use DistributedMinerWrapper and DistributedLossWrapper

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/distributed

Use the wrapped miner to get tuples, then pass these tuples to the wrapped loss function for training.

```python
# in each process
tuples = miner(embeddings, labels)
# pass into a DistributedLossWrapper
loss = loss_func(embeddings, labels, indices_tuple)
```

--------------------------------

### Loading StanfordOnlineProducts Dataset Splits - pytorch-metric-learning

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/datasets

Demonstrates how to load different splits of the StanfordOnlineProducts dataset. Set download=True for the initial load. The download process can take a significant amount of time.

```python
# The download takes a while - the dataset is very large
train_dataset = StanfordOnlineProducts(root="data", 
    split="train", 
    transform=None, 
    target_transform=None, 
    download=True
)
# No need to download the dataset after it is already downladed
test_dataset = StanfordOnlineProducts(root="data", 
    split="test", 
    transform=None, 
    target_transform=None, 
    download=False
)
train_and_test_dataset = StanfordOnlineProducts(root="data", 
    split="train+test", 
    transform=None, 
    target_transform=None, 
    download=False
) 

```

--------------------------------

### Initialize SphereFaceLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate SphereFaceLoss with required parameters. This loss requires an optimizer to be created and passed its parameters.

```python
losses.SphereFaceLoss(num_classes, 
                    embedding_size, 
                    margin=4, 
                    scale=1, 
                    **kwargs)
```

--------------------------------

### Initialize AccuracyCalculator

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/accuracy_calculation

Instantiate the AccuracyCalculator with optional parameters to customize metric calculation. Default metrics are calculated if 'include' is empty.

```python
from pytorch_metric_learning.utils.accuracy_calculator import AccuracyCalculator
AccuracyCalculator(include=(),
                    exclude=(),
                    avg_of_avgs=False,
                    return_per_class=False,
                    k=None,
                    label_comparison_fn=None,
                    device=None,
                    knn_func=None,
                    kmeans_func=None)
```

--------------------------------

### Initialize FaissKNN

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/inference_models

Instantiate FaissKNN for efficient k-nearest neighbor search using the Faiss library. Configure index initialization and GPU usage.

```python
from pytorch_metric_learning.utils.inference import FaissKNN
FaissKNN(reset_before=True,
            reset_after=True, 
            index_init_fn=None, 
            gpus=None)
```

--------------------------------

### Get Outliers and Dominant Centers from SubCenterArcFaceLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Compute outliers and dominant centers using the get_outliers method. This method can optionally return dominant centers.

```python
outliers, dominant_centers = loss_func.get_outliers(
    embeddings, labels, threshold=75, return_dominant_centers=True
)
```

--------------------------------

### Initialize PNPLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate the PNPLoss with specified parameters. The paper uses b=2, alpha=8, anneal=0.01, and variant='Dq'.

```python
losses.PNPLoss(b=2, alpha=1, anneal=0.01, variant="O", **kwargs)
```

--------------------------------

### Custom Floating Point Label Comparison Function

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/accuracy_calculation

Define a custom function for comparing floating-point labels. This example uses a tolerance threshold to determine equality.

```python
def label_comparison_fn(x, y):
    return torch.abs(x - y) < 1

# these are valid labels
labels = torch.tensor([
    10.0,
    0.03,
    0.04,
    0.05,
])
```

--------------------------------

### Initialize FastAPLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate FastAPLoss with the number of bins for histogram calculation.

```python
losses.FastAPLoss(num_bins=10, **kwargs)
```

--------------------------------

### StanfordOnlineProducts Dataset Initialization - pytorch-metric-learning

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/datasets

Initializes the StanfordOnlineProducts dataset. It supports loading 'train', 'test', or 'train+test' splits. Note that this dataset is very large.

```python
datasets.StanfordOnlineProducts(*args, **kwargs)

```

--------------------------------

### Instantiate SoftTripleLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Configure SoftTripleLoss with the number of classes, embedding size, and other hyperparameters like centers_per_class, la, gamma, and margin. This loss function requires an optimizer to be passed its parameters.

```python
losses.SoftTripleLoss(num_classes, 
                    embedding_size, 
                    centers_per_class=10, 
                    la=20, 
                    gamma=0.1, 
                    margin=0.01,
                    **kwargs)
```

--------------------------------

### Retrieve Loss History from HookContainer

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/logging_presets

Shows how to access recorded loss histories from the `HookContainer` instance, either to get all loss histories or to retrieve specific ones by name.

```python
# Get a dictionary mapping from loss names to lists
loss_histories = hooks.get_loss_history() 
```

```python
# You can also specify which loss histories you want
# It will still return a dictionary. In this case, the dictionary will contain only "total_loss"
loss_histories = hooks.get_loss_history(loss_names=["total_loss"])
```

--------------------------------

### TorchInitWrapper for Weight Initialization

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/common_functions

Shows how to use TorchInitWrapper to convert PyTorch weight initialization functions into a class-based format. This wrapper can be used within loss functions.

```python
from pytorch_metric_learning.utils import common_functions as c_f
import torch

# use kaiming_uniform, with a=1 and mode='fan_out'
weight_init_func = c_f.TorchInitWrapper(torch.nn.kaiming_uniform_, a=1, mode='fan_out')
loss_func = SomeClassificationLoss(..., weight_init_func=weight_init_func)
```

--------------------------------

### Integrate Logging Presets with Trainer and Tester

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/logging_presets

Demonstrates how to set up `record-keeper` and `tensorboard` logging, create a `HookContainer`, and integrate its hooks into `GlobalEmbeddingSpaceTester` and `MetricLossOnly` trainer for comprehensive logging and model saving.

```python
import pytorch_metric_learning.utils.logging_presets as LP
log_folder, tensorboard_folder = "example_logs", "example_tensorboard"
record_keeper, _, _ = LP.get_record_keeper(log_folder, tensorboard_folder)
hooks = LP.get_hook_container(record_keeper)
dataset_dict = {"val": val_dataset}
model_folder = "example_saved_models"

# Create the tester
tester = testers.GlobalEmbeddingSpaceTester(end_of_testing_hook=hooks.end_of_testing_hook)
end_of_epoch_hook = hooks.end_of_epoch_hook(tester, dataset_dict, model_folder)
trainer = trainers.MetricLossOnly(models,
                                optimizers,
                                batch_size,
                                loss_funcs,
                                mining_funcs,
                                train_dataset,
                                sampler=sampler,
                                end_of_iteration_hook=hooks.end_of_iteration_hook,
                                end_of_epoch_hook=end_of_epoch_hook)

trainer.train(num_epochs=num_epochs)
```

--------------------------------

### DoNothingReducer Initialization

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/reducers

Initializes the DoNothingReducer, which bypasses any reduction and returns the input loss dictionary unchanged. Useful when no aggregation is desired.

```python
reducers.DoNothingReducer(**kwargs)

```

--------------------------------

### Initialize ThresholdConsistentMarginLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate ThresholdConsistentMarginLoss with scaling coefficients and margin parameters for positive and negative pairs. This loss is typically used as regularization.

```python
losses.ThresholdConsistentMarginLoss(
    lambda_plus=1.0, 
    lambda_minus=1.0, 
    margin_plus=0.9, 
    margin_minus=0.5,
    **kwargs
)
```

--------------------------------

### Customized TripletMarginLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning

Create a customized TripletMarginLoss using CosineSimilarity, ThresholdReducer, and LpRegularizer. This example uses cosine similarity, discards triplets with loss > 0.3, and applies L2 regularization to embeddings.

```python
from pytorch_metric_learning.distances import CosineSimilarity
from pytorch_metric_learning.reducers import ThresholdReducer
from pytorch_metric_learning.regularizers import LpRegularizer
from pytorch_metric_learning import losses
loss_func = losses.TripletMarginLoss(distance = CosineSimilarity(), 
                                    reducer = ThresholdReducer(high=0.3), 
                                    embedding_regularizer = LpRegularizer())
```

--------------------------------

### Get Accuracy Metrics

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/accuracy_calculation

Call the get_accuracy method to compute accuracy metrics. Provide query and reference embeddings along with their labels. Optional 'include' and 'exclude' parameters can override initialization settings.

```python
def get_accuracy(self, 
    query,
    query_labels,       
    reference=None,
    reference_labels=None, 
    ref_includes_query=False,
    include=(),
    exclude=()
):
# returns a dictionary mapping from metric names to accuracy values
# The default metrics are:
# "NMI" (Normalized Mutual Information)
# "AMI" (Adjusted Mutual Information)
# "precision_at_1"
# "r_precision"
# "mean_average_precision_at_r"
```

--------------------------------

### Initialize MarginLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate the MarginLoss with specified parameters. Learnable beta can be enabled.

```python
losses.MarginLoss(margin=0.2, 
                nu=0, 
                beta=1.2, 
                triplets_per_anchor="all", 
                learn_beta=False, 
                num_classes=None, 
                **kwargs)
```

--------------------------------

### Instantiate LargeMarginSoftmaxLoss and Optimizer

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiates the LargeMarginSoftmaxLoss and its required optimizer. The optimizer must be configured with the loss function's parameters.

```python
loss_func = losses.LargeMarginSoftmaxLoss(...).to(torch.device('cuda'))
loss_optimizer = torch.optim.SGD(loss_func.parameters(), lr=0.01)
# then during training:
loss_optimizer.step()
```

--------------------------------

### Adding Custom Accuracy Metrics

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/accuracy_calculation

Extend the AccuracyCalculator class to add custom accuracy metrics. Methods starting with 'calculate_' are automatically recognized. Specify dependencies on k-nn search or clustering using 'requires_knn' and 'requires_clustering'.

```python
from pytorch_metric_learning.utils import accuracy_calculator

class YourCalculator(accuracy_calculator.AccuracyCalculator):
    def calculate_precision_at_2(self, knn_labels, query_labels, **kwargs):
        return accuracy_calculator.precision_at_k(knn_labels, query_labels[:, None], 2)

    def calculate_fancy_mutual_info(self, query_labels, cluster_labels, **kwargs):
        return fancy_computations

    def requires_clustering(self):
        return super().requires_clustering() + ["fancy_mutual_info"]

    def requires_knn(self):
        return super().requires_knn() + ["precision_at_2"]

```

--------------------------------

### Initialize HistogramLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate HistogramLoss with optional number of bins and delta for histogram construction.

```python
losses.HistogramLoss(n_bins=None, delta=None)
```

--------------------------------

### Get All Embeddings with BaseTester

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/testers

Use this function to retrieve all embeddings and labels from a dataset using a given model. Ensure that the dataset and trunk_model are provided. Optional arguments include an embedder_model, collate_fn, eval mode flag, and return format.

```python
embeddings, labels = tester.get_all_embeddings(
    dataset, # Any pytorch dataset
    trunk_model, # your model
    embedder_model=None, # by default this will be a no-op
    collate_fn=None, # custom collate_fn for the dataloader
    eval=True, # set models to eval mode
    return_as_numpy=False
 )
```

--------------------------------

### BaseTrainer Initialization

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/trainers

Initialize the BaseTrainer with models, optimizers, batch size, loss functions, and dataset. Optional parameters include mining functions, iterations per epoch, data device, dtype, loss weights, sampler, collate function, learning rate schedulers, gradient clippers, and freeze configurations.

```python
trainers.BaseTrainer(models,
                    optimizers,
                    batch_size,
                    loss_funcs,
                    dataset,
                    mining_funcs=None,
                    iterations_per_epoch=None,
                    data_device=None,
                    dtype=None,
                    loss_weights=None,
                    sampler=None,
                    collate_fn=None,
                    lr_schedulers=None,
                    gradient_clippers=None,
                    freeze_these=(),
                    freeze_trunk_batchnorm=False,
                    label_hierarchy_level=0,
                    dataloader_num_workers=2,
                    data_and_label_getter=None,
                    dataset_labels=None,
                    set_min_label_to_zero=False,
                    end_of_iteration_hook=None,
                    end_of_epoch_hook=None)
```

--------------------------------

### Initialize FaissKMeans

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/inference_models

Instantiate FaissKMeans for k-means clustering using Faiss. Any keyword arguments are passed directly to the faiss.Kmeans constructor.

```python
from pytorch_metric_learning.utils.inference import FaissKMeans
FaissKMeans(**kwargs)
```

--------------------------------

### PerAnchorReducer

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/reducers

Converts unreduced pairs to unreduced elements. For example, NTXentLoss returns losses per positive pair. If you used PerAnchorReducer with NTXentLoss, then the losses per pair would first be converted to losses per batch element, before being passed to the inner reducer.

```APIDOC
## PerAnchorReducer

### Description
This reducer converts unreduced pairs to unreduced elements. For example, NTXentLoss returns losses per positive pair. If you used PerAnchorReducer with NTXentLoss, then the losses per pair would first be converted to losses per batch element, before being passed to the inner reducer.

### Parameters
* **reducer** : The reducer that will be fed per-element losses. The default is MeanReducer.
* **aggregation_func** : A function that takes in `(x, num_per_row)` and returns a loss per row of `x`. The default is the `aggregation_func` defined in the code snippet above. It returns the mean per row.
    * `x` is an NxN array of pairwise losses, where N is the batch size.
    * `num_per_row` is a size N array which indicates how many non-zero losses there are per-row of `x`.

### Code Example
```python
def aggregation_func(x, num_per_row):
    zero_denom = num_per_row == 0
    x = torch.sum(x, dim=1) / num_per_row
    x[zero_denom] = 0
    return x

reducers.PerAnchorReducer(reducer=None, aggregation_func=aggregation_func, **kwargs)
```
```

--------------------------------

### Initialize HookContainer for Logging and Model Saving

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/logging_presets

Illustrates how to create an instance of `LP.HookContainer`, specifying parameters like `record_keeper`, `primary_metric`, `validation_split_name`, `save_models`, and `log_freq` to configure logging behavior.

```python
import pytorch_metric_learning.utils.logging_presets as LP
LP.HookContainer(record_keeper, 
    record_group_name_prefix=None, 
    primary_metric="mean_average_precision_at_r", 
    validation_split_name="val",
    save_models=True,
    log_freq=50)
```

--------------------------------

### Using the PML Logger

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/common_functions

Demonstrates how to use the default logger provided by the library. This logger is used across various components.

```python
from pytorch_metric_learning.utils import common_functions as c_f
c_f.LOGGER.info("Using the PML logger")
```

--------------------------------

### Loading CUB Dataset Splits - pytorch-metric-learning

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/datasets

Demonstrates how to load different splits of the CUB dataset. Set download=True for the initial load.

```python
train_dataset = CUB(root="data", 
    split="train", 
    transform=None, 
    target_transform=None, 
    download=True
)
# No need to download the dataset after it is already downladed
test_dataset = CUB(root="data", 
    split="test", 
    transform=None, 
    target_transform=None, 
    download=False
)
train_and_test_dataset = CUB(root="data", 
    split="train+test", 
    transform=None, 
    target_transform=None, 
    download=False
) 

```

--------------------------------

### Instantiate SmoothAPLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Initialize SmoothAPLoss with a specified temperature. This loss is suitable for large-scale image retrieval and requires labels with an equal number of elements per class.

```python
losses.SmoothAPLoss(
    temperature=0.01,
    **kwargs
)
```

--------------------------------

### Instantiate TwoStreamMetricLoss Trainer

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/trainers

Initialize the TwoStreamMetricLoss trainer with its arguments. Ensure that the models, loss functions, and miners meet the specified requirements.

```python
trainers.TwoStreamMetricLoss(*args, **kwargs)
```

--------------------------------

### Initialize HDCMiner

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/miners

Instantiate HDCMiner. This miner filters a percentage of the hardest pairs or triplets. It can use externally provided indices.

```python
miners.HDCMiner(filter_percentage=0.5, **kwargs)
```

--------------------------------

### Initialize NCALoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate the NCALoss with a specified softmax scale. This loss is based on Neighbourhood Components Analysis.

```python
losses.NCALoss(softmax_scale=1, **kwargs)
```

--------------------------------

### Initialize InferenceModel

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/inference_models

Instantiate InferenceModel with a trained trunk model. Optional parameters include an embedder, match finder, normalization settings, KNN function, data device, and data type.

```python
from pytorch_metric_learning.utils.inference import InferenceModel
InferenceModel(trunk,
                embedder=None,
                match_finder=None,
                normalize_embeddings=True,
                knn_func=None,
                data_device=None,
                dtype=None)
```

--------------------------------

### Instantiate VICRegLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate the VICRegLoss with weights for invariance, variance, and covariance terms. A small epsilon is included for numerical stability.

```python
losses.VICRegLoss(invariance_lambda=25, 
                variance_mu=25, 
                covariance_v=1, 
                eps=1e-4, 
                **kwargs)
```

--------------------------------

### InferenceModel Methods

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/inference_models

Demonstrates common methods of the InferenceModel class, including training KNN, adding data, retrieving nearest neighbors, checking for matches, and saving/loading the KNN function.

```python
# initialize with a model
im = InferenceModel(model)

# pass in a dataset to serve as the search space for k-nn
im.train_knn(dataset)

# add another dataset to the index
im.add_to_knn(dataset2)

# get the 10 nearest neighbors of a query
distances, indices = im.get_nearest_neighbors(query, k=10)

# determine if inputs are close to each other
is_match = im.is_match(x, y)

# determine "is_match" pairwise for all elements in a batch
match_matrix = im.get_matches(x)

# save and load the knn function (which is a faiss index by default)
im.save_knn_func("filename.index")
im.load_knn_func("filename.index")
```

--------------------------------

### AvgNonZeroReducer Initialization

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/reducers

Initializes the AvgNonZeroReducer, which computes the average of losses that are strictly greater than zero. This is equivalent to using ThresholdReducer with a low threshold of 0.

```python
reducers.AvgNonZeroReducer(**kwargs)

```

--------------------------------

### Initialize BaseTester

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/testers

Instantiate the BaseTester class with various configuration options. These parameters control normalization, batching, PCA, device, data types, and hooks for testing.

```python
testers.BaseTester(
                    normalize_embeddings=True,
                    use_trunk_output=False,
                    batch_size=32,
                    dataloader_num_workers=2,
                    pca=None,
                    data_device=None,
                    dtype=None,
                    data_and_label_getter=None,
                    label_hierarchy_level=0,
                    end_of_testing_hook=None,
                    dataset_labels=None,
                    set_min_label_to_zero=False,
                    accuracy_calculator=None,
                    visualizer=None,
                    visualizer_hook=None,)

```

--------------------------------

### ContrastiveLoss Initialization

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Initializes ContrastiveLoss with positive and negative margins. Default margins are suitable for LpDistance.

```python
losses.ContrastiveLoss(pos_margin=0, neg_margin=1, **kwargs):
```

--------------------------------

### CUB Dataset Initialization - pytorch-metric-learning

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/datasets

Initializes the CUB-200-2011 dataset. It supports loading 'train', 'test', or 'train+test' splits.

```python
datasets.CUB(*args, **kwargs)

```

--------------------------------

### Initialize NPairsLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate NPairsLoss. This loss is suitable for batches with more than 2 samples per label.

```python
losses.NPairsLoss(**kwargs)
```

--------------------------------

### Initialize SupConLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate SupConLoss with a temperature parameter. The default distance is CosineSimilarity.

```python
losses.SupConLoss(temperature=0.1, **kwargs)
```

--------------------------------

### Initialize GeneralizedLiftedStructureLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate GeneralizedLiftedStructureLoss with optional positive and negative margins.

```python
losses.GeneralizedLiftedStructureLoss(neg_margin=1, pos_margin=0, **kwargs)
```

--------------------------------

### Initialize RankedListLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate the RankedListLoss with the required margin and Tn parameters, along with optional imbalance and alpha.

```python
losses.RankedListLoss(margin, Tn, imbalance=0.5, alpha=None, Tp=0, **kwargs)
```

--------------------------------

### MeanReducer Initialization

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/reducers

Initializes the MeanReducer, which calculates the simple average of the input losses. This is a common and straightforward reduction method.

```python
reducers.MeanReducer(**kwargs)

```

--------------------------------

### Initialize SubCenterArcFaceLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate SubCenterArcFaceLoss with parameters including the number of sub-centers per class. This loss also requires an optimizer.

```python
losses.SubCenterArcFaceLoss(
    num_classes, 
    embedding_size, 
    margin=28.6, 
    scale=64, 
    sub_centers=3, 
    **kwargs
)
```

--------------------------------

### Initialize Miner and TripletMarginLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning

Combine a miner with TripletMarginLoss for training. The miner identifies difficult pairs, which are then used by the loss function.

```python
from pytorch_metric_learning import miners, losses
miner = miners.MultiSimilarityMiner()
loss_func = losses.TripletMarginLoss()
```

--------------------------------

### Initialize MultiSimilarityMiner

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/miners

Instantiate MultiSimilarityMiner. This miner selects pairs based on similarity thresholds relative to the hardest positive and negative pairs, using an epsilon margin.

```python
miners.MultiSimilarityMiner(epsilon=0.1, **kwargs)
```

--------------------------------

### Initialize FixedSetOfTriplets Sampler

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/samplers

Use FixedSetOfTriplets when you have a predefined set of ground truth triplets and want to evaluate algorithm performance on this fixed set. It requires a list of labels and the desired number of triplets.

```python
samplers.FixedSetOfTriplets(labels, num_triplets)
```

--------------------------------

### BatchEasyHardMiner Initialization

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/miners

Initializes BatchEasyHardMiner with strategies for selecting positive and negative pairs. It can be used with NTXentLoss for specific paper implementations. Restrictions apply to combinations of 'semihard' and 'all' strategies.

```python
miners.BatchEasyHardMiner(
        pos_strategy=BatchEasyHardMiner.EASY,
        neg_strategy=BatchEasyHardMiner.SEMIHARD,
        allowed_pos_range=None,
        allowed_neg_range=None,
        **kwargs
    )
```

--------------------------------

### Initialize MultiSimilarityLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate the MultiSimilarityLoss with custom weights for positive and negative pairs, and a base offset.

```python
losses.MultiSimilarityLoss(alpha=2, beta=50, base=0.5, **kwargs)
```

--------------------------------

### BaseTrainer Initialization

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/trainers

The BaseTrainer class serves as the foundation for all trainers. It accepts a variety of arguments for configuring models, optimizers, loss functions, datasets, and training parameters.

```APIDOC
## BaseTrainer

All trainers extend this class and therefore inherit its `__init__` arguments.

```python
trainers.BaseTrainer(models,
                    optimizers,
                    batch_size,
                    loss_funcs,
                    dataset,
                    mining_funcs=None,
                    iterations_per_epoch=None,
                    data_device=None,
                    dtype=None,
                    loss_weights=None,
                    sampler=None,
                    collate_fn=None,
                    lr_schedulers=None,
                    gradient_clippers=None,
                    freeze_these=(),
                    freeze_trunk_batchnorm=False,
                    label_hierarchy_level=0,
                    dataloader_num_workers=2,
                    data_and_label_getter=None,
                    dataset_labels=None,
                    set_min_label_to_zero=False,
                    end_of_iteration_hook=None,
                    end_of_epoch_hook=None)

```

### Parameters

*   **models** : A dictionary of the form:
    *   `{"trunk": trunk_model, "embedder": embedder_model}`
    *   The "embedder" key is optional.
*   **optimizers** : A dictionary mapping strings to optimizers. The base class does not require any specific keys. For example, you could provide an empty dictionary, in which case no optimization will happen. Or you could provide just an optimizer for your trunk_model. But most likely, you'll want to pass in:
    *   `{"trunk_optimizer": trunk_optimizer, "embedder_optimizer": embedder_optimizer}`.
*   **batch_size** : The number of elements that are retrieved at each iteration.
*   **loss_funcs** : A dictionary mapping strings to loss functions. The required keys depend on the training method, but all methods are likely to require at least:
    *   `{"metric_loss": loss_func}`.
*   **dataset** : The dataset you want to train on. Note that training methods do not perform validation, so do not pass in your validation or test set.
*   **mining_funcs** : A dictionary mapping strings to mining functions. Pass in an empty dictionary, or one or more of the following keys:
    *   `{"subset_batch_miner": mining_func1, "tuple_miner": mining_func2}`
*   **data_device** : The device that you want to put batches of data on. If not specified, the trainer will put the data on any available GPUs.
*   **dtype** : The type that the dataset output will be converted to, e.g. `torch.float16`. If set to `None`, then no type casting will be done.
*   **iterations_per_epoch** : Optional.
    *   If you don't specify `iterations_per_epoch`:
        *   1 epoch = 1 pass through the dataloader iterator. If `sampler=None`, then 1 pass through the iterator is 1 pass through the dataset.
        *   If you use a sampler, then 1 pass through the iterator is 1 pass through the iterable returned by the sampler.
    *   For samplers like `MPerClassSampler` or some offline mining method, the iterable returned might be very long or very short etc, and might not be related to the length of the dataset. The length of the epoch might vary each time the sampler creates a new iterable. In these cases, it can be useful to specify `iterations_per_epoch` so that each "epoch" is just a fixed number of iterations. The definition of epoch matters because there's various things like LR schedulers and hooks that depend on an epoch ending.
*   **loss_weights** : A dictionary mapping loss names to numbers. Each loss will be multiplied by the corresponding value in the dictionary. If not specified, then no loss weighting will occur. If not specified, then the original labels are used.
*   **sampler** : The sampler used by the dataloader. If not specified, then random sampling will be used.
*   **collate_fn** : The collate function used by the dataloader.
*   **lr_schedulers** : A dictionary of PyTorch learning rate schedulers. Your keys should be strings of the form `<model>_<step_type>`, where `<model>` is a key that comes from either the `models` or `loss_funcs` dictionary, and `<step_type>` is one of the following:
    *   "scheduler_by_iteration" (will be stepped at every iteration)
    *   "scheduler_by_epoch" (will be stepped at the end of each epoch)
    *   "scheduler_by_plateau" (will step if accuracy plateaus. This requires you to write your own end-of-epoch hook, compute validation accuracy, and call `trainer.step_lr_plateau_schedulers(validation_accuracy)`. Or you can use HookContainer.)
    *   Here are some example valid `lr_scheduler` keys:
        *   `trunk_scheduler_by_iteration`
        *   `metric_loss_scheduler_by_epoch`
        *   `embedder_scheduler_by_plateau`
*   **gradient_clippers** : A dictionary of gradient clipping functions. Each function will be called before the optimizers.
```

--------------------------------

### Initialize NormalizedSoftmaxLoss

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate NormalizedSoftmaxLoss with the number of classes and embedding size. It accepts optional temperature and other keyword arguments.

```python
losses.NormalizedSoftmaxLoss(num_classes, embedding_size, temperature=0.05, **kwargs)
```

--------------------------------

### Initialize TripletMarginMiner

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/miners

Instantiate TripletMarginMiner. This miner selects triplets based on a margin, with options to retrieve 'all', 'hard', 'semihard', or 'easy' triplets.

```python
miners.TripletMarginMiner(margin=0.2, type_of_triplets="all", **kwargs)
```

--------------------------------

### TuplesToWeightsSampler Initialization

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/samplers

Initializes TuplesToWeightsSampler for offline mining. It mines tuples, computes weights based on element appearance, and then samples using these weights. Consider using 'subset_size' to avoid memory issues.

```python
samplers.TuplesToWeightsSampler(model, 
                                miner, 
                                dataset, 
                                subset_size=None, 
                                **tester_kwargs)
```

--------------------------------

### Initialize BatchHardMiner

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/miners

Instantiate BatchHardMiner. This miner finds the hardest positive and negative for each element in a batch to form triplets.

```python
miners.BatchHardMiner(**kwargs)
```

--------------------------------

### Instantiating and Using Custom Calculator

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/accuracy_calculation

Instantiate the custom calculator and call 'get_accuracy'. The returned dictionary will include the custom metrics defined in the class.

```python
calculator = YourCalculator()
acc_dict = calculator.get_accuracy(query_embeddings,
    query_labels,
    reference_embeddings,
    reference_labels,
    ref_includes_query=True
)
# Now acc_dict contains the metrics "precision_at_2" and "fancy_mutual_info"

```

--------------------------------

### Loading Cars196 Dataset Splits - pytorch-metric-learning

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/datasets

Demonstrates how to load different splits of the Cars196 dataset. Set download=True for the initial load.

```python
train_dataset = Cars196(root="data", 
    split="train", 
    transform=None, 
    target_transform=None, 
    download=True
)
# No need to download the dataset after it is already downladed
test_dataset = Cars196(root="data", 
    split="test", 
    transform=None, 
    target_transform=None, 
    download=False
)
train_and_test_dataset = Cars196(root="data", 
    split="train+test", 
    transform=None, 
    target_transform=None, 
    download=False
) 

```

--------------------------------

### Basic Tester Usage

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/testers

Initialize a tester and run evaluations on a dataset dictionary. This can be done with a single model or a trunk and embedder.

```python
from pytorch_metric_learning import testers
t = testers.SomeTestingFunction(*args, **kwargs)
dataset_dict = {"train": train_dataset, "val": val_dataset}
all_accuracies = tester.test(dataset_dict, epoch, model)

# Or if your model is composed of a trunk + embedder
all_accuracies = tester.test(dataset_dict, epoch, trunk, embedder)
```

--------------------------------

### Instantiate WeightRegularizerMixin

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/losses

Instantiate the WeightRegularizerMixin, which is intended for use when a loss function has a learnable weight matrix. It accepts functions for weight initialization and regularization.

```python
losses.WeightRegularizerMixin(weight_init_func=None, weight_regularizer=None, weight_reg_weight=1, **kwargs)
```

--------------------------------

### BatchedDistance initialization and usage

Source: https://kevinmusgrave.github.io/pytorch-metric-learning/distances

Demonstrates how to use BatchedDistance with an iterator function to process distance matrices in batches. The iter_fn is called for each batch.

```python
from pytorch_metric_learning.distances import BatchedDistance, CosineSimilarity

def fn(mat, s, e):
    print(f"At query indices {s}:{e}")

distance = BatchedDistance(CosineSimilarity(), fn)

# Works like a regular distance function, except nothing is returned.
# So any persistent changes need to be done in the supplied iter_fn.
# query vs query
distance(embeddings)
# query vs ref
distance(embeddings, ref_emb)
```
Tokens:
Raw