### Launch Training with accelerate

Source: https://www.sbert.net/docs/sentence_transformer/training/distributed.html

Use this command to start distributed training with DDP using the accelerate library. Specify the number of processes required for your training setup.

```bash
accelerate launch --num_processes 4 train_script.py
```

--------------------------------

### Install Qdrant Client

Source: https://www.sbert.net/examples/sparse_encoder/applications/semantic_search/README.html

Install the Qdrant Python client using pip. This is a prerequisite for using Qdrant with the sparse encoder.

```bash
pip install qdrant-client

```

--------------------------------

### Install Sentence Transformers for Development

Source: https://www.sbert.net/docs/installation.html

Installs Sentence Transformers with all extras, including dependencies for development.

```bash
uv pip install -U "sentence-transformers[dev]"

```

--------------------------------

### Install OpenSearch Python Client

Source: https://www.sbert.net/examples/sparse_encoder/applications/semantic_search/README.html

Install the necessary Python client for OpenSearch. This is a prerequisite for using the OpenSearch integration.

```bash
pip install opensearch-py

```

--------------------------------

### Install Sentence Transformers with OpenVINO Support from Source

Source: https://www.sbert.net/docs/installation.html

Installs sentence-transformers with OpenVINO support from the source repository.

```bash
pip install -U "sentence-transformers[openvino] @ git+https://github.com/huggingface/sentence-transformers.git"

```

--------------------------------

### Install Sentence Transformers with Training Support from Source

Source: https://www.sbert.net/docs/installation.html

Installs sentence-transformers with additional dependencies for training from the source repository.

```bash
pip install -U "sentence-transformers[train] @ git+https://github.com/huggingface/sentence-transformers.git"

```

--------------------------------

### Install Sentence Transformers with Development Support from Source

Source: https://www.sbert.net/docs/installation.html

Installs sentence-transformers with development dependencies from the source repository.

```bash
pip install -U "sentence-transformers[dev] @ git+https://github.com/huggingface/sentence-transformers.git"

```

--------------------------------

### Install Sentence Transformers with Video Support from Source

Source: https://www.sbert.net/docs/installation.html

Installs sentence-transformers with additional dependencies for video processing from the source repository.

```bash
pip install -U "sentence-transformers[video] @ git+https://github.com/huggingface/sentence-transformers.git"

```

--------------------------------

### Install MTEB

Source: https://www.sbert.net/docs/sentence_transformer/usage/mteb_evaluation.html

Installs the MTEB library and its dependencies. Ensure you have version 2.0.0 or higher.

```bash
pip install mteb>=2.0.0
```

--------------------------------

### Install Sentence Transformers (Default)

Source: https://www.sbert.net/docs/installation.html

Installs the core Sentence Transformers library for loading, saving, and inference.

```bash
uv pip install -U sentence-transformers

```

--------------------------------

### Install Sentence Transformers with OpenVINO Support

Source: https://www.sbert.net/docs/installation.html

Installs Sentence Transformers with OpenVINO backend dependencies for inference.

```bash
uv pip install -U "sentence-transformers[openvino]"

```

--------------------------------

### Install Sentence Transformers with Training Support

Source: https://www.sbert.net/docs/installation.html

Installs Sentence Transformers with dependencies required for training and finetuning models.

```bash
uv pip install -U "sentence-transformers[train]"

```

--------------------------------

### Scalar Quantization Example

Source: https://www.sbert.net/examples/sentence_transformer/applications/embedding-quantization/README.html

Demonstrates how to quantize embeddings to int8 using a calibration dataset. Ensure you have the necessary libraries installed (`sentence-transformers`, `datasets`).

```python
from sentence_transformers import SentenceTransformer
from sentence_transformers.quantization import quantize_embeddings
from datasets import load_dataset

# 1. Load an embedding model
model = SentenceTransformer("mixedbread-ai/mxbai-embed-large-v1")

# 2. Prepare an example calibration dataset
corpus = load_dataset("google-research-datasets/nq_open", split="train[:1000]")["question"]
calibration_embeddings = model.encode(corpus)

# 3. Encode some text without quantization & apply quantization afterwards
embeddings = model.encode(["I am driving to the lake.", "It is a beautiful day."])
int8_embeddings = quantize_embeddings(
    embeddings,
    precision="int8",
    calibration_embeddings=calibration_embeddings,
)
```

--------------------------------

### Install Tracking Libraries for Training

Source: https://www.sbert.net/docs/installation.html

Install optional libraries like trackio for tracking training logs. Recommended for advanced logging.

```bash
pip install trackio

```

--------------------------------

### CachedGISTEmbedLoss Example

Source: https://www.sbert.net/docs/package_reference/sentence_transformer/losses.html

Demonstrates how to use CachedGISTEmbedLoss for training a Sentence Transformer model. This loss requires a primary model and a guide model, and allows for configurable margin strategies.

```python
from sentence_transformers import SentenceTransformer, SentenceTransformerTrainer, losses
from datasets import Dataset

model = SentenceTransformer("microsoft/mpnet-base")
guide = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
train_dataset = Dataset.from_dict({
    "anchor": ["It's nice weather outside today.", "He drove to work."],
    "positive": ["It's so sunny.", "He took the car to the office."],
})
loss = losses.CachedGISTEmbedLoss(
    model,
    guide,
    mini_batch_size=64,
    margin_strategy="absolute",   # or "relative" (e.g., margin=0.05 for max. 95% of positive similarity)
    margin=0.1
)

trainer = SentenceTransformerTrainer(
    model=model,
    train_dataset=train_dataset,
    loss=loss,
)
trainer.train()

```

--------------------------------

### Information Retrieval Evaluation Setup and Execution

Source: https://www.sbert.net/docs/package_reference/sentence_transformer/evaluation.html

Demonstrates how to set up and run the InformationRetrievalEvaluator using a subset of the Touche-2020 dataset. This involves loading a model, preparing the corpus and queries, and then performing the evaluation.

```python
import logging
import random

from datasets import load_dataset
from sentence_transformers import SentenceTransformer
from sentence_transformers.sentence_transformer.evaluation import InformationRetrievalEvaluator
from datasets import load_dataset

logging.basicConfig(format="%(asctime)s - %(message)s", datefmt="%Y-%m-%d %H:%M:%S", level=logging.INFO)

# Load a model
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

# Load the Touche-2020 IR dataset (https://huggingface.co/datasets/mteb/webis-touche2020-v3)
corpus = load_dataset("mteb/webis-touche2020-v3", "corpus", split="corpus")
queries = load_dataset("mteb/webis-touche2020-v3", "queries", split="train")
relevant_docs_data = load_dataset("mteb/webis-touche2020-v3", "default", split="test")

# For this dataset, we want to concatenate the title and texts for the corpus
corpus = corpus.map(lambda x: {"text": (x["title"] + " " + x["text"]).strip()}, remove_columns=["title"])

# Shrink the corpus size heavily to only the relevant documents + 30,000 random documents
required_corpus_ids = set(map(str, relevant_docs_data["corpus-id"]))
required_corpus_ids |= set(random.sample(corpus["_id"], k=30_000))
coprus = corpus.filter(lambda x: x["_id"] in required_corpus_ids)

# Convert the datasets to dictionaries
corpus = dict(zip(corpus["_id"], corpus["text"]))
queries = dict(zip(queries["_id"], queries["text"]))
relevant_docs = {}
for qid, corpus_ids in zip(relevant_docs_data["query-id"], relevant_docs_data["corpus-id"]):
    qid = str(qid)
    corpus_ids = str(corpus_ids)
    if qid not in relevant_docs:
        relevant_docs[qid] = set()
    relevant_docs[qid].add(corpus_ids)

# Given queries, a corpus and a mapping with relevant documents, the InformationRetrievalEvaluator computes different IR metrics.
ir_evaluator = InformationRetrievalEvaluator(
    queries=queries,
    corpus=corpus,
    relevant_docs=relevant_docs,
    name="mteb-touche2020-subset-test",
)
ir_evaluator(model)

print(ir_evaluator.primary_metric)
print(results[ir_evaluator.primary_metric])

```

--------------------------------

### Complete HPO Example for Sentence Transformers

Source: https://www.sbert.net/examples/sentence_transformer/training/hpo/README.html

This comprehensive example integrates all components required for hyperparameter search, including dataset loading, evaluator setup, search space definition, model and loss initialization, objective function, training arguments, trainer instantiation, and the search execution itself. Use this as a template for your own HPO tasks.

```python
from sentence_transformers import SentenceTransformer, SentenceTransformerTrainer, SentenceTransformerTrainingArguments
from sentence_transformers.sentence_transformer.evaluation import EmbeddingSimilarityEvaluator, SimilarityFunction
from sentence_transformers.sentence_transformer.losses import MultipleNegativesRankingLoss
from sentence_transformers.sentence_transformer.training_args import BatchSamplers
from datasets import load_dataset

# 1. Load the AllNLI dataset: https://huggingface.co/datasets/sentence-transformers/all-nli, only 10k train and 1k dev
train_dataset = load_dataset("sentence-transformers/all-nli", "triplet", split="train[:10000]")
eval_dataset = load_dataset("sentence-transformers/all-nli", "triplet", split="dev[:1000]")

# 2. Create an evaluator to perform useful HPO
stsb_eval_dataset = load_dataset("sentence-transformers/stsb", split="validation")
dev_evaluator = EmbeddingSimilarityEvaluator(
    sentences1=stsb_eval_dataset["sentence1"],
    sentences2=stsb_eval_dataset["sentence2"],
    scores=stsb_eval_dataset["score"],
    main_similarity=SimilarityFunction.COSINE,
    name="sts-dev",
)

# 3. Define the Hyperparameter Search Space
def hpo_search_space(trial):
    return {
        "num_train_epochs": trial.suggest_int("num_train_epochs", 1, 2),
        "per_device_train_batch_size": trial.suggest_int("per_device_train_batch_size", 32, 128),
        "warmup_steps": trial.suggest_float("warmup_steps", 0, 0.3),
        "learning_rate": trial.suggest_float("learning_rate", 1e-6, 1e-4, log=True),
    }

# 4. Define the Model Initialization
def hpo_model_init(trial):
    return SentenceTransformer("distilbert/distilbert-base-uncased")

# 5. Define the Loss Initialization
def hpo_loss_init(model):
    return MultipleNegativesRankingLoss(model)

# 6. Define the Objective Function
def hpo_compute_objective(metrics):
    """
    Valid keys are: 'eval_loss', 'eval_sts-dev_pearson_cosine', 'eval_sts-dev_spearman_cosine',
    'eval_sts-dev_pearson_manhattan', 'eval_sts-dev_spearman_manhattan', 'eval_sts-dev_pearson_euclidean',
    'eval_sts-dev_spearman_euclidean', 'eval_sts-dev_pearson_dot', 'eval_sts-dev_spearman_dot',
    'eval_sts-dev_pearson_max', 'eval_sts-dev_spearman_max', 'eval_runtime', 'eval_samples_per_second',
    'eval_steps_per_second', 'epoch'

    due to the evaluator that we're using.
    """
    return metrics["eval_sts-dev_spearman_cosine"]

# 7. Define the training arguments
args = SentenceTransformerTrainingArguments(
    # Required parameter:
    output_dir="checkpoints",
    # Optional training parameters:
    # max_steps=10000, # We might want to limit the number of steps for HPO
    fp16=True,  # Set to False if you get an error that your GPU can't run on FP16
    bf16=False,  # Set to True if you have a GPU that supports BF16
    batch_sampler=BatchSamplers.NO_DUPLICATES,  # MultipleNegativesRankingLoss benefits from no duplicate samples in a batch
    # Optional tracking/debugging parameters:
    eval_strategy="no", # We don't need to evaluate/save during HPO
    save_strategy="no",
    logging_steps=10,
    run_name="hpo",  # Will be used in W&B if `wandb` is installed
)

# 8. Create the trainer with model_init rather than model
trainer = SentenceTransformerTrainer(
    model=None,
    args=args,
    train_dataset=train_dataset,
    eval_dataset=eval_dataset,
    evaluator=dev_evaluator,
    model_init=hpo_model_init,
    loss=hpo_loss_init,
)

# 9. Perform the HPO
best_trial = trainer.hyperparameter_search(
    hp_space=hpo_search_space,
    compute_objective=hpo_compute_objective,
    n_trials=20,
    direction="maximize",
    backend="optuna",
)
print(best_trial)

```

--------------------------------

### MarginMSELoss with Teacher Model for Knowledge Distillation

Source: https://www.sbert.net/docs/package_reference/sentence_transformer/losses.html

Utilize MarginMSELoss in a knowledge distillation setup where a teacher model computes similarity scores to generate silver labels. This example shows how to use a teacher model to compute pairwise similarities and then use these as labels for the MarginMSELoss.

```python
from sentence_transformers import SentenceTransformer, SentenceTransformerTrainer, losses
from datasets import Dataset

student_model = SentenceTransformer("microsoft/mpnet-base")
teacher_model = SentenceTransformer("sentence-transformers/all-mpnet-base-v2")
train_dataset = Dataset.from_dict({
    "query": ["It's nice weather outside today.", "He drove to work."],
    "passage1": ["It's so sunny.", "He took the car to work."],
    "passage2": ["It's very sunny.", "She walked to the store."],
})

def compute_labels(batch):
    emb_queries = teacher_model.encode(batch["query"])
    emb_passages1 = teacher_model.encode(batch["passage1"])
    emb_passages2 = teacher_model.encode(batch["passage2"])
    return {
        "label": teacher_model.similarity_pairwise(emb_queries, emb_passages1) - teacher_model.similarity_pairwise(emb_queries, emb_passages2)
    }

train_dataset = train_dataset.map(compute_labels, batched=True)
loss = losses.MarginMSELoss(student_model)

trainer = SentenceTransformerTrainer(
    model=student_model,
    train_dataset=train_dataset,
    loss=loss,
)
trainer.train()

```

--------------------------------

### Install Trackio for Training Log Tracking

Source: https://www.sbert.net/docs/installation.html

Installs the trackio library to track training logs, recommended for use with training extras.

```bash
uv pip install trackio

```

--------------------------------

### Custom Loss Combining InfoNCE and GOR

Source: https://www.sbert.net/docs/package_reference/sentence_transformer/losses.html

This example demonstrates how to create a custom loss module that combines MultipleNegativesRankingLoss (InfoNCE) with GlobalOrthogonalRegularizationLoss. It's useful when you want to apply both losses simultaneously within a single training step. Ensure you have the 'datasets' and 'sentence-transformers' libraries installed.

```python
import torch
from datasets import Dataset
from torch import Tensor
from sentence_transformers import SentenceTransformer, SentenceTransformerTrainer
from sentence_transformers.sentence_transformer.losses import GlobalOrthogonalRegularizationLoss, MultipleNegativesRankingLoss
from sentence_transformers.util import cos_sim

model = SentenceTransformer("microsoft/mpnet-base")
train_dataset = Dataset.from_dict({
    "anchor": ["It's nice weather outside today.", "He drove to work."],
    "positive": ["It's so sunny.", "He took the car to the office."],
})

class InfoNCEGORLoss(torch.nn.Module):
    def __init__(self, model: SentenceTransformer, similarity_fct=cos_sim, scale=20.0) -> None:
        super().__init__()
        self.model = model
        self.info_nce_loss = MultipleNegativesRankingLoss(model, similarity_fct=similarity_fct, scale=scale)
        self.gor_loss = GlobalOrthogonalRegularizationLoss(model, similarity_fct=similarity_fct)

    def forward(self, sentence_features: list[dict[str, Tensor]], labels: Tensor | None = None) -> Tensor:
        embeddings = [self.model(sentence_featureization["sentence_embedding"] for sentence_feature in sentence_features]
        info_nce_loss: dict[str, Tensor] = {
            "info_nce": self.info_nce_loss.compute_loss_from_embeddings(embeddings, labels)
        }
        gor_loss: dict[str, Tensor] = self.gor_loss.compute_loss_from_embeddings(embeddings, labels)
        return {**info_nce_loss, **gor_loss}

loss = InfoNCEGORLoss(model)
trainer = SentenceTransformerTrainer(
    model=model,
    train_dataset=train_dataset,
    loss=loss,
)
trainer.train()

```

--------------------------------

### ContrastiveTensionLoss with Single Input Pairs and Random Pairing

Source: https://www.sbert.net/docs/package_reference/sentence_transformer/losses.html

This snippet demonstrates how to use ContrastiveTensionLoss with a dataset containing only single input sentences. It randomly pairs sentences to create positive and negative examples, suitable when identical pairs are not naturally present. Ensure the `datasets` library is installed.

```python
import random
from datasets import Dataset
from sentence_transformers import SentenceTransformer
from sentence_transformers.sentence_transformer.losses import ContrastiveTensionLoss
from sentence_transformers.sentence_transformer.training_args import SentenceTransformerTrainingArguments
from sentence_transformers.sentence_transformer.trainer import SentenceTransformerTrainer

model = SentenceTransformer('sentence-transformers/all-MiniLM-L6-v2')
train_dataset = Dataset.from_dict({
    "text1": [
        "It's nice weather outside today.",
        "He drove to work.",
        "It's so sunny.",
    ]
})
sentences = train_dataset['text1']

def to_ct_pairs(sample, pos_neg_ratio=8):
    pos_neg_ratio = 1 / pos_neg_ratio
    sample["text2"] = sample["text1"] if random.random() < pos_neg_ratio else random.choice(sentences)
    return sample

pos_neg_ratio = 8  # 1 positive pair for 7 negative pairs
train_dataset = train_dataset.map(to_ct_pairs, fn_kwargs={"pos_neg_ratio": pos_neg_ratio})

train_loss = ContrastiveTensionLoss(model=model)

args = SentenceTransformerTrainingArguments(
    num_train_epochs=10,
    per_device_train_batch_size=32,
    eval_steps=0.1,
    logging_steps=0.01,
    learning_rate=5e-5,
    save_strategy="no",
    fp16=True,
)
trainer = SentenceTransformerTrainer(model=model, args=args, train_dataset=train_dataset, loss=train_loss)
trainer.train()

```

--------------------------------

### Full SentenceTransformer Training Example

Source: https://www.sbert.net/docs/sentence_transformer/training_overview.html

This script demonstrates the complete process of fine-tuning a Sentence Transformer model. It covers loading a model, preparing a dataset, defining a loss function, specifying training arguments, creating an evaluator, training the model, and saving/pushing the results. Ensure all necessary libraries are installed.

```python
from datasets import load_dataset
from sentence_transformers import (
    SentenceTransformer,
    SentenceTransformerTrainer,
    SentenceTransformerTrainingArguments,
    SentenceTransformerModelCardData,
)
from sentence_transformers.sentence_transformer.losses import MultipleNegativesRankingLoss
from sentence_transformers.sentence_transformer.training_args import BatchSamplers
from sentence_transformers.sentence_transformer.evaluation import TripletEvaluator

# 1. Load a model to finetune with 2. (Optional) model card data
model = SentenceTransformer(
    "microsoft/mpnet-base",
    model_card_data=SentenceTransformerModelCardData(
        language="en",
        license="apache-2.0",
        model_name="MPNet base trained on AllNLI triplets",
    ),
    model_kwargs={\"torch_dtype\": \"float32\"},
)

# 3. Load a dataset to finetune on
dataset = load_dataset("sentence-transformers/all-nli", "triplet")
train_dataset = dataset["train"].select(range(100_000))
eval_dataset = dataset["dev"]
test_dataset = dataset["test"]

# 4. Define a loss function
loss = MultipleNegativesRankingLoss(model)

# 5. (Optional) Specify training arguments
args = SentenceTransformerTrainingArguments(
    # Required parameter:
    output_dir="models/mpnet-base-all-nli-triplet",
    # Optional training parameters:
    num_train_epochs=1,
    per_device_train_batch_size=16,
    per_device_eval_batch_size=16,
    learning_rate=2e-5,
    warmup_steps=0.1,
    fp16=True,  # Set to False if you get an error that your GPU can't run on FP16
    bf16=False,  # Set to True if you have a GPU that supports BF16
    batch_sampler=BatchSamplers.NO_DUPLICATES,  # MultipleNegativesRankingLoss benefits from no duplicate samples in a batch
    # Optional tracking/debugging parameters:
    eval_strategy="steps",
    eval_steps=100,
    save_strategy="steps",
    save_steps=100,
    save_total_limit=2,
    logging_steps=100,
    run_name="mpnet-base-all-nli-triplet",  # Will be used in W&B if `wandb` is installed
)

# 6. (Optional) Create an evaluator & evaluate the base model
dev_evaluator = TripletEvaluator(
    anchors=eval_dataset["anchor"],
    positives=eval_dataset["positive"],
    negatives=eval_dataset["negative"],
    name="all-nli-dev",
)
dev_evaluator(model)

# 7. Create a trainer & train
trainer = SentenceTransformerTrainer(
    model=model,
    args=args,
    train_dataset=train_dataset,
    eval_dataset=eval_dataset,
    loss=loss,
    evaluator=dev_evaluator,
)
trainer.train()

# (Optional) Evaluate the trained model on the test set
test_evaluator = TripletEvaluator(
    anchors=test_dataset["anchor"],
    positives=test_dataset["positive"],
    negatives=test_dataset["negative"],
    name="all-nli-test",
)
test_evaluator(model)

# 8. Save the trained model
model.save_pretrained("models/mpnet-base-all-nli-triplet/final")

# 9. (Optional) Push it to the Hugging Face Hub
model.push_to_hub("mpnet-base-all-nli-triplet")

```

--------------------------------

### Initialize Custom Multi-Dataset Batch Sampler with Class and Function

Source: https://www.sbert.net/docs/migration_guide.html

Demonstrates initializing a custom multi-dataset batch sampler using both a class and a function for v5.x.

```python
from sentence_transformers import SentenceTransformer, SentenceTransformerTrainer
from sentence_transformers.base.sampler import MultiDatasetDefaultBatchSampler
from sentence_transformers.training import SentenceTransformerTrainingArguments
from torch.utils.data import ConcatDataset, BatchSampler
import torch


class CustomMultiDatasetBatchSampler(MultiDatasetDefaultBatchSampler):
    def __init__(
        self,
        dataset: ConcatDataset,
        batch_samplers: list[BatchSampler],
        generator: torch.Generator | None = None,
        seed: int = 0,
    ):
        super().__init__(dataset, batch_samplers=batch_samplers, generator=generator, seed=seed)
        # Custom multi-dataset batch sampler logic here

args = SentenceTransformerTrainingArguments(
    # Other training arguments
    multi_dataset_batch_sampler=CustomMultiDatasetBatchSampler,
)
trainer = SentenceTransformerTrainer(
    model=model,
    args=args,
    train_dataset=train_dataset,
    ...
)
trainer.train()

# Or, use a function to initialize the batch sampler
def custom_batch_sampler(
    dataset: ConcatDataset,
    batch_samplers: list[BatchSampler],
    generator: torch.Generator | None = None,
    seed: int = 0,
):
    # Custom multi-dataset batch sampler logic here
    return ...

args = SentenceTransformerTrainingArguments(
    # Other training arguments
    multi_dataset_batch_sampler=custom_batch_sampler,  # Use the custom batch sampler function
)
trainer = SentenceTransformerTrainer(
    model=model,
    args=args,
    train_dataset=train_dataset,
    ...
)
trainer.train()
```

--------------------------------

### StaticEmbedding Initialization and Usage

Source: https://www.sbert.net/docs/package_reference/sentence_transformer/modules.html

Demonstrates how to initialize and use the StaticEmbedding module, which creates sentence embeddings by averaging trained per-token embeddings. It shows examples of initializing with pre-distilled embeddings, distilling custom embeddings, or starting with randomized embeddings. This module is highly efficient, and CPU usage might be preferable for inference and training due to low computational overhead.

```python
from sentence_transformers import SentenceTransformer
from sentence_transformers.sentence_transformer.modules import StaticEmbedding
from tokenizers import Tokenizer

# Pre-distilled embeddings:
static_embedding = StaticEmbedding.from_model2vec("minishlab/potion-base-8M")
# or distill your own embeddings:
static_embedding = StaticEmbedding.from_distillation("BAAI/bge-base-en-v1.5", device="cuda")
# or start with randomized embeddings:
tokenizer = Tokenizer.from_pretrained("FacebookAI/xlm-roberta-base")
static_embedding = StaticEmbedding(tokenizer, embedding_dim=512)

model = SentenceTransformer(modules=[static_embedding])

embeddings = model.encode(["What are Pandas?", "The giant panda, also known as the panda bear or simply the panda, is a bear native to south central China."])
similarity = model.similarity(embeddings[0], embeddings[1])
# tensor([[0.8093]]) (If you use potion-base-8M)
# tensor([[0.6234]]) (If you use the distillation method)
# tensor([[-0.0693]]) (For example, if you use randomized embeddings)
```

--------------------------------

### Install Sentence Transformers with Audio Support from Source

Source: https://www.sbert.net/docs/installation.html

Installs sentence-transformers with additional dependencies for audio processing from the source repository.

```bash
pip install -U "sentence-transformers[audio] @ git+https://github.com/huggingface/sentence-transformers.git"

```

--------------------------------

### Install Sentence Transformers with pip

Source: https://www.sbert.net/docs/installation.html

Install the base Sentence Transformers package using pip. This is the default installation method.

```bash
pip install -U sentence-transformers

```

--------------------------------

### Load and Prepare Dataset

Source: https://www.sbert.net/docs/sparse_encoder/training_overview.html

Loads the Natural Questions dataset and splits it into training and evaluation sets. Prints the first training example.

```python
# 3. Load a dataset to finetune on
full_dataset = load_dataset("sentence-transformers/natural-questions", split="train").select(range(100_000))
dataset_dict = full_dataset.train_test_split(test_size=1_000, seed=12)
train_dataset = dataset_dict["train"]
eval_dataset = dataset_dict["test"]
print(train_dataset)
print(train_dataset[0])
```

--------------------------------

### Install HPO Backend

Source: https://www.sbert.net/examples/sentence_transformer/training/hpo/README.html

Install a hyperparameter optimization backend like Optuna, Sigopt, Raytune, or Wandb. This command installs the chosen backend for use with SentenceTransformerTrainer.

```bash
pip install optuna/sigopt/wandb/ray[tune]
```

--------------------------------

### Install Seismic Python Package

Source: https://www.sbert.net/examples/sparse_encoder/applications/semantic_search/README.html

Install the necessary Seismic Python package using pip.

```bash
pip install pyseismic-lsr

```

--------------------------------

### Install Sentence Transformers with Video Support

Source: https://www.sbert.net/docs/installation.html

Installs Sentence Transformers with dependencies required for models that process video inputs.

```bash
uv pip install -U "sentence-transformers[video]"

```

--------------------------------

### Install splade-index Package

Source: https://www.sbert.net/examples/sparse_encoder/applications/semantic_search/README.html

Install the splade-index Python package using pip. This is a prerequisite for using the library.

```bash
pip install splade-index

```

--------------------------------

### Install Sentence Transformers with Training Support (pip)

Source: https://www.sbert.net/docs/installation.html

Install Sentence Transformers with optional support for training functionalities. Requires pip.

```bash
pip install -U "sentence-transformers[train]"

```