SetFit

https://huggingface.co/docs/setfit
Admin
SetFit is an efficient framework for few-shot fine-tuning of Sentence Transformers for text...
Tokens:103,127
Snippets:560
Trust Score:10
Update:1 day ago
Show doc for...
Tokens:
Raw
### End-to-end SetFit quickstart example

Source: https://huggingface.co/docs/setfit/quickstart

This snippet shows the complete SetFit workflow from model initialization to inference. It uses the 'BAAI/bge-small-en-v1.5' model and the 'SetFit/sst2' dataset, training on 8 samples per label. The expected accuracy is approximately 0.851, and the example predictions are ['positive', 'negative', 'negative']. It demonstrates saving locally with save_pretrained and pushing to the Hub with push_to_hub.

```python
from setfit import SetFitModel, Trainer, TrainingArguments, sample_dataset
from datasets import load_dataset

# Initializing a new SetFit model
model = SetFitModel.from_pretrained("BAAI/bge-small-en-v1.5", labels=["negative", "positive"])

# Preparing the dataset
dataset = load_dataset("SetFit/sst2")
train_dataset = sample_dataset(dataset["train"], label_column="label", num_samples=8)
test_dataset = dataset["test"]

# Preparing the training arguments
args = TrainingArguments(
    batch_size=32,
    num_epochs=10,
)

# Preparing the trainer
trainer = Trainer(
    model=model,
    args=args,
    train_dataset=train_dataset,
)
trainer.train()

# Evaluating
metrics = trainer.evaluate(test_dataset)
print(metrics)
# => {'accuracy': 0.8511806699615596}

# Saving the trained model
model.save_pretrained("setfit-bge-small-v1.5-sst2-8-shot")
# or
model.push_to_hub("tomaarsen/setfit-bge-small-v1.5-sst2-8-shot")

# Loading a trained model
model = SetFitModel.from_pretrained("tomaarsen/setfit-bge-small-v1.5-sst2-8-shot") # Load from the Hugging Face Hub
# or
model = SetFitModel.from_pretrained("setfit-bge-small-v1.5-sst2-8-shot") # Load from a local directory

# Performing inference
preds = model.predict([
    "It's a charming and often affecting journey.",
    "It's slow -- very, very slow.",
    "A sometimes tedious film.",
])
print(preds)
# => ["positive", "negative", "negative"]
```

--------------------------------

### Install Optuna backend

Source: https://huggingface.co/docs/setfit/how_to/hyperparameter_optimization

Install the optuna package required for hyperparameter optimization.

```bash
pip install optuna
```

--------------------------------

### Install SetFit and accelerate

Source: https://huggingface.co/docs/setfit/tutorials/onnx

Installs the required packages for the tutorial using pip. The -qqq flag suppresses output for a cleaner notebook experience.

```bash
!pip install setfit accelerate -qqq
```

--------------------------------

### Install SetFit from source (editable mode)

Source: https://huggingface.co/docs/setfit/installation

Clone the repository and install SetFit in editable mode to make changes to the code base. This sequence of commands clones the repo, changes into the directory, and installs the package with pip install -e .

```bash
git clone https://github.com/huggingface/setfit.git
cd setfit
pip install -e .
```

--------------------------------

### Install setfit and matplotlib

Source: https://huggingface.co/docs/setfit/tutorials/zero_shot

Uncomment and run this cell to install the required libraries when running the notebook on Colab or a similar cloud platform.

```bash
# %pip install setfit matplotlib
```

--------------------------------

### Install SetFit with pip

Source: https://huggingface.co/docs/setfit/installation

The most straightforward way to install SetFit is with pip. This command installs the latest version of the setfit package.

```bash
pip install setfit
```

--------------------------------

### Install SetFit from source (bleeding-edge)

Source: https://huggingface.co/docs/setfit/installation

Install the latest version directly from the GitHub repository without cloning. This is useful if you want the bleeding-edge version without making changes.

```bash
pip install git+https://github.com/huggingface/setfit.git
```

--------------------------------

### Install torch with CUDA support

Source: https://huggingface.co/docs/setfit/installation

Recommended if you have a CUDA-capable graphics card. This installs torch with CUDA support to speed up training and inference.

```bash
pip install torch --index-url https://download.pytorch.org/whl/cu118
```

--------------------------------

### Install SetFitABSA with pip

Source: https://huggingface.co/docs/setfit/how_to/absa

Install the setfit package with the absa extra, which also requires spaCy. The commented alternative shows installing spaCy separately.

```bash
!pip install "setfit[absa]"
# or
# !pip install spacy
```

--------------------------------

### to usage example

Source: https://huggingface.co/docs/setfit/reference/main

Shows how to move the model to a device (e.g., 'cpu') using the to method. The method returns the model itself after moving.

```python
>>> model = SetFitModel.from_pretrained(...)
>>> model.to("cpu")
>>> model(["cats are cute", "dogs are loyal"])
```

--------------------------------

### Install Optimum with ONNX Runtime GPU support

Source: https://huggingface.co/docs/setfit/tutorials/onnx

Installs the optimum library with ONNX Runtime GPU dependencies. Use this before exporting models to ONNX with CUDA support.

```bash
!pip install optimum[onnxruntime-gpu] -qqq
```

--------------------------------

### setfit.get_templated_dataset

Source: https://huggingface.co/docs/setfit/reference/utility

Creates templated examples for a reference dataset or reference labels. If candidate_labels is supplied, it uses them for generating templates; otherwise, it uses labels loaded from reference_dataset. If an input Dataset is supplied, the examples are added to it; otherwise, a new Dataset is created.

```APIDOC
## setfit.get_templated_dataset

### Description
Creates templated examples for a reference dataset or reference labels. If `candidate_labels` is supplied, it uses them for generating templates; otherwise, it uses labels loaded from `reference_dataset`. If an input Dataset is supplied, the examples are added to it; otherwise, a new Dataset is created.

### Method
Function

### Signature
```python
setfit.get_templated_dataset(
    dataset: typing.Optional[datasets.arrow_dataset.Dataset] = None,
    candidate_labels: typing.Optional[typing.List[str]] = None,
    reference_dataset: typing.Optional[str] = None,
    template: str = 'This sentence is {}',
    sample_size: int = 2,
    text_column: str = 'text',
    label_column: str = 'label',
    multi_label: bool = False,
    label_names_column: str = 'label_text'
) -> Dataset
```

### Parameters
- **dataset** (`Dataset`, optional) — A Dataset to add templated examples to.
- **candidate_labels** (`List[str]`, optional) — The list of candidate labels to be fed into the template to construct examples.
- **reference_dataset** (`str`, optional) — A dataset to take labels from, if `candidate_labels` is not supplied.
- **template** (`str`, optional, defaults to `"This sentence is {}"`) — The template used to turn each label into a synthetic training example. This template must include a `{}` for the candidate label to be inserted into the template. For example, the default template is “This sentence is {}.” With the candidate label “sports”, this would produce an example “This sentence is sports”.
- **sample_size** (`int`, optional, defaults to 2) — The number of examples to make for each candidate label.
- **text_column** (`str`, optional, defaults to `"text"`) — The name of the column containing the text of the examples.
- **label_column** (`str`, optional, defaults to `"label"`) — The name of the column in `dataset` containing the labels of the examples.
- **multi_label** (`bool`, optional, defaults to `False`) — Whether or not multiple candidate labels can be true.
- **label_names_column** (`str`, optional, defaults to `"label_text"`) — The name of the label column in the `reference_dataset`, to be used in case there is no ClassLabel feature for the label column.

### Returns
- **Dataset** — A copy of the input Dataset with templated examples added.

### Raises
- **ValueError** — If the input Dataset is not empty and one or both of the provided column names are missing.

```

--------------------------------

### Install SetFit with conda

Source: https://huggingface.co/docs/setfit/installation

Install SetFit using the conda package manager from the conda-forge channel.

```bash
conda install -c conda-forge setfit
```

--------------------------------

### Load training and eval datasets for SetFitABSA

Source: https://huggingface.co/docs/setfit/how_to/absa

Loads a dataset with text, span, label, and ordinal columns. The example uses the SemEval-2014 restaurants dataset and selects 128 training and 128 eval examples.

```python
from dataset import load_dataset

# The training/eval dataset must have `text`, `span`, `label`, and `ordinal` columns
dataset = load_dataset("tomaarsen/setfit-absa-semeval-restaurants", split="train")
train_dataset = dataset.select(range(128))
eval_dataset = dataset.select(range(128, 256))
```

--------------------------------

### Load SetFitModel with from_pretrained

Source: https://huggingface.co/docs/setfit/reference/main

Downloads a model from the Hugging Face Hub and instantiates it. The example uses the model 'sentence-transformers/paraphrase-mpnet-base-v2' with labels for binary classification.

```python
>>> from setfit import SetFitModel
>>> model = SetFitModel.from_pretrained(
...     "sentence-transformers/paraphrase-mpnet-base-v2",
...     labels=["positive", "negative"],
... )
```

--------------------------------

### Example model card metadata with custom metrics

Source: https://huggingface.co/docs/setfit/how_to/model_cards

Shows the YAML metadata that will be included in the model card when custom metrics are used. The values are examples and will vary based on your training results.

```yaml
    metrics:
    - type: accuracy
      value: 0.8061504667764964
      name: Accuracy
    - type: precision
      value: 0.7293729372937293
      name: Precision
    - type: recall
      value: 0.9724972497249725
      name: Recall
    - type: f1
      value: 0.8335690711928335
      name: F1
```

--------------------------------

### Sample a small labeled dataset with sample_dataset

Source: https://huggingface.co/docs/setfit/tutorials/zero_shot

Samples 8 labeled examples per class from the reference dataset using setfit.sample_dataset. The resulting dataset has 48 rows.

```python
from setfit import sample_dataset

train_dataset = sample_dataset(reference_dataset["train"])
train_dataset
```

```output
Dataset({
    features: ['text', 'label'],
    num_rows: 48
})
```

--------------------------------

### SetFitModelCardData usage example

Source: https://huggingface.co/docs/setfit/reference/main

Example of creating a SetFitModel with model_card_data to customize the model card. Requires installing codecarbon to track carbon emissions.

```python
>>> model = SetFitModel.from_pretrained(
...     "sentence-transformers/paraphrase-mpnet-base-v2",
...     labels=["negative", "positive"],
...     # Model card variables
...     model_card_data=SetFitModelCardData(
...         model_id="tomaarsen/setfit-paraphrase-mpnet-base-v2-sst2",
...         dataset_name="SST2",
...         dataset_id="sst2",
...         license="apache-2.0",
...         language="en",
...     ),
... )
```

--------------------------------

### from_pretrained

Source: https://huggingface.co/docs/setfit/reference/main

Download a model from the Hugging Face Hub and instantiate it.

```APIDOC
## from_pretrained

### Description
Download a model from the Hugging Face Hub and instantiate it.

### Method
class method

### Parameters
- **pretrained_model_name_or_path** (`str`, `Path`) - Either the `model_id` (string) of a model hosted on the Hub, e.g. `bigscience/bloom`, or a path to a `directory` containing model weights saved using `save_pretrained`, e.g., `../path/to/my_model_directory/`.
- **revision** (`str`, optional) - Revision of the model on the Hub. Can be a branch name, a git tag or any commit id. Defaults to the latest commit on `main` branch.
- **force_download** (`bool`, optional, defaults to `False`) - Whether to force (re-)downloading the model weights and configuration files from the Hub, overriding the existing cache.
- **proxies** (`Dict[str, str]`, optional) - A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}`. The proxies are used on every request.
- **token** (`str` or `bool`, optional) - The token to use as HTTP bearer authorization for remote files. By default, it will use the token cached when running `hf auth login`.
- **cache_dir** (`str`, `Path`, optional) - Path to the folder where cached files are stored.
- **local_files_only** (`bool`, optional, defaults to `False`) - If `True`, avoid downloading the file and return the path to the local cached file if it exists.
- **labels** (`List[str]`, optional) - If the labels are integers ranging from `0` to `num_classes-1`, then these labels indicate the corresponding labels.
- **model_card_data** (`SetFitModelCardData`, optional) - A `SetFitModelCardData` instance storing data such as model language, license, dataset name, etc. to be used in the automatically generated model cards.
- **use_differentiable_head** (`bool`, optional) - Whether to load SetFit using a differentiable (i.e., Torch) head instead of Logistic Regression.
- **normalize_embeddings** (`bool`, optional) - Whether to apply normalization on the embeddings produced by the Sentence Transformer body.
- **span_context** (`int`, defaults to `0`) - The number of words before and after the span candidate that should be prepended to the full sentence. By default, 0 for Aspect models and 3 for Polarity models.
- **device** (`Union[torch.device, str]`, optional) - The device on which to load the SetFit model, e.g. `"cuda:0"`, `"mps"` or `torch.device("cuda")`.
```

--------------------------------

### Zero-shot prediction example output

Source: https://huggingface.co/docs/setfit/tutorials/zero_shot

Example output of a zero-shot prediction, showing the ranked labels and their scores.

```text
{'sequence': 'im feeling rather rotten so im not very ambitious right now',
 'labels': ['sadness', 'anger', 'surprise', 'fear', 'joy', 'love'],
 'scores': [0.7367985844612122,
  0.10041674226522446,
  0.09770156443119049,
  0.05880110710859299,
  0.004266355652362108,
  0.0020156768150627613]}
```

--------------------------------

### Create a SetFit model with custom model card data

Source: https://huggingface.co/docs/setfit/how_to/model_cards

Use SetFitModel.from_pretrained with a model_card_data argument to include metadata in the generated model card. The example sets language, license, dataset ID, and dataset name.

```python
from setfit import SetFitModel

model = SetFitModel.from_pretrained(
    "BAAI/bge-small-en-v1.5",
    model_card_data=SetFitModelCardData(
        language="en",
        license="apache-2.0",
        dataset_id="sst2",
        dataset_name="SST2",
    )
)
```

--------------------------------

### Inspect synthetic dataset examples

Source: https://huggingface.co/docs/setfit/tutorials/zero_shot

Shuffles the synthetic dataset and displays the first three examples, showing the template text and corresponding label indices.

```python
train_dataset.shuffle()[:3]
```

```output
{'text': ['This sentence is love',
  'This sentence is fear',
  'This sentence is joy'],
 'label': [2, 4, 1]}
```

--------------------------------

### Inspect synthetic dataset

Source: https://huggingface.co/docs/setfit/how_to/zero_shot

Prints the dataset structure and the first example. The output shows the dataset has 48 rows and the first example maps 'This sentence is sadness' to label 0.

```python
print(train_dataset)
# => Dataset({
#     features: ['text', 'label'],
#     num_rows: 48
# })
print(train_dataset[0])
# {'text': 'This sentence is sadness', 'label': 0}
```

--------------------------------

### from_pretrained

Source: https://huggingface.co/docs/setfit/v1.1.3/en/reference/main

Downloads a model from the Hugging Face Hub and instantiates a SetFitModel. Accepts a model identifier or local directory path, along with various optional parameters for revision, caching, device, and model configuration.

```APIDOC
## from_pretrained

### Description
Downloads a model from the Hugging Face Hub and instantiates a SetFitModel.

### Method
from_pretrained

### Parameters
- **pretrained_model_name_or_path** (`str`, `Path`) - Required - Either the `model_id` (string) of a model hosted on the Hub, e.g. `bigscience/bloom`, or a path to a `directory` containing model weights saved using `save_pretrained`, e.g., `../path/to/my_model_directory/`.
- **revision** (`str`, optional) - Revision of the model on the Hub. Can be a branch name, a git tag or any commit id. Defaults to the latest commit on `main` branch.
- **force_download** (`bool`, optional, defaults to `False`) - Whether to force (re-)downloading the model weights and configuration files from the Hub, overriding the existing cache.
- **proxies** (`Dict[str, str]`, optional) - A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}`. The proxies are used on every request.
- **token** (`str` or `bool`, optional) - The token to use as HTTP bearer authorization for remote files. By default, it will use the token cached when running `hf auth login`.
- **cache_dir** (`str`, `Path`, optional) - Path to the folder where cached files are stored.
- **local_files_only** (`bool`, optional, defaults to `False`) - If `True`, avoid downloading the file and return the path to the local cached file if it exists.
- **labels** (`List[str]`, optional) - If the labels are integers ranging from `0` to `num_classes-1`, then these labels indicate the corresponding labels.
- **model_card_data** (`SetFitModelCardData`, optional) - A `SetFitModelCardData` instance storing data such as model language, license, dataset name, etc. to be used in the automatically generated model cards.
- **use_differentiable_head** (`bool`, optional) - Whether to load SetFit using a differentiable (i.e., Torch) head instead of Logistic Regression.
- **normalize_embeddings** (`bool`, optional) - Whether to apply normalization on the embeddings produced by the Sentence Transformer body.
- **span_context** (`int`, defaults to `0`) - The number of words before and after the span candidate that should be prepended to the full sentence. By default, 0 for Aspect models and 3 for Polarity models.
- **device** (`Union[torch.device, str]`, optional) - The device on which to load the SetFit model, e.g. `"cuda:0"`, `"mps"` or `torch.device("cuda")`.

### Returns
A SetFitModel instance.
```

--------------------------------

### Fine-tune base model from scratch

Source: https://huggingface.co/docs/setfit/tutorials/onnx

Commented-out code to fine-tune the base bge-small model using SetFit's Trainer. It loads the base model, sets training arguments with 20 iterations, trains, and then evaluates with a performance benchmark.

```python
# # Fine-tune the base model and Evaluate!
# from setfit import SetFitModel, Trainer, TrainingArguments

# # Load pretrained model from the Hub
# small_model = SetFitModel.from_pretrained(
#    "BAAI/bge-small-en-v1.5"
# )
# args = TrainingArguments(num_iterations=20)

# # Create trainer
# small_trainer = Trainer(
#    model=small_model, args=args, train_dataset=train_dataset
# )
# # Train!
# small_trainer.train()

# # Evaluate!
# pb = PerformanceBenchmark(
#    model=small_trainer.model, dataset=test_dataset, optim_type="bge-small (base)"
# )
# perf_metrics = pb.run_benchmark()
```

--------------------------------

### device

Source: https://huggingface.co/docs/setfit/reference/main

Get the Torch device that this model is on.

```APIDOC
## device

### Description
Get the Torch device that this model is on.

### Method
property

### Returns
- **torch.device** - The device that the model is on.
```

--------------------------------

### from_pretrained

Source: https://huggingface.co/docs/setfit/reference/main

Downloads a model from the Hugging Face Hub and instantiates a SetFitModel. Accepts a model identifier or local directory path, along with various optional parameters for revision, caching, token, and model configuration.

```APIDOC
## from_pretrained

### Description
Downloads a model from the Hugging Face Hub and instantiates a SetFitModel. Accepts a model identifier or local directory path, along with various optional parameters for revision, caching, token, and model configuration.

### Method
from_pretrained

### Signature
from_pretrained(force_download: bool = False, resume_download: Optional[bool] = None, proxies: Optional[Dict] = None, token: Union[bool, str, None] = None, cache_dir: Union[str, Path, None] = None, local_files_only: bool = False, revision: Optional[str] = None, **model_kwargs)

### Parameters
- **pretrained_model_name_or_path** (_str_, _Path_) - Required - Either the _model_id_ (string) of a model hosted on the Hub, e.g. _bigscience/bloom_, or a path to a _directory_ containing model weights saved using _~transformers.PreTrainedModel.save_pretrained_, e.g., _../path/to/my_model_directory/_.
- **revision** (_str_, _optional_) - Revision of the model on the Hub. Can be a branch name, a git tag or any commit id. Defaults to the latest commit on _main_ branch.
- **force_download** (_bool_, _optional_, defaults to _False_) - Whether to force (re-)downloading the model weights and configuration files from the Hub, overriding the existing cache.
- **proxies** (_Dict[str, str]_, _optional_) - A dictionary of proxy servers to use by protocol or endpoint, e.g., _{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}_. The proxies are used on every request.
- **token** (_str_ or _bool_, _optional_) - The token to use as HTTP bearer authorization for remote files. By default, it will use the token cached when running _hf auth login_.
- **cache_dir** (_str_, _Path_, _optional_) - Path to the folder where cached files are stored.
- **local_files_only** (_bool_, _optional_, defaults to _False_) - If _True_, avoid downloading the file and return the path to the local cached file if it exists.
- **labels** (_List[str]_, _optional_) - If the labels are integers ranging from _0_ to _num_classes-1_, then these labels indicate the corresponding labels.
- **model_card_data** (_SetFitModelCardData_, _optional_) - A _SetFitModelCardData_ instance storing data such as model language, license, dataset name, etc. to be used in the automatically generated model cards.
- **multi_target_strategy** (_str_, _optional_) - The strategy to use with multi-label classification. One of "one-vs-rest", "multi-output", or "classifier-chain".
- **use_differentiable_head** (_bool_, _optional_) - Whether to load SetFit using a differentiable (i.e., Torch) head instead of Logistic Regression.
- **normalize_embeddings** (_bool_, _optional_) - Whether to apply normalization on the embeddings produced by the Sentence Transformer body.
- **device** (_Union[torch.device, str]_, _optional_) - The device on which to load the SetFit model, e.g. _"cuda:0"_, _"mps"_ or _torch.device("cuda")_.
- **trust_remote_code** (_bool_, defaults to _False_) - Whether or not to allow for custom Sentence Transformers models defined on the Hub in their own modeling files. This option should only be set to True for repositories you trust and in which you have read the code, as it will execute code present on the Hub on your local machine. Defaults to False.

### Example
```python
>>> from setfit import SetFitModel
>>> model = SetFitModel.from_pretrained(
...     "sentence-transformers/paraphrase-mpnet-base-v2",
...     labels=["positive", "negative"],
... )
```
```

--------------------------------

### Create SetFit Trainer

Source: https://huggingface.co/docs/setfit/tutorials/zero_shot

Initializes a Trainer with the model, training dataset, and evaluation dataset. The train_dataset and reference_dataset variables must be defined beforehand.

```python
from setfit import Trainer

trainer = Trainer(
    model=model,
    train_dataset=train_dataset,
    eval_dataset=reference_dataset["test"]
)
```

--------------------------------

### Prediction output

Source: https://huggingface.co/docs/setfit/how_to/zero_shot

Shows the predicted class names for the example sentences.

```text
['sadness', 'joy', 'anger', 'fear', 'surprise']
```

--------------------------------

### Initialize SetFitModel with Custom Head (LinearSVC)

Source: https://huggingface.co/docs/setfit/how_to/classification_heads

Shows how to create a SetFit model with a custom non-differentiable head (LinearSVC) using the regular __init__ method. Requires a SentenceTransformer body and a head that implements the required methods.

```python
from setfit import SetFitModel
from sklearn.svm import LinearSVC
from sentence_transformers import SentenceTransformer

# Initializing a new SetFit model
model_body = SentenceTransformer("BAAI/bge-small-en-v1.5")
model_head = LinearSVC()
model = SetFitModel(model_body, model_head)
```

--------------------------------

### SetFitABSA Training Logs

Source: https://huggingface.co/docs/setfit/how_to/absa

Training logs for the aspect and polarity models, showing loss, learning rate, and epoch values. The aspect model trains on 249 examples for 5 epochs, and the polarity model on 39 examples. The best model is loaded from step 100 for aspect and step 50 for polarity.

```text
***** Running training *****
  Num examples = 249
  Num epochs = 5
  Total optimization steps = 1245
  Total train batch size = 128
{'aspect_embedding_loss': 0.2542, 'learning_rate': 1.6e-07, 'epoch': 0.0}                                                                                          
{'aspect_embedding_loss': 0.2437, 'learning_rate': 8.000000000000001e-06, 'epoch': 0.2}                                                                            
{'eval_aspect_embedding_loss': 0.2511, 'learning_rate': 8.000000000000001e-06, 'epoch': 0.2}                                                                       
{'aspect_embedding_loss': 0.2209, 'learning_rate': 1.6000000000000003e-05, 'epoch': 0.4}                                                                           
{'eval_aspect_embedding_loss': 0.2385, 'learning_rate': 1.6000000000000003e-05, 'epoch': 0.4}                                                                      
{'aspect_embedding_loss': 0.0165, 'learning_rate': 1.955357142857143e-05, 'epoch': 0.6}                                                                            
{'eval_aspect_embedding_loss': 0.2776, 'learning_rate': 1.955357142857143e-05, 'epoch': 0.6}                                                                       
{'aspect_embedding_loss': 0.0158, 'learning_rate': 1.8660714285714287e-05, 'epoch': 0.8}                                                                           
{'eval_aspect_embedding_loss': 0.2848, 'learning_rate': 1.8660714285714287e-05, 'epoch': 0.8}                                                                      
{'aspect_embedding_loss': 0.0015, 'learning_rate': 1.7767857142857143e-05, 'epoch': 1.0}                                                                           
{'eval_aspect_embedding_loss': 0.3133, 'learning_rate': 1.7767857142857143e-05, 'epoch': 1.0}                                                                      
{'aspect_embedding_loss': 0.0012, 'learning_rate': 1.6875e-05, 'epoch': 1.2}                                                                                       
{'eval_aspect_embedding_loss': 0.2966, 'learning_rate': 1.6875e-05, 'epoch': 1.2}                                                                                  
{'aspect_embedding_loss': 0.0009, 'learning_rate': 1.598214285714286e-05, 'epoch': 1.41}                                                                           
{'eval_aspect_embedding_loss': 0.2996, 'learning_rate': 1.598214285714286e-05, 'epoch': 1.41}                                                                      
 28%|██████████████████████████████████▎                                                                                       | 350/1245 [03:40<09:24,  1.59it/s] 
Loading best SentenceTransformer model from step 100.
{'train_runtime': 226.7429, 'train_samples_per_second': 702.822, 'train_steps_per_second': 5.491, 'epoch': 1.41}
***** Running training *****
  Num examples = 39
  Num epochs = 5
  Total optimization steps = 195
  Total train batch size = 128
{'polarity_embedding_loss': 0.2267, 'learning_rate': 1.0000000000000002e-06, 'epoch': 0.03}                                                                        
{'polarity_embedding_loss': 0.1038, 'learning_rate': 1.6571428571428574e-05, 'epoch': 1.28}                                                                        
{'eval_polarity_embedding_loss': 0.1946, 'learning_rate': 1.6571428571428574e-05, 'epoch': 1.28}                                                                   
{'polarity_embedding_loss': 0.0116, 'learning_rate': 1.0857142857142858e-05, 'epoch': 2.56}                                                                        
{'eval_polarity_embedding_loss': 0.2364, 'learning_rate': 1.0857142857142858e-05, 'epoch': 2.56}                                                                   
{'polarity_embedding_loss': 0.0059, 'learning_rate': 5.142857142857142e-06, 'epoch': 3.85}                                                                         
{'eval_polarity_embedding_loss': 0.2401, 'learning_rate': 5.142857142857142e-06, 'epoch': 3.85}                                                                    
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 195/195 [00:54<00:00,  3.58it/s]
Loading best SentenceTransformer model from step 50.
{'train_runtime': 54.4104, 'train_samples_per_second': 458.736, 'train_steps_per_second': 3.584, 'epoch': 5.0}
```

--------------------------------

### Augment dataset with get_templated_dataset

Source: https://huggingface.co/docs/setfit/tutorials/zero_shot

Generates synthetic examples using get_templated_dataset with candidate_labels and sample_size=8, resulting in a dataset with 96 rows.

```python
augmented_dataset = get_templated_dataset(train_dataset, candidate_labels=candidate_labels, sample_size=8)
augmented_dataset
```

```output
Dataset({
    features: ['text', 'label'],
    num_rows: 96
})
```

--------------------------------

### Train SetFit model and observe training logs

Source: https://huggingface.co/docs/setfit/quickstart

Runs the training loop. The output shows training progress, loss, learning rate, and runtime metrics.

```python
>>> trainer.train()
***** Running training *****
  Num examples = 5
  Num epochs = 10
  Total optimization steps = 50
  Total train batch size = 32
{'embedding_loss': 0.2077, 'learning_rate': 4.000000000000001e-06, 'epoch': 0.2}                                                                                                                
{'embedding_loss': 0.0097, 'learning_rate': 0.0, 'epoch': 10.0}                                                                                                                                 
{'train_runtime': 14.705, 'train_samples_per_second': 108.807, 'train_steps_per_second': 3.4, 'epoch': 10.0}
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [00:08<00:00,  5.70it/s]
```

--------------------------------

### Initialize SetFit Trainer

Source: https://huggingface.co/docs/setfit/quickstart

Creates a Trainer instance with the model, training arguments, and the sampled training dataset.

```python
>>> from setfit import Trainer

>>> trainer = Trainer(
...     model=model,
...     args=args,
...     train_dataset=train_dataset,
... )
```

--------------------------------

### Training and Inference with Custom Head

Source: https://huggingface.co/docs/setfit/v1.1.3/en/how_to/classification_heads

Example of training a SetFit model with a custom head using the Trainer, and performing evaluation and inference.

```APIDOC
## Training and Inference with Custom Head

### Description
Demonstrates how to train a SetFit model with a custom head, evaluate it, and perform inference.

### Code Example
```python
from setfit import Trainer, TrainingArguments, sample_dataset
from datasets import load_dataset

# Preparing the dataset
dataset = load_dataset("SetFit/sst2")
train_dataset = sample_dataset(dataset["train"], label_column="label", num_samples=32)
test_dataset = dataset["test"]

# Preparing the training arguments
args = TrainingArguments(
    batch_size=32,
    num_epochs=3,
)

# Preparing the trainer
trainer = Trainer(
    model=model,
    args=args,
    train_dataset=train_dataset,
)
trainer.train()

# Evaluating
metrics = trainer.evaluate(test_dataset)
print(metrics)
# => {'accuracy': 0.8638110928061504}

# Performing inference
preds = model.predict([
    "It's a charming and often affecting journey.",
    "It's slow -- very, very slow.",
    "A sometimes tedious film.",
])
print(preds)
# => tensor([1, 0, 0], dtype=torch.int32)
```
```
Tokens:
Raw