### End-to-end SetFit quickstart example

Source: https://huggingface.co/docs/setfit/quickstart

This snippet shows the complete SetFit workflow from model initialization to inference. It uses the 'BAAI/bge-small-en-v1.5' model and the 'SetFit/sst2' dataset, training on 8 samples per label. The expected accuracy is approximately 0.851, and the example predictions are ['positive', 'negative', 'negative']. It demonstrates saving locally with save_pretrained and pushing to the Hub with push_to_hub.

```python
from setfit import SetFitModel, Trainer, TrainingArguments, sample_dataset
from datasets import load_dataset

# Initializing a new SetFit model
model = SetFitModel.from_pretrained("BAAI/bge-small-en-v1.5", labels=["negative", "positive"])

# Preparing the dataset
dataset = load_dataset("SetFit/sst2")
train_dataset = sample_dataset(dataset["train"], label_column="label", num_samples=8)
test_dataset = dataset["test"]

# Preparing the training arguments
args = TrainingArguments(
    batch_size=32,
    num_epochs=10,
)

# Preparing the trainer
trainer = Trainer(
    model=model,
    args=args,
    train_dataset=train_dataset,
)
trainer.train()

# Evaluating
metrics = trainer.evaluate(test_dataset)
print(metrics)
# => {'accuracy': 0.8511806699615596}

# Saving the trained model
model.save_pretrained("setfit-bge-small-v1.5-sst2-8-shot")
# or
model.push_to_hub("tomaarsen/setfit-bge-small-v1.5-sst2-8-shot")

# Loading a trained model
model = SetFitModel.from_pretrained("tomaarsen/setfit-bge-small-v1.5-sst2-8-shot") # Load from the Hugging Face Hub
# or
model = SetFitModel.from_pretrained("setfit-bge-small-v1.5-sst2-8-shot") # Load from a local directory

# Performing inference
preds = model.predict([
    "It's a charming and often affecting journey.",
    "It's slow -- very, very slow.",
    "A sometimes tedious film.",
])
print(preds)
# => ["positive", "negative", "negative"]
```

--------------------------------

### Install Optuna backend

Source: https://huggingface.co/docs/setfit/how_to/hyperparameter_optimization

Install the optuna package required for hyperparameter optimization.

```bash
pip install optuna
```

--------------------------------

### Install SetFit and accelerate

Source: https://huggingface.co/docs/setfit/tutorials/onnx

Installs the required packages for the tutorial using pip. The -qqq flag suppresses output for a cleaner notebook experience.

```bash
!pip install setfit accelerate -qqq
```

--------------------------------

### Install SetFit from source (editable mode)

Source: https://huggingface.co/docs/setfit/installation

Clone the repository and install SetFit in editable mode to make changes to the code base. This sequence of commands clones the repo, changes into the directory, and installs the package with pip install -e .

```bash
git clone https://github.com/huggingface/setfit.git
cd setfit
pip install -e .
```

--------------------------------

### Install setfit and matplotlib

Source: https://huggingface.co/docs/setfit/tutorials/zero_shot

Uncomment and run this cell to install the required libraries when running the notebook on Colab or a similar cloud platform.

```bash
# %pip install setfit matplotlib
```

--------------------------------

### Install SetFit with pip

Source: https://huggingface.co/docs/setfit/installation

The most straightforward way to install SetFit is with pip. This command installs the latest version of the setfit package.

```bash
pip install setfit
```

--------------------------------

### Install SetFit from source (bleeding-edge)

Source: https://huggingface.co/docs/setfit/installation

Install the latest version directly from the GitHub repository without cloning. This is useful if you want the bleeding-edge version without making changes.

```bash
pip install git+https://github.com/huggingface/setfit.git
```

--------------------------------

### Install torch with CUDA support

Source: https://huggingface.co/docs/setfit/installation

Recommended if you have a CUDA-capable graphics card. This installs torch with CUDA support to speed up training and inference.

```bash
pip install torch --index-url https://download.pytorch.org/whl/cu118
```

--------------------------------

### Install SetFitABSA with pip

Source: https://huggingface.co/docs/setfit/how_to/absa

Install the setfit package with the absa extra, which also requires spaCy. The commented alternative shows installing spaCy separately.

```bash
!pip install "setfit[absa]"
# or
# !pip install spacy
```

--------------------------------

### to usage example

Source: https://huggingface.co/docs/setfit/reference/main

Shows how to move the model to a device (e.g., 'cpu') using the to method. The method returns the model itself after moving.

```python
>>> model = SetFitModel.from_pretrained(...)
>>> model.to("cpu")
>>> model(["cats are cute", "dogs are loyal"])
```

--------------------------------

### Install Optimum with ONNX Runtime GPU support

Source: https://huggingface.co/docs/setfit/tutorials/onnx

Installs the optimum library with ONNX Runtime GPU dependencies. Use this before exporting models to ONNX with CUDA support.

```bash
!pip install optimum[onnxruntime-gpu] -qqq
```

--------------------------------

### setfit.get_templated_dataset

Source: https://huggingface.co/docs/setfit/reference/utility

Creates templated examples for a reference dataset or reference labels. If candidate_labels is supplied, it uses them for generating templates; otherwise, it uses labels loaded from reference_dataset. If an input Dataset is supplied, the examples are added to it; otherwise, a new Dataset is created.

```APIDOC
## setfit.get_templated_dataset

### Description
Creates templated examples for a reference dataset or reference labels. If `candidate_labels` is supplied, it uses them for generating templates; otherwise, it uses labels loaded from `reference_dataset`. If an input Dataset is supplied, the examples are added to it; otherwise, a new Dataset is created.

### Method
Function

### Signature
```python
setfit.get_templated_dataset(
    dataset: typing.Optional[datasets.arrow_dataset.Dataset] = None,
    candidate_labels: typing.Optional[typing.List[str]] = None,
    reference_dataset: typing.Optional[str] = None,
    template: str = 'This sentence is {}',
    sample_size: int = 2,
    text_column: str = 'text',
    label_column: str = 'label',
    multi_label: bool = False,
    label_names_column: str = 'label_text'
) -> Dataset
```

### Parameters
- **dataset** (`Dataset`, optional) — A Dataset to add templated examples to.
- **candidate_labels** (`List[str]`, optional) — The list of candidate labels to be fed into the template to construct examples.
- **reference_dataset** (`str`, optional) — A dataset to take labels from, if `candidate_labels` is not supplied.
- **template** (`str`, optional, defaults to `"This sentence is {}"`) — The template used to turn each label into a synthetic training example. This template must include a `{}` for the candidate label to be inserted into the template. For example, the default template is “This sentence is {}.” With the candidate label “sports”, this would produce an example “This sentence is sports”.
- **sample_size** (`int`, optional, defaults to 2) — The number of examples to make for each candidate label.
- **text_column** (`str`, optional, defaults to `"text"`) — The name of the column containing the text of the examples.
- **label_column** (`str`, optional, defaults to `"label"`) — The name of the column in `dataset` containing the labels of the examples.
- **multi_label** (`bool`, optional, defaults to `False`) — Whether or not multiple candidate labels can be true.
- **label_names_column** (`str`, optional, defaults to `"label_text"`) — The name of the label column in the `reference_dataset`, to be used in case there is no ClassLabel feature for the label column.

### Returns
- **Dataset** — A copy of the input Dataset with templated examples added.

### Raises
- **ValueError** — If the input Dataset is not empty and one or both of the provided column names are missing.

```

--------------------------------

### Install SetFit with conda

Source: https://huggingface.co/docs/setfit/installation

Install SetFit using the conda package manager from the conda-forge channel.

```bash
conda install -c conda-forge setfit
```

--------------------------------

### SetFitModelCardData usage example

Source: https://huggingface.co/docs/setfit/reference/main

Example of creating a SetFitModel with model_card_data to customize the model card. Requires installing codecarbon to track carbon emissions.

```python
>>> model = SetFitModel.from_pretrained(
...     "sentence-transformers/paraphrase-mpnet-base-v2",
...     labels=["negative", "positive"],
...     # Model card variables
...     model_card_data=SetFitModelCardData(
...         model_id="tomaarsen/setfit-paraphrase-mpnet-base-v2-sst2",
...         dataset_name="SST2",
...         dataset_id="sst2",
...         license="apache-2.0",
...         language="en",
...     ),
... )
```

--------------------------------

### Load training and eval datasets for SetFitABSA

Source: https://huggingface.co/docs/setfit/how_to/absa

Loads a dataset with text, span, label, and ordinal columns. The example uses the SemEval-2014 restaurants dataset and selects 128 training and 128 eval examples.

```python
from dataset import load_dataset

# The training/eval dataset must have `text`, `span`, `label`, and `ordinal` columns
dataset = load_dataset("tomaarsen/setfit-absa-semeval-restaurants", split="train")
train_dataset = dataset.select(range(128))
eval_dataset = dataset.select(range(128, 256))
```

--------------------------------

### Load SetFitModel with from_pretrained

Source: https://huggingface.co/docs/setfit/reference/main

Downloads a model from the Hugging Face Hub and instantiates it. The example uses the model 'sentence-transformers/paraphrase-mpnet-base-v2' with labels for binary classification.

```python
>>> from setfit import SetFitModel
>>> model = SetFitModel.from_pretrained(
...     "sentence-transformers/paraphrase-mpnet-base-v2",
...     labels=["positive", "negative"],
... )
```

--------------------------------

### Example model card metadata with custom metrics

Source: https://huggingface.co/docs/setfit/how_to/model_cards

Shows the YAML metadata that will be included in the model card when custom metrics are used. The values are examples and will vary based on your training results.

```yaml
    metrics:
    - type: accuracy
      value: 0.8061504667764964
      name: Accuracy
    - type: precision
      value: 0.7293729372937293
      name: Precision
    - type: recall
      value: 0.9724972497249725
      name: Recall
    - type: f1
      value: 0.8335690711928335
      name: F1
```

--------------------------------

### Create a SetFit model with custom model card data

Source: https://huggingface.co/docs/setfit/how_to/model_cards

Use SetFitModel.from_pretrained with a model_card_data argument to include metadata in the generated model card. The example sets language, license, dataset ID, and dataset name.

```python
from setfit import SetFitModel

model = SetFitModel.from_pretrained(
    "BAAI/bge-small-en-v1.5",
    model_card_data=SetFitModelCardData(
        language="en",
        license="apache-2.0",
        dataset_id="sst2",
        dataset_name="SST2",
    )
)
```

--------------------------------

### Inspect synthetic dataset examples

Source: https://huggingface.co/docs/setfit/tutorials/zero_shot

Shuffles the synthetic dataset and displays the first three examples, showing the template text and corresponding label indices.

```python
train_dataset.shuffle()[:3]
```

```output
{'text': ['This sentence is love',
  'This sentence is fear',
  'This sentence is joy'],
 'label': [2, 4, 1]}
```

--------------------------------

### Zero-shot prediction example output

Source: https://huggingface.co/docs/setfit/tutorials/zero_shot

Example output of a zero-shot prediction, showing the ranked labels and their scores.

```text
{'sequence': 'im feeling rather rotten so im not very ambitious right now',
 'labels': ['sadness', 'anger', 'surprise', 'fear', 'joy', 'love'],
 'scores': [0.7367985844612122,
  0.10041674226522446,
  0.09770156443119049,
  0.05880110710859299,
  0.004266355652362108,
  0.0020156768150627613]}
```

--------------------------------

### Sample a small labeled dataset with sample_dataset

Source: https://huggingface.co/docs/setfit/tutorials/zero_shot

Samples 8 labeled examples per class from the reference dataset using setfit.sample_dataset. The resulting dataset has 48 rows.

```python
from setfit import sample_dataset

train_dataset = sample_dataset(reference_dataset["train"])
train_dataset
```

```output
Dataset({
    features: ['text', 'label'],
    num_rows: 48
})
```

--------------------------------

### Training and Inference with Custom Head

Source: https://huggingface.co/docs/setfit/v1.1.3/en/how_to/classification_heads

Example of training a SetFit model with a custom head using the Trainer, and performing evaluation and inference.

```APIDOC
## Training and Inference with Custom Head

### Description
Demonstrates how to train a SetFit model with a custom head, evaluate it, and perform inference.

### Code Example
```python
from setfit import Trainer, TrainingArguments, sample_dataset
from datasets import load_dataset

# Preparing the dataset
dataset = load_dataset("SetFit/sst2")
train_dataset = sample_dataset(dataset["train"], label_column="label", num_samples=32)
test_dataset = dataset["test"]

# Preparing the training arguments
args = TrainingArguments(
    batch_size=32,
    num_epochs=3,
)

# Preparing the trainer
trainer = Trainer(
    model=model,
    args=args,
    train_dataset=train_dataset,
)
trainer.train()

# Evaluating
metrics = trainer.evaluate(test_dataset)
print(metrics)
# => {'accuracy': 0.8638110928061504}

# Performing inference
preds = model.predict([
    "It's a charming and often affecting journey.",
    "It's slow -- very, very slow.",
    "A sometimes tedious film.",
])
print(preds)
# => tensor([1, 0, 0], dtype=torch.int32)
```
```

--------------------------------

### Prepare unlabeled dataset for knowledge distillation

Source: https://huggingface.co/docs/setfit/how_to/knowledge_distillation

Creates a dataset of 500 unlabeled examples by shuffling the training split and removing the 'label' column. The resulting dataset has only the 'text' feature and 500 rows.

```python
# Create a dataset of unlabeled examples to perform knowledge distillation
unlabeled_train_dataset = dataset["train"].shuffle(seed=0).select(range(500))
unlabeled_train_dataset = unlabeled_train_dataset.remove_columns("label")
# Dataset({
#     features: ['text'],
#     num_rows: 500
# })
```

--------------------------------

### Initializing a SetFit Model with a Custom Head

Source: https://huggingface.co/docs/setfit/how_to/classification_heads

Example of initializing a SetFit model with a custom non-differentiable head (LinearSVC) and then training and performing inference.

```APIDOC
## Initialize SetFitModel with Custom Head

### Description
Shows how to create a SetFitModel with a custom classification head (e.g., LinearSVC) and use it for training and inference.

### Code Example
```python
from setfit import SetFitModel
from sklearn.svm import LinearSVC
from sentence_transformers import SentenceTransformer

# Initializing a new SetFit model
model_body = SentenceTransformer("BAAI/bge-small-en-v1.5")
model_head = LinearSVC()
model = SetFitModel(model_body, model_head)
```

### Training and Inference Example
```python
from setfit import Trainer, TrainingArguments, sample_dataset
from datasets import load_dataset

# Preparing the dataset
dataset = load_dataset("SetFit/sst2")
train_dataset = sample_dataset(dataset["train"], label_column="label", num_samples=32)
test_dataset = dataset["test"]

# Preparing the training arguments
args = TrainingArguments(
    batch_size=32,
    num_epochs=3,
)

# Preparing the trainer
trainer = Trainer(
    model=model,
    args=args,
    train_dataset=train_dataset,
)
trainer.train()

# Evaluating
metrics = trainer.evaluate(test_dataset)
print(metrics)
# => {'accuracy': 0.8638110928061504}

# Performing inference
preds = model.predict([
    "It's a charming and often affecting journey.",
    "It's slow -- very, very slow.",
    "A sometimes tedious film.",
])
print(preds)
# => tensor([1, 0, 0], dtype=torch.int32)
```
```

--------------------------------

### Load SetFit model from Hub or local directory

Source: https://huggingface.co/docs/setfit/quickstart

Loads a SetFit model using SetFitModel.from_pretrained() either from the Hugging Face Hub or from a local directory. Two examples are shown.

```python
>>> model = SetFitModel.from_pretrained("tomaarsen/setfit-bge-small-v1.5-sst2-8-shot") # Load from the Hugging Face Hub

>>> model = SetFitModel.from_pretrained("setfit-bge-small-v1.5-sst2-8-shot") # Load from a local directory
```

--------------------------------

### Initializing a SetFit Model with a Custom Head

Source: https://huggingface.co/docs/setfit/v1.1.3/en/how_to/classification_heads

Example of initializing a SetFit model with a custom non-differentiable head (LinearSVC) using the regular __init__ method.

```APIDOC
## Initialize SetFitModel with Custom Head

### Description
Shows how to create a SetFitModel with a custom classification head using the `__init__` method.

### Code Example
```python
from setfit import SetFitModel
from sklearn.svm import LinearSVC
from sentence_transformers import SentenceTransformer

# Initializing a new SetFit model
model_body = SentenceTransformer("BAAI/bge-small-en-v1.5")
model_head = LinearSVC()
model = SetFitModel(model_body, model_head)
```
```

--------------------------------

### Distill knowledge with DistillationTrainer

Source: https://huggingface.co/docs/setfit/how_to/knowledge_distillation

Sets up and runs knowledge distillation using DistillationTrainer with a teacher model, student model, and unlabeled dataset. The training arguments use a batch size of 16 and max_steps of 500 to avoid overfitting, as the number of training steps grows exponentially with the number of unlabeled examples.

```python
from setfit import DistillationTrainer

distillation_args = TrainingArguments(
    batch_size=16,
    max_steps=500,
)

distillation_trainer = DistillationTrainer(
    teacher_model=teacher_model,
    student_model=model,
    args=distillation_args,
    train_dataset=unlabeled_train_dataset,
    eval_dataset=eval_dataset,
)
# Train student with knowledge distillation
distillation_trainer.train()
distillation_metrics = distillation_trainer.evaluate()
print(distillation_metrics)
```

--------------------------------

### Predict classes with SetFitModel.predict

Source: https://huggingface.co/docs/setfit/reference/main

Shows how to use the predict method to classify sentences. The example demonstrates both list and single-string inputs, returning labels when SetFitModel.labels is defined.

```python
>>> model = SetFitModel.from_pretrained(...)
>>> model.predict(["What a boring display", "Exhilarating through and through", "I'm wowed!"])
["negative", "positive", "positive"]
>>> model.predict("That was cool!")
"positive"
```

--------------------------------

### Initialize SetFit model with custom head_params for LogisticRegression

Source: https://huggingface.co/docs/setfit/how_to/classification_heads

Demonstrates passing additional parameters to the classification head via the head_params argument. The example sets solver to 'liblinear' and max_iter to 300.

```python
>>> from setfit import SetFitModel

>>> model = SetFitModel.from_pretrained("BAAI/bge-small-en-v1.5", head_params={"solver": "liblinear", "max_iter": 300})
>>> model.model_head
LogisticRegression(max_iter=300, solver='liblinear')
```

--------------------------------

### End-to-end knowledge distillation with SetFit

Source: https://huggingface.co/docs/setfit/how_to/knowledge_distillation

This snippet shows the entire knowledge distillation strategy in an end-to-end example. It loads the ag_news dataset, creates a few-shot training set, trains a student model, prepares unlabeled data, trains a teacher model, and then uses DistillationTrainer to distill knowledge from the teacher to the student.

```python
from datasets import load_dataset
from setfit import sample_dataset

# Load a dataset from the Hugging Face Hub
dataset = load_dataset("ag_news")

# Create a sample few-shot dataset to train with
train_dataset = sample_dataset(dataset["train"], label_column="label", num_samples=16)
# Dataset({
#     features: ['text', 'label'],
#     num_rows: 64
# })

# Dataset for evaluation
eval_dataset = dataset["test"]
# Dataset({
#     features: ['text', 'label'],
#     num_rows: 7600
# })

from setfit import SetFitModel, TrainingArguments, Trainer

model = SetFitModel.from_pretrained("sentence-transformers/paraphrase-MiniLM-L3-v2")

args = TrainingArguments(
    batch_size=64,
    num_epochs=5,
)

trainer = Trainer(
    model=model,
    args=args,
    train_dataset=train_dataset,
    eval_dataset=eval_dataset,
)
trainer.train()

metrics = trainer.evaluate()
print(metrics)

# Create a dataset of unlabeled examples to perform knowledge distillation
unlabeled_train_dataset = dataset["train"].shuffle(seed=0).select(range(500))
unlabeled_train_dataset = unlabeled_train_dataset.remove_columns("label")
# Dataset({
#     features: ['text'],
#     num_rows: 500
# })

from setfit import SetFitModel

teacher_model = SetFitModel.from_pretrained("sentence-transformers/paraphrase-mpnet-base-v2")

from setfit import TrainingArguments, Trainer

teacher_args = TrainingArguments(
    batch_size=16,
    num_epochs=2,
)

teacher_trainer = Trainer(
    model=teacher_model,
    args=teacher_args,
    train_dataset=train_dataset,
    eval_dataset=eval_dataset,
)

# Train teacher model
teacher_trainer.train()
teacher_metrics = teacher_trainer.evaluate()
print(teacher_metrics)

from setfit import DistillationTrainer

distillation_args = TrainingArguments(
    batch_size=16,
    max_steps=500,
)

distillation_trainer = DistillationTrainer(
    teacher_model=teacher_model,
    student_model=model,
    args=distillation_args,
    train_dataset=unlabeled_train_dataset,
    eval_dataset=eval_dataset,
)
# Train student with knowledge distillation
distillation_trainer.train()
distillation_metrics = distillation_trainer.evaluate()
print(distillation_metrics)
```

--------------------------------

### Train SetFit model with default arguments

Source: https://huggingface.co/docs/setfit/how_to/hyperparameter_optimization

Trains a SetFit model using the default Trainer settings on the SetFit/emotion dataset. The model is loaded from 'BAAI/bge-small-en-v1.5', and a sample of 8 examples per label is used for training. After training, the model is evaluated on the test set and the metrics are printed.

```python
from datasets import load_dataset
from setfit import SetFitModel, Trainer, sample_dataset

model = SetFitModel.from_pretrained("BAAI/bge-small-en-v1.5")

dataset = load_dataset("SetFit/emotion")
train_dataset = sample_dataset(dataset["train"], label_column="label", num_samples=8)
test_dataset = dataset["test"]

trainer = Trainer(
    model=model,
    train_dataset=train_dataset,
    eval_dataset=test_dataset,
)
trainer.train()

metrics = trainer.evaluate()
print(metrics)
```

--------------------------------

### SetFitModel prediction example

Source: https://huggingface.co/docs/setfit/reference/main

Shows how to use the __call__ method to predict labels for a list of sentences or a single sentence. The model must be loaded with SetFitModel.from_pretrained. Returns a list of string labels for multiple inputs, or a single string for a single input.

```python
>>> model = SetFitModel.from_pretrained(...)
>>> model(["What a boring display", "Exhilarating through and through", "I'm wowed!"])
["negative", "positive", "positive"]
>>> model("That was cool!")
"positive"
```

--------------------------------

### from_pretrained

Source: https://huggingface.co/docs/setfit/reference/main

Download a model from the Hugging Face Hub and instantiate it.

```APIDOC
## from_pretrained

### Description
Download a model from the Hugging Face Hub and instantiate it.

### Method
class method

### Parameters
- **pretrained_model_name_or_path** (`str`, `Path`) - Either the `model_id` (string) of a model hosted on the Hub, e.g. `bigscience/bloom`, or a path to a `directory` containing model weights saved using `save_pretrained`, e.g., `../path/to/my_model_directory/`.
- **revision** (`str`, optional) - Revision of the model on the Hub. Can be a branch name, a git tag or any commit id. Defaults to the latest commit on `main` branch.
- **force_download** (`bool`, optional, defaults to `False`) - Whether to force (re-)downloading the model weights and configuration files from the Hub, overriding the existing cache.
- **proxies** (`Dict[str, str]`, optional) - A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}`. The proxies are used on every request.
- **token** (`str` or `bool`, optional) - The token to use as HTTP bearer authorization for remote files. By default, it will use the token cached when running `hf auth login`.
- **cache_dir** (`str`, `Path`, optional) - Path to the folder where cached files are stored.
- **local_files_only** (`bool`, optional, defaults to `False`) - If `True`, avoid downloading the file and return the path to the local cached file if it exists.
- **labels** (`List[str]`, optional) - If the labels are integers ranging from `0` to `num_classes-1`, then these labels indicate the corresponding labels.
- **model_card_data** (`SetFitModelCardData`, optional) - A `SetFitModelCardData` instance storing data such as model language, license, dataset name, etc. to be used in the automatically generated model cards.
- **use_differentiable_head** (`bool`, optional) - Whether to load SetFit using a differentiable (i.e., Torch) head instead of Logistic Regression.
- **normalize_embeddings** (`bool`, optional) - Whether to apply normalization on the embeddings produced by the Sentence Transformer body.
- **span_context** (`int`, defaults to `0`) - The number of words before and after the span candidate that should be prepended to the full sentence. By default, 0 for Aspect models and 3 for Polarity models.
- **device** (`Union[torch.device, str]`, optional) - The device on which to load the SetFit model, e.g. `"cuda:0"`, `"mps"` or `torch.device("cuda")`.
```

--------------------------------

### from_pretrained

Source: https://huggingface.co/docs/setfit/v1.1.3/en/reference/main

Downloads a model from the Hugging Face Hub and instantiates a SetFitModel. Accepts a model identifier or local directory path, along with various optional parameters for revision, caching, device, and model configuration.

```APIDOC
## from_pretrained

### Description
Downloads a model from the Hugging Face Hub and instantiates a SetFitModel.

### Method
from_pretrained

### Parameters
- **pretrained_model_name_or_path** (`str`, `Path`) - Required - Either the `model_id` (string) of a model hosted on the Hub, e.g. `bigscience/bloom`, or a path to a `directory` containing model weights saved using `save_pretrained`, e.g., `../path/to/my_model_directory/`.
- **revision** (`str`, optional) - Revision of the model on the Hub. Can be a branch name, a git tag or any commit id. Defaults to the latest commit on `main` branch.
- **force_download** (`bool`, optional, defaults to `False`) - Whether to force (re-)downloading the model weights and configuration files from the Hub, overriding the existing cache.
- **proxies** (`Dict[str, str]`, optional) - A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}`. The proxies are used on every request.
- **token** (`str` or `bool`, optional) - The token to use as HTTP bearer authorization for remote files. By default, it will use the token cached when running `hf auth login`.
- **cache_dir** (`str`, `Path`, optional) - Path to the folder where cached files are stored.
- **local_files_only** (`bool`, optional, defaults to `False`) - If `True`, avoid downloading the file and return the path to the local cached file if it exists.
- **labels** (`List[str]`, optional) - If the labels are integers ranging from `0` to `num_classes-1`, then these labels indicate the corresponding labels.
- **model_card_data** (`SetFitModelCardData`, optional) - A `SetFitModelCardData` instance storing data such as model language, license, dataset name, etc. to be used in the automatically generated model cards.
- **use_differentiable_head** (`bool`, optional) - Whether to load SetFit using a differentiable (i.e., Torch) head instead of Logistic Regression.
- **normalize_embeddings** (`bool`, optional) - Whether to apply normalization on the embeddings produced by the Sentence Transformer body.
- **span_context** (`int`, defaults to `0`) - The number of words before and after the span candidate that should be prepended to the full sentence. By default, 0 for Aspect models and 3 for Polarity models.
- **device** (`Union[torch.device, str]`, optional) - The device on which to load the SetFit model, e.g. `"cuda:0"`, `"mps"` or `torch.device("cuda")`.

### Returns
A SetFitModel instance.
```

--------------------------------

### Inspect synthetic dataset

Source: https://huggingface.co/docs/setfit/how_to/zero_shot

Prints the dataset structure and the first example. The output shows the dataset has 48 rows and the first example maps 'This sentence is sadness' to label 0.

```python
print(train_dataset)
# => Dataset({
#     features: ['text', 'label'],
#     num_rows: 48
# })
print(train_dataset[0])
# {'text': 'This sentence is sadness', 'label': 0}
```

--------------------------------

### Fine-tune base model from scratch

Source: https://huggingface.co/docs/setfit/tutorials/onnx

Commented-out code to fine-tune the base bge-small model using SetFit's Trainer. It loads the base model, sets training arguments with 20 iterations, trains, and then evaluates with a performance benchmark.

```python
# # Fine-tune the base model and Evaluate!
# from setfit import SetFitModel, Trainer, TrainingArguments

# # Load pretrained model from the Hub
# small_model = SetFitModel.from_pretrained(
#    "BAAI/bge-small-en-v1.5"
# )
# args = TrainingArguments(num_iterations=20)

# # Create trainer
# small_trainer = Trainer(
#    model=small_model, args=args, train_dataset=train_dataset
# )
# # Train!
# small_trainer.train()

# # Evaluate!
# pb = PerformanceBenchmark(
#    model=small_trainer.model, dataset=test_dataset, optim_type="bge-small (base)"
# )
# perf_metrics = pb.run_benchmark()
```

--------------------------------

### device

Source: https://huggingface.co/docs/setfit/reference/main

Get the Torch device that this model is on.

```APIDOC
## device

### Description
Get the Torch device that this model is on.

### Method
property

### Returns
- **torch.device** - The device that the model is on.
```

--------------------------------

### from_pretrained

Source: https://huggingface.co/docs/setfit/reference/main

Downloads a model from the Hugging Face Hub and instantiates a SetFitModel. Accepts a model identifier or local directory path, along with various optional parameters for revision, caching, token, and model configuration.

```APIDOC
## from_pretrained

### Description
Downloads a model from the Hugging Face Hub and instantiates a SetFitModel. Accepts a model identifier or local directory path, along with various optional parameters for revision, caching, token, and model configuration.

### Method
from_pretrained

### Signature
from_pretrained(force_download: bool = False, resume_download: Optional[bool] = None, proxies: Optional[Dict] = None, token: Union[bool, str, None] = None, cache_dir: Union[str, Path, None] = None, local_files_only: bool = False, revision: Optional[str] = None, **model_kwargs)

### Parameters
- **pretrained_model_name_or_path** (_str_, _Path_) - Required - Either the _model_id_ (string) of a model hosted on the Hub, e.g. _bigscience/bloom_, or a path to a _directory_ containing model weights saved using _~transformers.PreTrainedModel.save_pretrained_, e.g., _../path/to/my_model_directory/_.
- **revision** (_str_, _optional_) - Revision of the model on the Hub. Can be a branch name, a git tag or any commit id. Defaults to the latest commit on _main_ branch.
- **force_download** (_bool_, _optional_, defaults to _False_) - Whether to force (re-)downloading the model weights and configuration files from the Hub, overriding the existing cache.
- **proxies** (_Dict[str, str]_, _optional_) - A dictionary of proxy servers to use by protocol or endpoint, e.g., _{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}_. The proxies are used on every request.
- **token** (_str_ or _bool_, _optional_) - The token to use as HTTP bearer authorization for remote files. By default, it will use the token cached when running _hf auth login_.
- **cache_dir** (_str_, _Path_, _optional_) - Path to the folder where cached files are stored.
- **local_files_only** (_bool_, _optional_, defaults to _False_) - If _True_, avoid downloading the file and return the path to the local cached file if it exists.
- **labels** (_List[str]_, _optional_) - If the labels are integers ranging from _0_ to _num_classes-1_, then these labels indicate the corresponding labels.
- **model_card_data** (_SetFitModelCardData_, _optional_) - A _SetFitModelCardData_ instance storing data such as model language, license, dataset name, etc. to be used in the automatically generated model cards.
- **multi_target_strategy** (_str_, _optional_) - The strategy to use with multi-label classification. One of "one-vs-rest", "multi-output", or "classifier-chain".
- **use_differentiable_head** (_bool_, _optional_) - Whether to load SetFit using a differentiable (i.e., Torch) head instead of Logistic Regression.
- **normalize_embeddings** (_bool_, _optional_) - Whether to apply normalization on the embeddings produced by the Sentence Transformer body.
- **device** (_Union[torch.device, str]_, _optional_) - The device on which to load the SetFit model, e.g. _"cuda:0"_, _"mps"_ or _torch.device("cuda")_.
- **trust_remote_code** (_bool_, defaults to _False_) - Whether or not to allow for custom Sentence Transformers models defined on the Hub in their own modeling files. This option should only be set to True for repositories you trust and in which you have read the code, as it will execute code present on the Hub on your local machine. Defaults to False.

### Example
```python
>>> from setfit import SetFitModel
>>> model = SetFitModel.from_pretrained(
...     "sentence-transformers/paraphrase-mpnet-base-v2",
...     labels=["positive", "negative"],
... )
```
```

--------------------------------

### Create SetFit Trainer

Source: https://huggingface.co/docs/setfit/tutorials/zero_shot

Initializes a Trainer with the model, training dataset, and evaluation dataset. The train_dataset and reference_dataset variables must be defined beforehand.

```python
from setfit import Trainer

trainer = Trainer(
    model=model,
    train_dataset=train_dataset,
    eval_dataset=reference_dataset["test"]
)
```

--------------------------------

### Prediction output

Source: https://huggingface.co/docs/setfit/how_to/zero_shot

Shows the predicted class names for the example sentences.

```text
['sadness', 'joy', 'anger', 'fear', 'surprise']
```

--------------------------------

### Initialize SetFitModel with Custom Head (LinearSVC)

Source: https://huggingface.co/docs/setfit/how_to/classification_heads

Shows how to create a SetFit model with a custom non-differentiable head (LinearSVC) using the regular __init__ method. Requires a SentenceTransformer body and a head that implements the required methods.

```python
from setfit import SetFitModel
from sklearn.svm import LinearSVC
from sentence_transformers import SentenceTransformer

# Initializing a new SetFit model
model_body = SentenceTransformer("BAAI/bge-small-en-v1.5")
model_head = LinearSVC()
model = SetFitModel(model_body, model_head)
```

--------------------------------

### SetFitABSA Training Logs

Source: https://huggingface.co/docs/setfit/how_to/absa

Training logs for the aspect and polarity models, showing loss, learning rate, and epoch values. The aspect model trains on 249 examples for 5 epochs, and the polarity model on 39 examples. The best model is loaded from step 100 for aspect and step 50 for polarity.

```text
***** Running training *****
  Num examples = 249
  Num epochs = 5
  Total optimization steps = 1245
  Total train batch size = 128
{'aspect_embedding_loss': 0.2542, 'learning_rate': 1.6e-07, 'epoch': 0.0}                                                                                          
{'aspect_embedding_loss': 0.2437, 'learning_rate': 8.000000000000001e-06, 'epoch': 0.2}                                                                            
{'eval_aspect_embedding_loss': 0.2511, 'learning_rate': 8.000000000000001e-06, 'epoch': 0.2}                                                                       
{'aspect_embedding_loss': 0.2209, 'learning_rate': 1.6000000000000003e-05, 'epoch': 0.4}                                                                           
{'eval_aspect_embedding_loss': 0.2385, 'learning_rate': 1.6000000000000003e-05, 'epoch': 0.4}                                                                      
{'aspect_embedding_loss': 0.0165, 'learning_rate': 1.955357142857143e-05, 'epoch': 0.6}                                                                            
{'eval_aspect_embedding_loss': 0.2776, 'learning_rate': 1.955357142857143e-05, 'epoch': 0.6}                                                                       
{'aspect_embedding_loss': 0.0158, 'learning_rate': 1.8660714285714287e-05, 'epoch': 0.8}                                                                           
{'eval_aspect_embedding_loss': 0.2848, 'learning_rate': 1.8660714285714287e-05, 'epoch': 0.8}                                                                      
{'aspect_embedding_loss': 0.0015, 'learning_rate': 1.7767857142857143e-05, 'epoch': 1.0}                                                                           
{'eval_aspect_embedding_loss': 0.3133, 'learning_rate': 1.7767857142857143e-05, 'epoch': 1.0}                                                                      
{'aspect_embedding_loss': 0.0012, 'learning_rate': 1.6875e-05, 'epoch': 1.2}                                                                                       
{'eval_aspect_embedding_loss': 0.2966, 'learning_rate': 1.6875e-05, 'epoch': 1.2}                                                                                  
{'aspect_embedding_loss': 0.0009, 'learning_rate': 1.598214285714286e-05, 'epoch': 1.41}                                                                           
{'eval_aspect_embedding_loss': 0.2996, 'learning_rate': 1.598214285714286e-05, 'epoch': 1.41}                                                                      
 28%|██████████████████████████████████▎                                                                                       | 350/1245 [03:40<09:24,  1.59it/s] 
Loading best SentenceTransformer model from step 100.
{'train_runtime': 226.7429, 'train_samples_per_second': 702.822, 'train_steps_per_second': 5.491, 'epoch': 1.41}
***** Running training *****
  Num examples = 39
  Num epochs = 5
  Total optimization steps = 195
  Total train batch size = 128
{'polarity_embedding_loss': 0.2267, 'learning_rate': 1.0000000000000002e-06, 'epoch': 0.03}                                                                        
{'polarity_embedding_loss': 0.1038, 'learning_rate': 1.6571428571428574e-05, 'epoch': 1.28}                                                                        
{'eval_polarity_embedding_loss': 0.1946, 'learning_rate': 1.6571428571428574e-05, 'epoch': 1.28}                                                                   
{'polarity_embedding_loss': 0.0116, 'learning_rate': 1.0857142857142858e-05, 'epoch': 2.56}                                                                        
{'eval_polarity_embedding_loss': 0.2364, 'learning_rate': 1.0857142857142858e-05, 'epoch': 2.56}                                                                   
{'polarity_embedding_loss': 0.0059, 'learning_rate': 5.142857142857142e-06, 'epoch': 3.85}                                                                         
{'eval_polarity_embedding_loss': 0.2401, 'learning_rate': 5.142857142857142e-06, 'epoch': 3.85}                                                                    
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 195/195 [00:54<00:00,  3.58it/s]
Loading best SentenceTransformer model from step 50.
{'train_runtime': 54.4104, 'train_samples_per_second': 458.736, 'train_steps_per_second': 3.584, 'epoch': 5.0}
```

--------------------------------

### Augment dataset with get_templated_dataset

Source: https://huggingface.co/docs/setfit/tutorials/zero_shot

Generates synthetic examples using get_templated_dataset with candidate_labels and sample_size=8, resulting in a dataset with 96 rows.

```python
augmented_dataset = get_templated_dataset(train_dataset, candidate_labels=candidate_labels, sample_size=8)
augmented_dataset
```

```output
Dataset({
    features: ['text', 'label'],
    num_rows: 96
})
```

--------------------------------

### Train SetFit model and observe training logs

Source: https://huggingface.co/docs/setfit/quickstart

Runs the training loop. The output shows training progress, loss, learning rate, and runtime metrics.

```python
>>> trainer.train()
***** Running training *****
  Num examples = 5
  Num epochs = 10
  Total optimization steps = 50
  Total train batch size = 32
{'embedding_loss': 0.2077, 'learning_rate': 4.000000000000001e-06, 'epoch': 0.2}                                                                                                                
{'embedding_loss': 0.0097, 'learning_rate': 0.0, 'epoch': 10.0}                                                                                                                                 
{'train_runtime': 14.705, 'train_samples_per_second': 108.807, 'train_steps_per_second': 3.4, 'epoch': 10.0}
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [00:08<00:00,  5.70it/s]
```

--------------------------------

### Initialize SetFit Trainer

Source: https://huggingface.co/docs/setfit/quickstart

Creates a Trainer instance with the model, training arguments, and the sampled training dataset.

```python
>>> from setfit import Trainer

>>> trainer = Trainer(
...     model=model,
...     args=args,
...     train_dataset=train_dataset,
... )
```

--------------------------------

### Training output

Source: https://huggingface.co/docs/setfit/how_to/zero_shot

Shows the training logs, including loss, learning rate, and progress bar. The training ran for 60 steps with a total runtime of about 15.5 seconds.

```text
***** Running training *****
  Num examples = 60
  Num epochs = 1
  Total optimization steps = 60
  Total train batch size = 32
{'embedding_loss': 0.2628, 'learning_rate': 3.3333333333333333e-06, 'epoch': 0.02}                                                                                 
{'embedding_loss': 0.0222, 'learning_rate': 3.7037037037037037e-06, 'epoch': 0.83}                                                                                 
{'train_runtime': 15.4717, 'train_samples_per_second': 124.098, 'train_steps_per_second': 3.878, 'epoch': 1.0}                                                     
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 60/60 [00:09<00:00,  6.35it/s]
```

--------------------------------

### Run hyperparameter search with Trainer

Source: https://huggingface.co/docs/setfit/how_to/hyperparameter_optimization

Instantiates a Trainer with the prepared datasets and model_init, then runs hyperparameter_search with the hp_space function for 10 trials, maximizing the objective.

```python
trainer = Trainer(
    train_dataset=train_dataset,
    eval_dataset=test_dataset,
    model_init=model_init,
)
best_run = trainer.hyperparameter_search(direction="maximize", hp_space=hp_space, n_trials=10)
```

--------------------------------

### Minimal AbsaModel instantiation

Source: https://huggingface.co/docs/setfit/how_to/absa

Creates an AbsaModel using a single Sentence Transformer model for both aspect filtering and polarity classification. This is the minimal example.

```python
from setfit import AbsaModel

model = AbsaModel.from_pretrained("BAAI/bge-small-en-v1.5")
```