Transformers

https://huggingface.co/docs/transformers/v4.56.2/en/
Admin
Transformers is a model-definition framework for state-of-the-art machine learning models across...
Tokens:4,092,384
Snippets:11,081
Trust Score:10
Update:10 months ago
Show doc for...
Tokens:
Raw
### Install Example-Specific Requirements

Source: https://huggingface.co/docs/transformers/v4.56.2/en/run_scripts

After setting up the correct version of Transformers, this command installs the Python dependencies required for a specific example script. These requirements are typically listed in a 'requirements.txt' file within the example's directory.

```bash
pip install -r requirements.txt
```

--------------------------------

### Editable Install Transformers

Source: https://huggingface.co/docs/transformers/v4.56.2/en/installation

Sets up an editable installation of the Transformers library, linking your local copy to the repository. This is ideal for local development and contributing to the library.

```bash
git clone https://github.com/huggingface/transformers.git
cd transformers
pip install -e .
```

--------------------------------

### Install Transformers from Source

Source: https://huggingface.co/docs/transformers/v4.56.2/en/installation

Installs the latest development version of the Transformers library directly from its GitHub repository. This is useful for accessing the newest features or bug fixes.

```bash
pip install git+https://github.com/huggingface/transformers
```

--------------------------------

### Install Transformers with PyTorch Support

Source: https://huggingface.co/docs/transformers/v4.56.2/en/installation

Installs the Transformers library along with PyTorch, enabling CPU-only machine learning operations. This command installs the 'torch' extra.

```bash
pip install 'transformers[torch]'
```

--------------------------------

### Install Transformers with PyTorch using UV

Source: https://huggingface.co/docs/transformers/v4.56.2/en/installation

Installs the Transformers library and PyTorch using the 'uv' package installer, which is known for its speed. This command also installs the 'torch' extra.

```bash
uv pip install 'transformers[torch]'
```

--------------------------------

### Install Libraries for Transformers and Audio Datasets

Source: https://huggingface.co/docs/transformers/v4.56.2/en/model_doc/mms

Installs the 'torch', 'accelerate', and 'datasets' libraries with audio support, and upgrades the 'transformers' library to the latest version. These are essential for running the subsequent code examples.

```bash
pip install torch accelerate datasets[audio]
pip install --upgrade transformers
```

--------------------------------

### Install Transformers Libraries

Source: https://huggingface.co/docs/transformers/v4.56.2/en/tasks/audio_classification

Installs the necessary Hugging Face libraries for transformers, datasets, and evaluation. These libraries are essential for loading models, datasets, and evaluating performance.

```bash
pip install transformers datasets evaluate
```

--------------------------------

### Install Necessary Libraries

Source: https://huggingface.co/docs/transformers/v4.56.2/en/tasks/idefics

Installs the required libraries for running IDEFICS examples, including bitsandbytes for quantization, sentencepiece for tokenization, accelerate for distributed training, and transformers for the core model.

```shell
pip install -q bitsandbytes sentencepiece accelerate transformers
```

--------------------------------

### Install Libraries for Image Segmentation with Transformers

Source: https://huggingface.co/docs/transformers/v4.56.2/en/tasks/semantic_segmentation

Installs the necessary libraries for working with Hugging Face Transformers, datasets, evaluate, and accelerate. This is a prerequisite for running the image segmentation examples.

```python
# uncomment to install the necessary libraries
!pip install -q datasets transformers evaluate accelerate
```

--------------------------------

### Install DeepSpeed using Pip

Source: https://huggingface.co/docs/transformers/v4.56.2/en/deepspeed

This command installs the DeepSpeed library via pip. Ensure you have a compatible Python environment and pip installed. This is a quick way to get started with DeepSpeed.

```bash
pip install deepspeed
```

--------------------------------

### Install Hugging Face Transformers and Ecosystem Libraries

Source: https://huggingface.co/docs/transformers/v4.56.2/en/quicktour

Installs or upgrades the Transformers library along with essential ecosystem tools: `datasets` for data handling, `evaluate` for model evaluation, `accelerate` for distributed training, and `timm` for vision models. This command ensures you have the latest versions for optimal performance.

```bash
!pip install -U transformers datasets evaluate accelerate timm
```

--------------------------------

### ReformerForQuestionAnswering Example: Question Answering

Source: https://huggingface.co/docs/transformers/v4.56.2/en/model_doc/reformer

This example demonstrates how to use the ReformerForQuestionAnswering model for a question answering task. It covers tokenizing input, performing inference to get answer start and end logits, predicting the answer tokens, and calculating the loss when target positions are provided.

```python
from transformers import AutoTokenizer, ReformerForQuestionAnswering
import torch

tokenizer = AutoTokenizer.from_pretrained("google/reformer-crime-and-punishment")
model = ReformerForQuestionAnswering.from_pretrained("google/reformer-crime-and-punishment")

question, text = "Who was Jim Henson?", "Jim Henson was a nice puppet"

inputs = tokenizer(question, text, return_tensors="pt")
with torch.no_grad():
    outputs = model(**inputs)

answer_start_index = outputs.start_logits.argmax()
answer_end_index = outputs.end_logits.argmax()

predict_answer_tokens = inputs.input_ids[0, answer_start_index : answer_end_index + 1]
predicted_answer = tokenizer.decode(predict_answer_tokens, skip_special_tokens=True)

# Example of calculating loss with target positions
target_start_index = torch.tensor([14])
target_end_index = torch.tensor([15])

outputs = model(**inputs, start_positions=target_start_index, end_positions=target_end_index)
loss = outputs.loss
print(f"Calculated loss: {round(loss.item(), 2)}")
# Expected output is around 0.0 for this specific example with correct target indices.
```

--------------------------------

### Install PyTorch

Source: https://huggingface.co/docs/transformers/v4.56.2/en/quicktour

Installs the PyTorch deep learning framework, a core dependency for many machine learning tasks, including those within the Hugging Face Transformers library. This command is typically run in a notebook or a similar environment.

```bash
!pip install torch
```

--------------------------------

### DeiTForImageClassificationWithTeacher Forward Pass Example - PyTorch

Source: https://huggingface.co/docs/transformers/v4.56.2/en/model_doc/deit

This snippet demonstrates how to use the DeiTForImageClassificationWithTeacher model for image classification. It involves loading a dataset, processing an image using AutoImageProcessor, and then passing the processed input to the model to get classification logits. The example assumes PyTorch is installed and configured.

```python
>>> from transformers import AutoImageProcessor, DeiTForImageClassificationWithTeacher
>>> import torch
>>> from datasets import load_dataset

>>> dataset = load_dataset("huggingface/cats-image")
>>> image = dataset["test"]["image"][0]

>>> image_processor = AutoImageProcessor.from_pretrained("facebook/deit-base-distilled-patch16-224")
>>> model = DeiTForImageClassificationWithTeacher.from_pretrained("facebook/deit-base-distilled-patch16-224")

>>> inputs = image_processor(image, return_tensors="pt")

>>> with torch.no_grad():
...     logits = model(**inputs).logits

>>> # model predicts one of the 1000 ImageNet classes
>>> predicted_label = logits.argmax(-1).item()
>>> print(model.config.id2label[predicted_label])
...
```

--------------------------------

### Set Up Development Environment

Source: https://huggingface.co/docs/transformers/v4.56.2/en/contributing

Installs the Transformers library in editable mode with development dependencies. This command ensures that your local changes are reflected immediately and sets up necessary tools for development.

```bash
pip install -e ".[dev]"

# If issues arise, especially with optional dependencies:
pip install -e ".[quality]"
```

--------------------------------

### RobertaPreLayerNormForQuestionAnswering Inference Example

Source: https://huggingface.co/docs/transformers/v4.56.2/en/model_doc/roberta-prelayernorm

Demonstrates how to use RobertaPreLayerNormForQuestionAnswering for question answering. It tokenizes input text, performs a forward pass to get answer logits, identifies the start and end tokens of the predicted answer, and decodes them. This example uses PyTorch and the Hugging Face Transformers library.

```python
from transformers import AutoTokenizer, RobertaPreLayerNormForQuestionAnswering
import torch

tokenizer = AutoTokenizer.from_pretrained("andreasmadsen/efficient_mlm_m0.40")
model = RobertaPreLayerNormForQuestionAnswering.from_pretrained("andreasmadsen/efficient_mlm_m0.40")

question, text = "Who was Jim Henson?", "Jim Henson was a nice puppet"

inputs = tokenizer(question, text, return_tensors="pt")
with torch.no_grad():
    outputs = model(**inputs)

answer_start_index = outputs.start_logits.argmax()
answer_end_index = outputs.end_logits.argmax()

predict_answer_tokens = inputs.input_ids[0, answer_start_index : answer_end_index + 1]
print(tokenizer.decode(predict_answer_tokens, skip_special_tokens=True))

```

--------------------------------

### Build Documentation Locally

Source: https://huggingface.co/docs/transformers/v4.56.2/en/contributing

Installs the documentation builder and builds the project documentation locally. This allows you to preview changes to the documentation before submitting a pull request.

```bash
pip install hf-doc-builder
doc-builder build transformers docs/source/en --build_dir ~/tmp/test-build
```

--------------------------------

### NezhaForNextSentencePrediction Forward Pass Example - Python

Source: https://huggingface.co/docs/transformers/v4.56.2/en/model_doc/nezha

This Python code snippet demonstrates how to use the NezhaForNextSentencePrediction model for next sentence prediction. It involves loading a tokenizer and the model, preparing input tensors, and then performing the forward pass to get predictions. The example assumes the user has the 'transformers' library installed and a PyTorch environment ready.

```python
>>> from transformers import AutoTokenizer, NezhaForNextSentencePrediction
>>> import torch

>>> tokenizer = AutoTokenizer.from_pretrained("Babelscape/mxbai-nezha-base-qa")
>>> model = NezhaForNextSentencePrediction.from_pretrained("Babelscape/mxbai-nezha-nezha-base-qa")

>>> inputs = tokenizer("Hello, my name is John. I live in New York.", "My favorite color is blue.", return_tensors="pt")
>>> labels = torch.tensor([1]) # next_sentence_b_is_logical

>>> outputs = model(**inputs, labels=labels)
>>> loss = outputs.loss
>>> logits = outputs.logits
```

--------------------------------

### UniSpeechSatForSequenceClassification Forward Pass Example (PyTorch)

Source: https://huggingface.co/docs/transformers/v4.56.2/en/model_doc/unispeech-sat

Demonstrates how to use the UniSpeechSatForSequenceClassification model for sequence classification. It shows tokenization, model inference to get logits, and calculating loss with provided labels. Ensure you have the 'transformers' library installed.

```python
import torch
from transformers import AutoTokenizer, UniSpeechSatForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("microsoft/unispeech-sat-base-100h-libri-ft")
model = UniSpeechSatForSequenceClassification.from_pretrained("microsoft/unispeech-sat-base-100h-libri-ft")

inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")

with torch.no_grad():
    logits = model(**inputs).logits

predicted_class_id = logits.argmax().item()
print(f"Predicted class ID: {predicted_class_id}")
print(f"Predicted label: {model.config.id2label[predicted_class_id]}")

# Example for training with labels
num_labels = len(model.config.id2label)
model = UniSpeechSatForSequenceClassification.from_pretrained("microsoft/unispeech-sat-base-100h-libri-ft", num_labels=num_labels)

labels = torch.tensor([1]) # Example label
loss = model(**inputs, labels=labels).loss
print(f"Loss: {round(loss.item(), 2)}")
```

--------------------------------

### Install Optimum Quanto and Dependencies

Source: https://huggingface.co/docs/transformers/v4.56.2/en/quantization/quanto

Installs the Optimum Quanto library along with accelerate and transformers. This is the initial step to use Quanto for model quantization.

```bash
pip install optimum-quanto accelerate transformers
```

--------------------------------

### Install Transformers with Pip

Source: https://huggingface.co/docs/transformers/v4.56.2/en/installation

Installs the Transformers library using pip within an activated virtual environment. This is the standard method for Python package installation.

```bash
pip install transformers
```

--------------------------------

### Build DeepSpeed Wheel for Distribution

Source: https://huggingface.co/docs/transformers/v4.56.2/en/debugging

Clones the DeepSpeed repository and builds a binary wheel package using `setup.py`. This wheel can then be installed locally or distributed to other machines with the same setup, simplifying deployment for multi-machine training.

```bash
git clone https://github.com/deepspeedai/DeepSpeed/
cd DeepSpeed
rm -rf build
TORCH_CUDA_ARCH_LIST="8.6" DS_BUILD_CPU_ADAM=1 DS_BUILD_UTILS=1 \
python setup.py build_ext -j8 bdist_wheel
```

--------------------------------

### Initialize and Use YosoForQuestionAncasting for Question Answering

Source: https://huggingface.co/docs/transformers/v4.56.2/en/model_doc/yoso

This example demonstrates how to initialize the YosoForQuestionAncasting model and tokenizer, process question and text inputs, perform inference to get answer logits, and decode the predicted answer. It also shows how to calculate the loss by providing target start and end positions.

```python
from transformers import AutoTokenizer, YosoForQuestionAnswering
import torch

tokenizer = AutoTokenizer.from_pretrained("uw-madison/yoso-4096")
model = YosoForQuestionAncasting.from_pretrained("uw-madison/yoso-4096")

question, text = "Who was Jim Henson?", "Jim Henson was a nice puppet"

inputs = tokenizer(question, text, return_tensors="pt")
with torch.no_grad():
    outputs = model(**inputs)

answer_start_index = outputs.start_logits.argmax()
answer_end_index = outputs.end_logits.argmax()

predict_answer_tokens = inputs.input_ids[0, answer_start_index : answer_end_index + 1]
print(tokenizer.decode(predict_answer_tokens, skip_special_tokens=True))

# target is "nice puppet"
target_start_index = torch.tensor([14])
target_end_index = torch.tensor([15])

outputs = model(**inputs, start_positions=target_start_index, end_positions=target_end_index)
loss = outputs.loss
print(round(loss.item(), 2))

```

--------------------------------

### Question Answering with BigBirdPegasusForQuestionAnswering (Python)

Source: https://huggingface.co/docs/transformers/v4.56.2/en/model_doc/bigbird_pegasus

This snippet demonstrates how to use the BigBirdPegasusForQuestionAnswering model for a question answering task. It includes tokenizing input questions and text, performing inference to get answer start and end logits, decoding the predicted answer, and calculating the loss when target positions are provided.

```python
>>> from transformers import AutoTokenizer, BigBirdPegasusForQuestionAnswering
>>> import torch

>>> tokenizer = AutoTokenizer.from_pretrained("google/bigbird-pegasus-large-arxiv")
>>> model = BigBirdPegasusForQuestionAnswering.from_pretrained("google/bigbird-pegasus-large-arxiv")

>>> question, text = "Who was Jim Henson?", "Jim Henson was a nice puppet"

>>> inputs = tokenizer(question, text, return_tensors="pt")
>>> with torch.no_grad():
...     outputs = model(**inputs)

>>> answer_start_index = outputs.start_logits.argmax()
>>> answer_end_index = outputs.end_logits.argmax()

>>> predict_answer_tokens = inputs.input_ids[0, answer_start_index : answer_end_index + 1]
>>> tokenizer.decode(predict_answer_tokens, skip_special_tokens=True)
...

>>> # target is "nice puppet"
>>> target_start_index = torch.tensor([14])
>>> target_end_index = torch.tensor([15])

>>> outputs = model(**inputs, start_positions=target_start_index, end_positions=target_end_index)
>>> loss = outputs.loss
>>> round(loss.item(), 2)
...
```

--------------------------------

### Install Transformers from Source

Source: https://huggingface.co/docs/transformers/v4.56.2/en/run_scripts

This code snippet demonstrates how to clone the Hugging Face Transformers repository and install it from source in a virtual environment. This is necessary to run the latest version of the example scripts.

```bash
git clone https://github.com/huggingface/transformers
cd transformers
pip install .
```

--------------------------------

### Clone and Set Up Transformers Repository

Source: https://huggingface.co/docs/transformers/v4.56.2/en/add_new_model

This section details how to fork the Transformers repository, clone it locally, and set up the upstream remote. It also covers creating a virtual environment and installing the library with development dependencies.

```bash
git clone https://github.com/[your Github handle]/transformers.git
cd transformers
git remote add upstream https://github.com/huggingface/transformers.git

python -m venv .env
source .env/bin/activate
pip install -e "".[dev]"
```

```bash
pip install -e "".[quality]"
```

--------------------------------

### Install Libraries for LLM Optimization

Source: https://huggingface.co/docs/transformers/v4.56.2/en/llm_tutorial_optimization

Installs necessary libraries for optimizing LLM performance and memory usage, including `transformers`, `accelerate`, `bitsandbytes`, and `optimum`.

```bash
!pip install transformers accelerate bitsandbytes optimum
```

--------------------------------

### Install Transformers with Conda

Source: https://huggingface.co/docs/transformers/v4.56.2/en/installation

Installs the Transformers library using the conda package manager from the conda-forge channel. This is an alternative to pip for managing Python environments and packages.

```bash
conda install conda-forge::transformers
```

--------------------------------

### Install bitsandbytes Library

Source: https://huggingface.co/docs/transformers/v4.56.2/en/llm_tutorial_optimization

Installs the 'bitsandbytes' library, which is required for loading models with 8-bit or 4-bit quantization. This is a prerequisite for the subsequent quantization steps.

```bash
!pip install bitsandbytes
```

--------------------------------

### Initialize and Run Trainer (Python)

Source: https://huggingface.co/docs/transformers/v4.56.2/en/quicktour

Instantiates the `Trainer` class with the model, training arguments, datasets, tokenizer, and data collator. The `trainer.train()` method is then called to start the training process.

```python
from transformers import Trainer

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=dataset["train"],
    eval_dataset=dataset["test"],
    tokenizer=tokenizer,
    data_collator=data_collator,
)

trainer.train()
```

--------------------------------

### PLBartForSequenceClassification Single-Label Classification Example

Source: https://huggingface.co/docs/transformers/v4.56.2/en/model_doc/plbart

Demonstrates how to use PLBartForSequenceClassification for single-label text classification. It shows loading a tokenizer and model, tokenizing input text, performing inference to get logits, predicting the class, and calculating the loss with provided labels. Assumes PyTorch and the transformers library are installed.

```python
import torch
from transformers import AutoTokenizer, PLBartForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("uclanlp/plbart-base")
model = PLBartForSequenceClassification.from_pretrained("uclanlp/plbart-base")

inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")

with torch.no_grad():
    logits = model(**inputs).logits

predicted_class_id = logits.argmax().item()
print(model.config.id2label[predicted_class_id])

# To train a model on `num_labels` classes, you can pass `num_labels=num_labels` to `.from_pretrained(...)`
num_labels = len(model.config.id2label)
model = PLBartForSequenceClassification.from_pretrained("uclanlp/plbart-base", num_labels=num_labels)

labels = torch.tensor([1])
loss = model(**inputs, labels=labels).loss
print(round(loss.item(), 2))
```

--------------------------------

### Blip2Model Forward Method Example (PyTorch)

Source: https://huggingface.co/docs/transformers/v4.56.2/en/model_doc/blip-2

Demonstrates how to use the forward method of the Blip2Model for image-to-text generation. This example requires PyTorch, Pillow, and the Hugging Face Transformers library. It shows the basic setup for device selection and loading the processor and model.

```python
from PIL import Image
import requests
from transformers import Blip2Processor, Blip2Model
import torch

device = "cuda" if torch.cuda.is_available() else "cpu"

```

--------------------------------

### Example Usage of RecurrentGemmaForCausalLM

Source: https://huggingface.co/docs/transformers/v4.56.2/en/model_doc/recurrent_gemma

Illustrates how to instantiate and use the RecurrentGemmaForCausalLM model with an AutoTokenizer for causal language modeling tasks. This example shows the basic setup for model inference.

```python
>>> from transformers import AutoTokenizer, RecurrentGemmaForCausalLM
>>> tokenizer = AutoTokenizer.from_pretrained("google/recurrentgemma-2b")
>>> model = RecurrentGemmaForCausalLM.from_pretrained("google/recurrentgemma-2b")
>>> inputs = tokenizer("", return_tensors="pt")
>>> # forward!: (input_ids=inputs.input_ids, attention_mask=inputs.attention_mask)
>>> # The above line is commented out to avoid execution errors as it requires model inputs.
```

--------------------------------

### Load Model, Tokenizer, and Dataset for Training (Python)

Source: https://huggingface.co/docs/transformers/v4.56.2/en/quicktour

Loads a pre-trained sequence classification model, its corresponding tokenizer, and a dataset using the `transformers` and `datasets` libraries. This is the initial setup for training a model.

```python
from transformers import AutoModelForSequenceClassification, AutoTokenizer
from datasets import load_dataset

model = AutoModelForSequenceClassification.from_pretrained("distilbert/distilbert-base-uncased")
tokenizer = AutoTokenizer.from_pretrained("distilbert/distilbert-base-uncased")
dataset = load_dataset("rotten_tomatoes")
```

--------------------------------

### MraForMultipleChoice Forward Pass Example

Source: https://huggingface.co/docs/transformers/v4.56.2/en/model_doc/mra

Demonstrates how to use the MraForMultipleChoice model's forward method with PyTorch. This example shows the basic setup for tokenization and passing data through the model.

```python
>>> from transformers import AutoTokenizer, MraForMultipleChoice
>>> import torch
>>> model = MraForMultipleChoice.from_pretrained("path/to/your/model")
>>> tokenizer = AutoTokenizer.from_pretrained("path/to/your/tokenizer")
>>> prompt = "Which sentence below expresses the same meaning as the following sentence?"
>>> choice1 = "The cat sat on the mat."
>>> choice2 = "The dog chased the ball."
>>> inputs = tokenizer(prompt, choice1, return_tensors="pt", padding=True, truncation=True)
>>> choice1_inputs = tokenizer(prompt, choice2, return_tensors="pt", padding=True, truncation=True)
>>> # For multiple choice, you typically need to structure inputs differently
>>> # This is a simplified conceptual example. Refer to specific MraForMultipleChoice examples for correct input formatting.
>>> # Example assuming correctly formatted input tensor `input_ids` and `attention_mask`
>>> # input_ids = torch.randint(0, tokenizer.vocab_size, (1, 2, 10)) # (batch_size, num_choices, sequence_length)
>>> # attention_mask = torch.ones((1, 2, 10)) # (batch_size, num_choices, sequence_length)
>>> # outputs = model(input_ids=input_ids, attention_mask=attention_mask)
>>> # print(outputs.logits)
```
Tokens:
Raw