### Full Training Script Example

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter8/4.mdx

This script demonstrates a typical setup for fine-tuning a DistilBERT model on the MNLI dataset. It includes dataset loading, preprocessing, model initialization, training arguments, metric computation, and the Trainer setup. Use this as a base for debugging.

```python
from datasets import load_dataset
import evaluate
from transformers import (
    AutoTokenizer,
    AutoModelForSequenceClassification,
    TrainingArguments,
    Trainer,
)

raw_datasets = load_dataset("glue", "mnli")

model_checkpoint = "distilbert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)


def preprocess_function(examples):
    return tokenizer(examples["premise"], examples["hypothesis"], truncation=True)


tokenized_datasets = raw_datasets.map(preprocess_function, batched=True)
model = AutoModelForSequenceClassification.from_pretrained(model_checkpoint)

args = TrainingArguments(
    f"distilbert-finetuned-mnli",
    evaluation_strategy="epoch",
    save_strategy="epoch",
    learning_rate=2e-5,
    num_train_epochs=3,
    weight_decay=0.01,
)

metric = evaluate.load("glue", "mnli")


def compute_metrics(eval_pred):
    predictions, labels = eval_pred
    return metric.compute(predictions=predictions, references=labels)


trainer = Trainer(
    model,
    args,
    train_dataset=raw_datasets["train"],
    eval_dataset=raw_datasets["validation_matched"],
    compute_metrics=compute_metrics,
)
trainer.train()
```

--------------------------------

### Basic SFTTrainer Implementation

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter11/3.mdx

This snippet shows a complete example of initializing and training a model using SFTTrainer with a dataset and basic configurations. Ensure you have the necessary libraries (datasets, trl, torch) installed and a compatible model and tokenizer loaded.

```python
from datasets import load_dataset
from trl import SFTConfig, SFTTrainer
import torch

# Set device
device = "cuda" if torch.cuda.is_available() else "cpu"

# Load dataset
dataset = load_dataset("HuggingFaceTB/smoltalk", "all")

# Configure model and tokenizer
model_name = "HuggingFaceTB/SmolLM2-135M"
model = AutoModelForCausalLM.from_pretrained(pretrained_model_name_or_path=model_name).to(
    device
)
tokenizer = AutoTokenizer.from_pretrained(pretrained_model_name_or_path=model_name)
# Setup chat template
model, tokenizer = setup_chat_format(model=model, tokenizer=tokenizer)

# Configure trainer
training_args = SFTConfig(
    output_dir="./sft_output",
    max_steps=1000,
    per_device_train_batch_size=4,
    learning_rate=5e-5,
    logging_steps=10,
    save_steps=100,
    eval_strategy="steps",
    eval_steps=50,
)

# Initialize trainer
trainer = SFTTrainer(
    model=model,
    args=training_args,
    train_dataset=dataset["train"],
    eval_dataset=dataset["test"],
    processing_class=tokenizer,
)

# Start training
trainer.train()
```

--------------------------------

### Display first example content

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter5/4.mdx

Show the content of the first example, which includes metadata and the abstract text.

```python
{
    'meta': {'pmid': 11409574, 'language': 'eng'},
    'text': 'Epidemiology of hypoxaemia in children with acute lower respiratory infection.\nTo determine the prevalence of hypoxaemia in children aged under 5 years suffering acute lower respiratory infections (ALRI), the risk factors for hypoxaemia in children under 5 years of age with ALRI, and the association of hypoxaemia with an increased risk of dying in children of the same age ...'
}
```

--------------------------------

### Peek at the first few examples

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter5/3.mdx

Displays the first three examples from the drug_sample dataset to inspect its structure and content.

```python
drug_sample[:3]
```

--------------------------------

### Install Unsloth and vLLM

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter12/6.mdx

Install the necessary libraries for accelerated LLM fine-tuning and fast inference.

```bash
pip install unsloth vllm
pip install --upgrade pillow
```

--------------------------------

### Install Course Dependencies

Source: https://github.com/huggingface/course/blob/main/README.md

Installs the necessary Python packages for the Hugging Face course from the requirements.txt file.

```bash
python -m pip install -r requirements.txt
```

--------------------------------

### Install Dependencies

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter12/5.mdx

Installs necessary libraries for the GRPO fine-tuning exercise, including datasets, transformers, trl, peft, accelerate, bitsandbytes, and wandb. It also installs flash-attn with no build isolation.

```bash
!pip install -qqq datasets==3.2.0 transformers==4.47.1 trl==0.14.0 peft==0.14.0 accelerate==1.2.1 bitsandbytes==0.45.2 wandb==0.19.7 --progress-bar off
!pip install -qqq flash-attn --no-build-isolation --progress-bar off
```

--------------------------------

### Install psutil library

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter5/4.mdx

Install the psutil library for monitoring system and process utilities, including memory usage.

```python
!pip install psutil
```

--------------------------------

### Instantiate Tokenizer and Tokenize Example Text

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter6/3.mdx

Loads a tokenizer and encodes an example sentence. The output is a BatchEncoding object.

```python
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("bert-base-cased")
example = "My name is Sylvain and I work at Hugging Face in Brooklyn."
encoding = tokenizer(example)
print(type(encoding))
```

--------------------------------

### Install Project Dependencies

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter0/1.mdx

Installs the 'transformers' library with 'sentencepiece' support using pip. This command should be run after activating the virtual environment.

```bash
pip install "transformers[sentencepiece]"
```

--------------------------------

### Install zstandard library

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter5/4.mdx

Install the zstandard library required for decompressing the dataset.

```python
!pip install zstandard
```

--------------------------------

### Install SacreBLEU

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter7/4.mdx

Install the SacreBLEU library using pip. This is a prerequisite for using the SacreBLEU metric.

```python
!pip install sacrebleu
```

--------------------------------

### Pre-tokenize Example String

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter6/8.mdx

Demonstrates how the Metaspace pre-tokenizer segments an example string. Each segment is shown with its corresponding character span.

```python
tokenizer.pre_tokenizer.pre_tokenize_str("Let's test the pre-tokenizer!")
```

--------------------------------

### Install Argilla Python SDK

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter10/2.mdx

Install the argilla library in your Python environment.

```bash
!pip install argilla
```

--------------------------------

### View a Single Example from the Dataset

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter5/2.mdx

Access and display the first example (index 0) from the 'train' split of the loaded dataset.

```python
squad_it_dataset["train"][0]
```

--------------------------------

### Print Example Function String

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter6/2.mdx

Prints the 'whole_func_string' for a specific example in the training split. Helps in viewing raw code samples.

```python
print(raw_datasets["train"][123456]["whole_func_string"])
```

--------------------------------

### Install Subtitle Translation Libraries

Source: https://github.com/huggingface/course/blob/main/subtitles/README.md

Installs the necessary Python libraries for fetching YouTube transcripts and searching for videos. Ensure you have Python and pip installed.

```bash
python -m pip install youtube_transcript_api youtube-search-python pandas tqdm
```

--------------------------------

### Full Training Setup with DataCollatorWithPadding

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter8/4.mdx

This comprehensive example sets up a Hugging Face Trainer for sequence classification, explicitly using DataCollatorWithPadding to ensure proper batch formation. It includes dataset loading, tokenization, model configuration, training arguments, and metric computation.

```python
from datasets import load_dataset
import evaluate
from transformers import (
    AutoTokenizer,
    AutoModelForSequenceClassification,
    DataCollatorWithPadding,
    TrainingArguments,
    Trainer,
)

raw_datasets = load_dataset("glue", "mnli")

model_checkpoint = "distilbert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)


def preprocess_function(examples):
    return tokenizer(examples["premise"], examples["hypothesis"], truncation=True)


tokenized_datasets = raw_datasets.map(preprocess_function, batched=True)
model = AutoModelForSequenceClassification.from_pretrained(model_checkpoint)

args = TrainingArguments(
    f"distilbert-finetuned-mnli",
    evaluation_strategy="epoch",
    save_strategy="epoch",
    learning_rate=2e-5,
    num_train_epochs=3,
    weight_decay=0.01,
)

metric = evaluate.load("glue", "mnli")


def compute_metrics(eval_pred):
    predictions, labels = eval_pred
    return metric.compute(predictions=predictions, references=labels)


data_collator = DataCollatorWithPadding(tokenizer=tokenizer)

trainer = Trainer(
    model,
    args,
    train_dataset=tokenized_datasets["train"],
    eval_dataset=tokenized_datasets["validation_matched"],
    compute_metrics=compute_metrics,
    data_collator=data_collator,
    tokenizer=tokenizer,
)
trainer.train()
```

--------------------------------

### Example Environment Information Output

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter8/5.mdx

This is an example of the output from the `transformers-cli env` command. You should fill in the last two points specific to your script.

```out
Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

- `transformers` version: 4.12.0.dev0
- Platform: Linux-5.10.61-1-MANJARO-x86_64-with-arch-Manjaro-Linux
- Python version: 3.7.9
- PyTorch version (GPU?): 1.8.1+cu111 (True)
- Tensorflow version (GPU?): 2.5.0 (True)
- Flax version (CPU?/GPU?/TPU?): 0.3.4 (cpu)
- Jax version: 0.2.13
- JaxLib version: 0.1.65
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>
```

--------------------------------

### Install Hugging Face Transformers Library

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter0/1.mdx

Use this command to install the core Hugging Face Transformers library using pip. This is a basic installation.

```bash
!pip install transformers
```

--------------------------------

### Displaying Sample Data

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter7/5.mdx

Shows a few examples from the dataset, including titles and reviews, to inspect content and language.

```python
show_samples(books_dataset)
```

--------------------------------

### Install seqeval library

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter7/2.mdx

Install the seqeval library, which is commonly used for evaluating token classification predictions. This is a prerequisite for using the seqeval metric.

```python
!pip install seqeval
```

--------------------------------

### Tokenize Example Code with New Tokenizer

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter6/2.mdx

Shows how the newly trained tokenizer tokenizes the same Python function example. Highlights improvements in handling indentation and domain-specific tokens.

```python
tokens = tokenizer.tokenize(example)
tokens
```

--------------------------------

### Tokenize Example Code with Pre-trained Tokenizer

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter6/2.mdx

Demonstrates how a pre-trained tokenizer (GPT-2) tokenizes a Python function example. Shows the tokenizer's handling of spaces and underscores.

```python
example = '''def add_numbers(a, b):
    """Add the two numbers `a` and `b`."""
    return a + b'''

tokens = old_tokenizer.tokenize(example)
tokens
```

--------------------------------

### ChatML Template Example

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter11/2.mdx

This is an example of the ChatML template format, commonly used by models like SmolLM2 and Qwen 2, showing how roles and content are delimited.

```text
<|im_start|>system
You are a helpful assistant.<|im_end|>
<|im_start|>user
Hello!<|im_end|>
<|im_start|>assistant
Hi! How can I help you today?<|im_end|>
<|im_start|>user
What's the weather?<|im_start|>assistant
```

--------------------------------

### Set up Local Development Environment

Source: https://github.com/huggingface/course/blob/main/README.md

Install the `doc-builder` tool locally using a virtual environment. This is required to preview the course documentation.

```bash
python -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install "git+https://github.com/huggingface/doc-builder.git"
```

--------------------------------

### Get Nearest Examples from FAISS Index

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter5/6.mdx

Retrieves the nearest examples from a FAISS index based on a query embedding. Returns similarity scores and the corresponding samples.

```python
scores, samples = embeddings_dataset.get_nearest_examples(
    "embeddings", question_embedding, k=5
)
```

--------------------------------

### Tokenize Input and Get Model Outputs

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter8/2.mdx

Tokenizes the input question and context, then passes it to the model to obtain start and end logits.

```python
import torch

inputs = tokenizer(question, context, add_special_tokens=True)
input_ids = inputs["input_ids"][0]
outputs = model(**inputs)
answer_start_scores = outputs.start_logits
answer_end_scores = outputs.end_logits
```

--------------------------------

### Full Training Script with Tokenization and Trainer Setup

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter8/4.mdx

This script demonstrates the complete process of loading a dataset, tokenizing it, defining a model, setting up training arguments, and initializing the Trainer. Ensure you use the tokenized dataset for training.

```python
from datasets import load_dataset
import evaluate
from transformers import (
    AutoTokenizer,
    AutoModelForSequenceClassification,
    TrainingArguments,
    Trainer,
)

raw_datasets = load_dataset("glue", "mnli")

model_checkpoint = "distilbert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)


def preprocess_function(examples):
    return tokenizer(examples["premise"], examples["hypothesis"], truncation=True)


tokenized_datasets = raw_datasets.map(preprocess_function, batched=True)
model = AutoModelForSequenceClassification.from_pretrained(model_checkpoint)

args = TrainingArguments(
    f"distilbert-finetuned-mnli",
    evaluation_strategy="epoch",
    save_strategy="epoch",
    learning_rate=2e-5,
    num_train_epochs=3,
    weight_decay=0.01,
)

metric = evaluate.load("glue", "mnli")


def compute_metrics(eval_pred):
    predictions, labels = eval_pred
    return metric.compute(predictions=predictions, references=labels)


trainer = Trainer(
    model,
    args,
    train_dataset=tokenized_datasets["train"],
    eval_dataset=tokenized_datasets["validation_matched"],
    compute_metrics=compute_metrics,
)
trainer.train()
```

--------------------------------

### Access First Element of Streamed Dataset

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter5/4.mdx

To access elements from an `IterableDataset`, you need to iterate over it. This example shows how to get the first element.

```python
next(iter(pubmed_dataset_streamed))
```

--------------------------------

### Pre-tokenize Example String

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter6/8.mdx

Demonstrates the output of the ByteLevel pre-tokenizer on a sample string. This shows how the text is initially split into tokens.

```python
tokenizer.pre_tokenizer.pre_tokenize_str("Let's test pre-tokenization!")
```

--------------------------------

### Create and Navigate to Project Directory

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter0/1.mdx

Use these commands to create a new directory for your project and navigate into it.

```bash
mkdir ~/transformers-course
cd ~/transformers-course
```

--------------------------------

### Logits Shapes Output (TensorFlow)

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter6/3b.mdx

Example output showing the shape of start and end logits for a TensorFlow model, indicating the batch size and number of tokens.

```python
(1, 66) (1, 66)
```

--------------------------------

### Logits Shapes Output (PyTorch)

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter6/3b.mdx

Example output showing the shape of start and end logits for a PyTorch model, indicating the batch size and number of tokens.

```python
torch.Size([1, 66]) torch.Size([1, 66])
```

--------------------------------

### Initialize GRPOTrainer with Reward Functions

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter12/6.mdx

Sets up the GRPOTrainer with specified model, tokenizer, dataset, and a list of reward functions. Configures training arguments like learning rate, batch size, and maximum steps.

```python
from trl import GRPOConfig, GRPOTrainer

max_prompt_length = 256

training_args = GRPOConfig(
    learning_rate=5e-6,
    adam_beta1=0.9,
    adam_beta2=0.99,
    weight_decay=0.1,
    warmup_ratio=0.1,
    lr_scheduler_type="cosine",
    optim="paged_adamw_8bit",
    logging_steps=1,
    per_device_train_batch_size=1,
    gradient_accumulation_steps=1,  # Increase to 4 for smoother training
    num_generations=6,  # Decrease if out of memory
    max_prompt_length=max_prompt_length,
    max_completion_length=max_seq_length - max_prompt_length,
    # num_train_epochs = 1, # Set to 1 for a full training run
    max_steps=250,
    save_steps=250,
    max_grad_norm=0.1,
    report_to="none",  # Can use Weights & Biases
    output_dir="outputs",
)

trainer = GRPOTrainer(
    model=model,
    processing_class=tokenizer,
    reward_funcs=[
        xmlcount_reward_func,
        soft_format_reward_func,
        strict_format_reward_func,
        int_reward_func,
        correctness_reward_func,
    ],
    args=training_args,
    train_dataset=dataset,
)
```

--------------------------------

### Execute Evaluation Function

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter7/6.mdx

Calls the previously defined `evaluate()` function to test its functionality before starting the main training loop. This helps verify that the evaluation setup is correct.

```python
evaluate()
```

--------------------------------

### Getting Model Outputs for Long Contexts (PyTorch)

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter6/3b.mdx

Passes tokenized inputs to the model to obtain start and end logits. The shapes of the output logits indicate the number of chunks and the sequence length.

```python
outputs = model(**inputs)

start_logits = outputs.start_logits
end_logits = outputs.end_logits
print(start_logits.shape, end_logits.shape)
```

--------------------------------

### Install and Build llama.cpp

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter2/8.mdx

Clone the llama.cpp repository, build the project using make, and download a GGUF model. This sets up the necessary components for running the llama.cpp server.

```shell
# Clone the repository
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp

# Build the project
make

# Download the SmolLM2-1.7B-Instruct-GGUF model
curl -L -O https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct-GGUF/resolve/main/smollm2-1.7b-instruct.Q4_K_M.gguf
```

--------------------------------

### Fetch a single GitHub issue

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter5/5.mdx

Retrieve the first issue from the 'huggingface/datasets' repository using the GitHub REST API. This example demonstrates making a GET request and setting query parameters for pagination.

```python
import requests

url = "https://api.github.com/repos/huggingface/datasets/issues?page=1&per_page=1"
response = requests.get(url)
```

--------------------------------

### Tokenized Text Output

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter6/7.mdx

Shows the result of tokenizing the example sentence "This is the Hugging Face course." using the Unigram model. The output is a list of tokens, including special tokens like '▁' which indicates the start of a word.

```python
['▁This', '▁is', '▁the', '▁Hugging', '▁Face', '▁', 'c', 'ou', 'r', 's', 'e', '.']
```

--------------------------------

### Tokenize inputs and get answer scores

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter8/2.mdx

Use `return_tensors='pt'` to ensure inputs are PyTorch tensors. This allows for tensor operations like `.size()` and enables the model to process the input correctly. The code then extracts the start and end logits from the model's output.

```python
inputs = tokenizer(question, context, add_special_tokens=True, return_tensors="pt")
input_ids = inputs["input_ids"][0]
outputs = model(**inputs)
answer_start_scores = outputs.start_logits
answer_end_scores = outputs.end_logits
```

--------------------------------

### Display First Training Example

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter7/7.mdx

Prints the 'context', 'question', and 'answers' fields for the first element in the training split of the SQuAD dataset.

```python
print("Context: ", raw_datasets["train"][0]["context"])
print("Question: ", raw_datasets["train"][0]["question"])
print("Answer: ", raw_datasets["train"][0]["answers"])
```

--------------------------------

### Initialize and Train the Model

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter7/2.mdx

Instantiate the Trainer with the model, arguments, datasets, data collator, metrics computation function, and tokenizer. Then, launch the training process.

```python
from transformers import Trainer

trainer = Trainer(
    model=model,
    args=args,
    train_dataset=tokenized_datasets["train"],
    eval_dataset=tokenized_datasets["validation"],
    data_collator=data_collator,
    compute_metrics=compute_metrics,
    processing_class=tokenizer,
)
trainer.train()
```

--------------------------------

### Find Start of Answer Span

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter8/2.mdx

Determines the most likely starting position of the answer span by finding the index with the highest score in the start logits.

```python
# Get the most likely beginning of answer with the argmax of the score
answer_start = torch.argmax(answer_start_scores)
```

--------------------------------

### Minimal GRPO Trainer Initialization and Training

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter12/4.mdx

This snippet shows the basic steps to set up and run GRPO training. It includes loading a dataset, defining a reward function, configuring the trainer, and initiating the training process.

```python
from trl import GRPOTrainer, GRPOConfig
from datasets import load_dataset

# 1. Load your dataset
dataset = load_dataset("your_dataset", split="train")


# 2. Define a simple reward function
def reward_func(completions, **kwargs):
    """Example: Reward longer completions"""
    return [float(len(completion)) for completion in completions]


# 3. Configure training
training_args = GRPOConfig(
    output_dir="output",
    num_train_epochs=3,
    per_device_train_batch_size=4,
    gradient_accumulation_steps=2,
    logging_steps=10,
)

# 4. Initialize and train
trainer = GRPOTrainer(
    model="your_model",  # e.g. "Qwen/Qwen2-0.5B-Instruct"
    args=training_args,
    train_dataset=dataset,
    reward_funcs=reward_func,
)
trainer.train()
```

--------------------------------

### Map Examples to Features

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter7/7.mdx

Creates a mapping from original example IDs to the indices of their corresponding features in the processed evaluation set. This is necessary because one example might be split into multiple features.

```python
import collections

example_to_features = collections.defaultdict(list)
for idx, feature in enumerate(eval_set):
    example_to_features[feature["example_id"]].append(idx)
```

--------------------------------

### Preview Course Documentation Locally

Source: https://github.com/huggingface/course/blob/main/README.md

Build and preview the translated course content locally using `doc-builder`. Ensure `_toctree.yml` only references existing files if content is partial.

```bash
doc-builder preview course ./chapters/LANG-ID --not_python_module
```

--------------------------------

### Initialize and Clone a Repository

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter4/3.mdx

Use the `Repository` class to initialize a local folder and clone a remote repository from the Hugging Face Hub. This requires git and git-lfs to be installed.

```python
from huggingface_hub import Repository

repo = Repository("<path_to_dummy_folder>", clone_from="<namespace>/dummy-model")
```

--------------------------------

### Example Code Snippet for Tokenization

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter6/2.mdx

A Python class definition used as an example for tokenization.

```python
example = """class LinearLayer():
    def __init__(self, input_size, output_size):

```

--------------------------------

### Create Optimizer and Compile Model

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter7/5.mdx

Sets up the optimizer and schedule for training and compiles the model. Mixed-precision training is enabled globally.

```python
num_train_epochs = 8
num_train_steps = len(tf_train_dataset) * num_train_epochs
model_name = model_checkpoint.split("/")[-1]

optimizer, schedule = create_optimizer(
    init_lr=5.6e-5,
    num_warmup_steps=0,
    num_train_steps=num_train_steps,
    weight_decay_rate=0.01,
)

model.compile(optimizer=optimizer)

# Train in mixed-precision float16
tf.keras.mixed_precision.set_global_policy("mixed_float16")
```

--------------------------------

### Launch llama.cpp Server

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter2/8.mdx

Start the llama.cpp server with the specified model, host, and port. Configure context size and GPU layers as needed. Set '--n-gpu-layers 0' for CPU inference.

```shell
# Start the server
./server \
    -m smollm2-1.7b-instruct.Q4_K_M.gguf \
    --host 0.0.0.0 \
    --port 8080 \
    -c 4096 \
    --n-gpu-layers 0  # Set to a higher number to use GPU
```

--------------------------------

### Initialize Hugging Face Hub Repository

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter7/5.mdx

Creates a local directory and clones a Hugging Face Hub repository into it, preparing it for pushing training artifacts.

```python
from huggingface_hub import Repository

output_dir = "results-mt5-finetuned-squad-accelerate"
repo = Repository(output_dir, clone_from=repo_name)
```

--------------------------------

### llama.cpp Server Launch Parameters

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter2/8.mdx

Configure llama.cpp server with parameters like context size, threads, and batch size.

```shell
./server \
    -m smollm2-1.7b-instruct.Q4_K_M.gguf \
    --host 0.0.0.0 \
    --port 8080 \
    -c 4096 	# Context size
    --threads 8 	# CPU threads to use
    --batch-size 512 	# Batch size for prompt evaluation
    --n-gpu-layers 0 	# GPU layers (0 = CPU only)
```

--------------------------------

### Example Perplexity Output

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter7/3.mdx

Shows example output for perplexity calculations across multiple training epochs.

```python
>>> Epoch 0: Perplexity: 11.397545307900472
>>> Epoch 1: Perplexity: 10.904909330983092
>>> Epoch 2: Perplexity: 10.729503505340409
```

--------------------------------

### Training Output Example

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter7/4.mdx

Example output showing the BLEU score computed at the end of each epoch during training.

```text
epoch 0, BLEU score: 53.47
epoch 1, BLEU score: 54.24
epoch 2, BLEU score: 54.44
```

--------------------------------

### Load Demo from Hugging Face Spaces

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter9/5.mdx

Loads a demo directly from a Hugging Face Space using `Interface.load()`. This is a quick way to run existing demos without local setup.

```python
gr.Interface.load("spaces/abidlabs/remove-bg").launch()
```

--------------------------------

### Input IDs Output Example

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter2/4.mdx

Example output of the `convert_tokens_to_ids()` method, showing the numerical representation of the tokens.

```python
[7993, 170, 11303, 1200, 2443, 1110, 3014]
```

--------------------------------

### Compute All Possible Scores

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter6/3b.mdx

Calculates the product of start and end probabilities for all possible start and end index combinations.

```python
scores = start_probabilities[:, None] * end_probabilities[None, :]
```

--------------------------------

### Install NLTK package

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter7/5.mdx

Install the NLTK library using pip. This is a prerequisite for using its sentence tokenization capabilities.

```python
!pip install nltk
```

--------------------------------

### GRPO Training Configuration with Key Parameters

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter12/4.mdx

Illustrates the GRPOConfig setup, highlighting essential parameters like `output_dir`, `num_train_epochs`, and `num_generation`, as well as optional ones for performance and GRPO-specific features.

```python
training_args = GRPOConfig(
    # Essential parameters
    output_dir="output",
    num_train_epochs=3,
    num_generation=4,  # Number of completions to generate for each prompt
    per_device_train_batch_size=4,  # We want to get all generations in one device batch
    # Optional but useful
    gradient_accumulation_steps=2,
    learning_rate=1e-5,
    logging_steps=10,
    # GRPO specific (optional)
    use_vllm=True,  # Speed up generation
)
```

--------------------------------

### Set up Hugging Face Hub Repository

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter7/2.mdx

Initialize a Hugging Face Hub repository for pushing model checkpoints. This involves getting the full repository name and cloning the repository locally.

```python
from huggingface_hub import Repository, get_full_repo_name

model_name = "bert-finetuned-ner-accelerate"
repo_name = get_full_repo_name(model_name)

output_dir = "bert-finetuned-ner-accelerate"
repo = Repository(output_dir, clone_from=repo_name)
```

--------------------------------

### Load and Customize Demo from Hugging Face Spaces

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter9/5.mdx

Loads a demo from Hugging Face Spaces and customizes its interface. This example sets a custom title and configures the input to use a webcam.

```python
gr.Interface.load(
    "spaces/abidlabs/remove-bg", inputs="webcam", title="Remove your webcam background!"
).launch()
```

--------------------------------

### Tokenizer Output Example

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter2/4.mdx

Example output of the tokenizer when encoding text, showing the generated 'input_ids', 'token_type_ids', and 'attention_mask'.

```python
{'input_ids': [101, 7993, 170, 11303, 1200, 2443, 1110, 3014, 102],
 'token_type_ids': [0, 0, 0, 0, 0, 0, 0, 0, 0],
 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1]}
```

--------------------------------

### Launching Distributed Training with Accelerate CLI

Source: https://github.com/huggingface/course/blob/main/subtitles/en/raw/chapter3/04b_accelerate.md

Use the Accelerate command-line interface to configure your distributed training setup and launch your training script. First, run 'accelerate config' to generate a configuration file.

```bash
accelerate config
accelerate launch path/to/your/training_script.py
```

--------------------------------

### Full Hugging Face Model Setup

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter8/4.mdx

Complete Python script for setting up a Hugging Face `Trainer` for sequence classification. Includes dataset loading, tokenization, model instantiation, training arguments, metric computation, and data collation. Ensure all imports are present before execution.

```python
from datasets import load_dataset
import evaluate
from transformers import (
    AutoTokenizer,
    AutoModelForSequenceClassification,
    DataCollatorWithPadding,
    TrainingArguments,
    Trainer,
)

raw_datasets = load_dataset("glue", "mnli")

model_checkpoint = "distilbert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)


def preprocess_function(examples):
    return tokenizer(examples["premise"], examples["hypothesis"], truncation=True)

tokenized_datasets = raw_datasets.map(preprocess_function, batched=True)
model = AutoModelForSequenceClassification.from_pretrained(model_checkpoint, num_labels=3)

args = TrainingArguments(
    f"distilbert-finetuned-mnli",
    evaluation_strategy="epoch",
    save_strategy="epoch",
    learning_rate=2e-5,
    num_train_epochs=3,
    weight_decay=0.01,
)

metric = evaluate.load("glue", "mnli")


def compute_metrics(eval_pred):
    predictions, labels = eval_pred
    return metric.compute(predictions=predictions, references=labels)

data_collator = DataCollatorWithPadding(tokenizer=tokenizer)

trainer = Trainer(
    model,
    args,
    train_dataset=tokenized_datasets["train"],
    eval_dataset=tokenized_datasets["validation_matched"],
    compute_metrics=compute_metrics,
    data_collator=data_collator,
    tokenizer=tokenizer,
)
```

--------------------------------

### Initialize SFTTrainer with LoRA Configuration

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter11/4.mdx

Set up the SFTTrainer by passing the base model, training arguments, dataset, the configured PEFT (LoRA) config, and maximum sequence length. This prepares the trainer for fine-tuning with LoRA.

```python
# Create SFTTrainer with LoRA configuration
trainer = SFTTrainer(
    model=model,
    args=args,
    train_dataset=dataset["train"],
    peft_config=peft_config,  # LoRA configuration
    max_seq_length=max_seq_length,  # Maximum sequence length
    processing_class=tokenizer,
)
```

--------------------------------

### Preprocess Validation Examples

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter7/7.mdx

This function preprocesses validation examples by tokenizing questions and contexts, adjusting offset mappings to exclude question parts, and associating each processed sample with its original example ID. Use this when preparing validation data for a question answering model.

```python
def preprocess_validation_examples(examples):
    questions = [q.strip() for q in examples["question"]]
    inputs = tokenizer(
        questions,
        examples["context"],
        max_length=max_length,
        truncation="only_second",
        stride=stride,
        return_overflowing_tokens=True,
        return_offsets_mapping=True,
        padding="max_length",
    )

    sample_map = inputs.pop("overflow_to_sample_mapping")
    example_ids = []

    for i in range(len(inputs["input_ids"])):
        sample_idx = sample_map[i]
        example_ids.append(examples["id"][sample_idx])

        sequence_ids = inputs.sequence_ids(i)
        offset = inputs["offset_mapping"][i]
        inputs["offset_mapping"][i] = [
            o if sequence_ids[k] == 1 else None for k, o in enumerate(offset)
        ]

    inputs["example_id"] = example_ids
    return inputs
```

--------------------------------

### Enabling Dataset Packing with SFTTrainer

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter11/3.mdx

This example demonstrates how to enable dataset packing for improved training efficiency by setting `packing=True` in SFTConfig. Be mindful that packing can lead to more epochs than anticipated when used with `max_steps`.

```python
# Configure packing
training_args = SFTConfig(packing=True)

trainer = SFTTrainer(model=model, train_dataset=dataset, args=training_args)

trainer.train()
```

--------------------------------

### Install rouge_score package

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter7/5.mdx

Install the necessary package to compute ROUGE scores. This is a prerequisite for using the ROUGE metric in 🤗 Datasets.

```python
!pip install rouge_score
```

--------------------------------

### Install requests library

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter5/5.mdx

Install the 'requests' library to make HTTP requests in Python. This is a prerequisite for fetching data from the GitHub API.

```python
!pip install requests
```

--------------------------------

### Set up Hugging Face Hub Repository

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter7/4.mdx

Determines the repository name for the model on the Hugging Face Hub and clones the repository locally. Ensure you are logged in to the Hugging Face Hub.

```python
from huggingface_hub import Repository, get_full_repo_name

model_name = "marian-finetuned-kde4-en-to-fr-accelerate"
repo_name = get_full_repo_name(model_name)

output_dir = "marian-finetuned-kde4-en-to-fr-accelerate"
repo = Repository(output_dir, clone_from=repo_name)
```

--------------------------------

### Load a Model with Transformers

Source: https://github.com/huggingface/course/blob/main/subtitles/en/raw/chapter4/01_huggingface-hub.md

Demonstrates how to load a pre-trained model using the Transformers library. Ensure the 'transformers' library is installed.

```python
from transformers import pipeline

classifier = pipeline("sentiment-analysis")
result = classifier("I've been learning to code and I love it!")
print(result)
```

--------------------------------

### Question Answering Output Example

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter7/7.mdx

Example of the output format from the question-answering pipeline, showing the answer, confidence score, and character indices.

```python
{
 'score': 0.9979003071784973,
 'start': 78,
 'end': 105,
 'answer': 'Jax, PyTorch and TensorFlow'
}
```

--------------------------------

### Extract Start and End Logits (TensorFlow)

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter7/7.mdx

Extracts the start and end logits from the model's output and converts them to NumPy arrays for further processing.

```python
start_logits = outputs.start_logits.numpy()
end_logits = outputs.end_logits.numpy()
```

--------------------------------

### Start GRPO Training

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter12/6.mdx

Initiates the training process using the configured GRPOTrainer. Note that improvements in rewards may not be immediately visible and can take several hundred steps.

```python
trainer.train()
```

--------------------------------

### Format Course Files for Pull Request

Source: https://github.com/huggingface/course/blob/main/README.md

Install project dependencies and run the `make style` command to ensure your translated files adhere to the project's formatting standards before submitting a pull request.

```bash
pip install -r requirements.txt
make style
```

--------------------------------

### Extract Start and End Logits (PyTorch)

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter7/7.mdx

Extracts the start and end logits from the model's output and converts them to NumPy arrays for further processing.

```python
start_logits = outputs.start_logits.cpu().numpy()
end_logits = outputs.end_logits.cpu().numpy()
```

--------------------------------

### Multi-step Demo: Speech-to-Text and Sentiment Analysis

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter9/7.mdx

This example creates a multi-step demo where audio input is converted to text, and then that text is analyzed for sentiment. The output of the speech-to-text model (`text`) is used as the input for the sentiment analysis model.

```python
from transformers import pipeline

import gradio as gr

asr = pipeline("automatic-speech-recognition", "facebook/wav2vec2-base-960h")
classifier = pipeline("text-classification")


def speech_to_text(speech):
    text = asr(speech)["text"]
    return text

def text_to_sentiment(text):
    return classifier(text)[0]["label"]


demo = gr.Blocks()

with demo:
    audio_file = gr.Audio(type="filepath")
    text = gr.Textbox()
    label = gr.Label()

    b1 = gr.Button("Recognize Speech")
    b2 = gr.Button("Classify Sentiment")

    b1.click(speech_to_text, inputs=audio_file, outputs=text)
    b2.click(text_to_sentiment, inputs=text, outputs=label)

demo.launch()
```

--------------------------------

### Test Tokenization on a Single Example

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter5/3.mdx

Applies the `tokenize_and_split` function to a single example from the dataset to observe the output structure and lengths of tokenized inputs.

```python
result = tokenize_and_split(drug_dataset["train"][0])
[len(inp) for inp in result["input_ids"]]
```

--------------------------------

### Example Reasoning Process

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter12/1.mdx

Illustrates a problem, the thought process to solve it, and the final answer. This format helps understand how LLMs can be trained to 'think' step-by-step.

```text
Problem: "I have 3 apples and 2 oranges. How many pieces of fruit do I have in total?"

Thought: "I need to add the number of apples and oranges to get the total number of pieces of fruit."

Answer: "5"
```

--------------------------------

### Configure PyTorch Training Arguments

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter7/6.mdx

Set up training parameters such as output directory, batch sizes, evaluation and logging steps, learning rate schedule, and whether to push to the Hub.

```python
from transformers import Trainer, TrainingArguments

args = TrainingArguments(
    output_dir="codeparrot-ds",
    per_device_train_batch_size=32,
    per_device_eval_batch_size=32,
    evaluation_strategy="steps",
    eval_steps=5_000,
    logging_steps=5_000,
    gradient_accumulation_steps=8,
    num_train_epochs=1,
    weight_decay=0.1,
    warmup_steps=1_000,
    lr_scheduler_type="cosine",
    learning_rate=5e-4,
    save_steps=5_000,
    fp16=True,
    push_to_hub=True,
)
```

--------------------------------

### Model Card Metadata Example

Source: https://github.com/huggingface/course/blob/main/chapters/en/chapter4/4.mdx

This is an example of metadata that can be included in a model card header. It specifies the language, license, and datasets the model is associated with.

```yaml
---
language: fr
license: mit
datasets:
- oscar
---
```