### Installing Dependencies for LLM Fine-tuning - Python

Source: https://github.com/mlabonne/llm-course/blob/main/Fine_tune_a_Mistral_7b_model_with_DPO.ipynb

This snippet installs the necessary Python libraries required for fine-tuning large language models, including `datasets` for data handling, `trl` for DPO training, `peft` for parameter-efficient fine-tuning, `bitsandbytes` for 4-bit quantization, `sentencepiece` for tokenization, and `wandb` for experiment tracking.

```python
!pip install -q datasets trl peft bitsandbytes sentencepiece wandb
```

--------------------------------

### Quantizing Llama 2 Models with GGUF and llama.cpp (Python)

Source: https://github.com/mlabonne/llm-course/blob/main/Quantize_Llama_2_models_using_GGUF_and_llama_cpp.ipynb

This snippet sets up the environment for Llama 2 model quantization. It defines model ID and quantization methods, installs `llama.cpp`, downloads the specified model from Hugging Face, converts it to FP16 format, and then quantizes it using the defined methods. It requires `git`, `make`, `pip`, and `huggingface_hub`.

```Python
# Variables
MODEL_ID = "mlabonne/EvolCodeLlama-7b"
QUANTIZATION_METHODS = ["q4_k_m", "q5_k_m"]

# Constants
MODEL_NAME = MODEL_ID.split('/')[-1]

# Install llama.cpp
!git clone https://github.com/ggerganov/llama.cpp
!cd llama.cpp && git pull && make clean && LLAMA_CUBLAS=1 make
!pip install -r llama.cpp/requirements.txt

# Download model
!git lfs install
!git clone https://huggingface.co/{MODEL_ID}

# Convert to fp16
fp16 = f"{MODEL_NAME}/{MODEL_NAME.lower()}.fp16.bin"
!python llama.cpp/convert.py {MODEL_NAME} --outtype f16 --outfile {fp16}

# Quantize the model for each method in the QUANTIZATION_METHODS list
for method in QUANTIZATION_METHODS:
    qtype = f"{MODEL_NAME}/{MODEL_NAME.lower()}.{method.upper()}.gguf"
    !./llama.cpp/quantize {fp16} {qtype} {method}
```

--------------------------------

### Setting Up Axolotl Environment in Colab

Source: https://github.com/mlabonne/llm-course/blob/main/Fine_tune_LLMs_with_Axolotl.ipynb

This snippet clones the Axolotl repository, changes the current directory to the cloned repository, and installs the necessary Python dependencies, including 'packaging', 'huggingface_hub', and optional components like 'flash-attn' and 'deepspeed' for optimized training.

```Python
!git clone -q https://github.com/OpenAccess-AI-Collective/axolotl
%cd axolotl
!pip install -qqq packaging huggingface_hub --progress-bar off
!pip install -qqq -e '.[flash-attn,deepspeed]' --progress-bar off
```

--------------------------------

### Installing AutoGPTQ and Transformers Libraries (Python)

Source: https://github.com/mlabonne/llm-course/blob/main/4_bit_LLM_Quantization_with_GPTQ.ipynb

This snippet installs the necessary `auto-gptq` and `transformers` libraries in a quiet mode. `BUILD_CUDA_EXT=0` ensures that CUDA extensions are not built, which is useful for environments without CUDA or to avoid compilation issues.

```python
!BUILD_CUDA_EXT=0 pip install -q auto-gptq transformers
```

--------------------------------

### Formatting DPO Dataset with ChatML - Python

Source: https://github.com/mlabonne/llm-course/blob/main/Fine_tune_a_Mistral_7b_model_with_DPO.ipynb

This snippet defines a `chatml_format` function to convert raw dataset examples into the ChatML format required for DPO training, separating system, user prompt, chosen, and rejected responses. It then loads the `Intel/orca_dpo_pairs` dataset, initializes the tokenizer, and applies the formatting function to prepare the data for the `DPOTrainer`.

```python
def chatml_format(example):
    # Format system
    if len(example['system']) > 0:
        message = {"role": "system", "content": example['system']}
        system = tokenizer.apply_chat_template([message], tokenize=False)
    else:
        system = ""

    # Format instruction
    message = {"role": "user", "content": example['question']}
    prompt = tokenizer.apply_chat_template([message], tokenize=False, add_generation_prompt=True)

    # Format chosen answer
    chosen = example['chosen'] + "<|im_end|>\n"

    # Format rejected answer
    rejected = example['rejected'] + "<|im_end|>\n"

    return {
        "prompt": system + prompt,
        "chosen": chosen,
        "rejected": rejected,
    }

# Load dataset
dataset = load_dataset("Intel/orca_dpo_pairs")['train']

# Save columns
original_columns = dataset.column_names

# Tokenizer
tokenizer = AutoTokenizer.from_pretrained(model_name)
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = "left"

# Format dataset
dataset = dataset.map(
    chatml_format,
    remove_columns=original_columns
)

# Print sample
dataset[1]
```

--------------------------------

### Installing mergekit for LLM Merging

Source: https://github.com/mlabonne/llm-course/blob/main/Mergekit.ipynb

This Python snippet executes shell commands to clone the mergekit repository from GitHub and then installs it in editable mode using pip. This step is a prerequisite for using mergekit functionalities, ensuring all necessary dependencies are installed for model merging operations.

```python
!git clone https://github.com/cg123/mergekit.git
!cd mergekit && pip install -q -e .
```

--------------------------------

### Installing Required Libraries for Llama 2 Fine-tuning (Python)

Source: https://github.com/mlabonne/llm-course/blob/main/Fine_tune_Llama_2_in_Google_Colab.ipynb

This snippet installs all necessary Python libraries for fine-tuning Llama 2, including `accelerate`, `peft`, `bitsandbytes`, `transformers`, and `trl`. These libraries are crucial for efficient model loading, quantization, parameter-efficient fine-tuning (PEFT), and supervised fine-tuning (SFT).

```Python
!pip install -q accelerate==0.21.0 peft==0.4.0 bitsandbytes==0.40.2 transformers==4.31.0 trl==0.4.7
```

--------------------------------

### Launching Axolotl Model Training

Source: https://github.com/mlabonne/llm-course/blob/main/Fine_tune_LLMs_with_Axolotl.ipynb

This command initiates the fine-tuning process using Axolotl's command-line interface. It leverages 'accelerate' for distributed training and points to the 'config.yaml' file created in the previous step, which contains all the necessary training configurations.

```Python
!accelerate launch -m axolotl.cli.train config.yaml
```

--------------------------------

### Importing Libraries and Defining Model Parameters (Python)

Source: https://github.com/mlabonne/llm-course/blob/main/4_bit_LLM_Quantization_with_GPTQ.ipynb

This snippet imports essential libraries like `auto_gptq`, `datasets`, `torch`, and `transformers`. It also defines the base model ID (`gpt2`) and the output directory name for the quantized model, setting up initial configurations for the quantization process.

```python
import random

from auto_gptq import AutoGPTQForCausalLM, BaseQuantizeConfig
from datasets import load_dataset
import torch
from transformers import AutoTokenizer


# Define base model and output directory
model_id = "gpt2"
out_dir = model_id + "-GPTQ"
```

--------------------------------

### Importing Libraries and Initializing Configuration - Python

Source: https://github.com/mlabonne/llm-course/blob/main/Fine_tune_a_Mistral_7b_model_with_DPO.ipynb

This code block imports all required Python libraries for the DPO fine-tuning process, including `transformers`, `datasets`, `peft`, and `trl`. It also retrieves API tokens for Hugging Face and Weights & Biases from Google Colab secrets and defines the base and new model names for the fine-tuning task.

```python
import os
import gc
import torch

import transformers
from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments, BitsAndBytesConfig
from datasets import load_dataset
from peft import LoraConfig, PeftModel, get_peft_model, prepare_model_for_kbit_training
from trl import DPOTrainer
import bitsandbytes as bnb
from google.colab import userdata
import wandb

# Defined in the secrets tab in Google Colab
hf_token = userdata.get('huggingface')
wb_token = userdata.get('wandb')
wandb.login(key=wb_token)

model_name = "teknium/OpenHermes-2.5-Mistral-7B"
new_model = "NeuralHermes-2.5-Mistral-7B"
```

--------------------------------

### Running Inference with Quantized Llama 2 Models (Python)

Source: https://github.com/mlabonne/llm-course/blob/main/Quantize_Llama_2_models_using_GGUF_and_llama_cpp.ipynb

This Python script demonstrates how to run inference using a quantized Llama 2 model. It lists available GGUF models, prompts the user to select one, and then executes the `llama.cpp/main` executable with the chosen model, offloading layers to the GPU for faster inference. It requires `llama.cpp` to be built and the quantized GGUF models to be present.

```Python
import os

model_list = [file for file in os.listdir(MODEL_NAME) if "gguf" in file]

prompt = input("Enter your prompt: ")
chosen_method = input("Name of the model (options: " + ", ".join(model_list) + "): ")

# Verify the chosen method is in the list
if chosen_method not in model_list:
    print("Invalid name")
else:
    qtype = f"{MODEL_NAME}/{MODEL_NAME.lower()}.{method.upper()}.gguf"
    !./llama.cpp/main -m {qtype} -n 128 --color -ngl 35 -p "{prompt}"
```

--------------------------------

### Configuring Axolotl Training Parameters with YAML

Source: https://github.com/mlabonne/llm-course/blob/main/Fine_tune_LLMs_with_Axolotl.ipynb

This code defines a comprehensive YAML configuration string for Axolotl, specifying parameters like the base model, QLoRA adapter settings, dataset path, training hyperparameters (e.g., num_epochs, learning_rate), and output directory. It then parses this string into a Python dictionary and writes it to a 'config.yaml' file, which Axolotl will use for training.

```Python
import yaml

new_model = "mlabonne/TinyAlpaca"
yaml_string = """
base_model: TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T
model_type: LlamaForCausalLM
tokenizer_type: LlamaTokenizer
is_llama_derived_model: true

load_in_8bit: false
load_in_4bit: true
strict: false

datasets:
  - path: mhenrichsen/alpaca_2k_test
    type: alpaca
dataset_prepared_path:
val_set_size: 0.05
output_dir: ./qlora-out

adapter: qlora
lora_model_dir:

sequence_len: 1096
sample_packing: true
pad_to_sequence_len: true

lora_r: 32
lora_alpha: 16
lora_dropout: 0.05
lora_target_modules:
lora_target_linear: true
lora_fan_in_fan_out:

wandb_project:
wandb_entity:
wandb_watch:
wandb_name:
wandb_log_model:

mlflow_experiment_name: colab-example

gradient_accumulation_steps: 1
micro_batch_size: 1
num_epochs: 4
max_steps: 20
optimizer: paged_adamw_32bit
lr_scheduler: cosine
learning_rate: 0.0002

train_on_inputs: false
group_by_length: false
bf16: false
fp16: true
tf32: false

gradient_checkpointing: true
early_stopping_patience:
resume_from_checkpoint:
local_rank:
logging_steps: 1
xformers_attention:
flash_attention: false

warmup_steps: 10
evals_per_epoch:
saves_per_epoch:
debug:
deepspeed:
weight_decay: 0.0
fsdp:
fsdp_config:
special_tokens:

"""

# Convert the YAML string to a Python dictionary
yaml_dict = yaml.safe_load(yaml_string)

# Specify your file path
yaml_file = 'config.yaml'

# Write the YAML file
with open(yaml_file, 'w') as file:
    yaml.dump(yaml_dict, file)
```

--------------------------------

### Training Mistral-7B with DPO using PEFT - Python

Source: https://github.com/mlabonne/llm-course/blob/main/Fine_tune_a_Mistral_7b_model_with_DPO.ipynb

This code block configures LoRA for parameter-efficient fine-tuning, loads the base Mistral-7B model with 4-bit quantization, and defines `TrainingArguments` for the DPO process. It then initializes the `DPOTrainer` with the prepared dataset, tokenizer, and LoRA configuration, and finally initiates the model training.

```python
# LoRA configuration
peft_config = LoraConfig(
    r=16,
    lora_alpha=16,
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM",
    target_modules=['k_proj', 'gate_proj', 'v_proj', 'up_proj', 'q_proj', 'o_proj', 'down_proj']
)

# Model to fine-tune
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.float16,
    load_in_4bit=True
)
model.config.use_cache = False

# Training arguments
training_args = TrainingArguments(
    per_device_train_batch_size=4,
    gradient_accumulation_steps=4,
    gradient_checkpointing=True,
    learning_rate=5e-5,
    lr_scheduler_type="cosine",
    max_steps=200,
    save_strategy="no",
    logging_steps=1,
    output_dir=new_model,
    optim="paged_adamw_32bit",
    warmup_steps=100,
    bf16=True,
    report_to="wandb",
)

# Create DPO trainer
dpo_trainer = DPOTrainer(
    model,
    args=training_args,
    train_dataset=dataset,
    tokenizer=tokenizer,
    peft_config=peft_config,
    beta=0.1,
    max_prompt_length=1024,
    max_length=1536,
)

# Fine-tune model with DPO
dpo_trainer.train()
```

--------------------------------

### Loading TensorBoard Extension (Python)

Source: https://github.com/mlabonne/llm-course/blob/main/Fine_tune_Llama_2_in_Google_Colab.ipynb

This commented-out snippet shows how to load the TensorBoard extension in a Jupyter/Colab environment and launch TensorBoard to visualize training logs. It's useful for monitoring the training progress and metrics stored in the specified log directory.

```Python
# %load_ext tensorboard
# %tensorboard --logdir results/runs
```

--------------------------------

### Performing Inference with Fine-tuned Model - Python

Source: https://github.com/mlabonne/llm-course/blob/main/Fine_tune_a_Mistral_7b_model_with_DPO.ipynb

This code demonstrates how to perform inference with the newly fine-tuned model. It formats a conversational prompt using the ChatML template, initializes a `text-generation` pipeline with the fine-tuned model and tokenizer, and then generates a response based on the provided prompt with specified sampling parameters.

```python
# Format prompt
message = [
    {"role": "system", "content": "You are a helpful assistant chatbot."},
    {"role": "user", "content": "What is a Large Language Model?"}
]
tokenizer = AutoTokenizer.from_pretrained(new_model)
prompt = tokenizer.apply_chat_template(message, add_generation_prompt=True, tokenize=False)

# Create pipeline
pipeline = transformers.pipeline(
    "text-generation",
    model=new_model,
    tokenizer=tokenizer
)

# Generate text
sequences = pipeline(
    prompt,
    do_sample=True,
    temperature=0.7,
    top_p=0.9,
    num_return_sequences=1,
    max_length=200,
)
print(sequences[0]['generated_text'])
```

--------------------------------

### Loading, Configuring, and Training Llama 2 with QLoRA (Python)

Source: https://github.com/mlabonne/llm-course/blob/main/Fine_tune_Llama_2_in_Google_Colab.ipynb

This comprehensive snippet loads the instruction dataset, initializes the Llama 2 model with `BitsAndBytesConfig` for 4-bit quantization, and sets up the tokenizer. It then defines the `LoraConfig` for PEFT and configures `TrainingArguments` for the `SFTTrainer`. Finally, it executes the training process and saves the fine-tuned model.

```Python
# Load dataset (you can process it here)
dataset = load_dataset(dataset_name, split="train")

# Load tokenizer and model with QLoRA configuration
compute_dtype = getattr(torch, bnb_4bit_compute_dtype)

bnb_config = BitsAndBytesConfig(
    load_in_4bit=use_4bit,
    bnb_4bit_quant_type=bnb_4bit_quant_type,
    bnb_4bit_compute_dtype=compute_dtype,
    bnb_4bit_use_double_quant=use_nested_quant,
)

# Check GPU compatibility with bfloat16
if compute_dtype == torch.float16 and use_4bit:
    major, _ = torch.cuda.get_device_capability()
    if major >= 8:
        print("=" * 80)
        print("Your GPU supports bfloat16: accelerate training with bf16=True")
        print("=" * 80)

# Load base model
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    quantization_config=bnb_config,
    device_map=device_map
)
model.config.use_cache = False
model.config.pretraining_tp = 1

# Load LLaMA tokenizer
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = "right" # Fix weird overflow issue with fp16 training

# Load LoRA configuration
peft_config = LoraConfig(
    lora_alpha=lora_alpha,
    lora_dropout=lora_dropout,
    r=lora_r,
    bias="none",
    task_type="CAUSAL_LM",
)

# Set training parameters
training_arguments = TrainingArguments(
    output_dir=output_dir,
    num_train_epochs=num_train_epochs,
    per_device_train_batch_size=per_device_train_batch_size,
    gradient_accumulation_steps=gradient_accumulation_steps,
    optim=optim,
    save_steps=save_steps,
    logging_steps=logging_steps,
    learning_rate=learning_rate,
    weight_decay=weight_decay,
    fp16=fp16,
    bf16=bf16,
    max_grad_norm=max_grad_norm,
    max_steps=max_steps,
    warmup_ratio=warmup_ratio,
    group_by_length=group_by_length,
    lr_scheduler_type=lr_scheduler_type,
    report_to="tensorboard"
)

# Set supervised fine-tuning parameters
trainer = SFTTrainer(
    model=model,
    train_dataset=dataset,
    peft_config=peft_config,
    dataset_text_field="text",
    max_seq_length=max_seq_length,
    tokenizer=tokenizer,
    args=training_arguments,
    packing=packing,
)

# Train model
trainer.train()

# Save trained model
trainer.model.save_pretrained(new_model)
```

--------------------------------

### Executing GPTQ Quantization and Saving Model (Python)

Source: https://github.com/mlabonne/llm-course/blob/main/4_bit_LLM_Quantization_with_GPTQ.ipynb

This snippet performs the actual 4-bit quantization of the loaded model using the prepared `examples_ids` and `use_triton=True` for optimized performance. After quantization, it saves the newly quantized model and its tokenizer to the specified output directory, utilizing `safetensors` for efficient storage.

```python
%%time

# Quantize with GPTQ
model.quantize(
    examples_ids,
    batch_size=1,
    use_triton=True,
)

# Save model and tokenizer
model.save_quantized(out_dir, use_safetensors=True)
tokenizer.save_pretrained(out_dir)
```

--------------------------------

### Loading Model, Tokenizer, and Quantization Configuration (Python)

Source: https://github.com/mlabonne/llm-course/blob/main/4_bit_LLM_Quantization_with_GPTQ.ipynb

This snippet initializes the `BaseQuantizeConfig` for 4-bit quantization, specifying parameters like `bits`, `group_size`, `damp_percent`, and `desc_act`. It then loads the pre-trained model (`gpt2`) with the defined quantization configuration and its corresponding tokenizer using `AutoGPTQForCausalLM` and `AutoTokenizer`.

```python
# Load quantize config, model and tokenizer
quantize_config = BaseQuantizeConfig(
    bits=4,
    group_size=128,
    damp_percent=0.01,
    desc_act=False,
)
model = AutoGPTQForCausalLM.from_pretrained(model_id, quantize_config)
tokenizer = AutoTokenizer.from_pretrained(model_id)
```

--------------------------------

### Preparing Dataset for Quantization (Python)

Source: https://github.com/mlabonne/llm-course/blob/main/4_bit_LLM_Quantization_with_GPTQ.ipynb

This snippet loads a subset of the C4 dataset, tokenizes the text, and then formats it into a list of dictionaries suitable for the GPTQ quantization process. It randomly selects `n_samples` segments of text, each of `tokenizer.model_max_length` tokens, along with their attention masks.

```python
# Load data and tokenize examples
n_samples = 1024
data = load_dataset("allenai/c4", data_files="en/c4-train.00001-of-01024.json.gz", split=f"train[:{n_samples*5}]")
tokenized_data = tokenizer("\n\n".join(data['text']), return_tensors='pt')

# Format tokenized examples
examples_ids = []
for _ in range(n_samples):
    i = random.randint(0, tokenized_data.input_ids.shape[1] - tokenizer.model_max_length - 1)
    j = i + tokenizer.model_max_length
    input_ids = tokenized_data.input_ids[:, i:j]
    attention_mask = torch.ones_like(input_ids)
    examples_ids.append({'input_ids': input_ids, 'attention_mask': attention_mask})
```

--------------------------------

### Uploading Fine-tuned Model to Hugging Face Hub

Source: https://github.com/mlabonne/llm-course/blob/main/Fine_tune_LLMs_with_Axolotl.ipynb

This snippet uses the 'huggingface_hub' library to programmatically upload the merged, fine-tuned model to the Hugging Face Hub. It creates a new model repository (or uses an existing one) and then uploads the contents of the 'qlora-out/merged' directory, making the fine-tuned model publicly accessible.

```Python
from huggingface_hub import HfApi
from google.colab import userdata

new_model = "mlabonne/TinyAlpaca"

# HF_TOKEN defined in the secrets tab in Google Colab
api = HfApi()

# Upload merge folder
api.create_repo(
    repo_id=new_model,
    repo_type="model",
    exist_ok=True,
)
api.upload_folder(
    repo_id=new_model,
    folder_path="qlora-out/merged",
)
```

--------------------------------

### Pushing Quantized Llama 2 Models to Hugging Face Hub (Python)

Source: https://github.com/mlabonne/llm-course/blob/main/Quantize_Llama_2_models_using_GGUF_and_llama_cpp.ipynb

This snippet uploads the quantized GGUF models to a new or existing repository on the Hugging Face Hub. It requires the `huggingface_hub` library and a Hugging Face token stored in Google Colab's 'Secrets' tab. The script creates a new model repository with a '-GGUF' suffix and then uploads all `.gguf` files from the model directory.

```Python
!pip install -q huggingface_hub
from huggingface_hub import create_repo, HfApi
from google.colab import userdata

# Defined in the secrets tab in Google Colab
hf_token = userdata.get('huggingface')

api = HfApi()
username = "mlabonne"

# Create empty repo
create_repo(
    repo_id = f"{username}/{MODEL_NAME}-GGUF",
    repo_type="model",
    exist_ok=True,
    token=hf_token
)

# Upload gguf files
api.upload_folder(
    folder_path=MODEL_NAME,
    repo_id=f"{username}/{MODEL_NAME}-GGUF",
    allow_patterns=f"*.gguf",
    token=hf_token
)
```

--------------------------------

### Performing Text Generation with Quantized Model (Python)

Source: https://github.com/mlabonne/llm-course/blob/main/4_bit_LLM_Quantization_with_GPTQ.ipynb

This snippet demonstrates how to use the reloaded 4-bit quantized model for text generation. It initializes a `text-generation` pipeline from the `transformers` library and then generates text based on a given prompt, printing the resulting output.

```python
from transformers import pipeline

generator = pipeline('text-generation', model=model, tokenizer=tokenizer)
result = generator("I have a dream", do_sample=True, max_length=50)[0]['generated_text']
print(result)
```

--------------------------------

### Merging and Uploading Fine-tuned Model to Hugging Face Hub - Python

Source: https://github.com/mlabonne/llm-course/blob/main/Fine_tune_a_Mistral_7b_model_with_DPO.ipynb

This snippet saves the fine-tuned adapter and tokenizer, then clears GPU memory. It reloads the base model in FP16, merges the trained LoRA adapter with the base model, and unloads the adapter. Finally, the merged model and tokenizer are saved locally and pushed to the Hugging Face Hub for sharing and deployment.

```python
# Save artifacts
dpo_trainer.model.save_pretrained("final_checkpoint")
tokenizer.save_pretrained("final_checkpoint")

# Flush memory
del dpo_trainer, model
gc.collect()
torch.cuda.empty_cache()

# Reload model in FP16 (instead of NF4)
base_model = AutoModelForCausalLM.from_pretrained(
    model_name,
    return_dict=True,
    torch_dtype=torch.float16,
)
tokenizer = AutoTokenizer.from_pretrained(model_name)

# Merge base model with the adapter
model = PeftModel.from_pretrained(base_model, "final_checkpoint")
model = model.merge_and_unload()

# Save model and tokenizer
model.save_pretrained(new_model)
tokenizer.save_pretrained(new_model)

# Push them to the HF Hub
model.push_to_hub(new_model, use_temp_dir=False, token=hf_token)
tokenizer.push_to_hub(new_model, use_temp_dir=False, token=hf_token)
```

--------------------------------

### Reloading Quantized Model and Tokenizer (Python)

Source: https://github.com/mlabonne/llm-course/blob/main/4_bit_LLM_Quantization_with_GPTQ.ipynb

This snippet determines the available device (CUDA or CPU) and then reloads the previously quantized model and its tokenizer from the saved directory. It uses `AutoGPTQForCausalLM.from_quantized` to load the 4-bit model, ensuring it's ready for inference.

```python
device = "cuda:0" if torch.cuda.is_available() else "cpu"

# Reload model and tokenizer
model = AutoGPTQForCausalLM.from_quantized(
    out_dir,
    device=device,
    use_triton=True,
    use_safetensors=True,
)
tokenizer = AutoTokenizer.from_pretrained(out_dir)
```

--------------------------------

### Executing LLM Merge with mergekit-yaml

Source: https://github.com/mlabonne/llm-course/blob/main/Mergekit.ipynb

This Python snippet executes the `mergekit-yaml` command-line tool to perform the model merge based on the `config.yaml` file. It includes flags to copy the tokenizer, allow 'crimes' (potential conflicts), set the output shard size to 1GB, and enable lazy unpickling for efficient memory usage during the merge process.

```python
!mergekit-yaml config.yaml merge --copy-tokenizer --allow-crimes --out-shard-size 1B --lazy-unpickle
```

--------------------------------

### Uploading Merged LLM to Hugging Face Hub

Source: https://github.com/mlabonne/llm-course/blob/main/Mergekit.ipynb

This Python snippet handles the authentication and upload of the merged model to the Hugging Face Hub. It uses the `huggingface_hub` library, authenticates with a token retrieved from Google Colab's user data, creates a new repository on the Hugging Face Hub, and then uploads the entire `merge` folder containing the merged model and its generated model card.

```python
from google.colab import userdata
from huggingface_hub import HfApi

username = "mlabonne"

# Defined in the secrets tab in Google Colab
api = HfApi(token=userdata.get("HF_TOKEN"))

api.create_repo(
    repo_id=f"{username}/{MODEL_NAME}",
    repo_type="model"
)
api.upload_folder(
    repo_id=f"{username}/{MODEL_NAME}",
    folder_path="merge",
)
```

--------------------------------

### Merging LoRA Adapters with Base Model

Source: https://github.com/mlabonne/llm-course/blob/main/Fine_tune_LLMs_with_Axolotl.ipynb

After training, this command uses Axolotl's utility to merge the trained LoRA adapters with the original base model. The 'config.yaml' specifies the model details, and '--lora_model_dir' indicates the directory where the QLoRA output (the trained adapters) is located, resulting in a single, fine-tuned model.

```Python
!python3 -m axolotl.cli.merge_lora config.yaml --lora_model_dir="./qlora-out"
```

--------------------------------

### Generating Hugging Face Model Card for Merged LLM

Source: https://github.com/mlabonne/llm-course/blob/main/Mergekit.ipynb

This Python snippet dynamically generates a Hugging Face Model Card for the merged LLM using Jinja2 templating. It extracts model names from the previously defined YAML configuration and populates a template with merge details, license, and tags. The generated content is then saved as `README.md` within the `merge` directory, preparing it for upload to the Hugging Face Hub.

```python
!pip install -qU huggingface_hub

from huggingface_hub import ModelCard, ModelCardData
from jinja2 import Template

username = "mlabonne"

template_text = """
---
license: apache-2.0
tags:
- merge
- mergekit
- lazymergekit
{%- for model in models %}
- {{ model }}
{%- endfor %}
---

# {{ model_name }}

{{ model_name }} is a merge of the following models using [mergekit](https://github.com/cg123/mergekit):

{%- for model in models %}
* [{{ model }}](https://huggingface.co/{{ model }})
{%- endfor %}

## 🧩 Configuration

```yaml
{{- yaml_config -}}
```
"""

# Create a Jinja template object
jinja_template = Template(template_text.strip())

# Get list of models from config
data = yaml.safe_load(yaml_config)
if "models" in data:
    models = [data["models"][i]["model"] for i in range(len(data["models"])) if "parameters" in data["models"][i]]
elif "parameters" in data:
    models = [data["slices"][0]["sources"][i]["model"] for i in range(len(data["slices"][0]["sources"]))]
elif "slices" in data:
    models = [data["slices"][i]["sources"][0]["model"] for i in range(len(data["slices"]))]
else:
    raise Exception("No models or slices found in yaml config")

# Fill the template
content = jinja_template.render(
    model_name=MODEL_NAME,
    models=models,
    yaml_config=yaml_config,
    username=username,
)

# Save the model card
card = ModelCard(content)
card.save('merge/README.md')
```

--------------------------------

### Defining and Saving mergekit Configuration in Python

Source: https://github.com/mlabonne/llm-course/blob/main/Mergekit.ipynb

This Python snippet defines a multi-line YAML configuration string for a SLERP merge, specifying source models, layer ranges, and interpolation parameters. It then saves this configuration to a file named `config.yaml`. This file will be used by the `mergekit-yaml` command-line tool to perform the model merging.

```python
import yaml

MODEL_NAME = "Marcoro14-7B-slerp"
yaml_config = """
slices:
  - sources:
      - model: AIDC-ai-business/Marcoroni-7B-v3
        layer_range: [0, 32]
      - model: EmbeddedLLM/Mistral-7B-Merge-14-v0.1
        layer_range: [0, 32]
merge_method: slerp
base_model: AIDC-ai-business/Marcoroni-7B-v3
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5
dtype: bfloat16

"""

# Save config as yaml file
with open('config.yaml', 'w', encoding="utf-8") as f:
    f.write(yaml_config)
```

--------------------------------

### Pushing Fine-tuned Llama 2 Model to Hugging Face Hub (Python)

Source: https://github.com/mlabonne/llm-course/blob/main/Fine_tune_Llama_2_in_Google_Colab.ipynb

This snippet facilitates pushing the fine-tuned Llama 2 model and its tokenizer to the Hugging Face Hub. It first requires logging in via `huggingface-cli login` (which prompts for a token) and then uses the `push_to_hub` method to upload the model and tokenizer, making them publicly or privately accessible.

```Python
!huggingface-cli login

model.push_to_hub(new_model, use_temp_dir=False)
tokenizer.push_to_hub(new_model, use_temp_dir=False)
```

--------------------------------

### Importing Core Libraries for LLM Fine-tuning (Python)

Source: https://github.com/mlabonne/llm-course/blob/main/Fine_tune_Llama_2_in_Google_Colab.ipynb

This code imports essential modules from `os`, `torch`, `datasets`, `transformers`, `peft`, and `trl`. These imports provide functionalities for system operations, tensor computations, dataset loading, model and tokenizer handling, quantization configuration, PEFT (LoRA), and supervised fine-tuning.

```Python
import os
import torch
from datasets import load_dataset
from transformers (
    AutoModelForCausalLM,
    AutoTokenizer,
    BitsAndBytesConfig,
    HfArgumentParser,
    TrainingArguments,
    pipeline,
    logging,
)
from peft import LoraConfig, PeftModel
from trl import SFTTrainer
```

--------------------------------

### Configuring TIES-Merging for LLMs with mergekit

Source: https://github.com/mlabonne/llm-course/blob/main/Mergekit.ipynb

This YAML configuration defines a TIES-Merging strategy for Large Language Models using mergekit. It specifies the base model and two additional models with their respective density and weight parameters, enabling a weighted average merge. The configuration also sets the normalization and data type for the merged model.

```yaml
models:
  - model: mistralai/Mistral-7B-v0.1
    # no parameters necessary for base model
  - model: OpenPipe/mistral-ft-optimized-1218
    parameters:
      density: 0.5
      weight: 0.5
  - model: mlabonne/NeuralHermes-2.5-Mistral-7B
    parameters:
      density: 0.5
      weight: 0.3
merge_method: ties
base_model: mistralai/Mistral-7B-v0.1
parameters:
  normalize: true
dtype: float16
```

--------------------------------

### Testing Fine-tuned Llama 2 Model with Text Generation (Python)

Source: https://github.com/mlabonne/llm-course/blob/main/Fine_tune_Llama_2_in_Google_Colab.ipynb

This snippet demonstrates how to use the fine-tuned Llama 2 model for text generation. It sets the logging verbosity to critical, defines a prompt, and then uses a Hugging Face `pipeline` to generate text based on the prompt, printing the generated output.

```Python
# Ignore warnings
logging.set_verbosity(logging.CRITICAL)

# Run text generation pipeline with our next model
prompt = "What is a large language model?"
pipe = pipeline(task="text-generation", model=model, tokenizer=tokenizer, max_length=200)
result = pipe(f"<s>[INST] {prompt} [/INST]")
print(result[0]['generated_text'])
```

--------------------------------

### Configuring Passthrough Merge for LLMs with mergekit

Source: https://github.com/mlabonne/llm-course/blob/main/Mergekit.ipynb

This YAML configuration demonstrates a Passthrough merge strategy, allowing specific layer ranges from different source models to be combined directly. It defines two slices, each pointing to a source model and a specific layer range, effectively creating a hybrid model by concatenating parts of different models. The configuration also specifies the data type for the merged model.

```yaml
slices:
  - sources:
    - model: OpenPipe/mistral-ft-optimized-1218
      layer_range: [0, 32]
  - sources:
    - model: mlabonne/NeuralHermes-2.5-Mistral-7B
      layer_range: [24, 32]
merge_method: passthrough
dtype: bfloat16
```

--------------------------------

### Defining Llama 2 QLoRA Fine-tuning Parameters (Python)

Source: https://github.com/mlabonne/llm-course/blob/main/Fine_tune_Llama_2_in_Google_Colab.ipynb

This snippet defines various parameters for the Llama 2 QLoRA fine-tuning process, including the base model name, dataset, new model name, LoRA configuration (r, alpha, dropout), bitsandbytes quantization settings (4-bit precision, compute dtype, quantization type, nested quantization), and `TrainingArguments` (output directory, epochs, batch sizes, learning rate, optimizer, etc.). These parameters control the training behavior and model configuration.

```Python
model_name = "NousResearch/Llama-2-7b-chat-hf"

dataset_name = "mlabonne/guanaco-llama2-1k"

new_model = "llama-2-7b-miniguanaco"

################################################################################
# QLoRA parameters
################################################################################

# LoRA attention dimension
lora_r = 64

# Alpha parameter for LoRA scaling
lora_alpha = 16

# Dropout probability for LoRA layers
lora_dropout = 0.1

################################################################################
# bitsandbytes parameters
################################################################################

# Activate 4-bit precision base model loading
use_4bit = True

# Compute dtype for 4-bit base models
bnb_4bit_compute_dtype = "float16"

# Quantization type (fp4 or nf4)
bnb_4bit_quant_type = "nf4"

# Activate nested quantization for 4-bit base models (double quantization)
use_nested_quant = False

################################################################################
# TrainingArguments parameters
################################################################################

# Output directory where the model predictions and checkpoints will be stored
output_dir = "./results"

# Number of training epochs
num_train_epochs = 1

# Enable fp16/bf16 training (set bf16 to True with an A100)
fp16 = False
bf16 = False

# Batch size per GPU for training
per_device_train_batch_size = 4

# Batch size per GPU for evaluation
per_device_eval_batch_size = 4

# Number of update steps to accumulate the gradients for
gradient_accumulation_steps = 1

# Enable gradient checkpointing
gradient_checkpointing = True

# Maximum gradient normal (gradient clipping)
max_grad_norm = 0.3

# Initial learning rate (AdamW optimizer)
learning_rate = 2e-4

# Weight decay to apply to all layers except bias/LayerNorm weights
weight_decay = 0.001

# Optimizer to use
optim = "paged_adamw_32bit"

# Learning rate schedule
lr_scheduler_type = "cosine"

# Number of training steps (overrides num_train_epochs)
max_steps = -1

# Ratio of steps for a linear warmup (from 0 to learning rate)
warmup_ratio = 0.03

# Group sequences into batches with same length
# Saves memory and speeds up training considerably
group_by_length = True

# Save checkpoint every X updates steps
save_steps = 0

# Log every X updates steps
logging_steps = 25

################################################################################
# SFT parameters
################################################################################

# Maximum sequence length to use
max_seq_length = None

# Pack multiple short examples in the same input sequence to increase efficiency
packing = False

# Load the entire model on the GPU 0
device_map = {"": 0}
```

--------------------------------

### Configuring SLERP for LLMs with mergekit

Source: https://github.com/mlabonne/llm-course/blob/main/Mergekit.ipynb

This YAML configuration outlines a Spherical Linear Interpolation (SLERP) merge for Large Language Models. It defines slices of models to be merged, specifying layer ranges for each source model. The configuration includes `t` parameters for fine-grained control over the interpolation across different model components like self-attention and MLP layers, and sets the data type.

```yaml
slices:
  - sources:
      - model: OpenPipe/mistral-ft-optimized-1218
        layer_range: [0, 32]
      - model: mlabonne/NeuralHermes-2.5-Mistral-7B
        layer_range: [0, 32]
merge_method: slerp
base_model: OpenPipe/mistral-ft-optimized-1218
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5
dtype: bfloat16
```

--------------------------------

### Reloading and Merging Fine-tuned Llama 2 Model (Python)

Source: https://github.com/mlabonne/llm-course/blob/main/Fine_tune_Llama_2_in_Google_Colab.ipynb

This snippet reloads the base Llama 2 model in FP16 precision and then merges the previously trained LoRA weights into it using `PeftModel.from_pretrained` and `model.merge_and_unload()`. It also reloads the tokenizer to ensure it's correctly configured for the merged model, preparing it for deployment or further use.

```Python
# Reload model in FP16 and merge it with LoRA weights
base_model = AutoModelForCausalLM.from_pretrained(
    model_name,
    low_cpu_mem_usage=True,
    return_dict=True,
    torch_dtype=torch.float16,
    device_map=device_map,
)
model = PeftModel.from_pretrained(base_model, new_model)
model = model.merge_and_unload()

# Reload tokenizer to save it
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = "right"
```

--------------------------------

### Emptying GPU VRAM (Python)

Source: https://github.com/mlabonne/llm-course/blob/main/Fine_tune_Llama_2_in_Google_Colab.ipynb

This snippet explicitly deletes the model, pipeline, and trainer objects from memory and then calls Python's garbage collector twice. This is a common practice in environments like Google Colab to free up GPU VRAM, especially before loading a new model or performing memory-intensive operations.

```Python
# Empty VRAM
del model
del pipe
del trainer
import gc
gc.collect()
gc.collect()
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.