### Install Unsloth and Dependencies for LLM Fine-tuning

Source: https://github.com/rasahq/notebooks/blob/main/cmd_gen_finetuning.ipynb

This script installs Unsloth, PyTorch, TRL, PEFT, Accelerate, BitsAndBytes, and Hugging Face Hub CLI, along with specific version requirements. It also uninstalls 'torch-xla' which is often pre-installed on GCP runtimes but not needed for GPU-only setups.

```Shell
%%sh
# install unsloth and other dependencies
pip install torch==2.2.2
pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
pip install --no-deps "xformers<=0.0.26" "trl<0.9.0" peft accelerate bitsandbytes huggingface_hub[cli]
# remove tpu-only package that is installed by default on gcp runtimes, even when only using gpu
pip uninstall torch-xla -y
```

--------------------------------

### Load and Format Datasets for LLM Training with TRL

Source: https://github.com/rasahq/notebooks/blob/main/cmd_gen_finetuning.ipynb

This code loads training and validation datasets from JSONL files using the `datasets` library. It then configures a tokenizer with a chat template (e.g., Llama-3) to format conversations, and applies a custom function to format prompts for each example, ensuring compatibility with TRL's SFTTrainer.

```python
import datasets
from trl.extras.dataset_formatting import get_formatting_func_from_dataset
from unsloth.chat_templates import get_chat_template

# Load the training and evaluation datasets from JSONL files on disk
train_dataset = datasets.load_dataset(
    "json", data_files={"train": "train.jsonl"}, split="train"
)
eval_dataset = datasets.load_dataset(
    "json", data_files={"eval": "val.jsonl"}, split="eval"
)

# Uncomment the following line if you want to test prompt formatting on a single example from the eval dataset
# print(get_formatting_func_from_dataset(train_dataset, tokenizer)(eval_dataset[0]))

# Get a tokenizer with a chat template to format conversations according to a specified structure
tokenizer = get_chat_template(
    tokenizer,
    chat_template="llama-3",  # Specifies the chat template format (options: zephyr, chatml, mistral, llama, alpaca, etc.)
    mapping={"role": "from", "content": "value", "user": "human", "assistant": "gpt"},  # Maps dataset roles and messages to expected format
)

# Define a function to format prompts for each example in the dataset
def formatting_prompts_func(examples):
    # Extract conversation messages from each example
    convos = examples["messages"]
    
    # Apply the chat template to each conversation without tokenizing or adding generation prompts
    texts = [tokenizer.apply_chat_template(convo, tokenize=False, add_generation_prompt=False) for convo in convos]
    
    # Return the formatted texts in a new dictionary key
    return {"text": texts}

# Apply the formatting function to both the training and evaluation datasets in batches
train_dataset = train_dataset.map(formatting_prompts_func, batched=True)
eval_dataset = eval_dataset.map(formatting_prompts_func, batched=True)
```

--------------------------------

### Install Python Libraries for Fine-tuning Metric Visualization

Source: https://github.com/rasahq/notebooks/blob/main/cmd_gen_finetuning.ipynb

This snippet installs the necessary Python libraries, `pandas` and `matplotlib`, required for visualizing the fine-tuning metrics. These libraries are used for data manipulation and plotting, enabling the analysis of training and validation losses over time. This step ensures that the environment has the tools needed for diagnostic plotting.

```python
!pip install pandas matplotlib
```

--------------------------------

### Export Fine-Tuned Model to Cloud Storage

Source: https://github.com/rasahq/notebooks/blob/main/cmd_gen_finetuning.ipynb

Provides commands to export a fine-tuned model directory to cloud object storage (Amazon S3 or Google Cloud Storage). It assumes the bucket already exists, the CLI tool for the cloud provider is installed, and the user has authenticated with sufficient permissions to write to the bucket. Users need to uncomment and update environment variables with their own values.

```bash
%%sh
export LOCAL_MODEL_PATH="./finetuned_model"

# if using amazon
# export S3_MODEL_URI="s3://CHANGEME" # update with your value
# aws s3 cp "${LOCAL_MODEL_PATH}" "${S3_MODEL_URI}" --recursive

# if using google
# export GCS_MODEL_URI="gs://CHANGEME" # update with your value
# gsutil cp -r "${LOCAL_MODEL_PATH}" "${GCS_MODEL_URI}"
```

--------------------------------

### Download Base LLM from Hugging Face Hub

Source: https://github.com/rasahq/notebooks/blob/main/cmd_gen_finetuning.ipynb

This script downloads a specified base LLM from Hugging Face Hub using the `huggingface-cli`. It requires a Hugging Face API token and the model name to be set as environment variables. The model is downloaded to a local directory named `./base_model`.

```Shell
%%sh
# TODO: update with your values
export HUGGINGFACE_TOKEN="CHANGEME"
export BASE_MODEL="meta-llama/Meta-Llama-3.1-8B-Instruct"

# download model
huggingface-cli download "${BASE_MODEL}" \
    --token "${HUGGINGFACE_TOKEN}" \
    --local-dir "./base_model"
```

--------------------------------

### Run Ad Hoc Inference with Fine-Tuned Unsloth Model

Source: https://github.com/rasahq/notebooks/blob/main/cmd_gen_finetuning.ipynb

Loads a fine-tuned Unsloth model from disk, enables inference optimizations, and demonstrates how to run ad hoc inference on individual inputs. It highlights the use of TRL conversational format for inputs and streams model outputs as they are generated.

```python
from transformers import TextStreamer
from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained("./finetuned_model")
FastLanguageModel.for_inference(model)  # enable inference optimizations
streamer = TextStreamer(tokenizer)  # stream model outputs as they are generated

# the content to include in the input prompt
# by default, a value from the validation dataset as example
content = eval_dataset["text"][0]

# apply prompt template and tokenize
input_ids = tokenizer.apply_chat_template(
    [{"role": "user", "content": content}],  # in the TRL conversational format
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt",
).to("cuda")

# generate model output from user input
_ = model.generate(
    input_ids=input_ids,
    streamer=streamer,  # remove streamer if you want whole output at end
    max_new_tokens=64,  # set the limit on how many tokens are generated
    do_sample=False  # disable random sampling for deterministic outputs
)
```

--------------------------------

### Execute Supervised Fine-tuning with SFTTrainer in Python

Source: https://github.com/rasahq/notebooks/blob/main/cmd_gen_finetuning.ipynb

This snippet initiates the supervised fine-tuning process by calling the `train()` method on the pre-configured `SFTTrainer` instance. It executes the training loop, applying the defined training arguments and datasets to fine-tune the model. The process duration is noted to be approximately 12 minutes for a specific dataset size and hardware configuration.

```python
# run fine-tuning
finetune_metrics = trainer.train()
```

--------------------------------

### Configure SFTTrainer Arguments for Supervised Fine-tuning in Python

Source: https://github.com/rasahq/notebooks/blob/main/cmd_gen_finetuning.ipynb

This snippet configures the `TrainingArguments` for the `SFTTrainer`, setting parameters such as batch size, learning rate, and evaluation strategy. It leverages `unsloth` to determine optimal data types (fp16/bf16) and initializes the `SFTTrainer` with the model, tokenizer, datasets, and configured arguments. The settings aim to balance performance and resource usage, with recommendations for adjustment based on OOM errors or desired training duration.

```python
import torch
from transformers import TrainingArguments
from trl import SFTTrainer
from unsloth import is_bfloat16_supported

# configure training args
args = TrainingArguments(
    ###### training
    seed = random_seed,
    per_device_train_batch_size = 2,
    gradient_accumulation_steps = 4,
    warmup_steps = 5,
    #max_steps = 60,
    num_train_epochs = 2,
    learning_rate = 2e-4,
    lr_scheduler_type = "linear",
    optim = "adamw_8bit",
    weight_decay = 0.01,
    ###### datatypes
    fp16 = not is_bfloat16_supported(),
    bf16 = is_bfloat16_supported(),
    ###### evaluation
    eval_strategy = "steps",
    eval_steps = 50,
    per_device_eval_batch_size = 8,
    ###### outputs
    logging_steps = 30,
    output_dir = "outputs",
)

# setup trainer
trainer = SFTTrainer(
    model = model,
    tokenizer = tokenizer,
    train_dataset = train_dataset,
    eval_dataset = eval_dataset,
    max_seq_length = max_seq_length,
    args = args,
)
```

--------------------------------

### Load and Quantize Base LLM with Unsloth

Source: https://github.com/rasahq/notebooks/blob/main/cmd_gen_finetuning.ipynb

This Python code loads a pre-trained language model from a local directory (`./base_model`) using Unsloth's `FastLanguageModel`. It configures 4-bit quantization via `BitsAndBytesConfig` to reduce GPU memory usage, and sets a maximum sequence length and random seed.

```Python
from transformers import BitsAndBytesConfig
from unsloth import FastLanguageModel

max_seq_length = 2048
random_seed = 42


# configure quantization method for base model
quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
)

# load quantized model and tokenizer from disk
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="./base_model",
    max_seq_length=max_seq_length,
    quantization_config=quantization_config,
)
```

--------------------------------

### Configure Base Model for PEFT with Unsloth LoRA

Source: https://github.com/rasahq/notebooks/blob/main/cmd_gen_finetuning.ipynb

This snippet demonstrates how to adapt a `FastLanguageModel` for Parameter Efficient Fine-Tuning (PEFT) using the Low-Rank Adaptation (LoRA) method via the `unsloth` library. It configures the model by specifying target modules, LoRA parameters like `r` and `lora_alpha`, and other settings to reduce GPU memory usage during fine-tuning.

```python
from unsloth import FastLanguageModel

# adapt model for peft
model = FastLanguageModel.get_peft_model(
    model,
    r=16,
    target_modules=[
        "q_proj",
        "k_proj",
        "v_proj",
        "o_proj",
        "gate_proj",
        "up_proj",
        "down_proj"
    ],
    lora_alpha=16,
    lora_dropout=0,
    bias="none",
    use_gradient_checkpointing="unsloth",
    random_state=random_seed,
    use_rslora=False,
    loftq_config=None,
)
```

--------------------------------

### Save Merged Fine-tuned Model to Disk in Python

Source: https://github.com/rasahq/notebooks/blob/main/cmd_gen_finetuning.ipynb

This snippet saves the fine-tuned model and its adapters to disk after the training process is complete. It merges the base model with the fine-tuned adapters into a 16-bit format, ensuring compatibility with model serving libraries like vLLM. The output path for the merged model is specified as './finetuned_model'.

```python
# save model to disk in 16-bit
model.save_pretrained_merged("./finetuned_model", tokenizer, save_method="merged_16bit")
```

--------------------------------

### Visualize Training and Validation Losses in Python

Source: https://github.com/rasahq/notebooks/blob/main/cmd_gen_finetuning.ipynb

This snippet visualizes the training and validation losses collected during the fine-tuning process using `pandas` and `matplotlib`. It plots the `eval_loss` and `loss` (training loss) against the training step number, providing insights into model convergence and potential issues like underfitting or overfitting. The plot helps diagnose the effectiveness of the fine-tuning configuration.

```python
import pandas as pd
import matplotlib.pyplot as plt

# plot step against train and val losses
fig, ax = plt.subplots()
log_history = pd.DataFrame(trainer.state.log_history)
eval_loss = log_history[["step", "eval_loss"]].dropna().plot(x="step", ax=ax)
train_loss = log_history[["step", "loss"]].dropna().plot(x="step", ax=ax)
fig.show()
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.