### Example Package Versions Output

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch05/11_qwen3/standalone-qwen3-moe.ipynb

This is an example output showing the versions of the required packages. Actual versions may vary based on your installation.

```text
Output:
huggingface_hub version: 0.35.0
tokenizers version: 0.22.1
torch version: 2.7.1+cu128


```

--------------------------------

### Example Package Versions Output

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch05/11_qwen3/standalone-qwen3-moe-plus-kvcache.ipynb

This is an example output showing the versions of the required packages after running the version check script. It confirms that the necessary libraries are installed and accessible.

```text
Output:
huggingface_hub version: 0.34.3
tokenizers version: 0.21.4
torch version: 2.7.1+cu128


```

--------------------------------

### Install Transformers Library

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch05/02_alternative_weight_loading/weight-loading-hf-transformers.ipynb

Install the transformers library using pip. This is a prerequisite for the subsequent code examples.

```python
# pip install transformers
```

--------------------------------

### Example Model Configuration Output

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch05/17_gemma4/standalone-gemma4-plus-kvcache.ipynb

An example of the resulting configuration dictionary after selecting the model and device settings.

```python
Result:
{'model': 'E2B',
 'instruct': True,
 'emb_dim': 1536,
 'n_layers': 35,
 'n_heads': 8,
 'n_kv_heads': 1,
 'global_head_dim': 512,
 'num_kv_shared_layers': 20,
 'device': 'mps',
 'dtype': 'torch.float32'}
```

--------------------------------

### Install a specific package with uv

Source: https://github.com/rasbt/llms-from-scratch/blob/main/setup/01_optional-python-setup-preferences/README.md

Installs a single Python package, 'packaging' in this example, using the uv package manager within the activated virtual environment.

```bash
uv pip install packaging
```

--------------------------------

### Install llms-from-scratch Package

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch05/07_gpt_to_llama/README.md

Install the necessary packages for using the llms-from-scratch library and loading model weights.

```bash
pip install llms_from_scratch blobfile
```

--------------------------------

### Install llms-from-scratch from PyPI

Source: https://github.com/rasbt/llms-from-scratch/blob/main/pkg/llms_from_scratch/README.md

Use this command to install the package from the Python Package Index.

```bash
pip install llms-from-scratch
```

--------------------------------

### Install and Check Package Versions

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch05/07_gpt_to_llama/converting-gpt-to-llama2.ipynb

Installs and prints the versions of necessary packages like huggingface_hub, sentencepiece, and torch. Ensure these packages are installed before proceeding.

```python
from importlib.metadata import version

pkgs = [
    "huggingface_hub",  # to download pretrained weights
    "sentencepiece",    # to implement the tokenizer
    "torch",            # to implement the model
]
for p in pkgs:
    print(f"{p} version: {version(p)}")
```

--------------------------------

### Install tokenizers library

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch05/14_ch05_with_other_llms/ch05-qwen3.ipynb

Installs the 'tokenizers' library, which is required for implementing the Qwen3 tokenizer.

```python
# pip install tokenizers
```

--------------------------------

### Initialize and Run Causal Attention Module

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch03/03_understanding-buffers/understanding-buffers.ipynb

Demonstrates initializing the `CausalAttentionWithoutBuffers` module and running it with example tensor data. Ensure PyTorch is installed.

```python
torch.manual_seed(123)

inputs = torch.tensor(
  [[0.43, 0.15, 0.89], # Your     (x^1)
   [0.55, 0.87, 0.66], # journey  (x^2)
   [0.57, 0.85, 0.64], # starts   (x^3)
   [0.22, 0.58, 0.33], # with     (x^4)
   [0.77, 0.25, 0.10], # one      (x^5)
   [0.05, 0.80, 0.55]] # step     (x^6)
)

batch = torch.stack((inputs, inputs), dim=0)
context_length = batch.shape[1]
d_in = inputs.shape[1]
d_out = 2

ca_without_buffer = CausalAttentionWithoutBuffers(d_in, d_out, context_length, 0.0)

with torch.no_grad():
    context_vecs = ca_without_buffer(batch)

print(context_vecs)
```

--------------------------------

### Install uv on macOS and Linux (wget)

Source: https://github.com/rasbt/llms-from-scratch/blob/main/setup/01_optional-python-setup-preferences/native-uv.md

Installs uv using a wget command. Ensure you have wget installed on your system.

```bash
wget -qO- https://astral.sh/uv/install.sh | sh
```

--------------------------------

### Install uv on Windows

Source: https://github.com/rasbt/llms-from-scratch/blob/main/setup/01_optional-python-setup-preferences/native-uv.md

Installs uv using a PowerShell command. This command downloads and executes the installation script.

```powershell
powershell -c "irm https://astral.sh/uv/install.ps1 | more"
```

--------------------------------

### Execute Miniforge Installer Script

Source: https://github.com/rasbt/llms-from-scratch/blob/main/setup/01_optional-python-setup-preferences/README.md

Run the Miniforge installer script from your terminal. Ensure the path points to your downloaded installer.

```bash
sh ~/Desktop/Miniforge3-MacOSX-arm64.sh
```

--------------------------------

### Install llms-from-scratch using uv

Source: https://github.com/rasbt/llms-from-scratch/blob/main/pkg/llms_from_scratch/README.md

Use this command with uv to install the package from the Python Package Index.

```bash
uv add llms-from-scratch
```

--------------------------------

### Editable Install from GitHub using uv

Source: https://github.com/rasbt/llms-from-scratch/blob/main/pkg/llms_from_scratch/README.md

Use this command with uv for an editable installation from a local GitHub repository.

```bash
uv add --editable . --dev
```

--------------------------------

### Generate Text with Simple Decoding (Painting Example)

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch05/14_ch05_with_other_llms/ch05-qwen3.ipynb

Generates text starting with the word 'Painting' using a simple decoding strategy. Ensure the model is on the correct device and in evaluation mode. The tokenizer and QWEN3_CONFIG must be accessible.

```python
token_ids = generate_text_simple(
    model=model,
    idx=text_to_token_ids("Painting", tokenizer).to(inference_device),
    max_new_tokens=25,
    context_size=QWEN3_CONFIG["train_context_length"]
)

print("Output text:\n", token_ids_to_text(token_ids, tokenizer))
```

--------------------------------

### Install Project Requirements

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch07/04_preference-tuning-with-dpo/dpo-from-scratch.ipynb

Installs necessary packages for the project from a requirements file hosted online. Run this before proceeding with the rest of the notebook.

```python
# !pip install -r https://raw.githubusercontent.com/rasbt/LLMs-from-scratch/main/requirements.txt
```

--------------------------------

### Install PyTorch Package

Source: https://github.com/rasbt/llms-from-scratch/blob/main/setup/02_installing-python-libraries/README.md

Install the PyTorch library using pip. For more complex installations involving specific CPU/GPU configurations, consult the official PyTorch website or the book's appendix.

```bash
pip install torch
```

--------------------------------

### Install packages from requirements.txt with uv

Source: https://github.com/rasbt/llms-from-scratch/blob/main/setup/01_optional-python-setup-preferences/README.md

Installs all Python packages listed in the 'requirements.txt' file using uv. Ensure the file is in your current directory.

```bash
uv pip install -r requirements.txt
```

--------------------------------

### Install Requirements

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch04/03_kv-cache/README.md

Installs the necessary Python packages from a requirements file hosted on GitHub.

```bash
pip install -r https://raw.githubusercontent.com/rasbt/LLMs-from-scratch/refs/heads/main/requirements.txt
```

--------------------------------

### Example Instruction Dataset Entry

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch07/05_dataset-generation/llama3-ollama.ipynb

This is an example of a single entry in an instruction dataset, featuring an 'instruction' and its corresponding 'output'.

```json
{
    "instruction": "What is the atomic number of helium?",
    "output": "The atomic number of helium is 2."
}
```

--------------------------------

### Install PyTorch for CPU

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch03/02_bonus_efficient-multihead-attention/mha-implementations.ipynb

Install PyTorch on a CPU machine using pip. This is a prerequisite for using FlexAttention.

```bash
pip install torch torchvision torchaudio
```

--------------------------------

### Install Tiktoken Library

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch02/01_main-chapter-code/ch02.ipynb

Install the tiktoken library using pip. This is a prerequisite for using the BPE tokenizer.

```python
# pip install tiktoken
```

--------------------------------

### Install Extra Requirements

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch02/02_bonus_bytepair-encoder/compare-bpe-tiktoken.ipynb

Uncomment and run this cell to install additional package requirements for the bonus notebook.

```python
# pip install -r requirements-extra.txt
```

--------------------------------

### Clone Repository and Install Dependencies

Source: https://github.com/rasbt/llms-from-scratch/blob/main/setup/README.md

Execute these commands in the terminal to set up your environment. These steps only need to be performed once as the environments are persistent.

```bash
git clone https://github.com/rasbt/LLMs-from-scratch.git
cd LLMs-from-scratch
pip install -r requirements.txt
```

--------------------------------

### Get tiktoken Version

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch02/02_bonus_bytepair-encoder/compare-bpe-tiktoken.ipynb

Prints the installed version of the tiktoken library. Ensure tiktoken is installed before running.

```python
from importlib.metadata import version

print("tiktoken version:", version("tiktoken"))
```

--------------------------------

### Run environment check script

Source: https://github.com/rasbt/llms-from-scratch/blob/main/setup/01_optional-python-setup-preferences/README.md

Executes a Python script to verify that the environment setup is correct and all necessary packages are installed. This script is located in the setup directory.

```bash
python setup/02_installing-python-libraries/python_environment_check.py
```

--------------------------------

### Download and Initialize Tokenizer

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch05/16_qwen3.5/qwen3.5.ipynb

Demonstrates downloading the tokenizer JSON file from Hugging Face Hub and initializing the Qwen3_5Tokenizer with specific chat template options.

```python
tokenizer_file_path = "Qwen3.5-0.8B/tokenizer.json"

hf_hub_download(
    repo_id=repo_id,
    filename="tokenizer.json",
    local_dir=local_dir,
)

tokenizer = Qwen3_5Tokenizer(
    tokenizer_file_path=tokenizer_file_path,
    repo_id=repo_id,
    apply_chat_template=True,
    add_generation_prompt=True,
    add_thinking=True,
)
```

--------------------------------

### Import and Get Tiktoken Version

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch02/01_main-chapter-code/ch02.ipynb

Import the tiktoken library and print its version. This confirms the library is installed and accessible.

```python
import importlib
import tiktoken

print("tiktoken version:", importlib.metadata.version("tiktoken"))
```

--------------------------------

### Download and Initialize Tokenizer

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch05/16_qwen3.5/qwen3.5-plus-kv-cache.ipynb

Downloads the tokenizer file from Hugging Face Hub and initializes the Qwen3.5Tokenizer. Ensure `repo_id` and `local_dir` are correctly set.

```python
from huggingface_hub import hf_hub_download

tokenizer_file_path = "Qwen3.5-0.8B/tokenizer.json"

hf_hub_download(
    repo_id=repo_id,
    filename="tokenizer.json",
    local_dir=local_dir,
)

tokenizer = Qwen3_5Tokenizer(
    tokenizer_file_path=tokenizer_file_path,
    repo_id=repo_id,
    apply_chat_template=True,
    add_generation_prompt=True,
    add_thinking=True,
)
```

--------------------------------

### Create and Process Batch Example

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch04/01_main-chapter-code/ch04.ipynb

Creates a batch of random examples and passes them through a linear layer followed by a ReLU activation. The output is then printed.

```python
batch_example = torch.randn(2, 5)
layer = nn.Sequential(nn.Linear(5, 6), nn.ReLU())
out = layer(batch_example)
print(out)
```

--------------------------------

### List and Print Package Versions

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch05/07_gpt_to_llama/converting-llama2-to-llama3.ipynb

Imports specified packages and prints their installed versions. This is useful for verifying the environment setup.

```python
from importlib.metadata import version

pkgs = [
    "blobfile",         # to download pretrained weights
    "huggingface_hub",  # to download pretrained weights
    "matplotlib",       # to visualize RoPE with different base frequencies
    "tiktoken",         # to implement the tokenizer
    "torch",            # to implement the model
]
for p in pkgs:
    print(f"{p} version: {version(p)}")
```

--------------------------------

### Run UI with Custom GPT-2 Weights

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch05/06_user_interface/README.md

Execute this command to start the UI server using custom GPT-2 weights generated in chapter 5. Ensure the chapter 5 notebook has been run first.

```bash
chainlit run app_own.py
```

--------------------------------

### Check Package Versions

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch07/04_preference-tuning-with-dpo/create-preference-data-ollama.ipynb

This Python script checks and prints the installed versions of specified packages, useful for verifying the environment setup.

```python
from importlib.metadata import version

pkgs = ["tqdm",    # Progress bar
        ]

for p in pkgs:
    print(f"{p} version: {version(p)}")
```

--------------------------------

### Initialize and Train LLM

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch05/14_ch05_with_other_llms/ch05-qwen3.ipynb

Sets up the LLM, optimizer, and initiates the training process for a specified number of epochs. Ensure `device`, `train_loader`, and `val_loader` are properly configured before execution.

```python
# Note:
# Uncomment the following code to calculate the execution time
# import time
# start_time = time.time()

torch.manual_seed(123)
model = Qwen3Model(QWEN3_CONFIG)
model.to(device)
optimizer = torch.optim.AdamW(model.parameters(), lr=0.0004, weight_decay=0.1)

num_epochs = 40
train_losses, val_losses, tokens_seen = train_model_simple(
    model, train_loader, val_loader, optimizer, device,
    num_epochs=num_epochs, eval_freq=5, eval_iter=5,
    start_context="Every effort moves you", tokenizer=tokenizer
)

# Note:
# Uncomment the following code to show the execution time
# end_time = time.time()
# execution_time_minutes = (end_time - start_time) / 60

```

--------------------------------

### Get Hugging Face Transformers Version

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch02/02_bonus_bytepair-encoder/compare-bpe-tiktoken.ipynb

Prints the installed version of the Hugging Face transformers library. This is useful for compatibility checks.

```python
import transformers

transformers.__version__
```

--------------------------------

### Run Simple Pretraining Script

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch05/03_bonus_pretraining_on_gutenberg/README.md

Initiates the pretraining process using the prepared dataset. Default values for arguments are shown for illustration. Consider using 'tee' to log output.

```bash
python pretraining_simple.py \
  --data_dir "gutenberg_preprocessed" \
  --n_epochs 1 \
  --batch_size 4 \
  --output_dir model_checkpoints
```

```bash
python -u pretraining_simple.py | tee log.txt
```

--------------------------------

### GPT-2 Configuration and Tokenizer Setup

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch05/01_main-chapter-code/exercise-solutions.ipynb

This snippet defines the configuration for a 124M GPT-2 model and initializes the tiktoken tokenizer. Ensure tiktoken and torch are installed.

```python
import tiktoken
import torch
from previous_chapters import GPTModel


GPT_CONFIG_124M = {
    "vocab_size": 50257,   # Vocabulary size
    "context_length": 256, # Shortened context length (orig: 1024)
    "emb_dim": 768,        # Embedding dimension
    "n_heads": 12,         # Number of attention heads
    "n_layers": 12,        # Number of layers
    "drop_rate": 0.1,      # Dropout rate
    "qkv_bias": False      # Query-key-value bias
}


tokenizer = tiktoken.get_encoding("gpt2")
```

--------------------------------

### Temperature Scaling and Top-K Sampling Setup

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch05/01_main-chapter-code/ch05.ipynb

This snippet illustrates the setup for temperature scaling and top-k sampling. It defines a vocabulary and its inverse, and calculates probabilities from logits using softmax. The `torch.argmax` is used to select the token with the highest probability.

```python
vocab = { 
    "closer": 0,
    "every": 1, 
    "effort": 2, 
    "forward": 3,
    "inches": 4,
    "moves": 5, 
    "pizza": 6,
    "toward": 7,
    "you": 8,
}

inverse_vocab = {v: k for k, v in vocab.items()}

# Suppose input is "every effort moves you", and the LLM
# returns the following logits for the next token:
next_token_logits = torch.tensor(
    [4.51, 0.89, -1.90, 6.75, 1.63, -1.62, -1.89, 6.28, 1.79]
)

probas = torch.softmax(next_token_logits, dim=0)
next_token_id = torch.argmax(probas).item()
```

--------------------------------

### Download Progress Indicator

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch05/15_tiny-aya/standalone-tiny-aya.ipynb

Displays the download progress for model weights. This snippet shows an example of an incomplete download status, indicating that the download has started but not yet finished.

```text
Result:
Downloading (incomplete total...): 0.00B [00:00, ?B/s]
```

--------------------------------

### Run UI with Original GPT-2 Weights

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch05/06_user_interface/README.md

Execute this command to start the UI server using the original GPT-2 weights from OpenAI. A browser tab should open automatically.

```bash
chainlit run app_orig.py
```

--------------------------------

### Manual Loss Calculation with torch.gather

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch07/04_preference-tuning-with-dpo/dpo-from-scratch.ipynb

This example shows how to manually compute the negative log-likelihood loss using `torch.gather`. It's useful for understanding the mechanics behind PyTorch's `cross_entropy` function. Ensure PyTorch is installed.

```python
import torch
import torch.nn.functional as F

# Sample data
logits = torch.tensor(
    [[2.0, 1.0, 0.1],
     [0.5, 2.5, 0.3]])  # Shape: (2, 3)
targets = torch.tensor([0, 2])  # Shape: (2,)


# Manual loss using torch.gather
log_softmax_logits = F.log_softmax(logits, dim=1)  # Shape: (2, 3)
selected_log_probs = torch.gather(
    input=log_softmax_logits,
    dim=1,
    index=targets.unsqueeze(1), # Shape 2, 1
).squeeze(1)  # Shape: (2,)
manual_loss = -selected_log_probs.mean()  # Averaging over the batch

```

--------------------------------

### Run First Token Finetuning Experiment

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch06/01_main-chapter-code/exercise-solutions.ipynb

Execute the experiment finetuning the first output token using the provided command-line argument. This can result in a substantial decrease in test accuracy.

```bash
python additional-experiments.py --trainable_token first
```

--------------------------------

### Download Tokenizer and Initialize Qwen3Tokenizer

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch05/14_ch05_with_other_llms/ch05-qwen3.ipynb

Downloads the tokenizer.json file from Hugging Face Hub and initializes the Qwen3Tokenizer. Requires 'pathlib' and 'huggingface_hub'.

```python
from pathlib import Path
from huggingface_hub import hf_hub_download

repo_id = "Qwen/Qwen3-0.6B-Base"
tokenizer_file_path = "tokenizer.json"
local_dir = "."

hf_hub_download(
    repo_id=repo_id,
    filename="tokenizer.json",
    local_dir=local_dir,
)

tokenizer = Qwen3Tokenizer(
    tokenizer_file_path=tokenizer_file_path,
    repo_id=repo_id,
    apply_chat_template=False,
    add_generation_prompt=False,
    add_thinking=False
)
```

--------------------------------

### Install uv on macOS and Linux

Source: https://github.com/rasbt/llms-from-scratch/blob/main/setup/01_optional-python-setup-preferences/native-uv.md

Installs uv using a curl command. Ensure you have curl installed on your system.

```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
```

--------------------------------

### Navigate to Project Directory

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch05/03_bonus_pretraining_on_gutenberg/README.md

Change the current working directory to the '03_bonus_pretraining_on_gutenberg' folder. This is a prerequisite for cloning the 'gutenberg' repository locally.

```bash
cd ch05/03_bonus_pretraining_on_gutenberg
```

--------------------------------

### DPO Training Loop Setup and Execution

Source: https://context7.com/rasbt/llms-from-scratch/llms.txt

Sets up policy and reference models, optimizer, and runs the DPO training loop. Loads pretrained weights and computes loss using the `compute_dpo_loss` and `compute_logprobs` functions.

```python
# DPO training setup
policy_model = GPTModel(BASE_CONFIG)
policy_model.load_state_dict(torch.load("gpt2-medium355M-sft.pth"))

reference_model = GPTModel(BASE_CONFIG)
reference_model.load_state_dict(torch.load("gpt2-medium355M-sft.pth"))
reference_model.eval()

optimizer = torch.optim.AdamW(policy_model.parameters(), lr=5e-6, weight_decay=0.01)

# Training loop
for batch in train_loader:
    optimizer.zero_grad()
    policy_chosen_logprobs = compute_logprobs(policy_model(batch["chosen"]), batch["chosen"], batch["chosen_mask"])
    policy_rejected_logprobs = compute_logprobs(policy_model(batch["rejected"]), batch["rejected"], batch["rejected_mask"])

    with torch.no_grad():
        ref_chosen_logprobs = compute_logprobs(reference_model(batch["chosen"]), batch["chosen"], batch["chosen_mask"])
        ref_rejected_logprobs = compute_logprobs(reference_model(batch["rejected"]), batch["rejected"], batch["rejected_mask"])

    loss, chosen_rewards, rejected_rewards = compute_dpo_loss(
        policy_chosen_logprobs, policy_rejected_logprobs,
        ref_chosen_logprobs, ref_rejected_logprobs, beta=0.1
    )
    loss.backward()
    optimizer.step()
```

--------------------------------

### Install Dependencies with Pip

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch05/11_qwen3/qwen3-chat-interface/README.md

Install the chainlit package and its dependencies from a requirements file using pip. Ensure you have pip installed.

```bash
pip install -r requirements-extra.txt
```

--------------------------------

### Install Dependencies with Uv

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch05/11_qwen3/qwen3-chat-interface/README.md

Install the chainlit package and its dependencies from a requirements file using uv. This is an alternative to pip for faster installation.

```bash
uv pip install -r requirements-extra.txt
```

--------------------------------

### Install JupyterLab and Watermark using Conda

Source: https://github.com/rasbt/llms-from-scratch/blob/main/setup/01_optional-python-setup-preferences/README.md

Install JupyterLab and the watermark package using the Conda package installer within your activated environment.

```bash
conda install jupyterlab watermark
```

--------------------------------

### DPO Training Setup and Execution

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch07/04_preference-tuning-with-dpo/dpo-from-scratch.ipynb

This Python code configures and runs the DPO training process. It initializes the optimizer, sets training parameters like epochs and beta, and calls the training function. Note that DPO is sensitive to hyperparameters and prone to collapse, requiring careful tuning.

```python
import time

start_time = time.time()

torch.manual_seed(123)


optimizer = torch.optim.AdamW(policy_model.parameters(), lr=5e-6, weight_decay=0.01)

num_epochs = 1
tracking = train_model_dpo_simple(
    policy_model=policy_model,
    reference_model=reference_model,
    train_loader=train_loader,
    val_loader=val_loader,
    optimizer=optimizer,
    num_epochs=num_epochs,
    beta=0.1, # value between 0.1 and 0.5
    eval_freq=5,
    eval_iter=5,
    start_context=format_input(val_data[2]),
    tokenizer=tokenizer
)

end_time = time.time()
execution_time_minutes = (end_time - start_time) / 60
print(f"Training completed in {execution_time_minutes:.2f} minutes.")
```

--------------------------------

### Check Installed Package Versions

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch07/03_model-evaluation/llm-instruction-eval-openai.ipynb

Verifies the installed versions of essential Python packages like 'openai' and 'tqdm'. Ensure these packages are installed before proceeding.

```python
from importlib.metadata import version

pkgs = ["openai",  # OpenAI API
        "tqdm",    # Progress bar
        ]

for p in pkgs:
    print(f"{p} version: {version(p)}")
```

--------------------------------

### Download and Initialize Qwen3 Tokenizer

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch05/11_qwen3/standalone-qwen3.ipynb

Downloads tokenizer.json from Hugging Face Hub and initializes the Qwen3Tokenizer. Configuration varies based on model type (reasoning, instruct, or base).

```python
if USE_REASONING_MODEL:
    tokenizer_file_path = f"Qwen3-{CHOOSE_MODEL}/tokenizer.json"
else:
    tokenizer_file_path = f"Qwen3-{CHOOSE_MODEL}-Base/tokenizer.json"

hf_hub_download(
    repo_id=repo_id,
    filename="tokenizer.json",
    local_dir=local_dir,
)

if USE_REASONING_MODEL or USE_INSTRUCT_MODEL:
    tokenizer = Qwen3Tokenizer(
        tokenizer_file_path=tokenizer_file_path,
        repo_id=repo_id,
        apply_chat_template=True,
        add_generation_prompt=True,
        add_thinking=USE_REASONING_MODEL
    )

else:
    tokenizer = Qwen3Tokenizer(
        tokenizer_file_path=tokenizer_file_path,
        repo_id=repo_id,
        apply_chat_template=False,
        add_generation_prompt=False,
        add_thinking=False
    )
```

--------------------------------

### Qwen3 Configuration and Initialization

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch05/14_ch05_with_other_llms/ch05-qwen3.ipynb

Defines the configuration dictionary for a 0.6B parameter Qwen3 model and initializes the model with specified parameters and a random seed. This setup is suitable for training or evaluation.

```python
#######################
### Initialize Qwen3 
#######################

# 0.6B model
QWEN3_CONFIG = {
    "vocab_size": 151_936,           # Vocabulary size
    "context_length": 40_960,        # Context length that was used to train the model
    "emb_dim": 1024,                 # Embedding dimension
    "n_heads": 16,                   # Number of attention heads
    "n_layers": 28,                  # Number of layers
    "hidden_dim": 3072,              # Size of the intermediate dimension in FeedForward
    "head_dim": 128,                 # Size of the heads in GQA
    "qk_norm": True,                 # Whether to normalize queries and keys in GQA
    "n_kv_groups": 8,                # Key-Value groups for grouped-query attention
    "rope_base": 1_000_000.0,        # The base in RoPE's "theta"
    "dtype": torch.bfloat16,         # Lower-precision dtype to reduce memory usage
}

QWEN3_CONFIG["train_context_length"] = 256  # It's a small dataset, and we also want to keep memory usage reasonable

torch.manual_seed(123)
model = Qwen3Model(QWEN3_CONFIG)
model.eval();
```

--------------------------------

### Calculating Cross-Entropy Loss with Three Examples

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch07/01_main-chapter-code/ch07.ipynb

Calculates the cross-entropy loss for three training examples, including a new third example. Requires `torch`.

```python
logits_2 = torch.tensor(
    [[-1.0, 1.0],
     [-0.5, 1.5],
     [-0.5, 1.5]]  # New 3rd training example
)
targets_2 = torch.tensor([0, 1, 1])

loss_2 = torch.nn.functional.cross_entropy(logits_2, targets_2)
print(loss_2)
```

--------------------------------

### Install packages from URL with pip

Source: https://github.com/rasbt/llms-from-scratch/blob/main/setup/01_optional-python-setup-preferences/README.md

Installs Python packages from a requirements file hosted online using pip. This is an alternative to using uv for remote installations.

```bash
pip install -U -r https://raw.githubusercontent.com/rasbt/LLMs-from-scratch/refs/heads/main/requirements.txt
```

--------------------------------

### Install uv package manager

Source: https://github.com/rasbt/llms-from-scratch/blob/main/setup/01_optional-python-setup-preferences/README.md

Installs the uv package manager, a fast Python package installer and resolver. This is the first step in setting up the virtual environment.

```bash
pip install uv
```

--------------------------------

### Example: Generate Text from Prompt

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch05/07_gpt_to_llama/standalone-llama32.ipynb

Demonstrates generating text using the `generate` function with a specific prompt, context size, and sampling parameters (top_k=1, temperature=0.0 for deterministic output). It also measures generation time and optionally reports GPU memory usage.

```python
import time


PROMPT = "What do llamas eat?"

torch.manual_seed(123)

start = time.time()

token_ids = generate(
    model=model,
    idx=text_to_token_ids(PROMPT, chat_tokenizer).to(device),
    max_new_tokens=150,
    context_size=LLAMA32_CONFIG["context_length"],
    top_k=1,
    temperature=0.
)

print(f"Time: {time.time() - start:.2f} sec")

if torch.cuda.is_available():
    max_mem_bytes = torch.cuda.max_memory_allocated()
    max_mem_gb = max_mem_bytes / (1024 ** 3)
    print(f"Max memory allocated: {max_mem_gb:.2f} GB")

output_text = token_ids_to_text(token_ids, tokenizer)
```

--------------------------------

### Install packages from a URL with uv

Source: https://github.com/rasbt/llms-from-scratch/blob/main/setup/01_optional-python-setup-preferences/README.md

Installs Python packages directly from a requirements file hosted online using uv. This is useful for installing dependencies from a GitHub repository.

```bash
uv pip install -r https://raw.githubusercontent.com/rasbt/LLMs-from-scratch/refs/heads/main/requirements.txt
```

--------------------------------

### From-Scratch Implementation of Qwen3 (0.6B and 30B-A3B)

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch05/README.md

A from-scratch implementation of Qwen3 models, including the 0.6B parameter version and the Mixture-of-Experts 30B-A3B variant. Code to load pretrained weights for base, reasoning, and coding models is provided.

```python
import torch

# Placeholder for Qwen3 model implementation
class Qwen3Model(torch.nn.Module):
    def __init__(self):
        super().__init__()
        # Model layers would be defined here
        pass

    def forward(self, x):
        # Forward pass logic
        return x

# Example of loading weights (conceptual)
# model = Qwen3Model()
# state_dict = torch.load("path/to/qwen3_weights.pth")
# model.load_state_dict(state_dict)

```

--------------------------------

### Run Full Model Finetuning Experiment

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch06/01_main-chapter-code/exercise-solutions.ipynb

Execute the finetuning experiment for the entire model using the specified command-line argument. This approach can yield improved test accuracy.

```bash
python additional-experiments.py --trainable_layers all
```

--------------------------------

### Print installed package versions

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch05/14_ch05_with_other_llms/ch05-llama32.ipynb

Prints the installed versions of essential Python packages including matplotlib, numpy, tiktoken, and torch. Ensure these packages are installed and up-to-date for compatibility.

```python
from importlib.metadata import version

pkgs = [
    "matplotlib", 
    "numpy", 
    "tiktoken", 
    "torch",
       ]
for p in pkgs:
    print(f"{p} version: {version(p)}")
```

--------------------------------

### Initialize Qwen3 Model

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch05/11_qwen3/standalone-qwen3-moe.ipynb

Initializes the Qwen3 model with the specified configuration and places it on the determined device. A manual seed is set for reproducibility.

```python
torch.manual_seed(123)

with device:
    model = Qwen3Model(QWEN3_CONFIG)

#model.to(device)
```

--------------------------------

### Install Dependencies from URL with uv pip

Source: https://github.com/rasbt/llms-from-scratch/blob/main/setup/01_optional-python-setup-preferences/native-uv.md

Install or upgrade dependencies from a remote requirements file URL using `uv pip install`. This method does not record dependencies in a lock file.

```bash
uv pip install -U -r https://raw.githubusercontent.com/rasbt/LLMs-from-scratch/refs/heads/main/requirements.txt
```

--------------------------------

### Install Hugging Face Transformers and Dependencies

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch05/17_gemma4/tests/test_e2b/gemma4-e2b-transformers-ref.ipynb

Install the necessary libraries for using Hugging Face Transformers, including PyTorch, transformers, and accelerate. Run this code if you haven't already installed these packages.

```python
# pip install -U torch transformers accelerate

```

--------------------------------

### Download and Initialize Tokenizer

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch05/11_qwen3/standalone-qwen3-plus-kvcache.ipynb

Downloads the tokenizer file from Hugging Face Hub and initializes the Qwen3Tokenizer. Conditional logic is used to set tokenizer parameters based on model type and reasoning capabilities.

```python
from huggingface_hub import hf_hub_download

# Assume USE_REASONING_MODEL, CHOOSE_MODEL, repo_id, and local_dir are defined elsewhere

if USE_REASONING_MODEL:
    tokenizer_file_path = f"Qwen3-{CHOOSE_MODEL}/tokenizer.json"
else:
    tokenizer_file_path = f"Qwen3-{CHOOSE_MODEL}-Base/tokenizer.json"

hf_hub_download(
    repo_id=repo_id,
    filename="tokenizer.json",
    local_dir=local_dir,
)

if USE_REASONING_MODEL or USE_INSTRUCT_MODEL:
    tokenizer = Qwen3Tokenizer(
        tokenizer_file_path=tokenizer_file_path,
        repo_id=repo_id,
        apply_chat_template=True,
        add_generation_prompt=True,
        add_thinking=USE_REASONING_MODEL
    )

else:
    tokenizer = Qwen3Tokenizer(
        tokenizer_file_path=tokenizer_file_path,
        repo_id=repo_id,
        apply_chat_template=False,
        add_generation_prompt=False,
        add_thinking=False
    )
```

--------------------------------

### Setup PyTorch Environment and Embeddings

Source: https://github.com/rasbt/llms-from-scratch/blob/main/ch03/02_bonus_efficient-multihead-attention/mha-implementations.ipynb

Initializes PyTorch, determines the optimal device (MPS, CUDA, or CPU), and creates random embeddings for demonstration. Ensure PyTorch version is at least 2.5 for FlexAttention.

```python
import torch

torch.manual_seed(123)
if torch.backends.mps.is_available():
    device = torch.device("mps")   # Apple Silicon GPU (Metal)
elif torch.cuda.is_available():
    device = torch.device("cuda")  # NVIDIA GPU
else:
    device = torch.device("cpu")   # CPU fallback

print(f"Using device: {device}")
print(f"PyTorch version: {torch.__version__}")

batch_size = 8
context_len = 1024
embed_dim = 768
embeddings = torch.randn((batch_size, context_len, embed_dim), device=device)
```