### Setup and Download GuppyLM

Source: https://github.com/arman-bd/guppylm/blob/main/use_guppylm.ipynb

Installs required dependencies and downloads the model files from Hugging Face to the local directory.

```python
# Setup + Download
!pip install -q torch tokenizers huggingface_hub
import os, shutil
if os.path.exists('/content/guppy'): shutil.rmtree('/content/guppy')
os.makedirs('/content/guppy'); os.chdir('/content/guppy')

from huggingface_hub import snapshot_download
snapshot_download(repo_id='arman-bd/guppylm-9M', local_dir='.')
print('Model downloaded.')
```

--------------------------------

### Install and Chat Locally with GuppyLM

Source: https://github.com/arman-bd/guppylm/blob/main/README.md

Install necessary packages and run the GuppyLM chat interface from the command line. This mode allows for interactive conversations.

```bash
pip install torch tokenizers
python -m guppylm chat
```

--------------------------------

### Setup Working Directory

Source: https://github.com/arman-bd/guppylm/blob/main/train_guppylm.ipynb

Cleans up any previous run's directory and creates a fresh working directory for the GuppyLM project. This ensures a clean state for training.

```python
import os, shutil

# Start fresh — removes stale files from previous runs
if os.path.exists('/content/guppy'):
    shutil.rmtree('/content/guppy')
os.makedirs('/content/guppy')
os.chdir('/content/guppy')
print(f'Working dir: {os.getcwd()}')
```

--------------------------------

### Install Dependencies and Check Environment

Source: https://github.com/arman-bd/guppylm/blob/main/train_guppylm.ipynb

Installs necessary Python libraries and checks for PyTorch and CUDA availability. Ensure you have a compatible environment for GPU acceleration.

```python
!pip install -q torch tokenizers tqdm numpy datasets huggingface_hub

import torch
print(f'PyTorch {torch.__version__}')
print(f'CUDA: {torch.cuda.is_available()}')
if torch.cuda.is_available():
    print(f'GPU: {torch.cuda.get_device_name(0)}')
```

--------------------------------

### Install GuppyLM dependencies

Source: https://context7.com/arman-bd/guppylm/llms.txt

Install the required Python packages to run GuppyLM.

```bash
pip install torch tokenizers tqdm numpy datasets
```

--------------------------------

### Run interactive chat

Source: https://context7.com/arman-bd/guppylm/llms.txt

Start an interactive session to chat with the Guppy model.

```bash
python -m guppylm chat
```

--------------------------------

### Example GuppyLM Conversations

Source: https://github.com/arman-bd/guppylm/blob/main/tools/dataset_card.md

Illustrates typical input-output pairs for the GuppyLM model, showcasing its character and response style.

```text
Input:  are you hungry
Output: yes. always yes. i will swim to the top right now.

Input:  what do you think about politics
Output: i don't know what politics is. is it wet.

Input:  goodnight guppy
Output: night. i'll rest near the bottom. maybe tomorrow there will be pellets.
```

--------------------------------

### GuppyLM Conversation Example

Source: https://github.com/arman-bd/guppylm/blob/main/tools/model_card.md

Sample interaction showing the model's fish-themed personality and response style.

```text
You> hi guppy
Guppy> hi there. i just found a nice spot near the rock. the temperature feels nice.

You> are you hungry
Guppy> yes. always yes. i will swim to the top right now. i promise to eat all of it.

You> what is the meaning of life
Guppy> food. the answer is always food.

You> tell me a joke
Guppy> what did the fish say when it hit the wall. dam.

You> goodnight guppy
Guppy> ok sleep time. i was following a bubble but now i'll stop. goodnight tank. goodnight water.
```

--------------------------------

### Chat with GuppyLM using a Single Prompt

Source: https://github.com/arman-bd/guppylm/blob/main/README.md

Invoke the GuppyLM chat interface with a specific prompt to get a single response. This is useful for quick queries without starting an interactive session.

```bash
python -m guppylm chat --prompt "tell me a joke"
```

--------------------------------

### Get DataLoader for Custom Training Loop

Source: https://context7.com/arman-bd/guppylm/llms.txt

Retrieves DataLoaders for training and evaluation datasets using specified data paths, tokenizer, sequence length, and batch size. Supports shuffling for the training loader.

```python
train_loader = get_dataloader("data/train.jsonl", "data/tokenizer.json",
                               mc.max_seq_len, tc.batch_size, shuffle=True)
eval_loader = get_dataloader("data/eval.jsonl", "data/tokenizer.json",
                              mc.max_seq_len, tc.batch_size, shuffle=False)
```

--------------------------------

### Prepare Dataset and Train Tokenizer

Source: https://github.com/arman-bd/guppylm/blob/main/train_guppylm.ipynb

Downloads the dataset, formats it into ChatML JSONL files, and trains a BPE tokenizer.

```python
import json, os
from datasets import load_dataset
from tokenizers import Tokenizer, models, trainers, pre_tokenizers, decoders, processors

# ── Download from HuggingFace ──
HF_DATASET = 'arman-bd/guppylm-60k-generic'
ds = load_dataset(HF_DATASET)
print(f'Downloaded: {len(ds["train"]):,} train, {len(ds["test"]):,} test samples')

# ── Format into ChatML and save as JSONL ──
os.makedirs('data', exist_ok=True)
texts = []

for split, path in [('train', 'data/train.jsonl'), ('test', 'data/eval.jsonl')]:
    with open(path, 'w') as f:
        for row in ds[split]:
            text = (
                f'<|im_start|>user\n{row["input"]}<|im_end|>\n'
                f'<|im_start|>assistant\n{row["output"]}<|im_end|>'
            )
            f.write(json.dumps({'text': text, 'category': row['category']}) + '\n')
            texts.append(text)
    print(f'  {path}: {len(ds[split]):,} samples')

# ── Train BPE tokenizer on the data ──
tokenizer = Tokenizer(models.BPE())
tokenizer.pre_tokenizer = pre_tokenizers.ByteLevel(add_prefix_space=False)
tokenizer.decoder = decoders.ByteLevel()

trainer = trainers.BpeTrainer(
    vocab_size=4096,
    special_tokens=['<pad>', '<|im_start|>', '<|im_end|>'],
    min_frequency=2,
    show_progress=True,
)
tokenizer.train_from_iterator(texts, trainer)
tokenizer.post_processor = processors.ByteLevel(trim_offsets=False)
tokenizer.save('data/tokenizer.json')
print(f'  Tokenizer: {tokenizer.get_vocab_size()} tokens')

# ── Preview ──
with open('data/train.jsonl') as f:
    sample = json.loads(f.readline())
print(f'\nSample ({sample["category"]}):\n{sample["text"]}')
```

--------------------------------

### Initialize GuppyLM Model and Training Configuration

Source: https://context7.com/arman-bd/guppylm/llms.txt

Initializes the GuppyLM model, training configuration, and device (GPU or CPU). Sets up the optimizer with specified learning rate and weight decay.

```python
import torch
from guppylm import GuppyConfig, TrainConfig, GuppyLM
from guppylm.dataset import get_dataloader
from guppylm.train import get_lr, evaluate

# Initialize
mc = GuppyConfig()
tc = TrainConfig(max_steps=5000, eval_interval=100)
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

model = GuppyLM(mc).to(device)
optimizer = torch.optim.AdamW(
    model.parameters(),
    lr=tc.learning_rate,
    weight_decay=tc.weight_decay,
    betas=(0.9, 0.95)
)
```

--------------------------------

### GuppyDataset and DataLoader

Source: https://context7.com/arman-bd/guppylm/llms.txt

Demonstrates creating a PyTorch Dataset from JSONL files and setting up a DataLoader with automatic padding for training. Requires guppylm.dataset and torch.

```python
from guppylm.dataset import GuppyDataset, get_dataloader

# Create dataset from JSONL file
dataset = GuppyDataset(
    path="data/train.jsonl",
    tokenizer_path="data/tokenizer.json",
    max_len=128
)

print(f"Dataset size: {len(dataset)}")
# Output: Dataset size: 57000

# Get a single sample (input, target shifted by 1)
x, y = dataset[0]
print(f"Input shape: {x.shape}")   # torch.Size([N]) where N varies
print(f"Target shape: {y.shape}")  # torch.Size([N])

# Create DataLoader with automatic padding
train_loader = get_dataloader(
    path="data/train.jsonl",
    tokenizer_path="data/tokenizer.json",
    max_len=128,
    batch_size=32,
    shuffle=True
)

```

--------------------------------

### Load Model and Tokenizer

Source: https://github.com/arman-bd/guppylm/blob/main/docs/index.html

Fetches the tokenizer and model files, initializes the inference session, and verifies the tokenizer output.

```javascript
// ── Load model + tokenizer ────────────────────────────────────────── async function load(ort) { const detail = document.getElementById("load-detail"); detail.textContent = "loading tokenizer..."; const tokResp = await fetch(`${MODEL_BASE}/tokenizer.json`); const tokJson = await tokResp.json(); tokenizer = buildTokenizer(tokJson); detail.textContent = "loading model (~10 MB)..."; const modelResp = await fetch(`${MODEL_BASE}/model.onnx`); const modelBuf = await modelResp.arrayBuffer(); session = await ort.InferenceSession.create(modelBuf, { executionProviders: ["wasm"], }); document.getElementById("loading").classList.add("hidden"); document.getElementById("samples").classList.remove("hidden"); document.getElementById("input-row").classList.remove("hidden"); document.getElementById("status").textContent = "ready — running locally in your browser"; document.getElementById("user-input").focus(); // Verify tokenizer const testIds = tokenizer.encode("<|im_start|>user\nhi guppy<|im_end|>\n<|im_start|>assistant\n"); const expected = [1, 88, 64, 779, 278, 2, 64, 1, 89, 64]; const match = testIds.length === expected.length && testIds.every((v, i) => v === expected[i]); console.log("Tokenizer check:", match ? "PASS" : "FAIL", testIds, "expected:", expected); }
```

--------------------------------

### Load GuppyLM Dataset with Datasets Library

Source: https://github.com/arman-bd/guppylm/blob/main/tools/dataset_card.md

Demonstrates how to load the GuppyLM dataset using the Hugging Face datasets library and print the first entry.

```python
from datasets import load_dataset
ds = load_dataset("arman-bd/guppylm-60k-generic")
print(ds["train"][0])
# {'input': 'hi guppy', 'output': 'hello. the water is nice today.', 'category': 'greeting'}
```

--------------------------------

### Prepare training data

Source: https://context7.com/arman-bd/guppylm/llms.txt

Generate synthetic conversation data and train the BPE tokenizer.

```bash
python -m guppylm prepare
```

--------------------------------

### Initialize ONNX Runtime

Source: https://github.com/arman-bd/guppylm/blob/main/docs/index.html

Loads the ONNX Runtime Web library and initializes the inference session.

```javascript
// ── ONNX Runtime ──────────────────────────────────────────────────── import("https://cdn.jsdelivr.net/npm/onnxruntime-web@1.21.0/dist/ort.min.mjs").then(async (ort) => { window.ort = ort; ort.env.wasm.wasmPaths = "https://cdn.jsdelivr.net/npm/onnxruntime-web@1.21.0/dist/"; await load(ort); });
```

--------------------------------

### Download pre-trained GuppyLM model

Source: https://context7.com/arman-bd/guppylm/llms.txt

Download the pre-trained model weights and tokenizer from HuggingFace.

```bash
python -m guppylm download
```

--------------------------------

### Handle Sample Prompts

Source: https://github.com/arman-bd/guppylm/blob/main/docs/index.html

Triggers a model response when a sample prompt pill is clicked.

```javascript
// ── Sample prompt pills ───────────────────────────────────────────── document.querySelectorAll(".pill").forEach(btn => { btn.addEventListener("click", () => { document.getElementById("user-input").value = btn.dataset.msg; send(); }); });
```

--------------------------------

### Default Training Configuration

Source: https://context7.com/arman-bd/guppylm/llms.txt

Instantiates the default training configuration and prints its parameters. Useful for understanding default hyperparameters.

```python
train_config = TrainConfig()
print(f"Batch size: {train_config.batch_size}")          # 32
print(f"Learning rate: {train_config.learning_rate}")    # 3e-4
print(f"Min LR: {train_config.min_lr}")                  # 3e-5
print(f"Weight decay: {train_config.weight_decay}")      # 0.1
print(f"Warmup steps: {train_config.warmup_steps}")      # 200
print(f"Max steps: {train_config.max_steps}")            # 10000
print(f"Eval interval: {train_config.eval_interval}")    # 200
print(f"Save interval: {train_config.save_interval}")    # 500
print(f"Gradient clip: {train_config.grad_clip}")        # 1.0
print(f"Device: {train_config.device}")                  # "auto"
print(f"Seed: {train_config.seed}")                      # 42
print(f"Data dir: {train_config.data_dir}")              # "data"
print(f"Output dir: {train_config.output_dir}")          # "checkpoints"
```

--------------------------------

### Format Sample for Training with Special Tokens

Source: https://context7.com/arman-bd/guppylm/llms.txt

Formats a generated sample into the required chat format by adding special tokens for user and assistant roles. This is necessary before feeding data into the model for training.

```python
# Format for training (adds special tokens)
formatted = format_sample(sample)
print(formatted)
# <|im_start|>user
# tell me a joke<|im_end|>
# <|im_start|>assistant
# what did the fish say when it hit the wall. dam.<|im_end|>
```

--------------------------------

### Run GuppyLM Inference

Source: https://github.com/arman-bd/guppylm/blob/main/tools/model_card.md

Initialize the inference engine with a model checkpoint and tokenizer, then generate a chat response.

```python
from inference import GuppyInference

engine = GuppyInference('checkpoints/best_model.pt', 'data/tokenizer.json')
r = engine.chat_completion([{'role': 'user', 'content': 'hi guppy'}])
print(r['choices'][0]['message']['content'])
# hi there. i just found a nice spot near the rock.
```

--------------------------------

### Train GuppyLM model

Source: https://context7.com/arman-bd/guppylm/llms.txt

Train the model from scratch using the prepared data and tokenizer.

```bash
python -m guppylm train
```

--------------------------------

### Integrate Model for Browser Inference

Source: https://context7.com/arman-bd/guppylm/llms.txt

Demonstrates loading an ONNX model in the browser and performing token generation. Requires the onnxruntime-web library.

```html
<!-- Load ONNX Runtime Web -->
<script src="https://cdn.jsdelivr.net/npm/onnxruntime-web/dist/ort.min.js"></script>

<script>
// Load model and tokenizer
const session = await ort.InferenceSession.create('model.onnx');
const tokenizer = await fetch('tokenizer.json').then(r => r.json());

// Encode prompt
function encode(text) {
    // Use tokenizer vocabulary for encoding
    // Returns array of token IDs
}

// Generate response
async function generate(prompt, maxTokens = 64) {
    let inputIds = encode(prompt);

    for (let i = 0; i < maxTokens; i++) {
        const tensor = new ort.Tensor('int64', BigInt64Array.from(inputIds.map(BigInt)), [1, inputIds.length]);
        const results = await session.run({ input_ids: tensor });
        const logits = results.logits.data;

        // Sample next token (with temperature and top-k)
        const nextToken = sampleToken(logits, inputIds.length - 1);

        if (nextToken === 2) break; // <|im_end|>
        inputIds.push(nextToken);
    }

    return decode(inputIds);
}

// Chat with Guppy
const response = await generate("<|im_start|>user\nhi guppy<|im_end|>\n<|im_start|>assistant\n");
console.log(response);
</script>
```

--------------------------------

### GuppyLM Model Initialization and Forward Pass

Source: https://context7.com/arman-bd/guppylm/llms.txt

Initializes the GuppyLM model with a configuration and performs a forward pass for both training (with loss) and inference (without loss). Requires torch and guppylm libraries.

```python
import torch
from guppylm import GuppyConfig, GuppyLM

# Initialize model
config = GuppyConfig()
model = GuppyLM(config)

# Check parameter count
print(model.param_summary())
# Output: GuppyLM: 8,728,576 params (8.7M)

# Forward pass with loss computation
input_ids = torch.randint(0, config.vocab_size, (2, 32))  # batch=2, seq=32
targets = torch.randint(0, config.vocab_size, (2, 32))

logits, loss = model(input_ids, targets)
print(f"Logits shape: {logits.shape}")  # torch.Size([2, 32, 4096])
print(f"Loss: {loss.item():.4f}")       # ~8.3 (untrained)

# Forward pass without loss (inference)
logits, _ = model(input_ids)
print(f"Logits shape: {logits.shape}")  # torch.Size([2, 32, 4096])
```

--------------------------------

### Load and Inspect GuppyLM Dataset

Source: https://github.com/arman-bd/guppylm/blob/main/README.md

Load the GuppyLM dataset from HuggingFace using the `datasets` library and print the first training sample. This demonstrates how to access the dataset for use.

```python
from datasets import load_dataset
ds = load_dataset("arman-bd/guppylm-60k-generic")
print(ds["train"][0])
```

--------------------------------

### Verify Model Architecture

Source: https://github.com/arman-bd/guppylm/blob/main/train_guppylm.ipynb

Performs a sanity check by building the model, printing parameter counts, and executing a dummy forward pass.

```python
from config import GuppyConfig
from model import GuppyLM
import torch

config = GuppyConfig()
model = GuppyLM(config)
print(model.param_summary())
print(f'  Layers: {config.n_layers}, Heads: {config.n_heads}, FFN: {config.ffn_hidden}')
print(f'  Vocab: {config.vocab_size}, Max seq: {config.max_seq_len}')

# Dummy forward pass
x = torch.randint(0, config.vocab_size, (2, 32))
logits, _ = model(x)
print(f'  Forward pass: {x.shape} -> {logits.shape} OK')
del model
```

--------------------------------

### Build ByteLevel BPE Tokenizer

Source: https://github.com/arman-bd/guppylm/blob/main/docs/index.html

Constructs a tokenizer based on HuggingFace ByteLevel BPE logic.

```javascript
// ── ByteLevel BPE tokenizer (JS port of HuggingFace tokenizers) ───── function buildTokenizer(json) { const vocab = json.model.vocab; const merges = json.model.merges; const added = {}; for (const t of json.added_tokens) added[t.content] = t.id; const id2token = {}; for (const [tok, id] of Object.entries(vocab)) id2token[id] = tok; for (const [tok, id] of Object.entries(added)) id2token[id] = tok; // Byte-to-unicode mapping (matches HuggingFace bytes_to_unicode) const byte2char = {}; const char2byte = {}; const ranges = [ [33, 126], [161, 172], [174, 255], ]; const direct = new Set(); for (const [lo, hi] of ranges) for (let b = lo; b <= hi; b++) direct.add(b); let n = 0; for (let b = 0; b < 256; b++) { if (direct.has(b)) { byte2char[b] = String.fromCharCode(b); } else { byte2char[b] = String.fromCharCode(256 + n); n++; } } for (const [b, c] of Object.entries(byte2char)) char2byte[c] = parseInt(b); // Merge rank lookup const mergeRank = {}; for (let i = 0; i < merges.length; i++) { const key = Array.isArray(merges[i]) ? merges[i].join(" ") : merges[i]; mergeRank[key] = i; } function bytesToTokenStr(bytes) { return Array.from(bytes).map(b => byte2char[b]).join(""); } function tokenStrToBytes(s) { return Uint8Array.from([...s].map(c => char2byte[c] ?? c.charCodeAt(0))); } function bpe(word) { if (word.length <= 1) return word; let pieces = word.slice(); while (pieces.length > 1) { let bestRank = Infinity, bestIdx = -1; for (let i = 0; i < pieces.length - 1; i++) { const pair = pieces[i] + " " + pieces[i + 1]; const rank = mergeRank[pair]; if (rank !== undefined && rank < bestRank) { bestRank = rank; bestIdx
```

--------------------------------

### Run Interactive Chat CLI

Source: https://github.com/arman-bd/guppylm/blob/main/train_guppylm.ipynb

Main entry point for running the GuppyLM chat interface via command line arguments.

```python
def main():
    import argparse
    p = argparse.ArgumentParser(description="Chat with Guppy")
    p.add_argument("--checkpoint", default="checkpoints/best_model.pt")
    p.add_argument("--tokenizer", default="data/tokenizer.json")
    p.add_argument("--device", default="cpu")
    args = p.parse_args()

    engine = GuppyInference(args.checkpoint, args.tokenizer, args.device)
    print("\nGuppy Chat (type 'quit' to exit)")
    msgs = []
    while True:
        inp = input("\nYou> ").strip()
        if inp.lower() in ("quit", "exit", "q"):
            break
        msgs.append({"role": "user", "content": inp})
        result = engine.chat_completion(msgs)
        msg = result["choices"][0]["message"]
        if msg.get("content"):
            print(f"Guppy> {msg['content']}")
        msgs.append(msg)


if __name__ == "__main__":
    main()
```

--------------------------------

### Initialize GuppyLM Model

Source: https://github.com/arman-bd/guppylm/blob/main/train_guppylm.ipynb

Loads the model state dictionary and initializes the evaluation mode.

```python
filtered = {k: v for k, v in state_dict.items() if k in self.model.state_dict()}
self.model.load_state_dict(filtered)
self.model.eval()

total, _ = self.model.param_count()
print(f"GuppyLM loaded: {total/1e6:.1f}M params")
```

--------------------------------

### Load GuppyLM Dataset from HuggingFace Hub

Source: https://context7.com/arman-bd/guppylm/llms.txt

Loads the official GuppyLM training dataset (60k generic samples) from the HuggingFace Hub. Provides access to training and testing splits, along with sample data and categories.

```python
from datasets import load_dataset

# Load the dataset
ds = load_dataset("arman-bd/guppylm-60k-generic")

print(f"Train samples: {len(ds['train'])}")  # 57000
print(f"Test samples: {len(ds['test'])}")    # 3000

# Inspect a sample
sample = ds["train"][0]
print(sample)
# {
#     'input': 'hi guppy',
#     'output': 'hello. the water is nice today.',
#     'category': 'greeting'
# }

# Get all categories
categories = set(ds["train"]["category"])
print(f"Categories: {len(categories)}")  # 60
print(sorted(categories)[:10])
# ['about', 'age', 'algae', 'bored', 'breathing', 'bubbles', 'bye', 'cat', 'children', 'colors']
```

--------------------------------

### GuppyLM Inference Initialization

Source: https://github.com/arman-bd/guppylm/blob/main/train_guppylm.ipynb

Initializes the GuppyInference class by loading the model checkpoint, tokenizer, and configuration. Handles different checkpoint formats and config loading strategies. Use this to set up GuppyLM for inference.

```python
%%writefile inference.py
"""GuppyLM inference — simple chat."""

import json
import time
import uuid

import torch
from tokenizers import Tokenizer

from config import GuppyConfig
from model import GuppyLM


class GuppyInference:
    def __init__(self, checkpoint_path, tokenizer_path, device="cpu"):
        self.device = torch.device(device)
        self.tokenizer = Tokenizer.from_file(tokenizer_path)

        import os
        ckpt = torch.load(checkpoint_path, map_location=self.device, weights_only=False)

        # Load config.json from same directory as the model file
        config_dir = os.path.dirname(os.path.abspath(checkpoint_path))
        config_path = os.path.join(config_dir, "config.json")

        # Extract state_dict — handle both legacy and standard formats
        if isinstance(ckpt, dict) and "model_state_dict" in ckpt:
            state_dict = ckpt["model_state_dict"]
        else:
            state_dict = ckpt

        # Load config — try config.json first, fall back to embedded config
        if os.path.exists(config_path):
            with open(config_path) as f:
                cfg = json.load(f)
            # Support both HF standard keys and our own keys
            self.config = GuppyConfig(
                vocab_size=cfg.get("vocab_size", 4096),
                max_seq_len=cfg.get("max_position_embeddings", cfg.get("max_seq_len", 128)),
                d_model=cfg.get("hidden_size", cfg.get("d_model", 384)),
                n_layers=cfg.get("num_hidden_layers", cfg.get("n_layers", 6)),
                n_heads=cfg.get("num_attention_heads", cfg.get("n_heads", 6)),
                ffn_hidden=cfg.get("intermediate_size", cfg.get("ffn_hidden", 768)),
                dropout=cfg.get("hidden_dropout_prob", cfg.get("dropout", 0.1)),
                pad_id=cfg.get("pad_token_id", cfg.get("pad_id", 0)),
                bos_id=cfg.get("bos_token_id", cfg.get("bos_id", 1)),
                eos_id=cfg.get("eos_token_id", cfg.get("eos_id", 2)),
            )
        elif isinstance(ckpt, dict) and "config" in ckpt:
            valid_fields = {f.name for f in GuppyConfig.__dataclass_fields__.values()}
            self.config = GuppyConfig(**{k: v for k, v in ckpt["config"].items() if k in valid_fields})
        else:
            print("Warning: No config found, using defaults")
            self.config = GuppyConfig()

        self.model = GuppyLM(self.config).to(self.device)

```

--------------------------------

### Train the Model

Source: https://github.com/arman-bd/guppylm/blob/main/train_guppylm.ipynb

Executes the training process for the model.

```python
from train import train
train()
```

--------------------------------

### Topics Hint Popup Functionality

Source: https://github.com/arman-bd/guppylm/blob/main/docs/index.html

Implements a topics hint popup that displays suggested conversation starters. Clicking a topic fills the input field and sends the message.

```javascript
const TOPICS = [
  ["greetings", "hi guppy"],
  ["food", "are you hungry"],
  ["temperature", "is it cold today"],
  ["water", "how is the water"],
  ["bubbles", "do you like bubbles"],
  ["light", "is it bright"],
  ["night", "goodnight guppy"],
  ["dreams", "what do you dream about"],
  ["jokes", "tell me a joke"],
  ["love", "do you love me"],
  ["loneliness", "do you get lonely"],
  ["meaning of life", "what is the meaning of life"],
  ["cats", "the cat is looking at you"],
  ["rain", "it is raining outside"],
  ["music", "do you like music"],
  ["feelings", "are you happy"],
  ["swimming", "do you like swimming"],
  ["plants", "tell me about your plants"],
  ["colors", "what is your favorite color"],
  ["memory", "do you remember me"],
  ["glass tapping", "sorry i tapped the glass"],
  ["reflection", "who is that fish in the glass"],
  ["size", "are you small"],
  ["scared", "are you scared"],
  ["bored", "are you bored"],
  ["intelligence", "are you smart"],
  ["friends", "do you have friends"],
  ["sleep", "are you tired"],
  ["time", "what time is it"],
  ["outside", "what is outside the tank"],
  ["noise", "it is loud today"],
  ["visitors", "someone new is here"],
  ["singing", "can you sing"],
  ["age", "how old are you"],
  ["name", "what is your name"],
  ["weather", "how is the weather"],
  ["future", "what will you do tomorrow"],
  ["past", "what did you do today"],
  ["filter", "do you hear the filter"],
  ["algae", "is there algae"],
  ["snails", "are there snails in the tank"],
  ["health", "are you feeling ok"],
  ["excited", "are you excited"],
  ["curious", "what are you curious about"],
  ["taste", "what does the water taste like"],
  ["breathing", "how do you breathe"],
  ["TV", "do you watch tv"],
  ["children", "do you like children"],
  ["seasons", "what season is it"],
  ["fear", "what are you afraid of"],
];
document.getElementById("hint-btn").addEventListener("click", () => {
  const overlay = document.getElementById("topics-overlay");
  overlay.innerHTML = "";
  overlay.classList.remove("hidden");
  const box = document.createElement("div");
  box.id = "topics-box";
  box.innerHTML = "<h3>&#x1F420; Guppy knows 60 topics — click one to ask</h3>";
  const grid = document.createElement("div");
  grid.className = "topics";
  for (const [label, msg] of TOPICS) {
    const btn = document.createElement("button");
    btn.className = "topic";
    btn.textContent = label;
    btn.addEventListener("click", () => {
      overlay.classList.add("hidden");
      document.getElementById("user-input").value = msg;
      send();
    });
    grid.appendChild(btn);
  }
  box.appendChild(grid);
  const close = document.createElement("button");
  close.className = "close-btn";
  close.textContent = "close";
  close.addEventListener("click", () => overlay.classList.add("hidden"));
  box.appendChild(close);
  overlay.appendChild(box);
  overlay.addEventListener("click", (e) => {
    if (e.target === overlay)
      overlay.classList.add("hidden");
  });
});
```

--------------------------------

### Test Model Inference

Source: https://github.com/arman-bd/guppylm/blob/main/train_guppylm.ipynb

Initializes the inference engine and runs a series of single-turn chat tests across various topics.

```python
from inference import GuppyInference
import torch

engine = GuppyInference(
    'checkpoints/best_model.pt', 'data/tokenizer.json',
    device='cuda' if torch.cuda.is_available() else 'cpu'
)

def chat(prompt):
    r = engine.chat_completion([{'role': 'user', 'content': prompt}], max_tokens=64)
    return r['choices'][0]['message'].get('content', '').strip()

# Test across different topics
tests = [
    ('hi guppy',                      'greeting'),
    ('are you hungry',                'food'),
    ('it is really hot today',        'temperature'),
    ('how is the water',              'water'),
    ('do you like bubbles',           'bubbles'),
    ('what is the internet',          'confused'),
    ('do you get lonely',             'lonely'),
    ('the cat is looking at you',     'cat'),
    ('tell me a joke',                'joke'),
    ('what do you dream about',       'dreams'),
    ('do you love me',                'love'),
    ('what is the meaning of life',   'meaning'),
    ('sorry i tapped the glass',      'glass_tap'),
    ('it is raining outside',         'rain'),
    ('goodnight guppy',               'night'),
]

print(f'{"Topic":<12s}  {"You":<35s}  Guppy')
print('=' * 100)
for prompt, topic in tests:
    reply = chat(prompt)
    print(f'{topic:<12s}  {prompt:<35s}  {reply[:128]}')
```

--------------------------------

### Generate Full Dataset with GuppyLM

Source: https://context7.com/arman-bd/guppylm/llms.txt

Generates a full dataset for training and evaluation using GuppyLM's data generation utility. Specifies the total number of samples and the ratio for the evaluation set.

```python
from guppylm.generate_data import generate_dataset, format_sample

# Generate full dataset
generate_dataset(
    n_samples=60000,   # Total samples to generate
    eval_ratio=0.05    # 5% for evaluation
)
# Creates: data/train.jsonl, data/eval.jsonl
#          data/train_openai.jsonl, data/eval_openai.jsonl
```

--------------------------------

### Export Model to HuggingFace

Source: https://github.com/arman-bd/guppylm/blob/main/train_guppylm.ipynb

Exports the model into PyTorch and quantized ONNX formats, then prepares the directory for HuggingFace upload.

```python
!pip install -q onnx onnxruntime onnxscript

from huggingface_hub import HfApi, login
import torch, json, os, shutil
from config import GuppyConfig
from model import GuppyLM

HF_TOKEN = os.environ.get('HF_TOKEN', '')  # Or paste your token here
HF_REPO = os.environ.get('HF_REPO', 'arman-bd/guppylm-9M')  # Or change this

# Load checkpoint
ckpt = torch.load('checkpoints/best_model.pt', map_location='cpu', weights_only=False)
cfg = ckpt['config']
os.makedirs('hf_export', exist_ok=True)

# ── PyTorch format ──
torch.save(ckpt['model_state_dict'], 'hf_export/pytorch_model.bin')

with open('hf_export/config.json', 'w') as f:
    json.dump({
        'model_type': 'guppylm',
        'architectures': ['GuppyLM'],
        'vocab_size': cfg['vocab_size'],
        'max_position_embeddings': cfg['max_seq_len'],
        'hidden_size': cfg['d_model'],
        'num_hidden_layers': cfg['n_layers'],
        'num_attention_heads': cfg['n_heads'],
        'intermediate_size': cfg['ffn_hidden'],
        'hidden_dropout_prob': cfg.get('dropout', 0.1),
        'pad_token_id': cfg['pad_id'],
        'bos_token_id': cfg['bos_id'],
        'eos_token_id': cfg['eos_id'],
    }, f, indent=2)

shutil.copy('data/tokenizer.json', 'hf_export/tokenizer.json')
print(f'pytorch_model.bin: {os.path.getsize("hf_export/pytorch_model.bin")/1e6:.1f} MB')

# ── ONNX format (quantized uint8) ──
valid_fields = {f.name for f in GuppyConfig.__dataclass_fields__.values()}
config = GuppyConfig(**{k: v for k, v in cfg.items() if k in valid_fields})
model = GuppyLM(config)
model.load_state_dict(ckpt['model_state_dict'])
model.eval()

dummy = torch.randint(0, config.vocab_size, (1, 32))
fp32_path = 'hf_export/model_fp32.onnx'
torch.onnx.export(
    model, (dummy,), fp32_path,
    input_names=['input_ids'], output_names=['logits'],
    dynamic_axes={'input_ids': {0: 'batch', 1: 'seq_len'},
                  'logits': {0: 'batch', 1: 'seq_len'}},
    opset_version=17,
)

from onnxruntime.quantization import quantize_dynamic, QuantType
quantize_dynamic(fp32_path, 'hf_export/model.onnx', weight_type=QuantType.QUInt8)
os.remove(fp32_path)
print(f'model.onnx:       {os.path.getsize("hf_export/model.onnx")/1e6:.1f} MB (uint8)')
```

--------------------------------

### Execute Training Loop with Cosine LR Schedule

Source: https://context7.com/arman-bd/guppylm/llms.txt

Implements the training loop including learning rate updates, gradient clipping, and periodic evaluation. Requires a pre-configured train_loader, model, and optimizer.

```python
model.train()
step = 0
for x, y in train_loader:
    if step >= tc.max_steps:
        break

    x, y = x.to(device), y.to(device)

    # Update learning rate (cosine with warmup)
    lr = get_lr(step, tc)
    for pg in optimizer.param_groups:
        pg["lr"] = lr

    # Forward + backward
    _, loss = model(x, y)
    loss.backward()
    torch.nn.utils.clip_grad_norm_(model.parameters(), tc.grad_clip)
    optimizer.step()
    optimizer.zero_grad(set_to_none=True)

    # Evaluate periodically
    if step % tc.eval_interval == 0:
        eval_loss = evaluate(model, eval_loader, device, max_batches=50)
        print(f"Step {step}: train={loss.item():.4f}, eval={eval_loss:.4f}, lr={lr:.6f}")

    step += 1

# Save checkpoint
torch.save({
    "step": step,
    "model_state_dict": model.state_dict(),
    "config": vars(mc),
}, "checkpoints/custom_model.pt")
```

--------------------------------

### Load Model and Run Inference

Source: https://github.com/arman-bd/guppylm/blob/main/use_guppylm.ipynb

Initializes the GuppyInference engine and defines a chat function to generate responses from the model.

```python
# Load model
from inference import GuppyInference
import torch

engine = GuppyInference('pytorch_model.bin', 'tokenizer.json',
                        device='cuda' if torch.cuda.is_available() else 'cpu')

def chat(prompt):
    return engine.chat_completion(
        [{'role': 'user', 'content': prompt}], max_tokens=64
    )['choices'][0]['message'].get('content', '').strip()

# Quick test
for p in ['hi guppy', 'are you hungry', 'tell me a joke', 'what is the internet', 'goodnight guppy']:
    print(f'You> {p}\nGuppy> {chat(p)}\n')
```

--------------------------------

### Generate Single Sample with GuppyLM

Source: https://context7.com/arman-bd/guppylm/llms.txt

Generates individual data samples using specific topic generators like greetings, food, or jokes. Useful for creating custom datasets or testing generation logic.

```python
from guppylm.generate_data import (
    gen_greeting, gen_food, gen_feeling, gen_confused,
    gen_bubbles, gen_meaning, gen_joke
)

# Generate single samples
sample = gen_greeting()
print(sample)
# {'input': 'hi guppy', 'output': 'hello. the water is nice today.', 'category': 'greeting'}

sample = gen_food()
print(sample)
# {'input': 'are you hungry', 'output': 'yes. always yes. give me the flakes.', 'category': 'food'}

sample = gen_joke()
print(sample)
# {'input': 'tell me a joke', 'output': 'what did the fish say when it hit the wall. dam.', 'category': 'joke'}
```

--------------------------------

### Download Model as Tarball

Source: https://github.com/arman-bd/guppylm/blob/main/train_guppylm.ipynb

Packages model checkpoints, configuration, and source files into a tar.gz archive. Automatically triggers a download if running within a Google Colab environment.

```python
import os

!cd /content && tar czf guppylm.tar.gz \
    guppy/checkpoints/best_model.pt \
    guppy/checkpoints/config.json \
    guppy/data/tokenizer.json \
    guppy/model.py \
    guppy/config.py \
    guppy/inference.py \
    guppy/hf_export/model.onnx

sz = os.path.getsize('/content/guppylm.tar.gz') / 1e6
print(f'Package: /content/guppylm.tar.gz ({sz:.1f} MB)')

try:
    from google.colab import files
    files.download('/content/guppylm.tar.gz')
except ImportError:
    print('Not in Colab — download manually from the file browser.')
```

--------------------------------

### GuppyLM Training Loop

Source: https://github.com/arman-bd/guppylm/blob/main/train_guppylm.ipynb

Implements the training loop for GuppyLM, including device selection, learning rate scheduling, evaluation, and mixed-precision training. Saves model and training configurations.

```python
%%writefile train.py
"""GuppyLM training loop."""

import json
import math
import os
import time

import torch

from config import GuppyConfig, TrainConfig
from dataset import get_dataloader
from model import GuppyLM


def get_device(config):
    if config.device == "auto":
        if torch.cuda.is_available():
            return torch.device("cuda")
        if hasattr(torch.backends, "mps") and torch.backends.mps.is_available():
            return torch.device("mps")
        return torch.device("cpu")
    return torch.device(config.device)


def get_lr(step, config):
    if step < config.warmup_steps:
        return config.learning_rate * step / config.warmup_steps
    progress = (step - config.warmup_steps) / max(1, config.max_steps - config.warmup_steps)
    coeff = 0.5 * (1 + math.cos(math.pi * progress))
    return config.min_lr + (config.learning_rate - config.min_lr) * coeff


@torch.no_grad()
def evaluate(model, loader, device, max_batches=50):
    model.eval()
    total_loss, n = 0, 0
    for x, y in loader:
        if n >= max_batches:
            break
        x, y = x.to(device), y.to(device)
        _, loss = model(x, y)
        total_loss += loss.item()
        n += 1
    model.train()
    return total_loss / max(1, n)


def train():
    mc = GuppyConfig()
    tc = TrainConfig()
    device = get_device(tc)
    torch.manual_seed(tc.seed)

    print(f"Device: {device}")

    tokenizer_path = os.path.join(tc.data_dir, "tokenizer.json")
    model = GuppyLM(mc).to(device)
    print(model.param_summary())

    train_loader = get_dataloader(
        os.path.join(tc.data_dir, "train.jsonl"), tokenizer_path,
        mc.max_seq_len, tc.batch_size, shuffle=True,
    )
    eval_loader = get_dataloader(
        os.path.join(tc.data_dir, "eval.jsonl"), tokenizer_path,
        mc.max_seq_len, tc.batch_size, shuffle=False,
    )
    print(f"Train: {len(train_loader.dataset):,}, Eval: {len(eval_loader.dataset):,}")

    optimizer = torch.optim.AdamW(
        model.parameters(), lr=tc.learning_rate,
        weight_decay=tc.weight_decay, betas=(0.9, 0.95),
    )

    use_amp = device.type == "cuda"
    scaler = torch.amp.GradScaler("cuda") if use_amp else None

    os.makedirs(tc.output_dir, exist_ok=True)
    with open(os.path.join(tc.output_dir, "config.json"), "w") as f:
        json.dump({"model": vars(mc), "train": vars(tc)}, f, indent=2)

    model.train()
    step, best_eval = 0, float("inf")
    losses = []
    t0 = time.time()

    print(f"\nTraining for {tc.max_steps} steps...")
    print(f"{ 'Step':>6} | {'LR':>10} | {'Train':>10} | {'Eval':>10} | {'Time':>8}")
    print("-" * 56)

    while step < tc.max_steps:
        for x, y in train_loader:

```

--------------------------------

### Format ChatML Prompts

Source: https://github.com/arman-bd/guppylm/blob/main/train_guppylm.ipynb

Converts a list of message dictionaries into a single ChatML-formatted string.

```python
def _format_prompt(self, messages):
    parts = []
    for msg in messages:
        role = msg.get("role", "user")
        content = msg.get("content") or ""
        if role == "system":
            continue
        parts.append(f"<|im_start|>{role}\n{content}<|im_end|>")
    parts.append("<|im_start|>assistant\n")
    return "\n".join(parts)
```

--------------------------------

### Define Model Configuration

Source: https://github.com/arman-bd/guppylm/blob/main/docs/index.html

Sets the hyperparameters for the model architecture and generation parameters.

```javascript
// ── Config ────────────────────────────────────────────────────────── const MODEL_BASE = "."; const CONFIG = { vocab_size: 4096, max_seq_len: 128, d_model: 384, n_layers: 6, n_heads: 6, ffn_hidden: 768, pad_id: 0, bos_id: 1, eos_id: 2, }; const GEN = { temperature: 0.7, top_k: 50, max_tokens: 32 };
```

--------------------------------

### Configure training parameters

Source: https://context7.com/arman-bd/guppylm/llms.txt

Define training hyperparameters using the TrainConfig dataclass.

```python
from guppylm import TrainConfig
```

--------------------------------

### Upload Model to Hugging Face

Source: https://github.com/arman-bd/guppylm/blob/main/train_guppylm.ipynb

Uses the Hugging Face API to create a repository and upload the contents of the hf_export folder. Requires the HF_TOKEN environment variable to be set.

```python
if HF_TOKEN:
    login(token=HF_TOKEN)
    api = HfApi()
    api.create_repo(HF_REPO, exist_ok=True)
    api.upload_folder(folder_path='hf_export', repo_id=HF_REPO, repo_type='model')
    print(f'Done! https://huggingface.co/{HF_REPO}')
else:
    print('No HF_TOKEN — exported locally to hf_export/')
```

--------------------------------

### Export GuppyLM Model to ONNX (No Quantization)

Source: https://context7.com/arman-bd/guppylm/llms.txt

Exports the trained GuppyLM model to ONNX format without quantization. This results in a larger file size (float32) but may offer higher precision for inference.

```bash
# Export without quantization (float32, ~35 MB)
python tools/export_onnx.py --no-quantize
```