### MoDA Model Training Example

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/api-reference-moda.md

Provides a complete example of setting up and training a MoDA model. This includes configuring the model with MoE parameters, moving it to CUDA, and performing a basic training loop with an optimizer.

```python
import torch
from open_mythos.moda import MoDAConfig, MoDAModel

# Configure model
cfg = MoDAConfig(
    vocab_size=10000,
    d_model=512,
    n_layers=6,
    n_heads_q=8,
    n_heads_kv=4,
    head_dim=64,
    max_seq_len=1024,
    n_shared_experts=2,
    n_routed_experts=32,
    n_activated_experts=4,
    expert_hidden_dim=1024,
    moe_balance_alpha=0.01,
)

model = MoDAModel(cfg).to("cuda")
optimizer = torch.optim.AdamW(model.parameters(), lr=1e-4)

# Training loop
for batch in dataloader:
    input_ids, labels = batch
    input_ids, labels = input_ids.to("cuda"), labels.to("cuda")
    
    logits, loss = model(input_ids, labels)
    
    loss.backward()
    optimizer.step()
    optimizer.zero_grad()
```

--------------------------------

### Initialize OpenMythos with Configuration

Source: https://github.com/kyegomez/openmythos/blob/main/docs/open_mythos.md

Demonstrates how to instantiate the OpenMythos model using either a minimal configuration for testing or a production-oriented setup.

```python
from open_mythos.main import OpenMythos, MythosConfig

# Minimal config for fast iteration / unit testing
small_cfg = MythosConfig(
    vocab_size=8192,
    dim=256,
    n_heads=4,
    n_kv_heads=2,
    max_seq_len=512,
    max_loop_iters=4,
    prelude_layers=1,
    coda_layers=1,
    attn_type="gqa",
    n_experts=8,
    n_shared_experts=1,
    n_experts_per_tok=2,
    expert_dim=64,
    lora_rank=4,
)
model = OpenMythos(small_cfg)
```

--------------------------------

### Complete OpenMythos Configuration Example

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/configuration.md

A comprehensive example demonstrating how to instantiate MythosConfig with a wide range of parameters for detailed model customization. This includes settings for vocabulary size, dimensions, attention heads, sequence length, MoE configuration, and more.

```python
from open_mythos import MythosConfig, OpenMythos

# Minimal configuration for testing
cfg = MythosConfig(
    vocab_size=1000,
    dim=256,
    n_heads=8,
    max_seq_len=128,
    max_loop_iters=4,
    prelude_layers=1,
    coda_layers=1,
    attn_type="mla",
    kv_lora_rank=32,
    q_lora_rank=64,
    qk_rope_head_dim=16,
    qk_nope_head_dim=16,
    v_head_dim=16,
    n_experts=8,
    n_shared_experts=1,
    n_experts_per_tok=2,
    expert_dim=64,
    act_threshold=0.99,
    rope_theta=500000.0,
    lora_rank=8,
    dropout=0.0,
)

model = OpenMythos(cfg)
```

--------------------------------

### Install Flash Attention

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/errors-and-exceptions.md

Optional installation command to include Flash Attention support for improved performance.

```bash
pip install open-mythos[flash]
```

--------------------------------

### Install OpenMythos

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/README.md

Install the OpenMythos package. Optionally, install with Flash Attention 2 for GQA speedup.

```bash
pip install open-mythos

# With Flash Attention 2 (optional, for GQA speedup)
pip install open-mythos[flash]
```

--------------------------------

### Install OpenMythos

Source: https://github.com/kyegomez/openmythos/blob/main/README.md

Install the OpenMythos library using pip. For Flash Attention 2 support, install with the `[flash]` extra.

```bash
pip install open-mythos

#uv pip install open-mythos
```

```bash
pip install open-mythos[flash]
```

--------------------------------

### Quick Navigation by Task

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/INDEX.md

Helps users find relevant documentation sections based on their immediate task, such as getting started, understanding architecture, using the model, or debugging errors.

```markdown
**I want to...**

- **Get started quickly** → [README.md](README.md) Quick Start section
- **Understand the architecture** → [README.md](README.md) Architecture Overview
- **Use the model** → [api-reference-openmythos.md](api-reference-openmythos.md)
- **Configure for my hardware** → [configuration.md](configuration.md)
- **Choose a model size** → [variants.md](variants.md)
- **Process text** → [api-reference-tokenizer.md](api-reference-tokenizer.md)
- **Debug an error** → [errors-and-exceptions.md](errors-and-exceptions.md)
- **Understand the types** → [types.md](types.md)
- **Explore MoDA** → [api-reference-moda.md](api-reference-moda.md)
```

--------------------------------

### OpenMythos Generation Example

Source: https://github.com/kyegomez/openmythos/blob/main/docs/open_mythos.md

Demonstrates how to initialize and use the OpenMythos model for text generation with specified parameters. Ensure you have a tokenizer of your choice.

```python
import torch
from open_mythos.main import OpenMythos, MythosConfig

model = OpenMythos(MythosConfig()).eval()

# Tokenized prompt (use your tokenizer of choice)
prompt = torch.tensor([[1, 450, 3118, 310, 278]])   # (1, 5)

output = model.generate(
    prompt,
    max_new_tokens=128,
    n_loops=16,        # deeper reasoning
    temperature=0.8,
    top_k=40,
)
# output.shape == (1, 133)
```

--------------------------------

### Instantiate OpenMythos Model with Custom Configuration

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/api-reference-openmythos.md

Example of creating an OpenMythos model instance with a custom MythosConfig. Ensure necessary imports are present.

```python
from open_mythos import MythosConfig, OpenMythos

cfg = MythosConfig(
    vocab_size=1000,
    dim=256,
    n_heads=8,
    max_seq_len=128,
    max_loop_iters=4,
    attn_type="mla",
    n_kv_heads=8,
    kv_lora_rank=32,
    q_lora_rank=64,
    qk_rope_head_dim=16,
    qk_nope_head_dim=16,
    v_head_dim=16,
)
model = OpenMythos(cfg)
```

--------------------------------

### Load Pre-configured Model Variants

Source: https://github.com/kyegomez/openmythos/blob/main/README.md

Instantiate OpenMythos models using pre-defined configurations for various parameter scales (e.g., 7B). This example shows how to load a configuration and then initialize the model.

```python
from open_mythos import (
    mythos_1b,
    mythos_3b,
    mythos_10b,
    mythos_50b,
    mythos_100b,
    mythos_500b,
    mythos_1t,
    OpenMythos,
)

cfg = mythos_7b()  # returns a MythosConfig
model = OpenMythos(cfg)

total = sum(p.numel() for p in model.parameters())
print(f"Parameters: {total:,}")
```

--------------------------------

### Initialize 1B Parameter Model

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/variants.md

Use the `mythos_1b()` function to get a configuration for a 1 billion parameter model. This is suitable for research or fine-tuning on limited hardware.

```python
from open_mythos import OpenMythos, mythos_1b

cfg = mythos_1b()
model = OpenMythos(cfg)

total_params = sum(p.numel() for p in model.parameters())
print(f"Parameters: {total_params:,}")  # ~1.0B
```

--------------------------------

### Train 3B Model on Single GPU

Source: https://github.com/kyegomez/openmythos/blob/main/README.md

Use this command to start training the 3B model on a single GPU. Ensure you are in the project root directory.

```bash
python training/3b_fine_web_edu.py
```

--------------------------------

### OpenMythos Training Example

Source: https://github.com/kyegomez/openmythos/blob/main/docs/open_mythos.md

Example of training the OpenMythos model using AdamW optimizer. Computes cross-entropy loss and performs backpropagation. Ensure model and input tensors are on the correct device (e.g., CUDA).

```python
import torch
from open_mythos.main import OpenMythos, MythosConfig

model = OpenMythos(MythosConfig()).cuda()
optimizer = torch.optim.AdamW(model.parameters(), lr=3e-4)

input_ids = torch.randint(0, 32000, (2, 512)).cuda()
labels    = torch.randint(0, 32000, (2, 512)).cuda()

logits = model(input_ids)                    # (2, 512, 32000)
loss   = torch.nn.functional.cross_entropy(
    logits.view(-1, 32000),
    labels.view(-1),
)
loss.backward()
optimizer.step()
```

--------------------------------

### MoE Balance Loss Example

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/api-reference-moda.md

Shows how to enable and use the balance loss for MoE FFNs in the MoDA model. This helps prevent routing collapse by encouraging equal expert usage. Requires importing torch.

```python
from open_mythos.moda import MoDAConfig, MoDAModel
import torch

cfg = MoDAConfig(
    vocab_size=512,
    d_model=128,
    n_layers=4,
    moe_balance_alpha=0.01,  # Enable balance loss
    moe_score_func="softmax",
)

model = MoDAModel(cfg)
input_ids = torch.randint(0, 512, (2, 32))
labels = torch.randint(0, 512, (2, 32))

logits, loss = model(input_ids, labels)
# Loss includes: LM loss + moe_balance_alpha * balance_loss
```

--------------------------------

### Quick Navigation by Audience

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/INDEX.md

Guides users to the most relevant documentation based on their role, such as ML Engineer, Researcher, Data Scientist, or Troubleshooter.

```markdown
**I'm a...**

- **First-time user** → Start with [README.md](README.md)
- **Machine Learning Engineer** → [configuration.md](configuration.md) + [variants.md](variants.md)
- **Researcher** → [api-reference-openmythos.md](api-reference-openmythos.md) + [api-reference-moda.md](api-reference-moda.md)
- **Data Scientist** → [api-reference-tokenizer.md](api-reference-tokenizer.md) + [README.md](README.md) patterns
- **Troubleshooter** → [errors-and-exceptions.md](errors-and-exceptions.md)
- **Type Checker / Validator** → [types.md](types.md)
```

--------------------------------

### Use Pre-configured Model Variants

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/configuration.md

Load pre-tuned configurations for different model scales using variants from the open_mythos.variants module. This simplifies setup for common model sizes.

```python
from open_mythos.variants import (
    mythos_1b,
    mythos_3b,
    mythos_10b,
    mythos_50b,
    mythos_100b,
    mythos_500b,
    mythos_1t,
)
from open_mythos import OpenMythos

# Use a pre-configured variant
cfg = mythos_3b()  # Returns MythosConfig optimized for ~3B parameters
model = OpenMythos(cfg)
```

--------------------------------

### Initialize 10B Parameter Model for General Use

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/variants.md

Use `mythos_10b()` to obtain a configuration for a 10 billion parameter model, suitable for general-purpose tasks balancing quality and compute. The example shows parameter count and deeper reasoning capabilities.

```python
from open_mythos import OpenMythos, mythos_10b

cfg = mythos_10b()
model = OpenMythos(cfg)

total_params = sum(p.numel() for p in model.parameters())
print(f"Parameters: {total_params:,}")  # ~10B

# Deeper reasoning at test time
ids = torch.randint(0, cfg.vocab_size, (2, 128))
logits = model(ids, n_loops=32)  # Extrapolate beyond training
```

--------------------------------

### Documentation Cross-References

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/INDEX.md

Highlights the consistent documentation standards across all files, including signatures, parameter tables, usage examples, and source code references.

```markdown
All documentation files include:
- ✓ Full signature in fenced code blocks
- ✓ Parameter tables (name | type | default | description)
- ✓ Return type documentation
- ✓ Usage examples with realistic code
- ✓ Source file references (e.g., `open_mythos/main.py:714`)
- ✓ Links to related classes/functions
- ✓ Constraint and error handling notes
```

--------------------------------

### Model Variants Imports

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/README.md

Imports pre-configured model scales from the variants module. Use these for quick setup of different model sizes.

```python
from open_mythos.variants import (
    mythos_1b,
    mythos_3b,
    mythos_10b,
    mythos_50b,
    mythos_100b,
    mythos_500b,
    mythos_1t,
)
```

--------------------------------

### Integrate OpenMythos with Tokenizer for Text Generation

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/README.md

Load a pre-configured model and tokenizer, encode input text, generate output, and decode the result back into human-readable text. This example demonstrates end-to-end text processing.

```python
from open_mythos import OpenMythos, MythosTokenizer, mythos_10b
import torch

cfg = mythos_10b()
model = OpenMythos(cfg).to("cuda")
tok = MythosTokenizer()

# Encode text
text = "The future of AI is"
ids = torch.tensor([tok.encode(text)]).to("cuda")

# Generate
generated = model.generate(ids, max_new_tokens=128, n_loops=16)

# Decode back
output = tok.decode(generated[0].tolist())
print(output)
```

--------------------------------

### OpenMythos Forward Pass Example

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/api-reference-openmythos.md

Demonstrates how to perform a forward pass through the OpenMythos model for prefilling sequences and for single-step decoding with KV caching. The KV cache is mutated in-place for efficient autoregressive generation.

```python
import torch
from open_mythos import OpenMythos, MythosConfig

cfg = MythosConfig(vocab_size=1000, dim=256)
model = OpenMythos(cfg)

# Prefill: process entire prompt
input_ids = torch.randint(0, 1000, (2, 16))  # batch_size=2, seq_len=16
logits = model(input_ids, n_loops=4)
print(logits.shape)  # (2, 16, 1000)

# Decode step with KV cache
kv_cache = {}
logits = model(input_ids, n_loops=4, kv_cache=kv_cache)
next_token_id = torch.randint(0, 1000, (2, 1))
logits = model(next_token_id, n_loops=4, kv_cache=kv_cache, start_pos=16)
```

--------------------------------

### KV Cache Structure Example

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/types.md

Illustrates the dictionary structure for storing key and value tensors across layers and loop iterations for autoregressive decoding.

```python
kv_cache = {
    "prelude_0": {"k": Tensor(B, S, n_heads, head_dim), "v": Tensor(...)},
    "prelude_1": {"k": Tensor(...), "v": Tensor(...)},
    "recurrent_loop_0": {"c_kv": Tensor(...), "k_rope": Tensor(...)},  # MLA
    "recurrent_loop_1": {"c_kv": Tensor(...), "k_rope": Tensor(...)},
    "coda_0": {"k": Tensor(...), "v": Tensor(...)},
    "coda_1": {"k": Tensor(...), "v": Tensor(...)},
}
```

--------------------------------

### Initialize 50B Parameter Model for Large Reasoning Tasks

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/variants.md

The `mythos_50b()` function returns a configuration for a 50 billion parameter model, designed for reasoning-heavy tasks requiring frontier-class quality. The example demonstrates multi-loop reasoning on a CUDA-enabled model.

```python
from open_mythos import OpenMythos, mythos_50b
import torch

cfg = mythos_50b()
model = OpenMythos(cfg).to("cuda")

# Multi-loop reasoning
prompt = torch.randint(0, cfg.vocab_size, (1, 128), device="cuda")
output = model.generate(
    prompt,
    max_new_tokens=256,
    n_loops=64,  # Deep reasoning
    temperature=0.7,
)
```

--------------------------------

### OpenMythos Model Integration and Inference

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/variants.md

Demonstrates how to load a specific OpenMythos model variant (e.g., 10B), tokenize input, perform inference, and decode the output. Ensure PyTorch and the open_mythos library are installed.

```python
import torch
from open_mythos import (
    OpenMythos,
    mythos_1b,
    mythos_3b,
    mythos_10b,
    mythos_50b,
    mythos_100b,
    mythos_500b,
    mythos_1t,
    MythosTokenizer,
)

# Select variant by model size
model_size = "10b"
variant_map = {
    "1b": mythos_1b,
    "3b": mythos_3b,
    "10b": mythos_10b,
    "50b": mythos_50b,
    "100b": mythos_100b,
    "500b": mythos_500b,
    "1t": mythos_1t,
}

cfg = variant_map[model_size]()
model = OpenMythos(cfg).eval().to("cuda")
tok = MythosTokenizer()

# Inference
prompt = "The future of AI is"
ids = torch.tensor([tok.encode(prompt)]).to("cuda")
generated = model.generate(ids, max_new_tokens=128, n_loops=16)
output = tok.decode(generated[0].tolist())
print(output)
```

--------------------------------

### OpenMythos Autoregressive Generation Example

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/api-reference-openmythos.md

Shows how to use the `generate` method for autoregressive token generation. This method handles KV caching internally and allows control over generation parameters like `max_new_tokens`, `n_loops`, `temperature`, and `top_k`.

```python
import torch
from open_mythos import OpenMythos, MythosConfig

cfg = MythosConfig(vocab_size=1000, dim=256)
model = OpenMythos(cfg)

prompt = torch.randint(0, 1000, (1, 16))
generated = model.generate(
    prompt,
    max_new_tokens=32,
    n_loops=8,
    temperature=0.7,
    top_k=50,
)
print(generated.shape)  # (1, 48)
```

--------------------------------

### Configure Multi-Latent Attention (MLA)

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/README.md

Example configuration for Multi-Latent Attention (MLA). MLA is recommended for optimizing memory usage, especially with long sequences, by compressing the KV cache.

```python
cfg = MythosConfig(
    attn_type="mla",
    kv_lora_rank=512,
    q_lora_rank=1536,
    qk_rope_head_dim=64,
    qk_nope_head_dim=128,
    v_head_dim=128,
)
```

--------------------------------

### Configure Grouped Query Attention (GQA)

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/README.md

Example configuration for Grouped Query Attention (GQA). Use GQA to optimize inference speed and reduce KV cache size when Flash Attention 2 is available.

```python
cfg = MythosConfig(
    attn_type="gqa",
    n_heads=16,
    n_kv_heads=4,  # 4:1 sharing
)
```

--------------------------------

### Fallback to Scaled Dot-Product Attention

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/errors-and-exceptions.md

When Flash Attention 2 is not installed, the model gracefully falls back to a slower manual scaled dot-product attention implementation. This snippet demonstrates the configuration and usage.

```python
from open_mythos import MythosConfig, OpenMythos

cfg = MythosConfig(attn_type="gqa")
model = OpenMythos(cfg)

# GQAttention will use fallback (slower) implementation
ids = torch.randint(0, 1000, (2, 16))
logits = model(ids)  # Works, but slower than with Flash Attention
```

--------------------------------

### MoDAModel Forward Pass Example

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/api-reference-moda.md

Demonstrates the forward pass of the MoDAModel. The model can compute logits without a loss, or with a language modeling loss if labels are provided. The returned loss includes both the language modeling loss and the expert balance loss.

```python
from open_mythos.moda import MoDAConfig, MoDAModel
import torch

cfg = MoDAConfig(vocab_size=512, d_model=128, n_layers=4)
model = MoDAModel(cfg)

# Forward pass without loss
input_ids = torch.randint(0, 512, (2, 32))
logits, _ = model(input_ids)
print(logits.shape)  # (2, 32, 512)

# Forward pass with loss
labels = torch.randint(0, 512, (2, 32))
logits, loss = model(input_ids, labels)
print(f"Loss: {loss.item():.4f}")

# Backward pass
loss.backward()
```

--------------------------------

### Batch Encoding for Performance

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/api-reference-tokenizer.md

Provides an example of using the underlying tokenizer's batch encoding method for improved performance when processing multiple texts. This method supports padding for consistent input lengths.

```python
batch_encoded = tok.tokenizer.batch_encode_plus(texts, padding=True)
```

--------------------------------

### Instantiate and Print Model Parameters

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/api-reference-moda.md

Demonstrates how to create a MoDA model with a specific configuration and then print its total trainable parameters. Ensure MoDAConfig and MoDAModel are imported.

```python
from open_pythos.moda import MoDAConfig, MoDAModel

cfg = MoDAConfig(vocab_size=512, d_model=128, n_layers=4)
model = MoDAModel(cfg)

total_params = model.num_parameters()
print(f"Total parameters: {total_params:,}")
```

--------------------------------

### LTIInjection Get A Method

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/api-reference-openmythos.md

Computes the discretized diagonal state matrix A. This method ensures that the spectral radius of A is less than 1.

```python
def get_A(self) -> torch.Tensor:
    """
    Compute discretized diagonal state matrix.

    Returns:
        1-D tensor of shape (dim,) with all values strictly in (0, 1),
        guaranteeing ρ(A) < 1 regardless of gradient steps.
    """
```

--------------------------------

### Use Pre-configured OpenMythos Variants

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/README.md

Instantiate a model using a pre-configured variant (e.g., mythos_3b) and print the total number of parameters. This is useful for quickly setting up models of specific sizes.

```python
from open_mythos import OpenMythos, mythos_3b

cfg = mythos_3b()  # Returns optimized 3B config
model = OpenMythos(cfg)

total_params = sum(p.numel() for p in model.parameters())
print(f"Parameters: {total_params:,}")
```

--------------------------------

### Get Injection Matrix 'A'

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/errors-and-exceptions.md

Retrieves the injection matrix 'A' from the model's recurrent component. This operation should be performed within the forward pass.

```python
A = model.recurrent.injection.get_A()
# A is computed correctly from current parameters
```

--------------------------------

### OpenMythos Model Initialization and Usage

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/types.md

Demonstrates how to initialize the OpenMythos model and use it for both prefilling (building the cache) and decoding (reusing the cache).

```python
import torch
from open_mythos import OpenMythos, MythosConfig

cfg = MythosConfig(vocab_size=1000, dim=256)
model = OpenMythos(cfg)

# Prefill: build cache
kv_cache = {}
input_ids = torch.randint(0, 1000, (1, 16))
logits = model(input_ids, kv_cache=kv_cache)

# Decode: reuse cache
next_id = torch.randint(0, 1000, (1, 1))
logits = model(next_id, kv_cache=kv_cache, start_pos=16)
```

--------------------------------

### Get Total Trainable Parameters

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/api-reference-moda.md

Retrieves the total number of trainable parameters in the MoDA model. This is useful for understanding model size and complexity.

```python
def num_parameters(self) -> int:
```
```

--------------------------------

### Instantiate OpenMythos Model with Custom Configuration

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/README.md

Create a custom configuration for the OpenMythos model and then instantiate the model. The output shows the shape of the logits from a forward pass.

```python
import torch
from open_mythos import OpenMythos, MythosConfig

# Create config
cfg = MythosConfig(
    vocab_size=1000,
    dim=256,
    n_heads=8,
    max_seq_len=128,
    max_loop_iters=4,
    attn_type="mla",
    n_kv_heads=8,
    kv_lora_rank=32,
    q_lora_rank=64,
    qk_rope_head_dim=16,
    qk_nope_head_dim=16,
    v_head_dim=16,
)

# Instantiate model
model = OpenMythos(cfg)

# Forward pass
ids = torch.randint(0, cfg.vocab_size, (2, 16))
logits = model(ids, n_loops=4)
print(logits.shape)  # (2, 16, 1000)

# Generation
generated = model.generate(ids, max_new_tokens=32, n_loops=8)
print(generated.shape)  # (2, 48)

# Check spectral radius for stability
A = model.recurrent.injection.get_A()
rho = torch.linalg.eigvals(A).abs().max().item()
print(f"Spectral radius ρ(A) = {rho:.4f} (must be < 1)")
```

--------------------------------

### Initialize OpenMythos Model

Source: https://github.com/kyegomez/openmythos/blob/main/docs/open_mythos.md

Instantiate the OpenMythos model with a given configuration. This builds sub-modules, precomputes RoPE frequency buffers, and runs weight initialization. Ensure MythosConfig is properly defined.

```python
OpenMythos(cfg: MythosConfig)
```

```python
from open_mythos.main import OpenMythos, MythosConfig

cfg = MythosConfig(
    vocab_size=32000,
    dim=2048,
    n_heads=16,
    n_kv_heads=4,
    max_loop_iters=16,
    attn_type="mla",
)
model = OpenMythos(cfg)
print(f"Parameters: {sum(p.numel() for p in model.parameters()):,}")
```

--------------------------------

### Get Tokenizer Vocabulary Size

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/api-reference-tokenizer.md

Retrieve the number of unique tokens in the tokenizer's vocabulary. This is useful for configuring models or understanding the tokenizer's capacity.

```python
tok = MythosTokenizer()
vocab = tok.vocab_size
print(f"Vocabulary size: {vocab}")
# Output: Vocabulary size: 20000
```

--------------------------------

### Initializing MythosTokenizer with Different Models

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/api-reference-tokenizer.md

Shows how to initialize the MythosTokenizer with various pre-trained models from the HuggingFace Hub. This allows flexibility in choosing tokenizers based on model size, performance, or specific training data.

```python
# LLaMA 2 tokenizer
tok = MythosTokenizer(model_id="meta-llama/Llama-2-7b-hf")
```

```python
# GPT-2 tokenizer (smaller, faster)
tok = MythosTokenizer(model_id="gpt2")
```

```python
# Falcon tokenizer
tok = MythosTokenizer(model_id="tiiuae/falcon-7b")
```

```python
# Custom local tokenizer
tok = MythosTokenizer(model_id="/path/to/local/tokenizer.json")
```

--------------------------------

### Mythos 1T Model Initialization and Usage

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/variants.md

Initializes the Mythos 1T model and demonstrates its parameter count and usage for extreme reasoning. Note the doubled loop iterations for testing.

```python
from open_mythos import OpenMythos, mythos_1t

cfg = mythos_1t()
model = OpenMythos(cfg)

# Maximum scale model
total_params = sum(p.numel() for p in model.parameters())
print(f"Total parameters: {total_params / 1e12:.2f}T")  # ~1.0T

# Extreme reasoning at test time (64+ loops)
ids = torch.randint(0, cfg.vocab_size, (1, 1024))
logits = model(ids, n_loops=128)  # 2x training depth
```

--------------------------------

### Basic Text Processing with Tokenizer and Model

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/api-reference-tokenizer.md

Demonstrates a basic workflow of encoding text using MythosTokenizer, preparing it as input for an OpenMythos model, and obtaining model logits.

```python
from open_mythos import MythosTokenizer, OpenMythos, MythosConfig
import torch

tok = MythosTokenizer()
cfg = MythosConfig(vocab_size=tok.vocab_size)
model = OpenMythos(cfg)

text = "Once upon a time"
token_ids = tok.encode(text)
input_ids = torch.tensor([token_ids])

logits = model(input_ids)
print(logits.shape)  # (1, 4, vocab_size)
```

--------------------------------

### Mythos 500B Model Initialization and Parameter Count

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/variants.md

Initializes the Mythos 500B model and prints its total parameter count. Note the sparse activation of MoE parameters.

```python
from open_mythos import OpenMythos, mythos_500b

cfg = mythos_500b()
model = OpenMythos(cfg)

# Ultra-large model: 500B parameter count
total_params = sum(p.numel() for p in model.parameters())
print(f"Total parameters: {total_params / 1e9:.1f}B")  # ~500B

# Note: MoE activation is sparse; only ~5% of parameters
# are activated per token
```

--------------------------------

### Configure Coda Transformer Layers

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/configuration.md

Set the number of standard transformer blocks after the recurrent loop for final refinement. Examples show minimal, standard, and deeper coda configurations.

```python
# Minimal coda
cfg = MythosConfig(coda_layers=1)

# Standard coda
cfg = MythosConfig(coda_layers=2)

# Deeper coda
cfg = MythosConfig(coda_layers=4)
```

--------------------------------

### Configure Model Dimension (dim)

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/configuration.md

Set the model's hidden dimension, which affects memory usage and compute. Examples demonstrate configurations for small, medium, and large models.

```python
# Small model
cfg = MythosConfig(dim=256)

# Medium model
cfg = MythosConfig(dim=2048)

# Large model
cfg = MythosConfig(dim=8192)
```

--------------------------------

### Configure GQA Key/Value Heads (n_kv_heads)

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/configuration.md

Set the number of key/value heads for Grouped Query Attention. This is used only when attn_type='gqa'. Examples show configurations for different head sharing ratios.

```python
# 16 Q heads, 4 KV heads (4:1 sharing)
cfg = MythosConfig(
    attn_type="gqa",
    n_heads=16,
    n_kv_heads=4,
)

# 32 Q heads, 8 KV heads (4:1 sharing)
cfg = MythosConfig(
    attn_type="gqa",
    n_heads=32,
    n_kv_heads=8,
)
```

--------------------------------

### Configure Prelude Transformer Layers

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/configuration.md

Set the number of standard transformer blocks before the recurrent loop for initial input processing. Examples show minimal, standard, and deeper prelude configurations.

```python
# Minimal prelude
cfg = MythosConfig(prelude_layers=1)

# Standard prelude
cfg = MythosConfig(prelude_layers=2)

# Deeper prelude
cfg = MythosConfig(prelude_layers=4)
```

--------------------------------

### Initialize 3B Parameter Model for Compact Inference

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/variants.md

The `mythos_3b()` function provides a configuration for a 3 billion parameter model, ideal for compact inference and deployment on devices with limited resources. It demonstrates memory-efficient generation.

```python
from open_mythos import OpenMythos, mythos_3b
import torch

cfg = mythos_3b()
model = OpenMythos(cfg)

# Compact model for deployment
ids = torch.randint(0, cfg.vocab_size, (1, 32))
logits = model(ids, n_loops=4)
print(f"Memory-efficient generation: {logits.shape}")
```

--------------------------------

### Configure Maximum Sequence Length (max_seq_len)

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/configuration.md

Set the maximum sequence length for RoPE precomputation, defining the context length. Examples show standard, long, and very long context configurations.

```python
# Standard context (4k tokens)
cfg = MythosConfig(max_seq_len=4096)

# Long context (8k tokens)
cfg = MythosConfig(max_seq_len=8192)

# Very long context (1M tokens)
cfg = MythosConfig(max_seq_len=1000000)
```

--------------------------------

### Text Generation using Tokenizer and Model

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/api-reference-tokenizer.md

Shows how to encode a prompt, generate new tokens using the model's `generate` method, and then decode the generated token IDs back into text.

```python
from open_mythos import MythosTokenizer, OpenMythos, MythosConfig
import torch

tok = MythosTokenizer()
cfg = MythosConfig(vocab_size=tok.vocab_size)
model = OpenMythos(cfg)

prompt = "The future of AI is"
token_ids = tok.encode(prompt)
input_ids = torch.tensor(token_ids)

# Generate new tokens
generated_ids = model.generate(input_ids, max_new_tokens=32)

# Decode back to text
generated_text = tok.decode(generated_ids[0].tolist())
print(generated_text)
```

--------------------------------

### Mythos 100B Model Initialization and Usage

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/variants.md

Initializes and uses the Mythos 100B model for processing ultra-long contexts and generating extended outputs. Requires CUDA.

```python
from open_mythos import OpenMythos, mythos_100b
import torch

cfg = mythos_100b()
model = OpenMythos(cfg).to("cuda")

# Ultra-long context
long_prompt = torch.randint(0, cfg.vocab_size, (1, 100000), device="cuda")

# Process million-token context
with torch.no_grad():
    logits = model(long_prompt, n_loops=32)

# Generate 128k output
generated = model.generate(
    long_prompt[-4096:],  # Last 4k tokens
    max_new_tokens=131072,
    n_loops=64,
    temperature=0.7,
)
```

--------------------------------

### Configure Maximum Recurrent Loop Iterations (max_loop_iters)

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/configuration.md

Set the default recurrent loop depth for inference. Each iteration applies the same transformer weights. Examples show default and deeper reasoning configurations.

```python
# Default training depth (16 iterations)
cfg = MythosConfig(max_loop_iters=16)

# Deeper reasoning (32 iterations)
cfg = MythosConfig(max_loop_iters=32)
```

--------------------------------

### Configure Number of Attention Heads (n_heads)

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/configuration.md

Set the number of query attention heads. For GQA, this must be divisible by n_kv_heads. Examples show configurations for different head counts and model dimensions.

```python
cfg = MythosConfig(dim=2048, n_heads=16)  # head_dim = 128

cfg = MythosConfig(dim=4096, n_heads=32)  # head_dim = 128
```

--------------------------------

### MoDA Configuration Variants

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/api-reference-moda.md

Shows how to configure MoDA models at different scales, from tiny for testing to medium. Adjust parameters like vocab_size, d_model, and n_layers based on your needs.

```python
from open_mythos.moda import MoDAConfig

# Tiny (testing)
cfg = MoDAConfig(
    vocab_size=512,
    d_model=128,
    n_layers=4,
    n_heads_q=4,
    n_heads_kv=2,
    head_dim=32,
)
```

```python
# Small
cfg = MoDAConfig(
    vocab_size=50257,
    d_model=768,
    n_layers=12,
    n_heads_q=12,
    n_heads_kv=4,
    head_dim=64,
)
```

```python
# Medium
cfg = MoDAConfig(
    vocab_size=50257,
    d_model=1024,
    n_layers=24,
    n_heads_q=16,
    n_heads_kv=8,
    head_dim=64,
)
```

--------------------------------

### Expert Initialization

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/api-reference-openmythos.md

Initializes a single SwiGLU feed-forward expert. This module is used both as a routed expert and within the dense FFN of the prelude and coda.

```python
class Expert(nn.Module):
    def __init__(self, dim: int, expert_dim: int) -> None:
        """
        Args:
            dim        -- input and output feature dimension
            expert_dim -- inner (hidden) dimension
        """

```

--------------------------------

### Configure Attention Type (attn_type)

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/configuration.md

Select the attention mechanism: 'gqa' for Grouped Query Attention (faster KV cache) or 'mla' for Multi-Latent Attention (smaller KV cache). Examples show configurations for both.

```python
# Grouped Query Attention (faster KV cache)
cfg = MythosConfig(
    attn_type="gqa",
    n_kv_heads=4,
)

# Multi-Latent Attention (smaller KV cache)
cfg = MythosConfig(
    attn_type="mla",
    kv_lora_rank=512,
    q_lora_rank=1536,
    qk_rope_head_dim=64,
    qk_nope_head_dim=128,
    v_head_dim=128,
)
```

--------------------------------

### OpenMythos Constructor

Source: https://github.com/kyegomez/openmythos/blob/main/docs/open_mythos.md

Initializes the OpenMythos model, building sub-modules, precomputing RoPE frequency buffers, and running weight initialization.

```APIDOC
## Constructor

### Description
Builds all sub-modules, precomputes RoPE frequency buffers, and runs weight initialization.

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **cfg** (MythosConfig) - Required - Configuration object for the model.

### Request Example
```python
from open_mythos.main import OpenMythos, MythosConfig

cfg = MythosConfig(
    vocab_size=32000,
    dim=2048,
    n_heads=16,
    n_kv_heads=4,
    max_loop_iters=16,
    attn_type="mla",
)
model = OpenMythos(cfg)
print(f"Parameters: {sum(p.numel() for p in model.parameters()):,}")
```

### Response
#### Success Response (200)
- **model** (OpenMythos) - The initialized OpenMythos model instance.

#### Response Example
```json
{
  "model": "<OpenMythos instance>"
}
```
```

--------------------------------

### Configure vocab_size for Tokenizer

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/configuration.md

Set the size of the token vocabulary. This value must match the tokenizer's output range. Examples show small, standard GPT-2 compatible, and large multilingual sizes.

```python
from open_mythos import MythosConfig, OpenMythos

# Small vocab for testing
cfg = MythosConfig(vocab_size=512)

# Standard size (GPT-2 compatible)
cfg = MythosConfig(vocab_size=50257)

# Large vocab for multilingual
cfg = MythosConfig(vocab_size=100000)
```

--------------------------------

### forward

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/api-reference-openmythos.md

Performs a forward pass through the OpenMythos model, processing input tokens to generate logits. It supports optional KV caching for efficient autoregressive decoding and specifies the starting position within a sequence.

```APIDOC
## forward

### Description
Performs a forward pass through Prelude → Recurrent Block → Coda. This method is used for both prefilling sequences and for individual decoding steps when KV caching is employed.

### Method
`forward` (Python method)

### Parameters
#### Input Parameters
- **input_ids** (torch.Tensor) - Required - Token indices of shape (B, T) where B is batch size, T is sequence length.
- **n_loops** (Optional[int]) - Optional - Recurrent loop depth; if None, uses cfg.max_loop_iters. Can be increased at inference for depth extrapolation.
- **kv_cache** (Optional[dict]) - Optional - Dict mutated in-place for autoregressive KV caching. Pass empty dict {} at start, reuse across decode steps.
- **start_pos** (int) - Optional - Default: 0 - Index of the first token within the full sequence; used to select correct RoPE frequencies during incremental decoding.

### Returns
`torch.Tensor` of shape (B, T, vocab_size) — logits over vocabulary at each position.

### Example
```python
import torch
from open_mythos import OpenMythos, MythosConfig

cfg = MythosConfig(vocab_size=1000, dim=256)
model = OpenMythos(cfg)

# Prefill: process entire prompt
input_ids = torch.randint(0, 1000, (2, 16))  # batch_size=2, seq_len=16
logits = model(input_ids, n_loops=4)
print(logits.shape)  # (2, 16, 1000)

# Decode step with KV cache
kv_cache = {}
logits = model(input_ids, n_loops=4, kv_cache=kv_cache)
next_token_id = torch.randint(0, 1000, (2, 1))
logits = model(next_token_id, n_loops=4, kv_cache=kv_cache, start_pos=16)
```
```

--------------------------------

### Sampling Strategy Logic

Source: https://github.com/kyegomez/openmythos/blob/main/docs/open_mythos.md

Illustrates the core logic for sampling the next token based on logits, temperature, and top-k filtering. This is a conceptual representation of the sampling process.

```python
logits = forward(cur_ids, n_loops, kv_cache)[:, -1, :] / temperature

if top_k > 0:
    threshold = logits.topk(top_k).values[:, -1:]
    logits[logits < threshold] = -inf

probs    = softmax(logits)
next_tok = multinomial(probs, num_samples=1)
```

--------------------------------

### Integrating Tokenizer with Model Training

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/api-reference-tokenizer.md

Illustrates the process of integrating the MythosTokenizer with OpenMythos model training. It ensures that the tokenizer's vocabulary size matches the model's configuration for compatibility.

```python
from open_mythos import MythosTokenizer, MythosConfig, OpenMythos

tok = MythosTokenizer()
cfg = MythosConfig(vocab_size=tok.vocab_size)
model = OpenMythos(cfg)

# Both now have matching vocabulary
assert cfg.vocab_size == tok.vocab_size
```

--------------------------------

### Common Errors and Exception Handling

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/INDEX.md

Details common errors encountered during configuration, input processing, caching, and runtime, along with troubleshooting tips and solutions.

```markdown
**Configuration Errors**
  - Dimension mismatch (dim % n_heads != 0)
  - GQA head ratio errors
  - Invalid attention type
  - Missing MLA parameters

- **Input Shape Errors**
  - Incorrect input_ids dimensions
  - Token IDs out of vocabulary range

- **Caching Errors**
  - KV cache corruption from reuse
  - Incorrect start_pos during decoding

- **Recurrence & Looping Errors**
  - Loop count constraints
  - Spectral radius divergence (ρ(A) ≥ 1)

- **Device & Memory Errors**
  - Device mismatch (GPU vs. CPU)
  - Out of memory (OOM) with solutions

- **Tokenizer Errors**
  - Missing tokenizer models
  - Type mismatches in encoding/decoding

- **Dependency Errors**
  - Optional Flash Attention 2 handling

- **Common Runtime Issues**
  - Spectral radius monitoring
  - ACT halting side effects
  - No-grad context misuse
```

--------------------------------

### OpenMythos Usage with MLA and GQA Attention

Source: https://github.com/kyegomez/openmythos/blob/main/README.md

Demonstrates how to initialize and use the OpenMythos model with different attention types (MLA and GQA). It shows model parameter count, forward pass, generation, and spectral radius calculation.

```python
import torch
from open_mythos.main import OpenMythos, MythosConfig


attn_type = "mla"  # or "gqa"

base = {
    "vocab_size": 1000,
    "dim": 256,
    "n_heads": 8,
    "max_seq_len": 128,
    "max_loop_iters": 4,
    "prelude_layers": 1,
    "coda_layers": 1,
    "n_experts": 8,
    "n_shared_experts": 1,
    "n_experts_per_tok": 2,
    "expert_dim": 64,
    "lora_rank": 8,
    "attn_type": attn_type,
}

if attn_type == "gqa":
    cfg = MythosConfig(**base, n_kv_heads=2)
else:
    cfg = MythosConfig(
        **base,
        n_kv_heads=8,
        kv_lora_rank=32,
        q_lora_rank=64,
        qk_rope_head_dim=16,
        qk_nope_head_dim=16,
        v_head_dim=16,
    )

model = OpenMythos(cfg)
total = sum(p.numel() for p in model.parameters())
print(f"\n[{attn_type.upper()}] Parameters: {total:,}")

ids = torch.randint(0, cfg.vocab_size, (2, 16))
logits = model(ids, n_loops=4)
print(f"[{attn_type.upper()}] Logits shape: {logits.shape}")

out = model.generate(ids, max_new_tokens=8, n_loops=8)
print(f"[{attn_type.upper()}] Generated shape: {out.shape}")

A = model.recurrent.injection.get_A()
rho = torch.linalg.eigvals(A).abs().max().item()
print(
    f"[{attn_type.upper()}] Spectral radius ρ(A) = {rho:.4f} (must be < 1)"
)
```

--------------------------------

### Batch Encoding and Padding

Source: https://github.com/kyegomez/openmythos/blob/main/_autodocs/api-reference-tokenizer.md

Illustrates how to encode multiple text strings, find the maximum length, and pad shorter sequences with zeros to create a uniform tensor for batch processing.

```python
from open_mythos import MythosTokenizer
import torch

tok = MythosTokenizer()

# Multiple texts
texts = [
    "Hello world",
    "How are you today?",
    "The quick brown fox",
]

# Encode each
batch_ids = [tok.encode(text) for text in texts]

# Pad to same length for batching
max_len = max(len(ids) for ids in batch_ids)
padded = [ids + [0] * (max_len - len(ids)) for ids in batch_ids]
batch_tensor = torch.tensor(padded)

print(batch_tensor.shape)  # (3, max_len)
```