### Install infini-transformer-pytorch

Source: https://github.com/lucidrains/infini-transformer-pytorch/blob/main/README.md

Install the library using pip.

```bash
$ pip install infini-transformer-pytorch
```

--------------------------------

### Install Infini-Transformer via pip

Source: https://context7.com/lucidrains/infini-transformer-pytorch/llms.txt

Use this command to install the package in your environment.

```bash
pip install infini-transformer-pytorch
```

--------------------------------

### Setup InfiniTransformerWrapper for generation

Source: https://context7.com/lucidrains/infini-transformer-pytorch/llms.txt

Initializes the wrapper for autoregressive text generation tasks.

```python
import torch
from infini_transformer_pytorch import InfiniTransformer, InfiniTransformerWrapper

# Initialize model and wrapper
model = InfiniTransformer(
    num_tokens=256,
    dim=512,
    depth=8,
    dim_head=128,
    heads=8,
    use_mem_delta_rule=True
)

wrapper = InfiniTransformerWrapper(
    model,
    segment_length=512,
    detach_mems_every_num_segments=2
).cuda()
```

--------------------------------

### Complete Training Example for Infini-Transformer

Source: https://context7.com/lucidrains/infini-transformer-pytorch/llms.txt

Demonstrates a full training loop for an Infini-Transformer on character-level text prediction using gradient accumulation and validation. Requires PyTorch and the infini-transformer-pytorch library. Ensure data is loaded and preprocessed appropriately.

```python
import torch
from torch.optim import Adam
from torch.utils.data import DataLoader, Dataset
from infini_transformer_pytorch import InfiniTransformer, InfiniTransformerWrapper

# Configuration
NUM_BATCHES = 10000
BATCH_SIZE = 4
GRADIENT_ACCUMULATE_EVERY = 4
LEARNING_RATE = 2e-4
VALIDATE_EVERY = 100
SEQ_LEN = 1024
SEGMENT_LENGTH = 128

# Simple dataset class
class TextDataset(Dataset):
    def __init__(self, data, seq_len):
        self.data = data
        self.seq_len = seq_len

    def __getitem__(self, index):
        start = torch.randint(0, len(self.data) - self.seq_len, (1,)) 
        return self.data[start:start + self.seq_len].long()

    def __len__(self):
        return len(self.data) // self.seq_len

# Initialize model with learned delta updates
model = InfiniTransformer(
    num_tokens=256,
    dim=512,
    depth=8,
    dim_head=64,
    heads=8,
    use_mem_delta_rule=True,
    learned_delta_update=True
)

wrapper = InfiniTransformerWrapper(
    model,
    segment_length=SEGMENT_LENGTH,
    detach_mems_every_num_segments=2
).cuda()

# Mock training data (replace with real data)
train_data = torch.randint(0, 256, (100000,))
val_data = torch.randint(0, 256, (10000,))

train_loader = DataLoader(TextDataset(train_data, SEQ_LEN), batch_size=BATCH_SIZE, shuffle=True)
val_loader = DataLoader(TextDataset(val_data, SEQ_LEN), batch_size=1)

# Optimizer
optimizer = Adam(model.parameters(), lr=LEARNING_RATE)

# Training loop
for batch_idx, batch in enumerate(train_loader):
    if batch_idx >= NUM_BATCHES:
        break

    batch = batch.cuda()

    # Gradient accumulation
    for _ in range(GRADIENT_ACCUMULATE_EVERY):
        loss = wrapper(
            batch,
            backward=True,
            grad_accum_scale=1.0 / GRADIENT_ACCUMULATE_EVERY
        )

    # Gradient clipping and optimization step
    torch.nn.utils.clip_grad_norm_(model.parameters(), 0.5)
    optimizer.step()
    optimizer.zero_grad()

    if batch_idx % VALIDATE_EVERY == 0:
        with torch.no_grad():
            wrapper.eval()
            val_batch = next(iter(val_loader)).cuda()
            val_loss = wrapper(val_batch)
            print(f"Batch {batch_idx} - Train loss: {loss.item():.4f}, Val loss: {val_loss.item():.4f}")
            wrapper.train()
```

--------------------------------

### Generate Sequence Without Prompt

Source: https://context7.com/lucidrains/infini-transformer-pytorch/llms.txt

Generates sequences starting from random input. Useful for unconditional generation or when no starting context is available.

```python
# Generate without prompt (random start)
output_random = wrapper.generate(
    seq_len=1024,
    batch_size=4,              # Generate 4 samples
    temperature=1.0
)
print(f"Random generation shape: {output_random.shape}")  # torch.Size([4, 1024])
```

--------------------------------

### Initialize and run InfiniTransformer

Source: https://context7.com/lucidrains/infini-transformer-pytorch/llms.txt

Demonstrates model instantiation and manual memory management across segments.

```python
import torch
from infini_transformer_pytorch import InfiniTransformer

# Initialize the Infini-Transformer model
transformer = InfiniTransformer(
    num_tokens=256,           # Vocabulary size
    dim=512,                  # Model dimension
    depth=8,                  # Number of transformer layers
    dim_head=128,             # Dimension per attention head (high capacity recommended)
    heads=8,                  # Number of attention heads
    use_mem_delta_rule=True,  # Enable delta rule for memory updates
    attn_dropout=0.1,         # Attention dropout rate
    ff_dropout=0.1,           # Feed-forward dropout rate
    ff_mult=4,                # Feed-forward dimension multiplier
    learned_delta_update=False  # Whether to learn delta update weights
)

# Create input sequence (batch_size=1, seq_len=1024)
x = torch.randint(0, 256, (1, 1024))

# Forward pass without creating new memories (builds cached key-values)
logits1, cached_kv1, mem1 = transformer(x, return_new_memories=False)
print(f"Logits shape: {logits1.shape}")  # torch.Size([1, 1024, 256])

# Continue with another segment, passing previous memories
logits2, cached_kv2, mem2 = transformer(x, past_memories=mem1, return_new_memories=False)

# Create new compressed memories from the current segment
logits3, cached_kv3, mem3 = transformer(x, past_memories=mem2, return_new_memories=True)
print(f"New memories created with {len(mem3)} layer memories")
```

--------------------------------

### Initialize and Use FastweightMemory

Source: https://context7.com/lucidrains/infini-transformer-pytorch/llms.txt

Initializes the FastweightMemory module and demonstrates creating and updating memories using keys and values. The delta rule can be enabled for efficient updates.

```python
import torch
from infini_transformer_pytorch.infini_transformer import FastweightMemory, Memories

# Initialize fastweight memory module
mem_module = FastweightMemory(
    heads=8,                      # Number of attention heads
    head_gate_init_value=10.0,    # Initial gate value (sigmoid applied)
    use_mem_delta_rule=True       # Enable delta rule for updates
)

# Simulated keys and values from attention (batch=2, heads=8, seq=128, dim=64)
keys = torch.randn(2, 8, 128, 64)
values = torch.randn(2, 8, 128, 64)

# Create initial memories
memories = mem_module.create_new_memories(keys, values, past_memories=None)
print(f"Memory KV shape: {memories.kv_mem.shape}")    # torch.Size([2, 8, 64, 64])
print(f"Memory norm shape: {memories.k_norm.shape}")  # torch.Size([2, 8, 64])

# Update memories with new segment (delta rule applied)
new_keys = torch.randn(2, 8, 128, 64)
new_values = torch.randn(2, 8, 128, 64)
updated_memories = mem_module.create_new_memories(new_keys, new_values, past_memories=memories)

# Retrieve and combine with attention output
queries = torch.randn(2, 8, 128, 64)
attention_out = torch.randn(2, 8, 128, 64)
combined_out = mem_module.retrieve_and_add_to_output(attention_out, queries, updated_memories)
print(f"Combined output shape: {combined_out.shape}")  # torch.Size([2, 8, 128, 64])
```

--------------------------------

### Generate Sequence with Prompt

Source: https://context7.com/lucidrains/infini-transformer-pytorch/llms.txt

Generates a sequence using a provided prompt and nucleus sampling. Ensure the prompt is a torch tensor on the correct device.

```python
prompt = torch.randint(0, 256, (1, 100)).cuda()  # Starting prompt

# Generate using nucleus sampling (default)
output = wrapper.generate(
    seq_len=8192,              # Total sequence length to generate
    prompt=prompt,             # Starting prompt
    temperature=0.9,           # Sampling temperature
    filter_kwargs=dict(thres=0.9),  # Top-p threshold
    exclude_prompt=True        # Exclude prompt from output
)
print(f"Generated shape: {output.shape}")  # torch.Size([1, 8091])
```

--------------------------------

### Basic InfiniTransformer Usage

Source: https://github.com/lucidrains/infini-transformer-pytorch/blob/main/README.md

Instantiate and use the InfiniTransformer model for processing sequences. It can handle past memories for recurrent processing.

```python
import torch
from infini_transformer_pytorch import InfiniTransformer

transformer = InfiniTransformer(
    num_tokens = 256,
    dim = 512,
    depth = 8,
    dim_head = 128,  # high head dimension may be part of the reason they got good results (kv has high capacity)
    heads = 8,
    use_mem_delta_rule = True
)

x = torch.randint(0, 256, (1, 1024))

logits1, _, mem1 = transformer(x, return_new_memories = False)
logits2, _, mem2 = transformer(x, past_memories = mem1, return_new_memories = False)
logits3, _, mem3 = transformer(x, past_memories = mem2, return_new_memories = True)
```

--------------------------------

### Train Autoregressive enwik8

Source: https://github.com/lucidrains/infini-transformer-pytorch/blob/main/README.md

Execute the training script for the autoregressive enwik8 task.

```bash
$ python train.py
```

--------------------------------

### Access TransformerReturn NamedTuple

Source: https://context7.com/lucidrains/infini-transformer-pytorch/llms.txt

Demonstrates how to initialize the InfiniTransformer and access its outputs, which are returned as a TransformerReturn namedtuple. This tuple contains logits, cached key-values, and memories.

```python
import torch
from infini_transformer_pytorch import InfiniTransformer
from infini_transformer_pytorch.infini_transformer import TransformerReturn, Memories

# Initialize model
transformer = InfiniTransformer(
    num_tokens=256,
    dim=512,
    depth=4,
    dim_head=64,
    heads=8,
    use_mem_delta_rule=True
)

x = torch.randint(0, 256, (1, 256))

# Forward returns a TransformerReturn namedtuple
result: TransformerReturn = transformer(x, return_new_memories=True)

# Access individual components
logits = result.logits              # Shape: (batch, seq_len, num_tokens)
cached_kvs = result.cached_kvs      # List of cached key-values per layer (None when returning memories)
past_memories = result.past_memories  # List of Memories per layer

print(f"Logits: {result.logits.shape}")
print(f"Number of layer memories: {len(result.past_memories)}")
```

--------------------------------

### Training with InfiniTransformerWrapper

Source: https://github.com/lucidrains/infini-transformer-pytorch/blob/main/README.md

Use InfiniTransformerWrapper for training transformers with recurrence. It automatically handles segmentation and gradient accumulation.

```python
import torch

from infini_transformer_pytorch import (
    InfiniTransformer,
    InfiniTransformerWrapper
)

# model and wrapper

model = InfiniTransformer(
    num_tokens = 256,
    dim = 512,
    depth = 8,
    dim_head = 128,
    heads = 8,
    use_mem_delta_rule = True
)

wrapper = InfiniTransformerWrapper(
    model,
    segment_length = 512,
    detach_mems_every_num_segments = 2 # greater than 1 so the network can learn how to 'write' to the fast weight memories
).cuda()

# mock input

seq = torch.randint(0, 256, (2, 10000)).cuda() # can be arbitrarily long sequence

# training

loss = wrapper(
    seq,
    backward = True # will automatically segment and accumulate gradients when it detaches the memories
)

# after much data...

# calculating eval loss

with torch.no_grad():
    wrapper.eval()
    eval_loss = wrapper(seq)

# generating is as easy as

output = wrapper.generate(seq_len = 8192, prompt = seq[:, :1])

output.shape # (2, 8192 - 1)
```

--------------------------------

### Train with InfiniTransformerWrapper

Source: https://context7.com/lucidrains/infini-transformer-pytorch/llms.txt

Handles automatic segmentation and gradient accumulation for training on long sequences.

```python
import torch
from infini_transformer_pytorch import InfiniTransformer, InfiniTransformerWrapper

# Create the base model
model = InfiniTransformer(
    num_tokens=256,
    dim=512,
    depth=8,
    dim_head=128,
    heads=8,
    use_mem_delta_rule=True,
    learned_delta_update=True
)

# Wrap with training helper
wrapper = InfiniTransformerWrapper(
    model,
    segment_length=512,                    # Length of each segment for memory compression
    detach_mems_every_num_segments=2,      # Detach gradients every N segments (allows learning to write to memory)
    ignore_index=-1                        # Index to ignore in loss calculation
).cuda()

# Create arbitrarily long input sequence
seq = torch.randint(0, 256, (2, 10000)).cuda()  # batch_size=2, seq_len=10000

# Training with automatic gradient accumulation
loss = wrapper(
    seq,
    backward=True,           # Automatically perform backward pass with gradient accumulation
    grad_accum_scale=1.0     # Scale factor for gradient accumulation
)
print(f"Training loss: {loss.item()}")

# Evaluation mode
with torch.no_grad():
    wrapper.eval()
    eval_loss = wrapper(seq)
    print(f"Evaluation loss: {eval_loss.item()}")
```

--------------------------------

### Iterate Through Past Memories

Source: https://context7.com/lucidrains/infini-transformer-pytorch/llms.txt

Iterates through the past memories of a result, printing the shapes of KV memory and K norm for each layer. Useful for inspecting memory states after processing.

```python
for i, mem in enumerate(result.past_memories):
    print(f"Layer {i} - KV memory: {mem.kv_mem.shape}, K norm: {mem.k_norm.shape}")
```

--------------------------------

### Detach Gradients from Memories

Source: https://context7.com/lucidrains/infini-transformer-pytorch/llms.txt

Detaches gradients from memory tensors in-place, crucial for truncated backpropagation through time. This prevents gradients from flowing to previous segments during training.

```python
import torch
from infini_transformer_pytorch import InfiniTransformer
from infini_transformer_pytorch.infini_transformer import detach_memories_, detach_cached_kv_

# Initialize model
transformer = InfiniTransformer(
    num_tokens=256,
    dim=512,
    depth=4,
    dim_head=64,
    heads=8,
    use_mem_delta_rule=True
)

# Process first segment and get memories
x1 = torch.randint(0, 256, (1, 512))
logits1, cached_kv, memories = transformer(x1, return_new_memories=True)

# Detach memories for truncated BPTT (prevents gradients from flowing to previous segments)
detach_memories_(memories)
print("Memories detached from computation graph")

# Process second segment with detached memories
x2 = torch.randint(0, 256, (1, 512))
logits2, new_cached_kv, new_memories = transformer(
    x2,
    past_memories=memories,
    return_new_memories=True,
    detach_memories=True  # Can also use built-in detach flag
)

# Detach cached key-values when using inference mode with caching
x3 = torch.randint(0, 256, (1, 512))
logits3, cached_kv3, mems3 = transformer(x3, return_new_memories=False)
detach_cached_kv_(cached_kv3)
print("Cached key-values detached")
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.