### Install Mogrifier

Source: https://github.com/lucidrains/mogrifier/blob/master/README.md

Install the package via pip.

```bash
$ pip install mogrifier
```

--------------------------------

### Initialize and Use Mogrifier

Source: https://context7.com/lucidrains/mogrifier/llms.txt

Basic setup and forward pass for the Mogrifier module.

```python
import torch
from mogrifier import Mogrifier

# Initialize Mogrifier with basic configuration
mogrify = Mogrifier(
    dim=512,           # dimension of the input vector
    dim_hidden=256,    # dimension of the hidden vector (defaults to dim if not specified)
    iters=5,           # number of iterations (paper recommends 5 for LSTM)
    factorize_k=16     # optional: factorize weights into (dim x k) and (k x dim)
)

# Create sample input and hidden tensors
# x: input tensor with shape (batch, sequence_length, dim)
# h: hidden state tensor with shape (batch, sequence_length, dim_hidden)
x = torch.randn(1, 16, 512)
h = torch.randn(1, 16, 256)

# Forward pass returns modulated versions of both tensors
out, hidden_out = mogrify(x, h)

print(f"Input shape: {x.shape}")        # torch.Size([1, 16, 512])
print(f"Output shape: {out.shape}")     # torch.Size([1, 16, 512])
print(f"Hidden input shape: {h.shape}") # torch.Size([1, 16, 256])
print(f"Hidden output shape: {hidden_out.shape}")  # torch.Size([1, 16, 256])
```

--------------------------------

### Initialize and Use Mogrifier

Source: https://github.com/lucidrains/mogrifier/blob/master/README.md

Configure the Mogrifier with dimensions and iterations, then pass input tensors to perform the modulation.

```python
import torch
from mogrifier import Mogrifier

mogrify = Mogrifier(
    dim = 512,
    dim_hidden = 256,
    iters = 5,          # number of iterations, defaults to 5 as paper recommended for LSTM
    factorize_k = 16    # factorize weight matrices into (dim x k) and (k x dim), if specified
)

x = torch.randn(1, 16, 512)
h = torch.randn(1, 16, 256)

out, hidden_out = mogrify(x, h) # (1, 16, 512), (1, 16, 256)

assert out.shape == x.shape
assert hidden_out.shape == h.shape
```

--------------------------------

### Use Mogrifier with Matching Dimensions

Source: https://context7.com/lucidrains/mogrifier/llms.txt

Simplified configuration when input and hidden dimensions are identical.

```python
import torch
from mogrifier import Mogrifier

# Simple configuration with matching dimensions
mogrify = Mogrifier(
    dim=256,
    iters=5
)

# Both tensors have the same dimension
x = torch.randn(32, 10, 256)  # batch=32, seq_len=10, dim=256
h = torch.randn(32, 10, 256)

modulated_x, modulated_h = mogrify(x, h)
# Both outputs maintain their original shapes
```

--------------------------------

### Apply Factorized Weight Matrices

Source: https://context7.com/lucidrains/mogrifier/llms.txt

Reduce memory usage by decomposing weight matrices using factorize_k.

```python
import torch
from mogrifier import Mogrifier

# Factorized configuration for memory efficiency
mogrify = Mogrifier(
    dim=1024,
    dim_hidden=512,
    iters=5,
    factorize_k=32,           # factorize Q projection weights
    hidden_factorize_k=16     # optionally use different k for R projection
)

x = torch.randn(8, 20, 1024)
h = torch.randn(8, 20, 512)

out, hidden_out = mogrify(x, h)

# Parameter reduction example:
# Without factorization: 1024 * 512 = 524,288 params per projection
# With factorize_k=32: (512 * 32) + (32 * 1024) = 49,152 params (~90% reduction)
```

--------------------------------

### Adjusting Mogrifier Iterations

Source: https://context7.com/lucidrains/mogrifier/llms.txt

Control the number of iterations to balance inference speed and modulation depth.

```python
# Override with fewer iterations for faster inference
out_fast, h_fast = mogrify(x, h, iters=3)

# Override with more iterations for potentially better modulation
out_deep, h_deep = mogrify(x, h, iters=7)
```

--------------------------------

### Integrating Mogrifier with PyTorch LSTM

Source: https://context7.com/lucidrains/mogrifier/llms.txt

Implement a custom LSTM module that applies the Mogrifier to inputs and hidden states before the LSTM cell step.

```python
import torch
import torch.nn as nn
from mogrifier import Mogrifier

class MogrifierLSTM(nn.Module):
    def __init__(self, input_dim, hidden_dim, iters=5, factorize_k=None):
        super().__init__()
        self.hidden_dim = hidden_dim

        # Mogrifier to modulate input and hidden state
        self.mogrifier = Mogrifier(
            dim=input_dim,
            dim_hidden=hidden_dim,
            iters=iters,
            factorize_k=factorize_k
        )

        # Standard LSTM cell
        self.lstm_cell = nn.LSTMCell(input_dim, hidden_dim)

    def forward(self, x, hidden=None):
        batch_size, seq_len, _ = x.shape

        if hidden is None:
            h = torch.zeros(batch_size, self.hidden_dim, device=x.device)
            c = torch.zeros(batch_size, self.hidden_dim, device=x.device)
        else:
            h, c = hidden

        outputs = []
        for t in range(seq_len):
            x_t = x[:, t, :]

            # Apply mogrifier before LSTM
            x_mod, h_mod = self.mogrifier(x_t.unsqueeze(1), h.unsqueeze(1))
            x_mod = x_mod.squeeze(1)
            h_mod = h_mod.squeeze(1)

            # LSTM step with modulated inputs
            h, c = self.lstm_cell(x_mod, (h_mod, c))
            outputs.append(h)

        return torch.stack(outputs, dim=1), (h, c)

# Usage example
model = MogrifierLSTM(input_dim=512, hidden_dim=256, iters=5, factorize_k=16)

x = torch.randn(8, 20, 512)  # batch=8, seq_len=20, input_dim=512
output, (h_n, c_n) = model(x)

print(f"Output shape: {output.shape}")  # torch.Size([8, 20, 256])
print(f"Final hidden: {h_n.shape}")     # torch.Size([8, 256])
print(f"Final cell: {c_n.shape}")       # torch.Size([8, 256])
```

--------------------------------

### Mogrifier Citation

Source: https://github.com/lucidrains/mogrifier/blob/master/README.md

BibTeX entry for the Mogrifier LSTM paper.

```bibtex
@inproceedings{Melis2020Mogrifier,
    title   = {Mogrifier LSTM},
    author  = {Gábor Melis and Tomáš Kočiský and Phil Blunsom},
    booktitle = {International Conference on Learning Representations},
    year    = {2020},
    url     = {https://openreview.net/forum?id=SJe5P6EYvS}
}
```

--------------------------------

### Override Iteration Count

Source: https://context7.com/lucidrains/mogrifier/llms.txt

Adjust modulation depth dynamically during the forward pass.

```python
import torch
from mogrifier import Mogrifier

mogrify = Mogrifier(
    dim=512,
    dim_hidden=256,
    iters=5  # default iterations
)

x = torch.randn(1, 16, 512)
h = torch.randn(1, 16, 256)

# Use default iterations (5)
out_default, h_default = mogrify(x, h)
```

--------------------------------

### Broadcast Hidden State

Source: https://context7.com/lucidrains/mogrifier/llms.txt

Automatically broadcast 2D hidden states to match 3D input sequence lengths.

```python
import torch
from mogrifier import Mogrifier

mogrify = Mogrifier(
    dim=512,
    dim_hidden=256,
    iters=5
)

# 3D input tensor (batch, sequence, features)
x = torch.randn(4, 32, 512)

# 2D hidden state (batch, features) - will be broadcast to match sequence length
h = torch.randn(4, 256)

out, hidden_out = mogrify(x, h)

print(f"Input shape: {x.shape}")            # torch.Size([4, 32, 512])
print(f"Hidden input shape: {h.shape}")     # torch.Size([4, 256])
print(f"Output shape: {out.shape}")         # torch.Size([4, 32, 512])
print(f"Hidden output shape: {hidden_out.shape}")  # torch.Size([4, 32, 256])
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.