### Install Lumiere-Pytorch

Source: https://github.com/lucidrains/lumiere-pytorch/blob/main/README.md

Install the package via pip.

```bash
$ pip install lumiere-pytorch
```

--------------------------------

### Temporal Downsampling and Upsampling Example

Source: https://context7.com/lucidrains/lumiere-pytorch/llms.txt

Demonstrates the usage of mp_downsample and mp_upsample for adjusting video frame counts. Ensure the input video frame count is divisible by 2^n, where n is the number of temporal downsample layers.

```python
video_8 = torch.randn(2, 64, 8, 32, 32)
video_4 = mp_downsample(video_8)
print(f"Downsample: {video_8.shape} -> {video_4.shape}")
```

```python
video_restored = mp_upsample(video_4)
print(f"Upsample: {video_4.shape} -> {video_restored.shape}")
```

--------------------------------

### Initialize and Run Lumiere Model

Source: https://github.com/lucidrains/lumiere-pytorch/blob/main/README.md

Configure the KarrasUnet and MPLumiere modules, then perform a forward pass with a noised video tensor.

```python
import torch
from lumiere_pytorch import MPLumiere

from denoising_diffusion_pytorch import KarrasUnet

karras_unet = KarrasUnet(
    image_size = 256,
    dim = 8,
    channels = 3,
    dim_max = 768,
)

lumiere = MPLumiere(
    karras_unet,
    image_size = 256,
    unet_time_kwarg = 'time',
    conv_module_names = [
        'downs.1',
        'ups.1',
        'downs.2',
        'ups.2',
    ],
    attn_module_names = [
        'mids.0'
    ],
    upsample_module_names = [
        'ups.2',
        'ups.1',
    ],
    downsample_module_names = [
        'downs.1',
        'downs.2'
    ]
)

noised_video = torch.randn(2, 3, 8, 256, 256)
time = torch.ones(2,)

denoised_video = lumiere(noised_video, time = time)

assert noised_video.shape == denoised_video.shape
```

--------------------------------

### Initialize and Use MPLumiere for Video Generation

Source: https://context7.com/lucidrains/lumiere-pytorch/llms.txt

Wraps a pretrained text-to-image U-Net with magnitude-preserving temporal layers for video generation. Requires specifying modules for temporal augmentation and configuration for convolution/attention inflation.

```python
import torch
from lumiere_pytorch import MPLumiere
from denoising_diffusion_pytorch import KarrasUnet

# Initialize the base text-to-image U-Net
karras_unet = KarrasUnet(
    image_size=256,
    dim=8,
    channels=3,
    dim_max=768,
)

# Wrap with MPLumiere for video generation
lumiere = MPLumiere(
    karras_unet,
    image_size=256,
    unet_time_kwarg='time',  # The kwarg name for timestep in your UNet
    conv_module_names=[
        'downs.1',
        'ups.1',
        'downs.2',
        'ups.2',
    ],
    attn_module_names=[
        'mids.0'
    ],
    upsample_module_names=[
        'ups.2',
        'ups.1',
    ],
    downsample_module_names=[
        'downs.1',
        'downs.2'
    ]
)

# Create noised video input: (batch, channels, time, height, width)
noised_video = torch.randn(2, 3, 8, 256, 256)
time = torch.ones(2,)  # Diffusion timestep

# Denoise the video
denois ed_video = lumiere(noised_video, time=time)

print(f"Input shape: {noised_video.shape}")   # torch.Size([2, 3, 8, 256, 256])
print(f"Output shape: {denoised_video.shape}") # torch.Size([2, 3, 8, 256, 256])
assert noised_video.shape == denoised_video.shape
```

--------------------------------

### Training Temporal Parameters with MPLumiere

Source: https://context7.com/lucidrains/lumiere-pytorch/llms.txt

Sets up and trains a Lumiere model, freezing pretrained text-to-image components and optimizing only temporal parameters. Requires KarrasUnet and MPLumiere from their respective libraries. The optimizer is configured to only use Lumiere's parameters.

```python
import torch
from lumiere_pytorch import MPLumiere
from denoising_diffusion_pytorch import KarrasUnet

# Setup model
karras_unet = KarrasUnet(image_size=64, dim=8, channels=3, dim_max=256)

lumiere = MPLumiere(
    karras_unet,
    image_size=64,
    unet_time_kwarg='time',
    conv_module_names=['downs.1', 'ups.1'],
    attn_module_names=['mids.0'],
    downsample_module_names=['downs.1'],
    upsample_module_names=['ups.1']
)

# Only temporal parameters are trainable
temporal_params = list(lumiere.parameters())
print(f"Number of trainable parameter groups: {len(temporal_params)}")

# Setup optimizer with only temporal parameters
optimizer = torch.optim.AdamW(lumiere.parameters(), lr=1e-4)

# Training loop example
lumiere.train()
for step in range(100):
    # Generate random video and timestep
    video = torch.randn(2, 3, 4, 64, 64)
    timestep = torch.rand(2)

    # Forward pass
    denoised = lumiere(video, time=timestep)

    # Compute loss (simplified - actual training uses proper diffusion loss)
    loss = (denoised - video).pow(2).mean()

    # Backward pass
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

    if step % 20 == 0:
        print(f"Step {step}, Loss: {loss.item():.4f}")
```

--------------------------------

### Initialize and Use Lumiere for Standard Video Generation

Source: https://context7.com/lucidrains/lumiere-pytorch/llms.txt

Wraps a text-to-image model with standard temporal convolution and attention layers for video generation. Requires specifying modules for temporal augmentation and configuration for inflation blocks.

```python
import torch
from lumiere_pytorch import Lumiere
from denoising_diffusion_pytorch import KarrasUnet

# Initialize base U-Net
unet = KarrasUnet(
    image_size=128,
    dim=32,
    channels=3,
    dim_max=512,
)

# Create Lumiere wrapper with temporal modules
video_model = Lumiere(
    unet,
    image_size=128,
    unet_time_kwarg='time',
    conv_module_names=[
        'downs.1',
        'downs.2',
        'ups.1',
        'ups.2',
    ],
    attn_module_names=[
        'mids.0'
    ],
    downsample_module_names=[
        'downs.1',
        'downs.2'
    ],
    upsample_module_names=[
        'ups.2',
        'ups.1',
    ],
    # Optional: customize inflation block parameters
    conv_inflation_kwargs={
        'conv2d_kernel_size': 3,
        'conv1d_kernel_size': 3,
    },
    attn_inflation_kwargs={
        'depth': 1,
        'prenorm': True,
        'residual_attn': True,
    }
)

# Process video through the model
# Time dimension must be divisible by 2^(num_downsample_layers)
video_input = torch.randn(1, 3, 4, 128, 128)  # 4 frames
timestep = torch.tensor([0.5])

output = video_model(video_input, time=timestep)
print(f"Denoised video shape: {output.shape}")  # torch.Size([1, 3, 4, 128, 128])
```

--------------------------------

### Initialize MP temporal sampling modules

Source: https://context7.com/lucidrains/lumiere-pytorch/llms.txt

Initializes magnitude-preserving temporal downsampling and upsampling modules with dirac-initialized convolutions.

```python
import torch
from lumiere_pytorch import MPTemporalDownsample, MPTemporalUpsample

# Create MP temporal modules
mp_downsample = MPTemporalDownsample(dim=64, channel_last=False)
mp_upsample = MPTemporalUpsample(dim=64, channel_last=False)
```

--------------------------------

### Initialize and use MPConvolutionInflationBlock

Source: https://context7.com/lucidrains/lumiere-pytorch/llms.txt

Uses forced weight normalization and magnitude-preserving additions for improved training dynamics.

```python
import torch
from lumiere_pytorch import MPConvolutionInflationBlock

# Create MP convolution inflation block
mp_conv_block = MPConvolutionInflationBlock(
    dim=64,
    conv2d_kernel_size=3,
    conv1d_kernel_size=3,
    channel_last=False,
    time_dim=8,
    mp_add_t=0.3,    # Magnitude-preserving residual weight (default 0.3)
    dropout=0.1      # Dropout rate
)

# Process video features
video = torch.randn(2, 64, 8, 32, 32)
output = mp_conv_block(video)

print(f"Input: {video.shape}")   # torch.Size([2, 64, 8, 32, 32])
print(f"Output: {output.shape}") # torch.Size([2, 64, 8, 32, 32])
```

--------------------------------

### Initialize ConvolutionInflationBlock

Source: https://context7.com/lucidrains/lumiere-pytorch/llms.txt

Adds spatial and temporal convolutions to process video features with optional time dimension specification.

```python
import torch
from lumiere_pytorch import ConvolutionInflationBlock

# Create convolution inflation block
conv_block = ConvolutionInflationBlock(
    dim=64,                    # Number of channels
    conv2d_kernel_size=3,      # Spatial convolution kernel
    conv1d_kernel_size=3,      # Temporal convolution kernel
    channel_last=False,        # Channel dimension position
    time_dim=8                 # Number of time frames (optional)
)

```

--------------------------------

### Initialize and use MPAttentionInflationBlock

Source: https://context7.com/lucidrains/lumiere-pytorch/llms.txt

Provides magnitude-preserving attention with pixel normalization for stable video diffusion training.

```python
import torch
from lumiere_pytorch import MPAttentionInflationBlock

# Create MP attention inflation block
mp_attn_block = MPAttentionInflationBlock(
    dim=128,
    depth=1,
    time_dim=8,
    channel_last=False,
    mp_add_t=0.3,    # Magnitude-preserving residual weight
    dropout=0.0      # Dropout rate for attention
)

# Process video features
video = torch.randn(2, 128, 8, 16, 16)
output = mp_attn_block(video)

print(f"Input: {video.shape}")   # torch.Size([2, 128, 8, 16, 16])
print(f"Output: {output.shape}") # torch.Size([2, 128, 8, 16, 16])
```

--------------------------------

### Initialize and use AttentionInflationBlock

Source: https://context7.com/lucidrains/lumiere-pytorch/llms.txt

Applies temporal self-attention across video frames. Requires specifying feature dimensions and temporal configuration.

```python
import torch
from lumiere_pytorch import AttentionInflationBlock

# Create attention inflation block
attn_block = AttentionInflationBlock(
    dim=128,              # Feature dimension
    depth=2,              # Number of attention layers
    prenorm=True,         # Apply RMSNorm before attention
    residual_attn=True,   # Use residual connections
    time_dim=8,           # Number of time frames
    channel_last=False    # Channel dimension position
)

# Process video features
video_features = torch.randn(2, 128, 8, 16, 16)
output = attn_block(video_features)

print(f"Input shape: {video_features.shape}")   # torch.Size([2, 128, 8, 16, 16])
print(f"Output shape: {output.shape}")          # torch.Size([2, 128, 8, 16, 16])
```

--------------------------------

### Process video tensor with convolution block

Source: https://context7.com/lucidrains/lumiere-pytorch/llms.txt

Demonstrates processing a video tensor using a convolution block, supporting both standard (batch, channels, time, height, width) and flattened (batch*time, channels, h, w) input formats.

```python
# Process video tensor: (batch, channels, time, height, width)
video_features = torch.randn(2, 64, 8, 32, 32)
output = conv_block(video_features)

print(f"Input: {video_features.shape}")   # torch.Size([2, 64, 8, 32, 32])
print(f"Output: {output.shape}")          # torch.Size([2, 64, 8, 32, 32])

# Alternative: process flattened batch*time tensor with batch_size
flat_features = torch.randn(16, 64, 32, 32)  # (batch*time, channels, h, w)
output_flat = conv_block(flat_features, batch_size=2)
print(f"Flat output: {output_flat.shape}")  # torch.Size([16, 64, 32, 32])
```

--------------------------------

### BibTeX Citations

Source: https://github.com/lucidrains/lumiere-pytorch/blob/main/README.md

Citations for the Lumiere and Karras papers.

```bibtex
@inproceedings{BarTal2024LumiereAS,
    title   = {Lumiere: A Space-Time Diffusion Model for Video Generation},
    author  = {Omer Bar-Tal and Hila Chefer and Omer Tov and Charles Herrmann and Roni Paiss and Shiran Zada and Ariel Ephrat and Junhwa Hur and Yuanzhen Li and Tomer Michaeli and Oliver Wang and Deqing Sun and Tali Dekel and Inbar Mosseri},
    year    = {2024},
    url     = {https://api.semanticscholar.org/CorpusID:267095113}
}
```

```bibtex
@article{Karras2023AnalyzingAI,
    title   = {Analyzing and Improving the Training Dynamics of Diffusion Models},
    author  = {Tero Karras and Miika Aittala and Jaakko Lehtinen and Janne Hellsten and Timo Aila and Samuli Laine},
    journal = {ArXiv},
    year    = {2023},
    volume  = {abs/2312.02696},
    url     = {https://api.semanticscholar.org/CorpusID:265659032}
}
```

--------------------------------

### Upsample temporal dimension

Source: https://context7.com/lucidrains/lumiere-pytorch/llms.txt

Increases the temporal dimension by a factor of 2 using transposed 1D convolution.

```python
import torch
from lumiere_pytorch import TemporalUpsample

# Create temporal upsample module
upsample = TemporalUpsample(
    dim=64,              # Number of channels
    channel_last=False,  # Channel dimension first
    time_dim=None        # Will infer from input
)

# Upsample from 4 frames to 8 frames
video = torch.randn(2, 64, 4, 32, 32)  # 4 time frames
upsampled = upsample(video)

print(f"Input: {video.shape}")       # torch.Size([2, 64, 4, 32, 32])
print(f"Output: {upsampled.shape}")  # torch.Size([2, 64, 8, 32, 32])
```

--------------------------------

### Downsample temporal dimension

Source: https://context7.com/lucidrains/lumiere-pytorch/llms.txt

Reduces the temporal dimension by a factor of 2 using strided 1D convolution.

```python
import torch
from lumiere_pytorch import TemporalDownsample

# Create temporal downsample module
downsample = TemporalDownsample(
    dim=64,              # Number of channels
    channel_last=False,  # Channel dimension first
    time_dim=None        # Will infer from input
)

# Downsample from 8 frames to 4 frames
video = torch.randn(2, 64, 8, 32, 32)  # 8 time frames
downsampled = downsample(video)

print(f"Input: {video.shape}")        # torch.Size([2, 64, 8, 32, 32])
print(f"Output: {downsampled.shape}") # torch.Size([2, 64, 4, 32, 32])
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.