### Install the package

Source: https://github.com/lucidrains/autoregressive-diffusion-pytorch/blob/main/README.md

Install the library via pip.

```bash
$ pip install autoregressive-diffusion-pytorch
```

--------------------------------

### Initialize Dataset and Model for Image Generation

Source: https://context7.com/lucidrains/autoregressive-diffusion-pytorch/llms.txt

Initializes the Oxford Flowers dataset and the ImageAutoregressiveFlow model for image generation tasks. Ensure the dataset and model parameters are configured according to your requirements.

```python
dataset = OxfordFlowersDataset(image_size = 64)

model = ImageAutoregressiveFlow(
    model = dict(
        dim = 1024,
        depth = 8,
        heads = 8,
        mlp_depth = 4,
        decoder_kwargs = dict(
            rotary_pos_emb = True
        )
    ),
    image_size = 64,
    patch_size = 8,
    model_output_clean = True
)
```

--------------------------------

### Initialize and train ImageAutoregressiveFlow

Source: https://context7.com/lucidrains/autoregressive-diffusion-pytorch/llms.txt

Configure the flow matching model for image generation and execute a training step.

```python
import torch
from autoregressive_diffusion_pytorch import ImageAutoregressiveFlow

# Initialize for image generation with flow matching
model = ImageAutoregressiveFlow(
    model = dict(
        dim = 1024,
        depth = 8,
        heads = 8,
        mlp_depth = 4,
        mlp_width = 1024,
        decoder_kwargs = dict(
            rotary_pos_emb = True  # Use rotary positional embeddings
        )
    ),
    image_size = 64,
    patch_size = 8,
    channels = 3,
    model_output_clean = True,  # Output in x-space
    train_max_noise = 0.0       # Optional: add noise during training (0 to 1)
)

# Training
images = torch.randn(4, 3, 64, 64)
loss = model(images)
loss.backward()

# Sampling
model.eval()
with torch.no_grad():
    generated = model.sample(batch_size = 4)
    print(generated.shape)  # torch.Size([4, 3, 64, 64])
```

--------------------------------

### Train Image Generation Model

Source: https://context7.com/lucidrains/autoregressive-diffusion-pytorch/llms.txt

Sets up and initiates the training process for an image generation model using the ImageTrainer. Configure training steps, learning rate, and batch size as needed.

```python
trainer = ImageTrainer(
    model,
    dataset = dataset,
    num_train_steps = 1_000_000,
    learning_rate = 7e-5,
    batch_size = 32
)

trainer()
```

--------------------------------

### Initialize and Train ImageAutoregressiveDiffusion

Source: https://context7.com/lucidrains/autoregressive-diffusion-pytorch/llms.txt

This wrapper handles image-to-patch tokenization for autoregressive generation. Ensure the image size is divisible by the patch size.

```python
import torch
from autoregressive_diffusion_pytorch import ImageAutoregressiveDiffusion

# Initialize for 64x64 images with 8x8 patches (64 tokens)
model = ImageAutoregressiveDiffusion(
    model = dict(
        dim = 1024,           # Transformer dimension
        depth = 12,           # Transformer depth
        heads = 12,           # Attention heads
        dim_head = 64,        # Dimension per head
        mlp_depth = 3,        # Denoiser MLP depth
        mlp_width = 1024      # Denoiser MLP width
    ),
    image_size = 64,          # Input image size
    patch_size = 8,           # Patch size (image_size must be divisible by patch_size)
    channels = 3              # Number of image channels
)

# Training: compute loss on a batch of images
images = torch.randn(4, 3, 64, 64)  # (batch, channels, height, width)
loss = model(images)
loss.backward()

# Sampling: generate new images
model.eval()
with torch.no_grad():
    generated_images = model.sample(batch_size = 4)
    print(generated_images.shape)  # torch.Size([4, 3, 64, 64])
    # Output is normalized to [0, 1] range
```

--------------------------------

### Initialize and use AutoregressiveDiffusion

Source: https://github.com/lucidrains/autoregressive-diffusion-pytorch/blob/main/README.md

Basic usage for sequence-based autoregressive diffusion models.

```python
import torch
from autoregressive_diffusion_pytorch import AutoregressiveDiffusion

model = AutoregressiveDiffusion(
    dim_input = 512,
    dim = 1024,
    max_seq_len = 32,
    depth = 8,
    mlp_depth = 3,
    mlp_width = 1024
)

seq = torch.randn(3, 32, 512)

loss = model(seq)
loss.backward()

sampled = model.sample(batch_size = 3)

assert sampled.shape == seq.shape
```

--------------------------------

### Initialize and use ImageAutoregressiveDiffusion

Source: https://github.com/lucidrains/autoregressive-diffusion-pytorch/blob/main/README.md

Usage for image generation where images are treated as a sequence of tokens.

```python
import torch
from autoregressive_diffusion_pytorch import ImageAutoregressiveDiffusion

model = ImageAutoregressiveDiffusion(
    model = dict(
        dim = 1024,
        depth = 12,
        heads = 12,
    ),
    image_size = 64,
    patch_size = 8
)

images = torch.randn(3, 3, 64, 64)

loss = model(images)
loss.backward()

sampled = model.sample(batch_size = 3)

assert sampled.shape == images.shape
```

--------------------------------

### Train models with ImageTrainer

Source: https://context7.com/lucidrains/autoregressive-diffusion-pytorch/llms.txt

Orchestrate training with distributed support, EMA, and automatic checkpointing.

```python
import torch
from autoregressive_diffusion_pytorch import (
    ImageDataset,
    ImageAutoregressiveFlow,
    ImageTrainer
)

# Prepare dataset
dataset = ImageDataset(
    folder = '/path/to/images',
    image_size = 128,
    augment_horizontal_flip = True
)

# Create model
model = ImageAutoregressiveFlow(
    model = dict(
        dim = 512,
        depth = 8,
        heads = 8
    ),
    image_size = 128,
    patch_size = 16
)

# Initialize trainer
trainer = ImageTrainer(
    model = model,
    dataset = dataset,
    num_train_steps = 70_000,          # Total training steps
    learning_rate = 3e-4,              # Learning rate
    batch_size = 16,                   # Batch size per GPU
    checkpoints_folder = './checkpoints',  # Checkpoint directory
    results_folder = './results',          # Generated samples directory
    save_results_every = 100,          # Save samples every N steps
    checkpoint_every = 1000,           # Save checkpoint every N steps
    num_samples = 16,                  # Number of samples to generate
    ema_kwargs = dict(
        beta = 0.9999                  # EMA decay rate
    )
)

# Start training (call the trainer)
trainer()
```

--------------------------------

### Load images with ImageDataset

Source: https://context7.com/lucidrains/autoregressive-diffusion-pytorch/llms.txt

Utility for loading and preprocessing images from a local directory.

```python
from pathlib import Path
from autoregressive_diffusion_pytorch import ImageDataset

# Load images from a folder
dataset = ImageDataset(
    folder = '/path/to/your/images',  # Folder containing images
    image_size = 128,                  # Target image size
    exts = ['jpg', 'jpeg', 'png'],     # Supported extensions
    augment_horizontal_flip = True,    # Random horizontal flip
    convert_image_to = 'RGB'           # Convert to RGB
)

# Access dataset
print(f'Dataset size: {len(dataset)}')
image_tensor = dataset[0]  # Returns tensor of shape (3, 128, 128)
```

--------------------------------

### Wrap HuggingFace datasets for training

Source: https://context7.com/lucidrains/autoregressive-diffusion-pytorch/llms.txt

Implement a custom dataset class to interface HuggingFace datasets with the trainer.

```python
import torchvision.transforms as T
from torch.utils.data import Dataset
from datasets import load_dataset

from autoregressive_diffusion_pytorch import ImageAutoregressiveFlow, ImageTrainer

# Custom dataset wrapper for HuggingFace dataset
class OxfordFlowersDataset(Dataset):
    def __init__(self, image_size):
        self.ds = load_dataset('nelorth/oxford-flowers')['train']
        self.transform = T.Compose([
            T.Resize((image_size, image_size)),
            T.PILToTensor()
        ])

    def __len__(self):
        return len(self.ds)

    def __getitem__(self, idx):
        pil = self.ds[idx]['image']
        tensor = self.transform(pil)
        return tensor / 255.0  # Normalize to [0, 1]
```

--------------------------------

### Initialize and Train AutoregressiveDiffusion

Source: https://context7.com/lucidrains/autoregressive-diffusion-pytorch/llms.txt

Use this model for generic continuous sequence generation. It requires defining transformer architecture parameters and diffusion sampling settings.

```python
import torch
from autoregressive_diffusion_pytorch import AutoregressiveDiffusion

# Initialize the model
model = AutoregressiveDiffusion(
    dim_input = 512,      # Input token dimension
    dim = 1024,           # Transformer hidden dimension
    max_seq_len = 32,     # Maximum sequence length
    depth = 8,            # Transformer decoder depth
    heads = 8,            # Number of attention heads
    dim_head = 64,        # Dimension per attention head
    mlp_depth = 3,        # MLP denoiser depth
    mlp_width = 1024,     # MLP hidden width
    diffusion_kwargs = dict(
        clamp_during_sampling = True,  # Clamp outputs to [-1, 1]
        num_sample_steps = 32          # Diffusion sampling steps
    )
)

# Training: compute loss on a batch of sequences
seq = torch.randn(4, 32, 512)  # (batch, seq_len, dim)
loss = model(seq)
loss.backward()

# Sampling: generate new sequences autoregressively
model.eval()
with torch.no_grad():
    sampled = model.sample(batch_size = 4)
    print(sampled.shape)  # torch.Size([4, 32, 512])

# Sampling with prompt: continue from a partial sequence
prompt = torch.randn(2, 8, 512)  # 8 tokens as prompt
with torch.no_grad():
    completed = model.sample(batch_size = 2, prompt = prompt)
    print(completed.shape)  # torch.Size([2, 32, 512])
```

--------------------------------

### Generate sequences using ODE integration

Source: https://context7.com/lucidrains/autoregressive-diffusion-pytorch/llms.txt

Perform sampling from a trained model using ODE integration within a no-gradient context.

```python
model.eval()
with torch.no_grad():
    sampled = model.sample(batch_size = 4)
    print(sampled.shape)  # torch.Size([4, 32, 512])
```

--------------------------------

### Use MLP denoiser network

Source: https://context7.com/lucidrains/autoregressive-diffusion-pytorch/llms.txt

Initialize and perform a forward pass with the internal MLP denoiser.

```python
import torch
from autoregressive_diffusion_pytorch import MLP

# Create standalone MLP denoiser
mlp = MLP(
    dim_cond = 256,      # Conditioning dimension
    dim_input = 512,     # Input/output dimension
    depth = 3,           # Number of MLP blocks
    width = 1024,        # Hidden layer width
    dropout = 0.0        # Dropout rate
)

# Forward pass
noised = torch.randn(4, 512)           # Noised input
times = torch.rand(4)                   # Time/noise level (0 to 1)
cond = torch.randn(4, 256)             # Conditioning embedding

denoised = mlp(noised, times = times, cond = cond)
print(denoised.shape)  # torch.Size([4, 512])
```

--------------------------------

### Initialize and Train AutoregressiveFlow

Source: https://context7.com/lucidrains/autoregressive-diffusion-pytorch/llms.txt

Use this model for flow-matching based generation. It supports ODE integration and axial positional embeddings.

```python
import torch
from autoregressive_diffusion_pytorch import AutoregressiveFlow

# Initialize the flow-based model
model = AutoregressiveFlow(
    dim = 1024,               # Transformer dimension
    dim_input = 512,          # Input token dimension
    max_seq_len = 32,         # Can be int or tuple for axial positions
    depth = 8,                # Transformer depth
    heads = 8,                # Attention heads
    dim_head = 64,            # Dimension per head
    mlp_depth = 3,            # Flow MLP depth
    mlp_width = 1024,         # Flow MLP width
    model_output_clean = True,  # Output in x-space (recommended)
    flow_kwargs = dict(
        atol = 1e-5,          # ODE solver absolute tolerance
        rtol = 1e-5,          # ODE solver relative tolerance
        method = 'midpoint'   # ODE solver method
    )
)

# Training: compute flow matching loss
seq = torch.randn(4, 32, 512)
loss = model(seq)
loss.backward()
```

--------------------------------

### Train image models

Source: https://github.com/lucidrains/autoregressive-diffusion-pytorch/blob/main/README.md

Use ImageTrainer to train image autoregressive models on a dataset.

```python
import torch

from autoregressive_diffusion_pytorch import (
    ImageDataset,
    ImageAutoregressiveDiffusion,
    ImageTrainer
)

dataset = ImageDataset(
    '/path/to/your/images',
    image_size = 128
)

model = ImageAutoregressiveDiffusion(
    model = dict(
        dim = 512
    ),
    image_size = 128,
    patch_size = 16
)

trainer = ImageTrainer(
    model = model,
    dataset = dataset
)

trainer()
```

```python
import torch

from autoregressive_diffusion_pytorch import (
    ImageDataset,
    ImageTrainer,
    ImageAutoregressiveFlow,
)

dataset = ImageDataset(
    '/path/to/your/images',
    image_size = 128
)

model = ImageAutoregressiveFlow(
    model = dict(
        dim = 512
    ),
    image_size = 128,
    patch_size = 16
)

trainer = ImageTrainer(
    model = model,
    dataset = dataset
)

trainer()
```

--------------------------------

### Citations

Source: https://github.com/lucidrains/autoregressive-diffusion-pytorch/blob/main/README.md

BibTeX citations for the underlying research.

```bibtex
@article{Li2024AutoregressiveIG,
    title   = {Autoregressive Image Generation without Vector Quantization},
    author  = {Tianhong Li and Yonglong Tian and He Li and Mingyang Deng and Kaiming He},
    journal = {ArXiv},
    year    = {2024},
    volume  = {abs/2406.11838},
    url     = {https://api.semanticscholar.org/CorpusID:270560593}
}
```

```bibtex
@article{Wu2023ARDiffusionAD,
    title     = {AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation},
    author    = {Tong Wu and Zhihao Fan and Xiao Liu and Yeyun Gong and Yelong Shen and Jian Jiao and Haitao Zheng and Juntao Li and Zhongyu Wei and Jian Guo and Nan Duan and Weizhu Chen},
    journal   = {ArXiv},
    year    = {2023},
    volume  = {abs/2305.09515},
    url     = {https://api.semanticscholar.org/CorpusID:258714669}
}
```

```bibtex
@article{Karras2022ElucidatingTD,
    title   = {Elucidating the Design Space of Diffusion-Based Generative Models},
    author  = {Tero Karras and Miika Aittala and Timo Aila and Samuli Laine},
    journal = {ArXiv},
    year    = {2022},
    volume  = {abs/2206.00364},
    url     = {https://api.semanticscholar.org/CorpusID:249240415}
}
```

```bibtex
@article{Liu2022FlowSA,
    title   = {Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow},
    author  = {Xingchao Liu and Chengyue Gong and Qiang Liu},
    journal = {ArXiv},
    year    = {2022},
    volume  = {abs/2209.03003},
    url     = {https://api.semanticscholar.org/CorpusID:252111177}
}
```

```bibtex
@article{Esser2024ScalingRF,
    title   = {Scaling Rectified Flow Transformers for High-Resolution Image Synthesis},
    author  = {Patrick Esser and Sumith Kulal and A. Blattmann and Rahim Entezari and Jonas Muller and Harry Saini and Yam Levi and Dominik Lorenz and Axel Sauer and Frederic Boesel and Dustin Podell and Tim Dockhorn and Zion English and Kyle Lacey and Alex Goodwin and Yannik Marek and Robin Rombach},
    journal = {ArXiv},
    year    = {2024},
    volume  = {abs/2403.03206},
    url     = {https://api.semanticscholar.org/CorpusID:268247980}
}
```

```bibtex
@misc{li2025basicsletdenoisinggenerative,
    title   = {Back to Basics: Let Denoising Generative Models Denoise}, 
    author  = {Tianhong Li and Kaiming He},
    year    = {2025},
    eprint  = {2511.13720},
    archivePrefix = {arXiv},
    primaryClass = {cs.CV},
    url     = {https://arxiv.org/abs/2511.13720}, 
}
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.