### Install DeepCache

Source: https://context7.com/horseee/deepcache/llms.txt

Install the DeepCache library using pip.

```bash
pip install DeepCache
```

--------------------------------

### Install DDPM Requirements

Source: https://github.com/horseee/deepcache/blob/master/experiments/README.md

Install necessary dependencies for running DDPM experiments.

```bash
pip install accelerate lmdb scipy diffusers pytorch_fid
```

--------------------------------

### Train Autoencoder Models

Source: https://github.com/horseee/deepcache/blob/master/experiments/ldm/README.md

Starts training for a KL-regularized autoencoder using a specified configuration file.

```bash
CUDA_VISIBLE_DEVICES=<GPU_ID> python main.py --base configs/autoencoder/<config_spec>.yaml -t --gpus 0,
```

--------------------------------

### Train Latent Diffusion Models

Source: https://github.com/horseee/deepcache/blob/master/experiments/ldm/README.md

Starts training for an LDM using a specified configuration file.

```shell
CUDA_VISIBLE_DEVICES=<GPU_ID> python main.py --base configs/latent-diffusion/<config_spec>.yaml -t --gpus 0,
```

--------------------------------

### Install Stable Diffusion Requirements

Source: https://github.com/horseee/deepcache/blob/master/experiments/README.md

Install dependencies required for Stable Diffusion experiments.

```bash
pip install diffusers==0.24.0 transformers open_clip_torch
```

--------------------------------

### Install DeepCache Dependencies

Source: https://github.com/horseee/deepcache/blob/master/README.md

Install the necessary libraries for DeepCache, including diffusers and transformers.

```bash
pip install diffusers==0.24.0 transformers
```

--------------------------------

### Run DDPM Experiments with DeepCache

Source: https://context7.com/horseee/deepcache/llms.txt

Install dependencies and execute DDPM experiments on CIFAR-10, LSUN Bedroom, and LSUN Church datasets, followed by FID score calculation.

```bash
# Install requirements
pip install accelerate lmdb scipy diffusers pytorch_fid

# Run DDPM with DeepCache on CIFAR-10
cd experiments/ddpm/
accelerate launch ddim.py \
    --config configs/cifar10.yml \
    --exp deepcache \
    --fid \
    --timesteps 100 \
    --eta 0 \
    --ni \
    --use_pretrained \
    --cache \
    --cache_interval 5 \
    --branch 2 \
    --skip_type quad

# Run on LSUN Bedroom dataset
accelerate launch ddim.py \
    --config configs/bedroom.yml \
    --exp deepcache \
    --fid \
    --timesteps 100 \
    --eta 0 \
    --ni \
    --use_pretrained \
    --cache \
    --cache_interval 5 \
    --branch 2

# Run on LSUN Church dataset
accelerate launch ddim.py \
    --config configs/church.yml \
    --exp deepcache \
    --fid \
    --timesteps 100 \
    --eta 0 \
    --ni \
    --use_pretrained \
    --cache \
    --cache_interval 5 \
    --branch 2

# Calculate FID score
python fid.py --path runtime_log/your_experiment/images npz/cifar10_fid.npz
```

--------------------------------

### Stable Diffusion v1.5 Output Log

Source: https://github.com/horseee/deepcache/blob/master/README.md

Example output log showing the performance comparison between baseline and DeepCache for Stable Diffusion v1.5.

```bash
2023-12-03 16:18:13,636 - INFO - Loaded safety_checker as StableDiffusionSafetyChecker from `safety_checker` subfolder of runwayml/stable-diffusion-v1-5.
2023-12-03 16:18:13,699 - INFO - Loaded vae as AutoencoderKL from `vae` subfolder of runwayml/stable-diffusion-v1-5.
Loading pipeline components...: 100%|██████████████████████████████████████████████████████████████████| 7/7 [00:01<00:00,  5.88it/s]
2023-12-03 16:18:22,837 - INFO - Running baseline...
100%|████████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [00:03<00:00, 15.33it/s]
2023-12-03 16:18:26,174 - INFO - Baseline: 3.34 seconds
2023-12-03 16:18:26,174 - INFO - Running DeepCache...
100%|████████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [00:01<00:00, 34.06it/s]
2023-12-03 16:18:27,718 - INFO - DeepCache: 1.54 seconds
2023-12-03 16:18:27,935 - INFO - Saved to output.png. Done!
```

--------------------------------

### Stable Diffusion XL Output Log

Source: https://github.com/horseee/deepcache/blob/master/README.md

Example output log showing the performance comparison between baseline and DeepCache for Stable Diffusion XL.

```bash
Loading pipeline components...: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:01<00:00,  6.62it/s]
2023-12-06 01:44:28,578 - INFO - Running baseline...
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [00:17<00:00,  2.93it/s]
2023-12-06 01:44:46,095 - INFO - Baseline: 17.52 seconds
Loading pipeline components...: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:00<00:00,  8.06it/s]
2023-12-06 01:45:02,865 - INFO - Running DeepCache...
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [00:06<00:00,  8.01it/s]
2023-12-06 01:45:09,573 - INFO - DeepCache: 6.71 seconds
2023-12-06 01:45:10,678 - INFO - Saved to output.png. Done!
```

--------------------------------

### Run Retrieval-Based Sampling

Source: https://github.com/horseee/deepcache/blob/master/experiments/ldm/README.md

Starts the retrieval-based text-guided sampling process with visual nearest neighbors.

```python
python scripts/knn2img.py  --prompt "a happy pineapple" --use_neighbors --knn <number_of_neighbors> 
```

--------------------------------

### Command-Line Usage for DeepCache

Source: https://context7.com/horseee/deepcache/llms.txt

Install DeepCache and related dependencies using pip. Then, run the main script with the --model_type argument to accelerate various diffusion models from the command line.

```bash
# Install dependencies
pip install DeepCache diffusers==0.24.0 transformers

# Stable Diffusion XL
python main.py --model_type sdxl

# Stable Diffusion v1.5
python main.py --model_type sd1.5

# Stable Diffusion v2.1
python main.py --model_type sd2.1

# Stable Video Diffusion
python main.py --model_type svd
```

--------------------------------

### Install Additional Python Packages

Source: https://github.com/horseee/deepcache/blob/master/experiments/ldm/README.md

Installs necessary Python packages for retrieval-augmented diffusion models into the active 'ldm' conda environment. Ensure the 'ldm' environment is activated before running these commands.

```shell
pip install transformers==4.19.2 scann kornia==0.6.4 torchmetrics==0.6.0
pip install git+https://github.com/arogozhnikov/einops.git
```

--------------------------------

### Create and Activate Conda Environment

Source: https://github.com/horseee/deepcache/blob/master/experiments/ldm/README.md

Creates a conda environment named 'ldm' from the 'environment.yaml' file and activates it. Ensure you have conda installed and the 'environment.yaml' file present.

```bash
conda env create -f environment.yaml
conda activate ldm
```

--------------------------------

### FLOPs Calculation Utility with DeepCache

Source: https://context7.com/horseee/deepcache/llms.txt

Calculate the computational cost (MACs/FLOPs) of U-Net models with and without DeepCache using the count_ops_and_params utility. Provide example inputs matching the model's expected format. Set layer_wise=True for a detailed per-layer breakdown.

```python
import torch
from DeepCache.flops import count_ops_and_params
from diffusers import UNet2DConditionModel

# Load U-Net model
unet = UNet2DConditionModel.from_pretrained(
    "runwayml/stable-diffusion-v1-5",
    subfolder="unet",
    torch_dtype=torch.float16
).to("cuda:0")

# Prepare example inputs matching the model's expected format
example_inputs = {
    'sample': torch.randn(1, 4, 64, 64, dtype=torch.float16).to("cuda:0"),
    'timestep': torch.tensor([1], dtype=torch.float16).to("cuda:0"),
    'encoder_hidden_states': torch.randn(1, 77, 768, dtype=torch.float16).to("cuda:0"),
}

# Calculate MACs and parameters
macs, nparams = count_ops_and_params(unet, example_inputs=example_inputs, layer_wise=False)

print(f"#Params: {nparams/1e6:.4f} M")
print(f"#MACs: {macs/1e9:.4f} G")

# Enable layer_wise=True for detailed per-layer breakdown
macs, nparams = count_ops_and_params(unet, example_inputs=example_inputs, layer_wise=True)
# This will print MACs and params for each layer in the model
```

--------------------------------

### Sample Inpainting Model

Source: https://github.com/horseee/deepcache/blob/master/experiments/ldm/README.md

Runs the inpainting script with specified input and output directories.

```python
python scripts/inpaint.py --indir data/inpainting_examples/ --outdir outputs/inpainting_results
```

--------------------------------

### Download Text-to-Image Weights

Source: https://github.com/horseee/deepcache/blob/master/experiments/ldm/README.md

Downloads the pre-trained model weights for the text-to-image generation pipeline.

```bash
mkdir -p models/ldm/text2img-large/
wget -O models/ldm/text2img-large/model.ckpt https://ommer-lab.com/files/latent-diffusion/nitro/txt2img-f8-large/model.ckpt
```

--------------------------------

### Run Text-to-Image Sampling

Source: https://github.com/horseee/deepcache/blob/master/experiments/ldm/README.md

Executes the text-to-image sampling script with specified prompt and generation parameters.

```python
python scripts/txt2img.py --prompt "a virus monster is playing guitar, oil on canvas" --ddim_eta 0.0 --n_samples 4 --n_iter 4 --scale 5.0  --ddim_steps 50
```

--------------------------------

### Download Inpainting Model Weights

Source: https://github.com/horseee/deepcache/blob/master/experiments/ldm/README.md

Downloads the pre-trained weights for the inpainting model.

```bash
wget -O models/ldm/inpainting_big/last.ckpt https://heibox.uni-heidelberg.de/f/4d9ac7ea40c64582b7c9/?dl=1
```

--------------------------------

### Sample Unconditional Models

Source: https://github.com/horseee/deepcache/blob/master/experiments/ldm/README.md

Executes the sampling script for unconditional LDMs with specific model and hardware configurations.

```shell
CUDA_VISIBLE_DEVICES=<GPU_ID> python scripts/sample_diffusion.py -r models/ldm/<model_spec>/model.ckpt -l <logdir> -n <\#samples> --batch_size <batch_size> -c <\#ddim steps> -e <\#eta>
```

--------------------------------

### Download RDM Retrieval Databases

Source: https://github.com/horseee/deepcache/blob/master/experiments/ldm/README.md

Downloads and extracts the ArtBench and OpenImages retrieval databases required for RDM conditioning.

```bash
mkdir -p data/rdm/retrieval_databases
wget -O data/rdm/retrieval_databases/artbench.zip https://ommer-lab.com/files/rdm/artbench_databases.zip
wget -O data/rdm/retrieval_databases/openimages.zip https://ommer-lab.com/files/rdm/openimages_database.zip
unzip data/rdm/retrieval_databases/artbench.zip -d data/rdm/retrieval_databases/
unzip data/rdm/retrieval_databases/openimages.zip -d data/rdm/retrieval_databases/
```

--------------------------------

### Run Stable Video Diffusion with DeepCache

Source: https://github.com/horseee/deepcache/blob/master/README.md

Executes the DeepCache acceleration script for Stable Video Diffusion.

```bash
python stable_video_diffusion.py
```

```bash
Loading pipeline components...: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00,  8.36it/s]
2023-12-21 04:56:47,329 - INFO - Running baseline...
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 25/25 [01:27<00:00,  3.49s/it]
2023-12-21 04:58:26,121 - INFO - Origin: 98.66 seconds
Loading pipeline components...: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 10.59it/s]
2023-12-21 04:58:27,202 - INFO - Running DeepCache...
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 25/25 [00:49<00:00,  1.96s/it]
2023-12-21 04:59:26,607 - INFO - DeepCache: 59.31 seconds
```

--------------------------------

### Download Pretrained RDM Model Weights

Source: https://github.com/horseee/deepcache/blob/master/experiments/ldm/README.md

Downloads the pre-trained weights for the retrieval-augmented diffusion model (RDM) and saves them to the specified directory. This command assumes you are in the project's root directory and have the necessary permissions.

```bash
mkdir -p models/rdm/rdm768x768/
wget -O models/rdm/rdm768x768/model.ckpt https://ommer-lab.com/files/rdm/model.ckpt
```

--------------------------------

### Download Pretrained Models

Source: https://github.com/horseee/deepcache/blob/master/experiments/ldm/README.md

Executes a shell script to download and extract all listed LDM model checkpoints.

```shell
bash scripts/download_models.sh
```

--------------------------------

### Run Stable Diffusion v2.1 with DeepCache

Source: https://github.com/horseee/deepcache/blob/master/README.md

Executes the DeepCache acceleration script for the specified Stable Diffusion v2.1 model.

```bash
python stable_diffusion.py --model stabilityai/stable-diffusion-2-1
```

```bash
2023-12-03 16:21:17,858 - INFO - Loaded feature_extractor as CLIPImageProcessor from `feature_extractor` subfolder of stabilityai/stable-diffusion-2-1.
2023-12-03 16:21:17,864 - INFO - Loaded scheduler as DDIMScheduler from `scheduler` subfolder of stabilityai/stable-diffusion-2-1.
Loading pipeline components...: 100%|██████████████████████████████████████████████████████████████████| 6/6 [00:01<00:00,  5.35it/s]
2023-12-03 16:21:49,770 - INFO - Running baseline...
100%|████████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [00:14<00:00,  3.42it/s]
2023-12-03 16:22:04,551 - INFO - Baseline: 14.78 seconds
2023-12-03 16:22:04,551 - INFO - Running DeepCache...
100%|████████████████████████████████████████████████████████████████████████████████████████████████| 50/08 [00:08<00:00,  6.10it/s]
2023-12-03 16:22:12,911 - INFO - DeepCache: 8.36 seconds
2023-12-03 16:22:13,417 - INFO - Saved to output.png. Done!
```

--------------------------------

### Accelerate Stable Video Diffusion with DeepCache

Source: https://context7.com/horseee/deepcache/llms.txt

Apply DeepCache acceleration to Stable Video Diffusion pipelines for faster video generation. Initialize DeepCache for video diffusion and configure parameters.

```python
import torch
from diffusers import StableVideoDiffusionPipeline
from diffusers.utils import load_image, export_to_video
from DeepCache import DeepCacheSDHelper

# Load SVD pipeline
pipe = StableVideoDiffusionPipeline.from_pretrained(
    "stabilityai/stable-video-diffusion-img2vid-xt",
    torch_dtype=torch.float16
).to("cuda:0")

# Initialize DeepCache for video diffusion
helper = DeepCacheSDHelper(pipe=pipe)
helper.set_params(
    cache_interval=3,
    cache_branch_id=0,
)
helper.enable()

# Load conditioning image
image = load_image("https://example.com/input_image.png")
image = image.resize((1024, 576))

# Generate video frames
generator = torch.manual_seed(42)
frames = pipe(
    image,
    decode_chunk_size=8,
    generator=generator,
).frames[0]

helper.disable()

# Export to video file
export_to_video(frames, "generated_video.mp4", fps=7)
```

--------------------------------

### Run General SD Script

Source: https://github.com/horseee/deepcache/blob/master/README.md

Execute the general script for Stable Diffusion models. Specify the model type using the --model_type argument.

```bash
python main.py --model_type sdxl
```

--------------------------------

### Run Stable Diffusion XL with DeepCache

Source: https://github.com/horseee/deepcache/blob/master/README.md

Generate images using the Stable Diffusion XL model with DeepCache. The model is specified via the --model argument.

```bash
python stable_diffusion_xl.py --model stabilityai/stable-diffusion-xl-base-1.0
```

--------------------------------

### Image-to-Image Pipeline Acceleration with DeepCache

Source: https://context7.com/horseee/deepcache/llms.txt

Accelerate Stable Diffusion img2img transformations using DeepCache. Initialize the pipeline and DeepCache helper, configure cache parameters, enable the helper, perform transformations, and then disable the helper.

```python
import torch
from diffusers import StableDiffusionImg2ImgPipeline
from DeepCache import DeepCacheSDHelper
from PIL import Image

# Load img2img pipeline
pipe = StableDiffusionImg2ImgPipeline.from_pretrained(
    "runwayml/stable-diffusion-v1-5",
    torch_dtype=torch.float16
).to("cuda:0")

# Initialize DeepCache
helper = DeepCacheSDHelper(pipe=pipe)
helper.set_params(
    cache_interval=3,
    cache_branch_id=0,
)
helper.enable()

# Load source image
init_image = Image.open("source_image.png").convert("RGB").resize((512, 512))

# Transform image with prompt guidance
prompt = "a fantasy landscape, trending on artstation"
image = pipe(
    prompt=prompt,
    image=init_image,
    strength=0.75,
    num_inference_steps=50,
).images[0]

helper.disable()
image.save("img2img_output.png")
```

--------------------------------

### Download Pretrained Models

Source: https://github.com/horseee/deepcache/blob/master/experiments/ldm/README.md

Executes a shell script to download and extract all available pretrained autoencoding models.

```shell
bash scripts/download_first_stages.sh
```

--------------------------------

### Inpainting Pipeline Acceleration with DeepCache

Source: https://context7.com/horseee/deepcache/llms.txt

Enable DeepCache acceleration for Stable Diffusion inpainting pipelines. Initialize DeepCache and configure parameters before loading images for inpainting.

```python
import torch
from diffusers import StableDiffusionInpaintPipeline
from DeepCache import DeepCacheSDHelper
from PIL import Image

# Load inpainting pipeline
pipe = StableDiffusionInpaintPipeline.from_pretrained(
    "runwayml/stable-diffusion-inpainting",
    torch_dtype=torch.float16
).to("cuda:0")

# Initialize DeepCache
helper = DeepCacheSDHelper(pipe=pipe)
helper.set_params(
    cache_interval=3,
    cache_branch_id=0,
)
helper.enable()

# Load image and mask
init_image = Image.open("input_image.png").convert("RGB")
mask_image = Image.open("mask.png").convert("RGB")

```

--------------------------------

### Run Stable Diffusion v1.5 with DeepCache

Source: https://github.com/horseee/deepcache/blob/master/README.md

Generate images using the Stable Diffusion v1.5 model with DeepCache. The model is specified via the --model argument.

```bash
python stable_diffusion.py --model runwayml/stable-diffusion-v1-5
```

--------------------------------

### Run High-Resolution Text-to-Image Sampling

Source: https://github.com/horseee/deepcache/blob/master/experiments/ldm/README.md

Generates images at custom resolutions by adjusting the H and W parameters.

```python
python scripts/txt2img.py --prompt "a sunset behind a mountain range, vector image" --ddim_eta 1.0 --n_samples 1 --n_iter 1 --H 384 --W 1024 --scale 5.0  
```

--------------------------------

### Run Inpainting and Image-to-Image Pipelines

Source: https://context7.com/horseee/deepcache/llms.txt

Execute specific diffusion pipelines using the main script with model type arguments.

```bash
python main.py --model_type sd-inpaint
python main.py --model_type sdxl-inpaint
```

```bash
python main.py --model_type sd-img2img
```

--------------------------------

### Perform Batch Generation and Evaluation

Source: https://context7.com/horseee/deepcache/llms.txt

Generate images using DeepCache or baseline methods and evaluate performance using CLIP scores.

```bash
# Install evaluation dependencies
pip install diffusers==0.24.0 transformers open_clip_torch

# Generate with DeepCache
python experiments/generate.py \
    --dataset coco2017 \
    --layer 0 \
    --block 0 \
    --update_interval 2 \
    --uniform \
    --steps 50 \
    --batch_size 16

# Generate baseline without caching
python experiments/generate.py \
    --dataset coco2017 \
    --original \
    --steps 50 \
    --batch_size 16

# Generate with BK-SDM baseline
python experiments/generate.py \
    --dataset coco2017 \
    --bk base \
    --steps 50 \
    --batch_size 16

# Evaluate CLIP score
python experiments/clip_score.py path/to/saved/images
```

--------------------------------

### Download ScaNN Search Indices

Source: https://github.com/horseee/deepcache/blob/master/experiments/ldm/README.md

Downloads and extracts pre-trained ScaNN search indices for the ArtBench dataset.

```bash
mkdir -p data/rdm/searchers
wget -O data/rdm/searchers/artbench.zip https://ommer-lab.com/files/rdm/artbench_searchers.zip
unzip data/rdm/searchers/artbench.zip -d data/rdm/searchers
```

--------------------------------

### Run Stable Diffusion Generation and Evaluation

Source: https://github.com/horseee/deepcache/blob/master/experiments/README.md

Commands to generate images with DeepCache or baselines and evaluate using CLIP score.

```bash
python generate.py --dataset coco2017 --layer 0 --block 0 --update_interval 2 --uniform --steps 50 --batch_size 16
```

```bash
python generate.py --dataset coco2017  --original --steps 50 --batch_size 16 # For original pipeline
python generate.py --dataset coco2017 --bk base --steps 50 --batch_size 16 # For BK-SDM
```

```bash
python clip_score.py PATH_TO_SAVED_IMAGES
```

--------------------------------

### Enable DeepCache in Stable Diffusion Pipeline

Source: https://github.com/horseee/deepcache/blob/master/README.md

Load a Stable Diffusion pipeline and then integrate DeepCache using the DeepCacheSDHelper. Configure cache parameters like interval and branch ID before enabling the cache.

```python
import torch

# Loading the original pipeline
from diffusers import StableDiffusionPipeline
pipe = StableDiffusionPipeline.from_pretrained('runwayml/stable-diffusion-v1-5', torch_dtype=torch.float16).to("cuda:0")

# Import the DeepCacheSDHelper
from DeepCache import DeepCacheSDHelper
helper = DeepCacheSDHelper(pipe=pipe)
helper.set_params(
    cache_interval=3,
    cache_branch_id=0,
)
helper.enable()

```

--------------------------------

### Run DDPM Sampling and FID Evaluation

Source: https://github.com/horseee/deepcache/blob/master/experiments/README.md

Commands to sample images using DeepCache and evaluate FID scores.

```bash
cd ddpm/
accelerate launch ddim.py --config configs/{DATASET_NAME}.yml --exp deepcache --fid --timesteps 100 --eta 0 --ni --use_pretrained --cache --cache_interval 5 --branch 2
```

```bash
python fid.py --path runtime_log/{YOUR_PATH_FOR_IMAGES}/images npz/cifar10_fid.npz
```

--------------------------------

### Accelerate Stable Diffusion XL with DeepCache

Source: https://context7.com/horseee/deepcache/llms.txt

Use DeepCacheSDHelper to accelerate Stable Diffusion XL pipelines. The interface is the same as for standard Stable Diffusion.

```python
import torch
from diffusers import StableDiffusionXLPipeline
from DeepCache import DeepCacheSDHelper

# Load SDXL pipeline
pipe = StableDiffusionXLPipeline.from_pretrained(
    'stabilityai/stable-diffusion-xl-base-1.0',
    torch_dtype=torch.float16
).to("cuda:0")

# Initialize and configure DeepCache
helper = DeepCacheSDHelper(pipe=pipe)
helper.set_params(
    cache_interval=3,
    cache_branch_id=0,
)
helper.enable()

# Generate with SDXL + DeepCache
prompt = "A cinematic shot of a baby racoon wearing an intricate italian priest robe"
image = pipe(
    prompt,
    num_inference_steps=50,
    guidance_scale=7.5,
).images[0]

helper.disable()
image.save("sdxl_deepcache_output.png")
```

--------------------------------

### Generate Image with DeepCache

Source: https://github.com/horseee/deepcache/blob/master/README.md

Basic usage of the DeepCache pipeline for image generation. Ensure helper is disabled after use.

```python
deepcache_image = pipe(
        prompt,
        output_type='pt'
).images[0]
helper.disable()
```

--------------------------------

### Run RDM Inference with Text Prompt

Source: https://github.com/horseee/deepcache/blob/master/experiments/ldm/README.md

Executes the retrieval-augmented diffusion model (RDM) for image generation using a text prompt. This mode does not require explicit retrieval as it conditions on the CLIP text embedding of the prompt.

```bash
python scripts/knn2img.py  --prompt "a happy bear reading a newspaper, oil on canvas"
```

--------------------------------

### DeepCacheSDHelper for Stable Diffusion

Source: https://context7.com/horseee/deepcache/llms.txt

Accelerate Stable Diffusion pipelines using DeepCacheSDHelper. Configure cache interval and branch ID, then enable caching before generation. Disable caching when done.

```python
import torch
from diffusers import StableDiffusionPipeline
from DeepCache import DeepCacheSDHelper

# Load any Stable Diffusion pipeline
pipe = StableDiffusionPipeline.from_pretrained(
    'runwayml/stable-diffusion-v1-5',
    torch_dtype=torch.float16
).to("cuda:0")

# Initialize the DeepCache helper
helper = DeepCacheSDHelper(pipe=pipe)

# Configure caching parameters
# cache_interval: How often to update the cache (higher = faster but lower quality)
# cache_branch_id: Which U-Net branch to cache (0 = shallowest, higher = deeper)
helper.set_params(
    cache_interval=3,  # Update cache every 3 steps
    cache_branch_id=0,  # Cache at the first skip branch
)

# Enable caching before generation
helper.enable()

# Generate image with DeepCache acceleration
prompt = "a photo of an astronaut riding a horse on mars"
image = pipe(
    prompt,
    num_inference_steps=50,
    output_type='pil'
).images[0]

# Disable caching when done
helper.disable()

# Save the accelerated result
image.save("output_deepcache.png")
```

--------------------------------

### Create OpenImages Search Index

Source: https://github.com/horseee/deepcache/blob/master/experiments/ldm/README.md

Generates the required search index for the OpenImages database, which is necessary for sampling.

```python
python scripts/train_searcher.py
```

--------------------------------

### Configure DeepCache set_params

Source: https://context7.com/horseee/deepcache/llms.txt

Configure DeepCache's caching behavior using set_params. Adjust cache_interval for frequency and cache_branch_id to select which layers to cache. Different branch IDs target different feature depths, affecting speed and quality.

```python
from DeepCache import DeepCacheSDHelper

helper = DeepCacheSDHelper(pipe=pipe)

# Basic configuration
helper.set_params(
    cache_interval=3,    # Update cache every N steps (1 = no caching, higher = more speedup)
    cache_branch_id=0,   # Select which skip branch to cache (0-11 for SD)
    skip_mode='uniform'  # Sampling strategy: 'uniform' for evenly spaced cache updates
)

# cache_branch_id breakdown:
# - Values 0-2: First down block (shallowest features)
# - Values 3-5: Second down block
# - Values 6-8: Third down block
# - Values 9-11: Fourth down block (deepest features)
# Lower values = more aggressive caching, higher speedup, potentially lower quality
# Higher values = more conservative caching, less speedup, better quality preservation

# Example: Conservative caching for higher quality
helper.set_params(
    cache_interval=2,
    cache_branch_id=3,
)

# Example: Aggressive caching for maximum speed
helper.set_params(
    cache_interval=5,
    cache_branch_id=0,
)
```

--------------------------------

### Generate Inpainted Image with DeepCache

Source: https://context7.com/horseee/deepcache/llms.txt

Use DeepCache to accelerate the inpainting process for diffusion models. Ensure the DeepCache helper is enabled before calling the pipeline and disabled afterward.

```python
prompt = "a beautiful garden with flowers"
image = pipe(
    prompt=prompt,
    image=init_image,
    mask_image=mask_image,
    num_inference_steps=50,
).images[0]

helper.disable()
image.save("inpainted_output.png")
```

--------------------------------

### Calculate MACs for DDPM and Stable Diffusion

Source: https://github.com/horseee/deepcache/blob/master/experiments/README.md

Code snippets to calculate model parameters and MACs by inserting them into the denoising loop.

```python
import sys
sys.path.append('../')
from flops import count_ops_and_params
example_inputs = {
    'x': torch.randn(1, 3, self.config.data.image_size, self.config.data.image_size).to(self.device), 
    't': torch.ones(1).to(self.device),
    'prv_f': [torch.randn(1, 256, 16, 16).to(self.device)],
    'branch': 2
}
macs, nparams = count_ops_and_params(model, example_inputs=example_inputs, layer_wise=False)
self.logger.log("#Params: {:.4f} M".format(nparams/1e6))
self.logger.log("#MACs: {:.4f} G".format(macs/1e9))
exit()
```

```python
from ..flops import count_ops_and_params
example_inputs = {
    'sample': latent_model_input, 
    'timestep': t,
    'encoder_hidden_states': prompt_embeds,
    'cross_attention_kwargs': cross_attention_kwargs,
    'replicate_prv_feature': prv_features,
    'quick_replicate': cache_interval>1,
    'cache_layer_id': cache_layer_id,
    'cache_block_id': cache_block_id,
    'return_dict': False,
}
macs, nparams = count_ops_and_params(self.unet, example_inputs=example_inputs, layer_wise=False)
print("#Params: {:.4f} M".format(nparams/1e6))
print("#MACs: {:.4f} G".format(macs/1e9))
exit() 
```

--------------------------------

### Enable/Disable DeepCache Methods

Source: https://context7.com/horseee/deepcache/llms.txt

Control DeepCache activation using enable() and disable() methods. This allows for baseline comparisons and switching caching strategies during runtime. Ensure to disable DeepCache when not needed to revert to normal pipeline operation.

```python
import torch
from diffusers import StableDiffusionPipeline
from DeepCache import DeepCacheSDHelper

pipe = StableDiffusionPipeline.from_pretrained(
    'runwayml/stable-diffusion-v1-5',
    torch_dtype=torch.float16
).to("cuda:0")

helper = DeepCacheSDHelper(pipe=pipe)
helper.set_params(cache_interval=3, cache_branch_id=0)

# Generate without DeepCache (baseline)
baseline_image = pipe("a sunset over mountains", num_inference_steps=50).images[0]
baseline_image.save("baseline.png")

# Enable DeepCache for accelerated generation
helper.enable()
accelerated_image = pipe("a sunset over mountains", num_inference_steps=50).images[0]
accelerated_image.save("accelerated.png")

# Disable to return to normal operation
helper.disable()

# Can re-enable with different parameters
helper.set_params(cache_interval=5, cache_branch_id=0)
helper.enable()
faster_image = pipe("a sunset over mountains", num_inference_steps=50).images[0]
helper.disable()
```

--------------------------------

### ImageNet Directory Structure

Source: https://github.com/horseee/deepcache/blob/master/experiments/ldm/README.md

Expected directory structure for the ImageNet dataset within the cache path.

```text
${XDG_CACHE}/autoencoders/data/ILSVRC2012_{split}/data/
├── n01440764
│   ├── n01440764_10026.JPEG
│   ├── n01440764_10027.JPEG
│   ├── ...
├── n01443537
│   ├── n01443537_10007.JPEG
│   ├── n01443537_10014.JPEG
│   ├── ...
├── ...
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.