### Install Example Dependencies

Source: https://github.com/tencentarc/brushnet/blob/main/examples/consistency_distillation/README.md

Install the specific requirements needed for the example scripts.

```bash
pip install -r requirements.txt
```

--------------------------------

### Install Example Requirements

Source: https://github.com/tencentarc/brushnet/blob/main/examples/custom_diffusion/README.md

After installing the main library, install specific requirements for the example scripts and the clip-retrieval library.

```bash
pip install -r requirements.txt 
pip install clip-retrieval
```

--------------------------------

### Install training dependencies

Source: https://github.com/tencentarc/brushnet/blob/main/docs/source/en/training/wuerstchen.md

Navigate to the Wuerstchen example directory and install the required packages.

```bash
cd examples/wuerstchen/text_to_image
pip install -r requirements.txt
```

--------------------------------

### Install Training Dependencies

Source: https://github.com/tencentarc/brushnet/blob/main/CONTRIBUTING.md

Install all necessary dependencies for a specific training example by referencing its `requirements.txt` file. Ensure this file lists all pip dependencies required for the example to run.

```bash
pip install -r /examples/<your-example-folder>/requirements.txt
```

--------------------------------

### Install SDXL Training Dependencies

Source: https://github.com/tencentarc/brushnet/blob/main/docs/source/en/training/sdxl.md

Navigate to the SDXL example directory and install the specific requirements for the training script.

```bash
cd examples/text_to_image
pip install -r requirements_sdxl.txt
```

--------------------------------

### Install training dependencies

Source: https://github.com/tencentarc/brushnet/blob/main/docs/source/en/training/kandinsky.md

Navigate to the Kandinsky 2.2 example directory and install the required packages.

```bash
cd examples/kandinsky2_2/text_to_image
pip install -r requirements.txt
```

--------------------------------

### Install script dependencies

Source: https://github.com/tencentarc/brushnet/blob/main/docs/source/en/training/overview.md

Navigate to a specific example directory and install the required packages defined in its requirements file.

```bash
cd examples/dreambooth
pip install -r requirements.txt
```

--------------------------------

### Install Training Script Dependencies

Source: https://github.com/tencentarc/brushnet/blob/main/docs/source/en/training/unconditional_training.md

Navigate to the specific example folder and install the required dependencies for the unconditional image generation training script.

```bash
cd examples/unconditional_image_generation
pip install -r requirements.txt
```

--------------------------------

### Install Script Dependencies

Source: https://github.com/tencentarc/brushnet/blob/main/docs/source/en/training/lcm_distill.md

Navigate to the consistency_distillation examples directory and install the required dependencies for the training script. Ensure you are in the correct directory before running this command.

```bash
cd examples/consistency_distillation
pip install -r requirements.txt
```

--------------------------------

### Install Project Dependencies

Source: https://github.com/tencentarc/brushnet/blob/main/README.md

Install the local package and additional requirements for the BrushNet examples.

```bash
pip install -e .
cd examples/brushnet/
pip install -r requirements.txt
```

--------------------------------

### Install Diffusers Training Dependencies

Source: https://github.com/tencentarc/brushnet/blob/main/examples/unconditional_image_generation/README.md

Clone the diffusers repository, install the library from source, and then install the example-specific requirements. This ensures compatibility with the latest example scripts.

```bash
git clone https://github.com/huggingface/diffusers
cd diffusers
pip install .

pip install -r requirements.txt
```

--------------------------------

### Install Custom Diffusion Dependencies

Source: https://github.com/tencentarc/brushnet/blob/main/docs/source/en/training/custom_diffusion.md

Navigate to the custom_diffusion example directory and install its specific dependencies, including clip-retrieval.

```bash
cd examples/custom_diffusion
pip install -r requirements.txt
pip install clip-retrieval
```

--------------------------------

### Install diffusers and dependencies

Source: https://github.com/tencentarc/brushnet/blob/main/examples/research_projects/intel_opts/README.md

Install the necessary libraries for diffusers, transformers, accelerate, scipy, and safetensors. This is a prerequisite for running the optimization examples.

```bash
pip install diffusers transformers accelerate scipy safetensors
```

--------------------------------

### Install Weights and Biases

Source: https://github.com/tencentarc/brushnet/blob/main/examples/kandinsky2_2/text_to_image/README.md

Install the wandb library for monitoring training progress.

```bash
pip install wandb
```

--------------------------------

### InstantID Pipeline Setup

Source: https://github.com/tencentarc/brushnet/blob/main/examples/community/README.md

This snippet prepares the necessary components for the InstantID pipeline, including face analysis models and pipeline configurations. It requires installing opencv-python, transformers, accelerate, and insightface.

```python
# !pip install opencv-python transformers accelerate insightface
import diffusers
from diffusers.utils import load_image
from diffusers.models import ControlNetModel

import cv2
import torch
import numpy as np
from PIL import Image

from insightface.app import FaceAnalysis
from pipeline_stable_diffusion_xl_instantid import StableDiffusionXLInstantIDPipeline, draw_kps

# prepare 'antelopev2' under ./models
# https://github.com/deepinsight/insightface/issues/1896#issuecomment-1023867304
app = FaceAnalysis(name='antelopev2', root='./', providers=['CUDAExecutionProvider', 'CPUExecutionProvider'])
app.prepare(ctx_id=0, det_size=(640, 640))

# prepare models under ./checkpoints
```

--------------------------------

### Install required libraries

Source: https://github.com/tencentarc/brushnet/blob/main/docs/source/en/tutorials/fast_diffusion.md

Install the necessary packages to run the optimization experiments.

```bash
pip install -U diffusers
```

```bash
pip install -U transformers accelerate peft
```

```bash
pip3 install --pre torch --index-url https://download.pytorch.org/whl/nightly/cu121
```

--------------------------------

### Install JAX and Dependencies

Source: https://github.com/tencentarc/brushnet/blob/main/examples/controlnet/README.md

Commands to install JAX and required training libraries on the TPU VM.

```bash
pip install "jax[tpu]==0.4.5" -f https://storage.googleapis.com/jax-releases/libtpu_releases.html
```

```python
import jax
jax.device_count()
```

```bash
git clone https://github.com/huggingface/diffusers
cd diffusers
pip install .
```

```bash
pip install -U -r requirements_flax.txt
```

```bash
pip install wandb
```

--------------------------------

### Install Documentation Dependencies

Source: https://github.com/tencentarc/brushnet/blob/main/docs/README.md

Install the necessary packages to build the documentation from the repository root.

```bash
pip install -e ".[docs]"
```

--------------------------------

### Install RealFill dependencies

Source: https://github.com/tencentarc/brushnet/blob/main/examples/research_projects/realfill/README.md

Navigate to the project directory and install the required Python packages.

```bash
cd realfill
pip install -r requirements.txt
```

--------------------------------

### Install Accelerate

Source: https://github.com/tencentarc/brushnet/blob/main/docs/source/zh/installation.md

Install the 'accelerate' library, which is a prerequisite for installing 🤗 Diffusers from source.

```bash
pip install accelerate
```

--------------------------------

### Install Training Dependencies

Source: https://github.com/tencentarc/brushnet/blob/main/docs/source/en/training/dreambooth.md

Install requirements specific to the chosen framework.

```bash
cd examples/dreambooth
pip install -r requirements.txt
```

```bash
cd examples/dreambooth
pip install -r requirements_flax.txt
```

--------------------------------

### Install Diffusers Training Dependencies

Source: https://github.com/tencentarc/brushnet/blob/main/examples/textual_inversion/README.md

Clone the diffusers repository, install from source, and then install example-specific requirements. Initialize an Accelerate environment for distributed training.

```bash
git clone https://github.com/huggingface/diffusers
cd diffusers
pip install .

```

```bash
pip install -r requirements.txt

```

```bash
accelerate config

```

--------------------------------

### Install required libraries

Source: https://github.com/tencentarc/brushnet/blob/main/docs/source/en/tutorials/using_peft_for_inference.md

Install the necessary packages to run the pipeline and manage adapters.

```bash
!pip install -q transformers accelerate peft diffusers
```

--------------------------------

### Install Diffusers Training Dependencies

Source: https://github.com/tencentarc/brushnet/blob/main/examples/t2i_adapter/README_sdxl.md

Clone the diffusers repository, install it from source, and then install the example-specific requirements for T2I-Adapter training.

```bash
git clone https://github.com/huggingface/diffusers
cd diffusers
pip install -e .
pip install -r requirements.txt
```

--------------------------------

### Install Diffusers and Training Dependencies

Source: https://github.com/tencentarc/brushnet/blob/main/examples/text_to_image/README_sdxl.md

Clone the diffusers repository, install it in editable mode, and then install the specific requirements for SDXL training. Ensure you are in a virtual environment.

```bash
git clone https://github.com/huggingface/diffusers
cd diffusers
pip install -e .
pip install -r examples/text_to_image/requirements_sdxl.txt
```

--------------------------------

### Install ControlNet Libraries

Source: https://github.com/tencentarc/brushnet/blob/main/docs/source/en/using-diffusers/controlnet.md

Install the necessary libraries for using ControlNet with Hugging Face Diffusers. Uncomment the line to install in a Colab environment.

```python
# uncomment to install the necessary libraries in Colab
#!pip install -q diffusers transformers accelerate opencv-python
```

--------------------------------

### Install Watchdog Module

Source: https://github.com/tencentarc/brushnet/blob/main/docs/README.md

Install the watchdog module to enable live previewing of documentation changes.

```bash
pip install watchdog
```

--------------------------------

### Install bitsandbytes library

Source: https://github.com/tencentarc/brushnet/blob/main/docs/source/en/api/pipelines/pixart.md

Install the bitsandbytes library to enable 8-bit precision loading.

```bash
pip install -U bitsandbytes
```

--------------------------------

### Install controlnet-aux

Source: https://github.com/tencentarc/brushnet/blob/main/docs/source/en/using-diffusers/controlnet.md

Install the necessary library for auxiliary conditioning models.

```bash
#!pip install -q controlnet-aux
```

--------------------------------

### Install training dependencies

Source: https://github.com/tencentarc/brushnet/blob/main/docs/source/en/training/t2i_adapters.md

Install the required packages for the T2I-Adapter training scripts.

```bash
cd examples/t2i_adapter
pip install -r requirements.txt
```

--------------------------------

### Install Documentation Builder

Source: https://github.com/tencentarc/brushnet/blob/main/docs/README.md

Install the open-source doc-builder tool required for generating documentation.

```bash
pip install git+https://github.com/huggingface/doc-builder
```

--------------------------------

### Install Optimum for ONNX Runtime

Source: https://github.com/tencentarc/brushnet/blob/main/docs/source/en/optimization/onnx.md

Install the necessary dependencies to enable ONNX Runtime support.

```bash
pip install -q optimum["onnxruntime"]
```

--------------------------------

### Install required libraries

Source: https://github.com/tencentarc/brushnet/blob/main/docs/source/en/using-diffusers/sdxl_turbo.md

Install the necessary dependencies for running SDXL Turbo in a Colab environment.

```python
# uncomment to install the necessary libraries in Colab
#!pip install -q diffusers transformers accelerate
```

--------------------------------

### Install Diffusers and Dependencies

Source: https://github.com/tencentarc/brushnet/blob/main/examples/research_projects/multi_subject_dreambooth/README.md

Clone the diffusers repository, install it in editable mode, and then install the requirements for the multi-subject DreamBooth example.

```bash
git clone https://github.com/huggingface/diffusers
cd diffusers
pip install -e .
cd diffusers/examples/research_projects/multi_subject_dreambooth
pip install -r requirements.txt
```

--------------------------------

### Install InstructPix2Pix Dependencies

Source: https://github.com/tencentarc/brushnet/blob/main/docs/source/en/training/instructpix2pix.md

Navigate to the examples/instruct_pix2pix directory and install the specific dependencies required for the InstructPix2Pix training script.

```bash
cd examples/instruct_pix2pix
pip install -r requirements.txt
```

--------------------------------

### Launch training with custom parameters

Source: https://github.com/tencentarc/brushnet/blob/main/docs/source/en/training/t2i_adapters.md

Example of running the training script with specific arguments like gradient accumulation.

```bash
accelerate launch train_t2i_adapter_sdxl.py \
  ----gradient_accumulation_steps=4
```

--------------------------------

### Setup Default Accelerate Environment

Source: https://github.com/tencentarc/brushnet/blob/main/docs/source/en/training/lora.md

Set up a default 🤗 Accelerate environment without interactive configuration. Useful for automated setups.

```bash
accelerate config default
```

--------------------------------

### Install Diffusers from Source

Source: https://github.com/tencentarc/brushnet/blob/main/examples/dreambooth/README_sdxl.md

Install the diffusers library from source and its training dependencies. This ensures you have the latest example scripts and necessary packages.

```bash
git clone https://github.com/huggingface/diffusers
cd diffusers
pip install -e .
```

```bash
pip install -r requirements_sdxl.txt
```

--------------------------------

### Install RL Requirements for Diffuser

Source: https://github.com/tencentarc/brushnet/blob/main/examples/reinforcement_learning/README.md

Install the required Python packages for running Diffuser examples, including PyTorch, free-mujoco-py, and d4rl.

```bash
pip install -f https://download.pytorch.org/whl/torch_stable.html \
                free-mujoco-py \
                einops \
                gym==0.24.1 \
                protobuf==3.20.1 \
                git+https://github.com/rail-berkeley/d4rl.git \
                mediapy \
                Pillow==9.0.0
```

--------------------------------

### CLIP Guided Stable Diffusion Pipeline Setup and Generation

Source: https://github.com/tencentarc/brushnet/blob/main/examples/community/README.md

Sets up and uses a CLIP guided stable diffusion pipeline for enhanced image realism. Requires `diffusers` and `transformers` libraries. This pipeline uses a CLIP model to guide the diffusion process at each step. It requires approximately 12GB of GPU RAM.

```python
from diffusers import DiffusionPipeline
from transformers import CLIPImageProcessor, CLIPModel
import torch


feature_extractor = CLIPImageProcessor.from_pretrained("laion/CLIP-ViT-B-32-laion2B-s34B-b79K")
clip_model = CLIPModel.from_pretrained("laion/CLIP-ViT-B-32-laion2B-s34B-b79K", torch_dtype=torch.float16)


guided_pipeline = DiffusionPipeline.from_pretrained(
    "runwayml/stable-diffusion-v1-5",
    custom_pipeline="clip_guided_stable_diffusion",
    clip_model=clip_model,
    feature_extractor=feature_extractor,

    torch_dtype=torch.float16,
)
guided_pipeline.enable_attention_slicing()
guided_pipeline = guided_pipeline.to("cuda")

prompt = "fantasy book cover, full moon, fantasy forest landscape, golden vector elements, fantasy magic, dark light night, intricate, elegant, sharp focus, illustration, highly detailed, digital painting, concept art, matte, art by WLOP and Artgerm and Albert Bierstadt, masterpiece"

generator = torch.Generator(device="cuda").manual_seed(0)
images = []
for i in range(4):
    image = guided_pipeline(
        prompt,
        num_inference_steps=50,
        guidance_scale=7.5,
        clip_guidance_scale=100,
        num_cutouts=4,
        use_cutouts=False,
        generator=generator,
    ).images[0]
    images.append(image)


```

--------------------------------

### Launch Gradio Demo

Source: https://context7.com/tencentarc/brushnet/llms.txt

Start the interactive web interface for BrushNet.

```bash
# Start the Gradio demo server
python examples/brushnet/app_brushnet.py

# Server runs at http://0.0.0.0:12345
```

--------------------------------

### Download Dataset Snapshot

Source: https://github.com/tencentarc/brushnet/blob/main/docs/source/en/training/text_inversion.md

Downloads a dataset snapshot from the Hugging Face Hub. Use this to get example images for training.

```python
from huggingface_hub import snapshot_download

local_dir = "./cat"
snapshot_download(
    "diffusers/cat_toy_example", local_dir=local_dir, repo_type="dataset", ignore_patterns=".gitattributes"
)
```

--------------------------------

### Stable Diffusion with K Diffusion (Text-to-Image)

Source: https://github.com/tencentarc/brushnet/blob/main/examples/community/README.md

Utilizes the k-diffusion library for Stable Diffusion text-to-image generation. Ensure k-diffusion is installed (`pip install k-diffusion`). This example demonstrates setting a specific scheduler and using a manual seed for reproducibility.

```python
from diffusers import DiffusionPipeline

pipe = DiffusionPipeline.from_pretrained("CompVis/stable-diffusion-v1-4", custom_pipeline="sd_text2img_k_diffusion")
pipe = pipe.to("cuda")

prompt = "an astronaut riding a horse on mars"
pipe.set_scheduler("sample_heun")
generator = torch.Generator(device="cuda").manual_seed(seed)
image = pipe(prompt, generator=generator, num_inference_steps=20).images[0]

image.save("./astronaut_heun_k_diffusion.png")
```

--------------------------------

### Launch Training Loop with Accelerate

Source: https://github.com/tencentarc/brushnet/blob/main/docs/source/en/tutorials/basic_training.md

Use `accelerate.notebook_launcher` to start the training loop. Pass the training function, its arguments, and the desired number of processes.

```python
from accelerate import notebook_launcher

args = (config, model, noise_scheduler, optimizer, train_dataloader, lr_scheduler)

notebook_launcher(train_loop, args, num_processes=1)
```

--------------------------------

### Clone Diffusers Repository

Source: https://github.com/tencentarc/brushnet/blob/main/CONTRIBUTING.md

Clone the Diffusers repository to access and contribute training examples. This is the initial step before installing dependencies or adding new scripts.

```bash
git clone https://github.com/huggingface/diffusers
```

--------------------------------

### Install Diffusers from source

Source: https://github.com/tencentarc/brushnet/blob/main/docs/source/en/training/overview.md

Clone the repository and install the library in a new virtual environment to ensure compatibility with the latest training scripts.

```bash
git clone https://github.com/huggingface/diffusers
cd diffusers
pip install .
```

--------------------------------

### DeepFloyd IF Text-to-Image + Upscaling Setup

Source: https://github.com/tencentarc/brushnet/blob/main/docs/source/en/optimization/torch2.0.md

Initializes and configures DeepFloyd IF and Stable Diffusion upscaler pipelines with torch.compile for performance. Ensure CUDA is available and PyTorch is installed with CUDA support.

```python
from diffusers import DiffusionPipeline
import torch

run_compile = True  # Set True / False

pipe_1 = DiffusionPipeline.from_pretrained("DeepFloyd/IF-I-M-v1.0", variant="fp16", text_encoder=None, torch_dtype=torch.float16, use_safetensors=True)
pipe_1.to("cuda")
pipe_2 = DiffusionPipeline.from_pretrained("DeepFloyd/IF-II-M-v1.0", variant="fp16", text_encoder=None, torch_dtype=torch.float16, use_safetensors=True)
pipe_2.to("cuda")
pipe_3 = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-x4-upscaler", torch_dtype=torch.float16, use_safetensors=True)
pipe_3.to("cuda")


pipe_1.unet.to(memory_format=torch.channels_last)
pipe_2.unet.to(memory_format=torch.channels_last)
pipe_3.unet.to(memory_format=torch.channels_last)

if run_compile:
    pipe_1.unet = torch.compile(pipe_1.unet, mode="reduce-overhead", fullgraph=True)
    pipe_2.unet = torch.compile(pipe_2.unet, mode="reduce-overhead", fullgraph=True)
    pipe_3.unet = torch.compile(pipe_3.unet, mode="reduce-overhead", fullgraph=True)

prompt = "the blue hulk"

prompt_embeds = torch.randn((1, 2, 4096), dtype=torch.float16)
neg_prompt_embeds = torch.randn((1, 2, 4096), dtype=torch.float16)

for _ in range(3):
    image_1 = pipe_1(prompt_embeds=prompt_embeds, negative_prompt_embeds=neg_prompt_embeds, output_type="pt").images
    image_2 = pipe_2(image=image_1, prompt_embeds=prompt_embeds, negative_prompt_embeds=neg_prompt_embeds, output_type="pt").images
    image_3 = pipe_3(prompt=prompt, image=image_1, noise_level=100).images
```

--------------------------------

### Preview Documentation

Source: https://github.com/tencentarc/brushnet/blob/main/docs/README.md

Run the preview command to serve documentation locally. The preview command requires existing files and may need a restart if the _toctree.yml is modified.

```bash
doc-builder preview {package_name} {path_to_docs}
```

```bash
doc-builder preview diffusers docs/source/en
```

--------------------------------

### Initialize Accelerate Environment

Source: https://github.com/tencentarc/brushnet/blob/main/docs/source/en/training/instructpix2pix.md

Configure the 🤗 Accelerate library for distributed training. Use 'accelerate config' for interactive setup or 'accelerate config default' for a basic configuration.

```bash
accelerate config
```

```bash
accelerate config default
```

--------------------------------

### Multi-GPU Training with Mixed Precision

Source: https://github.com/tencentarc/brushnet/blob/main/examples/unconditional_image_generation/README.md

Example command for launching distributed training with mixed precision (fp16) and specifying Weights and Biases (wandb) as the logger. Ensure wandb is installed separately.

```bash
accelerate launch --mixed_precision="fp16" --multi_gpu train_unconditional.py \
  --dataset_name="huggan/pokemon" \
  --resolution=64 --center_crop --random_flip \
  --output_dir="ddpm-ema-pokemon-64" \
  --train_batch_size=16 \
  --num_epochs=100 \
  --gradient_accumulation_steps=1 \
  --use_ema \
  --learning_rate=1e-4 \
  --lr_warmup_steps=500 \
  --mixed_precision="fp16" \
  --logger="wandb"
```

--------------------------------

### Start Tensorboard for Profiling

Source: https://github.com/tencentarc/brushnet/blob/main/examples/controlnet/README.md

Launch Tensorboard to visualize profiling data. Ensure the log directory points to your training runs.

```bash
tensorboard --logdir runs/fill-circle-100steps-20230411_165612/
```

--------------------------------

### Train Custom Diffusion with Validation and Wandb Reporting

Source: https://github.com/tencentarc/brushnet/blob/main/examples/custom_diffusion/README.md

This command extends the training example by adding validation prompts and configuring Weights and Biases for experiment tracking and reporting. Ensure wandb is installed and authorized.

```bash
export MODEL_NAME="CompVis/stable-diffusion-v1-4"
export OUTPUT_DIR="path-to-save-model"
export INSTANCE_DIR="./data/cat"

accelerate launch train_custom_diffusion.py \
  --pretrained_model_name_or_path=$MODEL_NAME  \
  --instance_data_dir=$INSTANCE_DIR \
  --output_dir=$OUTPUT_DIR \
  --class_data_dir=./real_reg/samples_cat/ \
  --with_prior_preservation --real_prior --prior_loss_weight=1.0 \
  --class_prompt="cat" --num_class_images=200 \
  --instance_prompt="photo of a <new1> cat"  \
  --resolution=512  \
  --train_batch_size=2  \
  --learning_rate=1e-5  \
  --lr_warmup_steps=0 \
  --max_train_steps=250 \
  --scale_lr --hflip  \
  --modifier_token "<new1>" \
  --validation_prompt="<new1> cat sitting in a bucket" \
  --report_to="wandb"
```

--------------------------------

### Install Optimum Intel for OpenVINO

Source: https://github.com/tencentarc/brushnet/blob/main/docs/source/en/optimization/open_vino.md

Install the required package with the eager upgrade strategy to ensure the latest version is used.

```bash
pip install --upgrade-strategy eager optimum["openvino"]
```

--------------------------------

### Configure Inpainting Pipeline with Offloading and Attention

Source: https://github.com/tencentarc/brushnet/blob/main/docs/source/en/using-diffusers/inpaint.md

Initializes the `AutoPipelineForInpainting` with specific configurations for memory efficiency and attention mechanisms. This setup is beneficial for optimizing performance on GPU, especially when dealing with large models or limited VRAM. Ensure `xFormers` is installed or use PyTorch 2.0+ for `enable_xformers_memory_efficient_attention`.

```python
import torch
from diffusers import AutoPipelineForInpainting
from diffusers.utils import load_image, make_image_grid

pipeline = AutoPipelineForInpainting.from_pretrained(
    "runwayml/stable-diffusion-inpainting", torch_dtype=torch.float16, variant="fp16"
)
pipeline.enable_model_cpu_offload()
# remove following line if xFormers is not installed or you have PyTorch 2.0 or higher installed
pipeline.enable_xformers_memory_efficient_attention()
```

--------------------------------

### Install Required Libraries

Source: https://github.com/tencentarc/brushnet/blob/main/docs/source/en/using-diffusers/diffedit.md

Before using DiffEdit, ensure you have the necessary libraries installed. This snippet shows the basic installation command.

```python
from huggingface_hub import hf_hub_download
from diffusers import DiffusionPipeline
import torch

pipe = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16, variant="fp16", use_safetensors=True)
pipe.to("cuda")

# Download the LoRA weights
l ora_path = hf_hub_download(
    repo_id="diffedit/diffedit-lora",
    filename="diffedit-lora.safetensors",
    subfolder="lora",
    cache_dir="./lora_weights",
)

# Load the LoRA weights into the pipeline
pipe.load_lora_weights(lora_path, adapter_name="diffedit")

# Set the adapter to use
pipe.set_adapters("diffedit")

# Enable attention slicing for memory efficiency
pipe.enable_attention_slicing()

# Enable xformers if available for further memory efficiency
try:
    pipe.enable_xformers_memory_efficient_attention()
except ImportError:
    print("xformers not available, proceeding without it.")

# Set the prompt and negative prompt
prompt = "a photo of a cat"
negative_prompt = "" # You can add a negative prompt here if needed

# Set the image to edit and the text prompt for editing
image_to_edit = "./cat.png" # Replace with your image path
edit_prompt = "a photo of a dog"

# Generate the edited image
# The output image will be saved as "edited_cat.png"
output = pipe(
    prompt=prompt,
    negative_prompt=negative_prompt,
    image=Image.open(image_to_edit).convert("RGB"),
    edit_prompt=edit_prompt,
    num_inference_steps=25,
    guidance_scale=7.5,
    cross_attention_kwargs={
        "scale": 0.8, # Adjust this scale to control the strength of the LoRA
    },
).images[0]

output.save("edited_cat.png")

```

--------------------------------

### Download Example Style Image

Source: https://github.com/tencentarc/brushnet/blob/main/examples/amused/README.md

Use this command to download the example style image for Styledrop. This image is used as the single training sample.

```sh
wget https://huggingface.co/datasets/diffusers/docs-images/resolve/main/amused/A%20mushroom%20in%20%5BV%5D%20style.png
```

--------------------------------

### Install Diffusers from Source

Source: https://github.com/tencentarc/brushnet/blob/main/examples/consistency_distillation/README.md

Clone the repository and install the library in editable mode to ensure access to the latest training scripts.

```bash
git clone https://github.com/huggingface/diffusers
cd diffusers
pip install -e .
```

--------------------------------

### Install xFormers with Pip

Source: https://github.com/tencentarc/brushnet/blob/main/docs/source/en/optimization/xformers.md

Use this command to install the latest version of xFormers. Ensure you have the latest PyTorch version installed.

```bash
pip install xformers
```

--------------------------------

### Install Diffusers with Flax

Source: https://github.com/tencentarc/brushnet/blob/main/README_original.md

Install the Diffusers library with Flax support using pip. Ensure you have Flax installed separately.

```bash
pip install --upgrade diffusers[flax]
```

--------------------------------

### Multi Subject DreamBooth Training Example

Source: https://github.com/tencentarc/brushnet/blob/main/examples/research_projects/multi_subject_dreambooth/README.md

Example command to launch the multi-subject DreamBooth training script. It demonstrates setting up environment variables for two subjects (a dog and a person) and configuring the training parameters.

```bash
export MODEL_NAME="CompVis/stable-diffusion-v1-4"
export OUTPUT_DIR="path-to-save-model"

# Subject 1
export INSTANCE_DIR_1="path-to-instance-images-concept-1"
export INSTANCE_PROMPT_1="a photo of a sks dog"
export CLASS_DIR_1="path-to-class-images-dog"
export CLASS_PROMPT_1="a photo of a dog"

# Subject 2
export INSTANCE_DIR_2="path-to-instance-images-concept-2"
export INSTANCE_PROMPT_2="a photo of a t@y person"
export CLASS_DIR_2="path-to-class-images-person"
export CLASS_PROMPT_2="a photo of a person"

accelerate launch train_multi_subject_dreambooth.py \
  --pretrained_model_name_or_path=$MODEL_NAME  \
  --instance_data_dir="$INSTANCE_DIR_1,$INSTANCE_DIR_2" \
  --output_dir=$OUTPUT_DIR \
  --train_text_encoder \
  --instance_prompt="$INSTANCE_PROMPT_1,$INSTANCE_PROMPT_2" \
  --with_prior_preservation \
  --prior_loss_weight=1.0 \
  --class_data_dir="$CLASS_DIR_1,$CLASS_DIR_2" \
  --class_prompt="$CLASS_PROMPT_1,$CLASS_PROMPT_2"
  --num_class_images=50 \
  --resolution=512 \
  --train_batch_size=1 \
  --gradient_accumulation_steps=1 \
  --learning_rate=1e-6 \
  --lr_scheduler="constant" \
  --lr_warmup_steps=0 \
  --max_train_steps=1500
```

--------------------------------

### Install Diffusers with PyTorch

Source: https://github.com/tencentarc/brushnet/blob/main/README_original.md

Install the Diffusers library with PyTorch support using pip. Ensure you have PyTorch installed separately.

```bash
pip install --upgrade diffusers[torch]
```

--------------------------------

### Install Tensorboard Profile Plugin

Source: https://github.com/tencentarc/brushnet/blob/main/examples/controlnet/README.md

Install necessary packages for Tensorboard profiling. This command installs TensorFlow and the Tensorboard profile plugin.

```bash
pip install tensorflow tensorboard-plugin-profile
```

--------------------------------

### Setup TPU and verify devices

Source: https://github.com/tencentarc/brushnet/blob/main/docs/source/en/using-diffusers/stable_diffusion_jax_how_to.md

Configure the TPU backend and verify the number and type of available JAX devices.

```python
import jax
import jax.tools.colab_tpu
jax.tools.colab_tpu.setup_tpu()

num_devices = jax.device_count()
device_type = jax.devices()[0].device_kind

print(f"Found {num_devices} JAX devices of type {device_type}.")
assert (
    "TPU" in device_type,
    "Available device is not a TPU, please select TPU from Runtime > Change runtime type > Hardware accelerator"
)
# Found 8 JAX devices of type Cloud TPU.
```

--------------------------------

### Install Libraries for Stable Video Diffusion

Source: https://github.com/tencentarc/brushnet/blob/main/docs/source/en/using-diffusers/svd.md

Installs the necessary libraries for using Stable Video Diffusion. Ensure these are installed before proceeding.

```python
!pip install -q -U diffusers transformers accelerate
```

--------------------------------

### Run SDXL DreamBooth training with EDM formulation

Source: https://github.com/tencentarc/brushnet/blob/main/examples/dreambooth/README_sdxl.md

Example command for training a LoRA with the Playground v2.5 model using EDM-style training.

```bash
accelerate launch train_dreambooth_lora_sdxl.py \
  --pretrained_model_name_or_path="playgroundai/playground-v2.5-1024px-aesthetic"  \
  --instance_data_dir="dog" \
  --output_dir="dog-playground-lora" \
  --mixed_precision="fp16" \
  --instance_prompt="a photo of sks dog" \
  --resolution=1024 \
  --train_batch_size=1 \
  --gradient_accumulation_steps=4 \
  --learning_rate=1e-4 \
  --use_8bit_adam \
  --report_to="wandb" \
  --lr_scheduler="constant" \
  --lr_warmup_steps=0 \
  --max_train_steps=500 \
  --validation_prompt="A photo of sks dog in a bucket" \
  --validation_epochs=25 \
  --seed="0" \
  --push_to_hub
```

--------------------------------

### Launch InstructPix2Pix Training

Source: https://github.com/tencentarc/brushnet/blob/main/docs/source/en/training/instructpix2pix.md

Use this command to launch the training script. Set MODEL_NAME and DATASET_ID environment variables beforehand. Consider using --multi_gpu for multi-GPU training and --report_to=wandb for monitoring.

```bash
accelerate launch --mixed_precision="fp16" train_instruct_pix2pix.py \
    --pretrained_model_name_or_path=$MODEL_NAME \
    --dataset_name=$DATASET_ID \
    --enable_xformers_memory_efficient_attention \
    --resolution=256 \
    --random_flip \
    --train_batch_size=4 \
    --gradient_accumulation_steps=4 \
    --gradient_checkpointing \
    --max_train_steps=15000 \
    --checkpointing_steps=5000 \
    --checkpoints_total_limit=1 \
    --learning_rate=5e-05 \
    --max_grad_norm=1 \
    --lr_warmup_steps=0 \
    --conditioning_dropout_prob=0.05 \
    --mixed_precision=fp16 \
    --seed=42 \
    --push_to_hub
```

--------------------------------

### Install Accelerate and Diffusers from Source

Source: https://github.com/tencentarc/brushnet/blob/main/docs/source/en/installation.md

Install the Accelerate library and then install Diffusers directly from its GitHub repository. This provides the latest development version.

```bash
pip install accelerate
```

```bash
pip install git+https://github.com/huggingface/diffusers
```

--------------------------------

### Launch Training with Accelerate

Source: https://github.com/tencentarc/brushnet/blob/main/examples/controlnet/README.md

Command to initiate the training process with memory optimization flags enabled.

```bash
export MODEL_DIR="runwayml/stable-diffusion-v1-5"
export OUTPUT_DIR="path to save model"

accelerate launch train_controlnet.py \
 --pretrained_model_name_or_path=$MODEL_DIR \
 --output_dir=$OUTPUT_DIR \
 --dataset_name=fusing/fill50k \
 --resolution=512 \
 --validation_image "./conditioning_image_1.png" "./conditioning_image_2.png" \
 --validation_prompt "red circle with blue background" "cyan circle with brown floral background" \
 --train_batch_size=1 \
 --gradient_accumulation_steps=4 \
 --gradient_checkpointing \
 --enable_xformers_memory_efficient_attention \
 --set_grads_to_none \
 --mixed_precision fp16
```