### Setup and Run Pre-Commit Hooks

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/developer/contribution_guide.md

Install pre-commit and run all checks to ensure code style consistency before committing. Re-run if checks fail initially.

```shell
pip install pre-commit  # This should already be installed with the development dependencies
pre-commit install
pre-commit run --all-files
```

--------------------------------

### Example ComfyUI Nunchaku Wheel Installation

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/installation/installation.md

Example command for installing Nunchaku for Python 3.11 and PyTorch 2.7 within ComfyUI. Adjust the wheel URL as needed.

```bat
"G:\ComfyUI\python\python.exe" -m pip install https://github.com/nunchaku-tech/nunchaku/releases/download/v0.3.1/nunchaku-0.3.1+torch2.7-cp311-cp311-linux_x86_64.whl
```

--------------------------------

### FLUX.1-Redux-Dev Pipeline Setup

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/usage/controlnet.md

Sets up the FLUX.1-Redux-Dev and FLUX.1-dev pipelines. This example initializes both the prior redux pipeline and the main flux pipeline with a custom transformer, automatically detecting GPU precision.

```python
import torch
from diffusers import FluxPipeline, FluxPriorReduxPipeline
from diffusers.utils import load_image

from nunchaku import NunchakuFluxTransformer2dModel
from nunchaku.utils import get_precision

precision = get_precision()
pipe_prior_redux = FluxPriorReduxPipeline.from_pretrained(
    "black-forest-labs/FLUX.1-Redux-dev", torch_dtype=torch.bfloat16
).to("cuda")
transformer = NunchakuFluxTransformer2dModel.from_pretrained(
    f"nunchaku-tech/nunchaku-flux.1-dev/svdq-{precision}_r32-flux.1-dev.safetensors"
)
pipe = FluxPipeline.from_pretrained(
    "black-forest-labs/FLUX.1-dev",
    text_encoder=None,
    text_encoder_2=None,
    transformer=transformer,
    torch_dtype=torch.bfloat16,
).to("cuda")
```

--------------------------------

### Install Nunchaku from Prebuilt Wheel or Source

Source: https://context7.com/nunchaku-ai/nunchaku/llms.txt

Instructions for installing Nunchaku using prebuilt wheels or building from source. Ensure PyTorch is installed first, matching your Python and CUDA versions.

```shell
# Install PyTorch first (example: PyTorch 2.7 + CUDA 12.8)
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128

# Install Nunchaku prebuilt wheel (example: v0.3.1, Python 3.11, PyTorch 2.7, Linux x86_64)
pip install https://github.com/nunchaku-tech/nunchaku/releases/download/v0.3.1/nunchaku-0.3.1+torch2.7-cp311-cp311-linux_x86_64.whl

# Or build from source
git clone --recurse-submodules https://github.com/nunchaku-tech/nunchaku.git
cd nunchaku
pip install -e "[dev,docs]"

# Build distribution wheel for all GPU architectures
NUNCHAKU_INSTALL_MODE=ALL NUNCHAKU_BUILD_WHEELS=1 python -m build --wheel --no-isolation
```

--------------------------------

### Install Image Quality Packages

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/app/flux.1/t2i/README.md

Installs the required Python packages for calculating image quality metrics. Ensure you have pip and Python installed.

```shell
pip install clean-fid torchmetrics image-reward clip datasets
```

--------------------------------

### Launch ComfyUI

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/installation/setup_windows.md

Command to start ComfyUI after installation using the CLI.

```bat
comfy launch
```

--------------------------------

### Install Development and Docs Dependencies

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/developer/build_docs.md

Install the project with development and documentation extras using pip. This ensures all necessary packages for building the docs are available.

```shell
pip install -e ".[dev,docs]"
```

--------------------------------

### FLUX.1-Canny-Dev ControlNet Example

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/usage/controlnet.md

Demonstrates using the FLUX.1-Canny-Dev model with a ControlNet pipeline. Ensure you have the necessary libraries installed and the model weights downloaded. The precision is auto-detected based on your GPU.

```python
import torch
from controlnet_aux import CannyDetector
from diffusers import FluxControlPipeline
from diffusers.utils import load_image

from nunchaku import NunchakuFluxTransformer2dModel
from nunchaku.utils import get_precision

precision = get_precision()  # auto-detect your precision is 'int4' or 'fp4' based on your GPU
transformer = NunchakuFluxTransformer2dModel.from_pretrained(
    f"nunchaku-tech/nunchaku-flux.1-canny-dev/svdq-{precision}_r32-flux.1-canny-dev.safetensors"
)
pipe = FluxControlPipeline.from_pretrained(
    "black-forest-labs/FLUX.1-Canny-dev", transformer=transformer, torch_dtype=torch.bfloat16
).to("cuda")

prompt = (
    "A robot made of exotic candies and chocolates of different kinds. "
    "The background is filled with confetti and celebratory gifts."
)
control_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/robot.png")

processor = CannyDetector()
control_image = processor(
    control_image, low_threshold=50, high_threshold=200, detect_resolution=1024, image_resolution=1024
)

image = pipe(
    prompt=prompt, control_image=control_image, height=1024, width=1024, num_inference_steps=50, guidance_scale=30.0
).images[0]
image.save(f"flux.1-canny-dev-{precision}.png")
```

--------------------------------

### Install Development Dependencies

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/developer/contribution_guide.md

Install all necessary development dependencies for the Nunchaku project.

```shell
pip install -e ".[dev]"
```

--------------------------------

### Install Build Tools for Nunchaku

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/installation/setup_windows.md

Installs necessary Python packages for building Nunchaku from source.

```bat
"G:\ComfyuI\python\python.exe" -m pip install ninja setuptools wheel build
```

--------------------------------

### Run ControlNet-Union-Pro with Nunchaku

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/usage/controlnet.md

This example demonstrates loading ControlNet-Union-Pro, setting up the Nunchaku pipeline with a transformer, and generating an image using both depth and Canny edge control maps. It includes logic for GPU memory optimization and offloading.

```python
import torch
from diffusers import FluxControlNetModel, FluxControlNetPipeline
from diffusers.models import FluxMultiControlNetModel
from diffusers.utils import load_image

from nunchaku import NunchakuFluxTransformer2dModel
from nunchaku.utils import get_gpu_memory, get_precision

base_model = "black-forest-labs/FLUX.1-dev"
controlnet_model_union = "Shakker-Labs/FLUX.1-dev-ControlNet-Union-Pro"

controlnet_union = FluxControlNetModel.from_pretrained(controlnet_model_union, torch_dtype=torch.bfloat16)
controlnet = FluxMultiControlNetModel([controlnet_union])  # we always recommend loading via FluxMultiControlNetModel

precision = get_precision()
need_offload = get_gpu_memory() < 36
transformer = NunchakuFluxTransformer2dModel.from_pretrained(
    f"nunchaku-tech/nunchaku-flux.1-dev/svdq-{precision}_r32-flux.1-dev.safetensors",
    torch_dtype=torch.bfloat16,
    offload=need_offload,
)
transformer.set_attention_impl("nunchaku-fp16")

pipeline = FluxControlNetPipeline.from_pretrained(
    base_model, transformer=transformer, controlnet=controlnet, torch_dtype=torch.bfloat16
)

if need_offload:
    pipeline.enable_sequential_cpu_offload()
else:
    pipeline = pipeline.to("cuda")

prompt = "A anime style girl with messy beach waves."
control_image_depth = load_image(
    "https://huggingface.co/Shakker-Labs/FLUX.1-dev-ControlNet-Union-Pro/resolve/main/assets/depth.jpg"
)
control_mode_depth = 2

control_image_canny = load_image(
    "https://huggingface.co/Shakker-Labs/FLUX.1-dev-ControlNet-Union-Pro/resolve/main/assets/canny.jpg"
)
control_mode_canny = 0

width, height = control_image_depth.size

image = pipeline(
    prompt,
    control_image=[control_image_depth, control_image_canny],
    control_mode=[control_mode_depth, control_mode_canny],
    width=width,
    height=height,
    controlnet_conditioning_scale=[0.3, 0.1],
    num_inference_steps=28,
    guidance_scale=3.5,
    generator=torch.manual_seed(233),
).images[0]


image.save(f"flux.1-dev-controlnet-union-pro-{precision}.png")
```

--------------------------------

### Build Nunchaku from Source

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/installation/setup_windows.md

Compiles and installs Nunchaku in development mode from the source code.

```bat
"G:\ComfyuI\python\python.exe" setup.py develop
```

--------------------------------

### Install Dependencies for Image Generation Aux

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/app/flux.1/depth_canny/README.md

Install the necessary Python packages for image generation auxiliary functions and control net models. Ensure you have git, controlnet_aux, and mediapipe installed.

```shell
pip install git+https://github.com/asomoza/image_gen_aux.git
pip install controlnet_aux mediapipe
```

--------------------------------

### Preview Documentation Locally

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/developer/build_docs.md

Serve the generated HTML documentation locally using Python's http.server. Navigate to the build/html directory and start the server on a specified port. You can then access the docs via your web browser.

```shell
cd build/html
python -m http.server 2333
```

--------------------------------

### Load Image and Generate with FLUX.1-tools

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/usage/controlnet.md

Demonstrates loading an image and using the FLUX.1-tools base model for image generation. Ensure the necessary libraries and models are installed.

```python
image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/robot.png")
pipe_prior_output = pipe_prior_redux(image)
images = pipe(guidance_scale=2.5, num_inference_steps=50, **pipe_prior_output).images
images[0].save(f"flux.1-redux-dev-{precision}.png")
```

--------------------------------

### Run 4-bit FLUX.1 on Default GPUs with Nunchaku

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/usage/basic_usage.md

This snippet shows the minimal setup for running FLUX.1 using Nunchaku on default GPU architectures (Ampere, Ada, Blackwell, etc.). It automatically detects the precision (int4 or fp4) based on the GPU. Ensure you have the necessary libraries installed.

```python
import torch
from diffusers import FluxPipeline

from nunchaku import NunchakuFluxTransformer2dModel
from nunchaku.utils import get_precision

precision = get_precision()  # auto-detect your precision is 'int4' or 'fp4' based on your GPU
transformer = NunchakuFluxTransformer2dModel.from_pretrained(
    f"nunchaku-tech/nunchaku-flux.1-dev/svdq-{precision}_r32-flux.1-dev.safetensors"
)
pipeline = FluxPipeline.from_pretrained(
    "black-forest-labs/FLUX.1-dev", transformer=transformer, torch_dtype=torch.bfloat16
).to("cuda")
image = pipeline("A cat holding a sign that says hello world", num_inference_steps=50, guidance_scale=3.5).images[0]
image.save(f"flux.1-dev-{precision}.png")
```

--------------------------------

### Verify Nunchaku Installation

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/installation/setup_windows.md

Confirms that the Nunchaku package has been successfully installed by attempting to import it.

```bat
"G:\ComfyuI\python\python.exe" -c "import nunchaku"
```

--------------------------------

### Load and Run Qwen-Image-Edit

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/usage/qwen-image-edit.md

Load the Qwen-Image-Edit model and perform an image editing task. This example includes logic for optimizing GPU memory usage based on available VRAM.

```python
import torch
from diffusers import QwenImageEditPipeline
from diffusers.utils import load_image

from nunchaku import NunchakuQwenImageTransformer2DModel
from nunchaku.utils import get_gpu_memory

num_inference_steps = 50
model_paths = {
    50: "nunchaku-tech/nunchaku-qwen-image-edit-lightning-4bit"
}
scheduler = FlowMatchEulerDiscreteScheduler.from_pretrained(
    model_paths[num_inference_steps], subfolder="scheduler"
)

# Load the model
transformer = NunchakuQwenImageTransformer2DModel.from_pretrained(model_paths[num_inference_steps])

pipeline = QwenImageEditPipeline.from_pretrained(
    "Qwen/Qwen-Image-Edit", transformer=transformer, scheduler=scheduler, torch_dtype=torch.bfloat16
)

if get_gpu_memory() > 18:
    pipeline.enable_model_cpu_offload()
else:
    # use per-layer offloading for low VRAM. This only requires 3-4GB of VRAM.
    transformer.set_offload(
        True, use_pin_memory=False, num_blocks_on_gpu=1
    )  # increase num_blocks_on_gpu if you have more VRAM
    pipeline._exclude_from_cpu_offload.append("transformer")
    pipeline.enable_sequential_cpu_offload()

image = load_image("https://huggingface.co/datasets/nunchaku-tech/test-data/resolve/main/inputs/neon_sign.png")
image = image.convert("RGB")
prompt = "change the text to read '双截棍 Qwen Image Edit is here'"
inputs = {
    "image": image,
    "prompt": prompt,
    "true_cfg_scale": 1,
    "num_inference_steps": num_inference_steps,
}

output = pipeline(**inputs)
output_image = output.images[0]
output_image.save(f"qwen-image-edit-lightning-r{rank}-{num_inference_steps}steps.png")
```

--------------------------------

### Install Test Dependencies

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/tests/README.md

Install necessary dependencies for running Nunchaku tests using pip.

```shell
pip install -r tests/requirements.txt
```

--------------------------------

### Install Nunchaku Wheel

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/installation/installation.md

Installs a specific Nunchaku wheel file from GitHub releases. Replace version numbers and architecture details with your environment's specifics.

```shell
pip install https://github.com/nunchaku-tech/nunchaku/releases/download/v0.3.1/nunchaku-0.3.1+torch2.7-cp311-cp311-linux_x86_64.whl
```

--------------------------------

### FLUX.1-Fill-Dev Inpainting Example

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/usage/controlnet.md

Illustrates using the FLUX.1-Fill-Dev pipeline for inpainting tasks. This requires an input image, a mask image, and a prompt. The precision is automatically detected.

```python
import torch
from diffusers import FluxFillPipeline
from diffusers.utils import load_image

from nunchaku import NunchakuFluxTransformer2dModel
from nunchaku.utils import get_precision

image = load_image("https://huggingface.co/datasets/diffusers/diffusers-images-docs/resolve/main/cup.png")
mask = load_image("https://huggingface.co/datasets/diffusers/diffusers-images-docs/resolve/main/cup_mask.png")

precision = get_precision()  # auto-detect your precision is 'int4' or 'fp4' based on your GPU
transformer = NunchakuFluxTransformer2dModel.from_pretrained(
    f"nunchaku-tech/nunchaku-flux.1-fill-dev/svdq-{precision}_r32-flux.1-fill-dev.safetensors"
)
pipe = FluxFillPipeline.from_pretrained(
    "black-forest-labs/FLUX.1-Fill-dev", transformer=transformer, torch_dtype=torch.bfloat16
).to("cuda")
image = pipe(
    prompt="a white paper cup",
    image=image,
    mask_image=mask,
    height=1024,
    width=1024,
    guidance_scale=30,
    num_inference_steps=50,
    max_sequence_length=512,
).images[0]
image.save(f"flux.1-fill-dev-{precision}.png")
```

--------------------------------

### Install PyTorch for Building

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/installation/installation.md

Installs PyTorch, torchvision, and torchaudio within the activated Conda environment. This is a prerequisite for building Nunchaku from source.

```shell
pip install torch torchvision torchaudio
```

--------------------------------

### Install PyTorch for General Use

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/installation/setup_windows.md

Installs PyTorch version 2.6 and compatible torchvision/torchaudio for most users.

```bat
"G:\ComfyuI\python\python.exe" -m pip install torch==2.6 torchvision==0.21 torchaudio==2.6
```

--------------------------------

### FLUX.1-Depth-Dev ControlNet Example

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/usage/controlnet.md

Shows how to integrate the FLUX.1-Depth-Dev model into a ControlNet pipeline. This example uses a pre-trained depth estimation model for the control image. Precision is automatically determined by the GPU.

```python
import torch
from diffusers import FluxControlPipeline
from diffusers.utils import load_image
from image_gen_aux import DepthPreprocessor

from nunchaku import NunchakuFluxTransformer2dModel
from nunchaku.utils import get_precision

precision = get_precision()  # auto-detect your precision is 'int4' or 'fp4' based on your GPU
transformer = NunchakuFluxTransformer2dModel.from_pretrained(
    f"nunchaku-tech/nunchaku-flux.1-depth-dev/svdq-{precision}_r32-flux.1-depth-dev.safetensors"
)

pipe = FluxControlPipeline.from_pretrained(
    "black-forest-labs/FLUX.1-Depth-dev",
    transformer=transformer,
    torch_dtype=torch.bfloat16,
).to("cuda")

prompt = (
    "A robot made of exotic candies and chocolates of different kinds. "
    "The background is filled with confetti and celebratory gifts."
)
control_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/robot.png")

processor = DepthPreprocessor.from_pretrained("LiheYoung/depth-anything-large-hf")
control_image = processor(control_image)[0].convert("RGB")

image = pipe(
    prompt=prompt, control_image=control_image, height=1024, width=1024, num_inference_steps=30, guidance_scale=10.0
).images[0]
image.save(f"flux.1-depth-dev-{precision}.png")
```

--------------------------------

### Download Standard FLUX.1-dev Models

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/installation/setup_windows.md

Use these commands to download the standard FLUX.1-dev text encoders, VAE, and diffusion models. Ensure you have the Hugging Face CLI installed.

```bat
huggingface-cli download comfyanonymous/flux_text_encoders clip_l.safetensors --local-dir models/text_encoders
```

```bat
huggingface-cli download comfyanonymous/flux_text_encoders t5xxl_fp16.safetensors --local-dir models/text_encoders
```

```bat
huggingface-cli download black-forest-labs/FLUX.1-schnell ae.safetensors --local-dir models/vae
```

```bat
huggingface-cli download black-forest-labs/FLUX.1-dev flux1-dev.safetensors --local-dir models/diffusion_models
```

--------------------------------

### Install ComfyUI-nunchaku Plugin

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/installation/setup_windows.md

Clones the ComfyUI-nunchaku plugin repository into the custom_nodes directory to integrate Nunchaku with ComfyUI.

```bat
cd ComfyUI/custom_nodes
git clone https://github.com/nunchaku-tech/ComfyUI-nunchaku.git
```

--------------------------------

### Load and Use FLUX.1-Kontext-dev Pipeline

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/usage/kontext.md

This snippet demonstrates how to load the FLUX.1-Kontext-dev pipeline with a Nunchaku transformer, apply a prompt to an image for editing, and save the result. Ensure you have the necessary libraries installed and CUDA available for GPU acceleration.

```python
import torch
from diffusers import FluxKontextPipeline
from diffusers.utils import load_image

from nunchaku import NunchakuFluxTransformer2dModel
from nunchaku.utils import get_precision

transformer = NunchakuFluxTransformer2dModel.from_pretrained(
    f"nunchaku-tech/nunchaku-flux.1-kontext-dev/svdq-{get_precision()}_r32-flux.1-kontext-dev.safetensors"
)

pipeline = FluxKontextPipeline.from_pretrained(
    "black-forest-labs/FLUX.1-Kontext-dev", transformer=transformer, torch_dtype=torch.bfloat16
).to("cuda")

image = load_image(
    "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/yarn-art-pikachu.png"
).convert("RGB")

prompt = "Make Pikachu hold a sign that says 'Nunchaku is awesome', yarn art style, detailed, vibrant colors"
image = pipeline(image=image, prompt=prompt, guidance_scale=2.5).images[0]
image.save("flux-kontext-dev.png")
```

--------------------------------

### Load and Run Qwen-Image-Edit-2509

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/usage/qwen-image-edit.md

Load the Qwen-Image-Edit-2509 model and perform an image editing task. This example demonstrates using multiple input images and includes GPU memory optimization.

```python
import torch
from diffusers import QwenImageEditPlusPipeline
from diffusers.utils import load_image

from nunchaku import NunchakuQwenImageTransformer2DModel
from nunchaku.utils import get_gpu_memory, get_precision

rank = 128  # you can also use rank=128 model to improve the quality

# Load the model
transformer = NunchakuQwenImageTransformer2DModel.from_pretrained(
    f"nunchaku-tech/nunchaku-qwen-image-edit-2509/svdq-{get_precision()}_r{rank}-qwen-image-edit-2509.safetensors"
)

pipeline = QwenImageEditPlusPipeline.from_pretrained(
    "Qwen/Qwen-Image-Edit-2509", transformer=transformer, torch_dtype=torch.bfloat16
)

if get_gpu_memory() > 18:
    pipeline.enable_model_cpu_offload()
else:
    # use per-layer offloading for low VRAM. This only requires 3-4GB of VRAM.
    transformer.set_offload(
        True, use_pin_memory=False, num_blocks_on_gpu=1
    )  # increase num_blocks_on_gpu if you have more VRAM
    pipeline._exclude_from_cpu_offload.append("transformer")
    pipeline.enable_sequential_cpu_offload()

image1 = load_image("https://huggingface.co/datasets/nunchaku-tech/test-data/resolve/main/inputs/man.png")
image1 = image1.convert("RGB")
image2 = load_image("https://huggingface.co/datasets/nunchaku-tech/test-data/resolve/main/inputs/puppy.png")
image2 = image2.convert("RGB")
image3 = load_image("https://huggingface.co/datasets/nunchaku-tech/test-data/resolve/main/inputs/sofa.png")
image3 = image3.convert("RGB")

prompt = "Let the man in image 1 lie on the sofa in image 3, and let the puppy in image 2 lie on the floor to sleep."
inputs = {
    "image": [image1, image2, image3],
    "prompt": prompt,
    "true_cfg_scale": 4.0,
    "negative_prompt": " ",
    "num_inference_steps": 40,
}

output = pipeline(**inputs)
output_image = output.images[0]
output_image.save(f"qwen-image-edit-2509-r{rank}.png")
```

--------------------------------

### Enable Cache-DiT for Qwen-Image Pipeline

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/usage/cache.md

Enables Cache-DiT with DB Cache configuration for a Qwen-Image pipeline. This example includes logic for enabling model CPU offloading based on available GPU memory.

```python
# Please install cache-dit with `pip install cache-dit`.
import cache_dit
import torch
from cache_dit import DBCacheConfig
from diffusers import QwenImagePipeline

from nunchaku.models.transformers.transformer_qwenimage import NunchakuQwenImageTransformer2DModel
from nunchaku.utils import get_gpu_memory, get_precision

rank = 32  # you can also use rank=128 model to improve the quality

# Load the model
transformer = NunchakuQwenImageTransformer2DModel.from_pretrained(
    f"nunchaku-tech/nunchaku-qwen-image/svdq-{get_precision()}_r{rank}-qwen-image.safetensors"
)

# currently, you need to use this pipeline to offload the model to CPU
pipe = QwenImagePipeline.from_pretrained("Qwen/Qwen-Image", transformer=transformer, torch_dtype=torch.bfloat16)

# Please check https://github.com/vipshop/cache-dit for more details about the parameters.

cache_dit.enable_cache(
    pipe,
    cache_config=DBCacheConfig(
        Fn_compute_blocks=8,
        Bn_compute_blocks=0,
        residual_diff_threshold=0.12,
    ),
)


if get_gpu_memory() > 18:
    pipe.enable_model_cpu_offload()
else:
    # use per-layer offloading for low VRAM. This only requires 3-4GB of VRAM.
    transformer.set_offload(
        True, use_pin_memory=False, num_blocks_on_gpu=1
    )  # increase num_blocks_on_gpu if you have more VRAM
    pipe._exclude_from_cpu_offload.append("transformer")
    pipe.enable_sequential_cpu_offload()

positive_magic = {
    "en": "Ultra HD, 4K, cinematic composition.",  # for english prompt,
    "zh": "超清，4K，电影级构图",  # for chinese prompt,
}

```

--------------------------------

### Set Up Visual Studio Environment for Building

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/installation/setup_windows.md

Configures the command prompt with the necessary Visual Studio environment variables for building C++ code on Windows.

```bat
"C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\Common7\Tools\VsDevCmd.bat" -startdir=none -arch=x64 -host_arch=x64
set DISTUTILS_USE_SDK=1
```

--------------------------------

### Install ComfyUI CLI

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/installation/setup_windows.md

Use pip to install the ComfyUI command-line interface for managing ComfyUI installations.

```bat
pip install comfy-cli
comfy install
```

--------------------------------

### Load and Use IP Adapter with FLUX.1-dev

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/usage/ip_adapter.md

This snippet demonstrates the complete process of initializing the FLUX pipeline, loading an IP Adapter, applying it, enabling caching, and generating an image using both a text prompt and an IP adapter image. Ensure the IP adapter image is in RGB format before passing it to the pipeline.

```python
import torch
from diffusers import FluxPipeline
from diffusers.utils import load_image

from nunchaku import NunchakuFluxTransformer2dModel
from nunchaku.caching.diffusers_adapters import apply_cache_on_pipe
from nunchaku.models.ip_adapter.diffusers_adapters import apply_IPA_on_pipe
from nunchaku.utils import get_precision

precision = get_precision()
transformer = NunchakuFluxTransformer2dModel.from_pretrained(
    f"nunchaku-tech/nunchaku-flux.1-dev/svdq-{precision}_r32-flux.1-dev.safetensors"
)
pipeline = FluxPipeline.from_pretrained(
    "black-forest-labs/FLUX.1-dev", transformer=transformer, torch_dtype=torch.bfloat16
).to("cuda")

pipeline.load_ip_adapter(
    pretrained_model_name_or_path_or_dict="XLabs-AI/flux-ip-adapter-v2",
    weight_name="ip_adapter.safetensors",
    image_encoder_pretrained_model_name_or_path="openai/clip-vit-large-patch14",
)

apply_IPA_on_pipe(pipeline, ip_adapter_scale=1.1, repo_id="XLabs-AI/flux-ip-adapter-v2")

apply_cache_on_pipe(
    pipeline,
    use_double_fb_cache=True,
    residual_diff_threshold_multi=0.09,
    residual_diff_threshold_single=0.12,
)

IP_image = load_image(
    "https://huggingface.co/datasets/nunchaku-tech/test-data/resolve/main/ComfyUI-nunchaku/inputs/monalisa.jpg"
)

image = pipeline(
    prompt="holding an sign saying 'SVDQuant is fast!'",
    ip_adapter_image=IP_image.convert("RGB"),
    num_inference_steps=50,
).images[0]

image.save(f"flux.1-dev-IP-adapter-{precision}.png")
```

--------------------------------

### Configure and Load Qwen-Image Pipeline

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/usage/qwen-image.md

Sets up the scheduler configuration, defines model paths based on inference steps and rank, and loads the transformer and pipeline. It includes logic for enabling model CPU offload based on available GPU memory.

```python
import math
import torch

from nunchaku_pipeline.qwen_image import QwenImagePipeline
from nunchaku_pipeline.qwen_image.transformer_2d import NunchakuQwenImageTransformer2DModel
from diffusers import FlowMatchEulerDiscreteScheduler

# Helper functions (assuming these are defined elsewhere)
def get_precision(): return "fp16"
def get_gpu_memory(): return 24

scheduler_config = {
    "base_image_seq_len": 256,
    "base_shift": math.log(3),  # We use shift=3 in distillation
    "invert_sigmas": False,
    "max_image_seq_len": 8192,
    "max_shift": math.log(3),  # We use shift=3 in distillation
    "num_train_timesteps": 1000,
    "shift": 1.0,
    "shift_terminal": None,  # set shift_terminal to None
    "stochastic_sampling": False,
    "time_shift_type": "exponential",
    "use_beta_sigmas": False,
    "use_dynamic_shifting": True,
    "use_exponential_sigmas": False,
    "use_karras_sigmas": False,
}
scheduler = FlowMatchEulerDiscreteScheduler.from_config(scheduler_config)

num_inference_steps = 4  # you can also use the 8-step model to improve the quality
rank = 32  # you can also use the rank=128 model to improve the quality
model_paths = {
    4: f"nunchaku-tech/nunchaku-qwen-image/svdq-{get_precision()}_r{rank}-qwen-image-lightningv1.0-4steps.safetensors",
    8: f"nunchaku-tech/nunchaku-qwen-image/svdq-{get_precision()}_r{rank}-qwen-image-lightningv1.1-8steps.safetensors",
}

# Load the model
transformer = NunchakuQwenImageTransformer2DModel.from_pretrained(model_paths[num_inference_steps])
pipe = QwenImagePipeline.from_pretrained(
    "Qwen/Qwen-Image", transformer=transformer, scheduler=scheduler, torch_dtype=torch.bfloat16
)

if get_gpu_memory() > 18:
    pipe.enable_model_cpu_offload()
else:
    # use per-layer offloading for low VRAM. This only requires 3-4GB of VRAM.
    transformer.set_offload(
        True, use_pin_memory=False, num_blocks_on_gpu=1
    )  # increase num_blocks_on_gpu if you have more VRAM
    pipe._exclude_from_cpu_offload.append("transformer")
    pipe.enable_sequential_cpu_offload()

```

--------------------------------

### Install Cache-DiT Package

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/usage/cache.md

Installs the necessary `cache-dit` package for enabling advanced caching strategies.

```bash
pip install cache-dit
```

--------------------------------

### Load and Run Quantized Qwen-Image-Edit

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/usage/qwen-image-edit.md

This snippet demonstrates loading a 4-bit quantized Qwen-Image-Edit model with Nunchaku. It includes configurations for VRAM management, such as model CPU offloading and per-layer offloading for low VRAM scenarios. Adjust `rank` to improve output quality.

```python
import torch
from diffusers import QwenImageEditPipeline
from diffusers.utils import load_image

from nunchaku import NunchakuQwenImageTransformer2DModel
from nunchaku.utils import get_gpu_memory, get_precision

rank = 128  # you can also use rank=128 model to improve the quality

# Load the model
transformer = NunchakuQwenImageTransformer2DModel.from_pretrained(
    f"nunchaku-tech/nunchaku-qwen-image-edit/svdq-{get_precision()}_r{rank}-qwen-image-edit.safetensors"
)

pipeline = QwenImageEditPipeline.from_pretrained(
    "Qwen/Qwen-Image-Edit", transformer=transformer, torch_dtype=torch.bfloat16
)

if get_gpu_memory() > 18:
    pipeline.enable_model_cpu_offload()
else:
    # use per-layer offloading for low VRAM. This only requires 3-4GB of VRAM.
    transformer.set_offload(
        True, use_pin_memory=False, num_blocks_on_gpu=1
    )  # increase num_blocks_on_gpu if you have more VRAM
    pipeline._exclude_from_cpu_offload.append("transformer")
    pipeline.enable_sequential_cpu_offload()

image = load_image("https://huggingface.co/datasets/nunchaku-tech/test-data/resolve/main/inputs/neon_sign.png")
image = image.convert("RGB")
prompt = "change the text to read '双截棍 Qwen Image Edit is here'"
inputs = {
    "image": image,
    "prompt": prompt,
    "true_cfg_scale": 4.0,
    "negative_prompt": " ",
    "num_inference_steps": 50,
}

output = pipeline(**inputs)
output_image = output.images[0]
output_image.save(f"qwen-image-edit-r{rank}.png")
```

--------------------------------

### Load and Configure Qwen Image Edit Model

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/usage/qwen-image-edit.md

This snippet shows how to set up the scheduler configuration, load the transformer model, and initialize the Qwen Image Edit Plus pipeline. It includes logic for optimizing model loading based on available GPU memory.

```python
import math
import torch

from diffusers import FlowMatchEulerDiscreteScheduler
from nunchaku_ai.pipelines import QwenImageEditPlusPipeline
from nunchaku_ai.models import NunchakuQwenImageTransformer2DModel
from nunchaku_ai.utils import get_precision, get_gpu_memory
from nunchaku_ai.image import load_image

scheduler_config = {
    "base_image_seq_len": 256,
    "base_shift": math.log(3),  # We use shift=3 in distillation
    "invert_sigmas": False,
    "max_image_seq_len": 8192,
    "max_shift": math.log(3),  # We use shift=3 in distillation
    "num_train_timesteps": 1000,
    "shift": 1.0,
    "shift_terminal": None,  # set shift_terminal to None
    "stochastic_sampling": False,
    "time_shift_type": "exponential",
    "use_beta_sigmas": False,
    "use_dynamic_shifting": True,
    "use_exponential_sigmas": False,
    "use_karras_sigmas": False,
}
scheduler = FlowMatchEulerDiscreteScheduler.from_config(scheduler_config)

num_inference_steps = 8  # you can also use the 8-step model to improve the quality
rank = 32  # you can also use the rank=128 model to improve the quality
model_path = f"nunchaku-tech/nunchaku-qwen-image-edit-2509/lightning-251115/svdq-{get_precision()}_r{rank}-qwen-image-edit-2509-lightning-{num_inference_steps}steps-251115.safetensors"

# Load the model
transformer = NunchakuQwenImageTransformer2DModel.from_pretrained(model_path)

pipeline = QwenImageEditPlusPipeline.from_pretrained(
    "Qwen/Qwen-Image-Edit-2509", transformer=transformer, scheduler=scheduler, torch_dtype=torch.bfloat16
)

if get_gpu_memory() > 18:
    pipeline.enable_model_cpu_offload()
else:
    # use per-layer offloading for low VRAM. This only requires 3-4GB of VRAM.
    transformer.set_offload(
        True, use_pin_memory=False, num_blocks_on_gpu=1
    )  # increase num_blocks_on_gpu if you have more VRAM
    pipeline._exclude_from_cpu_offload.append("transformer")
    pipeline.enable_sequential_cpu_offload()

```

--------------------------------

### Verify CUDA Installation

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/installation/setup_windows.md

Run this command in the terminal to check if the CUDA toolkit is installed correctly and accessible.

```bat
nvcc --version
```

--------------------------------

### Load and Run Original Qwen-Image with Nunchaku

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/usage/qwen-image.md

This snippet demonstrates loading a 4-bit quantized Qwen-Image model using Nunchaku's transformer. It includes logic for enabling CPU offloading based on available GPU memory and generates an image from a text prompt. Use this for standard image generation tasks.

```python
import torch
from diffusers import QwenImagePipeline

from nunchaku.models.transformers.transformer_qwenimage import NunchakuQwenImageTransformer2DModel
from nunchaku.utils import get_gpu_memory, get_precision

rank = 32  # you can also use rank=128 model to improve the quality

# Load the model
transformer = NunchakuQwenImageTransformer2DModel.from_pretrained(
    f"nunchaku-tech/nunchaku-qwen-image/svdq-{get_precision()}_r{rank}-qwen-image.safetensors"
)

# currently, you need to use this pipeline to offload the model to CPU
pipe = QwenImagePipeline.from_pretrained("Qwen/Qwen-Image", transformer=transformer, torch_dtype=torch.bfloat16)

if get_gpu_memory() > 18:
    pipe.enable_model_cpu_offload()
else:
    # use per-layer offloading for low VRAM. This only requires 3-4GB of VRAM.
    transformer.set_offload(
        True, use_pin_memory=False, num_blocks_on_gpu=1
    )  # increase num_blocks_on_gpu if you have more VRAM
    pipe._exclude_from_cpu_offload.append("transformer")
    pipe.enable_sequential_cpu_offload()

positive_magic = {
    "en": "Ultra HD, 4K, cinematic composition.",  # for english prompt,
    "zh": "超清，4K，电影级构图",  # for chinese prompt,
}

# Generate image
prompt = """Bookstore window display. A sign displays “New Arrivals This Week”. Below, a shelf tag with the text “Best-Selling Novels Here”. To the side, a colorful poster advertises “Author Meet And Greet on Saturday” with a central portrait of the author. There are four books on the bookshelf, namely “The light between worlds” “When stars are scattered” “The slient patient” “The night circus"""
negative_prompt = " "  # using an empty string if you do not have specific concept to remove

image = pipe(
    prompt=prompt + positive_magic["en"],
    negative_prompt=negative_prompt,
    width=1664,
    height=928,
    num_inference_steps=50,
    true_cfg_scale=4.0,
).images[0]

image.save(f"qwen-image-r{rank}.png")
```

--------------------------------

### Install PyTorch for RTX 50-series GPUs

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/installation/setup_windows.md

Installs the latest nightly build of PyTorch with CUDA 12.8 support, suitable for RTX 50-series GPUs.

```bat
"G:\ComfyuI\python\python.exe" -m pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128
```

--------------------------------

### Set Up Conda Environment for Building

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/installation/installation.md

Creates and activates a Conda environment with Python 3.11. This is the first step for building Nunchaku from source.

```shell
conda create -n nunchaku python=3.11
conda activate nunchaku
```

--------------------------------

### Install PyTorch with CUDA

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/installation/installation.md

Installs PyTorch, torchvision, and torchaudio with CUDA 12.8 support. Ensure your environment meets PyTorch's CUDA requirements.

```shell
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128
```

--------------------------------

### Initialize SVDQW4A4Linear Quantized Layer

Source: https://context7.com/nunchaku-ai/nunchaku/llms.txt

Demonstrates creating a low-level SVDQuant quantized linear layer with W4A4 quantization. Supports direct initialization or conversion from an existing `nn.Linear` layer.

```python
import torch
from nunchaku.models.linear import SVDQW4A4Linear

# Create a quantized linear layer (INT4, rank-32 SVD branch)
layer = SVDQW4A4Linear(
    in_features=3072,
    out_features=3072,
    rank=32,
    precision="int4",         # or "nvfp4" for Blackwell
    torch_dtype=torch.bfloat16,
    device="cuda",
)

# Create from an existing nn.Linear (preserves shape, dummy weights)
import torch.nn as nn
original = nn.Linear(3072, 3072, bias=True)
layer = SVDQW4A4Linear.from_linear(original, rank=32, precision="int4")

# Forward pass (accepts BF16/FP16 input of shape [B, S, in_features])
x = torch.randn(1, 512, 3072, dtype=torch.bfloat16, device="cuda")
# out = layer(x)  # shape: [1, 512, 3072]

print(layer)
# SVDQW4A4Linear(in_features=3072, out_features=3072, rank=32, precision=int4, act_unsigned=False)
```

--------------------------------

### Install Recent GCC/G++ on Linux

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/installation/installation.md

Installs GCC and G++ version 11 on Linux using Conda, if the system's default versions are insufficient for building Nunchaku.

```shell
conda install -c conda-forge gxx=11 gcc=11
```

--------------------------------

### Install Nunchaku Wheel in ComfyUI

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/installation/installation.md

Manually installs a Nunchaku wheel file into a ComfyUI Python environment. Use the Python executable path found in the ComfyUI console log.

```bat
"G:\ComfyUI\python\python.exe" -m pip install <your-wheel-file>.whl
```

--------------------------------

### Build HTML Documentation with Sphinx

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/developer/build_docs.md

Navigate to the docs directory and use make to clean previous builds and generate new HTML documentation. The output will be in the docs/build/html directory.

```shell
cd docs
make clean
make html
```

--------------------------------

### Build Nunchaku Wheel for Distribution

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/installation/installation.md

Builds a distributable wheel file for Nunchaku. Set NUNCHAKU_INSTALL_MODE=ALL to ensure compatibility across supported GPU architectures.

```shell
NUNCHAKU_INSTALL_MODE=ALL NUNCHAKU_BUILD_WHEELS=1 python -m build --wheel --no-isolation
```

--------------------------------

### Set Up ComfyUI Workflows

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/installation/setup_windows.md

Copy the official Nunchaku workflows from the ComfyUI-nunchaku repository into your ComfyUI user workflows directory. This command should be run from the root of your ComfyUI folder.

```bat
# From the root of your ComfyUI folder
cp -r custom_nodes/ComfyUI-nunchaku/example_workflows user/default/workflows/nunchaku_examples
```

--------------------------------

### Check Python Version

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/installation/installation.md

Displays the currently active Python version. This is useful for verifying compatibility when installing Nunchaku wheels.

```shell
python --version
```

--------------------------------

### Docstring Improvement Prompt

Source: https://github.com/nunchaku-ai/nunchaku/blob/main/docs/source/developer/docstring.md

A prompt for improving the writing of a docstring according to a specific guide, emphasizing conciseness and clear organization.

```text
Improve the writing of the docstring according to this guide. Be concise. Organize my comments clearly.

Write the docstring for this module (every class and functions) according to this guide. Each function and class that can be shwon properly and beautifully in sphinx html. Be concise. Organize my comments clearly.
```