### Launch Local Gradio Demo for DemoFusion Source: https://github.com/pris-cv/demofusion/blob/main/README.md Starts the interactive Gradio web interface for DemoFusion. Ensure you have installed 'gradio' and 'gradio_imageslider' packages. This demo provides a user-friendly way to interact with the model. ```bash python gradio_demo.py ``` -------------------------------- ### Launch Local Gradio Demo for Image2Image with DemoFusion Source: https://github.com/pris-cv/demofusion/blob/main/README.md Starts the interactive Gradio web interface for DemoFusion's Image2Image functionality. Requires 'gradio' and 'gradio_imageslider' to be installed. Note that Image2Image performance is sensitive to prompt accuracy and SDXL's training data. ```bash python gradio_demo_img2img.py ``` -------------------------------- ### Install DemoFusion Dependencies Source: https://github.com/pris-cv/demofusion/blob/main/README.md Set up the Python environment and install necessary packages for DemoFusion. Ensure you are using Python 3.9. ```bash conda create -n demofusion python=3.9 conda activate demofusion pip install -r requirements.txt ``` -------------------------------- ### Initialize and Run DemoFusion SDXL Pipeline Source: https://github.com/pris-cv/demofusion/blob/main/demo.ipynb Initializes the DemoFusionSDXLPipeline with a specified model checkpoint and moves it to the CUDA device. It then generates an image using a detailed prompt, negative prompt, and various generation parameters including dimensions, batch size, stride, inference steps, guidance scale, and custom scaling factors for denoising. Ensure CUDA is available and the necessary libraries are installed. ```python import os import torch from PIL import Image import matplotlib.pyplot as plt from pipeline_demofusion_sdxl import DemoFusionSDXLPipeline model_ckpt = "stabilityai/stable-diffusion-xl-base-1.0" pipe = DemoFusionSDXLPipeline.from_pretrained(model_ckpt, torch_dtype=torch.float16) pipe = pipe.to("cuda") prompt = "Envision a portrait of an elderly woman, her face a canvas of time, framed by a headscarf with muted tones of rust and cream. Her eyes, blue like faded denim. Her attire, simple yet dignified." negative_prompt = "blurry, ugly, duplicate, poorly drawn, deformed, mosaic" generator = torch.Generator(device='cuda') generator = generator.manual_seed(522) images = pipe(prompt, negative_prompt=negative_prompt, generator=generator, height=3072, width=3072, view_batch_size=16, stride=64, num_inference_steps=50, guidance_scale = 7.5, cosine_scale_1=3, cosine_scale_2=1, cosine_scale_3=1, sigma=0.8, multi_decoder=True, show_image=True ) ``` -------------------------------- ### Install Dependencies for DemoFusion on Windows Source: https://github.com/pris-cv/demofusion/blob/main/README.md Installs necessary Python packages for running DemoFusion, including specific versions of xformers, diffusers, and transformers. Ensure you have a compatible CUDA version (e.g., cu118) for xformers. ```bash git clone "https://github.com/PRIS-CV/DemoFusion" cd DemoFusion python -m venv venv venv\Scripts\activate pip install -U "xformers==0.0.22.post7+cu118" --index-url https://download.pytorch.org/whl/cu118 pip install "diffusers==0.21.4" "matplotlib==3.8.2" "transformers==4.35.2" "accelerate==0.25.0" ``` -------------------------------- ### Text-to-Image Generation with DemoFusion (Low VRAM) Source: https://github.com/pris-cv/demofusion/blob/main/README.md Launches the text-to-image generation pipeline using DemoFusion's low VRAM optimized script. This example demonstrates generating a high-resolution image with specific parameters for VAE, model checkpoint, and generation settings. ```python from pipeline_demofusion_sdxl import DemoFusionSDXLPipeline import torch from diffusers.models import AutoencoderKL vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16) model_ckpt = "stabilityai/stable-diffusion-xl-base-1.0" pipe = DemoFusionSDXLPipeline.from_pretrained(model_ckpt, torch_dtype=torch.float16, vae=vae) pipe = pipe.to("cuda") prompt = "Envision a portrait of an elderly woman, her face a canvas of time, framed by a headscarf with muted tones of rust and cream. Her eyes, blue like faded denim. Her attire, simple yet dignified." negative_prompt = "blurry, ugly, duplicate, poorly drawn, deformed, mosaic" images = pipe(prompt, negative_prompt=negative_prompt, height=2048, width=2048, view_batch_size=4, stride=64, num_inference_steps=40, guidance_scale=7.5, cosine_scale_1=3, cosine_scale_2=1, cosine_scale_3=1, sigma=0.8, multi_decoder=True, show_image=False, lowvram=True ) for i, image in enumerate(images): image.save('image_' + str(i) + '.png') ``` -------------------------------- ### Text-to-Image Generation with DemoFusion Source: https://github.com/pris-cv/demofusion/blob/main/README.md Load the DemoFusion SDXL pipeline and generate images from a text prompt. Adjust parameters like height, width, and inference steps for desired output. For resolutions exceeding 3072*3072 on an RTX 3090, `multi_decoder=True` is generally required. ```python from pipeline_demofusion_sdxl import DemoFusionSDXLPipeline import torch model_ckpt = "stabilityai/stable-diffusion-xl-base-1.0" pipe = DemoFusionSDXLPipeline.from_pretrained(model_ckpt, torch_dtype=torch.float16) pipe = pipe.to("cuda") prompt = "Envision a portrait of an elderly woman, her face a canvas of time, framed by a headscarf with muted tones of rust and cream. Her eyes, blue like faded denim. Her attire, simple yet dignified." negative_prompt = "blurry, ugly, duplicate, poorly drawn, deformed, mosaic" images = pipe(prompt, negative_prompt=negative_prompt, height=3072, width=3072, view_batch_size=16, stride=64, num_inference_steps=50, guidance_scale=7.5, cosine_scale_1=3, cosine_scale_2=1, cosine_scale_3=1, sigma=0.8, multi_decoder=True, show_image=True ) for i, image in enumerate(images): image.save('image_' + str(i) + '.png') ``` -------------------------------- ### Create an Image Grid from Generated Images Source: https://github.com/pris-cv/demofusion/blob/main/demo.ipynb A utility function to arrange multiple generated images into a single grid. It calculates the total width required based on the input images and pastes them sequentially. The function can optionally save each individual image with a timestamped filename if a save path is provided. This is useful for visualizing multiple outputs or intermediate steps. ```python def image_grid(imgs, save_path=None): w = 0 for i, img in enumerate(imgs): h_, w_ = imgs[i].size w += w_ h = h_ grid = Image.new('RGB', size=(w, h)) grid_w, grid_h = grid.size w = 0 for i, img in enumerate(imgs): h_, w_ = imgs[i].size grid.paste(img, box=(w, h - h_)) if save_path != None: img.save(save_path + "/img_{}.jpg".format((i + 1) * 1024)) w += w_ return grid image_grid(images, save_path="./outputs/") ``` -------------------------------- ### DemoFusion Citation Source: https://github.com/pris-cv/demofusion/blob/main/README.md BibTeX entry for citing the DemoFusion paper in academic research. Use this in your LaTeX documents when referencing the DemoFusion project. ```bibtex @inproceedings{du2024demofusion, title={DemoFusion: Democratising High-Resolution Image Generation With No \$\$\$}, author={Du, Ruoyi and Chang, Dongliang and Hospedales, Timothy and Song, Yi-Zhe and Ma, Zhanyu}, booktitle={CVPR}, year={2024} } ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.