### Install Dependencies Source: https://github.com/zxyin/consistedit_code/blob/main/README.md Install all required Python packages using pip. ```bash pip install -r requirements.txt ``` -------------------------------- ### Use SD3 Solver's Denoise Method Source: https://context7.com/zxyin/consistedit_code/llms.txt Demonstrates using the denoise() method added by the SD3 solver for image or video generation. Specify prompts, starting latents, guidance scale, and inference steps. ```python # Assuming pipe is already registered with register_sd3_solver # prompts, start_code, etc. are defined elsewhere images = pipe.denoise( prompts, latents=start_code, guidance_scale=7.5, num_inference_steps=28, output_type="pil" # or "latent" for latent output ).images ``` -------------------------------- ### SD3 Image Synthesis Editing with ConsistEdit Source: https://context7.com/zxyin/consistedit_code/llms.txt Use this script to perform consistent editing on images synthesized by Stable Diffusion 3. Specify source and target prompts, the object to edit, and an alpha value for consistency strength. Ensure the model path is correctly set and necessary libraries are installed. ```python # run_synthesis_sd3.py - Basic SD3 consistent editing import torch from diffusers import StableDiffusion3Pipeline from consistEdit.global_var import GlobalVars from consistEdit.attention_control import regiter_attention_editor_diffusers_sd3_mask from consistEdit.solver import register_sd3_solver from consistEdit.utils import get_t5_token_indices, get_clip_token_indices, setup_seed # Initialize pipeline model_path = "/path/to/stable-diffusion-3-medium-diffusers" dtype = torch.float16 device = torch.device("cuda") pipe = StableDiffusion3Pipeline.from_pretrained(model_path, torch_dtype=dtype).to(device) register_sd3_solver(pipe) regiter_attention_editor_diffusers_sd3_mask(pipe) # Configure editing parameters setup_seed(42) src_prompt = "a portrait of a woman in a red dress in a forest, best quality" tgt_prompt = "a portrait of a woman in a yellow dress in a forest, best quality" edit_object = "dress" alpha = 1.0 # Consistency strength (0.0-1.0) # Set up latent space width, height = 1024, 1024 total_steps = 28 vae_scale_factor, channel_num = 8, 16 start_code = torch.randn([1, channel_num, width // vae_scale_factor, height // vae_scale_factor], device=device, dtype=dtype) start_code = start_code.expand(2, -1, -1, -1) # Configure global variables GlobalVars.WIDTH = width // 8 // 2 GlobalVars.HEIGHT = height // 8 // 2 GlobalVars.TEST_QK_STEP = int(total_steps * alpha) # Get token indices for masking tokenizer_path = f"{model_path}/tokenizer" clip_indices = get_clip_token_indices(pipe, src_prompt, edit_object, tokenizer_path) t5_indices = [x + 77 for x in get_t5_token_indices(pipe, src_prompt, edit_object)] GlobalVars.TOKEN_IDS = clip_indices + t5_indices # Generate mask GlobalVars.GENERATE_MASK = True GlobalVars.MASK_OUTPUT_PATH = "output/mask.png" pipe.denoise([src_prompt], latents=start_code[:1], guidance_scale=7.5) # Generate edited image GlobalVars.GENERATE_MASK = False prompts = [src_prompt, tgt_prompt] images = pipe.denoise(prompts, latents=start_code, guidance_scale=7.5).images images[0].save("output/source.png") # Original images[1].save("output/target.png") # Edited ``` -------------------------------- ### FLUX Image Synthesis Editing with ConsistEdit Source: https://context7.com/zxyin/consistedit_code/llms.txt Perform consistent editing on images synthesized by FLUX.1-dev using this script. It includes memory optimization via sequential CPU offloading and uses T5 tokenizer. Ensure the model path is correct and necessary libraries are installed. ```python # run_synthesis_flux.py - FLUX consistent editing import torch from diffusers import FluxPipeline from consistEdit.global_var import GlobalVars from consistEdit.attention_control import regiter_attention_editor_diffusers_flux_mask from consistEdit.solver import register_flux_solver from consistEdit.utils import get_t5_token_indices, setup_seed # Initialize FLUX pipeline with memory optimization model_path = "/path/to/FLUX.1-dev" dtype = torch.float16 device = torch.device("cuda") pipe = FluxPipeline.from_pretrained(model_path, torch_dtype=dtype).to(device) pipe.enable_sequential_cpu_offload() # Memory optimization register_flux_solver(pipe) regiter_attention_editor_diffusers_flux_mask(pipe) # Configure editing setup_seed(42) src_prompt = "a portrait of a woman in a red dress in a forest, best quality" tgt_prompt = "a portrait of a woman in a yellow dress in a forest, best quality" edit_object = "dress" alpha = 1.0 # Set up latents (FLUX uses packed latents) width, height = 1024, 1024 steps = 28 vae_scale_factor, channel_num = 8, 16 start_code = torch.randn([1, channel_num, width // vae_scale_factor, height // vae_scale_factor], device=device, dtype=dtype) start_code = start_code.expand(2, -1, -1, -1) start_code = pipe._pack_latents(start_code, 2, channel_num, width // vae_scale_factor, height // vae_scale_factor) ``` -------------------------------- ### Get T5 Token Indices for CogVideoX Source: https://context7.com/zxyin/consistedit_code/llms.txt Retrieves T5 tokenizer indices specifically for the CogVideoX pipeline, used in conjunction with its tokenizer path for mask generation. ```python from consistEdit.utils import get_cog_t5_token_indices # Assuming pipe, prompt, edit_word, tokenizer_path are defined cog_indices = get_cog_t5_token_indices(pipe, prompt, edit_word, tokenizer_path) # GlobalVars.TOKEN_IDS = cog_indices ``` -------------------------------- ### Get T5 Token Indices for SD3/FLUX Source: https://context7.com/zxyin/consistedit_code/llms.txt Retrieves T5 tokenizer indices for a prompt and edit word, with an offset for SD3's combined embedding. For FLUX, use the is_flux=True flag. ```python from consistEdit.utils import get_t5_token_indices # For SD3: # prompt = "a portrait of a woman in a red dress in a forest, best quality" # edit_word = "dress" # t5_indices = [x + 77 for x in get_t5_token_indices(pipe, prompt, edit_word)] # Example output: [85] # For FLUX: # flux_indices = get_t5_token_indices(pipe, prompt, edit_word, is_flux=True) # GlobalVars.TOKEN_IDS = flux_indices ``` -------------------------------- ### Get CLIP Token Indices for SD3 Source: https://context7.com/zxyin/consistedit_code/llms.txt Retrieves CLIP tokenizer indices for a given prompt and edit word, used for SD3 mask generation. The tokenizer path must be specified. ```python from consistEdit.utils import get_clip_token_indices # Assuming pipe, prompt, edit_word, tokenizer_path are defined clip_indices = get_clip_token_indices(pipe, prompt, edit_word, tokenizer_path) # Example output: [8] ``` -------------------------------- ### Inconsistent Editing Command Line Source: https://context7.com/zxyin/consistedit_code/llms.txt Perform inconsistent editing, allowing for style changes by using a lower alpha value. This example demonstrates editing a dress from a realistic to a cartoon style. ```bash python run_synthesis_sd3.py \ --src_prompt "a portrait of a woman in a red dress, realistic style, best quality" \ --tgt_prompt "a portrait of a woman in a yellow dress, cartoon style, best quality" \ --edit_object "dress" \ --out_dir "output" \ --alpha 0.3 \ --no_mask \ --model_path "/path/to/stable-diffusion-3-medium-diffusers" ``` -------------------------------- ### Initialize and Configure CogVideoX Pipeline Source: https://context7.com/zxyin/consistedit_code/llms.txt Sets up the CogVideoX pipeline for video editing, including model loading, device configuration, and registering necessary attention control and solver functions. Ensure the model path is correct and the device is available. ```python # run_synthesis_cog.py - CogVideoX video editing import torch import numpy as np from diffusers import CogVideoXPipeline from consistEdit.global_var import GlobalVars from consistEdit.attention_control import regiter_attention_editor_diffusers_cog_mask from consistEdit.solver import register_cog_solver from consistEdit.utils import get_cog_t5_token_indices, save_numpy_arrays_as_mp4, setup_seed # Initialize CogVideoX pipeline model_path = "/path/to/CogVideoX-2b" dtype = torch.float16 device = torch.device("cuda") pipe = CogVideoXPipeline.from_pretrained(model_path, torch_dtype=dtype).to(device) pipe.enable_sequential_cpu_offload() register_cog_solver(pipe) regiter_attention_editor_diffusers_cog_mask(pipe) # Configure video editing setup_seed(42) src_prompt = "a portrait of a woman in a red dress in a forest, best quality" tgt_prompt = "a portrait of a woman in a yellow dress in a forest, best quality" edit_object = "dress" alpha = 1.0 # Video dimensions width, height = 720, 480 steps = 50 num_frames = 49 T = (num_frames - 1) // 4 + 1 # Temporal latent frames vae_scale_factor, channel_num = 8, 16 # Set up video latents start_code = torch.randn([1, T, channel_num, height // vae_scale_factor, width // vae_scale_factor], device=device, dtype=dtype) start_code = start_code.expand(2, -1, -1, -1, -1) # Configure globals for CogVideoX GlobalVars.WIDTH = width // 8 // 2 GlobalVars.HEIGHT = height // 8 // 2 GlobalVars.TEXT_LENGTH = 226 GlobalVars.TEST_QK_STEP = int(steps * alpha) # Get token indices tokenizer_path = f"{model_path}/tokenizer" GlobalVars.TOKEN_IDS = get_cog_t5_token_indices(pipe, src_prompt, edit_object, tokenizer_path) # Generate frame masks GlobalVars.GENERATE_MASK = True GlobalVars.MASK_OUTPUT_PATH = "output/mask" # Directory for frame masks pipe.denoise([src_prompt], latents=start_code[:1], guidance_scale=6, num_inference_steps=steps) # Generate edited video GlobalVars.GENERATE_MASK = False prompts = [src_prompt, tgt_prompt] videos = pipe.denoise(prompts, latents=start_code, guidance_scale=6, num_inference_steps=steps).frames # Save videos save_numpy_arrays_as_mp4(np.clip(videos[0], 0, 255), "output/source.mp4") save_numpy_arrays_as_mp4(np.clip(videos[1], 0, 255), "output/target.mp4") ``` -------------------------------- ### Calculate PIE-Bench Evaluation Metrics Source: https://context7.com/zxyin/consistedit_code/llms.txt Command-line script to calculate evaluation metrics for the PIE-Bench dataset. Requires annotation mapping, source/mask image folders, and specified metrics. ```bash python evaluate_sd3.py \ --annotation_mapping_file "/path/to/pie-bench/mapping_file.json" \ --src_image_folder "ours/source_images" \ --mask_image_folder "ours/dilate_mask_images" \ --tgt_methods ours \ --result_path "evaluation_result.csv" \ --metrics canny_ssim structure_distance psnr_unedit_part ssim_unedit_part \ clip_similarity_target_image clip_similarity_target_image_edit_part \ --edit_category_list 0 1 2 3 4 5 6 7 8 9 ``` -------------------------------- ### Run PIE-Bench Evaluation Source: https://github.com/zxyin/consistedit_code/blob/main/README.md Use this script to process the PIE-Bench dataset and generate edited images for quantitative evaluation. Ensure you provide the correct paths to your model and the dataset. ```bash python run_metric.py \ --model_path "/path/to/stable-diffusion-3-medium-diffusers" \ --data_path "/path/to/pie-bench-dataset" ``` -------------------------------- ### Manual Synthesis with Stable Diffusion 3 (Color Change) Source: https://github.com/zxyin/consistedit_code/blob/main/README.md Perform manual synthesis for a color change using Stable Diffusion 3. Ensure the model path is correctly set. ```python python run_synthesis_sd3.py \ --src_prompt "a portrait of a woman in a red dress in a forest, best quality" \ --tgt_prompt "a portrait of a woman in a yellow dress in a forest, best quality" \ --edit_object "dress" \ --out_dir "output" \ --alpha 1.0 \ --model_path "/path/to/stable-diffusion-3-medium-diffusers" ``` -------------------------------- ### Manual Synthesis with CogVideo Source: https://github.com/zxyin/consistedit_code/blob/main/README.md Perform manual synthesis for a color change using the CogVideo model. Ensure the model path is correctly set. ```python python run_synthesis_cog.py \ --src_prompt "a portrait of a woman in a red dress in a forest, best quality" \ --tgt_prompt "a portrait of a woman in a yellow dress in a forest, best quality" \ --edit_object "dress" \ --out_dir "output" \ --alpha 1.0 \ --model_path "/path/to/CogVideoX-2b" ``` -------------------------------- ### Initialize and Configure Stable Diffusion 3 Pipeline for Real Image Editing Source: https://context7.com/zxyin/consistedit_code/llms.txt Sets up the Stable Diffusion 3 pipeline for editing real images using DDIM inversion. It initializes the pipeline with dual schedulers and registers necessary attention control and solver functions. Note the lower alpha value for real images to mitigate inversion information leakage. ```python # run_real_sd3.py - Real image editing with inversion import torch from PIL import Image from diffusers import StableDiffusion3Pipeline from diffusers.schedulers import FlowMatchEulerDiscreteScheduler from consistEdit.global_var import GlobalVars from consistEdit.attention_control import regiter_attention_editor_diffusers_sd3_mask_real from consistEdit.solver import register_sd3_solver from consistEdit.scheduler import UniInvEulerScheduler from consistEdit.utils import get_t5_token_indices, get_clip_token_indices, latent2image, setup_seed # Initialize pipeline with dual schedulers model_path = "/path/to/stable-diffusion-3-medium-diffusers" dtype = torch.float16 device = torch.device("cuda") pipe = StableDiffusion3Pipeline.from_pretrained(model_path, torch_dtype=dtype).to(device) scheduler = FlowMatchEulerDiscreteScheduler.from_pretrained(model_path, subfolder='scheduler') invert_scheduler = UniInvEulerScheduler.from_pretrained(model_path, subfolder='scheduler') register_sd3_solver(pipe) regiter_attention_editor_diffusers_sd3_mask_real(pipe) # Configure editing setup_seed(42) src_prompt = "a girl with a red hat and red t-shirt is sitting in a park, best quality" tgt_prompt = "a girl with a yellow hat and red t-shirt is sitting in a park, best quality" edit_object = "hat" alpha = 0.1 # Lower alpha for real images due to inversion information leakage ``` -------------------------------- ### Use SD3 Solver's Invert Method for Editing Source: https://context7.com/zxyin/consistedit_code/llms.txt Shows how to use the invert() method from the SD3 solver for real image editing via DDIM inversion. This method takes source prompts and latents to prepare for editing. ```python # Assuming pipe is already registered with register_sd3_solver # src_prompt, image_latent are defined elsewhere inverted_latent = pipe.invert( [src_prompt], latents=image_latent, guidance_scale=1, num_inference_steps=28, output_type="latent" ).images ``` -------------------------------- ### Manual Synthesis with FLUX Source: https://github.com/zxyin/consistedit_code/blob/main/README.md Perform manual synthesis for a color change using the FLUX model. Ensure the model path is correctly set. ```python python run_synthesis_flux.py \ --src_prompt "a portrait of a woman in a red dress in a forest, best quality" \ --tgt_prompt "a portrait of a woman in a yellow dress in a forest, best quality" \ --edit_object "dress" \ --out_dir "output" \ --alpha 1.0 \ --model_path "/path/to/FLUX.1-dev" ``` -------------------------------- ### Run Consistent Editing Script (SD3) Source: https://github.com/zxyin/consistedit_code/blob/main/README.md Execute the script for consistent editing using Stable Diffusion 3, suitable for changing color or material. ```bash bash script/sd3_consist_edit.sh ``` -------------------------------- ### Manual Synthesis with Stable Diffusion 3 (Style Change) Source: https://github.com/zxyin/consistedit_code/blob/main/README.md Perform manual synthesis for a style change using Stable Diffusion 3. Adjust alpha for blending effect. Ensure the model path is correctly set. ```python python run_synthesis_sd3.py \ --src_prompt "a portrait of a woman in a red dress, realistic style, best quality" \ --tgt_prompt "a portrait of a woman in a yellow dress, cartoon style, best quality" \ --edit_object "dress" \ --out_dir "output" \ --alpha 0.3 \ --model_path "/path/to/stable-diffusion-3-medium-diffusers" ``` -------------------------------- ### Calculate Evaluation Metrics Source: https://github.com/zxyin/consistedit_code/blob/main/README.md This command executes the script for computing evaluation metrics for the edited images. ```bash python evaluate_sd3.py ``` -------------------------------- ### Register Cog Solver for CogVideoX Source: https://context7.com/zxyin/consistedit_code/llms.txt Registers the Cog solver, enabling denoise() and invert() methods on the CogVideoX pipeline. This facilitates image/video generation and DDIM inversion. ```python from diffusers import CogVideoXPipeline from consistEdit.solver import register_cog_solver model_path = "path/to/your/model" pipe = CogVideoXPipeline.from_pretrained(model_path, torch_dtype=torch.float16) register_cog_solver(pipe) ``` -------------------------------- ### Generate Edited Images for PIE-Bench Source: https://context7.com/zxyin/consistedit_code/llms.txt Command-line script to generate edited images for the PIE-Bench dataset. Specify model, data, and output paths, along with edit categories. ```bash python run_metric.py \ --model_path "/path/to/stable-diffusion-3-medium-diffusers" \ --data_path "/path/to/pie-bench-dataset" \ --output_path "result_output" \ --edit_category_list 0 1 2 3 4 5 6 7 8 9 ``` -------------------------------- ### Register SD3 Solver for Denoising and Inversion Source: https://context7.com/zxyin/consistedit_code/llms.txt Registers the SD3 solver, adding denoise() and invert() methods to the StableDiffusion3Pipeline. This enables forward diffusion for generation and backward diffusion for DDIM inversion. ```python from diffusers import StableDiffusion3Pipeline from consistEdit.solver import register_sd3_solver model_path = "path/to/your/model" pipe = StableDiffusion3Pipeline.from_pretrained(model_path, torch_dtype=torch.float16) register_sd3_solver(pipe) ``` -------------------------------- ### Register CogVideoX Pipeline with Mask Support Source: https://context7.com/zxyin/consistedit_code/llms.txt Registers the CogVideoX pipeline with mask support, using vanilla attention for improved consistency. The model path must be valid. ```python from diffusers import CogVideoXPipeline from consistEdit.editor import regiter_attention_editor_diffusers_cog_mask model_path = "path/to/your/model" pipe = CogVideoXPipeline.from_pretrained(model_path, torch_dtype=torch.float16) regiter_attention_editor_diffusers_cog_mask(pipe, use_old_version=False) ``` -------------------------------- ### Run Inconsistent Editing Script (SD3) Source: https://github.com/zxyin/consistedit_code/blob/main/README.md Execute the script for inconsistent editing using Stable Diffusion 3, suitable for changing style or objects. ```bash bash script/sd3_inconsist_edit.sh ``` -------------------------------- ### Register FLUX Solver Source: https://context7.com/zxyin/consistedit_code/llms.txt Registers the FLUX solver, adding denoise() and invert() methods to the FLUX pipeline for diffusion and inversion tasks. Ensure the pipeline is loaded correctly. ```python from diffusers import FluxPipeline from consistEdit.solver import register_flux_solver model_path = "path/to/your/model" pipe = FluxPipeline.from_pretrained(model_path, torch_dtype=torch.float16) register_flux_solver(pipe) ``` -------------------------------- ### Load and Encode Source Image for Editing Source: https://context7.com/zxyin/consistedit_code/llms.txt This Python code loads a source image, resizes it, converts it to RGB, and then encodes it into a latent representation suitable for diffusion models. It also configures global variables for editing parameters. ```python source_image_path = "assets/red_hat_girl.png" width, height = 1024, 1024 steps = 28 source_image = Image.open(source_image_path).resize((width, height)).convert("RGB") source_tensor = pipe.image_processor.preprocess(source_image).to(dtype=dtype, device=device) image_latent = pipe.vae.encode(source_tensor)['latent_dist'].mean image_latent = (image_latent - pipe.vae.config.shift_factor) * pipe.vae.config.scaling_factor # Configure globals GlobalVars.WIDTH = width // 8 // 2 GlobalVars.HEIGHT = height // 8 // 2 GlobalVars.TEXT_LENGTH = 333 GlobalVars.TEST_QK_STEP = int(steps * alpha) # Get token indices tokenizer_path = f"{model_path}/tokenizer" clip_indices = get_clip_token_indices(pipe, src_prompt, edit_object, tokenizer_path) t5_indices = [x + 77 for x in get_t5_token_indices(pipe, src_prompt, edit_object)] GlobalVars.TOKEN_IDS = clip_indices + t5_indices GlobalVars.MASK_OUTPUT_PATH = "output/mask.png" # Invert the image to get starting latent pipe.scheduler = invert_scheduler start_latent = pipe.invert([src_prompt], latents=image_latent, guidance_scale=1, num_inference_steps=steps, output_type="latent").images torch.save(start_latent, "output/latent.pt") # Generate edited image target_prompts = [src_prompt, tgt_prompt] start_latent = start_latent.expand(2, -1, -1, -1) pipe.scheduler = scheduler image_latents = pipe.denoise(target_prompts, latents=start_latent, num_inference_steps=steps, guidance_scale=1, output_type="latent").images source_image = latent2image(pipe, image_latents[:1], device, dtype) source_image.save("output/source.png") edit_image = latent2image(pipe, image_latents[1:2], device, dtype) edit_image.save("output/target.png") ``` -------------------------------- ### Configure Globals for FLUX Source: https://context7.com/zxyin/consistedit_code/llms.txt Sets global variables for image dimensions, text length, and token generation steps specific to the FLUX model. Ensure GlobalVars are configured before generating masks or edited images. ```python GlobalVars.WIDTH = width // 8 // 2 GlobalVars.HEIGHT = height // 8 // 2 GlobalVars.TEXT_LENGTH = 512 # FLUX uses 512 text tokens GlobalVars.TEST_QK_STEP = int(steps * alpha) # Get T5 token indices (FLUX uses tokenizer_2) GlobalVars.TOKEN_IDS = [x for x in get_t5_token_indices(pipe, src_prompt, edit_object, is_flux=True)] # Generate mask GlobalVars.GENERATE_MASK = True GlobalVars.MASK_OUTPUT_PATH = "output/mask.png" pipe.denoise([src_prompt], width=width, height=height, latents=start_code[:1], guidance_scale=7.5, num_inference_steps=steps) # Generate edited image GlobalVars.GENERATE_MASK = False prompts = [src_prompt, tgt_prompt] images = pipe.denoise(prompts, width=width, height=height, latents=start_code, guidance_scale=7.5, num_inference_steps=steps).images images[0].save("output/source.png") images[1].save("output/target.png") ``` -------------------------------- ### Register Attention Editors for Diffusion Models Source: https://context7.com/zxyin/consistedit_code/llms.txt This Python code demonstrates how to register custom attention control mechanisms for various diffusion models, including SD3, FLUX, and CogVideoX. It shows how to enable masking for more precise editing. ```python # Register attention editors for different models from consistEdit.attention_control import ( regiter_attention_editor_diffusers_sd3, # SD3 without masking regiter_attention_editor_diffusers_sd3_mask, # SD3 with masking (recommended) regiter_attention_editor_diffusers_sd3_mask_real,# SD3 for real images regiter_attention_editor_diffusers_flux, # FLUX without masking regiter_attention_editor_diffusers_flux_mask, # FLUX with masking (recommended) regiter_attention_editor_diffusers_cog, # CogVideoX without masking regiter_attention_editor_diffusers_cog_mask, # CogVideoX with masking (recommended) ) # Example: Register SD3 with mask support pipe = StableDiffusion3Pipeline.from_pretrained(model_path, torch_dtype=torch.float16) regiter_attention_editor_diffusers_sd3_mask(pipe, use_old_version=False) # use_old_version=True: Uses scaled dot-product attention (faster but less accurate) ``` -------------------------------- ### Disable Masking with No Mask Option Source: https://github.com/zxyin/consistedit_code/blob/main/README.md Run synthesis disabling all masking and content fusion using the `--no_mask` flag. This may lead to uncontrolled color changes in non-editing regions. ```bash python run_synthesis_sd3.py --no_mask --alpha 0.3 ... ``` -------------------------------- ### Configure GlobalVars for Editing Source: https://context7.com/zxyin/consistedit_code/llms.txt Sets global configuration variables for editing behavior, including dimensions, text length, attention steps, and mask generation parameters. Ensure values are model-specific. ```python from consistEdit.global_var import GlobalVars # Image/video dimensions (latent space) GlobalVars.WIDTH = 64 # Width / 8 / 2 for SD3/FLUX GlobalVars.HEIGHT = 64 # Height / 8 / 2 for SD3/FLUX # Text sequence length (model-specific) # GlobalVars.TEXT_LENGTH = 333 # SD3 # GlobalVars.TEXT_LENGTH = 512 # FLUX # GlobalVars.TEXT_LENGTH = 226 # CogVideoX # Attention injection steps GlobalVars.TEST_QK_STEP = 28 # Higher = more consistency, less editing # Mask generation GlobalVars.GENERATE_MASK = True GlobalVars.MASK_OUTPUT_PATH = "output/mask.png" GlobalVars.TOKEN_IDS = [8, 85] # Token indices for mask generation # Reset all variables # GlobalVars.reset_global_vars() # Get attention controller # attn_controller = GlobalVars.get_attn_controller() ``` -------------------------------- ### Real Image Editing with Stable Diffusion 3 Source: https://github.com/zxyin/consistedit_code/blob/main/README.md Edit a real image using Stable Diffusion 3, specifying the source image and the object to edit. Adjust alpha for desired effect. Ensure the model path is correctly set. ```python python run_real_sd3.py \ --src_prompt "a girl with a red hat and red t-shirt is sitting in a park, best quality" \ --tgt_prompt "a girl with a yellow hat and red t-shirt is sitting in a park, best quality" \ --edit_object "hat" \ --source_image_path "assets/red_hat_girl.png" \ --out_dir "output" \ --alpha 0.1 \ --model_path "/path/to/stable-diffusion-3-medium-diffusers" ``` -------------------------------- ### Convert Latent Image to PIL Image Source: https://context7.com/zxyin/consistedit_code/llms.txt Converts latent image output from a denoise process to a PIL image. Supports custom shapes for non-square images. Requires torch and the latent2image utility. ```python image_latents = pipe.denoise(prompts, latents=start_code, guidance_scale=7.5, output_type="latent").images device = torch.device("cuda") dtype = torch.float16 pil_image = latent2image(pipe, image_latents[:1], device, dtype) pil_image.save("output/result.png") ``` ```python pil_image = latent2image(pipe, image_latents[:1], device, dtype, custom_shape=(height, width)) ``` -------------------------------- ### Convert Latent Tensors to PIL Images Source: https://context7.com/zxyin/consistedit_code/llms.txt Utility function to convert latent tensors back into PIL images after diffusion editing processes. Ensure the consistEdit.utils module is imported. ```python from consistEdit.utils import latent2image # Assuming latent_tensor is available # pil_image = latent2image(latent_tensor) ``` -------------------------------- ### Register FLUX Pipeline with Mask Support Source: https://context7.com/zxyin/consistedit_code/llms.txt Registers the FLUX pipeline with mask support, utilizing vanilla attention for consistency. Ensure the model path is correctly specified. ```python from diffusers import FluxPipeline from consistEdit.editor import regiter_attention_editor_diffusers_flux_mask model_path = "path/to/your/model" pipe = FluxPipeline.from_pretrained(model_path, torch_dtype=torch.float16) regiter_attention_editor_diffusers_flux_mask(pipe, use_old_version=False) ``` -------------------------------- ### Save Numpy Frames as MP4 Video Source: https://context7.com/zxyin/consistedit_code/llms.txt Saves a list of numpy frame arrays as an MP4 video using the libx264 encoder. Ensure frames are in (H, W, 3) uint8 format and within the 0-255 range. ```python from consistEdit.utils import save_numpy_arrays_as_mp4 import numpy as np # frames: list of numpy arrays, each (H, W, 3) uint8 frames = videos[0] # Output from CogVideoX pipeline frames = np.clip(frames, 0, 255) # Ensure valid range # Save as MP4 with libx264 encoding save_numpy_arrays_as_mp4(frames, "output/video.mp4", fps=8) ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.