### Install Compel and dependencies Source: https://github.com/damian0815/compel/blob/main/compel-demo.ipynb Install the required packages using pip. ```bash !pip install torch diffusers transformers compel ``` -------------------------------- ### Style Prompt with SDXL Source: https://github.com/damian0815/compel/blob/main/README.md Demonstrates how to use style prompts with SDXL models using the CompelForSDXL class. Ensure you have the necessary libraries installed and models downloaded. ```python from diffusers import DiffusionPipeline from compel import CompelForSDXL import torch device = 'cuda' pipeline = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", variant="fp16", use_safetensors=True, torch_dtype=torch.float16).to(device) prompt = "a cat playing with a ball++ in the forest" style_prompt = "painting by van gogh, impasto, thick brush strokes" negative_prompt = "low quality, blurry" negative_style_prompt = "photography" compel = CompelForSDXL(pipeline) # or CompelForFlux conditioning = compel(prompt=prompt, negative_prompt=negative_prompt, style_prompt=style_prompt, negative_style_prompt=negative_style_prompt) generator = torch.Generator().manual_seed(42) image = pipeline(prompt_embeds=conditioning.embeds, pooled_prompt_embeds=conditioning.pooled_embeds, negative_prompt_embeds=conditioning.negative_embeds, negative_pooled_prompt_embeds=conditioning.negative_pooled_embeds, num_inference_steps=25, width=1024, height=1024, generator=generator).images[0] image.save('sdxl_image.jpg') ``` -------------------------------- ### Basic Prompting with Compel for Stable Diffusion Source: https://github.com/damian0815/compel/blob/main/README.md Demonstrates how to use Compel to upweight a specific word in a prompt for Stable Diffusion. Ensure you have diffusers version 0.12 or higher installed. ```python from diffusers import StableDiffusionPipeline from compel import CompelForSD pipeline = StableDiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5") compel = CompelForSD(pipeline) # upweight "ball" prompt = "a cat playing with a ball++ in the forest" conditioning = compel(prompt) # generate image images = pipeline(prompt_embeds=conditioning.embeds, num_inference_steps=20).images images[0].save("image.jpg") ``` -------------------------------- ### Build Conditioning Tensor with Modified Prompt Source: https://github.com/damian0815/compel/blob/main/compel-demo.ipynb Build a conditioning tensor using Compel with a modified prompt. This example demonstrates how to use Compel to process prompts with special characters or syntax, such as '++' and '-----', which might influence the generation process. ```python prompt = "a cat playing with a ball++ (in the forest)-----" seed = 123 embeds = compel.build_conditioning_tensor(prompt) fixed_seed_generator = Generator(device="cpu" if compel.device.type == "mps" else compel.device).manual_seed(seed) image = pipeline(prompt_embeds=embeds, num_inference_steps=7, generator=fixed_seed_generator).images[0] display(image) ``` -------------------------------- ### Initialize Compel with SDXL Pipeline Source: https://github.com/damian0815/compel/blob/main/README.md Demonstrates how to initialize the Compel library with an SDXL diffusion pipeline. Ensure to pass device='cuda' for optimal performance. ```python from compel import Compel, ReturnedEmbeddingsType from diffusers import DiffusionPipeline import torch pipeline = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", variant="fp16", use_safetensors=True, torch_dtype=torch.float16).to("cuda") compel = Compel(tokenizer=[pipeline.tokenizer, pipeline.tokenizer_2] , text_encoder=[pipeline.text_encoder, pipeline.text_encoder_2], returned_embeddings_type=ReturnedEmbeddingsType.PENULTIMATE_HIDDEN_STATES_NON_NORMALIZED, requires_pooled=[False, True]) # upweight "ball" prompt = "a cat playing with a ball++ in the forest" conditioning, pooled = compel(prompt) # generate image image = pipeline(prompt_embeds=conditioning, pooled_prompt_embeds=pooled, num_inference_steps=30).images[0] ``` -------------------------------- ### Initialize FLUX.1 Pipeline Source: https://github.com/damian0815/compel/blob/main/compel-demo-flux.ipynb Load the FLUX.1-schnell model and move it to the MPS device. ```python from diffusers import DiffusionPipeline import torch pipe = DiffusionPipeline.from_pretrained("black-forest-labs/FLUX.1-schnell", torch_dtype=torch.bfloat16) pipe = pipe.to('mps') ``` -------------------------------- ### Initialize Stable Diffusion pipeline Source: https://github.com/damian0815/compel/blob/main/compel-demo.ipynb Import necessary modules and initialize the Stable Diffusion pipeline on the specified device. ```python import torch from compel import Compel, SplitLongTextMode from diffusers import StableDiffusionPipeline, DPMSolverMultistepScheduler from torch import Generator device = "mps" pipeline: StableDiffusionPipeline = StableDiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5").to(device) ``` -------------------------------- ### Initialize SDXL Pipeline Source: https://github.com/damian0815/compel/blob/main/compel-demo-sdxl.ipynb Loads the Stable Diffusion XL base model using the diffusers library. ```python from diffusers import DiffusionPipeline import torch device='mps' pipeline = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", variant="fp16", use_safetensors=True, torch_dtype=torch.float16).to(device) ``` -------------------------------- ### SDXL Prompting with Compel Source: https://github.com/damian0815/compel/blob/main/README.md Demonstrates using Compel for SDXL models, including weighted prompts and negative prompts. Requires `torch` and `diffusers` libraries. Ensure the pipeline is moved to the appropriate device (e.g., 'cuda'). ```python from diffusers import DiffusionPipeline from compel import CompelForSDXL import torch device = 'cuda' pipeline = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", variant="fp16", use_safetensors=True, torch_dtype=torch.float16).to(device) prompt = "a cat playing with a ball++ in the forest" negative_prompt = "low quality, blurry" compel = CompelForSDXL(pipeline) conditioning = compel(prompt, negative_prompt=negative_prompt) generator = torch.Generator().manual_seed(42) image = pipeline(prompt_embeds=conditioning.embeds, pooled_prompt_embeds=conditioning.pooled_embeds, negative_prompt_embeds=conditioning.negative_embeds, negative_pooled_prompt_embeds=conditioning.negative_pooled_embeds, num_inference_steps=25, width=1024, height=1024, generator=generator).images[0] image.save('sdxl_image.jpg') ``` -------------------------------- ### Enable Clip Skip Source: https://github.com/damian0815/compel/blob/main/README.md Demonstrates how to enable the 'clip skip' feature by passing `use_penultimate_clip_layer=True` during Compel initialization. Note that this is not needed for SD2.0/SD2.1 as diffusers handles it. ```python compel = Compel(tokenizer=..., text_encoder=..., use_penultimate_clip_layer=True) ``` -------------------------------- ### Blend Prompts with Weights Source: https://github.com/damian0815/compel/blob/main/doc/syntax.md Mathematically merge multiple conditioning prompts by appending `.blend()` to a list of quoted prompts with specified weights. Weights are normalized by default. ```python ("spider man", "robot mech").blend(1, 0.8) ``` -------------------------------- ### Batched Prompting with Compel for Stable Diffusion Source: https://github.com/damian0815/compel/blob/main/README.md Illustrates how to generate multiple images with different prompts and negative prompts simultaneously using Compel for Stable Diffusion. This is useful for exploring variations or generating a set of related images. ```python from diffusers import StableDiffusionPipeline from compel import CompelForSD pipeline = StableDiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5") compel = CompelForSD(pipeline) prompt = ["a cat playing with a ball++ in the forest", "a dog wearing a hat++"] negative_prompt = ["blurry, low quality, deformed", "painting"] # or: negative_prompt = "blurry, low quality, deformed" # shared across all generated images conditioning = compel(prompt, negative_prompt=negative_prompt) # generate image images = pipeline(prompt_embeds=conditioning.embeds, negative_prompt_embeds=conditioning.negative_embeds, num_inference_steps=20).images images[0].save("image.jpg") ``` -------------------------------- ### Weighted Prompt Concatenation with .and() Source: https://github.com/damian0815/compel/blob/main/README.md Shows how to assign specific weights to different parts of a prompt when using the .and() method for concatenation. This allows for fine-grained control over the influence of each segment. ```python ("a man eating an apple", "sitting on the roof of a car", "high quality, trending on artstation, 8K UHD").and(1, 0.5, 0.5) ``` -------------------------------- ### Describe tokenization Source: https://github.com/damian0815/compel/blob/main/doc/compel.md Returns a list of tokens for the given text, marking word boundaries with . ```python def describe_tokenization(text: str) -> List[str] ``` -------------------------------- ### Flux Prompting with Compel Source: https://github.com/damian0815/compel/blob/main/README.md Shows how to use Compel with the FluxPipeline, including weighted prompts and pooled embeddings. Note that `bfloat16` can cause NaN on MPS devices, so `float32` is recommended. ```python from diffusers import FluxPipeline from compel import CompelForFlux import torch device = "mps" pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-schnell", torch_dtype=torch.float32).to(device) # bfloat16 causes NaN on MPS compel = CompelForFlux(pipe) prompt = "Astronaut---- in a jungle++++, cold color palette, muted colors, detailed, 8k" conditioning = compel(prompt) generator = torch.Generator().manual_seed(42) images = pipe(prompt_embeds=conditioning.embeds, pooled_prompt_embeds=conditioning.pooled_embeds, num_inference_steps=4, width=512, height=512, generator=generator) ``` -------------------------------- ### Describe Tokenization Source: https://github.com/damian0815/compel/blob/main/doc/compel.md Provides a list of strings showing how the input text will be tokenized. ```APIDOC ## POST /api/describe_tokenization ### Description For the given text, return a list of strings showing how it will be tokenized. Word boundaries are indicated in the output with `` strings. ### Method `describe_tokenization` ### Endpoint N/A (This is a method, not a direct API endpoint.) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body * **text** (str) - Required - The text that is to be tokenized. ### Request Example ```python compel.describe_tokenization("a cat playing in the forest") ``` ### Response #### Success Response (List[str]) * A list of strings representing the output of the tokenizer. #### Response Example ```json { "example": ["a", "cat", "playing", "in", "the", "forest"] } ``` ``` -------------------------------- ### Prompting with Negative Prompts using Compel for Stable Diffusion Source: https://github.com/damian0815/compel/blob/main/README.md Shows how to incorporate a negative prompt alongside a weighted positive prompt for Stable Diffusion using Compel. This helps refine the generated image by specifying undesirable elements. ```python from diffusers import StableDiffusionPipeline from compel import CompelForSD pipeline = StableDiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5") compel = CompelForSD(pipeline) # upweight "ball" prompt = "a cat playing with a ball++ in the forest" negative_prompt = "blurry, low quality, deformed" conditioning = compel(prompt, negative_prompt=negative_prompt) # generate image images = pipeline(prompt_embeds=conditioning.embeds, negative_prompt_embeds=conditioning.negative_embeds, num_inference_steps=20).images images[0].save("image.jpg") ``` -------------------------------- ### Generate Conditioning Tensors Source: https://github.com/damian0815/compel/blob/main/doc/README.md Create conditioning tensors from a prompt string using the Compel instance. ```python prompt = "a cat playing in the forest" conditioning = compel(prompt) # or: conditioning = compel.build_conditioning_tensor(prompt) ``` -------------------------------- ### Execute Standard Pipeline Inference Source: https://github.com/damian0815/compel/blob/main/compel-demo-flux.ipynb Run image generation using the base pipeline without additional prompt conditioning. ```python prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" generator = torch.Generator().manual_seed(42) image = pipe(prompt, num_inference_steps=4, width=512, height=512, generator=generator).images[0] image ``` -------------------------------- ### Handle Batched Inputs with Padding Source: https://github.com/damian0815/compel/blob/main/compel-demo-sdxl.ipynb Demonstrates automatic padding of prompt tensors when using batched inputs with varying lengths. ```python from compel import CompelForSDXL compel = CompelForSDXL(pipeline) prompt = "a cat playing with a ball++ in the forest" negative_prompt = "a long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long long negative prompt" # use batched input - Compel will automatically pad the shorter main prompt to the length of the longer negative prompt (or vice versa) # otherwise you'll hae to use `pad_conditioning_tensors_to_same_length` from `compel.utils` conditioning = compel([prompt, negative_prompt]) print(conditioning.embeds.shape, conditioning.pooled_embeds.shape) image = pipeline(prompt_embeds=conditioning.embeds[0:1], pooled_prompt_embeds=conditioning.pooled_embeds[0:1], negative_prompt_embeds=conditioning.embeds[1:2], negative_pooled_prompt_embeds=conditioning.pooled_embeds[1:2], num_inference_steps=24, width=768, height=768).images[0] image ``` -------------------------------- ### Use CompelForSDXL for dual-prompt conditioning Source: https://github.com/damian0815/compel/blob/main/compel-demo-sdxl.ipynb Utilize the high-level CompelForSDXL class to process main and style prompts simultaneously. This approach simplifies the pipeline integration by returning combined embeddings. ```python from compel import CompelForSDXL compel = CompelForSDXL(pipeline) main_prompt = "a cat playing with a ball++ in the forest" style_prompt = "forest ambience, high quality, detailed, intricate, artstation, 8k" conditioning = compel(main_prompt=main_prompt, style_prompt=style_prompt) image = pipeline(prompt_embeds=conditioning.embeds, pooled_prompt_embeds=conditioning.pooled_embeds, num_inference_steps=30).images[0] image ``` -------------------------------- ### Parse Prompt String with Lora Support Source: https://github.com/damian0815/compel/blob/main/README.md The `parse_prompt_string` method now returns a `Conjunction` and parses `withLora` or `useLora` syntax into `LoraWeight` instances. ```python from compel import Compel compel = Compel() prompt = "a cat withLora(my_lora, 0.8) playing with a ball" conjunction = compel.parse_prompt_string(prompt) print(conjunction) ``` -------------------------------- ### Initialize Compel Object Source: https://github.com/damian0815/compel/blob/main/doc/compel.md Initializes the Compel object with tokenizers, text encoders, and optional managers. Supports multiple tokenizers/encoders for SDXL. Configure prompt truncation, padding, downweighting, and embedding return types. ```python def __init__( tokenizer: Union[CLIPTokenizer, List[CLIPTokenizer]], text_encoder: Union[CLIPTextModel, List[CLIPTextModel]], textual_inversion_manager: Optional[ BaseTextualInversionManager] = None, dtype_for_device_getter: Callable[ [torch.device], torch.dtype] = lambda device: torch.float32, truncate_long_prompts: bool = True, padding_attention_mask_value: int = 1, downweight_mode: DownweightMode = DownweightMode.MASK, returned_embeddings_type: ReturnedEmbeddingsType = ReturnedEmbeddingsType .LAST_HIDDEN_STATES_NORMALIZED, requires_pooled: Union[bool, List[bool]] = False, device: Optional[str] = None) ``` -------------------------------- ### Compel Class Initialization Source: https://github.com/damian0815/compel/blob/main/doc/compel.md Initializes the Compel object with tokenizer and text encoder configurations. ```APIDOC ## __init__ ### Description Initialize the Compel class with necessary components for prompt conditioning. ### Parameters - **tokenizer** (Union[CLIPTokenizer, List[CLIPTokenizer]]) - Required - Tokenizer(s) from a DiffusionPipeline. - **text_encoder** (Union[CLIPTextModel, List[CLIPTextModel]]) - Required - Text encoder(s) from a DiffusionPipeline. - **textual_inversion_manager** (BaseTextualInversionManager) - Optional - Manager for multi-vector textual inversion tokens. - **truncate_long_prompts** (bool) - Optional - Whether to truncate prompts to 77 tokens. - **device** (str) - Optional - Torch device for tensor creation. ``` -------------------------------- ### Build Conditioning Tensor with Tokenization Source: https://github.com/damian0815/compel/blob/main/compel-demo.ipynb Builds a conditioning tensor from a given prompt and returns the tokenization details. This is useful for debugging and understanding how prompts are processed. ```python compel.build_conditioning_tensor("a test", return_tokenization=True) ``` -------------------------------- ### Build conditioning tensor for prompt object Source: https://github.com/damian0815/compel/blob/main/doc/compel.md Generates a conditioning tensor and options dictionary from a Blend or FlattenedPrompt object. ```python def build_conditioning_tensor_for_prompt_object( prompt: Union[Blend, FlattenedPrompt]) -> Tuple[torch.Tensor, dict] ``` -------------------------------- ### Configure DPMSolverMultistepScheduler Source: https://github.com/damian0815/compel/blob/main/compel-demo.ipynb Sets the scheduler to use the DPMSolverMultistepScheduler with the algorithm_type set to 'dpmsolver++'. This is useful for fine-tuning the diffusion process. ```python pipeline.scheduler = DPMSolverMultistepScheduler.from_config(pipeline.scheduler.config, algorithm_type="dpmsolver++" ) ``` -------------------------------- ### Handle Long Prompts with Truncation Disabled Source: https://github.com/damian0815/compel/blob/main/README.md Initialize Compel with `truncate_long_prompts=False` to chunk and pad prompts longer than the model's `max_token_length`. Ensure conditioning tensors are padded to the same length, even for empty negative prompts. ```python from compel import Compel compel = Compel(truncate_long_prompts=False) prompt = "a cat playing with a ball++ in the forest, amazing, exquisite, stunning, masterpiece, skilled, powerful, incredible, amazing, trending on gregstation, greg, greggy, greggson, greggy mcgregface, ..." # very long prompt conditioning = compel.build_conditioning_tensor(prompt) negative_prompt = "" # it's necessary to create an empty prompt - it can also be very long, if you want negative_conditioning = compel.build_conditioning_tensor(negative_prompt) [conditioning, negative_conditioning] = compel.pad_conditioning_tensors_to_same_length([conditioning, negative_conditioning]) ``` -------------------------------- ### Run Local Checkpoint Smoke Tests Source: https://github.com/damian0815/compel/blob/main/README.md Executes local smoke tests for Diffusers components using environment variable to opt-in. This is useful for validating real local SDXL .safetensors checkpoints. ```bash COMPEL_RUN_LOCAL_CHECKPOINT_TESTS=1 python -m unittest test.test_diffusers_smoke -v ``` -------------------------------- ### build_conditioning_tensor Source: https://github.com/damian0815/compel/blob/main/doc/compel.md Builds a conditioning tensor from a provided prompt string. ```APIDOC ## build_conditioning_tensor ### Description Parses the input text for Compel syntax and constructs a conditioning tensor. ### Parameters - **text** (str) - Required - The prompt string to be parsed and converted into a tensor. ### Response - **torch.Tensor** - The resulting conditioning tensor. ``` -------------------------------- ### Instantiate Compel Source: https://github.com/damian0815/compel/blob/main/doc/README.md Initialize the Compel object using the tokenizer and text encoder from a StableDiffusionPipeline. ```python from diffusers import StableDiffusionPipeline from compel import Compel pipeline = StableDiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5") compel = Compel(tokenizer=pipeline.tokenizer, text_encoder=pipeline.text_encoder) ``` -------------------------------- ### Concatenate Embeddings with .and() Source: https://github.com/damian0815/compel/blob/main/README.md Illustrates how to use the .and() method to concatenate multiple prompt segments for improved image generation quality, especially with SD2.1. Segments can be weighted individually. ```python ("a moist sloppy pindlesackboy sloppy hamblin' bogomadong", "Clem Fandango is pissed-off", "Wario's Woods in background", "making a noise like ga-woink-a").and() ``` ```python ("A dream of a distant galaxy, by Caspar David Friedrich, matte painting", "trending on artstation, HQ").and() ``` -------------------------------- ### Run Pipeline Without Compel Source: https://github.com/damian0815/compel/blob/main/compel-demo-sdxl.ipynb Executes image generation using standard pipeline parameters. ```python image = pipeline(prompt="A cat playing with a ball in the forest", negative_prompt="deformed, ugly", num_inference_steps=10, width=512, height=512).images[0] image ``` -------------------------------- ### Apply Compel Conditioning Source: https://github.com/damian0815/compel/blob/main/compel-demo-sdxl.ipynb Uses Compel to generate prompt embeddings for SDXL without specific weighting. ```python from compel import CompelForSDXL compel = CompelForSDXL(pipeline) conditioning = compel("A cat playing with a ball in the forest") negative_conditioning = compel("deformed, ugly") # you could also use batched input: # conditioning = compel(["A cat playing with a ball in the forest", "deformed, ugly"]) # and then use conditioning.embeds[0:1] for positive and conditioning.embeds[1:2] for negative image = pipeline(prompt_embeds=conditioning.embeds, pooled_prompt_embeds=conditioning.pooled_embeds, negative_prompt_embeds=negative_conditioning.embeds, negative_pooled_prompt_embeds=negative_conditioning.pooled_embeds, num_inference_steps=10, width=512, height=512).images[0] image ``` -------------------------------- ### Initialize Compel with Custom Downweighting Mode Source: https://github.com/damian0815/compel/blob/main/README.md The default downweighting algorithm applies an attention mask. To use the older behavior of literally removing tokens, initialize Compel with `downweight_mode=DownweightMode.REMOVE`. ```python from compel import Compel, DownweightMode compel = Compel(downweight_mode=DownweightMode.REMOVE) # ... rest of your code ``` -------------------------------- ### Configure Custom Embedding Types Source: https://github.com/damian0815/compel/blob/main/compel-demo-sdxl.ipynb Initializes Compel with specific text encoders and embedding types for advanced pipeline control. ```python from compel import Compel, ReturnedEmbeddingsType compel = Compel(tokenizer=[pipeline.tokenizer, pipeline.tokenizer_2] , text_encoder=[pipeline.text_encoder, pipeline.text_encoder_2], returned_embeddings_type=ReturnedEmbeddingsType.PENULTIMATE_HIDDEN_STATES_NON_NORMALIZED, requires_pooled=[False, True], device="cuda") pipeline.enable_sequential_cpu_offload() prompt = "a cat playing with a ball++ in the forest" negative_prompt = "deformed, ugly" conditioning, pooled = compel([prompt, negative_prompt]) image = pipeline(prompt_embeds=conditioning[0:1], pooled_prompt_embeds=pooled[0:1], negative_prompt_embeds=conditioning[1:2], negative_pooled_prompt_embeds=pooled[1:2], num_inference_steps=24, width=768, height=768).images[0] image ``` -------------------------------- ### Build Conditioning Tensor for Prompt Object Source: https://github.com/damian0815/compel/blob/main/doc/compel.md Builds a conditioning tensor for a given prompt object (Blend or FlattenedPrompt). ```APIDOC ## POST /api/build_conditioning_tensor_for_prompt_object ### Description Build a conditioning tensor for the given prompt object (either a Blend or a FlattenedPrompt). ### Method `build_conditioning_tensor_for_prompt_object` ### Endpoint N/A (This is a method, not a direct API endpoint.) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body * **prompt** (Union[Blend, FlattenedPrompt]) - Required - The prompt object (Blend or FlattenedPrompt) to process. ### Request Example ```python # Assuming blend_obj is a Blend object and flattened_prompt_obj is a FlattenedPrompt object compel.build_conditioning_tensor_for_prompt_object(blend_obj) compel.build_conditioning_tensor_for_prompt_object(flattened_prompt_obj) ``` ### Response #### Success Response (Tuple[torch.Tensor, dict]) - A tuple containing: - conditioning tensor (torch.Tensor) - options dict (dict) - Contains cross-attention control conditioning data if applicable. #### Response Example ```json { "example": ["conditioning tensor", {"swap": "..."}] } ``` ``` -------------------------------- ### Compel Prompt Parser Classes Source: https://github.com/damian0815/compel/blob/main/doc/prompt_parser.md Overview of the core classes used in the Compel prompt parsing library. ```APIDOC ## Conjunction Objects ### Description Storage for one or more Prompts or Blends, each of which is to be separately diffused and then the results merged by weighted sum in latent space. ```python class Conjunction() ``` ## Prompt Objects ### Description Mid-level structure for storing the tree-like result of parsing a prompt. A Prompt may not represent the whole of the singular user-defined "prompt string" (although it can) - for example, if the user specifies a Blend, the objects that are to be blended together are stored individually as Prompt objects. Nesting makes this object not suitable for directly tokenizing; instead call flatten() on the containing Conjunction to produce a FlattenedPrompt. ```python class Prompt() ``` ## FlattenedPrompt Objects ### Description A Prompt that has been passed through flatten(). Its children can be readily tokenized. ```python class FlattenedPrompt() ``` ## Fragment Objects ### Description A Fragment is a chunk of plain text and an optional weight. The text should be passed as-is to the CLIP tokenizer. ```python class Fragment(BaseFragment) ``` ## Attention Objects ### Description Nestable weight control for fragments. Each object in the children array may in turn be an Attention object; weights should be considered to accumulate as the tree is traversed to deeper levels of nesting. Do not traverse directly; instead obtain a FlattenedPrompt by calling Flatten() on a top-level Conjunction object. ```python class Attention() ``` ## CrossAttentionControlSubstitute Objects ### Description A Cross-Attention Controlled ('prompt2prompt') fragment, for use inside a Prompt, Attention, or FlattenedPrompt. Representing an "original" word sequence that supplies feature vectors for an initial diffusion operation, and an "edited" word sequence, to which the attention maps produced by the "original" word sequence are applied. Intuitively, the result should be an "edited" image that looks like the "original" image with concepts swapped. eg "a cat sitting on a car" (original) -> "a smiling dog sitting on a car" (edited): the edited image should look almost exactly the same as the original, but with a smiling dog rendered in place of the cat. The CrossAttentionControlSubstitute object representing this swap may be confined to the tokens being swapped: CrossAttentionControlSubstitute(original=[Fragment('cat')], edited=[Fragment('dog')]) or it may represent a larger portion of the token sequence: CrossAttentionControlSubstitute(original=[Fragment('a cat sitting on a car')], edited=[Fragment('a smiling dog sitting on a car')]) In either case expect it to be embedded in a Prompt or FlattenedPrompt: FlattenedPrompt([ Fragment('a'), CrossAttentionControlSubstitute(original=[Fragment('cat')], edited=[Fragment('dog')], Fragment('sitting on a car') ]) ```python class CrossAttentionControlSubstitute(CrossAttentionControlledFragment) ``` ## Blend Objects ### Description Stores a Blend of multiple Prompts. To apply, build feature vectors for each of the child Prompts and then perform a weighted blend of the feature vectors to produce a single feature vector that is effectively a lerp between the Prompts. ```python class Blend() ``` ``` -------------------------------- ### Generate Images with Compel (SplitLongTextMode.COPY_FIRST_CLS_TOKEN) Source: https://github.com/damian0815/compel/blob/main/compel-demo.ipynb Generates images using a prompt and negative prompt with Compel configured for sentence splitting and copying the first CLS token. This mode is useful for preserving the initial context of long prompts. ```python compel.compel.conditioning_provider.split_long_text_mode = SplitLongTextMode.SENTENCES | SplitLongTextMode.COPY_FIRST_CLS_TOKEN conditioning = compel(prompt, negative_prompt="ugly, deformed, misshapen, blurry") fixed_seed_generator = Generator(device="cpu").manual_seed(seed) pipeline(prompt_embeds=conditioning.embeds, negative_prompt_embeds=conditioning.negative_embeds, num_inference_steps=30, num_images_per_prompt=3, generator=fixed_seed_generator).images ``` -------------------------------- ### Textual Inversion Support with Diffusers Source: https://github.com/damian0815/compel/blob/main/README.md Integrates Diffusers textual inversions with Compel by instantiating DiffusersTextualInversionManager. This allows Compel to access and utilize textual inversions from the Diffusers library. ```python from diffusers import StableDiffusionPipeline from compel import CompelForSD, DiffusersTextualInversionManager pipeline = StableDiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5") textual_inversion_manager = DiffusersTextualInversionManager(pipeline) compel = CompelForSD(pipe=pipeline, textual_inversion_manager=textual_inversion_manager) ``` -------------------------------- ### Create Conditioning Scheduler Source: https://github.com/damian0815/compel/blob/main/doc/compel.md Generates a ConditioningScheduler object from positive and negative prompts. This scheduler is intended for managing conditioning tensors across different diffusion steps. ```python def make_conditioning_scheduler( positive_prompt: str, negative_prompt: str = '') -> ConditioningScheduler ``` -------------------------------- ### Configure IPython Autoreload Source: https://github.com/damian0815/compel/blob/main/compel-demo-flux.ipynb Enable automatic reloading of modules in an IPython environment. ```python %load_ext autoreload %autoreload 2 ``` -------------------------------- ### Generate Image with Compel and Negative Prompt Source: https://github.com/damian0815/compel/blob/main/compel-demo.ipynb Use Compel to create conditioning for a Stable Diffusion pipeline, including a negative prompt. This method is useful for generating images with specific stylistic controls. ```python compel = CompelForSD(pipeline) conditioning = compel(prompt, negative_prompt="ugly") fixed_seed_generator = Generator(device="cpu").manual_seed(seed) image = pipeline(prompt_embeds=conditioning.embeds, negative_prompt_embeds=conditioning.negative_embeds, num_inference_steps=30, generator=fixed_seed_generator).images[0] display(image) ``` -------------------------------- ### Call Compel instance Source: https://github.com/damian0815/compel/blob/main/doc/compel.md Builds conditioning tensors from a string or list of strings, padding them to match lengths if necessary. ```python @torch.no_grad() def __call__(text: Union[str, List[str]]) -> torch.FloatTensor ``` -------------------------------- ### Pad Conditioning Tensors to Same Length Source: https://github.com/damian0815/compel/blob/main/doc/compel.md Pads conditioning tensors to ensure they all have the same length, which is necessary when combining positive and negative prompts. ```APIDOC ## POST /api/pad_conditioning_tensors_to_same_length ### Description If `truncate_long_prompts` was set to False on initialization, or if your prompt includes a `.and()` operator, conditioning tensors do not have a fixed length. This function pads any of the passed-in tensors, as necessary, to ensure they all have the same length, returning the padded tensors in the same order they are passed. ### Method `pad_conditioning_tensors_to_same_length` ### Endpoint N/A (This is a method, not a direct API endpoint.) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body * **conditionings** (List[torch.Tensor]) - Required - A list of conditioning tensors to pad. ### Request Example ```python embeds = compel('("a cat playing in the forest", "an impressionist oil painting").and()') negative_embeds = compel("ugly, deformed, distorted") [embeds, negative_embeds] = compel.pad_conditioning_tensors_to_same_length([embeds, negative_embeds]) ``` ### Response #### Success Response (List[torch.Tensor]) - A list of padded conditioning tensors in the same order as the input. #### Response Example ```json { "example": ["padded tensor 1", "padded tensor 2"] } ``` ``` -------------------------------- ### PromptParser Methods Source: https://github.com/damian0815/compel/blob/main/doc/prompt_parser.md Methods available in the PromptParser class for parsing and flattening prompts. ```APIDOC ## PromptParser Objects ### Description ```python class PromptParser() ``` #### parse_conjunction ### Description Parses a prompt string into a Conjunction object. ### Method `parse_conjunction` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Arguments - `prompt` (str) - The prompt string to parse. - `verbose` (bool) - Optional flag for verbose output. Defaults to False. ### Returns - `Conjunction` - A Conjunction object representing the parsed results. ```python def parse_conjunction(prompt: str, verbose: bool = False) -> Conjunction ``` #### flatten ### Description Flattens a prompt structure into a more tokenizable format. ### Method `flatten` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Returns - `FlattenedPrompt` - A FlattenedPrompt object. ```python def flatten(prompt: Prompt) -> FlattenedPrompt ``` ``` -------------------------------- ### Define PromptParser Object Source: https://github.com/damian0815/compel/blob/main/doc/prompt_parser.md Handles the parsing of prompt strings into structured objects. ```python class PromptParser() ``` -------------------------------- ### Build conditioning tensor for conjunction Source: https://github.com/damian0815/compel/blob/main/doc/compel.md Generates a conditioning tensor and options dictionary from a Conjunction object. ```python def build_conditioning_tensor_for_conjunction( conjunction: Conjunction) -> Tuple[torch.Tensor, dict] ``` -------------------------------- ### Execute Pipeline with Compel Conditioning Source: https://github.com/damian0815/compel/blob/main/compel-demo-flux.ipynb Use Compel to process prompts for the FLUX pipeline, supporting both unweighted and weighted inputs. ```python from compel import CompelForFlux compel = CompelForFlux(pipe=pipe) conditioning = compel(prompt) generator = torch.Generator().manual_seed(42) image = pipe(prompt_embeds=conditioning.embeds, pooled_prompt_embeds=conditioning.pooled_embeds, num_inference_steps=4, width=512, height=512, generator=generator).images[0] image ``` ```python weighted_prompt = "Astronaut++++ in a jungle----, cold color palette, muted colors, detailed, 8k" weighted_conditioning = compel(weighted_prompt) generator = torch.Generator().manual_seed(42) image = pipe(prompt_embeds=weighted_conditioning.embeds, pooled_prompt_embeds=weighted_conditioning.pooled_embeds, num_inference_steps=4, width=512, height=512, generator=generator).images[0] image ``` -------------------------------- ### Apply Compel with Weighting Source: https://github.com/damian0815/compel/blob/main/compel-demo-sdxl.ipynb Uses Compel syntax to apply emphasis weights to specific prompt tokens. ```python from compel import CompelForSDXL compel = CompelForSDXL(pipeline) conditioning = compel("A cat playing with a ball++ in the forest") compel = CompelForSDXL(pipeline) negative_conditioning = compel("deformed, ugly") # you could also use batched input: # conditioning = compel(["A cat playing with a ball++ in the forest", "deformed, ugly"]) # and then use conditioning.embeds[0:1] for positive and conditioning.embeds[1:2] for negative image = pipeline(prompt_embeds=conditioning.embeds, pooled_prompt_embeds=conditioning.pooled_embeds, negative_prompt_embeds=negative_conditioning.embeds, negative_pooled_prompt_embeds=negative_conditioning.pooled_embeds, num_inference_steps=10, width=512, height=512).images[0] image ``` -------------------------------- ### Conjunction of Prompts Source: https://github.com/damian0815/compel/blob/main/doc/syntax.md Break a prompt into multiple clauses and pass them to CLIP separately by appending `.and()` to a list of quoted prompts. This can create subtle differences in the output compared to a single string prompt. ```python ("A dream of a distant galaxy, by Caspar David Friedrich, matte painting", "trending on artstation, HQ").and() ``` ```text A dream of a distant galaxy, by Caspar David Friedrich, matte painting, trending on artstation, HQ ``` -------------------------------- ### Run Stable Diffusion Pipeline Source: https://github.com/damian0815/compel/blob/main/doc/README.md Pass the generated conditioning tensors to the pipeline to generate and save images. ```python images = pipeline(prompt_embeds=conditioning, negative_prompt_embeds=negative_conditioning).images images[0].save("image.jpg") ``` -------------------------------- ### Pad conditioning tensors Source: https://github.com/damian0815/compel/blob/main/doc/compel.md Ensures all provided conditioning tensors have the same length by padding them. ```python def pad_conditioning_tensors_to_same_length( conditionings: List[torch.Tensor]) -> List[torch.Tensor] ``` ```python embeds = compel('("a cat playing in the forest", "an impressionist oil painting").and()') negative_embeds = compel("ugly, deformed, distorted") [embeds, negative_embeds] = compel.pad_conditioning_tensors_to_same_length([embeds, negative_embeds]) ``` -------------------------------- ### make_conditioning_scheduler Source: https://github.com/damian0815/compel/blob/main/doc/compel.md Creates a scheduler for conditioning tensors across diffusion steps. ```APIDOC ## make_conditioning_scheduler ### Description Returns a ConditioningScheduler object for managing conditioning tensors across diffusion steps. ### Parameters - **positive_prompt** (str) - Required - The positive prompt string. - **negative_prompt** (str) - Optional - The negative prompt string. ### Response - **ConditioningScheduler** - An object providing conditioning tensors. ``` -------------------------------- ### Parse prompt string Source: https://github.com/damian0815/compel/blob/main/doc/compel.md Converts a prompt string into a structured Conjunction object. ```python @classmethod def parse_prompt_string(cls, prompt_string: str) -> Conjunction ``` -------------------------------- ### Parse Prompt String Source: https://github.com/damian0815/compel/blob/main/doc/compel.md Parses a prompt string into a structured Conjunction object. ```APIDOC ## POST /api/parse_prompt_string ### Description Parse the given prompt string and return a structured Conjunction object that represents the prompt it contains. ### Method `parse_prompt_string` (classmethod) ### Endpoint N/A (This is a class method, not a direct API endpoint in the traditional sense. It operates on the class itself.) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body * **prompt_string** (str) - Required - The prompt string to parse. ### Request Example ```python Compel.parse_prompt_string("a photo of a cat AND a dog") ``` ### Response #### Success Response (Conjunction) * A Conjunction object representing the parsed prompt. #### Response Example ```json { "example": "Conjunction object" } ``` ``` -------------------------------- ### Generate Images with Compel (SplitLongTextMode.MERGE_CLS_TOKENS) Source: https://github.com/damian0815/compel/blob/main/compel-demo.ipynb Generates images using a prompt and negative prompt with Compel configured for sentence splitting and merging CLS tokens. This mode is suitable for handling long prompts by merging embeddings. ```python from compel import CompelForSD, SplitLongTextMode #prompt = "a cat playing with a ball in the forest" prompt = "photo, awesome, enchantress in her cluttered workshop , grey background, headshot, head++. tabletop roleplaying artwork. Dynamic, high-fantasy digital painting with dramatic lighting, painterly textures, and intricate detail. , intense action, and blends realism with impressionistic brushstrokes for an immersive composition. , concept art, clean outlines, painterly, highlights, shadows" seed = 123 compel = CompelForSD(pipeline) compel.compel.conditioning_provider.split_long_text_mode = SplitLongTextMode.SENTENCES | SplitLongTextMode.MERGE_CLS_TOKENS conditioning = compel(prompt, negative_prompt="ugly, deformed, misshapen, blurry") fixed_seed_generator = Generator(device="cpu").manual_seed(seed) pipeline(prompt_embeds=conditioning.embeds, negative_prompt_embeds=conditioning.negative_embeds, num_inference_steps=30, num_images_per_prompt=3, generator=fixed_seed_generator).images ``` -------------------------------- ### Define Prompt Object Source: https://github.com/damian0815/compel/blob/main/doc/prompt_parser.md Stores the tree-like result of parsing a prompt. Not suitable for direct tokenization; use flatten() on the containing Conjunction. ```python class Prompt() ``` -------------------------------- ### Build Conditioning Tensor with Compel Source: https://github.com/damian0815/compel/blob/main/compel-demo.ipynb Build a conditioning tensor using Compel for a given prompt. This is a foundational step for generating images with custom prompts and controlling generation parameters like the number of inference steps and the random seed. ```python prompt = "a cat playing with a ball (in the forest)-----" seed = 123 embeds = compel.build_conditioning_tensor(prompt) fixed_seed_generator = Generator(device="cpu" if compel.device.type == "mps" else compel.device).manual_seed(seed) image = pipeline(prompt_embeds=embeds, num_inference_steps=7, generator=fixed_seed_generator).images[0] display(image) ``` -------------------------------- ### Pad Conditioning Tensors Source: https://github.com/damian0815/compel/blob/main/doc/README.md Ensure conditioning and negative conditioning tensors have the same length, required when using conjunctions or disabled truncation. ```python [conditioning, negative_conditioning] = compel.pad_conditioning_tensors_to_same_length([conditioning, negative_conditioning]) ``` -------------------------------- ### Build Conditioning Tensor from Text Source: https://github.com/damian0815/compel/blob/main/doc/compel.md Parses a given text string using Compel's syntax, constructs a Conjunction object, and then generates a conditioning tensor from it. ```python def build_conditioning_tensor(text: str) -> torch.Tensor ``` -------------------------------- ### Define FlattenedPrompt Object Source: https://github.com/damian0815/compel/blob/main/doc/prompt_parser.md A Prompt that has been flattened, making its children readily tokenizable. ```python class FlattenedPrompt() ``` -------------------------------- ### Compel Call Source: https://github.com/damian0815/compel/blob/main/doc/compel.md Builds conditioning tensors from text. Handles single strings or lists of strings, padding them to the same length if necessary. ```APIDOC ## CALL __call__ ### Description Takes a string or a list of strings and builds conditioning tensors to match. If multiple strings are passed, the resulting tensors will be padded until they have the same length. ### Method `__call__` (implicitly a method of the Compel object) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body * **text** (Union[str, List[str]]) - Required - The input text or list of texts to condition. ### Request Example ```python compel("a photo of a cat") compel(["a photo of a cat", "a drawing of a dog"]) ``` ### Response #### Success Response (torch.FloatTensor) - A tensor consisting of conditioning tensors for each of the passed-in strings, concatenated along dim 0. #### Response Example ```json { "example": "torch.FloatTensor representing conditioning tensors" } ``` ``` -------------------------------- ### Define CrossAttentionControlSubstitute Object Source: https://github.com/damian0815/compel/blob/main/doc/prompt_parser.md Represents a 'prompt2prompt' fragment for swapping concepts in diffusion. Can be confined to swapped tokens or represent a larger sequence. ```python class CrossAttentionControlSubstitute(CrossAttentionControlledFragment) ``` -------------------------------- ### Generate Images with Compel (SplitLongTextMode.SENTENCES) Source: https://github.com/damian0815/compel/blob/main/compel-demo.ipynb Generates images using a prompt and negative prompt with Compel configured for sentence splitting only. This mode processes each sentence independently without specific CLS token handling. ```python compel.compel.conditioning_provider.split_long_text_mode = SplitLongTextMode.SENTENCES conditioning = compel(prompt, negative_prompt="ugly, deformed, misshapen, blurry") fixed_seed_generator = Generator(device="cpu").manual_seed(seed) pipeline(prompt_embeds=conditioning.embeds, negative_prompt_embeds=conditioning.negative_embeds, num_inference_steps=30, num_images_per_prompt=3, generator=fixed_seed_generator).images ``` -------------------------------- ### Initialize Compel for Prompt Engineering Source: https://github.com/damian0815/compel/blob/main/compel-demo.ipynb Initializes the Compel object with the pipeline's tokenizer and text encoder. Set truncate_long_prompts to False to handle longer prompts. ```python compel = Compel(tokenizer=pipeline.tokenizer, text_encoder=pipeline.text_encoder, truncate_long_prompts=False) ``` -------------------------------- ### Build Conditioning Tensor for Conjunction Source: https://github.com/damian0815/compel/blob/main/doc/compel.md Builds a conditioning tensor for a given Conjunction object. ```APIDOC ## POST /api/build_conditioning_tensor_for_conjunction ### Description Build a conditioning tensor for the given Conjunction object. ### Method `build_conditioning_tensor_for_conjunction` ### Endpoint N/A (This is a method, not a direct API endpoint.) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body * **conjunction** (Conjunction) - Required - The Conjunction object for which to build the tensor. ### Request Example ```python conjunction_obj = Compel.parse_prompt_string("a photo of a cat") compel.build_conditioning_tensor_for_conjunction(conjunction_obj) ``` ### Response #### Success Response (Tuple[torch.Tensor, dict]) - A tuple containing: - conditioning tensor (torch.Tensor) - options dict (dict) - Contains cross-attention control conditioning data if applicable. #### Response Example ```json { "example": ["conditioning tensor", {"swap": "..."}] } ``` ``` -------------------------------- ### Save Generated Images Source: https://github.com/damian0815/compel/blob/main/compel-demo.ipynb Saves the generated images to disk with a specified quality. This snippet is typically used after generating images with the pipeline. ```python for i, image in enumerate(images): display(image) image.save(f'sd1-mergecls-{i}.jpg', quality=95) ``` ```python for i, image in enumerate(images): display(image) image.save(f'sd1-copycls0-{i}.jpg', quality=95) ``` ```python for i, image in enumerate(images): display(image) image.save(f'sd1-nocopycls0-b-{i}.jpg', quality=95) ``` -------------------------------- ### Define Attention Object Source: https://github.com/damian0815/compel/blob/main/doc/prompt_parser.md Handles nestable weight control for fragments. Weights accumulate with nesting depth. Obtain a FlattenedPrompt via Flatten() on a Conjunction. ```python class Attention() ```