### Train Text-Prompt Conditioned Diffusion Generator Source: https://github.com/style3d/garmagenet-impl/blob/main/README.md Trains a GarmaGenet model conditioned on text prompts using a CLIP encoder. This setup requires additional parameters for text encoding and layer configuration. ```bash python src/ldm.py --data /garmages --use_data_root \ --list --option garmagenet \ --surfvae \ --cache_dir log/garmagenet_vae_surf_256_xyz_mask_unet6_latent_1/cache/GarmageNet_xyz_mask_caption_cond/encoder_mode \ --expr GarmageNet_xyz_mask_pad_zero_caption_cond \ --train_nepoch 200000 --test_nepoch 200 --save_nepoch 10000 --batch_size 1230 --chunksize -1 \ --padding zero --bbox_scaled 1.0 --z_scaled 1.0 \ --block_dims 16 32 32 64 64 128 --latent_channels 1 --max_face 32 \ --embed_dim 768 --num_layer 12 --dropout 0.1 \ --text_encoder CLIP \ --data_fields surf_ncs surf_mask surf_bbox_wcs surf_uv_bbox_wcs caption \ --gpu 0 ``` -------------------------------- ### Prepare Point Cloud Conditioning Samples (Python) Source: https://context7.com/style3d/garmagenet-impl/llms.txt Samples point clouds from garment meshes for use as conditioning input during the training of generative models. This script helps prepare data for point cloud-conditioned generation. ```bash python data_process/prepare_pc_cond_sample.py \ --dataset_folder /path/to/garmageset/raw \ --pc_output_folder /path/to/garmageset/pc_cond_sample_uniform ``` -------------------------------- ### Initialize and Use VaeData Dataset Source: https://context7.com/style3d/garmagenet-impl/llms.txt Initializes and uses the VaeData dataset class for training VAE models. It requires input data paths, data fields, and optional arguments for validation, augmentation, and chunk size. The dataset provides access to samples, length, number of channels, and resolution. ```python from src.datasets.garmage import VaeData import argparse # Create args namespace with required parameters args = argparse.Namespace( use_data_root=True, data_fields=['surf_ncs', 'surf_mask'] ) # Initialize dataset train_dataset = VaeData( input_data='/path/to/garmages', input_list='/path/to/datalist.pkl', data_fields=['surf_ncs', 'surf_mask'], validate=False, aug=True, chunksize=512, args=args ) # Access data sample = train_dataset[0] print(f"Sample shape: {sample.shape}") # (256, 256, 4) print(f"Dataset length: {len(train_dataset)}") print(f"Num channels: {train_dataset.num_channels}") print(f"Resolution: {train_dataset.resolution}") ``` -------------------------------- ### Prepare Training Data List (Python) Source: https://context7.com/style3d/garmagenet-impl/llms.txt Creates train/validation splits from processed Garmage files. It uses a 90/10 split ratio to divide the dataset for model training and validation. ```bash python data_process/prepare_data_list.py \ --garmage_dir /path/to/garmageset/garmages \ --output_dir /path/to/garmageset/datalist \ --output_name garmageset_split_9_1 ``` -------------------------------- ### Initialize and Use GarmageNetData Dataset Source: https://context7.com/style3d/garmagenet-impl/llms.txt Initializes and uses the GarmageNetData dataset class for training the GarmageNet diffusion model. This class supports various conditioning modes and requires extensive configuration through an argparse.Namespace object. It allows initialization of encoders and access to various data components like position, latents, masks, and features. ```python from src.datasets.garmage import GarmageNetData import argparse # Create args namespace with required parameters args = argparse.Namespace( use_data_root=True, max_face=32, bbox_scaled=1.0, data_fields=['surf_ncs', 'surf_mask', 'surf_bbox_wcs', 'surf_uv_bbox_wcs', 'caption'], padding='zero', condition_type='summary', feature_kwd='0', cache_dir='/path/to/cache', surfvae='/path/to/vae.pt' ) # Initialize dataset train_dataset = GarmageNetData( input_data='/path/to/garmages', input_list='/path/to/datalist.pkl', validate=False, aug=True, args=args ) # Initialize encoder and load cached latents train_dataset.init_encoder(vae_encoder, text_encoder, z_scaled=1.0) # Access data (returns tuple) surf_pos, surf_latents, pad_mask, surf_cls, caption, pc_feature, sketch_feature = train_dataset[0] print(f"Position shape: {surf_pos.shape}") # (32, 10) print(f"Latent shape: {surf_latents.shape}") # (32, 64) print(f"Pad mask shape: {pad_mask.shape}") # (32,) print(f"Caption: {caption}") ``` -------------------------------- ### Initialize and Use PointcloudEncoder Source: https://context7.com/style3d/garmagenet-impl/llms.txt Initializes and uses the PointcloudEncoder for encoding point cloud data. The encoder takes N x 3 coordinates and outputs point cloud features. It requires specifying the encoder type and device. ```python from src.network import PointcloudEncoder import numpy as np # Initialize pointcloud encoder pc_enc = PointcloudEncoder(encoder='POINT_E', device='cuda') # Encode point cloud (N x 3 coordinates) point_cloud = np.random.randn(2048, 3).astype(np.float32) pc_features = pc_enc(point_cloud) print(f"Pointcloud embedding shape: {pc_features.shape}") # (1, 512) ``` -------------------------------- ### Initialize TextEncoder Source: https://context7.com/style3d/garmagenet-impl/llms.txt Initializes the CLIP-based text encoder used for text-conditioned garment generation. ```python from src.network import TextEncoder text_enc = TextEncoder(encoder='CLIP', device='cuda') ``` -------------------------------- ### Initialize and Run GarmageNet Transformer Source: https://context7.com/style3d/garmagenet-impl/llms.txt Initializes the main diffusion transformer model and performs a forward pass with conditioning. Handles panel positions, latents, and global CLIP embeddings for generation. ```python from src.network import GarmageNet import torch model = GarmageNet( p_dim=8, z_dim=64, embed_dim=768, condition_dim=768, num_layer=12, num_heads=12, num_cf=-1 ) model = model.cuda().eval() batch_size = 4 max_panels = 32 pos = torch.randn(batch_size, max_panels, 8).cuda() z = torch.randn(batch_size, max_panels, 64).cuda() timesteps = torch.randint(0, 1000, (batch_size,)).cuda() mask = torch.zeros(batch_size, max_panels, dtype=torch.bool).cuda() cond_global = torch.randn(batch_size, 768).cuda() with torch.no_grad(): pred_noise = model( pos=pos, z=z, timesteps=timesteps, mask=mask, cond_global=cond_global, is_train=False ) print(f"Predicted noise shape: {pred_noise.shape}") ``` -------------------------------- ### Perform Sketch-Conditioned Batch Inference Source: https://context7.com/style3d/garmagenet-impl/llms.txt Executes batch inference using cached features to generate Garmages from line-art sketch images. Requires path configurations for VAE and GarmageNet checkpoints. ```bash python src/experiments/batch_inference/batch_inference.py \ --vae /path/to/vae_checkpoint.pt \ --garmagenet /path/to/garmagenet_sketch_checkpoint.pt \ --cache log/garmagenet_vae/cache/GarmageNet_xyz_mask_sketchCond_laion2b/encoder_mode/garmagenet_validate.pkl \ --sketch_encoder LAION2B \ --output generated/sketch_cond \ --padding zero \ --block_dims 16 32 32 64 64 128 \ --img_channels 4 \ --garmage_data_fields surf_ncs surf_mask \ --latent_data_fields latent64 ``` -------------------------------- ### Train Pointcloud Conditioned Diffusion Generator Source: https://github.com/style3d/garmagenet-impl/blob/main/README.md Prepares pointcloud samples and trains a model conditioned on pointcloud features using the POINT_E encoder. It involves a two-step process of data preparation followed by training. ```bash # Prepare pointcloud sampling (surface uniform sampling). python data_process/prepare_pc_cond_sample.py \ --dataset_folder /raw \ --pc_output_folder /pc_cond_sample_uniform # Run training python src/ldm.py --data /garmages --use_data_root \ --list --option garmagenet \ --surfvae \ --cache_dir log/garmagenet_vae_surf_256_xyz_mask_unet6_latent_1/cache/GarmageNet_xyz_mask_pccond/encoder_mode \ --expr GarmageNet_xyz_mask_pad_zero_pccond \ --train_nepoch 200000 --test_nepoch 200 --save_nepoch 10000 --batch_size 1230 --chunksize -1 \ --padding zero --bbox_scaled 1.0 --z_scaled 1.0 \ --block_dims 16 32 32 64 64 128 --latent_channels 1 --max_face 32 \ --embed_dim 768 --num_layer 12 --dropout 0.1 \ --pointcloud_encoder POINT_E --pointcloud_sampled_dir /data/AIGP/GarmageSet_Opensource/pc_cond_sample_uniform \ --data_fields surf_ncs surf_mask surf_bbox_wcs surf_uv_bbox_wcs pointcloud_feature \ --gpu 0 ``` -------------------------------- ### Initialize and Use AutoencoderKLFastDecode Source: https://context7.com/style3d/garmagenet-impl/llms.txt Initializes the fast decoding-only VAE variant and reconstructs Garmage images from latent representations. Optimized for efficient reconstruction during inference. ```python from src.network import AutoencoderKLFastDecode import torch decoder = AutoencoderKLFastDecode( in_channels=4, out_channels=4, down_block_types=['DownEncoderBlock2D'] * 6, up_block_types=['UpDecoderBlock2D'] * 6, block_out_channels=[16, 32, 32, 64, 64, 128], layers_per_block=2, act_fn='silu', latent_channels=1, norm_num_groups=8, sample_size=256 ) decoder.load_state_dict(torch.load('vae_checkpoint.pt'), strict=False) decoder = decoder.cuda().eval() latent_z = torch.randn(8, 1, 8, 8).cuda() with torch.no_grad(): decoded_garmage = decoder(latent_z) print(f"Decoded shape: {decoded_garmage.shape}") ``` -------------------------------- ### Initialize and Use SketchEncoder Source: https://context7.com/style3d/garmagenet-impl/llms.txt Initializes and uses the SketchEncoder, a Vision Transformer-based encoder for sketch image conditioning. It takes the path to a sketch image and extracts spatial features. The encoder can be initialized with different pre-trained models. ```python from src.network import SketchEncoder # Initialize sketch encoder (LAION2B ViT) sketch_enc = SketchEncoder(encoder='LAION2B', device='cuda') # Extract features from sketch image sketch_features = sketch_enc('/path/to/sketch.png') print(f"Sketch spatial features: {sketch_features['spatial'].shape}") # (256, 1280) ``` -------------------------------- ### Process OBJ Files to Garmage Format (Python) Source: https://context7.com/style3d/garmagenet-impl/llms.txt Converts 3D garment OBJ files into the Garmage representation by rasterizing mesh geometry into structured 2D images. These images contain position, UV coordinates, normals, and panel masks. Supports processing all files or a specific range. ```bash python data_process/process_garmage.py \ -i /path/to/garmageset/raw \ -o /path/to/garmageset/garmages \ --nf 256 python data_process/process_garmage.py \ -i /path/to/garmageset/raw \ -o /path/to/garmageset/garmages \ -r "0,100" ``` -------------------------------- ### Train Sketch Conditioned Diffusion Generator Source: https://github.com/style3d/garmagenet-impl/blob/main/README.md Trains a model conditioned on line-art sketches. Requires pre-processing sketch features using ViT and configuring the training script with the LAION2B encoder. ```bash # Prepare sketch feature. python data_process/prepare_sketch_feature_vit.py \ --root_dir /images \ --output_dir /feature_laion2b # Run training python src/ldm.py \ --data /garmages --use_data_root \ --list --option garmagenet \ --surfvae \ --cache_dir log/garmagenet_vae_surf_256_xyz_mask_unet6_latent_1/cache/GarmageNet_xyz_mask_sketchCond_laion2b/encoder_mode \ --expr GarmageNet_xyz_mask_pad_zero_sketchCond_laion2b \ --train_nepoch 200000 --test_nepoch 200 --save_nepoch 10000 --batch_size 1230 --chunksize -1 \ --padding zero --bbox_scaled 1.0 --z_scaled 1.0 \ --block_dims 16 32 32 64 64 128 --latent_channels 1 --max_face 32 \ --sketch_encoder LAION2B --sketch_feature_dir /feature_laion2b \ --condition_type spatial --feature_kwd 0 \ --data_fields surf_ncs surf_mask surf_bbox_wcs surf_uv_bbox_wcs sketch_feature \ --gpu 0 ``` -------------------------------- ### Perform Direct Sketch Inference from Images Source: https://context7.com/style3d/garmagenet-impl/llms.txt Generates Garmages directly from a directory of sketch images without relying on cached features. Configures the task as image-based and specifies the inference device. ```bash python src/experiments/batch_inference/batch_inference.py \ --task image \ --vae /path/to/vae_checkpoint.pt \ --garmagenet /path/to/garmagenet_sketch_checkpoint.pt \ --inference_data /path/to/sketch_images/ \ --sketch_encoder LAION2B \ --condition_type spatial \ --output generated/sketch_from_images \ --padding zero \ --block_dims 16 32 32 64 64 128 \ --img_channels 4 \ --garmage_data_fields surf_ncs surf_mask \ --latent_data_fields latent64 \ --device cuda ``` -------------------------------- ### Train Unconditional Diffusion Generator Source: https://github.com/style3d/garmagenet-impl/blob/main/README.md Executes the training process for an unconditional GarmaGenet diffusion model. It requires a dataset path, VAE checkpoint, and specific training hyperparameters. ```bash python src/ldm.py \ --data /garmages --use_data_root \ --list --option garmagenet --lr 5e-4 \ --surfvae \ --cache_dir log/garmagenet_vae_surf_256_xyz_mask_unet6_latent_1/cache/GarmageNet_xyz_mask_uncond/encoder_mode \ --expr GarmageNet_xyz_mask_pad_zero_uncond \ --train_nepoch 200000 --test_nepoch 200 --save_nepoch 10000 --batch_size 1230 --chunksize -1 \ --padding zero --bbox_scaled 1.0 --z_scaled 1.0 \ --block_dims 16 32 32 64 64 128 --latent_channels 1 --max_face 32 \ --embed_dim 768 \ --data_fields surf_ncs surf_mask surf_bbox_wcs surf_uv_bbox_wcs \ --gpu 0 ``` -------------------------------- ### Sketch-Conditioned Generation Source: https://context7.com/style3d/garmagenet-impl/llms.txt Generates Garmages from line-art sketch images, with a preference for front-view sketches. This script utilizes pre-trained VAE and Garmagenet models. ```APIDOC ## Sketch-Conditioned Generation ### Description Generate Garmages from line-art sketch images (front-view preferred). ### Method `python` ### Endpoint `src/experiments/batch_inference/batch_inference.py` ### Parameters #### Command Line Arguments - `--vae` (path) - Required - Path to the VAE checkpoint. - `--garmagenet` (path) - Required - Path to the Garmagenet sketch checkpoint. - `--cache` (path) - Required - Path to the cached features. - `--sketch_encoder` (string) - Required - Type of sketch encoder (e.g., `LAION2B`). - `--output` (path) - Required - Directory to save generated images. - `--padding` (string) - Optional - Padding mode (e.g., `zero`). - `--block_dims` (list of ints) - Optional - Dimensions for model blocks. - `--img_channels` (int) - Optional - Number of image channels. - `--garmage_data_fields` (list of strings) - Optional - Fields for Garmage data. - `--latent_data_fields` (list of strings) - Optional - Fields for latent data. ### Request Example ```bash python src/experiments/batch_inference/batch_inference.py \ --vae /path/to/vae_checkpoint.pt \ --garmagenet /path/to/garmagenet_sketch_checkpoint.pt \ --cache log/garmagenet_vae/cache/GarmageNet_xyz_mask_sketchCond_laion2b/encoder_mode/garmagenet_validate.pkl \ --sketch_encoder LAION2B \ --output generated/sketch_cond \ --padding zero \ --block_dims 16 32 32 64 64 128 \ --img_channels 4 \ --garmage_data_fields surf_ncs surf_mask \ --latent_data_fields latent64 ``` ### Response N/A (This is a script execution, not an API endpoint with a direct response body.) #### Success Response (Exit Code 0) - Indicates successful execution of the script. #### Response Example Output files will be saved in the specified `--output` directory. ``` -------------------------------- ### Initialize and Use AutoencoderKLFastEncode Source: https://context7.com/style3d/garmagenet-impl/llms.txt Initializes the fast encoding-only VAE variant and encodes Garmage images into latent representations. This component is optimized for efficient latent extraction during training. ```python from src.network import AutoencoderKLFastEncode import torch encoder = AutoencoderKLFastEncode( in_channels=4, out_channels=4, down_block_types=['DownEncoderBlock2D'] * 6, up_block_types=['UpDecoderBlock2D'] * 6, block_out_channels=[16, 32, 32, 64, 64, 128], layers_per_block=2, act_fn='silu', latent_channels=1, norm_num_groups=8, sample_size=256 ) encoder.load_state_dict(torch.load('vae_checkpoint.pt'), strict=False) encoder = encoder.cuda().eval() garmage_input = torch.randn(8, 4, 256, 256).cuda() with torch.no_grad(): latent_z = encoder(garmage_input) print(f"Latent shape: {latent_z.shape}") ``` -------------------------------- ### Prepare Sketch Feature Extraction (Python) Source: https://context7.com/style3d/garmagenet-impl/llms.txt Extracts visual features from sketch images using a Vision Transformer (ViT) encoder. These features are used for sketch-conditioned garment generation. ```bash python data_process/prepare_sketch_feature_vit.py \ --root_dir /path/to/garmageset/images \ --output_dir /path/to/garmageset/feature_laion2b ``` -------------------------------- ### Train Unconditional GarmageNet Diffusion Model (Python) Source: https://context7.com/style3d/garmagenet-impl/llms.txt Trains the latent diffusion transformer (GarmageNet) for unconditional garment generation. This model generates garment panels without any specific conditioning input, using a VAE-encoded latent space. ```bash python src/ldm.py \ --data /path/to/garmageset/garmages \ --use_data_root \ --list /path/to/datalist.pkl \ --option garmagenet \ --lr 5e-4 \ --surfvae /path/to/vae_checkpoint.pt \ --cache_dir log/garmagenet_vae/cache/GarmageNet_xyz_mask_uncond/encoder_mode \ --expr GarmageNet_xyz_mask_pad_zero_uncond \ --train_nepoch 200000 \ --test_nepoch 200 \ --save_nepoch 10000 \ --batch_size 1230 \ --chunksize -1 \ --padding zero \ --bbox_scaled 1.0 \ --z_scaled 1.0 \ --block_dims 16 32 32 64 64 128 \ --latent_channels 1 \ --max_face 32 \ --embed_dim 768 \ --data_fields surf_ncs surf_mask surf_bbox_wcs surf_uv_bbox_wcs \ --gpu 0 ``` -------------------------------- ### Encode Text Prompts with Text Encoder Source: https://context7.com/style3d/garmagenet-impl/llms.txt Encodes a list of text prompts into embeddings using a text encoder. This is useful for conditioning generative models with textual descriptions. The output shape indicates the number of prompts and the embedding dimension. ```python prompts = ["a long red dress with sleeves", "casual blue t-shirt"] text_embeddings = text_enc(prompts) print(f"Text embedding shape: {text_embeddings.shape}") # (2, 768) ``` -------------------------------- ### Train Geometry Encoding VAE (Python) Source: https://context7.com/style3d/garmagenet-impl/llms.txt Trains a Variational Autoencoder (VAE) to encode Garmage images (256x256 resolution) into a compact latent representation. Each garment is represented as a tensor (N, 256, 256, C). Supports standard training and finetuning from existing checkpoints. ```bash # Train VAE with standard configuration python src/vae.py \ --data /path/to/garmageset/garmages \ --use_data_root \ --list /path/to/datalist.pkl \ --expr garmagenet_vae_surf_256_xyz_mask_unet6_latent_1 \ --batch_size 64 \ --block_dims 16 32 32 64 64 128 \ --latent_channels 1 \ --test_nepoch 10 \ --save_nepoch 50 \ --train_nepoch 2000 \ --data_fields surf_ncs surf_mask \ --chunksize 512 # Finetune from existing checkpoint python src/vae.py \ --data /path/to/garmageset/garmages \ --use_data_root \ --list /path/to/datalist.pkl \ --expr garmagenet_vae_finetuned \ --finetune \ --weight /path/to/vae_checkpoint.pt \ --batch_size 64 \ --block_dims 16 32 32 64 64 128 \ --latent_channels 1 \ --train_nepoch 500 \ --data_fields surf_ncs surf_mask ``` -------------------------------- ### Direct Sketch Inference from Images Source: https://context7.com/style3d/garmagenet-impl/llms.txt Generates Garmages directly from a folder of sketch images without using cached features. This method is useful for direct inference when pre-computed features are not available. ```APIDOC ## Direct Sketch Inference from Images ### Description Generate Garmages directly from a folder of sketch images without using cached features. ### Method `python` ### Endpoint `src/experiments/batch_inference/batch_inference.py` ### Parameters #### Command Line Arguments - `--task` (string) - Required - Set to `image` for direct image inference. - `--vae` (path) - Required - Path to the VAE checkpoint. - `--garmagenet` (path) - Required - Path to the Garmagenet sketch checkpoint. - `--inference_data` (path) - Required - Path to the folder containing sketch images. - `--sketch_encoder` (string) - Required - Type of sketch encoder (e.g., `LAION2B`). - `--condition_type` (string) - Required - Type of conditioning (e.g., `spatial`). - `--output` (path) - Required - Directory to save generated images. - `--padding` (string) - Optional - Padding mode (e.g., `zero`). - `--block_dims` (list of ints) - Optional - Dimensions for model blocks. - `--img_channels` (int) - Optional - Number of image channels. - `--garmage_data_fields` (list of strings) - Optional - Fields for Garmage data. - `--latent_data_fields` (list of strings) - Optional - Fields for latent data. - `--device` (string) - Optional - Device to run inference on (e.g., `cuda`). ### Request Example ```bash python src/experiments/batch_inference/batch_inference.py \ --task image \ --vae /path/to/vae_checkpoint.pt \ --garmagenet /path/to/garmagenet_sketch_checkpoint.pt \ --inference_data /path/to/sketch_images/ \ --sketch_encoder LAION2B \ --condition_type spatial \ --output generated/sketch_from_images \ --padding zero \ --block_dims 16 32 32 64 64 128 \ --img_channels 4 \ --garmage_data_fields surf_ncs surf_mask \ --latent_data_fields latent64 \ --device cuda ``` ### Response N/A (This is a script execution, not an API endpoint with a direct response body.) #### Success Response (Exit Code 0) - Indicates successful execution of the script. #### Response Example Output files will be saved in the specified `--output` directory. ``` -------------------------------- ### TextEncoder Source: https://context7.com/style3d/garmagenet-impl/llms.txt CLIP-based text encoder for text-conditioned generation. ```APIDOC ## TextEncoder ### Description CLIP-based text encoder for text-conditioned generation. ### Method `python` ### Endpoint `src/network.py` (Class Definition) ### Parameters #### Initialization Arguments - `encoder` (string) - Required - Type of encoder to use (e.g., `CLIP`). - `device` (string) - Required - Device to run the encoder on (e.g., `cuda`). ### Request Example (Initialization) ```python from src.network import TextEncoder # Initialize CLIP text encoder text_enc = TextEncoder(encoder='CLIP', device='cuda') ``` ### Response N/A (This is a class initialization, not an endpoint with a direct response.) #### Success Response - An initialized `TextEncoder` object ready for use. ``` -------------------------------- ### GarmageNet Transformer Source: https://context7.com/style3d/garmagenet-impl/llms.txt The main diffusion transformer model for generating garment latents with optional conditioning. ```APIDOC ## GarmageNet Transformer ### Description The main diffusion transformer model for generating garment latents with optional conditioning. ### Method `python` ### Endpoint `src/network.py` (Class Definition) ### Parameters #### Initialization Arguments - `p_dim` (int) - Required - Dimension of panel properties (e.g., 3D bbox + 2D scale). - `z_dim` (int) - Required - Dimension of the latent space for each panel. - `embed_dim` (int) - Required - Transformer embedding dimension. - `condition_dim` (int) - Required - Dimension of the conditioning embedding (e.g., CLIP text embedding). - `num_layer` (int) - Required - Number of transformer layers. - `num_heads` (int) - Required - Number of attention heads. - `num_cf` (int) - Optional - Number of classes for class conditioning (-1 for no class conditioning). ### Request Example (Forward Pass) ```python from src.network import GarmageNet import torch # Initialize GarmageNet model = GarmageNet( p_dim=8, # 3D bbox (6) + 2D scale (2) z_dim=64, # Latent dimension (8*8*1 = 64) embed_dim=768, # Transformer embedding dimension condition_dim=768, # Conditioning embedding dimension (CLIP) num_layer=12, # Number of transformer layers num_heads=12, # Number of attention heads num_cf=-1 # Number of classes (-1 for no class conditioning) ) model = model.cuda().eval() # Forward pass with conditioning batch_size = 4 max_panels = 32 pos = torch.randn(batch_size, max_panels, 8).cuda() # Panel positions/bboxes z = torch.randn(batch_size, max_panels, 64).cuda() # Panel latents timesteps = torch.randint(0, 1000, (batch_size,)).cuda() # Diffusion timesteps mask = torch.zeros(batch_size, max_panels, dtype=torch.bool).cuda() # Padding mask cond_global = torch.randn(batch_size, 768).cuda() # CLIP text embedding with torch.no_grad(): pred_noise = model( pos=pos, z=z, timesteps=timesteps, mask=mask, cond_global=cond_global, is_train=False ) print(f"Predicted noise shape: {pred_noise.shape}") # (4, 32, 72) = latent + position ``` ### Response #### Success Response (Output of `model(...)`) - `pred_noise` (torch.Tensor) - The predicted noise, which is part of the diffusion process. Shape: `(batch_size, max_panels, z_dim + p_dim)`. ``` -------------------------------- ### AutoencoderKLFastDecode Source: https://context7.com/style3d/garmagenet-impl/llms.txt Provides a fast decoding-only VAE variant for efficient latent reconstruction during inference. ```APIDOC ## AutoencoderKLFastDecode ### Description Fast decoding-only VAE variant for efficient latent reconstruction during inference. ### Method `python` ### Endpoint `src/network.py` (Class Definition) ### Parameters #### Initialization Arguments - `in_channels` (int) - Required - Number of input channels (e.g., 4 for xyz + mask). - `out_channels` (int) - Required - Number of output channels. - `down_block_types` (list of strings) - Required - Types of down-sampling blocks. - `up_block_types` (list of strings) - Required - Types of up-sampling blocks. - `block_out_channels` (list of ints) - Required - Output channels for each block. - `layers_per_block` (int) - Required - Number of layers per block. - `act_fn` (string) - Required - Activation function (e.g., `silu`). - `latent_channels` (int) - Required - Number of latent channels. - `norm_num_groups` (int) - Required - Number of normalization groups. - `sample_size` (int) - Required - The spatial size of the output samples. ### Request Example (Initialization and Decoding) ```python from src.network import AutoencoderKLFastDecode import torch # Initialize decoder decoder = AutoencoderKLFastDecode( in_channels=4, out_channels=4, down_block_types=['DownEncoderBlock2D'] * 6, up_block_types=['UpDecoderBlock2D'] * 6, block_out_channels=[16, 32, 32, 64, 64, 128], layers_per_block=2, act_fn='silu', latent_channels=1, norm_num_groups=8, sample_size=256 ) # Load pretrained weights decoder.load_state_dict(torch.load('vae_checkpoint.pt'), strict=False) decoder = decoder.cuda().eval() # Decode latent to Garmage images latent_z = torch.randn(8, 1, 8, 8).cuda() # (batch, latent_channels, H/32, W/32) with torch.no_grad(): decoded_garmage = decoder(latent_z) print(f"Decoded shape: {decoded_garmage.shape}") # (8, 4, 256, 256) ``` ### Response #### Success Response (Output of `decoder(latent_z)`) - `decoded_garmage` (torch.Tensor) - The reconstructed Garmage images from the latent representation. Shape: `(batch_size, out_channels, sample_size, sample_size)`. ``` -------------------------------- ### AutoencoderKLFastEncode Source: https://context7.com/style3d/garmagenet-impl/llms.txt Provides a fast encoding-only VAE variant for efficient latent extraction during training. ```APIDOC ## AutoencoderKLFastEncode ### Description Fast encoding-only VAE variant for efficient latent extraction during training. ### Method `python` ### Endpoint `src/network.py` (Class Definition) ### Parameters #### Initialization Arguments - `in_channels` (int) - Required - Number of input channels (e.g., 4 for xyz + mask). - `out_channels` (int) - Required - Number of output channels. - `down_block_types` (list of strings) - Required - Types of down-sampling blocks. - `up_block_types` (list of strings) - Required - Types of up-sampling blocks. - `block_out_channels` (list of ints) - Required - Output channels for each block. - `layers_per_block` (int) - Required - Number of layers per block. - `act_fn` (string) - Required - Activation function (e.g., `silu`). - `latent_channels` (int) - Required - Number of latent channels. - `norm_num_groups` (int) - Required - Number of normalization groups. - `sample_size` (int) - Required - The spatial size of the input samples. ### Request Example (Initialization and Encoding) ```python from src.network import AutoencoderKLFastEncode import torch # Initialize encoder encoder = AutoencoderKLFastEncode( in_channels=4, # xyz + mask out_channels=4, down_block_types=['DownEncoderBlock2D'] * 6, up_block_types=['UpDecoderBlock2D'] * 6, block_out_channels=[16, 32, 32, 64, 64, 128], layers_per_block=2, act_fn='silu', latent_channels=1, norm_num_groups=8, sample_size=256 ) # Load pretrained weights encoder.load_state_dict(torch.load('vae_checkpoint.pt'), strict=False) encoder = encoder.cuda().eval() # Encode Garmage images to latent garmage_input = torch.randn(8, 4, 256, 256).cuda() # (batch, channels, H, W) with torch.no_grad(): latent_z = encoder(garmage_input) # Returns sampled latent print(f"Latent shape: {latent_z.shape}") # (8, 1, 8, 8) ``` ### Response #### Success Response (Output of `encoder(garmage_input)`) - `latent_z` (torch.Tensor) - The encoded latent representation of the input Garmage images. Shape: `(batch_size, latent_channels, height/32, width/32)`. ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.