PartCrafter

https://github.com/wgsxm/partcrafter
Admin
PartCrafter is a structured 3D mesh generation model that uses compositional latent diffusion...
Tokens:14,263
Snippets:123
Trust Score:7.9
License:MIT
Update:5 months ago
Show doc for...
Tokens:
Raw
### Clone Repository and Setup Script

Source: https://github.com/wgsxm/partcrafter/blob/main/README.md

Clones the PartCrafter repository from GitHub and executes the setup script to install remaining dependencies. Navigate into the cloned directory before running the script.

```bash
git clone https://github.com/wgsxm/PartCrafter.git
cd PartCrafter
bash settings/setup.sh
```

--------------------------------

### Minimal Dataset Configuration Example

Source: https://github.com/wgsxm/partcrafter/blob/main/datasets/README.md

An example of a minimal, valid dataset configuration file required for training. It lists essential paths and properties for each object.

```json
[
    {
        "mesh_path": "/path/to/object.glb",
        "surface_path": "/path/to/object.npy",
        "image_path": "/path/to/object.png",
        "num_parts": 4,
        "iou_mean": 0.5,
        "iou_max": 0.9,
        "valid": true
    },
    {
        "...": "..."
    },
    ...
]
```

--------------------------------

### Training Configuration YAML

Source: https://context7.com/wgsxm/partcrafter/llms.txt

An example YAML configuration file for training PartCrafter, detailing model parameters, dataset settings, optimizer, learning rate scheduler, and training/validation hyperparameters.

```yaml
model:
  pretrained_model_name_or_path: 'pretrained_weights/TripoSG'
  vae:
    num_tokens: 1024
  transformer:
    enable_local_cross_attn: true
    global_attn_block_ids: [0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20]

dataset:
  config:
    - 'datasets/object_part_configs.json'
  training_ratio: 0.9
  min_num_parts: 1
  max_num_parts: 16
  max_iou_mean: 0.2
  max_iou_max: 0.2
  shuffle_parts: true
  object_ratio: 0.3

optimizer:
  name: "adamw"
  lr: 5e-5
  betas: [0.9, 0.999]
  weight_decay: 0.01

lr_scheduler:
  name: "constant_warmup"
  num_warmup_steps: 1000

train:
  batch_size_per_gpu: 32
  epochs: 10
  grad_checkpoint: true
  weighting_scheme: "logit_normal"
  cfg_dropout_prob: 0.1
  training_objective: "-v"
  log_freq: 1
  eval_freq: 1000
  save_freq: 2000
  ema_kwargs:
    decay: 0.9999
    use_ema_warmup: true

val:
  batch_size_per_gpu: 1
  num_inference_steps: 50
  rendering:
    radius: 4.0
    num_views: 36
    fps: 18
```

--------------------------------

### Finetune PartCrafter with more parts (16 parts)

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Finetunes PartCrafter with a configuration supporting 16 parts, starting from a previously trained baseline. It specifies the config file, enables EMA, sets gradient accumulation steps, output directory, and loads a pretrained model checkpoint.

```bash
# Finetune with more parts (16 parts)
bash scripts/train_partcrafter.sh \
  --config configs/mp16_nt512.yaml \
  --use_ema \
  --gradient_accumulation_steps 4 \
  --output_dir output_partcrafter \
  --load_pretrained_model baseline_mp8_nt512 \
  --load_pretrained_model_ckpt 10000 \
  --tag finetune_mp16_nt512
```

--------------------------------

### Train PartCrafter from TripoSG

Source: https://github.com/wgsxm/partcrafter/blob/main/README.md

This script trains PartCrafter starting from TripoSG weights. Modify the config file path and other parameters as needed. It's recommended to use 8 H20 GPUs with 96G VRAM each.

```bash
bash scripts/train_partcrafter.sh --config configs/mp8_nt512.yaml --use_ema \
  --gradient_accumulation_steps 4 \
  --output_dir output_partcrafter \
  --tag scaleup_mp8_nt512
```

--------------------------------

### Install Graphics Libraries with Conda

Source: https://github.com/wgsxm/partcrafter/blob/main/README.md

Installs required graphics libraries (libegl, libglu, pyopengl) using conda-forge. This is useful if you lack root access and are using a conda environment.

```bash
conda install -c conda-forge libegl libglu pyopengl
```

--------------------------------

### Install PyTorch with CUDA Support

Source: https://github.com/wgsxm/partcrafter/blob/main/README.md

Installs PyTorch version 2.5.1 with torchvision and torchaudio, specifically compiled for CUDA 12.4. Ensure your system has compatible NVIDIA drivers.

```bash
pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 --index-url https://download.pytorch.org/whl/cu124
```

--------------------------------

### Train PartCrafter from TripoSG base (8 GPUs)

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Initiates the training of PartCrafter from the TripoSG base model, recommended for 8 GPUs. This command uses a specified configuration file, enables EMA, sets gradient accumulation steps, and defines output directory and a tag for the experiment.

```bash
# Train from TripoSG base (8 GPUs recommended)
bash scripts/train_partcrafter.sh \
  --config configs/mp8_nt512.yaml \
  --use_ema \
  --gradient_accumulation_steps 4 \
  --output_dir output_partcrafter \
  --tag baseline_mp8_nt512
```

--------------------------------

### Finetune PartCrafter with more tokens (higher quality)

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Finetunes PartCrafter for higher quality by increasing the number of tokens, using a configuration for 16 parts and 1024 tokens. It loads a pretrained model from a previous finetuning step.

```bash
# Finetune with more tokens (higher quality)
bash scripts/train_partcrafter.sh \
  --config configs/mp16_nt1024.yaml \
  --use_ema \
  --gradient_accumulation_steps 4 \
  --output_dir output_partcrafter \
  --load_pretrained_model finetune_mp16_nt512 \
  --load_pretrained_model_ckpt 10000 \
  --tag finetune_mp16_nt1024
```

--------------------------------

### Finetune PartCrafter with More Tokens

Source: https://github.com/wgsxm/partcrafter/blob/main/README.md

This command finetunes PartCrafter with an increased number of tokens. It requires loading a previously finetuned model and checkpoint.

```bash
bash scripts/train_partcrafter.sh --config configs/mp16_nt1024.yaml --use_ema \
  --gradient_accumulation_steps 4 \
  --output_dir output_partcrafter \
  --load_pretrained_model scaleup_mp16_nt512 \
  --load_pretrained_model_ckpt 10 \
  --tag scaleup_mp16_nt1024
```

--------------------------------

### Step-by-step preprocessing: Sample surface points

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Samples surface points from a single mesh file (e.g., GLB) and saves them to a specified output location. This is the first step in preparing a custom dataset.

```bash
# 1. Sample surface points
python datasets/preprocess/mesh_to_point.py \
  --input assets/objects/scissors.glb \
  --output preprocessed_data
```

--------------------------------

### Step-by-step preprocessing: Render images

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Renders images of a single mesh from various viewpoints and saves them to the specified output directory. This is part of the dataset preparation pipeline.

```bash
# 2. Render images
python datasets/preprocess/render.py \
  --input assets/objects/scissors.glb \
  --output preprocessed_data
```

--------------------------------

### Download TripoSG Pretrained Models

Source: https://github.com/wgsxm/partcrafter/blob/main/README.md

Use this command to download the TripoSG model weights from Hugging Face. Ensure the weights are stored in the specified directory.

```bash
huggingface-cli download VAST-AI/TripoSG --local-dir pretrained_weights/TripoSG
```

--------------------------------

### Finetune PartCrafter with More Parts

Source: https://github.com/wgsxm/partcrafter/blob/main/README.md

Use this command to finetune PartCrafter when dealing with a larger number of parts. This command loads a previously trained model and checkpoint.

```bash
bash scripts/train_partcrafter.sh --config configs/mp16_nt512.yaml --use_ema \
  --gradient_accumulation_steps 4 \
  --output_dir output_partcrafter \
  --load_pretrained_model scaleup_mp8_nt512 \
  --load_pretrained_model_ckpt 10 \
  --tag scaleup_mp16_nt512
```

--------------------------------

### Download TripoSG base model

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Downloads the TripoSG base model from Hugging Face to a local directory. This is a prerequisite for training PartCrafter from a pretrained checkpoint.

```bash
# Download TripoSG base model first
huggingface-cli download VAST-AI/TripoSG --local-dir pretrained_weights/TripoSG
```

--------------------------------

### Implement OpenAI VLM Provider in Python

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Create a new Python file for the OpenAI provider, defining functions for suggesting part counts and stylizing images. This provider can be integrated into the PartCrafter system by registering its module path.

```python
# Create src/utils/providers/openai_provider.py

import os
from PIL import Image

DEFAULT_VLM_MODEL = "gpt-4-vision-preview"

def suggest_num_parts(
    image: Image.Image,
    max_num_parts: int,
    mode: str = "object",
    model_name: str = DEFAULT_VLM_MODEL,
) -> int:
    """
    Use OpenAI VLM to suggest number of parts.

    Args:
        image: PIL Image to analyze
        max_num_parts: Maximum allowed parts
        mode: "object" or "scene"
        model_name: Override default model

    Returns:
        Suggested number of parts (clamped to [1, max_num_parts])
    """
    # Implement your OpenAI API call here
    # Return an integer
    pass

def stylize_for_objaverse(
    image: Image.Image,
    model_name: str = "dall-e-3",
) -> Image.Image:
    """
    Apply style transfer using OpenAI image generation.

    Returns:
        Stylized PIL Image
    """
    # Implement your style transfer here
    pass

```

```python
# Register in src/utils/providers/__init__.py:
PROVIDERS = {
    "gemini": "src.utils.providers.gemini_provider",
    "openai": "src.utils.providers.openai_provider",  # Add this
}

```

--------------------------------

### PartCrafterPipeline API Initialization

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Initialize the PartCrafter pipeline in Python for programmatic 3D generation. Requires PyTorch, PIL, and huggingface_hub.

```python
import torch
from PIL import Image
from huggingface_hub import snapshot_download
from src.pipelines.pipeline_partcrafter import PartCrafterPipeline
from src.utils.data_utils import get_colored_mesh_composition

```

--------------------------------

### Resume PartCrafter training from checkpoint

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Resumes PartCrafter training from a specified iteration checkpoint. This command requires the configuration file, output directory, and a tag for the experiment, along with the iteration number to resume from.

```bash
# Resume training from checkpoint
bash scripts/train_partcrafter.sh \
  --config configs/mp16_nt1024.yaml \
  --use_ema \
  --output_dir output_partcrafter \
  --tag my_experiment \
  --resume_from_iter 5000
```

--------------------------------

### Generate 3D Scene from Image

Source: https://github.com/wgsxm/partcrafter/blob/main/README.md

Use this command to generate a 3D scene from a given image. The required PartCrafter-Scene model weights are downloaded automatically. The generated results are saved to a specified output directory.

```bash
python scripts/inference_partcrafter_scene.py \
  --image_path assets/images_scene/np6_0192a842-531c-419a-923e-28db4add8656_DiningRoom-31158.png \
  --num_parts 6 --tag dining_room --render
```

--------------------------------

### Sample Points from Mesh Surface

Source: https://github.com/wgsxm/partcrafter/blob/main/datasets/README.md

Use this script to sample points from the surface of a single 3D mesh. Specify the input GLB file and the output directory.

```python
python datasets/preprocess/mesh_to_point.py --input assets/objects/scissors.glb --output preprocessed_data
```

--------------------------------

### Mesh Utilities: Composition, Normalization, and Sampling

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Provides utilities for working with 3D meshes, including creating colored compositions, normalizing scale, sampling surface points, and extracting parts from scenes. Requires trimesh.

```python
import trimesh
import numpy as np
from src.utils.data_utils import (
    normalize_mesh,
    get_colored_mesh_composition,
    mesh_to_surface,
    scene_to_parts,
    move_meshes_by_center
)

# Load generated parts
meshes = [trimesh.load(f"part_{i:02d}.glb") for i in range(4)]

# Create colored composition with default colors
colored_scene = get_colored_mesh_composition(
    meshes,
    is_random=False,  # Use predefined color palette
    is_sorted=True    # Sort by volume (largest first)
)
colored_scene.export("colored_composition.glb")

# Explode parts outward for visualization
exploded = move_meshes_by_center(meshes, scale=1.5)
exploded.export("exploded_view.glb")

# Normalize mesh to unit scale
mesh = trimesh.load("input.glb")
normalized = normalize_mesh(mesh, scale=2.0)

# Sample surface points from mesh
points, normals = mesh_to_surface(
    mesh,
    num_pc=204800,  # Number of points
    clip_to_num_vertices=False
)
print(f"Sampled {len(points)} surface points")

# Extract parts from a scene
scene = trimesh.load("multi_part_scene.glb")
parts = scene_to_parts(
    scene,
    normalize=True,
    scale=2.0,
    return_type="mesh"  # or "point" for point clouds
)
print(f"Extracted {len(parts)} parts from scene")
```

--------------------------------

### Suggest Number of Parts for Object/Scene

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Programmatically suggests the number of parts for an object or objects in a scene using a VLM provider like Gemini. Requires setting the GEMINI_API_KEY environment variable.

```python
import os
from src.utils.vlm_utils import suggest_num_parts

# Set API key
os.environ["GEMINI_API_KEY"] = "your_api_key"

# Suggest parts for an object
num_parts = suggest_num_parts(
    image_path="input_object.png",
    max_num_parts=16,
    mode="object",  # "object" for single objects, "scene" for scenes
    provider="gemini",
    model_name="gemini-3-flash-preview"  # Optional, uses default if None
)
print(f"Suggested number of parts: {num_parts}")

# Suggest objects for a scene
num_objects = suggest_num_parts(
    image_path="input_scene.png",
    max_num_parts=8,
    mode="scene",
    provider="gemini"
)
print(f"Suggested number of objects: {num_objects}")
```

--------------------------------

### PartCrafter Model Generation

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Demonstrates how to download, load, and use the PartCrafter model to generate 3D parts from an input image.

```APIDOC
## PartCrafter Model Generation

### Description
This section shows how to download and load the PartCrafter model, prepare an input image, and generate 3D parts. It also covers accessing the generated meshes and exporting them.

### Method
N/A (Python script)

### Endpoint
N/A

### Parameters
N/A

### Request Example
```python
# Download and load the model
snapshot_download(repo_id="wgsxm/PartCrafter", local_dir="pretrained_weights/PartCrafter")
pipe = PartCrafterPipeline.from_pretrained("pretrained_weights/PartCrafter")
pipe = pipe.to("cuda", torch.float16)

# Load and prepare input image
image = Image.open("input.png")
num_parts = 4

# Generate 3D parts
outputs = pipe(
    image=[image] * num_parts,  # Replicate image for each part
    attention_kwargs={"num_parts": num_parts},
    num_tokens=1024,
    generator=torch.Generator(device="cuda").manual_seed(42),
    num_inference_steps=50,
    guidance_scale=7.0,
)

# Access generated meshes
meshes = outputs.meshes  # List of trimesh.Trimesh objects

# Export individual parts
for i, mesh in enumerate(meshes):
    if mesh is not None:
        mesh.export(f"part_{i:02d}.glb")

# Create colored composite mesh
composite = get_colored_mesh_composition(meshes)
composite.export("composite.glb")
```

### Response
N/A (Generates files)
```

--------------------------------

### Style Transfer for Real-World Images

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Apply style transfer to real-world photos before generation using the --style_transfer flag. Requires setting GEMINI_API_KEY.

```bash
GEMINI_API_KEY=your_api_key python scripts/inference_partcrafter.py \
  --image_path real_photo.jpg \
  --num_parts 4 \
  --style_transfer \
  --rmbg \
  --render \
  --tag styled_object
```

--------------------------------

### Combine Style Transfer and Part Suggestion

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Integrate style transfer with automatic part suggestion for a comprehensive pipeline. Specify --style_provider and --style_model for VLM customization.

```bash
GEMINI_API_KEY=your_api_key python scripts/inference_partcrafter.py \
  --image_path real_photo.jpg \
  --part_suggest \
  --style_transfer \
  --style_provider gemini \
  --style_model gemini-3.1-flash-image-preview \
  --rmbg \
  --render \
  --tag full_pipeline
```

--------------------------------

### Rendering Utilities

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Renders generated meshes with customizable camera positions and lighting.

```APIDOC
## Rendering Utilities

### Description
Render generated meshes with customizable camera positions and lighting. These utilities provide flexible options for visualizing 3D models from various viewpoints.

### Method
N/A (Python script)

### Endpoint
N/A

### Parameters
N/A

### Request Example
```python
from src.utils.render_utils import (
    render_views_around_mesh,
    render_normal_views_around_mesh,
    render_single_view,
    export_renderings,
    make_grid_for_images_or_videos
)
import trimesh
import numpy as np

# Load mesh
mesh = trimesh.load("object.glb")

# Example usage (specific functions not detailed in source text)
# render_views_around_mesh(mesh)
# render_normal_views_around_mesh(mesh)
# render_single_view(mesh, camera_position=[...])
# export_renderings(...)
# make_grid_for_images_or_videos(...)
```

### Response
N/A (Rendering functions typically return image data or save files, specific return types depend on the function called.)
```

--------------------------------

### VLM-Based Automatic Part Suggestion

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Automatically determine the number of parts using a Vision Language Model by setting the --part_suggest flag. Requires setting GEMINI_API_KEY.

```bash
GEMINI_API_KEY=your_api_key python scripts/inference_partcrafter.py \
  --image_path assets/images/np3_2f6ab901c5a84ed6bbdf85a67b22a2ee.png \
  --part_suggest \
  --tag auto_parts \
  --rmbg \
  --render
```

--------------------------------

### Rendering Utilities for Meshes

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Provides functions for rendering generated meshes from various camera angles and exporting the renderings. Requires trimesh.

```python
from src.utils.render_utils import (
    render_views_around_mesh,
    render_normal_views_around_mesh,
    render_single_view,
    export_renderings,
    make_grid_for_images_or_videos
)
import trimesh
import numpy as np

# Load mesh
mesh = trimesh.load("object.glb")

```

--------------------------------

### Generate and Export 3D Parts

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Loads a PartCrafter model, generates 3D parts from an input image, and exports them as GLB files. Requires PyTorch and trimesh.

```python
from huggingface_hub import snapshot_download
from partcrafter import PartCrafterPipeline
import torch
from PIL import Image
from partcrafter.utils.mesh_utils import get_colored_mesh_composition

# Download and load the model
snapshot_download(repo_id="wgsxm/PartCrafter", local_dir="pretrained_weights/PartCrafter")
pipe = PartCrafterPipeline.from_pretrained("pretrained_weights/PartCrafter")
pipe = pipe.to("cuda", torch.float16)

# Load and prepare input image
image = Image.open("input.png")
num_parts = 4

# Generate 3D parts
outputs = pipe(
    image=[image] * num_parts,  # Replicate image for each part
    attention_kwargs={"num_parts": num_parts},
    num_tokens=1024,
    generator=torch.Generator(device="cuda").manual_seed(42),
    num_inference_steps=50,
    guidance_scale=7.0,
)

# Access generated meshes
meshes = outputs.meshes  # List of trimesh.Trimesh objects

# Export individual parts
for i, mesh in enumerate(meshes):
    if mesh is not None:
        mesh.export(f"part_{i:02d}.glb")

# Create colored composite mesh
composite = get_colored_mesh_composition(meshes)
composite.export("composite.glb")
```

--------------------------------

### Preprocess a folder of GLB meshes

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Preprocesses a folder containing GLB mesh files for PartCrafter training. This script converts meshes into a format suitable for the dataset, outputting to a specified directory.

```bash
# Preprocess a folder of GLB meshes
python datasets/preprocess/preprocess.py \
  --input path/to/glb_files \
  --output preprocessed_data
```

--------------------------------

### Prepare Image with Background Removal

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Uses the BriaRMBG model to remove the background from an input image and save the processed image. Requires PyTorch and Pillow.

```python
import torch
import numpy as np
from PIL import Image
from huggingface_hub import snapshot_download
from src.models.briarmbg import BriaRMBG
from src.utils.image_utils import prepare_image

# Load RMBG model
snapshot_download(repo_id="briaai/RMBG-1.4", local_dir="pretrained_weights/RMBG-1.4")
rmbg_net = BriaRMBG.from_pretrained("pretrained_weights/RMBG-1.4").to("cuda")
rmbg_net.eval()

# Prepare image with background removal
processed_image = prepare_image(
    image_path="input_with_background.jpg",
    bg_color=np.array([1.0, 1.0, 1.0]),  # White background
    rmbg_net=rmbg_net,
    padding_ratio=0.1,
    device="cuda"
)

# Save processed image
processed_image.save("processed_input.png")
```

--------------------------------

### Scene Generation with Automatic Object Count

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Generate a 3D scene with automatic object count suggestion using VLM. Requires setting GEMINI_API_KEY and enabling --part_suggest and --style_transfer.

```bash
GEMINI_API_KEY=your_api_key python scripts/inference_partcrafter_scene.py \
  --image_path scene_photo.jpg \
  --part_suggest \
  --style_transfer \
  --tag auto_scene \
  --render
```

--------------------------------

### Generate 3D Part-Level Object from Image

Source: https://github.com/wgsxm/partcrafter/blob/main/README.md

Use this command to generate a 3D part-level object from a given image. The required model weights are downloaded automatically. The generated results are saved to a specified output directory.

```bash
python scripts/inference_partcrafter.py \
  --image_path assets/images/np3_2f6ab901c5a84ed6bbdf85a67b22a2ee.png \
  --num_parts 3 --tag robot --render
```

--------------------------------

### Preprocess Folder of Meshes

Source: https://github.com/wgsxm/partcrafter/blob/main/datasets/README.md

This script preprocesses an entire folder of 3D meshes, generating necessary data and a configuration file. Specify the input folder containing GLB files and the output directory.

```python
python datasets/preprocess/preprocess.py --input assets/objects --output preprocessed_data
```

--------------------------------

### Render 360-degree rotation and export as GIF

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Renders a mesh from multiple viewpoints to create a 360-degree rotation animation and exports it as a GIF. Requires a mesh object and specifies rendering parameters like number of views, radius, image size, field of view, and light intensity.

```python
rendered_images = render_views_around_mesh(
    mesh,
    num_views=36,
    radius=4.0,
    image_size=(512, 512),
    fov=40.0,
    light_intensity=5.0
)

export_renderings(rendered_images, "rotation.gif", fps=18)
```

--------------------------------

### VLM-Based Part Suggestion for 3D Object Generation

Source: https://github.com/wgsxm/partcrafter/blob/main/README.md

Instead of manually specifying the number of parts, use a VLM to automatically suggest the part count. This requires setting the GEMINI_API_KEY and uses the VLM to analyze the object.

```bash
GEMINI_API_KEY=your_key python scripts/inference_partcrafter.py \
  --image_path assets/images/np3_2f6ab901c5a84ed6bbdf85a67b22a2ee.png \
  --part_suggest --tag robot --rmbg --render
```

--------------------------------

### Style Transfer for Real-World Images

Source: https://github.com/wgsxm/partcrafter/blob/main/README.md

Apply style transfer to real-world photos to bridge the domain gap between real images and the Objaverse-style renderings PartCrafter was trained on. This converts the input photo to an Objaverse-style 3D rendering before feeding it to the pipeline.

```bash
GEMINI_API_KEY=your_key python scripts/inference_partcrafter.py \
  --image_path real_photo.jpg \
  --num_parts 4 --style_transfer --rmbg --render
```

--------------------------------

### Scene Generation with High Token Count

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Generate a 3D scene with higher quality by increasing the number of tokens using --num_tokens. The scene model defaults to 2048 tokens.

```bash
python scripts/inference_partcrafter_scene.py \
  --image_path scene_input.png \
  --num_parts 7 \
  --num_tokens 2048 \
  --num_inference_steps 50 \
  --guidance_scale 7.0 \
  --tag bedroom_scene \
  --render
```

--------------------------------

### Generate 3D Object with Custom Image and Options

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Generate a 3D object from a custom image, optionally removing the background with --rmbg. A seed can be set for reproducibility.

```bash
python scripts/inference_partcrafter.py \
  --image_path my_custom_image.png \
  --num_parts 4 \
  --tag custom_object \
  --rmbg \
  --render \
  --seed 42
```

--------------------------------

### Create image grid from multiple renders

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Creates a grid of images from a list of rendered images or video frames. Useful for visualizing multiple views or steps of a process. Requires a list of images, number of rows, and padding.

```python
grid = make_grid_for_images_or_videos(
    rendered_images[:9],  # First 9 views
    nrow=3,
    padding=0
)
grid.save("render_grid.png")
```

--------------------------------

### Advanced Object Generation Parameters

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Control generation quality and output location using advanced parameters like --num_tokens, --num_inference_steps, --guidance_scale, and --output_dir.

```bash
python scripts/inference_partcrafter.py \
  --image_path input.png \
  --num_parts 5 \
  --num_tokens 1024 \
  --num_inference_steps 50 \
  --guidance_scale 7.0 \
  --output_dir ./my_results \
  --tag experiment_01 \
  --render
```

--------------------------------

### Render Images from 3D Model

Source: https://github.com/wgsxm/partcrafter/blob/main/datasets/README.md

This script renders images from a given 3D GLB model. Provide the input mesh path and the desired output directory.

```python
python datasets/preprocess/render.py --input assets/objects/scissors.glb --output preprocessed_data
```

--------------------------------

### Dataset Configuration Format

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Defines the JSON format for configuring custom datasets for PartCrafter training. Each entry specifies paths to mesh, surface points, and images, along with part count and IoU metrics.

```json
[
    {
        "mesh_path": "/path/to/object.glb",
        "surface_path": "/path/to/object.npy",
        "image_path": "/path/to/object.png",
        "num_parts": 4,
        "iou_mean": 0.15,
        "iou_max": 0.18,
        "valid": true
    },
    {
        "mesh_path": "/path/to/another_object.glb",
        "surface_path": "/path/to/another_object.npy",
        "image_path": "/path/to/another_object.png",
        "num_parts": 6,
        "iou_mean": 0.12,
        "iou_max": 0.16,
        "valid": true
    }
]
```

--------------------------------

### Generate 3D Object with Multiple Parts

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Use the inference script for basic object generation. Specify image path, number of parts, and a tag. The --render flag generates a 360-degree animation.

```bash
python scripts/inference_partcrafter.py \
  --image_path assets/images/np3_2f6ab901c5a84ed6bbdf85a67b22a2ee.png \
  --num_parts 3 \
  --tag robot \
  --render
```

--------------------------------

### Combine VLM Part Suggestion and Style Transfer

Source: https://github.com/wgsxm/partcrafter/blob/main/README.md

Both VLM-based part suggestion and style transfer features can be combined for generating 3D objects from real-world images.

```bash
GEMINI_API_KEY=your_key python scripts/inference_partcrafter.py \
  --image_path real_photo.jpg \
  --part_suggest --style_transfer --rmbg --render
```

--------------------------------

### Apply Objaverse-Style Transfer

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Applies Objaverse-style rendering to real-world images programmatically using a VLM provider. Requires setting the GEMINI_API_KEY environment variable.

```python
import os
from src.utils.style_transfer_utils import stylize_for_objaverse

# Set API key
os.environ["GEMINI_API_KEY"] = "your_api_key"

# Apply style transfer
stylized_image = stylize_for_objaverse(
    image_path="real_world_photo.jpg",
    output_path="stylized_output.png",
    provider="gemini",
    model_name="gemini-3.1-flash-image-preview"  # Optional
)

# The stylized image is saved and returned as PIL Image
print(f"Stylized image saved to stylized_output.png")
```

--------------------------------

### Remove Background and Resize Rendered Images

Source: https://github.com/wgsxm/partcrafter/blob/main/datasets/README.md

Applies background removal to rendered images and resizes them to 90% of their original dimensions. Specify the input image path and the output directory.

```python
python datasets/preprocess/rmbg.py --input preprocessed_data/scissors/rendering.png --output preprocessed_data
```

--------------------------------

### VLM Provider and Model Override

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Customize the VLM provider and model used for part suggestion by specifying --part_provider and --part_model.

```bash
GEMINI_API_KEY=your_api_key python scripts/inference_partcrafter.py \
  --image_path input.png \
  --part_suggest \
  --part_provider gemini \
  --part_model gemini-3-flash-preview \
  --tag custom_vlm \
  --render
```

--------------------------------

### Background Removal and Image Preparation

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Utilizes the RMBG model for automatic background removal and image preparation.

```APIDOC
## Background Removal and Image Preparation

### Description
Prepare input images with automatic background removal using the RMBG model. This utility helps in isolating the subject of an image by removing its background and applying a specified background color.

### Method
N/A (Python script)

### Endpoint
N/A

### Parameters
N/A

### Request Example
```python
import torch
import numpy as np
from PIL import Image
from huggingface_hub import snapshot_download
from src.models.briarmbg import BriaRMBG
from src.utils.image_utils import prepare_image

# Load RMBG model
snapshot_download(repo_id="briaai/RMBG-1.4", local_dir="pretrained_weights/RMBG-1.4")
rmbg_net = BriaRMBG.from_pretrained("pretrained_weights/RMBG-1.4").to("cuda")
rmbg_net.eval()

# Prepare image with background removal
processed_image = prepare_image(
    image_path="input_with_background.jpg",
    bg_color=np.array([1.0, 1.0, 1.0]),  # White background
    rmbg_net=rmbg_net,
    padding_ratio=0.1,
    device="cuda"
)

# Save processed image
processed_image.save("processed_input.png")
```

### Response
N/A (Saves a processed image file)
```

--------------------------------

### Style Transfer for Real-World Images

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Convert real-world photographs to Objaverse-style 3D renderings before generation to bridge the domain gap. Supports custom style providers and models.

```APIDOC
## Style Transfer for Real-World Images

### Description
Convert real-world photographs to Objaverse-style 3D renderings before generation to bridge the domain gap.

### Method
`python scripts/inference_partcrafter.py`

### Parameters
#### Path Parameters
- **image_path** (string) - Required - Path to the input image.

#### Query Parameters
- **num_parts** (integer) - Required - The number of parts to generate.
- **style_transfer** (boolean) - Required - Flag to enable style transfer.
- **rmbg** (boolean) - Optional - Whether to remove the background from the image.
- **render** (boolean) - Optional - Whether to render the output.
- **tag** (string) - Required - A tag for the generation experiment.
- **style_provider** (string) - Optional - The style transfer provider to use (e.g., 'gemini').
- **style_model** (string) - Optional - The style transfer model to use (e.g., 'gemini-3.1-flash-image-preview').

### Environment Variables
- **GEMINI_API_KEY** (string) - Required for Gemini provider - Your Gemini API key.

### Request Example
```bash
# Apply style transfer to real-world photos
GEMINI_API_KEY=your_api_key python scripts/inference_partcrafter.py \
  --image_path real_photo.jpg \
  --num_parts 4 \
  --style_transfer \
  --rmbg \
  --render \
  --tag styled_object

# Combine style transfer with automatic part suggestion
GEMINI_API_KEY=your_api_key python scripts/inference_partcrafter.py \
  --image_path real_photo.jpg \
  --part_suggest \
  --style_transfer \
  --style_provider gemini \
  --style_model gemini-3.1-flash-image-preview \
  --rmbg \
  --render \
  --tag full_pipeline
```

### Response
#### Success Response (200)
- **Output Files** (various) - Generated 3D assets and potentially a stylized input image (`styled_input.png`).
```

--------------------------------

### Generate 3D Scene with Multiple Objects

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Use the scene inference script to generate a 3D scene with multiple objects. Specify the scene image path and the desired number of objects.

```bash
python scripts/inference_partcrafter_scene.py \
  --image_path assets/images_scene/np6_0192a842-531c-419a-923e-28db4add8656_DiningRoom-31158.png \
  --num_parts 6 \
  --tag dining_room \
  --render
```

--------------------------------

### Object Generation from Image

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Generate a structured 3D object with multiple parts from a single image using the inference script. Supports basic usage, custom images with background removal, and advanced parameters for fine-tuning generation.

```APIDOC
## Object Generation from Image

### Description
Generate a structured 3D object with multiple parts from a single image using the inference script.

### Method
`python scripts/inference_partcrafter.py`

### Parameters
#### Path Parameters
- **image_path** (string) - Required - Path to the input image.
- **output_dir** (string) - Optional - Directory to save the results.

#### Query Parameters
- **num_parts** (integer) - Required - The number of parts to generate.
- **tag** (string) - Required - A tag for the generation experiment.
- **render** (boolean) - Optional - Whether to render the output.
- **rmbg** (boolean) - Optional - Whether to remove the background from the image.
- **seed** (integer) - Optional - Random seed for reproducibility.
- **num_tokens** (integer) - Optional - Number of tokens for generation (default: 1024).
- **num_inference_steps** (integer) - Optional - Number of inference steps (default: 50).
- **guidance_scale** (float) - Optional - Guidance scale for generation (default: 7.0).

### Request Example
```bash
# Basic usage: Generate a 3D object with 3 parts
python scripts/inference_partcrafter.py \
  --image_path assets/images/np3_2f6ab901c5a84ed6bbdf85a67b22a2ee.png \
  --num_parts 3 \
  --tag robot \
  --render

# With custom image (includes background removal)
python scripts/inference_partcrafter.py \
  --image_path my_custom_image.png \
  --num_parts 4 \
  --tag custom_object \
  --rmbg \
  --render \
  --seed 42

# Advanced parameters
python scripts/inference_partcrafter.py \
  --image_path input.png \
  --num_parts 5 \
  --num_tokens 1024 \
  --num_inference_steps 50 \
  --guidance_scale 7.0 \
  --output_dir ./my_results \
  --tag experiment_01 \
  --render
```

### Response
#### Success Response (200)
- **Output Files** (various) - GLB mesh files for each part, composite colored mesh, manifest.json, rendering.gif, rendering.png.

#### Response Example
```
./results/robot/
  part_00.glb
  part_01.glb
  part_02.glb
  object.glb
  manifest.json
  rendering.gif
  rendering.png
```
```

--------------------------------

### VLM-Based Automatic Part Suggestion

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Automatically determine the optimal number of parts using a Vision Language Model (VLM) instead of manually specifying the count. Supports custom VLM providers and models.

```APIDOC
## VLM-Based Automatic Part Suggestion

### Description
Automatically determine the optimal number of parts using a Vision Language Model instead of manually specifying the count.

### Method
`python scripts/inference_partcrafter.py`

### Parameters
#### Path Parameters
- **image_path** (string) - Required - Path to the input image.

#### Query Parameters
- **part_suggest** (boolean) - Required - Flag to enable VLM-based part suggestion.
- **tag** (string) - Required - A tag for the generation experiment.
- **rmbg** (boolean) - Optional - Whether to remove the background from the image.
- **render** (boolean) - Optional - Whether to render the output.
- **part_provider** (string) - Optional - The VLM provider to use (e.g., 'gemini').
- **part_model** (string) - Optional - The VLM model to use (e.g., 'gemini-3-flash-preview').

### Environment Variables
- **GEMINI_API_KEY** (string) - Required for Gemini provider - Your Gemini API key.

### Request Example
```bash
# Use VLM to suggest number of parts automatically
GEMINI_API_KEY=your_api_key python scripts/inference_partcrafter.py \
  --image_path assets/images/np3_2f6ab901c5a84ed6bbdf85a67b22a2ee.png \
  --part_suggest \
  --tag auto_parts \
  --rmbg \
  --render

# Override the VLM provider or model
GEMINI_API_KEY=your_api_key python scripts/inference_partcrafter.py \
  --image_path input.png \
  --part_suggest \
  --part_provider gemini \
  --part_model gemini-3-flash-preview \
  --tag custom_vlm \
  --render
```

### Response
#### Success Response (200)
- **Suggestion** (string) - The VLM's suggested number of parts (e.g., "VLM suggested 3 parts").
```

--------------------------------

### Calculate IoU for Object Parts (Optional)

Source: https://github.com/wgsxm/partcrafter/blob/main/datasets/README.md

An optional script to calculate the Intersection over Union (IoU) for object parts. Requires the input GLB file and an output directory.

```python
python datasets/preprocess/calculate_iou.py --input assets/objects/scissors.glb --output preprocessed_data
```

--------------------------------

### Create and Activate Conda Environment

Source: https://github.com/wgsxm/partcrafter/blob/main/README.md

Use this command to create a new conda environment named 'partcrafter' with Python 3.11.13 and activate it. This is an optional step for managing dependencies.

```bash
conda create -n partcrafter python=3.11.13
conda activate partcrafter
```

--------------------------------

### Render single view with depth map

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Renders a single view of a mesh and returns both the image and its corresponding depth map. The depth map can be normalized. Both the image and depth map are saved as PNG files.

```python
image, depth = render_single_view(
    mesh,
    azimuth=0.0,
    elevation=0.0,
    radius=4.0,
    return_depth=True,
    normalize_depth=True
)
image.save("render.png")
depth.save("depth.png")
```

--------------------------------

### Mesh Utilities

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Provides utilities for mesh manipulation, including normalization, coloring, and composition.

```APIDOC
## Mesh Utilities

### Description
Utilities for working with generated meshes including normalization, coloring, and composition. These functions assist in processing and preparing 3D mesh data for various applications.

### Method
N/A (Python script)

### Endpoint
N/A

### Parameters
N/A

### Request Example
```python
import trimesh
import numpy as np
from src.utils.data_utils import (
    normalize_mesh,
    get_colored_mesh_composition,
    mesh_to_surface,
    scene_to_parts,
    move_meshes_by_center
)

# Load generated parts
meshes = [trimesh.load(f"part_{i:02d}.glb") for i in range(4)]

# Create colored composition with default colors
colored_scene = get_colored_mesh_composition(
    meshes,
    is_random=False,  # Use predefined color palette
    is_sorted=True    # Sort by volume (largest first)
)
colored_scene.export("colored_composition.glb")

# Explode parts outward for visualization
exploded = move_meshes_by_center(meshes, scale=1.5)
exploded.export("exploded_view.glb")

# Normalize mesh to unit scale
mesh = trimesh.load("input.glb")
normalized = normalize_mesh(mesh, scale=2.0)

# Sample surface points from mesh
points, normals = mesh_to_surface(
    mesh,
    num_pc=204800,  # Number of points
    clip_to_num_vertices=False
)
print(f"Sampled {len(points)} surface points")

# Extract parts from a scene
scene = trimesh.load("multi_part_scene.glb")
parts = scene_to_parts(
    scene,
    normalize=True,
    scale=2.0,
    return_type="mesh"  # or "point" for point clouds
)
print(f"Extracted {len(parts)} parts from scene")
```

### Response
- **colored_scene** (trimesh.Scene) - A scene object with colored meshes.
- **exploded** (trimesh.Scene) - A scene object with meshes exploded outwards.
- **normalized** (trimesh.Trimesh) - The normalized mesh.
- **points, normals** (numpy.ndarray) - Sampled surface points and their normals.
- **parts** (list) - A list of extracted mesh or point cloud parts.
```

--------------------------------

### Step-by-step preprocessing: Remove background

Source: https://context7.com/wgsxm/partcrafter/llms.txt

Removes the background from a rendered image using a background removal model. The processed image is saved to the specified output path. This is useful for isolating the object of interest.

```bash
# 3. Remove background
python datasets/preprocess/rmbg.py \
  --input preprocessed_data/scissors/rendering.png \
  --output preprocessed_data
```
Tokens:
Raw