### Install ComfyUI-Sharp

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/00-index.md

Clone the repository, navigate to the directory, and install dependencies using pip. Then run the install script.

```bash
cd ComfyUI/custom_nodes
git clone https://github.com/PozzettiAndrea/ComfyUI-Sharp.git
cd ComfyUI-Sharp
pip install -r requirements.txt
python install.py
```

--------------------------------

### Manual Installation of ComfyUI-Sharp

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/README.md

Use this method for the most reliable installation. It involves cloning the repository, navigating to the directory, and installing requirements and the Python script.

```bash
cd ComfyUI/custom_nodes
git clone https://github.com/PozzettiAndrea/ComfyUI-Sharp.git
cd ComfyUI-Sharp
pip install -r requirements.txt --upgrade
python install.py
```

--------------------------------

### ComfyUI Workflow Usage Example

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/04-load-image-exif.md

Demonstrates how to use the LoadImageWithExif node in a ComfyUI workflow, passing its output to a SharpPredict node.

```python
# In ComfyUI workflow
# Assuming LoadImageWithExif and SharpPredict are available nodes

# Instantiate the node (in a real workflow, this is handled by ComfyUI)
# load_node = LoadImageWithExif()

# Execute the node with a specific image and default focal length
# image, mask, focal_mm = load_node.execute(
#     image="photo.jpg",
#     default_focal_mm=35.0
# )
# focal_mm extracted from EXIF if available, else 35.0

# Pass the output to another node like SharpPredict
# predict_node = SharpPredict()
# model_config = "path/to/model/config"
# ply_path = predict_node.execute(
#     model=model_config,
#     image=image,
#     focal_length_mm=focal_mm  # From EXIF or default
# )

# Note: The above is illustrative Python code representing node execution. 
# In ComfyUI, nodes are connected visually, and execution is managed by the framework.
# The actual Python code would look more like:

# Example of how ComfyUI might internally represent and execute nodes:
class MockLoadImageWithExif:
    def execute(self, image: str, default_focal_mm: float = 30.0):
        print(f"Mock executing LoadImageWithExif for {image}")
        # Simulate output
        mock_image_tensor = torch.rand(1, 100, 100, 3)
        mock_mask_tensor = torch.zeros(1, 100, 100)
        mock_focal_mm = 50.0 # Assume EXIF extraction yields 50mm
        return mock_image_tensor, mock_mask_tensor, mock_focal_mm

class MockSharpPredict:
    def execute(self, model: str, image: torch.Tensor, focal_length_mm: float):
        print(f"Mock executing SharpPredict with focal length: {focal_length_mm}")
        # Simulate output
        return "output.ply"

# --- Workflow Simulation ---
# load_node_instance = MockLoadImageWithExif()
# predict_node_instance = MockSharpPredict()
# model_config_path = "dummy_model_path"

# image_output, mask_output, focal_mm_output = load_node_instance.execute(
#     image="path/to/your/photo.jpg",
#     default_focal_mm=35.0
# )

# ply_result = predict_node_instance.execute(
#     model=model_config_path,
#     image=image_output,
#     focal_length_mm=focal_mm_output
# )

# print(f"Generated PLY path: {ply_result}")

```

--------------------------------

### ComfyUI-Sharp Panorama Workflow Example

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/08-project-depth-panorama.md

Demonstrates a typical workflow for creating a panoramic depth map using comfyui-sharp, including loading images, sampling, depth prediction, and projection.

```python
# Panorama workflow
load_node = LoadImageWithExif()
panorama, _, _ = load_node.execute("equirect.jpg", 30.0)

sample_node = SamplePanorama()
images, extrinsics, intrinsics, n_h, n_v = sample_node.execute(
    panorama=panorama,
    fov_degrees=65.0,
    overlap_percent=10.0
)

# Extract depth instead of Gaussians
depth_node = SharpPredictDepth()
depth_maps, ext, intr, align_maps = depth_node.execute(
    model=model_config,
    image=images,
    extrinsics=extrinsics,
    intrinsics=intrinsics
)

# Project to panorama and check consistency
project_node = ProjectDepthToPanorama()
pano_depth, debug, heatmap = project_node.execute(
    depth_maps=depth_maps,
    extrinsics=extrinsics,
    intrinsics=intrinsics,
    panorama_width=2048,
    blend_mode="gaussian",
    show_borders=True,
    border_color="red",
    disagreement_scale=0.2
)

# pano_depth: reconstructed panoramic depth
# debug: with sample region borders for visualization
# heatmap: color disagreement map (red=bad alignment, blue=good)
# If heatmap mostly blue: safe to merge Gaussians
# If heatmap mostly red: alignment problem, consider AlignDepthMaps
```

--------------------------------

### ComfyUI Workflow Usage Example

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/02-load-sharp-model.md

Demonstrates how to use the LoadSharpModel node within a ComfyUI workflow to obtain model configuration and pass it to a prediction node.

```python
# In ComfyUI workflow
config_node = LoadSharpModel(precision="auto")
model_config = config_node.execute()  # Returns io.NodeOutput

# Pass to SharpPredict or SharpPredictDepth
predict_node = SharpPredict()
ply_path = predict_node.execute(
    model=model_config[0],  # Extract dict from NodeOutput
    image=image_tensor,
    focal_length_mm=35.0
)
```

--------------------------------

### Python Usage Example: Depth Map Alignment Workflow

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/09-align-depth-maps.md

Demonstrates a typical workflow for aligning depth maps using SharpPredictDepth, AlignDepthMaps, and ProjectDepthToPanorama nodes.

```python
# After SharpPredictDepth
depth_node = SharpPredictDepth()
depth_maps, ext, intr, _ = depth_node.execute(
    model=model_config,
    image=panorama_samples,
    extrinsics=extrinsics_from_panorama,
    intrinsics=intrinsics_from_panorama
)

# Align depth maps before projection
align_node = AlignDepthMaps()
aligned_depths, ext, intr, info = align_node.execute(
    depth_maps=depth_maps,
    extrinsics=extrinsics_from_panorama,
    intrinsics=intrinsics_from_panorama,
    method="global_optimization",
    reference_view=0
)

# Project aligned depth to panorama
project_node = ProjectDepthToPanorama()
pano_depth, debug, heatmap = project_node.execute(
    depth_maps=aligned_depths,
    extrinsics=extrinsics_from_panorama,
    intrinsics=intrinsics_from_panorama
)

# Heatmap should be mostly blue (good agreement) after alignment
```

--------------------------------

### Example Calculation for Panorama Sampling

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/05-sample-panorama.md

Illustrates the calculation of step degrees, number of horizontal and vertical samples, and total samples for a given FOV and overlap.

```text
step_degrees = 65 * (1 - 0.1) = 58.5°
num_horizontal = ceil(360 / 58.5) = 7
num_vertical = ceil(150 / 58.5) = 3 (or ceil(180 / 58.5) = 4 if not skip_poles)
total = 7 × 3 = 21 samples
```

--------------------------------

### Python Example: Merging Gaussian PLY Files

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/07-merge-gaussians.md

Demonstrates how to use the SharpPredict and MergeGaussians nodes in Python. First, SharpPredict processes a panorama to generate individual PLY files in a specified folder. Then, MergeGaussians takes this folder path and merges all PLY files into a single output PLY file, with options to filter by maximum depth and minimum opacity.

```python
# Process panorama with SharpPredict
predict_node = SharpPredict()
ply_folder = predict_node.execute(
    model=model_config,
    image=panorama_samples,  # [N, H, W, 3]
    extrinsics=extrinsics,   # [N, 4, 4]
    intrinsics=intrinsics
)
# Returns folder path like "output/sharp_1717680000123/"
# Contains: 001.ply, 002.ply, 003.ply, ...

# Merge all samples into one scene
merge_node = MergeGaussians()
merged_ply = merge_node.execute(
    ply_folder=ply_folder,
    output_prefix="panorama_merged",
    max_depth=50.0,      # Remove anything >50 meters away
    min_opacity=0.1      # Remove mostly transparent Gaussians
)
# Output: "output/panorama_merged_1717680001456.ply"

# Result is a unified 3D Gaussian scene ready for viewing in Gaussian viewers
# (e.g., web viewer, Nerfstudio, custom C++ renderer)
```

--------------------------------

### ComfyUI-Sharp Panorama Sampling and SHARP Prediction Workflow

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/05-sample-panorama.md

Demonstrates loading a panorama, sampling perspective views, and processing them with the SHARP model using ComfyUI-Sharp nodes.

```python
# Load 360 panorama
load_node = LoadImageWithExif()
panorama, _, _ = load_node.execute("equirect_360.jpg", 30.0)

# Sample perspective views
sample_node = SamplePanorama()
images, extrinsics, intrinsics, n_h, n_v = sample_node.execute(
    panorama=panorama,
    fov_degrees=65.0,
    overlap_percent=10.0,
    output_size=1536,
    skip_poles=True
)
# images: [21, 1536, 1536, 3]
# extrinsics: [21, 4, 4]
# intrinsics: [4, 4]

# Process all samples with SHARP
predict_node = SharpPredict()
ply_folder = predict_node.execute(
    model=model_config,
    image=images,  # Batch of 21 images
    extrinsics=extrinsics,
    intrinsics=intrinsics
)
# Outputs folder with 001.ply through 021.ply
```

--------------------------------

### Registering ComfyUI-Sharp Nodes

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/01-nodes-overview.md

Shows how custom nodes are imported and aggregated in the '__init__.py' file for registration with ComfyUI.

```python
from .load_model import NODE_CLASS_MAPPINGS as LOAD_MODEL_MAPPINGS
from .predict import NODE_CLASS_MAPPINGS as PREDICT_MAPPINGS
# ... etc for all 8 nodes
```

--------------------------------

### SamplePanorama Configuration

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/00-index.md

Configuration for panorama sampling, controlling field of view and overlap.

```APIDOC
## SamplePanorama Configuration

### Description
Configures the `SamplePanorama` node, allowing adjustment of the field of view for sampled perspectives and the overlap between adjacent samples.

### Parameters
- **fov_degrees** (float): The Field of View in degrees for each perspective view. Lower values result in more detail but slower processing.
- **overlap_percent** (float): The percentage of overlap between adjacent sampled views. Higher overlap leads to smoother results but increases the number of samples and processing time.
```

--------------------------------

### SamplePanorama Node Execute Method

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/05-sample-panorama.md

Details the execution of the SamplePanorama node, including its parameters, return types, and the underlying behavior for sampling perspective views.

```APIDOC
## SamplePanorama Node Execute Method

This section details the `execute` method of the `SamplePanorama` node, which performs the core functionality of sampling perspective views from an equirectangular panorama.

### Method Signature

```python
@classmethod
execute(
    cls,
    panorama: torch.Tensor,
    fov_degrees: float = 65.0,
    overlap_percent: float = 10.0,
    output_size: int = 1536,
    skip_poles: bool = True
) -> io.NodeOutput
```

### Parameters

- **panorama** (torch.Tensor): IMAGE tensor [B,H,W,C] or [H,W,C], representing the equirectangular input.
- **fov_degrees** (float): Field of view for each perspective view in degrees. Default: 65.0.
- **overlap_percent** (float): Overlap between adjacent samples, as a percentage of FOV. Range: 0.0-50.0. Default: 10.0.
- **output_size** (int): The desired output resolution for each perspective view (square). Default: 1536.
- **skip_poles** (bool): If True, skips samples pointing directly up or down. Default: True.

### Return Type

`io.NodeOutput` containing the following:

- **images** (torch.Tensor): A tensor of shape [N, H, W, 3] containing the sampled perspective images.
- **extrinsics** (torch.Tensor): A tensor of shape [N, 4, 4] representing the world-to-camera matrices for each view.
- **intrinsics** (torch.Tensor): A tensor of shape [4, 4] containing the shared camera intrinsics matrix.
- **num_horizontal** (int): The number of samples taken horizontally.
- **num_vertical** (int): The number of samples taken vertically.

### Behavior

1.  **Grid Calculation**: Determines the number of horizontal and vertical samples based on `fov_degrees`, `overlap_percent`, and `skip_poles` to cover the 360° panorama.
2.  **Perspective Sampling**: For each output view, it calculates rays, rotates them to world space, and samples the input panorama using spherical coordinates and bilinear interpolation.
3.  **Camera Geometry**: Computes focal length, principal point, and constructs the shared intrinsics matrix and individual extrinsics matrices for each sampled view.
```

--------------------------------

### Minimal ComfyUI Workflow

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/00-index.md

A basic workflow for generating 3D Gaussians using ComfyUI-Sharp. It involves loading an image, configuring the SHARP model, and predicting the PLY output.

```text
Load Image → LoadSharpModel → SharpPredict → Output PLY
```

--------------------------------

### SamplePanorama Execute Method Signature

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/05-sample-panorama.md

Shows the signature for the execute method of the SamplePanorama node, detailing its parameters and return type.

```python
import torch
import io

# Assuming io and torch are available in the context

class SamplePanorama:
    @classmethod
    def execute(
        cls,
        panorama: torch.Tensor,
        fov_degrees: float = 65.0,
        overlap_percent: float = 10.0,
        output_size: int = 1536,
        skip_poles: bool = True
    ) -> io.NodeOutput:
        # Method implementation would go here
        pass

```

--------------------------------

### Recommended Panorama Reconstruction Pipeline

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/09-align-depth-maps.md

Outlines a step-by-step pipeline for panorama reconstruction, integrating depth map alignment and projection.

```text
1. Load panorama
2. SamplePanorama → perspective views
3. SharpPredictDepth → depth maps per view
4. AlignDepthMaps → scale correction
5. ProjectDepthToPanorama → check consistency (heatmap)
6. SharpPredict → PLY files
7. Optionally: ProjectDepthToPanorama → visualization
8. MergeGaussians → final scene
```

--------------------------------

### Quick Panorama Preview

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/12-workflows.md

Generate a quick preview of a panorama scene, skipping depth analysis for faster processing. This is useful for rapidly assessing alignment before committing to a full, high-quality reconstruction.

```python
# Fast path: skip alignment checking
pano, _, _ = LoadImageWithExif().execute("equirect.jpg")

images, extrinsics, intrinsics, _, _ = SamplePanorama().execute(
    panorama=pano,
    fov_degrees=65.0,
    overlap_percent=10.0
)

model_config = LoadSharpModel().execute()

# Skip depth analysis, go straight to Gaussians
ply_folder = SharpPredict().execute(
    model=model_config,
    image=images,
    extrinsics=extrinsics,
    intrinsics=intrinsics
)

merged_ply, num_g = MergeGaussians().execute(
    ply_folder=ply_folder,
    output_prefix="quick_panorama"
)
# 3-4 minutes total (skip depth analysis)
```

--------------------------------

### create_initializer

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/10-sharp-parameters.md

Creates the Gaussian initializer layer.

```APIDOC
## create_initializer

### Description
Creates the Gaussian initializer layer.

### Parameters
- **params** (params) - Parameters for the initializer.
```

--------------------------------

### LoadSharpModel

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/00-index.md

Configuration for loading Sharp models, allowing selection of precision for optimization.

```APIDOC
## LoadSharpModel Configuration

### Description
Configures the loading of Sharp models, allowing users to specify the desired precision for inference. This impacts performance and memory usage.

### Parameters
- **precision** (string): The precision mode for model loading. Options include:
    - `"auto"`: Automatically selects the best precision based on GPU capabilities (bf16 for Ampere+, fp16 for Volta/Turing, fp32 for older).
    - `"bf16"`: Uses bfloat16 precision (requires Ampere GPUs or newer).
    - `"fp16"`: Uses float16 precision (Volta, Turing, Ampere GPUs).
    - `"fp32"`: Uses float32 precision (compatible with all GPUs, but slower and uses more memory).
```

--------------------------------

### Panorama ComfyUI Workflow

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/00-index.md

A workflow for reconstructing 3D Gaussians from panoramic images. This involves loading the panorama, sampling it, and then using the SHARP model for prediction and merging.

```text
Load Panorama → SamplePanorama → [LoadSharpModel, SharpPredict] → MergeGaussians → Output PLY
```

--------------------------------

### Core Function: Sample Perspective from Equirectangular

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/05-sample-panorama.md

Samples a single perspective view from an equirectangular panorama. Returns the sampled image, extrinsics, and intrinsics.

```python
import torch

# Assuming torch is available in the context

def sample_perspective_from_equirectangular(
    panorama: torch.Tensor,
    yaw: float,  # radians
    pitch: float,  # radians
    fov_radians: float,
    output_size: int
) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor]:
    # Function implementation would go here
    pass

```

--------------------------------

### Single Image to 3D Gaussians

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/12-workflows.md

Converts a single photograph into a 3D Gaussian splat PLY file. Requires loading an image, loading the SHARP model, and then predicting the Gaussians.

```python
# Load image
image = LoadImage().execute("photo.jpg")[0]  # [1, H, W, 3]

# Load model
model_config = LoadSharpModel(precision="auto").execute()

# Generate Gaussians (focal_length defaults to 30mm)
ply_path, _, _ = SharpPredict().execute(
    model=model_config,
    image=image,
    focal_length_mm=35.0,
    output_prefix="my_scene"
)
# Output: output/my_scene_<timestamp>.ply
```

--------------------------------

### Model Loading Precision Configuration

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/00-index.md

Configure the precision for loading Sharp models. Options range from auto-detection to specific float types like bf16, fp16, and fp32, affecting performance and memory usage.

```python
precision: str = "auto"
# "auto": GPU-optimized (bf16 for Ampere+, fp16 for Volta/Turing, fp32 for older)
# "bf16": bfloat16 (Ampere GPUs and newer)
# "fp16": float16 (Volta, Turing, Ampere)
# "fp32": float32 (all GPUs, slower, less memory benefit)
```

--------------------------------

### create_monodepth_dpt

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/10-sharp-parameters.md

Creates a full monodepth network, including the encoder and decoder.

```APIDOC
## create_monodepth_dpt

### Description
Creates full monodepth network (encoder + decoder).

### Parameters
- **params** (MonodepthParams | None) - Parameters for the monodepth network.
- **dtype** - Data type.
- **device** - Device to use.
- **operations** - Operations to perform.
```

--------------------------------

### SamplePanorama Node Schema

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/05-sample-panorama.md

Defines the input and output schema for the SamplePanorama node, specifying parameters like FOV, overlap, and output size, as well as the types of data it produces.

```APIDOC
## SamplePanorama Node Schema

This section describes the schema definition for the `SamplePanorama` node, outlining its inputs and outputs.

### Inputs

- **panorama** (Image): The input equirectangular panorama image.
- **fov_degrees** (Float): Field of view for each perspective view in degrees. Range: [30.0, 120.0], Default: 65.0.
- **overlap_percent** (Float): Overlap between adjacent samples in percentage of FOV. Range: [0.0, 50.0], Default: 10.0.
- **output_size** (Int): Output resolution per view (square). Range: [256, 2048], Step: 64, Default: 1536.
- **skip_poles** (Boolean): Optional. Whether to skip samples pointing straight up/down. Default: True.

### Outputs

- **images** (Image): Batched perspective images [N, H, W, 3].
- **extrinsics** (Custom): Camera extrinsics [N, 4, 4] (world-to-camera matrices).
- **intrinsics** (Custom): Shared camera intrinsics [4, 4] (all views share the same intrinsics).
- **num_horizontal** (Int): Number of horizontal samples generated.
- **num_vertical** (Int): Number of vertical samples generated.
```

--------------------------------

### ComfyUI Sharp Predictor Creation (Default)

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/10-sharp-parameters.md

Standard usage for creating a predictor with default parameters. Ensures compatibility with the loaded checkpoint.

```python
# Standard usage (no custom config needed)
params = PredictorParams()  # Use defaults matching checkpoint
predictor = create_predictor(
    params,
    dtype=torch.float16,
    device="cuda:0",
    operations=comfy.ops.PickOperations(...)
)
```

--------------------------------

### SamplePanorama Node Schema Definition

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/05-sample-panorama.md

Defines the input and output schema for the SamplePanorama node, specifying parameters like FOV, overlap, and output size.

```python
import torch
import io

# Assuming io and torch are available in the context

class SamplePanorama:
    @classmethod
    def define_schema(cls):
        return io.Schema(
            node_id="SamplePanorama",
            display_name="Sample Panorama (Equirect -> Perspective)",
            category="SHARP",
            description="Sample perspective views from a 360 equirectangular panorama...",
            inputs=[
                io.Image.Input("panorama"),
                io.Float.Input("fov_degrees", default=65.0, min=30.0, max=120.0),
                io.Float.Input("overlap_percent", default=10.0, min=0.0, max=50.0),
                io.Int.Input("output_size", default=1536, min=256, max=2048, step=64),
                io.Boolean.Input("skip_poles", default=True, optional=True)
            ],
            outputs=[
                io.Image.Output(display_name="images"),
                io.Custom("EXTRINSICS").Output(display_name="extrinsics"),
                io.Custom("INTRINSICS").Output(display_name="intrinsics"),
                io.Int.Output(display_name="num_horizontal"),
                io.Int.Output(display_name="num_vertical")
            ]
        )

```

--------------------------------

### ProjectDepthToPanorama Execute Method Signature

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/08-project-depth-panorama.md

Defines the signature for the execute method, detailing the types and default values of parameters for projecting depth maps to a panorama.

```python
import torch
import io

class ProjectDepthToPanorama:
    @classmethod
    def execute(
        cls,
        depth_maps: torch.Tensor,
        extrinsics: torch.Tensor,
        intrinsics: torch.Tensor,
        panorama_width: int = 2048,
        blend_mode: str = "gaussian",
        show_borders: bool = True,
        border_color: str = "red",
        disagreement_scale: float = 0.2
    ) -> io.NodeOutput:
        pass

```

--------------------------------

### SamplePanorama

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/00-index.md

Samples a panorama from a 360° image, extracting views with specified field of view and overlap.

```APIDOC
## SamplePanorama

### Description
Samples a panorama from a 360° input image. It generates multiple perspective views based on the provided Field of View (FOV) and overlap settings.

### Inputs
- **panorama**: The input 360° panorama image.
- **FOV**: The Field of View in degrees for each sampled perspective view.
- **overlap**: The percentage of overlap between adjacent sampled views.

### Outputs
- **images**: The sampled perspective images.
- **extrinsics**: Extrinsic camera parameters for each sampled image.
- **intrinsics**: Intrinsic camera parameters for each sampled image.
```

--------------------------------

### LoadImageWithExif

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/00-index.md

Loads an image and its EXIF data. Can be used for inputting images into the pipeline.

```APIDOC
## LoadImageWithExif

### Description
Loads an image and its EXIF data. This node is typically used as the first step in a workflow to input image data.

### Inputs
- **image**: The input image.
- **default_focal**: A default focal length to use if EXIF data is missing.

### Outputs
- **image**: The loaded image.
- **mask**: A mask associated with the image, if any.
- **focal_mm**: The focal length in millimeters extracted from EXIF or provided as default.
```

--------------------------------

### Draw Sample Borders on Panorama

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/08-project-depth-panorama.md

Draws colored borders around each sample region's coverage area on the panorama image. Helps visualize sample coverage.

```python
def draw_sample_borders(panorama, extrinsics, intrinsics, border_color, border_width):
```

--------------------------------

### Execute Method for LoadImageWithExif Node

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/04-load-image-exif.md

The execute method processes the image input, extracts focal length from EXIF, and returns the image, mask, and focal length.

```python
import io
import torch
from PIL import Image, ImageOps

# Assume necessary helper functions and classes like io.NodeOutput, _get_image_files are defined

class LoadImageWithExif:
    # ... (define_schema method would be here)

    @classmethod
    def execute(cls, image: str, default_focal_mm: float = 30.0) -> io.NodeOutput:
        # Placeholder for actual implementation logic
        print(f"Loading image: {image} with default focal length: {default_focal_mm}")
        
        # Simulate image loading and processing
        try:
            img_pil = Image.open(image) # In a real scenario, this would use ComfyUI's path handling
            img_pil = ImageOps.exif_transpose(img_pil)
            
            # Simulate focal length extraction
            focal_length_mm = cls.extract_focal_length_mm(img_pil, default_focal_mm)
            
            # Simulate image and mask conversion
            # This is a highly simplified representation
            width, height = img_pil.size
            image_tensor = torch.rand(1, height, width, 3) # Placeholder tensor
            mask_tensor = torch.zeros(1, height, width) # Placeholder tensor
            
            return io.NodeOutput(image=image_tensor, mask=mask_tensor, focal_length_mm=focal_length_mm)
        except FileNotFoundError:
            return io.NodeOutput(error="Image file not found")
        except Exception as e:
            return io.NodeOutput(error=f"Error processing image: {e}")

    @classmethod
    def extract_focal_length_mm(cls, img_pil: Image.Image, default_mm: float = 30.0) -> float:
        # Placeholder for actual EXIF extraction logic
        exif_data = cls.extract_exif(img_pil)
        focal_mm = default_mm
        if "FocalLengthIn35mmFilm" in exif_data:
            focal_mm = exif_data["FocalLengthIn35mmFilm"]
        elif "FocalLenIn35mmFilm" in exif_data:
            focal_mm = exif_data["FocalLenIn35mmFilm"]
        elif "FocalLength" in exif_data:
            raw_focal = exif_data["FocalLength"]
            if raw_focal < 10.0:
                focal_mm = raw_focal * 8.4
            else:
                focal_mm = raw_focal
        print(f"Extracted focal length: {focal_mm} mm")
        return focal_mm

    @classmethod
    def extract_exif(cls, img_pil: Image.Image) -> dict:
        # Placeholder for actual EXIF extraction
        # In reality, this would parse img_pil.info or use piexif library
        return img_pil.getexif() or {}


```

--------------------------------

### Create SHARP Predictor

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/10-sharp-parameters.md

Factory function to create and initialize the SHARP predictor model. Accepts configuration parameters, dtype, device, and ComfyUI operations.

```python
def create_predictor(
    params: PredictorParams,
    dtype=None,
    device=None,
    operations=None
) -> RGBGaussianPredictor:
```

--------------------------------

### Factory Functions for Sharp Nodes

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/00-index.md

Provides factory functions to create various components of the Sharp system, including predictors, vision transformers, depth estimators, and initializers.

```python
create_predictor(params, dtype, device, operations) → RGBGaussianPredictor
create_vit(config, preset, ...) → VisionTransformer
create_monodepth_dpt(...) → MonodepthDensePredictionTransformer
create_gaussian_decoder(...) → GaussianDensePredictionTransformer
create_initializer(...) → MultiLayerInitializer
create_alignment(...) → LearnedAlignment | None
```

--------------------------------

### create_initializer Function Signature

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/10-sharp-parameters.md

Function signature for creating the Gaussian initializer layer. Takes AlignmentParams as input.

```python
def create_initializer(params) -> MultiLayerInitializer
```

--------------------------------

### create_gaussian_decoder

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/10-sharp-parameters.md

Creates a Gaussian decoder, which outputs positions, colors, scales, and opacities.

```APIDOC
## create_gaussian_decoder

### Description
Creates Gaussian decoder (outputs positions, colors, scales, rotations, opacities).

### Parameters
- **params** (GaussianDecoderParams) - Parameters for the Gaussian decoder.
- **dims_depth_features** (list[int]) - Dimensions of depth features.
- **dtype** - Data type.
- **device** - Device to use.
- **operations** - Operations to perform.
```

--------------------------------

### LoadSharpModel Execute Method Signature

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/02-load-sharp-model.md

Shows the class method signature for executing the LoadSharpModel node, including its parameters and return type.

```python
@classmethod
def execute(cls, precision: str = "auto") -> io.NodeOutput
```

--------------------------------

### Third-Party Dependencies for ComfyUI-Sharp

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/00-index.md

Outlines the essential third-party Python libraries that ComfyUI-Sharp depends on for its operations, including tensor computations, image processing, array manipulation, file I/O, and model downloading.

```python
torch                      # Tensor operations
PIL                        # Image loading, EXIF extraction
numpy                      # Array operations
plyfile                    # PLY file I/O
huggingface_hub            # Model downloading
```

--------------------------------

### JSON Output Structure for Alignment Information

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/09-align-depth-maps.md

Illustrates the JSON output format for alignment statistics, showing per-view scale and shift values.

```json
{
  "method": "global_optimization",
  "view_0": {
    "scale": 1.0,
    "shift": 0.0
  },
  "view_1": {
    "scale": 1.052,
    "shift": 0.0
  },
  ...
}
```

--------------------------------

### Utility Functions

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/00-index.md

Provides utility functions for image conversion, focal length calculation, and color space manipulation.

```APIDOC
## Utility Functions

### Description
This section details utility functions for common tasks such as image format conversion, focal length calculations, and color space transformations.

### Image Utilities (Module: nodes.utils.image)
- **comfy_to_numpy_rgb(tensor) → np.ndarray**: Converts a ComfyUI tensor to a NumPy RGB array.
- **convert_focallength(width, height, f_mm) → float**: Converts focal length from millimeters to pixels based on image dimensions.

### Color Space Utilities (Module: nodes.sharp.color_space)
- **sRGB2linearRGB(tensor) → tensor**: Converts a tensor from sRGB color space to linear RGB.
- **linearRGB2sRGB(tensor) → tensor**: Converts a tensor from linear RGB color space to sRGB.
- **encode_color_space(name) → int**: Encodes a color space name into an integer representation.
- **decode_color_space(int) → name**: Decodes an integer representation back into a color space name.
```

--------------------------------

### Batch Process Images from Folder

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/12-workflows.md

Process a folder of independent images sequentially. This script iterates through image files, loads them with EXIF data, and performs Sharp prediction, saving each result as a separate PLY file.

```python
import os
from pathlib import Path

image_folder = "input_images/"
model_config = LoadSharpModel(precision="auto").execute()

for image_file in sorted(os.listdir(image_folder)):
    if not image_file.lower().endswith((".jpg", ".png")):
        continue
    
    # Load
    image, _, focal_mm = LoadImageWithExif().execute(
        image=image_file,
        default_focal_mm=35.0
    )
    
    # Process
    ply_path, _, _ = SharpPredict().execute(
        model=model_config,
        image=image,
        focal_length_mm=focal_mm,
        output_prefix=Path(image_file).stem
    )
    
    print(f"✓ {image_file} → {ply_path}")
```

--------------------------------

### Raw Depth Extraction with SharpPredictDepth

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/06-sharp-predict-depth.md

Use this snippet to perform raw depth estimation from images using the SharpPredictDepth node. Ensure model_config, images, extrinsics, and intrinsics are properly defined.

```python
predict_depth = SharpPredictDepth()
depth_maps, ext, intr, align_maps = predict_depth.execute(
    model=model_config,
    image=images,  # From SamplePanorama
    extrinsics=extrinsics,  # From SamplePanorama
    intrinsics=intrinsics    # From SamplePanorama
)
```

--------------------------------

### ComfyUI Sharp Predictor Creation (Custom)

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/10-sharp-parameters.md

Advanced usage for creating a predictor with custom parameters, such as adjusting scale limits or color space.

```python
# Custom configuration (advanced)
params = PredictorParams(
    max_scale=15.0,
    min_scale=-5.0,
    color_space="sRGB"  # Switch to sRGB instead of linearRGB
)
predictor = create_predictor(params, ...)
```

--------------------------------

### AlignmentParams Configuration

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/10-sharp-parameters.md

Configuration for learned depth alignment, used when reference depth is provided. It includes parameters like kernel size, stride, and activation type.

```python
import dataclasses

@dataclasses.dataclass
class AlignmentParams:
    kernel_size: int = 16
    stride: int = 1
    frozen: bool = False
    steps: int = 4
    activation_type: str = "exp"
    depth_decoder_features: bool = False
    base_width: int = 16
```

--------------------------------

### SharpPredictDepth Node Execute Method Signature

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/06-sharp-predict-depth.md

Provides the signature for the execute method of the SharpPredictDepth node, outlining the parameters and their types for processing image data to generate depth maps.

```python
import torch
import io

@classmethod
@torch.no_grad()
def execute(
    cls,
    model: dict,
    image: torch.Tensor,
    extrinsics: torch.Tensor = None,
    intrinsics: torch.Tensor = None,
    reference_depth: torch.Tensor = None
) -> io.NodeOutput
```

--------------------------------

### Intrinsics Scaling Calculation

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/06-sharp-predict-depth.md

Demonstrates how to scale camera intrinsics (focal length and principal point) from the original image resolution to the depth map output resolution (1536x1536).

```python
scale_factor = output_size (1536) / input_width
intrinsics_scaled = intrinsics.clone()
intrinsics_scaled[0, 0] *= scale_factor  # f_px
intrinsics_scaled[1, 1] *= scale_factor  # f_py
intrinsics_scaled[0, 2] *= scale_factor  # cx
intrinsics_scaled[1, 2] *= scale_factor  # cy
```

--------------------------------

### Core Function: Create Rotation Matrix

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/05-sample-panorama.md

Creates a 3x3 camera-to-world rotation matrix from yaw and pitch angles. Follows a specific angle convention for yaw and pitch.

```python
import torch

# Assuming torch is available in the context

def create_rotation_matrix(yaw: float, pitch: float) -> torch.Tensor:
    # Function implementation would go here
    pass

```

--------------------------------

### ComfyUI APIs Used in ComfyUI-Sharp

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/00-index.md

Lists the core ComfyUI modules utilized by ComfyUI-Sharp for various functionalities like VRAM management, model patching, operations, utilities, and directory management.

```python
comfy.model_management     # VRAM management, dtype selection
comfy.model_patcher        # Model wrapping for VRAM
comfy.ops                  # Device/dtype aware operations
comfy.utils                # File loading, progress bars
folder_paths               # Input/output directory management
comfy_api.latest.io        # Node schema and I/O
```

--------------------------------

### Color Space Utilities

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/11-utility-functions.md

Provides functions for encoding, decoding, and converting between sRGB and linear RGB color spaces, along with type definitions for supported color spaces.

```APIDOC
## Color Space Utilities

### Module
`nodes/sharp/color_space.py`

### Description
Color space conversion functions.

### Types

#### `ColorSpace`
```python
ColorSpace = Literal["sRGB", "linearRGB"]
```
Type alias for supported color spaces.

### Functions

#### `encode_color_space(color_space: ColorSpace) -> int`
Encodes color space to integer for storage.
- "sRGB" → 0
- "linearRGB" → 1

#### `decode_color_space(color_space_index: int) -> ColorSpace`
Decodes integer back to color space name.
- 0 → "sRGB"
- 1 → "linearRGB"

#### `sRGB2linearRGB(sRGB: torch.Tensor) -> torch.Tensor`
Converts sRGB to linear RGB.

**Formula**:
```
if x ≤ 0.04045:
    y = x / 12.92
else:
    y = ((x + 0.055) / 1.055) ^ 2.4
```

#### `linearRGB2sRGB(linearRGB: torch.Tensor) -> torch.Tensor`
Converts linear RGB to sRGB.

**Formula**:
```
if x ≤ 0.0031308:
    y = x * 12.92
else:
    y = 1.055 * x^(1/2.4) - 0.055
```
```

--------------------------------

### Image Format Conversions

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/11-utility-functions.md

Provides standard ComfyUI format conversion utilities for images and masks, including conversions between ComfyUI tensors, NumPy arrays, PIL Images, and CHW formats.

```APIDOC
## Image Format Conversions

### Module
`nodes/image_utils.py`

### Description
Standard ComfyUI format conversions for images and masks.

### Functions

#### `comfy_to_numpy(image: torch.Tensor, index: int = 0) -> np.ndarray`
Converts ComfyUI IMAGE [B, H, W, C] float32 to NumPy [H, W, C] uint8.

#### `numpy_to_comfy(image: np.ndarray) -> torch.Tensor`
Converts NumPy [H, W, C] uint8 to ComfyUI IMAGE [1, H, W, C] float32.

#### `comfy_to_chw(image: torch.Tensor) -> torch.Tensor`
Converts IMAGE [B, H, W, C] to model input [B, C, H, W].

#### `chw_to_comfy(image: torch.Tensor) -> torch.Tensor`
Converts model output [B, C, H, W] to IMAGE [B, H, W, C].

#### `comfy_to_pil(image: torch.Tensor, index: int = 0) -> PIL.Image`
Converts IMAGE to PIL Image (RGB).

#### `pil_to_comfy(image: PIL.Image) -> torch.Tensor`
Converts PIL Image to IMAGE [1, H, W, C].

#### `mask_to_image(mask: torch.Tensor) -> torch.Tensor`
Converts MASK [B, H, W] to IMAGE [B, H, W, 3] (grayscale RGB).

#### `image_to_mask(image: torch.Tensor) -> torch.Tensor`
Converts IMAGE [B, H, W, C] to MASK [B, H, W] using ITU-R BT.601 luminance formula.
```

--------------------------------

### ViTConfig Configuration

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/10-sharp-parameters.md

Configuration for the Vision Transformer backbone. Requires input channels, embedding dimension, depth, number of heads, and layer scale initialization.

```python
import dataclasses

@dataclasses.dataclass
class ViTConfig:
    in_chans: int  # Input channels (always 3 for RGB)
    embed_dim: int  # Embedding dimension
    depth: int  # Number of transformer blocks
    num_heads: int  # Number of attention heads
    init_values: float  # Layer scale initialization
    img_size: int = 384  # Input size (always 384)
    patch_size: int = 16  # Patch size (always 16)
    num_classes: int = 21841  # Classification head size
    mlp_ratio: float = 4.0  # MLP hidden dimension ratio
    qkv_bias: bool = True  # Use bias in QKV projection
    global_pool: str = "avg"  # Global pooling strategy
    mlp_mode: MLPMode = "vanilla"  # "vanilla" or "glu"
    intermediate_features_ids: list[int] | None = None  # Hook layer indices
```

--------------------------------

### Data Structures

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/00-index.md

Defines key data structures used within the Sharp system, including Gaussians3D and SceneMetaData.

```APIDOC
## Data Structures (Module: nodes.sharp.gaussians)

### Description
Defines essential data structures used for representing 3D Gaussians and scene metadata within the Sharp system.

### Structures
- **Gaussians3D(NamedTuple)**:
    - `mean_vectors`: Array of mean vectors for each Gaussian (shape [N, 3]).
    - `singular_values`: Singular values for each Gaussian (shape [N, 3]).
    - `quaternions`: Quaternions representing rotation for each Gaussian (shape [N, 4]).
    - `colors`: RGB colors for each Gaussian (shape [N, 3]).
    - `opacities`: Opacity values for each Gaussian (shape [N]).

- **SceneMetaData(NamedTuple)**:
    - `focal_length_px`: Focal length in pixels.
    - `resolution_px`: Resolution of the scene in pixels (width, height).
    - `color_space`: The color space of the scene data.
```

--------------------------------

### Define Schema for LoadImageWithExif Node

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/04-load-image-exif.md

Defines the input and output schema for the LoadImageWithExif node, specifying image input, a default focal length, and image, mask, and focal length outputs.

```python
import io

# Assuming _get_image_files is defined elsewhere and returns a list of image file options
def _get_image_files():
    # Placeholder for actual implementation
    return ["image1.png", "image2.jpg"]

class LoadImageWithExif:
    @classmethod
    def define_schema(cls):
        return io.Schema(
            node_id="LoadImageWithExif",
            display_name="Load Image with EXIF (Focal Length)",
            category="SHARP",
            description="Load an image and extract focal length from EXIF metadata...",
            inputs=[
                io.Combo.Input("image", options=_get_image_files(), upload=io.UploadType.image),
                io.Float.Input("default_focal_mm", default=30.0, min=1.0, max=500.0, step=0.1)
            ],
            outputs=[
                io.Image.Output(display_name="image"),
                io.Mask.Output(display_name="mask"),
                io.Float.Output(display_name="focal_length_mm")
            ]
        )
```

--------------------------------

### Create Disagreement Heatmap

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/08-project-depth-panorama.md

Generates a color visualization of depth disagreement across the panorama. Useful for identifying alignment issues.

```python
def create_disagreement_heatmap(
    disagreement: torch.Tensor,
    overlap_count: torch.Tensor,
    max_disagreement: float = 0.2,
    add_legend: bool = True
) -> torch.Tensor:
```

--------------------------------

### InitializerParams Dataclass

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/10-sharp-parameters.md

Dataclass for configuring the Gaussian initializer layer. Includes settings for scale, disparity, stride, and depth initialization options.

```python
@dataclasses.dataclass
class InitializerParams:
    scale_factor: float = 1.0
    disparity_factor: float = 1.0
    stride: int = 2
    num_layers: int = 2
    first_layer_depth_option: DepthInitOption = "surface_min"
    rest_layer_depth_option: DepthInitOption = "surface_min"
    color_option: ColorInitOption = "all_layers"
    base_depth: float = 10.0
    feature_input_stop_grad: bool = False
    normalize_depth: bool = True
    output_inpainted_layer_only: bool = False
    set_uninpainted_opacity_to_zero: bool = False
    concat_inpainting_mask: bool = False
```

--------------------------------

### ProjectDepthToPanorama Schema Definition

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/08-project-depth-panorama.md

Defines the input and output schema for the ProjectDepthToPanorama node, including image inputs, camera extrinsics and intrinsics, panorama dimensions, blend mode, border options, and disagreement scale.

```python
import torch
import io

class ProjectDepthToPanorama:
    @classmethod
    def define_schema(cls):
        return io.Schema(
            node_id="ProjectDepthToPanorama",
            display_name="Project Depth to Panorama",
            category="SHARP",
            description="Project depth maps to panorama. Outputs disagreement heatmap...",
            inputs=[
                io.Image.Input("depth_maps"),
                io.Custom("EXTRINSICS").Input("extrinsics"),
                io.Custom("INTRINSICS").Input("intrinsics"),
                io.Int.Input("panorama_width", default=2048, min=512, max=8192, step=64),
                io.Combo.Input("blend_mode",
                    options=["gaussian", "cosine", "quadratic", "linear", "feather", "none"],
                    default="gaussian"),
                io.Boolean.Input("show_borders", default=True, optional=True),
                io.Combo.Input("border_color",
                    options=["red", "green", "blue", "white", "yellow"],
                    default="red", optional=True),
                io.Float.Input("disagreement_scale", default=0.2, min=0.01, max=1.0)
            ],
            outputs=[
                io.Image.Output(display_name="panoramic_depth"),
                io.Image.Output(display_name="debug_overlay"),
                io.Image.Output(display_name="disagreement_heatmap")
            ]
        )

```

--------------------------------

### Horizontal Loop Logic for Panorama Sampling

Source: https://github.com/pozzettiandrea/comfyui-sharp/blob/main/_autodocs/05-sample-panorama.md

Defines the calculation for the horizontal index (h_idx) to cover a full 360-degree range for panorama sampling.

```text
yaw = -180° + (h_idx + 0.5) * step_degrees
covers: -180° to +180° (full circle)
```