### Install ComfyUI Gaussian Preview Source: https://github.com/yichengup/comfyui-gaussian_preview/blob/main/README.md Clone the repository into the ComfyUI custom_nodes directory to install the node. ```bash cd ComfyUI/custom_nodes git clone https://github.com/your-username/ComfyUI-gaussian_preview.git ``` -------------------------------- ### Frontend Preview Control API Source: https://context7.com/yichengup/comfyui-gaussian_preview/llms.txt Demonstrates how to interact with the preview window using postMessage for recording and data loading. ```javascript // 预览窗口控件功能: // 1. Scale 缩放控制 (0.01 - 100) // 调整高斯点云的显示比例,实时生效 gaussianScaleValue.value = 1.0; // 默认值 // 输入新值后自动应用到所有高斯点 // 2. Reset View 重置视角 // 点击按钮重置相机到初始位置(基于 intrinsics/extrinsics) document.getElementById('resetCamera').click(); // 3. Max (s) 最大录制时长 (1-600秒) maxRecordTime.value = 30; // 默认30秒 // 4. Start Record / Stop Record 视频录制 // 录制格式:MP4 (H.264) // 帧率:30 FPS // 码率:5 Mbps // 分辨率:与预览窗口一致(由 preview_width 参数控制) // 视频录制消息通信示例: window.addEventListener('message', (event) => { if (event.data.type === 'VIDEO_RECORDING') { // event.data.video: Base64 编码的视频数据 // event.data.mimeType: 'video/mp4' // event.data.timestamp: 录制时间戳 console.log('视频大小:', event.data.video.length); } }); // 加载 PLY 数据消息格式: iframe.contentWindow.postMessage({ type: "LOAD_MESH_DATA", data: arrayBuffer, // PLY 文件的 ArrayBuffer filename: "model.ply", // 文件名 extrinsics: extrinsicsMatrix, // 4x4 相机外参 intrinsics: intrinsicsMatrix, // 3x3 相机内参 timestamp: Date.now() }, "*", [arrayBuffer]); ``` -------------------------------- ### Initialize Gaussian Viewer and Listen for Messages Source: https://github.com/yichengup/comfyui-gaussian_preview/blob/main/web/viewer_gaussian.html Initializes the viewer and sets up a message listener to receive mesh data, extrinsics, and intrinsics from a parent window for loading. Logs viewer readiness. ```javascript og('\[GaussianViewer\] originalScales:', !!originalScales); } } scaleInput.addEventListener('change', (e) => updateGaussianScale(e.target.value)); // Event listeners document.getElementById('resetCamera').addEventListener('click', resetCamera); // 截图按钮已删除,事件监听器已移除 // document.getElementById('screenshot').addEventListener('click', takeScreenshot); // Listen for messages from parent window.addEventListener('message', (event) => { const { type, data, filename, extrinsics, intrinsics } = event.data; if (type === 'LOAD\_MESH\_DATA' && data) { console.log('\x5BGaussianViewer\x5D Received LOAD\_MESH\_DATA, size:', data.byteLength); console.log('\x5BGaussianViewer\x5D Extrinsics:', extrinsics); console.log('\x5BGaussianViewer\x5D Intrinsics:', intrinsics); loadPLYFromData(data, filename || 'gaussian.ply', extrinsics, intrinsics); } }); // Initialize on load initViewer(); console.log('\x5BGaussianViewer\x5D Ready - waiting for LOAD\_MESH message'); ``` -------------------------------- ### Integrate with ComfyUI-Sharp Source: https://github.com/yichengup/comfyui-gaussian_preview/blob/main/README.md Workflow representation for connecting the SHARP Predict node to the Gaussian Preview node. ```text [Load Image] → [SHARP Predict] → [Preview Gaussian] ↓ [video_path 输出] ``` -------------------------------- ### Preview Gaussian Node Configuration Source: https://context7.com/yichengup/comfyui-gaussian_preview/llms.txt Defines the input structure and camera parameter matrices for the YCGaussianPreview node. ```python # 基本使用 - 从 PLY 文件路径加载 # 在 ComfyUI 工作流中连接节点 # 节点输入参数: # - ply_path (STRING, 可选): 高斯点云 PLY 文件路径 # - gaussians (GAUSSIANS_3D, 可选): 来自 SHARP Predict 的高斯对象 # - extrinsics (EXTRINSICS, 可选): 4x4 相机外参矩阵 # - intrinsics (INTRINSICS, 可选): 3x3 相机内参矩阵 # - preview_width (INT, 默认512): 预览窗口宽度 (256-4096px) # 节点输出: # - video_path (STRING): 最新录制视频的文件路径 # - save_ply_path (STRING): PLY 文件保存路径 # 工作流示例 1: 使用 ComfyUI-Sharp # [Load Image] → [SHARP Predict] → [Preview Gaussian] # ↓ # video_path → [下游节点] # 工作流示例 2: 直接加载 PLY 文件 # [String Input: "/path/to/model.ply"] → [Preview Gaussian] # 相机参数格式示例: extrinsics = [ [1.0, 0.0, 0.0, 0.0], # R11, R12, R13, tx [0.0, 1.0, 0.0, 0.0], # R21, R22, R23, ty [0.0, 0.0, 1.0, 0.0], # R31, R32, R33, tz [0.0, 0.0, 0.0, 1.0] # 齐次坐标 ] intrinsics = [ [512.0, 0.0, 512.0], # fx, 0, cx (cx = image_width/2) [0.0, 512.0, 512.0], # 0, fy, cy (cy = image_height/2) [0.0, 0.0, 1.0] # 0, 0, 1 ] ``` -------------------------------- ### Integrate with ComfyUI-GeometryPack Source: https://github.com/yichengup/comfyui-gaussian_preview/blob/main/README.md Workflow representation for connecting GeometryPack nodes to the Gaussian Preview node. ```text [相关几何节点] → [高斯点云生成] → [Preview Gaussian] ↓ [video_path 输出] ``` -------------------------------- ### 初始化 gsplat.js 查看器与渲染循环 Source: https://context7.com/yichengup/comfyui-gaussian_preview/llms.txt 设置 WebGL 渲染环境、场景、相机及轨道控制器,并启动渲染循环以实现实时交互。 ```javascript // 初始化 gsplat.js 查看器 const SPLAT = window.GSPLAT; // 创建场景、相机、渲染器和控制器 const scene = new SPLAT.Scene(); const camera = new SPLAT.Camera(); const renderer = new SPLAT.WebGLRenderer(canvas); const controls = new SPLAT.OrbitControls(camera, canvas); // 设置默认焦距 camera.data.fx = 30; // 水平焦距 camera.data.fy = 30; // 垂直焦距 // 渲染循环 function frame() { controls.update(); renderer.render(scene, camera); requestAnimationFrame(frame); } frame(); // 从 ArrayBuffer 加载 PLY 文件 async function loadPLYFromData(arrayBuffer, filename) { const blob = new Blob([arrayBuffer], { type: 'application/octet-stream' }); const blobUrl = URL.createObjectURL(blob); await SPLAT.PLYLoader.LoadAsync(blobUrl, scene); URL.revokeObjectURL(blobUrl); // 获取加载的高斯点云对象 const splat = scene.objects[scene.objects.length - 1]; // 访问高斯点云数据 console.log('高斯点数量:', splat.data.scales.length / 3); console.log('场景中心:', splat.bounds.center()); console.log('场景尺寸:', splat.bounds.size()); } // 设置相机位置和目标 function setCameraFromExtrinsics(extrinsics, intrinsics, splat) { // 从 4x4 外参矩阵提取相机位置 const R = [ [extrinsics[0][0], extrinsics[0][1], extrinsics[0][2]], [extrinsics[1][0], extrinsics[1][1], extrinsics[1][2]], [extrinsics[2][0], extrinsics[2][1], extrinsics[2][2]] ]; const t = [extrinsics[0][3], extrinsics[1][3], extrinsics[2][3]]; // 相机位置 = -R^T * t const camPosX = -(R[0][0]*t[0] + R[1][0]*t[1] + R[2][0]*t[2]); const camPosY = -(R[0][1]*t[0] + R[1][1]*t[1] + R[2][1]*t[2]); const camPosZ = -(R[0][2]*t[0] + R[1][2]*t[1] + R[2][2]*t[2]); // 从内参计算焦距缩放 const fx = intrinsics[0][0]; const imageWidth = intrinsics[0][2] * 2; const scale = canvas.clientWidth / imageWidth; camera.data.fx = fx * scale; camera.data.fy = fx * scale; camera.position.x = camPosX; camera.position.y = -camPosY; // Y轴翻转 camera.position.z = camPosZ; controls.setCameraTarget(new SPLAT.Vector3(0, 0, targetZ)); } ``` -------------------------------- ### Frontend Preview Control API Source: https://context7.com/yichengup/comfyui-gaussian_preview/llms.txt API for controlling the interactive preview window and video recording features via JavaScript and `postMessage`. ```APIDOC ## Frontend Preview Control API ### Description This API describes the controls available in the preview window's footer, which allow real-time adjustment of Gaussian Splatting display and video recording. Communication between the frontend controls and the backend nodes is handled via the `postMessage` mechanism. ### Controls and Functionality 1. **Scale Control** * **Range**: 0.01 - 100 * **Functionality**: Adjusts the display scale of the Gaussian Splatting model. Changes are applied in real-time. * **Example**: `gaussianScaleValue.value = 1.0;` (Sets the default value) 2. **Reset View Button** * **Functionality**: Resets the camera to its initial position, determined by the `intrinsics` and `extrinsics` parameters. * **Example**: `document.getElementById('resetCamera').click();` 3. **Max Record Time Input** * **Range**: 1 - 600 seconds * **Default**: 30 seconds * **Functionality**: Sets the maximum duration for video recordings. * **Example**: `maxRecordTime.value = 30;` 4. **Start Record / Stop Record Buttons** * **Functionality**: Initiates or terminates video recording. * **Format**: MP4 (H.264 codec) * **Frame Rate**: 30 FPS * **Bitrate**: 5 Mbps * **Resolution**: Matches the preview window size (controlled by `preview_width`). ### Message Communication Examples **Receiving Video Recording Data:** ```javascript window.addEventListener('message', (event) => { if (event.data.type === 'VIDEO_RECORDING') { // event.data.video: Base64 encoded video data // event.data.mimeType: 'video/mp4' // event.data.timestamp: Recording timestamp console.log('Video size:', event.data.video.length); } }); ``` **Sending PLY Data to Load:** ```javascript iframe.contentWindow.postMessage({ type: "LOAD_MESH_DATA", data: arrayBuffer, // PLY file's ArrayBuffer filename: "model.ply", // Filename extrinsics: extrinsicsMatrix, // 4x4 camera extrinsic matrix intrinsics: intrinsicsMatrix, // 3x3 camera intrinsic matrix timestamp: Date.now() }, "*", [arrayBuffer]); ``` ``` -------------------------------- ### Preview Gaussian Node (YCGaussianPreview) Source: https://context7.com/yichengup/comfyui-gaussian_preview/llms.txt The main node for previewing and recording Gaussian Splatting models within ComfyUI. It supports loading data from PLY file paths or GAUSSIANS_3D objects and handles camera parameters for accurate perspective. ```APIDOC ## Preview Gaussian Node (YCGaussianPreview) ### Description This node displays and records 3D Gaussian Splatting models in ComfyUI. It can load data from PLY file paths or GAUSSIANS_3D objects, automatically processing camera parameters to restore the original viewpoint. It also provides video recording capabilities. ### Method (Custom Node - No direct HTTP method) ### Endpoint (Custom Node - Operates within ComfyUI) ### Parameters #### Input Parameters - **ply_path** (STRING, Optional) - Path to the Gaussian Splatting PLY file. - **gaussians** (GAUSSIANS_3D, Optional) - Gaussian object from SHARP Predict. - **extrinsics** (EXTRINSICS, Optional) - 4x4 camera extrinsic matrix. - **intrinsics** (INTRINSICS, Optional) - 3x3 camera intrinsic matrix. - **preview_width** (INT, Default: 512) - Width of the preview window (range: 256-4096px). #### Output Parameters - **video_path** (STRING) - File path of the latest recorded video. - **save_ply_path** (STRING) - File path where the PLY file is saved. ### Workflow Examples **Example 1: Using ComfyUI-Sharp** `[Load Image] -> [SHARP Predict] -> [Preview Gaussian]` ` | V video_path -> [Downstream Node]` **Example 2: Loading PLY File Directly** `[String Input: "/path/to/model.ply"] -> [Preview Gaussian]` ### Camera Parameter Format Examples ```python extrinsics = [ [1.0, 0.0, 0.0, 0.0], [0.0, 1.0, 0.0, 0.0], [0.0, 0.0, 1.0, 0.0], [0.0, 0.0, 0.0, 1.0] ] intrinsics = [ [512.0, 0.0, 512.0], [0.0, 512.0, 512.0], [0.0, 0.0, 1.0] ] ``` ``` -------------------------------- ### Save PLY Node Configuration Source: https://context7.com/yichengup/comfyui-gaussian_preview/llms.txt Outlines the input parameters and usage scenarios for the SavePLYNode to persist Gaussian data. ```python # 节点输入参数: # - ply_path (STRING, 可选): 现有 PLY 文件路径,直接返回 # - gaussians (GAUSSIANS_3D, 可选): 需要保存的高斯对象 # - extrinsics (EXTRINSICS, 可选): 相机外参(用于元数据) # - intrinsics (INTRINSICS, 可选): 相机内参(用于提取焦距和图像尺寸) # - output_prefix (STRING, 默认"gaussians"): 输出文件名前缀 # 节点输出: # - ply_path (STRING): 保存的 PLY 文件完整路径 # 使用场景 1: 保存 SHARP 生成的高斯对象 # [SHARP Predict] → gaussians → [Save PLY] → ply_path → [Preview Gaussian] # 使用场景 2: 传递已有 PLY 文件 # [String: "model.ply"] → ply_path → [Save PLY] → ply_path (原样返回) # 输出文件命名格式:{output_prefix}_{timestamp}.ply # 示例:gaussians_1704067200000.ply # 文件保存位置:ComfyUI 输出目录 (output/) ``` -------------------------------- ### Save PLY Node (SavePLYNode) Source: https://context7.com/yichengup/comfyui-gaussian_preview/llms.txt Converts and saves GAUSSIANS_3D objects to PLY files, or passes through existing PLY file paths. Useful for persisting Gaussian Splatting data. ```APIDOC ## Save PLY Node (SavePLYNode) ### Description This node converts GAUSSIANS_3D objects into PLY files for persistent storage. It can also pass through existing PLY file paths without modification. It supports including camera extrinsic and intrinsic metadata. ### Method (Custom Node - No direct HTTP method) ### Endpoint (Custom Node - Operates within ComfyUI) ### Parameters #### Input Parameters - **ply_path** (STRING, Optional) - Path to an existing PLY file. If provided, this path is returned directly. - **gaussians** (GAUSSIANS_3D, Optional) - The Gaussian object to be saved. - **extrinsics** (EXTRINSICS, Optional) - Camera extrinsic matrix (4x4) to be included as metadata. - **intrinsics** (INTRINSICS, Optional) - Camera intrinsic matrix (3x3) used to extract focal length and image dimensions for metadata. - **output_prefix** (STRING, Default: "gaussians") - Prefix for the output PLY filename. #### Output Parameters - **ply_path** (STRING) - The full path to the saved PLY file. ### Usage Scenarios **Scenario 1: Saving a Gaussian Object from SHARP** `[SHARP Predict] -> gaussians -> [Save PLY] -> ply_path -> [Preview Gaussian]` **Scenario 2: Passing Through an Existing PLY File** `[String: "model.ply"] -> ply_path -> [Save PLY] -> ply_path (Returned as is)` ### Output File Naming Files are named using the format: `{output_prefix}_{timestamp}.ply`. Example: `gaussians_1704067200000.ply` Files are saved in the ComfyUI output directory (`output/`). ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.