### Install Dependencies for Ubuntu 22.04 Source: https://github.com/graphdeco-inria/gaussian-splatting/blob/main/README.md Installs necessary dependencies for building SIBR viewers on Ubuntu 22.04. This includes libraries for OpenGL, Assimp, Boost, GTK, OpenCV, GLFW, FFmpeg, and Eigen. ```shell # Dependencies sudo apt install -y libglew-dev libassimp-dev libboost-all-dev libgtk-3-dev libopencv-dev libglfw3-dev libavdevice-dev libavcodec-dev libeigen3-dev libxxf86vm-dev libembree-dev ``` -------------------------------- ### Run Gaussian Splatting Optimizer Source: https://github.com/graphdeco-inria/gaussian-splatting/blob/main/README.md Use this command to start the training process. Replace `` with the actual path to your dataset. ```shell python train.py -s ``` -------------------------------- ### Install Accelerated Rasterizer Source: https://github.com/graphdeco-inria/gaussian-splatting/blob/main/README.md Follow these steps to uninstall the default rasterizer, navigate to the submodule, checkout the accelerated branch, and install the custom rasterizer for faster training. ```bash pip uninstall diff-gaussian-rasterization -y cd submodules/diff-gaussian-rasterization rm -r build git checkout 3dgs_accel pip install . ``` -------------------------------- ### Install Submodules on Windows Source: https://github.com/graphdeco-inria/gaussian-splatting/blob/main/README.md Use these commands to install the necessary submodules on Windows if you encounter 'cl.exe' errors. Ensure you have the correct Visual Studio path added to your environment variables and start a new conda prompt. ```bash conda activate gaussian_splatting cd /gaussian-splatting pip install submodules\diff-gaussian-rasterization pip install submodules\simple-knn ``` -------------------------------- ### Build Viewers on Windows Source: https://github.com/graphdeco-inria/gaussian-splatting/blob/main/README.md Builds the SIBR viewers on Windows using CMake. Ensure you have Visual Studio and CMake installed. Specify a different configuration like 'Debug' if needed. ```shell cd SIBR_viewers cmake -Bbuild . cmake --build build --target install --config RelWithDebInfo ``` -------------------------------- ### Initialize and Manage `GaussianModel` Source: https://context7.com/graphdeco-inria/gaussian-splatting/llms.txt The `GaussianModel` class manages learnable 3D Gaussian parameters. It supports initialization from point clouds, property accessors with activations, optimizer setup, learning rate scheduling, adaptive density control, and serialization. ```python from scene.gaussian_model import GaussianModel from arguments import OptimizationParams from argparse import ArgumentParser # Instantiation gaussians = GaussianModel(sh_degree=3, optimizer_type="default") # optimizer_type: "default" (Adam) | "sparse_adam" (requires 3dgs_accel) # Key property accessors xyz = gaussians.get_xyz # [N, 3] 3D positions scale = gaussians.get_scaling # [N, 3] exp-activated scales rot = gaussians.get_rotation # [N, 4] normalized quaternions opac = gaussians.get_opacity # [N, 1] sigmoid-activated opacities feats = gaussians.get_features # [N, (sh_degree+1)^2, 3] SH color coefficients cov3d = gaussians.get_covariance(scaling_modifier=1.0) # [N, 6] upper-triangular covariance # Initialize from a COLMAP point cloud from utils.graphics_utils import BasicPointCloud import numpy as np pcd = BasicPointCloud( points=np.random.rand(10000, 3), colors=np.random.rand(10000, 3), normals=np.zeros((10000, 3)) ) gaussians.create_from_pcd(pcd, cam_infos=train_cam_list, spatial_lr_scale=2.4) # Set up optimizer with per-parameter learning rates parser = ArgumentParser() from arguments import OptimizationParams opt = OptimizationParams(parser).extract(parser.parse_args([])) gaussians.training_setup(opt) # Per-iteration learning rate scheduling gaussians.update_learning_rate(iteration=5000) # Promote SH degree every 1000 iterations (starts at 0, grows to sh_degree) gaussians.oneupSHdegree() # Adaptive density control gaussians.add_densification_stats(viewspace_point_tensor, visibility_filter) if iteration % 100 == 0: gaussians.densify_and_prune( max_grad=0.0002, min_opacity=0.005, extent=scene.cameras_extent, max_screen_size=20, radii=radii ) gaussians.reset_opacity() # periodically reset all opacities to 0.01 # Checkpoint save / restore torch.save((gaussians.capture(), iteration), "chkpnt30000.pth") model_args, iter = torch.load("chkpnt30000.pth") gaussians.restore(model_args, opt) # PLY serialization gaussians.save_ply("/output/garden/point_cloud/iteration_30000/point_cloud.ply") gaussians.load_ply("/output/garden/point_cloud/iteration_30000/point_cloud.ply") ``` -------------------------------- ### Create Conda Environment for Gaussian Splatting Source: https://github.com/graphdeco-inria/gaussian-splatting/blob/main/README.md Set up the Conda environment using the provided environment.yml file. Ensure CUDA SDK 11 is installed. For Windows, set the DISTUTILS_USE_SDK environment variable. ```shell SET DISTUTILS_USE_SDK=1 # Windows only conda env create --file environment.yml conda activate gaussian_splatting ``` -------------------------------- ### Run convert.py for Scene Processing Source: https://github.com/graphdeco-inria/gaussian-splatting/blob/main/README.md Use this script to convert input images into the COLMAP dataset structure. It can optionally resize images and requires COLMAP and ImageMagick to be installed. Specify custom executable paths if they are not in your system's PATH. ```shell python convert.py -s [--resize] #If not resizing, ImageMagick is not needed ``` ```shell python convert.py -s --skip_matching [--resize] #If not resizing, ImageMagick is not needed ``` -------------------------------- ### Clone Depth Anything V2 Repository Source: https://github.com/graphdeco-inria/gaussian-splatting/blob/main/README.md Clone the Depth Anything V2 repository to generate depth maps for real-world datasets. Ensure you have Git installed. ```bash git clone https://github.com/DepthAnything/Depth-Anything-V2.git ``` -------------------------------- ### Initialize and Connect to Network GUI Server Source: https://context7.com/graphdeco-inria/gaussian-splatting/llms.txt Initializes the TCP socket server for the SIBR network viewer and attempts to connect to a running training process. The `init` function should be called once before the training loop, and `try_connect` is used within the loop for non-blocking connection attempts. ```python from gaussian_renderer import network_gui # Initialize server (called once before training loop) network_gui.init(wish_host="127.0.0.1", wish_port=6009) # Inside training loop if network_gui.conn is None: network_gui.try_connect() # non-blocking accept() ``` -------------------------------- ### Run Gaussian Viewer with Model Path Source: https://github.com/graphdeco-inria/gaussian-splatting/blob/main/README.md Launches the viewer, specifying the path to a trained model. Use -s to override the source dataset path if needed. ```shell .//bin/SIBR_gaussianViewer_app -m ``` -------------------------------- ### Train a 3D Gaussian Model Source: https://context7.com/graphdeco-inria/gaussian-splatting/llms.txt Use this command to initiate a standard full training run. Specify dataset and output directories. Optional flags control evaluation, iteration counts, saving checkpoints, SH degree, learning rates, densification thresholds, and network viewer settings. ```bash python train.py \ -s /data/garden \ -m /output/garden \ --eval \ --iterations 30000 \ --test_iterations 7000 30000 \ --save_iterations 7000 30000 \ --checkpoint_iterations 15000 \ --sh_degree 3 \ --position_lr_init 0.00016 \ --densify_grad_threshold 0.0002 \ --lambda_dssim 0.2 \ --ip 127.0.0.1 --port 6009 ``` ```bash python train.py -s /data/garden -m /output/garden \ --start_checkpoint /output/garden/chkpnt15000.pth ``` ```bash python train.py -s /data/garden -m /output/garden \ --optimizer_type sparse_adam ``` ```bash python train.py -s /data/garden -m /output/garden \ -d /data/garden/depths \ --depth_l1_weight_init 1.0 --depth_l1_weight_final 0.01 ``` ```bash python train.py -s /data/garden -m /output/garden \ --exposure_lr_init 0.001 --exposure_lr_final 0.0001 \ --exposure_lr_delay_steps 5000 --exposure_lr_delay_mult 0.001 \ --train_test_exp ``` ```bash python train.py -s /data/garden -m /output/garden --antialiasing ``` -------------------------------- ### Build Viewers on Ubuntu 22.04 Source: https://github.com/graphdeco-inria/gaussian-splatting/blob/main/README.md Sets up and builds the SIBR viewers on Ubuntu 22.04 using CMake. You can add '-G Ninja' for faster builds. The '-j24' flag specifies the number of parallel jobs. ```shell # Project setup cd SIBR_viewers cmake -Bbuild . -DCMAKE_BUILD_TYPE=Release # add -G Ninja to build faster cmake --build build -j24 --target install ``` -------------------------------- ### Run Network Viewer Source: https://github.com/graphdeco-inria/gaussian-splatting/blob/main/README.md Launches the remote Gaussian Splatting network viewer application. By default, it attempts to connect to the optimizer on localhost:6009. Use command-line arguments to specify IP, port, or data source path. ```shell .//bin/SIBR_remoteGaussian_app ``` -------------------------------- ### Full Benchmark Evaluation Source: https://context7.com/graphdeco-inria/gaussian-splatting/llms.txt Executes the full evaluation pipeline, including training, rendering, and metrics calculation for benchmark scenes. This command requires specifying paths for MipNeRF360, Tanks&Temples, and Deep Blending datasets, along with an output path. The evaluation can take approximately 7 hours on an A6000 GPU. ```bash python full_eval.py \ -m360 /data/mipnerf360 \ -tat /data/tanks_and_temples \ -db /data/deep_blending \ --output_path /results ``` -------------------------------- ### Compute Metrics on Pre-rendered Images (Skip Training and Rendering) Source: https://context7.com/graphdeco-inria/gaussian-splatting/llms.txt Computes metrics on images that have already been rendered, skipping both the training and rendering phases. This is useful for analyzing results or re-calculating metrics. Provide the paths to the pre-rendered image directories. ```bash python full_eval.py \ --skip_training --skip_rendering \ -m /results/garden /results/bicycle /results/room ``` -------------------------------- ### Programmatic Rendering with Gaussian Splatting Source: https://context7.com/graphdeco-inria/gaussian-splatting/llms.txt This Python code demonstrates how to programmatically load a trained Gaussian model and render specific camera views. It requires importing necessary classes and setting up rendering parameters. ```python from scene import Scene from gaussian_renderer import render, GaussianModel from arguments import ModelParams, PipelineParams, get_combined_args import torch from argparse import ArgumentParser parser = ArgumentParser() model_params = ModelParams(parser, sentinel=True) pipeline_params = PipelineParams(parser) args = get_combined_args(parser) gaussians = GaussianModel(model_params.extract(args).sh_degree) scene = Scene(model_params.extract(args), gaussians, load_iteration=-1, shuffle=False) bg = torch.tensor([0, 0, 0], dtype=torch.float32, device="cuda") pipe = pipeline_params.extract(args) for view in scene.getTestCameras(): pkg = render(view, gaussians, pipe, bg) image = pkg["render"] # [3, H, W] float tensor, values in [0, 1] depth = pkg["depth"] # [1, H, W] inverse-depth map ``` -------------------------------- ### Camera Geometry Transformations in Python Source: https://context7.com/graphdeco-inria/gaussian-splatting/llms.txt Provides utilities for camera transformations, including world-to-view matrix generation, OpenGL-style projection matrix creation, focal length/FoV conversion, and point transformation. ```python from utils.graphics_utils import ( getWorld2View2, getProjectionMatrix, fov2focal, focal2fov, geom_transform_points ) import numpy as np import torch # World-to-view matrix (with optional scene re-centering) R = np.eye(3) t = np.array([0.0, 0.0, -3.0]) W2V = getWorld2View2(R, t, translate=np.zeros(3), scale=1.0) # [4,4] float32 # OpenGL-style projection matrix P = getProjectionMatrix(znear=0.01, zfar=100.0, fovX=1.047, fovY=0.785) # [4,4] tensor # Convert between focal length and FoV fov = focal2fov(focal=800.0, pixels=1920) # radians focal = fov2focal(fov=1.047, pixels=1920) # pixels # Homogeneous point transform pts = torch.rand(1000, 3, device="cuda") M = torch.eye(4, device="cuda") pts_out = geom_transform_points(pts, M) # [1000, 3] ``` -------------------------------- ### Use Sparse Adam Optimizer Source: https://github.com/graphdeco-inria/gaussian-splatting/blob/main/README.md Add this flag to your train.py command to enable the sparse Adam optimizer, which can significantly speed up training times. ```bash --optimizer_type sparse_adam ``` -------------------------------- ### Enable Exposure Compensation during Training Source: https://github.com/graphdeco-inria/gaussian-splatting/blob/main/README.md Add these parameters to your training command to enable exposure compensation, which optimizes an affine transformation for each image to handle exposure changes. ```bash --exposure_lr_init 0.001 --exposure_lr_final 0.0001 --exposure_lr_delay_steps 5000 --exposure_lr_delay_mult 0.001 --train_test_exp ``` -------------------------------- ### Receive and Send Data with Network GUI Source: https://context7.com/graphdeco-inria/gaussian-splatting/llms.txt Handles receiving custom camera viewpoints and training control signals from the SIBR viewer, rendering frames, and sending them back. The `receive` function returns camera data, training flags, and scaling modifiers. Frames are sent back as bytes, and the loop breaks if `do_training` is true or an exception occurs. ```python while network_gui.conn is not None: try: # receive() returns (MiniCam | None, do_training, do_shs_python, # do_rot_scale_python, keep_alive, scaling_modifier) custom_cam, do_training, shs_python, cov_python, keep_alive, scale_mod = \ network_gui.receive() if custom_cam is not None: img = render(custom_cam, gaussians, pipe, background, scaling_modifier=scale_mod)["render"] img_bytes = memoryview( (torch.clamp(img, 0, 1) * 255).byte() .permute(1, 2, 0).contiguous().cpu().numpy() ) else: img_bytes = None # send rendered frame + source_path as verification string network_gui.send(img_bytes, dataset.source_path) if do_training: break # resume training iteration except Exception: network_gui.conn = None ``` -------------------------------- ### Calculate Loss Functions in Python Source: https://context7.com/graphdeco-inria/gaussian-splatting/llms.txt Demonstrates the usage of L1, L2, SSIM, and fused-SSIM loss functions. The default training objective combines L1 and SSIM losses. ```python from utils.loss_utils import l1_loss, l2_loss, ssim, fast_ssim import torch rendered = torch.rand(3, 1080, 1920, device="cuda") gt = torch.rand(3, 1080, 1920, device="cuda") # Pixel-wise L1 loss_l1 = l1_loss(rendered, gt) # scalar tensor # Pixel-wise L2 (MSE) loss_l2 = l2_loss(rendered, gt) # scalar tensor # Structural similarity (window_size=11, range [0,1]; higher = more similar) loss_ssim = ssim(rendered, gt) # scalar tensor # Per-pixel SSIM map (no spatial averaging) ssim_map = ssim(rendered, gt, size_average=False) # [B] tensor # Fused CUDA SSIM (faster, requires diff-gaussian-rasterization with fusedssim) loss_fast_ssim = fast_ssim( rendered.unsqueeze(0), # expects [B, C, H, W] gt.unsqueeze(0) ) # Combined training loss lambda_dssim = 0.2 loss = (1.0 - lambda_dssim) * loss_l1 + lambda_dssim * (1.0 - loss_ssim) loss.backward() ``` -------------------------------- ### Full Evaluation of Pre-trained Models Source: https://github.com/graphdeco-inria/gaussian-splatting/blob/main/README.md Perform a full evaluation of pre-trained models by specifying their location and skipping the training stage. This allows for direct metric computation on existing models. ```shell python full_eval.py -o --skip_training -m360 -tat -db ``` -------------------------------- ### Render Pre-trained Models Source: https://github.com/graphdeco-inria/gaussian-splatting/blob/main/README.md Render pre-trained models by specifying the model path and the source dataset location. Note that metrics may differ from the paper due to codebase improvements. ```shell python render.py -m -s ``` -------------------------------- ### Parse Hyperparameters with Argument Groups in Python Source: https://context7.com/graphdeco-inria/gaussian-splatting/llms.txt Illustrates how to use `ParamGroup` subclasses to parse and manage hyperparameters for model, pipeline, and optimization settings. Uses `argparse` for command-line argument handling. ```python from arguments import ModelParams, PipelineParams, OptimizationParams, get_combined_args from argparse import ArgumentParser parser = ArgumentParser() lp = ModelParams(parser) # -s, -m, -i, --sh_degree, --eval, --data_device, … op = OptimizationParams(parser) # --iterations, --position_lr_init, --lambda_dssim, … pp = PipelineParams(parser) # --convert_SHs_python, --antialiasing, --debug, … args = parser.parse_args([ "-s", "/data/garden", "-m", "/output/garden", "--iterations", "30000", "--optimizer_type", "sparse_adam", "--antialiasing", ]) dataset = lp.extract(args) # GroupParams with source_path, model_path, sh_degree, … opt = op.extract(args) # GroupParams with iterations=30000, lambda_dssim=0.2, … pipe = pp.extract(args) # GroupParams with antialiasing=True, debug=False, … # get_combined_args merges cfg_args saved alongside the model with new CLI flags # (used by render.py and metrics.py so re-specifying training params is optional) args2 = get_combined_args(parser) ``` -------------------------------- ### Evaluate with Pre-rendered Images Source: https://github.com/graphdeco-inria/gaussian-splatting/blob/main/README.md Compute metrics directly from evaluation images without rendering. This is useful if you already have the desired renderings or evaluation images. ```shell python full_eval.py -m /garden ... --skip_training --skip_rendering ``` -------------------------------- ### Load and Manage Scene Data with `Scene` Source: https://context7.com/graphdeco-inria/gaussian-splatting/llms.txt Use the `Scene` class to load camera data from COLMAP or NeRF Synthetic formats. It supports training (shuffling cameras) and inference (loading specific iterations), and allows saving the Gaussian model and exposure data. ```python from scene import Scene from scene.gaussian_model import GaussianModel from arguments import ModelParams from argparse import ArgumentParser, Namespace parser = ArgumentParser() mp = ModelParams(parser) args = parser.parse_args([ "-s", "/data/garden", "-m", "/output/garden", "--eval", "--sh_degree", "3" ]) dataset = mp.extract(args) gaussians = GaussianModel(dataset.sh_degree) # Training: load from scratch, shuffle cameras scene = Scene(dataset, gaussians) # scene.cameras_extent — float, scene radius used for learning rate scaling # scene.getTrainCameras() — list of Camera objects at scale 1.0 # scene.getTestCameras() — list of Camera objects at scale 1.0 # Inference: load existing model at iteration 30000 (pass -1 for latest) scene = Scene(dataset, gaussians, load_iteration=30000, shuffle=False) print(scene.loaded_iter) # 30000 # Save Gaussians + exposure.json at a given iteration scene.save(iteration=30000) # writes: /output/garden/point_cloud/iteration_30000/point_cloud.ply # writes: /output/garden/exposure.json # Multi-resolution loading (e.g., for progressive training) scene = Scene(dataset, gaussians, resolution_scales=[1.0, 2.0, 4.0]) train_cams_half_res = scene.getTrainCameras(scale=2.0) ``` -------------------------------- ### Evaluate Pre-trained Models (Skip Training) Source: https://context7.com/graphdeco-inria/gaussian-splatting/llms.txt Evaluates pre-trained models without re-running the training process. This is useful for benchmarking or testing existing models. The `--skip_training` flag prevents the training phase, and dataset paths must still be provided. ```bash python full_eval.py \ -o /pretrained_models \ --skip_training \ -m360 /data/mipnerf360 \ -tat /data/tanks_and_temples \ -db /data/deep_blending ``` -------------------------------- ### Full Evaluation Script Source: https://github.com/graphdeco-inria/gaussian-splatting/blob/main/README.md Run the `full_eval.py` script for a comprehensive evaluation routine, specifying paths to various datasets. This script can take a significant amount of time. ```shell python full_eval.py -m360 -tat -db ``` -------------------------------- ### Generate Depth Maps with Depth Anything V2 Source: https://github.com/graphdeco-inria/gaussian-splatting/blob/main/README.md Use the Depth Anything V2 run script to generate depth maps. Specify the model encoder, image path, and output directory. The --pred-only and --grayscale flags are recommended for this use case. ```bash python Depth-Anything-V2/run.py --encoder vitl --pred-only --grayscale --img-path --outdir ``` -------------------------------- ### Render a Trained 3D Gaussian Model Source: https://context7.com/graphdeco-inria/gaussian-splatting/llms.txt Generate renderings for train and test splits from a trained model. Specify the model path and iteration. Use `--skip_train` to omit training set rendering and `--quiet` for less verbose output. The source dataset path can be overridden if the model was trained elsewhere. ```bash python render.py \ -m /output/garden \ --iteration 30000 \ --skip_train \ --quiet ``` ```bash python render.py -m /output/garden -s /data/garden ``` -------------------------------- ### Render and Compute Metrics Source: https://github.com/graphdeco-inria/gaussian-splatting/blob/main/README.md Generate renderings of a trained model and compute error metrics. This is typically done after training with an evaluation split. ```shell python render.py -m # Generate renderings ``` ```shell python metrics.py -m # Compute error metrics on renderings ``` -------------------------------- ### Render 3D Gaussians with `render()` Source: https://context7.com/graphdeco-inria/gaussian-splatting/llms.txt Use the `render()` function to rasterize 3D Gaussians onto an image plane. It requires camera parameters, the Gaussian model, pipeline settings, and background color. Optional parameters control scaling, SH evaluation, and color overrides. ```python from gaussian_renderer import render import torch # render() signature out = render( viewpoint_camera, pc, pipe, bg_color, scaling_modifier=1.0, separate_sh=False, override_color=None, use_trained_exp=False, ) # Return dict keys: # out["render"] — [3, H, W] rendered image clamped to [0, 1] # out["viewspace_points"] — [N, 3] screen-space means with gradients attached # out["visibility_filter"] — indices of Gaussians with radius > 0 # out["radii"] — [N] screen-space radii in pixels # out["depth"] — [1, H, W] depth image # Example: custom camera with white background from scene.cameras import MiniCam import math cam = MiniCam( width=1920, height=1080, fovy=math.radians(60), fovx=math.radians(90), znear=0.01, zfar=100.0, world_view_transform=view_matrix.cuda(), full_proj_transform=proj_matrix.cuda() ) bg = torch.ones(3, device="cuda") out = render(cam, gaussians, pipe, bg, scaling_modifier=0.5) ``` -------------------------------- ### Spherical Harmonics Evaluation and Conversion in Python Source: https://context7.com/graphdeco-inria/gaussian-splatting/llms.txt Demonstrates evaluating view-dependent color from Spherical Harmonics (SH) coefficients and converting between linear RGB and SH coefficients. Supports SH up to degree 4. ```python from utils.sh_utils import eval_sh, RGB2SH, SH2RGB import torch # Evaluate view-dependent color from SH coefficients N = 10000 deg = 3 sh_coeffs = torch.randn(N, 3, (deg + 1) ** 2, device="cuda") # [N, 3, 16] view_dirs = torch.nn.functional.normalize( torch.randn(N, 3, device="cuda"), dim=-1 ) # [N, 3] # eval_sh expects [..., C, num_coeffs] and [..., 3] direction rgb = eval_sh(deg, sh_coeffs.transpose(1, 2), view_dirs) # [N, 3] rgb = torch.clamp_min(rgb + 0.5, 0.0) # shift + clamp # Initialize SH DC coefficient from an RGB value rgb_init = torch.tensor([[0.8, 0.5, 0.2]]) # linear RGB sh_dc = RGB2SH(rgb_init) # tensor([[ 1.0611, 0.0000, -0.8489]]) # Recover RGB from DC coefficient rgb_back = SH2RGB(sh_dc) # tensor([[0.8000, 0.5000, 0.2000]]) ``` -------------------------------- ### Force High-Resolution Training Source: https://github.com/graphdeco-inria/gaussian-splatting/blob/main/README.md To ensure training uses original, higher-resolution images, set the resolution flag to '1'. This overrides the default behavior of resizing images wider than 1.6K pixels. ```shell python train.py -s -r 1 ``` -------------------------------- ### Calculate PSNR in Python Source: https://context7.com/graphdeco-inria/gaussian-splatting/llms.txt Shows how to compute the Peak Signal-to-Noise Ratio (PSNR) between rendered images and ground truth. The result is a tensor of PSNR values in decibels. ```python from utils.image_utils import psnr import torch rendered = torch.rand(1, 3, 1080, 1920, device="cuda") # [B, C, H, W] gt = torch.rand(1, 3, 1080, 1920, device="cuda") psnr_val = psnr(rendered, gt) # [B, 1] tensor, dB print(psnr_val.mean().item()) # e.g., 28.73 ``` -------------------------------- ### Set Rendering Resolution and Aspect Ratio Source: https://github.com/graphdeco-inria/gaussian-splatting/blob/main/README.md Configures the rendering resolution and enforces a specific aspect ratio. The default width is 1200. ```shell .//bin/SIBR_gaussianViewer_app --rendering-size --force-aspect-ratio ``` -------------------------------- ### render() — Core Differentiable Gaussian Rasterizer Source: https://context7.com/graphdeco-inria/gaussian-splatting/llms.txt Splats all 3D Gaussians onto the image plane through the CUDA rasterizer, returning the rendered image, screen-space points, per-Gaussian radii, and a depth map. ```APIDOC ## `render()` — Core Differentiable Gaussian Rasterizer Splats all 3D Gaussians onto the image plane through the CUDA rasterizer, returning the rendered image, screen-space points (for gradient-based densification), per-Gaussian radii, and a depth map. ### Signature ```python out = render( viewpoint_camera, # Camera object with FoVx/y, image_height/width, transforms pc, # GaussianModel instance pipe, # PipelineParams (debug, antialiasing, convert_SHs_python, …) bg_color, # torch.Tensor([R, G, B], device="cuda") scaling_modifier=1.0, # multiplies all Gaussian scales (viewer zoom) separate_sh=False, # True when using SparseGaussianAdam override_color=None, # precomputed [N,3] colors, bypasses SH evaluation use_trained_exp=False # apply per-image affine exposure correction ) ``` ### Return Values - `out["render"]` — [3, H, W] rendered image clamped to [0, 1] - `out["viewspace_points"]` — [N, 3] screen-space means with gradients attached - `out["visibility_filter"]` — indices of Gaussians with radius > 0 - `out["radii"]` — [N] screen-space radii in pixels - `out["depth"]` — [1, H, W] depth image ### Example ```python from gaussian_renderer import render from scene.cameras import MiniCam import torch import math # Assuming viewpoint_camera, pc, pipe, view_matrix, proj_matrix are defined cam = MiniCam( width=1920, height=1080, fovy=math.radians(60), fovx=math.radians(90), znear=0.01, zfar=100.0, world_view_transform=view_matrix.cuda(), full_proj_transform=proj_matrix.cuda() ) bg = torch.ones(3, device="cuda") out = render(cam, pc, pipe, bg, scaling_modifier=0.5) ``` ``` -------------------------------- ### Evaluate Trained Models with `metrics.py` Source: https://context7.com/graphdeco-inria/gaussian-splatting/llms.txt The `evaluate()` function in `metrics.py` computes SSIM, PSNR, and LPIPS metrics by comparing rendered images against ground truth. It outputs results to `results.json` and `per_view.json` for each specified model path. ```bash # Compute metrics on one or more trained models python metrics.py -m /output/garden /output/bicycle /output/room # Output per model: /output/garden/results.json # { # "ours_30000": { # "SSIM": 0.8312, # "PSNR": 27.45, # "LPIPS": 0.1823 # } # } # Programmatic usage from metrics import evaluate evaluate(model_paths=["/output/garden", "/output/bicycle"]) ``` -------------------------------- ### Evaluate Pre-trained Models Source: https://github.com/graphdeco-inria/gaussian-splatting/blob/main/README.md Compute error metrics for pre-trained models. The source dataset path is not required if only computing metrics. ```shell python metrics.py -m ``` -------------------------------- ### Enable Depth Regularization during Training Source: https://github.com/graphdeco-inria/gaussian-splatting/blob/main/README.md Activate depth regularization when training a scene by specifying the path to your depth maps using the -d flag. ```bash -d ``` -------------------------------- ### Specify Conda Package and Environment Paths Source: https://github.com/graphdeco-inria/gaussian-splatting/blob/main/README.md Customize Conda's package and environment directories to avoid using the main system hard drive. This is useful for managing disk space. ```shell conda config --add pkgs_dirs / conda env create --file environment.yml --prefix //gaussian_splatting conda activate //gaussian_splatting ``` -------------------------------- ### Align Monocular Depth Maps with `make_depth_scale.py` Source: https://context7.com/graphdeco-inria/gaussian-splatting/llms.txt This script aligns monocular inverse-depth maps to COLMAP's metric depth. It computes per-image scale and offset, saving them to `depth_params.json`, which is used by the trainer when the `-d` argument is provided. The `get_scales` function offers programmatic access to this alignment logic. ```bash # Generate depth_params.json for a COLMAP scene python utils/make_depth_scale.py \ --base_dir /data/garden \ --depths_dir /data/garden/depths \ --model_type bin # Output: /data/garden/sparse/0/depth_params.json # { # "DSC_0001": {"scale": 0.42, "offset": -0.07}, # "DSC_0002": {"scale": 0.39, "offset": -0.05}, # ... # } # Programmatic access to the alignment logic: from utils.make_depth_scale import get_scales result = get_scales( key=image_key, cameras=cam_intrinsics, images=images_metas, points3d_ordered=pts_array, # shape [max_id+1, 3] args=args ) # result: {"image_name": "DSC_0001", "scale": 0.42, "offset": -0.07} # or None if insufficient valid points ``` -------------------------------- ### Clone Gaussian Splatting Repository Source: https://github.com/graphdeco-inria/gaussian-splatting/blob/main/README.md Clone the repository including submodules using either SSH or HTTPS. ```shell # SSH git clone git@github.com:graphdeco-inria/gaussian-splatting.git --recursive ``` ```shell # HTTPS git clone https://github.com/graphdeco-inria/gaussian-splatting --recursive ``` -------------------------------- ### Checkout Fossa Compatibility Branch Source: https://github.com/graphdeco-inria/gaussian-splatting/blob/main/README.md Checks out the 'fossa_compatibility' branch for building SIBR on Ubuntu 20.04. This is for backwards compatibility and may not be fully tested. ```shell git checkout fossa_compatibility ``` -------------------------------- ### Enable Anti-aliasing during Training or Rendering Source: https://github.com/graphdeco-inria/gaussian-splatting/blob/main/README.md Include the --antialiasing flag when training or rendering a scene to apply the EWA Filter for aliasing removal. This feature is disabled by default. ```bash --antialiasing ``` -------------------------------- ### Convert Raw Images to COLMAP Format Source: https://context7.com/graphdeco-inria/gaussian-splatting/llms.txt Use `convert.py` to process raw images into the `sparse/0/` structure required by `Scene`. This script automates COLMAP's feature extraction, matching, bundle adjustment, and undistortion. Ensure COLMAP and ImageMagick are in your PATH or specify their locations. ```bash # Full pipeline: raw images → COLMAP-ready dataset # Requires COLMAP and (optionally) ImageMagick on PATH python convert.py \ -s /data/my_scene \ --camera OPENCV \ --resize # GPU-less machines python convert.py -s /data/my_scene --no_gpu # Skip feature extraction/matching (COLMAP data already in distorted/) python convert.py -s /data/my_scene --skip_matching --resize # Custom COLMAP binary path (Windows .bat wrapper) python convert.py -s /data/my_scene \ --colmap_executable "C:/COLMAP/COLMAP.bat" \ --magick_executable "C:/ImageMagick/magick.exe" # Expected output structure after conversion: # /data/my_scene/ # images/ ← undistorted full-resolution images # images_2/ ← 50 % (if --resize) # images_4/ ← 25 % (if --resize) # images_8/ ← 12.5 % (if --resize) # sparse/0/ # cameras.bin # images.bin # points3D.bin ``` -------------------------------- ### GaussianModel — Learnable 3D Gaussian Representation Source: https://context7.com/graphdeco-inria/gaussian-splatting/llms.txt Stores and manages all optimizable Gaussian parameters, exposing property accessors with activation functions. ```APIDOC ## `GaussianModel` — Learnable 3D Gaussian Representation Stores all optimizable Gaussian parameters as `nn.Parameter` tensors and exposes property accessors with activation functions (exp for scale, sigmoid for opacity, normalize for rotation). ### Instantiation ```python from scene.gaussian_model import GaussianModel gaussians = GaussianModel(sh_degree=3, optimizer_type="default") # optimizer_type: "default" (Adam) | "sparse_adam" (requires 3dgs_accel) ``` ### Key Property Accessors - `xyz` = `gaussians.get_xyz` # [N, 3] 3D positions - `scale` = `gaussians.get_scaling` # [N, 3] exp-activated scales - `rot` = `gaussians.get_rotation` # [N, 4] normalized quaternions - `opac` = `gaussians.get_opacity` # [N, 1] sigmoid-activated opacities - `feats` = `gaussians.get_features` # [N, (sh_degree+1)^2, 3] SH color coefficients - `cov3d` = `gaussians.get_covariance(scaling_modifier=1.0)` # [N, 6] upper-triangular covariance ### Initialization ```python from scene.gaussian_model import GaussianModel from utils.graphics_utils import BasicPointCloud import numpy as np # Assuming pcd, cam_infos, train_cam_list are defined pcd = BasicPointCloud( points=np.random.rand(10000, 3), colors=np.random.rand(10000, 3), normals=np.zeros((10000, 3)) ) gaussians.create_from_pcd(pcd, cam_infos=train_cam_list, spatial_lr_scale=2.4) ``` ### Optimizer Setup ```python from arguments import OptimizationParams from argparse import ArgumentParser # Assuming parser is set up parser = ArgumentParser() opt = OptimizationParams(parser).extract(parser.parse_args([])) gaussians.training_setup(opt) ``` ### Learning Rate Scheduling ```python # Per-iteration learning rate scheduling gaussians.update_learning_rate(iteration=5000) ``` ### SH Degree Promotion ```python # Promote SH degree every 1000 iterations (starts at 0, grows to sh_degree) gaussians.oneupSHdegree() ``` ### Adaptive Density Control ```python # Assuming viewspace_point_tensor, visibility_filter, radii, scene.cameras_extent are defined gaussians.add_densification_stats(viewspace_point_tensor, visibility_filter) if iteration % 100 == 0: gaussians.densify_and_prune( max_grad=0.0002, min_opacity=0.005, extent=scene.cameras_extent, max_screen_size=20, # None before opacity_reset_interval radii=radii ) gaussians.reset_opacity() # periodically reset all opacities to 0.01 ``` ### Checkpoint Save/Restore ```python import torch # Save torch.save((gaussians.capture(), iteration), "chkpnt30000.pth") # Restore model_args, iter = torch.load("chkpnt30000.pth") gaussians.restore(model_args, opt) ``` ### PLY Serialization ```python # Save gaussians.save_ply("/output/garden/point_cloud/iteration_30000/point_cloud.ply") # Load gaussians.load_ply("/output/garden/point_cloud/iteration_30000/point_cloud.ply") ``` ``` -------------------------------- ### Train with Evaluation Split Source: https://github.com/graphdeco-inria/gaussian-splatting/blob/main/README.md Use the `--eval` flag during training to withhold a test set for evaluation. This enables rendering of training and test sets and computation of error metrics. ```shell python train.py -s --eval # Train with train/test split ``` -------------------------------- ### Generate Depth Parameters JSON Source: https://github.com/graphdeco-inria/gaussian-splatting/blob/main/README.md Create a depth_params.json file required for depth regularization. This script uses the base directory of your COLMAP data and the directory where depth maps were generated. ```bash python utils/make_depth_scale.py --base_dir --depths_dir ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.