### GET /config/app Source: https://context7.com/niessner/bundlefusion/llms.txt Retrieves or initializes the global application state for depth sensing and reconstruction parameters. ```APIDOC ## GET /config/app ### Description Accesses the GlobalAppState singleton to manage reconstruction parameters such as voxel size, truncation, and sensor-specific depth thresholds. ### Method GET ### Endpoint /config/app ### Parameters #### Query Parameters - **config_file** (string) - Optional - Path to the parameter file (e.g., zParametersDefault.txt). ### Request Example { "config_file": "zParametersDefault.txt" } ### Response #### Success Response (200) - **s_SDFVoxelSize** (float) - Voxel size in meters. - **s_SDFTruncation** (float) - SDF truncation distance in meters. - **s_sensorIdx** (int) - Identifier for the active depth sensor. #### Response Example { "s_SDFVoxelSize": 0.010, "s_SDFTruncation": 0.06, "s_sensorIdx": 0 } ``` -------------------------------- ### GET /config/bundling Source: https://context7.com/niessner/bundlefusion/llms.txt Retrieves or initializes the global bundling state for feature extraction and pose graph optimization. ```APIDOC ## GET /config/bundling ### Description Accesses the GlobalBundlingState singleton to configure SIFT feature matching thresholds and bundle adjustment optimization iterations. ### Method GET ### Endpoint /config/bundling ### Parameters #### Query Parameters - **config_file** (string) - Optional - Path to the bundling parameter file (e.g., zParametersBundlingDefault.txt). ### Request Example { "config_file": "zParametersBundlingDefault.txt" } ### Response #### Success Response (200) - **s_siftMatchThresh** (float) - Threshold for SIFT feature matching. - **s_numGlobalNonLinIterations** (int) - Number of global non-linear optimization iterations. - **s_submapSize** (int) - Number of frames per submap. #### Response Example { "s_siftMatchThresh": 0.7, "s_numGlobalNonLinIterations": 3, "s_submapSize": 10 } ``` -------------------------------- ### RGBDSensor - Depth Sensor Interface API Source: https://context7.com/niessner/bundlefusion/llms.txt This section outlines the abstract interface for RGB-D depth sensors and provides examples for common implementations like Kinect, Structure Sensor, and file readers. ```APIDOC ## RGBDSensor - Depth Sensor Interface ### Description Defines the interface for RGB-D depth sensors. Implementations include Kinect, Kinect One, Structure Sensor, PrimeSense, and file readers. ### Method N/A (Class Usage) ### Endpoint N/A (Class Usage) ### Parameters N/A (Class Usage) ### Request Example ```cpp #include "RGBDSensor.h" #include "StructureSensor.h" #include "BinaryDumpReader.h" #include "SensorDataReader.h" // Create sensor based on configuration RGBDSensor* sensor = nullptr; unsigned int sensorIdx = GlobalAppState::get().s_sensorIdx; switch (sensorIdx) { case 3: // Binary dump reader (legacy format) sensor = new BinaryDumpReader(); break; case 7: // Structure Sensor (iOS) sensor = new StructureSensor(); break; case 8: // SensorData reader (.sens files) sensor = new SensorDataReader(); break; // ... other sensors } // Initialize sensor connection sensor->createFirstConnected(); // Main acquisition loop while (running) { // Process depth and color data bool hasDepth = sensor->processDepth(); bool hasColor = sensor->processColor(); if (hasDepth && hasColor) { // Access raw sensor data float* depthData = sensor->getDepthFloat(); vec4uc* colorData = sensor->getColorRGBX(); unsigned int depthWidth = sensor->getDepthWidth(); unsigned int depthHeight = sensor->getDepthHeight(); unsigned int colorWidth = sensor->getColorWidth(); unsigned int colorHeight = sensor->getColorHeight(); // Get camera intrinsics const mat4f& depthIntrinsics = sensor->getDepthIntrinsics(); const mat4f& colorIntrinsics = sensor->getColorIntrinsics(); const mat4f& depthExtrinsics = sensor->getDepthExtrinsics(); } } // Recording functionality if (GlobalAppState::get().s_recordData) { sensor->recordFrame(); // Record current frame } // Save recorded frames to file std::vector trajectory = ...; // Camera trajectory sensor->saveRecordedFramesToFile("output/recording.sens", trajectory, true); // Save point cloud sensor->savePointCloud("output/pointcloud.ply", mat4f::identity()); // Reset sensor state sensor->reset(); // Cleanup delete sensor; ``` ### Response N/A (Class Usage) ### Response Example N/A (Class Usage) ``` -------------------------------- ### Bundler Data Access and Validation Source: https://context7.com/niessner/bundlefusion/llms.txt Provides methods to access the optimized trajectory on the GPU, retrieve the number of frames, get a list of valid images, and check the overall validity of the bundler's data. ```APIDOC ## Bundler Data Access and Validation ### Description Provides access to the results of the bundle adjustment, including the optimized camera trajectory, frame counts, and validation status. ### Method `getTrajectoryGPU`, `getNumFrames`, `getValidImages`, `isValid` methods. ### Endpoint N/A (C++ class) ### Parameters None for these getter methods. ### Request Example ```cpp // Access optimized trajectory on GPU const float4x4* d_trajectory = localBundler.getTrajectoryGPU(); unsigned int numFrames = localBundler.getNumFrames(); // Get list of valid (non-invalidated) images const std::vector& validImages = localBundler.getValidImages(); // Check if bundler has valid data bool isValid = localBundler.isValid(); ``` ### Response #### Success Response (200) - **d_trajectory** (const float4x4*) - Pointer to the GPU array containing the optimized camera poses (trajectory). - **numFrames** (unsigned int) - The total number of frames processed. - **validImages** (const std::vector&) - A vector of integers representing the indices of valid (non-invalidated) images. - **isValid** (bool) - True if the bundler contains valid data, false otherwise. #### Response Example ```cpp // d_trajectory points to the optimized poses. // numFrames indicates the count of processed frames. // validImages contains indices like [0, 1, 3, 5] if frames 2 and 4 were invalidated. // isValid will be true if optimization was successful and data is consistent. ``` ``` -------------------------------- ### Initialize GlobalAppState Configuration Source: https://context7.com/niessner/bundlefusion/llms.txt Demonstrates how to access the GlobalAppState singleton and load reconstruction parameters from a configuration file. This allows for dynamic adjustment of voxel hashing, SDF integration, and sensor-specific settings. ```cpp #include "GlobalAppState.h" // Access global configuration singleton GlobalAppState& state = GlobalAppState::get(); // Read parameters from a configuration file ParameterFile paramFile("zParametersDefault.txt"); state.readMembers(paramFile); // Access reconstruction parameters float voxelSize = state.s_SDFVoxelSize; // e.g., 0.010f meters float truncation = state.s_SDFTruncation; // e.g., 0.06f meters float maxIntegrationDist = state.s_SDFMaxIntegrationDistance; // e.g., 3.0f meters unsigned int hashBuckets = state.s_hashNumBuckets; // e.g., 800000 unsigned int sdfBlocks = state.s_hashNumSDFBlocks; // e.g., 200000 // Sensor configuration unsigned int sensorIdx = state.s_sensorIdx; // 0=Kinect, 7=StructureSensor, 8=SensorDataReader float depthMin = state.s_sensorDepthMin; // 0.1f meters float depthMax = state.s_sensorDepthMax; // 4.0f meters // Integration resolution unsigned int integrationWidth = state.s_integrationWidth; // e.g., 320 unsigned int integrationHeight = state.s_integrationHeight; // e.g., 240 // Print all current settings state.print(); ``` -------------------------------- ### Initialize and Execute OnlineBundler Pipeline Source: https://context7.com/niessner/bundlefusion/llms.txt This snippet demonstrates how to initialize the OnlineBundler with sensor and image manager instances, run the main reconstruction loop, and perform bundle adjustment. It covers processing input frames, executing optimization iterations, retrieving camera transforms for integration, and managing the bundling thread lifecycle. ```cpp #include "OnlineBundler.h" const RGBDSensor* sensor = ...; const CUDAImageManager* imageManager = ...; OnlineBundler bundler(sensor, imageManager); while (running) { bundler.processInput(); unsigned int localNonLinIters = GlobalBundlingState::get().s_numLocalNonLinIterations; unsigned int localLinIters = GlobalBundlingState::get().s_numLocalLinIterations; unsigned int globalNonLinIters = GlobalBundlingState::get().s_numGlobalNonLinIterations; unsigned int globalLinIters = GlobalBundlingState::get().s_numGlobalLinIterations; bundler.process(localNonLinIters, localLinIters, globalNonLinIters, globalLinIters); mat4f siftTransform; unsigned int frameIdx; bool globalTrackingLost; if (bundler.getCurrentIntegrationFrame(siftTransform, frameIdx, globalTrackingLost)) { if (!globalTrackingLost) { sceneRep.integrate(siftTransform, depthCameraData, depthCameraParams, d_bitMask); } } bool processed = bundler.hasProcssedInputFrame(); bundler.confirmProcessedInputFrame(); unsigned int currFrame = bundler.getCurrProcessedFrame(); } TrajectoryManager* trajManager = bundler.getTrajectoryManager(); bundler.saveGlobalSparseCorrsToFile("output/global_corrs.bin"); bundler.exitBundlingThread(); ``` -------------------------------- ### Initialize GlobalBundlingState Configuration Source: https://context7.com/niessner/bundlefusion/llms.txt Shows how to configure the bundling pipeline using the GlobalBundlingState singleton. It covers SIFT feature extraction parameters, matching thresholds, and pose graph optimization iteration counts. ```cpp #include "GlobalBundlingState.h" // Access bundling configuration singleton GlobalBundlingState& bundlingState = GlobalBundlingState::get(); // Read bundling parameters from configuration file ParameterFile bundlingParams("zParametersBundlingDefault.txt"); bundlingState.readMembers(bundlingParams); // SIFT feature extraction settings unsigned int siftWidth = bundlingState.s_widthSIFT; // e.g., 640 unsigned int siftHeight = bundlingState.s_heightSIFT; // e.g., 480 unsigned int maxKeysPerImage = bundlingState.s_maxNumKeysPerImage; // e.g., 1024 float minKeyScale = bundlingState.s_minKeyScale; // e.g., 3.0f // Feature matching thresholds float siftMatchThresh = bundlingState.s_siftMatchThresh; // e.g., 0.7f float matchRatioLocal = bundlingState.s_siftMatchRatioMaxLocal; // e.g., 0.8f float matchRatioGlobal = bundlingState.s_siftMatchRatioMaxGlobal; // e.g., 0.8f unsigned int minMatchesLocal = bundlingState.s_minNumMatchesLocal; // e.g., 5 unsigned int minMatchesGlobal = bundlingState.s_minNumMatchesGlobal; // e.g., 5 // Optimization iterations unsigned int localNonLinIters = bundlingState.s_numLocalNonLinIterations; // e.g., 2 unsigned int localLinIters = bundlingState.s_numLocalLinIterations; // e.g., 100 unsigned int globalNonLinIters = bundlingState.s_numGlobalNonLinIterations; // e.g., 3 unsigned int globalLinIters = bundlingState.s_numGlobalLinIterations; // e.g., 150 // Submap configuration unsigned int submapSize = bundlingState.s_submapSize; // e.g., 10 frames unsigned int maxImages = bundlingState.s_maxNumImages; // e.g., 1200 ``` -------------------------------- ### Initialize and Optimize Bundler in BundleFusion Source: https://context7.com/niessner/bundlefusion/llms.txt This snippet demonstrates how to instantiate a Bundler, detect SIFT features, perform feature matching, and execute bundle adjustment optimization. It also covers utility functions for frame management, trajectory retrieval, and data fusion. ```cpp #include "Bundler.h" unsigned int maxNumImages = GlobalBundlingState::get().s_maxNumImages; unsigned int maxKeysPerImage = GlobalBundlingState::get().s_maxNumKeysPerImage; mat4f siftIntrinsicsInv = ...; bool isLocal = true; Bundler localBundler(maxNumImages, maxKeysPerImage, siftIntrinsicsInv, imageManager, isLocal); float* d_intensitySift = ...; const float* d_inputDepthFilt = ...; localBundler.detectFeatures(d_intensitySift, d_inputDepthFilt); unsigned int depthWidth = imageManager->getSIFTDepthWidth(); unsigned int depthHeight = imageManager->getSIFTDepthHeight(); localBundler.storeCachedFrame(depthWidth, depthHeight, d_inputColor, colorWidth, colorHeight, d_inputDepthRaw); unsigned int numMatches = localBundler.matchAndFilter(); unsigned int numNonLinIters = 2; unsigned int numLinIters = 100; bool useVerify = true; bool removeMaxResidual = true; bool isScanDone = false; bool optRemoved = false; bool success = localBundler.optimize(numNonLinIters, numLinIters, useVerify, removeMaxResidual, isScanDone, optRemoved); const float4x4* d_trajectory = localBundler.getTrajectoryGPU(); unsigned int numFrames = localBundler.getNumFrames(); const std::vector& validImages = localBundler.getValidImages(); localBundler.fuseToGlobal(globalBundler); std::vector sparseWeights = {1.0f, 1.0f, 1.0f}; std::vector denseDepthWeights = {0.5f, 0.5f, 0.5f}; std::vector denseColorWeights = {0.1f, 0.1f, 0.1f}; localBundler.setSolveWeights(sparseWeights, denseDepthWeights, denseColorWeights); localBundler.saveSparseCorrsToFile("output/local_corrs.txt"); ``` -------------------------------- ### Interfacing with RGB-D Sensors Source: https://context7.com/niessner/bundlefusion/llms.txt Shows how to instantiate specific RGBDSensor implementations, process sensor data streams, and handle recording or saving point cloud data. ```cpp #include "RGBDSensor.h" #include "StructureSensor.h" #include "BinaryDumpReader.h" #include "SensorDataReader.h" RGBDSensor* sensor = new SensorDataReader(); sensor->createFirstConnected(); while (running) { if (sensor->processDepth() && sensor->processColor()) { float* depthData = sensor->getDepthFloat(); vec4uc* colorData = sensor->getColorRGBX(); } } sensor->savePointCloud("output/pointcloud.ply", mat4f::identity()); sensor->reset(); delete sensor; ``` -------------------------------- ### Bundler Class Initialization and Feature Detection Source: https://context7.com/niessner/bundlefusion/llms.txt This snippet demonstrates how to initialize the Bundler for local or global optimization and how to detect SIFT features in an image. ```APIDOC ## Bundler Initialization and Feature Detection ### Description Initializes the Bundler for local or global optimization and performs SIFT feature detection on an input image. ### Method Constructor and `detectFeatures` method. ### Endpoint N/A (C++ class) ### Parameters #### Constructor Parameters - **maxNumImages** (unsigned int) - Required - Maximum number of images to consider. - **maxKeysPerImage** (unsigned int) - Required - Maximum number of SIFT keys per image. - **siftIntrinsicsInv** (mat4f) - Required - Inverse SIFT camera intrinsics. - **imageManager** (ImageManager*) - Required - Pointer to the ImageManager. - **isLocal** (bool) - Required - Flag to indicate if it's for local submap optimization. #### `detectFeatures` Parameters - **d_intensitySift** (float*) - Required - GPU intensity image for SIFT. - **d_inputDepthFilt** (const float*) - Required - Filtered depth map for feature depth lookup. ### Request Example ```cpp unsigned int maxNumImages = GlobalBundlingState::get().s_maxNumImages; unsigned int maxKeysPerImage = GlobalBundlingState::get().s_maxNumKeysPerImage; mat4f siftIntrinsicsInv = ...; // Inverse SIFT camera intrinsics bool isLocal = true; Bundler localBundler(maxNumImages, maxKeysPerImage, siftIntrinsicsInv, imageManager, isLocal); float* d_intensitySift = ...; // GPU intensity image for SIFT const float* d_inputDepthFilt = ...; // Filtered depth for feature depth lookup localBundler.detectFeatures(d_intensitySift, d_inputDepthFilt); ``` ### Response #### Success Response (200) Feature detection is performed in-place. No direct return value for detection success, but subsequent methods will reflect the detected features. #### Response Example N/A ``` -------------------------------- ### Configuration Parameters Source: https://context7.com/niessner/bundlefusion/llms.txt Reference for the configuration files used to control sensor input, SDF voxel hashing, and bundle adjustment behavior. ```APIDOC ## Configuration Parameters ### Description BundleFusion uses two primary configuration files: zParametersDefault.txt for reconstruction and zParametersBundlingDefault.txt for optimization. ### Parameters #### Reconstruction (zParametersDefault.txt) - **s_sensorIdx** (int) - Sensor type identifier. - **s_SDFVoxelSize** (float) - Voxel resolution in meters. - **s_hashNumBuckets** (int) - Hash table size for SDF blocks. #### Bundling (zParametersBundlingDefault.txt) - **s_maxNumKeysPerImage** (int) - SIFT feature extraction limit. - **s_numLocalNonLinIterations** (int) - Iterations for local bundle adjustment. - **s_denseDistThresh** (float) - Distance threshold for dense alignment. ### Usage These parameters are loaded at runtime to initialize the reconstruction engine and optimize the camera trajectory. ``` -------------------------------- ### Bundler Fusion and Frame Management Source: https://context7.com/niessner/bundlefusion/llms.txt Demonstrates how to fuse results from a local bundler to a global bundler, and how to manage frame validity (adding invalid frames, invalidating frames, and attempting revalidation). ```APIDOC ## Bundler Fusion and Frame Management ### Description Enables merging the results of a local bundle adjustment into a global bundler, and provides utilities for marking frames as invalid, adding them, and attempting to revalidate previously invalidated frames. ### Method `fuseToGlobal`, `addInvalidFrame`, `invalidateLastFrame`, `tryRevalidation` methods. ### Endpoint N/A (C++ class) ### Parameters #### `fuseToGlobal` Parameters - **globalBundler** (Bundler*) - Required - Pointer to the global Bundler instance to fuse into. #### `addInvalidFrame` Parameters None #### `invalidateLastFrame` Parameters None #### `tryRevalidation` Parameters - **curGlobalFrame** (unsigned int) - Required - The current global frame index. - **isScanDone** (bool) - Required - Flag indicating if the scan is complete. ### Request Example ```cpp // Fuse local bundler results to global bundler Bundler* globalBundler = ...; localBundler.fuseToGlobal(globalBundler); // Handle invalid frames localBundler.addInvalidFrame(); localBundler.invalidateLastFrame(); // Try to revalidate previously invalidated frames unsigned int curGlobalFrame = ...; unsigned int revalidatedIdx = localBundler.tryRevalidation(curGlobalFrame, isScanDone); ``` ### Response #### Success Response (200) - **revalidatedIdx** (unsigned int) - The index of the revalidated frame, or a special value (e.g., -1) if no frame was revalidated. #### Response Example ```cpp // After calling tryRevalidation, revalidatedIdx will hold the index of a frame that was successfully revalidated, or an indicator that none were. ``` ``` -------------------------------- ### SBA: GPU-Accelerated Sparse Bundle Adjustment Source: https://context7.com/niessner/bundlefusion/llms.txt This C++ code demonstrates the initialization, configuration, and execution of the SBA class for sparse bundle adjustment. It supports both sparse SIFT correspondences and dense photometric/geometric alignment, leveraging CUDA solvers for performance. The snippet covers setting optimization weights, running the alignment process with various parameters, and accessing optimization statistics like convergence and residual analysis. It also includes saving logs for removed correspondences and evaluating solver timings. ```cpp #include "SBA.h" // Create and initialize SBA optimizer SBA optimizer; unsigned int maxImages = GlobalBundlingState::get().s_maxNumImages; unsigned int maxResiduals = maxImages * maxImages * MAX_MATCHES_PER_IMAGE_PAIR_FILTERED; optimizer.init(maxImages, maxResiduals); // Configure optimization weights std::vector sparseWeights = {1.0f, 0.5f, 0.25f}; // Per iteration std::vector denseDepthWeights = {0.5f, 0.25f, 0.1f}; // Per iteration std::vector denseColorWeights = {0.1f, 0.05f, 0.02f}; // Per iteration bool useGlobalDenseOpt = true; optimizer.setGlobalWeights(sparseWeights, denseDepthWeights, denseColorWeights, useGlobalDenseOpt); // Run alignment SIFTImageManager* siftManager = ...; // Feature manager const CUDACache* cudaCache = ...; // Dense image cache float4x4* d_transforms = ...; // GPU transform array (output) unsigned int maxIters = 3; // Nonlinear iterations unsigned int pcgIters = 150; // PCG linear solver iterations bool useVerify = true; // Verify alignment quality bool isLocal = false; // Global optimization bool recordConvergence = true; bool isStart = true; // First optimization bool isEnd = false; // Last optimization bool isScanDone = false; bool removedResidual = optimizer.align( siftManager, cudaCache, d_transforms, maxIters, pcgIters, useVerify, isLocal, recordConvergence, isStart, isEnd, isScanDone ); // Get optimization statistics float maxResidual = optimizer.getMaxResidual(); bool usedVerification = optimizer.useVerification(); // Access convergence data const std::vector& convergence = optimizer.getLinearConvergenceAnalysis(); // Print convergence to file optimizer.printConvergence("output/convergence.txt"); // Evaluate timing statistics optimizer.evaluateSolverTimings(); // Save removed correspondences log optimizer.saveLogRemovedCorrToFile("output/removed_corrs"); ``` -------------------------------- ### Bundler Optimization Weights and Reset Source: https://context7.com/niessner/bundlefusion/llms.txt Details how to set custom weights for different components (sparse, dense depth, dense color) in hybrid optimization and how to reset the bundler for a new sequence. ```APIDOC ## Bundler Optimization Weights and Reset ### Description Allows customization of optimization weights for dense-sparse hybrid optimization and provides a method to reset the bundler's internal state for processing a new sequence of images. ### Method `setSolveWeights` and `reset` methods. ### Endpoint N/A (C++ class) ### Parameters #### `setSolveWeights` Parameters - **sparseWeights** (std::vector) - Required - Weights for sparse feature optimization. - **denseDepthWeights** (std::vector) - Required - Weights for dense depth alignment. - **denseColorWeights** (std::vector) - Required - Weights for dense color alignment. #### `reset` Parameters None ### Request Example ```cpp // Set optimization weights for dense-sparse hybrid optimization std::vector sparseWeights = {1.0f, 1.0f, 1.0f}; std::vector denseDepthWeights = {0.5f, 0.5f, 0.5f}; std::vector denseColorWeights = {0.1f, 0.1f, 0.1f}; localBundler.setSolveWeights(sparseWeights, denseDepthWeights, denseColorWeights); // Reset for new sequence localBundler.reset(); ``` ### Response #### Success Response (200) These methods typically perform actions and do not return specific data, but indicate success by not throwing exceptions. #### Response Example N/A ``` -------------------------------- ### Manage SIFT Features with SIFTImageManager Source: https://context7.com/niessner/bundlefusion/llms.txt Demonstrates how to initialize the SIFTImageManager, allocate GPU storage for features, perform match filtering, and access global correspondences. This class is essential for managing feature-based alignment in the BundleFusion pipeline. ```cpp #include "SIFTImageManager.h" unsigned int maxImages = 500; unsigned int maxKeysPerImage = 4096; SIFTImageManager siftManager(maxImages, maxKeysPerImage); SIFTImageGPU& siftImage = siftManager.createSIFTImageGPU(); unsigned int numKeyPoints = ...; siftManager.finalizeSIFTImageGPU(numKeyPoints); unsigned int prevIdx = 5; unsigned int curIdx = 10; uint2 keyPointOffset; ImagePairMatch& match = siftManager.getImagePairMatch(prevIdx, curIdx, keyPointOffset); siftManager.FilterKeyPointMatchesCU(currentFrame, 0, numImages, siftIntrinsicsInv, minMatches, maxKabschRes); siftManager.FilterMatchesBySurfaceAreaCU(currentFrame, 0, numImages, colorIntrinsicsInv, areaThresh); siftManager.AddCurrToResidualsCU(currentFrame, 0, numImages, colorIntrinsicsInv); const EntryJ* d_correspondences = siftManager.getGlobalCorrespondencesGPU(); siftManager.saveToFile("output/sift_data.bin"); ``` -------------------------------- ### CUDASceneRepHashSDF: Volumetric Hashing SDF Initialization and Integration Source: https://context7.com/niessner/bundlefusion/llms.txt Initializes and uses the CUDASceneRepHashSDF class for spatial hashing-based volumetric SDF representation. It covers creating hash parameters, initializing the scene, integrating depth frames, de-integrating when poses are updated, and performing garbage collection. ```cpp #include "CUDASceneRepHashSDF.h" // Create hash parameters from global application state HashParams hashParams = CUDASceneRepHashSDF::parametersFromGlobalAppState(GlobalAppState::get()); // Parameters include: // - hashNumBuckets: 800000 // - hashBucketSize: HASH_BUCKET_SIZE (compile-time constant) // - virtualVoxelSize: 0.010f meters // - truncation: 0.06f meters // - maxIntegrationDistance: 3.0f meters // - integrationWeightSample: 1 // - integrationWeightMax: 99999999 // Create the scene representation CUDASceneRepHashSDF sceneRep(hashParams); // Prepare depth camera data and parameters DepthCameraData depthCameraData; depthCameraData.d_depthData = d_depthFloat; // GPU depth buffer depthCameraData.d_colorData = d_colorRGBX; // GPU color buffer DepthCameraParams depthCameraParams; depthCameraParams.m_imageWidth = integrationWidth; depthCameraParams.m_imageHeight = integrationHeight; depthCameraParams.m_intrinsics = depthIntrinsics; depthCameraParams.m_intrinsicsInverse = depthIntrinsicsInv; // Integrate a depth frame with its camera transform mat4f cameraToWorld = trajectory[frameIdx]; sceneRep.integrate(cameraToWorld, depthCameraData, depthCameraParams, d_bitMask); // De-integrate a frame when pose is updated (for re-integration) sceneRep.deIntegrate(oldCameraToWorld, depthCameraData, depthCameraParams, d_bitMask); // Re-integrate with corrected pose sceneRep.integrate(newCameraToWorld, depthCameraData, depthCameraParams, d_bitMask); // Run garbage collection to free unused voxel blocks sceneRep.garbageCollect(); // Reset the entire scene sceneRep.reset(); // Get statistics unsigned int numIntegratedFrames = sceneRep.getNumIntegratedFrames(); unsigned int heapFreeCount = sceneRep.getHeapFreeCount(); ``` -------------------------------- ### Extracting Meshes with CUDAMarchingCubesHashSDF Source: https://context7.com/niessner/bundlefusion/llms.txt Demonstrates how to initialize the marching cubes parameters, extract an isosurface from a HashSDF representation, and export the resulting mesh to a PLY file. ```cpp #include "CUDAMarchingCubesHashSDF.h" MarchingCubesParams mcParams = CUDAMarchingCubesHashSDF::parametersFromGlobalAppState(GlobalAppState::get()); CUDAMarchingCubesHashSDF marchingCubes(mcParams); const HashDataStruct& hashData = sceneRep.getHashData(); const HashParams& hashParams = sceneRep.getHashParams(); const RayCastData& rayCastData = rayCaster.getRayCastData(); marchingCubes.extractIsoSurface(hashData, hashParams, rayCastData); marchingCubes.copyTrianglesToCPU(); marchingCubes.saveMesh("output/reconstruction.ply", &mat4f::identity(), true); marchingCubes.clearMeshBuffer(); ``` -------------------------------- ### Configure Bundle Adjustment Parameters Source: https://context7.com/niessner/bundlefusion/llms.txt Sets parameters for SIFT extraction, feature matching, submap configuration, and optimization iterations used during the bundle adjustment process. ```ini s_widthSIFT = 640; s_heightSIFT = 480; s_maxNumKeysPerImage = 1024; s_siftMatchThresh = 0.7f; s_submapSize = 10; s_numLocalNonLinIterations = 2; s_numGlobalNonLinIterations = 3; s_denseDistThresh = 0.15f; s_depthFilter = true; ``` -------------------------------- ### Configure BundleFusion Data Recording Source: https://github.com/niessner/bundlefusion/blob/master/FriedLiver/zParametersDefault.txt Sets the master flags and file paths for recording sensor data. Includes options for compression and resolution settings for non-compressed output. ```cpp s_recordData = false; s_recordCompression = true; s_recordDataWidth = 640; s_recordDataHeight = 480; s_recordDataFile = "dump/recording.sens"; s_reconstructionEnabled = true; ``` -------------------------------- ### Configure BundleFusion Global Parameters Source: https://github.com/niessner/bundlefusion/blob/master/FriedLiver/zParametersBundlingDefault.txt This C++ header-style configuration defines the core operational constants for the BundleFusion reconstruction engine. It includes settings for SIFT feature detection, dense tracking thresholds, and optimization iteration counts. ```cpp s_verbose = false; s_erodeSIFTdepth = true; s_widthSIFT = 640; s_heightSIFT = 480; s_minKeyScale = 3.0f; s_siftMatchThresh = 0.7f; s_numLocalNonLinIterations = 2; s_numLocalLinIterations = 100; s_denseDepthMin = 0.5f; s_denseDepthMax = 4.0f; s_depthFilter = true; ``` -------------------------------- ### CUDARayCastSDF: Volumetric SDF Raycasting for Visualization Source: https://context7.com/niessner/bundlefusion/llms.txt This C++ code snippet illustrates the usage of the CUDARayCastSDF class for rendering a volumetric Signed Distance Function (SDF) representation. It details how to set up raycasting parameters from global application state and camera intrinsics, create a raycaster instance, and render the SDF to generate depth, normal, and color maps. The code also shows how to access the rendered data, update rendering parameters like depth range and intrinsics, and retrieve the current camera intrinsics used for rendering. ```cpp #include "CUDARayCastSDF.h" // Create raycast parameters from global state mat4f intrinsics = imageManager.getDepthIntrinsics(); mat4f intrinsicsInv = imageManager.getDepthIntrinsicsInv(); RayCastParams rayCastParams = CUDARayCastSDF::parametersFromGlobalAppState( GlobalAppState::get(), intrinsics, intrinsicsInv ); // Create raycaster CUDARayCastSDF rayCaster(rayCastParams); // Render the SDF from current viewpoint mat4f lastRigidTransform = sceneRep.getLastRigidTransform(); rayCaster.render(sceneRep.getHashData(), sceneRep.getHashParams(), lastRigidTransform); // Access rendered data const RayCastData& rayData = rayCaster.getRayCastData(); // rayData.d_depth - rendered depth map // rayData.d_normals - rendered normal map // rayData.d_colors - rendered color map // Update depth range for rendering rayCaster.updateRayCastMinMax(0.5f, 4.0f); // Update intrinsics for different resolution rayCaster.setRayCastIntrinsics(width, height, newIntrinsics, newIntrinsicsInv); // Get current intrinsics mat4f currentIntrinsics = rayCaster.getIntrinsics(); mat4f currentIntrinsicsInv = rayCaster.getIntrinsicsInv(); ``` -------------------------------- ### Define SDF and Hash Table Parameters Source: https://github.com/niessner/bundlefusion/blob/master/FriedLiver/zParametersDefault.txt Configures the voxel size, truncation distances, and hash table capacity for the volumetric representation. These settings are fundamental to the memory usage and spatial resolution of the reconstructed scene. ```cpp s_SDFVoxelSize = 0.010f; s_SDFTruncation = 0.06f; s_hashNumBuckets = 800000; s_hashNumSDFBlocks = 200000; ``` -------------------------------- ### Bundler Optimization Source: https://context7.com/niessner/bundlefusion/llms.txt This section details the bundle adjustment optimization process, including parameters for non-linear and linear iterations, and options for verification and outlier removal. ```APIDOC ## Bundler Optimization ### Description Performs bundle adjustment optimization to refine camera poses and 3D structure. Allows configuration of iteration counts and outlier removal strategies. ### Method `optimize` method. ### Endpoint N/A (C++ class) ### Parameters #### `optimize` Parameters - **numNonLinIters** (unsigned int) - Required - Number of non-linear optimization iterations. - **numLinIters** (unsigned int) - Required - Number of linear optimization iterations. - **useVerify** (bool) - Required - Whether to use verification during optimization. - **removeMaxResidual** (bool) - Required - Whether to remove outliers based on maximum residual. - **isScanDone** (bool) - Required - Flag indicating if the scan is complete. - **optRemoved** (bool) - Output parameter, will be true if outliers were removed. ### Request Example ```cpp unsigned int numNonLinIters = 2; unsigned int numLinIters = 100; bool useVerify = true; bool removeMaxResidual = true; bool isScanDone = false; bool optRemoved = false; bool success = localBundler.optimize(numNonLinIters, numLinIters, useVerify, removeMaxResidual, isScanDone, optRemoved); ``` ### Response #### Success Response (200) - **success** (bool) - True if the optimization completed successfully, false otherwise. - **optRemoved** (bool) - True if outliers were removed during optimization. #### Response Example ```cpp // 'success' will be true if optimization ran without critical errors. // 'optRemoved' indicates if any points were flagged as outliers. ``` ``` -------------------------------- ### Configure BundleFusion Sensor and Integration Settings Source: https://github.com/niessner/bundlefusion/blob/master/FriedLiver/zParametersDefault.txt Sets the active sensor type and defines the resolution for depth integration and raycasting. Adjusting these values allows for balancing between reconstruction performance and fidelity. ```cpp s_sensorIdx = 7; // 7=StructureSensor s_windowWidth = 640; s_windowHeight = 480; s_integrationWidth = 320; s_integrationHeight = 240; s_rayCastWidth = 320; s_rayCastHeight = 240; ``` -------------------------------- ### Configure Reconstruction Parameters Source: https://context7.com/niessner/bundlefusion/llms.txt Defines the main reconstruction parameters including sensor selection, voxel hashing settings, and mesh extraction limits for the BundleFusion system. ```ini s_sensorIdx = 8; s_binaryDumpSensorFile = "../data/sequence.sens"; s_windowWidth = 640; s_windowHeight = 480; s_SDFVoxelSize = 0.010f; s_SDFTruncation = 0.06f; s_hashNumBuckets = 800000; s_hashNumSDFBlocks = 200000; s_marchingCubesMaxNumTriangles = 3000000; ``` -------------------------------- ### Configure BundleFusion Streaming Parameters Source: https://github.com/niessner/bundlefusion/blob/master/FriedLiver/zParametersDefault.txt Defines the streaming radius, position, and frame sweep count for the reconstruction system. These parameters are dependent on the depth range configuration. ```cpp s_streamingRadius = 5.0f; s_streamingPos = 0.0f 0.0f 3.0f 1.0f; s_streamingOutParts = 80; ``` -------------------------------- ### Bundler Frame Caching and Matching Source: https://context7.com/niessner/bundlefusion/llms.txt This snippet shows how to store cached frame data for dense alignment and how to match detected features with previous frames, followed by filtering. ```APIDOC ## Bundler Frame Caching and Matching ### Description Stores cached frame data (depth and color) for dense alignment and performs feature matching between the current frame and previous frames, followed by filtering of matches. ### Method `storeCachedFrame` and `matchAndFilter` methods. ### Endpoint N/A (C++ class) ### Parameters #### `storeCachedFrame` Parameters - **depthWidth** (unsigned int) - Required - Width of the depth image. - **depthHeight** (unsigned int) - Required - Height of the depth image. - **d_inputColor** (const float*) - Required - GPU color image data. - **colorWidth** (unsigned int) - Required - Width of the color image. - **colorHeight** (unsigned int) - Required - Height of the color image. - **d_inputDepthRaw** (const float*) - Required - Raw depth image data. #### `matchAndFilter` Parameters None ### Request Example ```cpp // Store cached frame data for dense alignment unsigned int depthWidth = imageManager->getSIFTDepthWidth(); unsigned int depthHeight = imageManager->getSIFTDepthHeight(); localBundler.storeCachedFrame(depthWidth, depthHeight, d_inputColor, colorWidth, colorHeight, d_inputDepthRaw); // Match features with previous frames and filter matches unsigned int numMatches = localBundler.matchAndFilter(); ``` ### Response #### Success Response (200) - **numMatches** (unsigned int) - The number of valid matches found after filtering. #### Response Example ```cpp // numMatches will contain the count of successfully matched and filtered features. ``` ``` -------------------------------- ### Set Rendering and Material Properties Source: https://github.com/niessner/bundlefusion/blob/master/FriedLiver/zParametersDefault.txt Defines the visual properties of the reconstructed mesh, including lighting, material shininess, and depth discontinuity thresholds for rendering. ```cpp s_materialShininess = 16.0f; s_lightDirection = 0.0f -1.0f 2.0f; s_renderingDepthDiscontinuityThresOffset = 0.012f; ``` -------------------------------- ### CUDAImageManager: RGB-D Frame Processing and Management Source: https://context7.com/niessner/bundlefusion/llms.txt Manages input RGB-D frames, handling filtering, resampling, and GPU memory. It provides a buffered interface for sensor data, allowing access to processed frames on both GPU and CPU, and facilitates synchronization between threads. ```cpp #include "CUDAImageManager.h" // Create image manager with target integration resolution unsigned int integrationWidth = 320; unsigned int integrationHeight = 240; unsigned int siftWidth = 640; unsigned int siftHeight = 480; bool storeFramesOnGPU = true; CUDAImageManager imageManager( integrationWidth, integrationHeight, siftWidth, siftHeight, sensor, // RGBDSensor* - the active depth sensor storeFramesOnGPU // whether to keep frames in GPU memory ); // Initialize D3D11 resources imageManager.OnD3D11CreateDevice(pd3dDevice); // Process next frame from sensor if (imageManager.process()) { // Frame successfully processed unsigned int frameNumber = imageManager.getCurrFrameNumber(); // Access the processed frame data CUDAImageManager::ManagedRGBDInputFrame& frame = imageManager.getLastIntegrateFrame(); // Get depth/color data on GPU const float* d_depth = frame.getDepthFrameGPU(); const uchar4* d_color = frame.getColorFrameGPU(); // Or get data on CPU if needed const float* h_depth = frame.getDepthFrameCPU(); const uchar4* h_color = frame.getColorFrameCPU(); } // Access camera intrinsics (adapted for integration resolution) const mat4f& depthIntrinsics = imageManager.getDepthIntrinsics(); const mat4f& depthIntrinsicsInv = imageManager.getDepthIntrinsicsInv(); const mat4f& depthExtrinsics = imageManager.getDepthExtrinsics(); // Get SIFT processing dimensions and intrinsics unsigned int siftDepthWidth = imageManager.getSIFTDepthWidth(); unsigned int siftDepthHeight = imageManager.getSIFTDepthHeight(); const mat4f& siftIntrinsics = imageManager.getSIFTDepthIntrinsics(); // Copy raw frame data for bundling thread float* d_depthRaw, *d_depthFilt; uchar4* d_color; imageManager.copyToBundling(d_depthRaw, d_depthFilt, d_color); // Synchronization between depth sensing and bundling threads imageManager.setBundlingFrameRdy(); // Signal frame is ready bool hasFrame = imageManager.hasBundlingFrameRdy(); imageManager.confirmRdyBundlingFrame(); // Bundling confirms it read the frame // Reset all buffered frames imageManager.reset(); ``` -------------------------------- ### CUDAMarchingCubesHashSDF - Mesh Extraction API Source: https://context7.com/niessner/bundlefusion/llms.txt This section details the usage of the CUDAMarchingCubesHashSDF class for extracting triangle meshes from volumetric SDF data using the marching cubes algorithm on the GPU. ```APIDOC ## CUDAMarchingCubesHashSDF - Mesh Extraction ### Description Extracts triangle meshes from volumetric SDF data using the marching cubes algorithm on the GPU. ### Method N/A (Class Usage) ### Endpoint N/A (Class Usage) ### Parameters N/A (Class Usage) ### Request Example ```cpp #include "CUDAMarchingCubesHashSDF.h" // Create marching cubes parameters from global state MarchingCubesParams mcParams = CUDAMarchingCubesHashSDF::parametersFromGlobalAppState( GlobalAppState::get() ); // Create marching cubes extractor CUDAMarchingCubesHashSDF marchingCubes(mcParams); // Extract isosurface from entire scene const HashDataStruct& hashData = sceneRep.getHashData(); const HashParams& hashParams = sceneRep.getHashParams(); const RayCastData& rayCastData = rayCaster.getRayCastData(); // Full extraction marchingCubes.extractIsoSurface(hashData, hashParams, rayCastData); // Or extract within a bounding box vec3f minCorner(-2.0f, -1.0f, 0.0f); vec3f maxCorner(2.0f, 2.0f, 5.0f); bool boxEnabled = true; marchingCubes.extractIsoSurface(hashData, hashParams, rayCastData, minCorner, maxCorner, boxEnabled); // Copy extracted triangles to CPU marchingCubes.copyTrianglesToCPU(); // Save mesh to file mat4f worldTransform = mat4f::identity(); bool overwrite = true; marchingCubes.saveMesh("output/reconstruction.ply", &worldTransform, overwrite); // Clear mesh buffer for next extraction marchingCubes.clearMeshBuffer(); ``` ### Response N/A (Class Usage) ### Response Example N/A (Class Usage) ``` -------------------------------- ### Bundler Data Saving Source: https://context7.com/niessner/bundlefusion/llms.txt Provides a method to save the sparse correspondences detected and matched by the bundler to a file for analysis. ```APIDOC ## Bundler Data Saving ### Description Saves the sparse feature correspondences, including matches and their associated data, to a specified text file for offline analysis and debugging. ### Method `saveSparseCorrsToFile` method. ### Endpoint N/A (C++ class) ### Parameters #### `saveSparseCorrsToFile` Parameters - **filename** (const char*) - Required - The path and name of the file to save the correspondences to. ### Request Example ```cpp // Save sparse correspondences for analysis localBundler.saveSparseCorrsToFile("output/local_corrs.txt"); ``` ### Response #### Success Response (200) Indicates successful file write operation. No specific return value, but the file will be created or updated. #### Response Example N/A ``` -------------------------------- ### SIFTImageManager API Source: https://context7.com/niessner/bundlefusion/llms.txt The SIFTImageManager class handles GPU-resident storage for SIFT features, cross-frame matching, and geometric filtering for the BundleFusion pipeline. ```APIDOC ## SIFTImageManager API ### Description Manages GPU memory for SIFT keypoints, descriptors, and image-to-image correspondences. Provides methods for filtering matches and updating global residuals. ### Method C++ Class Interface ### Parameters #### Constructor Parameters - **maxImages** (unsigned int) - Required - Maximum number of frames to store. - **maxKeysPerImage** (unsigned int) - Required - Maximum SIFT keypoints per frame. ### Key Methods - **createSIFTImageGPU()** - Allocates GPU memory for a new frame. - **finalizeSIFTImageGPU(numKeyPoints)** - Finalizes the feature count for a frame. - **FilterKeyPointMatchesCU(...)** - Performs GPU-accelerated Kabsch-based match filtering. - **AddCurrToResidualsCU(...)** - Adds valid correspondences to the global optimization system. - **saveToFile(path)** / **loadFromFile(path)** - Persistence for debugging. ### Response - **d_keyPoints** (GPU Array) - Keypoint coordinates. - **d_keyPointDescs** (GPU Array) - SIFT descriptors. - **d_correspondences** (EntryJ*) - Global correspondence list. ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.