### Install and Run Development Server Source: https://github.com/marcusandreassvensson/gaussian-splatting-webgpu/blob/main/README.md Navigate to the code directory and run these commands to install dependencies and start the development server. The dev server will be accessible at localhost:5173. ```bash npm install npm run dev ``` -------------------------------- ### Development and Build Commands Source: https://context7.com/marcusandreassvensson/gaussian-splatting-webgpu/llms.txt A set of bash commands for managing the project's dependencies, starting the development server, building for production, and previewing the production build. ```bash # Install dependencies (uses pnpm) pnpm install # Start the Vite dev server (hot-reload, available at http://localhost:5173) pnpm dev # Production build (TypeScript check + Vite bundle) pnpm build # Preview the production build locally (http://localhost:4173) pnpm preview ``` -------------------------------- ### Initialize and Use Gaussian Splatting Renderer Source: https://context7.com/marcusandreassvensson/gaussian-splatting-webgpu/llms.txt Sets up the WebGPU renderer, including allocating resources and starting the animation loop. Demonstrates invalidating the renderer for re-renders and clean destruction. ```typescript import { Renderer } from '@/splatting-web/renderer' import { InteractiveCamera } from '@/splatting-web/camera' import { PackedGaussians, loadFileAsArrayBuffer } from '@/splatting-web/ply' import { EngineContext } from '@/radix-sort/EngineContext' const canvas = document.querySelector('#gpu-canvas')! const fpsLabel = document.querySelector('#fps')! const file: File = /* from */ someFile const arrayBuffer = await loadFileAsArrayBuffer(file) const gaussians = new PackedGaussians(arrayBuffer) const gpuContext = await Renderer.requestContext(gaussians) // EngineContext is needed by the radix sorter's bind-group layout cache const engineCtx = new EngineContext() await engineCtx.initialize() const interactiveCamera = InteractiveCamera.default(canvas) // Registers mouse/keyboard event listeners on canvas automatically const renderer = new Renderer(canvas, interactiveCamera, gaussians, gpuContext, fpsLabel) // Animation loop is now running; frames are submitted on camera movement. // Force a re-render (e.g., after adjusting a slider): renderer.invalidate() // Tear down cleanly (waits for the current frame to finish): await renderer.destroy() ``` -------------------------------- ### React Canvas Component Integration Example Source: https://context7.com/marcusandreassvensson/gaussian-splatting-webgpu/llms.txt This is a simplified integration example of the Canvas component within a React application. It demonstrates how to use the Canvas component to render a full-screen WebGPU view. ```tsx import Canvas from '@/components/Canvas' // Full-page usage (as in App.tsx): export default function App() { return (
{/* Canvas fills the viewport; sidebar controls are rendered internally */}
) } ``` -------------------------------- ### Renderer Constructor Source: https://context7.com/marcusandreassvensson/gaussian-splatting-webgpu/llms.txt The main renderer class constructor. It allocates all necessary WebGPU resources, including storage buffers, uniform buffers, textures, and compute/render pipelines. It automatically starts the `requestAnimationFrame` loop and performs dirty-checking for rendering. ```APIDOC ## `new Renderer(canvas, interactiveCamera, gaussians, context, fpsCounter)` The main renderer class. Its constructor allocates all WebGPU resources: point-data storage buffer (uploaded from `gaussians.gaussiansBuffer`), uniform buffer (camera matrices and screen size), an intermediate RGBA8 color texture, the rasterization compute pipeline (`rasterize.wgsl`), the screen blit render pipeline (`screen.wgsl`), and a `Preprocessor` instance. It immediately starts the `requestAnimationFrame` loop. Rendering is dirty-checked — a new frame is only submitted when the camera has moved or `invalidate()` is called. ``` -------------------------------- ### Build and Preview Project Source: https://github.com/marcusandreassvensson/gaussian-splatting-webgpu/blob/main/README.md Navigate to the code directory and run these commands to build the project for production and preview the build locally. The preview will be accessible at localhost:4173. ```bash npm install npm run build npm run preview ``` -------------------------------- ### Initialize WebGPU Context with Memory Limits Source: https://context7.com/marcusandreassvensson/gaussian-splatting-webgpu/llms.txt Requests a WebGPU device with storage buffer and buffer sizes scaled to the Gaussian data. Falls back to a plain device if timestamp query is not supported. ```typescript import { Renderer } from '@/splatting-web/renderer' import { PackedGaussians } from '@/splatting-web/ply' async function initGpu(gaussians: PackedGaussians) { if (!navigator.gpu) { throw new Error('WebGPU is not supported in this browser.') } // Requests a device with maxStorageBufferBindingSize and maxBufferSize // set to 1.5× the Gaussian data size to give the GPU enough headroom. const context = await Renderer.requestContext(gaussians) console.log('Timestamp query available:', context.hasTimestampQuery) // true only when Chrome is launched with --enable-dawn-features=allow_unsafe_apis console.log('Max buffer size (bytes):', context.device.limits.maxBufferSize) return context } ``` -------------------------------- ### Renderer.requestContext Source: https://context7.com/marcusandreassvensson/gaussian-splatting-webgpu/llms.txt A static asynchronous factory method to negotiate a WebGPU adapter and device. It attempts to acquire the `timestamp-query` feature for profiling and falls back to a plain device if unsupported. It returns a `GpuContext` containing the `GPUAdapter`, `GPUDevice`, and `GPUQueue`. ```APIDOC ## `Renderer.requestContext(gaussians: PackedGaussians): Promise` Static async factory that negotiates a WebGPU adapter and device with memory limits scaled to the loaded scene. It first tries to acquire the `timestamp-query` optional feature for GPU pipeline profiling; if the browser does not support it, it falls back to a plain device. Returns a `GpuContext` wrapping the `GPUAdapter`, `GPUDevice`, and `GPUQueue`. ``` -------------------------------- ### Initialize and Control Interactive Camera Source: https://context7.com/marcusandreassvensson/gaussian-splatting-webgpu/llms.txt Sets up an interactive camera for a canvas and demonstrates programmatic control. Handles keyboard and mouse input for navigation. ```typescript import { Camera, InteractiveCamera, CameraFileParser } from '@/splatting-web/camera' const canvas = document.getElementById('gpu-canvas') as HTMLCanvasElement // Create a default camera at a sensible starting position (960×960 canvas): const interactiveCamera = InteractiveCamera.default(canvas) // Keyboard controls: W/S forward/back, A/D left/right, Q/E spin, Z/X up/down // Mouse: left-click drag to look around, scroll wheel to dolly // Programmatically set a new camera (e.g., from a preset list): const rawCamera = { id: 0, img_name: 'frame_00001', width: 960, height: 960, position: [0.13, -1.18, 3.39], rotation: [[0.58, -0.32, 0.74], [0.24, 0.94, 0.22], [-0.77, 0.04, 0.62]], fx: 960, fy: 960, } // Load presets from a cameras.json file (drag-and-drop onto the UI): const parser = new CameraFileParser(canvas, (cam) => { interactiveCamera.setNewCamera(cam) }) // After loading a cameras.json via an : const cameraList: Camera[] = await parser.handleFileInputChange(inputChangeEvent) // cameraList[n] can then be passed to interactiveCamera.setNewCamera(cameraList[n]) // Read the current camera for manual uniform uploads: const cam = interactiveCamera.getCamera() console.log('Camera position:', cam.getPosition()) // Vec3 in world space console.log('View matrix:', cam.viewMatrix) // column-major Mat4 console.log('Proj*View matrix:', cam.getProjMatrix()) // combined for shaders ``` -------------------------------- ### Enabling GPU Timestamp Profiling Source: https://context7.com/marcusandreassvensson/gaussian-splatting-webgpu/llms.txt Instructions for launching Chrome with specific flags to enable GPU pipeline timestamp profiling for performance analysis. This is useful for debugging and optimizing rendering passes. ```bash # macOS / Linux google-chrome --enable-dawn-features=allow_unsafe_apis # Then open http://localhost:5173 and check the browser console for per-pass timings ``` -------------------------------- ### Initialize and Use GpuContext for WebGPU Operations Source: https://context7.com/marcusandreassvensson/gaussian-splatting-webgpu/llms.txt GpuContext wraps WebGPU objects and provides helpers for buffer creation and timestamp queries. It's typically constructed by a renderer but can be used standalone. ```typescript import { GpuContext } from '@/point-preprocessor/gpuContext' // Constructed by Renderer.requestContext(); not instantiated directly in userland. // But can be used standalone: const gpu = navigator.gpu const adapter = await gpu.requestAdapter() const device = await adapter!.requestDevice() const ctx = new GpuContext(gpu, adapter!, device) // Size-aligned buffer creation (pads to 4 bytes): const buf = ctx.createBuffer0( 100, // requested bytes — padded to 100 (already aligned) GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_SRC, ) // Upload-at-creation convenience: const data = new Float32Array([1, 2, 3, 4]) const uploadBuf = ctx.createBuffer( data, GPUBufferUsage.STORAGE, 0, // source offset data.byteLength, ) // Check whether timestamp profiling is available: if (ctx.hasTimestampQuery) { console.log('GPU pipeline timing is enabled.') } // Destroy device when done (called internally by renderer.destroy()): ctx.destroy() ``` -------------------------------- ### Orchestrate GPU Gaussian Preprocessing Pipeline with Preprocessor Source: https://context7.com/marcusandreassvensson/gaussian-splatting-webgpu/llms.txt The Preprocessor class manages a multi-pass compute shader pipeline for projecting 3D Gaussians to 2D, calculating intersections, and sorting them. It requires a GpuContext, Gaussian data, uniform buffer, and canvas. ```typescript import { Preprocessor } from '@/point-preprocessor/preproces' import { GpuContext } from '@/point-preprocessor/gpuContext' import { PackedGaussians } from '@/splatting-web/ply' // Preprocessor is created inside the Renderer constructor. Standalone usage: const preprocessor = new Preprocessor( gpuContext, // GpuContext gaussians, // PackedGaussians pointDataBuffer, // GPUBuffer — pre-uploaded Gaussian data uniformBuffer, // GPUBuffer — camera/screen uniforms canvas, // HTMLCanvasElement — to derive tile grid dimensions renderer, // Renderer — for optional timestamp insertion ) // Each frame, called from Renderer.draw(): const result = await preprocessor.run() // result contains: // { // gaussData: GPUBuffer — projected 2D Gaussian properties // gaussDataLayout: StaticArray // intersectionKeys: GPUBuffer — sorted (tile | depth) u32 pairs // intersectionKeysLayout: StaticArray // intersectionOffsets: GPUBuffer — per-tile start indices into sorted list // intersectionOffsetsLayout: StaticArray // aux: GPUBuffer — { num_intersections, num_visible_gaussians, min/max_depth } // auxLayout: Struct // } console.log('Tile grid:', Math.ceil(canvas.width / 16), '×', Math.ceil(canvas.height / 16)) console.log('Max intersection array length:', 14e6) ``` -------------------------------- ### loadFileAsArrayBuffer Source: https://context7.com/marcusandreassvensson/gaussian-splatting-webgpu/llms.txt Reads a browser File object as a raw binary ArrayBuffer. This is the entry point for loading a .ply scene file from a file-picker element before passing it to PackedGaussians. ```APIDOC ## loadFileAsArrayBuffer(file: File): Promise ### Description Reads a browser `File` object as a raw binary `ArrayBuffer`. This is the entry point for loading a `.ply` scene file from a file-picker `` element before passing it to `PackedGaussians`. ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```typescript import { loadFileAsArrayBuffer } from '@/splatting-web/ply' // Wired to an onChange handler async function handlePlyChange(event: React.ChangeEvent) { const file = event.target.files?.[0] if (!file) return try { const arrayBuffer = await loadFileAsArrayBuffer(file) // arrayBuffer is now a raw binary blob ready for PLY parsing const gaussians = new PackedGaussians(arrayBuffer) console.log(`Loaded ${gaussians.numGaussians} Gaussians`) } catch (error) { // FileReader errors surface here console.error('Failed to load PLY file:', error) } } ``` ### Response #### Success Response (200) - **arrayBuffer** (ArrayBuffer) - The raw binary data of the file. #### Response Example (Binary data, not representable as JSON) ``` -------------------------------- ### Define and Pack Uniform Buffer Layout Source: https://context7.com/marcusandreassvensson/gaussian-splatting-webgpu/llms.txt Defines a struct layout mirroring WGSL uniform structs for packing data into ArrayBuffers. Used for `device.queue.writeBuffer`. ```typescript import { Struct, StaticArray, Mat4x4, f32, u32, vec2, vec3, vec4 } from '@/splatting-web/packing' // --- Define the uniform layout (mirrors the WGSL uniform struct) --- const uniformLayout = new Struct([ ['viewMatrix', new Mat4x4(f32)], // 64 bytes, alignment 16 ['projMatrix', new Mat4x4(f32)], ['cameraPosition', new vec3(f32)], // 12 bytes, alignment 16 ['tanHalfFovX', f32], ['tanHalfFovY', f32], ['focalX', f32], ['focalY', f32], ['scaleModifier', f32], ['screen_size', new vec2(u32)], // 8 bytes, alignment 8 ]) console.log('Uniform buffer size (bytes):', uniformLayout.size) // e.g. 176 // --- Pack data into an ArrayBuffer for writeBuffer --- const buf = new ArrayBuffer(uniformLayout.size) const view = new DataView(buf) uniformLayout.pack(0, { viewMatrix: [[1,0,0,0],[0,1,0,0],[0,0,1,0],[0,0,0,1]], projMatrix: [[1,0,0,0],[0,1,0,0],[0,0,1,0],[0,0,0,1]], cameraPosition: [0, 0, 3], tanHalfFovX: 0.5, tanHalfFovY: 0.5, focalX: 960, focalY: 960, scaleModifier: 1.0, screen_size: [1840, 960], }, view) device.queue.writeBuffer(uniformGPUBuffer, 0, buf, 0, buf.byteLength) // --- Define and size a point-cloud array layout --- const gaussianLayout = new Struct([ ['position', new vec3(f32)], ['logScale', new vec3(f32)], ['rotQuat', new vec4(f32)], ['opacityLogit', f32], ['shCoeffs', new StaticArray(new vec3(f32), 16)], // SH degree 3 ]) const arrayLayout = new StaticArray(gaussianLayout, 3_000_000) console.log('Total GPU buffer size:', arrayLayout.size, 'bytes') // ≈ 912 MB for 3M Gaussians at degree 3 ``` -------------------------------- ### Internal Canvas Component Logic Flow Source: https://context7.com/marcusandreassvensson/gaussian-splatting-webgpu/llms.txt This outlines the internal steps the Canvas component takes when a .ply file is selected for rendering. It details the sequence from file loading to renderer initialization and state management. ```plaintext // What Canvas does internally when a .ply file is selected: // // 1. loadFileAsArrayBuffer(file) → ArrayBuffer // 2. new PackedGaussians(arrayBuffer) → packed GPU-ready scene data // 3. Renderer.requestContext(gaussians) → GpuContext (negotiates device memory limits) // 4. new EngineContext(); await init() → sets up radix-sort bind-group layout cache // 5. new Renderer(canvas, cam, gaussians, ctx, fpsLabel) → starts rAF loop // 6. renderStore.setState({ renderer }) → makes renderer accessible to other components // // Camera scale slider wired to: // camera.setScale(value) // renderer.invalidate() // triggers a re-render without camera movement // // Camera preset list wired to: // interactiveCamera.setNewCamera(presetCamera) ``` -------------------------------- ### PackedGaussians Source: https://context7.com/marcusandreassvensson/gaussian-splatting-webgpu/llms.txt Parses a binary PLY file from an ArrayBuffer, automatically detecting the spherical harmonics degree and packing Gaussian data into a contiguous ArrayBuffer for WebGPU. ```APIDOC ## new PackedGaussians(arrayBuffer: ArrayBuffer) ### Description Parses a binary PLY file from an `ArrayBuffer`, automatically detecting the spherical harmonics degree from the file's vertex properties, and packs all Gaussian data (position, log-scale, rotation quaternion, opacity logit, SH coefficients) into a contiguous `ArrayBuffer` laid out for direct upload to a WebGPU `GPUBuffer`. The `gaussiansBuffer` and `positionsBuffer` properties are ready for GPU consumption immediately after construction. ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```typescript import { PackedGaussians } from '@/splatting-web/ply' const arrayBuffer: ArrayBuffer = await loadFileAsArrayBuffer(file) const gaussians = new PackedGaussians(arrayBuffer) console.log('Number of Gaussians:', gaussians.numGaussians) // e.g. 3_000_000 console.log('SH degree detected:', gaussians.sphericalHarmonicsDegree) // 0, 1, 2, or 3 — inferred from f_rest_* property count in PLY header console.log('SH coefficients per Gaussian:', gaussians.nShCoeffs) // degree 0 → 1, degree 1 → 4, degree 2 → 9, degree 3 → 16 console.log('GPU buffer byte size:', gaussians.gaussiansBuffer.byteLength) // Ready for: device.createBuffer({ mappedAtCreation: true, ... }) // Layout descriptor (used by the Renderer for buffer sizing): console.log('Total byte size:', gaussians.gaussianArrayLayout.size) ``` ### Response #### Success Response (200) - **numGaussians** (number) - The total number of Gaussians parsed from the file. - **sphericalHarmonicsDegree** (number) - The detected degree of spherical harmonics (0, 1, 2, or 3). - **nShCoeffs** (number) - The number of spherical harmonics coefficients per Gaussian. - **gaussiansBuffer** (ArrayBuffer) - A contiguous buffer containing Gaussian data ready for GPU upload. - **positionsBuffer** (ArrayBuffer) - A buffer containing only the positions of the Gaussians, ready for GPU upload. - **gaussianArrayLayout** (object) - An object describing the layout and size of the Gaussian data. #### Response Example (Object properties with numerical values, not representable as JSON) ``` -------------------------------- ### Load File as ArrayBuffer Source: https://context7.com/marcusandreassvensson/gaussian-splatting-webgpu/llms.txt Reads a browser File object into an ArrayBuffer. Use this to load `.ply` scene files before parsing them. ```typescript import { loadFileAsArrayBuffer } from '@/splatting-web/ply' // Wired to an onChange handler async function handlePlyChange(event: React.ChangeEvent) { const file = event.target.files?.[0] if (!file) return try { const arrayBuffer = await loadFileAsArrayBuffer(file) // arrayBuffer is now a raw binary blob ready for PLY parsing const gaussians = new PackedGaussians(arrayBuffer) console.log(`Loaded ${gaussians.numGaussians} Gaussians`) } catch (error) { // FileReader errors surface here console.error('Failed to load PLY file:', error) } } ``` -------------------------------- ### Perform In-Place GPU Radix Sort on Apple Silicon Source: https://context7.com/marcusandreassvensson/gaussian-splatting-webgpu/llms.txt RadixSort implements a 4-bit chunk LSD radix sort optimized for Apple Silicon GPUs, sorting a GPUBuffer of 32-bit integer keys in-place. It's constructed once and reused, with an option to sort only active elements. ```typescript import { RadixSort } from '@/radix-sort/radixSortApple' import { GpuContext } from '@/point-preprocessor/gpuContext' const numElements = 14_000_000 // max tile-splat intersections const numIntsPerElement = 2 // each key is a (tile_id, depth) u32 pair const sorter = new RadixSort(gpuContext, numElements, numIntsPerElement, renderer) // Called once per frame inside Preprocessor.run(): const tileDepthKeyBuffer: GPUBuffer = /* ... preprocessed key buffer ... */ someBuffer const { values: sortedBuffer } = sorter.sort(tileDepthKeyBuffer) // sortedBuffer === tileDepthKeyBuffer (sorted in-place, returned for convenience) // To sort only the active intersections this frame (avoids sorting padding zeros): const activeIntersections = 2_500_000 // from auxBuffer.num_intersections sorter.sort(tileDepthKeyBuffer, activeIntersections) ``` -------------------------------- ### Parse PLY File into PackedGaussians Source: https://context7.com/marcusandreassvensson/gaussian-splatting-webgpu/llms.txt Parses a binary PLY file from an ArrayBuffer, packing Gaussian data for GPU consumption. Automatically detects SH degree and prepares buffers for WebGPU. ```typescript import { PackedGaussians } from '@/splatting-web/ply' const arrayBuffer: ArrayBuffer = await loadFileAsArrayBuffer(file) const gaussians = new PackedGaussians(arrayBuffer) console.log('Number of Gaussians:', gaussians.numGaussians) // e.g. 3_000_000 console.log('SH degree detected:', gaussians.sphericalHarmonicsDegree) // 0, 1, 2, or 3 — inferred from f_rest_* property count in PLY header console.log('SH coefficients per Gaussian:', gaussians.nShCoeffs) // degree 0 → 1, degree 1 → 4, degree 2 → 9, degree 3 → 16 console.log('GPU buffer byte size:', gaussians.gaussiansBuffer.byteLength) // Ready for: device.createBuffer({ mappedAtCreation: true, ... }) // Layout descriptor (used by the Renderer for buffer sizing): console.log('Total byte size:', gaussians.gaussianArrayLayout.size) ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.