### Install fisheye.js using npm Source: https://context7.com/gyeonghokim/fisheye.js/llms.txt This command installs the fisheye.js library from the npm registry. Ensure you have Node.js and npm installed on your system. ```bash npm install @gyeonghokim/fisheye.js ``` -------------------------------- ### Install fisheye.js and WebGPU Types Source: https://github.com/gyeonghokim/fisheye.js/blob/main/README.md Instructions for installing the fisheye.js library and optional WebGPU types for TypeScript projects. This includes npm commands and configuring tsconfig.json. ```bash npm install @gyeonghokim/fisheye.js # optional npm install --save-dev @webgpu/types ``` ```json { "compilerOptions": { "types": ["@webgpu/types"] } } ``` -------------------------------- ### Create VideoFrame from YUV Data (TypeScript) Source: https://github.com/gyeonghokim/fisheye.js/blob/main/README.md Provides an example of using the `createVideoFrameFromYUV` utility to convert raw YUV binary data into a VideoFrame object. This is useful when receiving video data in YUV format from external sources. ```typescript import { Fisheye, createVideoFrameFromYUV } from "@gyeonghokim/fisheye.js"; const dewarper = new Fisheye({ k1: 0.5, width: 1920, height: 1080 }); // Example: Receiving NV12 data from a server const response = await fetch("/api/camera/frame"); const yuvBuffer = await response.arrayBuffer(); const frame = createVideoFrameFromYUV(new Uint8Array(yuvBuffer), { format: "NV12", // YUV format width: 1920, height: 1080, timestamp: performance.now() * 1000, // microseconds }); const dewarpedFrame = await dewarper.dewarp(frame); frame.close(); // Don't forget to close the original frame ``` -------------------------------- ### Initialize Fisheye Dewarping Instance (TypeScript) Source: https://context7.com/gyeonghokim/fisheye.js/llms.txt Demonstrates how to create an instance of the Fisheye class with OpenCV fisheye model parameters. This class initializes WebGPU resources and compute shader pipelines for frame processing. It accepts parameters like distortion coefficients (k1-k4), output resolution, field of view, lens center offsets, and zoom. ```typescript import { Fisheye } from "@gyeonghokim/fisheye.js"; // Create a dewarper with OpenCV fisheye model parameters const dewarper = new Fisheye({ k1: 0.5, // Distortion coefficient k1 (range: -1.0 to 1.0) k2: 0.0, // Distortion coefficient k2 k3: 0.0, // Distortion coefficient k3 k4: 0.0, // Distortion coefficient k4 width: 1920, // Output frame width (default: 300) height: 1080, // Output frame height (default: 150) fov: 180, // Field of view in degrees (default: 180) centerX: 0, // X offset of lens center, normalized -1.0 to 1.0 (default: 0) centerY: 0, // Y offset of lens center, normalized -1.0 to 1.0 (default: 0) zoom: 1.0, // Zoom factor (default: 1.0) }); // Process a video frame async function processFrame(inputFrame: VideoFrame): Promise { const dewarpedFrame = await dewarper.dewarp(inputFrame); return dewarpedFrame; } // Clean up when done dewarper.destroy(); ``` -------------------------------- ### Initialize Fisheye Dewarping Instance (TypeScript) Source: https://github.com/gyeonghokim/fisheye.js/blob/main/README.md Demonstrates how to import and initialize the Fisheye dewarper in TypeScript. It shows the constructor with various configuration options for distortion, output dimensions, field of view, lens center offset, and zoom. ```typescript import { Fisheye } from "@gyeonghokim/fisheye.js"; const dewarper = new Fisheye({ k1: 0.5, k2: 0.0, k3: 0.0, k4: 0.0, width: 1920, height: 1080, fov: 180, // Field of view in degrees centerX: 0, // X offset of lens center (-1.0 to 1.0) centerY: 0, // Y offset of lens center (-1.0 to 1.0) zoom: 1.0, // Zoom factor }); const renderLoop = (timestamp: DOMHighResTimestamp) => { // your render logic const dewarpedVideoFrame: VideoFrame = await dewarper.dewarp(yourVideoFrame); yourYUVPlayer.draw(dewarpedVideoFrame); requestAnimationFrame(renderLoop); }; ``` -------------------------------- ### Dynamically Update Fisheye Configuration (TypeScript) Source: https://context7.com/gyeonghokim/fisheye.js/llms.txt Shows how to use the `updateConfig()` method to adjust dewarping parameters in real-time without re-initializing the `Fisheye` object. This method allows for dynamic changes to distortion coefficients, zoom levels, lens center offsets, and output resolution. ```typescript import { Fisheye } from "@gyeonghokim/fisheye.js"; const dewarper = new Fisheye({ k1: 0.5, width: 1920, height: 1080, zoom: 1.0, }); // Update distortion parameters dynamically function onDistortionSliderChange(value: number) { dewarper.updateConfig({ k1: value }); } // Update zoom level function onZoomChange(zoomLevel: number) { dewarper.updateConfig({ zoom: zoomLevel }); } // Update lens center offset function onCenterOffsetChange(x: number, y: number) { dewarper.updateConfig({ centerX: x, // -1.0 to 1.0 centerY: y, // -1.0 to 1.0 }); } // Change output resolution (triggers texture recreation) function onResolutionChange(width: number, height: number) { dewarper.updateConfig({ width, height }); } ``` -------------------------------- ### Create VideoFrame from YUV Data using fisheye.js Source: https://context7.com/gyeonghokim/fisheye.js/llms.txt Generates VideoFrame objects from raw YUV binary data. Supports various YUV formats like NV12 and I420, and allows specifying dimensions, timestamp, and optional buffer transfer for zero-copy operations. Useful for processing camera or server streams. ```typescript import { Fisheye, createVideoFrameFromYUV } from "@gyeonghokim/fisheye.js"; const dewarper = new Fisheye({ k1: 0.5, width: 1920, height: 1080 }); // Example: Processing YUV data from a WebSocket stream async function processYUVStream(websocket: WebSocket) { websocket.onmessage = async (event) => { const yuvBuffer = await event.data.arrayBuffer(); // Create VideoFrame from NV12 data (common camera format) const frame = createVideoFrameFromYUV(new Uint8Array(yuvBuffer), { format: "NV12", // Supported: I420, NV12, I420A, I422, I444 width: 1920, height: 1080, timestamp: performance.now() * 1000, // microseconds duration: 33333, // Optional: ~30fps frame duration transfer: true, // Optional: zero-copy buffer transfer }); const dewarpedFrame = await dewarper.dewarp(frame); frame.close(); // Render or encode the dewarped frame // yourRenderer.draw(dewarpedFrame); dewarpedFrame.close(); }; } // Example: Fetching YUV data from REST API async function fetchAndDewarp(url: string) { const response = await fetch(url); const yuvBuffer = await response.arrayBuffer(); const frame = createVideoFrameFromYUV(new Uint8Array(yuvBuffer), { format: "I420", width: 1280, height: 720, timestamp: 0, }); const dewarped = await dewarper.dewarp(frame); frame.close(); return dewarped; } ``` -------------------------------- ### Clean up GPU Resources Source: https://github.com/gyeonghokim/fisheye.js/blob/main/README.md Demonstrates the proper way to release GPU resources allocated by the Fisheye dewarper instance. The `destroy` method should be called when the dewarper is no longer needed to prevent memory leaks. ```typescript dewarper.destroy(); ``` -------------------------------- ### Update Fisheye Dewarping Configuration Source: https://github.com/gyeonghokim/fisheye.js/blob/main/README.md Illustrates how to dynamically update the configuration of an existing Fisheye dewarper instance using the `updateConfig` method. This allows modifying parameters like distortion coefficients or zoom without re-initializing. ```typescript dewarper.updateConfig({ k1: 0.6, zoom: 1.2 }); ``` -------------------------------- ### Dewarp VideoFrame using fisheye.js Source: https://github.com/gyeonghokim/fisheye.js/blob/main/README.md Shows the core functionality of dewarping a VideoFrame using the `dewarp` method of the Fisheye instance. This asynchronous method takes an input VideoFrame and returns a Promise resolving to the dewarped VideoFrame. ```typescript const dewarpedVideoFrame: VideoFrame = await dewarper.dewarp(yourVideoFrame); ``` -------------------------------- ### Calculate YUV Data Buffer Size using fisheye.js Source: https://context7.com/gyeonghokim/fisheye.js/llms.txt Calculates the required byte size for YUV data based on the specified format and dimensions. This is essential for allocating appropriate buffers before processing or storing YUV data. Supports I420, NV12, I422, I444, and I420A formats. ```typescript import { calculateYUVDataSize } from "@gyeonghokim/fisheye.js"; // Calculate buffer sizes for different YUV formats const width = 1920; const height = 1080; const i420Size = calculateYUVDataSize("I420", width, height); // Result: 3110400 bytes (width * height * 1.5) const nv12Size = calculateYUVDataSize("NV12", width, height); // Result: 3110400 bytes (width * height * 1.5) const i422Size = calculateYUVDataSize("I422", width, height); // Result: 4147200 bytes (width * height * 2) const i444Size = calculateYUVDataSize("I444", width, height); // Result: 6220800 bytes (width * height * 3) const i420aSize = calculateYUVDataSize("I420A", width, height); // Result: 5184000 bytes (width * height * 2.5, includes alpha) // Use for buffer allocation const buffer = new ArrayBuffer(calculateYUVDataSize("NV12", 1920, 1080)); ``` -------------------------------- ### Convert RGBA to YUV Format using fisheye.js Source: https://context7.com/gyeonghokim/fisheye.js/llms.txt Converts RGBA pixel data into the I420 YUV format using ITU-R BT.601 color space conversion. This utility is useful for processing image data from sources like HTML canvas elements or image files before further manipulation or encoding. ```typescript import { convertRGBAtoYUV, createVideoFrameFromYUV, Fisheye } from "@gyeonghokim/fisheye.js"; const dewarper = new Fisheye({ k1: 0.4, width: 1920, height: 1080 }); // Convert canvas image to YUV and process async function processCanvasImage(canvas: HTMLCanvasElement) { const ctx = canvas.getContext("2d"); if (!ctx) throw new Error("Could not get canvas context"); const width = canvas.width; const height = canvas.height; // Get RGBA pixel data from canvas const imageData = ctx.getImageData(0, 0, width, height); // Convert to I420 YUV format (BT.601 color space) const yuvData = convertRGBAtoYUV(imageData.data, width, height, "I420"); // Create VideoFrame from YUV data const frame = createVideoFrameFromYUV(yuvData, { format: "I420", width, height, timestamp: performance.now() * 1000, }); // Dewarp the frame const dewarpedFrame = await dewarper.dewarp(frame); frame.close(); return dewarpedFrame; } // Process an image file async function processImageFile(file: File) { const bitmap = await createImageBitmap(file); const canvas = document.createElement("canvas"); canvas.width = bitmap.width; canvas.height = bitmap.height; const ctx = canvas.getContext("2d")!; ctx.drawImage(bitmap, 0, 0); return processCanvasImage(canvas); } ``` -------------------------------- ### Calculate YUV Data Size with TypeScript Source: https://github.com/gyeonghokim/fisheye.js/blob/main/README.md Calculates the expected byte size for YUV data based on the specified format, width, and height. This function is useful for pre-allocating buffers for YUV data. It requires the fisheye.js library to be imported. ```typescript import { calculateYUVDataSize } from "@gyeonghokim/fisheye.js"; const size = calculateYUVDataSize("NV12", 1920, 1080); // 3110400 bytes ``` -------------------------------- ### TypeScript Fisheye Lens Correction Integration Source: https://context7.com/gyeonghokim/fisheye.js/llms.txt This snippet demonstrates how to use the fisheye.js library to perform real-time fisheye lens correction. It includes a FisheyeProcessor class that handles initialization, NV12 frame processing, distortion updates, and cleanup. The `processNV12Frame` method takes raw YUV data and returns a dewarped VideoFrame. Ensure VideoFrame objects are closed after use to prevent memory leaks. ```typescript import { Fisheye, createVideoFrameFromYUV, calculateYUVDataSize } from "@gyeonghokim/fisheye.js"; class FisheyeProcessor { private dewarper: Fisheye; private isProcessing = false; constructor(config: { k1: number; width: number; height: number }) { this.dewarper = new Fisheye({ k1: config.k1, k2: 0, k3: 0, k4: 0, width: config.width, height: config.height, fov: 180, zoom: 1.0, }); } async processNV12Frame( data: Uint8Array, width: number, height: number ): Promise { // Validate buffer size const expectedSize = calculateYUVDataSize("NV12", width, height); if (data.byteLength < expectedSize) { throw new Error(`Invalid buffer size: ${data.byteLength} < ${expectedSize}`); } const inputFrame = createVideoFrameFromYUV(data, { format: "NV12", width, height, timestamp: performance.now() * 1000, }); try { return await this.dewarper.dewarp(inputFrame); } finally { inputFrame.close(); } } updateDistortion(k1: number, k2 = 0, k3 = 0, k4 = 0): void { this.dewarper.updateConfig({ k1, k2, k3, k4 }); } destroy(): void { this.dewarper.destroy(); } } // Usage const processor = new FisheyeProcessor({ k1: 0.5, width: 1920, height: 1080, }); // Process frames from your video source // const dewarpedFrame = await processor.processNV12Frame(yuvData, 1920, 1080); // Clean up when done // processor.destroy(); ``` -------------------------------- ### Dewarp Video Frames using Fisheye.dewarp() (TypeScript) Source: https://context7.com/gyeonghokim/fisheye.js/llms.txt Illustrates the usage of the `dewarp()` method to correct fisheye distortion in a VideoFrame. This method handles GPU texture creation, compute shader execution, and data readback. It's designed for real-time video processing loops, requiring proper closing of both input and output VideoFrames to manage GPU resources. ```typescript import { Fisheye } from "@gyeonghokim/fisheye.js"; const dewarper = new Fisheye({ k1: 0.3, width: 1920, height: 1080, }); // Real-time video processing loop async function startVideoProcessing(videoElement: HTMLVideoElement) { const renderLoop = async () => { // Create VideoFrame from video element const inputFrame = new VideoFrame(videoElement, { timestamp: performance.now() * 1000, // microseconds }); try { // Dewarp the frame const dewarpedFrame = await dewarper.dewarp(inputFrame); // Use the dewarped frame (e.g., render to canvas or send to encoder) // yourRenderer.draw(dewarpedFrame); dewarpedFrame.close(); // Release GPU resources } finally { inputFrame.close(); // Always close input frame } requestAnimationFrame(renderLoop); }; renderLoop(); } ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.