### Install Face Decoration Example Dependencies Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-examples/faceDecoration/README.md Navigate to the faceDecoration example directory within the cloned Paddle.js repository and install the necessary npm dependencies to run the face decoration examples. ```sh cd Paddle.js/packages/paddlejs-examples/faceDecoration && npm install ``` -------------------------------- ### Install and Run Gesture Demo Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-models/gesture/README.md Instructions to install dependencies and run the gesture recognition demo locally. This involves using npm to install packages and then starting the development server. The demo can then be accessed via a local URL. ```bash npm install npm run dev ``` -------------------------------- ### Install @paddlejs-mediapipe/camera Package Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-mediapipe/camera/README.md This snippet shows the command to install the @paddlejs-mediapipe/camera package using npm. This is the first step to using the camera functionality in your project. ```bash npm install @paddlejs-mediapipe/camera ``` -------------------------------- ### Initialize Runner with WebGL Backend (Simplified) Source: https://context7.com/paddlepaddle/paddle.js/llms.txt A simplified example of initializing the `Runner` with the WebGL backend, focusing on model path, feed shape, fill color, and WebGL specific processing. This setup is suitable for scenarios involving video elements for prediction. ```typescript import { Runner } from '@paddlejs/paddlejs-core'; import '@paddlejs/paddlejs-backend-webgl'; const runner = new Runner({ modelPath: '/model/mobilenetv2', feedShape: { fw: 256, fh: 256 }, fill: '#fff', webglFeedProcess: true }); await runner.init(); const result = await runner.predict(videoElement); ``` -------------------------------- ### Initialize and Control Camera with JavaScript Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-mediapipe/camera/README.md Demonstrates how to import and initialize the Camera class from the @paddlejs-mediapipe/camera package. It includes an example of configuring camera options, starting, pausing, and switching cameras. The `onSuccess`, `onError`, and `onFrame` callbacks are crucial for handling video stream events. ```javascript import Camera from '@paddlejs-mediapipe/camera'; const option = { // video width width?: number, // video height height?: number, // mirror or not mirror?: boolean, // if enable capture stream when page hidden, default is false enableOnInactiveState?: boolean, // canvas DOM targetCanvas?: HTMLCanvasElement, // video rendered successfully onSuccess?: () => void, // video rendering failed onError?: NavigatorUserMediaErrorCallback, // browser does not support the getusermedia API onNotSupported?: () => void, // get every frame of video onFrame?: (canvas: HTMLCanvasElement) => void, // switch camera error switchError?: () => void, // video loadedData videoLoaded?: () => void }; const camera = new Camera(video, option); // video play camera.start(); // video pause camera.pause(); // cameras switch camera.switchCameras(); ``` -------------------------------- ### Clone Paddle.js Repository Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-examples/faceDecoration/README.md This command clones the Paddle.js GitHub repository to your local machine, allowing you to access its various packages and examples. ```sh git clone https://github.com/PaddlePaddle/Paddle.js.git ``` -------------------------------- ### Install @paddlejs-mediapipe/data-processor Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-mediapipe/data-processor/README.md Installs the @paddlejs-mediapipe/data-processor npm package using npm. This is the initial step before using its data processing capabilities. ```bash npm install @paddlejs-mediapipe/data-processor ``` -------------------------------- ### Install and Build HumanSeg with npm Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-examples/humanseg/README.md This snippet demonstrates the npm commands required to set up and build the HumanSeg project. `npm install` installs dependencies, while `npm run dev` compiles for development with hot-reloading, and `npm run build` compiles for production. ```bash npm install ## Compiles and hot-reloads for development npm run dev ## Compiles and minifies for production npm run build ``` -------------------------------- ### Install npm dependencies for Paddle.js Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-examples/gesture/README.md Installs the necessary dependencies for the Paddle.js project using npm. This command should be run in the project's root directory. ```bash npm install ``` -------------------------------- ### Run Development Server using npm Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-examples/webglWorker/README.md This command starts the development server, compiles the project, and makes it available at the specified URL. It's typically used for local development and testing. Visit `http://0.0.0.0:8866/` in your browser to view the application. ```bash npm run dev # visit http://0.0.0.0:8866/ ``` -------------------------------- ### Compile and hot-reload Paddle.js for development Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-examples/gesture/README.md Starts the development server, enabling live-reloading for rapid development cycles. This command is typically used during the development phase of the project. ```bash npm run dev ``` -------------------------------- ### Initialize and Use OCR Module in JavaScript Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-examples/ocr/README.md Demonstrates how to initialize the OCR module and get text recognition results from an image. It shows how to import the module, initialize it, and call the recognize function, with options for canvas drawing and styling. ```javascript import * as ocr from '@paddlejs-models/ocr'; // Model initialization await ocr.init(); // Get the text recognition result API, img is the user's upload picture, and option is an optional parameter // option.canvas as HTMLElementCanvas:if the user needs to draw the selected area of the text box, pass in the canvas element // option.style as object:if the user needs to configure the canvas style, pass in the style object // option.style.strokeStyle as string:select a color for the text box // option.style.lineWidth as number:width of selected line segment in text box // option.style.fillStyle as string:select the fill color for the text box const res = await ocr.recognize(img, option?); // character recognition results console.log(res.text); // text area points console.log(res.points); ``` -------------------------------- ### Import WebGL Backend via NPM Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-backend-webgl/README.md This snippet demonstrates how to import the registered WebGL backend for Paddle.js using NPM. Ensure you have the necessary package installed. ```javascript import '@paddlejs/paddlejs-backend-webgl'; ``` -------------------------------- ### Install OpenCV npm Package Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-mediapipe/opencv/README.md Instructions to install the OpenCV npm package for use with Paddle.js. This package provides access to various OpenCV modules. ```bash npm install @paddlejs-mediapipe/opencv ``` -------------------------------- ### Include WebGL Backend via Script Tag Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-backend-webgl/README.md This snippet shows how to include the Paddle.js WebGL backend using a script tag, referencing a CDN. This is an alternative to NPM installation. ```html ``` -------------------------------- ### Set and Get Environment Variables in Paddle.js Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-core/README.md Demonstrates how to use the global env module to set and retrieve environment variables in Paddle.js. The env.set() method stores key-value pairs, while env.get() retrieves values by key. This is essential for configuring performance flags and other runtime settings. ```javascript // set env key/flag and value env.set(key, value); // get value by key/flag env.get(key); ``` -------------------------------- ### Import NodeGL Backend in Node.js Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-backend-nodegl/README.md This code snippet demonstrates how to import the registered NodeGL backend for Paddle.js. Ensure the '@paddlejs/paddlejs-backend-nodegl' package is installed via npm before using this import statement. ```javascript import '@paddlejs/paddlejs-backend-nodegl'; ``` -------------------------------- ### Apply Gaussian Blur using OpenCV.js Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-mediapipe/opencv/README.md Example demonstrating how to apply a Gaussian blur effect to an image using the OpenCV.js library imported from the Paddle.js package. It involves reading an image from a canvas, creating a destination matrix, defining kernel size and anchor points, and then applying the blur operation. ```javascript import cv from '@paddlejs-mediapipe/opencv/library/opencv_blur'; let logit = cv.imread('canvas'); let dst = new cv.Mat(); let ksize = new cv.Size(5, 5); let anchor = new cv.Point(-1, -1); cv.blur(logit, dst, ksize, anchor, cv.BORDER_DEFAULT); ``` -------------------------------- ### Initialize Runner with WebGPU Backend Source: https://context7.com/paddlepaddle/paddle.js/llms.txt Shows how to initialize the `Runner` for inference using the experimental WebGPU backend, which offers next-generation GPU acceleration. This requires a compatible browser environment (e.g., Chrome Canary with WebGPU enabled). ```typescript import { Runner } from '@paddlejs/paddlejs-core'; import '@paddlejs/paddlejs-backend-webgpu'; const runner = new Runner({ modelPath: 'https://example.com/model', feedShape: { fw: 224, fh: 224 } }); await runner.init(); const result = await runner.predict(imageElement); ``` -------------------------------- ### Paddle.js Core Initialization and Usage Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-core/README.md This section details how to initialize and use the Paddle.js Core runner. It covers importing necessary packages, configuring the runner with `RunnerConfig`, and performing predictions. ```APIDOC ## Paddle.js Core Initialization and Usage ### Description This section details how to initialize and use the Paddle.js Core runner. It covers importing necessary packages, configuring the runner with `RunnerConfig`, and performing predictions. ### Method ```typescript import { Runner } from '@paddlejs/paddlejs-core'; import '@paddlejs/paddlejs-backend-webgl'; // Example backend import // Runner configuration interface interface RunnerConfig { modelPath: string; // model path (local or web address) modelName?: string; // model name feedShape: { fc?: number; // feed channel, default is 3. fw: number; // feed width fh: number; // feed height }; fill?: Color; // the color used to padding mean?: number[]; // mean value std?: number[]; // standard deviation bgr?: boolean; // whether the image channel alignment is BGR, default is false (RGB) type?: GraphType; // model structure, default is singleInput and singleOutput needPreheat?: boolean; // whether to warm up the engine during initialization, default is true plugins?: { preTransforms?: Transformer[]; transforms?: Transformer[]; postTransforms?: Transformer[]; }; webglFeedProcess?: boolean; // Turn on `webglFeedProcess` to convert all pre-processing parts of the model to shader processing, and keep the original image texture. } // Example usage const runner = new Runner({ modelPath: '/model/mobilenetv2', // model path, e.g. http://xx.cc/path, http://xx.cc/path/model.json, /localModelDir/model.json, /localModelDir feedShape: { // input shape fw: 256, fh: 256 }, fill: '#fff', // fill color when resize image, default value is #fff webglFeedProcess: true }); // init runner await runner.init(); // predict and get result const res = await runner.predict(mediadata, callback?); ``` ### Parameters No direct parameters for initialization, but `RunnerConfig` object is required. ### Request Example ```json { "modelPath": "/model/mobilenetv2", "feedShape": { "fw": 256, "fh": 256 }, "fill": "#fff", "webglFeedProcess": true } ``` ### Response #### Success Response (200) - **runner** (Runner) - An initialized Paddle.js runner instance. - **res** (any) - The result of the prediction. #### Response Example ```json { "predictionResult": "example" } ``` ### Notes - If you are importing the Core package, you also need to import a backend (e.g., `@paddlejs/paddlejs-backend-webgl`, `@paddlejs/paddlejs-backend-webgpu`). - The `RunnerConfig` interface defines the structure for configuring the engine. Key required fields include `modelPath` and `feedShape`. ``` -------------------------------- ### Initialize Runner and Perform Model Prediction in JavaScript Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-core/README.md Demonstrates how to import @paddlejs/paddlejs-core and a WebGL backend, instantiate the Runner class with configuration parameters, initialize the runner, and execute model predictions on media data. The webglFeedProcess option enables shader-based preprocessing while preserving the original image texture. ```javascript // Import @paddlejs/paddlejs-core import { Runner } from '@paddlejs/paddlejs-core'; // Import the registered WebGL backend. import '@paddlejs/paddlejs-backend-webgl'; const runner = new Runner({ modelPath: '/model/mobilenetv2', // model path, e.g. http://xx.cc/path, http://xx.cc/path/model.json, /localModelDir/model.json, /localModelDir feedShape: { // input shape fw: 256, fh: 256 }, fill?: '#fff', // fill color when resize image, default value is #fff webglFeedProcess?: true // Turn on `webglFeedProcess` to convert all pre-processing parts of the model to shader processing, and keep the original image texture. }); // init runner await runner.init(); // predict and get result const res = await runner.predict(mediadata, callback?); ``` -------------------------------- ### Initialize Runner with WebGL Backend Source: https://context7.com/paddlepaddle/paddle.js/llms.txt Demonstrates initializing the `Runner` class for model inference using the WebGL backend. It specifies model path, input shape, normalization parameters, and other configuration options. The `init()` method prepares the model for execution, and `predict()` performs inference on an image element. ```typescript import { Runner } from '@paddlejs/paddlejs-core'; import '@paddlejs/paddlejs-backend-webgl'; const runner = new Runner({ modelPath: 'https://example.com/model/model.json', feedShape: { fw: 224, fh: 224, fc: 3 }, mean: [0.485, 0.456, 0.406], std: [0.229, 0.224, 0.225], fill: '#fff', bgr: false, needPreheat: true, webglFeedProcess: false, keepRatio: true }); await runner.init(); const imageElement = document.getElementById('myImage'); const result = await runner.predict(imageElement); console.log(result); ``` -------------------------------- ### Compile and minify Paddle.js for production Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-examples/gesture/README.md Builds the project for production, generating minified and optimized code. This command should be used before deploying the application. ```bash npm run build ``` -------------------------------- ### Initialize Runner with WASM Backend Source: https://context7.com/paddlepaddle/paddle.js/llms.txt This snippet demonstrates initializing the `Runner` with the WebAssembly (WASM) backend for CPU-based inference. It includes specifying the model path, feed shape, and a WASM-specific memory type configuration. ```typescript import { Runner } from '@paddlejs/paddlejs-core'; import '@paddlejs/paddlejs-backend-wasm'; const runner = new Runner({ modelPath: '/model/path', feedShape: { fw: 224, fh: 224 }, wasmMemoryType: 'memory300' }); await runner.init(); const result = await runner.predict(imageElement); ``` -------------------------------- ### Convert PaddlePaddle Models to Browser Format Source: https://context7.com/paddlepaddle/paddle.js/llms.txt Converts PaddlePaddle inference models to a format suitable for web deployment using the paddlejsconverter Python package. Requires specifying input model and parameter paths, and an output directory. The output includes model.json and chunked data files. ```bash pip install paddlejsconverter paddlejsconverter \ --modelPath=/path/to/inference_model \ --paramPath=/path/to/inference_params \ --outputDir=./output \ --useGPUOpt=True ls output/ # model.json chunk_1.dat chunk_2.dat chunk_3.dat ``` -------------------------------- ### Load Mobilenet Model and Classify Image with Paddle.js Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-examples/wine/README.md Demonstrates how to load a mobilenet model and classify an image using the Paddle.js library. It requires the model path and optionally provides mean and standard deviation for normalization. The output is the classification result for the input image. ```javascript import * as mobilenet from '@paddlejs-models/mobilenet'; // model load const path = 'https://paddlejs.bj.bcebos.com/models/fuse/mobilenet/wine_fuse_activation/model.json'; await mobilenet.load({ path, mean: [0.485, 0.456, 0.406], std: [0.229, 0.224, 0.225] }, map); // get classification results const res = await mobilenet.classify(img); ``` -------------------------------- ### HTML Video and Control Elements Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-mediapipe/data-processor/demo/index.html HTML markup structure for the Paddle web camera demo, including a video element for displaying the camera feed and control buttons for play/pause functionality and camera switching. Includes a loading indicator div with Chinese text. ```html
loading中……
``` -------------------------------- ### Enable WebGL Pack Output Performance Flag Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-core/README.md Activates the webgl_pack_output flag to migrate NHWC to NCHW layout transformation to the GPU and pack results into four-channel layout. This optimization reduces loop processing overhead when reading results from GPU memory. ```javascript env.set('webgl_pack_output', true); ``` -------------------------------- ### Convert Model for CPU/WASM Backend using PaddleJSConverter Source: https://context7.com/paddlepaddle/paddle.js/llms.txt Converts a Paddle model for CPU or WebAssembly backends by disabling GPU optimizations. This command-line tool requires the paths to the model and parameter files and specifies an output directory. ```bash paddlejsconverter \ --modelPath=/path/to/inference_model \ --paramPath=/path/to/inference_params \ --outputDir=./output_cpu \ --useGPUOpt=False ``` -------------------------------- ### Loading Overlay and Modal CSS Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-mediapipe/data-processor/demo/index.html CSS styles for a full-screen loading overlay that covers the entire viewport with a semi-transparent dark background. Includes styling for loading text displayed in white with 24px font size, creating a modal loading indicator. ```css #isLoading { position: fixed; top: 0; left: 0; right: 0; bottom: 0; background-color: rgba(0, 0, 0, .5); } #isLoading .loading-text { color: #fff; font-size: 24px; } ``` -------------------------------- ### Load and classify gestures using Paddle.js in JavaScript Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-examples/gesture/README.md Demonstrates how to use the Paddle.js gesture recognition module. It involves loading both the gesture detection and recognition models, and then classifying gestures from an input image. The output includes palm frame coordinates and recognition results. ```javascript import * as gesture from '@paddlejs-models/gesture'; // Load gesture_detect model and gesture_rec model await gesture.load(); // Get the image recognition results. The results include: palm frame coordinates and recognition results const res = await gesture.classify(img); ``` -------------------------------- ### Enable WebGL GPU Pipeline Performance Flag Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-core/README.md Activates the webgl_gpu_pipeline flag to convert model pre-processing to shader processing and render results to WebGL2RenderingContext. This enables GPU-accelerated pipeline combining pre-processing, inference, and post-processing for maximum performance, with output textures available for custom post-processing operations. ```javascript env.set('webgl_gpu_pipeline', true); ``` -------------------------------- ### Initialize and Detect Faces with Paddle.js Facedetect Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-models/facedetect/README.md Demonstrates how to import and initialize the FaceDetector from the '@paddlejs-models/facedetect' package. It shows the process of detecting faces in an image element, specifying optional parameters like 'shrink' and 'threshold', and explains the structure of the returned face area information (left, top, width, height, confidence). ```javascript import { FaceDetector } from '@paddlejs-models/facedetect'; const faceDetector = new FaceDetector(); await faceDetector.init(); // Required parameter:imgEle(HTMLImageElement) // Optional parameter: shrink, threshold // Result is face area information. It includes left, top, width, height, confidence const res = await faceDetector.detect( imgEle, { shrink: 0.4, threshold: 0.6 } ); ``` -------------------------------- ### CSS Layout and Positioning - Gesture Demo Interface Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-models/gesture/demo/index.html Comprehensive CSS styling for a full-screen gesture demonstration interface. Includes body layout, canvas positioning, loading overlay, and responsive centering using flexbox and CSS transforms. The styling creates a fixed full-screen container with absolute positioning for interactive elements. ```css body { margin: 0 auto; position: fixed; width: 100%; height: 100%; } #rect { position: absolute; top: 0; left: 0; width: 100px; height: 100px; } .wrapper { position: relative; width: 100%; height: 100%; margin: 0 auto; } #isLoading { position: fixed; top: 0; left: 0; right: 0; bottom: 0; background-color: rgba(0, 0, 0, .5); } #isLoading .loading-text { color: #fff; font-size: 24px; } .center { position: absolute; top: 50%; left: 50%; -webkit-transform: translate(-50%, -50%); transform: translate(-50%, -50%); } #canvas { position: absolute; left: 0; top: 0; width: 256px; } ``` -------------------------------- ### GPU Pipeline with Custom Post-Processing in Paddle.js Source: https://context7.com/paddlepaddle/paddle.js/llms.txt Implements end-to-end GPU processing with custom rendering on output textures. It configures WebGL rendering context, sets environment variables for GPU pipeline, initializes a Runner, and processes video input, drawing the output to a canvas. Dependencies include '@paddlejs/paddlejs-core' and '@paddlejs/paddlejs-backend-webgl'. ```typescript import { Runner, env } from '@paddlejs/paddlejs-core'; import { GLHelper } from '@paddlejs/paddlejs-backend-webgl'; import '@paddlejs/paddlejs-backend-webgl'; const canvas = document.createElement('canvas'); canvas.width = 640; canvas.height = 480; const gl = canvas.getContext('webgl2', { alpha: true, antialias: false, premultipliedAlpha: false, preserveDrawingBuffer: true, depth: false, stencil: false }); GLHelper.setWebGLRenderingContext(gl); env.set('webgl_pack_channel', true); env.set('webgl_gpu_pipeline', true); env.set('webgl_force_half_float_texture', true); env.set('webgl_pack_output', true); const runner = new Runner({ modelPath: '/model/path', feedShape: { fw: 640, fh: 480 }, webglFeedProcess: true }); await runner.init(); const videoElement = document.getElementById('webcam'); await runner.predict(videoElement); const outputCanvas = document.getElementById('output'); const ctx = outputCanvas.getContext('2d'); ctx.drawImage(gl.canvas, 0, 0, outputCanvas.width, outputCanvas.height); ``` -------------------------------- ### OCR Detect Model Usage in JavaScript Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-examples/ocrdetection/README.md Demonstrates how to use the ocr_detect model from the @paddlejs-models/ocrdet package. It shows the steps to load the model and then use it to detect text areas within an image. ```javascript import * as ocr from '@paddlejs-models/ocrdet'; // Load ocr_detect model await ocr.load(); // Get text area points const res = await ocr.detect(img); ``` -------------------------------- ### Full-Screen Video Layout CSS Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-mediapipe/data-processor/demo/index.html CSS stylesheet that establishes a full-screen video container with fixed positioning, ensuring the video element stretches to fill the entire viewport. This provides a responsive base layout for the camera demo application with zero margins. ```css body { margin: 0 auto; position: fixed; width: 100%; height: 100%; } video { width: 100%; } ``` -------------------------------- ### Load and Run Detection with PaddleJS Detect Model Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-models/detect/README.md This snippet demonstrates how to import the PaddleJS detect model, load the pre-trained model, and perform object detection on an image. The detect function returns an array of results where each item contains the label index, confidence score, and bounding box coordinates (left, top, right, bottom) for detected objects. ```javascript import * as det from '@paddlejs-models/detect'; // Load model await det.load(); // Get label index, confidence and coordinates const res = await det.detect(img); res.forEach(item => { // Get label index console.log(item[0]); // Get label confidence console.log(item[1]); // Get label left coordinates console.log(item[2]); // Get label top coordinates console.log(item[3]); // Get label right coordinates console.log(item[4]); // Get label bottom coordinates console.log(item[5]); }); ``` -------------------------------- ### CSS Styling for Paddle.js Demos Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-models/facedetect/demo/index.html This CSS code provides styling for the Paddle.js demo pages. It defines styles for navigation elements, demo cards, information boxes, and a loading indicator, ensuring a consistent and user-friendly presentation. ```css .demo-nav { background-color: #2932e1; color: #fff; } .paddle-demo-card { margin-right: 0; margin-left: 0; background-color: #fff; } .infobox { background-color: #fff; border-radius: 6px; margin-bottom: 30px; padding: 20px; position: relative; min-height: 350px; box-sizing: border-box; color: #707070; } #loading { position: fixed; top: 0; left: 0; z-index: 10; width: 100vw; height: 100vh; line-height: 100vh; background-color: rgba(0, 0, 0, .4); color: #fff; text-align: center; font-size: 16px; } #resCanvas { margin-top: 2vw; } ``` -------------------------------- ### Generate and Process Data with @paddlejs-mediapipe/data-processor (JavaScript) Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-mediapipe/data-processor/README.md Demonstrates how to import and use data processing functions from the @paddlejs-mediapipe/data-processor package in JavaScript. It covers generating feed data with specified options and performing array operations using numjs, such as reshaping, transposing, and flattening. ```javascript import { genFeedData, nj } from '@paddlejs-mediapipe/data-processor'; const options = { targetShape: [1, 3, 224, 224], // required mean: [0.5, 0.5, 0.5], std: [1, 1, 1], colorType: 0, // 0: rgb; 1: bgr; default 0 normalizeType: 0 // data normalize type: 0: 0~1; 1:-1~1; default 0 }; // Array data that the length should match the targetShape in options. const data = Array.from(new Array(3 * 224 * 224), () => 244); const feedData = genFeedData(data, options); // use numjs const numData = nj.array([1, 2, 3, 4], 'float32'); const reshapedData = nj.reshape(numData, [2, 2]); const transposedData = nj.transpose(1, 0); const flattenedData = transposedData.flatten(); ``` -------------------------------- ### Load and Classify Images with Paddle.js MobileNet Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-examples/mobilenet/README.md Demonstrates how to load a MobileNet model and classify an image using Paddle.js. Requires specifying model path, mean/std parameters, and a classification map. Outputs the classification results for the input image. ```javascript import * as mobilenet from '@paddlejs-models/mobilenet'; // You need to specify your model path and the binary file count // If your has mean and std params, you need to specify them. // map is the results your model can classify. await mobilenet.load({ path, mean: [0.485, 0.456, 0.406], std: [0.229, 0.224, 0.225] }, map); // get the result the mobilenet model classified. const res = await mobilenet.classify(img); ``` -------------------------------- ### Load Converted Model with Paddle.js Source: https://context7.com/paddlepaddle/paddle.js/llms.txt Loads a converted model for use in a web application. It initializes a Runner with model path and configuration, then performs inference on an image element. Dependencies include '@paddlejs/paddlejs-core' and '@paddlejs/paddlejs-backend-webgl'. ```typescript import { Runner } from '@paddlejs/paddlejs-core'; import '@paddlejs/paddlejs-backend-webgl'; const runner = new Runner({ modelPath: 'https://example.com/models/output/model.json', feedShape: { fw: 224, fh: 224 }, mean: [0.485, 0.456, 0.406], std: [0.229, 0.224, 0.225], needPreheat: true }); await runner.init(); const result = await runner.predict(imageElement); ``` -------------------------------- ### Main Wrapper and Layout Structure - CSS Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-examples/ocr/index.html Defines the primary container layout using flexbox with responsive wrapping. Sets up relative positioning, text alignment, and minimum height constraints for the main demo interface. ```css .wrapper { position: relative; display: flex; text-align: left; padding: 2%; flex-wrap: wrap; min-height: 500px; } ``` -------------------------------- ### Flexbox Display Utility CSS Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-mediapipe/data-processor/demo/index.html Simple CSS utility class that applies flexbox display property to any element. The .flex class enables flexible box layout for quick responsive positioning and alignment of child elements. ```css .flex { display: flex; } ``` -------------------------------- ### Loading Overlay Component - CSS Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-examples/ocr/index.html Creates a full-screen semi-transparent loading overlay with centered text. Uses fixed positioning and viewport units to cover entire screen with semi-opaque dark background. ```css #isLoading { position: fixed; top: 0; left: 0; right: 0; bottom: 0; width: 100vw; height: 100vh; background-color: rgba(0, 0, 0, .5); } #isLoading .loading-text { color: #fff; font-size: 24px; text-align: center; line-height: 100vh; } ``` -------------------------------- ### Enable WebGL Pack Channel Performance Flag Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-core/README.md Activates the webgl_pack_channel flag to enable packing shader transformations for eligible conv2d operations. This optimization improves performance through vectorization calculations by packing multiple values into single operations. ```javascript env.set('webgl_pack_channel', true); ``` -------------------------------- ### Generate feed data with genFeedData in JavaScript Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-mediapipe/data-processor/README_cn.md Demonstrates how to use the genFeedData function to process image data with specified target shape, normalization, and color type options. The function accepts raw data array and configuration options to produce processed feed data compatible with Paddle.js models. ```javascript import { genFeedData } from '@paddlejs-mediapipe/data-processor'; const options = { targetShape: [1, 3, 224, 224], // 必选 mean: [0.5, 0.5, 0.5], // 可选 std: [1, 1, 1], // 可选 colorType: 0, // 0: rgb; 1: bgr; default 0 normalizeType: 0 // data normalize type: 0: 0~1; 1:-1~1; default 0 }; // 长度为与targetShape一致的数组 const data = Array.from(new Array(3 * 224 * 224), () => 244); const feedData = genFeedData(data, options); ``` -------------------------------- ### Load and Use HumanSeg Model in JavaScript Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-examples/humanseg/README.md This JavaScript code illustrates how to load and use the HumanSeg model. It shows loading with default parameters, faster loading with slight precision loss, and then demonstrates the usage of `drawHumanSeg` for segmented background drawing, `drawHumanSeg` for blurred backgrounds, and `drawMask` for extracting the background. ```javascript import * as humanseg from '@paddlejs-models/humanseg/lib/index_gpu'; // load humanseg model, use 398x224 shape model, and preheat await humanseg.load(); // use 288x160 shape model, preheat and predict faster with a little loss of precision // await humanseg.load(true, true); // background canvas const back_canvas = document.getElementById('background') as HTMLCanvasElement; // draw human segmentation const canvas1 = document.getElementById('back') as HTMLCanvasElement; await humanseg.drawHumanSeg(input, canvas1, back_canvas) ; // blur background const canvas2 = document.getElementById('blur') as HTMLCanvasElement; await humanseg.drawHumanSeg(input, canvas2) ; // draw the mask with background const canvas3 = document.getElementById('mask') as HTMLCanvasElement; await humanseg.drawMask(input, canvas3, back_canvas); ``` -------------------------------- ### Paddle.js MobileNet Demo - Image Classification Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-models/facedetect/demo/index.html This snippet relates to the MobileNet demo for Paddle.js, likely involved in image classification. It requires the Paddle.js library and potentially a pre-trained MobileNet model. The output would be the classification result of an input image. ```javascript // This section likely contains JavaScript code for initializing and running the MobileNet model // on user-provided images. It would involve loading the model, processing the image, // and displaying the classification results. ``` -------------------------------- ### Array manipulation with numjs reshape, transpose, and flatten Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-mediapipe/data-processor/README_cn.md Shows how to use the numjs library for advanced array operations including creating typed arrays, reshaping dimensions, transposing axes, and flattening multi-dimensional arrays. These operations are useful for preparing data in the correct format for neural network models. ```javascript // 使用 numjs const numData = nj.array([1, 2, 3, 4], 'float32'); const reshapedData = nj.reshape(numData, [2, 2]); const transposedData = nj.transpose(1, 0); const flattenedData = transposedData.flatten(); ``` -------------------------------- ### Configure Paddle.js Performance Environment Variables Source: https://context7.com/paddlepaddle/paddle.js/llms.txt Sets environment variables to optimize inference speed and memory usage for the Paddle.js runner, particularly for WebGL backend. It configures texture packing, half-float precision, GPU pipeline, and maximum texture size before initializing the runner. ```typescript import { Runner, env } from '@paddlejs/paddlejs-core'; import '@paddlejs/paddlejs-backend-webgl'; env.set('webgl_pack_channel', true); env.set('webgl_force_half_float_texture', true); env.set('webgl_gpu_pipeline', true); env.set('webgl_pack_output', true); env.set('MAX_TEXTURE_SIZE', 2048); const runner = new Runner({ modelPath: '/model/path', feedShape: { fw: 224, fh: 224 }, webglFeedProcess: true }); await runner.init(); const result = await runner.predict(imageElement); ``` -------------------------------- ### Paddle.js FaceDetect Demo - Face Detection Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-models/facedetect/demo/index.html This snippet is associated with the FaceDetect demo of Paddle.js, which performs real-time face detection. It would utilize the Paddle.js library and a face detection model. The input is typically a video stream or an image, and the output highlights detected faces. ```javascript // This section likely contains JavaScript code for initializing and running the face detection // model. It would handle video stream access or image input, process frames/images, // and draw bounding boxes around detected faces on a canvas. ``` -------------------------------- ### Basic HumanSeg Usage in JavaScript Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-models/humanseg/README.md Demonstrates how to load the HumanSeg model, predict human segmentation from an image, and utilize various drawing APIs. Supports different model shapes and includes options for preheating. ```javascript import * as humanseg from '@paddlejs-models/humanseg'; // load humanseg model, use 398x224 shape model, and preheat await humanseg.load(); // use 288x160 shape model, preheat and predict faster with a little loss of precision // await humanseg.load(true, true); // get the gray value [2, 398, 224] or [2, 288, 160]; const { data } = await humanseg.getGrayValue(img); // background canvas const back_canvas = document.getElementById('background') as HTMLCanvasElement; // draw human segmentation const canvas1 = document.getElementById('back') as HTMLCanvasElement; humanseg.drawHumanSeg(data, canvas1, back_canvas) ; // blur background const canvas2 = document.getElementById('blur') as HTMLCanvasElement; humanseg.drawHumanSeg(data, canvas2) ; // draw the mask with background const canvas3 = document.getElementById('mask') as HTMLCanvasElement; humanseg.drawMask(data, canvas3, back_canvas); ``` -------------------------------- ### Enable WebGL Half Float Texture Performance Flag Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-core/README.md Activates the webgl_force_half_float_texture flag to use half float (HALF_FLOAT) precision for feature maps instead of full float. This reduces memory usage and bandwidth requirements while maintaining acceptable accuracy for inference. ```javascript env.set('webgl_force_half_float_texture', true); ``` -------------------------------- ### HumanSeg GPU Pipeline Usage in JavaScript Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-models/humanseg/README.md Illustrates how to use the HumanSeg model with its GPU pipeline for potentially faster inference. It involves importing the GPU-specific module and using similar APIs for segmentation and drawing. ```javascript // 引入 humanseg sdk import * as humanseg from '@paddlejs-models/humanseg/lib/index_gpu'; // load humanseg model, use 398x224 shape model, and preheat await humanseg.load(); // use 288x160 shape model, preheat and predict faster with a little loss of precision // await humanseg.load(true, true); // background canvas const back_canvas = document.getElementById('background') as HTMLCanvasElement; // draw human segmentation const canvas1 = document.getElementById('back') as HTMLCanvasElement; await humanseg.drawHumanSeg(input, canvas1, back_canvas) ; // blur background const canvas2 = document.getElementById('blur') as HTMLCanvasElement; await humanseg.drawHumanSeg(input, canvas2) ; // draw the mask with background const canvas3 = document.getElementById('mask') as HTMLCanvasElement; await humanseg.drawMask(input, canvas3, back_canvas); ``` -------------------------------- ### WebWorker Integration for Paddle.js Inference Source: https://context7.com/paddlepaddle/paddle.js/llms.txt Demonstrates how to run Paddle.js inference in a background WebWorker to prevent blocking the main thread. Includes code for both the main thread (index.ts) and the worker thread (worker.ts), handling model initialization and prediction. ```typescript // Main thread: index.ts import Worker from './worker'; const worker = new Worker(); const canvas = document.getElementById('canvas'); const offscreen = canvas.transferControlToOffscreen(); worker.postMessage({ event: 'init', data: { canvas: offscreen, modelPath: 'https://paddlejs.cdn.bcebos.com/models/mobileNetV2Opt/model.json', feedShape: { fw: 224, fh: 224 }, mean: [0.485, 0.456, 0.406], std: [0.229, 0.224, 0.225] } }, [offscreen]); worker.addEventListener('message', (msg) => { if (msg.data.event === 'init') { console.log('Worker initialized'); } else if (msg.data.event === 'predict') { console.log('Prediction result:', msg.data.data); } }); const image = document.getElementById('photo'); createImageBitmap(image, 0, 0, image.naturalWidth, image.naturalHeight) .then(imageBitmap => { worker.postMessage({ event: 'predict', data: imageBitmap }, [imageBitmap]); }); // Worker thread: worker.ts import { Runner, env } from '@paddlejs/paddlejs-core'; import { GLHelper } from '@paddlejs/paddlejs-backend-webgl'; const webWorker: Worker = self as any; let runner: Runner; const WEBGL_ATTRIBUTES = { alpha: false, antialias: false, premultipliedAlpha: false, preserveDrawingBuffer: false, depth: false, stencil: false, failIfMajorPerformanceCaveat: true }; webWorker.addEventListener('message', async (msg) => { const { event, data } = msg.data; if (event === 'init') { const offscreenCanvasFor2D = new OffscreenCanvas(1, 1); env.set('canvas2d', offscreenCanvasFor2D); const gl = data.canvas.getContext('webgl2', WEBGL_ATTRIBUTES); GLHelper.setWebGLRenderingContext(gl); runner = new Runner({ modelPath: data.modelPath, feedShape: data.feedShape, mean: data.mean, std: data.std }); await runner.init(); webWorker.postMessage({ event: 'init' }); } else if (event === 'predict') { const result = await runner.predict(data); webWorker.postMessage({ event: 'predict', data: result }); } }); ``` -------------------------------- ### Video Controls Flexbox Layout CSS Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-mediapipe/data-processor/demo/index.html CSS styling for the video tool control section using flexbox to distribute buttons evenly across the container. The #video-tool element uses space-around justification to create equal spacing between control buttons. ```css #video-tool { display: flex; justify-content: space-around; } ``` -------------------------------- ### Title and Section Styling - CSS Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-examples/ocr/index.html Provides styling for heading elements within the demo, including font weight, size, and bottom margin spacing to separate title from content below. ```css .title { font-weight: 600; font-size: 20px; margin-bottom: 10px; } ``` -------------------------------- ### Post Message with Transferable ArrayBuffer in JavaScript Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-examples/webglWorker/README.md Demonstrates transferring an ArrayBuffer to a WebWorker using `postMessage`. This method is efficient for large data as it transfers ownership rather than copying, improving performance. The second argument to `postMessage` specifies the transferable objects. ```javascript worker.postMessage(arrayBuffer, [arrayBuffer]); ``` -------------------------------- ### Load Model from In-Memory Objects Source: https://context7.com/paddlepaddle/paddle.js/llms.txt Illustrates loading a Paddle.js model directly from JavaScript objects (model JSON and parameters) rather than fetching them from a URL. This is useful for offline applications or when models are bundled with the application code. ```typescript import { Runner } from '@paddlejs/paddlejs-core'; import '@paddlejs/paddlejs-backend-webgl'; import modelJson from './model.json'; import modelParams from './params.bin'; const runner = new Runner({ modelObj: { model: modelJson, params: new Float32Array(modelParams) }, feedShape: { fw: 224, fh: 224 }, mean: [0.485, 0.456, 0.406], std: [0.229, 0.224, 0.225] }); await runner.init(); const result = await runner.predict(imageElement); ``` -------------------------------- ### Predict with Raw Numerical Data or ImageData Source: https://context7.com/paddlepaddle/paddle.js/llms.txt Shows how to perform model prediction using raw numerical data (e.g., Float32Array) or `ImageData` objects, bypassing the need for media elements like `` or `