### Install Face Decoration Example Dependencies
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-examples/faceDecoration/README.md
Navigate to the faceDecoration example directory within the cloned Paddle.js repository and install the necessary npm dependencies to run the face decoration examples.
```sh
cd Paddle.js/packages/paddlejs-examples/faceDecoration && npm install
```
--------------------------------
### Install and Run Gesture Demo
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-models/gesture/README.md
Instructions to install dependencies and run the gesture recognition demo locally. This involves using npm to install packages and then starting the development server. The demo can then be accessed via a local URL.
```bash
npm install
npm run dev
```
--------------------------------
### Install @paddlejs-mediapipe/camera Package
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-mediapipe/camera/README.md
This snippet shows the command to install the @paddlejs-mediapipe/camera package using npm. This is the first step to using the camera functionality in your project.
```bash
npm install @paddlejs-mediapipe/camera
```
--------------------------------
### Initialize Runner with WebGL Backend (Simplified)
Source: https://context7.com/paddlepaddle/paddle.js/llms.txt
A simplified example of initializing the `Runner` with the WebGL backend, focusing on model path, feed shape, fill color, and WebGL specific processing. This setup is suitable for scenarios involving video elements for prediction.
```typescript
import { Runner } from '@paddlejs/paddlejs-core';
import '@paddlejs/paddlejs-backend-webgl';
const runner = new Runner({
modelPath: '/model/mobilenetv2',
feedShape: { fw: 256, fh: 256 },
fill: '#fff',
webglFeedProcess: true
});
await runner.init();
const result = await runner.predict(videoElement);
```
--------------------------------
### Initialize and Control Camera with JavaScript
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-mediapipe/camera/README.md
Demonstrates how to import and initialize the Camera class from the @paddlejs-mediapipe/camera package. It includes an example of configuring camera options, starting, pausing, and switching cameras. The `onSuccess`, `onError`, and `onFrame` callbacks are crucial for handling video stream events.
```javascript
import Camera from '@paddlejs-mediapipe/camera';
const option = {
// video width
width?: number,
// video height
height?: number,
// mirror or not
mirror?: boolean,
// if enable capture stream when page hidden, default is false
enableOnInactiveState?: boolean,
// canvas DOM
targetCanvas?: HTMLCanvasElement,
// video rendered successfully
onSuccess?: () => void,
// video rendering failed
onError?: NavigatorUserMediaErrorCallback,
// browser does not support the getusermedia API
onNotSupported?: () => void,
// get every frame of video
onFrame?: (canvas: HTMLCanvasElement) => void,
// switch camera error
switchError?: () => void,
// video loadedData
videoLoaded?: () => void
};
const camera = new Camera(video, option);
// video play
camera.start();
// video pause
camera.pause();
// cameras switch
camera.switchCameras();
```
--------------------------------
### Clone Paddle.js Repository
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-examples/faceDecoration/README.md
This command clones the Paddle.js GitHub repository to your local machine, allowing you to access its various packages and examples.
```sh
git clone https://github.com/PaddlePaddle/Paddle.js.git
```
--------------------------------
### Install @paddlejs-mediapipe/data-processor
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-mediapipe/data-processor/README.md
Installs the @paddlejs-mediapipe/data-processor npm package using npm. This is the initial step before using its data processing capabilities.
```bash
npm install @paddlejs-mediapipe/data-processor
```
--------------------------------
### Install and Build HumanSeg with npm
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-examples/humanseg/README.md
This snippet demonstrates the npm commands required to set up and build the HumanSeg project. `npm install` installs dependencies, while `npm run dev` compiles for development with hot-reloading, and `npm run build` compiles for production.
```bash
npm install
## Compiles and hot-reloads for development
npm run dev
## Compiles and minifies for production
npm run build
```
--------------------------------
### Install npm dependencies for Paddle.js
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-examples/gesture/README.md
Installs the necessary dependencies for the Paddle.js project using npm. This command should be run in the project's root directory.
```bash
npm install
```
--------------------------------
### Run Development Server using npm
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-examples/webglWorker/README.md
This command starts the development server, compiles the project, and makes it available at the specified URL. It's typically used for local development and testing. Visit `http://0.0.0.0:8866/` in your browser to view the application.
```bash
npm run dev
# visit http://0.0.0.0:8866/
```
--------------------------------
### Compile and hot-reload Paddle.js for development
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-examples/gesture/README.md
Starts the development server, enabling live-reloading for rapid development cycles. This command is typically used during the development phase of the project.
```bash
npm run dev
```
--------------------------------
### Initialize and Use OCR Module in JavaScript
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-examples/ocr/README.md
Demonstrates how to initialize the OCR module and get text recognition results from an image. It shows how to import the module, initialize it, and call the recognize function, with options for canvas drawing and styling.
```javascript
import * as ocr from '@paddlejs-models/ocr';
// Model initialization
await ocr.init();
// Get the text recognition result API, img is the user's upload picture, and option is an optional parameter
// option.canvas as HTMLElementCanvas:if the user needs to draw the selected area of the text box, pass in the canvas element
// option.style as object:if the user needs to configure the canvas style, pass in the style object
// option.style.strokeStyle as string:select a color for the text box
// option.style.lineWidth as number:width of selected line segment in text box
// option.style.fillStyle as string:select the fill color for the text box
const res = await ocr.recognize(img, option?);
// character recognition results
console.log(res.text);
// text area points
console.log(res.points);
```
--------------------------------
### Import WebGL Backend via NPM
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-backend-webgl/README.md
This snippet demonstrates how to import the registered WebGL backend for Paddle.js using NPM. Ensure you have the necessary package installed.
```javascript
import '@paddlejs/paddlejs-backend-webgl';
```
--------------------------------
### Install OpenCV npm Package
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-mediapipe/opencv/README.md
Instructions to install the OpenCV npm package for use with Paddle.js. This package provides access to various OpenCV modules.
```bash
npm install @paddlejs-mediapipe/opencv
```
--------------------------------
### Include WebGL Backend via Script Tag
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-backend-webgl/README.md
This snippet shows how to include the Paddle.js WebGL backend using a script tag, referencing a CDN. This is an alternative to NPM installation.
```html
```
--------------------------------
### Set and Get Environment Variables in Paddle.js
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-core/README.md
Demonstrates how to use the global env module to set and retrieve environment variables in Paddle.js. The env.set() method stores key-value pairs, while env.get() retrieves values by key. This is essential for configuring performance flags and other runtime settings.
```javascript
// set env key/flag and value
env.set(key, value);
// get value by key/flag
env.get(key);
```
--------------------------------
### Import NodeGL Backend in Node.js
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-backend-nodegl/README.md
This code snippet demonstrates how to import the registered NodeGL backend for Paddle.js. Ensure the '@paddlejs/paddlejs-backend-nodegl' package is installed via npm before using this import statement.
```javascript
import '@paddlejs/paddlejs-backend-nodegl';
```
--------------------------------
### Apply Gaussian Blur using OpenCV.js
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-mediapipe/opencv/README.md
Example demonstrating how to apply a Gaussian blur effect to an image using the OpenCV.js library imported from the Paddle.js package. It involves reading an image from a canvas, creating a destination matrix, defining kernel size and anchor points, and then applying the blur operation.
```javascript
import cv from '@paddlejs-mediapipe/opencv/library/opencv_blur';
let logit = cv.imread('canvas');
let dst = new cv.Mat();
let ksize = new cv.Size(5, 5);
let anchor = new cv.Point(-1, -1);
cv.blur(logit, dst, ksize, anchor, cv.BORDER_DEFAULT);
```
--------------------------------
### Initialize Runner with WebGPU Backend
Source: https://context7.com/paddlepaddle/paddle.js/llms.txt
Shows how to initialize the `Runner` for inference using the experimental WebGPU backend, which offers next-generation GPU acceleration. This requires a compatible browser environment (e.g., Chrome Canary with WebGPU enabled).
```typescript
import { Runner } from '@paddlejs/paddlejs-core';
import '@paddlejs/paddlejs-backend-webgpu';
const runner = new Runner({
modelPath: 'https://example.com/model',
feedShape: { fw: 224, fh: 224 }
});
await runner.init();
const result = await runner.predict(imageElement);
```
--------------------------------
### Paddle.js Core Initialization and Usage
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-core/README.md
This section details how to initialize and use the Paddle.js Core runner. It covers importing necessary packages, configuring the runner with `RunnerConfig`, and performing predictions.
```APIDOC
## Paddle.js Core Initialization and Usage
### Description
This section details how to initialize and use the Paddle.js Core runner. It covers importing necessary packages, configuring the runner with `RunnerConfig`, and performing predictions.
### Method
```typescript
import { Runner } from '@paddlejs/paddlejs-core';
import '@paddlejs/paddlejs-backend-webgl'; // Example backend import
// Runner configuration interface
interface RunnerConfig {
modelPath: string; // model path (local or web address)
modelName?: string; // model name
feedShape: {
fc?: number; // feed channel, default is 3.
fw: number; // feed width
fh: number; // feed height
};
fill?: Color; // the color used to padding
mean?: number[]; // mean value
std?: number[]; // standard deviation
bgr?: boolean; // whether the image channel alignment is BGR, default is false (RGB)
type?: GraphType; // model structure, default is singleInput and singleOutput
needPreheat?: boolean; // whether to warm up the engine during initialization, default is true
plugins?: {
preTransforms?: Transformer[];
transforms?: Transformer[];
postTransforms?: Transformer[];
};
webglFeedProcess?: boolean; // Turn on `webglFeedProcess` to convert all pre-processing parts of the model to shader processing, and keep the original image texture.
}
// Example usage
const runner = new Runner({
modelPath: '/model/mobilenetv2', // model path, e.g. http://xx.cc/path, http://xx.cc/path/model.json, /localModelDir/model.json, /localModelDir
feedShape: { // input shape
fw: 256,
fh: 256
},
fill: '#fff', // fill color when resize image, default value is #fff
webglFeedProcess: true
});
// init runner
await runner.init();
// predict and get result
const res = await runner.predict(mediadata, callback?);
```
### Parameters
No direct parameters for initialization, but `RunnerConfig` object is required.
### Request Example
```json
{
"modelPath": "/model/mobilenetv2",
"feedShape": {
"fw": 256,
"fh": 256
},
"fill": "#fff",
"webglFeedProcess": true
}
```
### Response
#### Success Response (200)
- **runner** (Runner) - An initialized Paddle.js runner instance.
- **res** (any) - The result of the prediction.
#### Response Example
```json
{
"predictionResult": "example"
}
```
### Notes
- If you are importing the Core package, you also need to import a backend (e.g., `@paddlejs/paddlejs-backend-webgl`, `@paddlejs/paddlejs-backend-webgpu`).
- The `RunnerConfig` interface defines the structure for configuring the engine. Key required fields include `modelPath` and `feedShape`.
```
--------------------------------
### Initialize Runner and Perform Model Prediction in JavaScript
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-core/README.md
Demonstrates how to import @paddlejs/paddlejs-core and a WebGL backend, instantiate the Runner class with configuration parameters, initialize the runner, and execute model predictions on media data. The webglFeedProcess option enables shader-based preprocessing while preserving the original image texture.
```javascript
// Import @paddlejs/paddlejs-core
import { Runner } from '@paddlejs/paddlejs-core';
// Import the registered WebGL backend.
import '@paddlejs/paddlejs-backend-webgl';
const runner = new Runner({
modelPath: '/model/mobilenetv2', // model path, e.g. http://xx.cc/path, http://xx.cc/path/model.json, /localModelDir/model.json, /localModelDir
feedShape: { // input shape
fw: 256,
fh: 256
},
fill?: '#fff', // fill color when resize image, default value is #fff
webglFeedProcess?: true // Turn on `webglFeedProcess` to convert all pre-processing parts of the model to shader processing, and keep the original image texture.
});
// init runner
await runner.init();
// predict and get result
const res = await runner.predict(mediadata, callback?);
```
--------------------------------
### Initialize Runner with WebGL Backend
Source: https://context7.com/paddlepaddle/paddle.js/llms.txt
Demonstrates initializing the `Runner` class for model inference using the WebGL backend. It specifies model path, input shape, normalization parameters, and other configuration options. The `init()` method prepares the model for execution, and `predict()` performs inference on an image element.
```typescript
import { Runner } from '@paddlejs/paddlejs-core';
import '@paddlejs/paddlejs-backend-webgl';
const runner = new Runner({
modelPath: 'https://example.com/model/model.json',
feedShape: {
fw: 224,
fh: 224,
fc: 3
},
mean: [0.485, 0.456, 0.406],
std: [0.229, 0.224, 0.225],
fill: '#fff',
bgr: false,
needPreheat: true,
webglFeedProcess: false,
keepRatio: true
});
await runner.init();
const imageElement = document.getElementById('myImage');
const result = await runner.predict(imageElement);
console.log(result);
```
--------------------------------
### Compile and minify Paddle.js for production
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-examples/gesture/README.md
Builds the project for production, generating minified and optimized code. This command should be used before deploying the application.
```bash
npm run build
```
--------------------------------
### Initialize Runner with WASM Backend
Source: https://context7.com/paddlepaddle/paddle.js/llms.txt
This snippet demonstrates initializing the `Runner` with the WebAssembly (WASM) backend for CPU-based inference. It includes specifying the model path, feed shape, and a WASM-specific memory type configuration.
```typescript
import { Runner } from '@paddlejs/paddlejs-core';
import '@paddlejs/paddlejs-backend-wasm';
const runner = new Runner({
modelPath: '/model/path',
feedShape: { fw: 224, fh: 224 },
wasmMemoryType: 'memory300'
});
await runner.init();
const result = await runner.predict(imageElement);
```
--------------------------------
### Convert PaddlePaddle Models to Browser Format
Source: https://context7.com/paddlepaddle/paddle.js/llms.txt
Converts PaddlePaddle inference models to a format suitable for web deployment using the paddlejsconverter Python package. Requires specifying input model and parameter paths, and an output directory. The output includes model.json and chunked data files.
```bash
pip install paddlejsconverter
paddlejsconverter \
--modelPath=/path/to/inference_model \
--paramPath=/path/to/inference_params \
--outputDir=./output \
--useGPUOpt=True
ls output/
# model.json chunk_1.dat chunk_2.dat chunk_3.dat
```
--------------------------------
### Load Mobilenet Model and Classify Image with Paddle.js
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-examples/wine/README.md
Demonstrates how to load a mobilenet model and classify an image using the Paddle.js library. It requires the model path and optionally provides mean and standard deviation for normalization. The output is the classification result for the input image.
```javascript
import * as mobilenet from '@paddlejs-models/mobilenet';
// model load
const path = 'https://paddlejs.bj.bcebos.com/models/fuse/mobilenet/wine_fuse_activation/model.json';
await mobilenet.load({
path,
mean: [0.485, 0.456, 0.406],
std: [0.229, 0.224, 0.225]
}, map);
// get classification results
const res = await mobilenet.classify(img);
```
--------------------------------
### HTML Video and Control Elements
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-mediapipe/data-processor/demo/index.html
HTML markup structure for the Paddle web camera demo, including a video element for displaying the camera feed and control buttons for play/pause functionality and camera switching. Includes a loading indicator div with Chinese text.
```html
loading中……
```
--------------------------------
### Enable WebGL Pack Output Performance Flag
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-core/README.md
Activates the webgl_pack_output flag to migrate NHWC to NCHW layout transformation to the GPU and pack results into four-channel layout. This optimization reduces loop processing overhead when reading results from GPU memory.
```javascript
env.set('webgl_pack_output', true);
```
--------------------------------
### Convert Model for CPU/WASM Backend using PaddleJSConverter
Source: https://context7.com/paddlepaddle/paddle.js/llms.txt
Converts a Paddle model for CPU or WebAssembly backends by disabling GPU optimizations. This command-line tool requires the paths to the model and parameter files and specifies an output directory.
```bash
paddlejsconverter \
--modelPath=/path/to/inference_model \
--paramPath=/path/to/inference_params \
--outputDir=./output_cpu \
--useGPUOpt=False
```
--------------------------------
### Loading Overlay and Modal CSS
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-mediapipe/data-processor/demo/index.html
CSS styles for a full-screen loading overlay that covers the entire viewport with a semi-transparent dark background. Includes styling for loading text displayed in white with 24px font size, creating a modal loading indicator.
```css
#isLoading {
position: fixed;
top: 0;
left: 0;
right: 0;
bottom: 0;
background-color: rgba(0, 0, 0, .5);
}
#isLoading .loading-text {
color: #fff;
font-size: 24px;
}
```
--------------------------------
### Load and classify gestures using Paddle.js in JavaScript
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-examples/gesture/README.md
Demonstrates how to use the Paddle.js gesture recognition module. It involves loading both the gesture detection and recognition models, and then classifying gestures from an input image. The output includes palm frame coordinates and recognition results.
```javascript
import * as gesture from '@paddlejs-models/gesture';
// Load gesture_detect model and gesture_rec model
await gesture.load();
// Get the image recognition results. The results include: palm frame coordinates and recognition results
const res = await gesture.classify(img);
```
--------------------------------
### Enable WebGL GPU Pipeline Performance Flag
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-core/README.md
Activates the webgl_gpu_pipeline flag to convert model pre-processing to shader processing and render results to WebGL2RenderingContext. This enables GPU-accelerated pipeline combining pre-processing, inference, and post-processing for maximum performance, with output textures available for custom post-processing operations.
```javascript
env.set('webgl_gpu_pipeline', true);
```
--------------------------------
### Initialize and Detect Faces with Paddle.js Facedetect
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-models/facedetect/README.md
Demonstrates how to import and initialize the FaceDetector from the '@paddlejs-models/facedetect' package. It shows the process of detecting faces in an image element, specifying optional parameters like 'shrink' and 'threshold', and explains the structure of the returned face area information (left, top, width, height, confidence).
```javascript
import { FaceDetector } from '@paddlejs-models/facedetect';
const faceDetector = new FaceDetector();
await faceDetector.init();
// Required parameter:imgEle(HTMLImageElement)
// Optional parameter: shrink, threshold
// Result is face area information. It includes left, top, width, height, confidence
const res = await faceDetector.detect(
imgEle,
{ shrink: 0.4, threshold: 0.6 }
);
```
--------------------------------
### CSS Layout and Positioning - Gesture Demo Interface
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-models/gesture/demo/index.html
Comprehensive CSS styling for a full-screen gesture demonstration interface. Includes body layout, canvas positioning, loading overlay, and responsive centering using flexbox and CSS transforms. The styling creates a fixed full-screen container with absolute positioning for interactive elements.
```css
body {
margin: 0 auto;
position: fixed;
width: 100%;
height: 100%;
}
#rect {
position: absolute;
top: 0;
left: 0;
width: 100px;
height: 100px;
}
.wrapper {
position: relative;
width: 100%;
height: 100%;
margin: 0 auto;
}
#isLoading {
position: fixed;
top: 0;
left: 0;
right: 0;
bottom: 0;
background-color: rgba(0, 0, 0, .5);
}
#isLoading .loading-text {
color: #fff;
font-size: 24px;
}
.center {
position: absolute;
top: 50%;
left: 50%;
-webkit-transform: translate(-50%, -50%);
transform: translate(-50%, -50%);
}
#canvas {
position: absolute;
left: 0;
top: 0;
width: 256px;
}
```
--------------------------------
### GPU Pipeline with Custom Post-Processing in Paddle.js
Source: https://context7.com/paddlepaddle/paddle.js/llms.txt
Implements end-to-end GPU processing with custom rendering on output textures. It configures WebGL rendering context, sets environment variables for GPU pipeline, initializes a Runner, and processes video input, drawing the output to a canvas. Dependencies include '@paddlejs/paddlejs-core' and '@paddlejs/paddlejs-backend-webgl'.
```typescript
import { Runner, env } from '@paddlejs/paddlejs-core';
import { GLHelper } from '@paddlejs/paddlejs-backend-webgl';
import '@paddlejs/paddlejs-backend-webgl';
const canvas = document.createElement('canvas');
canvas.width = 640;
canvas.height = 480;
const gl = canvas.getContext('webgl2', {
alpha: true,
antialias: false,
premultipliedAlpha: false,
preserveDrawingBuffer: true,
depth: false,
stencil: false
});
GLHelper.setWebGLRenderingContext(gl);
env.set('webgl_pack_channel', true);
env.set('webgl_gpu_pipeline', true);
env.set('webgl_force_half_float_texture', true);
env.set('webgl_pack_output', true);
const runner = new Runner({
modelPath: '/model/path',
feedShape: { fw: 640, fh: 480 },
webglFeedProcess: true
});
await runner.init();
const videoElement = document.getElementById('webcam');
await runner.predict(videoElement);
const outputCanvas = document.getElementById('output');
const ctx = outputCanvas.getContext('2d');
ctx.drawImage(gl.canvas, 0, 0, outputCanvas.width, outputCanvas.height);
```
--------------------------------
### OCR Detect Model Usage in JavaScript
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-examples/ocrdetection/README.md
Demonstrates how to use the ocr_detect model from the @paddlejs-models/ocrdet package. It shows the steps to load the model and then use it to detect text areas within an image.
```javascript
import * as ocr from '@paddlejs-models/ocrdet';
// Load ocr_detect model
await ocr.load();
// Get text area points
const res = await ocr.detect(img);
```
--------------------------------
### Full-Screen Video Layout CSS
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-mediapipe/data-processor/demo/index.html
CSS stylesheet that establishes a full-screen video container with fixed positioning, ensuring the video element stretches to fill the entire viewport. This provides a responsive base layout for the camera demo application with zero margins.
```css
body {
margin: 0 auto;
position: fixed;
width: 100%;
height: 100%;
}
video {
width: 100%;
}
```
--------------------------------
### Load and Run Detection with PaddleJS Detect Model
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-models/detect/README.md
This snippet demonstrates how to import the PaddleJS detect model, load the pre-trained model, and perform object detection on an image. The detect function returns an array of results where each item contains the label index, confidence score, and bounding box coordinates (left, top, right, bottom) for detected objects.
```javascript
import * as det from '@paddlejs-models/detect';
// Load model
await det.load();
// Get label index, confidence and coordinates
const res = await det.detect(img);
res.forEach(item => {
// Get label index
console.log(item[0]);
// Get label confidence
console.log(item[1]);
// Get label left coordinates
console.log(item[2]);
// Get label top coordinates
console.log(item[3]);
// Get label right coordinates
console.log(item[4]);
// Get label bottom coordinates
console.log(item[5]);
});
```
--------------------------------
### CSS Styling for Paddle.js Demos
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-models/facedetect/demo/index.html
This CSS code provides styling for the Paddle.js demo pages. It defines styles for navigation elements, demo cards, information boxes, and a loading indicator, ensuring a consistent and user-friendly presentation.
```css
.demo-nav {
background-color: #2932e1;
color: #fff;
}
.paddle-demo-card {
margin-right: 0;
margin-left: 0;
background-color: #fff;
}
.infobox {
background-color: #fff;
border-radius: 6px;
margin-bottom: 30px;
padding: 20px;
position: relative;
min-height: 350px;
box-sizing: border-box;
color: #707070;
}
#loading {
position: fixed;
top: 0;
left: 0;
z-index: 10;
width: 100vw;
height: 100vh;
line-height: 100vh;
background-color: rgba(0, 0, 0, .4);
color: #fff;
text-align: center;
font-size: 16px;
}
#resCanvas {
margin-top: 2vw;
}
```
--------------------------------
### Generate and Process Data with @paddlejs-mediapipe/data-processor (JavaScript)
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-mediapipe/data-processor/README.md
Demonstrates how to import and use data processing functions from the @paddlejs-mediapipe/data-processor package in JavaScript. It covers generating feed data with specified options and performing array operations using numjs, such as reshaping, transposing, and flattening.
```javascript
import { genFeedData, nj } from '@paddlejs-mediapipe/data-processor';
const options = {
targetShape: [1, 3, 224, 224], // required
mean: [0.5, 0.5, 0.5],
std: [1, 1, 1],
colorType: 0, // 0: rgb; 1: bgr; default 0
normalizeType: 0 // data normalize type: 0: 0~1; 1:-1~1; default 0
};
// Array data that the length should match the targetShape in options.
const data = Array.from(new Array(3 * 224 * 224), () => 244);
const feedData = genFeedData(data, options);
// use numjs
const numData = nj.array([1, 2, 3, 4], 'float32');
const reshapedData = nj.reshape(numData, [2, 2]);
const transposedData = nj.transpose(1, 0);
const flattenedData = transposedData.flatten();
```
--------------------------------
### Load and Classify Images with Paddle.js MobileNet
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-examples/mobilenet/README.md
Demonstrates how to load a MobileNet model and classify an image using Paddle.js. Requires specifying model path, mean/std parameters, and a classification map. Outputs the classification results for the input image.
```javascript
import * as mobilenet from '@paddlejs-models/mobilenet';
// You need to specify your model path and the binary file count
// If your has mean and std params, you need to specify them.
// map is the results your model can classify.
await mobilenet.load({
path,
mean: [0.485, 0.456, 0.406],
std: [0.229, 0.224, 0.225]
}, map);
// get the result the mobilenet model classified.
const res = await mobilenet.classify(img);
```
--------------------------------
### Load Converted Model with Paddle.js
Source: https://context7.com/paddlepaddle/paddle.js/llms.txt
Loads a converted model for use in a web application. It initializes a Runner with model path and configuration, then performs inference on an image element. Dependencies include '@paddlejs/paddlejs-core' and '@paddlejs/paddlejs-backend-webgl'.
```typescript
import { Runner } from '@paddlejs/paddlejs-core';
import '@paddlejs/paddlejs-backend-webgl';
const runner = new Runner({
modelPath: 'https://example.com/models/output/model.json',
feedShape: { fw: 224, fh: 224 },
mean: [0.485, 0.456, 0.406],
std: [0.229, 0.224, 0.225],
needPreheat: true
});
await runner.init();
const result = await runner.predict(imageElement);
```
--------------------------------
### Main Wrapper and Layout Structure - CSS
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-examples/ocr/index.html
Defines the primary container layout using flexbox with responsive wrapping. Sets up relative positioning, text alignment, and minimum height constraints for the main demo interface.
```css
.wrapper {
position: relative;
display: flex;
text-align: left;
padding: 2%;
flex-wrap: wrap;
min-height: 500px;
}
```
--------------------------------
### Flexbox Display Utility CSS
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-mediapipe/data-processor/demo/index.html
Simple CSS utility class that applies flexbox display property to any element. The .flex class enables flexible box layout for quick responsive positioning and alignment of child elements.
```css
.flex {
display: flex;
}
```
--------------------------------
### Loading Overlay Component - CSS
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-examples/ocr/index.html
Creates a full-screen semi-transparent loading overlay with centered text. Uses fixed positioning and viewport units to cover entire screen with semi-opaque dark background.
```css
#isLoading {
position: fixed;
top: 0;
left: 0;
right: 0;
bottom: 0;
width: 100vw;
height: 100vh;
background-color: rgba(0, 0, 0, .5);
}
#isLoading .loading-text {
color: #fff;
font-size: 24px;
text-align: center;
line-height: 100vh;
}
```
--------------------------------
### Enable WebGL Pack Channel Performance Flag
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-core/README.md
Activates the webgl_pack_channel flag to enable packing shader transformations for eligible conv2d operations. This optimization improves performance through vectorization calculations by packing multiple values into single operations.
```javascript
env.set('webgl_pack_channel', true);
```
--------------------------------
### Generate feed data with genFeedData in JavaScript
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-mediapipe/data-processor/README_cn.md
Demonstrates how to use the genFeedData function to process image data with specified target shape, normalization, and color type options. The function accepts raw data array and configuration options to produce processed feed data compatible with Paddle.js models.
```javascript
import { genFeedData } from '@paddlejs-mediapipe/data-processor';
const options = {
targetShape: [1, 3, 224, 224], // 必选
mean: [0.5, 0.5, 0.5], // 可选
std: [1, 1, 1], // 可选
colorType: 0, // 0: rgb; 1: bgr; default 0
normalizeType: 0 // data normalize type: 0: 0~1; 1:-1~1; default 0
};
// 长度为与targetShape一致的数组
const data = Array.from(new Array(3 * 224 * 224), () => 244);
const feedData = genFeedData(data, options);
```
--------------------------------
### Load and Use HumanSeg Model in JavaScript
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-examples/humanseg/README.md
This JavaScript code illustrates how to load and use the HumanSeg model. It shows loading with default parameters, faster loading with slight precision loss, and then demonstrates the usage of `drawHumanSeg` for segmented background drawing, `drawHumanSeg` for blurred backgrounds, and `drawMask` for extracting the background.
```javascript
import * as humanseg from '@paddlejs-models/humanseg/lib/index_gpu';
// load humanseg model, use 398x224 shape model, and preheat
await humanseg.load();
// use 288x160 shape model, preheat and predict faster with a little loss of precision
// await humanseg.load(true, true);
// background canvas
const back_canvas = document.getElementById('background') as HTMLCanvasElement;
// draw human segmentation
const canvas1 = document.getElementById('back') as HTMLCanvasElement;
await humanseg.drawHumanSeg(input, canvas1, back_canvas) ;
// blur background
const canvas2 = document.getElementById('blur') as HTMLCanvasElement;
await humanseg.drawHumanSeg(input, canvas2) ;
// draw the mask with background
const canvas3 = document.getElementById('mask') as HTMLCanvasElement;
await humanseg.drawMask(input, canvas3, back_canvas);
```
--------------------------------
### Paddle.js MobileNet Demo - Image Classification
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-models/facedetect/demo/index.html
This snippet relates to the MobileNet demo for Paddle.js, likely involved in image classification. It requires the Paddle.js library and potentially a pre-trained MobileNet model. The output would be the classification result of an input image.
```javascript
// This section likely contains JavaScript code for initializing and running the MobileNet model
// on user-provided images. It would involve loading the model, processing the image,
// and displaying the classification results.
```
--------------------------------
### Array manipulation with numjs reshape, transpose, and flatten
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-mediapipe/data-processor/README_cn.md
Shows how to use the numjs library for advanced array operations including creating typed arrays, reshaping dimensions, transposing axes, and flattening multi-dimensional arrays. These operations are useful for preparing data in the correct format for neural network models.
```javascript
// 使用 numjs
const numData = nj.array([1, 2, 3, 4], 'float32');
const reshapedData = nj.reshape(numData, [2, 2]);
const transposedData = nj.transpose(1, 0);
const flattenedData = transposedData.flatten();
```
--------------------------------
### Configure Paddle.js Performance Environment Variables
Source: https://context7.com/paddlepaddle/paddle.js/llms.txt
Sets environment variables to optimize inference speed and memory usage for the Paddle.js runner, particularly for WebGL backend. It configures texture packing, half-float precision, GPU pipeline, and maximum texture size before initializing the runner.
```typescript
import { Runner, env } from '@paddlejs/paddlejs-core';
import '@paddlejs/paddlejs-backend-webgl';
env.set('webgl_pack_channel', true);
env.set('webgl_force_half_float_texture', true);
env.set('webgl_gpu_pipeline', true);
env.set('webgl_pack_output', true);
env.set('MAX_TEXTURE_SIZE', 2048);
const runner = new Runner({
modelPath: '/model/path',
feedShape: { fw: 224, fh: 224 },
webglFeedProcess: true
});
await runner.init();
const result = await runner.predict(imageElement);
```
--------------------------------
### Paddle.js FaceDetect Demo - Face Detection
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-models/facedetect/demo/index.html
This snippet is associated with the FaceDetect demo of Paddle.js, which performs real-time face detection. It would utilize the Paddle.js library and a face detection model. The input is typically a video stream or an image, and the output highlights detected faces.
```javascript
// This section likely contains JavaScript code for initializing and running the face detection
// model. It would handle video stream access or image input, process frames/images,
// and draw bounding boxes around detected faces on a canvas.
```
--------------------------------
### Basic HumanSeg Usage in JavaScript
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-models/humanseg/README.md
Demonstrates how to load the HumanSeg model, predict human segmentation from an image, and utilize various drawing APIs. Supports different model shapes and includes options for preheating.
```javascript
import * as humanseg from '@paddlejs-models/humanseg';
// load humanseg model, use 398x224 shape model, and preheat
await humanseg.load();
// use 288x160 shape model, preheat and predict faster with a little loss of precision
// await humanseg.load(true, true);
// get the gray value [2, 398, 224] or [2, 288, 160];
const { data } = await humanseg.getGrayValue(img);
// background canvas
const back_canvas = document.getElementById('background') as HTMLCanvasElement;
// draw human segmentation
const canvas1 = document.getElementById('back') as HTMLCanvasElement;
humanseg.drawHumanSeg(data, canvas1, back_canvas) ;
// blur background
const canvas2 = document.getElementById('blur') as HTMLCanvasElement;
humanseg.drawHumanSeg(data, canvas2) ;
// draw the mask with background
const canvas3 = document.getElementById('mask') as HTMLCanvasElement;
humanseg.drawMask(data, canvas3, back_canvas);
```
--------------------------------
### Enable WebGL Half Float Texture Performance Flag
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-core/README.md
Activates the webgl_force_half_float_texture flag to use half float (HALF_FLOAT) precision for feature maps instead of full float. This reduces memory usage and bandwidth requirements while maintaining acceptable accuracy for inference.
```javascript
env.set('webgl_force_half_float_texture', true);
```
--------------------------------
### HumanSeg GPU Pipeline Usage in JavaScript
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-models/humanseg/README.md
Illustrates how to use the HumanSeg model with its GPU pipeline for potentially faster inference. It involves importing the GPU-specific module and using similar APIs for segmentation and drawing.
```javascript
// 引入 humanseg sdk
import * as humanseg from '@paddlejs-models/humanseg/lib/index_gpu';
// load humanseg model, use 398x224 shape model, and preheat
await humanseg.load();
// use 288x160 shape model, preheat and predict faster with a little loss of precision
// await humanseg.load(true, true);
// background canvas
const back_canvas = document.getElementById('background') as HTMLCanvasElement;
// draw human segmentation
const canvas1 = document.getElementById('back') as HTMLCanvasElement;
await humanseg.drawHumanSeg(input, canvas1, back_canvas) ;
// blur background
const canvas2 = document.getElementById('blur') as HTMLCanvasElement;
await humanseg.drawHumanSeg(input, canvas2) ;
// draw the mask with background
const canvas3 = document.getElementById('mask') as HTMLCanvasElement;
await humanseg.drawMask(input, canvas3, back_canvas);
```
--------------------------------
### WebWorker Integration for Paddle.js Inference
Source: https://context7.com/paddlepaddle/paddle.js/llms.txt
Demonstrates how to run Paddle.js inference in a background WebWorker to prevent blocking the main thread. Includes code for both the main thread (index.ts) and the worker thread (worker.ts), handling model initialization and prediction.
```typescript
// Main thread: index.ts
import Worker from './worker';
const worker = new Worker();
const canvas = document.getElementById('canvas');
const offscreen = canvas.transferControlToOffscreen();
worker.postMessage({
event: 'init',
data: {
canvas: offscreen,
modelPath: 'https://paddlejs.cdn.bcebos.com/models/mobileNetV2Opt/model.json',
feedShape: { fw: 224, fh: 224 },
mean: [0.485, 0.456, 0.406],
std: [0.229, 0.224, 0.225]
}
}, [offscreen]);
worker.addEventListener('message', (msg) => {
if (msg.data.event === 'init') {
console.log('Worker initialized');
} else if (msg.data.event === 'predict') {
console.log('Prediction result:', msg.data.data);
}
});
const image = document.getElementById('photo');
createImageBitmap(image, 0, 0, image.naturalWidth, image.naturalHeight)
.then(imageBitmap => {
worker.postMessage({
event: 'predict',
data: imageBitmap
}, [imageBitmap]);
});
// Worker thread: worker.ts
import { Runner, env } from '@paddlejs/paddlejs-core';
import { GLHelper } from '@paddlejs/paddlejs-backend-webgl';
const webWorker: Worker = self as any;
let runner: Runner;
const WEBGL_ATTRIBUTES = {
alpha: false,
antialias: false,
premultipliedAlpha: false,
preserveDrawingBuffer: false,
depth: false,
stencil: false,
failIfMajorPerformanceCaveat: true
};
webWorker.addEventListener('message', async (msg) => {
const { event, data } = msg.data;
if (event === 'init') {
const offscreenCanvasFor2D = new OffscreenCanvas(1, 1);
env.set('canvas2d', offscreenCanvasFor2D);
const gl = data.canvas.getContext('webgl2', WEBGL_ATTRIBUTES);
GLHelper.setWebGLRenderingContext(gl);
runner = new Runner({
modelPath: data.modelPath,
feedShape: data.feedShape,
mean: data.mean,
std: data.std
});
await runner.init();
webWorker.postMessage({ event: 'init' });
} else if (event === 'predict') {
const result = await runner.predict(data);
webWorker.postMessage({ event: 'predict', data: result });
}
});
```
--------------------------------
### Video Controls Flexbox Layout CSS
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-mediapipe/data-processor/demo/index.html
CSS styling for the video tool control section using flexbox to distribute buttons evenly across the container. The #video-tool element uses space-around justification to create equal spacing between control buttons.
```css
#video-tool {
display: flex;
justify-content: space-around;
}
```
--------------------------------
### Title and Section Styling - CSS
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-examples/ocr/index.html
Provides styling for heading elements within the demo, including font weight, size, and bottom margin spacing to separate title from content below.
```css
.title {
font-weight: 600;
font-size: 20px;
margin-bottom: 10px;
}
```
--------------------------------
### Post Message with Transferable ArrayBuffer in JavaScript
Source: https://github.com/paddlepaddle/paddle.js/blob/release/v2.2.5/packages/paddlejs-examples/webglWorker/README.md
Demonstrates transferring an ArrayBuffer to a WebWorker using `postMessage`. This method is efficient for large data as it transfers ownership rather than copying, improving performance. The second argument to `postMessage` specifies the transferable objects.
```javascript
worker.postMessage(arrayBuffer, [arrayBuffer]);
```
--------------------------------
### Load Model from In-Memory Objects
Source: https://context7.com/paddlepaddle/paddle.js/llms.txt
Illustrates loading a Paddle.js model directly from JavaScript objects (model JSON and parameters) rather than fetching them from a URL. This is useful for offline applications or when models are bundled with the application code.
```typescript
import { Runner } from '@paddlejs/paddlejs-core';
import '@paddlejs/paddlejs-backend-webgl';
import modelJson from './model.json';
import modelParams from './params.bin';
const runner = new Runner({
modelObj: {
model: modelJson,
params: new Float32Array(modelParams)
},
feedShape: { fw: 224, fh: 224 },
mean: [0.485, 0.456, 0.406],
std: [0.229, 0.224, 0.225]
});
await runner.init();
const result = await runner.predict(imageElement);
```
--------------------------------
### Predict with Raw Numerical Data or ImageData
Source: https://context7.com/paddlepaddle/paddle.js/llms.txt
Shows how to perform model prediction using raw numerical data (e.g., Float32Array) or `ImageData` objects, bypassing the need for media elements like `` or `