### React: Basic Setup for Realtime Digital Human Demo

Source: https://cloud.baidu.com/doc/AI_DH/s/ylywx77oh

This section outlines the steps to set up and run the React version of the realtime digital human demo. It includes environment checks, dependency installation via yarn, starting the development server, and configuring the iframe parameters like token, figureId, and cameraId.

```bash
1. 解压后，进入目录：React版本目录名称：realtime-digital-human-demo
2. 判断环境是否有 node(22 版本以上)和 yarn，如没有则进行 node, yarn 的安装： brew install node; npm install --global yarn
3. 命令行执行 yarn，安装所需依赖包
4. 命令行执行 npm run dev，启动项目
5. 在公共形象库中查找组件对应的人像类型的人像 id 和机位 id，替换 demo 文件中 figureId 和 cameraId 参数，文件位置：React 版本是 src/template
6. 根据接口通用说明生成鉴权参数 token，替换 demo 文件中 token 参数，文件位置：React 版本是 src/template
```

--------------------------------

### Vue: Basic Setup for Realtime Digital Human Demo

Source: https://cloud.baidu.com/doc/AI_DH/s/ylywx77oh

This section outlines the steps to set up and run the Vue version of the realtime digital human demo. It includes environment checks, dependency installation via yarn, starting the development server, and configuring the iframe parameters like token, figureId, and cameraId.

```bash
1. 解压后，进入目录：Vue 版本目录名称：realtime-digital-human-vue-demo
2. 判断环境是否有 node(22 版本以上)和 yarn，如没有则进行 node, yarn 的安装： brew install node; npm install --global yarn
3. 命令行执行 yarn，安装所需依赖包
4. 命令行执行 npm run dev，启动项目
5. 在公共形象库中查找组件对应的人像类型的人像 id 和机位 id，替换 demo 文件中 figureId 和 cameraId 参数，文件位置：Vue 版本是 /src/components/Iframe
6. 根据接口通用说明生成鉴权参数 token，替换 demo 文件中 token 参数，文件位置：Vue 版本是 /src/components/Iframe
```

--------------------------------

### SDK Installation

Source: https://cloud.baidu.com/doc/AI_DH/s/6m2wziq1k

Instructions for installing the SDK using npm or yarn package managers.

```APIDOC
## SDK Installation

This section details how to install the Digital Human SDK using either npm or yarn.

### Method
NPM / Yarn

### Usage

**Using npm:**
```bash
npm i @bddh/starling-dhiframe
```

**Using yarn:**
```bash
yarn add @bddh/starling-dhiframe
```
```

--------------------------------

### Audio Stream Start Request (JavaScript)

Source: https://cloud.baidu.com/doc/AI_DH/s/Sm1h9a4dh

This object initiates an audio stream for rendering. It includes a `streamId` for tracing, the event type 'START', and optionally the `sampleRate` if it deviates from the default 16000 Hz. After receiving a 'READY' event from the server, the client can send audio data via the binary channel.

```javascript
{
    "streamId": "stream id", // Custom ID for problem tracing
    "event": "START", // START, COMPLETE
    "sampleRate": 16000 // Default 16000
}
```

--------------------------------

### Digital Human API Task Query GET Request Example

Source: https://cloud.baidu.com/doc/AI_DH/s/5m0ggor0z

Example of a GET request to query the status of a Digital Human API task. It requires a taskId to identify the specific task and an optional requestId for correlation.

```http
GET /api/digitalhuman/open/v1/figure/image/task?taskId=abc123&requestId=123e4567-e89b-12d3-a456-426614174000
```

--------------------------------

### GET Request Example for Task Query (JavaScript)

Source: https://cloud.baidu.com/doc/AI_DH/s/ilywwq0jz

This JavaScript example demonstrates how to query the status of a digital human video synthesis task using its ID. It's a GET request to the specified API endpoint with the taskId as a query parameter.

```javascript
GET /api/digitalhuman/open/v1/video/task?taskId=abc123&requestId=123e4567-e89b-12d3-a456-426614174000
```

--------------------------------

### SDK Initialization and Usage

Source: https://cloud.baidu.com/doc/AI_DH/s/6m2wziq1k

Guide on how to initialize the SDK, load models, and handle text rendering and callbacks.

```APIDOC
## SDK Initialization and Usage

This guide explains how to initialize the SDK, load digital human models, and manage text rendering and event callbacks.

### Method
JavaScript

### Initialization
To initialize the SDK, you need to create an instance of `DHIframeV2` by providing the `id` attribute of your iframe element.

```javascript
import { DHIframeV2 } from '@bddh/starling-dhiframe';

const dhIframe = new DHIframeV2('iframeId'); // Replace 'iframeId' with your actual iframe's id attribute
```

### Loading Models
Model initialization is done using the `load` method after the engine has been initialized. The `load` method requires configuration parameters such as `backgroundImageUrl` and `modelUrl`.

```javascript
// Example of loading a model after engine initialization
dhIframe.load({
    backgroundImageUrl: 'https://digital-human-js-cdn.cdn.bcebos.com/meta_human_editor/20240903202457/digital-human-open-console-web/img/bg1.png',
    modelUrl: 'URL_to_your_custom_model.zip',
    cameraId: 'face_cam' // Optional: specifies the initial camera position
});
```

**Parameters for `load` method:**

*   **modelUrl** (string) - Required - The CDN or downloadable URL for your custom digital human model package (zip file). Local file paths are not supported.
*   **backgroundImageUrl** (string) - Optional - The URL for a background image. If not provided, the background will be transparent. Local file paths are not supported.
*   **cameraId** (string) - Optional - Specifies the initial camera view (e.g., full body, half body). This value should correspond to settings in your model's `config.json`.

**Important:** The `load` method should only be called after the engine initialization status (`INIT`) has been received.

### Text Rendering and Callbacks
To drive the digital human with text and receive status updates, use the `textRender` method and register message listeners.

```javascript
// Example of text rendering
dhIframe.textRender('Hello, how can I help you today?');

// Registering a listener for messages
dhIframe.registerMessageReceived((data: any) => {
    // Handle different message types, e.g., state updates
    if (data.type === 'state') {
        console.log('Engine status:', data.content.status);
        // Handle state transitions like INIT, AVATAR_LOAD, CLOSE etc.
    }
});
```
```

--------------------------------

### Initialize Device License (Java)

Source: https://cloud.baidu.com/doc/AI_DH/s/cm4i2ln3e

Initializes the SDK using device-specific `appKey` and `seceretKey`. Requires a valid context, keys, and an authentication callback to handle success or failure.

```java
boolean initLicenseBatch(final Context context, final String appKey,
                             final String seceretKey, final VirtualAuth.AuthCallBack authCallback);
```

```java
String ak = deviceAppKeyEdit.getText().toString().trim();
String sk = deviceSeceretKeyEdit.getText().toString().trim();
ChirologyAvatar.getInstance().initLicenseBatch(MainActivity.this, ak, sk, new VirtualAuth.AuthCallBack() {
     @Override
     public void onMessage(int code, int subCode, String msg) {
                        
      }
});
```

--------------------------------

### Query Task Status (GET Request)

Source: https://cloud.baidu.com/doc/AI_DH/s/Ylywv4uqg

This example shows how to query the status of a digital human video generation task using a GET request. It includes the endpoint and required query parameters such as taskId and requestId.

```http
GET /api/digitalhuman/open/v1/video/image/task?taskId=abc123&requestId=123e4567-e89b-12d3-a456-426614174000
```

--------------------------------

### 1.2 Initialize SDK - Project Authorization

Source: https://cloud.baidu.com/doc/AI_DH/s/Em488seof

Initializes digital human and avatar resources. Project AK and SK are required for authorization, which in turn obtains appKey and secretKey.

```APIDOC
## 1.2 Initialize SDK - Project Authorization

### Description
Initializes digital human and avatar resources. Project AK and SK are required for authorization, which in turn obtains appKey and secretKey. It is recommended to instantiate `VirtualAuth` beforehand as it needs to operate on the main thread.

### Method
`void initProjectAuth(Context context, VirtualAuth virtualAuth, VirtualInitParam virtualInitParam, VirtualAuth.AuthCallBack authCallBack)`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **context** (Context) - Required - The application context.
- **virtualAuth** (VirtualAuth) - Required - An instance of `VirtualAuth`.
- **virtualInitParam** (VirtualInitParam) - Required - Initialization parameters.
- **authCallBack** (VirtualAuth.AuthCallBack) - Required - Callback for authorization status (code, subCode, msg).

**VirtualInitParam Fields:**
- **appKey** (String) - Required - Your project's unique `projectAk`, provided by Baidu.
- **seceretKey** (String) - Required - Your project's unique `projectSk`, provided by Baidu.
- **ttsId** (String) - Optional - TTS application ID, required for text-to-speech. Obtain from the component platform.
- **ttsKey** (String) - Optional - TTS application key, required for text-to-speech. Obtain from the component platform.

### Request Example
```java
VirtualFactory.getEngine().initProjectAuth(this, vhAuth, Config.projectAk, Config.projectSK, new VirtualAuth.AuthCallBack() {
    @Override
    public void onMessage(int code, int subCode, String msg) {
        // Handle authorization response
    }
});
```

### Response
#### Success Response (200)
- **code** (int) - 0: Initialization successful.

#### Error Codes:
- **-1000**: Context is null.
- **-1001**: AppKey is empty.
- **-1002**: SecretKey is empty.
- **-1003**: AppKey length error.
- **-1004**: SecretKey length error.
- **-1005**: Failed to set file path.
- **-1006**: Engine state error, duplicate initialization.
- **-1007**: Callback is null.
- **-1008**: Project authorization request result error.
- **-1009**: ProjectAK is empty.
- **-1010**: ProjectSK is empty.
- **-1011**: ProjectAK length error.
- **-1012**: ProjectSK length error.
- **-1013**: Context is null during project authorization.
- **-1014**: VirtualAuth is null during project authorization.
- **-1015**: VirtualInitParam is null.
- **-1016**: ttsId is empty.
- **-1017**: ttsKey is empty.

#### Response Example
(Callback provides status: `code`, `subCode`, `msg`)
```

--------------------------------

### Query Task Status Request Example

Source: https://cloud.baidu.com/doc/AI_DH/s/9m5c8ntvh

This is an example of a GET request to query the status of a background removal task. It utilizes the taskId parameter to specify which task's status to retrieve. If taskId is omitted, it queries all tasks for the account.

```http
GET /api/digitalhuman/open/v1/figure/lite2d/background/task?taskId=bkg-qmur7u3her5cn255
```

--------------------------------

### Query Digital Human Task Status (HTTP GET)

Source: https://cloud.baidu.com/doc/AI_DH/s/pm5xoetu7

This example shows how to query the status of a digital human video generation task using its unique task ID via an HTTP GET request. The endpoint requires the taskId as a query parameter.

```http
GET /api/digitalhuman/open/v1/video/task?taskId=vf3-rajqt8jue9aiqu0p
```

--------------------------------

### SDK Initialization

Source: https://cloud.baidu.com/doc/AI_DH/s/Em488seof

Details on how to initialize the SDK for device authorization and TTS (Text-to-Speech) functionality.

```APIDOC
## SDK Initialization and Authorization

### Initialize SDK - Device Authorization

Initializes the digital human resources and persona resources.

#### Method Signature
```java
int init(Context context, VirtualInitParam virtualInitParam);
```

#### Parameters
- **context** (`Context`): Non-null. The application context.
- **virtualInitParam** (`VirtualInitParam`): Non-null. Contains initialization parameters.

#### `VirtualInitParam` Fields
- **appKey** (`String`): Non-null. Unique key for the device, provided by Baidu. Essential for audio-driven features.
- **seceretKey** (`String`): Non-null. Corresponding secret key for `appKey`.
- **ttsId** (`String`): Required for text-driven features. The App ID of the TTS synthesis application created on the component platform. Refer to https://cloud.baidu.com/doc/AI_DH/s/plyy6xhi0 for details.
- **ttsKey** (`String`): Required for text-driven features. The App Key of the TTS synthesis application.

#### Return Values
- **0**: Initialization successful.
- **-1000**: Context is null.
- **-1001**: AppKey is empty.
- **-1002**: SecretKey is empty.
- **-1003**: AppKey length error.
- **-1004**: SecretKey length error.
- **-1005**: Failed to set file path.
- **-1006**: Engine state error, duplicate initialization.
- **-1007**: Callback is null.
- **-1008**: Project authorization request result error.
- **-1009**: ProjectAK is empty.
- **-1010**: ProjectSK is empty.
- **-1011**: ProjectAK length error.
- **-1012**: ProjectSK length error.
- **-1013**: Context is null during project authorization.
- **-1014**: VirtualAuth is null during project authorization.
- **-1015**: VirtualInitParam is null.
- **-1016**: ttsId is empty.
- **-1017**: ttsKey is empty.

#### Initialization Example
```java
public class MainApplication extends Application {
    @Override
    protected void attachBaseContext(Context base) {
        super.attachBaseContext(base);
        // For audio-driven features:
        // VirtualFactory.getEngine().init(this, Config.APP_KEY, Config.SECERET_KEY);

        // For text-driven features (including audio):
        VirtualFactory.getEngine().init(this, new VirtualInitParam(Config.APP_KEY, Config.SECERET_KEY, Config.TTS_ID, Config.TTS_KEY));
    }
}
```
```

--------------------------------

### Query Digital Human API Request GET

Source: https://cloud.baidu.com/doc/AI_DH/s/Ym410mx30

This is an example of a GET request to query digital humans. It demonstrates how to specify a particular figureId to retrieve information about a specific digital human. Other parameters like systemFigure, trainSuccess, pageNo, and pageSize can also be used for filtering and pagination.

```http
GET /api/digitalhuman/open/v1/figure/lite2d/query?figureId=931
```

--------------------------------

### SDK Initialization Parameters

Source: https://cloud.baidu.com/doc/AI_DH/s/bmg52lb7y

This section details the parameters required for initializing the Digital Human SDK, including mandatory and optional configurations for rendering and callbacks.

```APIDOC
## SDK Initialization Parameters

### Description
Parameters for initializing the Digital Human SDK, including wrapper ID, token, and callback functions.

### Parameters
#### Path Parameters
- None

#### Query Parameters
- None

#### Request Body
- **wrapperId** (string) - Required - The ID of the DOM element where the digital human will be rendered.
- **token** (string) - Required - The authentication token obtained from the application management portal.
- **connectParams** (object) - Optional - Configuration parameters for connecting to the digital human.
- **renderParams** (object) - Optional - Configuration parameters for rendering the digital human.
- **onDigitalHumanCallback** (function) - Required - A callback function to listen for digital human status updates.
```

--------------------------------

### Initialize Project Authorization (Java)

Source: https://cloud.baidu.com/doc/AI_DH/s/cm4i2ln3e

Authorizes the SDK for a specific project using `projectAk` and `projectSk`. This method requires context, project-specific keys, and an authentication callback.

```java
boolean initProjectAuth(final Context context, final String projectAk, final String projectSk,
                           final VirtualAuth.AuthCallBack authCallback);
```

```java
String projectAk = projectAkEdit.getText().toString().trim();
String projectsk = projectskEdit.getText().toString().trim();
ChirologyAvatar.getInstance().initProjectAuth(MainActivity.this, projectAk, projectsk, new VirtualAuth.AuthCallBack() {
   @Override
   public void onMessage(int code, int subCode, String msg) {
   
   }
});
```

--------------------------------

### Query Voice Cloning Tasks API Request Example

Source: https://cloud.baidu.com/doc/AI_DH/s/em9v0ob8g

This example demonstrates a GET request to query voice cloning tasks. The 'isSuccess' query parameter can be set to 'false' to retrieve all tasks, regardless of their completion status, or 'true' to only fetch successfully cloned voices. An optional 'perId' parameter can be added to filter results for a specific voice cloning task.

```HTTP
GET /api/digitalhuman/open/v1/tts/clone?isSuccess=false
```

--------------------------------

### Query Audio Synthesis Task Status Request Example (Plain Text)

Source: https://cloud.baidu.com/doc/AI_DH/s/um3ztkf8x

This shows a simple GET request to query the status of an audio synthesis task using its taskId. The taskId is passed as a query parameter.

```Plain Text
GET /api/digitalhuman/open/v1/tts/text2audio/task?taskId=t2a-xxx
```

--------------------------------

### Initialize SDK with Device and TTS Credentials

Source: https://cloud.baidu.com/doc/AI_DH/s/Em488seof

This Java code demonstrates the initialization of the digital human SDK. It requires context, unique device keys (`appKey`, `seceretKey`), and optional TTS credentials (`ttsId`, `ttsKey`) for text-driven speech. Proper initialization is crucial for SDK functionality.

```java
public class MainApplication extends Application {
    @Override
    protected void attachBaseContext(Context base) {
        super.attachBaseContext(base);
        VirtualFactory.getEngine().init(this, Config.APP_KEY, Config.SECERET_KEY);
    }
}
```

--------------------------------

### Check and Initialize DigitalHumanEngine

Source: https://cloud.baidu.com/doc/AI_DH/s/Tm3o7d45f

Retrieves the singleton instance of DigitalHumanEngine and performs a preliminary check of the DigitalHumanSettings and EngineCallback. Following the check, it proceeds with the actual initialization of the engine using the provided context, settings, and callback. This initialization step must be followed by calling `onSurfaceTextureAvailable`.

```Java
// SplashActivity.initDigitalHumanEngine
// 检查
DigitalHumanEngine digitalHumanEngine = DigitalHumanEngine.getInstance();
digitalHumanEngine.checkDigitalHumanSettings(digitalHumanSettings, engineCallback);
       
// 初始化       
digitalHumanEngine.init(this, digitalHumanSettings, engineCallback);

//init 调用后需要调用下文的DigitalHumanEngine.onSurfaceTextureAvailable，完成初始化
```

--------------------------------

### SDK Integration

Source: https://cloud.baidu.com/doc/AI_DH/s/Em488seof

Information for developers on integrating the SDK, including supported system and hardware versions, build configurations, and necessary permissions.

```APIDOC
## SDK Integration for Developers

### Supported Systems and Hardware Versions
- **Target Platforms:** Android mobile phones and Android large screens (RK3588+).
- **System:** Android 10.0+ (API Level 24) and above.
- **CPU Architecture:** `armeabi-v7a`, `arm64-v8a`.
- **Hardware Requirements:** 8-core CPU or higher. Recommended chipsets: Qualcomm Snapdragon 660+, HiSilicon Kirin 980+, MediaTek Dimensity 800U+.
- **Network:** Wi-Fi and mobile networks. For cloud TTS, a bandwidth of 10mbps or higher is recommended.
- **Development IDE:** Android Studio Flamingo 2022.2.1 Patch 1.
- **Memory:** Minimum 600MB RAM for the SDK.

### Importing SDK AAR Packages
- Include `module_core-jnidev-release.aar` and `module_virtual-release.aar` in the `libs` directory of your app module.
- Add the following configurations to your `build.gradle` (module level):
```gradle
implementation fileTree(dir: "libs", include: ["*.jar"])
implementation files('libs/module_core-jnidev-release.aar')
implementation files('libs/module_virtual-release.aar')

implementation 'com.google.code.gson:gson:2.8.9'
implementation("com.squareup.okhttp3:okhttp:4.11.0")
```

### Android Studio Configuration
- **Gradle JDK:** Set to Java 11 (File -> Settings -> Build, Execution, Deployment -> Build Tools -> Gradle -> Gradle JDK).
- **Android Gradle plugin Version:** Configure in project settings (Cmd + ; on Mac).
- **Android SDK & Java Version:** Use recommended versions for optimal performance.

### System Permissions
#### Permissions Configuration (`AndroidManifest.xml`)
```xml
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />
<uses-permission android:name="android.permission.CHANGE_WIFI_STATE" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.ACCESS_WIFI_STATE" />
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.READ_PHONE_STATE" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.MANAGE_EXTERNAL_STORAGE" />
```

#### Permissions Request
- Refer to the `BaseActivity` example in the demo.
- Ensure all listed permissions are granted to avoid SDK errors.
```

--------------------------------

### Close Session API Response Example

Source: https://cloud.baidu.com/doc/AI_DH/s/2m9gtysrn

This snippet provides an example of a successful response from the DELETE /api/digitalhuman/open/v1/session/interaction endpoint. A 'code' of 0 indicates success, and the 'message' provides details.

```json
{
    "code": 0,
    "message": {
        "global": "OK"
    },
    "result": null,
    "requestId": null,
    "success": true
}
```

--------------------------------

### Initialize and Use DH SDK

Source: https://cloud.baidu.com/doc/AI_DH/s/6m2wziq1k

Example of initializing the DHIframeV2 instance and calling core methods like load() for model initialization and textRender() for text-driven rendering. Ensure the iframe ID is correctly provided.

```javascript
1import {DHIframeV2} from '@bddh/starling-dhiframe';
2
3const dhIframe = new DHIframeV2('iframeId'); // 这里修改成 iframe 的 id属性值
4// 模型的初始化
5dhIframe.load(...)
6// 文本驱动和回调
7dhIframe.textRender(...)
```

--------------------------------

### Text-to-Speech Response Example (JavaScript)

Source: https://cloud.baidu.com/doc/AI_DH/s/um3ztkf8x

This example demonstrates the JSON response received after initiating a text-to-speech request. It includes a status code, a message, and a result object containing a taskId for tracking the synthesis process.

```JavaScript
{
    "code": 0,
    "message": {
        "global": "success"
    },
    "result": {
        "taskId": "t2a-xxx"
    },
    "requestId": "a829316c-aa14-42be-839e-d2e432b58cca",
    "success": true
}
```

--------------------------------

### Listen for Engine Initialization and Load Model

Source: https://cloud.baidu.com/doc/AI_DH/s/6m2wziq1k

This snippet demonstrates how to register a message listener to detect when the engine is initialized (status 'INIT'). Upon successful initialization, it proceeds to load the digital human model using the `load` method, specifying background and model URLs. It also logs other significant status updates like model loading completion and destruction.

```javascript
1dhIframe.registerMessageReceived((data: any) => {
2    if (data.type === 'state') {
3        if (data.content.status === 'INIT') {
4            console.log('引擎初始化完成')
5            dhIframe.load({
6                backgroundImageUrl: 'https://digital-human-js-cdn.cdn.bcebos.com/meta_human_editor/20240903202457/digital-human-open-console-web/img/bg1.png',
7                modelUrl: '定制人像包的zip的cdn地址'
8            });
9        }
10        if (data.content.status === 'AVATAR_LOAD') {
11            console.log('模型数字人加载完成')
12        }
13        if (data.content.status === 'CLOSE') {
14            console.log('模型数字人销毁完成')
15        }
16    }
17});
```

--------------------------------

### Close Session API Request Example

Source: https://cloud.baidu.com/doc/AI_DH/s/2m9gtysrn

This snippet shows an example of how to make a DELETE request to the /api/digitalhuman/open/v1/session/interaction endpoint to close a session. It requires the 'sessionId' parameter to identify the session to be closed.

```http
DELETE /api/digitalhuman/open/v1/session/interaction?sessionId=r-xxx
```

--------------------------------

### State Listener Callback - JavaScript

Source: https://cloud.baidu.com/doc/AI_DH/s/Clywx7q0f

Illustrates how to monitor the state changes of a digital human in JavaScript using the `onStateChange` callback. It includes handling initialization (`INIT`) and error states, demonstrating how to set up an initial greeting after the digital human is ready.

```javascript
<digitalhuman-view
    id="digitalhuman"
    class="digitalhuman-view-container"
    settings="{{digitalHumanSetting}}"
    bind:onMessage="onMessage"
    bind:onStateChange="onStateChange"
/>

Page({
    data: {
        digitalHumanSetting: {...}
    },
    onLoad(options) {
        const digitalhuman = this.selectComponent('#digitalhuman');
    }
    onStateChange(e) {
        const {type, error = {}, loading} = e.detail;
        const {action, message, code} = error;
        if (type === 'INIT') {
            // 通过监听 INIT 消息，来实现数字人的开场白设置
            this.digitalhuman.textRender('数字人初始化完成，这是数字人的开场白播报，可以自行修改开场白内容');
        }
        if (type === 'ERROR'){
            console.log(type, error);
        }
    },
});
```