Doubao IME ASR

https://github.com/starccy/doubaoime-asr
Admin
Doubao IME ASR is a Python client library for automatic speech recognition that provides both...
Tokens:12,761
Snippets:78
Trust Score:7.1
Update:5 months ago
Show doc for...
Tokens:
Raw
### Install Example Dependencies for Real-time Recognition

Source: https://github.com/starccy/doubaoime-asr/blob/main/README.md

Install additional libraries required for running real-time microphone recognition examples. This includes `sounddevice` and `numpy`.

```bash
pip install sounddevice numpy
```

```bash
# 或
pip install doubaoime-asr[examples]
```

--------------------------------

### Install DoubaoIME ASR from Local Source

Source: https://github.com/starccy/doubaoime-asr/blob/main/README.md

Install the library from a local clone of the Git repository. This is useful for development or when you need to modify the source code.

```bash
git clone https://github.com/starccy/doubaoime-asr.git
cd doubaoime-asr
pip install -e .
```

--------------------------------

### Install DoubaoIME ASR from Git Repository

Source: https://github.com/starccy/doubaoime-asr/blob/main/README.md

Install the library directly from the Git repository using pip. This is a convenient way to get the latest version.

```bash
pip install git+https://github.com/starccy/doubaoime-asr.git
```

--------------------------------

### Install doubaoime-asr and Dependencies

Source: https://context7.com/starccy/doubaoime-asr/llms.txt

Install system dependencies for Opus codec and the doubaoime-asr Python library. Additional dependencies are required for real-time microphone recognition.

```bash
# Install system dependencies (Opus codec library)
# Debian/Ubuntu:
sudo apt install libopus0
# macOS:
brew install opus

# Install doubaoime-asr
pip install git+https://github.com/starccy/doubaoime-asr.git

# Install extra dependencies for real-time microphone recognition
pip install doubaoime-asr[examples]
```

--------------------------------

### Install Opus Audio Codec Library (Debian/Ubuntu)

Source: https://github.com/starccy/doubaoime-asr/blob/main/README.md

Install the Opus audio codec library on Debian or Ubuntu systems. This is a system dependency for the ASR client.

```bash
sudo apt install libopus0
```

--------------------------------

### Install Opus Audio Codec Library (macOS)

Source: https://github.com/starccy/doubaoime-asr/blob/main/README.md

Install the Opus audio codec library on macOS using Homebrew. This is a system dependency for the ASR client.

```bash
brew install opus
```

--------------------------------

### Install Opus Audio Codec Library (Arch Linux)

Source: https://github.com/starccy/doubaoime-asr/blob/main/README.md

Install the Opus audio codec library on Arch Linux systems. This is a system dependency for the ASR client.

```bash
sudo pacman -S opus
```

--------------------------------

### Example Encrypted HTTP Request

Source: https://github.com/starccy/doubaoime-asr/blob/main/wave_protocol.md

Illustrates a complete HTTP POST request with Wave protocol encryption headers, including `x-tt-e-b`, `x-tt-e-t`, `x-tt-e-p`, and `x-ss-stub`, along with API-specific headers.

```http
POST /api/v3/context/ime/ner?device_platform=android&... HTTP/1.1
Host: speech.bytedance.com
Content-Type: application/json
x-tt-e-b: 1
x-tt-e-t: MIGYBAwrT...（ticket）
x-tt-e-p: abc123...（nonce Base64）
x-ss-stub: A1B2C3D4E5F6...（MD5 大写）
x-api-app-key: SYlxZr6LnvBaIVmF
x-api-token: eyJhbGciOiJFUzI1NiI...

<加密后的请求体>
```

--------------------------------

### Real-time Microphone Streaming Transcription

Source: https://context7.com/starccy/doubaoime-asr/llms.txt

Transcribes audio from a microphone in real-time using an asynchronous generator. Requires `sounddevice` and `numpy`. Handles session start, interim, and final results, as well as errors.

```python
import asyncio
import sounddevice as sd
import numpy as np
from doubaoime_asr import transcribe_realtime, ASRConfig, ResponseType

async def mic_audio_generator(sample_rate=16000, channels=1, frame_duration_ms=20):
    """麦克风异步音频生成器"""
    samples_per_frame = sample_rate * frame_duration_ms // 1000
    queue: asyncio.Queue[bytes] = asyncio.Queue()
    loop = asyncio.get_event_loop()

    def audio_callback(indata, frames, time_info, status):
        loop.call_soon_threadsafe(queue.put_nowait, indata.tobytes())

    with sd.InputStream(
        samplerate=sample_rate,
        channels=channels,
        dtype=np.int16,
        blocksize=samples_per_frame,
        callback=audio_callback,
    ):
        while True:
            yield await queue.get()


async def main():
    config = ASRConfig(credential_path="./credentials.json")

    print("开始实时识别，按 Ctrl+C 停止...")
    try:
        async for response in transcribe_realtime(
            mic_audio_generator(
                sample_rate=config.sample_rate,
                channels=config.channels,
                frame_duration_ms=config.frame_duration_ms,
            ),
            config=config,
        ):
            if response.type == ResponseType.SESSION_STARTED:
                print("会话已启动，请开始说话...\n")
            elif response.type == ResponseType.INTERIM_RESULT:
                # 使用回车符在同一行更新中间结果
                print(f"\r[识别中] {response.text}", end="", flush=True)
            elif response.type == ResponseType.FINAL_RESULT:
                print(f"\r[最终结果] {response.text}          ")
            elif response.type == ResponseType.ERROR:
                print(f"\n[错误] {response.error_msg}")
    except KeyboardInterrupt:
        print("\n已停止录音")

asyncio.run(main())
```

--------------------------------

### Streaming Audio File Recognition

Source: https://github.com/starccy/doubaoime-asr/blob/main/README.md

Perform streaming speech recognition on an audio file to get intermediate and final results. This is useful for real-time feedback or detailed status.

```python
import asyncio
from doubaoime_asr import transcribe_stream, ASRConfig, ResponseType

async def main():
    config = ASRConfig(credential_path="./credentials.json")

    async for response in transcribe_stream("audio.wav", config=config):
        match response.type:
            case ResponseType.INTERIM_RESULT:
                print(f"[中间结果] {response.text}")
            case ResponseType.FINAL_RESULT:
                print(f"[最终结果] {response.text}")
            case ResponseType.ERROR:
                print(f"[错误] {response.error_msg}")

asyncio.run(main())
```

--------------------------------

### Configure ASR with ASRConfig

Source: https://context7.com/starccy/doubaoime-asr/llms.txt

Initialize ASRConfig for managing credentials and audio parameters. Supports automatic device registration, credential caching, and overriding audio settings.

```python
from doubaoime_asr import ASRConfig

# Minimal configuration: automatic device registration, no credential persistence
config = ASRConfig()

# Recommended configuration: persist credentials to a file to avoid re-registration
config = ASRConfig(credential_path="~/.config/doubaoime-asr/credentials.json")

# Full configuration example: override audio parameters and connection settings
config = ASRConfig(
    credential_path="./credentials.json",
    sample_rate=16000,          # Sample rate (Hz), default 16000
    channels=1,                 # Number of channels, default 1 (mono)
    frame_duration_ms=20,       # Frame duration (ms), default 20
    enable_punctuation=True,    # Enable punctuation, default True
    enable_speech_rejection=False,
    connect_timeout=10.0,       # WebSocket connection timeout (seconds)
    recv_timeout=10.0,          # Receive timeout (seconds)
    app_name="com.android.chrome",  # Simulated host application
)

# Manually provide existing credentials (skip automatic registration)
config = ASRConfig(
    device_id="7123456789012345",
    token="eyJhbGci...",
)

# Verify credential loading
config.ensure_credentials()
print(f"device_id: {config.device_id}")
print(f"token: {config.token[:20]}...")
```

--------------------------------

### ASRConfig - Configuration and Credential Management

Source: https://context7.com/starccy/doubaoime-asr/llms.txt

The ASRConfig class is central to all functionalities, handling automatic device registration, credential caching, and overriding audio parameters and connection settings. It supports direct parameter input, credential file paths, and automatic registration, with a priority given to directly provided parameters.

```APIDOC
## ASRConfig

### Description
Handles core configuration, including automatic device registration, credential caching, and overriding audio/connection parameters. Credentials can be provided directly, loaded from a file, or obtained via automatic registration.

### Usage
```python
from doubaoime_asr import ASRConfig

# Minimal configuration: automatic device registration, no credential persistence
config = ASRConfig()

# Recommended configuration: persistent credentials to avoid re-registration
config = ASRConfig(credential_path="~/.config/doubaoime-asr/credentials.json")

# Full configuration example: overriding audio parameters and connection settings
config = ASRConfig(
    credential_path="./credentials.json",
    sample_rate=16000,          # Sample rate (Hz), default 16000
    channels=1,                 # Number of channels, default 1 (mono)
    frame_duration_ms=20,       # Frame duration (ms), default 20
    enable_punctuation=True,    # Enable punctuation, default True
    enable_speech_rejection=False,
    connect_timeout=10.0,       # WebSocket connection timeout (seconds)
    recv_timeout=10.0,          # Receive timeout (seconds)
    app_name="com.android.chrome",  # Simulated host application
)

# Manually provide existing credentials (skips automatic registration)
config = ASRConfig(
    device_id="7123456789012345",
    token="eyJhbGci...",
)

# Verify credential loading
config.ensure_credentials()
print(f"device_id: {config.device_id}")
print(f"token: {config.token[:20]}...")
```

### Parameters
- **credential_path** (str, optional): Path to a file for persisting and loading credentials.
- **sample_rate** (int, optional): Audio sample rate in Hz. Defaults to 16000.
- **channels** (int, optional): Number of audio channels. Defaults to 1 (mono).
- **frame_duration_ms** (int, optional): Duration of audio frames in milliseconds. Defaults to 20.
- **enable_punctuation** (bool, optional): Whether to enable punctuation. Defaults to True.
- **enable_speech_rejection** (bool, optional): Whether to enable speech rejection. Defaults to False.
- **connect_timeout** (float, optional): Timeout for WebSocket connection in seconds.
- **recv_timeout** (float, optional): Timeout for receiving data in seconds.
- **app_name** (str, optional): The name of the simulated host application.
- **device_id** (str, optional): Manually provided device ID.
- **token** (str, optional): Manually provided authentication token.
```

--------------------------------

### ASRConfig

Source: https://github.com/starccy/doubaoime-asr/blob/main/README.md

Configuration class for ASR settings.

```APIDOC
## ASRConfig

### Description
Configuration class for ASR settings.

### Parameters
- **credential_path** (str) - Optional - Path to the credential cache file. Defaults to None.
- **device_id** (str) - Optional - Device ID. If empty, a new device will be registered. Defaults to None.
- **token** (str) - Optional - Authentication token. If empty, the token will be automatically obtained. Defaults to None.
- **sample_rate** (int) - Optional - Sample rate of the audio. Defaults to 16000.
- **channels** (int) - Optional - Number of audio channels. Defaults to 1.
- **enable_punctuation** (bool) - Optional - Whether to enable punctuation. Defaults to True.
```

--------------------------------

### Batch and Streaming Transcription with DoubaoASR Client

Source: https://context7.com/starccy/doubaoime-asr/llms.txt

Utilizes the `DoubaoASR` class as an asynchronous context manager for efficient reuse of configurations across multiple audio files. Supports both batch file processing and streaming transcription.

```python
import asyncio
from doubaoime_asr import DoubaoASR, ASRConfig, ASRError, ResponseType

async def main():
    config = ASRConfig(credential_path="./credentials.json")

    async with DoubaoASR(config) as asr:
        # 批量处理多个文件
        audio_files = ["audio1.wav", "audio2.wav", "audio3.wav"]
        for audio_path in audio_files:
            try:
                result = await asr.transcribe(audio_path)
                print(f"{audio_path}: {result}")
            except ASRError as e:
                print(f"{audio_path} 识别失败: {e}")
                if e.response:
                    print(f"  错误详情: {e.response.error_msg}")

        # 流式识别并访问详细信息
        async for resp in asr.transcribe_stream("long_audio.wav"):
            if resp.type == ResponseType.FINAL_RESULT and resp.extra:
                print(f"文本: {resp.text}")
                print(f"音频时长: {resp.extra.audio_duration} ms")
                print(f"模型 RTF: {resp.extra.model_avg_rtf:.4f}")
                print(f"首响应时间: {resp.extra.model_send_first_response} ms")

asyncio.run(main())
```

--------------------------------

### ASRResponse and Data Model Explanation

Source: https://context7.com/starccy/doubaoime-asr/llms.txt

Details the structure and fields of `ASRResponse`, `ASRResult`, `ASRAlternative`, `ASRWord`, and `ASRExtra` objects returned by the ASR functions. Explains response types, text content, timing, confidence, and performance metrics.

```python
from doubaoime_asr import (
    ASRResponse, ASRResult, ASRAlternative, ASRWord,
    ASRExtra, ResponseType
)

# ASRResponse 字段说明
resp: ASRResponse
resp.type          # ResponseType 枚举，表示响应类型
resp.text          # str，当前识别文本（中间或最终）
resp.is_final      # bool，是否为最终结果
resp.vad_finished  # bool，VAD 是否已结束
resp.error_msg     # str，错误信息（type == ERROR 时有值）
resp.results       # List[ASRResult]，详细识别结果列表
resp.extra         # ASRExtra，附加性能数据
resp.raw_json      # dict，原始 JSON 数据

# ASRResult 字段说明
result: ASRResult
result.text         # str，该段识别文本
result.start_time   # float，起始时间（秒）
result.end_time     # float，结束时间（秒）
result.confidence   # float，置信度
result.is_interim   # bool，是否为中间结果
result.alternatives # List[ASRAlternative]，候选结果
```

--------------------------------

### DoubaoASR Client Class

Source: https://context7.com/starccy/doubaoime-asr/llms.txt

The `DoubaoASR` class provides a lower-level interface for ASR tasks and can be used as an asynchronous context manager for efficient reuse of configurations across multiple audio processing operations.

```APIDOC
## DoubaoASR — 底层客户端类

`DoubaoASR` 是三个便捷函数的底层实现，支持作为异步上下文管理器使用。当需要复用同一配置对多个音频进行识别时，直接使用该类可避免重复初始化。

```python
import asyncio
from doubaoime_asr import DoubaoASR, ASRConfig, ASRError, ResponseType

async def main():
    config = ASRConfig(credential_path="./credentials.json")

    async with DoubaoASR(config) as asr:
        # 批量处理多个文件
        audio_files = ["audio1.wav", "audio2.wav", "audio3.wav"]
        for audio_path in audio_files:
            try:
                result = await asr.transcribe(audio_path)
                print(f"{audio_path}: {result}")
            except ASRError as e:
                print(f"{audio_path} 识别失败: {e}")
                if e.response:
                    print(f"  错误详情: {e.response.error_msg}")

        # 流式识别并访问详细信息
        async for resp in asr.transcribe_stream("long_audio.wav"):
            if resp.type == ResponseType.FINAL_RESULT and resp.extra:
                print(f"文本: {resp.text}")
                print(f"音频时长: {resp.extra.audio_duration} ms")
                print(f"模型 RTF: {resp.extra.model_avg_rtf:.4f}")
                print(f"首响应时间: {resp.extra.model_send_first_response} ms")

asyncio.run(main())
```
```

--------------------------------

### Cache Credentials for ASR Configuration

Source: https://github.com/starccy/doubaoime-asr/blob/main/README.md

Specify a `credential_path` in `ASRConfig` to automatically cache device registration and authentication tokens. This avoids repeated registration.

```python
config = ASRConfig(credential_path="~/.config/doubaoime-asr/credentials.json")
```

--------------------------------

### ASRResponse and Data Models

Source: https://context7.com/starccy/doubaoime-asr/llms.txt

Details the structure and fields of `ASRResponse` and its associated data models (`ASRResult`, `ASRAlternative`, `ASRWord`, `ASRExtra`), which are used for unified response handling in streaming recognition.

```APIDOC
## ASRResponse 与数据模型

`ASRResponse` 是所有流式识别函数产出的统一响应对象，包含响应类型、识别文本、详细结果列表和附加性能信息。

```python
from doubaoime_asr import (
    ASRResponse, ASRResult, ASRAlternative, ASRWord,
    ASRExtra, ResponseType
)

# ASRResponse 字段说明
resp: ASRResponse
resp.type          # ResponseType 枚举，表示响应类型
resp.text          # str，当前识别文本（中间或最终）
resp.is_final      # bool，是否为最终结果
resp.vad_finished  # bool，VAD 是否已结束
resp.error_msg     # str，错误信息（type == ERROR 时有值）
resp.results       # List[ASRResult]，详细识别结果列表
resp.extra         # ASRExtra，附加性能数据
resp.raw_json      # dict，原始 JSON 数据

# ASRResult 字段说明
result: ASRResult
result.text         # str，该段识别文本
result.start_time   # float，起始时间（秒）
result.end_time     # float，结束时间（秒）
result.confidence   # float，置信度
result.is_interim   # bool，是否为中间结果
result.alternatives # List[ASRAlternative]，候选结果
```
```

--------------------------------

### List Response Types

Source: https://context7.com/starccy/doubaoime-asr/llms.txt

Prints a list of all available ResponseType enumerations. Useful for understanding the different stages and types of responses from the ASR system.

```python
print(list(ResponseType))
```

--------------------------------

### Basic Audio File Transcription

Source: https://github.com/starccy/doubaoime-asr/blob/main/README.md

Perform non-streaming speech recognition on an audio file. The `ASRConfig` handles device registration and credential caching.

```python
import asyncio
from doubaoime_asr import transcribe, ASRConfig

async def main():
    # 配置（首次运行会自动注册设备，并将凭据保存到指定文件）
    config = ASRConfig(credential_path="./credentials.json")

    # 识别音频文件
    result = await transcribe("audio.wav", config=config)
    print(f"识别结果: {result}")

asyncio.run(main())
```

--------------------------------

### Named Entity Recognition (NER) on Text

Source: https://context7.com/starccy/doubaoime-asr/llms.txt

Performs Named Entity Recognition on provided text using Doubao's SAMI service. Automatically handles authentication and token management. Can be used independently or in conjunction with ASR results.

```python
from doubaoime_asr import ASRConfig, ner, NerResponse

config = ASRConfig(credential_path="./credentials.json")

# 对识别出的文本进行 NER，用于输入法联想优化
text = "张三李四以及张三在使用 Chrome 浏览器"
response: NerResponse = ner(config, text)

# 输出 NerResponse 结构
for result in response.results:
    print(f"原文: {result.text}")
    for word in result.words:
        print(f"  实体: '{word.word}' (出现 {word.freq} 次)")

# 输出示例:
# 原文: 张三李四以及张三在使用 Chrome 浏览器
#   实体: '张三' (出现 2 次)
#   实体: '李四' (出现 1 次)
#   实体: 'Chrome 浏览器' (出现 1 次)
#   实体: '张三李四' (出现 1 次)
#   实体: 'Chrome' (出现 1 次)

# 配合 ASR 使用：先转写再实体提取
import asyncio
from doubaoime_asr import transcribe

async def transcribe_and_ner(audio_path: str):
    text = await transcribe(audio_path, config=config)
    print(f"识别文本: {text}")
    entities = ner(config, text)
    print(f"提取实体: {[w.word for r in entities.results for w in r.words]}")

asyncio.run(transcribe_and_ner("meeting_record.wav"))
```

--------------------------------

### transcribe_realtime

Source: https://github.com/starccy/doubaoime-asr/blob/main/README.md

Performs real-time streaming speech recognition, accepting an asynchronous iterator of PCM audio data.

```APIDOC
## transcribe_realtime

### Description
Real-time streaming speech recognition, accepts an asynchronous iterator of PCM audio data.

### Method Signature
```python
async def transcribe_realtime(
    audio_source: AsyncIterator[bytes],
    *,
    config: ASRConfig | None = None,
) -> AsyncIterator[ASRResponse]
```

### Parameters
- **audio_source** (AsyncIterator[bytes]) - Required - An asynchronous iterator yielding raw PCM byte data.
- **config** (ASRConfig | None) - Optional - ASR configuration object.
```

--------------------------------

### Perform Streaming Speech Recognition with transcribe_stream

Source: https://context7.com/starccy/doubaoime-asr/llms.txt

Utilize `transcribe_stream` for streaming ASR, yielding `ASRResponse` objects asynchronously. This is suitable for real-time progress display and handling multi-segment audio, providing detailed events like VAD, interim/final results, and errors.

```python
import asyncio
from doubaoime_asr import transcribe_stream, ASRConfig, ResponseType

async def main():
    config = ASRConfig(credential_path="./credentials.json")

    with open("audio.wav", "rb") as f:
        audio_data = f.read()

    async for response in transcribe_stream(audio_data, config=config, realtime=False):
        match response.type:
            case ResponseType.TASK_STARTED:
                print("[系统] 任务已启动")
            case ResponseType.SESSION_STARTED:
                print("[系统] 会话已启动")
            case ResponseType.VAD_START:
                print("[VAD] 检测到语音活动开始")
            case ResponseType.INTERIM_RESULT:
                print(f"[中间] {response.text}")
            case ResponseType.FINAL_RESULT:
                # Get timestamps for recognition results
                if response.results:
                    r = response.results[0]
                    print(f"[最终] ({r.start_time:.2f}s ~ {r.end_time:.2f}s) {response.text}")
                    # Access alternative results
                    for alt in r.alternatives:
                        print(f"  候选: {alt.text}")
                    # Access word-level timestamps
                    if r.alternatives:
                        for word in r.alternatives[0].words:
                            print(f"  词: '{word.word}' [{word.start_time:.2f}s-{word.end_time:.2f}s]")
            case ResponseType.SESSION_FINISHED:
                print("[系统] 会话结束")
            case ResponseType.ERROR:
                print(f"[错误] {response.error_msg}")

asyncio.run(main())
```

--------------------------------

### Calculate MD5 Hash for x-ss-stub Header

Source: https://github.com/starccy/doubaoime-asr/blob/main/wave_protocol.md

Computes the MD5 hash of the ciphertext and converts it to an uppercase hexadecimal string. This hash is used for the `x-ss-stub` HTTP header.

```python
import hashlib

stub = hashlib.md5(ciphertext).hexdigest().upper()
```

--------------------------------

### transcribe

Source: https://github.com/starccy/doubaoime-asr/blob/main/README.md

Performs non-streaming speech recognition and returns the final result.

```APIDOC
## transcribe

### Description
Non-streaming speech recognition, directly returns the final result.

### Method Signature
```python
async def transcribe(
    audio: str | Path | bytes,
    *, 
    config: ASRConfig | None = None,
    on_interim: Callable[[str], None] | None = None,
    realtime: bool = False,
) -> str
```

### Parameters
- **audio** (str | Path | bytes) - Required - Path to the audio file or raw PCM byte data.
- **config** (ASRConfig | None) - Optional - ASR configuration object.
- **on_interim** (Callable[[str], None] | None) - Optional - Callback function for interim results.
- **realtime** (bool) - Optional - If True, simulates real-time sending with fixed delays between audio frames. If False, sends data as quickly as possible.
```

--------------------------------

### Handshake Endpoint

Source: https://github.com/starccy/doubaoime-asr/blob/main/wave_protocol.md

Initiates the secure connection by exchanging cryptographic parameters. Requires a POST request to the handshake endpoint with a JSON payload containing client information and key shares.

```APIDOC
## POST https://keyhub.zijieapi.com/handshake

### Description
Initiates the secure connection by exchanging cryptographic parameters. Requires a POST request to the handshake endpoint with a JSON payload containing client information and key shares.

### Method
POST

### Endpoint
https://keyhub.zijieapi.com/handshake

### Parameters
#### Request Body
- **version** (integer) - Required - Protocol version.
- **random** (string) - Required - Base64 encoded 32-byte client random nonce.
- **app_id** (string) - Required - Application identifier.
- **did** (string) - Required - Device ID.
- **key_shares** (array) - Required - List of key shares.
  - **curve** (string) - Required - Elliptic curve name (e.g., "secp256r1").
  - **pubkey** (string) - Required - Base64 encoded 65-byte uncompressed public key.
- **cipher_suites** (array) - Required - List of supported cipher suites (e.g., [4097]).

### Request Example
```json
{
  "version": 2,
  "random": "<base64(32字节客户端随机数)>",
  "app_id": "401734",
  "did": "<设备ID>",
  "key_shares": [
    {
      "curve": "secp256r1",
      "pubkey": "<base64(65字节未压缩公钥)>"
    }
  ],
  "cipher_suites": [4097]
}
```

### Response
#### Success Response (200)
- **version** (integer) - Protocol version.
- **random** (string) - Base64 encoded 32-byte server random nonce.
- **key_share** (object) - Server's key share.
  - **curve** (string) - Elliptic curve name.
  - **pubkey** (string) - Base64 encoded 65-byte uncompressed server public key.
- **cipher_suite** (integer) - Negotiated cipher suite.
- **ticket** (string) - Base64 encoded session ticket.
- **expire_time** (integer) - Session ticket expiration timestamp.

### Response Example
```json
{
  "version": 2,
  "random": "<base64(32字节服务器随机数)>",
  "key_share": {
    "curve": "secp256r1",
    "pubkey": "<base64(65字节服务器公钥)>"
  },
  "cipher_suite": 4097,
  "ticket": "<base64(会话 ticket)>",
  "expire_time": 1738763280
}
```
```

--------------------------------

### Derive Encryption Key using HKDF-SHA256

Source: https://github.com/starccy/doubaoime-asr/blob/main/wave_protocol.md

Derives the 32-byte encryption key from the ECDH shared secret using HKDF with SHA256. The salt is a concatenation of client and server random nonces, and a hardcoded info string is used.

```python
from cryptography.hazmat.primitives.kdf.hkdf import HKDF
from cryptography.hazmat.primitives import hashes

# 参数
salt = client_random + server_random  # 64 字节
info = b"4e30514609050cd3"            # 16 字节，硬编码常量

# 派生
encryption_key = HKDF(
    algorithm=hashes.SHA256(),
    length=32,
    salt=salt,
    info=info
).derive(shared_key)
```

--------------------------------

### Perform Non-Streaming Speech Recognition with transcribe

Source: https://context7.com/starccy/doubaoime-asr/llms.txt

Use the `transcribe` function for non-streaming ASR. It accepts audio data (bytes, file path, or Path object) and returns the final text. An `on_interim` callback can be provided for intermediate results.

```python
import asyncio
import requests
from doubaoime_asr import transcribe, ASRConfig

async def main():
    config = ASRConfig(credential_path="./credentials.json")

    # Get test audio from URL
    audio_data = requests.get(
        "https://github.com/liangstein/Chinese-speech-to-text/raw/refs/heads/master/1.wav"
    ).content

    # Basic usage: get final recognition result directly
    result = await transcribe(audio_data, config=config)
    print(f"识别结果: {result}")
    # Output example: 识别结果: 今天天气怎么样

    # With interim results callback
    def on_interim(text: str):
        print(f"  [中间] {text}")

    result = await transcribe(
        audio_data,
        config=config,
        on_interim=on_interim,
        realtime=False,  # False (default): send as fast as possible, faster speed
    )
    print(f"  [最终] {result}")

    # Pass file path
    result = await transcribe("recording.wav", config=config)
    print(f"文件识别: {result}")

asyncio.run(main())
```

--------------------------------

### ChaCha20 Encryption/Decryption Function

Source: https://github.com/starccy/doubaoime-asr/blob/main/wave_protocol.md

Implements ChaCha20 stream encryption/decryption. Note that the cryptography library requires a 16-byte nonce, which is formed by prepending 4 null bytes to the 12-byte Wave protocol nonce.

```python
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms

def chacha20_crypt(key: bytes, nonce: bytes, data: bytes) -> bytes:
    """ChaCha20 加密/解密 (对称操作)"""
    # Python cryptography 库需要 16 字节 nonce
    # 格式: 4 字节 counter (固定为 0) + 12 字节 nonce
    nonce_16 = b'\x00\x00\x00\x00' + nonce

    cipher = Cipher(
        algorithms.ChaCha20(key, nonce_16),
        mode=None
    )
    encryptor = cipher.encryptor()
    return encryptor.update(data) + encryptor.finalize()
```

--------------------------------

### Wave Protocol Handshake Request Format

Source: https://github.com/starccy/doubaoime-asr/blob/main/wave_protocol.md

Defines the JSON structure for the handshake request, including version, client random nonce, app ID, device ID, key shares, and supported cipher suites.

```json
{
  "version": 2,
  "random": "<base64(32字节客户端随机数)>",
  "app_id": "401734",
  "did": "<设备ID>",
  "key_shares": [
    {
      "curve": "secp256r1",
      "pubkey": "<base64(65字节未压缩公钥)>"
    }
  ],
  "cipher_suites": [4097]
}
```

--------------------------------

### transcribe_stream — Streaming Speech Recognition

Source: https://context7.com/starccy/doubaoime-asr/llms.txt

This function provides streaming speech recognition by yielding ASRResponse objects as an asynchronous iterator. It's suitable for real-time display of recognition progress and handling multi-segment audio, providing detailed event sequences including task/session status, VAD events, and results with timestamps.

```APIDOC
## transcribe_stream

### Description
Performs streaming speech recognition, yielding `ASRResponse` objects as an asynchronous iterator. This is ideal for real-time display and handling multi-segment audio, providing a full sequence of events like task/session start, VAD detection, intermediate and final results with timestamps, and errors.

### Method
`async transcribe_stream(audio_data: Union[str, Path, bytes], config: ASRConfig, realtime: bool = False) -> AsyncIterator[ASRResponse]`

### Parameters
- **audio_data** (Union[str, Path, bytes]): Path to the audio file, a `Path` object, or raw PCM audio bytes.
- **config** (ASRConfig): An instance of `ASRConfig` for configuration and credentials.
- **realtime** (bool, optional): If True, simulates sending audio at its real-time rate. Defaults to False (sends as fast as possible).

### Response
- **AsyncIterator[ASRResponse]**: An asynchronous iterator yielding `ASRResponse` objects.

### ASRResponse Structure
- **type** (ResponseType): The type of the response event (e.g., TASK_STARTED, SESSION_STARTED, VAD_START, INTERIM_RESULT, FINAL_RESULT, SESSION_FINISHED, ERROR).
- **text** (str, optional): The recognized text for result events.
- **results** (List[RecognitionResult], optional): List of recognition results for FINAL_RESULT events.
- **error_msg** (str, optional): Error message for ERROR events.

### RecognitionResult Structure
- **start_time** (float): Start time of the recognized segment in seconds.
- **end_time** (float): End time of the recognized segment in seconds.
- **alternatives** (List[AlternativeResult]): List of alternative transcriptions.

### AlternativeResult Structure
- **text** (str): The alternative transcription text.
- **words** (List[WordInfo]): List of word-level information including timestamps.

### WordInfo Structure
- **word** (str): The recognized word.
- **start_time** (float): Start time of the word in seconds.
- **end_time** (float): End time of the word in seconds.

### Request Example
```python
import asyncio
from doubaoime_asr import transcribe_stream, ASRConfig, ResponseType

async def main():
    config = ASRConfig(credential_path="./credentials.json")

    with open("audio.wav", "rb") as f:
        audio_data = f.read()

    async for response in transcribe_stream(audio_data, config=config, realtime=False):
        match response.type:
            case ResponseType.TASK_STARTED:
                print("[System] Task started")
            case ResponseType.SESSION_STARTED:
                print("[System] Session started")
            case ResponseType.VAD_START:
                print("[VAD] Voice activity detected start")
            case ResponseType.INTERIM_RESULT:
                print(f"[Interim] {response.text}")
            case ResponseType.FINAL_RESULT:
                if response.results:
                    r = response.results[0]
                    print(f"[Final] ({r.start_time:.2f}s ~ {r.end_time:.2f}s) {response.text}")
                    for alt in r.alternatives:
                        print(f"  Alternative: {alt.text}")
                    if r.alternatives and r.alternatives[0].words:
                        for word in r.alternatives[0].words:
                            print(f"  Word: '{word.word}' [{word.start_time:.2f}s-{word.end_time:.2f}s]")
            case ResponseType.SESSION_FINISHED:
                print("[System] Session finished")
            case ResponseType.ERROR:
                print(f"[Error] {response.error_msg}")

asyncio.run(main())
```
```

--------------------------------

### transcribe_stream

Source: https://github.com/starccy/doubaoime-asr/blob/main/README.md

Performs streaming speech recognition and returns an asynchronous iterator of ASRResponse objects.

```APIDOC
## transcribe_stream

### Description
Streaming speech recognition, returns an asynchronous iterator of `ASRResponse`.

### Method Signature
```python
async def transcribe_stream(
    audio: str | Path | bytes,
    *,
    config: ASRConfig | None = None,
    realtime: bool = False,
) -> AsyncIterator[ASRResponse]
```

### Parameters
- **audio** (str | Path | bytes) - Required - Path to the audio file or raw PCM byte data.
- **config** (ASRConfig | None) - Optional - ASR configuration object.
- **realtime** (bool) - Optional - If True, simulates real-time sending with fixed delays between audio frames. If False, sends data as quickly as possible.
```

--------------------------------

### ner

Source: https://context7.com/starccy/doubaoime-asr/llms.txt

Performs Named Entity Recognition (NER) on a given text using Doubao's SAMI service, returning a list of high-frequency words and entities along with their frequencies.

```APIDOC
## ner — 命名实体识别

`ner` 函数通过豆包的 SAMI 服务对文本进行命名实体识别，返回文本中出现的高频词和实体列表及其出现频次。内部自动管理 Wave 加密握手、SAMI token 获取与缓存，无需手动处理。

```python
from doubaoime_asr import ASRConfig, ner, NerResponse

config = ASRConfig(credential_path="./credentials.json")

# 对识别出的文本进行 NER，用于输入法联想优化
text = "张三李四以及张三在使用 Chrome 浏览器"
response: NerResponse = ner(config, text)

# 输出 NerResponse 结构
for result in response.results:
    print(f"原文: {result.text}")
    for word in result.words:
        print(f"  实体: '{word.word}' (出现 {word.freq} 次)")

# 输出示例:
# 原文: 张三李四以及张三在使用 Chrome 浏览器
#   实体: '张三' (出现 2 次)
#   实体: '李四' (出现 1 次)
#   实体: 'Chrome 浏览器' (出现 1 次)
#   实体: '张三李四' (出现 1 次)
#   实体: 'Chrome' (出现 1 次)

# 配合 ASR 使用：先转写再实体提取
import asyncio
from doubaoime_asr import transcribe

async def transcribe_and_ner(audio_path: str):
    text = await transcribe(audio_path, config=config)
    print(f"识别文本: {text}")
    entities = ner(config, text)
    print(f"提取实体: {[w.word for r in entities.results for w in r.words]}")

asyncio.run(transcribe_and_ner("meeting_record.wav"))
```
```

--------------------------------

### Wave Protocol Handshake Response Format

Source: https://github.com/starccy/doubaoime-asr/blob/main/wave_protocol.md

Defines the JSON structure for the handshake response, including version, server random nonce, server public key share, cipher suite, session ticket, and expiration time.

```json
{
  "version": 2,
  "random": "<base64(32字节服务器随机数)>",
  "key_share": {
    "curve": "secp256r1",
    "pubkey": "<base64(65字节服务器公钥)>"
  },
  "cipher_suite": 4097,
  "ticket": "<base64(会话 ticket)>",
  "expire_time": 1738763280
}
```

--------------------------------

### ResponseType

Source: https://github.com/starccy/doubaoime-asr/blob/main/README.md

Enum for different response types in streaming recognition.

```APIDOC
## ResponseType

### Description
Enum for different response types in streaming recognition.

### Enum Values
- **TASK_STARTED**: Task has started.
- **SESSION_STARTED**: Session has started.
- **VAD_START**: Voice activity detection start.
- **INTERIM_RESULT**: Interim recognition result.
- **FINAL_RESULT**: Final recognition result.
- **SESSION_FINISHED**: Session has finished.
- **ERROR**: An error occurred.
```

--------------------------------

### HKDF Key Derivation

Source: https://github.com/starccy/doubaoime-asr/blob/main/wave_protocol.md

Derives the encryption key from the ECDH shared secret using HKDF-SHA256 with a specific salt and info string.

```APIDOC
## HKDF Key Derivation

### Description
Derives the encryption key from the ECDH shared secret using HKDF-SHA256 with a specific salt and info string.

### Method
HKDF-SHA256 Key Derivation

### Parameters
- **shared_key** (bytes) - The 32-byte shared secret key from ECDH.
- **client_random** (bytes) - The 32-byte client random nonce from the handshake request.
- **server_random** (bytes) - The 32-byte server random nonce from the handshake response.

### Key Parameters
- **Salt**: Concatenation of `client_random` and `server_random` (64 bytes).
- **Info String**: A hardcoded 16-byte string, e.g., `"4e30514609050cd3"`.
- **Output Length**: 32 bytes for the encryption key.

### Code Example
```python
from cryptography.hazmat.primitives.kdf.hkdf import HKDF
from cryptography.hazmat.primitives import hashes

# 参数
salt = client_random + server_random  # 64 字节
info = b"4e30514609050cd3"            # 16 字节，硬编码常量

# 派生
encryption_key = HKDF(
    algorithm=hashes.SHA256(),
    length=32,
    salt=salt,
    info=info
).derive(shared_key)
```

### Output
- **encryption_key** (bytes) - The derived 32-byte encryption key.
```

--------------------------------

### ECDH Shared Key Calculation

Source: https://github.com/starccy/doubaoime-asr/blob/main/wave_protocol.md

Derives a shared secret key using the Elliptic Curve Diffie-Hellman (ECDH) algorithm between the client's private key and the server's public key.

```APIDOC
## ECDH Shared Key Calculation

### Description
Derives a shared secret key using the Elliptic Curve Diffie-Hellman (ECDH) algorithm between the client's private key and the server's public key.

### Method
ECDH Key Exchange

### Parameters
- **client_private_key** (object) - The client's ephemeral private key.
- **server_public_key** (object) - The server's public key received during the handshake.

### Code Example
```python
from cryptography.hazmat.primitives.asymmetric import ec

# 使用客户端私钥和服务器公钥计算共享密钥
shared_key = client_private_key.exchange(
    ec.ECDH(),
    server_public_key
)
# shared_key: 32 字节
```

### Output
- **shared_key** (bytes) - The derived 32-byte shared secret key.
```

--------------------------------

### ChaCha20 Encryption/Decryption

Source: https://github.com/starccy/doubaoime-asr/blob/main/wave_protocol.md

Performs symmetric encryption and decryption using the ChaCha20 stream cipher. The same key is used for both encryption and decryption, with different nonces.

```APIDOC
## ChaCha20 Encryption/Decryption

### Description
Performs symmetric encryption and decryption using the ChaCha20 stream cipher. The same key is used for both encryption and decryption, with different nonces.

### Algorithm
ChaCha20 (pure stream cipher, no authentication tag)

### Parameters
- **key** (bytes) - The 32-byte encryption key derived via HKDF.
- **nonce** (bytes) - The 12-byte nonce. Must be unique for each encryption/decryption operation with the same key.
- **data** (bytes) - The plaintext data to encrypt or ciphertext to decrypt.

### Code Example
```python
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms

def chacha20_crypt(key: bytes, nonce: bytes, data: bytes) -> bytes:
    """ChaCha20 加密/解密 (对称操作)"""
    # Python cryptography 库需要 16 字节 nonce
    # 格式: 4 字节 counter (固定为 0) + 12 字节 nonce
    nonce_16 = b'\x00\x00\x00\x00' + nonce

    cipher = Cipher(
        algorithms.ChaCha20(key, nonce_16),
        mode=None
    )
    encryptor = cipher.encryptor()
    return encryptor.update(data) + encryptor.finalize()
```

### Usage
- **Encryption**: Encrypt plaintext to ciphertext.
- **Decryption**: Decrypt ciphertext back to plaintext.

### Nonce Format
- **Length**: 12 bytes.
- **Generation**: Randomly generated for each request.
- **Transmission**: Base64 encoded and placed in the `x-tt-e-p` header for requests, and the response header for responses.
```

--------------------------------

### Encrypted Request Headers

Source: https://github.com/starccy/doubaoime-asr/blob/main/wave_protocol.md

Specifies the HTTP headers required for sending an encrypted request.

```APIDOC
## Encrypted Request Headers

### Description
Specifies the HTTP headers required for sending an encrypted request.

### Headers
- **`x-tt-e-b`**: (string) - Indicates that the request body is encrypted. Value is typically "1".
- **`x-tt-e-t`**: (string) - The session ticket obtained during the handshake.
- **`x-tt-e-p`**: (string) - Base64 encoded 12-byte nonce used for encrypting the request body.
- **`x-ss-stub`**: (string) - Uppercase MD5 hash of the ciphertext.

### x-ss-stub Calculation
```python
import hashlib

stub = hashlib.md5(ciphertext).hexdigest().upper()
```

### Full Request Example
```http
POST /api/v3/context/ime/ner?device_platform=android&... HTTP/1.1
Host: speech.bytedance.com
Content-Type: application/json
x-tt-e-b: 1
x-tt-e-t: MIGYBAwrT...（ticket）
x-tt-e-p: abc123...（nonce Base64）
x-ss-stub: A1B2C3D4E5F6...（MD5 大写）
x-api-app-key: SYlxZr6LnvBaIVmF
x-api-token: eyJhbGciOiJFUzI1NiI... 

<加密后的请求体>
```
```
Tokens:
Raw