### Install Example Dependencies for Real-time Recognition Source: https://github.com/starccy/doubaoime-asr/blob/main/README.md Install additional libraries required for running real-time microphone recognition examples. This includes `sounddevice` and `numpy`. ```bash pip install sounddevice numpy ``` ```bash # 或 pip install doubaoime-asr[examples] ``` -------------------------------- ### Install DoubaoIME ASR from Local Source Source: https://github.com/starccy/doubaoime-asr/blob/main/README.md Install the library from a local clone of the Git repository. This is useful for development or when you need to modify the source code. ```bash git clone https://github.com/starccy/doubaoime-asr.git cd doubaoime-asr pip install -e . ``` -------------------------------- ### Install DoubaoIME ASR from Git Repository Source: https://github.com/starccy/doubaoime-asr/blob/main/README.md Install the library directly from the Git repository using pip. This is a convenient way to get the latest version. ```bash pip install git+https://github.com/starccy/doubaoime-asr.git ``` -------------------------------- ### Install doubaoime-asr and Dependencies Source: https://context7.com/starccy/doubaoime-asr/llms.txt Install system dependencies for Opus codec and the doubaoime-asr Python library. Additional dependencies are required for real-time microphone recognition. ```bash # Install system dependencies (Opus codec library) # Debian/Ubuntu: sudo apt install libopus0 # macOS: brew install opus # Install doubaoime-asr pip install git+https://github.com/starccy/doubaoime-asr.git # Install extra dependencies for real-time microphone recognition pip install doubaoime-asr[examples] ``` -------------------------------- ### Install Opus Audio Codec Library (Debian/Ubuntu) Source: https://github.com/starccy/doubaoime-asr/blob/main/README.md Install the Opus audio codec library on Debian or Ubuntu systems. This is a system dependency for the ASR client. ```bash sudo apt install libopus0 ``` -------------------------------- ### Install Opus Audio Codec Library (macOS) Source: https://github.com/starccy/doubaoime-asr/blob/main/README.md Install the Opus audio codec library on macOS using Homebrew. This is a system dependency for the ASR client. ```bash brew install opus ``` -------------------------------- ### Install Opus Audio Codec Library (Arch Linux) Source: https://github.com/starccy/doubaoime-asr/blob/main/README.md Install the Opus audio codec library on Arch Linux systems. This is a system dependency for the ASR client. ```bash sudo pacman -S opus ``` -------------------------------- ### Example Encrypted HTTP Request Source: https://github.com/starccy/doubaoime-asr/blob/main/wave_protocol.md Illustrates a complete HTTP POST request with Wave protocol encryption headers, including `x-tt-e-b`, `x-tt-e-t`, `x-tt-e-p`, and `x-ss-stub`, along with API-specific headers. ```http POST /api/v3/context/ime/ner?device_platform=android&... HTTP/1.1 Host: speech.bytedance.com Content-Type: application/json x-tt-e-b: 1 x-tt-e-t: MIGYBAwrT...(ticket) x-tt-e-p: abc123...(nonce Base64) x-ss-stub: A1B2C3D4E5F6...(MD5 大写) x-api-app-key: SYlxZr6LnvBaIVmF x-api-token: eyJhbGciOiJFUzI1NiI... <加密后的请求体> ``` -------------------------------- ### Real-time Microphone Streaming Transcription Source: https://context7.com/starccy/doubaoime-asr/llms.txt Transcribes audio from a microphone in real-time using an asynchronous generator. Requires `sounddevice` and `numpy`. Handles session start, interim, and final results, as well as errors. ```python import asyncio import sounddevice as sd import numpy as np from doubaoime_asr import transcribe_realtime, ASRConfig, ResponseType async def mic_audio_generator(sample_rate=16000, channels=1, frame_duration_ms=20): """麦克风异步音频生成器""" samples_per_frame = sample_rate * frame_duration_ms // 1000 queue: asyncio.Queue[bytes] = asyncio.Queue() loop = asyncio.get_event_loop() def audio_callback(indata, frames, time_info, status): loop.call_soon_threadsafe(queue.put_nowait, indata.tobytes()) with sd.InputStream( samplerate=sample_rate, channels=channels, dtype=np.int16, blocksize=samples_per_frame, callback=audio_callback, ): while True: yield await queue.get() async def main(): config = ASRConfig(credential_path="./credentials.json") print("开始实时识别,按 Ctrl+C 停止...") try: async for response in transcribe_realtime( mic_audio_generator( sample_rate=config.sample_rate, channels=config.channels, frame_duration_ms=config.frame_duration_ms, ), config=config, ): if response.type == ResponseType.SESSION_STARTED: print("会话已启动,请开始说话...\n") elif response.type == ResponseType.INTERIM_RESULT: # 使用回车符在同一行更新中间结果 print(f"\r[识别中] {response.text}", end="", flush=True) elif response.type == ResponseType.FINAL_RESULT: print(f"\r[最终结果] {response.text} ") elif response.type == ResponseType.ERROR: print(f"\n[错误] {response.error_msg}") except KeyboardInterrupt: print("\n已停止录音") asyncio.run(main()) ``` -------------------------------- ### Streaming Audio File Recognition Source: https://github.com/starccy/doubaoime-asr/blob/main/README.md Perform streaming speech recognition on an audio file to get intermediate and final results. This is useful for real-time feedback or detailed status. ```python import asyncio from doubaoime_asr import transcribe_stream, ASRConfig, ResponseType async def main(): config = ASRConfig(credential_path="./credentials.json") async for response in transcribe_stream("audio.wav", config=config): match response.type: case ResponseType.INTERIM_RESULT: print(f"[中间结果] {response.text}") case ResponseType.FINAL_RESULT: print(f"[最终结果] {response.text}") case ResponseType.ERROR: print(f"[错误] {response.error_msg}") asyncio.run(main()) ``` -------------------------------- ### Configure ASR with ASRConfig Source: https://context7.com/starccy/doubaoime-asr/llms.txt Initialize ASRConfig for managing credentials and audio parameters. Supports automatic device registration, credential caching, and overriding audio settings. ```python from doubaoime_asr import ASRConfig # Minimal configuration: automatic device registration, no credential persistence config = ASRConfig() # Recommended configuration: persist credentials to a file to avoid re-registration config = ASRConfig(credential_path="~/.config/doubaoime-asr/credentials.json") # Full configuration example: override audio parameters and connection settings config = ASRConfig( credential_path="./credentials.json", sample_rate=16000, # Sample rate (Hz), default 16000 channels=1, # Number of channels, default 1 (mono) frame_duration_ms=20, # Frame duration (ms), default 20 enable_punctuation=True, # Enable punctuation, default True enable_speech_rejection=False, connect_timeout=10.0, # WebSocket connection timeout (seconds) recv_timeout=10.0, # Receive timeout (seconds) app_name="com.android.chrome", # Simulated host application ) # Manually provide existing credentials (skip automatic registration) config = ASRConfig( device_id="7123456789012345", token="eyJhbGci...", ) # Verify credential loading config.ensure_credentials() print(f"device_id: {config.device_id}") print(f"token: {config.token[:20]}...") ``` -------------------------------- ### ASRConfig - Configuration and Credential Management Source: https://context7.com/starccy/doubaoime-asr/llms.txt The ASRConfig class is central to all functionalities, handling automatic device registration, credential caching, and overriding audio parameters and connection settings. It supports direct parameter input, credential file paths, and automatic registration, with a priority given to directly provided parameters. ```APIDOC ## ASRConfig ### Description Handles core configuration, including automatic device registration, credential caching, and overriding audio/connection parameters. Credentials can be provided directly, loaded from a file, or obtained via automatic registration. ### Usage ```python from doubaoime_asr import ASRConfig # Minimal configuration: automatic device registration, no credential persistence config = ASRConfig() # Recommended configuration: persistent credentials to avoid re-registration config = ASRConfig(credential_path="~/.config/doubaoime-asr/credentials.json") # Full configuration example: overriding audio parameters and connection settings config = ASRConfig( credential_path="./credentials.json", sample_rate=16000, # Sample rate (Hz), default 16000 channels=1, # Number of channels, default 1 (mono) frame_duration_ms=20, # Frame duration (ms), default 20 enable_punctuation=True, # Enable punctuation, default True enable_speech_rejection=False, connect_timeout=10.0, # WebSocket connection timeout (seconds) recv_timeout=10.0, # Receive timeout (seconds) app_name="com.android.chrome", # Simulated host application ) # Manually provide existing credentials (skips automatic registration) config = ASRConfig( device_id="7123456789012345", token="eyJhbGci...", ) # Verify credential loading config.ensure_credentials() print(f"device_id: {config.device_id}") print(f"token: {config.token[:20]}...") ``` ### Parameters - **credential_path** (str, optional): Path to a file for persisting and loading credentials. - **sample_rate** (int, optional): Audio sample rate in Hz. Defaults to 16000. - **channels** (int, optional): Number of audio channels. Defaults to 1 (mono). - **frame_duration_ms** (int, optional): Duration of audio frames in milliseconds. Defaults to 20. - **enable_punctuation** (bool, optional): Whether to enable punctuation. Defaults to True. - **enable_speech_rejection** (bool, optional): Whether to enable speech rejection. Defaults to False. - **connect_timeout** (float, optional): Timeout for WebSocket connection in seconds. - **recv_timeout** (float, optional): Timeout for receiving data in seconds. - **app_name** (str, optional): The name of the simulated host application. - **device_id** (str, optional): Manually provided device ID. - **token** (str, optional): Manually provided authentication token. ``` -------------------------------- ### ASRConfig Source: https://github.com/starccy/doubaoime-asr/blob/main/README.md Configuration class for ASR settings. ```APIDOC ## ASRConfig ### Description Configuration class for ASR settings. ### Parameters - **credential_path** (str) - Optional - Path to the credential cache file. Defaults to None. - **device_id** (str) - Optional - Device ID. If empty, a new device will be registered. Defaults to None. - **token** (str) - Optional - Authentication token. If empty, the token will be automatically obtained. Defaults to None. - **sample_rate** (int) - Optional - Sample rate of the audio. Defaults to 16000. - **channels** (int) - Optional - Number of audio channels. Defaults to 1. - **enable_punctuation** (bool) - Optional - Whether to enable punctuation. Defaults to True. ``` -------------------------------- ### Batch and Streaming Transcription with DoubaoASR Client Source: https://context7.com/starccy/doubaoime-asr/llms.txt Utilizes the `DoubaoASR` class as an asynchronous context manager for efficient reuse of configurations across multiple audio files. Supports both batch file processing and streaming transcription. ```python import asyncio from doubaoime_asr import DoubaoASR, ASRConfig, ASRError, ResponseType async def main(): config = ASRConfig(credential_path="./credentials.json") async with DoubaoASR(config) as asr: # 批量处理多个文件 audio_files = ["audio1.wav", "audio2.wav", "audio3.wav"] for audio_path in audio_files: try: result = await asr.transcribe(audio_path) print(f"{audio_path}: {result}") except ASRError as e: print(f"{audio_path} 识别失败: {e}") if e.response: print(f" 错误详情: {e.response.error_msg}") # 流式识别并访问详细信息 async for resp in asr.transcribe_stream("long_audio.wav"): if resp.type == ResponseType.FINAL_RESULT and resp.extra: print(f"文本: {resp.text}") print(f"音频时长: {resp.extra.audio_duration} ms") print(f"模型 RTF: {resp.extra.model_avg_rtf:.4f}") print(f"首响应时间: {resp.extra.model_send_first_response} ms") asyncio.run(main()) ``` -------------------------------- ### ASRResponse and Data Model Explanation Source: https://context7.com/starccy/doubaoime-asr/llms.txt Details the structure and fields of `ASRResponse`, `ASRResult`, `ASRAlternative`, `ASRWord`, and `ASRExtra` objects returned by the ASR functions. Explains response types, text content, timing, confidence, and performance metrics. ```python from doubaoime_asr import ( ASRResponse, ASRResult, ASRAlternative, ASRWord, ASRExtra, ResponseType ) # ASRResponse 字段说明 resp: ASRResponse resp.type # ResponseType 枚举,表示响应类型 resp.text # str,当前识别文本(中间或最终) resp.is_final # bool,是否为最终结果 resp.vad_finished # bool,VAD 是否已结束 resp.error_msg # str,错误信息(type == ERROR 时有值) resp.results # List[ASRResult],详细识别结果列表 resp.extra # ASRExtra,附加性能数据 resp.raw_json # dict,原始 JSON 数据 # ASRResult 字段说明 result: ASRResult result.text # str,该段识别文本 result.start_time # float,起始时间(秒) result.end_time # float,结束时间(秒) result.confidence # float,置信度 result.is_interim # bool,是否为中间结果 result.alternatives # List[ASRAlternative],候选结果 ``` -------------------------------- ### DoubaoASR Client Class Source: https://context7.com/starccy/doubaoime-asr/llms.txt The `DoubaoASR` class provides a lower-level interface for ASR tasks and can be used as an asynchronous context manager for efficient reuse of configurations across multiple audio processing operations. ```APIDOC ## DoubaoASR — 底层客户端类 `DoubaoASR` 是三个便捷函数的底层实现,支持作为异步上下文管理器使用。当需要复用同一配置对多个音频进行识别时,直接使用该类可避免重复初始化。 ```python import asyncio from doubaoime_asr import DoubaoASR, ASRConfig, ASRError, ResponseType async def main(): config = ASRConfig(credential_path="./credentials.json") async with DoubaoASR(config) as asr: # 批量处理多个文件 audio_files = ["audio1.wav", "audio2.wav", "audio3.wav"] for audio_path in audio_files: try: result = await asr.transcribe(audio_path) print(f"{audio_path}: {result}") except ASRError as e: print(f"{audio_path} 识别失败: {e}") if e.response: print(f" 错误详情: {e.response.error_msg}") # 流式识别并访问详细信息 async for resp in asr.transcribe_stream("long_audio.wav"): if resp.type == ResponseType.FINAL_RESULT and resp.extra: print(f"文本: {resp.text}") print(f"音频时长: {resp.extra.audio_duration} ms") print(f"模型 RTF: {resp.extra.model_avg_rtf:.4f}") print(f"首响应时间: {resp.extra.model_send_first_response} ms") asyncio.run(main()) ``` ``` -------------------------------- ### Cache Credentials for ASR Configuration Source: https://github.com/starccy/doubaoime-asr/blob/main/README.md Specify a `credential_path` in `ASRConfig` to automatically cache device registration and authentication tokens. This avoids repeated registration. ```python config = ASRConfig(credential_path="~/.config/doubaoime-asr/credentials.json") ``` -------------------------------- ### ASRResponse and Data Models Source: https://context7.com/starccy/doubaoime-asr/llms.txt Details the structure and fields of `ASRResponse` and its associated data models (`ASRResult`, `ASRAlternative`, `ASRWord`, `ASRExtra`), which are used for unified response handling in streaming recognition. ```APIDOC ## ASRResponse 与数据模型 `ASRResponse` 是所有流式识别函数产出的统一响应对象,包含响应类型、识别文本、详细结果列表和附加性能信息。 ```python from doubaoime_asr import ( ASRResponse, ASRResult, ASRAlternative, ASRWord, ASRExtra, ResponseType ) # ASRResponse 字段说明 resp: ASRResponse resp.type # ResponseType 枚举,表示响应类型 resp.text # str,当前识别文本(中间或最终) resp.is_final # bool,是否为最终结果 resp.vad_finished # bool,VAD 是否已结束 resp.error_msg # str,错误信息(type == ERROR 时有值) resp.results # List[ASRResult],详细识别结果列表 resp.extra # ASRExtra,附加性能数据 resp.raw_json # dict,原始 JSON 数据 # ASRResult 字段说明 result: ASRResult result.text # str,该段识别文本 result.start_time # float,起始时间(秒) result.end_time # float,结束时间(秒) result.confidence # float,置信度 result.is_interim # bool,是否为中间结果 result.alternatives # List[ASRAlternative],候选结果 ``` ``` -------------------------------- ### List Response Types Source: https://context7.com/starccy/doubaoime-asr/llms.txt Prints a list of all available ResponseType enumerations. Useful for understanding the different stages and types of responses from the ASR system. ```python print(list(ResponseType)) ``` -------------------------------- ### Basic Audio File Transcription Source: https://github.com/starccy/doubaoime-asr/blob/main/README.md Perform non-streaming speech recognition on an audio file. The `ASRConfig` handles device registration and credential caching. ```python import asyncio from doubaoime_asr import transcribe, ASRConfig async def main(): # 配置(首次运行会自动注册设备,并将凭据保存到指定文件) config = ASRConfig(credential_path="./credentials.json") # 识别音频文件 result = await transcribe("audio.wav", config=config) print(f"识别结果: {result}") asyncio.run(main()) ``` -------------------------------- ### Named Entity Recognition (NER) on Text Source: https://context7.com/starccy/doubaoime-asr/llms.txt Performs Named Entity Recognition on provided text using Doubao's SAMI service. Automatically handles authentication and token management. Can be used independently or in conjunction with ASR results. ```python from doubaoime_asr import ASRConfig, ner, NerResponse config = ASRConfig(credential_path="./credentials.json") # 对识别出的文本进行 NER,用于输入法联想优化 text = "张三李四以及张三在使用 Chrome 浏览器" response: NerResponse = ner(config, text) # 输出 NerResponse 结构 for result in response.results: print(f"原文: {result.text}") for word in result.words: print(f" 实体: '{word.word}' (出现 {word.freq} 次)") # 输出示例: # 原文: 张三李四以及张三在使用 Chrome 浏览器 # 实体: '张三' (出现 2 次) # 实体: '李四' (出现 1 次) # 实体: 'Chrome 浏览器' (出现 1 次) # 实体: '张三李四' (出现 1 次) # 实体: 'Chrome' (出现 1 次) # 配合 ASR 使用:先转写再实体提取 import asyncio from doubaoime_asr import transcribe async def transcribe_and_ner(audio_path: str): text = await transcribe(audio_path, config=config) print(f"识别文本: {text}") entities = ner(config, text) print(f"提取实体: {[w.word for r in entities.results for w in r.words]}") asyncio.run(transcribe_and_ner("meeting_record.wav")) ``` -------------------------------- ### transcribe_realtime Source: https://github.com/starccy/doubaoime-asr/blob/main/README.md Performs real-time streaming speech recognition, accepting an asynchronous iterator of PCM audio data. ```APIDOC ## transcribe_realtime ### Description Real-time streaming speech recognition, accepts an asynchronous iterator of PCM audio data. ### Method Signature ```python async def transcribe_realtime( audio_source: AsyncIterator[bytes], *, config: ASRConfig | None = None, ) -> AsyncIterator[ASRResponse] ``` ### Parameters - **audio_source** (AsyncIterator[bytes]) - Required - An asynchronous iterator yielding raw PCM byte data. - **config** (ASRConfig | None) - Optional - ASR configuration object. ``` -------------------------------- ### Perform Streaming Speech Recognition with transcribe_stream Source: https://context7.com/starccy/doubaoime-asr/llms.txt Utilize `transcribe_stream` for streaming ASR, yielding `ASRResponse` objects asynchronously. This is suitable for real-time progress display and handling multi-segment audio, providing detailed events like VAD, interim/final results, and errors. ```python import asyncio from doubaoime_asr import transcribe_stream, ASRConfig, ResponseType async def main(): config = ASRConfig(credential_path="./credentials.json") with open("audio.wav", "rb") as f: audio_data = f.read() async for response in transcribe_stream(audio_data, config=config, realtime=False): match response.type: case ResponseType.TASK_STARTED: print("[系统] 任务已启动") case ResponseType.SESSION_STARTED: print("[系统] 会话已启动") case ResponseType.VAD_START: print("[VAD] 检测到语音活动开始") case ResponseType.INTERIM_RESULT: print(f"[中间] {response.text}") case ResponseType.FINAL_RESULT: # Get timestamps for recognition results if response.results: r = response.results[0] print(f"[最终] ({r.start_time:.2f}s ~ {r.end_time:.2f}s) {response.text}") # Access alternative results for alt in r.alternatives: print(f" 候选: {alt.text}") # Access word-level timestamps if r.alternatives: for word in r.alternatives[0].words: print(f" 词: '{word.word}' [{word.start_time:.2f}s-{word.end_time:.2f}s]") case ResponseType.SESSION_FINISHED: print("[系统] 会话结束") case ResponseType.ERROR: print(f"[错误] {response.error_msg}") asyncio.run(main()) ``` -------------------------------- ### Calculate MD5 Hash for x-ss-stub Header Source: https://github.com/starccy/doubaoime-asr/blob/main/wave_protocol.md Computes the MD5 hash of the ciphertext and converts it to an uppercase hexadecimal string. This hash is used for the `x-ss-stub` HTTP header. ```python import hashlib stub = hashlib.md5(ciphertext).hexdigest().upper() ``` -------------------------------- ### transcribe Source: https://github.com/starccy/doubaoime-asr/blob/main/README.md Performs non-streaming speech recognition and returns the final result. ```APIDOC ## transcribe ### Description Non-streaming speech recognition, directly returns the final result. ### Method Signature ```python async def transcribe( audio: str | Path | bytes, *, config: ASRConfig | None = None, on_interim: Callable[[str], None] | None = None, realtime: bool = False, ) -> str ``` ### Parameters - **audio** (str | Path | bytes) - Required - Path to the audio file or raw PCM byte data. - **config** (ASRConfig | None) - Optional - ASR configuration object. - **on_interim** (Callable[[str], None] | None) - Optional - Callback function for interim results. - **realtime** (bool) - Optional - If True, simulates real-time sending with fixed delays between audio frames. If False, sends data as quickly as possible. ``` -------------------------------- ### Handshake Endpoint Source: https://github.com/starccy/doubaoime-asr/blob/main/wave_protocol.md Initiates the secure connection by exchanging cryptographic parameters. Requires a POST request to the handshake endpoint with a JSON payload containing client information and key shares. ```APIDOC ## POST https://keyhub.zijieapi.com/handshake ### Description Initiates the secure connection by exchanging cryptographic parameters. Requires a POST request to the handshake endpoint with a JSON payload containing client information and key shares. ### Method POST ### Endpoint https://keyhub.zijieapi.com/handshake ### Parameters #### Request Body - **version** (integer) - Required - Protocol version. - **random** (string) - Required - Base64 encoded 32-byte client random nonce. - **app_id** (string) - Required - Application identifier. - **did** (string) - Required - Device ID. - **key_shares** (array) - Required - List of key shares. - **curve** (string) - Required - Elliptic curve name (e.g., "secp256r1"). - **pubkey** (string) - Required - Base64 encoded 65-byte uncompressed public key. - **cipher_suites** (array) - Required - List of supported cipher suites (e.g., [4097]). ### Request Example ```json { "version": 2, "random": "", "app_id": "401734", "did": "<设备ID>", "key_shares": [ { "curve": "secp256r1", "pubkey": "" } ], "cipher_suites": [4097] } ``` ### Response #### Success Response (200) - **version** (integer) - Protocol version. - **random** (string) - Base64 encoded 32-byte server random nonce. - **key_share** (object) - Server's key share. - **curve** (string) - Elliptic curve name. - **pubkey** (string) - Base64 encoded 65-byte uncompressed server public key. - **cipher_suite** (integer) - Negotiated cipher suite. - **ticket** (string) - Base64 encoded session ticket. - **expire_time** (integer) - Session ticket expiration timestamp. ### Response Example ```json { "version": 2, "random": "", "key_share": { "curve": "secp256r1", "pubkey": "" }, "cipher_suite": 4097, "ticket": "", "expire_time": 1738763280 } ``` ``` -------------------------------- ### Derive Encryption Key using HKDF-SHA256 Source: https://github.com/starccy/doubaoime-asr/blob/main/wave_protocol.md Derives the 32-byte encryption key from the ECDH shared secret using HKDF with SHA256. The salt is a concatenation of client and server random nonces, and a hardcoded info string is used. ```python from cryptography.hazmat.primitives.kdf.hkdf import HKDF from cryptography.hazmat.primitives import hashes # 参数 salt = client_random + server_random # 64 字节 info = b"4e30514609050cd3" # 16 字节,硬编码常量 # 派生 encryption_key = HKDF( algorithm=hashes.SHA256(), length=32, salt=salt, info=info ).derive(shared_key) ``` -------------------------------- ### Perform Non-Streaming Speech Recognition with transcribe Source: https://context7.com/starccy/doubaoime-asr/llms.txt Use the `transcribe` function for non-streaming ASR. It accepts audio data (bytes, file path, or Path object) and returns the final text. An `on_interim` callback can be provided for intermediate results. ```python import asyncio import requests from doubaoime_asr import transcribe, ASRConfig async def main(): config = ASRConfig(credential_path="./credentials.json") # Get test audio from URL audio_data = requests.get( "https://github.com/liangstein/Chinese-speech-to-text/raw/refs/heads/master/1.wav" ).content # Basic usage: get final recognition result directly result = await transcribe(audio_data, config=config) print(f"识别结果: {result}") # Output example: 识别结果: 今天天气怎么样 # With interim results callback def on_interim(text: str): print(f" [中间] {text}") result = await transcribe( audio_data, config=config, on_interim=on_interim, realtime=False, # False (default): send as fast as possible, faster speed ) print(f" [最终] {result}") # Pass file path result = await transcribe("recording.wav", config=config) print(f"文件识别: {result}") asyncio.run(main()) ``` -------------------------------- ### ChaCha20 Encryption/Decryption Function Source: https://github.com/starccy/doubaoime-asr/blob/main/wave_protocol.md Implements ChaCha20 stream encryption/decryption. Note that the cryptography library requires a 16-byte nonce, which is formed by prepending 4 null bytes to the 12-byte Wave protocol nonce. ```python from cryptography.hazmat.primitives.ciphers import Cipher, algorithms def chacha20_crypt(key: bytes, nonce: bytes, data: bytes) -> bytes: """ChaCha20 加密/解密 (对称操作)""" # Python cryptography 库需要 16 字节 nonce # 格式: 4 字节 counter (固定为 0) + 12 字节 nonce nonce_16 = b'\x00\x00\x00\x00' + nonce cipher = Cipher( algorithms.ChaCha20(key, nonce_16), mode=None ) encryptor = cipher.encryptor() return encryptor.update(data) + encryptor.finalize() ``` -------------------------------- ### Wave Protocol Handshake Request Format Source: https://github.com/starccy/doubaoime-asr/blob/main/wave_protocol.md Defines the JSON structure for the handshake request, including version, client random nonce, app ID, device ID, key shares, and supported cipher suites. ```json { "version": 2, "random": "", "app_id": "401734", "did": "<设备ID>", "key_shares": [ { "curve": "secp256r1", "pubkey": "" } ], "cipher_suites": [4097] } ``` -------------------------------- ### transcribe_stream — Streaming Speech Recognition Source: https://context7.com/starccy/doubaoime-asr/llms.txt This function provides streaming speech recognition by yielding ASRResponse objects as an asynchronous iterator. It's suitable for real-time display of recognition progress and handling multi-segment audio, providing detailed event sequences including task/session status, VAD events, and results with timestamps. ```APIDOC ## transcribe_stream ### Description Performs streaming speech recognition, yielding `ASRResponse` objects as an asynchronous iterator. This is ideal for real-time display and handling multi-segment audio, providing a full sequence of events like task/session start, VAD detection, intermediate and final results with timestamps, and errors. ### Method `async transcribe_stream(audio_data: Union[str, Path, bytes], config: ASRConfig, realtime: bool = False) -> AsyncIterator[ASRResponse]` ### Parameters - **audio_data** (Union[str, Path, bytes]): Path to the audio file, a `Path` object, or raw PCM audio bytes. - **config** (ASRConfig): An instance of `ASRConfig` for configuration and credentials. - **realtime** (bool, optional): If True, simulates sending audio at its real-time rate. Defaults to False (sends as fast as possible). ### Response - **AsyncIterator[ASRResponse]**: An asynchronous iterator yielding `ASRResponse` objects. ### ASRResponse Structure - **type** (ResponseType): The type of the response event (e.g., TASK_STARTED, SESSION_STARTED, VAD_START, INTERIM_RESULT, FINAL_RESULT, SESSION_FINISHED, ERROR). - **text** (str, optional): The recognized text for result events. - **results** (List[RecognitionResult], optional): List of recognition results for FINAL_RESULT events. - **error_msg** (str, optional): Error message for ERROR events. ### RecognitionResult Structure - **start_time** (float): Start time of the recognized segment in seconds. - **end_time** (float): End time of the recognized segment in seconds. - **alternatives** (List[AlternativeResult]): List of alternative transcriptions. ### AlternativeResult Structure - **text** (str): The alternative transcription text. - **words** (List[WordInfo]): List of word-level information including timestamps. ### WordInfo Structure - **word** (str): The recognized word. - **start_time** (float): Start time of the word in seconds. - **end_time** (float): End time of the word in seconds. ### Request Example ```python import asyncio from doubaoime_asr import transcribe_stream, ASRConfig, ResponseType async def main(): config = ASRConfig(credential_path="./credentials.json") with open("audio.wav", "rb") as f: audio_data = f.read() async for response in transcribe_stream(audio_data, config=config, realtime=False): match response.type: case ResponseType.TASK_STARTED: print("[System] Task started") case ResponseType.SESSION_STARTED: print("[System] Session started") case ResponseType.VAD_START: print("[VAD] Voice activity detected start") case ResponseType.INTERIM_RESULT: print(f"[Interim] {response.text}") case ResponseType.FINAL_RESULT: if response.results: r = response.results[0] print(f"[Final] ({r.start_time:.2f}s ~ {r.end_time:.2f}s) {response.text}") for alt in r.alternatives: print(f" Alternative: {alt.text}") if r.alternatives and r.alternatives[0].words: for word in r.alternatives[0].words: print(f" Word: '{word.word}' [{word.start_time:.2f}s-{word.end_time:.2f}s]") case ResponseType.SESSION_FINISHED: print("[System] Session finished") case ResponseType.ERROR: print(f"[Error] {response.error_msg}") asyncio.run(main()) ``` ``` -------------------------------- ### transcribe_stream Source: https://github.com/starccy/doubaoime-asr/blob/main/README.md Performs streaming speech recognition and returns an asynchronous iterator of ASRResponse objects. ```APIDOC ## transcribe_stream ### Description Streaming speech recognition, returns an asynchronous iterator of `ASRResponse`. ### Method Signature ```python async def transcribe_stream( audio: str | Path | bytes, *, config: ASRConfig | None = None, realtime: bool = False, ) -> AsyncIterator[ASRResponse] ``` ### Parameters - **audio** (str | Path | bytes) - Required - Path to the audio file or raw PCM byte data. - **config** (ASRConfig | None) - Optional - ASR configuration object. - **realtime** (bool) - Optional - If True, simulates real-time sending with fixed delays between audio frames. If False, sends data as quickly as possible. ``` -------------------------------- ### ner Source: https://context7.com/starccy/doubaoime-asr/llms.txt Performs Named Entity Recognition (NER) on a given text using Doubao's SAMI service, returning a list of high-frequency words and entities along with their frequencies. ```APIDOC ## ner — 命名实体识别 `ner` 函数通过豆包的 SAMI 服务对文本进行命名实体识别,返回文本中出现的高频词和实体列表及其出现频次。内部自动管理 Wave 加密握手、SAMI token 获取与缓存,无需手动处理。 ```python from doubaoime_asr import ASRConfig, ner, NerResponse config = ASRConfig(credential_path="./credentials.json") # 对识别出的文本进行 NER,用于输入法联想优化 text = "张三李四以及张三在使用 Chrome 浏览器" response: NerResponse = ner(config, text) # 输出 NerResponse 结构 for result in response.results: print(f"原文: {result.text}") for word in result.words: print(f" 实体: '{word.word}' (出现 {word.freq} 次)") # 输出示例: # 原文: 张三李四以及张三在使用 Chrome 浏览器 # 实体: '张三' (出现 2 次) # 实体: '李四' (出现 1 次) # 实体: 'Chrome 浏览器' (出现 1 次) # 实体: '张三李四' (出现 1 次) # 实体: 'Chrome' (出现 1 次) # 配合 ASR 使用:先转写再实体提取 import asyncio from doubaoime_asr import transcribe async def transcribe_and_ner(audio_path: str): text = await transcribe(audio_path, config=config) print(f"识别文本: {text}") entities = ner(config, text) print(f"提取实体: {[w.word for r in entities.results for w in r.words]}") asyncio.run(transcribe_and_ner("meeting_record.wav")) ``` ``` -------------------------------- ### Wave Protocol Handshake Response Format Source: https://github.com/starccy/doubaoime-asr/blob/main/wave_protocol.md Defines the JSON structure for the handshake response, including version, server random nonce, server public key share, cipher suite, session ticket, and expiration time. ```json { "version": 2, "random": "", "key_share": { "curve": "secp256r1", "pubkey": "" }, "cipher_suite": 4097, "ticket": "", "expire_time": 1738763280 } ``` -------------------------------- ### ResponseType Source: https://github.com/starccy/doubaoime-asr/blob/main/README.md Enum for different response types in streaming recognition. ```APIDOC ## ResponseType ### Description Enum for different response types in streaming recognition. ### Enum Values - **TASK_STARTED**: Task has started. - **SESSION_STARTED**: Session has started. - **VAD_START**: Voice activity detection start. - **INTERIM_RESULT**: Interim recognition result. - **FINAL_RESULT**: Final recognition result. - **SESSION_FINISHED**: Session has finished. - **ERROR**: An error occurred. ``` -------------------------------- ### HKDF Key Derivation Source: https://github.com/starccy/doubaoime-asr/blob/main/wave_protocol.md Derives the encryption key from the ECDH shared secret using HKDF-SHA256 with a specific salt and info string. ```APIDOC ## HKDF Key Derivation ### Description Derives the encryption key from the ECDH shared secret using HKDF-SHA256 with a specific salt and info string. ### Method HKDF-SHA256 Key Derivation ### Parameters - **shared_key** (bytes) - The 32-byte shared secret key from ECDH. - **client_random** (bytes) - The 32-byte client random nonce from the handshake request. - **server_random** (bytes) - The 32-byte server random nonce from the handshake response. ### Key Parameters - **Salt**: Concatenation of `client_random` and `server_random` (64 bytes). - **Info String**: A hardcoded 16-byte string, e.g., `"4e30514609050cd3"`. - **Output Length**: 32 bytes for the encryption key. ### Code Example ```python from cryptography.hazmat.primitives.kdf.hkdf import HKDF from cryptography.hazmat.primitives import hashes # 参数 salt = client_random + server_random # 64 字节 info = b"4e30514609050cd3" # 16 字节,硬编码常量 # 派生 encryption_key = HKDF( algorithm=hashes.SHA256(), length=32, salt=salt, info=info ).derive(shared_key) ``` ### Output - **encryption_key** (bytes) - The derived 32-byte encryption key. ``` -------------------------------- ### ECDH Shared Key Calculation Source: https://github.com/starccy/doubaoime-asr/blob/main/wave_protocol.md Derives a shared secret key using the Elliptic Curve Diffie-Hellman (ECDH) algorithm between the client's private key and the server's public key. ```APIDOC ## ECDH Shared Key Calculation ### Description Derives a shared secret key using the Elliptic Curve Diffie-Hellman (ECDH) algorithm between the client's private key and the server's public key. ### Method ECDH Key Exchange ### Parameters - **client_private_key** (object) - The client's ephemeral private key. - **server_public_key** (object) - The server's public key received during the handshake. ### Code Example ```python from cryptography.hazmat.primitives.asymmetric import ec # 使用客户端私钥和服务器公钥计算共享密钥 shared_key = client_private_key.exchange( ec.ECDH(), server_public_key ) # shared_key: 32 字节 ``` ### Output - **shared_key** (bytes) - The derived 32-byte shared secret key. ``` -------------------------------- ### ChaCha20 Encryption/Decryption Source: https://github.com/starccy/doubaoime-asr/blob/main/wave_protocol.md Performs symmetric encryption and decryption using the ChaCha20 stream cipher. The same key is used for both encryption and decryption, with different nonces. ```APIDOC ## ChaCha20 Encryption/Decryption ### Description Performs symmetric encryption and decryption using the ChaCha20 stream cipher. The same key is used for both encryption and decryption, with different nonces. ### Algorithm ChaCha20 (pure stream cipher, no authentication tag) ### Parameters - **key** (bytes) - The 32-byte encryption key derived via HKDF. - **nonce** (bytes) - The 12-byte nonce. Must be unique for each encryption/decryption operation with the same key. - **data** (bytes) - The plaintext data to encrypt or ciphertext to decrypt. ### Code Example ```python from cryptography.hazmat.primitives.ciphers import Cipher, algorithms def chacha20_crypt(key: bytes, nonce: bytes, data: bytes) -> bytes: """ChaCha20 加密/解密 (对称操作)""" # Python cryptography 库需要 16 字节 nonce # 格式: 4 字节 counter (固定为 0) + 12 字节 nonce nonce_16 = b'\x00\x00\x00\x00' + nonce cipher = Cipher( algorithms.ChaCha20(key, nonce_16), mode=None ) encryptor = cipher.encryptor() return encryptor.update(data) + encryptor.finalize() ``` ### Usage - **Encryption**: Encrypt plaintext to ciphertext. - **Decryption**: Decrypt ciphertext back to plaintext. ### Nonce Format - **Length**: 12 bytes. - **Generation**: Randomly generated for each request. - **Transmission**: Base64 encoded and placed in the `x-tt-e-p` header for requests, and the response header for responses. ``` -------------------------------- ### Encrypted Request Headers Source: https://github.com/starccy/doubaoime-asr/blob/main/wave_protocol.md Specifies the HTTP headers required for sending an encrypted request. ```APIDOC ## Encrypted Request Headers ### Description Specifies the HTTP headers required for sending an encrypted request. ### Headers - **`x-tt-e-b`**: (string) - Indicates that the request body is encrypted. Value is typically "1". - **`x-tt-e-t`**: (string) - The session ticket obtained during the handshake. - **`x-tt-e-p`**: (string) - Base64 encoded 12-byte nonce used for encrypting the request body. - **`x-ss-stub`**: (string) - Uppercase MD5 hash of the ciphertext. ### x-ss-stub Calculation ```python import hashlib stub = hashlib.md5(ciphertext).hexdigest().upper() ``` ### Full Request Example ```http POST /api/v3/context/ime/ner?device_platform=android&... HTTP/1.1 Host: speech.bytedance.com Content-Type: application/json x-tt-e-b: 1 x-tt-e-t: MIGYBAwrT...(ticket) x-tt-e-p: abc123...(nonce Base64) x-ss-stub: A1B2C3D4E5F6...(MD5 大写) x-api-app-key: SYlxZr6LnvBaIVmF x-api-token: eyJhbGciOiJFUzI1NiI... <加密后的请求体> ``` ```