### Setup and Installation

Source: https://huggingface.co/FireRedTeam/FireRedPunc/commit/bbceaf3e89869d1d26b1207320275102f9e53b48

Instructions for setting up the environment, installing dependencies, and downloading models for FireRedPunc.

```APIDOC
## Setup

1. Create a clean Python environment:
```bash
$ conda create --name fireredasr2s python=3.10
$ conda activate fireredasr2s
$ git clone https://github.com/FireRedTeam/FireRedASR2S.git
$ cd FireRedASR2S # or fireredasr2s
```

2. Install dependencies and set up PATH and PYTHONPATH:
```bash
$ pip install -r requirements.txt
$ export PATH=$PWD/fireredasr2s/:$PATH
$ export PYTHONPATH=$PWD/:$PYTHONPATH
```

3. Download models:
```bash
# Download via ModelScope (recommended for users in China)
pip install -U modelscope
modelscope download --model FireRedTeam/FireRedASR2-AED --local_dir ./pretrained_models/FireRedASR2-AED
modelscope download --model FireRedTeam/FireRedVAD --local_dir ./pretrained_models/FireRedVAD
modelscope download --model FireRedTeam/FireRedLID --local_dir ./pretrained_models/FireRedLID
modelscope download --model FireRedTeam/FireRedPunc --local_dir ./pretrained_models/FireRedPunc

# Download via Hugging Face
pip install -U "huggingface_hub[cli]"
huggingface-cli download FireRedTeam/FireRedASR2-AED --local-dir ./pretrained_models/FireRedASR2-AED
huggingface-cli download FireRedTeam/FireRedVAD --local-dir ./pretrained_models/FireRedVAD
huggingface-cli download FireRedTeam/FireRedLID --local-dir ./pretrained_models/FireRedLID
huggingface-cli download FireRedTeam/FireRedPunc --local-dir ./pretrained_models/FireRedPunc
```

4. Convert your audio to **16kHz 16-bit mono PCM** format if needed:
```bash
$ ffmpeg -i <input_audio_path> -ar 16000 -ac 1 -acodec pcm_s16le -f wav <output_wav_path>
```
```

--------------------------------

### Install Dependencies and Set Up PATH

Source: https://huggingface.co/FireRedTeam/FireRedPunc/raw/main/README.md

Install project dependencies using pip and set the PATH and PYTHONPATH environment variables.

```bash
pip install -r requirements.txt
export PATH=$PWD/fireredasr2s/:$PATH
export PYTHONPATH=$PWD/:$PYTHONPATH
```

--------------------------------

### Dependency Installation

Source: https://huggingface.co/FireRedTeam/FireRedPunc/blob/main/README.md

Install project requirements and configure environment paths.

```bash
$ pip install -r requirements.txt
$ export PATH=$PWD/fireredasr2s/:$PATH
$ export PYTHONPATH=$PWD/:$PYTHONPATH
```

--------------------------------

### Environment Setup

Source: https://huggingface.co/FireRedTeam/FireRedPunc/blob/main/README.md

Commands to create a conda environment and clone the repository.

```bash
$ conda create --name fireredasr2s python=3.10
$ conda activate fireredasr2s
$ git clone https://github.com/FireRedTeam/FireRedASR2S.git
$ cd FireRedASR2S  # or fireredasr2s
```

--------------------------------

### Configure and Initialize FireRedAsr2System

Source: https://huggingface.co/FireRedTeam/FireRedPunc/commit/bbceaf3e89869d1d26b1207320275102f9e53b48

Setup the punctuation and system configurations before initializing the ASR system.

```python
eos_penalty=1.0,
return_timestamp=True
)
punc_config = FireRedPuncConfig(use_gpu=True)

asr_system_config = FireRedAsr2SystemConfig(
"pretrained_models/FireRedVAD/VAD",
"pretrained_models/FireRedLID",
"aed", "pretrained_models/FireRedASR2-AED",
"pretrained_models/FireRedPunc",
vad_config, lid_config, asr_config, punc_config,
enable_vad=1, enable_lid=1, enable_punc=1
)
asr_system = FireRedAsr2System(asr_system_config)
```

--------------------------------

### Run ASR Inference Script (LLM)

Source: https://huggingface.co/FireRedTeam/FireRedPunc/blame/main/README.md

Navigate to the ASR inference examples directory and execute the LLM inference script.

```bash
cd examples_infer/asr
bash inference_asr_llm.sh
```

--------------------------------

### Run Punc Inference Script

Source: https://huggingface.co/FireRedTeam/FireRedPunc/blame/main/README.md

Navigate to the Punc inference examples directory and execute the Punc inference script.

```bash
cd examples_infer/punc
bash inference_punc.sh
```

--------------------------------

### Download Models via Hugging Face

Source: https://huggingface.co/FireRedTeam/FireRedPunc

Install the huggingface_hub library and download the required models for ASR, VAD, LID, and Punc.

```bash
pip install -U "huggingface_hub[cli]"
huggingface-cli download FireRedTeam/FireRedASR2-AED --local-dir ./pretrained_models/FireRedASR2-AED
huggingface-cli download FireRedTeam/FireRedASR2-AED --local-dir ./pretrained_models/FireRedASR2-AED
huggingface-cli download FireRedTeam/FireRedVAD --local-dir ./pretrained_models/FireRedVAD
huggingface-cli download FireRedTeam/FireRedLID --local-dir ./pretrained_models/FireRedLID
huggingface-cli download FireRedTeam/FireRedPunc --local-dir ./pretrained_models/FireRedPunc
huggingface-cli download FireRedTeam/FireRedASR2-LLM --local-dir ./pretrained_models/FireRedASR2-LLM
```

--------------------------------

### Run ASR System Inference Script

Source: https://huggingface.co/FireRedTeam/FireRedPunc/blame/main/README.md

Navigate to the ASR inference examples directory and execute the main inference script.

```bash
cd examples_infer/asr_system
bash inference_asr_system.sh
```

--------------------------------

### Run ASR Inference Script

Source: https://huggingface.co/FireRedTeam/FireRedPunc/raw/main/README.md

Navigate to the ASR examples directory and run the inference script for ASR tasks.

```bash
cd examples_infer/asr
bash inference_asr_aed.sh
bash inference_asr_llm.sh
```

--------------------------------

### Run LID Inference Script

Source: https://huggingface.co/FireRedTeam/FireRedPunc/blame/main/README.md

Navigate to the LID inference examples directory and execute the LID inference script.

```bash
cd examples_infer/lid
bash inference_lid.sh
```

--------------------------------

### Download Models with Hugging Face CLI

Source: https://huggingface.co/FireRedTeam/FireRedPunc/commit/e448fd967f44182a1c323cc30f5d89f2400c28da

Use the Hugging Face CLI to download pre-trained models. Ensure you have the huggingface_hub package installed with the 'cli' extra.

```bash
pip install -U "huggingface_hub[cli]"
huggingface-cli download FireRedTeam/FireRedVAD --local-dir ./pretrained_models/FireRedVAD
huggingface-cli download FireRedTeam/FireRedLID --local-dir ./pretrained_models/FireRedLID
huggingface-cli download FireRedTeam/FireRedPunc --local-dir ./pretrained_models/FireRedPunc
huggingface-cli download FireRedTeam/FireRedASR2-LLM --local-dir ./pretrained_models/FireRedASR2-LLM
```

--------------------------------

### Download Models with Hugging Face CLI

Source: https://huggingface.co/FireRedTeam/FireRedPunc/blame/main/README.md

Install the huggingface_hub library and use the CLI to download specific FireRedPunc models to a local directory.

```bash
pip install -U "huggingface_hub[cli]"
huggingface-cli download FireRedTeam/FireRedASR2-AED --local-dir ./pretrained_models/FireRedASR2-AED
huggingface-cli download FireRedTeam/FireRedVAD --local-dir ./pretrained_models/FireRedVAD
huggingface-cli download FireRedTeam/FireRedLID --local-dir ./pretrained_models/FireRedLID
huggingface-cli download FireRedTeam/FireRedPunc --local-dir ./pretrained_models/FireRedPunc
huggingface-cli download FireRedTeam/FireRedASR2-LLM --local-dir ./pretrained_models/FireRedASR2-LLM
```

--------------------------------

### Python API Usage

Source: https://huggingface.co/FireRedTeam/FireRedPunc/commit/bbceaf3e89869d1d26b1207320275102f9e53b48

Example of how to use the FireRedPunc model within a Python application.

```APIDOC
### Python API Usage
```python
from fireredasr2s import FireRedAsr2System, FireRedAsr2SystemConfig
```
```

--------------------------------

### Python API Inference Output

Source: https://huggingface.co/FireRedTeam/FireRedPunc/commit/e448fd967f44182a1c323cc30f5d89f2400c28da

Example output structure from the Python API for audio processing.

```python
# {'uttid': 'hello_en', 'text': 'Hello speech.', 'sentences': [{'start_ms': 260, 'end_ms': 1820, 'text': 'Hello speech.', 'asr_confidence': 0.933, 'lang': 'en', 'lang_confidence': 0.993}], 'vad_segments_ms': [(260, 1820)], 'dur_s': 2.24, 'words': [{'start_ms': 400, 'end_ms': 960, 'text': 'hello'}, {'start_ms': 960, 'end_ms': 1666, 'text': 'speech'}], 'wav_path': 'assets/hello_en.wav'}
```

--------------------------------

### Download Models with ModelScope CLI

Source: https://huggingface.co/FireRedTeam/FireRedPunc/commit/e448fd967f44182a1c323cc30f5d89f2400c28da

Use the ModelScope CLI to download pre-trained models for ASR, VAD, LID, and Punctuation. Ensure you have the modelscope package installed.

```bash
pip install -U modelscope
modelscope download --model xukaituo/FireRedASR2-AED --local_dir ./pretrained_models/FireRedASR2-AED
modelscope download --model xukaituo/FireRedVAD --local_dir ./pretrained_models/FireRedVAD
modelscope download --model xukaituo/FireRedLID --local_dir ./pretrained_models/FireRedLID
modelscope download --model xukaituo/FireRedPunc --local_dir ./pretrained_models/FireRedPunc
modelscope download --model xukaituo/FireRedASR2-LLM --local_dir ./pretrained_models/FireRedASR2-LLM
```

--------------------------------

### Run ASR Inference Script (AED)

Source: https://huggingface.co/FireRedTeam/FireRedPunc/blame/main/README.md

Navigate to the ASR inference examples directory and execute the AED inference script.

```bash
cd examples_infer/asr
bash inference_asr_aed.sh
```

--------------------------------

### Serve FireRedASR2-LLM with vLLM

Source: https://huggingface.co/FireRedTeam/FireRedPunc/blame/main/README.md

Serve the FireRedASR2-LLM model using vLLM for high-performance inference. This requires installing vLLM and potentially adjusting tensor parallelism and data types.

```shell
vllm serve allendou/FireRedASR2-LLM-vllm -tp=2 --dtype=float32
python3 examples/online_serving/openai_transcription_client.py --repetition_penalty=1.0 --audio_path=/root/hello_zh.wav
```

--------------------------------

### Run VAD Inference Scripts

Source: https://huggingface.co/FireRedTeam/FireRedPunc/blame/main/README.md

Navigate to the VAD inference examples directory and execute the VAD, stream VAD, and Audio Event Detection (AED) inference scripts.

```bash
cd examples_infer/vad
bash inference_vad.sh
bash inference_streamvad.sh
bash inference_aed.sh
```

--------------------------------

### Initialize and Process Audio with FireRedASR2

Source: https://huggingface.co/FireRedTeam/FireRedPunc/blob/main/README.md

Demonstrates initializing the FireRedASR2 system with various configurations for VAD, LID, ASR, and Punctuation, and then processing a batch of audio files. Ensure the system is tested on Linux Ubuntu 22.04.

```python
from fireredasr2s.fireredasr2 import FireRedAsr2Config
from fireredasr2s.fireredlid import FireRedLidConfig
from fireredasr2s.fireredpunc import FireRedPuncConfig
from fireredasr2s.fireredvad import FireRedVadConfig
from fireredasr2s import FireRedAsr2System, FireRedAsr2SystemConfig

vad_config = FireRedVadConfig(
    use_gpu=False,
    smooth_window_size=5,
    speech_threshold=0.4,
    min_speech_frame=20,
    max_speech_frame=2000,
    min_silence_frame=20,
    merge_silence_frame=0,
    extend_speech_frame=0,
    chunk_max_frame=30000
)
lid_config = FireRedLidConfig(use_gpu=True, use_half=False)
asr_config = FireRedAsr2Config(
    use_gpu=True,
    use_half=False,
    beam_size=3,
    nbest=1,
    decode_max_len=0,
    softmax_smoothing=1.25,
    aed_length_penalty=0.6,
    eos_penalty=1.0,
    return_timestamp=True
)
punc_config = FireRedPuncConfig(use_gpu=True)

asr_system_config = FireRedAsr2SystemConfig(
    "pretrained_models/FireRedVAD/VAD",
    "pretrained_models/FireRedLID",
    "aed", "pretrained_models/FireRedASR2-AED",
    "pretrained_models/FireRedPunc",
    vad_config, lid_config, asr_config, punc_config,
    enable_vad=1, enable_lid=1, enable_punc=1
)
asr_system = FireRedAsr2System(asr_system_config)

batch_uttid = ["hello_zh", "hello_en"]
batch_wav_path = ["assets/hello_zh.wav", "assets/hello_en.wav"]
for wav_path, uttid in zip(batch_wav_path, batch_uttid):
    result = asr_system.process(wav_path, uttid)
    print(result)

```

--------------------------------

### Command-line Interface Usage

Source: https://huggingface.co/FireRedTeam/FireRedPunc/blob/main/README.md?code=true

Use the CLI tool to process audio files and view the resulting JSONL output.

```bash
$ fireredasr2s-cli --help
$ fireredasr2s-cli --wav_paths "assets/hello_zh.wav" "assets/hello_en.wav" --outdir output
$ cat output/result.jsonl
# {"uttid": "hello_zh", "text": "你好世界。", "sentences": [{"start_ms": 310, "end_ms": 1840, "text": "你好世界。", "asr_confidence": 0.875, "lang": "zh mandarin", "lang_confidence": 0.999}], "vad_segments_ms": [[310, 1840]], "dur_s": 2.32, "words": [{"start_ms": 490, "end_ms": 690, "text": "你"}, {"start_ms": 690, "end_ms": 1090, "text": "好"}, {"start_ms": 1090, "end_ms": 1330, "text": "世"}, {"start_ms": 1330, "end_ms": 1795, "text": "界"}], "wav_path": "assets/hello_zh.wav"}
# {"uttid": "hello_en", "text": "Hello speech.", "sentences": [{"start_ms": 120, "end_ms": 1840, "text": "Hello speech.", "asr_confidence": 0.833, "lang": "en", "lang_confidence": 0.998}], "vad_segments_ms": [[120, 1840]], "dur_s": 2.24, "words": [{"start_ms": 340, "end_ms": 1020, "text": "hello"}, {"start_ms": 1020, "end_ms": 1666, "text": "speech"}], "wav_path": "assets/hello_en.wav"}
```

--------------------------------

### Model Download

Source: https://huggingface.co/FireRedTeam/FireRedPunc/blob/main/README.md

Download pretrained models using ModelScope or Hugging Face CLI.

```bash
# Download via ModelScope (recommended for users in China)
pip install -U modelscope
modelscope download --model xukaituo/FireRedASR2-AED --local_dir ./pretrained_models/FireRedASR2-AED
modelscope download --model xukaituo/FireRedVAD --local_dir ./pretrained_models/FireRedVAD
modelscope download --model xukaituo/FireRedLID --local_dir ./pretrained_models/FireRedLID
modelscope download --model xukaituo/FireRedPunc --local_dir ./pretrained_models/FireRedPunc
modelscope download --model xukaituo/FireRedASR2-LLM --local_dir ./pretrained_models/FireRedASR2-LLM

# Download via Hugging Face
pip install -U "huggingface_hub[cli]"
huggingface-cli download FireRedTeam/FireRedASR2-AED --local-dir ./pretrained_models/FireRedASR2-AED
huggingface-cli download FireRedTeam/FireRedVAD --local-dir ./pretrained_models/FireRedVAD
huggingface-cli download FireRedTeam/FireRedLID --local-dir ./pretrained_models/FireRedLID
huggingface-cli download FireRedTeam/FireRedPunc --local-dir ./pretrained_models/FireRedPunc
huggingface-cli download FireRedTeam/FireRedASR2-LLM --local-dir ./pretrained_models/FireRedASR2-LLM
```

--------------------------------

### Initialize and Run FireRedASR2S System

Source: https://huggingface.co/FireRedTeam/FireRedPunc/blame/main/README.md

Configures the ASR system components and processes a batch of audio files. Ensure all model paths are correctly specified before initialization.

```python
from fireredasr2s.fireredasr2 import FireRedAsr2Config
from fireredasr2s.fireredlid import FireRedLidConfig
from fireredasr2s.fireredpunc import FireRedPuncConfig
from fireredasr2s.fireredvad import FireRedVadConfig
from fireredasr2s import FireRedAsr2System, FireRedAsr2SystemConfig

vad_config = FireRedVadConfig(
    use_gpu=False,
    smooth_window_size=5,
    speech_threshold=0.4,
    min_speech_frame=20,
    max_speech_frame=2000,
    min_silence_frame=20,
    merge_silence_frame=0,
    extend_speech_frame=0,
    chunk_max_frame=30000
)
lid_config = FireRedLidConfig(use_gpu=True, use_half=False)
asr_config = FireRedAsr2Config(
    use_gpu=True,
    use_half=False,
    beam_size=3,
    nbest=1,
    decode_max_len=0,
    softmax_smoothing=1.25,
    aed_length_penalty=0.6,
    eos_penalty=1.0,
    return_timestamp=True
)
punc_config = FireRedPuncConfig(use_gpu=True)

asr_system_config = FireRedAsr2SystemConfig(
    "pretrained_models/FireRedVAD/VAD",
    "pretrained_models/FireRedLID",
    "aed", "pretrained_models/FireRedASR2-AED",
    "pretrained_models/FireRedPunc",
    vad_config, lid_config, asr_config, punc_config,
    enable_vad=1, enable_lid=1, enable_punc=1
)
asr_system = FireRedAsr2System(asr_system_config)

batch_uttid = ["hello_zh", "hello_en"]
batch_wav_path = ["assets/hello_zh.wav", "assets/hello_en.wav"]
for wav_path, uttid in zip(batch_wav_path, batch_uttid):
    result = asr_system.process(wav_path, uttid)
    print(result)
```

--------------------------------

### Download Pretrained Models

Source: https://huggingface.co/FireRedTeam/FireRedPunc/resolve/main/README.md?download=true

Commands to download models via ModelScope or Hugging Face.

```bash
# Download via ModelScope (recommended for users in China)
pip install -U modelscope
modelscope download --model xukaituo/FireRedASR2-AED --local_dir ./pretrained_models/FireRedASR2-AED
modelscope download --model xukaituo/FireRedVAD --local_dir ./pretrained_models/FireRedVAD
modelscope download --model xukaituo/FireRedLID --local_dir ./pretrained_models/FireRedLID
modelscope download --model xukaituo/FireRedPunc --local_dir ./pretrained_models/FireRedPunc
modelscope download --model xukaituo/FireRedASR2-LLM --local_dir ./pretrained_models/FireRedASR2-LLM

# Download via Hugging Face
pip install -U "huggingface_hub[cli]"
huggingface-cli download FireRedTeam/FireRedASR2-AED --local-dir ./pretrained_models/FireRedASR2-AED
huggingface-cli download FireRedTeam/FireRedVAD --local-dir ./pretrained_models/FireRedVAD
huggingface-cli download FireRedTeam/FireRedLID --local-dir ./pretrained_models/FireRedLID
huggingface-cli download FireRedTeam/FireRedPunc --local-dir ./pretrained_models/FireRedPunc
huggingface-cli download FireRedTeam/FireRedASR2-LLM --local_dir ./pretrained_models/FireRedASR2-LLM
```

--------------------------------

### Initialize and Use FireRedAsr2System

Source: https://huggingface.co/FireRedTeam/FireRedPunc/raw/main/README.md

Configures the full ASR system including VAD, LID, ASR, and Punctuation models to process audio files.

```python
from fireredasr2s.fireredasr2 import FireRedAsr2Config
from fireredasr2s.fireredlid import FireRedLidConfig
from fireredasr2s.fireredpunc import FireRedPuncConfig
from fireredasr2s.fireredvad import FireRedVadConfig
from fireredasr2s import FireRedAsr2System, FireRedAsr2SystemConfig

vad_config = FireRedVadConfig(
    use_gpu=False,
    smooth_window_size=5,
    speech_threshold=0.4,
    min_speech_frame=20,
    max_speech_frame=2000,
    min_silence_frame=20,
    merge_silence_frame=0,
    extend_speech_frame=0,
    chunk_max_frame=30000
)
lid_config = FireRedLidConfig(use_gpu=True, use_half=False)
asr_config = FireRedAsr2Config(
    use_gpu=True,
    use_half=False,
    beam_size=3,
    nbest=1,
    decode_max_len=0,
    softmax_smoothing=1.25,
    aed_length_penalty=0.6,
    eos_penalty=1.0,
    return_timestamp=True
)
punc_config = FireRedPuncConfig(use_gpu=True)

asr_system_config = FireRedAsr2SystemConfig(
    "pretrained_models/FireRedVAD/VAD",
    "pretrained_models/FireRedLID",
    "aed", "pretrained_models/FireRedASR2-AED",
    "pretrained_models/FireRedPunc",
    vad_config, lid_config, asr_config, punc_config,
    enable_vad=1, enable_lid=1, enable_punc=1
)
asr_system = FireRedAsr2System(asr_system_config)

batch_uttid = ["hello_zh", "hello_en"]
batch_wav_path = ["assets/hello_zh.wav", "assets/hello_en.wav"]
for wav_path, uttid in zip(batch_wav_path, batch_uttid):
    result = asr_system.process(wav_path, uttid)
    print(result)
# {'uttid': 'hello_zh', 'text': '你好世界。', 'sentences': [{'start_ms': 440, 'end_ms': 1820, 'text': '你好世界。', 'asr_confidence': 0.868, 'lang': 'zh mandarin', 'lang_confidence': 0.999}], 'vad_segments_ms': [(440, 1820)], 'dur_s': 2.32, 'words': [{'start_ms': 540, 'end_ms': 700, 'text': '你'}, {'start_ms': 700, 'end_ms': 1100, 'text': '好'}, {'start_ms': 1100, 'end_ms': 1300, 'text': '世'}, {'start_ms': 1300, 'end_ms': 1765, 'text': '界'}], 'wav_path': 'assets/hello_zh.wav'}
# {'uttid': 'hello_en', 'text': 'Hello speech.', 'sentences': [{'start_ms': 260, 'end_ms': 1820, 'text': 'Hello speech.', 'asr_confidence': 0.933, 'lang': 'en', 'lang_confidence': 0.993}], 'vad_segments_ms': [(260, 1820)], 'dur_s': 2.24, 'words': [{'start_ms': 400, 'end_ms': 960, 'text': 'hello'}, {'start_ms': 960, 'end_ms': 1666, 'text': 'speech'}], 'wav_path': 'assets/hello_en.wav'}
```

--------------------------------

### Create Python Environment

Source: https://huggingface.co/FireRedTeam/FireRedPunc/blob/main/README.md?code=true

Initialize a dedicated Conda environment and clone the repository.

```bash
$ conda create --name fireredasr2s python=3.10
$ conda activate fireredasr2s
$ git clone https://github.com/FireRedTeam/FireRedASR2S.git
$ cd FireRedASR2S # or fireredasr2s
```

--------------------------------

### vLLM Serving

Source: https://huggingface.co/FireRedTeam/FireRedPunc/blob/main/README.md

Serve the model using vLLM and run a client transcription.

```bash
# Serving FireRedASR2-LLM with latest vLLM for the highest performance.
# For more details, see https://github.com/vllm-project/vllm/pull/35727.
$ vllm serve allendou/FireRedASR2-LLM-vllm -tp=2 --dtype=float32
$ python3 examples/online_serving/openai_transcription_client.py --repetition_penalty=1.0 --audio_path=/root/hello_zh.wav
```

--------------------------------

### Download Pretrained Models

Source: https://huggingface.co/FireRedTeam/FireRedPunc/commit/e448fd967f44182a1c323cc30f5d89f2400c28da

Commands to download FireRedTeam models using either ModelScope or Hugging Face CLI.

```bash
# Download via ModelScope (recommended for users in China)
pip install -U modelscope
modelscope download --model ~~FireRedTeam~~ /FireRedASR2-AED --local_dir ./pretrained_models/FireRedASR2-AED
modelscope download --model ~~FireRedTeam~~ /FireRedVAD --local_dir ./pretrained_models/FireRedVAD
modelscope download --model ~~FireRedTeam~~ /FireRedLID --local_dir ./pretrained_models/FireRedLID
modelscope download --model ~~FireRedTeam~~ /FireRedPunc --local_dir ./pretrained_models/FireRedPunc

# Download via Hugging Face
pip install -U "huggingface_hub[cli]"
huggingface-cli download FireRedTeam/FireRedASR2-AED --local-dir ./pretrained_models/FireRedASR2-AED
huggingface-cli download FireRedTeam/FireRedVAD --local-dir ./pretrained_models/FireRedVAD
huggingface-cli download FireRedTeam/FireRedLID --local-dir ./pretrained_models/FireRedLID
huggingface-cli download FireRedTeam/FireRedPunc --local-dir ./pretrained_models/FireRedPunc
```

--------------------------------

### Download Models via ModelScope

Source: https://huggingface.co/FireRedTeam/FireRedPunc/commit/bbceaf3e89869d1d26b1207320275102f9e53b48

Downloads pre-trained models using the ModelScope CLI. Recommended for users in China.

```bash
# Download via ModelScope (recommended for users in China)
pip install -U modelscope
modelscope download --model FireRedTeam/FireRedASR2-AED --local_dir ./pretrained_models/FireRedASR2-AED
modelscope download --model FireRedTeam/FireRedVAD --local_dir ./pretrained_models/FireRedVAD
modelscope download --model FireRedTeam/FireRedLID --local_dir ./pretrained_models/FireRedLID
modelscope download --model FireRedTeam/FireRedPunc --local_dir ./pretrained_models/FireRedPunc
```

--------------------------------

### ASR System Configuration

Source: https://huggingface.co/FireRedTeam/FireRedPunc/commit/bbceaf3e89869d1d26b1207320275102f9e53b48

Initializes the configuration objects required for the full ASR system pipeline.

```python
from fireredasr2s.fireredasr2 import FireRedAsr2Config
from fireredasr2s.fireredlid import FireRedLidConfig
from fireredasr2s.fireredpunc import FireRedPuncConfig
from fireredasr2s.fireredvad import FireRedVadConfig
from fireredasr2s import FireRedAsr2System, FireRedAsr2SystemConfig

vad_config = FireRedVadConfig(
use_gpu=False,
smooth_window_size=5,
speech_threshold=0.4,
min_speech_frame=20,
max_speech_frame=2000,
min_silence_frame=20,
merge_silence_frame=0,
extend_speech_frame=0,
chunk_max_frame=30000
)
lid_config = FireRedLidConfig(use_gpu=True, use_half=False)
asr_config = FireRedAsr2Config(
use_gpu=True,
use_half=False,
beam_size=3,
nbest=1,
decode_max_len=0,
softmax_smoothing=1.25,
aed_length_penalty=0.6,
```

--------------------------------

### Python API Usage

Source: https://huggingface.co/FireRedTeam/FireRedPunc/blob/main/README.md?code=true

Initialize the ASR system and process an audio file using the Python library.

```python
from fireredasr2s import FireRedAsr2System, FireRedAsr2SystemConfig

asr_system_config = FireRedAsr2SystemConfig() # Use default config
asr_system = FireRedAsr2System(asr_system_config)

result = asr_system.process("assets/hello_zh.wav")
print(result)
```

--------------------------------

### Python API Initialization

Source: https://huggingface.co/FireRedTeam/FireRedPunc/commit/bbceaf3e89869d1d26b1207320275102f9e53b48

Initializes the FireRedASR2 system using the FireRedAsr2System and FireRedAsr2SystemConfig classes.

```python
from fireredasr2s import FireRedAsr2System, FireRedAsr2SystemConfig
```

--------------------------------

### Audio Preprocessing

Source: https://huggingface.co/FireRedTeam/FireRedPunc/blob/main/README.md

Convert audio files to the required 16kHz 16-bit mono PCM format using ffmpeg.

```bash
$ ffmpeg -i <input_audio_path> -ar 16000 -ac 1 -acodec pcm_s16le -f wav <output_wav_path>
```

--------------------------------

### Script and Command-line Usage

Source: https://huggingface.co/FireRedTeam/FireRedPunc/commit/bbceaf3e89869d1d26b1207320275102f9e53b48

How to use FireRedPunc via shell scripts and the command-line interface.

```APIDOC
### Script Usage
```bash
$ cd examples_infer/asr_system
$ bash inference_asr_system.sh
```

### Command-line Usage
```bash
$ fireredasr2s-cli --help
$ fireredasr2s-cli --wav_paths "assets/hello_zh.wav" "assets/hello_en.wav" --outdir output
$ cat output/result.jsonl
# {"uttid": "hello_zh", "text": "你好世界。", "sentences": [{"start_ms": 310, "end_ms": 1840, "text": "你好世界。", "asr_confidence": 0.875, "lang": "zh mandarin", "lang_confidence": 0.999}], "vad_segments_ms": [[310, 1840]], "dur_s": 2.32, "words": [{"start_ms": 490, "end_ms": 690, "text": "你"}, {"start_ms": 690, "end_ms": 1090, "text": "好"}, {"start_ms": 1090, "end_ms": 1330, "text": "世"}, {"start_ms": 1330, "end_ms": 1795, "text": "界"}], "wav_path": "assets/hello_zh.wav"}
# {"uttid": "hello_en", "text": "Hello speech.", "sentences": [{"start_ms": 120, "end_ms": 1840, "text": "Hello speech.", "asr_confidence": 0.833, "lang": "en", "lang_confidence": 0.998}], "vad_segments_ms": [[120, 1840]], "dur_s": 2.24, "words": [{"start_ms": 340, "end_ms": 1020, "text": "hello"}, {"start_ms": 1020, "end_ms": 1666, "text": "speech"}], "wav_path": "assets/hello_en.wav"}
```
```

--------------------------------

### CLI Usage

Source: https://huggingface.co/FireRedTeam/FireRedPunc/blob/main/README.md

Perform inference using the command-line interface.

```bash
$ fireredasr2s-cli --help
$ fireredasr2s-cli --wav_paths "assets/hello_zh.wav" "assets/hello_en.wav" --outdir output
$ cat output/result.jsonl 
# {"uttid": "hello_zh", "text": "你好世界。", "sentences": [{"start_ms": 310, "end_ms": 1840, "text": "你好世界。", "asr_confidence": 0.875, "lang": "zh mandarin", "lang_confidence": 0.999}], "vad_segments_ms": [[310, 1840]], "dur_s": 2.32, "words": [{"start_ms": 490, "end_ms": 690, "text": "你"}, {"start_ms": 690, "end_ms": 1090, "text": "好"}, {"start_ms": 1090, "end_ms": 1330, "text": "世"}, {"start_ms": 1330, "end_ms": 1795, "text": "界"}], "wav_path": "assets/hello_zh.wav"}
# {"uttid": "hello_en", "text": "Hello speech.", "sentences": [{"start_ms": 120, "end_ms": 1840, "text": "Hello speech.", "asr_confidence": 0.833, "lang": "en", "lang_confidence": 0.998}], "vad_segments_ms": [[120, 1840]], "dur_s": 2.24, "words": [{"start_ms": 340, "end_ms": 1020, "text": "hello"}, {"start_ms": 1020, "end_ms": 1666, "text": "speech"}], "wav_path": "assets/hello_en.wav"}
```

--------------------------------

### Serve FireRedASR2-LLM with vLLM

Source: https://huggingface.co/FireRedTeam/FireRedPunc/commit/e448fd967f44182a1c323cc30f5d89f2400c28da

Commands to serve the model using vLLM and run a transcription client.

```shell
$ vllm serve allendou/FireRedASR2-LLM-vllm -tp=2 --dtype=float32
$ python3 examples/online_serving/openai_transcription_client.py --repetition_penalty=1.0 --audio_path=/root/hello_zh.wav
```

--------------------------------

### Run Full ASR System Pipeline

Source: https://huggingface.co/FireRedTeam/FireRedPunc/blob/main/README.md?code=true

Integrates VAD, LID, ASR, and Punctuation modules into a single processing pipeline.

```python
from fireredasr2s.fireredasr2 import FireRedAsr2Config
from fireredasr2s.fireredlid import FireRedLidConfig
from fireredasr2s.fireredpunc import FireRedPuncConfig
from fireredasr2s.fireredvad import FireRedVadConfig
from fireredasr2s import FireRedAsr2System, FireRedAsr2SystemConfig

vad_config = FireRedVadConfig(
 use_gpu=False,
 smooth_window_size=5,
 speech_threshold=0.4,
 min_speech_frame=20,
 max_speech_frame=2000,
 min_silence_frame=20,
 merge_silence_frame=0,
 extend_speech_frame=0,
 chunk_max_frame=30000
)
lid_config = FireRedLidConfig(use_gpu=True, use_half=False)
asr_config = FireRedAsr2Config(
 use_gpu=True,
 use_half=False,
 beam_size=3,
 nbest=1,
 decode_max_len=0,
 softmax_smoothing=1.25,
 aed_length_penalty=0.6,
 eos_penalty=1.0,
 return_timestamp=True
)
punc_config = FireRedPuncConfig(use_gpu=True)

asr_system_config = FireRedAsr2SystemConfig(
 "pretrained_models/FireRedVAD/VAD",
 "pretrained_models/FireRedLID",
 "aed", "pretrained_models/FireRedASR2-AED",
 "pretrained_models/FireRedPunc",
 vad_config, lid_config, asr_config, punc_config,
 enable_vad=1, enable_lid=1, enable_punc=1
)
asr_system = FireRedAsr2System(asr_system_config)

batch_uttid = ["hello_zh", "hello_en"]
batch_wav_path = ["assets/hello_zh.wav", "assets/hello_en.wav"]
for wav_path, uttid in zip(batch_wav_path, batch_uttid):
 result = asr_system.process(wav_path, uttid)
 print(result)
# {'uttid': 'hello_zh', 'text': '你好世界。', 'sentences': [{'start_ms': 440, 'end_ms': 1820, 'text': '你好世界。', 'asr_confidence': 0.868, 'lang': 'zh mandarin', 'lang_confidence': 0.999}], 'vad_segments_ms': [(440, 1820)], 'dur_s': 2.32, 'words': [{'start_ms': 540, 'end_ms': 700, 'text': '你'}, {'start_ms': 700, 'end_ms': 1100, 'text': '好'}, {'start_ms': 1100, 'end_ms': 1300, 'text': '世'}, {'start_ms': 1300, 'end_ms': 1765, 'text': '界'}], 'wav_path': 'assets/hello_zh.wav'}
# {'uttid': 'hello_en', 'text': 'Hello speech.', 'sentences': [{'start_ms': 260, 'end_ms': 1820, 'text': 'Hello speech.', 'asr_confidence': 0.933, 'lang': 'en', 'lang_confidence': 0.993}], 'vad_segments_ms': [(260, 1820)], 'dur_s': 2.24, 'words': [{'start_ms': 400, 'end_ms': 960, 'text': 'hello'}, {'start_ms': 960, 'end_ms': 1666, 'text': 'speech'}], 'wav_path': 'assets/hello_en.wav'}
```

--------------------------------

### Python API Usage for FireRedAsr2

Source: https://huggingface.co/FireRedTeam/FireRedPunc

Initialize the FireRedAsr2 model with a configuration and process batch audio files. Ensure PYTHONPATH is set.

```python
from fireredasr2s.fireredasr2 import FireRedAsr2, FireRedAsr2Config

batch_uttid = ["hello_zh", "hello_en"]
batch_wav_path = ["assets/hello_zh.wav", "assets/hello_en.wav"]

```

--------------------------------

### Initialize and Run Audio Event Detection (AED)

Source: https://huggingface.co/FireRedTeam/FireRedPunc/blob/main/README.md?code=true

Configures and runs the FireRedAed model to detect speech, singing, and music events in an audio file.

```python
from fireredasr2s.fireredvad import FireRedAed, FireRedAedConfig

aed_config=FireRedAedConfig(
 use_gpu=False,
 smooth_window_size=5,
 speech_threshold=0.4,
 singing_threshold=0.5,
 music_threshold=0.5,
 min_event_frame=20,
 max_event_frame=2000,
 min_silence_frame=20,
 merge_silence_frame=0,
 extend_speech_frame=0,
 chunk_max_frame=30000)
aed = FireRedAed.from_pretrained("pretrained_models/FireRedVAD/AED", aed_config)

result, probs = aed.detect("assets/event.wav")

print(result)
# {'dur': 22.016, 'event2timestamps': {'speech': [(0.4, 3.56), (3.66, 9.08), (9.27, 9.77), (10.78, 21.76)], 'singing': [(1.79, 19.96), (19.97, 22.016)], 'music': [(0.09, 12.32), (12.33, 22.016)]}, 'event2ratio': {'speech': 0.848, 'singing': 0.905, 'music': 0.991}, 'wav_path': 'assets/event.wav'}
```

--------------------------------

### Python API for ASR System

Source: https://huggingface.co/FireRedTeam/FireRedPunc/blame/main/README.md

Initialize the FireRedAsr2System with a default configuration and process audio files for ASR. The output is a dictionary containing transcription results.

```python
from fireredasr2s import FireRedAsr2System, FireRedAsr2SystemConfig

asr_system_config = FireRedAsr2SystemConfig()  # Use default config
asr_system = FireRedAsr2System(asr_system_config)

result = asr_system.process("assets/hello_zh.wav")
print(result)

result = asr_system.process("assets/hello_en.wav")
print(result)
```

--------------------------------

### Stream VAD Initialization and Detection

Source: https://huggingface.co/FireRedTeam/FireRedPunc/commit/bbceaf3e89869d1d26b1207320275102f9e53b48

Configures and runs the Stream VAD model to detect speech segments in an audio file.

```python
from fireredasr2s.fireredvad import FireRedStreamVad, FireRedStreamVadConfig

vad_config=FireRedStreamVadConfig(
use_gpu=False,
smooth_window_size=5,
speech_threshold=0.4,
pad_start_frame=5,
min_speech_frame=8,
max_speech_frame=2000,
min_silence_frame=20,
chunk_max_frame=30000)
stream_vad = FireRedStreamVad.from_pretrained("pretrained_models/FireRedVAD/Stream-VAD", vad_config)

frame_results, result = stream_vad.detect_full("assets/hello_zh.wav")

print(result)
# {'dur': 2.32, 'timestamps': [(0.46, 1.84)], 'wav_path': 'assets/hello_zh.wav'}
```

--------------------------------

### Audio Event Detection (AED) Initialization and Detection

Source: https://huggingface.co/FireRedTeam/FireRedPunc/commit/bbceaf3e89869d1d26b1207320275102f9e53b48

Configures and runs the AED model to identify speech, singing, and music events in an audio file.

```python
from fireredasr2s.fireredvad import FireRedAed, FireRedAedConfig

aed_config=FireRedAedConfig(
use_gpu=False,
smooth_window_size=5,
speech_threshold=0.4,
singing_threshold=0.5,
music_threshold=0.5,
min_event_frame=20,
max_event_frame=2000,
min_silence_frame=20,
merge_silence_frame=0,
extend_speech_frame=0,
chunk_max_frame=30000)
aed = FireRedAed.from_pretrained("pretrained_models/FireRedVAD/AED", aed_config)

result, probs = aed.detect("assets/event.wav")

print(result)
# {'dur': 22.016, 'event2timestamps': {'speech': [(0.4, 3.56), (3.66, 9.08), (9.27, 9.77), (10.78, 21.76)], 'singing': [(1.79, 19.96), (19.97, 22.016)], 'music': [(0.09, 12.32), (12.33, 22.016)]}, 'event2ratio': {'speech': 0.848, 'singing': 0.905, 'music': 0.991}, 'wav_path': 'assets/event.wav'}
```

--------------------------------

### Run ASR LLM Inference

Source: https://huggingface.co/FireRedTeam/FireRedPunc/commit/e448fd967f44182a1c323cc30f5d89f2400c28da

Execute the inference script for the FireRedASR2-LLM model. Ensure your audio is in the correct format (16kHz 16-bit mono PCM).

```bash
bash inference_asr_llm.sh
```

--------------------------------

### Process Audio Files with FireRedAsr2System

Source: https://huggingface.co/FireRedTeam/FireRedPunc/commit/bbceaf3e89869d1d26b1207320275102f9e53b48

Iterate through a list of audio paths and identifiers to perform speech recognition.

```python
batch_uttid = ["hello_zh", "hello_en"]
batch_wav_path = ["assets/hello_zh.wav", "assets/hello_en.wav"]
for wav_path, uttid in zip(batch_wav_path, batch_uttid):
result = asr_system.process(wav_path, uttid)
print(result)
# {'uttid': 'hello_zh', 'text': '你好世界。', 'sentences': [{'start_ms': 440, 'end_ms': 1820, 'text': '你好世界。', 'asr_confidence': 0.868, 'lang': 'zh mandarin', 'lang_confidence': 0.999}], 'vad_segments_ms': [(440, 1820)], 'dur_s': 2.32, 'words': [{'start_ms': 540, 'end_ms': 700, 'text': '你'}, {'start_ms': 700, 'end_ms': 1100, 'text': '好'}, {'start_ms': 1100, 'end_ms': 1300, 'text': '世'}, {'start_ms': 1300, 'end_ms': 1765, 'text': '界'}], 'wav_path': 'assets/hello_zh.wav'}
# {'uttid': 'hello_en', 'text': 'Hello speech.', 'sentences': [{'start_ms': 260, 'end_ms': 1820, 'text': 'Hello speech.', 'asr_confidence': 0.933, 'lang': 'en', 'lang_confidence': 0.993}], 'vad_segments_ms': [(260, 1820)], 'dur_s': 2.24, 'words': [{'start_ms': 400, 'end_ms': 960, 'text': 'hello'}, {'start_ms': 960, 'end_ms': 1666, 'text': 'speech'}], 'wav_path': 'assets/hello_en.wav'}
```