### TensorRT Installation Output

Source: https://github.com/shashikg/whispers2t/blob/main/notebooks/WhisperS2T_TensorRT_LLM.ipynb

This output shows the process of downloading the TensorRT installation script and the subsequent installation of build tools and other necessary packages on an Ubuntu system.

```text
Output:
--2024-01-28 12:18:44--  https://github.com/shashikg/WhisperS2T/raw/main/install_tensorrt.sh
Resolving github.com (github.com)... 140.82.114.3
Connecting to github.com (github.com)|140.82.114.3|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://raw.githubusercontent.com/shashikg/WhisperS2T/main/install_tensorrt.sh [following]
--2024-01-28 12:18:44--  https://raw.githubusercontent.com/shashikg/WhisperS2T/main/install_tensorrt.sh
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.111.133, 185.199.109.133, 185.199.110.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.111.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1206 (1.2K) [text/plain]
Saving to: ‘install_tensorrt.sh’

install_tensorrt.sh 100%[===================>]   1.18K  --.-KB/s    in 0s      

2024-01-28 12:18:45 (68.4 MB/s) - ‘install_tensorrt.sh’ saved [1206/1206] 


###########################[ Installing Build Tools ]##########################
Hit:1 https://cloud.r-project.org/bin/linux/ubuntu jammy-cran40/ InRelease
Hit:2 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  InRelease
Hit:3 http://archive.ubuntu.com/ubuntu jammy InRelease
Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease
Hit:5 http://archive.ubuntu.com/ubuntu jammy-updates InRelease
Hit:6 http://archive.ubuntu.com/ubuntu jammy-backports InRelease
Hit:7 https://ppa.launchpadcontent.net/c2d4u.team/c2d4u4.0+/ubuntu jammy InRelease
Hit:8 https://ppa.launchpadcontent.net/deadsnakes/ppa/ubuntu jammy InRelease
Hit:9 https://ppa.launchpadcontent.net/graphics-drivers/ppa/ubuntu jammy InRelease
Hit:10 https://ppa.launchpadcontent.net/ubuntugis/ppa/ubuntu jammy InRelease
Reading package lists... Done
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
build-essential is already the newest version (12.9ubuntu3).
wget is already the newest version (1.21.2-2ubuntu1).
ca-certificates is already the newest version (20230311ubuntu0.22.04.1).
cmake is already the newest version (3.22.1-1ubuntu1.22.04.1).
curl is already the newest version (7.81.0-1ubuntu1.15).
gnupg2 is already the newest version (2.2.27-3ubuntu2.1).
The following additional packages will be installed:
  libbabeltrace1 libc6-dbg libdebuginfod-common libdebuginfod1 libhiredis0.14 libipt2
  libsource-highlight-common libsource-highlight4v5
Suggested packages:
  distcc | icecc gdb-doc gdbserver
The following NEW packages will be installed:
  ccache gdb libbabeltrace1 libc6-dbg libdebuginfod-common libdebuginfod1 libhiredis0.14 libipt2
  libsource-highlight-common libsource-highlight4v5
0 upgraded, 10 newly installed, 0 to remove and 81 not upgraded.
Need to get 18.8 MB of archives.
After this operation, 33.4 MB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 libdebuginfod-common all 0.186-1build1 [7,878 B]
Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libhiredis0.14 amd64 0.14.1-2 [32.8 kB]
Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 ccache amd64 4.5.1-1 [495 kB]
Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 libbabeltrace1 amd64 1.5.8-2build1 [160 kB]

```

--------------------------------

### Install system dependencies and WhisperS2T

Source: https://github.com/shashikg/whispers2t/blob/main/notebooks/WhisperS2T_CTranslate2.ipynb

Updates the system's package list and installs essential libraries like libsndfile1 and ffmpeg. It then installs the WhisperS2T package from its GitHub repository.

```bash
!apt-get update && apt-get install -y libsndfile1 ffmpeg
!pip install -U git+https://github.com/shashikg/WhisperS2T.git
```

--------------------------------

### Install MPI4PY

Source: https://github.com/shashikg/whispers2t/blob/main/notebooks/WhisperS2T_TensorRT_LLM.ipynb

Installs the mpi4py library from a local source directory.

```bash
Processing /tmp/mpi4py-3.1.5
  Installing build dependencies ... [?25l[?25hdone
  Getting requirements to build wheel ... [?25l[?25hdone
  Installing backend dependencies ... [?25l[?25hdone
  Preparing metadata (pyproject.toml) ... [?25l[?25hdone
Building wheels for collected packages: mpi4py
  Building wheel for mpi4py (pyproject.toml) ... [?25l[?25hdone
  Created wheel for mpi4py: filename=mpi4py-3.1.5-cp310-cp310-linux_x86_64.whl size=2746210 sha256=adb8e616a82eb29c5277bbcba9d0e195e5a470911b78680f18368c6e55216d97
  Stored in directory: /root/.cache/pip/wheels/54/23/8d/2c18465d0c7f5d94d6b3ceca3fbbd63e823e38a7f012793810
Successfully built mpi4py
Installing collected packages: mpi4py
Successfully installed mpi4py-3.1.5
```

--------------------------------

### Install TensorRT-LLM

Source: https://github.com/shashikg/whispers2t/blob/main/notebooks/WhisperS2T_TensorRT_LLM.ipynb

Installs the TensorRT-LLM package and its dependencies from the NVIDIA PyPI index.

```bash
Looking in indexes: https://pypi.org/simple, https://pypi.nvidia.com
Collecting tensorrt_llm==0.8.0.dev2024012301
  Downloading https://pypi.nvidia.com/tensorrt-llm/tensorrt_llm-0.8.0.dev2024012301-cp310-cp310-linux_x86_64.whl (1063.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 GB 1.5 MB/s eta 0:00:00
[?25hCollecting accelerate==0.25.0 (from tensorrt_llm==0.8.0.dev2024012301)
  Downloading accelerate-0.25.0-py3-none-any.whl (265 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 265.7/265.7 kB 6.9 MB/s eta 0:00:00
[?25hRequirement already satisfied: build in /usr/local/lib/python3.10/dist-packages (from tensorrt_llm==0.8.0.dev2024012301) (1.0.3)
Collecting colored (from tensorrt_llm==0.8.0.dev2024012301)
  Downloading colored-2.2.4-py3-none-any.whl (16 kB)
Collecting cuda-python (from tensorrt_llm==0.8.0.dev2024012301)
  Downloading cuda_python-12.3.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (23.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 23.6/23.6 MB 57.4 MB/s eta 0:00:00
[?25hCollecting diffusers==0.15.0 (from tensorrt_llm==0.8.0.dev2024012301)
  Downloading diffusers-0.15.0-py3-none-any.whl (851 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 851.8/851.8 kB 74.0 MB/s eta 0:00:00
[?25hCollecting lark (from tensorrt_llm==0.8.0.dev2024012301)
  Downloading lark-1.1.9-py3-none-any.whl (111 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 111.7/111.7 kB 17.3 MB/s eta 0:00:00
[?25hRequirement already satisfied: mpi4py in /usr/local/lib/python3.10/dist-packages (from tensorrt_llm==0.8.0.dev2024012301) (3.1.5)
Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from tensorrt_llm==0.8.0.dev2024012301) (1.23.5)
Collecting onnx>=1.12.0 (from tensorrt_llm==0.8.0.dev2024012301)
  Downloading onnx-1.15.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (15.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 15.7/15.7 MB 59.8 MB/s eta 0:00:00
[?25hCollecting polygraphy (from tensorrt_llm==0.8.0.dev2024012301)
  Downloading https://pypi.nvidia.com/polygraphy/polygraphy-0.49.0-py2.py3-none-any.whl (327 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 327.9/327.9 kB 35.2 MB/s eta 0:00:00
[?25hRequirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from tensorrt_llm==0.8.0.dev2024012301) (5.9.5)
Collecting pynvml>=11.5.0 (from tensorrt_llm==0.8.0.dev2024012301)
  Downloading pynvml-11.5.0-py3-none-any.whl (53 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 53.1/53.1 kB 7.9 MB/s eta 0:00:00
```

--------------------------------

### Install Audio Packages on Ubuntu

Source: https://github.com/shashikg/whispers2t/blob/main/README.md

Install necessary audio packages (libsndfile1 and ffmpeg) on Ubuntu systems for audio file processing.

```sh
apt-get install -y libsndfile1 ffmpeg
```

--------------------------------

### Install TensorRT-LLM Backend

Source: https://github.com/shashikg/whispers2t/blob/main/README.md

Execute the provided bash script to install the TensorRT-LLM backend. For other systems or if the script fails, refer to the official TensorRT-LLM GitHub repository.

```sh
bash <repo_dir>/install_tensorrt.sh
```

--------------------------------

### Install ipython_autotime

Source: https://github.com/shashikg/whispers2t/blob/main/notebooks/WhisperS2T_CTranslate2.ipynb

Installs the ipython_autotime package, which can be used to automatically time code execution in IPython environments. Ensure you have pip installed.

```python
!pip install ipython_autotime
```

--------------------------------

### Install WhisperS2T from Latest Commit

Source: https://github.com/shashikg/whispers2t/blob/main/README.md

Install the WhisperS2T package directly from the latest commit in the GitHub repository using pip.

```sh
pip install -U git+https://github.com/shashikg/WhisperS2T.git
```

--------------------------------

### Download and Execute TensorRT Installation Script

Source: https://github.com/shashikg/whispers2t/blob/main/notebooks/WhisperS2T_TensorRT_LLM.ipynb

Downloads the install_tensorrt.sh script using wget and then executes it with bash. This script is necessary for setting up TensorRT, which is a prerequisite for optimized performance.

```bash
!wget https://github.com/shashikg/WhisperS2T/raw/main/install_tensorrt.sh
!bash install_tensorrt.sh
```

--------------------------------

### Install FFmpeg on MAC

Source: https://github.com/shashikg/whispers2t/blob/main/README.md

Install FFmpeg on macOS using Homebrew for audio file processing.

```sh
brew install ffmpeg
```

--------------------------------

### Install WhisperS2T and Dependencies

Source: https://github.com/shashikg/whispers2t/blob/main/notebooks/WhisperS2T_TensorRT_LLM.ipynb

Installs the whisper-s2t and openai-whisper packages along with their dependencies using pip. It also notes potential dependency conflicts.

```bash
pip install --upgrade pip
pip install -r requirements.txt
```

--------------------------------

### Transcribe Audio with VAD using TensorRT-LLM Backend

Source: https://github.com/shashikg/whispers2t/blob/main/README.md

Perform speech-to-text transcription on audio files using the loaded model with VAD enabled. This example demonstrates specifying language codes, tasks, and initial prompts. The output contains text, start, and end times for each utterance.

```python
files = ['data/KINCAID46/audio/1.wav']
lang_codes = ['en']
tasks = ['transcribe']
initial_prompts = [None]

out = model.transcribe_with_vad(files,
                                lang_codes=lang_codes,
                                tasks=tasks,
                                initial_prompts=initial_prompts,
                                batch_size=24)

print(out[0][0]) # Print first utterance for first file
```

--------------------------------

### Install FFmpeg with Conda

Source: https://github.com/shashikg/whispers2t/blob/main/README.md

Install FFmpeg using Conda, which is compatible with Ubuntu, MAC, Windows, and other systems.

```sh
conda install conda-forge::ffmpeg
```

--------------------------------

### Install Latest WhisperS2T Release

Source: https://github.com/shashikg/whispers2t/blob/main/README.md

Install or update to the latest released version of the WhisperS2T Python package using pip.

```sh
pip install -U whisper-s2t
```

--------------------------------

### Displaying Processing Time

Source: https://github.com/shashikg/whispers2t/blob/main/notebooks/WhisperS2T_CTranslate2.ipynb

This output indicates the total time taken for a processing step, including the start time.

```text
time: 39.6 s (started: 2024-01-28 12:02:33 +00:00)
```

--------------------------------

### Build Docker Container with TensorRT-LLM Support

Source: https://github.com/shashikg/whispers2t/blob/main/README.md

Build a Docker container from the main branch with TensorRT-LLM support enabled. This command assumes the TensorRT-LLM installation script is available.

```sh
docker build --build-arg WHISPER_S2T_VER=main -t whisper_s2t:main-trtllm .
```

--------------------------------

### Load WhisperS2T Model with TensorRT-LLM Backend

Source: https://github.com/shashikg/whispers2t/blob/main/notebooks/WhisperS2T_TensorRT_LLM.ipynb

Load the WhisperS2T model using the specified identifier and backend. Ensure TensorRT-LLM is installed and configured.

```python
import whisper_s2t
model = whisper_s2t.load_model(model_identifier="large-v2", backend='TensorRT-LLM')
```

--------------------------------

### Displaying Total Transcription Time

Source: https://github.com/shashikg/whispers2t/blob/main/notebooks/WhisperS2T_CTranslate2.ipynb

This output indicates the total time taken for the transcription process, including the start time.

```text
time: 3min 22s (started: 2024-01-28 12:03:24 +00:00)
```

--------------------------------

### Return Word-Alignments with CTranslate2/TensorRT

Source: https://github.com/shashikg/whispers2t/blob/main/docs.md

Use this for CTranslate2 and TensorRT backends to get word-level timestamps. Ensure 'word_timestamps': True is set in ASR options.

```python
import whisper_s2t

model = whisper_s2t.load_model(model_identifier="large-v2", asr_options={'word_timestamps': True})

files = ['sample_1.wav']
lang_codes = ['en']
tasks = ['transcribe']
initial_prompts = [None]

out = model.transcribe_with_vad(files,
                                lang_codes=lang_codes, # pass lang_codes for each file
                                tasks=tasks, # pass transcribe/translate 
                                initial_prompts=initial_prompts, # to do prompting (currently only supported for CTranslate2 backend)
                                batch_size=24)

print(out[0][0]) # Print first utterance for first file

```

--------------------------------

### Set LD_LIBRARY_PATH for CUDNN and CUBLAS

Source: https://github.com/shashikg/whispers2t/blob/main/README.md

Export the LD_LIBRARY_PATH environment variable to include CUDNN and CUBLAS library paths, typically needed when installations are done via pip wheels.

```sh
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:`python3 -c 'import os; import nvidia.cublas.lib; import nvidia.cudnn.lib; print(os.path.dirname(nvidia.cublas.lib.__file__) + ":" + os.path.dirname(nvidia.cudnn.lib.__file__))'`
```

--------------------------------

### Transcription Result Output

Source: https://github.com/shashikg/whispers2t/blob/main/notebooks/WhisperS2T_CTranslate2.ipynb

This is the output of the transcription, showing the transcribed text, average log probability, probability of no speech, start time, and end time for a segment of the audio.

```text
{'text': 'We have been a misunderstood and badly mocked org for a long time. Like when we started, we like announced the org at the end of 2015 and said we were going to work on AGI, like people thought we were batshit insane. Yeah. You know, like I remember at the time an eminent AI scientist at a', 'avg_logprob': -0.26604661215906555, 'no_speech_prob': 0.0018663406372070312, 'start_time': 0.0, 'end_time': 21.26}
time: 823 µs (started: 2024-01-28 12:10:14 +00:00)
```

--------------------------------

### Download Audio File

Source: https://github.com/shashikg/whispers2t/blob/main/notebooks/WhisperS2T_CTranslate2.ipynb

Downloads a sample audio file from Hugging Face using wget.

```bash
!wget https://huggingface.co/datasets/reach-vb/random-audios/resolve/main/sam_altman_lex_podcast_367.flac
```

--------------------------------

### Load and Preprocess Audio Files

Source: https://context7.com/shashikg/whispers2t/llms.txt

Utilize load_audio for automatic resampling and pad_or_trim to normalize audio length for model input. These utilities support various formats via FFmpeg.

```python
from whisper_s2t.audio import load_audio, pad_or_trim
from whisper_s2t.configs import N_SAMPLES, SAMPLE_RATE
import numpy as np

# Load audio file (automatically resamples to 16kHz mono)
audio_signal = load_audio('audio.mp3', sr=16000)
print(f"Audio shape: {audio_signal.shape}")
print(f"Duration: {len(audio_signal) / 16000:.2f} seconds")

# Load with duration info
audio_signal, duration = load_audio('audio.wav', sr=16000, return_duration=True)
print(f"Audio duration: {duration:.2f} seconds")

# Pad or trim audio to expected length (30 seconds = 480000 samples)
audio_padded = pad_or_trim(audio_signal, length=N_SAMPLES)
print(f"Padded audio shape: {audio_padded.shape}")  # (480000,)

# Works with numpy arrays
short_audio = np.random.randn(160000)  # 10 seconds
padded = pad_or_trim(short_audio, length=N_SAMPLES)
print(f"Short audio padded: {padded.shape}")  # (480000,)

# Trim long audio
long_audio = np.random.randn(800000)  # 50 seconds
trimmed = pad_or_trim(long_audio, length=N_SAMPLES)
print(f"Long audio trimmed: {trimmed.shape}")  # (480000,)
```

--------------------------------

### Displaying Model File Download Progress

Source: https://github.com/shashikg/whispers2t/blob/main/notebooks/WhisperS2T_CTranslate2.ipynb

This output shows the download progress for a specific model file, 'model.bin'. It includes the file size and download speed.

```text
model.bin:   0%|          | 0.00/3.09G [00:00<?, ?B/s]
```

--------------------------------

### Initialize WhisperS2T with HuggingFace

Source: https://github.com/shashikg/whispers2t/blob/main/docs.md

Load the model using the HuggingFace backend, which defaults to FlashAttention2 on compatible Nvidia GPUs.

```python
model = whisper_s2t.load_model(model_identifier="large-v2", backend='HuggingFace') # Supported backends ['CTranslate2', 'HuggingFace', 'OpenAI']
```

--------------------------------

### Initialize Multilingual Transcription Model

Source: https://context7.com/shashikg/whispers2t/llms.txt

Basic initialization for multilingual transcription and translation tasks.

```python
import whisper_s2t

model = whisper_s2t.load_model(model_identifier="large-v2", backend='CTranslate2')
```

--------------------------------

### Configure TensorRT-LLM Backend

Source: https://context7.com/shashikg/whispers2t/llms.txt

Set up the TensorRT-LLM backend for optimized performance on NVIDIA GPUs, including loading from pre-built engines.

```python
import whisper_s2t
from whisper_s2t.backends.tensorrt.engine_builder import TRTBuilderConfig

# Define TensorRT build configuration
trt_build_args = TRTBuilderConfig(
    max_output_len=448,      # Maximum output token length
    max_beam_width=5,        # Maximum beam search width
    # Additional TRT options can be configured here
)

# Load model with TensorRT-LLM backend
model = whisper_s2t.load_model(
    model_identifier="large-v2",
    backend='TensorRT-LLM',
    asr_options={
        'beam_size': 5,
        'word_timestamps': True,
    },
    trt_build_args=trt_build_args
)

# Alternatively, load from pre-built engine directory
model_from_engine = whisper_s2t.load_model(
    model_identifier="/path/to/tensorrt/engine",
    backend='TensorRT-LLM'
)

# Transcribe with TensorRT backend
files = ['audio.wav']
lang_codes = ['en']
tasks = ['transcribe']
initial_prompts = [None]

out = model.transcribe_with_vad(
    files,
    lang_codes=lang_codes,
    tasks=tasks,
    initial_prompts=initial_prompts,
    batch_size=24
)

# Output format same as other backends
for utterance in out[0]:
    print(f"[{utterance['start_time']:.2f}s]: {utterance['text']}")
```

--------------------------------

### Pass Custom Model Configuration

Source: https://github.com/shashikg/whispers2t/blob/main/docs.md

Apply custom configurations during model loading or update them dynamically.

```python
import whisper_s2t
from whisper_s2t.backends.ctranslate2.model import BEST_ASR_CONFIG

model_kwargs = {
    'compute_type': 'int8', # Note int8 is only supported for CTranslate2 backend, for others only float16 is supported for lower precision.
    'asr_options': BEST_ASR_CONFIG
}

model = whisper_s2t.load_model(model_identifier="large-v2", backend='CTranslate2', **model_kwargs)
```

```python
model.update_params(model_kwargs)
```

--------------------------------

### Displaying Config JSON Download Progress

Source: https://github.com/shashikg/whispers2t/blob/main/notebooks/WhisperS2T_CTranslate2.ipynb

This output indicates the download progress for the 'config.json' file, showing its size and download speed.

```text
config.json:   0%|          | 0.00/2.80k [00:00<?, ?B/s]
```

--------------------------------

### Displaying Vocabulary File Download Progress

Source: https://github.com/shashikg/whispers2t/blob/main/notebooks/WhisperS2T_CTranslate2.ipynb

This output indicates the download progress for the 'vocabulary.txt' file, showing its size and download speed.

```text
vocabulary.txt:   0%|          | 0.00/460k [00:00<?, ?B/s]
```

--------------------------------

### Displaying Tokenizer JSON Download Progress

Source: https://github.com/shashikg/whispers2t/blob/main/notebooks/WhisperS2T_CTranslate2.ipynb

This output shows the download progress for the 'tokenizer.json' file, including its size and download speed.

```text
tokenizer.json:   0%|          | 0.00/2.20M [00:00<?, ?B/s]
```

--------------------------------

### Configure Speech Segmenter Options

Source: https://context7.com/shashikg/whispers2t/llms.txt

Define parameters for speech segmentation and update them dynamically after model initialization.

```python
speech_segmenter_options = {
    'eos_thresh': 0.2,       # End-of-speech threshold (lower = more sensitive)
    'bos_thresh': 0.2,       # Beginning-of-speech threshold
    'max_seg_len': 29.0,     # Maximum segment length in seconds
    'min_seg_len': 0.1,      # Minimum segment length in seconds
    'max_silent_region': 0.8, # Maximum silent region to merge segments
    'padding': 0.2,          # Padding around detected speech in seconds
}

model = whisper_s2t.load_model(
    model_identifier="large-v2",
    backend='CTranslate2',
    speech_segmenter_options=speech_segmenter_options
)

# Update segmenter options after loading
model.speech_segmenter.update_params({
    'eos_thresh': 0.15,
    'bos_thresh': 0.15,
})

# Update underlying VAD model parameters
model.speech_segmenter.update_vad_model_params({
    'batch_size': 8,
    'chunk_size': 20.0,
    'margin_size': 1.5,
})

# Transcribe with configured segmenter
files = ['long_audio.wav']
out = model.transcribe_with_vad(files, lang_codes=['en'], tasks=['transcribe'], batch_size=32)
print(f"Detected {len(out[0])} speech segments")
```

--------------------------------

### TensorRT-LLM Build Configuration

Source: https://github.com/shashikg/whispers2t/blob/main/notebooks/WhisperS2T_TensorRT_LLM.ipynb

Displays the configuration parameters used for building the TensorRT-LLM engine. These settings optimize model performance for specific hardware.

```json
{
  'max_batch_size': 24,
  'max_beam_width': 1,
  'max_input_len': 4,
  'max_output_len': 448,
  'world_size': 1,
  'dtype': 'float16',
  'quantize_dir': 'quantize/1-gpu',
  'use_gpt_attention_plugin': 'float16',
  'use_bert_attention_plugin': None,
  'use_context_fmha_enc': False,
  'use_context_fmha_dec': False,
  'use_gemm_plugin': 'float16',
  'use_layernorm_plugin': False,
  'remove_input_padding': False,
  'use_weight_only_enc': False,
  'use_weight_only_dec': False,
  'weight_only_precision': 'int8',
  'int8_kv_cache': False,
  'debug_mode': False,
  'cuda_compute_capability': [7, 5],
  'output_dir': '/root/.cache/whisper_s2t/models/trt/large-v2/54692706a4675997e48447facdd30a40',
  'model_path': '/root/.cache/whisper_s2t/models/trt/large-v2/pt_ckpt.pt'
}
```

--------------------------------

### Configure VAD Parameters

Source: https://github.com/shashikg/whispers2t/blob/main/docs.md

Adjust VAD sensitivity thresholds during model initialization.

```python
speech_segmenter_options = {
    'eos_thresh': 0.1,
    'bos_thresh': 0.1,
}

model = whisper_s2t.load_model(speech_segmenter_options=speech_segmenter_options)
```

--------------------------------

### Load Whisper Model with Different Backends

Source: https://context7.com/shashikg/whispers2t/llms.txt

Demonstrates loading a Whisper model using the `load_model` function with various backends like CTranslate2, HuggingFace, and TensorRT-LLM. It also shows how to enable word timestamps and load specific model versions.

```python
import whisper_s2t

# Load model with CTranslate2 backend (default, fastest for most use cases)
model = whisper_s2t.load_model(model_identifier="large-v2", backend='CTranslate2')

# Load model with HuggingFace backend (uses FlashAttention2 on Ampere/Hopper GPUs)
model_hf = whisper_s2t.load_model(model_identifier="large-v2", backend='HuggingFace')

# Load model with TensorRT-LLM backend (requires TensorRT installation)
model_trt = whisper_s2t.load_model(model_identifier="large-v2", backend='TensorRT-LLM')

# Load model with word timestamps enabled
model_with_words = whisper_s2t.load_model(
    model_identifier="large-v2",
    backend='CTranslate2',
    asr_options={'word_timestamps': True}
)

# Load Whisper Large-V3 (automatically sets n_mels=128)
model_v3 = whisper_s2t.load_model(model_identifier="large-v3", backend='CTranslate2')

# Load Distil-Whisper (automatically switches to HuggingFace backend)
model_distil = whisper_s2t.load_model(model_identifier="distil-large-v2")

print(f"Model loaded successfully on device: {model.device}")
```

--------------------------------

### Build Custom Docker Container (Specific Release)

Source: https://github.com/shashikg/whispers2t/blob/main/README.md

Build a Docker container from a specific release tag (e.g., v1.3.0). Ensure you checkout the tag before building. Set SKIP_TENSORRT_LLM to 1 to exclude TensorRT-LLM support.

```sh
git checkout v1.3.0
docker build --build-arg WHISPER_S2T_VER=v1.3.0 --build-arg SKIP_TENSORRT_LLM=1 -t whisper_s2t:1.3.0 .
```

--------------------------------

### Displaying File Fetch Progress

Source: https://github.com/shashikg/whispers2t/blob/main/notebooks/WhisperS2T_CTranslate2.ipynb

This output indicates the progress of fetching multiple files. It shows the percentage completion and the number of files being processed.

```text
Fetching 4 files:   0%|          | 0/4 [00:00<?, ?it/s]
```

--------------------------------

### Configure Speech Segmenter

Source: https://context7.com/shashikg/whispers2t/llms.txt

Initialize the speech segmenter configuration for VAD. This uses NVIDIA's NeMo VAD model.

```python
import whisper_s2t
```

--------------------------------

### Initialize Whisper-S2T Model

Source: https://github.com/shashikg/whispers2t/blob/main/notebooks/WhisperS2T_CTranslate2.ipynb

Loads the whisper-s2t model with the CTranslate2 backend.

```python
import whisper_s2t
model = whisper_s2t.load_model(model_identifier="large-v2", backend='CTranslate2')
```

--------------------------------

### Initialize WhisperS2T with CTranslate2

Source: https://github.com/shashikg/whispers2t/blob/main/docs.md

Load the model using the CTranslate2 backend and perform transcription with VAD enabled.

```python
import whisper_s2t

model = whisper_s2t.load_model(model_identifier="large-v2", backend='CTranslate2')

files = ['sample_1.wav']
lang_codes = ['en']
tasks = ['transcribe']
initial_prompts = [None]

out = model.transcribe_with_vad(files,
                                lang_codes=lang_codes, # pass lang_codes for each file
                                tasks=tasks, # pass transcribe/translate 
                                initial_prompts=initial_prompts, # to do prompting (currently only supported for CTranslate2 backend)
                                batch_size=16)

print(out[0][0]) # Print first utterance for first file
```

--------------------------------

### Build Custom Docker Container (Main Branch)

Source: https://github.com/shashikg/whispers2t/blob/main/README.md

Build a Docker container from the main branch of the WhisperS2T repository. Set SKIP_TENSORRT_LLM to 1 to exclude TensorRT-LLM support.

```sh
docker build --build-arg WHISPER_S2T_VER=main --build-arg SKIP_TENSORRT_LLM=1 -t whisper_s2t:main .
```

--------------------------------

### Use Custom VAD Model

Source: https://github.com/shashikg/whispers2t/blob/main/docs.md

Integrate a custom VAD implementation by passing it during model initialization.

```python
vad_model = CustomVAD()
model = whisper_s2t.load_model(model_identifier="large-v2", backend='CTranslate2', vad_model=vad_model)
```

--------------------------------

### Transcribe with VAD using Whispers2t

Source: https://github.com/shashikg/whispers2t/blob/main/notebooks/WhisperS2T_CTranslate2.ipynb

This Python code snippet demonstrates how to transcribe audio files using the `transcribe_with_vad` method from the Whispers2t model. It specifies the audio files, language codes, tasks, initial prompts, and batch size. Note that the VAD model may have slower inference speed on the first run due to JIT tracing.

```python
# Note: For first run the model may give slightly slower inference speed. After 1-2 runs it gives better inference speed. This is due to the JIT tracing of the VAD model.

files = ['sam_altman_lex_podcast_367.flac']
lang_codes = ['en']
tasks = ['transcribe']
initial_prompts = [None]

out = model.transcribe_with_vad(files,
                                lang_codes=lang_codes,
                                tasks=tasks,
                                initial_prompts=initial_prompts,
                                batch_size=24)
```

--------------------------------

### Enable Word-Level Timestamps

Source: https://context7.com/shashikg/whispers2t/llms.txt

Configure the model to return precise timing for each word using an alignment model. This requires setting word_timestamps to True and specifying a word_aligner_model.

```python
import whisper_s2t

# Load model with word timestamps enabled
model = whisper_s2t.load_model(
    model_identifier="large-v2",
    backend='CTranslate2',
    asr_options={
        'word_timestamps': True,
        'word_aligner_model': 'tiny'  # Uses tiny model for alignment
    }
)

files = ['speech.wav']
lang_codes = ['en']
tasks = ['transcribe']
initial_prompts = [None]

out = model.transcribe_with_vad(
    files,
    lang_codes=lang_codes,
    tasks=tasks,
    initial_prompts=initial_prompts,
    batch_size=24
)

# Access word-level timestamps
for utterance in out[0]:
    print(f"Utterance: {utterance['text']}")
    print("Word timestamps:")
    for word_info in utterance.get('word_timestamps', []):
        print(f"  '{word_info['word']}': {word_info['start']:.2f}s - {word_info['end']:.2f}s (prob: {word_info['prob']:.2f})")

# Output:
# Utterance: Let's bring in Phil Mackie who is there at the palace.
# Word timestamps:
#   'Let's': 0.00s - 0.24s (prob: 0.63)
#   'bring': 0.24s - 0.40s (prob: 0.96)
#   'in': 0.40s - 0.52s (prob: 0.71)
#   'Phil': 0.52s - 0.66s (prob: 0.46)
#   'Mackie': 0.66s - 1.02s (prob: 0.27)
```

--------------------------------

### Pull Prebuilt Docker Container

Source: https://github.com/shashikg/whispers2t/blob/main/README.md

Use this command to pull the latest development version of the WhisperS2T Docker container from Docker Hub.

```sh
docker pull shashikg/whisper_s2t:dev-trtllm
```

--------------------------------

### TensorRT Engine Build Times

Source: https://github.com/shashikg/whispers2t/blob/main/notebooks/WhisperS2T_TensorRT_LLM.ipynb

Shows the time taken to build the encoder and decoder components of the TensorRT engine. These times indicate the complexity of the model and the build process.

```text
Time taken for building Encoder: 113.45 seconds.
Time taken for building Decoder: 94.83 seconds.
```

--------------------------------

### Integrate Custom VAD Model

Source: https://context7.com/shashikg/whispers2t/llms.txt

Implement the VADBaseClass interface to use a custom Voice Activity Detection model for speech segmentation.

```python
import numpy as np
import whisper_s2t
from whisper_s2t.speech_segmenter import VADBaseClass

class CustomVAD(VADBaseClass):
    """Custom VAD model implementation."""

    def __init__(self, sampling_rate=16000, threshold=0.5):
        super().__init__(sampling_rate=sampling_rate)
        self.threshold = threshold
        # Initialize your custom VAD model here
        # self.model = load_your_vad_model()

    def update_params(self, params={}):
        """Update VAD parameters."""
        for key, value in params.items():
            setattr(self, key, value)

    def __call__(self, audio_signal):
        """
        Process audio and return speech probabilities.

        Args:
            audio_signal: numpy array of audio samples

        Returns:
            numpy array of shape (T, 3) where each row is:
            [speech_probability, frame_start_time, frame_end_time]
        """
        frame_size = 0.02  # 20ms frames
        audio_duration = len(audio_signal) / self.sampling_rate
        num_frames = int(audio_duration / frame_size)

        vad_output = []
        for idx in range(num_frames):
            start_time = idx * frame_size
            end_time = min((idx + 1) * frame_size, audio_duration)

            # Replace with your actual VAD inference
            # speech_prob = self.model.predict(audio_signal[...])
            speech_prob = 0.8  # Placeholder

            vad_output.append([speech_prob, start_time, end_time])

        return np.array(vad_output)

# Use custom VAD with WhisperS2T
custom_vad = CustomVAD(threshold=0.4)
model = whisper_s2t.load_model(
    model_identifier="large-v2",
    backend='CTranslate2',
    vad_model=custom_vad
)

files = ['audio.wav']
out = model.transcribe_with_vad(files, lang_codes=['en'], tasks=['transcribe'], batch_size=16)
```

--------------------------------

### Load Whisper Model with TensorRT-LLM Backend

Source: https://github.com/shashikg/whispers2t/blob/main/README.md

Load the WhisperS2T model specifying the 'TensorRT-LLM' backend. Note that initial inference runs may be slower due to JIT tracing of the VAD model, but speed improves after a few runs.

```python
import whisper_s2t

model = whisper_s2t.load_model(model_identifier="large-v2", backend='TensorRT-LLM')
```

--------------------------------

### Load autotime extension

Source: https://github.com/shashikg/whispers2t/blob/main/notebooks/WhisperS2T_CTranslate2.ipynb

Loads the autotime extension in IPython, enabling automatic timing of code cells. This is useful for performance analysis.

```python
%load_ext autotime
```

--------------------------------

### Transcribe Audio with CTranslate2 Backend and VAD

Source: https://github.com/shashikg/whispers2t/blob/main/README.md

Load a Whisper model using the CTranslate2 backend and transcribe audio files with Voice Activity Detection (VAD). Specify language codes, tasks, and initial prompts. The output contains detailed information per utterance.

```python
import whisper_s2t

model = whisper_s2t.load_model(model_identifier="large-v2", backend='CTranslate2')

files = ['data/KINCAID46/audio/1.wav']
lang_codes = ['en']
tasks = ['transcribe']
initial_prompts = [None]

out = model.transcribe_with_vad(files,
                                lang_codes=lang_codes,
                                tasks=tasks,
                                initial_prompts=initial_prompts,
                                batch_size=32)

print(out[0][0]) # Print first utterance for first file

```

--------------------------------

### Transcribe Audio with Voice Activity Detection (VAD)

Source: https://context7.com/shashikg/whispers2t/llms.txt

Performs speech-to-text transcription using the `transcribe_with_vad` method, which leverages built-in VAD to segment audio efficiently. This method is recommended for variable-length audio files and supports specifying language codes, tasks (transcribe/translate), and initial prompts.

```python
import whisper_s2t

model = whisper_s2t.load_model(model_identifier="large-v2", backend='CTranslate2')

# Prepare input files and parameters
files = ['audio/meeting.wav', 'audio/podcast.mp3', 'audio/interview.m4a']
lang_codes = ['en', 'en', 'es']  # Language codes for each file
tasks = ['transcribe', 'transcribe', 'translate']  # transcribe or translate
initial_prompts = [None, "Technical discussion about AI", None]  # Optional prompts

# Transcribe with VAD-based segmentation
out = model.transcribe_with_vad(
    files,
    lang_codes=lang_codes,
    tasks=tasks,
    initial_prompts=initial_prompts,
    batch_size=32  # Adjust based on GPU memory
)

# Process results for each file
for file_idx, file_transcripts in enumerate(out):
    print(f"\n=== File: {files[file_idx]} ===")
    for utterance in file_transcripts:
        print(f"[{utterance['start_time']:.2f}s - {utterance['end_time']:.2f}s]: {utterance['text']}")
        # Additional fields available: avg_logprob, no_speech_prob

# Example output:
# === File: audio/meeting.wav ===
# [0.00s - 24.80s]: Let's bring in Phil Mackie who is there at the palace...
# [25.00s - 48.50s]: The ceremony is about to begin...
```

--------------------------------

### Transcribe Audio with VAD using WhisperS2T

Source: https://github.com/shashikg/whispers2t/blob/main/notebooks/WhisperS2T_TensorRT_LLM.ipynb

Perform speech-to-text transcription on audio files using the loaded model with Voice Activity Detection (VAD). Note that initial runs may be slower due to JIT tracing and engine warmups.

```python
# Note: For first run the model may give slightly slower inference speed. After 1-2 runs it gives better inference speed.
# This is due to the JIT tracing of the VAD model and TensorRT Engine -- they need some warmups before running actual inference.

files = ['sam_altman_lex_podcast_367.flac']
lang_codes = ['en']
tasks = ['transcribe']
initial_prompts = [None]

out = model.transcribe_with_vad(files,
                                lang_codes=lang_codes,
                                tasks=tasks,
                                initial_prompts=initial_prompts,
                                batch_size=24)
```

--------------------------------

### Configure Custom Model Parameters

Source: https://context7.com/shashikg/whispers2t/llms.txt

Adjust ASR options such as beam search, compute type, and repetition penalties. Parameters can be updated during model loading or via update_params and update_generation_kwargs.

```python
import whisper_s2t
from whisper_s2t.backends.ctranslate2.model import BEST_ASR_CONFIG, FAST_ASR_OPTIONS

# Use BEST_ASR_CONFIG for highest accuracy (slower)
model_kwargs = {
    'compute_type': 'int8',  # Options: float16, int8 (CTranslate2 only)
    'asr_options': BEST_ASR_CONFIG  # beam_size=5, patience=2
}

model = whisper_s2t.load_model(
    model_identifier="large-v2",
    backend='CTranslate2',
    **model_kwargs
)

# Or use FAST_ASR_OPTIONS for speed (default)
# FAST_ASR_OPTIONS: beam_size=1, patience=1

# Custom ASR options
custom_options = {
    'beam_size': 3,
    'patience': 1.5,
    'length_penalty': 1.0,
    'repetition_penalty': 1.05,
    'no_repeat_ngram_size': 3,
    'suppress_blank': True,
    'without_timestamps': True,
    'sampling_temperature': 0.8,
    'word_timestamps': False,
}

model_custom = whisper_s2t.load_model(
    model_identifier="large-v2",
    backend='CTranslate2',
    compute_type='float16',
    asr_options=custom_options
)

# Update parameters after model loading
model.update_params({
    'max_speech_len': 25.0,  # Maximum speech segment length
    'merge_chunks': True,     # Merge adjacent speech chunks
})

# Update generation kwargs
model.update_generation_kwargs({
    'beam_size': 5,
    'max_length': 256
})
```

--------------------------------

### Load Model for Word Timestamps

Source: https://github.com/shashikg/whispers2t/blob/main/README.md

Load the Whisper model with specific ASR options to enable word-level timestamps during transcription. This is useful for detailed alignment of speech to text.

```python
model = whisper_s2t.load_model("large-v2", asr_options={'word_timestamps': True})
```

--------------------------------

### Export Transcripts to Files

Source: https://context7.com/shashikg/whispers2t/llms.txt

Use the write_outputs function to save transcription results into formats like VTT, SRT, JSON, TSV, or TXT.

```python
import whisper_s2t

model = whisper_s2t.load_model(model_identifier="large-v2", backend='CTranslate2')

files = ['video.mp4', 'podcast.mp3']
lang_codes = ['en', 'en']
tasks = ['transcribe', 'transcribe']
initial_prompts = [None, None]

out = model.transcribe_with_vad(
    files,
    lang_codes=lang_codes,
    tasks=tasks,
    initial_prompts=initial_prompts,
    batch_size=24
)

# Export to VTT format (WebVTT subtitles)
whisper_s2t.write_outputs(out, format='vtt', ip_files=files, save_dir="./subtitles")
# Creates: ./subtitles/0_video.vtt, ./subtitles/1_podcast.vtt

# Export to SRT format (SubRip subtitles)
whisper_s2t.write_outputs(out, format='srt', ip_files=files, save_dir="./subtitles")

# Export to JSON format (full data with metadata)
whisper_s2t.write_outputs(out, format='json', ip_files=files, save_dir="./output")

# Export to TSV format (tab-separated values)
whisper_s2t.write_outputs(out, format='tsv', ip_files=files, save_dir="./output")

# Export to TXT format (readable text with timestamps)
whisper_s2t.write_outputs(out, format='txt', ip_files=files, save_dir="./output")

# Custom output file names
custom_files = ['meeting_transcript.vtt', 'podcast_transcript.vtt']
whisper_s2t.write_outputs(out, format='vtt', op_files=custom_files)

# Export with single sentence per utterance (requires word_timestamps)
whisper_s2t.write_outputs(
    out,
    format='srt',
    ip_files=files,
    save_dir="./subtitles",
    single_sentence_in_one_utterance=True,
    end_punct_marks=["?", "."]
)
```