### Install Dependencies

Source: https://github.com/nvidia/tensorrt/blob/main/tools/Polygraphy/examples/api/07_tensorrt_and_dynamic_shapes/README.md

Install the necessary dependencies for the example. Ensure TensorRT is installed separately.

```bash
python3 -m pip install -r requirements.txt
```

--------------------------------

### Setup and Asset Folder Creation

Source: https://github.com/nvidia/tensorrt/blob/main/tools/tensorflow-quantization/docs/source/notebooks/simple_network_quantize_full.md

Initializes TensorFlow, imports necessary libraries, and creates directories for saving model artifacts. Ensure TensorFlow and TensorFlow Quantization are installed.

```python
#
# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# [http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

import tensorflow as tfn
from tensorflow_quantization import quantize_model
import tiny_resnet
from tensorflow_quantization import utilsn
import os

tf.keras.backend.clear_session()

# Create folders to save TF and ONNX models
assets = utils.CreateAssetsFolders(os.path.join(os.getcwd(), "tutorials"))
assets.add_folder("simple_network_quantize_full")
```

--------------------------------

### Setup and Asset Folder Creation

Source: https://github.com/nvidia/tensorrt/blob/main/tools/tensorflow-quantization/docs/source/notebooks/simple_network_quantize_full.ipynb

Initializes TensorFlow and creates necessary asset folders for saving models. Ensure TensorFlow is installed.

```python
#
# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

import tensorflow as tf
from tensorflow_quantization import quantize_model
import tiny_resnet
from tensorflow_quantization import utils
import os

tf.keras.backend.clear_session()

# Create folders to save TF and ONNX models
assets = utils.CreateAssetsFolders(os.path.join(os.getcwd(), "tutorials"))
assets.add_folder("simple_network_quantize_full")
```

--------------------------------

### Setup Benchmark UI Widgets

Source: https://github.com/nvidia/tensorrt/blob/main/demo/BERT/notebooks/benchmark.ipynb

Initializes radio buttons for engine and batch size selection, an integer slider for iterations, a button to start the benchmark, and an output area. This setup is essential for user interaction.

```python
engine_selector = widgets.RadioButtons(
    options=['GPU - TensorRT FP16',
             'GPU - TensorRT INT8'],
    description='Engine:',
    disabled=False
)

batchsize_selector = widgets.RadioButtons(
    options=['1', '32', '64', '128'],
    description='Batch size:',
    disabled=False
)

iteration_selector = widgets.IntSlider(
    value=500,
    min=100,
    max=1000,
    step=1,
    description='Iterations:',
    disabled=False,
    continuous_update=False,
    orientation='horizontal',
    readout=True,
    readout_format='d'
)

button = widgets.Button(description="Run benchmark")
output = widgets.Output()
#box = widgets.HBox(children=[button],layout=box_layout)
```

--------------------------------

### Install TensorFlow Quantization Toolkit

Source: https://github.com/nvidia/tensorrt/blob/main/tools/experimental/trt-engine-explorer/examples/tensorflow/resnet/README.md

Clone the TensorRT repository and install the TensorFlow Quantization Toolkit using the provided script. This is a prerequisite for running the example.

```bash
git clone https://github.com/NVIDIA/TensorRT.git
cd TensorRT/tools/tensorflow-quantization
./install.sh
```

--------------------------------

### Install PyTorch and TorchVision

Source: https://github.com/nvidia/tensorrt/blob/main/tools/experimental/trt-engine-explorer/examples/pytorch/resnet/README.md

Install PyTorch and TorchVision using pip. These are prerequisites for the example.

```bash
python3 -m pip install torch torchvision
```

--------------------------------

### Install TensorFlow Quantization Locally

Source: https://github.com/nvidia/tensorrt/blob/main/tools/tensorflow-quantization/docs/source/docs/installation.md

Clone the repository locally, navigate to the tools directory, run the installation script, and execute the tests to verify the installation.

```bash
cd ~/
git clone https://github.com/NVIDIA/TensorRT.git
cd TensorRT/tools/tensorflow-quantization
./install.sh
cd tests
python3 -m pytest quantize_test.py -rP
```

--------------------------------

### Install Sample Executable

Source: https://github.com/nvidia/tensorrt/blob/main/samples/sampleCharRNN/CMakeLists.txt

Installs the sample_char_rnn executable as an optional component.

```cmake
installLibraries(
    TARGETS sample_char_rnn
    OPTIONAL
    COMPONENT internal
)
```

--------------------------------

### Install TensorFlow Quantization Toolkit Locally

Source: https://github.com/nvidia/tensorrt/blob/main/tools/tensorflow-quantization/README.md

Clones the TensorRT repository, navigates to the quantization tools directory, and executes the installation script. Runs tests to verify the installation.

```bash
$ cd ~/
$ git clone https://github.com/NVIDIA/TensorRT.git
$ cd TensorRT/tools/tensorflow-quantization
$ ./install.sh
$ cd tests
$ python3 -m pytest quantize_test.py -rP
```

--------------------------------

### Run Example Script

Source: https://github.com/nvidia/tensorrt/blob/main/tools/Polygraphy/examples/api/01_comparing_frameworks/README.md

Execute the main example script to perform model inference and comparison.

```bash
python3 example.py
```

--------------------------------

### Install Sample Targets

Source: https://github.com/nvidia/tensorrt/blob/main/samples/CMakeSamplesTemplate.txt

Installs the built sample targets to specified runtime, library, and archive destinations. This is part of the CMake installation process.

```cmake
install(TARGETS ${TARGET_NAME}
        RUNTIME DESTINATION bin
        LIBRARY DESTINATION lib
        ARCHIVE DESTINATION lib
)
```

--------------------------------

### Install ONNX GraphSurgeon using Prebuilt Wheels

Source: https://github.com/nvidia/tensorrt/blob/main/tools/onnx-graphsurgeon/README.md

Install ONNX GraphSurgeon using pip with the NVIDIA PyPI index. This is the recommended installation method.

```bash
python3 -m pip install onnx_graphsurgeon --extra-index-url https://pypi.ngc.nvidia.com
```

--------------------------------

### Install Sample Executable

Source: https://github.com/nvidia/tensorrt/blob/main/samples/sampleIOFormats/CMakeLists.txt

Installs the sample_io_formats executable as an optional component. This is part of the build configuration for the sample.

```cmake
installLibraries(
    TARGETS sample_io_formats
    OPTIONAL
    COMPONENT internal
)
```

--------------------------------

### Install Dependencies with Options

Source: https://github.com/nvidia/tensorrt/blob/main/demo/Diffusion/README.md

Install dependencies with additional options such as forcing a reinstall or specifying a custom installation path for dependencies.

```bash
# Force reinstall even if already installed
python3 setup.py all --force
```

```bash
# Install dependencies to a custom location
# Option 1 (recommended): set the env var and install
export TENSORRT_DIFFUSION_DEPS_ROOT=/custom/path/deps
python3 setup.py all
```

```bash
# Option 2: install to a path without changing this shell
# Remember to export the env var in the environment that runs the demos,
# so deps.configure() can find the custom path.
python3 setup.py all --deps-root /custom/path/deps
# Then, before running any demo scripts:
export TENSORRT_DIFFUSION_DEPS_ROOT=/custom/path/deps
```

--------------------------------

### Build and Install Extension Module

Source: https://github.com/nvidia/tensorrt/blob/main/tools/Polygraphy/examples/dev/02_extending_polygraphy_run/README.md

Builds a Python wheel for the extension module and installs it. Use `--force-reinstall --no-deps` to update an existing installation.

```bash
python3 extension_module/setup.py bdist_wheel
```

```bash
python3 -m pip install extension_module/dist/polygraphy_reshape_destroyer-0.0.1-py3-none-any.whl \
    --extra-index-url https://pypi.ngc.nvidia.com
```

```bash
python3 -m pip install extension_module/dist/polygraphy_reshape_destroyer-0.0.1-py3-none-any.whl \
    --force-reinstall --no-deps
```

--------------------------------

### Install Sphinx and Extensions

Source: https://github.com/nvidia/tensorrt/blob/main/tools/tensorflow-quantization/docs/README.md

Execute this bash script to install Sphinx and all necessary extensions for building the documentation.

```bash
$ ./setup_sphinx.sh
```

--------------------------------

### Install Sample Library (CMake)

Source: https://github.com/nvidia/tensorrt/blob/main/samples/sampleOnnxMnistCoordConvAC/CMakeLists.txt

Installs the compiled sample library for internal components when the new samples flow is enabled.

```cmake
installLibraries(
    TARGETS sample_onnx_mnist_coord_conv_ac
    OPTIONAL
    COMPONENT internal
)
```

--------------------------------

### Install Python Dependencies

Source: https://github.com/nvidia/tensorrt/blob/main/tools/Polygraphy/polygraphy/README.md

Install the required Python packages for building documentation. This should be run before building the docs.

```bash
python -m pip install -r docs/requirements.txt
```

--------------------------------

### Install Notebook and Navigate to Sample

Source: https://github.com/nvidia/tensorrt/blob/main/samples/python/refactored/README.md

Install the notebook package and navigate to a specific sample directory to launch the Jupyter Notebook server.

```bash
pip install notebook
cd 1_run_onnx_with_tensorrt # or any other sample
jupyter notebook
```

--------------------------------

### Sample Help Options

Source: https://github.com/nvidia/tensorrt/blob/main/samples/sampleCudla/README.md

Display all available command-line options and their descriptions for the sample. Use this to understand customization possibilities.

```bash
./sample_cudla -h
```

--------------------------------

### Sample Help Options

Source: https://github.com/nvidia/tensorrt/blob/main/samples/python/simple_progress_monitor/README.md

View the available command-line options and their descriptions for the simple_progress_monitor script.

```bash
usage: simple_progress_monitor.py [-h] [-d DATADIR]

Runs a ResNet50 network with a TensorRT inference engine. Displays intermediate build progress.

optional arguments:
 -h, --help            show this help message and exit
 -d DATADIR, --datadir DATADIR
                       Location of the TensorRT sample data directory.
                       (default: /usr/src/tensorrt/data)
```

--------------------------------

### Install PyTorch with CUDA Support

Source: https://github.com/nvidia/tensorrt/blob/main/quickstart/IntroNotebooks/0. Running This Guide.ipynb

Install PyTorch, torchvision, and torchaudio, ensuring compatibility with your CUDA version (cu124 in this example).

```python
%pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124
```

--------------------------------

### Display Sample Help Options

Source: https://github.com/nvidia/tensorrt/blob/main/samples/python/strongly_type_autocast/README.md

Use the --help or -h flag to view all available command-line options for the sample. This is useful for understanding configuration and customization.

```bash
python3 sample.py -h
```

--------------------------------

### Install Required Python Packages

Source: https://github.com/nvidia/tensorrt/blob/main/quickstart/IntroNotebooks/0. Running This Guide.ipynb

Install essential Python libraries for the examples, including cuda-python, onnx, scikit-image, and matplotlib.

```python
%pip install cuda-python onnx scikit-image matplotlib
```

--------------------------------

### Install ONNX Runtime

Source: https://github.com/nvidia/tensorrt/blob/main/tools/onnx-graphsurgeon/examples/05_folding_constants/README.md

Install ONNX Runtime, which is used by ONNX GraphSurgeon to evaluate constant expressions. This is a prerequisite for running the example.

```bash
python3 -m pip install onnxruntime
```

--------------------------------

### Sample Help Options

Source: https://github.com/nvidia/tensorrt/blob/main/samples/sampleNonZeroPlugin/README.md

Display the full list of available command-line options and their descriptions for the sample.

```bash
./sample_non_zero_plugin -h
```

```bash
./sample_non_zero_plugin --help
```

--------------------------------

### Display sample --help options

Source: https://github.com/nvidia/tensorrt/blob/main/samples/sampleCharRNN/README.md

Use the -h or --help command line option to view all available options and their descriptions for the sample.

```bash
-h or --help
```

--------------------------------

### Sample Help Options

Source: https://github.com/nvidia/tensorrt/blob/main/samples/python/sample_weight_stripping/README.md

View all available command-line options and their descriptions for the sample by using the -h or --help flag.

```bash
python3 your_script_name.py --help
```

--------------------------------

### Display Sample Help Options

Source: https://github.com/nvidia/tensorrt/blob/main/samples/sampleOnnxMNIST/README.md

To view all available command-line options and their descriptions for the sample, use the -h or --help flag.

```bash
./sample_onnx_mnist -h
```

--------------------------------

### Setup Python Virtual Environment for trex

Source: https://github.com/nvidia/tensorrt/blob/main/tools/experimental/trt-engine-explorer/README.md

Create and activate a Python virtual environment for installing and using the trex package. Python 3.8+ is required.

```bash
cd TensorRT/tools/experimental/trt-engine-explorer
python3 -m virtualenv env_trex
source env_trex/bin/activate
```

--------------------------------

### Quantized Fine-tuning Setup and Training

Source: https://github.com/nvidia/tensorrt/blob/main/tools/pytorch-quantization/examples/finetune_quant_resnet50.ipynb

Sets up the loss function, optimizer, and learning rate scheduler for fine-tuning. It then prepares the data loader and initiates the training process for one epoch.

```python
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.SGD(model.parameters(), lr=0.0001)
lr_scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=1, gamma=0.1)

data_loader = torch.utils.data.DataLoader(
    dataset, batch_size=128,
    sampler=train_sampler, num_workers=16, pin_memory=True)

# Training takes about one and half hour per epoch on single V100
train_one_epoch(model, criterion, optimizer, data_loader, "cuda", 0, 100)
```

--------------------------------

### Sample Help Options

Source: https://github.com/nvidia/tensorrt/blob/main/samples/python/introductory_parser_samples/README.md

View the available command-line options and their descriptions for the onnx_resnet50.py script using the -h or --help flag.

```bash
usage: onnx_resnet50.py [-h] [-d DATADIR]

Runs a ResNet50 network with a TensorRT inference engine.

optional arguments:
  -h, --help            show this help message and exit
  -d DATADIR, --datadir DATADIR
                        Location of the TensorRT sample data directory.
                        (default: /usr/src/tensorrt/data)
```

--------------------------------

### Build Documentation

Source: https://github.com/nvidia/tensorrt/blob/main/tools/tensorflow-quantization/docs/README.md

Run this make command to clean previous builds and generate the HTML documentation.

```bash
$ make clean html
```

--------------------------------

### Create and Save ONNX Model

Source: https://github.com/nvidia/tensorrt/blob/main/tools/onnx-graphsurgeon/examples/01_creating_a_model/README.md

This Python script utilizes ONNX GraphSurgeon to build a graph containing a GlobalLpPool node and then saves it to an ONNX file. Ensure ONNX GraphSurgeon is installed to run this example.

```python
import onnx_graphsurgeon as gs
import numpy as np

# Define input tensor
input_tensor = gs.Variable(name="input", dtype=np.float32, shape=[1, 3, 224, 224])

# Create a GlobalLpPool node
# The 'lp_norm' parameter is set to 2 for Lp norm, and 'axis' is set to 0 to pool over the batch dimension.
# The output shape is inferred automatically.
global_lp_pool = gs.Node(
    op="GlobalLpPool",
    inputs=[input_tensor],
    outputs=["output"],
    attrs={"lp_norm": 2, "axis": 0}
)

# Build the graph
graph = gs.Graph(nodes=[global_lp_pool], inputs=[input_tensor], outputs=[global_lp_pool.outputs[0]])

# Build the ONNX model
model = gs.export_onnx(graph)

# Save the model to a file
with open("test_globallppool.onnx", "wb") as f:
    f.write(model.SerializeToString())
```

--------------------------------

### Run Inference with ONNXClassifierWrapper

Source: https://github.com/nvidia/tensorrt/blob/main/quickstart/IntroNotebooks/1. Introduction.ipynb

Perform inference on a batch of dummy input data using the initialized ONNXClassifierWrapper. This example shows how to get softmax probability predictions for the first 10 classes of the first sample.

```python
# Warm up:
trt_model.predict(dummy_input_batch)[:10] # softmax probability predictions for the first 10 classes of the first sample
```

--------------------------------

### TensorRT OSS Environment Setup

Source: https://github.com/nvidia/tensorrt/blob/main/demo/DeBERTa/README.md

Builds and launches a Docker container for TensorRT OSS development, then builds the OSS and installs Python dependencies for the DeBERTa demo. Ensure PyTorch 1.11.0 is used for proper ONNX export.

```bash
# Download this TensorRT OSS repo
git clone -b main https://github.com/nvidia/TensorRT TensorRT
cd TensorRT
git submodule update --init --recursive

## at root of TensorRT OSS
# build container
./docker/build.sh --file docker/ubuntu-20.04.Dockerfile --tag tensorrt-ubuntu20.04-cuda12.8

# launch container
./docker/launch.sh --tag tensorrt-ubuntu20.04-cuda12.8 --gpus all

## now inside container
# build OSS (only required for pre-8.4.3 TensorRT versions)
cd $TRT_OSSPATH
mkdir -p build && cd build
cmake .. -DTRT_LIB_DIR=$TRT_LIBPATH -DTRT_OUT_DIR=`pwd`/out
make -j$(nproc)

# polygraphy bin location & trtexec bin location
export PATH="~/.local/bin":"${TRT_OSSPATH}/build/out":$PATH 

# navigate to the demo folder install additional dependencies (note PyTorch 1.11.0 is recommended for onnx to export properly)
cd $TRT_OSSPATH/demo/DeBERTa
pip install -r requirements.txt
```

--------------------------------

### Sample Help Options

Source: https://github.com/nvidia/tensorrt/blob/main/samples/sampleOnnxMnistCoordConvAC/README.md

View available command-line options for the sample by using the -h or --help flag. This displays usage information and descriptions for arguments like --datadir and --useDLACore.

```bash
Usage: ./sample_onnx_mnist_coord_conv_ac [-h or --help] [-d or --datadir=<path to data directory>] [--useDLACore=<int>]
--help Display help information
--datadir Specify path to a data directory, overriding the default. This option can be used multiple times to add multiple directories. If no data directories are given, the default is to use (data/samples/mnist/, data/mnist/)
--useDLACore=N Specify a DLA engine for layers that support DLA. Value can range from 0 to n-1, where n is the number of DLA engines on the platform.
```

--------------------------------

### Sample Progress Monitor Help Options

Source: https://github.com/nvidia/tensorrt/blob/main/samples/sampleProgressMonitor/README.md

Display available command-line options for the sample_progress_monitor, including help, data directory specification, and DLA core selection.

```text
Usage: ./sample_progress_monitor [-h or --help] [-d or --datadir=<path to data directory>] [--useDLACore=<int>]
--help Display help information
--datadir Specify path to a data directory, overriding the default. This option can be used multiple times to add multiple directories. If no data directories are given, the default is to use (data/samples/mnist/, data/mnist/)
--useDLACore=N Specify a DLA engine for layers that support DLA. Value can range from 0 to n-1, where n is the number of DLA engines on the platform.
```

--------------------------------

### Install Transformers and PyTorch

Source: https://github.com/nvidia/tensorrt/blob/main/demo/BERT/notebooks/BERT-TRT-FP16.ipynb

Clones the transformers repository, installs it, and installs a specific version of PyTorch with CUDA support.

```python
!cd /tmp && git clone https://github.com/vinhngx/transformers && cd transformers && pip install .
!pip install torch==1.8.1+cu111 torchvision==0.9.1+cu111 torchaudio===0.8.1 -f https://download.pytorch.org/whl/torch_stable.html
```

--------------------------------

### Install Polygraphy using Make (Linux)

Source: https://github.com/nvidia/tensorrt/blob/main/tools/Polygraphy/README.md

Installs Polygraphy from source using the make install target on Linux systems.

```bash
make install
```

--------------------------------

### Check Installation Status

Source: https://github.com/nvidia/tensorrt/blob/main/demo/Diffusion/README.md

Verify the installation status of the demoDiffusion dependencies. This command checks if all required packages are correctly installed.

```bash
python3 -c "from demo_diffusion import deps; deps.print_status()"
```

--------------------------------

### Run Example Tool

Source: https://github.com/nvidia/tensorrt/blob/main/tools/Polygraphy/examples/dev/01_writing_cli_tools/README.md

Execute the custom 'gen-data' tool to generate data and specify the output file and number of values.

```bash
./gen-data -o data.json --num-values 25
```

--------------------------------

### Install Dependencies with pip

Source: https://github.com/nvidia/tensorrt/blob/main/samples/python/detectron2/README.md

Installs all project dependencies listed in the requirements.txt file. Ensure TensorRT is installed prior to running this command.

```bash
pip install -r requirements.txt
```

--------------------------------

### Setup and Asset Folder Creation

Source: https://github.com/nvidia/tensorrt/blob/main/tools/tensorflow-quantization/docs/source/notebooks/simple_network_quantize_specific_class.ipynb

Initializes TensorFlow and creates necessary folders for saving models and assets. Imports required libraries for quantization and model utilities.

```python
#
# SPDX-FileCopyrightText: Copyright (c) 1993-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

import tensorflow as tf
from tensorflow_quantization import quantize_model, QuantizationSpec
import tiny_resnet
from tensorflow_quantization import utils
import os

tf.keras.backend.clear_session()

# Create folders to save TF and ONNX models
assets = utils.CreateAssetsFolders(os.path.join(os.getcwd(), "tutorials"))
assets.add_folder("simple_network_quantize_specific_class")
```

--------------------------------

### Install Dependencies

Source: https://github.com/nvidia/tensorrt/blob/main/samples/python/dds_faster_rcnn/README.md

Install all project dependencies listed in the requirements.txt file. Ensure TensorRT and its Python bindings are correctly installed beforehand.

```bash
pip3 install -r requirements.txt
```

--------------------------------

### Initialize Tokenizer and Parameters

Source: https://github.com/nvidia/tensorrt/blob/main/demo/BERT/notebooks/Q-and-A.ipynb

Sets up the tokenizer and defines key parameters for the Q&A task, including document stride, maximum query length, and the path to the vocabulary file.

```python
doc_stride = 128
max_query_length = 64

vocab_file = "models/fine-tuned/bert_tf_ckpt_large_qa_squad2_amp_384_v19.03.1/vocab.txt"
tokenizer = tokenization.FullTokenizer(vocab_file=vocab_file, do_lower_case=True)
```

--------------------------------

### Install and Test TensorFlow Quantization inside Docker

Source: https://github.com/nvidia/tensorrt/blob/main/tools/tensorflow-quantization/docs/source/docs/installation.md

After entering the Docker container, navigate to the mounted repository, run the installation script, and execute the tests to verify the installation.

```bash
cd /home/tensorflow-quantization
./install.sh
cd tests
python3 -m pytest quantize_test.py -rP
```

--------------------------------

### Sample Plugin Safe Build Help Options

Source: https://github.com/nvidia/tensorrt/blob/main/samples/sampleSafePluginV3/README.md

Use this command to view all available options for building safe plugins. Supports long flags and limited short flags.

```bash
Usage: ./sample_plugin_safe_build [-h or --help] [-d or --datadir=<path to data directory>]
--help or -h    Display help information
--datadir or -d Specify path to a data directory, overriding the default. This option can be used multiple times to add multiple directories. If no data directories are given, the default is to use (data/samples/safe_plugin/, data/safe_plugin/)
--saveEngine    Save the serialized engine to the file, the default is to use safe_plugin.engine.
--remoteAutoTuningConfig  Set the remote auto tuning config. Format: protocol://username[:password]@hostname[:port]?param1=value1&param2=value2
                    Example: ssh://user:pass@192.0.2.100:22?remote_exec_path=/opt/tensorrt/bin&remote_lib_path=/opt/tensorrt/lib
```

--------------------------------

### Install ONNX GraphSurgeon

Source: https://github.com/nvidia/tensorrt/blob/main/samples/python/tensorflow_object_detection_api/README.md

Install the specific version of the onnx-graphsurgeon module required by the sample. Use the provided NGC index URL if the module is not already installed by TensorRT.

```bash
pip install onnx-graphsurgeon==0.3.10 --index-url https://pypi.ngc.nvidia.com
```

--------------------------------

### Check TensorRT Installation

Source: https://github.com/nvidia/tensorrt/blob/main/quickstart/IntroNotebooks/2. Using PyTorch through ONNX.ipynb

Verifies that the TensorRT library is installed and accessible.

```python
import tensorrt
```

--------------------------------

### Setup Notebook Environment

Source: https://github.com/nvidia/tensorrt/blob/main/tools/experimental/trt-engine-explorer/notebooks/compare_engines.ipynb

Imports necessary libraries and configures the display for wide graphs in the Jupyter notebook.

```python
import IPython
from ipywidgets import widgets
from trex import *
from trex.notebook import *
from trex.report_card import *
from trex.compare_engines import *

# Configure a wider output (for the wide graphs)
set_wide_display()
```

--------------------------------

### Install onnx-graphsurgeon

Source: https://github.com/nvidia/tensorrt/blob/main/samples/python/efficientdet/README.md

Manually install or update the onnx-graphsurgeon Python module if it's not already installed or if a version less than 0.3.9 is detected. This module is crucial for ONNX graph manipulation.

```bash
pip3 install onnx-graphsurgeon --index-url https://pypi.ngc.nvidia.com
```

--------------------------------

### Configure Quantization

Source: https://github.com/nvidia/tensorrt/blob/main/tools/tensorflow-quantization/docs/source/notebooks/simple_network_quantize_partial.md

Set up quantization configuration. This example enables INT8 quantization and specifies a calibration dataset.

```python
config.set_flag(trt.BuilderFlag.INT8)
# For INT8 calibration, you need to provide a calibration dataset.
# This is a placeholder for the actual calibration process.
# In a real scenario, you would use a representative dataset to calibrate the scales.
class MyCalibrator(trt.IInt8EntropyCalibrator2):
    def get_batch_size(self):
        return 1
    def get_batch(self, names):
        # Replace with your actual calibration data loading logic
        dummy_data = np.random.rand(1, 1, 28, 28).astype(np.float32)
        return [dummy_data]
    def read_calibration_cache(self):
        return None # No cache to read for this example
    def write_calibration_cache(self, cache):
        pass # No cache to write for this example

calibrator = MyCalibrator()
config.int8_calibrator = calibrator
```

--------------------------------

### Configure Quantization

Source: https://github.com/nvidia/tensorrt/blob/main/tools/tensorflow-quantization/docs/source/notebooks/simple_network_quantize_partial.md

Set up the quantization configuration. This example enables INT8 quantization and specifies a calibration dataset.

```python
config.set_flag(trt.BuilderFlag.INT8)
config.int8_calibrator = MyCalibrator(training_data, batch_size=32, cache_file='int8_calibration.cache')
```

--------------------------------

### Install Demo Dependencies

Source: https://github.com/nvidia/tensorrt/blob/main/demo/Diffusion/README.md

Install the required Python packages for the demoDiffusion application. Use 'all' to install for all model families or specify individual families like 'sd', 'flux', or 'cosmos'.

```bash
python3 setup.py all
```

```bash
# SD family: SD 1.4, SDXL, SD3, SD3.5, SVD (Stable Video Diffusion), Stable Cascade
python3 setup.py sd
```

```bash
# Flux family: Flux.1-dev, Flux.1-schnell, Flux.1-Canny, Flux.1-Depth, Flux.1-Kontext
python3 setup.py flux
```

```bash
# Cosmos family: Cosmos-Predict2 text2image, video2world
python3 setup.py cosmos
```

--------------------------------

### Install Graphviz

Source: https://github.com/nvidia/tensorrt/blob/main/tools/experimental/trt-engine-explorer/utils/README.md

Command to install Graphviz, a dependency for drawing engine graphs.

```bash
$ sudo apt-get --yes install graphviz
```

--------------------------------

### Run Pre-built Engine on QNX-safe Platform

Source: https://github.com/nvidia/tensorrt/blob/main/samples/sampleSafePluginV3/README.md

Execute this command to load and run a pre-built engine on a QNX-safe platform. This requires the runtime-only safety plugin library.

```bash
trtexec_safe --loadEngine=sample.engine --safetyPlugins=libsample_safe_plugin_v3_safe.so[::MaxPoolPlugin]
```

--------------------------------

### Install Graphviz

Source: https://github.com/nvidia/tensorrt/blob/main/tools/experimental/trt-engine-explorer/README.md

Installs Graphviz, a dependency required for generating dot and SVG graphs.

```bash
$ sudo apt --yes install graphviz
```

--------------------------------

### Running the qdp_runner.py Sample

Source: https://github.com/nvidia/tensorrt/blob/main/samples/python/quickly_deployable_plugins/README.md

Use this command to run the sample. The `--save_engine` flag saves the engine, and `--load_engine` loads a previously saved engine. The `--mode` flag specifies whether to use ONNX or INetworkDefinition APIs. `--artifacts_dir` is used for storing or retrieving engine files. `-v` enables verbose logging.

```bash
python3 qdp_runner.py circ_pad [--save_engine] [--load_engine] --mode {onnx,inetdef} [--artifacts_dir ARTIFACTS_DIR]  [-v]

options:
  --save_engine         Save engine to the artifacts_dir.
  --load_engine         Load engine from the artifacts_dir. Ignores all other options.
  --artifacts_dir ARTIFACTS_DIR
                        Whether to store (or retrieve) artifacts.
  --mode {onnx,inetdef} Whether to use ONNX parser or INetworkDefinition APIs to construct the network.
  --aot                 Use the AOT implementation of the plugin.
  -v, --verbose         Enable verbose log output.
```

--------------------------------

### start

Source: https://github.com/nvidia/tensorrt/blob/main/tools/tensorflow-quantization/docs/source/logger/toc.md

Logs a message to stdout with START severity. This indicates the beginning of a significant operation.

```APIDOC
## start(message, mode=LogMode.EACH)

Logs a message to stdout with START severity.

### Parameters:
* **message** (Union[str, Callable[[], str]]) – A string or callable which returns a string of the message to log.
* **mode** (LogMode) – Controls how the message is logged. See LogMode for details.
```

--------------------------------

### Install trtexec Extension Wheel

Source: https://github.com/nvidia/tensorrt/blob/main/tools/polygraphy-extension-trtexec/README.md

Installs the built trtexec extension wheel and its dependencies from NGC.

```bash
python3 -m pip install dist/polygraphy_trtexec-*.whl \
        --extra-index-url https://pypi.ngc.nvidia.com
```