NVIDIA TensorRT for RTX

https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/
Admin
NVIDIA TensorRT for RTX is an inference library that simplifies the deployment of AI models on...
Tokens:493,796
Snippets:4,086
Trust Score:10
Update:2 months ago
Show doc for...
Tokens:
Raw
### Install Python bindings on Windows

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/installing-tensorrt-rtx/installing.html

Use PowerShell to install the TensorRT-RTX wheel file from the extracted SDK directory.

```powershell
$version = "1.5.0.0" # Replace with newest version
$arch = "amd64" # Replace with your architecture
$pyversion = "311" # For Python 3.11. Replace with your
                   # Python version for other versions.
$wheel = Join-Path "TensorRT-RTX-$version" "tensorrt_rtx-$version-cp$pyversion-none-win_$arch.whl"
python3 -m pip install $wheel
```

--------------------------------

### Install via PyPI

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/installing-tensorrt-rtx/installing.html

Install the tensorrt-rtx package directly from the Python Package Index.

```bash
python3 -m pip install tensorrt-rtx
```

--------------------------------

### FillOperation kLINSPACE Example

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/cpp-api/namespacenvinfer1.html

Example output matrix for the kLINSPACE fill operation with specific parameters.

```text
 1  11  21  31

101 111 121 131 201 211 221 231
```

--------------------------------

### Install Python bindings on Linux

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/installing-tensorrt-rtx/installing.html

Install the Python wheel included within the extracted Linux tarball directory.

```bash
pyversion="311" # Assuming Python 3.11, else replace with your
                # Python version
wheel="tensorrt_rtx-${version}-cp${pyversion}-none-linux_${arch}.whl"
python3 -m pip install TensorRT-RTX-${version}/python/${wheel}
```

--------------------------------

### Max Pooling Example

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/operators/Pooling.html

Demonstrates the configuration of a max pooling layer with a 3x3 window.

```python
in1 = network.add_input("input1", dtype=trt.float32, shape=(1, 1, 5, 5))
layer = network.add_pooling_nd(in1, trt.PoolingType.MAX, trt.tensorrt.DimsHW(3, 3))
network.mark_output(layer.get_output(0))

inputs[in1.name] = np.array(
    [
        [
            [
                [-10.0, -9.0, -8.0, -7.0, -6.0],
                [-5.0, -4.0, -3.0, -2.0, -1.0],
                [0.0, 1.0, 2.0, 3.0, 4.0],
                [5.0, 6.0, 7.0, 8.0, 9.0],
                [10.0, 11.0, 12.0, 13.0, 14.0],
            ]
        ]
    ]
)
np.reshape(np.arange(-10, 15, dtype=np.float32), newshape=(1, 1, 5, 5))

outputs[layer.get_output(0).name] = layer.get_output(0).shape

expected[layer.get_output(0).name] = np.array([[[[2.0, 3.0, 4.0], [7.0, 8.0, 9.0], [12.0, 13.0, 14.0]]]])
```

--------------------------------

### Define output comparison examples

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/polygraphy/comparator/compare_func.html

Example data structures illustrating how index tolerance affects output comparison logic.

```python
output0 = [0, 1, 2]
output1 = [1, 2, 0]
```

--------------------------------

### Install Torch-TensorRT-RTX

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/installing-tensorrt-rtx/torch-trt-rtx.html

Use pip to install the necessary Python packages for TensorRT-RTX support.

```bash
python -m pip install torch-tensorrt-rtx
```

--------------------------------

### Padding Layer Usage Example

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/operators/Padding.html

Demonstrates adding a padding layer to a network, defining pre and post padding values, and setting up input/output expectations.

```python
in1 = network.add_input("input1", dtype=trt.float32, shape=(1, 1, 3, 5))
layer = network.add_padding_nd(in1, pre_padding=(-1, 3), post_padding=(3, -2))
network.mark_output(layer.get_output(0))

inputs[in1.name] = np.array(
    [[[[-3.0, -2.0, -1.0, 10.0, -25.0], [-4.0, -9.0, -1.0, 10.0, -25.0], [0.0, 1.0, 2.0, -2.0, -1.0]]]]
)

outputs[layer.get_output(0).name] = layer.get_output(0).shape
expected[layer.get_output(0).name] = np.array(
    [
        [
            [
                [0.0, 0.0, 0.0, -4.0, -9.0, -1.0],
                [0.0, 0.0, 0.0, 0.0, 1.0, 2.0],
                [0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
                [0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
                [0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
            ]
        ]
    ]
)
```

--------------------------------

### Average Pooling Example

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/operators/Pooling.html

Demonstrates the configuration of an average pooling layer with padding and exclusion settings.

```python
in1 = network.add_input("input1", dtype=trt.float32, shape=(1, 1, 5, 5))
layer = network.add_pooling_nd(in1, trt.PoolingType.AVERAGE, trt.tensorrt.DimsHW(3, 3))
layer.post_padding = (1, 1)
layer.pre_padding = (1, 1)
layer.average_count_excludes_padding = True
network.mark_output(layer.get_output(0))

inputs[in1.name] = np.array(
    [
        [
            [
                [-10.0, -9.0, -8.0, -7.0, -6.0],
                [-5.0, -4.0, -3.0, -2.0, -1.0],
                [0.0, 1.0, 2.0, 3.0, 4.0],
                [5.0, 6.0, 7.0, 8.0, 9.0],
                [10.0, 11.0, 12.0, 13.0, 14.0],
            ]
        ]
    ]
)
np.reshape(np.arange(-10, 15, dtype=np.float32), newshape=(1, 1, 5, 5))

outputs[layer.get_output(0).name] = layer.get_output(0).shape

expected[layer.get_output(0).name] = np.array(
    [
        [
            [
                [-7.0, -6.5, -5.5, -4.5, -4.0],
                [-4.5, -4.0, -3.0, -2.0, -1.5],
                [0.5, 1.0, 2.0, 3.0, 3.5],
                [5.5, 6.0, 7.0, 8.0, 8.5],
                [8.0, 8.5, 9.5, 10.5, 11.0],
            ]
        ]
    ]
)
```

--------------------------------

### Get Builder

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/cpp-api/_nv_infer_8h_source.html

Retrieves the builder instance.

```cpp
virtual IBuilder& getBuilder() const noexcept
{
return mImpl->getBuilder();
}
```

--------------------------------

### Registering a Custom Class

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/polygraphy/_modules/polygraphy/json/serde.html

Example of defining a class to be used with the custom serialization framework.

```python
class Dummy:
    def __init__(self, x):
        self.x = x
```

--------------------------------

### Default Comparison Setup

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/polygraphy/_modules/polygraphy/comparator/comparator.html

Generates a list of tuples representing default comparisons between sequential runners.

```python
    @staticmethod
    def default_comparisons(run_results):
        # Sets up default comparisons - which is to compare each runner to the subsequent one.
        return [(i, i + 1) for i in range(len(run_results) - 1)]
```

--------------------------------

### Configuration Methods

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/cpp-api/classnvonnxparser_1_1_i_parser.html

Methods for setting and getting parser flags and builder configurations.

```APIDOC
## void setFlags(OnnxParserFlags onnxParserFlags)
Sets the parser flags.

## OnnxParserFlags getFlags()
Gets the current parser flags.

## bool setBuilderConfig(const nvinfer1::IBuilderConfig *const builderConfig)
Sets the BuilderConfig for the parser.
```

--------------------------------

### Example Postprocess Mapping

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/polygraphy/tool/args/comparator/postprocess.html

A dictionary mapping postprocessing function names to their respective output parameters.

```json
{"top_k": {"output1": 5, "output2": 6}}
```

--------------------------------

### Implement supportsFormatCombination for FP16 NCHW

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/cpp-api/classnvinfer1_1_1v__1__0_1_1_i_plugin_v3_one_build.html

Example implementation for a plugin that supports only FP16 NCHW format.

```cpp
return inOut.format[pos] == TensorFormat::kLINEAR && inOut.type[pos] == DataType::kHALF;
```

--------------------------------

### Implement supportsFormatCombination for mixed precision

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/cpp-api/classnvinfer1_1_1v__1__0_1_1_i_plugin_v3_one_build.html

Example implementation for a plugin supporting FP16 NCHW for two inputs and FP32 NCHW for one output.

```cpp
return inOut.format[pos] == TensorFormat::kLINEAR && (inOut.type[pos] == pos < 2 ?  DataType::kHALF :
  DataType::kFLOAT);
```

--------------------------------

### Example of index comparison outputs

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/polygraphy/comparator/compare_func.html

Illustrates the structure of outputs compared by the index comparison function.

```python
output0 = [0, 1, 2]
output1 = [1, 0, 2]
```

--------------------------------

### Assertion Not Firing Example

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/operators/Assertion.html

Demonstrates a network configuration where the assertion condition is met, preventing a failure.

```python
in1 = network.add_input("input1", dtype=trt.float32, shape=(3, 4, 4))
shape = network.add_shape(in1)
identity = network.add_identity(in1)
cond = network.add_elementwise(shape.get_output(0), shape.get_output(0), op=trt.ElementWiseOperation.EQUAL)
assertion = network.add_assertion(cond.get_output(0), message="Shouldn't fail")
network.mark_output(identity.get_output(0))

inputs[in1.name] = np.zeros(shape=(2, 4))
outputs[identity.get_output(0).name] = identity.get_output(0).shape
```

--------------------------------

### getPrePaddingNd

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/cpp-api/classnvinfer1_1_1_i_padding_layer.html

Gets the padding that is applied at the start of the tensor.

```APIDOC
## Dims getPrePaddingNd()

### Description
Retrieves the padding dimensions currently applied at the start of the tensor.
```

--------------------------------

### Initialize Builder and Network Objects

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/inference-library/cpp-api-docs.html

Set up smart pointers for the builder and network definition for manual network construction.

```cpp
std::unique_ptr<Logger> myLogger;
auto builder = std::unique_ptr<nvinfer1::IBuilder>(nvinfer1::createInferBuilder(&myLogger));
auto network = std::unique_ptr<nvinfer1::INetworkDefinition>(0);
```

--------------------------------

### Nearest Resize with Scales Example

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/operators/Resize.html

Demonstrates performing a nearest neighbor resize where the output shape is determined by applying scale factors to the input dimensions.

```python
input = network.add_input("input", dtype=trt.float32, shape=(1, 1, 3, 3))
layer = network.add_resize(input)
layer.resize_mode = trt.InterpolationMode.NEAREST
layer.scales = (1, 1, 2, 2)
layer.coordinate_transformation = trt.ResizeCoordinateTransformation.ALIGN_CORNERS
network.mark_output(layer.get_output(0))

inputs[input.name] = np.array([[[[0, 1, 2], [3, 4, 5], [6, 7, 8]]]])

outputs[layer.get_output(0).name] = layer.get_output(0).shape
expected[layer.get_output(0).name] = np.array(
    [
        [
            [
                [0.0, 0.0, 0.0, 1.0, 1.0, 2.0],
                [0.0, 0.0, 0.0, 1.0, 1.0, 2.0],
                [0.0, 0.0, 0.0, 1.0, 1.0, 2.0],
                [3.0, 3.0, 3.0, 4.0, 4.0, 5.0],
                [3.0, 3.0, 3.0, 4.0, 4.0, 5.0],
                [6.0, 6.0, 6.0, 7.0, 7.0, 8.0],
            ]
        ]
    ]
)
```

--------------------------------

### Install Linux tarball and set environment variables

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/installing-tensorrt-rtx/installing.html

Extract the tarball and update PATH and LD_LIBRARY_PATH to include the TensorRT-RTX binaries and libraries.

```bash
version="1.5.0.0" # Replace with newest version
arch="x86_64"      # Replace with your architecture
cuda="12.9"        # Replace with your CUDA version
tarfile="TensorRT-RTX-${version}.Linux.${arch}-gnu-${cuda}.tar.gz"
tar -xzf $tarfile
export PATH=$PATH:$PWD/TensorRT-RTX-${version}/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$PWD/TensorRT-RTX-${version}/lib
```

--------------------------------

### Create and Configure Builder

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/inference-library/python-api-docs.html

Initialize the builder configuration and set memory limits for the workspace.

```python
config = builder.create_builder_config()
```

```python
config.set_memory_pool_limit(trt.MemoryPoolType.WORKSPACE, 1 << 20) # 1 MiB
```

--------------------------------

### Slice Fill Mode Example

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/operators/Slice.html

Demonstrates using FILL mode to handle out-of-bounds coordinates by providing a fill constant input.

```python
in1 = network.add_input("input1", dtype=trt.float32, shape=(2, 2))
layer = network.add_slice(in1, start=(0, 0), shape=(3, 3), stride=(1, 1))
fill_constant = network.add_input("fill_constant", dtype=trt.float32, shape=())
layer.mode = trt.SampleMode.FILL
layer.set_input(4, fill_constant)
network.mark_output(layer.get_output(0))

inputs[in1.name] = np.zeros(shape=(2, 2))
inputs[fill_constant.name] = np.array([1.0])

outputs[layer.get_output(0).name] = layer.get_output(0).shape
expected[layer.get_output(0).name] = np.array([[0, 0, 1], [0, 0, 1], [1, 1, 1]])
```

--------------------------------

### Assertion Fired At Build Time Example

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/operators/Assertion.html

Demonstrates a network configuration where the assertion condition is expected to fail during the build stage.

```python
# This test should fail during build stage
in1 = network.add_input("input1", dtype=trt.float32, shape=(3, 4, 4))
shape1 = network.add_shape(in1)
in2 = network.add_input("input2", dtype=trt.float32, shape=(3, 3, 4))
shape2 = network.add_shape(in2)
identity = network.add_identity(in1)
cond = network.add_elementwise(shape1.get_output(0), shape2.get_output(0), op=trt.ElementWiseOperation.EQUAL)
assertion = network.add_assertion(cond.get_output(0), message="Should fail")
network.mark_output(identity.get_output(0))

inputs[in1.name] = np.zeros(shape=(2, 4))
outputs[identity.get_output(0).name] = identity.get_output(0).shape
```

--------------------------------

### Implement supportsFormatCombination for polymorphic types

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/cpp-api/classnvinfer1_1_1v__1__0_1_1_i_plugin_v3_one_build.html

Example implementation for a plugin requiring inputs and outputs to share the same format and type.

```cpp
return pos == 0 || (inOut.format[pos] == inOut.format[0] && inOut.type[pos] == inOut.type[0]);
```

--------------------------------

### Runtime.__init__

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/python-api/infer/Core/Runtime.html

Initializes a new Runtime instance with a specified logger.

```APIDOC
## Runtime(logger: ILogger)

### Description
Creates a new Runtime object used for deserializing ICudaEngine instances.

### Parameters
- **logger** (ILogger) - Required - The logger instance to be used by the runtime.
```

--------------------------------

### Linear Resize Example

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/operators/Resize.html

Demonstrates performing a linear resize on a 3x3 input tensor to a 5x5 output shape using the TensorRT Python API.

```python
input = network.add_input("input", dtype=trt.float32, shape=(1, 1, 3, 3))
layer = network.add_resize(input)
layer.resize_mode = trt.InterpolationMode.LINEAR
layer.shape = (1, 1, 5, 5)
layer.coordinate_transformation = trt.ResizeCoordinateTransformation.ALIGN_CORNERS
network.mark_output(layer.get_output(0))

inputs[input.name] = np.array([[[[0, 1, 2], [3, 4, 5], [6, 7, 8]]]])

outputs[layer.get_output(0).name] = layer.get_output(0).shape
expected[layer.get_output(0).name] = np.array(
    [
        [
            [0, 0.5, 1, 1.5, 2],
            [1.5, 2, 2.5, 3, 3.5],
            [3, 3.5, 4, 4.5, 5],
            [4.5, 5, 5.5, 6, 6.5],
            [6, 6.5, 7, 7.5, 8],
        ]
    ]
)
```

--------------------------------

### Refitter.__init__

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/python-api/infer/Core/Refitter.html

Initializes a new Refitter instance for a given engine and logger.

```APIDOC
## __init__(engine, logger)

### Description
Initializes the Refitter object to manage weight updates for the specified engine.

### Parameters
- **engine** (ICudaEngine) - Required - The engine to refit.
- **logger** (ILogger) - Required - The logger to use.
```

--------------------------------

### Normalization Implementation and Reference Example

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/operators/Normalization.html

Demonstrates how to define a normalization reference function and integrate the normalization layer into a TensorRT network.

```python
def convert_axes_to_tuple(axes, num_dims):
    converted_axes = []
    for i in range(num_dims):
        ref = 1 << i
        if axes & ref == ref:
            converted_axes.append(i)
    return tuple(converted_axes)

def normalization_reference(x, s, bias, axes, epsilon=1e-5):  # type: ignore
    num_dims = len(x.shape)
    converted_axes = convert_axes_to_tuple(axes, num_dims)
    mean = np.mean(x, axis=converted_axes, keepdims=True)
    var = np.var(x, axis=converted_axes, keepdims=True)
    return s * (x - mean) / np.sqrt(var + epsilon) + bias

input = network.add_input("input1", dtype=trt.float32, shape=(2, 3, 2, 2))
scale = network.add_input("scale", dtype=trt.float32, shape=(1, 3, 1, 1))
bias = network.add_input("bias", dtype=trt.float32, shape=(1, 3, 1, 1))

axes = 1 << 2 | 1 << 3
layer = network.add_normalization(input, scale, bias, axes)
network.mark_output(layer.get_output(0))

input_vals = np.random.rand(2, 3, 2, 2)
scale_vals = np.array([1.0, 2.0, 3.0]).reshape((1, 3, 1, 1))
bias_vals = np.array([-3.0, -2.0, -1.0]).reshape((1, 3, 1, 1))

inputs[input.name] = input_vals
inputs[scale.name] = scale_vals
inputs[bias.name] = bias_vals

outputs[layer.get_output(0).name] = layer.get_output(0).shape

ref = normalization_reference(input_vals, scale_vals, bias_vals, axes)

expected[layer.get_output(0).name] = ref
```

--------------------------------

### phaseStart

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/cpp-api/classnvinfer1_1_1v__1__0_1_1_i_progress_monitor.html

Signals that a phase of the optimizer has started.

```APIDOC
## virtual void phaseStart(char const *phaseName, char const *parentPhase, int32_t nbSteps) noexcept = 0

### Description
Signals that a phase of the optimizer has started.

### Parameters
- **phaseName** (char const *) - The name of the phase that is starting.
- **parentPhase** (char const *) - The name of the parent phase, if applicable.
- **nbSteps** (int32_t) - The total number of steps in this phase.
```

--------------------------------

### Run basic performance benchmarking

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/performance/best-practices.html

Execute a quick performance check on an ONNX model with detailed profiling enabled.

```bash
tensorrt_rtx --onnx=your_model.onnx --profilingVerbosity=detailed
```

--------------------------------

### Load Plugins Immediately

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/polygraphy/_modules/polygraphy/backend/trt/loader.html

Use this function for immediate execution of plugin loading from specified library paths.

```python
load_plugins(plugins=["/path/to/my/plugin.so", "/path/to/my/other_plugin.so"])
```

--------------------------------

### Define Plugin Format Support

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/cpp-api/classnvinfer1_1_1_i_plugin_v2_i_o_ext.html

Examples of implementing supportsFormatCombination to validate input and output tensor formats and data types.

```cpp
return inOut.format[pos] == TensorFormat::kLINEAR && inOut.type[pos] == DataType::kHALF;
```

```cpp
return inOut.format[pos] == TensorFormat::kLINEAR &&
         (inOut.type[pos] == (pos < 2 ?  DataType::kHALF : DataType::kFLOAT));
```

```cpp
return pos == 0 || (inOut.format[pos] == inOut.format[0] && inOut.type[pos] == inOut.type[0]);
```

--------------------------------

### IProgressMonitor.phase_start

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/python-api/infer/Core/ProgressMonitor.html

Signals that a specific optimization phase has started.

```APIDOC
## phase_start(phase_name, parent_phase, num_steps)

### Description
Signals that a phase of the optimizer has started. This function indicates the beginning of a phase and specifies the number of steps involved.

### Parameters
- **phase_name** (str) - Required - The name of this phase for tracking purposes.
- **parent_phase** (str) - Required - The parent phase that this phase belongs to, or None if there is no parent.
- **num_steps** (SupportsInt) - Required - The number of steps that are involved in this phase.
```

--------------------------------

### Create a Builder Instance

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/inference-library/cpp-api-docs.html

Initialize the builder using the previously defined logger instance.

```cpp
IBuilder* builder = createInferBuilder(logger);
```

--------------------------------

### setStart()

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/cpp-api/classnvinfer1_1_1_i_slice_layer.html

Sets the start offset for the slice layer.

```APIDOC
## void nvinfer1::ISliceLayer::setStart(Dims const & start)

### Description
Sets the start offset that the slice layer uses to create the output slice. If a second input had been used to create this layer, that input is reset to null.

### Parameters
- **start** (Dims const &) - The start offset to read data from the input tensor.
```

--------------------------------

### IPluginV3OneBuild Interface

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/cpp-api/_nv_infer_runtime_8h_source.html

Build-time interface for V3 plugins, handling configuration, shape inference, and format validation.

```cpp
862class IPluginV3OneBuild : public IPluginCapability
863{
864public:
870 static constexpr int32_t kDEFAULT_FORMAT_COMBINATION_LIMIT = 100;
871
875 InterfaceInfo getInterfaceInfo() const noexcept override
876 {
877 return InterfaceInfo{"PLUGIN_V3ONE_BUILD", 1, 0};
878 }
879
899 virtual int32_t configurePlugin(DynamicPluginTensorDesc const* in, int32_t nbInputs,
900 DynamicPluginTensorDesc const* out, int32_t nbOutputs) noexcept = 0;
901
918 virtual int32_t getOutputDataTypes(
919 DataType* outputTypes, int32_t nbOutputs, const DataType* inputTypes, int32_t nbInputs) const noexcept = 0;
920
942 virtual int32_t getOutputShapes(DimsExprs const* inputs, int32_t nbInputs, DimsExprs const* shapeInputs,
943 int32_t nbShapeInputs, DimsExprs* outputs, int32_t nbOutputs, IExprBuilder& exprBuilder) noexcept = 0;
944
979 virtual bool supportsFormatCombination(
980 int32_t pos, DynamicPluginTensorDesc const* inOut, int32_t nbInputs, int32_t nbOutputs) noexcept = 0;
981
987 virtual int32_t getNbOutputs() const noexcept = 0;
988
998 virtual size_t getWorkspaceSize(DynamicPluginTensorDesc const* /* inputs */, int32_t /* nbInputs */,
999 DynamicPluginTensorDesc const* /* outputs */, int32_t /* nbOutputs */) const noexcept
1000 {
1001 return 0;
1002 }
1003
1035 virtual int32_t getValidTactics(int32_t* /* tactics */, int32_t /* nbTactics */) noexcept
1036 {
1037 return 0;
1038 }
1039
1043 virtual int32_t getNbTactics() noexcept
1044 {
1045 return 0;
1046 }
1047
1059 virtual char const* getTimingCacheID() noexcept
1060 {
1061 return nullptr;
1062 }
1063
1067 virtual int32_t getFormatCombinationLimit() noexcept
1068 {
1069 return kDEFAULT_FORMAT_COMBINATION_LIMIT;
1070 }
1071
1078 virtual char const* getMetadataString() noexcept
1079 {
1080 return nullptr;
1081 }
1082};
```

--------------------------------

### getStart()

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/cpp-api/classnvinfer1_1_1_i_slice_layer.html

Retrieves the start offset for the slice layer.

```APIDOC
## Dims getStart()

### Description
Returns the start offset for the slice. If the second input is present and non-null, this returns a Dims with nbDims = -1.
```

--------------------------------

### Scale 5D Example

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/operators/Scale.html

Demonstrates applying a channel-wise scale, shift, and power transformation to a 5D input tensor using add_scale_nd.

```python
in1 = network.add_input("input1", dtype=trt.float32, shape=(1, 2, 1, 3, 3))
scale_u = np.array([1, 2], np.float32).reshape(-1)
shift_u = np.array([0, 1], np.float32).reshape(-1)
power_u = np.array([1, 2], np.float32).reshape(-1)
layer = network.add_scale_nd(
    in1, trt.ScaleMode.CHANNEL, shift=shift_u, scale=scale_u, power=power_u, channel_axis=1
)
network.mark_output(layer.get_output(0))

inputs[in1.name] = np.array(
    [
        [
            [
                [
                    [1.0, 2.0, 3.0],
                    [4.0, 5.0, 6.0],
                    [7.0, 8.0, 9.0],
                ]
            ],
            [
                [
                    [1.0, 2.0, 3.0],
                    [4.0, 5.0, 6.0],
                    [7.0, 8.0, 9.0],
                ]
            ],
        ]
    ]
)

outputs[layer.get_output(0).name] = layer.get_output(0).shape

expected[layer.get_output(0).name] = np.array(
    [
        [
            [
                [
                    [1.0, 2.0, 3.0],
                    [4.0, 5.0, 6.0],
                    [7.0, 8.0, 9.0],
                ]
            ],
            [[[9.0, 25.0, 49.0], [81.0, 121.0, 169.0], [225.0, 289.0, 361.0]]],
        ]
    ]
)
```

--------------------------------

### Create a Network Definition

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/inference-library/python-api-docs.html

Initialize a network definition using the builder instance.

```python
network = builder.create_network(flag)
```

--------------------------------

### getPrePaddingNd

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/cpp-api/classnvinfer1_1_1_i_padding_layer.html

Retrieves the padding applied at the start of the tensor.

```APIDOC
### Dims getPrePaddingNd() const

Returns the padding applied at the start of the tensor. Note that only 2-dimensional padding is currently supported.
```

--------------------------------

### IShapeLayer

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/python-api/infer/Graph/Layers.html

A layer used for getting the shape of a tensor, outputting a one-dimensional int64 tensor containing the input dimensions.

```APIDOC
## class tensorrt_rtx.IShapeLayer

### Description
A shape layer in an INetworkDefinition. Used for getting the shape of a tensor. This class sets the output to a one-dimensional tensor with the dimensions of the input tensor.
```

--------------------------------

### IPluginV3OneBuild

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/cpp-api/_nv_infer_runtime_8h_source.html

Interface for plugin configuration and build-time logic.

```APIDOC
## IPluginV3OneBuild

### Description
Defines methods for configuring the plugin, determining output shapes, and validating format combinations during the build phase.

### Methods
- **configurePlugin(...)** (int32_t) - Configures the plugin based on input and output tensor descriptors.
- **getOutputShapes(...)** (int32_t) - Calculates output tensor shapes based on input shapes.
- **supportsFormatCombination(...)** (bool) - Checks if a specific format combination is supported by the plugin.
- **getWorkspaceSize(...)** (size_t) - Returns the required workspace size for the plugin.
```

--------------------------------

### setPrePaddingNd

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/cpp-api/classnvinfer1_1_1_i_padding_layer.html

Sets the padding that is applied at the start of the tensor.

```APIDOC
## void setPrePaddingNd(Dims const &padding)

### Description
Sets the padding that is applied at the start of the tensor.

### Parameters
- **padding** (Dims const &) - The dimensions of the padding to apply.
```

--------------------------------

### getNamedWeights

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/cpp-api/classnvinfer1_1_1_i_refitter.html

Get weights associated with the given name.

```APIDOC
## Weights getNamedWeights(char const *weightsName)

### Description
Retrieves the weights currently associated with the specified name.
```

--------------------------------

### IPluginV3OneBuild Version 2.0

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/cpp-api/_nv_infer_runtime_8h_source.html

Extended build interface for V3 plugins, adding support for aliased inputs.

```cpp
1181class IPluginV3OneBuild : public v_1_0::IPluginV3OneBuild
1182{
1183public:
1184 InterfaceInfo getInterfaceInfo() const noexcept override
1185 {
1186 return InterfaceInfo{"PLUGIN_V3ONE_BUILD", 2, 0};
1187 }
1188
1218 virtual int32_t getAliasedInput(int32_t /* outputIndex */) noexcept
1219 {
1220 return -1;
1221 }
1222};
```

--------------------------------

### getPostPaddingNd

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/cpp-api/classnvinfer1_1_1_i_padding_layer.html

Gets the padding that is applied at the end of the tensor.

```APIDOC
## Dims getPostPaddingNd()

### Description
Retrieves the padding dimensions currently applied at the end of the tensor.
```

--------------------------------

### getPluginRegistry

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/cpp-api/classnvinfer1_1_1_i_runtime.html

Get the local plugin registry that can be used by the runtime.

```APIDOC
## IPluginRegistry& getPluginRegistry()

### Description
Get the local plugin registry that can be used by the runtime.
```

--------------------------------

### getMissing

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/cpp-api/classnvinfer1_1_1_i_refitter.html

Get description of missing weights that are required for refitting.

```APIDOC
## int32_t getMissing(int32_t size, char const **layerNames, WeightsRole *roles)

### Description
Retrieves the names and roles of weights that are missing and must be provided before the engine can be refit.
```

--------------------------------

### getOptimizationProfile

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/cpp-api/classnvinfer1_1_1_i_execution_context.html

Gets the index of the currently selected optimization profile.

```APIDOC
## int32_t getOptimizationProfile()

### Description
Returns the index of the currently selected optimization profile. Returns -1 if the profile index has not been set.
```

--------------------------------

### getMaxDLABatchSize()

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/cpp-api/classnvinfer1_1_1_i_builder.html

Gets the maximum batch size supported by DLA.

```APIDOC
## int32_t getMaxDLABatchSize() const

### Description
Returns the maximum batch size that DLA can support. Note that this function does not support dynamic shapes.

### Returns
- **int32_t** - The maximum supported DLA batch size.
```

--------------------------------

### Create a Constant Layer with Int4 Weights

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/operators/Constant.html

Shows how to pack int4 weights and follow the constant layer with a dequantize node.

```python
def pack_int4(array: np.ndarray):
    result = []
    array = array.flatten()
    for low, high in zip(array[::2], array[1::2]):
        low = np.rint(np.clip(low, -8, 7)).astype(np.int8)
        high = np.rint(np.clip(high, -8, 7)).astype(np.int8)
        result.append(high << 4 | low & 0x0F)
    return np.asarray(result, dtype=np.int8)

w = np.array([[ 0,  1,  2,  3,  4,  5,  6,  7],
              [-1, -2, -3, -4, -5, -6, -7, -8],
              [ 7,  6,  5,  4,  3,  2,  1,  0],
              [-7, -6, -5, -4, -3, -2, -1,  0]], dtype=np.int8)

w_packed = pack_int4(w)
weights = network.add_constant(shape=w.shape, weights=trt.Weights(trt.int4, w_packed.ctypes.data, w.size))

# Quantized weights must be followed by a DQ node
scale = network.add_constant(shape=(), weights=np.ones(shape=(1), dtype=np.float32))
dequantize = network.add_dequantize(weights.get_output(0), scale.get_output(0), trt.float32)
network.mark_output(dequantize.get_output(0))

outputs[dequantize.get_output(0).name] = dequantize.get_output(0).shape
expected[dequantize.get_output(0).name] = w
```

--------------------------------

### Initialize Logger, Builder, and Network

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/inference-library/python-api-docs.html

Set up the core TensorRT objects required for network construction.

```python
TRT_LOGGER = trt.Logger(trt.Logger.ERROR)
builder = trt.Builder(TRT_LOGGER)
network = builder.create_network(0)
```

--------------------------------

### Enqueue Inference

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/inference-library/cpp-api-docs.html

Starts the inference process on the specified CUDA stream.

```cpp
context->enqueueV3(stream);
```

--------------------------------

### ISliceLayer.set_input

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/python-api/infer/Graph/Layers.html

Sets the input tensor for the slice layer at a specified index. This method can be used to convert a static slice layer into a dynamic one by providing inputs for start, size, stride, or axes.

```APIDOC
## set_input(index, tensor)

### Description
Sets the input tensor for the given index. The index must be 0 or 4 for a static slice layer. A static slice layer is converted to a dynamic slice layer by calling set_input() with an index between 1 and 3.

### Parameters
- **index** (SupportsInt) - Required - The index of the input tensor (0-5).
- **tensor** (ITensor) - Required - The input tensor to be set.

### Indices
- **0**: Data or Shape tensor to be sliced.
- **1**: The start tensor to begin slicing.
- **2**: The size tensor of the resulting slice.
- **3**: The stride of the slicing operation.
- **4**: Value for the SampleMode.FILL slice mode.
- **5**: The axes tensor indicating the axes that starts, sizes, and strides correspond to.
```

--------------------------------

### Tactic Sources Configuration

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/cpp-api/_nv_infer_8h_source.html

Methods to set and get tactic sources.

```cpp
 bool setTacticSources(TacticSources tacticSources) noexcept
 {
 return mImpl->setTacticSources(tacticSources);
 }

 TacticSources getTacticSources() const noexcept
 {
 return mImpl->getTacticSources();
 }
```

--------------------------------

### Create a Constant Layer

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/operators/Constant.html

Demonstrates initializing a constant layer with a specific shape and float32 weights.

```python
input_shape = [1, 3, 3, 3]
w = np.arange(0.0, 27.0, dtype=np.dtype("f4"))
layer = network.add_constant(shape=input_shape, weights=trt.Weights(w))
network.mark_output(layer.get_output(0))

outputs[layer.get_output(0).name] = layer.get_output(0).shape
expected[layer.get_output(0).name] = w.reshape(input_shape)
```

--------------------------------

### Get Error Recorder

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/cpp-api/_nv_infer_8h_source.html

Retrieves the error recorder associated with the implementation.

```cpp
IErrorRecorder* getErrorRecorder() const noexcept
{
return mImpl->getErrorRecorder();
}
```

--------------------------------

### Network Name Management

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/cpp-api/_nv_infer_8h_source.html

Methods to set and get the network name.

```cpp
void setName(char const* name) noexcept
{
mImpl->setName(name);
}
```

```cpp
char const* getName() const noexcept
{
return mImpl->getName();
}
```

--------------------------------

### Refitter.get_all

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/python-api/infer/Core/Refitter.html

Retrieves descriptions of all weights that can be refitted.

```APIDOC
## get_all()

### Description
Returns the names of layers with refittable weights and their corresponding roles.

### Returns
- **tuple[list[str], list[WeightsRole]]** - A tuple containing the list of layer names and the list of weights roles.
```

--------------------------------

### Get Dimension Name

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/inference-library/work-with-dynamic-shapes.html

Retrieves the name assigned to a specific tensor dimension.

```C++
tensor.getDimensionName(2) // returns the name of the third dimension of the tensor, or nullptr if it does not have a name.
```

```Python
tensor.get_dimension_name(2) # returns the name of the third dimension of the tensor, or None if it does not have a name.
```

--------------------------------

### Create Runtime Configuration

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/inference-library/work-with-runtime-cache.html

Initialize the runtime configuration object and set the execution context allocation strategy.

```C++
1IRuntimeConfig* runtimeConfig = engine->createRuntimeConfig();
2
3runtimeConfig->setExecutionContextAllocationStrategy(ExecutionContextAllocationStrategy::kSTATIC);
```

```Python
1runtime_config = engine.create_runtime_config()
2
3runtime_config.set_execution_context_allocation_strategy(trt.ExecutionContextAllocationStrategy.STATIC)
```

--------------------------------

### Set Input Metadata and Initialize Cache

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/polygraphy/_modules/polygraphy/comparator/data_loader.html

Configures input metadata and populates the input cache from the data loader.

```python
def set_input_metadata(self, input_metadata):
    """
    Set the input metadata for the data loader.

    Args:
        input_metadata (TensorMetadata):
                Input Metadata, including shape and type information. The cache may attempt to transform inputs to
                match the specified input_metadata when data already in the cache does not exactly match.
    """
    self.input_metadata = input_metadata
    with contextlib.suppress(AttributeError):
        self.data_loader.input_metadata = input_metadata

    if not self.cache:
        G_LOGGER.verbose("Loading inputs from data loader")
        self.cache = list(self.data_loader)

        def _is_feed_dict(inp):
            if isinstance(inp, RunResults):
                return False

            try:
                for name, _ in inp.items():
                    if not isinstance(name, str):
                        return False
            except:
                return False
            else:
                return True

        if not self.cache:
            G_LOGGER.warning("Data loader did not yield any input data.")
        elif not _is_feed_dict(self.cache[0]):
            G_LOGGER.critical(
                f"Data loader returned an object that cannot be recognized as a feed_dict (Dict[str, Union[np.ndarray, torch.Tensor, DeviceView]]):"
                f"\nNote: The object was:\n{self.cache[0]}.\n"
                f"\nHint: If this is a `RunReults` object (e.g. generated with `--save-outputs`), try using the "
                f"`data merge` tool to convert it to a feed_dict compatible format. "
            )

        # Only save inputs the first time the cache is generated
        if self.save_inputs_path is not None:
            save_json(self.cache, self.save_inputs_path, "inference input data")
```

--------------------------------

### Builder Optimization Level

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/cpp-api/_nv_infer_8h_source.html

Methods to set and get the builder optimization level.

```cpp
 void setBuilderOptimizationLevel(int32_t level) noexcept
 {
 mImpl->setBuilderOptimizationLevel(level);
 }

 int32_t getBuilderOptimizationLevel() noexcept
 {
 return mImpl->getBuilderOptimizationLevel();
 }
```

--------------------------------

### Create Runtime Instance

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/inference-library/cpp-api-docs.html

Initialize the Runtime interface using a logger instance before deserializing an engine.

```cpp
IRuntime* runtime = createInferRuntime(logger);
```

--------------------------------

### Memory Pool Limits

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/cpp-api/_nv_infer_8h_source.html

Methods to set and get memory pool limits.

```cpp
 void setMemoryPoolLimit(MemoryPoolType pool, std::size_t poolSize) noexcept
 {
 mImpl->setMemoryPoolLimit(pool, poolSize);
 }

 std::size_t getMemoryPoolLimit(MemoryPoolType pool) const noexcept
 {
 return mImpl->getMemoryPoolLimit(pool);
 }
```

--------------------------------

### Create Builder Configuration

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/inference-library/cpp-api-docs.html

Initialize the builder configuration object to define optimization parameters.

```cpp
IBuilderConfig* config = builder->createBuilderConfig();
```

--------------------------------

### Profiling Verbosity Configuration

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/cpp-api/_nv_infer_8h_source.html

Methods to set and get the profiling verbosity level.

```cpp
 void setProfilingVerbosity(ProfilingVerbosity verbosity) noexcept
 {
 mImpl->setProfilingVerbosity(verbosity);
 }

 ProfilingVerbosity getProfilingVerbosity() const noexcept
 {
 return mImpl->getProfilingVerbosity();
 }
```

--------------------------------

### Build and Save Engine

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/inference-library/python-api-docs.html

Serialize the network into an engine and write it to a file.

```python
serialized_engine = builder.build_serialized_network(network, config)
```

```python
with open("sample.engine", "wb") as f:
    f.write(serialized_engine)
```

--------------------------------

### Profile Stream Configuration

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/cpp-api/_nv_infer_8h_source.html

Methods to set and get the CUDA stream for profiling.

```cpp
 void setProfileStream(const cudaStream_t stream) noexcept
 {
 return mImpl->setProfileStream(stream);
 }

 cudaStream_t getProfileStream() const noexcept
 {
 return mImpl->getProfileStream();
 }
```

--------------------------------

### enqueue()

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/cpp-api/functions_func_e.html

Executes the plugin. This method is part of the IPluginV2, IPluginV2DynamicExt, and IPluginV3OneRuntime interfaces.

```APIDOC
## enqueue()

### Description
Executes the plugin. This method is available in nvinfer1::IPluginV2, nvinfer1::IPluginV2DynamicExt, and nvinfer1::v_1_0::IPluginV3OneRuntime interfaces.
```

--------------------------------

### Create Runtime Configuration

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/inference-library/work-with-cuda-graphs.html

Initialize the runtime configuration object from the deserialized engine.

```C++
IRuntimeConfig* runtimeConfig = engine->createRuntimeConfig();
```

```Python
runtime_config = engine.create_runtime_config()
```

--------------------------------

### Builder Flags Management

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/cpp-api/_nv_infer_8h_source.html

Methods to set, get, and clear builder flags.

```cpp
 void setFlags(BuilderFlags builderFlags) noexcept
 {
 mImpl->setFlags(builderFlags);
 }

 BuilderFlags getFlags() const noexcept
 {
 return mImpl->getFlags();
 }

 void clearFlag(BuilderFlag builderFlag) noexcept
 {
 mImpl->clearFlag(builderFlag);
 }

 void setFlag(BuilderFlag builderFlag) noexcept
 {
 mImpl->setFlag(builderFlag);
 }

 bool getFlag(BuilderFlag builderFlag) const noexcept
 {
 return mImpl->getFlag(builderFlag);
 }
```

--------------------------------

### getMaxOutputSize

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/cpp-api/classnvinfer1_1_1_i_execution_context.html

Gets the upper bound on an output tensor's size in bytes.

```APIDOC
## int64_t getMaxOutputSize(char const * tensorName)

### Description
Returns the upper bound on an output tensor's size, in bytes, based on the current optimization profile and input dimensions. Returns -1 if the profile or input dimensions are not set, or if the name does not map to an output.

### Parameters
- **tensorName** (char const *) - Required - The name of an output tensor (must be null-terminated, max 4096 bytes).
```

--------------------------------

### Create a Network Definition

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/inference-library/cpp-api-docs.html

Initialize the network definition object using the builder.

```cpp
INetworkDefinition* network = builder->createNetworkV2(flag);
```

--------------------------------

### Error Recorder Management

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/cpp-api/classnvinfer1_1_1_i_engine_inspector-members.html

Methods for getting and setting the error recorder for the inspector.

```APIDOC
### Methods
IErrorRecorder* getErrorRecorder() const noexcept
void setErrorRecorder(IErrorRecorder *recorder) noexcept

### Description
These methods allow for retrieving or setting the IErrorRecorder instance used to capture errors during inspector operations.
```

--------------------------------

### NetworkFromOnnxBytes.__init__

Source: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/_static/polygraphy/_modules/polygraphy/backend/trt/loader.html

Initializes the NetworkFromOnnxBytes functor with the model data and configuration options.

```APIDOC
## Class: NetworkFromOnnxBytes

### Description
Parses an ONNX model from bytes or a callable to create a trt.INetworkDefinition.

### Constructor
`NetworkFromOnnxBytes(model_bytes, flags=None, plugin_instancenorm=None, strongly_typed=None, mark_unfused_tensors_as_debug_tensors=None, enable_uint8_asymmetric_quantization_dla=None, config=None)`

### Parameters
- **model_bytes** (Union[bytes, Callable]) - Required - A serialized ONNX model or a callable that returns one.
- **flags** (List[trt.OnnxParserFlag]) - Optional - A list of OnnxParserFlag to modify parsing behavior.
- **plugin_instancenorm** (bool) - Optional - Whether to force usage of the plugin implementation of ONNX InstanceNorm.
- **strongly_typed** (bool) - Optional - Whether to mark the network as being strongly typed.
- **mark_unfused_tensors_as_debug_tensors** (bool) - Optional - Whether to mark unfused tensors as debug tensors.
- **enable_uint8_asymmetric_quantization_dla** (bool) - Optional - Deprecated. Use flags instead.
- **config** (Union[trt.IBuilderConfig, Callable]) - Optional - A TensorRT builder configuration or a callable that returns one.
```
Tokens:
Raw