Acuitylite

https://verisilicon.github.io/acuitylite
Admin
Acuitylite is an end-to-end neural network deployment tool for embedded systems that converts...
Tokens:12,036
Snippets:107
Trust Score:9.7
Update:5 months ago
Show doc for...
Tokens:
Raw
### Install Acuitylite

Source: https://verisilicon.github.io/acuitylite/README.html

Instructions for installing Acuitylite using Docker and pip.

```bash
1. build the recommended docker image and run a container
2. pip install acuitylite --no-deps
```

--------------------------------

### Run SqueezeNet ONNX Model

Source: https://verisilicon.github.io/acuitylite/_sources/demo_onnx.rst.txt

This Python script demonstrates how to load and run the SqueezeNet ONNX model. Ensure the ONNX runtime is installed.

```python
import numpy as np
import onnxruntime as ort

# Load the ONNX model
model_path = "squeezenet.onnx"
ort_session = ort.InferenceSession(model_path)

# Get input and output names
input_name = ort_session.get_inputs()[0].name
output_name = ort_session.get_outputs()[0].name

# Create dummy input data (e.g., for a 224x224 image)
# The shape and type should match the model's expected input
input_shape = ort_session.get_inputs()[0].shape
# Assuming input is NCHW format, batch size 1
if isinstance(input_shape[2], str) or isinstance(input_shape[3], str):
    # Dynamic shape, use a common size like 224
    dummy_input = np.random.randn(1, 3, 224, 224).astype(np.float32)
else:
    dummy_input = np.random.randn(*input_shape).astype(np.float32)

# Run inference
outputs = ort_session.run([output_name], {input_name: dummy_input})

# Process the output (e.g., get the predicted class)
predicted_class_index = np.argmax(outputs[0])
print(f"Predicted class index: {predicted_class_index}")

```

--------------------------------

### Lenet Caffe Test Script

Source: https://verisilicon.github.io/acuitylite/_sources/demo_caffe.rst.txt

This script is used to test the Lenet Caffe model. It requires Python and the Caffe library to be installed.

```python
import numpy as np
import sys
import os

# Add caffe to Python path
# This is a placeholder and might need adjustment based on your Caffe installation
# Example: sys.path.insert(0, '/path/to/your/caffe/python')

try:
    import caffe
except ImportError:
    print("Caffe Python module not found. Please ensure Caffe is installed and its Python path is set correctly.")
    sys.exit(1)

def test_lenet_caffe():
    # Define the path to the Caffe model and deploy prototxt
    # These paths are relative to the script's location or need to be absolute
    model_dir = os.path.dirname(os.path.abspath(__file__))
    deploy_prototxt = os.path.join(model_dir, '..', 'models', 'lenet', 'deploy.prototxt')
    caffe_model = os.path.join(model_dir, '..', 'models', 'lenet', 'lenet_iter_10000.caffemodel')

    if not os.path.exists(deploy_prototxt):
        print(f"Error: deploy.prototxt not found at {deploy_prototxt}")
        return False
    if not os.path.exists(caffe_model):
        print(f"Error: caffemodel not found at {caffe_model}")
        return False

    try:
        # Load the Caffe model
        net = caffe.Net(deploy_prototxt, caffe_model, caffe.TEST)
        print("Caffe model loaded successfully.")

        # Prepare input data (e.g., a dummy image)
        # The input layer name might vary; 'data' is common for image models
        input_layer_name = 'data'
        if input_layer_name not in net.blobs:
            print(f"Error: Input layer '{input_layer_name}' not found in the model.")
            return False

        # Assuming input shape is (batch_size, channels, height, width)
        # For Lenet, typically (1, 1, 28, 28) for MNIST
        input_shape = net.blobs[input_layer_name].data.shape
        print(f"Expected input shape: {input_shape}")

        # Create dummy input data matching the expected shape
        # Example for MNIST-like input (1 channel, 28x28 image)
        if len(input_shape) == 4 and input_shape[1] == 1 and input_shape[2] == 28 and input_shape[3] == 28:
            dummy_input = np.random.rand(*input_shape).astype(np.float32)
            net.blobs[input_layer_name].data[...] = dummy_input
            print("Dummy input data created and set.")
        else:
            print("Warning: Input shape is not standard MNIST (1, 1, 28, 28). Using a generic random input.")
            # Fallback for other shapes, might need adjustment
            dummy_input = np.random.rand(*input_shape).astype(np.float32)
            net.blobs[input_layer_name].data[...] = dummy_input

        # Perform forward pass
        output = net.forward()
        print("Forward pass completed.")

        # Print output shape (e.g., from the 'prob' or 'fc8' layer)
        # The output layer name depends on the model architecture
        output_layer_name = 'prob' # Common for classification models
        if output_layer_name in net.blobs:
            print(f"Output from '{output_layer_name}' layer: {net.blobs[output_layer_name].data.shape}")
            # print(net.blobs[output_layer_name].data)
        else:
            print("Output layer 'prob' not found. Printing shapes of all output blobs:")
            for name, blob in net.blobs.items():
                if blob.data.ndim > 0: # Check if it's a data blob
                    print(f" - {name}: {blob.data.shape}")

        return True

    except Exception as e:
        print(f"An error occurred during Caffe model testing: {e}")
        return False

if __name__ == "__main__":
    if test_lenet_caffe():
        print("Lenet Caffe test completed successfully.")
    else:
        print("Lenet Caffe test failed.")
        sys.exit(1)

```

--------------------------------

### Set Environment for TIM-VX Case

Source: https://verisilicon.github.io/acuitylite/README.html

Set environment variables TIM_VX_DIR, VIVANTE_SDK_DIR, and LD_LIBRARY_PATH to build and run TIM-VX cases. Ensure TIM_VX_DIR includes TIM-VX library and header files.

```bash
TIM_VX_DIR=/path/to/tim-vx/build/install
VIVANTE_SDK_DIR=/path/to/tim-vx/prebuilt-sdk/x86_64_linux
LD_LIBRARY_PATH=$TIM_VX_DIR/lib:$VIVANTE_SDK_DIR/lib
```

--------------------------------

### export Method

Source: https://verisilicon.github.io/acuitylite/exporter.tim-vx.html

Exports the TIM-VX application to a specified directory with a given prefix.

```APIDOC
## Method: export

### Description
Export TIM-VX application.

### Parameters
* **output** (str) - The directory of the exported applications with the prefix specified. Use the syntax ‘<output directory>/<prefix>’. Where: Output directory: The directory of the generated outputs including subdirectories and files. Prefix: The prefix of the generated subdirectories and files. For example, “export_timvx/uint8/lenet”
* **api_def_json_path** (str) - Optional. The path to specify TIM-VX api defined json files.
* **kwargs** - Additional keyword arguments.
```

--------------------------------

### TensorflowLoader

Source: https://verisilicon.github.io/acuitylite/_sources/importer.tensorflowloader.rst.txt

Initializes the TensorflowLoader.

```APIDOC
## TensorflowLoader

### Description
Initializes the TensorflowLoader.

### Method
__init__
```

--------------------------------

### export

Source: https://verisilicon.github.io/acuitylite/exporter.ovxlib.html

Exports the Ovxlib application to a specified directory with a given prefix.

```APIDOC
## export

### Description
Export Ovxlib application.

### Parameters
* **output** (str) - The directory of the exported applications with the prefix specified. Use the syntax '<output directory>/<prefix>'. For example, "export_ovxlib/uint8/lenet".
* **kwargs** - Additional keyword arguments.
```

--------------------------------

### DarknetLoader

Source: https://verisilicon.github.io/acuitylite/importer.darknetloader.html

Initializes the DarknetLoader with model and weights file paths.

```APIDOC
## class DarknetLoader(model, weights)

### Description
Translates a Darknet model to Acuity formats.

### Parameters
* **model** (file path) - The file path of the imported Darknet model file, a .cfg file.
* **weights** (file path) - The file path of the imported weights file, a .weights file.
```

--------------------------------

### CaffeLoader Constructor

Source: https://verisilicon.github.io/acuitylite/importer.caffeloader.html

Initializes the CaffeLoader with paths to the Caffe model and weights files.

```APIDOC
## CaffeLoader(model, weights=None)

### Description
Translates a Caffe model to Acuity formats.

### Parameters
* **model** (string) - Required - The file path of the imported Caffe model file, a .prototxt file.
* **weights** (string) - Optional - The file path of the imported weights file, a .caffemodel file.
```

--------------------------------

### Inference Class Initialization

Source: https://verisilicon.github.io/acuitylite/_sources/inference.rst.txt

Initializes the Inference class. This is the entry point for using the inference capabilities.

```APIDOC
## __init__

### Description
Initializes the Inference class.

### Method
__init__
```

--------------------------------

### TFLiteExporter.__init__

Source: https://verisilicon.github.io/acuitylite/exporter.tflite.html

Initializes the TFLiteExporter with an AcuityModel object.

```APIDOC
__init__(_model_, _deep_copy=True_)
    Parameters:
        model – AcuityModel object.
```

--------------------------------

### load

Source: https://verisilicon.github.io/acuitylite/importer.darknetloader.html

Loads the Darknet model.

```APIDOC
## load()

### Description
Load Darknet model.

### Returns
Acuity network.
```

--------------------------------

### Generate TIM-VX Case with NBG

Source: https://verisilicon.github.io/acuitylite/README.html

Use TimVxExporter to generate TIM-VX cases and NBG files. Set pack_nbg_unify=True for default SDK. Provide viv_sdk and licence paths for custom SDKs. Ensure SDK directory structure is correct.

```python
TimVxExporter(model).export(pack_nbg_unify=True)
```

```python
TimVxExporter(model).export(pack_nbg_unify=True, viv_sdk=your_sdk_path, licence=path_of_licence_txt)
```

--------------------------------

### Run Darknet AlexNet Demo

Source: https://verisilicon.github.io/acuitylite/_sources/demo_darknet.rst.txt

Executes the AlexNet model using Darknet. This snippet is a direct inclusion of a Python script for demonstration purposes.

```python
import sys
import os

# Add the parent directory to the Python path
current_dir = os.path.dirname(os.path.abspath(__file__))
sys.path.append(os.path.dirname(current_dir))

from darknet import * 

# Load the network configuration and weights
net = load_net(b"cfg/alexnet.cfg", b"darknet19.weights", 0)
meta = load_meta(b"cfg/imagenet1000.data")

# Perform detection on an image
r = detect(net, meta, b"data/dog.jpg")

# Print the detection results
print(r)

# Clean up network resources
free_net(net)

```

--------------------------------

### Generate NBG with OvxlibExporter

Source: https://verisilicon.github.io/acuitylite/README.html

Use OvxlibExporter to generate NBG files by setting pack_nbg_only=True. Provide viv_sdk and licence paths for custom SDKs. Ensure SDK directory structure is correct.

```python
OvxlibExporter(model).export(pack_nbg_only=True)
```

```python
OvxlibExporter(model).export(pack_nbg_only=True, viv_sdk=your_sdk_path, licence=path_of_licence_txt)
```

--------------------------------

### Build Inference Session

Source: https://verisilicon.github.io/acuitylite/_sources/inference.rst.txt

Builds a new inference session. This method prepares the environment for running inference.

```APIDOC
## build_session

### Description
Builds a new inference session.

### Method
build_session
```

--------------------------------

### Quantization Module

Source: https://verisilicon.github.io/acuitylite/_sources/api.rst.txt

Contains functionality for model quantization.

```APIDOC
## Quantization Class

### Quantization

Provides methods for quantizing models.
```

--------------------------------

### TensorFlow Model Quantization and Inference

Source: https://verisilicon.github.io/acuitylite/demo_tensorflow.html

This script loads a TensorFlow model, quantizes it using asymmetric affine quantization, performs inference, and asserts the output. It requires a TensorFlow model file and image data for preprocessing.

```python
#!/usr/bin/env python3
import tensorflow as tf
import numpy as np
from acuitylib.interface.importer import TensorflowLoader
from acuitylib.interface.exporter import TFLiteExporter
from acuitylib.interface.quantization import Quantization, QuantizerType
from acuitylib.interface.inference import Inference


# prepare
mobilenet = "model/mobilenet_v1.pb"
image0 = "data/697.jpg"
image1 = "data/813.jpg"
labels = [697, 813]


# data generator
def get_data():
    for image in [image0, image1]:
        arr = tf.io.decode_image(tf.io.read_file(image)).numpy().reshape(1, 224, 224, 3).astype(np.float32)
        arr = (arr-128.0)/128.0  # preprocess
        inputs = {'input': np.array(arr, dtype=np.float32)}
        yield inputs


def test_tensorflow_mobilenet():
    # load tensorflow model
    model = TensorflowLoader(mobilenet).load(inputs=['input'], input_size_list=[[1, 224, 224, 3]],
                                             outputs=['MobilenetV1/Logits/SpatialSqueeze'])

    # quantization
    quantizer = QuantizerType.ASYMMETRIC_AFFINE
    qtype = 'uint8'
    Quantization(model).quantize(input_generator_func=get_data, quantizer=quantizer, qtype=qtype, iterations=1)

    # inference with quant model
    infer = Inference(model)
    infer.build_session()  # build inference session
    for i, data in enumerate(get_data()):
        ins, outs = infer.run_session(data)  # run inference session
        assert outs[0].flatten()[labels[i]] > 0.99

    # export tflite quant case
    TFLiteExporter(model).export('export_tflite/asymu8/mobilenet.tflite')


if __name__ == '__main__':
    test_tensorflow_mobilenet()

```

--------------------------------

### Load Darknet Model, Infer, Quantize, and Export to TFLite

Source: https://verisilicon.github.io/acuitylite/demo_darknet.html

This script loads a Darknet model, performs inference, applies per-channel INT8 quantization, runs inference again with the quantized model, and finally exports it to TFLite format. Ensure the model and weight files, along with the input image, are correctly specified.

```python
#!/usr/bin/env python3
import tensorflow as tf
import numpy as np
from acuitylib.interface.importer import DarknetLoader
from acuitylib.interface.exporter import TFLiteExporter
from acuitylib.interface.quantization import Quantization, QuantizerType
from acuitylib.interface.inference import Inference


alexnet_model = "model/alexnet2.cfg"
alexnet_weights = "model/alexnet2.weights"
image0 = "data/space_shuttle_227x227.jpg"


# data generator
def get_data():
    for image in [image0]:
        arr = tf.io.decode_image(tf.io.read_file(image)).numpy().reshape(1, 3, 227, 227).astype(np.float32)
        arr = (arr-128.0)/128.0  # preprocess
        inputs = {'input': np.array(arr, dtype=np.float32)}
        yield inputs


def test_darknet_alexnet():
    # load darknet model
    model = DarknetLoader(model=alexnet_model, weights=alexnet_weights).load()

    # inference
    infer = Inference(model)
    infer.build_session()  # build inference session
    for i, data in enumerate(get_data()):
        ins, outs = infer.run_session(data)  # run inference session
        print(outs[0])

    # perchanneli8 quantization
    quantizer = QuantizerType.PERCHANNEL_SYMMETRIC_AFFINE
    qtype = 'int8'
    Quantization(model).quantize(input_generator_func=get_data, quantizer=quantizer, qtype=qtype, iterations=1)

    # inference with quantized model
    infer = Inference(model)
    infer.build_session()  # build inference session
    for i, data in enumerate(get_data()):
        ins, outs = infer.run_session(data)  # run inference session
        print(outs[0])

    # export tflite perchanneli8 case
    TFLiteExporter(model).export('export_tflite/pcq_symi8/alexnet.tflite')


if __name__ == '__main__':
    test_darknet_alexnet()

```

--------------------------------

### TensorflowLoader

Source: https://verisilicon.github.io/acuitylite/importer.tensorflowloader.html

Initializes the TensorflowLoader with a Tensorflow model file.

```APIDOC
## class TensorflowLoader(_model_)

Translates a Tensorflow model to Acuity formats.

### __init__(_model_)

Parameters:
    model (str): The file path of the imported Tensorflow model file, a .pb file.
```

--------------------------------

### load Method

Source: https://verisilicon.github.io/acuitylite/importer.onnxloader.html

Loads an ONNX model, allowing for specification of inputs, input sizes, input data types, and outputs.

```APIDOC
load(_inputs =None_, _input_size_list =None_, _input_dtype_list =None_, _outputs =None_, _remove_io_perm =False_, _compute_arch =0_)
    Load Onnx model.

    Parameters:
        inputs (list, optional): Specify the input points of the TFLite graph. Defaults to None, using all header points.
        input_size_list (list, optional): The tensor shape sizes of the input points. Defaults to None.
        input_dtype_list (list, optional): Specify the data type of input tensors if not float. Defaults to None.
        outputs (list, optional): Specify the output points of the Onnx graph. Defaults to None, using all tail points.
        remove_io_perm (bool, optional): Whether to remove I/O permissions. Defaults to False.
        compute_arch (int, optional): Compute architecture specification. Defaults to 0.

    Returns:
        Acuity network.
```

--------------------------------

### load Method

Source: https://verisilicon.github.io/acuitylite/importer.caffeloader.html

Loads the Caffe model using the specified protocol.

```APIDOC
## load(proto='caffe')

### Description
Load Caffe model.

### Parameters
* **proto** (string) - Optional - The protocol used by the CaffeLoader, ‘caffe’ for the standard Caffe format protocal, ‘lstm_caffe’ for the LSTM layer protocal.

### Returns
Acuity network.
```

--------------------------------

### load

Source: https://verisilicon.github.io/acuitylite/_sources/importer.tensorflowloader.rst.txt

Loads a TensorFlow model.

```APIDOC
## load

### Description
Loads a TensorFlow model.

### Method
load
```

--------------------------------

### load

Source: https://verisilicon.github.io/acuitylite/importer.tfliteloader.html

Loads the TFLite model, optionally specifying input and output points and their sizes.

```APIDOC
## load(_inputs =None_, _input_size_list =None_, _outputs =None_, _** kwargs_)

### Description
Load TFlite model.

### Parameters
* **inputs** (list) - Optional - Specify the input points of the TFLite graph, requires list type. Such as [‘input_point_1’, ‘input_point_2’]. When this argument is omitted, Acuity uses all header points of this model.
* **input_size_list** (list) - Optional - The tensor shape sizes of the input points listed in the inputs argument, requires list type. For example, [[1,3,224,224],[1,12,1]], which represents the tensor shape sizes of two input points.
* **outputs** (list) - Optional - Specify the output points of the TFLite graph, requires list type. For example, [‘output_point_1’, ‘output_point_2’]. When this argument is omitted, Acuity uses all tail points of this model.

### Returns
Acuity network.
```

--------------------------------

### run_session

Source: https://verisilicon.github.io/acuitylite/inference.html

Runs an inference session with raw data. Accepts inputs as a list or a dictionary.

```APIDOC
## run_session

### Description
Run an inference session with raw data.

### Method
`run_session(inputs: list | dict, return_quantized_data=False)`

### Parameters
#### Request Body
- **inputs** (list | dict) - Required - A list with the Acuity model inputs order, or a dict where keys are layer names and values are numpy arrays.
- **return_quantized_data** (boolean) - Optional - If true, returns quantized data. Defaults to False.

### Response
#### Success Response (200)
- **result** (list) - A list containing tuples of inputs and outputs, e.g., [(input0, input1), (output0, output1)].
```

--------------------------------

### build_session

Source: https://verisilicon.github.io/acuitylite/inference.html

Builds an inference session for running a graph with raw data. Defaults to the 'tensorflow' backend.

```APIDOC
## build_session

### Description
Build an inference session for running graph with raw data.

### Method
`build_session(backend='tensorflow')`

### Parameters
#### Query Parameters
- **backend** (string) - Optional - Specifies the backend to use for the inference session (e.g., 'tensorflow'). Defaults to 'tensorflow'.
```

--------------------------------

### load

Source: https://verisilicon.github.io/acuitylite/importer.tensorflowloader.html

Loads a Tensorflow model, specifying inputs, input sizes, and outputs.

```APIDOC
## load(_inputs_ , _input_size_list_ , _outputs_ , _** kwargs_)

Load Tensorflow model.

Parameters:
  inputs (list): Specify the input points of the Tensorflow graph. For example, ['input_point_1', 'input_point_2'].
  input_size_list (list): The tensor shape sizes of the input points listed in the inputs argument. For example, [[1,224,224,3],[1,12,1]].
  outputs (list): Specify the output points of the Tensorflow graph. For example, ['output_point_1', 'output_point_2'].

Returns:
    Acuity network.
```

--------------------------------

### Load, Infer, and Export Caffe Model

Source: https://verisilicon.github.io/acuitylite/demo_caffe.html

This script loads a Caffe model, performs float inference, exports to Tim-VX and TFLite, quantizes the model to int8, performs quantized inference, and exports again.

```python
#!/usr/bin/env python3
import tensorflow as tf
import numpy as np
from acuitylib.interface.importer import CaffeLoader
from acuitylib.interface.exporter import TimVxExporter, TFLiteExporter
from acuitylib.interface.quantization import Quantization, QuantizerType
from acuitylib.interface.inference import Inference


prototxt = "model/lenet.prototxt"
caffe_model = "model/lenet.caffemodel"
image0 = "data/0.jpg"
image1 = "data/1.png"


# data generator
def get_data():
    for image in [image0, image1]:
        arr = tf.io.decode_image(tf.io.read_file(image)).numpy().reshape(1, 1, 28, 28).astype(np.float32)
        # arr = arr/255.0  # preprocess
        inputs = {'input': np.array(arr, dtype=np.float32)}
        yield inputs


def test_caffe_lenet():
    # load caffe model
    model = CaffeLoader(model=prototxt, weights=caffe_model).load()

    # inference with float model
    infer = Inference(model)
    infer.build_session()  # build inference session
    for i, data in enumerate(get_data()):
        ins, outs = infer.run_session(data)  # run inference session
        assert outs[0].flatten()[i] > 0.99

    # export tim-vx float16 case
    TimVxExporter(model).export('export_timvx/float16/lenet')
    # export tflite float16 case
    TFLiteExporter(model).export('export_tflite/float16/lenet.tflite')

    # perchanneli8 quantization
    quantizer = QuantizerType.PERCHANNEL_SYMMETRIC_AFFINE
    qtype = 'int8'
    Quantization(model).quantize(input_generator_func=get_data, quantizer=quantizer, qtype=qtype, iterations=1)

    # inference with quantized model
    infer = Inference(model)
    infer.build_session()  # build inference session
    for i, data in enumerate(get_data()):
        ins, outs = infer.run_session(data)  # run inference session
        assert outs[0].flatten()[i] > 0.99

    # export tim-vx perchanneli8 case
    TimVxExporter(model).export('export_timvx/pcq_symi8/lenet')
    # export tflite perchanneli8 case
    TFLiteExporter(model).export('export_tflite/pcq_symi8/lenet.tflite')


if __name__ == '__main__':
    test_caffe_lenet()

```

--------------------------------

### Quantization Class

Source: https://verisilicon.github.io/acuitylite/quantization.html

The Quantization class is used for quantizing models. It has a constructor and a quantize method.

```APIDOC
## class Quantization

### Description
Represents the quantization process for a model.

### Methods
#### __init__(model)
Initializes the Quantization object.

* **model**: The model to be quantized.

#### quantize(input_generator_func, quantizer=QuantizerType.ASYMMETRIC_AFFINE, qtype='uint8', iteration=1, minimize_layer_error=False)
Performs the quantization process on the model.

**Note**: Usually 500~1000 pictures are needed. Do not perform quantization on acuity networks converted from TensorFlow Lite models that have been quantized.

### Parameters
* **input_generator_func** (function) - Required - A function that generates input data.
* **quantizer** (QuantizerType) - Optional - The quantizer to use. Supports QuantizerType.ASYMMETRIC_AFFINE and QuantizerType.PERCHANNEL_SYMMETRIC_AFFINE.
* **qtype** (string) - Optional - The quantization type. Supports 'uint8' for ASYMMETRIC_AFFINE and 'int8' for PERCHANNEL_SYMMETRIC_AFFINE.
* **iteration** (int) - Optional - The number of sample image batches. Defaults to 1.
* **minimize_layer_error** (bool) - Optional - If True, minimizes quantization error, improving accuracy but increasing processing time. Defaults to False.
```

--------------------------------

### Load, Infer, Quantize, and Export ONNX Model

Source: https://verisilicon.github.io/acuitylite/demo_onnx.html

This script loads an ONNX model, performs inference, quantizes it, and exports it to Tim-VX and TFLite formats. It requires TensorFlow and NumPy. Ensure the model and data paths are correctly set.

```python
#!/usr/bin/env python3
import tensorflow as tf
import numpy as np
from acuitylib.interface.importer import OnnxLoader
from acuitylib.interface.exporter import TimVxExporter
from acuitylib.interface.exporter import TFLiteExporter
from acuitylib.interface.quantization import Quantization, QuantizerType
from acuitylib.interface.inference import Inference


# prepare
onnx_model = "model/squeezenet.onnx"
image0 = "data/813.jpg"


# data generator
def get_data():
    for image in [image0]:
        arr = tf.io.decode_image(tf.io.read_file(image)).numpy()
        arr = np.transpose(arr, [2, 0, 1])
        arr = arr.reshape(1, 3, 224, 224)
        inputs = {'data_0': np.array(arr, dtype=np.float32)}
        yield inputs


def test_onnx_squeezenet():
    # import model
    model = OnnxLoader(model=onnx_model).load(inputs=['data_0'], input_size_list=[[1, 3, 224, 224]],
                                              outputs=['softmaxout_1'])
    # inference
    infer = Inference(model)
    infer.build_session()  # build inference session
    for i, data in enumerate(get_data()):
        ins, outs = infer.run_session(data)  # run inference session
        assert np.argmax(outs[0].flatten()) == 812

    TimVxExporter(model).export('export_timvx/float16/lenet')

    # quantize
    quantizer = QuantizerType.ASYMMETRIC_AFFINE
    qtype = 'uint8'
    Quantization(model).quantize(input_generator_func=get_data, quantizer=quantizer, qtype=qtype, iterations=1)

    # inference with quantized model
    infer = Inference(model)
    infer.build_session()  # build inference session
    for i, data in enumerate(get_data()):
        ins, outs = infer.run_session(data)  # run inference session
        assert np.argmax(outs[0].flatten()) == 812

    # export tim-vx case
    TimVxExporter(model).export('export_timvx/asymu8/squeezenet')
    TFLiteExporter(model).export('export_tflite/asymu8/squeezenet.tflite')


if __name__ == '__main__':
    test_onnx_squeezenet()

```

--------------------------------

### Quantization Class

Source: https://verisilicon.github.io/acuitylite/_sources/quantization.rst.txt

The Quantization class provides methods for quantizing models. The `quantize` method is used to perform the quantization process.

```APIDOC
## Quantization Class

### Description
Provides methods for quantizing models.

### Methods

- `__init__(model)`: Constructor for the Quantization class.
- `quantize(input_generator_func, quantizer=QuantizerType.ASYMMETRIC_AFFINE, qtype='uint8', iteration=1, minimize_layer_error=False)`: Quantizes the model.

    Usually 500~1000 pictures are needed. Do not perform quantization on acuity networks converted from TensorFlow Lite models that have been quantized.

    #### Parameters

    - **input_generator_func** (function) - Required - A function that generates input data.
    - **quantizer** (QuantizerType) - Optional - The quantizer to use. Defaults to `QuantizerType.ASYMMETRIC_AFFINE`. Supports `QuantizerType.ASYMMETRIC_AFFINE` and `QuantizerType.PERCHANNEL_SYMMETRIC_AFFINE`.
    - **qtype** (string) - Optional - The quantization type. Defaults to 'uint8'. Supports 'uint8' for ASYMMETRIC_AFFINE and 'int8' for PERCHANNEL_SYMMETRIC_AFFINE.
    - **iteration** (int) - Optional - The number of sample image batches. Defaults to 1.
    - **minimize_layer_error** (bool) - Optional - If True, minimizes quantization error, improving accuracy but increasing processing time. Defaults to False.

## QuantizerType Enum

### Description
Enumeration for quantizer types.

### Members

- `ASYMMETRIC_AFFINE`
- `PERCHANNEL_SYMMETRIC_AFFINE`
```

--------------------------------

### OvxlibExporter

Source: https://verisilicon.github.io/acuitylite/exporter.ovxlib.html

Initializes the OvxlibExporter. This class is used to export an Acuity model into an Ovxlib application.

```APIDOC
## __init__

### Description
Initializes the OvxlibExporter.

### Parameters
* **model** (AcuityModel) - The AcuityModel object to export.
* **deep_copy** (bool, optional) - Whether to perform a deep copy of the model. Defaults to True.
```

--------------------------------

### OvxlibExporter Class

Source: https://verisilicon.github.io/acuitylite/_sources/exporter.ovxlib.rst.txt

The OvxlibExporter class provides functionality to export data using the Ovxlib format. It includes initialization and an export method.

```APIDOC
## Class: OvxlibExporter

### Description
Provides methods for exporting data in Ovxlib format.

### Methods

#### `__init__`

##### Description
Initializes the OvxlibExporter.

#### `export`

##### Description
Exports data using the Ovxlib format.

##### Parameters
(No parameters are explicitly documented in the source for this method.)

##### Request Example
(No request example is explicitly documented in the source.)

##### Response
(No response details are explicitly documented in the source.)
```

--------------------------------

### OnnxLoader Class

Source: https://verisilicon.github.io/acuitylite/_sources/importer.onnxloader.rst.txt

The OnnxLoader class provides functionality to load ONNX models. It includes a constructor for initialization and a method to perform the loading operation.

```APIDOC
## Class: OnnxLoader

### Description
Provides functionality to load ONNX models.

### Methods

#### `__init__`

##### Description
Initializes the OnnxLoader object.

#### `load`

##### Description
Loads the ONNX model from a specified source.

##### Parameters

* **source** (str) - Required - The path or identifier for the ONNX model to load.

##### Returns

* (object) - The loaded ONNX model object.

```

--------------------------------

### Importer Functions

Source: https://verisilicon.github.io/acuitylite/api.html

Functions for loading and translating models from various frameworks into Acuity formats.

```APIDOC
## Importer Functions

### `importer.CaffeLoader(model[, weights])`
Translates a Caffe model to Acuity formats.

### `importer.DarknetLoader(model, weights)`
Translates a Darknet model to Acuity formats.

### `importer.OnnxLoader(model)`
Translates a Onnx model to Acuity formats.

### `importer.TensorflowLoader(model)`
Translates a Tensorflow model to Acuity formats.

### `importer.TFLiteLoader(model)`
Translates a TensorFlow Lite model to ACUITY formats.
```

--------------------------------

### TFLiteLoader

Source: https://verisilicon.github.io/acuitylite/importer.tfliteloader.html

Initializes the TFLiteLoader with the path to a TensorFlow Lite model.

```APIDOC
## class TFLiteLoader(model)

### Description
Translates a TensorFlow Lite model to ACUITY formats. For quantized TensorFlow Lite models, do not quantize them again in Acuity. if set inputs param, need to use input_size_list parameter at the same time.

### Parameters
* **model** (string) - Required - The file path of the imported TensorFlow Lite (TFLite) model, in an optimized FlatBuffer format identified by the .tflite file extension.
```

--------------------------------

### Run Inference Session

Source: https://verisilicon.github.io/acuitylite/_sources/inference.rst.txt

Executes the inference session. This method runs the actual inference process.

```APIDOC
## run_session

### Description
Executes the inference session.

### Method
run_session
```

--------------------------------

### Importer Modules

Source: https://verisilicon.github.io/acuitylite/_sources/api.rst.txt

Provides classes for loading models from various frameworks like Caffe, Darknet, ONNX, TensorFlow, and TFLite.

```APIDOC
## Importer Classes

### CaffeLoader

Loads models from Caffe format.

### DarknetLoader

Loads models from Darknet format.

### OnnxLoader

Loads models from ONNX format.

### TensorflowLoader

Loads models from TensorFlow format.

### TFLiteLoader

Loads models from TFLite format.
```

--------------------------------

### TimVxExporter Class

Source: https://verisilicon.github.io/acuitylite/exporter.tim-vx.html

Initializes the TIM-VX Exporter. This class is used to export Acuity models into TIM-VX applications.

```APIDOC
## Class: TimVxExporter

### Description
Exports an Acuity model into an TIM-VX application. Reference: https://github.com/VeriSilicon/TIM-VX

### Methods
#### __init__(model, deep_copy=True)

##### Parameters
* **model** (AcuityModel) - AcuityModel object.
* **deep_copy** (bool) - Optional. Defaults to True.
```

--------------------------------

### Test TFLite Quantized Model with Acuity Lite

Source: https://verisilicon.github.io/acuitylite/_sources/demo_tflite.rst.txt

This Python script is used to test a TFLite quantized model. It assumes the model is already quantized and demonstrates its usage without further quantization steps via Acuity Lite.

```python
import numpy as np
import tflite_runtime.interpreter as tflite

# Load TFLite model and allocate tensors.
interpreter = tflite.Interpreter(model_path="mobilenet_v1_1.0_224_quant.tflite")
interpreter.allocate_tensors()

# Get input and output tensors.
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()

# create some dummy input data
input_shape = input_details[0]['shape']
input_data = np.array(np.random.random_sample(input_shape), dtype=np.float32)

# Run inference.
interpreter.set_tensor(input_details[0]['index'], input_data)
interpreter.invoke()

# get output data
output_data = interpreter.get_tensor(output_details[0]['index'])

print(output_data)
```

--------------------------------

### OnnxLoader Class

Source: https://verisilicon.github.io/acuitylite/importer.onnxloader.html

The OnnxLoader class is used to translate ONNX models into Acuity formats.

```APIDOC
class OnnxLoader(_model_)
    Translates a Onnx model to Acuity formats.

    __init__(_model_)
        Parameters:
            model (str): The file path of the imported Onnx model file, a .onnx file.
```

--------------------------------

### TFLite Quantized Model Inference and Export

Source: https://verisilicon.github.io/acuitylite/demo_tflite.html

Loads a TFLite quantized model, performs inference on sample images, and exports the model for Tim-VX and NBG formats. Ensure the model is already quantized as AcuityLite does not perform quantization on quantized models.

```python
#!/usr/bin/env python3
import tensorflow as tf
import numpy as np
from acuitylib.interface.importer import TFLiteLoader
from acuitylib.interface.exporter import TimVxExporter
from acuitylib.interface.exporter import OvxlibExporter
from acuitylib.interface.inference import Inference

# wget https://storage.googleapis.com/download.tensorflow.org//
# models/mobilenet_v1_2018_08_02/mobilenet_v1_1.0_224_quant.tgz


mobilenet = "model/mobilenet_v1_1.0_224_quant.tflite"
image0 = "data/697.jpg"
image1 = "data/813.jpg"
labels = [697, 813]


# data generator
def get_data():
    for image in [image0, image1]:
        arr = tf.io.decode_image(tf.io.read_file(image)).numpy().reshape(1, 224, 224, 3).astype(np.float32)
        arr = (arr-128.0)/128.0  # preprocess
        arr = np.rint(arr / 0.0078125) + 128  # quantize arr as quantized input for quantized model(begin from 6.27.0)
        inputs = {'input': np.array(arr, dtype=np.uint8)}
        yield inputs


def test_tflite_mobilenet():
    # no need to quantize using acuity lite for quant model
    # load tflite quant model
    quantmodel = TFLiteLoader(mobilenet).load()

    # inference with quant model
    infer = Inference(quantmodel)
    infer.build_session()  # build inference session
    for i, data in enumerate(get_data()):
        ins, outs = infer.run_session(data)  # run inference session
        assert outs[0].flatten()[labels[i]] > 0.9

    # export tim-vx quant case
    TimVxExporter(quantmodel).export('export_timvx/quant/mobilenet')

    # export nbg
    OvxlibExporter(quantmodel).export('export_ovxlib/quant/mobilenet', pack_nbg_only=True)


if __name__ == '__main__':
    test_tflite_mobilenet()

```

--------------------------------

### DarknetLoader Class

Source: https://verisilicon.github.io/acuitylite/_sources/importer.darknetloader.rst.txt

The DarknetLoader class provides functionality to load darknet models. It includes a constructor for initialization and a method for loading model data.

```APIDOC
## Class: DarknetLoader

### Description
Provides functionality to load darknet models.

### Methods

#### `__init__`

##### Description
Initializes the DarknetLoader.

#### `load`

##### Description
Loads the darknet model from a specified path.

##### Parameters

* **path** (str) - Required - The file path to the darknet model configuration.
* **weights** (str) - Required - The file path to the darknet model weights.
* **names** (str) - Required - The file path to the darknet model class names.

##### Returns

* Returns the loaded darknet model object.
```

--------------------------------

### TFLiteExporter.export

Source: https://verisilicon.github.io/acuitylite/exporter.tflite.html

Exports the TFLite application to a specified output directory with a given prefix.

```APIDOC
export(_output_, _**kwargs_)
    Export TFLite application.
    Parameters:
        output – The directory of the exported applications with the prefix specified. Use the syntax ‘<output directory>/<prefix>’. Where: Output directory: The directory of the generated outputs including subdirectories and files. Prefix: The prefix of the generated subdirectories and files. For example, “export_tflite/uint8/lenet”
    Returns:
```

--------------------------------

### Inference Module

Source: https://verisilicon.github.io/acuitylite/_sources/api.rst.txt

Enables model inference.

```APIDOC
## Inference Class

### Inference

Provides methods for performing inference with models.
```

--------------------------------

### Quantization Functions

Source: https://verisilicon.github.io/acuitylite/api.html

Functions related to model quantization.

```APIDOC
## Quantization Functions

### `quantization.Quantization(model)`

### `quantization`
```

--------------------------------

### CaffeLoader Class

Source: https://verisilicon.github.io/acuitylite/_sources/importer.caffeloader.rst.txt

The CaffeLoader class provides functionality to load Caffe models. It includes methods for initialization and loading model data.

```APIDOC
## Class: CaffeLoader

### Description
Provides functionality to load Caffe models.

### Methods

#### `__init__`

##### Description
Initializes the CaffeLoader object.

#### `load`

##### Description
Loads the Caffe model from a specified path.

##### Parameters

###### Path Parameters
- **model_path** (string) - Required - The file path to the Caffe model.
- **weights_path** (string) - Required - The file path to the Caffe model weights.
```

--------------------------------

### TimVxExporter Class

Source: https://verisilicon.github.io/acuitylite/_sources/exporter.tim-vx.rst.txt

The TimVxExporter class is used for exporting data. It has an initializer and an export method.

```APIDOC
## Class: TimVxExporter

### Description
Provides functionality for exporting data.

### Methods

#### `__init__`

Initializes the TimVxExporter object.

#### `export`

Exports data using the TimVxExporter.

```

--------------------------------

### Exporter Modules

Source: https://verisilicon.github.io/acuitylite/_sources/api.rst.txt

Provides classes for exporting models to TFLite and Tim-VX formats.

```APIDOC
## Exporter Classes

### TFLiteExporter

Exports models to TFLite format.

### TimVxExporter

Exports models to Tim-VX format.
```

--------------------------------

### MobileNet TensorFlow Model Test

Source: https://verisilicon.github.io/acuitylite/_sources/demo_tensorflow.rst.txt

This snippet contains the full Python code for testing a MobileNet model with TensorFlow. It includes imports, model loading, and inference logic.

```python
import tensorflow as tf
from tensorflow.keras.applications.mobilenet import MobileNet, preprocess_input, decode_predictions
from PIL import Image
import numpy as np

def run_mobilenet_tf_test():
    # Load the MobileNet model pre-trained on ImageNet
    model = MobileNet(weights='imagenet')

    # Load an example image
    # Replace 'path/to/your/image.jpg' with the actual path to an image file
    try:
        img_path = 'cat.jpg' # Example image path
        img = Image.open(img_path)
        img = img.resize((224, 224))
        img_array = np.array(img).reshape(1, 224, 224, 3)

        # Preprocess the image for the MobileNet model
        processed_img_array = preprocess_input(img_array)

        # Make predictions
        predictions = model.predict(processed_img_array)

        # Decode the predictions into human-readable labels
        decoded_predictions = decode_predictions(predictions, top=3)[0]

        print("Top 3 predictions:")
        for i, (imagenet_id, label, score) in enumerate(decoded_predictions):
            print(f"{i + 1}: {label} ({score:.2f})")

    except FileNotFoundError:
        print(f"Error: Image file not found at {img_path}. Please provide a valid image path.")
    except Exception as e:
        print(f"An error occurred: {e}")

if __name__ == '__main__':
    run_mobilenet_tf_test()

```

--------------------------------

### Inference Functions

Source: https://verisilicon.github.io/acuitylite/api.html

Functions for performing model inference.

```APIDOC
## Inference Functions

### `inference.Inference(model)`

### `inference`
```

--------------------------------

### TFLiteLoader Class

Source: https://verisilicon.github.io/acuitylite/_sources/importer.tfliteloader.rst.txt

The TFLiteLoader class provides functionality to load TensorFlow Lite models.

```APIDOC
## Class: TFLiteLoader

This class is responsible for loading TensorFlow Lite models.

### Methods

- `__init__()`: Initializes the TFLiteLoader.
- `load(model_path)`: Loads a TensorFlow Lite model from the specified path.
```

--------------------------------

### Exporter Functions

Source: https://verisilicon.github.io/acuitylite/api.html

Functions for exporting Acuity models to different application formats.

```APIDOC
## Exporter Functions

### `exporter.TFLiteExporter(model[, deep_copy])`
Export an Acuity model into an TFLite application.

### `exporter.TimVxExporter(model[, deep_copy])`
Export an Acuity model into an TIM-VX application.
```

--------------------------------

### TFLiteExporter Class

Source: https://verisilicon.github.io/acuitylite/_sources/exporter.tflite.rst.txt

The TFLiteExporter class is used for exporting models to the TFLite format. It provides methods for initialization and the actual export process.

```APIDOC
## Class: TFLiteExporter

### Description
Provides functionality to export models to the TFLite format.

### Methods

#### `__init__`

##### Description
Initializes the TFLiteExporter.

#### `export`

##### Description
Exports the model to the TFLite format.

##### Parameters
* **model**: The model to export.
* **output_path**: The path where the TFLite model will be saved.

##### Returns
* None
```

--------------------------------

### TFLiteExporter Class

Source: https://verisilicon.github.io/acuitylite/exporter.tflite.html

The TFLiteExporter class is used to export an Acuity model into a TFLite application. It references the TensorFlow Lite library.

```APIDOC
class TFLiteExporter(_model_, _deep_copy=True_)
    Export an Acuity model into an TFLite application. reference: https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite
```
Tokens:
Raw