### Larq QuantConv2D with DoReFa Quantizer Example Source: https://wiki.st.com/stm32mcu/wiki/AI%3ADeep_Quantized_Neural_Network_support Example of using the QuantConv2D layer from the Larq library with the DoReFa quantizer for both activations and weights. This snippet demonstrates a typical Keras/TensorFlow setup for quantized convolutional layers. ```python x = tf.keras.Input(shape=(..)) y = larq.layers.QuantConv2D(.., input_quantizer=larq.Dorefa(k_bits=8, mode="activations", kernel_quantizer=larq.Dorefa(k_bits=8, mode="weights", use_bias=False, )(x) ... ``` -------------------------------- ### QKeras Fully Binarized Network with Binary Quantizer Source: https://wiki.st.com/stm32mcu/wiki/AI%3ADeep_Quantized_Neural_Network_support Shows how to implement a fully binarized network using QKeras's binary quantizer for activations and convolutional layers. This example includes settings for kernel and bias quantizers, disabling bias, and applying pooling and batch normalization. ```python ... y = qkeras.QActivation(qkeras.binary(alpha=1))(y) y = qkeras.QConv2D( kernel_quantizer="binary(alpha=1)", bias_quantizer=qkeras.binary(alpha=1), use_bias=False, )(y) y = keras.MaxPooling2D(...)(y) y = keras.BatchNormalization()(y) y = qkeras.QActivation(qkeras.binary(alpha=1))(y) ... ``` -------------------------------- ### Example of Consolidated Classification Output Source: https://wiki.st.com/stm32mcu/wiki/AI%3ADatalogging_guidelines_for_a_successful_NanoEdge_AI_project Illustrates how to consolidate multiple sequential classification inferences to produce a reliable final result. This method helps mitigate false positives by requiring a consistent output over a short period. ```text rock rock [...x7] rock ``` ```text rock rock [...x5] paper rock rock [...x6] paper ``` -------------------------------- ### Quantized Dense Layer with 8-bit Input and Binary Weights (TensorFlow/QKeras) Source: https://wiki.st.com/stm32mcu/wiki/AI%3ADeep_Quantized_Neural_Network_support This example demonstrates a QDense layer with 8-bit quantized input and 1-bit quantized (binary) weights. It includes an optional QActivation for input format, a QDense layer, BatchNormalization, and a final Activation. The output format is f32. ```python x = tf.keras.Input(shape=shape_in) y = qkeras.QActivation(qkeras.quantized_bits(bits=8, integer=7))(x) y = tf.keras.layers.Flatten()(y) y = qkeras.QDense(16, kernel_quantizer=qkeras.binary(alpha=1), bias_quantizer=qkeras.binary(alpha=1), use_bias=True, name="dense_0")(y) y = tf.keras.layers.BatchNormalization()(y) y = tf.keras.layers.Activation("softmax")(y) model = tf.keras.Model(inputs=x, outputs=y) ``` -------------------------------- ### Sequential Quantized Dense Layers with Binary Weights (TensorFlow/QKeras) Source: https://wiki.st.com/stm32mcu/wiki/AI%3ADeep_Quantized_Neural_Network_support This example shows two sequential QDense layers. The first uses binary weights with an 8-bit quantized input to produce a binary output. The second layer uses the binary output to compute an f32 output. Both layers utilize binary weights and include BatchNormalization. ```python x = tf.keras.Input(shape=shape_in) y = tf.keras.layers.Flatten()(x) y = qkeras.QActivation(qkeras.quantized_bits(bits=8, integer=7))(y) y = qkeras.QDense(64, kernel_quantizer=qkeras.binary(alpha=1), bias_quantizer=qkeras.binary(alpha=1), use_bias=True, name="dense_0")(y) y = tf.keras.layers.BatchNormalization()(y) y = qkeras.QActivation(qkeras.binary(alpha=1))(y) y = qkeras.QDense(10, kernel_quantizer=qkeras.binary(alpha=1), bias_quantizer=qkeras.binary(alpha=1), use_bias=True, name="dense_1")(y) y = tf.keras.layers.Activation("softmax")(y) ``` -------------------------------- ### Quantized Convolution with Binary Input/Weight and MaxPooling2D (Keras) Source: https://wiki.st.com/stm32mcu/wiki/AI%3ADeep_Quantized_Neural_Network_support This example demonstrates a quantized 2D convolution layer using binary inputs and weights, followed by MaxPooling2D for activation compaction. It utilizes QKeras for quantization and TensorFlow Keras for model definition. The output is a normalized binary value. ```python x = tf.keras.Input(shape=shape_in) y = qkeras.QActivation(qkeras.binary(alpha=1))(x) y = qkeras.QConv2D(filters=16, kernel_size=(3, 3), kernel_quantizer=qkeras.binary(alpha=1), bias_quantizer=qkeras.binary(alpha=1), use_bias=False, padding="same", name="dense_0")(y) y = tf.keras.layers.MaxPooling2D(pool_size=(2, 2))(y) y = tf.keras.layers.BatchNormalization()(y) y = qkeras.QActivation(qkeras.binary(alpha=1))(y) model = tf.keras.Model(inputs=x, outputs=y) ``` -------------------------------- ### Quantized Convolution with Binary Input/Weight and Strides (Keras) Source: https://wiki.st.com/stm32mcu/wiki/AI%3ADeep_Quantized_Neural_Network_support This example shows a quantized 2D convolution layer with binary inputs and weights, using strides to potentially replace MaxPooling2D. It leverages QKeras for quantization and TensorFlow Keras for model construction. The output is a normalized binary value. ```python x = tf.keras.Input(shape=shape_in) y = qkeras.QActivation(qkeras.binary(alpha=1))(x) y = qkeras.QConv2D(filters=16, kernel_size=(3, 3), kernel_quantizer=qkeras.binary(alpha=1), bias_quantizer=qkeras.binary(alpha=1), use_bias=False, strides=(2, 2), padding="same", name="dense_0")(y) y = tf.keras.layers.BatchNormalization()(y) y = qkeras.QActivation(qkeras.binary(alpha=1))(y) model = tf.keras.Model(inputs=x, outputs=y) ``` -------------------------------- ### Fuse Activation Layer with QuantConv2D in TensorFlow/Keras Source: https://wiki.st.com/stm32mcu/wiki/AI%3ADeep_Quantized_Neural_Network_support This example demonstrates how TensorFlow/Keras code using QActivation and QConv2D can be represented equivalently from a code generation perspective using Larq's QuantConv2D with specific input quantizers. This fusion is a common pattern for optimizing activation layers. ```python x = tf.keras.Input(shape=(..)) y = qkeras.QActivation(qkeras.binary(alpha=1))(x) y = qkeras.QConv2D(..)(y) ... is equivalent to (code gen point of view): x = tf.keras.Input(shape=(..)) y = larq.layers.QuantConv2D(.., input_quantizer='ste_sign',..)(x) ... ``` -------------------------------- ### Create Quantized Model with QKeras Source: https://wiki.st.com/stm32mcu/wiki/AI%3ADeep_Quantized_Neural_Network_support This Python snippet demonstrates how to build a quantized Keras model using the QKeras library. It showcases the use of QActivation and QConv2D layers with binary quantization for kernel and bias, along with standard Keras layers like MaxPooling2D and BatchNormalization. The model is configured for 8-bit activations and binary weights/biases. ```python import tensorflow as tf import qkeras x = tf.keras.Input(shape=(28, 28, 1)) y = qkeras.QActivation(qkeras.quantized_relu(bits=8, alpha=1))(x) y = qkeras.QConv2D(16, (3, 3), kernel_quantizer=qkeras.binary(alpha=1), bias_quantizer=qkeras.binary(alpha=1), use_bias=False, name="conv2d_0")(y) y = tf.keras.layers.MaxPooling2D(pool_size=(2,2))(y) y = tf.keras.layers.BatchNormalization()(y) y = qkeras.QActivation(qkeras.binary(alpha=1))(y) model = tf.keras.Model(inputs=x, outputs=y) ``` -------------------------------- ### Quantized Convolution with Fallback to 32-bit Float (Keras) Source: https://wiki.st.com/stm32mcu/wiki/AI%3ADeep_Quantized_Neural_Network_support This example illustrates a scenario where a specific quantization configuration ('s8xs1->s8') is unsupported, leading to a fallback to 32-bit floating-point kernels. It uses QKeras for quantization and TensorFlow Keras for model definition, allowing users to manage precision loss by keeping layers in float. ```python x = tf.keras.Input(shape=shape_in) y = qkeras.QActivation(qkeras.quantized_bits(bits=8, integer=7))(x) y = qkeras.QConv2D(filters=16, kernel_size=(3, 3), strides=(2, 2), kernel_quantizer=qkeras.binary(alpha=1), bias_quantizer=qkeras.binary(alpha=1), use_bias=True, padding="same", name="dense_0")(y) y = tf.keras.layers.BatchNormalization()(y) y = qkeras.QActivation(qkeras.quantized_bits(bits=8, integer=7))(y) model = tf.keras.Model(inputs=x, outputs=y) ``` -------------------------------- ### Signal Capture Logic with Presence Detection (Pseudo-code) Source: https://wiki.st.com/stm32mcu/wiki/AI%3ADatalogging_guidelines_for_a_successful_NanoEdge_AI_project This pseudo-code demonstrates a loop that continuously checks for presence. When presence is detected, it collects a buffer and then waits until the presence is no longer detected before continuing the loop. This prevents continuous buffer collection and handles static presence. ```pseudo-code while (1) { if (presence) { collect_buffer(); do { wait_ms(1000); } while (presence); } } ``` -------------------------------- ### Analyze Model Operations and Memory with stm32ai Source: https://wiki.st.com/stm32mcu/wiki/AI%3ADeep_Quantized_Neural_Network_support This command-line output from stm32ai provides a comprehensive analysis of a Keras model. It details parameter counts, memory allocation for weights and activations, and a breakdown of operations per C-layer and by operation type, highlighting the usage of optimized kernels. ```shell $ stm32ai ... params #  : 93,556 items (365.45 KiB) macc  : 2,865,718 weights (ro)  : 14,496 B (14.16 KiB) (1 segment) / -359,728(-96.1%) vs float model activations (rw)  : 86,528 B (84.50 KiB) (1 segment) ram (total)  : 89,704 B (87.60 KiB) = 86,528 + 3,136 + 40 ... Number of operations and param per C-layer ------------------------------------------------------------------------------------------- c_id m_id name (type) #op (type) ------------------------------------------------------------------------------------------- 0 2 quant_conv2d_conv2d (conv2d) 194,720 (smul_f32_f32) 1 3 quant_conv2d_1_conv (conv) 43,264 (conv_f32_s1) 2 1 max_pooling2d (pool) 21,632 (op_s1_s1) 3 3 quant_conv2d_1_conv2d (conv2d_dqnn) 2,230,272 (sxor_s1_s1) 4 5 max_pooling2d_1 (pool) 6,400 (op_s1_s1) 5 7 quant_conv2d_2_conv2d (conv2d_dqnn) 331,776 (sxor_s1_s1) 6 10 quant_dense_quantdense (dense_dqnn_dqnn) 36,864 (sxor_s1_s1) 7 13 quant_dense_1_quantdense (dense_dqnn_dqnn) 640 (sxor_s1_s1) 8 15 activation (nl) 150 (op_f32_f32) ------------------------------------------------------------------------------------------- total 2,865,718 Number of operation types --------------------------------------------- smul_f32_f32 194,720 6.8% conv_f32_s1 43,264 1.5% op_s1_s1 28,032 1.0% sxor_s1_s1 2,599,552 90.7% op_f32_f32 150 0.0% ``` -------------------------------- ### QKeras Quantized ReLU Activation and Convolution Source: https://wiki.st.com/stm32mcu/wiki/AI%3ADeep_Quantized_Neural_Network_support Demonstrates the use of QKeras's quantized_relu quantizer for activations and QConv2D for convolutional layers. It specifies that quantized_relu with bits=8 and integer=0 is suitable for inputs normalized between 0.0 and 1.0. ```python x = tf.keras.Input(shape=(..)) y = qkeras.QActivation(qkeras.quantized_relu(bits=8, integer=0))(x) y = qkeras.QConv2D(..) ... ```