### List Return Example (Utility Functions)

Source: https://github.com/google/qkeras/blob/master/_autodocs/types.md

Certain utility functions within QKeras return lists. Examples include retrieving quantizers or prunable weights.

```python
quantizers: List[Quantizer] = layer.get_quantizers()
weights: List[tf.Variable] = layer.get_prunable_weights()
```

--------------------------------

### Example Usage of QConv2DTranspose

Source: https://github.com/google/qkeras/blob/master/_autodocs/api-reference/qconvolutional.md

Demonstrates how to instantiate a QConv2DTranspose layer with specific quantization and activation settings. This example is suitable for decoder layers in generative models.

```python
import qkeras as qk

# Upsampling layer for decoder
layer = qk.QConv2DTranspose(
    filters=64,
    kernel_size=4,
    strides=2,
    padding='same',
    kernel_quantizer='quantized_bits(8,4,1)',
    activation='relu'
)
```

--------------------------------

### Dictionary Return Example (Configuration)

Source: https://github.com/google/qkeras/blob/master/_autodocs/types.md

Configuration methods in QKeras return dictionaries. These are useful for inspecting or modifying layer settings.

```python
config: Dict = layer.get_config()
quant_config: Dict = layer.get_quantization_config()
```

--------------------------------

### Example Usage of QSeparableConv1D

Source: https://github.com/google/qkeras/blob/master/_autodocs/api-reference/qconvolutional.md

Demonstrates how to create an instance of the QSeparableConv1D layer with specific quantization settings and activation function. This example uses 8-bit quantization for both depthwise and pointwise convolutions with a ReLU activation.

```python
import qkeras as qk

layer = qk.QSeparableConv1D(
    filters=64,
    kernel_size=3,
    padding='same',
    depthwise_quantizer='quantized_bits(8,4,1)',
    pointwise_quantizer='quantized_bits(8,4,1)',
    activation='relu'
)
```

--------------------------------

### Example Usage of QDepthwiseConv2D

Source: https://github.com/google/qkeras/blob/master/_autodocs/api-reference/qconvolutional.md

Instantiates a QDepthwiseConv2D layer with quantization for efficient mobile inference. This example uses a 3x3 kernel and ReLU activation.

```python
import qkeras as qk

# Efficient mobile-friendly depthwise convolution
layer = qk.QDepthwiseConv2D(
    kernel_size=3,
    strides=1,
    padding='same',
    depth_multiplier=1,
    depthwise_quantizer='quantized_bits(8,4,1)',
    activation='relu'
)
```

--------------------------------

### Import Libraries and Setup

Source: https://github.com/google/qkeras/blob/master/notebook/QRNNTutorial.ipynb

Imports necessary libraries like TensorFlow, NumPy, and QKeras, and sets up TensorFlow version compatibility and warnings.

```python
import warnings
warnings.filterwarnings("ignore")
import tempfile

import numpy as np
import tensorflow.compat.v2 as tf
tf.enable_v2_behavior()

from tensorflow.keras.layers import Input, Dense, Embedding, SimpleRNN, GRU, LSTM, Bidirectional
from tensorflow.keras.optimizers import *
from tensorflow.keras.datasets import imdb
from tensorflow.keras.preprocessing import sequence

from qkeras.autoqkeras import *
from qkeras import *

print("using tensorflow", tf.__version__)
```

--------------------------------

### QAdaptiveActivation Example

Source: https://github.com/google/qkeras/blob/master/_autodocs/api-reference/qlayers.md

Demonstrates adaptive quantization with automatic range learning using QAdaptiveActivation. Shows how to use it in a Keras Sequential model.

```python
import qkeras as qk
import tensorflow as tf

# Adaptive quantization with automatic range learning
layer = qk.QAdaptiveActivation(
    activation='quantized_relu',
    total_bits=8,
    current_step=model.optimizer.iterations,
    quantization_delay=1000,
    ema_decay=0.9999,
    per_channel=True
)

# Use in model
model = tf.keras.Sequential([
    qk.QDense(64, kernel_quantizer='quantized_bits(8,4,1)')
    qk.QAdaptiveActivation('quantized_relu', total_bits=8,
                          current_step=None, quantization_delay=500),
])
```

--------------------------------

### Model Quantization Configuration Example

Source: https://github.com/google/qkeras/blob/master/notebook/QKerasTutorial.ipynb

An example of a quantization configuration dictionary used with the `model_quantize` function. This configuration specifies different quantization methods for convolutional layers, dense layers, and activations.

```python
config = {
  "conv2d_1": {
      "kernel_quantizer": "stochastic_binary",
      "bias_quantizer": "quantized_po2(4)"
  },
  "QConv2D": {
      "kernel_quantizer": "stochastic_ternary",
      "bias_quantizer": "quantized_po2(4)"
  },
  "QDense": {
      "kernel_quantizer": "quantized_bits(4,0,1)",
      "bias_quantizer": "quantized_bits(4)"
  },
  "QActivation": { "relu": "binary" },
  "act_2": "quantized_relu(3)",
}
```

--------------------------------

### QActivation Examples

Source: https://github.com/google/qkeras/blob/master/_autodocs/api-reference/qlayers.md

Demonstrates how to use QActivation with different quantizer configurations, including using quantizer strings and quantizer objects, and building a fully quantized network.

```python
import tensorflow as tf
from tensorflow import keras
import qkeras as qk

# Using quantizer string
model = keras.Sequential([
    keras.layers.Dense(64),
    qk.QActivation('quantized_relu(bits=8)'),
])

# Using quantizer object
q = qk.quantized_bits(bits=8, integer=4)
model = keras.Sequential([
    keras.layers.Dense(64),
    qk.QActivation(q),
])

# Full quantized network
model = keras.Sequential([
    qk.QDense(64, kernel_quantizer='quantized_bits(8,4,1 ברירת מחדל=1)'),
    qk.QActivation('quantized_relu(bits=8)'),
    qk.QDense(32, kernel_quantizer='quantized_bits(8,3,1)'),
    qk.QActivation('quantized_bits(bits=8,integer=4,symmetric=1)'),
    qk.QDense(10, kernel_quantizer='quantized_bits(8,2,1)',
              activation='softmax'),
])
```

--------------------------------

### Standard Keras Initializer Options

Source: https://github.com/google/qkeras/blob/master/_autodocs/configuration.md

Examples of using standard Keras initializer strings for layer weights.

```python
kernel_initializer='glorot_uniform'      # Standard initializer string
```

```python
kernel_initializer='he_normal'           # He initialization
```

```python
kernel_initializer='he_uniform'
```

```python
kernel_initializer='random_normal'
```

```python
kernel_initializer='random_uniform'
```

```python
kernel_initializer='zeros'
```

```python
kernel_initializer='ones'
```

--------------------------------

### Quantizer-Aware Initializer Configuration

Source: https://github.com/google/qkeras/blob/master/_autodocs/configuration.md

Example of configuring a quantizer-aware initializer using QInitializer with a standard Keras initializer and a quantizer.

```python
from qkeras import QInitializer
kernel_initializer = QInitializer(
    initializer=tf.keras.initializers.GlorotUniform(),
    use_scale=True,
    quantizer=qk.quantized_bits(8, 4)
)
```

--------------------------------

### QActivation Get Config Method

Source: https://github.com/google/qkeras/blob/master/_autodocs/api-reference/qlayers.md

Returns a serializable configuration dictionary for the QActivation layer. This is useful for saving and loading models.

```python
def get_config(self)
```

--------------------------------

### QSeparableConv2D Usage Example

Source: https://github.com/google/qkeras/blob/master/_autodocs/api-reference/qconvolutional.md

Demonstrates how to use the QSeparableConv2D layer to create an efficient, mobile-friendly layer and build a MobileNet-style architecture with quantized activations.

```python
import qkeras as qk
from tensorflow import keras

# Efficient mobile-friendly layer
layer = qk.QSeparableConv2D(
    filters=128,
    kernel_size=3,
    strides=1,
    padding='same',
    depthwise_quantizer='quantized_bits(8,4,1)',
    pointwise_quantizer='quantized_bits(8,4,1)',
    activation='relu'
)

# MobileNet-style architecture
model = keras.Sequential([
    keras.layers.Input(shape=(224, 224, 3)),
    qk.QSeparableConv2D(32, 3, padding='same',
                       depthwise_quantizer='quantized_bits(8,4,1)',
                       pointwise_quantizer='quantized_bits(8,4,1)',
                       activation='relu'),
    keras.layers.MaxPooling2D(2),
    qk.QSeparableConv2D(64, 3, padding='same',
                       depthwise_quantizer='quantized_bits(8,4,1)',
                       pointwise_quantizer='quantized_bits(8,4,1)',
                       activation='relu'),
    keras.layers.GlobalAveragePooling2D(),
    qk.QDense(10, activation='softmax'),
])
```

--------------------------------

### Example Usage of QBatchNormalization Layer

Source: https://github.com/google/qkeras/blob/master/_autodocs/api-reference/other-layers.md

Shows how to instantiate a QBatchNormalization layer with a specified output quantizer. This layer is experimental and intended for applying quantization to batch normalization outputs.

```python
import qkeras as qk

layer = qk.QBatchNormalization(
    axis=-1,
    quantizer='quantized_bits(8,4,1)'
)
```

--------------------------------

### QKeras Clip Constraint Configuration

Source: https://github.com/google/qkeras/blob/master/_autodocs/configuration.md

Example of configuring a QKeras Clip constraint with min/max values and an optional quantizer.

```python
from qkeras import Clip
kernel_constraint = Clip(
    min_value=-1.0,
    max_value=1.0,
    quantizer='quantized_bits(8,4,1)'
)
```

--------------------------------

### Example Usage of QOctaveConv2D

Source: https://github.com/google/qkeras/blob/master/_autodocs/api-reference/other-layers.md

Demonstrates how to instantiate and configure a QOctaveConv2D layer with specific quantization and frequency parameters. This layer is suitable for multi-frequency convolution tasks.

```python
import qkeras as qk

# Multi-frequency convolution
layer = qk.QOctaveConv2D(
    filters=64,
    kernel_size=3,
    padding='same',
    kernel_quantizer='quantized_bits(8,4,1)',
    alpha_in=0.5,
    alpha_out=0.5,
)
```

--------------------------------

### Quantized Bits Parameter Example

Source: https://github.com/google/qkeras/blob/master/_autodocs/README.md

Example of using the quantized_bits function with both positional and named parameters.

```python
quantized_bits(total_bits, integer_bits, symmetric)
quantized_bits(8, 4, 1)
```

--------------------------------

### Example: Creating and Using QConv2D Layer

Source: https://github.com/google/qkeras/blob/master/_autodocs/api-reference/qconvolutional.md

Demonstrates how to instantiate a QConv2D layer with specific quantization and activation functions. It also shows how to build a simple quantized Convolutional Neural Network (CNN) model for image classification using QConv2D and other Keras layers.

```python
import qkeras as qk
from tensorflow import keras

# Quantized 2D convolution
layer = qk.QConv2D(
    filters=32,
    kernel_size=3,
    strides=1,
    padding='same',
    kernel_quantizer='quantized_bits(8,4,1)',
    activation='relu'
)

# Build quantized CNN for image classification
model = keras.Sequential([
    keras.layers.Input(shape=(224, 224, 3)),
    qk.QConv2D(64, 3, padding='same',
               kernel_quantizer='quantized_bits(8,4,1)', activation='relu'),
    keras.layers.MaxPooling2D(2),
    qk.QConv2D(128, 3, padding='same',
               kernel_quantizer='quantized_bits(8,4,1)', activation='relu'),
    keras.layers.MaxPooling2D(2),
    keras.layers.Flatten(),
    qk.QDense(10, activation='softmax'),
])
```

--------------------------------

### Accumulator Analysis Example

Source: https://github.com/google/qkeras/blob/master/_autodocs/README.md

Estimate accumulator bit width requirements for deployed inference using sample data.

```python
# Estimate from sample data
config = qk.analyze_accumulator(model, x_sample)

# For conv layer with quantized_bits(8, 4, 1) weights/activations:
# Output range: [-8*N, 8*N] where N = receptive field product
# Accumulator: 16-20 bits typically sufficient
```

--------------------------------

### Example Usage of QSeparableConv2DTranspose

Source: https://github.com/google/qkeras/blob/master/_autodocs/api-reference/other-layers.md

Demonstrates how to instantiate the QSeparableConv2DTranspose layer with specific quantization settings for depthwise and pointwise operations. This is useful for creating quantized models for efficient inference.

```python
import qkeras as qk

layer = qk.QSeparableConv2DTranspose(
    filters=64,
    kernel_size=4,
    strides=2,
    padding='same',
    depthwise_quantizer='quantized_bits(8,4,1)',
    pointwise_quantizer='quantized_bits(8,4,1)',
)
```

--------------------------------

### Binary and Stochastic Binary Quantizer Configuration Strings

Source: https://github.com/google/qkeras/blob/master/_autodocs/types.md

Examples of string formats for configuring binary and stochastic binary quantizers, specifying the alpha parameter.

```python
'binary(alpha=1.0)'
'binary(alpha=auto)'
'stochastic_binary(alpha=auto)'
```

--------------------------------

### Dynamic Quantizer Configuration

Source: https://github.com/google/qkeras/blob/master/notebook/QRNNTutorial.ipynb

Sets up a dynamic quantization configuration using f-strings for bit precision. This example configures quantization for bidirectional, dense layers, and a specific activation.

```python
bits = 4
quantizer_config = {
  "bidirectional": {
      'activation' : f"quantized_tanh({bits})",
      'recurrent_activation' : f"quantized_relu(4,0,1)",
      'kernel_quantizer' : f"quantized_bits({bits}, alpha='auto')",
      'recurrent_quantizer' : f"quantized_bits({bits}, alpha='auto')",
      'bias_quantizer' : f"quantized_bits({bits}, alpha='auto')",
  },
  "dense": {
      'kernel_quantizer' : f"quantized_bits({bits}), alpha='auto'",
      'bias_quantizer' : f"quantized_bits({bits}), alpha='auto'"
  },
  "embedding_act": f"quantized_bits({bits}), alpha='auto'",
}
```

--------------------------------

### QKeras Quantizer Function Example

Source: https://github.com/google/qkeras/blob/master/_autodocs/README.md

Demonstrates the creation of a quantizer object using the `quantized_bits` function, specifying the total bit width and the number of integer bits.

```python
quantizer = qk.quantized_bits(bits=8, integer=4)
# bits=8: total bit width
# integer=4: bits to left of binary point
```

--------------------------------

### Configure Fixed-Point Quantizer

Source: https://github.com/google/qkeras/blob/master/_autodocs/README.md

Example of configuring a standard fixed-point quantizer using `quantized_bits`. This specifies the total bit width, the number of integer bits, and whether the range should be symmetric.

```python
q = qk.quantized_bits(bits=8, integer=4, symmetric=1)
# 8-bit total, 4 integer bits (left of point), symmetric range
# Range: [-8.0, 7.9375]
```

--------------------------------

### TPU or CPU/GPU Strategy Setup

Source: https://github.com/google/qkeras/blob/master/notebook/QRNNTutorial.ipynb

Configures the TensorFlow distribution strategy, attempting to connect to a TPU if available, otherwise defaulting to CPU/GPU.

```python
try:
  device_name = os.environ['COLAB_TPU_ADDR']
  TPU_ADDRESS = 'grpc://' + device_name
  print('Found TPU at: {}'.format(TPU_ADDRESS))
  resolver = tf.distribute.cluster_resolver.TPUClusterResolver(TPU_ADDRESS)
  tf.config.experimental_connect_to_cluster(resolver)
  # This is the TPU initialization code that has to be at the beginning.
  tf.tpu.experimental.initialize_tpu_system(resolver)
  print("All devices: ", tf.config.list_logical_devices('TPU'))
  strategy = tf.distribute.experimental.TPUStrategy(resolver)  
except KeyError:
  print('TPU not found')
  strategy = tf.distribute.get_strategy()
```

--------------------------------

### Quantized PO2 and Quantized ReLU PO2 Quantizer Configuration Strings

Source: https://github.com/google/qkeras/blob/master/_autodocs/types.md

Examples of string formats for configuring `quantized_po2` and `quantized_relu_po2` quantizers, specifying bit width and maximum value.

```python
'quantized_po2(bits=8, max_value=-1)'
'quantized_relu_po2(bits=8)'
```

--------------------------------

### Enabling Verbose Output for Debugging

Source: https://github.com/google/qkeras/blob/master/_autodocs/errors.md

A simple Python snippet demonstrating how to import TensorFlow, which can be a starting point for enabling more verbose logging or debugging output.

```Python
import tensorflow as tf
```

--------------------------------

### Quantized uLaw Quantizer Configuration String

Source: https://github.com/google/qkeras/blob/master/_autodocs/types.md

Example of a string format for configuring a `quantized_ulaw` quantizer, specifying bit width and the u parameter.

```python
'quantized_ulaw(bits=8, u=255.0)'
```

--------------------------------

### Configure a Quantized Dense Layer

Source: https://github.com/google/qkeras/blob/master/_autodocs/README.md

Example of creating a QKeras QDense layer with specified units and quantizers for weights and biases. This layer integrates quantization directly into the Keras layer structure.

```python
layer = qk.QDense(
    units=128,
    kernel_quantizer='quantized_bits(8,4,1)',  # Quantize weights
    bias_quantizer='quantized_bits(8,4,1)',    # Quantize biases
    activation='relu'                           # Post-layer activation
)
```

--------------------------------

### Ternary and Stochastic Ternary Quantizer Configuration Strings

Source: https://github.com/google/qkeras/blob/master/_autodocs/types.md

Examples of string formats for configuring ternary and stochastic ternary quantizers, including alpha and threshold parameters.

```python
'ternary(alpha=auto, threshold=0.33)'
'stochastic_ternary(alpha=auto_po2)'
```

--------------------------------

### Build a Quantized Model with QKeras

Source: https://github.com/google/qkeras/blob/master/_autodocs/README.md

Demonstrates how to build a quantized convolutional neural network using QKeras layers with 8-bit fixed-point quantization. This snippet includes model compilation and training setup.

```python
import qkeras as qk
from tensorflow import keras

# 8-bit fixed-point quantization
model = keras.Sequential([
    qk.QConv2D(32, 3, padding='same',
               kernel_quantizer='quantized_bits(8,4,1)',
               activation='relu'),
    keras.layers.MaxPooling2D(2),
    
    qk.QConv2D(64, 3, padding='same',
               kernel_quantizer='quantized_bits(8,4,1)',
               activation='relu'),
    keras.layers.GlobalAveragePooling2D(),
    
    qk.QDense(10, kernel_quantizer='quantized_bits(8,4,1)',
             activation='softmax')
])

model.compile(optimizer='adam', loss='categorical_crossentropy',
              metrics=['accuracy'])
model.fit(x_train, y_train, epochs=50, batch_size=32)
```

--------------------------------

### Quantized ReLU Quantizer Configuration Strings

Source: https://github.com/google/qkeras/blob/master/_autodocs/types.md

Examples of string formats for configuring `quantized_relu` quantizers, with options for bit width, integer bits, and activation properties.

```python
'quantized_relu(8)'
'quantized_relu(bits=8, integer=2, negative_slope=0.1)'
'quantized_relu(bits=8, use_sigmoid=1, alpha=auto)'
```

--------------------------------

### Instantiate and Use QConv1D in a Model

Source: https://github.com/google/qkeras/blob/master/_autodocs/api-reference/qconvolutional.md

Demonstrates how to instantiate a QConv1D layer with 8-bit quantization and integrate it into a Keras Sequential model for time-series data. This example shows setting up a model with multiple quantized convolutional layers and pooling.

```python
import qkeras as qk
from tensorflow import keras

# 1D temporal convolution with 8-bit quantization
layer = qk.QConv1D(
    filters=64,
    kernel_size=3,
    strides=1,
    padding='same',
    kernel_quantizer='quantized_bits(8,4,1)',
    bias_quantizer='quantized_bits(8,4,1)',
    activation='relu'
)

# Use in time-series model
model = keras.Sequential([
    keras.layers.Input(shape=(100, 16)),  # (time_steps, features)
    qk.QConv1D(32, 3, padding='same',
               kernel_quantizer='quantized_bits(8,4,1)', activation='relu'),
    keras.layers.MaxPooling1D(2),
    qk.QConv1D(64, 3, padding='same',
               kernel_quantizer='quantized_bits(8,4,1)', activation='relu'),
    keras.layers.GlobalAveragePooling1D(),
    qk.QDense(10, activation='softmax'),
])
```

--------------------------------

### Example Usage of QConv2DBatchnorm

Source: https://github.com/google/qkeras/blob/master/_autodocs/api-reference/other-layers.md

Shows how to create a QConv2DBatchnorm layer with specified filters, kernel size, quantization, and batch normalization parameters. This layer is useful for fused quantized convolution and batch normalization operations.

```python
import qkeras as qk

# Fused conv+bn layer
layer = qk.QConv2DBatchnorm(
    filters=64,
    kernel_size=3,
    strides=1,
    padding='same',
    kernel_quantizer='quantized_bits(8,4,1)',
    activation='relu',
    bn_momentum=0.99,
)
```

--------------------------------

### Quantized Keras Network with QKeras

Source: https://github.com/google/qkeras/blob/master/README.md

This example demonstrates how to quantize the previously defined Keras network using QKeras layers and activation functions. It shows the application of various quantization parameters for convolutional, separable convolutional, dense layers, and activations.

```python
from keras.layers import *
from qkeras import *

x = x_in = Input(shape)
x = QConv2D(18, (3, 3),
        kernel_quantizer="stochastic_ternary",
        bias_quantizer="ternary", name="first_conv2d")(x)
x = QActivation("quantized_relu(3)")(x)
x = QSeparableConv2D(32, (3, 3),
        depthwise_quantizer=quantized_bits(4, 0, 1),
        pointwise_quantizer=quantized_bits(3, 0, 1),
        bias_quantizer=quantized_bits(3),
        depthwise_activation=quantized_tanh(6, 2, 1))(x)
x = QActivation("quantized_relu(3)")(x)
x = Flatten()(x)
x = QDense(NB_CLASSES,
        kernel_quantizer=quantized_bits(3),
        bias_quantizer=quantized_bits(3))(x)
x = QActivation("quantized_bits(20, 5)")(x)
x = Activation("softmax")(x)
```

--------------------------------

### Example Usage of QAveragePooling2D in a Sequential Model

Source: https://github.com/google/qkeras/blob/master/_autodocs/api-reference/other-layers.md

Demonstrates how to use QAveragePooling2D layers within a Keras Sequential model. This example shows its application after QConv2D layers for downsampling and quantization.

```python
import qkeras as qk
from tensorflow import keras

model = keras.Sequential([
    qk.QConv2D(64, 3, padding='same', activation='relu'),
    qk.QAveragePooling2D(pool_size=2, strides=2),
    qk.QConv2D(128, 3, padding='same', activation='relu'),
    qk.QAveragePooling2D(pool_size=2, strides=2),
])
```

--------------------------------

### QDense Layer Initialization

Source: https://github.com/google/qkeras/blob/master/_autodocs/api-reference/qlayers.md

Illustrates the initialization of a QDense layer with specific kernel and bias quantizers, and a ReLU activation function.

```python
import qkeras as qk
from tensorflow import keras

# Quantized dense with 8-bit weights and bias
layer = qk.QDense(
    units=128,
    kernel_quantizer='quantized_bits(bits=8, integer=1, symmetric=1)',
    bias_quantizer='quantized_bits(bits=8, integer=2, symmetric=1)',
    activation='relu'
)
```

--------------------------------

### Standard Keras Network

Source: https://github.com/google/qkeras/blob/master/README.md

This is an example of a standard Keras network before quantization.

```python
from keras.layers import *

x = x_in = Input(shape)
x = Conv2D(18, (3, 3), name="first_conv2d")(x)
x = Activation("relu")(x)
x = SeparableConv2D(32, (3, 3))(x)
x = Activation("relu")(x)
x = Flatten()(x)
x = Dense(NB_CLASSES)(x)
x = Activation("softmax")(x)
```

--------------------------------

### Configure AutoQKeras Run Settings

Source: https://github.com/google/qkeras/blob/master/notebook/AutoQKeras.ipynb

Sets up the configuration dictionary for AutoQKeras, specifying output directory, goal, quantization configuration, tuning mode, seed, filter tuning options, distribution strategy, layer indexes for quantization, and maximum trials.

```python
run_config = {
  "output_dir": tempfile.mkdtemp(),
  "goal": goal,
  "quantization_config": quantization_config,
  "learning_rate_optimizer": False,
  "transfer_weights": False,
  "mode": "random",
  "seed": 42,
  "limit": limit,
  "tune_filters": "layer",
  "tune_filters_exceptions": "^dense",
  "distribution_strategy": cur_strategy,
  # first layer is input, layer two layers are softmax and flatten
  "layer_indexes": range(1, len(model.layers) - 1),
  "max_trials": 20
}

print("quantizing layers:", [model.layers[i].name for i in run_config["layer_indexes"]])
```

--------------------------------

### QLSTM Layer Usage Example

Source: https://github.com/google/qkeras/blob/master/_autodocs/api-reference/qrecurrent.md

Demonstrates how to use the QLSTM layer within a Keras Sequential model for sequence-to-sequence or time series tasks. This example shows two stacked QLSTM layers followed by a QDense output layer, with quantization applied to kernels, recurrent weights, and biases.

```python
import qkeras as qk
from tensorflow import keras

# LSTM for sequence-to-sequence or time series
model = keras.Sequential([
    keras.layers.Input(shape=(50, 10)),  # (timesteps, features)
    qk.QLSTM(
        units=128,
        kernel_quantizer='quantized_bits(8,4,1)',
        recurrent_quantizer='quantized_bits(8,4,1)',
        bias_quantizer='quantized_bits(8,4,1)',
        return_sequences=True,
    ),
    qk.QLSTM(
        units=64,
        kernel_quantizer='quantized_bits(8,4,1)',
        recurrent_quantizer='quantized_bits(8,4,1)',
        bias_quantizer='quantized_bits(8,4,1)',
        return_sequences=False,
    ),
    qk.QDense(10, activation='softmax'),
])
```

--------------------------------

### Get Model for MNIST Dataset

Source: https://github.com/google/qkeras/blob/master/notebook/AutoQKeras.ipynb

Instantiates and builds a ConvBlockNetwork model configured for the MNIST dataset.

```python
def get_model(dataset):
  """Returns a model for the demo of AutoQKeras."""
  if dataset == "mnist":
    model = ConvBlockNetwork(
        shape=(28, 28, 1),
        nb_classes=10,
        kernel_size=3,
        filters=[16, 32, 48, 64, 128],
        dropout_rate=0.2,
        with_maxpooling=False,
        with_batchnorm=True,
        kernel_initializer="he_uniform",
        bias_initializer="zeros",
    ).build()

  elif dataset == "fashion_mnist":
    model = ConvBlockNetwork(
        shape=(28, 28, 1),
        nb_classes=10,
        kernel_size=3,
        filters=[16, [32]*3, [64]*3],
        dropout_rate=0.2,
        with_maxpooling=True,
        with_batchnorm=True,
        use_separable="mobilenet",
        kernel_initializer="he_uniform",
        bias_initializer="zeros",
        use_xnornet_trick=True
    ).build()

  elif dataset == "cifar10":
    model = ConvBlockNetwork(
        shape=(32, 32, 3),
        nb_classes=10,
        kernel_size=3,
        filters=[16, [32]*3, [64]*3, [128]*3],
        dropout_rate=0.2,
        with_maxpooling=True,
        with_batchnorm=True,
        use_separable="mobilenet",
        kernel_initializer="he_uniform",
        bias_initializer="zeros",
        use_xnornet_trick=True
    ).build()

  elif dataset == "cifar100":
    model = ConvBlockNetwork(
        shape=(32, 32, 3),
        nb_classes=100,
        kernel_size=3,
        filters=[16, [32]*3, [64]*3, [128]*3, [256]*3],
        dropout_rate=0.2,
        with_maxpooling=True,
        with_batchnorm=True,
        use_separable="mobilenet",
        kernel_initializer="he_uniform",
        bias_initializer="zeros",
        use_xnornet_trick=True
    ).build()

  model.summary()

  return model
```

--------------------------------

### Fit Model with AutoQKeras

Source: https://github.com/google/qkeras/blob/master/notebook/QRNNTutorial.ipynb

Initializes AutoQKeras with the model and configuration, then fits the model using provided training and validation datasets. This step performs quantization-aware training.

```python
autoqk = AutoQKeras(model, metrics=["acc"], custom_objects={}, **run_config)
autoqk.fit(
  train_dataset, 
  validation_data=test_dataset, 
  batch_size=BATCH_SIZE, 
  epochs=10,
  verbose=2)
```

--------------------------------

### Tensor Return Example

Source: https://github.com/google/qkeras/blob/master/_autodocs/types.md

Most layer calls in QKeras return a tf.Tensor. This is the standard output for many operations.

```python
output: tf.Tensor = layer(inputs)
```

--------------------------------

### Configure AutoQKeras Run Settings

Source: https://github.com/google/qkeras/blob/master/notebook/AutoQKeras.ipynb

Sets up the configuration for an AutoQKeras run, including output directory, quantization configuration, learning rate optimizer, and distribution strategy. It also specifies layer indexing and the maximum number of trials.

```python
run_config = {
  "output_dir": tempfile.mkdtemp(),
  "goal": goal,
  "quantization_config": quantization_config,
  "learning_rate_optimizer": False,
  "transfer_weights": False,
  "mode": "random",
  "seed": 42,
  "limit": limit,
  "tune_filters": "layer",
  "tune_filters_exceptions": "^dense",
  "distribution_strategy": cur_strategy,
  "layer_indexes": range(1, len(model.layers) - 1),
  "max_trials": 40
}
```

--------------------------------

### Standard Keras Constraint Options

Source: https://github.com/google/qkeras/blob/master/_autodocs/configuration.md

Examples of using standard Keras constraint strings for layer weights, including None for no constraint.

```python
kernel_constraint=None                   # No constraint
```

```python
kernel_constraint='max_norm'             # Standard constraint string
```

```python
kernel_constraint='non_neg'
```

```python
kernel_constraint='unit_norm'
```

--------------------------------

### Quantized Bits Quantizer Configuration Strings

Source: https://github.com/google/qkeras/blob/master/_autodocs/types.md

Examples of string formats for configuring `quantized_bits` quantizers, including options for symmetry and rounding.

```python
'quantized_bits(8,4,1)'
'quantized_bits(bits=8, integer=4, symmetric=1)'
'quantized_bits(bits=8, integer=4, symmetric=1, keep_negative=1)'
'quantized_bits(bits=8, alpha=auto, use_stochastic_rounding=1)'
'quantized_bits(bits=8, alpha=auto_po2, scale_axis=0, per_channel_scale=1)'
```

--------------------------------

### Initialize QKeras QTools for Energy Estimation

Source: https://github.com/google/qkeras/blob/master/notebook/AutoQKeras.ipynb

Initialize the QTools class from QKeras for energy calculation. This involves specifying the model, process, quantizers, and whether to calculate a reference energy.

```python
reference_internal = "fp32"
reference_accumulator = "fp32"

 q = run_qtools.QTools(
    model,
    # energy calculation using a given process
    # "horowitz" refers to 45nm process published at
    # M. Horowitz, "1.1 Computing's energy problem (and what we can do about
    # it), "2014 IEEE International Solid-State Circuits Conference Digest of
    # Technical Papers (ISSCC), San Francisco, CA, 2014, pp. 10-14, 
    # doi: 10.1109/ISSCC.2014.6757323.
    process="horowitz",
    # quantizers for model input
    source_quantizers=[quantized_bits(8, 0, 1)],
    is_inference=False,
    # absolute path (including filename) of the model weights
    # in the future, we will attempt to optimize the power model
    # by using weight information, although it can be used to further
    # optimize QBatchNormalization.
    weights_path=None,
    # keras_quantizer to quantize weight/bias in un-quantized keras layers
    keras_quantizer=reference_internal,
    # keras_quantizer to quantize MAC in un-quantized keras layers
    keras_accumulator=reference_accumulator,
    # whether calculate baseline energy
    for_reference=True)
  
```

--------------------------------

### TPU Distributed Training Strategy

Source: https://github.com/google/qkeras/blob/master/notebook/AutoQKeras.ipynb

Configures and initializes a TPUStrategy for distributed training if TPUs are available. Otherwise, it gets the default strategy.

```python
has_tpus = np.any([d.device_type == "TPU" for d in physical_devices])

if has_tpus:
  TPU_WORKER = 'local'

  resolver = tf.distribute.cluster_resolver.TPUClusterResolver(
      tpu=TPU_WORKER, job_name='tpu_worker')
  if TPU_WORKER != 'local':
    tf.config.experimental_connect_to_cluster(resolver, protocol='grpc+loas')
  tf.tpu.experimental.initialize_tpu_system(resolver)
  strategy = tf.distribute.experimental.TPUStrategy(resolver)
  print('Number of devices: {}'.format(strategy.num_replicas_in_sync))

  cur_strategy = strategy
else:
  cur_strategy = tf.distribute.get_strategy()
```

--------------------------------

### QKeras Quantization Schemes

Source: https://github.com/google/qkeras/blob/master/_autodocs/README.md

Illustrates various quantization schemes available in QKeras, including fixed-point, binary, ternary, power-of-two, and quantized ReLU activations. It also shows how to define an adaptive activation layer.

```python
import qkeras as qk

# Fixed-point 8-bit (4 integer bits)
q_fixed = 'quantized_bits(8,4,1)'

# Binary weights (-1 or +1)
q_binary = 'binary(alpha=auto)'

# Ternary weights (-1, 0, +1)
q_ternary = 'ternary(alpha=auto, threshold=0.33)'

# Power-of-two (hardware efficient)
q_po2 = 'quantized_po2(bits=8, max_value=-1)'

# ReLU with quantization
q_relu = 'quantized_relu(bits=8, integer=2, negative_slope=0.1)'

# Adaptive (learns quantization range)
layer = qk.QAdaptiveActivation(
    activation='quantized_relu',
    total_bits=8,
    current_step=model.optimizer.iterations
)
```

--------------------------------

### Initialize and Fit AutoQKeras Model

Source: https://github.com/google/qkeras/blob/master/notebook/AutoQKeras.ipynb

Initializes the AutoQKeras model with the specified configuration and metrics, then fits the model to the training data, using validation data for evaluation.

```python
autoqk = AutoQKeras(model, metrics=["acc"], custom_objects=custom_objects, **run_config)
autoqk.fit(x_train, y_train, validation_data=(x_test, y_test), batch_size=1024, epochs=20)
```

--------------------------------

### Build Sequential Model with QDense Layers

Source: https://github.com/google/qkeras/blob/master/_autodocs/api-reference/qlayers.md

Demonstrates how to construct a Keras Sequential model using QKeras's QDense layers with specified quantization parameters for kernels and biases.

```python
import qkeras as qk
from tensorflow import keras

model = keras.Sequential([
    qk.QDense(64, kernel_quantizer='quantized_bits(8,4,1)', 
              activation='relu'),
    qk.QDense(64, kernel_quantizer='quantized_bits(8,4,1)',
              bias_quantizer='quantized_bits(8,4,1)',
              activation='relu'),
    qk.QDense(10, kernel_quantizer='quantized_bits(8,3,1)',
              activation='softmax')
])
```

--------------------------------

### Quantized Hard Swish Quantizer Configuration String

Source: https://github.com/google/qkeras/blob/master/_autodocs/types.md

Example of a string format for configuring a `quantized_hswish` quantizer, specifying bit width and integer bits.

```python
'quantized_hswish(bits=8, integer=2)'
```

--------------------------------

### Quantized Tanh Quantizer Configuration String

Source: https://github.com/google/qkeras/blob/master/_autodocs/types.md

Example of a string format for configuring a `quantized_tanh` quantizer, specifying bit width, integer bits, and symmetry.

```python
'quantized_tanh(bits=8, integer=0, symmetric=1)'
```

--------------------------------

### QGRUCell Constructor

Source: https://github.com/google/qkeras/blob/master/_autodocs/api-reference/qrecurrent.md

Initializes a QGRUCell with specified parameters for units, activations, initializers, regularizers, constraints, dropout, and quantization.

```APIDOC
## QGRUCell Constructor

### Description
Initializes a QGRUCell, a quantized Gated Recurrent Unit cell for sequence processing. It accepts parameters for units, activation functions, initializers, regularizers, constraints, dropout rates, and quantization configurations.

### Parameters

#### Initialization Parameters
- **units** (int) - Required - Output dimensionality.
- **activation** (str) - Optional - Defaults to 'tanh'. Cell activation function.
- **recurrent_activation** (str) - Optional - Defaults to 'sigmoid'. Gate activation function.
- **use_bias** (bool) - Optional - Defaults to True. Whether to include a bias vector.
- **kernel_initializer** (str) - Optional - Defaults to 'glorot_uniform'. Initializer for the kernel weights.
- **recurrent_initializer** (str) - Optional - Defaults to 'orthogonal'. Initializer for the recurrent kernel weights.
- **bias_initializer** (str) - Optional - Defaults to 'zeros'. Initializer for the bias vector.
- **kernel_regularizer** (Regularizer) - Optional - Defaults to None. Regularizer function applied to the kernel weights.
- **recurrent_regularizer** (Regularizer) - Optional - Defaults to None. Regularizer function applied to the recurrent kernel weights.
- **bias_regularizer** (Regularizer) - Optional - Defaults to None. Regularizer function applied to the bias vector.
- **kernel_constraint** (Constraint) - Optional - Defaults to None. Constraint function applied to the kernel weights.
- **recurrent_constraint** (Constraint) - Optional - Defaults to None. Constraint function applied to the recurrent kernel weights.
- **bias_constraint** (Constraint) - Optional - Defaults to None. Constraint function applied to the bias vector.
- **dropout** (float) - Optional - Defaults to 0.0. Dropout rate for the input units.
- **recurrent_dropout** (float) - Optional - Defaults to 0.0. Dropout rate for the recurrent units.
- **reset_after** (bool) - Optional - Defaults to True. Whether to apply reset after (GRUv2) or before (GRUv1) the gates.
- **kernel_quantizer** (str or Quantizer) - Optional - Defaults to None. Quantizer for the kernel weights.
- **recurrent_quantizer** (str or Quantizer) - Optional - Defaults to None. Quantizer for the recurrent kernel weights.
- **bias_quantizer** (str or Quantizer) - Optional - Defaults to None. Quantizer for the bias weights.
- **kwargs** (dict) - Optional - Additional keyword arguments.

### Example

```python
import qkeras as qk

cell = qk.QGRUCell(
    units=128,
    kernel_quantizer='quantized_bits(8,4,1)',
    recurrent_quantizer='quantized_bits(8,4,1)',
    bias_quantizer='quantized_bits(8,4,1)',
    reset_after=True
)
```

### Source
`qkeras/qrecurrent.py:QGRUCell class`
```

--------------------------------

### Quantizing and Compiling a Model

Source: https://github.com/google/qkeras/blob/master/notebook/QRNNTutorial.ipynb

Demonstrates quantizing an existing Keras model using a defined configuration and then compiling it for training. The model is then trained on provided datasets.

```python
tf.keras.backend.clear_session()
with strategy.scope():
  model = create_model(BATCH_SIZE)
  custom_objects = {}
  
  qmodel = model_quantize(model, quantizer_config, bits, custom_objects)
  qmodel.compile(
      optimizer=Adam(learning_rate=0.01),
      loss=loss,
      metrics=['acc'])
  
qmodel.summary()
print('Train...')
qmodel.fit(train_dataset,
          batch_size=BATCH_SIZE,
          epochs=10,
          verbose=2,
          validation_data=test_dataset)
```

--------------------------------

### Get Quantization Configuration

Source: https://github.com/google/qkeras/blob/master/_autodocs/api-reference/qconvolutional.md

Retrieves the current quantization configuration for the QConv2D layer. This method is useful for inspecting or saving the layer's quantization settings.

```python
def get_quantization_config()
    Returns quantization configuration.
```

--------------------------------

### QSimpleRNN Layer Initialization

Source: https://github.com/google/qkeras/blob/master/_autodocs/api-reference/qrecurrent.md

Initializes a QSimpleRNN layer with various parameters for units, activation, bias, initializers, regularizers, constraints, dropout, and quantization.

```python
class QSimpleRNN(RNN, PrunableLayer):
    def __init__(self, units, activation='tanh', use_bias=True,
                 kernel_initializer='glorot_uniform',
                 recurrent_initializer='orthogonal',
                 bias_initializer='zeros',
                 kernel_regularizer=None, recurrent_regularizer=None,
                 bias_regularizer=None, activity_regularizer=None,
                 kernel_constraint=None, recurrent_constraint=None,
                 bias_constraint=None, dropout=0.0, recurrent_dropout=0.0,
                 return_sequences=False, return_state=False,
                 go_backwards=False, stateful=False,
                 unroll=False, kernel_quantizer=None,
                 recurrent_quantizer=None, bias_quantizer=None,
                 **kwargs)
```

--------------------------------

### Configure QAdaptiveActivation Layer

Source: https://github.com/google/qkeras/blob/master/_autodocs/configuration.md

Sets up the QAdaptiveActivation layer with options for activation type, bit precision, EMA decay, and quantization delay. Sync the training step with the optimizer's iterations for accurate quantization.

```python
qk.QAdaptiveActivation(
    activation='quantized_bits',         # or 'quantized_relu'
    total_bits=8,                        # Total quantization bits
    
    # Step tracking
    current_step=None,                   # tf.Variable for training step
    
    # EMA configuration
    ema_decay=0.9999,                    # Exponential moving average decay
    quantization_delay=0,                # Steps before quantization starts
    ema_freeze_delay=None,               # Steps before freezing EMA
    
    # Quantization options
    symmetric=True,                      # Symmetric quantization
    po2_rounding=False,                  # Round vs ceil to power-of-2
    per_channel=False,                   # Per-channel vs global
    
    # ReLU-specific options (if activation='quantized_relu')
    relu_neg_slope=0.0,                  # Leaky ReLU slope
    relu_upper_bound=None,               # Optional upper bound
)
```

--------------------------------

### Configure AutoQKerasScheduler for Block Quantization

Source: https://github.com/google/qkeras/blob/master/notebook/AutoQKeras.ipynb

Set up the `run_config` dictionary to define parameters for block quantization, including output directory, goal, quantization configuration, learning rate optimization, weight transfer, mode, seed, limits, filter tuning, distribution strategy, layer indexes, max trials, blocks to quantize (using regex), and the scheduling mode for blocks.

```python
run_config = {
  "output_dir": tempfile.mkdtemp(),
  "goal": goal,
  "quantization_config": quantization_config,
  "learning_rate_optimizer": False,
  "transfer_weights": False,
  "mode": "random",
  "seed": 42,
  "limit": limit,
  "tune_filters": "layer",
  "tune_filters_exceptions": "^dense",
  "distribution_strategy": cur_strategy,
  "layer_indexes": range(1, len(model.layers) - 1),
  "max_trials": 40,

  "blocks": [
    "^.*_0$",
    "^.*_1$",
    "^.*_2$",
    "^.*_3$",
    "^.*_4$",
    "^dense"
  ],
  "schedule_block": "cost"
}
```

--------------------------------

### Quantized Sequence Classification Model

Source: https://github.com/google/qkeras/blob/master/_autodocs/api-reference/qrecurrent.md

Example of building a Keras Sequential model for quantized sequence classification using QSimpleRNN and QDense layers with specified quantization configurations.

```python
import qkeras as qk
from tensorflow import keras

# Quantized sequence classification
model = keras.Sequential([
    keras.layers.Input(shape=(100, 10)),  # (timesteps, features)
    qk.QSimpleRNN(
        units=64,
        kernel_quantizer='quantized_bits(8,4,1)',
        recurrent_quantizer='quantized_bits(8,4,1)',
        bias_quantizer='quantized_bits(8,4,1)',
        return_sequences=False,
        activation='relu'
    ),
    qk.QDense(10, activation='softmax'),
])
```

--------------------------------

### Scale Initializers for Quantized Layers

Source: https://github.com/google/qkeras/blob/master/_autodocs/api-reference/qlayers.md

Illustrates how to wrap Keras initializers with QKeras's QInitializer to scale weights appropriately for the specified quantizer's range. This ensures initialized values are suitable for quantized layers.

```python
import qkeras as qk
from tensorflow.keras import initializers

# Scale glorot_uniform initialization for 8-bit quantization
quantizer = qk.quantized_bits(bits=8, integer=4)
init = qk.QInitializer(
    initializer=initializers.GlorotUniform(),
    use_scale=True,
    quantizer=quantizer
)

# Use in layer
layer = qk.QDense(
    64,
    kernel_initializer=init,
    kernel_quantizer=quantizer
)
```

--------------------------------

### Get and Save Quantized Model

Source: https://github.com/google/qkeras/blob/master/notebook/QRNNTutorial.ipynb

Retrieves the best quantized model found by AutoQKeras and saves its weights to a file. This allows for later loading and use of the quantized model.

```python
qmodel = autoqk.get_best_model()
qmodel.save_weights("qmodel.h5")
```

--------------------------------

### Instantiate and prepare the Keras model

Source: https://github.com/google/qkeras/blob/master/notebook/QKerasTutorial.ipynb

Calls the data loading function and creates an instance of the Keras model.

```python
(x_train, y_train), (x_test, y_test) = get_data()

model = CreateModel(x_train.shape[1:], y_train.shape[-1])
```

--------------------------------

### Get Quantizer from Identifier

Source: https://github.com/google/qkeras/blob/master/_autodocs/utilities.md

Retrieves or instantiates a quantizer object from a string identifier or by passing an existing quantizer object. This function is essential for dynamically applying quantization schemes.

```python
import qkeras as qk

# String identifier
q1 = qk.get_quantizer('quantized_bits(8,4,1)')
q2 = qk.get_quantizer('binary(alpha=auto)')

# Object passthrough
q = qk.quantized_bits(8, 4)
q_same = qk.get_quantizer(q)
```

--------------------------------

### Import Required Packages

Source: https://github.com/google/qkeras/blob/master/notebook/AutoQKeras.ipynb

Loads essential libraries for running AutoQKeras, including TensorFlow.

```python
import warnings
warnings.filterwarnings("ignore")

import json
import pprint
import numpy as np
import six
import tempfile
import tensorflow.compat.v2 as tf

```

--------------------------------

### Tuple Return Example (RNN Layers)

Source: https://github.com/google/qkeras/blob/master/_autodocs/types.md

RNN layers configured with return_state=True return tuples. The exact tuple structure depends on the RNN cell type (e.g., LSTM).

```python
output, hidden_state = lstm_layer(inputs)
output, (hidden_state, cell_state) = lstm_layer(inputs)  # LSTM
```

--------------------------------

### Instantiate the QKeras model

Source: https://github.com/google/qkeras/blob/master/notebook/QKerasTutorial.ipynb

Creates an instance of the quantized QKeras model.

```python
qmodel = CreateQModel(x_train.shape[1:], y_train.shape[-1])
```