### Install HGQ2 (Development and Regular)

Source: https://github.com/calad0i/hgq2/blob/master/docs/install.md

Use the development installation for active contribution. For general use, a regular installation is sufficient.

```bash
glt clone https://github.com/calad0i/HGQ2
cd HGQ2
pip install -e .
```

```bash
pip install HGQ2
```

--------------------------------

### Install HGQ2 and optional hardware synthesis backends

Source: https://context7.com/calad0i/hgq2/llms.txt

Install the core HGQ2 library and optional backends for FPGA synthesis. Development installation from source is also supported.

```bash
# Core library
pip install HGQ2

# Optional: FPGA synthesis backends
pip install da4ml>=0.6
pip install hls4ml>=1.2.0

# Development install from source
git clone https://github.com/calad0i/HGQ2
cd HGQ2
pip install -e .

```

--------------------------------

### Install Optional Dependencies (da4ml, hls4ml)

Source: https://github.com/calad0i/hgq2/blob/master/docs/install.md

Install da4ml or hls4ml for model conversion and synthesis. Ensure you meet the minimum version requirements.

```bash
pip install da4ml>=0.6
```

```bash
pip install hls4ml>=1.2.0
```

--------------------------------

### Using LayerConfigScope and QuantizerConfigScope for Model Training

Source: https://context7.com/calad0i/hgq2/llms.txt

Demonstrates how to configure layer-level training settings and quantizer parameters using context managers. This setup is useful for applying specific quantization and regularization settings to a model during its definition.

```python
from hgq.config import LayerConfigScope, QuantizerConfigScope
import keras
from hgq.layers import QDense, QConv2D

with (
    QuantizerConfigScope(place='weight', overflow_mode='SAT_SYM', b0=6, i0=2),
    QuantizerConfigScope(place='datalane', overflow_mode='WRAP', i0=4, f0=4),
    LayerConfigScope(enable_ebops=True, beta0=1e-5),   # EBOPs regularization ON, small initial beta
):
    model = keras.Sequential([
        QConv2D(32, (3, 3), activation='relu', padding='same'),
        keras.layers.MaxPooling2D((2, 2)),
        QConv2D(64, (3, 3), activation='relu', padding='same'),
        keras.layers.MaxPooling2D((2, 2)),
        keras.layers.Flatten(),
        QDense(128, activation='relu'),
        QDense(10),
    ])

model.compile(
    optimizer='adam',
    loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
    metrics=['accuracy'],
)
```

--------------------------------

### End-to-End Workflow: Model Training with PID-Controlled EBOPs

Source: https://context7.com/calad0i/hgq2/llms.txt

This comprehensive example demonstrates building a quantized model, loading data, and training it using `BetaPID` for automatic EBOPs targeting and `EarlyStoppingWithEbopsThres` for conditional early stopping. It also includes `FreeEBOPs` for logging.

```python
import numpy as np
import keras
from hgq.layers import QConv2D, QDense
from hgq.config import QuantizerConfigScope, LayerConfigScope
from hgq.utils import trace_minmax
from hgq.utils.sugar import BetaPID, EarlyStoppingWithEbopsThres, FreeEBOPs

# ── 1. Build model ──────────────────────────────────────────────────────────
with (
    QuantizerConfigScope(place='weight', overflow_mode='SAT_SYM', b0=8, i0=3),
    QuantizerConfigScope(place='datalane', overflow_mode='WRAP', i0=4, f0=4),
    LayerConfigScope(enable_ebops=True, beta0=1e-6),
):
    model = keras.Sequential([
        keras.layers.InputLayer((28, 28, 1)),
        QConv2D(32, (3, 3), activation='relu', padding='same'),
        keras.layers.MaxPooling2D((2, 2)),
        QConv2D(64, (3, 3), activation='relu', padding='same'),
        keras.layers.MaxPooling2D((2, 2)),
        keras.layers.Flatten(),
        QDense(128, activation='relu'),
        QDense(10),
    ])

# ── 2. Load data ────────────────────────────────────────────────────────────
(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
x_train = x_train[..., None].astype('float32') / 255.0
x_test  = x_test[..., None].astype('float32') / 255.0

# ── 3. Train with PID-controlled EBOPs budget ───────────────────────────────
model.compile(
    optimizer=keras.optimizers.Adam(1e-3),
    loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
    metrics=['accuracy'],
)

callbacks = [
    BetaPID(target_ebops=2_000_000, init_beta=1e-7, warmup=5),
    EarlyStoppingWithEbopsThres(ebops_threshold=2_000_000, monitor='val_loss', patience=5, restore_best_weights=True),
    FreeEBOPs(),
]

model.fit(x_train, y_train, batch_size=256, epochs=50,
          validation_data=(x_test, y_test), callbacks=callbacks)
```

--------------------------------

### Install HGQ2 Package

Source: https://github.com/calad0i/hgq2/blob/master/README.md

Install the HGQ2 library using pip. Ensure da4ml is at least v0.6 and hls4ml is at least v1.2.0 if you are using them.

```bash
pip install HGQ2
```

```bash
pip install da4ml>=0.6
```

```bash
pip install hls4ml>=1.2.0
```

--------------------------------

### Install or Update hls4ml from Repository

Source: https://github.com/calad0i/hgq2/blob/master/docs/install.md

Install the latest version of hls4ml directly from its GitHub repository to access the newest features. This is an alternative to installing a specific version via pip.

```bash
pip install 'git+https://github.com/fastmachinelearning/hls4ml.git'
```

--------------------------------

### Minimal Keras Model with HGQ2 Layers

Source: https://github.com/calad0i/hgq2/blob/master/README.md

Set up quantization configurations using QuantizerConfigScope and LayerConfigScope before defining a Keras Sequential model with HGQ2's QConv2D and QDense layers. This example demonstrates default quantization settings for demonstration.

```python
import keras
from hgq.layers import QDense, QConv2D
from hgq.config import LayerConfigScope, QuantizerConfigScope

# Setup quantization configuration
# These values are the defaults, just for demonstration purposes here
with (
   # Configuration scope for setting the default quantization type and overflow mode
   # The second configuration scope overrides the first one for the 'datalane' place
   QuantizerConfigScope(place='all', default_q_type='kbi', overflow_mode='SAT_SYM'),
   # Configuration scope for enabling EBOPs and setting the beta0 value
   QuantizerConfigScope(place='datalane', default_q_type='kif', overflow_mode='WRAP'),
   LayerConfigScope(enable_ebops=True, beta0=1e-5),
):
   model = keras.Sequential([
      QConv2D(32, (3, 3), activation='relu'),
      keras.layers.MaxPooling2D((2, 2)),
      keras.layers.Flatten(),
      QDense(10)
   ])
```

--------------------------------

### Set Keras Backend and Import Libraries

Source: https://github.com/calad0i/hgq2/blob/master/example/small_jet_tagger.ipynb

Configures the Keras backend to JAX for optimal performance and imports necessary libraries for HGQ2 and general machine learning tasks. Ensure JAX is installed for the fastest execution.

```python
import os
import random

# JAX runs the fastest for hgq in general based on our experience
# If you don't have jax, or if you want to use another backend, you can change this to 'tensorflow' or 'torch'
os.environ['KERAS_BACKEND'] = 'jax'
# tested for tensorflow, jax, torch. Openvino support is not tested yet.
# For the best performance, we recommend using jax, or tensorflow with XLA enabled.
# Jit compilation for torch (torch dynamo) is not supported yet.

import keras
import numpy as np
from matplotlib import pyplot as plt

from hgq.config import QuantizerConfig, QuantizerConfigScope
from hgq.layers import QDense, QSoftmax
from hgq.utils.sugar import FreeEBOPs, PBar
```

--------------------------------

### Check Keras Version

Source: https://github.com/calad0i/hgq2/blob/master/docs/install.md

Verify that your installed Keras version is 3.x, which is required by HGQ2. Incompatible versions can lead to errors.

```python
import keras
print(keras.__version__)  # Should be 3.x
```

--------------------------------

### Configuring QDense Layer with Quantizer Overrides

Source: https://context7.com/calad0i/hgq2/llms.txt

Shows how to instantiate a QDense layer and override its default kernel (kq) and input (iq) quantizer configurations. This allows fine-grained control over quantization parameters for specific layers.

```python
from hgq.layers import QDense
from hgq.config import QuantizerConfig, QuantizerConfigScope, LayerConfigScope
import keras

kq = QuantizerConfig('kbi', 'weight', b0=6, i0=2, overflow_mode='SAT_SYM')
iq = QuantizerConfig('kif', 'datalane', i0=3, f0=5, overflow_mode='WRAP')

with LayerConfigScope(enable_ebops=True, beta0=5e-6):
    layer = QDense(
        units=64,
        activation='relu',
        kq_conf=kq,    # override kernel quantizer
        iq_conf=iq,    # override input quantizer
        parallelization_factor=1,  # for EBOPs: how many parallel MAC ops
    )

# Inspect quantizer state after build
x = keras.ops.ones((4, 32))
y = layer(x)
print(layer.kq)   # Quantizer(q_type=kbi, name=q_dense_kq, built=True)
print(layer.qkernel.shape)  # quantized kernel tensor
```

--------------------------------

### Prepare and Create Datasets

Source: https://github.com/calad0i/hgq2/blob/master/example/small_jet_tagger.ipynb

Converts the target labels to categorical format and creates training and testing datasets using the Dataset utility. Specifies batch size and device for data loading.

```python
_y_train = keras.utils.to_categorical(y_train, 5)
_y_test = keras.utils.to_categorical(y_test, 5)

dataset_train = Dataset(X_train, _y_train, batch_size=33200, device='gpu:0')
dataset_test = Dataset(X_test, _y_test, batch_size=33200, device='gpu:0')
```

--------------------------------

### Create DataLoaders

Source: https://github.com/calad0i/hgq2/blob/master/example/larger_jet_tagger.ipynb

Initializes Dataset objects for training, validation, and testing with specified batch sizes and device allocation.

```python
train_data = Dataset(X_train, y_train, batch_size=2480, device='gpu:0')
val_data = Dataset(X_val, y_val, batch_size=2480, device='gpu:0')
test_data = Dataset(X_test, y_test, batch_size=2600, device='gpu:0')
```

--------------------------------

### Trace Model and Generate Verilog

Source: https://github.com/calad0i/hgq2/blob/master/example/larger_jet_tagger.ipynb

Traces a model using specified solver options and hardware configuration, then generates a Verilog model. The Verilog model is saved to a binary file and written to a project directory.

```python
inp, out = trace_model(model, solver_options={'hard_dc': 2}, hwconf=HWConfig(1, -1, -1))
solution = comb_trace(inp, out)
solution.save_binary('/tmp/emulator.bin')
verilog_model = VerilogModel(
    solution,
    prj_name='jet_classifier_large',
    path='/tmp/verilog_test',
    part_name='xcvu13p-flga2577-2-e',
    clock_period=2,
    clock_uncertainty=0.0,
    latency_cutoff=2,
)
verilog_model.write()
```

--------------------------------

### Display Verilog Model Info

Source: https://github.com/calad0i/hgq2/blob/master/example/larger_jet_tagger.ipynb

Prints information about the generated Verilog model, including top module, input/output dimensions, stages, and estimated resource cost.

```python
verilog_model
```

--------------------------------

### Load and Preprocess Data Paths

Source: https://github.com/calad0i/hgq2/blob/master/example/larger_jet_tagger.ipynb

Loads H5 file paths for training and validation datasets from specified directories.

```python
train_path_root = Path('/tmp/train')
test_path_root = Path('/tmp/val')
train_paths = list(train_path_root.glob('**/*.h5'))
test_paths = list(test_path_root.glob('**/*.h5'))
```

--------------------------------

### Create Save Directory

Source: https://github.com/calad0i/hgq2/blob/master/example/larger_jet_tagger.ipynb

Creates a directory for saving model checkpoints and results if it doesn't exist.

```python
save_path = Path('/tmp/results')
save_path.mkdir(parents=True, exist_ok=True)
```

--------------------------------

### Implement Quantized Einsum-Dense Layers with QEinsumDense and QEinsumDenseBatchnorm

Source: https://context7.com/calad0i/hgq2/llms.txt

Use QEinsumDense for quantized arbitrary tensor contractions and QEinsumDenseBatchnorm to fuse batch normalization for hardware efficiency. These layers support folding BN statistics into the quantized kernel. Configure weight and datalane quantizers using QuantizerConfigScope.

```python
import keras
from hgq.layers import QEinsumDense, QEinsumDenseBatchnorm
from hgq.config import QuantizerConfigScope, LayerConfigScope

with (
    QuantizerConfigScope(place='weight', b0=8, i0=3, overflow_mode='SAT_SYM'),
    QuantizerConfigScope(place='datalane', i0=4, f0=4, overflow_mode='WRAP'),
    LayerConfigScope(enable_ebops=True, beta0=1e-5),
):
    inputs = keras.Input(shape=(16, 32))

    # Standard einsum dense: 'abc,cd->abd' maps (batch, seq, in) -> (batch, seq, out)
    x = QEinsumDense(equation='abc,cd->abd', output_shape=(16, 64), bias_axes='d')(inputs)

    # Fused BN+Dense variant: merges batchnorm into weight quantization
    x = QEinsumDenseBatchnorm(equation='abc,cd->abd', output_shape=(16, 10), bias_axes='d')(x)

    model = keras.Model(inputs, x)

model.compile(optimizer='adam', loss='mse')
```

--------------------------------

### Softmax Layer Usage Note

Source: https://github.com/calad0i/hgq2/blob/master/example/small_jet_tagger.ipynb

A note regarding the use of QSoftmax, highlighting its bit-accurate nature but cautioning about potential zero outputs that can cause divergence in cross-entropy calculations.

```python
use_softmax = False
# QSoftmax is bit-accurate, but it can give exactly zero now: xentropy will diverge and thus USE WITH CAUTION
```

--------------------------------

### Define Callbacks for Model Training

Source: https://github.com/calad0i/hgq2/blob/master/example/larger_jet_tagger.ipynb

Sets up various Keras callbacks including progress bar, early stopping on NaN, Pareto front checkpointing, free EBOPs calculation, and a beta learning rate scheduler.

```python
pbar = PBar(metric='loss: {loss:.2f}/{val_loss:.2f} - acc: {accuracy:.2%}/{val_accuracy:.2%} - beta: {beta:.2e}')

terminate_on_nan = keras.callbacks.TerminateOnNaN()

save = ParetoFront(
    path=save_path / 'ckpts',
    fname_format='epoch={epoch}-acc={accuracy:.2%}-val_acc={val_accuracy:.2%}-EBOPs={ebops}.keras',
    metrics=['val_accuracy', 'ebops'],
    enable_if=lambda x: x['val_accuracy'] > 0.5,
    sides=[1, -1],
)

ebops = FreeEBOPs()
beta_sched = BetaScheduler(PieceWiseSchedule([[0, 1e-7, 'linear'], [1000, 5.0e-7, 'constant']]))

callbacks = [beta_sched, ebops, save, pbar, terminate_on_nan]
```

--------------------------------

### Import Dataset Utility

Source: https://github.com/calad0i/hgq2/blob/master/example/small_jet_tagger.ipynb

Imports the Dataset utility class from hgq.utils.sugar, which is used for creating data loaders for training and testing.

```python
from hgq.utils.sugar import Dataset
```

--------------------------------

### Define Callbacks

Source: https://github.com/calad0i/hgq2/blob/master/example/small_jet_tagger.ipynb

Sets up custom callbacks for training, including a progress bar (`PBar`), event-based operations (`FreeEBOPs`), and termination on NaN values.

```python
pbar = PBar('loss: {loss:.3f}/{val_loss:.3f} - acc: {accuracy:.3f}/{val_accuracy:.3f}')
ebops = FreeEBOPs()
nan_terminate = keras.callbacks.TerminateOnNaN()
callbacks = [ebops, pbar, nan_terminate]
```

--------------------------------

### Using Quantized Convolutional Layers (QConv1D, QConv2D)

Source: https://context7.com/calad0i/hgq2/llms.txt

Illustrates the use of QConv1D and QConv2D layers within a Keras model, demonstrating how to apply quantization configurations and standard convolution arguments. This is suitable for image and sequence processing tasks.

```python
import keras
from hgq.layers import QConv1D, QConv2D
from hgq.config import QuantizerConfigScope, LayerConfigScope

with (
    QuantizerConfigScope(place='weight', b0=8, i0=3, overflow_mode='SAT_SYM'),
    QuantizerConfigScope(place='datalane', i0=4, f0=4, overflow_mode='WRAP'),
    LayerConfigScope(enable_ebops=True, beta0=1e-5),
):
    # 1D convolution for sequence/time-series data
    conv1d = QConv1D(filters=32, kernel_size=3, activation='relu', padding='causal')

    # 2D convolution for images
    inputs = keras.Input(shape=(28, 28, 1))
    x = QConv2D(32, (3, 3), activation='relu', padding='same')(inputs)
    x = keras.layers.MaxPooling2D((2, 2))(x)
    x = QConv2D(64, (3, 3), activation='relu', padding='same')(x)
    x = keras.layers.GlobalAveragePooling2D()(x)
    outputs = keras.layers.Dense(10)(x)
    model = keras.Model(inputs, outputs)

model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
model.summary()
```

--------------------------------

### Create Quantized Recurrent Layers with QSimpleRNN and QGRU

Source: https://context7.com/calad0i/hgq2/llms.txt

Employ QSimpleRNN and QGRU for quantized sequence modeling. These layers are functional for training and simulation but lack full hls4ml deployment support. Configure weight and datalane quantizers using QuantizerConfigScope.

```python
import keras
from hgq.layers import QSimpleRNN, QGRU
from hgq.config import QuantizerConfigScope, LayerConfigScope

with (
    QuantizerConfigScope(place='weight', b0=8, i0=3, overflow_mode='SAT_SYM'),
    QuantizerConfigScope(place='datalane', i0=4, f0=4, overflow_mode='WRAP'),
    LayerConfigScope(enable_ebops=True, beta0=1e-5),
):
    inputs = keras.Input(shape=(20, 16))  # (timesteps, features)
    x = QGRU(units=32, return_sequences=True)(inputs)
    x = QSimpleRNN(units=16)(x)
    outputs = keras.layers.Dense(4)(x)
    model = keras.Model(inputs, outputs)

model.compile(optimizer='adam', loss='mse')
```

--------------------------------

### Configure per-quantizer settings with QuantizerConfig

Source: https://context7.com/calad0i/hgq2/llms.txt

Define specific quantization parameters for individual layers using QuantizerConfig. Supports 'kbi', 'kif', and 'float' quantization types. Apply custom configurations to layer arguments like `iq_conf` and `kq_conf`.

```python
from hgq.config import QuantizerConfig
from hgq.layers import QDense

# KBI config: total bits (b) and integer bits (i) are trainable
kbi_conf = QuantizerConfig(
    q_type='kbi',
    place='weight',
    b0=8,            # initial total bits (excl. sign)
    i0=3,            # initial integer bits (excl. sign)
    k0=True,         # allow negative values (signed)
    overflow_mode='SAT_SYM',
    round_mode='RND',
)

# KIF config: integer bits (i) and fractional bits (f) are trainable
kif_conf = QuantizerConfig(
    q_type='kif',
    place='datalane',
    i0=4,            # initial integer bits
    f0=4,            # initial fractional bits
    overflow_mode='WRAP',
    round_mode='RND',
)

# Float/minifloat config (training only; no hls4ml synthesis support)
float_conf = QuantizerConfig(
    q_type='float',
    place='datalane',
    m0=3,            # initial mantissa bits
    e0=4,            # initial exponent bits
)

# Apply per-layer overrides: custom kernel and input quantizer configs
layer = QDense(64, kq_conf=kbi_conf, iq_conf=kif_conf)

```

--------------------------------

### Per-Batch Quantization for Activations

Source: https://github.com/calad0i/hgq2/blob/master/docs/getting_started.md

Configure activations for per-batch quantization by specifying the 'datalane' place and setting homogeneous_axis to a tuple containing the batch dimension (e.g., (0,)). This applies quantization parameters across the batch dimension.

```python
# Per-batch quantization for activations
with QuantizerConfigScope(place='datalane', heterogeneous_axis=None, homogeneous_axis=(0,)):
    # Model creation
```

--------------------------------

### Train Quantized Model with Beta Scheduler

Source: https://github.com/calad0i/hgq2/blob/master/docs/getting_started.md

Train a quantized model using standard Keras `fit` method. Utilize the `FreeEBOPs` callback for dynamic beta scheduling to manage the resource-performance tradeoff during training.

```python
# Prepare data (example for MNIST)
(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
x_train = x_train.astype('float32') / 255.0
x_test = x_test.astype('float32') / 255.0

from hgq.utils.sugar import FreeEBOPs
ebops = FreeEBOPs()

# Train model with beta scheduler
history = model.fit(
    x_train, y_train,
    batch_size=128,
    epochs=15,
    validation_data=(x_test, y_test),
    callbacks=[ebops] # It is recommended to use the FreeEBOPs callback to monitor EBOPs during training
    verbose=2
)
```

--------------------------------

### Import Verilog Code Generator

Source: https://github.com/calad0i/hgq2/blob/master/example/larger_jet_tagger.ipynb

Imports the VerilogModel class for potential hardware code generation from the trained model.

```python
from da4ml.codegen import VerilogModel
```

--------------------------------

### Integrating QBatchNormalization in a Model

Source: https://context7.com/calad0i/hgq2/llms.txt

Demonstrates how to incorporate the QBatchNormalization layer into a Keras model, typically after a quantized convolutional layer. This layer supports quantization and can be fused with dense layers for optimized deployments.

```python
import keras
from hgq.layers import QBatchNormalization, QConv2D, QDense
from hgq.config import QuantizerConfigScope, LayerConfigScope

with (
    QuantizerConfigScope(place='weight', overflow_mode='SAT_SYM', b0=8, i0=3),
    QuantizerConfigScope(place='datalane', overflow_mode='WRAP', i0=4, f0=4),
    LayerConfigScope(enable_ebops=True, beta0=1e-5),
):
    inputs = keras.Input(shape=(32, 32, 3))
    x = QConv2D(64, (3, 3), padding='same', use_bias=False)(inputs)
    x = QBatchNormalization()(x)
    x = keras.layers.Activation('relu')(x)
    x = keras.layers.GlobalAveragePooling2D()(x)
    x = QDense(10)(x)
    model = keras.Model(inputs, x)

model.compile(optimizer='adam', loss='sparse_categorical_crossentropy')
```

--------------------------------

### Per-Channel Quantization for Weights

Source: https://github.com/calad0i/hgq2/blob/master/docs/getting_started.md

Configure weights for per-channel quantization by setting heterogeneous_axis to None and homogeneous_axis to an empty tuple within a QuantizerConfigScope. This ensures that quantization parameters are unique for each channel of the weights.

```python
# Per-channel quantization for weights
with QuantizerConfigScope(heterogeneous_axis=None, homogeneous_axis=()):
    # Model creation
```

--------------------------------

### Manual Beta Scheduling with BetaScheduler Callback

Source: https://context7.com/calad0i/hgq2/llms.txt

The `BetaScheduler` callback allows manual adjustment of the `beta` regularization parameter for quantized layers based on a user-defined schedule. Higher `beta` increases compression, while lower `beta` prioritizes accuracy. The schedule can be constant, logarithmic, or piecewise.

```python
from hgq.utils.sugar import BetaScheduler, PieceWiseSchedule, FreeEBOPs

# Piecewise schedule: warm up for 5 epochs, then ramp from 1e-6 to 1e-3 logarithmically
schedule = PieceWiseSchedule([
    (0,  0.0,  'constant'),   # epochs 0-4: beta=0 (no regularization, warm-up)
    (5,  1e-6, 'log'),        # epochs 5-24: log ramp to 1e-3
    (25, 1e-3, 'constant'),   # epochs 25+: hold at 1e-3
])

beta_cb   = BetaScheduler(schedule)
ebops_cb  = FreeEBOPs()

history = model.fit(
    x_train, y_train,
    batch_size=256,
    epochs=40,
    validation_data=(x_val, y_val),
    callbacks=[beta_cb, ebops_cb],
)
```

--------------------------------

### Import Libraries for Keras Jet Tagger

Source: https://github.com/calad0i/hgq2/blob/master/example/larger_jet_tagger.ipynb

Imports necessary libraries for Keras, JAX backend, data handling, and quantization utilities.

```python
import os

os.environ['KERAS_BACKEND'] = 'jax'

from pathlib import Path

import h5py as h5
import keras
import numpy as np

from hgq.config import QuantizerConfigScope
from hgq.layers import QAdd, QAveragePooling1D, QEinsumDenseBatchnorm, QGlobalAveragePooling1D
from hgq.utils import trace_minmax
from hgq.utils.sugar import BetaScheduler, Dataset, FreeEBOPs, ParetoFront, PBar, PieceWiseSchedule
```

--------------------------------

### Implement Quantized Lookup-Table Activation with QUnaryFunctionLUT

Source: https://context7.com/calad0i/hgq2/llms.txt

Use QUnaryFunctionLUT for non-linear activations like sigmoid or tanh when bit-exact hls4ml synthesis is required. Both input and output quantizers are trainable. Configure output table and input quantizer precision using QuantizerConfig.

```python
import keras
from hgq.layers import QDense, QUnaryFunctionLUT
from hgq.config import QuantizerConfig, LayerConfigScope

lut_conf = QuantizerConfig('kif', 'table', i0=1, f0=6)  # output table precision
iq_conf  = QuantizerConfig('kif', 'datalane', i0=3, f0=4, overflow_mode='SAT_SYM')

with LayerConfigScope(enable_ebops=True, beta0=1e-5):
    model = keras.Sequential([
        QDense(64),
        QUnaryFunctionLUT(
            activation='sigmoid',
            iq_conf=iq_conf,
            oq_conf=lut_conf,
            allow_heterogeneous_input=True,   # per-neuron input precision
            allow_heterogeneous_table=True,   # per-neuron table precision
        ),
        QDense(10),
    ])

model.compile(optimizer='adam', loss='sparse_categorical_crossentropy')
```

--------------------------------

### Load and Preprocess LHC Jet Dataset

Source: https://github.com/calad0i/hgq2/blob/master/example/small_jet_tagger.ipynb

Loads the LHC jet dataset from OpenML, handles potential zstd compression, and preprocesses the data including feature scaling. This function is essential for preparing the dataset for model training.

```python
import pickle as pkl
from pathlib import Path

from sklearn.datasets import fetch_openml
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler


def get_data(data_path: Path, seed=42):
    try:
        import zstd
    except ImportError:
        zstd = None
    if not os.path.exists(data_path):
        print('Downloading data...')
        data = fetch_openml('hls4ml_lhc_jets_hlf')
        buf = pkl.dumps(data)
        with open(data_path, 'wb') as f:
            if zstd is not None:
                buf = zstd.compress(buf)
            f.write(buf)
    else:
        os.makedirs(data_path.parent, exist_ok=True)
        with open(data_path, 'rb') as f:
            buf = f.read()
            if zstd is not None:
                buf = zstd.decompress(buf)
            data = pkl.loads(buf)

    X, y = data['data'], data['target']
    codecs = {'g': 0, 'q': 1, 't': 4, 'w': 2, 'z': 3}
    y = np.array([codecs[i] for i in y])

    X_train_val, X_test, y_train_val, y_test = train_test_split(X, y, test_size=0.2, random_state=seed)

    X_train_val, X_test, y_train_val, y_test = X_train_val.astype(np.float32), X_test.astype(np.float32), y_train_val, y_test

    scaler = StandardScaler()
    X_train_val = scaler.fit_transform(X_train_val)
    X_test = scaler.transform(X_test)

    X_train_val = X_train_val.astype(np.float32)
    y_train_val = y_train_val.astype(np.float32)

    return X_train_val, X_test, y_train_val, y_test


X_train, X_test, y_train, y_test = get_data(Path('/tmp/inp_data.zst'))
```

--------------------------------

### Calibrate Integer Bits for WRAP Activations

Source: https://context7.com/calad0i/hgq2/llms.txt

Calibrates integer bits for WRAP activations using trace_minmax. Ensure the model and test data are prepared before use.

```python
total_ebops = trace_minmax(model, x_test, batch_size=512, verbose=True)
print(f"Calibrated EBOPs: {total_ebops:,}")
```

--------------------------------

### Plot Training History

Source: https://github.com/calad0i/hgq2/blob/master/example/small_jet_tagger.ipynb

Visualizes the training history, plotting EBOPs and accuracy over epochs, and validation accuracy against training accuracy.

```python
plt.plot(history.history['ebops'], '.')
plt.ylabel('EBOPs')
plt.yscale('log')
plt.xlabel('Epoch')
plt.show()

plt.plot(history.history['val_accuracy'], '.', label='val')
plt.plot(history.history['accuracy'], '.', label='train')
plt.legend()
plt.ylabel('Accuracy')
plt.ylim(0.65, 0.77)
plt.xlabel('Epoch')
plt.show()
```

```python
plt.plot(history.history['ebops'], history.history['val_accuracy'], '.')
plt.xscale('log')
plt.xlabel('EBOPs')
plt.ylabel('Validation accuracy')
plt.ylim(0.7, 0.765)
plt.xticks([3000, 10000, 30000, 100000], ['3k', '10k', '30k', '100k'])
plt.show()
```

--------------------------------

### Trace Minimum and Maximum of a Model

Source: https://github.com/calad0i/hgq2/blob/master/example/small_jet_tagger.ipynb

Use `trace_minmax` to analyze the minimum and maximum values of a model with training data. Ensure `hgq.utils` is imported.

```python
from hgq.utils import trace_minmax

trace_minmax(model, dataset_train)
```

--------------------------------

### Load Best Model Weights

Source: https://github.com/calad0i/hgq2/blob/master/example/larger_jet_tagger.ipynb

Loads the model weights corresponding to the best performance saved by the ParetoFront callback.

```python
model.load_weights(save.paths[0])
```

--------------------------------

### Compile Verilog Model

Source: https://github.com/calad0i/hgq2/blob/master/example/larger_jet_tagger.ipynb

Compiles the Verilog model. This step is necessary before running predictions with the Verilog model.

```python
verilog_model._compile()
```

--------------------------------

### Create a Quantized Model with HGQ2

Source: https://github.com/calad0i/hgq2/blob/master/docs/getting_started.md

Define a Keras model using HGQ2's quantized layers (QConv2D, QDense) with specific quantization configurations for weights and activations. Ensure LayerConfigScope is used for enabling EBOPs.

```python
import keras
import numpy as np
from hgq.layers import QConv2D, QDense
from hgq.config import LayerConfigScope, QuantizerConfigScope

# First, set up quantization configuration
# For weights, use SAT_SYM overflow mode
with QuantizerConfigScope(q_type='kif', place='weight', overflow_mode='SAT_SYM', round_mode='RND'):
    # For activations, use different config
    with QuantizerConfigScope(q_type='kif', place='datalane', overflow_mode='WRAP', round_mode='RND'):
        with LayerConfigScope(enable_ebops=True, beta0=1e-5):
            # Create model with quantized layers
            model = keras.Sequential([
                keras.layers.Reshape((28, 28, 1)),
                keras.layers.MaxPooling2D((2, 2)),
                QConv2D(16, (3, 3), activation='relu'),
                keras.layers.MaxPooling2D((2, 2)),
                QConv2D(32, (3, 3), activation='relu'),
                keras.layers.MaxPooling2D((2, 2)),
                keras.layers.Flatten(),
                QDense(10)
            ])

# Compile model as usual
model.compile(
    optimizer=keras.optimizers.Adam(learning_rate=0.001),
    loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
    metrics=['accuracy']
)
```

--------------------------------

### Calibrate Activation Range with trace_minmax Utility

Source: https://context7.com/calad0i/hgq2/llms.txt

Use trace_minmax to calibrate activation quantizers for bit-exact inference when using WRAP overflow mode. This utility determines minimum integer bits required by each activation quantizer. It can optionally return model predictions on the calibration data.

```python
import numpy as np
from hgq.utils import trace_minmax

# Assume `model` is a trained HGQ2 model and `x_test` is a representative dataset
# verbose=True prints per-layer EBOPs and shows a progress bar
total_ebops = trace_minmax(model, x_test, batch_size=512, verbose=True)
print(f"Total EBOPs after calibration: {total_ebops:,}")

# Get model predictions on calibration data at the same time
preds = trace_minmax(model, x_test, batch_size=512, return_results=True)
# preds is a numpy array of shape matching model output

# Example output:
# q_conv2d_iq  : 124928
```

--------------------------------

### Create Quantized Multi-Head Attention with QMultiHeadAttention

Source: https://context7.com/calad0i/hgq2/llms.txt

Utilize QMultiHeadAttention for transformer architectures targeting FPGA deployment, offering bit-accurate softmax and scaled dot-product attention. Currently alpha quality and only supports io_parallel in hls4ml. Configure weight and datalane quantizers using QuantizerConfigScope.

```python
import keras
from hgq.layers import QMultiHeadAttention, QDense
from hgq.config import QuantizerConfigScope, LayerConfigScope

with (
    QuantizerConfigScope(place='weight', b0=8, i0=3, overflow_mode='SAT_SYM'),
    QuantizerConfigScope(place='datalane', i0=4, f0=4, overflow_mode='WRAP'),
    LayerConfigScope(enable_ebops=True, beta0=1e-6),
):
    seq_len, d_model = 16, 32
    inputs = keras.Input(shape=(seq_len, d_model))
    attn_out = QMultiHeadAttention(num_heads=4, key_dim=8)(inputs, inputs)
    outputs = QDense(d_model)(attn_out)
    model = keras.Model(inputs, outputs)

model.compile(optimizer='adam', loss='mse')
```

--------------------------------

### Split Data and Normalize

Source: https://github.com/calad0i/hgq2/blob/master/example/larger_jet_tagger.ipynb

Splits the training data into training and validation sets, then normalizes the data using the mean and standard deviation of the training set.

```python
N_train = int(0.8 * len(X_train))
order = np.random.permutation(len(X_train))
X_train = X_train[order]
y_train = y_train[order]
X_val = X_train[N_train:]
y_val = y_train[N_train:]
X_train = X_train[:N_train]
y_train = y_train[:N_train]

_std, _bias = np.std(X_train.astype(np.float32), axis=(0, 1)), np.mean(X_train.astype(np.float32), axis=(0, 1))
X_train = (X_train - _bias) / _std
X_val = (X_val - _bias) / _std
X_test = (X_test - _bias) / _std
```

--------------------------------

### Convert Keras Model to hls4ml for Hardware

Source: https://github.com/calad0i/hgq2/blob/master/docs/getting_started.md

Convert a trained Keras model to an hls4ml model for hardware deployment. This involves tracing activation ranges and then using `convert_from_keras_model` with specified backend and I/O types. Finally, write and compile the C++ model and test for mismatches.

```python
from hgq.utils import trace_minmax
from hls4ml.converters import convert_from_keras_model

# Trace the required number of integer bits for activations
# This step is only necessary when using WRAP overflow mode (recommended) for data.

trace_minmax(model, x_test, verbose=True)

# Convert to hls4ml model
hls_model = convert_from_keras_model(
    model,
    output_dir='hls_project',
    backend='Vitis',
    io_type='io_parallel'  # or 'io_stream' for streaming interface
)

# Write out and compile the C++ model for simulation
hls_model.write()
hls_model._compile()

# Test for mismatches
# HGQ2 and hls4ml should produce the same output, up to machine precision
# Notice that due to the quantization, internal mismatches may be amplified, but the vast majority of the output should match
keras_pred = model.predict(x_test)
hls_pred = hls_model.predict(x_test)
print(f"{np.sum(keras_pred != hls_pred)} / {np.prod(keras_pred.shape)} value mismatches")
```

--------------------------------

### Compare Keras and Verilog Predictions

Source: https://github.com/calad0i/hgq2/blob/master/example/larger_jet_tagger.ipynb

Compares the predictions from the Keras model and the Verilog model to ensure consistency.

```python
np.all(r_keras == r_verilog)
```

--------------------------------

### Automatic Beta Adjustment with BetaPID Controller

Source: https://context7.com/calad0i/hgq2/llms.txt

The `BetaPID` callback automatically tunes the `beta` regularization strength using a PID control loop to meet a target EBOPs budget. This removes the need for manual schedule tuning. It requires setting `target_ebops`, `init_beta`, PID gains (`p`, `i`, `d`), `warmup` epochs, and optionally `log` scale and `max_beta`/`min_beta` limits.

```python
from hgq.utils.sugar import BetaPID, EarlyStoppingWithEbopsThres

target_ebops = 500_000  # desired resource budget

pid_cb = BetaPID(
    target_ebops=target_ebops,
    init_beta=1e-6,     # initial beta before PID kicks in
    p=1.0,              # proportional gain
    i=2e-3,             # integral gain
    d=0.0,              # derivative gain (keep 0 due to EBOPs noise)
    warmup=10,          # epochs before PID activates
    log=True,           # operate in log scale for smoother control
    max_beta=1e-1,
    min_beta=0.0,
)

# Stop training when EBOPs < target AND val_loss stops improving
early_stop = EarlyStoppingWithEbopsThres(
    ebops_threshold=target_ebops,
    monitor='val_loss',
    patience=5,
    restore_best_weights=True,
)

history = model.fit(
    x_train, y_train,
    batch_size=256,
    epochs=200,
    validation_data=(x_val, y_val),
    callbacks=[pid_cb, early_stop],
)
```

--------------------------------

### Compile Model

Source: https://github.com/calad0i/hgq2/blob/master/example/small_jet_tagger.ipynb

Compiles the Keras model with specified optimizer, loss function, and metrics. `jit_compile=True` and `steps_per_execution=4` are used for performance optimization.

```python
model = build_model(use_softmax=use_softmax, beta0=0.5e-5)
if not use_softmax:
    loss = keras.losses.CategoricalCrossentropy(from_logits=True)
else:
    loss = keras.losses.CategoricalHinge()
opt = keras.optimizers.Adam(learning_rate=5e-3)

model.compile(opt, loss, metrics=['accuracy'], jit_compile=True, steps_per_execution=4)
```

--------------------------------

### Train the Keras Model

Source: https://github.com/calad0i/hgq2/blob/master/example/larger_jet_tagger.ipynb

Trains the Keras model for 200 epochs using the prepared training and validation data, along with the defined callbacks.

```python
model.fit(train_data, epochs=200, validation_data=val_data, callbacks=callbacks, verbose=0)
```

--------------------------------

### Trace Min/Max Values

Source: https://github.com/calad0i/hgq2/blob/master/example/larger_jet_tagger.ipynb

Traces the minimum and maximum values of model weights and activations for the training and validation datasets.

```python
trace_minmax(model, train_data)
trace_minmax(model, val_data, reset=False)
```

--------------------------------

### Display Training Data Shape

Source: https://github.com/calad0i/hgq2/blob/master/example/larger_jet_tagger.ipynb

Prints the shape of the training data array.

```python
X_train.shape
```

--------------------------------

### Train Model

Source: https://github.com/calad0i/hgq2/blob/master/example/small_jet_tagger.ipynb

Trains the compiled model using the provided training and validation datasets. `epochs`, `batch_size`, and `callbacks` are specified.

```python
history = model.fit(dataset_train, epochs=3000, batch_size=33200, validation_data=dataset_test, verbose=0, callbacks=callbacks)
```

--------------------------------

### Configure Quantization Type with Scope

Source: https://github.com/calad0i/hgq2/blob/master/docs/getting_started.md

Use QuantizerConfigScope to set global quantization parameters like type, overflow, and rounding mode for model creation. This is useful for applying consistent settings across a model or a section of it.

```python
# Configure for specific quantization types
with QuantizerConfigScope(q_type='kif', overflow_mode='SAT_SYM', round_mode='RND'):
    # Model creation
```

--------------------------------

### Configure Quantizer Scopes and Layers

Source: https://github.com/calad0i/hgq2/blob/master/example/small_jet_tagger.ipynb

Defines QuantizerConfigScope and QuantizerConfig objects to customize quantization parameters for different layers or parts of the model. These configurations influence precision and overflow handling.

```python
from hgq.regularizers import MonoL1

# Skipping these should also work.
# Usually, the default configs are good enough for most cases, but the initial number of bits, `[bif]0`
# may need to be increased. If you see that the model is not converging, you can try increasing these values.
scope0 = QuantizerConfigScope(place='all', k0=1, b0=3, i0=0, default_q_type='kbi', overflow_mode='sat_sym')
scope1 = QuantizerConfigScope(place='datalane', k0=0, default_q_type='kif', overflow_mode='wrap', f0=3, i0=3)

exp_table_conf = QuantizerConfig('kif', 'table', k0=0, i0=1, f0=8, overflow_mode='sat_sym')
inv_table_conf = QuantizerConfig('kif', 'table', k0=1, i0=4, f0=4, overflow_mode='sat_sym')
# Layer scope will over formal one. When using scope0, scope1, 'datalane' config will be overriden with config in scope1
```

--------------------------------

### Build Quantized Model

Source: https://github.com/calad0i/hgq2/blob/master/example/small_jet_tagger.ipynb

Defines a sequential Keras model with quantized layers. Use `use_softmax=False` for classification tasks where softmax is applied in the loss function.

```python
def build_model(use_softmax=False, beta0=1e-5):
    with scope0, scope1:
        iq_conf = QuantizerConfig(place='datalane', k0=1)
        oq_conf = QuantizerConfig(place='datalane', k0=1, fr=MonoL1(1e-3))
        layers = [
            QDense(64, beta0=beta0, iq_conf=iq_conf, activation='relu', name='dense_0'),
            QDense(32, beta0=beta0, activation='relu', name='dense_1'),
            QDense(32, beta0=beta0, activation='relu', name='dense_2'),
            QDense(5, beta0=beta0, enable_oq=not use_softmax, name='dense_3', oq_conf=oq_conf),
            # QEinsumDense('...c,oc->...o', 64, bias_axes='o', beta0=beta0, iq_conf=iq_conf, activation='relu', bame='dense_0'),
            # QEinsumDense('...c,oc->...o', 32, bias_axes='o', beta0=beta0, activation='relu', name='dense_1'),
            # QEinsumDense('...c,oc->...o', 32, bias_axes='o', beta0=beta0, activation='relu', name='dense_2'),
            # QEinsumDense('...c,oc->...o', 5, bias_axes='o', beta0=beta0, enable_oq=not use_softmax, name='dense_3'),
        ]
        if use_softmax:
            layers.append(QSoftmax(exp_oq_conf=exp_table_conf, inv_oq_conf=inv_table_conf))

    model = keras.models.Sequential(layers)
    return model
```

--------------------------------

### Verify Bit-Exact Match Between Keras and hls4ml Models

Source: https://context7.com/calad0i/hgq2/llms.txt

Verifies bit-exact match by comparing predictions from Keras and hls4ml models. Requires both models to be loaded and test data to be available.

```python
keras_pred = model.predict(x_test[:100])
hls_pred   = hls_model.predict(x_test[:100])
mismatches  = np.sum(keras_pred != hls_pred)
print(f"Value mismatches: {mismatches} / {keras_pred.size}")
# Expected: 0 or near-zero mismatches
```

--------------------------------

### Compile hls4ml Model

Source: https://github.com/calad0i/hgq2/blob/master/example/larger_jet_tagger.ipynb

Compiles the hls4ml model. This step is necessary before running predictions with the hls4ml model.

```python
hls4ml_model.compile()
```

--------------------------------

### Log EBOPs per Epoch with FreeEBOPs Callback

Source: https://context7.com/calad0i/hgq2/llms.txt

Use `FreeEBOPs` to log the total model EBOPs at the end of each epoch. This makes the EBOPs metric available for other callbacks like `ModelCheckpoint` or `TensorBoard` and for plotting.

```python
from hgq.utils.sugar import FreeEBOPs

ebops_cb = FreeEBOPs()

history = model.fit(
    x_train, y_train,
    batch_size=256,
    epochs=30,
    validation_data=(x_val, y_val),
    callbacks=[ebops_cb],
    verbose=1,
)

# 'ebops' is now in history.history
import matplotlib.pyplot as plt
plt.plot(history.history['ebops'])
plt.xlabel('Epoch')
plt.ylabel('EBOPs')
plt.title('Resource cost over training')
plt.show()
```