### Install Required Packages

Source: https://github.com/bamler-lab/constriction/blob/main/examples/python/03-tutorial-autoregressive-nlp-compression.ipynb

Install `constriction`, `tqdm`, and `unidecode` using pip. Restart the Jupyter kernel after installation.

```python
%pip install constriction~=0.3.5 tqdm~=4.62.3 unidecode~=1.3.2
```

--------------------------------

### Start Python REPL

Source: https://github.com/bamler-lab/constriction/blob/main/README.md

Starts an IPython REPL session that includes the compiled Python module of constriction, allowing for interactive testing.

```shell
poetry run ipython
```

--------------------------------

### Import PyTorch

Source: https://github.com/bamler-lab/constriction/blob/main/examples/python/03-tutorial-autoregressive-nlp-compression.ipynb

Import the PyTorch library to begin. Ensure PyTorch is installed before running this code.

```python
import torch
```

--------------------------------

### Install constriction for Python

Source: https://github.com/bamler-lab/constriction/blob/main/README-python.md

Install the constriction library using pip. Ensure you use a version compatible with '~=0.4.2'.

```bash
pip install constriction~=0.4.2
```

--------------------------------

### Install Constriction and SciPy

Source: https://github.com/bamler-lab/constriction/blob/main/examples/python/02-custom-entropy-models.ipynb

Installs the constriction library and SciPy, which are necessary for using custom entropy models. Remember to restart your Jupyter kernel after installation.

```python
%pip install constriction~=0.3.5 scipy  # (this will automatically also install numpy)
```

--------------------------------

### Hello World: Encode and Decode Message

Source: https://github.com/bamler-lab/constriction/blob/main/examples/python/01-hello-world.ipynb

Implements a basic encoding-decoding round trip using Constriction's ANS coder with a QuantizedGaussian model. This example demonstrates how to define a message, model, encode it, and then reconstruct the original message.

```python
import constriction
import numpy as np

# Define some example message and entropy model:
message = np.array([6,   10,   -4,    2,   -9,   41,   3,    0,    2  ], dtype=np.int32)
means   = np.array([2.5, 13.1, -1.1, -3.0, -6.1, 34.2, 2.8, -6.4, -3.1], dtype=np.float64)
stds    = np.array([4.1,  8.7,  6.2,  5.4, 24.1, 12.7, 4.9, 28.9,  4.2], dtype=np.float64)
model_family = constriction.stream.model.QuantizedGaussian(-100, 100) # We'll provide `means` and `stds` when encoding/decoding.
print(f"Original message: {message}")

# Encode the message:
encoder = constriction.stream.stack.AnsCoder()
encoder.encode_reverse(message, model_family, means, stds)

# Get and print the compressed representation:
compressed = encoder.get_compressed()
print(f"compressed representation: {compressed}")
print(f"(in binary: {[bin(word) for word in compressed]})")

# Decode the message:
decoder = constriction.stream.stack.AnsCoder(compressed) # (we could also just reuse `encoder`.)
reconstructed = decoder.decode(model_family, means, stds)
print(f"Reconstructed message: {reconstructed}")
assert np.all(reconstructed == message)
```

--------------------------------

### Start Model Training

Source: https://github.com/bamler-lab/constriction/blob/main/examples/python/03-tutorial-autoregressive-nlp-compression.ipynb

Execute the training script with your text corpus. This process can take a significant amount of time (around 30 minutes). The trained model will be saved to a file.

```python
!python char-rnn.pytorch/train.py shakespeare_train.txt
```

--------------------------------

### Install Constriction

Source: https://github.com/bamler-lab/constriction/blob/main/examples/python/01-hello-world.ipynb

Installs the Constriction library and its dependencies. Restart your Jupyter kernel after installation.

```python
%pip install --upgrade constriction~=0.3.5  # (this will automatically also install numpy)
```

--------------------------------

### Install Python Dependencies

Source: https://github.com/bamler-lab/constriction/blob/main/README.md

Installs Python dependencies using Poetry. Ensure you are in the repository's root directory. This command does not install the root package itself.

```shell
poetry install --no-root
```

--------------------------------

### Test Constriction Import

Source: https://github.com/bamler-lab/constriction/blob/main/examples/python/02-custom-entropy-models.ipynb

Verifies that the constriction library has been successfully installed and can be imported without errors.

```python
import constriction # This cell should produce no output (in particular, no error messages).
```

--------------------------------

### Python ANS Coder Example

Source: https://github.com/bamler-lab/constriction/blob/main/README.md

Demonstrates encoding and decoding using the ANS coder in Python. Ensure the message and entropy model are defined before use.

```python
encoder = constriction.stream.stack.AnsCoder()
encoder.encode_reverse(message, entropy_model)

compressed = encoder.get_compressed()
print(f"compressed representation: {compressed}")
print(f"(in binary: {[bin(word) for word in compressed]})")

decoder = constriction.stream.stack.AnsCoder(compressed)
decoded = decoder.decode(entropy_model, 9) # (decodes 9 symbols)
assert np.all(decoded == message) # (verifies correctness)
```

--------------------------------

### Rust ANS Coder Setup

Source: https://github.com/bamler-lab/constriction/blob/main/README.md

Add the constriction and probability crates to your Cargo.toml for Rust projects. For no_std mode, disable default features.

```toml
[dependencies]
constriction = "0.4.2"
probability = "0.20" # Not strictly required but used in many code examples.
```

```toml
[dependencies]
constriction = {version = "0.4.2", default-features = false} # for `no_std` mode
```

--------------------------------

### Handling Entropy Model Errors in Autoregressive Compression

Source: https://github.com/bamler-lab/constriction/blob/main/examples/python/03-tutorial-autoregressive-nlp-compression.ipynb

This example illustrates a common error scenario in autoregressive compression where the decoder encounters invalid compressed data due to discrepancies in entropy models. It highlights the brittleness of entropy coders and the necessity for careful implementation of entropy models.

```python
from constriction import RangeCoder
from pathlib import Path

# Assume 'model' is a trained autoregressive model and 'data' is the input text
# For demonstration, we'll simulate a scenario leading to an error

# Simulate compressed data that might become invalid
compressed_data = b'...' # Placeholder for actual compressed data

# Simulate an entropy model that might have subtle issues
class FaultyEntropyModel:
    def __init__(self):
        self.prob_table = {'a': 0.5, 'b': 0.5} # Simplified probability table

    def prob(self, char):
        return self.prob_table.get(char, 0.0001) # Return a small probability for unknown chars

    def encode(self, coder, char):
        # Simplified encoding logic that might introduce errors
        prob = self.prob(char)
        if prob > 0:
            coder.encode_symbol(prob)
        else:
            raise ValueError("Cannot encode symbol with zero probability")

    def decode(self, coder):
        # Simplified decoding logic that might fail on slightly off probabilities
        # In a real scenario, this would involve complex probability lookups and range updates
        # For this example, we simulate a failure point
        try:
            # Simulate decoding a character, which might fail if probabilities don't align
            # This is a conceptual representation of the error described in the text
            symbol = coder.decode_symbol(self.prob_table)
            return symbol
        except Exception as e:
            # This catch block is conceptual; the actual error is an AssertionError from the library
            print(f"Decoding error: {e}")
            raise


# --- Simulation of the error scenario --- 
# This part is conceptual and aims to represent the described error
# The actual error occurs within the constriction library's internal decoding process
# when the provided compressed data does not perfectly match the expected probabilities
# from the entropy model.

# Example of how the error might manifest (conceptual):
# try:
#     model = FaultyEntropyModel()
#     coder = RangeCoder()
#     # ... (code to write compressed data to a file) ...
#     # Assume compressed_data was generated with slight inaccuracies
#     # Now try to decode it with the same model
#     # The following line conceptually represents the decoding process that fails:
#     # decoded_char = model.decode(coder)
# except AssertionError as e:
#     print(f"Caught expected error: {e}")

# The provided text describes an AssertionError: "Tried to decode from compressed data that is invalid for the employed entropy model."
# This typically happens when the bitstream produced by the encoder, when interpreted using the decoder's probability model,
# falls outside the expected ranges, indicating a mismatch. This mismatch can arise from floating-point inaccuracies,
# incorrect probability updates, or using a different model for encoding and decoding.

# The text mentions decoding the first 883 characters successfully before failure.
# This implies that the initial part of the compressed data was consistent with the model,
# but a divergence occurred later, leading to the error.

# The core issue is the brittleness of entropy coders to small errors in the compressed data stream
# when compared against the entropy model used for decoding.

print("The provided text describes an error scenario related to entropy model mismatches during decoding.")
print("Implementing entropy models with care and ensuring consistency between encoder and decoder are crucial.")

```

--------------------------------

### Compress with General-Purpose Tools

Source: https://github.com/bamler-lab/constriction/blob/main/examples/python/03-tutorial-autoregressive-nlp-compression.ipynb

Compresses the first 10,000 characters of the test data using standard compression utilities like gzip, bzip2, and xz for comparison.

```bash
!head -c 10000 shakespeare_test.txt | gzip --best > shakespeare_test.txt.gzip
!head -c 10000 shakespeare_test.txt | bzip2 --best > shakespeare_test.txt.bz2
!head -c 10000 shakespeare_test.txt | xz --best > shakespeare_test.txt.xz
```

--------------------------------

### Huffman Coding (Python) - Symbol Encoding and Decoding

Source: https://context7.com/bamler-lab/constriction/llms.txt

Demonstrates building Huffman encoder and decoder trees from probabilities and then encoding/decoding a sequence of symbols. Ensure the number of symbols is provided for decoding.

```python
import constriction
import numpy as np

# Build encoder and decoder Huffman trees from the same probabilities
probabilities = np.array([0.3, 0.2, 0.2, 0.15, 0.15], dtype=np.float64)
encoder_tree = constriction.symbol.huffman.EncoderHuffmanTree(probabilities)
decoder_tree = constriction.symbol.huffman.DecoderHuffmanTree(probabilities)

# Encode a sequence of symbols (integers in {0,...,4})
symbols = np.array([0, 2, 3, 1, 4, 0], dtype=np.int32)
compressed, bits_written = constriction.symbol.huffman.huffman_encode(
    symbols, encoder_tree)

# Decode
decoded = constriction.symbol.huffman.huffman_decode(
    compressed, decoder_tree, len(symbols))
assert np.all(decoded == symbols)

```

--------------------------------

### Switching to Range Coder for Encoding/Decoding

Source: https://github.com/bamler-lab/constriction/blob/main/README-python.md

Shows how to switch from an ANS coder to a Range Coder for encoding and decoding while keeping the message and entropy model representation the same. Requires numpy.

```python
import constriction
import numpy as np

# Same representation of message and entropy model as in the previous example:
message = np.array([6, 10, -4, 2, 5, 2, 1, 0, 2], dtype=np.int32)
entropy_model = constriction.stream.model.QuantizedGaussian(-50, 50, 3.2, 9.6)

# Let's use a Range coder now:
encoder = constriction.stream.queue.RangeEncoder()         # <-- CHANGED LINE
encoder.encode(message, entropy_model)          # <-- (slightly) CHANGED LINE

compressed = encoder.get_compressed()
print(f"compressed representation: {compressed}")
print(f"(in binary: {[bin(word) for word in compressed]})")

decoder = constriction.stream.queue.RangeDecoder(compressed) #<--CHANGED LINE
decoded = decoder.decode(entropy_model, 9) # (decodes 9 symbols)
assert np.all(decoded == message)
```

--------------------------------

### Other Entropy Models

Source: https://context7.com/bamler-lab/constriction/llms.txt

Shows instantiation of QuantizedLaplace, QuantizedCauchy, and Binomial models with their respective parameters.

```python
import constriction

laplace  = constriction.stream.model.QuantizedLaplace(-50, 50, 0.0, 3.0)
cauchy   = constriction.stream.model.QuantizedCauchy(-50, 50, 0.0, 2.0)
binomial = constriction.stream.model.Binomial(20, 0.4)    # n=20, p=0.4
```

--------------------------------

### Add Constriction to Rust Project

Source: https://context7.com/bamler-lab/constriction/llms.txt

Add the Constriction library as a dependency in your Rust project's Cargo.toml file. The 'probability' crate is optional but used in many examples.

```toml
[dependencies]
constriction = "0.4.2"
probability = "0.20"   # optional, used in many examples
```

--------------------------------

### Compile and Benchmark Rust Library

Source: https://github.com/bamler-lab/constriction/blob/main/README.md

Use this command to compile the Rust library in release mode with optimizations and run the benchmarks.

```rust
cargo bench
```

--------------------------------

### Quantizing and Encoding with DefaultLeakyQuantizer

Source: https://context7.com/bamler-lab/constriction/llms.txt

Demonstrates quantizing Gaussian and Laplace distributions and encoding symbols using heterogeneous models with DefaultAnsCoder. Requires the probability and constriction crates.

```rust
use constriction::stream::{
    model::{DefaultLeakyQuantizer, DefaultContiguousCategoricalEntropyModel},
    stack::DefaultAnsCoder,
    Decode,
};
use probability::distribution::{Gaussian, Laplace};

let quantizer = DefaultLeakyQuantizer::new(-100i32..=100);

// Quantize a Gaussian
let gauss_model = quantizer.quantize(Gaussian::new(0.0f64, 10.0));

// Quantize a Laplace
let laplace_model = quantizer.quantize(Laplace::new(5.0f64, 3.0));

// Encode with heterogeneous models
let mut coder = DefaultAnsCoder::new();
coder.encode_symbols_reverse([
    (7i32,  gauss_model),
    (-3i32, laplace_model),
]).unwrap();

let (s1, s2): (i32, i32) = {
    let mut iter = coder.decode_symbols([gauss_model, laplace_model]);
    (iter.next().unwrap().unwrap(), iter.next().unwrap().unwrap())
};
assert_eq!((s1, s2), (7, -3));

// Categorical model from floating-point probabilities
let probs = [0.1f32, 0.4, 0.2, 0.3];
let cat = DefaultContiguousCategoricalEntropyModel
    ::from_floating_point_probabilities_fast(&probs, None)
    .unwrap();
let mut coder2 = DefaultAnsCoder::new();
coder2.encode_iid_symbols_reverse(&[0usize, 3, 1], &cat).unwrap();
let decoded: Vec<usize> = coder2
    .decode_iid_symbols(3, &cat)
    .collect::<Result<_, _>>()
    .unwrap();
assert_eq!(decoded, [0, 3, 1]);
```

--------------------------------

### Rust ANS Coder Encode and Decode

Source: https://github.com/bamler-lab/constriction/blob/main/README.md

Example of encoding and decoding symbols using the ANS coder in Rust. Requires defining symbols and entropy models. The coder operates as a stack, so symbols are encoded in reverse order.

```rust
use constriction::stream::{model::DefaultLeakyQuantizer, stack::DefaultAnsCoder, Decode};

// Let's use an ANS Coder in this example. Constriction also provides a Range
// Coder, a Huffman Coder, and an experimental new "Chain Coder".
let mut coder = DefaultAnsCoder::new();
 
// Define some data and a sequence of entropy models. We use quantized Gaussians here,
// but `constriction` also provides other models and allows you to implement your own.
let symbols = [23i32, -15, 78, 43, -69];
let quantizer = DefaultLeakyQuantizer::new(-100..=100);
let means = [35.2f64, -1.7, 30.1, 71.2, -75.1];
let stds = [10.1f64, 25.3, 23.8, 35.4, 3.9];
let models = means.iter().zip(&stds).map(
    |(&mean, &std)| quantizer.quantize(probability::distribution::Gaussian::new(mean, std))
);

// Encode symbols (in *reverse* order, because ANS Coding operates as a stack).
coder.encode_symbols_reverse(symbols.iter().zip(models.clone())).unwrap();

// Obtain temporary shared access to the compressed bit string. If you want ownership of the
// compressed bit string, call `.into_compressed()` instead of `.get_compressed()`.
println!("Encoded into {} bits: {:?}", coder.num_bits(), &*coder.get_compressed().unwrap());

// Decode the symbols and verify correctness.
let reconstructed = coder.decode_symbols(models).collect::<Result<Vec<_>, _>>().unwrap();
assert_eq!(reconstructed, symbols);
```

--------------------------------

### Basic Encoding and Decoding with ANS Coder

Source: https://github.com/bamler-lab/constriction/blob/main/README-python.md

Demonstrates a simple encoding-decoding round trip using an ANS coder and a QuantizedGaussian entropy model. Requires numpy.

```python
import constriction
import numpy as np

message = np.array([6, 10, -4, 2, 5, 2, 1, 0, 2], dtype=np.int32)

# Define an i.i.d. entropy model (see below for more complex models):
entropy_model = constriction.stream.model.QuantizedGaussian(-50, 50, 3.2, 9.6)

# Let's use an ANS coder in this example. See below for a Range Coder example.
encoder = constriction.stream.stack.AnsCoder()
encoder.encode_reverse(message, entropy_model)

compressed = encoder.get_compressed()
print(f"compressed representation: {compressed}")
print(f"(in binary: {[bin(word) for word in compressed]})")

decoder = constriction.stream.stack.AnsCoder(compressed)
decoded = decoder.decode(entropy_model, 9) # (decodes 9 symbols)
assert np.all(decoded == message)
```

--------------------------------

### Decode Sample Data using ANS Coder in Rust

Source: https://github.com/bamler-lab/constriction/blob/main/README-rust.md

Demonstrates decoding compressed data back into symbols using a DefaultAnsCoder. The same entropy models and quantizer used during encoding must be provided.

```rust
use constriction::stream::{stack::DefaultAnsCoder, model::DefaultLeakyQuantizer, Decode};
use probability::distribution::Gaussian;

fn decode_sample_data(compressed: Vec<u32>) -> Vec<i32> {
    // Create an ANS Coder with default word and state size from the compressed data:
    // (ANS uses the same type for encoding and decoding, which makes the method very flexible
    // and allows interleaving small encoding and decoding chunks, e.g., for bits-back coding.)
    let mut coder = DefaultAnsCoder::from_compressed(compressed).unwrap();

    // Same entropy models and quantizer we used for encoding:
    let means = [35.2, -1.7, 30.1, 71.2, -75.1];
    let stds = [10.1, 25.3, 23.8, 35.4, 3.9];
    let quantizer = DefaultLeakyQuantizer::new(-100..=100);

    // Decode the data:
    coder.decode_symbols(
        means.iter().zip(&stds).map(
            |(&mean, &std)| quantizer.quantize(Gaussian::new(mean, std))
    )).collect::<Result<Vec<_>, _>>().unwrap()
}

assert_eq!(decode_sample_data(vec![0x421C_7EC3, 0x000B_8ED1]), [23, -15, 78, 43, -69]);
```

--------------------------------

### QuantizedGaussian Model Usage

Source: https://context7.com/bamler-lab/constriction/llms.txt

Illustrates creating concrete QuantizedGaussian models and model families. Model families require per-symbol parameters during encoding and decoding.

```python
import constriction
import numpy as np

# Concrete model (fixed mean=2.0, std=5.0, domain {-10,...,10})
model_concrete = constriction.stream.model.QuantizedGaussian(-10, 10, 2.0, 5.0)

# Model family (domain fixed, mean/std provided per symbol at encode/decode time)
model_family = constriction.stream.model.QuantizedGaussian(-10, 10)
means = np.array([2.3, -1.7,  0.1,  2.2, -5.1], dtype=np.float32)
stds  = np.array([1.1,  5.3,  3.8,  1.4,  3.9], dtype=np.float32)

ans = constriction.stream.stack.AnsCoder()
symbols = np.array([2, -1, 0, 2, 3], dtype=np.int32)
ans.encode_reverse(symbols, model_family, means, stds)
decoded = ans.decode(model_family, means, stds)
assert np.all(decoded == symbols)
```

--------------------------------

### Huffman Coding (Python)

Source: https://context7.com/bamler-lab/constriction/llms.txt

Demonstrates building and using Huffman encoder and decoder trees for symbol coding in Python.

```APIDOC
## Huffman Coding (Python)

### Description
Symbol code (not a stream code) that builds optimal prefix-free code books from a probability distribution. Each symbol is assigned an integer number of bits, resulting in up to ~1 bit overhead per symbol in low-entropy regimes. Mainly provided for teaching purposes; prefer ANS or Range Coding for production use.

### Usage Example
```python
import constriction
import numpy as np

# Build encoder and decoder Huffman trees from the same probabilities
probabilities = np.array([0.3, 0.2, 0.2, 0.15, 0.15], dtype=np.float64)
encoder_tree = constriction.symbol.huffman.EncoderHuffmanTree(probabilities)
decoder_tree = constriction.symbol.huffman.DecoderHuffmanTree(probabilities)

# Encode a sequence of symbols (integers in {0,...,4})
symbols = np.array([0, 2, 3, 1, 4, 0], dtype=np.int32)
compressed, bits_written = constriction.symbol.huffman.huffman_encode(
    symbols, encoder_tree)

# Decode
decoded = constriction.symbol.huffman.huffman_decode(
    compressed, decoder_tree, len(symbols))
assert np.all(decoded == symbols)
```
```

--------------------------------

### Encode Sample Data using ANS Coder in Rust

Source: https://github.com/bamler-lab/constriction/blob/main/README-rust.md

Demonstrates encoding sample data using a DefaultAnsCoder and a quantized Gaussian distribution model. Ensure the `probability` crate is added as a dependency for Gaussian distributions.

```rust
use constriction::stream::{stack::DefaultAnsCoder, model::DefaultLeakyQuantizer};
use probability::distribution::Gaussian;

fn encode_sample_data() -> Vec<u32> {
    // Create an empty ANS Coder with default word and state size:
    let mut coder = DefaultAnsCoder::new();

    // Some made up data and entropy models for demonstration purpose:
    let symbols = [23i32, -15, 78, 43, -69];
    let means = [35.2, -1.7, 30.1, 71.2, -75.1];
    let stds = [10.1, 25.3, 23.8, 35.4, 3.9];

    // Create an adapter that integrates 1-d probability density functions over bins
    // `[n - 0.5, n + 0.5)` for all integers `n` from `-100` to `100` using fixed point
    // arithmetic with default precision, guaranteeing a nonzero probability for each bin:
    let quantizer = DefaultLeakyQuantizer::new(-100..=100);

    // Encode the data (in reverse order, since ANS Coding operates as a stack):
    coder.encode_symbols_reverse(
        symbols.iter().zip(&means).zip(&stds).map(
            |((&sym, &mean), &std)| (sym, quantizer.quantize(Gaussian::new(mean, std)))
    )).unwrap();

    // Retrieve the compressed representation (filling it up to full words with zero bits).
    coder.into_compressed().unwrap()
}

assert_eq!(encode_sample_data(), [0x421C_7EC3, 0x000B_8ED1]);
```

--------------------------------

### DefaultLeakyQuantizer Usage

Source: https://context7.com/bamler-lab/constriction/llms.txt

Demonstrates how to use DefaultLeakyQuantizer to quantize continuous and discrete probability distributions and encode/decode symbols using heterogeneous models.

```APIDOC
## Leaky Quantizer (Rust) — `stream::model::DefaultLeakyQuantizer`

Adapter that converts any one-dimensional continuous or discrete probability distribution (e.g., from the `probability` crate) into an exactly invertible fixed-point entropy model. The "leaky" property ensures every integer in the supplied range receives at least the smallest representable nonzero probability, preventing encoding errors.

```rust
use constriction::stream::{
    model::{DefaultLeakyQuantizer, DefaultContiguousCategoricalEntropyModel},
    stack::DefaultAnsCoder,
    Decode,
};
use probability::distribution::{Gaussian, Laplace};

let quantizer = DefaultLeakyQuantizer::new(-100i32..=100);

// Quantize a Gaussian
let gauss_model = quantizer.quantize(Gaussian::new(0.0f64, 10.0));

// Quantize a Laplace
let laplace_model = quantizer.quantize(Laplace::new(5.0f64, 3.0));

// Encode with heterogeneous models
let mut coder = DefaultAnsCoder::new();
coder.encode_symbols_reverse([
    (7i32,  gauss_model),
    (-3i32, laplace_model),
]).unwrap();

let (s1, s2): (i32, i32) = {
    let mut iter = coder.decode_symbols([gauss_model, laplace_model]);
    (iter.next().unwrap().unwrap(), iter.next().unwrap().unwrap())
};
assert_eq!((s1, s2), (7, -3));

// Categorical model from floating-point probabilities
let probs = [0.1f32, 0.4, 0.2, 0.3];
let cat = DefaultContiguousCategoricalEntropyModel
    ::from_floating_point_probabilities_fast(&probs, None)
    .unwrap();
let mut coder2 = DefaultAnsCoder::new();
coder2.encode_iid_symbols_reverse(&[0usize, 3, 1], &cat).unwrap();
let decoded: Vec<usize> = coder2
    .decode_iid_symbols(3, &cat)
    .collect::<Result<_, _>>()
    .unwrap();
assert_eq!(decoded, [0, 3, 1]);
```
```

--------------------------------

### Custom Entropy Models (Python)

Source: https://context7.com/bamler-lab/constriction/llms.txt

Explains how to use `CustomModel` and `ScipyModel` for wrapping arbitrary Python CDF/PPF callbacks into entropy models.

```APIDOC
## Custom Entropy Models (Python)

### Description
`CustomModel` wraps arbitrary Python CDF/PPF callback pairs into an exactly invertible entropy model. `ScipyModel` is a convenience wrapper for distributions from `scipy.stats`. Both support the model-family pattern for per-symbol parameters.

### Usage Example
```python
import constriction
import numpy as np
import scipy.stats

# ── ScipyModel: concrete instance ────────────────────────────────────────────
scipy_dist = scipy.stats.cauchy(loc=6.7, scale=12.4)
model = constriction.stream.model.ScipyModel(scipy_dist, -100, 100)

symbols = np.array([22, 14, 5, -3, 19, 7], dtype=np.int32)
coder = constriction.stream.stack.AnsCoder()
coder.encode_reverse(symbols, model)
assert np.all(coder.decode(model, 6) == symbols)
```
```

--------------------------------

### Huffman Coding (Rust) - Building Encoder/Decoder Trees

Source: https://context7.com/bamler-lab/constriction/llms.txt

Shows how to construct Huffman encoder and decoder trees from float probabilities in Rust. The encoder maps symbols to codewords, and the decoder maps bits back to symbols.

```rust
use constriction::symbol::huffman::{EncoderHuffmanTree, DecoderHuffmanTree};

let probabilities: Vec<f32> = vec![0.3, 0.2, 0.2, 0.15, 0.15];

let encoder_tree = EncoderHuffmanTree::from_float_probabilities::<f32, _>(&probabilities)
    .expect("valid probabilities");
let decoder_tree = DecoderHuffmanTree::from_float_probabilities::<f32, _>(&probabilities)
    .expect("valid probabilities");

// Encode symbol 2 (look up its codeword)
let codeword = encoder_tree.encode_symbol(2usize);
println!("Symbol 2 codeword: {:?}", codeword);

// Decode one symbol from a bit reader
// (In practice you'd feed bits from your compressed buffer via a `BitReader`.)

```

--------------------------------

### Compile and Test Rust Library

Source: https://github.com/bamler-lab/constriction/blob/main/README.md

Use this command to compile the Rust library in development mode and execute all associated tests.

```rust
cargo test
```

--------------------------------

### Encode and Decode Data with QuantizedGaussian Model in Python

Source: https://github.com/bamler-lab/constriction/blob/main/README.md

Demonstrates encoding and decoding data using a QuantizedGaussian entropy model. Ensure the message array and entropy model are defined before use.

```python
import constriction
import numpy as np

message = np.array([6, 10, -4, 2, 5, 2, 1, 0, 2], dtype=np.int32)

# Define an i.i.d. entropy model (see links below for more complex models):
entropy_model = constriction.stream.model.QuantizedGaussian(-50, 50, 3.2, 9.6)
```

--------------------------------

### Categorical Model Usage

Source: https://context7.com/bamler-lab/constriction/llms.txt

Shows how to use the Categorical model, both as a concrete instance with fixed probabilities and as a model family where probabilities vary per symbol.

```python
import constriction
import numpy as np

# Concrete model over alphabet {0,1,2,3}
probs = np.array([0.2, 0.4, 0.1, 0.3], dtype=np.float32)
cat_model = constriction.stream.model.Categorical(probs, perfect=False)

# Model family: different probability vector per symbol (2-D array)
cat_family = constriction.stream.model.Categorical(perfect=False)
per_symbol_probs = np.array(
    [[0.3, 0.1, 0.1, 0.3, 0.2],
     [0.1, 0.4, 0.2, 0.1, 0.2],
     [0.4, 0.2, 0.1, 0.2, 0.1]], dtype=np.float32)
syms = np.array([0, 4, 1], dtype=np.int32)
ans2 = constriction.stream.stack.AnsCoder()
ans2.encode_reverse(syms, cat_family, per_symbol_probs)
assert np.all(ans2.decode(cat_family, per_symbol_probs) == syms)
```

--------------------------------

### Download Shakespeare Dataset

Source: https://github.com/bamler-lab/constriction/blob/main/examples/python/03-tutorial-autoregressive-nlp-compression.ipynb

Downloads the 'input.txt' file from the char-rnn GitHub repository. This is the raw text data used for training.

```python
!wget -O shakespeare_all.txt https://raw.githubusercontent.com/karpathy/char-rnn/master/data/tinyshakespeare/input.txt
```

--------------------------------

### Custom Entropy Models with ScipyModel (Python)

Source: https://context7.com/bamler-lab/constriction/llms.txt

Demonstrates wrapping a scipy.stats distribution into an exactly invertible entropy model for use with Constriction's stream coders. Requires specifying bounds for the model.

```python
import constriction
import numpy as np
import scipy.stats

# ── ScipyModel: concrete instance ────────────────────────────────────────────
scipy_dist = scipy.stats.cauchy(loc=6.7, scale=12.4)
model = constriction.stream.model.ScipyModel(scipy_dist, -100, 100)

symbols = np.array([22, 14, 5, -3, 19, 7], dtype=np.int32)
coder = constriction.stream.stack.AnsCoder()
coder.encode_reverse(symbols, model)
assert np.all(coder.decode(model, 6) == symbols)

```

--------------------------------

### Split Data into Training and Test Sets

Source: https://github.com/bamler-lab/constriction/blob/main/examples/python/03-tutorial-autoregressive-nlp-compression.ipynb

Splits the downloaded text file into training and testing sets. It randomly assigns lines to ensure reproducibility with a fixed random seed.

```python
import numpy as np

target_test_set_ratio = 0.1 # means that about 10% of the lines will end up in the test set.
train_count, test_count = 0, 0
rng = np.random.RandomState(830472) # (always set a random seed to make results reproducible)

with open("shakespeare_all.txt", "r") as in_file, \
     open("shakespeare_train.txt", "w") as train_file, \
     open("shakespeare_test.txt", "w") as test_file:
    for line in in_file.readlines():
        if line == "\n" or line == "":
            continue # Let's leave out empty lines
        if rng.uniform() < target_test_set_ratio:
            test_file.write(line)
            test_count += 1
        else:
            train_file.write(line)
            train_count += 1

total_count = train_count + test_count
print(f"Total number of non-empty lines in the data set: {total_count}")
print(f"File `shakespeare_train.txt` has {train_count} lines ({100 * train_count / total_count:.1f}%).")
print(f"File `shakespeare_test.txt` has {test_count} lines ({100 * test_count / total_count:.1f}%).")
```

--------------------------------

### Range Coding with Simple and Complex Entropy Models

Source: https://github.com/bamler-lab/constriction/blob/main/README-python.md

Demonstrates encoding and decoding a message using Range Coding with both a simple and a complex entropy model. Ensure the correct parameters are provided for each model during encoding and decoding.

```python
encoder = constriction.stream.queue.RangeEncoder()
encoder.encode(message[0:5], entropy_model1, means, stds) # per-symbol params.
encoder.encode(message[5:9], entropy_model2)

compressed = encoder.get_compressed()
print(f"compressed representation: {compressed}")
print(f"(in binary: {[bin(word) for word in compressed]})")

decoder = constriction.stream.queue.RangeDecoder(compressed)
decoded_part1 = decoder.decode(entropy_model1, means, stds)
decoded_part2 = decoder.decode(entropy_model2, 4)
assert np.all(np.concatenate((decoded_part1, decoded_part2)) == message)
```

--------------------------------

### List Compressed File Sizes

Source: https://github.com/bamler-lab/constriction/blob/main/examples/python/03-tutorial-autoregressive-nlp-compression.ipynb

Lists the sizes of all compressed files, including the autoregressive model's output and general-purpose compression methods, for comparison.

```bash
!ls -l shakespeare_test.txt.*
```

--------------------------------

### Import Libraries for NLP Compression

Source: https://github.com/bamler-lab/constriction/blob/main/examples/python/03-tutorial-autoregressive-nlp-compression.ipynb

Imports necessary libraries for the Constriction framework, PyTorch, NumPy, and progress bar visualization.

```python
import constriction
import torch
import numpy as np
from tqdm import tqdm
```

--------------------------------

### CustomModel with Explicit CDF and PPF

Source: https://github.com/bamler-lab/constriction/blob/main/examples/python/02-custom-entropy-models.ipynb

Illustrates how to define a custom entropy model using `CustomModel` by providing explicit Python functions for the cumulative distribution function (CDF) and its approximate inverse (PPF). This offers flexibility for models not available in SciPy.

```python
model = constriction.stream.model.CustomModel(
    lambda x: ... TODO ...,  # define your CDF here
    lambda xi: ... TODO ..., # define an (approximate) inverse of the CDF here
    -100, 100) # (or on whichever range you want to define your model)
```

--------------------------------

### CustomModel with CDF/PPF

Source: https://context7.com/bamler-lab/constriction/llms.txt

Creates a custom entropy model by providing CDF and approximate PPF functions, along with the symbol domain.

```python
import math
import constriction

custom = constriction.stream.model.CustomModel(
    lambda x: 0.5 * (1.0 + math.erf((x + 0.5) / (3.0 * math.sqrt(2)))),  # CDF
    lambda xi: round(3.0 * math.sqrt(2) * math.erfinv(2*xi - 1)),         # approx PPF
    -30, 30)
```

--------------------------------

### Pos and Seek for Random Access Decoding

Source: https://context7.com/bamler-lab/constriction/llms.txt

Illustrates how to use the Pos and Seek traits with DefaultAnsCoder for random access decoding by capturing and restoring encoder states.

```APIDOC
## Pos / Seek — Random Access Decoding

Both ANS and Range Coding support checkpointing via the `Pos` and `Seek` traits. Calling `pos()` during encoding captures a `(position, state)` snapshot. The decoder can later `seek` to that snapshot for random access. `DefaultAnsCoder` does not implement `Seek` directly but exposes `as_seekable_decoder()` / `into_seekable_decoder()`.

```rust
use constriction::{
    stream::{
        model::DefaultContiguousCategoricalEntropyModel,
        stack::DefaultAnsCoder,
        Decode,
    },
    Pos, Seek,
};

let mut ans = DefaultAnsCoder::new();
let probs = vec![0.03, 0.07, 0.1, 0.1, 0.2, 0.2, 0.1, 0.15, 0.05];
let model = DefaultContiguousCategoricalEntropyModel
    ::from_floating_point_probabilities_fast(&probs, None)
    .unwrap();

// Encode two chunks and capture snapshots
let chunk1 = vec![8usize, 2, 0, 7];
ans.encode_iid_symbols_reverse(&chunk1, &model).unwrap();
let snapshot1 = ans.pos();   // <-- (pos_in_words, coder_state)

let chunk2 = vec![3usize, 1, 5];
ans.encode_iid_symbols_reverse(&chunk2, &model).unwrap();
let snapshot2 = ans.pos();

// Convert to a seekable decoder (loses encoding ability)
let mut decoder = ans.as_seekable_decoder();

// Jump to snapshot1 and decode chunk1
decoder.seek(snapshot1);
let decoded1: Vec<usize> = decoder
    .decode_iid_symbols(4, &model)
    .collect::<Result<_, _>>()
    .unwrap();
assert_eq!(decoded1, chunk1);
assert!(decoder.is_empty());

// Jump to snapshot2 and decode both chunks in order
decoder.seek(snapshot2);
let decoded_both: Vec<usize> = decoder
    .decode_iid_symbols(7, &model)
    .map(Result::unwrap)
    .collect();
assert!(decoded_both.iter().zip(chunk2.iter().chain(&chunk1)).all(|(a, b)| a == b));
```
```

--------------------------------

### Print All Characters

Source: https://github.com/bamler-lab/constriction/blob/main/examples/python/03-tutorial-autoregressive-nlp-compression.ipynb

Displays the set of all characters recognized by the model.

```python
print(all_characters)
```

--------------------------------

### Fixed Custom Entropy Model with ScipyModel

Source: https://github.com/bamler-lab/constriction/blob/main/examples/python/02-custom-entropy-models.ipynb

Creates and uses a fixed entropy model based on SciPy's Cauchy distribution. This model is then used for encoding and decoding a message, demonstrating lossless compression and reconstruction.

```python
import constriction
import numpy as np
import scipy.stats

# Get a scipy model with fixed parameters (`loc` and `scale`). See below
# for an example of a model *family* with variable model parameters.
scipy_model = scipy.stats.cauchy(loc=10.2, scale=30.9)

# Quantize the model to bins of size 1 centered at integers from -100 to 100:
model = constriction.stream.model.ScipyModel(scipy_model, -100, 100)

# Encode and decode some message using the above model for all model parameters:
message = np.array([3, 2, 6, -51, -19, 5, 87], dtype=np.int32)
print(f"Original message: {message}")
encoder = constriction.stream.queue.RangeEncoder()
encoder.encode(message, model)
compressed = encoder.get_compressed()
print(f"compressed representation: {compressed}")

decoder = constriction.stream.queue.RangeDecoder(compressed)
reconstructed = decoder.decode(model, 7) # (decodes 7 symbols)
print(f"Reconstructed message: {reconstructed}")
assert np.all(reconstructed == message)
```

--------------------------------

### DefaultChainCoder (Rust)

Source: https://context7.com/bamler-lab/constriction/llms.txt

Demonstrates the usage of `DefaultChainCoder` for decoding symbols independently, contrasting it with ANS coding.

```APIDOC
## DefaultChainCoder (Rust)

### Description
Experimental stream coder that treats each decoded symbol independently. Unlike ANS, modifying the entropy model for one symbol does not produce a ripple effect on subsequently decoded symbols. This makes it useful for advanced joint inference, quantization, and bits-back techniques.

### Usage Example
```rust
use constriction::stream::{
    model::DefaultContiguousCategoricalEntropyModel,
    stack::DefaultAnsCoder,
    chain::DefaultChainCoder,
    Decode,
};

fn decode_categoricals<D: Decode<24, Word = u32>>(
    decoder: &mut D,
    probabilities: &[[f32; 4]],
) -> Vec<usize> {
    let models = probabilities.iter().map(|probs|
        DefaultContiguousCategoricalEntropyModel
            ::from_floating_point_probabilities_fast(probs, None).unwrap()
    );
    decoder.decode_symbols(models).collect::<Result<Vec<_>, _>>().unwrap()
}

let data = vec![0x80d1_4131u32, 0xdda9_7c6c, 0x5017_a640, 0x0117_0a3e];
let mut probabilities = [
    [0.1f32, 0.7, 0.1, 0.1],
    [0.2,    0.2, 0.1, 0.5],
    [0.2,    0.1, 0.4, 0.3],
];

// ANS: changing one model has a ripple effect on ALL subsequent symbols
let mut ans = DefaultAnsCoder::from_binary(data.clone()).unwrap();
assert_eq!(decode_categoricals(&mut ans, &probabilities), [0, 0, 2]);

probabilities[0] = [0.09, 0.71, 0.1, 0.1]; // only first model changed
let mut ans2 = DefaultAnsCoder::from_binary(data.clone()).unwrap();
assert_eq!(decode_categoricals(&mut ans2, &probabilities), [1, 0, 0]); // all changed!

// ChainCoder: changing one model affects ONLY that symbol
probabilities[0] = [0.1, 0.7, 0.1, 0.1]; // restore
let mut chain = DefaultChainCoder::from_binary(data.clone()).unwrap();
assert_eq!(decode_categoricals(&mut chain, &probabilities), [0, 3, 3]);

probabilities[0] = [0.09, 0.71, 0.1, 0.1]; // only first model changed
let mut chain2 = DefaultChainCoder::from_binary(data).unwrap();
assert_eq!(decode_categoricals(&mut chain2, &probabilities), [1, 3, 3]); // only first changed
```
```

--------------------------------

### Run Python Unit Tests

Source: https://github.com/bamler-lab/constriction/blob/main/README.md

Executes the Python unit tests for the constriction library using Poetry.

```shell
poetry run pytest tests/python
```

--------------------------------

### Huffman Coding (Rust)

Source: https://context7.com/bamler-lab/constriction/llms.txt

Provides the Rust API for constructing and utilizing Huffman codebooks from probability inputs.

```APIDOC
## Huffman Coding (Rust)

### Description
Rust API for building and using Huffman codebooks from integer-valued or float-valued probability inputs. `EncoderHuffmanTree` maps symbols to codewords; `DecoderHuffmanTree` maps incoming bits back to symbols one at a time.

### Usage Example
```rust
use constriction::symbol::huffman::{EncoderHuffmanTree, DecoderHuffmanTree};

let probabilities: Vec<f32> = vec![0.3, 0.2, 0.2, 0.15, 0.15];

let encoder_tree = EncoderHuffmanTree::from_float_probabilities::<f32, _>(&probabilities)
    .expect("valid probabilities");
let decoder_tree = DecoderHuffmanTree::from_float_probabilities::<f32, _>(&probabilities)
    .expect("valid probabilities");

// Encode symbol 2 (look up its codeword)
let codeword = encoder_tree.encode_symbol(2usize);
println!("Symbol 2 codeword: {:?}", codeword);

// Decode one symbol from a bit reader
// (In practice you'd feed bits from your compressed buffer via a `BitReader`.)
```
```

--------------------------------

### Load Trained Model

Source: https://github.com/bamler-lab/constriction/blob/main/examples/python/03-tutorial-autoregressive-nlp-compression.ipynb

Loads a pre-trained PyTorch model from a file. Ensure the model file exists in the specified path.

```python
model = torch.load("shakespeare_train.pt")
```

--------------------------------

### Add char-rnn.pytorch to Python Path

Source: https://github.com/bamler-lab/constriction/blob/main/examples/python/03-tutorial-autoregressive-nlp-compression.ipynb

Appends the 'char-rnn.pytorch' subdirectory to sys.path to allow importing necessary modules.

```python
import sys
import os
sys.path.append(os.path.join(os.getcwd(), "char-rnn.pytorch"))
from model import *
from helpers import read_file, all_characters
```

--------------------------------

### Encode and Decode with RangeCoder

Source: https://context7.com/bamler-lab/constriction/llms.txt

Demonstrates encoding and decoding of integer arrays using RangeEncoder and RangeDecoder with Categorical and QuantizedGaussian models. Ensure the compressed data is correctly retrieved and used for decoding.

```python
import constriction
import numpy as np

message_part1       = np.array([1, 2, 0, 3, 2, 3, 0], dtype=np.int32)
probabilities_part1 = np.array([0.2, 0.4, 0.1, 0.3], dtype=np.float32)
model_part1 = constriction.stream.model.Categorical(probabilities_part1, perfect=False)

message_part2      = np.array([6,    10,    -4,    2  ], dtype=np.int32)
means_part2        = np.array([2.5,  13.1,  -1.1, -3.0], dtype=np.float32)
stds_part2         = np.array([4.1,   8.7,   6.2,  5.4], dtype=np.float32)
model_family_part2 = constriction.stream.model.QuantizedGaussian(-100, 100)

encoder = constriction.stream.queue.RangeEncoder()
encoder.encode(message_part1, model_part1)                                 # forward order
encoder.encode(message_part2, model_family_part2, means_part2, stds_part2)

compressed = encoder.get_compressed()   # dtype=uint32 numpy array
print(f"compressed: {compressed}")

# --- Decoding ---
decoder = constriction.stream.queue.RangeDecoder(compressed)
decoded_part1 = decoder.decode(model_part1, 7)
decoded_part2 = decoder.decode(model_family_part2, means_part2, stds_part2)

assert np.all(decoded_part1 == message_part1)
assert np.all(decoded_part2 == message_part2)
print("Decoded:", np.concatenate([decoded_part1, decoded_part2]))
```