### Install Pomegranate from GitHub

Source: https://github.com/jmschrei/pomegranate/blob/master/docs/install.rst

Installs the latest version of Pomegranate directly from its GitHub repository. This involves cloning the repository and running the setup script.

```bash
git clone https://github.com/jmschrei/pomegranate
cd pomegranate
python setup.py install
```

--------------------------------

### Setup and Environment Check for Pomegranate

Source: https://github.com/jmschrei/pomegranate/blob/master/docs/tutorials/B_Model_Tutorial_5_Markov_Chains.ipynb

This snippet sets up the environment for using Pomegranate and PyTorch, including inline plotting and checking installed versions. It's a common starting point for running examples.

```Python
%pylab inline
import seaborn; seaborn.set_style('whitegrid')
import torch

%load_ext watermark
%watermark -m -n -p torch,pomegranate
```

--------------------------------

### Setup and Watermark for Pomegranate

Source: https://github.com/jmschrei/pomegranate/blob/master/benchmarks/Benchmark_1_Distributions.ipynb

Initializes the environment by setting random seeds, configuring print options, and displaying package versions using the watermark extension. This is a common setup step for reproducible scientific computing.

```Python
import numpy
import scipy
import torch

from torchegranate.distributions import *

numpy.random.seed(0)
numpy.set_printoptions(suppress=True)

%load_ext watermark
%watermark -m -n -p numpy,scipy,torch,pomegranate
```

--------------------------------

### Install Pomegranate using Pip

Source: https://github.com/jmschrei/pomegranate/blob/master/docs/install.rst

Installs the Pomegranate library and its dependencies using the pip package manager.

```bash
pip install pomegranate
```

--------------------------------

### Setup and Environment Check

Source: https://github.com/jmschrei/pomegranate/blob/master/docs/tutorials/C_Feature_Tutorial_3_Out_Of_Core_Learning.ipynb

Initializes the environment for out-of-core learning examples, including setting random seeds, display options, and loading necessary libraries like PyTorch and Pomegranate.

```Python
%pylab inline
import torch

numpy.random.seed(0)
numpy.set_printoptions(suppress=True)

%load_ext watermark
%watermark -m -n -p torch,pomegranate
```

--------------------------------

### Setup and Watermark

Source: https://github.com/jmschrei/pomegranate/blob/master/benchmarks/Benchmark_5_Hidden_Markov_Model.ipynb

Initializes the environment with pylab and seaborn, imports necessary libraries from torch and pomegranate, and displays system and package information using the watermark extension.

```python
%pylab inline
import seaborn; seaborn.set_style('whitegrid')

import time
import torch

from tqdm import tqdm

from torchegranate.distributions import *
from torchegranate.hmm import HiddenMarkovModel, Node

from pomegranate import State, NormalDistribution, IndependentComponentsDistribution, DiscreteDistribution
from pomegranate import HiddenMarkovModel as HMM

numpy.random.seed(0)
numpy.set_printoptions(suppress=True)

%load_ext watermark
%watermark -m -n -p numpy,scipy,torch,pomegranate
```

--------------------------------

### Gamma Distribution

Source: https://github.com/jmschrei/pomegranate/blob/master/benchmarks/Benchmark_1_Distributions.ipynb

Demonstrates the Gamma distribution, including parameter setup and log probability calculations. Performance is compared across Pomegranate, PyTorch, and SciPy implementations.

```Python
shapes = torch.abs(torch.randn(d))
rates = torch.abs(torch.randn(d))
```

```Python
%timeit Gamma(shapes, rates).log_probability(X)
%timeit torch.distributions.Gamma(shapes, rates).log_prob(X)
%timeit scipy.stats.gamma.logpdf(X, shapes, rates)
```

--------------------------------

### Setup and Environment Check for Pomegranate

Source: https://github.com/jmschrei/pomegranate/blob/master/docs/tutorials/B_Model_Tutorial_2_General_Mixture_Models.ipynb

This snippet sets up the necessary environment for running Pomegranate examples, including importing libraries like PyTorch and Seaborn, and checking system information using watermark.

```Python
%pylab inline
import seaborn; seaborn.set_style('whitegrid')

import torch

from pomegranate.gmm import GeneralMixtureModel
from pomegranate.distributions import *


numpy.random.seed(0)
numpy.set_printoptions(suppress=True)

%load_ext watermark
%watermark -m -n -p numpy,scipy,torch,pomegranate
```

--------------------------------

### Import Libraries and Setup

Source: https://github.com/jmschrei/pomegranate/blob/master/benchmarks/Benchmark_4_Bayes_Classifier.ipynb

Imports necessary libraries such as numpy, scipy, torch, scikit-learn, and Pomegranate. It also sets up random seeds, print options, and loads environment information using %watermark.

```Python
import numpy
import scipy
import torch

from sklearn.datasets import make_blobs

from torchegranate.distributions import *
from torchegranate.bayes_classifier import BayesClassifier

from sklearn.naive_bayes import GaussianNB, BernoulliNB

import matplotlib.pyplot as plt
import seaborn; seaborn.set_style('whitegrid')

numpy.random.seed(0)
numpy.set_printoptions(suppress=True)

%load_ext watermark
%watermark -m -n -p numpy,scipy,torch,pomegranate
```

--------------------------------

### Setup and Environment Check

Source: https://github.com/jmschrei/pomegranate/blob/master/docs/tutorials/B_Model_Tutorial_6_Bayesian_Networks.ipynb

Imports necessary libraries (pylab, seaborn, torch) and loads the watermark extension to display environment information, including versions of torch and pomegranate.

```Python
%pylab inline
import seaborn; seaborn.set_style('whitegrid')
import torch

%load_ext watermark
%watermark -m -n -p torch,pomegranate
```

--------------------------------

### Setup and Environment Information

Source: https://github.com/jmschrei/pomegranate/blob/master/docs/tutorials/B_Model_Tutorial_7_Factor_Graphs.ipynb

This snippet sets up the environment for plotting inline, applies a white grid style using Seaborn, and displays watermark information including the Python version, machine type, and installed package versions (torch, pomegranate).

```Python
%pylab inline
import seaborn; seaborn.set_style('whitegrid')
import torch

%load_ext watermark
%watermark -m -n -p torch,pomegranate
```

--------------------------------

### Normal Distribution with Diagonal Covariance

Source: https://github.com/jmschrei/pomegranate/blob/master/benchmarks/Benchmark_1_Distributions.ipynb

Demonstrates the setup for a Normal distribution with diagonal covariance. It generates random data and parameters, then times the log probability calculation using Pomegranate, PyTorch, and SciPy.

```Python
n, d = 100000, 500

X = torch.randn(n, d)
Xn = X.numpy()

mus = torch.randn(d)
covs = torch.abs(torch.randn(d))
stds = torch.sqrt(covs)
```

```Python
%timeit Normal(mus, covs, covariance_type='diag').log_probability(X)
%timeit torch.distributions.Normal(mus, stds).log_prob(X).sum(dim=-1)
%timeit scipy.stats.norm.logpdf(Xn, mus, stds).sum(axis=1)
```

--------------------------------

### GPU Support Example in Pomegranate

Source: https://github.com/jmschrei/pomegranate/blob/master/README.md

Demonstrates how to utilize GPU support in Pomegranate models by moving data to the GPU. This example shows fitting an Exponential distribution to random data on the CPU.

```Python
>>> X = torch.exp(torch.randn(50, 4))

# Will execute on the CPU
>>> d = Exponential().fit(X)
>>> d.scales
Parameter containing:
tensor([1.8627, 1.3132, 1.7187, 1.4957])
```

--------------------------------

### Exponential Distribution

Source: https://github.com/jmschrei/pomegranate/blob/master/benchmarks/Benchmark_1_Distributions.ipynb

Shows the usage of the Exponential distribution. It generates positive data and parameters, then benchmarks the log probability calculation using Pomegranate, PyTorch, and SciPy.

```Python
X = torch.abs(torch.randn(n, d))
Xn = X.numpy()

means = torch.abs(torch.randn(d))
```

```Python
%timeit Exponential(means).log_probability(X)
%timeit torch.distributions.Exponential(means).log_prob(X)
%timeit scipy.stats.expon.logpdf(X, means)
```

--------------------------------

### Setup and Environment Check for Pomegranate

Source: https://github.com/jmschrei/pomegranate/blob/master/docs/tutorials/B_Model_Tutorial_4_Hidden_Markov_Models.ipynb

This Python snippet sets up the inline plotting environment, imports necessary libraries like seaborn and torch, and displays version information for numpy, scipy, torch, and pomegranate using the watermark extension. It's a common starting point for using the library.

```Python
%pylab inline
import seaborn; seaborn.set_style('whitegrid')

import torch

numpy.random.seed(0)
numpy.set_printoptions(suppress=True)

%load_ext watermark
%watermark -m -n -p numpy,scipy,torch,pomegranate
```

--------------------------------

### Setup and Environment Information

Source: https://github.com/jmschrei/pomegranate/blob/master/benchmarks/Benchmark_3_KMeans.ipynb

Imports necessary libraries (NumPy, SciPy, PyTorch, Scikit-learn, Matplotlib, Seaborn) and sets up the environment. It also loads the 'watermark' extension to display package versions.

```python
import numpy
import scipy
import torch

from sklearn.datasets import make_blobs

from sklearn.cluster import KMeans
from torchegranate.kmeans import KMeans as KMeans2

import matplotlib.pyplot as plt
import seaborn; seaborn.set_style('whitegrid')

numpy.random.seed(0)
numpy.set_printoptions(suppress=True)

%load_ext watermark
%watermark -m -n -p numpy,scipy,torch,pomegranate
```

--------------------------------

### Import Libraries and Setup

Source: https://github.com/jmschrei/pomegranate/blob/master/docs/tutorials/B_Model_Tutorial_1_Distributions.ipynb

Imports necessary libraries (numpy, torch, pomegranate distributions) and sets up random seeds and print options for reproducible results. It also loads and displays version information for the libraries.

```Python
import numpy
import torch
from pomegranate.distributions import *

numpy.random.seed(0)
numpy.set_printoptions(suppress=True)

%load_ext watermark
%watermark -m -n -p numpy,torch,pomegranate
```

--------------------------------

### Install Pomegranate

Source: https://github.com/jmschrei/pomegranate/blob/master/README.md

Installs the pomegranate library using pip. For users needing the last Cython release before the PyTorch rewrite, a specific version (0.14.8) can be installed. Manual installation of a Cython version before v3 might be necessary.

```Shell
pip install pomegranate
```

```Shell
pip install pomegranate==0.14.8
```

--------------------------------

### Bernoulli Distribution

Source: https://github.com/jmschrei/pomegranate/blob/master/benchmarks/Benchmark_1_Distributions.ipynb

Illustrates the Bernoulli distribution, focusing on binary data generation and probability calculations. It compares the log probability/log PMF computation using Pomegranate, PyTorch, and SciPy.

```Python
X = torch.tensor(numpy.random.choice(2, size=(n, d)), dtype=torch.float32)
probs = torch.mean(X, dim=0)
```

```Python
%timeit Bernoulli(probs).log_probability(X)
%timeit torch.distributions.Bernoulli(probs).log_prob(X)
%timeit scipy.stats.bernoulli.logpmf(X, probs)
```

--------------------------------

### Setup and Environment Configuration

Source: https://github.com/jmschrei/pomegranate/blob/master/docs/tutorials/C_Feature_Tutorial_2_Mixed_Precision_and_DataTypes.ipynb

This snippet sets up the environment by specifying the CUDA device, importing necessary libraries (pylab, seaborn, torch), and configuring numpy for printing. It also loads the 'watermark' extension to display package versions.

```Python
import os
os.environ['CUDA_VISIBLE_DEVICES'] = '0'

%pylab inline
import seaborn; seaborn.set_style('whitegrid')

import torch

numpy.random.seed(0)
numpy.set_printoptions(suppress=True)

%load_ext watermark
%watermark -m -n -p numpy,scipy,torch,pomegranate
```

--------------------------------

### Setup PyTorch and Pomegranate for GPU

Source: https://github.com/jmschrei/pomegranate/blob/master/docs/tutorials/C_Feature_Tutorial_1_GPU_Usage.ipynb

Initializes the environment for GPU usage by setting the CUDA visible devices and importing necessary libraries like PyTorch and Pomegranate. It also includes commands for displaying system information and package versions.

```Python
import os
os.environ['CUDA_VISIBLE_DEVICES'] = '0'

%pylab inline
import seaborn; seaborn.set_style('whitegrid')

import torch

numpy.random.seed(0)
numpy.set_printoptions(suppress=True)

%load_ext watermark
%watermark -m -n -p numpy,scipy,torch,pomegranate
```

--------------------------------

### Import Libraries and Setup

Source: https://github.com/jmschrei/pomegranate/blob/master/docs/tutorials/B_Model_Tutorial_3_Bayes_Classifier.ipynb

Imports necessary libraries including torch, seaborn, and components from pomegranate. It also sets up inline plotting and displays system information using watermark.

```Python
%pylab inline
import seaborn; seaborn.set_style('whitegrid')

import torch

from pomegranate.bayes_classifier import BayesClassifier
from pomegranate.distributions import *


numpy.random.seed(0)
numpy.set_printoptions(suppress=True)

%load_ext watermark
%watermark -m -n -p numpy,scipy,torch,pomegranate
```

--------------------------------

### Normal Distribution with Full Covariance

Source: https://github.com/jmschrei/pomegranate/blob/master/benchmarks/Benchmark_1_Distributions.ipynb

Illustrates fitting and calculating log probabilities for a Normal distribution with full covariance. It shows how to extract means and covariances from a fitted model and compares performance across Pomegranate, PyTorch, and SciPy.

```Python
d0 = Normal().fit(X)

mu, cov = d0.means, d0.covs
```

```Python
%timeit Normal(mu, cov).log_probability(X)
%timeit torch.distributions.MultivariateNormal(mu, cov).log_prob(X).sum(dim=-1)
%timeit scipy.stats.multivariate_normal.logpdf(Xn, mu, cov).sum(axis=-1)
```

--------------------------------

### Categorical Distribution HMM Creation

Source: https://github.com/jmschrei/pomegranate/blob/master/benchmarks/Benchmark_5_Hidden_Markov_Model.ipynb

Creates three Hidden Markov Models using Categorical distributions. It initializes states with IndependentComponentsDistribution and Categorical from pomegranate. The models are then constructed using from_matrix and HiddenMarkovModel constructors.

```python
n, l, d = 200, 50, 10
k = 50


X = numpy.random.choice(4, size=(n, l, d))
Xt = torch.tensor(X)

d1s, d2s, d3s = [], [], []
for i in range(k):
    mus = numpy.abs(numpy.random.randn(d, 4))
    mus = mus / mus.sum(axis=1, keepdims=True)

    d_ = [DiscreteDistribution({i: mu[i] for i in range(4)}) for mu in mus]
    d1 = IndependentComponentsDistribution(d_)
    d2 = Categorical(mus)
    d3 = Categorical(mus)

    d1s.append(d1)
    d2s.append(d2)
    d3s.append(d3)


starts = torch.ones(k, dtype=torch.float64) / k
ends = torch.ones(k, dtype=torch.float64) / k
edges = torch.ones(k, k, dtype=torch.float64) / k


model1 = HMM.from_matrix(edges, d1s, starts, ends)
model2 = HiddenMarkovModel(nodes=d2s, edges=edges, starts=starts, ends=ends, kind='sparse', max_iter=10, verbose=True)
model2.bake()
model3 = HiddenMarkovModel(nodes=d3s, edges=torch.clone(edges), starts=torch.clone(starts), ends=torch.clone(ends), max_iter=10, kind='dense', verbose=True)
model3.bake()
```

--------------------------------

### Setup Environment with PyLab and Torch

Source: https://github.com/jmschrei/pomegranate/blob/master/examples/Bayesian_Network_Monty_Hall.ipynb

This snippet sets up the Python environment for the simulation, enabling inline plotting with PyLab and loading necessary libraries like seaborn and torch. It also loads the watermark extension to display package versions.

```Python
%pylab inline
import seaborn; seaborn.set_style('whitegrid')
import torch

%load_ext watermark
%watermark -m -n -p torch,torchegranate
```

--------------------------------

### Import Libraries and Setup

Source: https://github.com/jmschrei/pomegranate/blob/master/benchmarks/Benchmark_2_General_Mixture_Models.ipynb

Imports necessary libraries such as numpy, scipy, torch, scikit-learn, pomegranate, and matplotlib. It also sets up random seeds, print options, and loads environment information using %load_ext and %watermark.

```Python
import numpy
import scipy
import torch

from sklearn.datasets import make_blobs
from torchegranate.distributions import *

from sklearn.mixture import GaussianMixture
from torchegranate.gmm import GeneralMixtureModel

import matplotlib.pyplot as plt
import seaborn; seaborn.set_style('whitegrid')

numpy.random.seed(0)
numpy.set_printoptions(suppress=True)

%load_ext watermark
%watermark -m -n -p numpy,scipy,torch,pomegranate
```

--------------------------------

### Mixed Precision Training with torch.autocast

Source: https://github.com/jmschrei/pomegranate/blob/master/README.md

Shows how to potentially use mixed precision training with Pomegranate models by leveraging PyTorch's `torch.autocast` context manager. This example uses `torch.bfloat16` for computations on the GPU.

```Python
>>> X = torch.randn(100, 4)
>>> d = Normal(covariance_type='diag')
>>> 
>>> with torch.autocast('cuda', dtype=torch.bfloat16):
>>>     d.fit(X)
```

--------------------------------

### Normal Distribution HMM Forward-Backward Pass Timing

Source: https://github.com/jmschrei/pomegranate/blob/master/benchmarks/Benchmark_5_Hidden_Markov_Model.ipynb

Measures the execution time of the forward-backward algorithm for the three created Normal distribution HMMs using %timeit. This benchmarks the combined inference process.

```python
%timeit [model1.forward_backward(x) for x in X]
%timeit model2.forward_backward(X)
%timeit model3.forward_backward(X)
```

--------------------------------

### Initialize Probability Distributions

Source: https://github.com/jmschrei/pomegranate/blob/master/docs/tutorials/B_Model_Tutorial_1_Distributions.ipynb

Demonstrates the initialization of various probability distributions (Normal, Exponential, Categorical) with different data types for parameters, including lists, tuples, numpy arrays, and torch tensors. It shows examples for both diagonal and full covariance types for the Normal distribution.

```Python
d1 = Normal([0.3, 0.7, 1.1], [1.1, 0.3, 1.8], covariance_type='diag')
d2 = Exponential([0.8, 1.4, 4.1])
d3 = Categorical([[0.3, 0.2, 0.5], [0.2, 0.1, 0.7]])

d11 = Normal((0.3, 0.7, 1.1), (1.1, 0.3, 1.8), covariance_type='diag')
d12 = Normal(numpy.array([0.3, 0.7, 1.1]), numpy.array([1.1, 0.3, 1.8]), covariance_type='diag')
d13 = Normal(torch.tensor([0.3, 0.7, 1.1]), torch.tensor([1.1, 0.3, 1.8]), covariance_type='diag')
```

--------------------------------

### Benchmark HMM Forward Time vs. Transition Matrix Size (Python)

Source: https://github.com/jmschrei/pomegranate/blob/master/benchmarks/Benchmark_5_Hidden_Markov_Model.ipynb

Measures and compares the time taken by Pomegranate and Torchegranate HMMs (sparse and dense) to execute the forward algorithm as the number of nodes in the transition matrix increases. This helps understand scalability with model complexity.

```Python
times1, times2, times3 = [], [], []

for k in tqdm(range(10, 351, 10)):
    n, l, d = 200, 50, 10

    X = numpy.random.choice(4, size=(n, l, d))
    Xt = torch.tensor(X)

    d1s, d2s, d3s = [], [], []
    for i in range(k):
        mus = numpy.abs(numpy.random.randn(d, 4))
        mus = mus / mus.sum(axis=1, keepdims=True)

        d_ = [DiscreteDistribution({i: mu[i] for i in range(4)}) for mu in mus]
        d1 = IndependentComponentsDistribution(d_)
        d2 = Categorical(mus)
        d3 = Categorical(mus)

        d1s.append(d1)
        d2s.append(d2)
        d3s.append(d3)


    starts = torch.ones(k, dtype=torch.float64) / k
    ends = torch.ones(k, dtype=torch.float64) / (k+1)
    edges = torch.ones(k, k, dtype=torch.float64) / (k+1)


    model1 = HMM.from_matrix(edges, d1s, starts, ends)
    model2 = HiddenMarkovModel(nodes=d2s, edges=edges, starts=starts, ends=ends, kind='sparse', max_iter=10, verbose=True)
    model2.bake()
    model3 = HiddenMarkovModel(nodes=d3s, edges=torch.clone(edges), starts=torch.clone(starts), ends=torch.clone(ends), max_iter=10, kind='dense', verbose=True)
    model3.bake()
    
    tic = time.time()
    [model1.forward(x) for x in X]
    times1.append(time.time() - tic)
    
    tic = time.time()
    model2.forward(X)
    times2.append(time.time() - tic)
    
    tic = time.time()
    model3.forward(X)
    times3.append(time.time() - tic)
```

```Python
x = range(10, 351, 10)

plt.figure(figsize=(6, 3))
plt.title("HMM Forward Algorithm Time", fontsize=12)
plt.plot(x, times1, label="pomegranate")
plt.plot(x, times2, label="torchegranate sparse")
plt.plot(x, times3, label="torchegranate dense")

plt.xlabel("Number of Nodes", fontsize=10)
plt.ylabel("Time (s)", fontsize=10)
plt.legend()

plt.tight_layout()
plt.show()
```

--------------------------------

### Benchmark HMM Forward Time vs. Transition Matrix Sparsity (Python)

Source: https://github.com/jmschrei/pomegranate/blob/master/benchmarks/Benchmark_5_Hidden_Markov_Model.ipynb

Evaluates the impact of transition matrix sparsity on the forward algorithm's execution time for Pomegranate and Torchegranate HMMs. It iterates through different proportions of non-zero elements in the transition matrix.

```Python
times1, times2, times3 = [], [], []
ps = 1. / numpy.exp(numpy.arange(3, 10.1, 0.25))

for p in tqdm(ps):
    n, l, d = 200, 50, 10
    k = 2000

    X = numpy.random.choice(4, size=(n, l, d))
    Xt = torch.tensor(X)

    d1s, d2s, d3s = [], [], []
    for i in range(k):
        mus = numpy.abs(numpy.random.randn(d, 4))
        mus = mus / mus.sum(axis=1, keepdims=True)

        d_ = [DiscreteDistribution({i: mu[i] for i in range(4)}) for mu in mus]
        d1 = IndependentComponentsDistribution(d_)
        d2 = Categorical(mus)
        d3 = Categorical(mus)

        d1s.append(d1)
        d2s.append(d2)
        d3s.append(d3)


    starts = torch.ones(k, dtype=torch.float64) / k
    ends = torch.ones(k, dtype=torch.float64)
    
    edges = numpy.ones((k, k), dtype=numpy.float64)
    mask = numpy.random.choice(2, size=(k, k), replace=True, p=[1-p, p])
    
    edges = edges * mask
    z = edges.sum(axis=1, keepdims=True) + 1
    
    edges = torch.tensor(edges / z)
    ends = torch.tensor(ends / z[:, 0])    

    model1 = HMM.from_matrix(edges, d1s, starts, ends)
    model2 = HiddenMarkovModel(nodes=d2s, edges=edges, starts=starts, ends=ends, kind='sparse', max_iter=10, verbose=True)
    model2.bake()
    model3 = HiddenMarkovModel(nodes=d3s, edges=torch.clone(edges), starts=torch.clone(starts), ends=torch.clone(ends), max_iter=10, kind='dense', verbose=True)
    model3.bake()
    
    tic = time.time()
    [model1.forward(x) for x in X]
    times1.append(time.time() - tic)
    
    tic = time.time()
    model2.forward(X)
    times2.append(time.time() - tic)
    
    tic = time.time()
    model3.forward(X)
    times3.append(time.time() - tic)
```

```Python
plt.figure(figsize=(6, 3))
plt.title("HMM Forward Algorithm Time", fontsize=12)
plt.plot(ps, times1, label="pomegranate")
plt.plot(ps, times2, label="torchegranate sparse")
plt.plot(ps, times3, label="torchegranate dense")

plt.xlabel("Proportion Non-zero", fontsize=10)
plt.ylabel("Time (s)", fontsize=10)
plt.xscale('log')
plt.yscale('log')
plt.legend()

plt.tight_layout()
plt.show()
```

--------------------------------

### Categorical Distribution HMM Forward Pass Timing

Source: https://github.com/jmschrei/pomegranate/blob/master/benchmarks/Benchmark_5_Hidden_Markov_Model.ipynb

Measures the execution time of the forward pass for the three created Categorical distribution HMMs using %timeit. This compares the performance of different HMM implementations for categorical data.

```python
%timeit [model1.forward(x) for x in X]
%timeit model2.forward(X)
%timeit model3.forward(X)
```

--------------------------------

### Fitting a Normal Distribution with bfloat16

Source: https://github.com/jmschrei/pomegranate/blob/master/docs/tutorials/C_Feature_Tutorial_2_Mixed_Precision_and_DataTypes.ipynb

This example demonstrates fitting a Normal distribution with 'diag' covariance type using data cast to `torch.bfloat16`. It shows how to convert the distribution object to `bfloat16` before fitting and accessing its parameters.

```Python
from pomegranate.distributions import Normal

X = torch.randn(1000, 5)

d = Normal(covariance_type='diag').fit(X)
d.means, d.covs
```

```Python
X = X.to(torch.bfloat16)

d = Normal(covariance_type='diag').to(torch.bfloat16).fit(X)
d.means, d.covs
```

--------------------------------

### Normal Distribution HMM Creation

Source: https://github.com/jmschrei/pomegranate/blob/master/benchmarks/Benchmark_5_Hidden_Markov_Model.ipynb

Creates three Hidden Markov Models using Normal distributions with diagonal covariance. It initializes states with IndependentComponentsDistribution, Normal (diag), and Normal (diag) from pomegranate. The models are then constructed using from_matrix and HiddenMarkovModel constructors.

```python
n, l, d = 200, 50, 10
k = 50


X = numpy.random.randn(n, l, d)
Xt = torch.tensor(X)

d1s, d2s, d3s = [], [], []
for i in range(k):
    mus = numpy.random.randn(d)

    d1 = IndependentComponentsDistribution([NormalDistribution(mu, 1) for mu in mus])
    d2 = Normal(torch.tensor(mus), torch.ones(d), covariance_type='diag')
    d3 = Normal(torch.tensor(mus), torch.ones(d), covariance_type='diag')

    d1s.append(d1)
    d2s.append(d2)
    d3s.append(d3)


starts = torch.ones(k, dtype=torch.float64) / k
ends = torch.ones(k, dtype=torch.float64) / (k+1)
edges = torch.ones(k, k, dtype=torch.float64) / (k+1)


model1 = HMM.from_matrix(edges, d1s, starts, ends)
model2 = HiddenMarkovModel(nodes=d2s, edges=edges, starts=starts, ends=ends, kind='sparse', max_iter=10, verbose=True)
model2.bake()
model3 = HiddenMarkovModel(nodes=d3s, edges=torch.clone(edges), starts=torch.clone(starts), ends=torch.clone(ends), max_iter=10, kind='dense', verbose=True)
model3.bake()
```

--------------------------------

### Initialize and Fit DenseHMM Model

Source: https://github.com/jmschrei/pomegranate/blob/master/docs/tutorials/B_Model_Tutorial_4_Hidden_Markov_Models.ipynb

Demonstrates the initialization of a DenseHMM model with specified categorical distributions, transition edges, start probabilities, and end probabilities, followed by fitting the model to data X. The `verbose` parameter is set to True for detailed output during fitting.

```Python
d1 = Categorical([[0.25, 0.25, 0.25, 0.25]])
d2 = Categorical([[0.10, 0.40, 0.40, 0.10]])

edges = [[0.89, 0.1], [0.1, 0.9]]
starts = [0.5, 0.5]
ends = [0.01, 0.0]

model = DenseHMM([d1, d2], edges=edges, starts=starts, ends=ends, verbose=True)
model.fit(X)
```

--------------------------------

### Normal Distribution HMM Fitting (Model 1)

Source: https://github.com/jmschrei/pomegranate/blob/master/benchmarks/Benchmark_5_Hidden_Markov_Model.ipynb

Fits the first Normal distribution HMM (model1) to the data using the fit method with specified maximum iterations and verbosity. It also calculates and prints the sum of log probabilities for the data.

```python
_ = model1.fit(X, max_iterations=10, verbose=True)
sum(model1.log_probability(x) for x in X)
```

--------------------------------

### Time Gaussian Naive Bayes Training

Source: https://github.com/jmschrei/pomegranate/blob/master/benchmarks/Benchmark_4_Bayes_Classifier.ipynb

Measures the time taken to train Gaussian Naive Bayes models using both scikit-learn and Pomegranate. This helps in comparing the training efficiency of the two libraries.

```Python
%timeit model_sklearn = GaussianNB().fit(X, y)
%timeit model_pom = BayesClassifier([Normal(covariance_type='diag') for i in range(k)]).fit(X, y)
```

--------------------------------

### Learn Pomegranate Model Directly from Data

Source: https://github.com/jmschrei/pomegranate/blob/master/docs/faq.rst

Shows the standard method for training a Pomegranate model from scratch using the `fit` function, adhering to the scikit-learn API. It highlights that some models require initialization parameters.

```Python
from pomegranate import *

# Example: Learn a NormalDistribution from data
X = [[1.0], [2.0], [3.0], [4.0], [5.0]]
model = NormalDistribution().fit(X)

# Example: Mixture model requires specifying distributions
# distributions = [NormalDistribution(mu=0, sigma=1), NormalDistribution(mu=5, sigma=2)]
# mixture_model = GeneralMixtureModel(distributions).fit(X)
```

--------------------------------

### Create and Use Pomegranate Model with Custom Parameters

Source: https://github.com/jmschrei/pomegranate/blob/master/docs/faq.rst

Demonstrates how to initialize a Pomegranate model with known parameters for tasks like inference. Users can pass parameters directly or leave the model uninitialized to fit it to data.

```Python
from pomegranate import *

# Example: Initialize a NormalDistribution with known parameters
model = NormalDistribution.from_samples([[1.0], [2.0], [3.0]])

# Perform inference using log_probability
log_prob = model.log_probability([1.5])

# Or predict values
prediction = model.predict([1.5])
```

--------------------------------

### Calculate Probability

Source: https://github.com/jmschrei/pomegranate/blob/master/docs/api.rst

Calculates the probability of input examples using the model. The input can be 2D or 3D depending on the model. Returns a vector of probabilities.

```python
>>> model.probability(X)
```

--------------------------------

### Initialize Factor Graph with Distributions and Edges

Source: https://github.com/jmschrei/pomegranate/blob/master/docs/tutorials/B_Model_Tutorial_7_Factor_Graphs.ipynb

Demonstrates the initialization of a FactorGraph by providing lists of factors, marginal distributions, and edges connecting them. This sets up the structure of the probabilistic model.

```Python
from pomegranate.distributions import Categorical
from pomegranate.distributions import JointCategorical
from pomegranate.factor_graph import FactorGraph

m1 = Categorical([[0.5, 0.5]])
m2 = Categorical([[0.5, 0.5]])

f1 = JointCategorical([[0.1, 0.3], [0.2, 0.4]])

model = FactorGraph([f1], [m1, m2], [(m1, f1), (m2, f1)])
```

--------------------------------

### Initialize Bayes Classifier with Distributions

Source: https://github.com/jmschrei/pomegranate/blob/master/docs/tutorials/B_Model_Tutorial_3_Bayes_Classifier.ipynb

Initializes a BayesClassifier with pre-defined Gaussian distributions. This setup is for a classifier where the distributions are already specified, not learned from data.

```Python
d1 = Normal([1.1, 1.3], [0.3, 0.9], covariance_type='diag')
d2 = Normal([1.3, 1.8], [1.1, 1.5], covariance_type='diag')
d3 = Normal([2.1, 2.3], [0.5, 1.8], covariance_type='diag')

model = BayesClassifier([d1, d2, d3])
```

--------------------------------

### Get Predicted Log Probabilities with Factor Graph

Source: https://github.com/jmschrei/pomegranate/blob/master/docs/tutorials/B_Model_Tutorial_7_Factor_Graphs.ipynb

Demonstrates how to calculate the log probabilities of predictions using the `predict_log_proba` method with a MaskedTensor.

```Python
model.predict_log_proba(X_masked)
```

--------------------------------

### Calculate Log Probability

Source: https://github.com/jmschrei/pomegranate/blob/master/docs/api.rst

Calculates the log probability of input examples for improved numerical stability. This is the basis for the `probability` method. Returns a vector of log probabilities.

```python
>>> model.log_probability(X)
```

--------------------------------

### Normal Distribution HMM Fitting (Model 3)

Source: https://github.com/jmschrei/pomegranate/blob/master/benchmarks/Benchmark_5_Hidden_Markov_Model.ipynb

Fits the third Normal distribution HMM (model3) to the data using the fit method and calculates the sum of log probabilities.

```python
model3.fit(X)
model3.log_probability(X).sum()
```

--------------------------------

### Initialize Bayesian Network with Distributions and Edges

Source: https://github.com/jmschrei/pomegranate/blob/master/docs/tutorials/B_Model_Tutorial_6_Bayesian_Networks.ipynb

Demonstrates initializing a Bayesian Network by providing a list of distributions (Categorical and ConditionalCategorical) and a list of edges defining the network structure.

```Python
from pomegranate.distributions import Categorical
from pomegranate.distributions import ConditionalCategorical
from pomegranate.bayesian_network import BayesianNetwork

d1 = Categorical([[0.1, 0.9]])
d2 = ConditionalCategorical([[[0.4, 0.6], [0.3, 0.7]]])

model = BayesianNetwork([d1, d2], [(d1, d2)])
```

--------------------------------

### Normal Distribution HMM Fitting (Model 2)

Source: https://github.com/jmschrei/pomegranate/blob/master/benchmarks/Benchmark_5_Hidden_Markov_Model.ipynb

Fits the second Normal distribution HMM (model2) to the data using the fit method and calculates the sum of log probabilities.

```python
model2.fit(X)
model2.log_probability(X).sum()
```

--------------------------------

### Normal Distribution HMM Backward Pass Timing

Source: https://github.com/jmschrei/pomegranate/blob/master/benchmarks/Benchmark_5_Hidden_Markov_Model.ipynb

Measures the execution time of the backward pass for the three created Normal distribution HMMs using %timeit. This is useful for performance benchmarking.

```python
%timeit [model1.backward(x) for x in X]
%timeit model2.backward(X)
%timeit model3.backward(X)
```

--------------------------------

### Fit Normal Distribution on CPU

Source: https://github.com/jmschrei/pomegranate/blob/master/docs/tutorials/C_Feature_Tutorial_1_GPU_Usage.ipynb

Demonstrates fitting a Normal distribution from the Pomegranate library using PyTorch tensors on the CPU. It shows how to initialize the distribution and then fit it with sample data.

```Python
from pomegranate.distributions import Normal

X = torch.randn(5000, 5)
dist = Normal().fit(X)

dist.means, dist.covs
```

--------------------------------

### Categorical Distribution HMM Backward Pass Timing

Source: https://github.com/jmschrei/pomegranate/blob/master/benchmarks/Benchmark_5_Hidden_Markov_Model.ipynb

Measures the execution time of the backward pass for the three created Categorical distribution HMMs using %timeit. This benchmarks the backward inference for categorical HMMs.

```python
%timeit [model1.backward(x) for x in X]
%timeit model2.backward(X)
%timeit model3.backward(X)
```

--------------------------------

### Initialize Bayesian Network with Structure and Fit

Source: https://github.com/jmschrei/pomegranate/blob/master/docs/tutorials/B_Model_Tutorial_6_Bayesian_Networks.ipynb

Demonstrates initializing a Bayesian Network directly with a specified structure and then fitting it to data. This is useful when the structure is known but parameters are not.

```Python
model3 = BayesianNetwork(structure=[(), (0,)])
model3.fit(X)

model3.distributions[1].probs[0]
```

--------------------------------

### Using GeneralMixtureModel with bfloat16

Source: https://github.com/jmschrei/pomegranate/blob/master/docs/tutorials/C_Feature_Tutorial_2_Mixed_Precision_and_DataTypes.ipynb

This snippet illustrates how to initialize and fit a `GeneralMixtureModel` with `bfloat16` data types. It shows converting the model to `bfloat16` and then fitting it with the prepared data.

```Python
from pomegranate.gmm import GeneralMixtureModel

model = GeneralMixtureModel([Normal(covariance_type='diag'), Normal(covariance_type='diag')], verbose=True)
model = model.to(torch.bfloat16)
model.fit(X)
```

--------------------------------

### rebuildconda script for environment management

Source: https://github.com/jmschrei/pomegranate/blob/master/docs/whats_new.rst

A script to remove and create conda environments for development. It defaults to rebuilding the 'py2.7' environment but can be configured for different versions. Example usage shows creating, testing, and removing an environment.

```Shell
./rebuildconda 2.7.9 ; make PY2_ENV=py2.7.9 bigclean py2build py2test py2install nbtest ; source deactivate ; conda env remove --name py2.7.9
```

--------------------------------

### Out-of-Core Summarization of Bayesian Networks

Source: https://github.com/jmschrei/pomegranate/blob/master/docs/tutorials/B_Model_Tutorial_6_Bayesian_Networks.ipynb

Illustrates how to learn Bayesian network parameters using out-of-core summarization. This involves calling the `summarize` method on batches of data and then using `from_summaries` to build the model. It also shows how to inspect learned parameters and compare them to data averages.

```Torch
X = torch.randint(2, size=(20, 4))

d1 = Categorical([[0.1, 0.9]])
d2 = Categorical([[0.3, 0.7]])
d3 = ConditionalCategorical([[[0.4, 0.6], [0.3, 0.7]]])
d4 = ConditionalCategorical([[[[0.8, 0.2], [0.1, 0.9]], [[0.1, 0.9], [0.5, 0.5]]]])

model = BayesianNetwork([d1, d2, d3, d4], [(d1, d3), (d2, d4), (d3, d4)])
model.summarize(X[:5])
model.summarize(X[5:])
model.from_summaries()
```

```Torch
model.distributions[0].probs, model.distributions[1].probs
```

```Torch
torch.mean(X.type(torch.float32), dim=0)
```

--------------------------------

### Normal Distribution HMM Forward Pass Timing

Source: https://github.com/jmschrei/pomegranate/blob/master/benchmarks/Benchmark_5_Hidden_Markov_Model.ipynb

Measures the execution time of the forward pass for the three created Normal distribution HMMs using %timeit. This helps compare the performance of different HMM implementations.

```python
%timeit [model1.forward(x) for x in X]
%timeit model2.forward(X)
%timeit model3.forward(X)
```

--------------------------------

### Summarize and Update Normal Distribution

Source: https://github.com/jmschrei/pomegranate/blob/master/docs/tutorials/C_Feature_Tutorial_3_Out_Of_Core_Learning.ipynb

Demonstrates the out-of-core learning process for a Normal distribution. It shows how to use `summarize` with data batches and then `from_summaries` to update the distribution's parameters, comparing the results to direct in-memory training.

```Python
from pomegranate.distributions import Normal

X = torch.randn(1000, 5)

dist = Normal()

dist.summarize(X[:200])
dist.summarize(X[200:])

dist.from_summaries()
print(dist.means, dist.covs)

dist = Normal()
dist.summarize(X)
dist.from_summaries()
print(dist.means, dist.covs)
```

--------------------------------

### Time Bernoulli Naive Bayes Prediction

Source: https://github.com/jmschrei/pomegranate/blob/master/benchmarks/Benchmark_4_Bayes_Classifier.ipynb

Evaluates and compares the prediction speed of Bernoulli Naive Bayes models from scikit-learn and Pomegranate. The time taken to predict on the dataset is measured.

```Python
model_sklearn = BernoulliNB().fit(X, y)
model_pom = BayesClassifier([Bernoulli() for i in range(k)]).fit(X, y)

%timeit model_sklearn.predict(X)
%timeit model_pom.predict(X)
```

--------------------------------

### Get Predicted Probabilities with Factor Graph

Source: https://github.com/jmschrei/pomegranate/blob/master/docs/tutorials/B_Model_Tutorial_7_Factor_Graphs.ipynb

Illustrates how to obtain the full predicted probabilities for each category of variables using the `predict_proba` method with a MaskedTensor. The output is a list of tensors, where each tensor corresponds to a data dimension.

```Python
model.predict_proba(X_masked)
```

--------------------------------

### Calculate Probability with Initialized Normal Distribution

Source: https://github.com/jmschrei/pomegranate/blob/master/docs/tutorials/B_Model_Tutorial_1_Distributions.ipynb

Computes the probability density for data points in tensor X using the pre-initialized Normal distribution `d1`. This shows how to get the probability density for a distribution with specified parameters.

```Python
d1.probability(X)
```

--------------------------------

### Model Serialization (Loading)

Source: https://github.com/jmschrei/pomegranate/blob/master/README.md

Illustrates how to load a previously saved Pomegranate model from a file using PyTorch's `torch.load` function.

```Python
>>> model = torch.load("test.torch")
```

--------------------------------

### Initialize and Time GMM Models

Source: https://github.com/jmschrei/pomegranate/blob/master/benchmarks/Benchmark_2_General_Mixture_Models.ipynb

Initializes parameters for Gaussian Mixture Models (means, covariances, priors) for both scikit-learn and Pomegranate. It then uses the %timeit magic command to measure the execution time of fitting these models to the generated data.

```Python
means = X[numpy.random.choice(X.shape[0], replace=False, size=k)]
means2 = numpy.copy(means)

covs = numpy.array([numpy.eye(d) for i in range(k)])
covs2 = numpy.copy(covs)

priors = numpy.ones(k) / k

%timeit -n 1 -r 1 GaussianMixture(k, tol=1e-2, max_iter=20, means_init=means, precisions_init=covs, weights_init=priors).fit(X)
%timeit -n 1 -r 1 GeneralMixtureModel([Normal(means2[i], covs2[i]) for i in range(k)], tol=1e-2, max_iter=20, init='random').fit(X)
```