### Install PyRIID from PyPI

Source: https://github.com/sandialabs/pyriid/blob/main/README.md

Use this command to install the latest stable version of PyRIID from the Python Package Index.

```sh
pip install riid
```

--------------------------------

### Install PyRIID for Development

Source: https://github.com/sandialabs/pyriid/blob/main/README.md

Install PyRIID in editable mode for development, including development dependencies. This is recommended if you plan to contribute to the project.

```sh
pip install -e ".[dev]"
```

--------------------------------

### Build Local Documentation

Source: https://github.com/sandialabs/pyriid/blob/main/README.md

Build the API documentation locally. This requires installing dependencies from pdoc/requirements.txt and then running the pdoc command.

```sh
pip install -r pdoc/requirements.txt
pdoc riid -o docs/ --html --template-dir pdoc
```

--------------------------------

### Install PyRIID for Development with Compat Mode

Source: https://github.com/sandialabs/pyriid/blob/main/README.md

If you encounter Pylance issues during development, use this command to install PyRIID in editable mode with compatibility settings.

```sh
pip install -e ".[dev]" --config-settings editable_mode=compat
```

--------------------------------

### Install Latest PyRIID from GitHub

Source: https://github.com/sandialabs/pyriid/blob/main/README.md

Install the most recent features directly from the main branch of the GitHub repository. Changes may appear here before they are released on PyPI.

```sh
pip install git+https://github.com/sandialabs/pyriid.git@main
```

--------------------------------

### Select and Copy Data from SampleSet

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 2/Primer 2.ipynb

Demonstrates how to select all samples from a SampleSet using slicing `[:]` for a full copy. Be cautious with large datasets. Examples of selecting specific samples by index are also commented out.

```python
seeds = simple_seeds[:]  # Select all samples; how you make a full copy; careful with large datasets...
# seeds[0].sources  # select first sample
# seeds[:4].sources  # select first 4 samples
# seeds[-2:].sources  # select last 2 samples

```

--------------------------------

### Check GADRAS Installation Path

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 2/Primer 2.ipynb

Prints the current GADRAS installation path. If GADRAS is installed in a non-default location, set the GADRAS_DIR environment variable to point to it.

```python
from riid.gadras.api import GADRAS_INSTALL_PATH

print(GADRAS_INSTALL_PATH)

# If you have a non-default GADRAS installation, set an environment variable
# named "GADRAS_DIR" to point to your custom install location

```

--------------------------------

### Inspecting SampleSet Data

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 1/Primer1.ipynb

Provides examples of how to inspect the contents of a `SampleSet` object, including its spectra, sources, information, and prediction probabilities. This is useful for understanding the data structure after synthesis or loading.

```python
"""Inspecting SampleSets"""
seeds_ss
# seeds_ss.spectra
# seeds_ss.sources
# seeds_ss.info
# seeds_ss.prediction_probas
```

--------------------------------

### Predict and Get Confidence Scores with MLPClassifier

Source: https://context7.com/sandialabs/pyriid/llms.txt

Illustrates predicting with a trained MLPClassifier and then obtaining prediction confidence scores using `get_confidences`. This requires a trained model and SampleSets.

```python
from riid import get_dummy_seeds
from riid.data.synthetic.static import StaticSynthesizer
from riid.models.neural_nets.basic import MLPClassifier

seeds_ss = get_dummy_seeds(n_channels=512, normalize=True)
fg_seeds_ss, bg_seeds_ss = seeds_ss.split_fg_and_bg()
synth = StaticSynthesizer(samples_per_seed=100, return_fg=True)
fg_ss, _ = synth.generate(fg_seeds_ss, bg_seeds_ss)
fg_ss.normalize()

model = MLPClassifier()
model.fit(fg_ss, epochs=20, verbose=False)
model.predict(fg_ss)

# The get_confidences method would be called here on fg_ss after prediction.
```

--------------------------------

### Construct and Manipulate SampleSet Objects

Source: https://context7.com/sandialabs/pyriid/llms.txt

Demonstrates the construction of a SampleSet using dummy seeds, along with common operations like slicing, accessing properties, normalization, downsampling, serialization, splitting, concatenation, difficulty scoring, energy calibration re-binning, and region of interest extraction. Use `get_dummy_seeds` for testing and demos.

```python
import numpy as np
import pandas as pd
from riid import SampleSet, SpectraState, SpectraType, get_dummy_seeds

# --- Construction via get_dummy_seeds (testing / demos) ---
seeds_ss = get_dummy_seeds(
    n_channels=512,
    live_time=600.0,
    count_rate=1000.0,
    normalize=True,
    rng=np.random.default_rng(42)
)
print(seeds_ss)
# SampleSet Summary:
#  - # of channels:  512
#  - # of samples:   10
#  - Present sources: Am241, Ba133, K40, Pu239, Ra226, Th232, U238

# --- Slicing ---
first_two = seeds_ss[:2]
print(first_two.n_samples)   # 2

# --- Properties ---
print(seeds_ss.n_channels)           # 512
print(seeds_ss.spectra_state)        # SpectraState.L1Normalized
print(seeds_ss.spectra_type)         # SpectraType.BackgroundForeground
print(seeds_ss.get_labels("Isotope").values[:3])  # ['Am241' 'Ba133' 'K40']

# --- Normalise spectra (L1 / L2) ---
seeds_ss.normalize(p=1)  # in-place; sets spectra_state = L1Normalized

# --- Downsample channels ---
seeds_ss.downsample_spectra(target_bins=128)
print(seeds_ss.n_channels)  # 128

# --- Serialise / deserialise ---
seeds_ss.to_hdf("seeds.h5")
seeds_ss.to_json("seeds.json")
seeds_ss.to_pcf("seeds.pcf")

from riid import read_hdf, read_json, read_pcf
loaded_ss = read_hdf("seeds.h5")

# --- Split foreground and background seeds ---
fg_ss, bg_ss = seeds_ss.split_fg_and_bg(
    bg_seed_names=["PotassiumInSoil", "UraniumInSoil", "ThoriumInSoil", "Cosmic"]
)
print(fg_ss.spectra_type)   # SpectraType.Foreground
print(bg_ss.spectra_type)   # SpectraType.Background

# --- Concatenate multiple SampleSets ---
combined = SampleSet()
combined.concat([fg_ss, bg_ss])
print(combined.n_samples)   # fg + bg sample count

# --- Difficulty score (0 = easy, 1 = hard) ---
score = seeds_ss.difficulty_score
print(f"Difficulty: {score:.3f}")

# --- Energy calibration re-binning ---
rebinned = seeds_ss.as_ecal(
    new_offset=0.0, new_gain=3000.0,
    new_quadratic=0.0, new_cubic=0.0, new_low_energy=0.0
)

# --- Region of interest extraction ---
roi_ss = rebinned.as_regions([(100, 400), (1000, 1500)])  # keV ranges

```

--------------------------------

### Get Labels from SampleSet

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 2/Primer 2.ipynb

Extracts labels from a SampleSet at different hierarchical levels (Isotope, Category, Seed) and with or without associated values. The default level is 'Isotope'.

```python
seeds.get_labels()  # By default, the isotope level is extracted
seeds.get_labels(target_level="Isotope")
seeds.get_labels(target_level="Category")
seeds.get_labels(target_level="Seed")
seeds.get_labels(include_value=True)

# More arguments will be explored later as they become relevant

```

--------------------------------

### Splitting Foreground and Background Seeds

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 1/Primer1.ipynb

Demonstrates how to separate a `SampleSet` containing both foreground and background spectra into two distinct `SampleSet` objects. This is a common preprocessing step before further analysis.

```python
"""Separating our background seeds from our foreground seeds."""
fg_seeds_ss, bg_seeds_ss = seeds_ss.split_fg_and_bg()

print(fg_seeds_ss)
print(bg_seeds_ss)
```

--------------------------------

### Predict and Get Predictions for IND Data

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 2/Primer 2.ipynb

Generates predictions for the in-distribution test data using the loaded model and retrieves these predictions, optionally including their associated values.

```python
model.predict(testing_data)
```

```python
testing_data.get_predictions(include_value=True)
```

--------------------------------

### Run All Unit Tests

Source: https://github.com/sandialabs/pyriid/blob/main/README.md

Execute all unit tests for the PyRIID package using the unittest module. The -v flag provides verbose output.

```sh
python -m unittest tests/*.py -v
```

--------------------------------

### Synthesizing Seeds with PyRIID

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 1/Primer1.ipynb

Initializes the `SeedSynthesizer` to generate seed spectra. It checks for GADRAS API availability and falls back to dummy seeds if not found. This is useful for creating template spectra for further analysis.

```python
"""Synthesizing seeds"""
from riid.gadras.api import GADRAS_API_SEEMINGLY_AVAILABLE

if GADRAS_API_SEEMINGLY_AVAILABLE:
    from riid import SeedSynthesizer
    seed_syn = SeedSynthesizer()
    # The YAML file defining the seed synthesis specification is ultimately parsed into a dictionary.
    # You can also load it yourself and pass in the dictionary instead - this is useful for varying detector parameters!
    seeds_ss = seed_syn.generate("./spec_nai_few_sources.yaml")
else:
    # If you don't have Windows with GADRAS installed, this will use the dummy seeds below which are not actual gamma spectra.
    # Another option would be to load a seeds file obtained elsewhere.
    from riid import get_dummy_seeds
    seeds_ss = get_dummy_seeds()
```

--------------------------------

### Prepare Post-Processing Data with JSD and Model Probabilities

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 2/Primer 2.ipynb

Reads seed data, downsamples spectra, calculates multi-class JSDs, and combines them with model probabilities and SNR into a DataFrame for further analysis.

```python
fg_seeds = read_hdf(FG_SEEDS_PATH)
fg_seeds.downsample_spectra(128)
# We specifically do NOT want to use `sources` for confidence as we won't have that in practice
sample_jsds = testing_data.get_multiclass_jsds(fg_seeds, model.target_level)

def multiclass_jsds_to_top_jsd(jsds):
    post = []
    for d in jsds:
        min_key = min(d, key=d.get)
        post.append((min_key, d[min_key]))
    post_df = pd.DataFrame(post, columns=["seed", "jsd"])
    return post_df

post_df = multiclass_jsds_to_top_jsd(sample_jsds)
post_df["model_proba"] = testing_data.prediction_probas.T.groupby(model.target_level).sum().max()
post_df["snr"] = testing_data.info.snr
```

--------------------------------

### Load SampleSet Objects from Files

Source: https://context7.com/sandialabs/pyriid/llms.txt

Provides functions to load `SampleSet` objects from HDF5, JSON, and PCF formats. HDF5 is recommended for large datasets due to compression, while JSON is suitable for smaller sets, and PCF reads the GADRAS binary format.

```python
from riid import read_hdf, read_json, read_pcf

# Load from HDF (preferred for large data)
try:
    ss_from_hdf = read_hdf("measurements.h5")
    print(f"Loaded {ss_from_hdf.n_samples} samples from HDF")
except FileNotFoundError as e:
    print(e)

# Load from JSON
ss_from_json = read_json("seeds.json")
print(ss_from_json.n_samples)

# Load from PCF (GADRAS binary format)
ss_from_pcf = read_pcf("gadras_output.pcf", verbose=False)
print(ss_from_pcf.detector_info)

```

--------------------------------

### Save and Load SampleSets to/from HDF5 and PCF

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 1/Primer1.ipynb

Persist SampleSets to disk using HDF5 for numerical data and PCF for configuration. This allows for later retrieval and reuse of spectral datasets.

```python
"""Saving and loading SampleSets"""
fg_seeds_ss.to_hdf("./fg_seeds.h5")
bg_seeds_ss.to_hdf("./bg_seeds.h5")

fg_seeds_ss.to_pcf("./fg_seeds.pcf")
bg_seeds_ss.to_pcf("./bg_seeds.pcf")
```

--------------------------------

### Inspect Sample Information

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 2/Primer 2.ipynb

Shows the 'info' DataFrame, which provides additional descriptive information for each sample in the SampleSet.

```python
simple_seeds.info

```

--------------------------------

### Build and Set Sources DataFrame

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 2/Primer 2.ipynb

Manually construct a sources DataFrame using labels and a mapping function. The DataFrame is then sorted, grouped, and summed.

```python
labels = ["K40", "Am241", "Ba133", "U235", "U238", "Mo99", "Tc99m", "WGPu", "K40"]
n_samples = len(labels)
column_tuples = [label_to_index_element(l, label_level="Seed") for l in labels]
columns = pd.MultiIndex.from_tuples(column_tuples, names=SampleSet.SOURCES_MULTI_INDEX_NAMES)

sources_df = pd.DataFrame(np.identity(n_samples), columns=columns)
    .sort_index(axis=1)
    .T.groupby(level=SampleSet.SOURCES_MULTI_INDEX_NAMES)
    .sum().T
sources_df
```

--------------------------------

### Energy Calibration in PyRIID

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 2/Primer 2.ipynb

Demonstrates how to access and manipulate energy calibration information for spectra. Use `as_ecal` to interpolate spectra to a new energy calibration, useful when dealing with SampleSets that have disparate but known energy calibration parameters.

```python
"""Energy calibration information"""
seeds.info  # A lot of noise
seeds.info.loc[:, seeds.ECAL_INFO_COLUMNS]  # Better
seeds.ecal  # Succinct, but not a DF
seeds.get_channel_energies(0)  # Converting channels to energy
seeds.get_all_channel_energies()  # Converting channels to energy for all samples
new_ecal_seeds = seeds.as_ecal(0, 1000, 0, 0, 0)  # Interpolating spectra to a new energy calibration
# Note: if you have a SampleSet with DISPARATE BUT KNOWN energy calibration parameters,
#   `as_ecal()` is useful for transforming all the spectra such that they have the same energy calibration.

# PyRIID defines energy calibration in the same terms as GADRAS, which uses the Full Range Fraction defined in ANSI N42.42-2006.
# However, PyRIID does not currently support deviation pairs.

# Plot in channel space
IDX = 0
fig, ax = plt.subplots()
ax.plot(simple_seeds.spectra.iloc[IDX], label="original")
ax.plot(new_ecal_seeds.spectra.iloc[IDX], label="new")
ax.set_yscale("log")
ax.set_title("Spectra when plotted using channels")
ax.legend()
plt.show()

# Plot in energy space (note that we have cut off our data)
fig, ax = plt.subplots()
ax.plot(simple_seeds.get_channel_energies(IDX), simple_seeds.spectra.iloc[IDX], label="original")
ax.plot(new_ecal_seeds.get_channel_energies(IDX), new_ecal_seeds.spectra.iloc[IDX], label="new")
ax.set_yscale("log")
ax.set_title("Spectra when plotted using energy")
ax.legend()
plt.show()
```

--------------------------------

### Fit and Predict with PoissonBayesClassifier

Source: https://context7.com/sandialabs/pyriid/llms.txt

Demonstrates fitting a PoissonBayesClassifier with dummy seeds and predicting on generated gross measurements. Requires paired gross and background SampleSets for prediction.

```python
from riid import get_dummy_seeds
from riid.data.synthetic.static import StaticSynthesizer
from riid.models.bayes import PoissonBayesClassifier

seeds_ss = get_dummy_seeds(n_channels=512, normalize=True)
fg_seeds_ss, bg_seeds_ss = seeds_ss.split_fg_and_bg()

synth = StaticSynthesizer(
    samples_per_seed=100,
    return_fg=False,
    return_gross=True
)
_, gross_ss = synth.generate(fg_seeds_ss, bg_seeds_ss)

# Fit (builds the TF graph from seeds; no gradient descent)
model = PoissonBayesClassifier()
model.fit(fg_seeds_ss)

# Predict (requires paired gross + background SampleSets)
bg_repeat = bg_seeds_ss.sample(n_samples=gross_ss.n_samples, random_seed=0)
model.predict(gross_ss, bg_repeat, normalize_scores=True, verbose=False)

predictions = gross_ss.get_predictions(target_level="Seed")
labels = gross_ss.get_labels(target_level="Seed")
accuracy = (predictions == labels).mean()
print(f"Poisson-Bayes Accuracy: {accuracy:.2%}")
```

--------------------------------

### Model Architecture Summary

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 2/Primer 2.ipynb

Prints a summary of the model's architecture, including layer types, output shapes, and parameter counts.

```python
model.model.summary()
```

--------------------------------

### Managing Sources in PyRIID

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 2/Primer 2.ipynb

Covers the management of ground truth sources, particularly relevant when loading data from custom files or synthesizing data. Includes functions for generating data with specified source normalization, dropping sources, and cleaning up zero-valued columns.

```python
"""Managing sources

Tracking ground truth is important and can be a lot of work.
You may find yourself managing it yourself if you're loading data from your custom files.

The truth is, for better or worse, what you make it when you synthesize data.
"""
from riid.data.labeling import label_to_index_element

seeds_sources_as_counts = seed_synth.generate(seed_synth_config, normalize_sources=False)

# seeds.normalize_sources()
# seeds.drop_sources()  # By default, drops background seeds at seed level; can provide your own; normalizes by default
# seeds.drop_spectra_with_no_contributors()
# seeds.drop_sources_columns_with_all_zeros()

```

--------------------------------

### Static Synthesis of Training Data

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 2/Primer 2.ipynb

Generates foreground measurement data using a StaticSynthesizer with specified background counts per second, live time, and signal-to-noise ratio parameters. The generated data is saved to HDF5 files for training and independent testing.

```python
"""Static synthesis"""
mixed_bg_seeds = read_hdf(MIXED_BG_SEEDS_PATH)
fg_seeds = read_hdf(FG_SEEDS_PATH)

static_synth = StaticSynthesizer(
    samples_per_seed=500,
    bg_cps=300,
    live_time_function="uniform",
    live_time_function_args=(1, 10),
    snr_function="log10",
    snr_function_args=(MIN_SNR, MAX_SNR),
    long_bg_live_time=120,  # adjust this to make background subtraction worse
    return_fg=True,
    return_gross=False,
)

foregrounds, _ = static_synth.generate(fg_seeds, mixed_bg_seeds)
foregrounds.to_hdf(TRAIN_PATH)

static_synth.samples_per_seed //= 4
foregrounds, _ = static_synth.generate(fg_seeds, mixed_bg_seeds)
foregrounds.to_hdf(IND_TEST_PATH)
```

--------------------------------

### Compare SampleSets and Plot Comparison

Source: https://context7.com/sandialabs/pyriid/llms.txt

Compares two SampleSet objects by computing Jensen-Shannon distances for metadata columns and visualizes the comparison. Requires `riid.visualize`.

```python
ss1_stats, ss2_stats, col_comparisons = train_ss.compare_to(test_ss, n_bins=20)

for col, jsd in col_comparisons.items():
    print(f"{col:20s}: JSD = {jsd:.4f}")

import riid.visualize as viz
viz.plot_ss_comparison(ss1_stats, ss2_stats, col_comparisons, target_col="snr", show=False)
```

--------------------------------

### Generate Basic Seeds

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 2/Primer 2.ipynb

Synthesizes basic gamma seeds using a configuration file loaded with YAML. Ensure the 'basic.yaml' configuration file exists in the specified SEED_CONFIGS_DIR.

```python
with open(BASIC_SEED_CONFIG_PATH) as fin:
    seed_synth_config = yaml.safe_load(fin)
seed_synth = SeedSynthesizer()
simple_seeds = seed_synth.generate(seed_synth_config)

```

--------------------------------

### Inspect Dataset Information

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 2/Primer 2.ipynb

Display information about the synthesized foreground dataset, including data types, non-null counts, and memory usage.

```python
"""Inspect the dataset"""
foregrounds.info
```

--------------------------------

### Plotting Spectra with PyRIID

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 1/Primer1.ipynb

Visualizes foreground and background spectra using `plot_spectra` from `riid.visualize`. Allows customization of plot parameters like energy range, limits, and target level. Includes adding specific energy markers and legends.

```python
"""Plotting spectra"""
from riid.visualize import plot_spectra
import matplotlib.pyplot as plt

# Plot foreground(s)
am241_only_ss = fg_seeds_ss[fg_seeds_ss.get_labels() == "Am241"]
fig, ax = plot_spectra(
    am241_only_ss,
    in_energy=True,
    figsize=(9.6, 4.8),
    ylim=(1e-10, None),
    xlim=(0, 1000),
    target_level="Seed",
    show=False
)
ax.axvline(59, color="green", label="59 KeV")
ax.legend()
plt.show()

# Plot background(s)
_ = plot_spectra(
    bg_seeds_ss,
    in_energy=True,
    figsize=(9.6, 4.8),
    ylim=(1e-5, None),
    title="Backgrounds"
)
```

--------------------------------

### Load and Preprocess In-Distribution Test Data

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 2/Primer 2.ipynb

Loads and preprocesses in-distribution test data from a specified path. This is a prerequisite for evaluating model performance on known data distributions.

```python
testing_data = load_and_preprocess_data(IND_TEST_PATH)
```

--------------------------------

### Visualize Seed Mixtures with Bar Chart

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 2/Primer 2.ipynb

Visualize seed mixtures using a horizontal bar chart, displaying partitions and their proportions. This plot helps in understanding the composition of the mixed seeds.

```python
"""Another way to visualize mixtures"""
barh_kwargs = {
    "height": 1.0,
    "edgecolor": "black",
    "linewidth": 0.5,
}
bar_x = np.arange(mixed_bg_seeds.n_samples)+1
props = mixed_bg_seeds.sources.to_numpy(float)
cols = mixed_bg_seeds.sources.columns.get_level_values("Seed")

fig, ax = plt.subplots(figsize=(8, 8), sharey=True, sharex=True)

ax.barh(bar_x, props[:,0], left=0, label=cols[0], **barh_kwargs)
ax.barh(bar_x, props[:,1], left=props[:,0], label=cols[1], **barh_kwargs)
ax.barh(bar_x, props[:,2], left=props[:,:2].sum(axis=1), label=cols[2], **barh_kwargs)
ax.barh(bar_x, props[:,3], left=props[:,:3].sum(axis=1), label=cols[3], **barh_kwargs)

ax.set_title(rf"$\\alpha$ = {alphas}")
ax.set_xlabel("Partitions")
ax.set_xlim((0, 1))
ax.set_ylim((0, mixed_bg_seeds.n_samples))
ax.set_ylabel("Sample #")
ax.legend()
fig.tight_layout()
plt.show()
```

--------------------------------

### Generate Seed Templates with SeedSynthesizer (GADRAS-Based)

Source: https://context7.com/sandialabs/pyriid/llms.txt

Drive the GADRAS API to simulate detector responses for specified radioisotope sources, producing seed templates. Configuration can be a YAML path or a dictionary. Requires Windows and pythonnet.

```python
from riid.data.synthetic.seed import SeedSynthesizer

synth = SeedSynthesizer()

# Config can be a YAML path or a dict
config = "path/to/seed_config.yaml"
# Minimal YAML structure:
# gamma_detector:
#   name: "NaI 2x4x16"
#   parameters:
#     distance_cm: 100
#     height_cm: 100
# sources:
#   - isotope: Am241
#     ...

seeds_ss = synth.generate(
    config=config,
    normalize_spectra=True,
    normalize_sources=True,
    verbose=True
)
print(seeds_ss.n_samples)
print(seeds_ss.detector_info)
```

--------------------------------

### Split and Mix Seed Data

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 2/Primer 2.ipynb

Reads seed data, downsamples it, and splits it into foreground and background sets. It then creates a SeedMixer to generate mixed background spectra, saving both foreground and mixed background seeds to HDF5 files.

```python
"""Split and mix"""
seeds = read_hdf(SEEDS_PATH)
# Downsample at the seed stage!  It speeds everything up.
seeds.downsample_spectra(128)g_seeds, bg_seeds = seeds.split_fg_and_bg()
fg_seeds.to_hdf(FG_SEEDS_PATH)
bg_seeds.to_hdf(BG_SEEDS_PATH)

mixer = SeedMixer(
    bg_seeds,
    mixture_size=bg_seeds.n_samples,
    dirichlet_alpha=2,
)
mixed_bg_seeds = mixer.generate(1)
mixed_bg_seeds.to_hdf(MIXED_BG_SEEDS_PATH)
```

--------------------------------

### Generate Static SampleSets with StaticSynthesizer

Source: https://context7.com/sandialabs/pyriid/llms.txt

Use StaticSynthesizer to generate static gamma-ray spectra. Configure parameters like samples per seed, live time, SNR, and background counts. Useful for creating datasets with controlled source characteristics.

```python
from riid import get_dummy_seeds
from riid.data.synthetic.static import StaticSynthesizer

seeds_ss = get_dummy_seeds(n_channels=512, normalize=True)
fg_seeds_ss, bg_seeds_ss = seeds_ss.split_fg_and_bg()

synth = StaticSynthesizer(
    samples_per_seed=500,
    live_time_function="uniform",
    live_time_function_args=(0.25, 8.0),
    snr_function="log10",
    snr_function_args=(0.01, 100.0),
    bg_cps=300.0,
    apply_poisson_noise=True,
    normalize_sources=True,
    return_fg=True,
    return_gross=True,
)

fg_ss, gross_ss = synth.generate(
    fg_seeds_ss,
    bg_seeds_ss,
    skip_health_check=False,
    verbose=True
)
# Synthesis complete in X.XX seconds.

print(f"Foreground samples: {fg_ss.n_samples}")
print(f"Gross samples:      {gross_ss.n_samples}")
print(fg_ss.spectra.shape)   # (n_seeds * 500, 512)

# Shuffle for training
fg_ss.shuffle(inplace=True, random_state=42)
```

--------------------------------

### Generate Mixture Seeds with SeedMixer

Source: https://context7.com/sandialabs/pyriid/llms.txt

Create multi-isotope mixture seeds by randomly mixing existing seed spectra using Dirichlet-sampled proportions. Configure mixture size, Dirichlet alpha, and restricted isotope pairs. Supports batch generation for memory efficiency.

```python
from riid import get_dummy_seeds
from riid.data.synthetic.seed import SeedMixer
import numpy as np

seeds_ss = get_dummy_seeds(n_channels=512, normalize=True)
fg_seeds_ss, _ = seeds_ss.split_fg_and_bg()

mixer = SeedMixer(
    seeds_ss=fg_seeds_ss,
    mixture_size=2,          # mix 2 isotopes at a time
    dirichlet_alpha=2.0,     # higher = more equal proportions
    restricted_isotope_pairs=[("U238", "Pu239")],  # never mix these
    rng=np.random.default_rng(0)
)

# Option 1: generate all at once
mixtures_ss = mixer.generate(n_samples=1000)
print(mixtures_ss.n_samples)   # 1000

# Option 2: stream in batches (memory-efficient)
for batch_ss in mixer(n_samples=1000, max_batch_size=100):
    print(batch_ss.n_samples)  # up to 100 per batch
    # process batch_ss ...
```

--------------------------------

### Generate Seed Data

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 2/Primer 2.ipynb

Loads a configuration file to generate synthetic radiation measurement seeds using the SeedSynthesizer. The generated seeds are then saved to an HDF5 file.

```python
"""Seeds"""
with open(PROBLEM_SEED_CONFIG_PATH) as fin:
    seed_synth_config = yaml.safe_load(fin)
seed_synth = SeedSynthesizer()
seeds = seed_synth.generate(seed_synth_config, verbose=True)
seeds.to_hdf(SEEDS_PATH)
```

--------------------------------

### read_hdf / read_json / read_pcf - File I/O Functions

Source: https://context7.com/sandialabs/pyriid/llms.txt

Top-level convenience functions for loading SampleSet objects from HDF5, JSON, and PCF formats. These functions provide easy access to saved SampleSet data.

```APIDOC
## read_hdf / read_json / read_pcf

### Description
Top-level convenience functions for loading `SampleSet` objects from HDF5, JSON, and PCF (Spectral Data File) formats. `read_hdf` is recommended for large datasets due to compression; `read_json` suits small sets; `read_pcf` reads the GADRAS PCF binary format.

### Parameters
- **`path`** (string) - The file path to load the SampleSet from.
- **`verbose`** (bool, optional) - If `read_pcf` is used, controls verbosity. Defaults to `False`.

### Functions
- **`read_hdf(path)`**: Loads a SampleSet from an HDF5 file.
- **`read_json(path)`**: Loads a SampleSet from a JSON file.
- **`read_pcf(path, verbose=False)`**: Loads a SampleSet from a GADRAS PCF file.

### Example
```python
from riid import read_hdf, read_json, read_pcf

# Load from HDF (preferred for large data)
try:
    ss_from_hdf = read_hdf("measurements.h5")
    print(f"Loaded {ss_from_hdf.n_samples} samples from HDF")
except FileNotFoundError as e:
    print(e)

# Load from JSON
ss_from_json = read_json("seeds.json")
print(ss_from_json.n_samples)

# Load from PCF (GADRAS binary format)
ss_from_pcf = read_pcf("gadras_output.pcf", verbose=False)
print(ss_from_pcf.detector_info)
```
```

--------------------------------

### Plot Seed Spectra

Source: https://context7.com/sandialabs/pyriid/llms.txt

Generates a log-scaled plot of seed spectra, saving it to a file. Ensure `riid.visualize` is imported.

```python
viz.plot_spectra(
    fg_seeds_ss,
    in_energy=False,
    yscale="log",
    title="Seed Spectra",
    save_file_path="spectra.png",
    show=False
)
```

--------------------------------

### Validate and Expand Inject Configuration

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 2/Primer 2.ipynb

Validate an injection configuration and expand it using PyRIID's API. The random seed can be set in the configuration for reproducible results.

```python
from riid.gadras.api import validate_inject_config, get_expanded_config

with open(ADVANCED_SEED_CONFIG_PATH) as fin:
    seed_synth_config = yaml.safe_load(fin)

validate_inject_config(seed_synth_config)

get_expanded_config(seed_synth_config)

# You can set the random seed in the config for reproducibility
```

--------------------------------

### Static Synthesis of Spectra

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 2/Primer 2.ipynb

Perform static synthesis to combine foreground and background seeds, simulating variations in live time, SNR, and background count rate. This allows for obtaining foreground, background, or gross spectra.

```python
"""Static synthesis

Static synthesis takes your foreground seeds and your background seeds and adds them together, randomly capturing variation in:
- live time
- signal-to-noise ratio (SNR)
- background count rate (effectively)
- Poisson fluctuations

You can obtain foreground, background, or gross spectra.
"""
static_synth = StaticSynthesizer(
    samples_per_seed=100,
    bg_cps=300.0,
    live_time_function="uniform",
    live_time_function_args=(0.25, 8),
    snr_function="log10",
    snr_function_args=(1, 200),
    long_bg_live_time=120,  # adjust this to make background subtraction worse
    return_fg=True,
    return_gross=False,
)
foregrounds, _ = static_synth.generate(fg_seeds, mixed_bg_seeds)
```

--------------------------------

### Plot Model Learning Curve

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 1/Primer1.ipynb

Visualizes the training and validation loss over epochs. Requires the training history object from model.fit().

```python
"""Model learning curve"""
from riid.visualize import plot_learning_curve

_ = plot_learning_curve(history.history["loss"], history.history["val_loss"])
```

--------------------------------

### StaticSynthesizer - Static Spectrum Synthesis

Source: https://context7.com/sandialabs/pyriid/llms.txt

Generates batches of synthetic gamma spectra from seed templates by sampling live-time and SNR. This is a primary tool for producing training data.

```APIDOC
## StaticSynthesizer

### Description
`StaticSynthesizer` generates large batches of labelled synthetic gamma spectra from seed templates by randomly sampling live-time and SNR spaces. It is the primary tool for producing training data for RIID classifiers.

### Methods
- **`__init__(...)`**: Constructor for StaticSynthesizer.
- **`synthesize(...)`**: Generates synthetic spectra.

### Example
```python
from riid import get_dummy_seeds
from riid.data.synthetic.static import StaticSynthesizer

# Example usage would go here, assuming synthesizer is initialized and used to generate data.
# For instance:
# synthesizer = StaticSynthesizer()
# synthetic_data = synthesizer.synthesize(seed_samples=get_dummy_seeds())
```
```

--------------------------------

### Inspect Seed Data

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 2/Primer 2.ipynb

Prints summary statistics of the generated seed data, including maximum dead time, number of distinct seeds, and number of distinct isotopes. It also plots the first 7 spectra.

```python
"""Inspect seeds"""
print(f"Maximum dead time present: {seeds.info.dead_time_prop.max():.4f}")
print(f"# of distinct seeds:       {seeds.n_samples}")
print(f"# of distinct isotopes:    {seeds.get_labels().unique().shape[0]}")
_ = plot_spectra(seeds[:7], in_energy=True, target_level="Seed")
```

--------------------------------

### Visualize Spectra Data

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 2/Primer 2.ipynb

Accesses and plots the gamma spectra from the 'simple_seeds' SampleSet object. The plot displays spectra in energy units.

```python
simple_seeds.spectra

_ = plot_spectra(simple_seeds, in_energy=True)

```

--------------------------------

### Spectra States and Types in PyRIID

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 2/Primer 2.ipynb

Explains the concepts of spectra 'state' and 'type' within PyRIID. 'State' refers to the form of the spectra, while 'Type' categorizes them as Background, Foreground, or Gross. These are crucial for PyRIID operations.

```python
"""Spectra states and types

State: the form in which the spectra exist
Type: Background, Foreground, Gross

Both are used by PyRIID to track and properly carry out certain operations.
"""
simple_seeds.spectra_state, simple_seeds.spectra_type, fg_seeds.spectra_type, bg_seeds.spectra_type, new_ss.spectra_type
```

--------------------------------

### Process JSDs and Identify OOD Samples

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 2/Primer 2.ipynb

Converts JSDs to a DataFrame, adds model probabilities, signal-to-noise ratio (SNR), and determines if samples are OOD based on JSD and SNR thresholds. Prints the count of OOD samples.

```python
new_post_df = multiclass_jsds_to_top_jsd(ood_src_data_jsds)
new_post_df["model_proba"] = ood_src_data.prediction_probas.max(axis=1)
new_post_df["snr"] = ood_src_data.info.snr
new_post_df["ood"] = (new_post_df.jsd > spline(new_post_df.snr)) | ~new_post_df.snr.between(MIN_SNR, MAX_SNR)
print(new_post_df.ood.value_counts())
```

--------------------------------

### Plot Learning Curve

Source: https://context7.com/sandialabs/pyriid/llms.txt

Displays the training and validation loss over epochs to visualize the learning process. Requires the training history object.

```python
viz.plot_learning_curve(
    train_loss=history.history["loss"],
    validation_loss=history.history["val_loss"],
    save_file_path="learning_curve.png",
    show=False
)
```

--------------------------------

### Real-time Anomaly Detection with PoissonNChannelEventDetector

Source: https://context7.com/sandialabs/pyriid/llms.txt

Demonstrates using PoissonNChannelEventDetector for streaming anomaly detection. It simulates a measurement stream and flags events based on statistical anomalies.

```python
import numpy as np
from riid.anomaly import PoissonNChannelEventDetector

detector = PoissonNChannelEventDetector(
    long_term_duration=120,     # seconds of background history
    short_term_duration=1,      # seconds in "foreground" window
    pre_event_duration=5,
    max_event_duration=120,
    post_event_duration=1.5,
    tolerable_false_alarms_per_day=1.0,
    anomaly_threshold_update_interval=60,
)

n_channels = 1024
sample_interval = 0.25   # seconds
bg_cps = 300.0

# Stream measurements through the detector
for t in range(3000):   # 750 seconds at 0.25s intervals
    # Simulate background (Poisson noise)
    bg_counts = np.random.poisson(bg_cps * sample_interval / n_channels, n_channels)
    # Inject a source anomaly for 10 seconds around t=500
    if 500 <= t < 540:
        bg_counts += np.random.poisson(50, n_channels)

    event_result = detector.add_measurement(
        measurement_id=t,
        measurement=bg_counts,
        duration=sample_interval,
        verbose=False
    )

    if event_result is not None:
        event_spectrum, bg_spectrum, event_duration, meas_ids = event_result
        print(f"Event detected! Duration={event_duration:.1f}s, "
              f"Total counts={event_spectrum.sum()}")

print(f"Background fill: {detector.background_percent_complete:.1f}%")
```

--------------------------------

### Load Model from JSON

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 2/Primer 2.ipynb

Loads a previously saved model from a JSON file. Ensure the model architecture is defined before loading.

```python
model = MLPClassifier()
model.load(MODEL_JSON_PATH)
```

--------------------------------

### PyRIIDModel.save / load / to_onnx / to_tflite

Source: https://context7.com/sandialabs/pyriid/llms.txt

Methods for model serialization, allowing models to be saved to disk, reloaded, and exported to ONNX and TFLite formats for deployment.

```APIDOC
## PyRIIDModel.save / load / to_onnx / to_tflite

### Description
All PyRIID models inherit `save`, `load`, `to_onnx`, and `to_tflite` from `PyRIIDModel`, enabling persistence and deployment to ONNX and TFLite runtimes.

### Method
`save(path, overwrite=False)`

### Parameters
- **path** (str) - Required - The file path to save the model to.
- **overwrite** (bool) - Optional - Whether to overwrite the file if it already exists.

### Method
`load(path)`

### Parameters
- **path** (str) - Required - The file path to load the model from.

### Method
`to_onnx(path)`

### Parameters
- **path** (str) - Required - The file path to export the model to in ONNX format.

### Method
`to_tflite(path, quantize=True, prune=False)`

### Parameters
- **path** (str) - Required - The file path to export the model to in TFLite format.
- **quantize** (bool) - Optional - Whether to quantize the model.
- **prune** (bool) - Optional - Whether to prune the model.

### Example
```python
from riid.models.neural_nets.basic import MLPClassifier

model = MLPClassifier()
# ... train model ...

# Save to JSON (includes architecture + weights + metadata)
model.save("my_model.json", overwrite=False)

# Reload
model2 = MLPClassifier()
model2.load("my_model.json")
print(model2.target_level)   # "Isotope"
print(model2.model_outputs)  # list of class names

# Export for deployment
model.to_onnx("my_model.onnx")
model.to_tflite("my_model.tflite", quantize=True, prune=False)
```
```

--------------------------------

### Generate Mixed Background Seeds with SeedMixer

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 1/Primer1.ipynb

Use SeedMixer to randomly combine background seeds, ensuring no seeds from the same isotope are mixed. Useful for generating realistic background spectra.

```python
"""Seed mixing"""
from riid import SeedMixer

mixed_bg_seeds_ss = SeedMixer(
    bg_seeds_ss,
    mixture_size=3,
    dirichlet_alpha=2,
).generate(n_samples=10)

print(mixed_bg_seeds_ss.sources)
_ = plot_spectra(mixed_bg_seeds_ss, in_energy=True, ylim=(1e-5, None), title="Backgrounds")
```

--------------------------------

### Separate Foreground and Background Seeds

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 2/Primer 2.ipynb

Splits a SampleSet into two new SampleSet objects: one containing foreground seeds ('fg_seeds') and the other containing background seeds ('bg_seeds').

```python
fg_seeds, bg_seeds = simple_seeds.split_fg_and_bg()

print(fg_seeds)
print(bg_seeds)

```

--------------------------------

### Generate Confusion Matrix

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 1/Primer1.ipynb

Visualizes the confusion matrix for the model's predictions on the test dataset. Requires the test SampleSet.

```python
"""Confusion Matrix"""
from riid.visualize import confusion_matrix

_ = confusion_matrix(test_fg_ss)
```

--------------------------------

### Save Model in Multiple Formats

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 1/Primer1.ipynb

Saves the trained model to JSON, TFLite, and ONNX formats. Includes logic to delete existing files before saving.

```python
"""Save model"""
from pathlib import Path


def _delete_if_exists(path: Path):
    if path.exists():
        path.unlink()

model_path_json = Path("./model.json")
model_path_tflite = model_path_json.with_suffix(".tflite")
model_path_onnx = model_path_json.with_suffix(".onnx")
_delete_if_exists(model_path_json)
_delete_if_exists(model_path_tflite)
_delete_if_exists(model_path_onnx)

model.save(str(model_path_json))
model.to_tflite(str(model_path_tflite))
model.to_onnx(str(model_path_onnx))
```

--------------------------------

### Plot Sampled Spectra

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 2/Primer 2.ipynb

Sample a subset of the synthesized foreground spectra and plot them. Note that negative values may appear in the plotted spectra.

```python
"""Sample our dataset to plot"""
_ = plot_spectra(foregrounds.sample(3), in_energy=True)  # Note the negatives
```

--------------------------------

### Generate Noisy Spectra with StaticSynthesizer

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 1/Primer1.ipynb

Create synthetic spectra with controlled noise levels (SNR) and live times using StaticSynthesizer. This function allows for generating realistic datasets for model training and testing.

```python
"""Static Synthesis"""
from riid import StaticSynthesizer

static_syn = StaticSynthesizer(
    samples_per_seed=100,
    bg_cps=300,
    live_time_function="uniform",
    live_time_function_args=(0.25, 8),
    snr_function="log10",
    snr_function_args=(0.1, 100),
    apply_poisson_noise=True,
    return_fg=True,
    return_gross=True,
)
fg_ss, gross_ss = static_syn.generate(fg_seeds_ss, mixed_bg_seeds_ss)  # Note: be sure to use the mixed background seeds!

# Adjust ground truth
# gross_ss.sources.drop(bg_seeds_ss.sources.columns, axis=1, inplace=True)  # Remove background ground truth to target model focus
# gross_ss.normalize_sources()

# You can perform sample-wise, channel-wise addition and subtraction between two compatible SampleSets
bg_ss = gross_ss - fg_ss
```

--------------------------------

### Downsampling Spectra in PyRIID

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 2/Primer 2.ipynb

Reduces the number of channels in spectra, also known as down-binning or rebinning. This operation is destructive, so it's recommended to work on a copy of the SampleSet.

```python
"""Downsampling (AKA down-binning, AKA rebinning into less channels) """
downsampled_seeds = simple_seeds[:]  # Make a copy, we're about to be destructive
downsampled_seeds.downsample_spectra(128)
downsampled_seeds
```

--------------------------------

### Import Libraries and Define Constants

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 2/Primer 2.ipynb

Imports essential libraries like pathlib, matplotlib, numpy, pandas, yaml, and specific modules from the riid package. It also defines constants for minimum and maximum SNR values and specifies directories and file paths for seed configurations and data.

```python
from pathlib import Path

import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import yaml

from riid import (
    SampleSet,
    SeedMixer,
    SeedSynthesizer,
    StaticSynthesizer,
    read_hdf,
    read_json,
    read_pcf,
)
from riid.models import MLPClassifier
from riid.visualize import plot_spectra

# SNR values for example problem
MIN_SNR = 5
MAX_SNR = 300

# Directories
SEED_CONFIGS_DIR = Path("./seed_configs/")
assert SEED_CONFIGS_DIR.exists()
DATA_DIR = Path("./problem_data")
DATA_DIR.mkdir(exist_ok=True)

# Files
BASIC_SEED_CONFIG_PATH = SEED_CONFIGS_DIR.joinpath("basic.yaml")
ADVANCED_SEED_CONFIG_PATH = SEED_CONFIGS_DIR.joinpath("advanced.yaml")
PROBLEM_SEED_CONFIG_PATH = SEED_CONFIGS_DIR.joinpath("problem.yaml")
SEEDS_PATH = str(DATA_DIR.joinpath("seeds.h5"))
FG_SEEDS_PATH = str(DATA_DIR.joinpath("fg_seeds.h5"))
BG_SEEDS_PATH = str(DATA_DIR.joinpath("bg_seeds.h5"))
MIXED_BG_SEEDS_PATH = str(DATA_DIR.joinpath("mixed_bg_seeds.h5"))
TRAIN_PATH = str(DATA_DIR.joinpath("train.h5"))
MODEL_JSON_PATH = str(DATA_DIR.joinpath("model.json"))
MODEL_ONNX_PATH = str(DATA_DIR.joinpath("model.onnx"))
MODEL_TFLITE_PATH = str(DATA_DIR.joinpath("model.tflite"))
IND_TEST_PATH = str(DATA_DIR.joinpath("test.h5"))  # Hold out, not seen in training

```

--------------------------------

### Compute Jensen-Shannon Divergences (JSDs)

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 2/Primer 2.ipynb

Computes the multiclass Jensen-Shannon Divergences (JSDs) for the OOD SNR data using the provided foreground seeds and the model's target level. This is a key step in quantifying the difference between predicted and true distributions.

```python
ood_snr_data_jsds = ood_snr_data.get_multiclass_jsds(fg_seeds, model.target_level)
```

--------------------------------

### Save Model in Multiple Formats

Source: https://github.com/sandialabs/pyriid/blob/main/examples/courses/Primer 2/Primer 2.ipynb

Saves the trained model in JSON, ONNX, and TFLite formats. Use JSON for PyRIID compatibility, ONNX for interoperability, and TFLite for edge devices.

```python
model.save(MODEL_JSON_PATH)
```

```python
model.to_onnx(MODEL_ONNX_PATH)
```

```python
model.to_tflite(MODEL_TFLITE_PATH)
```