### Install matplotlib

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/learn/pls_regression.ipynb

Install the matplotlib library for data visualization.

```bash
pip install matplotlib
```

--------------------------------

### Install Dependencies

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/learn/pls_classification.ipynb

Install necessary libraries for the project using pip.

```bash
pip install chemotools
pip install matplotlib
pip install numpy
pip install pandas
pip install scikit-learn
```

--------------------------------

### Install Chemotools

Source: https://context7.com/paucablop/chemotools/llms.txt

Install the chemotools library from PyPI or Conda. Visualization support can be included with `[viz]`.

```bash
pip install chemotools
```

```bash
conda install -c conda-forge chemotools
```

```bash
pip install chemotools[viz]
```

--------------------------------

### Install Astartes

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/explore/astartes.md

Command to install the Astartes library via pip.

```bash
pip install astartes
```

--------------------------------

### Install chemotools via Package Managers

Source: https://github.com/paucablop/chemotools/blob/main/README.md

Commands to install the library using pip or conda.

```bash
pip install chemotools
```

```bash
conda install -c conda-forge chemotools
```

--------------------------------

### Install project dependencies

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/learn/pls_regression.ipynb

List of required Python packages for the chemotools project.

```text
chemotools
matplotlib
numpy
pandas
scikit-learn
```

--------------------------------

### Install Chemotools using pip

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/user/install.md

Use this command to install the chemotools package via pip. Ensure pip is up-to-date.

```bash
pip install chemotools
```

--------------------------------

### Install Chemotools using conda

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/user/install.md

Use this command to install the chemotools package from the conda-forge channel. This is recommended for users managing environments with conda.

```bash
conda install -c conda-forge chemotools
```

--------------------------------

### Configure RangeCut Transformer

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/_static/images/explore/pipelines/pipelines_pipeline_visualization.html

Configuration for the RangeCut transformer, specifying the start and end points for the range.

```python
RangeCut(end=1550, start=950)
```

--------------------------------

### Get Explained Variance Ratio

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/explore/inspector_fundamentals.md

Retrieves the explained variance ratio for both X and Y spaces. This helps in understanding how much variance is captured by the model components.

```python
x_variance = inspector.get_explained_x_variance_ratio()
y_variance = inspector.get_explained_y_variance_ratio()
```

--------------------------------

### Initialize PCA Model Analysis

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/explore/plotting_fundamentals.md

Import necessary libraries for analyzing chemometric models.

```python
from sklearn.decomposition import PCA
import matplotlib.pyplot as plt
```

--------------------------------

### Run development tasks with Taskfile

Source: https://github.com/paucablop/chemotools/blob/main/CONTRIBUTING.md

Use these commands to manage dependencies, run quality checks, and execute tests.

```bash
task install     # install dependencies
task check       # run formatting, linting, typing, tests
 task test        # quick test run in the current environment
task test:matrix # run the nox compatibility matrix locally
task coverage    # run tests with coverage
task build       # build the package
```

--------------------------------

### Run project tests

Source: https://github.com/paucablop/chemotools/blob/main/CONTRIBUTING.md

Execute the test suite for the project.

```bash
task test
```

--------------------------------

### Importing Chemotools Classes

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/api/index.md

Demonstrates the direct import pattern for various modules and classes available in the chemotools library.

```python
from chemotools.baseline import AirPls, ArPls
from chemotools.augmentation import AddNoise, BaselineShift
from chemotools.scale import MinMaxScaler, NormScaler
from chemotools.outliers import HotellingT2, Leverage
from chemotools.plotting import SpectraPlot, ScoresPlot
from chemotools.inspector import PCAInspector, PLSRegressionInspector
# ... and so on
```

--------------------------------

### Load Sample Datasets

Source: https://context7.com/paucablop/chemotools/llms.txt

Load built-in fermentation and coffee spectral datasets. Data can be loaded as pandas DataFrames or polars DataFrames.

```python
from chemotools.datasets import load_fermentation_train, load_fermentation_test, load_coffee

spectra_train, hplc_train = load_fermentation_train()
print(f"Training spectra shape: {spectra_train.shape}")

spectra_test, hplc_test = load_fermentation_test()
print(f"Test spectra shape: {spectra_test.shape}")

coffee_spectra, coffee_labels = load_coffee()
print(f"Coffee spectra shape: {coffee_spectra.shape}")

spectra_polars, hplc_polars = load_fermentation_train(set_output="polars")
```

--------------------------------

### Build and Apply PLSRegression Model

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/explore/sklearn.md

Demonstrates the three-step workflow of defining, fitting, and predicting using a PLSRegression model.

```python
# 1. Define the PLS model with two components
pls = PLSRegression(n_components=2)

# 2. Fit the PLS model to the training data
pls.fit(X_scaled, y)

# 3. Apply the PLS model to new data
y_pred = pls.predict(X_new_scaled)
```

--------------------------------

### GET /inspector/data

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/explore/inspector_fundamentals.md

Retrieves underlying model data such as scores, loadings, and variance ratios for custom analysis.

```APIDOC
## GET /inspector/data

### Description
Provides methods to extract raw data from the inspector instance.

### Methods
- **get_scores(dataset)**: Returns scores for a specific dataset.
- **get_loadings(components)**: Returns loadings, optionally filtered by component indices.
- **get_explained_variance_ratio()**: Returns the explained variance ratio.
- **get_x_scores(dataset)**: Returns X-space scores for PLS models.
- **get_x_loadings()**: Returns X-space loadings for PLS models.
- **get_x_weights()**: Returns X-space weights for PLS models.
- **get_x_rotations()**: Returns X-space rotations for PLS models.
- **get_y_scores(dataset)**: Returns Y-space scores for PLS models.
```

--------------------------------

### Initialize and Inspect Datasets

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/explore/inspector_fundamentals.md

Initialize the PLSRegressionInspector with training, testing, and validation data, then generate combined plots.

```python
inspector = PLSRegressionInspector(
    pls,
    X_train=X_train,
    y_train=y_train,
    X_test=X_test,
    y_test=y_test,
    X_val=X_val,
    y_val=y_val,
    x_axis=wn,
)

# Inspect all datasets together
figures = inspector.inspect(dataset=['train', 'test', 'val'])
```

--------------------------------

### Example Output of PLS-DA Evaluation

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/learn/pls_classification.ipynb

This is the expected output format for the accuracy score and confusion matrix after evaluating the PLS-DA model.

```text
Accuracy:  1.0
Confusion matrix: 
 [[2 0 0]
 [0 4 0]
 [0 0 6]]
```

--------------------------------

### Get Regression Coefficients

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/explore/inspector_fundamentals.md

Retrieves the regression coefficients from the model inspector. Use this to understand the linear relationship between predictors and the response variable.

```python
coefficients = inspector.get_regression_coefficients()
```

--------------------------------

### Get Model Summary Object

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/explore/inspector_fundamentals.md

Retrieves a summary object for the model. This object contains various attributes and methods to inspect model properties.

```python
# Get model summary
summary = inspector.summary()
```

--------------------------------

### Create a preprocessing pipeline

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/learn/pls_regression.ipynb

Chains multiple preprocessing steps into a single scikit-learn pipeline for spectral data.

```python
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

from chemotools.baseline import LinearCorrection
from chemotools.derivative import SavitzkyGolay
from chemotools.feature_selection import RangeCut

# create a pipeline that scales the data
preprocessing = make_pipeline(
    RangeCut(start=950, end=1500, wavenumbers=wavenumbers),
    LinearCorrection(),
    SavitzkyGolay(window_size=15, polynomial_order=2, derivate_order=1),
    StandardScaler(with_std=False),
)
```

--------------------------------

### Get Regression Metrics as Pandas DataFrame

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/explore/inspector_fundamentals.md

Retrieves regression metrics in a format suitable for pandas DataFrames. This is useful for further analysis and visualization of model performance across different datasets.

```python
import pandas as pd

# Get metrics in DataFrame-friendly format
pd.DataFrame(inspector.summary().metrics).T
```

--------------------------------

### Apply StandardScaler Preprocessing

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/explore/sklearn.md

Demonstrates the three-step workflow of defining, fitting, and transforming data using StandardScaler.

```python
# 1. Define the preprocessor
scaler = StandardScaler(with_mean=True, with_std=False)

# 2. Fit the preprocessor to the data
scaler.fit(X)

# 3. Apply the processor to the data
X_scaled = scaler.transform(X)

# ... or to new new data
X_new_scaled = scaler.transform(X_new)
```

--------------------------------

### Import Derivative Classes from chemotools

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/api/derivative.md

Import the NorrisWilliams and SavitzkyGolay classes for derivative calculations. These are essential for applying derivative methods to spectral data.

```python
from chemotools.derivative import (
    NorrisWilliams,
    SavitzkyGolay,
)
```

--------------------------------

### Load Coffee Dataset

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/learn/datasets.md

Loads the coffee dataset containing ATR-FTIR spectra from various coffee samples.

```python
from chemotools.datasets import load_coffee

spectra, labels = load_coffee()
```

```python
from chemotools.datasets import load_coffee

spectra, labels = load_coffee(set_output="polars")
```

--------------------------------

### Run Compatibility Tests with nox

Source: https://github.com/paucablop/chemotools/blob/main/README.md

Commands to manage and execute cross-version compatibility tests using nox.

```bash
uv run nox --list               # show available sessions
uv run nox -s tests-3.12       # run tests on a specific Python version
uv run nox -s tests-min-sklearn-3.10
uv run nox -s tests-min-sklearn-3.12
```

--------------------------------

### Execute Development Tasks

Source: https://github.com/paucablop/chemotools/blob/main/README.md

Common development workflow commands using the Task runner.

```bash
task install     # install all dependencies
task check       # run formatting, linting, typing, and tests
task test        # quick test run in the current environment
task test:matrix # run the nox compatibility matrix locally
task coverage    # run tests with coverage reporting
task build       # build the package for distribution
```

--------------------------------

### Build and fit a processing pipeline

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/user/user.md

Combines preprocessing and PLSRegression modeling into a single pipeline, then fits it to the training data.

```python
from chemotools.derivate import SavitzkyGolay
from sklearn.cross_decomposition import PLSRegression
from sklearn.pipeline import make_pipeline

# Define the pipeline
pipeline = make_pipeline(
    SavitzkyGolay(window_size=3, polynomial_order=1, derivate_order=1),
    PLSRegression(n_components=2)
)

# Fit the pipeline to the training data
pipeline.fit(X_train, y_train)

# Prediction on the training data
y_train_pred = pipeline.predict(X_train)
```

--------------------------------

### Load fermentation dataset

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/user/user.md

Imports and loads the fermentation training dataset containing mid infrared spectra and reference values.

```python
from chemotools.datasets import load_fermentation_train

X_train, y_train = load_fermentation_train()
```

--------------------------------

### Instantiate and Fit PLS Regression Model

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/learn/pls_regression.ipynb

Instantiates a PLSRegression model with a specified number of components and fits it to the preprocessed spectral data and corresponding glucose concentrations. Ensure data is preprocessed before fitting.

```python
# instanciate a PLSRegression object with 6 components
pls = PLSRegression(n_components=6, scale=False)

# fit the model to the data
pls.fit(spectra_preprocessed, hplc_np)
```

--------------------------------

### Persist and Load Model using Pickle

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/explore/persist.md

Demonstrates how to save a trained scikit-learn pipeline to a file using Python's pickle module and subsequently load it back. Be cautious when unpickling data from untrusted sources due to potential security risks.

```python
import pickle

# persist model
filename = 'model.pkl'

with open(filename, 'wb') as file:
    pickle.dump(pipeline, file)

# load model
with open(filename, 'rb') as file:
    pipeline = pickle.load(file)
```

--------------------------------

### Initialize PreprocessingInspector

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/explore/inspector_fundamentals.md

Configures the inspector with training and testing datasets to evaluate preprocessing pipelines.

```python
inspector = PreprocessingInspector(
    pca,
    X_train=X_train,
    y_train=y_train,
    X_test=X_test,
    y_test=y_test,
    x_axis=wn,
)
figures = inspector.inspect(dataset=['train', 'test'])
```

--------------------------------

### Support for multivariate targets

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/methods/orthogonal_signal_correction.md

Shows how to use the transformer with multiple target variables.

```pycon
>>> y_multi = np.column_stack([y, y**2])
>>> osc = OrthogonalSignalCorrection(n_components=2, method="fearn")
>>> osc.fit(X, y_multi)
OrthogonalSignalCorrection(method='fearn')
```

--------------------------------

### Run compatibility checks with nox

Source: https://github.com/paucablop/chemotools/blob/main/CONTRIBUTING.md

Execute tests across different Python versions and dependency configurations.

```bash
uv run nox --list
uv run nox -s tests-3.12
uv run nox -s tests-min-sklearn-3.10
uv run nox -s tests-min-sklearn-3.12
```

--------------------------------

### Importing smoothing classes

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/api/smooth.md

Import the available smoothing filters from the chemotools.smooth module.

```python
from chemotools.smooth import (
    MeanFilter,
    MedianFilter,
    ModifiedSincFilter,
    SavitzkyGolayFilter,
    WhittakerSmooth,
)
```

--------------------------------

### Perform manual preprocessing workflow

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/explore/pipelines.md

Applies multiple preprocessing steps individually to spectra before fitting a PLS regression model.

```python
from chemotools.feature_selection import RangeCut
from chemotools.baseline import LinearCorrection
from chemotools.derivative import SavitzkyGolay
from sklearn.cross_decomposition import PLSRegression
from sklearn.preprocessing import StandardScaler

# Range Cut
# Define the Range Cut
range_cut = RangeCut(start=950, end=1550, wavenumbers=wavenumbers)

# Fit and apply Range Cut
spectra_cut = range_cut.fit_transform(spectra)

# Linear Correction
# Define the Linear Correction
linear_correction = LinearCorrection()

# Fit and apply Linear Correction
spectra_corrected = linear_correction.fit_transform(spectra_cut)

# Savitzky-Golay
# Define the Savitzky-Golay
savitzky_golay = SavitzkyGolay(window_size=21, polynomial_order=2, derivate_order=1)

# Fit and apply Savitzky-Golay
spectra_derivate = savitzky_golay.fit_transform(spectra_corrected)

# Mean Centering (Standard Scaler)
# Define the Standard Scaler
standard_scaler = StandardScaler(with_mean=True, with_std=False)

# Fit and apply Standard Scaler
spectra_centered = standard_scaler.fit_transform(spectra_derivate)

# PLS regression
# Define the PLS regression
pls = PLSRegression(n_components=2, scale=False)

# Fit the model
pls.fit(spectra_centered, reference)

# Apply model to make predictions
prediction = pls.predict(spectra_centered)
```

--------------------------------

### Create Preprocessing Pipeline with Chemotools and Scikit-learn

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/learn/pls_classification.ipynb

Build a preprocessing pipeline using `make_pipeline` from `sklearn.pipeline`. This pipeline includes Standard Normal Variate, Savitzky-Golay derivative, Range Cut, and StandardScaler for spectral data.

```python
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

from chemotools.derivative import SavitzkyGolay
from chemotools.feature_selection import RangeCut
from chemotools.scatter import StandardNormalVariate

pipeline = make_pipeline(
    StandardNormalVariate(),
    SavitzkyGolay(window_size=21, polynomial_order=1),
    RangeCut(start=10, end=1350),
    StandardScaler(with_std=False),
)

preprocessed_spectra = pipeline.fit_transform(spectra)
```

--------------------------------

### Visualize PCA Loadings

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/explore/plotting_fundamentals.md

Plots loadings to identify which spectral features contribute most to the model.

```python
from chemotools.plotting import LoadingsPlot

loadings = pca.components_.T

# Plot loadings for the first component
plot = LoadingsPlot(loadings, feature_names=wavenumbers, components=0)
fig = plot.show(title="PC1 Loadings", ylabel="Loading Coefficient")
```

--------------------------------

### Explore Dataset Size

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/learn/pls_classification.ipynb

Print the number of samples and features in the loaded spectra dataset.

```python
print(f"The spectra dataset has {spectra.shape[0]} samples")
print(f"The spectra dataset has {spectra.shape[1]} features")
```

--------------------------------

### Apply preprocessing to spectra

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/learn/pls_regression.ipynb

Executes the fitted pipeline to transform raw spectral data.

```python
spectra_preprocessed = preprocessing.fit_transform(spectra_np)
```

--------------------------------

### Visualize Spectra with SpectraPlot

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/explore/plotting_fundamentals.md

Create and display spectral plots using various configuration options for titles, axes, and coloring.

```python
# Create plot object
plot = SpectraPlot(x=wavenumbers, y=X)

# Display it
fig = plot.show(title="All Spectra", ylabel="Absorbance")
```

```python
# Display it
fig = plot.show(title="All Spectra", ylabel="Absorbance", xlim=(900, 1500))
```

```python
# Create plot object
plot = SpectraPlot(x=wavenumbers, y=X, color_by=y)

# Display it
fig = plot.show(title="All Spectra", ylabel="Absorbance", xlim=(900, 1500))
```

```python
# Create plot object
plot = SpectraPlot(x=wavenumbers, y=X, color_by=measuring_date, color_mode="categorical")

# Display it
fig = plot.show(title="All Spectra", ylabel="Absorbance", xlim=(900, 1500))
```

--------------------------------

### Examine spectral data rows

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/learn/pls_regression.ipynb

Display the first 5 rows of the spectral data to understand the structure of samples and wavenumbers.

```python
spectra.head()
```

--------------------------------

### Load training data

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/learn/pls_regression.ipynb

Import and load the fermentation training dataset using the chemotools library.

```python
from chemotools.datasets import load_fermentation_train

spectra, hplc = load_fermentation_train()
```

--------------------------------

### Fit and Transform Spectra with Pipeline (Pandas Output)

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/explore/dataframes.md

Fit the configured pipeline to the spectral data and transform it. The result will be a pandas DataFrame.

```python
# Fit the pipeline and transform the spectra
output = pipeline.fit_transform(spectra)
```

--------------------------------

### Import Baseline Correction Classes

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/api/baseline.md

Import various baseline correction algorithms from the chemotools.baseline module. Ensure these classes are available in your environment.

```python
from chemotools.baseline import (
    AirPls,
    ArPls,
    AsLs,
    ConstantBaselineCorrection,
    CubicSplineCorrection,
    LinearCorrection,
    NonNegative,
    PolynomialCorrection,
    SubtractReference,
)
```

--------------------------------

### Perform Kennard-Stone Splitting with Chemotools

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/explore/astartes.md

Demonstrates splitting a dataset using the Kennard-Stone algorithm via Astartes before passing the data into a scikit-learn pipeline for preprocessing.

```python
from astartes import train_test_split
from chemotools.datasets import load_fermentation_train
from chemotools.baseline import AirPls
from sklearn.pipeline import make_pipeline
from sklearn.decomposition import PCA

# Load the data (returns pandas DataFrames)
X, y = load_fermentation_train()

# Split with Kennard-Stone instead of random
X_train, X_test, y_train, y_test = train_test_split(
    X.values,  # astartes expects numpy arrays
    y.values,
    train_size=0.75,
    sampler='kennard_stone'
)

# Preprocess and reduce dimensionality
pipeline = make_pipeline(
    AirPls(),
    PCA(n_components=3)
)

pipeline.fit(X_train)
scores = pipeline.transform(X_test)
```

--------------------------------

### Fit a PCA Model

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/explore/plotting_fundamentals.md

Initializes and fits a PCA model to the input data X.

```python
pca = PCA(n_components=3)
scores = pca.fit_transform(X)
```

--------------------------------

### Compute Norris-Williams Derivatives

Source: https://context7.com/paucablop/chemotools/llms.txt

Calculate Norris-Williams gap-segment derivatives for spectral data.

```python
from chemotools.derivative import NorrisWilliams
from chemotools.datasets import load_fermentation_train

X, _ = load_fermentation_train()

# Norris-Williams derivative
nw_deriv = NorrisWilliams()
X_deriv = nw_deriv.fit_transform(X)
```

--------------------------------

### Import scaling classes from chemotools.scale

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/api/scale.md

Import the available scaling classes for use in spectral data preprocessing.

```python
from chemotools.scale import (
    BandScaler,
    MinMaxScaler,
    NormScaler,
    ParetoScaler,
    PointScaler,
)
```

--------------------------------

### Compare Raw vs Preprocessed Spectra

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/explore/inspector_fundamentals.md

Visualizes the difference between raw input data and preprocessed data, available when using a sklearn Pipeline.

```python
# Compare raw vs preprocessed spectra
spectra_figures = inspector.inspect_spectra()
```

--------------------------------

### Load Fermentation Train Dataset

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/learn/datasets.md

Loads the fermentation training set containing 21 synthetic spectra and reference glucose concentrations.

```python
from chemotools.datasets import load_fermentation_train

X_train, y_train = load_fermentation_train()
```

```python
from chemotools.datasets import load_fermentation_train

X_train, y_train = load_fermentation_train(set_output="polars")
```

--------------------------------

### Load Fermentation Test Dataset

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/learn/datasets.md

Loads the fermentation test set containing over 1000 spectra and 35 reference glucose concentrations.

```python
from chemotools.datasets import load_fermentation_test

X_test, y_test = load_fermentation_test()
```

```python
from chemotools.datasets import load_fermentation_test

X_test, y_test = load_fermentation_test(set_output="polars")
```

--------------------------------

### Building Full Modeling Pipelines with Pipeline

Source: https://context7.com/paucablop/chemotools/llms.txt

Constructs a complete modeling pipeline including preprocessing steps (baseline, scatter correction, smoothing, derivative) and a PLS regression model. Supports cross-validation.

```python
from sklearn.pipeline import Pipeline
from sklearn.cross_decomposition import PLSRegression
from sklearn.model_selection import cross_val_score

from chemotools.baseline import AirPls
from chemotools.scatter import StandardNormalVariate
from chemotools.smooth import SavitzkyGolayFilter
from chemotools.derivative import SavitzkyGolay
from chemotools.datasets import load_fermentation_train

X, y = load_fermentation_train()
y_values = y.iloc[:, 0].values

# Full modeling pipeline with PLS
full_pipeline = Pipeline([
    ('baseline', AirPls(lam=1e5)),
    ('scatter', StandardNormalVariate()),
    ('smooth', SavitzkyGolayFilter(window_length=11, polyorder=2)),
    ('derivative', SavitzkyGolay(window_length=15, polyorder=2, deriv=1)),
    ('pls', PLSRegression(n_components=5))
])

# Fit and predict
full_pipeline.fit(X, y_values)
y_pred = full_pipeline.predict(X)

# Cross-validation
scores = cross_val_score(full_pipeline, X, y_values, cv=5, scoring='r2')
print(f"Cross-validation R2: {scores.mean():.3f} (+/- {scores.std()*2:.3f})")
```

--------------------------------

### Scale Spectra to [0, 1] Range

Source: https://context7.com/paucablop/chemotools/llms.txt

Utilize MinMaxScaler to scale spectra. Set use_min=True to scale to [0, 1] or use_min=False to scale to [0, max].

```python
from chemotools.scale import MinMaxScaler
from chemotools.datasets import load_fermentation_train

X, _ = load_fermentation_train()

# Scale each spectrum to [0, 1]
scaler = MinMaxScaler(use_min=True)
X_scaled = scaler.fit_transform(X)

# Scale by maximum only (no subtraction of minimum)
scaler_max = MinMaxScaler(use_min=False)
X_scaled_max = scaler_max.fit_transform(X)
```

--------------------------------

### Inspect Preprocessing Steps

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/explore/inspector_fundamentals.md

Initializes and uses PreprocessingInspector to visualize the effect of each preprocessing step on the PCA pipeline. Requires the trained pipeline and data.

```python
# Inspect the preprocessing steps of the PCA pipeline
inspector = PreprocessingInspector(pca, X_train=X, y_train=y, x_axis=wn)
figures = inspector.inspect()
```

--------------------------------

### Create a Pipeline with Pandas Output

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/explore/dataframes.md

Construct a scikit-learn pipeline including MultiplicativeScatterCorrection and StandardScaler, then configure the pipeline to output pandas DataFrames.

```python
import pandas as pd
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from chemotools.scatter import MultiplicativeScatterCorrection

# Make the pipeline
pipeline = make_pipeline(MultiplicativeScatterCorrection(), StandardScaler())

# Set the output to pandas
pipeline.set_output(transform="pandas")
```

--------------------------------

### Load Coffee Dataset

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/learn/pls_classification.ipynb

Load the coffee dataset using the `load_coffee()` function from the `chemotools.datasets` module. This function returns spectra and labels.

```python
from chemotools.datasets import load_coffee

spectra, labels = load_coffee()
```

--------------------------------

### Apply Savitzky-Golay preprocessing

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/user/user.md

Configures and applies a Savitzky-Golay filter to compute the first derivative of the sample spectra.

```python
from chemotools.derivate import SavitzkyGolay

# Configure the preprocessing step
sg = SavitzkyGolay(window_size=3, polynomial_order=1, derivate_order=1)

# Fit and transform the training data
X_train_preprocessed = sg.fit_transform(X_train)
```

--------------------------------

### Fit EPO with sample identifiers

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/methods/external_parameter_orthogonalization.md

Shows how to use sample_ids to estimate the nuisance subspace from within-sample differences.

```pycon
>>> sample_ids = np.array([0, 0, 1, 1, 2, 2])
>>> epo = ExternalParameterOrthogonalization(n_components=1)
>>> epo.fit(X, X_external=X_external, sample_ids=sample_ids)
ExternalParameterOrthogonalization(n_components=1)
```

--------------------------------

### Select Components for Visualization

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/explore/inspector_fundamentals.md

Specifies which components to display in scores and loadings plots using indices or tuples.

```python
# Plot LV2 vs LV3 for scores, and the first 2 components for loadings
inspector.inspect(
    components_scores=(1, 2),
    loadings_components=[0, 1]
)
```

--------------------------------

### Create a scikit-learn Preprocessing Pipeline

Source: https://github.com/paucablop/chemotools/blob/main/README.md

Integrate chemotools transformers into a standard scikit-learn pipeline for spectral data processing.

```python
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import make_pipeline

from chemotools.baseline import AirPls
from chemotools.scatter import MultiplicativeScatterCorrection

preprocessing = make_pipeline(
    AirPls(),
    MultiplicativeScatterCorrection(),
    StandardScaler(with_std=False),
)

spectra_transformed = preprocessing.fit_transform(spectra)
```

--------------------------------

### Split Data into Training and Testing Sets

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/learn/pls_classification.ipynb

Use train_test_split from scikit-learn to divide the preprocessed spectra and encoded labels into training and testing sets. Set test_size to 0.2 for 80% training data and use random_state for reproducibility.

```python
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
    preprocessed_spectra, labels_encoded, test_size=0.2, random_state=42
)
```

--------------------------------

### Load Data and Train PCA/PLS Models

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/explore/inspector_fundamentals.md

Loads fermentation data and trains PCA and PLS regression models using scikit-learn pipelines. Requires importing necessary modules from sklearn and chemotools.

```python
from sklearn.cross_decomposition import PLSRegression
from sklearn.decomposition import PCA
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

from chemotools.datasets import load_fermentation_train
from chemotools.derivative import SavitzkyGolay
from chemotools.feature_selection import RangeCut
from chemotools.inspector import PCAInspector, PLSRegressionInspector, PreprocessingInspector


# 1. Load Data
X, y = load_fermentation_train()
wn = X.columns

# 2. Fit the PCA Model
pca = make_pipeline(
    RangeCut(start=900, end=1400, wavenumbers=wn),
    SavitzkyGolay(window_size=21, polynomial_order=2, derivate_order=0),
    StandardScaler(with_std=False),
    PCA(n_components=3),
)
pca.fit(X)

# 3. Fit the PLS regression model
pls = make_pipeline(
    RangeCut(start=900, end=1400, wavenumbers=wn),
    SavitzkyGolay(window_size=21, polynomial_order=2, derivate_order=1),
    PLSRegression(n_components=3, scale=False),
)
pls.fit(X, y)
```

--------------------------------

### Import plotting classes

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/api/plotting.md

Import the available visualization classes from the chemotools.plotting module.

```python
from chemotools.plotting import (
    SpectraPlot,
    ScoresPlot,
    LoadingsPlot,
    DistancesPlot,
    ExplainedVariancePlot,
    FeatureSelectionPlot,
    PredictedVsActualPlot,
    YResidualsPlot,
    QQPlot,
    ResidualDistributionPlot,
)
```

--------------------------------

### Configure SavitzkyGolay Filter

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/_static/images/explore/pipelines/pipelines_pipeline_visualization.html

Configuration for the SavitzkyGolay filter, specifying the polynomial order and window size for smoothing.

```python
SavitzkyGolay(polynomial_order=2, window_size=21)
```

--------------------------------

### Set Color Mode

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/explore/inspector_fundamentals.md

Toggles between categorical and continuous color mapping for sample visualization.

```python
# Use categorical coloring (discrete colors for each unique value)
inspector.inspect(color_by='y', color_mode='categorical')

# Use continuous coloring (gradient based on values) - default
inspector.inspect(color_by='y', color_mode='continuous')
```

--------------------------------

### Scale Spectra by Band Integral

Source: https://context7.com/paucablop/chemotools/llms.txt

Apply BandScaler to scale spectra based on the integral of a specific spectral region.

```python
from chemotools.scale import BandScaler
from chemotools.datasets import load_fermentation_train

X, _ = load_fermentation_train()

# Scale by band integral
band_scaler = BandScaler()
X_scaled = band_scaler.fit_transform(X)
```

--------------------------------

### Configure PLSRegression

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/_static/images/explore/pipelines/pipelines_pipeline_visualization.html

Configuration for PLSRegression, with `scale` set to False to disable scaling of the variables.

```python
PLSRegression(scale=False)
```

--------------------------------

### Create Composite Figures

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/explore/plotting_fundamentals.md

Demonstrates how to use the render method to place multiple plots onto existing matplotlib axes.

```python
import matplotlib.pyplot as plt

# Create a figure with 2 subplots
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 5))

# Plot 1: All spectra
SpectraPlot(x=wavenumbers, y=X, color='lightgray').render(ax1)
ax1.set_title("Raw Spectra")

# Plot 2: Mean spectrum
SpectraPlot(x=wavenumbers, y=X.mean(axis=0), color='black').render(ax2)
ax2.set_title("Mean Spectrum")

plt.tight_layout()
plt.show()
```

--------------------------------

### Persist and Load Model using Joblib

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/explore/persist.md

Shows how to efficiently save and load a scikit-learn pipeline using the joblib library, which is optimized for NumPy arrays. This method is recommended for larger models.

```python
from joblib import dump, load

# persist model
filename = 'model.joblib'

with open(filename, 'wb') as file:
    dump(pipeline, file)

# load model
with open(filename, 'rb') as file:
    pipeline = load(file)
```

--------------------------------

### Import chemotools utilities

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/api/utils.md

Import statement for accessing internal utility functions. Note that these are intended for internal use.

```python
from chemotools.utils import (
    # Linear algebra utilities
)
```

--------------------------------

### Import Augmentation Classes from chemotools

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/api/augmentation.md

Import necessary data augmentation classes from the chemotools.augmentation module. These classes are used to apply various transformations to spectral data.

```python
from chemotools.augmentation import (
    AddNoise,
    BaselineShift,
    FractionalShift,
    GaussianBroadening,
    IndexShift,
    SpectrumScale,
)
```

--------------------------------

### Style Pipeline Visualization

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/explore/pipelines.md

CSS definitions for rendering scikit-learn pipeline diagrams, including support for light and dark themes.

```css
#sk-container-id-4 {
  /* Definition of color scheme common for light and dark mode */
  --sklearn-color-text: #000;
  --sklearn-color-text-muted: #666;
  --sklearn-color-line: gray;
  /* Definition of color scheme for unfitted estimators */
  --sklearn-color-unfitted-level-0: #fff5e6;
  --sklearn-color-unfitted-level-1: #f6e4d2;
  --sklearn-color-unfitted-level-2: #ffe0b3;
  --sklearn-color-unfitted-level-3: chocolate;
  /* Definition of color scheme for fitted estimators */
  --sklearn-color-fitted-level-0: #f0f8ff;
  --sklearn-color-fitted-level-1: #d4ebff;
  --sklearn-color-fitted-level-2: #b3dbfd;
  --sklearn-color-fitted-level-3: cornflowerblue;

  /* Specific color for light theme */
  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, rgb(255, 255, 255))));
  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));
  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));
  --sklearn-color-icon: #696969;

  @media (prefers-color-scheme: dark) {
    /* Redefinition of color scheme for dark theme */
    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));
    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));
    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));
    --sklearn-color-icon: #878787;
  }
}

#sk-container-id-4 {
  color: var(--sklearn-color-text);
}

#sk-container-id-4 pre {
  padding: 0;
}

#sk-container-id-4 input.sk-hidden--visually {
  border: 0;
  clip: rect(1px 1px 1px 1px);
  clip: rect(1px, 1px, 1px, 1px);
  height: 1px;
  margin: -1px;
  overflow: hidden;
  padding: 0;
  position: absolute;
  width: 1px;
}

#sk-container-id-4 div.sk-dashed-wrapped {
  border: 1px dashed var(--sklearn-color-line);
  margin: 0 0.4em 0.5em 0.4em;
  box-sizing: border-box;
  padding-bottom: 0.4em;
  background-color: var(--sklearn-color-background);
}

#sk-container-id-4 div.sk-container {
  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`
     but bootstrap.min.css set `[hidden] { display: none !important; }`
     so we also need the `!important` here to be able to override the
     default hidden behavior on the sphinx rendered scikit-learn.org.
     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */
  display: inline-block !important;
  position: relative;
}

#sk-container-id-4 div.sk-text-repr-fallback {
  display: none;
}

div.sk-parallel-item,
div.sk-serial,
div.sk-item {
  /* draw centered vertical line to link estimators */
  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));
  background-size: 2px 100%;
  background-repeat: no-repeat;
  background-position: center center;
}

/* Parallel-specific style estimator block */

#sk-container-id-4 div.sk-parallel-item::after {
  content: "";
  width: 100%;
  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);
  flex-grow: 1;
}

#sk-container-id-4 div.sk-parallel {
  display: flex;
  align-items: stretch;
  justify-content: center;
  background-color: var(--sklearn-color-background);
  position: relative;
}

#sk-container-id-4 div.sk-parallel-item {
  display: flex;
  flex-direction: column;
}

#sk-container-id-4 div.sk-parallel-item:first-child::after {
  align-self: flex-end;
  width: 50%;
}

#sk-container-id-4 div.sk-parallel-item:last-child::after {
  align-self: flex-start;
  width: 50%;
}

#sk-container-id-4 div.sk-parallel-item:only-child::after {
  width: 0;
}

/* Serial-specific style estimator block */

#sk-container-id-4 div.sk-serial {
  display: flex;
  flex-direction: column;
  align-items: center;
  background-color: var(--sklearn-color-background);
  padding-right: 1em;
  padding-left: 1em;
}


/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is
clickable and can be expanded/collapsed.
- Pipeline and ColumnTransformer use this feature and define the default style
- Estimators will overwrite some part of the style using the `sk-estimator` class
*/

/* Pipeline and ColumnTransformer style (default) */

#sk-container-id-4 div.sk-toggleable {
  /* Default theme specific background. It is overwritten whether we have a
  specific estimator or a Pipeline/ColumnTransformer */
  background-color: var(--sklearn-color-background);
}

/* Toggleable label */
#sk-container-id-4 label.sk-toggleable__label {
  cursor: pointer;
  display: flex;
  width: 100%;
```

--------------------------------

### Apply GaussianBroadening

Source: https://context7.com/paucablop/chemotools/llms.txt

Applies Gaussian broadening to spectral peaks.

```python
from chemotools.augmentation import GaussianBroadening
from chemotools.datasets import load_fermentation_train

X, _ = load_fermentation_train()

# Apply Gaussian broadening
broadening = GaussianBroadening()
X_broadened = broadening.fit_transform(X)
```

--------------------------------

### Plotting Coffee Spectra by Origin

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/learn/pls_classification.ipynb

Plots coffee spectra using a predefined color dictionary for each origin. Ensure 'spectra' and 'labels' DataFrames are available, and 'plt' is imported.

```python
color_dict = {
    "Brasil": "Violet",
    "Ethiopia": "Salmon",
    "Vietnam": "LightBlue",
}

fig, ax = plt.subplots(figsize=(10, 3))
for i, row in enumerate(spectra.iterrows()):
    ax.plot(row[1].values, color=color_dict[labels.iloc[i].values[0]])

ax.set_xlabel("Wavelength (a.u.)")
ax.set_ylabel("Absorbance (a.u.)")
ax.set_title("Coffee spectra")
```

--------------------------------

### Initialize OrthogonalSignalCorrection

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/methods/orthogonal_signal_correction.md

Constructor for the OSC transformer to define the number of components and memory handling.

```APIDOC
## Constructor: OrthogonalSignalCorrection

### Description
Initialize the Orthogonal Signal Correction (OSC) transformer.

### Parameters
- **n_components** (int) - Optional (default=2) - Number of orthogonal components to remove. Must be a positive integer.
- **copy** (bool) - Optional (default=True) - Whether to copy X and Y in fit before applying centering.
```

--------------------------------

### Inspect Multi-Output PLS

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/explore/inspector_fundamentals.md

Use the target_index parameter to specify which target variable to inspect in multi-output models.

```python
# Inspect the second target variable (index 1)
inspector.inspect(target_index=1)
```

--------------------------------

### Load Coffee Dataset with Polars

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/learn/pls_classification.ipynb

Load the coffee dataset and set the output to use `polars.DataFrame` by passing `set_output="polars"` to the `load_coffee()` function.

```python
from chemotools.datasets import load_coffee

spectra, labels = load_coffee(set_output="polars")
```

--------------------------------

### Print Best Parameters and Score

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/learn/pls_regression.ipynb

Prints the optimal hyperparameters and the corresponding best score obtained from a grid search. This is useful for understanding the results of hyperparameter tuning.

```python
print("Best parameters: ", grid_search.best_params_)
print("Best score: ", np.abs(grid_search.best_score_))
```

--------------------------------

### Plot Explained Variance

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/explore/plotting_fundamentals.md

Visualizes the explained variance ratio to determine the optimal number of components.

```python
from chemotools.plotting import ExplainedVariancePlot

# Plot explained variance ratio
plot = ExplainedVariancePlot(pca.explained_variance_ratio_)
fig = plot.show(title="Explained Variance")
```

--------------------------------

### Train PLS-DA Model

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/learn/pls_classification.ipynb

Initialize and train a PLS-DA model using PLSRegression from scikit-learn. Set n_components to 2 and scale to False as data is already scaled. Fit the model to the training data.

```python
from sklearn.cross_decomposition import PLSRegression

# Make a PLSRegression object
pls = PLSRegression(n_components=2, scale=False)

# Fit the PLSRegression object to the training data
pls.fit(X_train, y_train)
```

--------------------------------

### Import outlier detection classes

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/api/outliers.md

Import the available outlier detection classes from the chemotools.outliers module.

```python
from chemotools.outliers import (
    DModX,
    HotellingT2,
    Leverage,
    QResiduals,
    StudentizedResiduals,
)
```

--------------------------------

### Import Scikit-Learn Modules

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/explore/sklearn.md

Required imports for using StandardScaler and PLSRegression in a chemometric workflow.

```python
from sklearn.preprocessing import StandardScaler
from sklearn.cross_decomposition import PLSRegression
```

--------------------------------

### Import feature selection classes

Source: https://github.com/paucablop/chemotools/blob/main/docs/source/api/feature_selection.md

Import the available feature selection selectors from the chemotools.feature_selection module.

```python
from chemotools.feature_selection import (
    IndexSelector,
    RangeCut,
    SRSelector,
    VIPSelector,
)
```