### Install osfclient using pip

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/GLHMM_example.ipynb

This snippet checks if the 'osfclient' library is installed and installs it using pip if it's not found. It's a prerequisite for fetching data from the OSF.

```python
import sys
import pip

def install(package):
    pip.main(['install', package])

try:
    import osfclient
except ImportError:
    print('osfclient is not installed, installing it now')
    install('osfclient')
```

--------------------------------

### Install osfclient if not present

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/Preprocessing.ipynb

Checks if the 'osfclient' package is installed and installs it using pip if it's not found. This is used for downloading example data from OSF for the tutorial.

```python
# checks if osfclient is installed and otherwise installs it using pip install
```

--------------------------------

### Example: Train Gaussian HMM (DO NOT RUN)

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/Prediction_tutorial.ipynb

This code block provides an example of how to initialize and train a Gaussian HMM. It is commented out and marked as 'DO NOT RUN' as it's for illustrative purposes and might require specific data or environment setup. It sets a random seed for reproducibility and defines an HMM with specific parameters.

```python
# DO NOT RUN
# %%capture
# np.random.seed(123)
# hmm = glhmm.glhmm(model_beta='no', K=6, covtype='full')
# hmm.train(X=None, Y=data_preproc, indices=T_t)
```

--------------------------------

### Download Example Data using osfclient

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/Testing_across_subjects.ipynb

Sets up a data directory and downloads necessary example data files (time series, HMM outputs, confounds, behavioral data) from the Open Science Framework (OSF) using the osfclient package. Skips download if files already exist.

```python
# Set up data directory
data_dir = Path.cwd() / "files" / "data_statistical_testing"

if not data_dir.exists():
    print(f"Creating {data_dir}...")
    data_dir.mkdir(parents=True, exist_ok=True)
else:
    print(f"Data directory {data_dir} already exists.")

# Files to download
files = [
    "tc_forpred.csv",
    "T_forpred.csv",
    "Y_forpred.csv",
    "confounds_forpred.csv",
    "hmm_pred.pkl"
]


```

--------------------------------

### Install osfclient using pip

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/HMM-TDE_vs_HMM-MAR_example.ipynb

This function installs the 'osfclient' package using pip if it's not already present. It's a utility for fetching data from OSF. It requires the pip module to be available.

```python
def install(package):
    pip.main(['install', package])

try:
    import osfclient
except ImportError:
    print('osfclient is not installed, installing it now')
    install('osfclient')
```

--------------------------------

### Download and Load Pre-trained HMM

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/HMM-TDE_vs_HMM-MAR_example.ipynb

This snippet demonstrates how to download a pre-trained HMM using the 'osf' command-line tool and then load it into the Python environment using pickle. It assumes the 'osf' tool is installed and configured.

```python
! osf -p 8qcyj fetch MEG_data/hmm_tde.pkl ./example_data/hmm_tde.pkl

```

```python
# load into the notebook
import pickle
with open('./example_data/hmm_tde.pkl', "rb") as f:
    hmm_dict = pickle.load(f)

TDE_hmm = hmm_dict['hmm']
stc_tde = hmm_dict['stc']
vpath_tde = hmm_dict['vpath']

```

--------------------------------

### Install GLHMM Package using Pip

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/HMM-TDE_vs_HMM-MAR_example.ipynb

This command installs the GLHMM package directly from its GitHub repository. It requires pip to be installed and an active internet connection. The output shows the cloning process, dependency resolution, and confirmation of successful installation.

```python
!pip install git+https://github.com/vidaurre/glhmm
```

--------------------------------

### Download Data using OSF CLI

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/HMM-TDE_vs_HMM-MAR_example.ipynb

Downloads MEG data from an OSF project using the OSF command-line interface. Requires the OSF CLI to be installed and configured. Saves the data to the 'example_data' folder.

```bash
! osf -p 8qcyj fetch MEG_data/data_MEG_MAR.pkl ./example_data/data_MEG_MAR.pkl
```

--------------------------------

### Install GLHMM Development Version (Bash)

Source: https://github.com/vidaurre/glhmm/blob/main/README.md

Installs the latest development version of the GLHMM toolbox directly from its GitHub repository. This command uses pip to fetch and install the package.

```bash
pip install git+https://github.com/vidaurre/glhmm
```

--------------------------------

### Import GLHMM Modules

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/GLHMM_example.ipynb

Imports necessary libraries for the GLHMM example, including numpy, pandas, matplotlib, seaborn, and specific modules from the glhmm package.

```python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sb
from glhmm import glhmm, preproc, utils, graphics
```

--------------------------------

### Install GLHMM Stable Release (Bash)

Source: https://github.com/vidaurre/glhmm/blob/main/README.md

Installs the latest stable release of the GLHMM toolbox from the Python Package Index (PyPI). This command uses pip to download and install the package.

```bash
pip install glhmm
```

--------------------------------

### Example: Save Gaussian HMM (DO NOT RUN)

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/Prediction_tutorial.ipynb

This commented-out code block demonstrates how to save a trained HMM model to a pickle file. It uses the 'pickle' library for serialization. This is typically done after training a model to persist it for later use.

```python
# DO NOT RUN
# import pickle
# with open('./data_prediction/hmm_pred.pkl', 'wb') as outp:

```

--------------------------------

### Fetch GLHMM data using OSF client

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/GLHMM_example.ipynb

These commands use the OSF client to fetch necessary data files (data.csv, dataX.csv, T.csv) for the GLHMM model. The data is downloaded to the './example_data/' directory. Note that these commands will error if the files already exist.

```bash
! osf -p 8qcyj fetch GLHMM/data.csv ./example_data/data.csv
! osf -p 8qcyj fetch GLHMM/dataX.csv ./example_data/dataX.csv
! osf -p 8qcyj fetch GLHMM/T.csv ./example_data/T.csv
```

--------------------------------

### Set Up Data Directory and Download Files

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/Testing_across_sessions_within_subject.ipynb

Creates a directory for storing data if it doesn't exist and defines a list of files to be downloaded from Open Science Framework (OSF). This setup is crucial for loading the synthetic data required for statistical testing.

```python
# Set up data directory
data_dir = Path.cwd() / "files" / "data_statistical_testing"

if not data_dir.exists():
    print(f"Creating {data_dir}...")
    data_dir.mkdir(parents=True, exist_ok=True)
else:
    print(f"Data directory {data_dir} already exists.")

# Files to download
files = [
    "Gamma_sessions.npy",
    "R_sessions.npy",
    "idx_sessions.npy",
]

```

--------------------------------

### Install GLHMM and osfclient in Google Colab

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/Testing_across_visits.ipynb

This code snippet checks if the environment is Google Colab and installs the GLHMM package from GitHub if it's not already present. It also installs the 'osfclient' package if it's missing, which is used for downloading data from Open Science Framework. Finally, it imports necessary libraries.

```python
# Install packages if needed
try:
    import google.colab
    IN_COLAB = True
except ImportError:
    IN_COLAB = False

if IN_COLAB:
    print("Running in Google Colab. Installing GLHMM...")
    !pip install git+https://github.com/vidaurre/glhmm

# Install osfclient if missing
try:
    import osfclient
except ImportError:
    print("Installing osfclient...")
    !pip install osfclient

# Now import everything we need
import numpy as np
from pathlib import Path
from glhmm import glhmm, preproc, io, graphics, statistics
```

--------------------------------

### Print GLHMM Hyperparameters

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/GLHMM_example.ipynb

Prints the hyperparameters of the initialized GLHMM object to verify the model configuration. This is useful for confirming settings like the number of states (`K`), covariance type (`covtype`), and interaction modeling (`model_beta`).

```python
print(brainphys_glhmm.hyperparameters)
```

--------------------------------

### Download Trained Model using OSF CLI

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/HMM-TDE_vs_HMM-MAR_example.ipynb

Downloads a pre-trained HMM model file from an OSF repository using the OSF command-line interface. This is a convenient way to obtain model files for analysis. Requires the OSF CLI to be installed and configured.

```shell
# or download the trained model 
! osf -p 8qcyj fetch MEG_data/hmm_mar.pkl ./example_data/hmm_mar.pkl
```

--------------------------------

### Get State Activation Onsets (Python)

Source: https://github.com/vidaurre/glhmm/blob/main/docs/_build/html/_modules/glhmm/utils.html

Calculates the time points when each state activates for each trial/session. It uses the `get_visits` function to determine state activations and filters them based on a threshold.

```python
def get_state_onsets(vpath, indices, get_visits):
    """Calculates the time points when each state activates for each trial/session.

    Parameters:
    ----------
    vpath : array-like
        The state path for all time points.
    indices : array-like of shape (n_sessions, 2)
        The start and end indices of each trial/session in the input data.
    get_visits : function
        A function that computes state visits and filters them by a threshold.

    Returns:
    --------
    onsets : list of lists of ints
        A list of the time points when each state activates for each trial/session.
    """

    N = indices.shape[0]
    K = vpath.shape[1]
    onsets = []
    for j in range(N):
        onsets_j = []
        ind = np.arange(indices[j,0],indices[j,1]).astype(int)
        for k in range(K):
            _, onsets_k = get_visits(vpath[ind, :], k, threshold=0)
            onsets_j.append(onsets_k)
        onsets.append(onsets_j)
    return onsets
```

--------------------------------

### Install GLHMM and osfclient, Import Libraries

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/Testing_across_subjects.ipynb

Installs the GLHMM package and osfclient if not already present, particularly for Google Colab environments. It then imports necessary libraries including numpy, pandas, matplotlib, and components from the glhmm package. Sets a random seed for reproducibility.

```python
# Install packages if needed
try:
    import google.colab
    IN_COLAB = True
except ImportError:
    IN_COLAB = False

if IN_COLAB:
    print("Running in Google Colab. Installing GLHMM...")
    !pip install git+https://github.com/vidaurre/glhmm

# Install osfclient if missing
try:
    import osfclient
except ImportError:
    print("Installing osfclient...")
    !pip install osfclient

# Now import everything we need
import numpy as np
from pathlib import Path
import pandas as pd
import matplotlib.pyplot as plt
from glhmm import glhmm, preproc, io, graphics, statistics
np.random.seed(0)  # For reproducibility

```

--------------------------------

### Initialize GLHMM Model

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/GLHMM_example.ipynb

Initializes a GLHMM object with specified parameters for modeling time-varying interactions between two sets of variables. It requires setting `model_beta` to 'state' for state-dependent regression coefficients and `K` for the number of states. Dependencies include the `glhmm` library.

```python
brainphys_glhmm = glhmm.glhmm(model_beta='state', K=4, covtype='full', preproclogX=preproclogX, preproclogY=preproclogY)
```

--------------------------------

### General data preprocessing in GLHMM

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/Preprocessing.ipynb

This example shows the general syntax for the `preprocess_data` function in the GLHMM toolbox. It takes raw data, indices, and a list of preprocessing steps as input, returning preprocessed data, indices, and a log of operations. The log is essential for relating HMM parameters back to the original data.

```python
from glhmm import glhmm

# Assuming data, indices, and preprocessing_steps are defined
# data_preprocessed, indices_preprocessed, log = preprocess_data(data, indices, preprocessing_steps)
# hmm = glhmm.glhmm(K=6, preproclogY=log)
```

--------------------------------

### Stochastic Initialization of GLHMM (Python)

Source: https://github.com/vidaurre/glhmm/blob/main/docs/_build/html/_modules/glhmm/glhmm.html

Performs the initial setup for stochastic training of the Gaussian Linear Hidden Markov Model (GLHMM). This involves loading data, initializing Gamma, priors, dynamics, observation distributions, and updating priors. It's a comprehensive initialization step for the training process.

```python
def __init_stochastic(self, files, options):
        N = len(files)
        I = np.random.choice(np.arange(N), size=options["initNbatch"], replace=False)
        X, Y, indices, _ = io.load_files(files, I)
        Gamma = self.__init_Gamma(X, Y, indices, options)
        self.__init_priors(files=files)
        self.__init_dynamics(Gamma, indices=indices)
        self.__init_obsdist_stochastic(files, I, Gamma)
        self.__update_priors()
```

--------------------------------

### Analyze and Plot Transition Probabilities in Python

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/GLHMM_example.ipynb

This code calculates and visualizes the transition probability matrices for a GLMM. It includes plots for both the full transition probabilities and those excluding self-transitions, normalized for clarity. Dependencies include numpy and matplotlib.

```python
TP = brainphys_glhmm.P.copy() # the transition probability matrix

# Plot Transition Probabilities
plt.figure(figsize=(7, 4))

# Plot 1: Full Transition Probabilities
plt.subplot(1, 2, 1)
plt.imshow(TP, cmap=cmap, interpolation='nearest')  # Improved color mapping
plt.title('Transition Probabilities')
plt.xlabel('To State')
plt.ylabel('From State')
plt.colorbar(fraction=0.046, pad=0.04)

# Plot 2: Transition Probabilities without Self-Transitions
TP_noself = TP - np.diag(np.diag(TP))  # Remove self-transitions
TP_noself2 = TP_noself / TP_noself.sum(axis=1, keepdims=True)  # Normalize probabilities
plt.subplot(1, 2, 2)
plt.imshow(TP_noself2, cmap=cmap, interpolation='nearest')  # Improved color mapping
plt.title('Transition Probabilities\nwithout Self-Transitions')
plt.xlabel('To State')
plt.ylabel('From State')
plt.colorbar(fraction=0.046, pad=0.04)

plt.tight_layout()  # Adjust layout for better spacing
plt.show()
```

--------------------------------

### Retrieve and Plot State Means and Covariances in Python

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/GLHMM_example.ipynb

This snippet demonstrates how to retrieve the mean activation levels and covariance matrices for each state in a GLMM. It then visualizes the state means using a heatmap. Dependencies include numpy and matplotlib.

```python
state_means = np.zeros(shape=(q, K))
for k in range(K):
    state_means[:,k] = brainphys_glhmm.get_mean(k) # the state means in the shape (no. features, no. states)
state_FC = np.zeros(shape=(q, q, K))
for k in range(K):
    state_FC[:,:,k] = brainphys_glhmm.get_covariance_matrix(k=k) # the state covariance matrices in the shape (no. features, no. features, no. states)
```

```python
plt.imshow(state_means,cmap=cmap, interpolation="none")
plt.colorbar(label='Activation Level') # Label for color bar
plt.title("State mean activation")
plt.xticks(np.arange(K), np.arange(1,K+1))
plt.gca().set_xlabel('State')
plt.gca().set_ylabel('Brain region')
plt.tight_layout()  # Adjust layout for better spacing
plt.show()
```

```python
for k in range(K):
    plt.subplot(2,2,k+1)
    plt.imshow(state_FC[:,:,k], cmap=cmap, interpolation="none")
    plt.xlabel('Brain region')
    plt.ylabel('Brain region')
    plt.colorbar()
    plt.title("State covariance\nstate #%s" % (k+1))
plt.subplots_adjust(hspace=0.7, wspace=0.8)
plt.show()
```

--------------------------------

### Decode and Plot Viterbi Path in Python

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/GLHMM_example.ipynb

This snippet shows how to decode the most likely sequence of states (Viterbi path) given the observed data and then visualize this path. It also demonstrates plotting a segment of the Viterbi path for a specific subject. Dependencies include the 'graphics' module and the GLMM model object.

```python
vpath = brainphys_glhmm.decode(X=phys_data, Y=brain_data, indices=T_t, viterbi=True)
```

```python
graphics.plot_vpath(vpath, title="Viterbi path")
```

```python
num_subject = 0
graphics.plot_vpath(vpath[T_t[num_subject,0]:T_t[num_subject,1],:], title="Viterbi path")
```

--------------------------------

### Get State Onsets (Python)

Source: https://github.com/vidaurre/glhmm/blob/main/docs/_build/html/_modules/glhmm/utils.html

Calculates the time points when each state activates within sessions, given a Viterbi path and session indices. This function is intended to identify the start times of state occurrences, potentially filtering short durations.

```python
import numpy as np

def get_state_onsets(vpath, indices, threshold=0):
    """Calculates the state onsets, i.e., the time points when each state activates.

    Parameters:
    ---------------
    vpath : array-like of shape (n_samples, n_states)
        The viterbi path represents the most likely state sequence.
    indices : array-like of shape (n_sessions, 2)
        The start and end indices of each trial/session in the input data.
    threshold : int, optional, default=0
        A threshold value used to exclude visits with a duration below this value.

    """
    # This function's implementation is not provided in the input text.
```

--------------------------------

### Fetch data using osf client

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/HMM-TDE_vs_HMM-MAR_example.ipynb

This command-line instruction uses the 'osf' tool to fetch a data file named 'data_MEG_TDE.pkl' from the OSF project '8qcyj' and saves it locally as './example_data/data_MEG_TDE.pkl'. This is useful for downloading datasets for analysis.

```bash
! osf -p 8qcyj fetch MEG_data/data_MEG_TDE.pkl ./example_data/data_MEG_TDE.pkl
```

--------------------------------

### Train GLHMM Model

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/GLHMM_example.ipynb

Trains the GLHMM model using both primary (`Y`, e.g., brain data) and secondary (`X`, e.g., physiological data) time-series. It requires session start and end indices (`indices`) to be provided. The `%%capture` magic command suppresses training progress output.

```python
%%capture
np.random.seed(123)
brainphys_glhmm.train(X=phys_data, Y=brain_data, indices=T_t)
```

--------------------------------

### Get GLHMM State Betas

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/GLHMM_example.ipynb

Retrieves the beta values (parameters describing the interaction) for each state from a trained GLHMM model. This function is essential for understanding how the secondary time-series (`X`) influences the primary time-series (`Y`) within each hidden state. It returns a 3D array where dimensions represent physiological measures, brain regions, and states.

```python
K = brainphys_glhmm.hyperparameters["K"] # the number of states
q = brain_data.shape[1] # the number of parcels/channels
state_betas = np.zeros(shape=(2,q,K))
state_betas = brainphys_glhmm.get_betas()
```

--------------------------------

### Install GLHMM and osfclient Packages

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/Testing_across_sessions_within_subject.ipynb

Installs the GLHMM package from GitHub and the osfclient package using pip. It checks if the code is running in Google Colab to conditionally install GLHMM. Imports necessary libraries like numpy, graphics, and statistics from glhmm.

```python
# Install required packages
import sys
import subprocess
from pathlib import Path

def install(package):
    subprocess.check_call([sys.executable, "-m", "pip", "install", package])

# Install GLHMM if using Google Colab
try:
    import google.colab
    IN_COLAB = True
except ImportError:
    IN_COLAB = False

if IN_COLAB:
    print("Detected Google Colab. Installing GLHMM...")
    install("git+https://github.com/vidaurre/glhmm")

# Install osfclient if missing
try:
    import osfclient
except ImportError:
    print("Installing osfclient...")
    install("osfclient")

# Import libraries
import numpy as np
from glhmm import graphics, statistics

```

--------------------------------

### Set up data directory and download files

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/Testing_across_visits.ipynb

This Python code sets up a local directory for storing statistical testing data. It checks if the directory exists and creates it if necessary. It then defines a list of files to be downloaded using the 'osfclient' package from Open Science Framework (OSF).

```python
# Set up data directory
data_dir = Path.cwd() / "files" / "data_statistical_testing"

if not data_dir.exists():
    print(f"Creating {data_dir}...")
    data_dir.mkdir(parents=True, exist_ok=True)
else:
    print(f"Data directory {data_dir} already exists.")

# Files to download
files = [
    "vpath.npy",
    "sig_data.npy",
]
```

--------------------------------

### Set Up Data Directory and Download Files

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/Testing_across_trials_within_session.ipynb

Sets up a local directory for storing statistical testing data and downloads necessary .npy files (Gamma_trials.npy, R_trials.npy, idx_trials_session.npy) from the Open Science Framework (OSF) using the osfclient package. It checks if the directory and files already exist to avoid redundant downloads.

```python
# Set up data directory
data_dir = Path.cwd() / "files" / "data_statistical_testing"

if not data_dir.exists():
    print(f"Creating {data_dir}...")
    data_dir.mkdir(parents=True, exist_ok=True)
else:
    print(f"Data directory {data_dir} already exists.")

# Files to download
files = [
    "Gamma_trials.npy",
    "R_trials.npy",
    "idx_trials_session.npy",
]

```

--------------------------------

### Load Tapping Onset Data

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/HMM-TDE_vs_HMM-MAR_example.ipynb

Loads the tapping onset data, previously downloaded from OSF, into the notebook using the pickle library. This data represents the timing of behavioral events (finger taps).

```python
# load the onset data into the notebook
with open('./example_data/onset_tap_MAR.pkl', "rb") as f:
    onset = pickle.load(f)
```

--------------------------------

### glhmm.sample()

Source: https://github.com/vidaurre/glhmm/blob/main/docs/_build/html/glhmm.html

Samples data from the GLHM model.

```APIDOC
## POST /glhmm/sample

### Description
Samples new time series data from a trained GLHM model. This is useful for simulation and generating synthetic data.

### Method
POST

### Endpoint
/glhmm/sample

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **n_samples** (int) - Required - The number of samples to generate.
- **n_states** (int) - Required - The number of hidden states.
- **n_parcels** (int) - Required - The number of parcels (variables).
- **model_params** (dict) - Required - Dictionary containing the trained GLHM model parameters (e.g., means, covariances, transition probabilities).
- **n_sessions** (int) - Optional - The number of sessions to simulate.

### Request Example
```json
{
  "n_samples": 100,
  "n_states": 2,
  "n_parcels": 10,
  "model_params": {
    "means": [[0.1, 0.2], [0.3, 0.4]],
    "covariances": [[[1.0, 0.1], [0.1, 1.0]], [[1.0, -0.1], [-0.1, 1.0]]],
    "transition_matrix": [[0.9, 0.1], [0.1, 0.9]]
  },
  "n_sessions": 5
}
```

### Response
#### Success Response (200)
- **X** (array-like) - The generated timeseries data for variable set 1.
- **Y** (array-like) - The generated timeseries data for variable set 2.
- **states** (array-like) - The sequence of hidden states.

#### Response Example
```json
{
  "X": [[0.15, 0.25], ...],
  "Y": [[0.35, 0.45], ...],
  "states": [0, 0, 1, 1, 0, ...]
}
```
```

--------------------------------

### Install GLHMM and Dependencies in Colab

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/Testing_across_trials_within_session.ipynb

Installs the GLHMM package and the osfclient library if running in a Google Colab environment. It also imports necessary libraries like numpy and pathlib for data manipulation and statistical analysis. The code sets a random seed for reproducibility.

```python
# Install packages if needed
try:
    import google.colab
    IN_COLAB = True
except ImportError:
    IN_COLAB = False

if IN_COLAB:
    print("Running in Google Colab. Installing GLHMM...")
    !pip install git+https://github.com/vidaurre/glhmm

# Install osfclient if missing
try:
    import osfclient
except ImportError:
    print("Installing osfclient...")
    !pip install osfclient

# Now import everything we need
import numpy as np
from pathlib import Path
from glhmm import graphics, statistics
np.random.seed(0)  # For reproducibility

```

--------------------------------

### Download Tapping Onset Data using OSF CLI

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/HMM-TDE_vs_HMM-MAR_example.ipynb

Downloads tapping onset data, stored as a boolean vector indicating tap timings, from an OSF repository using the OSF CLI. This data is used to align state time courses with behavioral events. Requires the OSF CLI.

```shell
# download onset
! osf -p 8qcyj fetch MEG_data/onset_tap_MAR.pkl ./example_data/onset_tap_MAR.pkl
```

--------------------------------

### slice_matrix

Source: https://github.com/vidaurre/glhmm/blob/main/docs/_build/html/auxiliary.html

Slices rows of an input matrix M based on provided start and end indices for each session.

```APIDOC
## slice_matrix

### Description
Slices rows of input matrix M based on indices array along axis 0.

### Method
N/A (This appears to be a function call, not a REST endpoint)

### Endpoint
N/A

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
```json
{
  "M": "array-like of shape (n_samples, n_parcels)",
  "indices": "array-like of shape (n_sessions, 2)"
}
```

### Response
#### Success Response (200)
- **M_sliced** (array-like) - The sliced matrix.

#### Response Example
```json
{
  "M_sliced": "array-like of shape (n_total_samples, n_parcels)"
}
```
```

--------------------------------

### Display Session Indices

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/HMM-TDE_vs_HMM-MAR_example.ipynb

Displays the start and end indices for each subject session within the concatenated MEG data. This is useful for understanding data segmentation.

```python
# show the indices of the subject/session dat
idx_data
```

--------------------------------

### Concatenate and Index MEG Data

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/HMM-TDE_vs_HMM-MAR_example.ipynb

Concatenates MEG data from multiple subjects into a single array and generates start and end indices for each session. This prepares the data for further processing by grouping sessions.

```python
# concatenate subject data
X_concat = []
for j in range(len(data_meg_mar)):
    X_concat.extend(data_meg_mar[j])

X_concat = np.array(X_concat)

# Get the start and end indices for each session
idx_data = statistics.get_indices_from_list(data_meg_mar)
```

--------------------------------

### Create Indices from Trial Lengths (Python)

Source: https://github.com/vidaurre/glhmm/blob/main/docs/_build/html/_modules/glhmm/auxiliary.html

Generates a 2D array of start and end indices for each trial or session, given an array of their lengths. This is useful for organizing data into distinct segments for processing.

```python
def make_indices_from_T(T):
    """Creates indices array from trials/sessions lengths.
    
    Parameters:
    -----------
    T : array-like of shape (n_sessions,)
        Contains the lengths of each trial/session.

    Returns:
    --------
    indices : array-like of shape (n_sessions, 2)
        The start and end indices of each trial/session in the input data.

    """

    
    N = T.shape[0]
    indices = np.zeros((N,2),dtype=int)
    acc = 0
    for j in range(N):
        indices[j,0] = acc
        indices[j,1] = acc + T[j]
        acc += T[j]
    return indices

```

--------------------------------

### Initialize GLHMM from Dictionary in Python

Source: https://github.com/vidaurre/glhmm/blob/main/docs/_build/html/_modules/glhmm/io_glhmm.html

This snippet demonstrates the initialization of a Gaussian Linear Hidden Markov Model (GLHMM) object. It parses parameters from a 'hmm_mat' dictionary to configure the model's means, betas, covariance matrices, transition probabilities (P), and initial state distribution (Pi). Dependencies include the 'glhmm' library and 'numpy'.

```python
prior_Omega_Gam_shape = hmm_mat["prior_Omega_Gam_shape"][0][0]
else:
    prior_Omega_Gam_rate = hmm_mat["state_0_prior_Omega_Gam_rate"]
    prior_Omega_Gam_shape = hmm_mat["state_0_prior_Omega_Gam_shape"][0][0]
if diagonal_covmat: prior_Omega_Gam_rate = np.squeeze(prior_Omega_Gam_rate)
q = prior_Omega_Gam_rate.shape[0]

if "state_0_Mu_W" in hmm_mat:
    p = hmm_mat["state_0_Mu_W"].shape[0]
    if model_mean == 'state': p -= 1
else: p = 0

hmm = glhmm.glhmm(
    K=K,
    covtype=covtype,
    model_mean=model_mean,
    model_beta=model_beta,
    dirichlet_diag=dirichlet_diag,
    connectivity=connectivity,
    Pstructure=Pstructure,
    Pistructure=Pistructure
    )

# mean 
if model_mean == 'state':
    hmm.mean = []
    for k in range(K):
        hmm.mean.append({})
        Sigma_W = np.squeeze(hmm_mat["state_" + str(k) + "_S_W"])
        Mu_W = np.squeeze(hmm_mat["state_" + str(k) + "_Mu_W"])
        if model_beta == 'state':
            if q==1: hmm.mean[k]["Mu"] = np.array(Mu_W[0])
            else: hmm.mean[k]["Mu"] = Mu_W[0,:]
        else:
            if q==1: hmm.mean[k]["Mu"] = np.array(Mu_W)
            else: hmm.mean[k]["Mu"] = Mu_W
        if diagonal_covmat:
            if model_beta == 'state':
                if q==1: hmm.mean[k]["Sigma"] = np.array([[Sigma_W[0,0]]])
                else: hmm.mean[k]["Sigma"] = np.diag(Sigma_W[:,0,0])
            else:
                if q==1: hmm.mean[k]["Sigma"] = np.array([[Sigma_W]])
                hmm.mean[k]["Sigma"] = np.diag(Sigma_W)
        else:
            if q==1: np.array([[Sigma_W[0,0]]])
            else: hmm.mean[k]["Sigma"] = Sigma_W[0:q,0:q]

# beta
if model_beta == 'state':
    if model_mean == 'state': j0 = 1
    else: j0 = 0
    hmm.beta = []
    for k in range(K):
        hmm.beta.append({})
        Sigma_W = hmm_mat["state_" + str(k) + "_S_W"]
        Mu_W = hmm_mat["state_" + str(k) + "_Mu_W"]
        hmm.beta[k]["Mu"] = np.zeros((p,q))
        hmm.beta[k]["Mu"][:,:] = Mu_W[j0:,:]
        if diagonal_covmat:
            hmm.beta[k]["Sigma"] = np.zeros((p,p,q))
            if q==1:
                hmm.beta[k]["Sigma"][:,:,0] = Sigma_W[j0:,j0:]
            else:
                for j in range(q):
                    hmm.beta[k]["Sigma"][:,:,j] = Sigma_W[j,j0:,j0:]
        else:
            hmm.beta[k]["Sigma"] = Sigma_W[(j0*q):,(j0*q):]

hmm._glhmm__init_priors_sub(prior_Omega_Gam_rate,prior_Omega_Gam_shape,p,q)
hmm._glhmm__update_priors()

# covmatrix
hmm.Sigma = []
if diagonal_covmat and shared_covmat:
    hmm.Sigma.append({})
    hmm.Sigma[0]["rate"] = np.zeros(q)
    hmm.Sigma[0]["rate"][:] = hmm_mat["Omega_Gam_rate"]
    hmm.Sigma[0]["shape"] = hmm_mat["Omega_Gam_shape"][0][0]
elif diagonal_covmat and not shared_covmat:
    for k in range(K):
        hmm.Sigma.append({})
        hmm.Sigma[k]["rate"] = np.zeros(q)
        hmm.Sigma[k]["rate"][:] = hmm_mat["state_" + str(k) + "_Omega_Gam_rate"]
        hmm.Sigma[k]["shape"] = hmm_mat["state_" + str(k) + "_Omega_Gam_shape"][0][0]
elif not diagonal_covmat and shared_covmat:
    hmm.Sigma.append({})
    hmm.Sigma[0]["rate"] = hmm_mat["Omega_Gam_rate"]
    hmm.Sigma[0]["irate"] = hmm_mat["Omega_Gam_irate"]
    hmm.Sigma[0]["shape"] = hmm_mat["Omega_Gam_shape"][0][0]
else: # not diagonal_covmat and not shared_covmat
    for k in range(K):
        hmm.Sigma.append({})
        hmm.Sigma[k]["rate"] = hmm_mat["state_" + str(k) + "_Omega_Gam_rate"]
        hmm.Sigma[k]["irate"] = hmm_mat["state_" + str(k) + "_Omega_Gam_irate"]
        hmm.Sigma[k]["shape"] = hmm_mat["state_" + str(k) + "_Omega_Gam_shape"][0][0]

#hmm.init_dynamics()
hmm.P = hmm_mat["P"]
hmm.Pi = np.squeeze(hmm_mat["Pi"])
hmm.Dir2d_alpha = hmm_mat["Dir2d_alpha"]
hmm.Dir_alpha = np.squeeze(hmm_mat["Dir_alpha"])

return hmm
```

--------------------------------

### Get Mean in glhmm

Source: https://github.com/vidaurre/glhmm/blob/main/docs/_build/html/_modules/glhmm/glhmm.html

Retrieves the mean value for a specified state in the glhmm model. This function is intended to return the mean of a given state, but the implementation details are truncated in the provided text.

```python
def get_mean(self,k=0):

        """Returns the mean of the specified state.
        
```

--------------------------------

### Initialize and Train Gaussian Linear HMM with glhmm.glhmm

Source: https://context7.com/vidaurre/glhmm/llms.txt

Demonstrates how to initialize and train a Gaussian Linear Hidden Markov Model using the `glhmm.glhmm` class. It covers data generation, model configuration (number of states, covariance type, mean and beta models), training options, and the training process itself. The output includes the trained model's active states and final free energy.

```python
import numpy as np
from glhmm.glhmm import glhmm

# Create synthetic data
np.random.seed(42)
n_samples = 1000
n_features = 10
n_regressors = 5
n_sessions = 4

# Generate random data
Y = np.random.randn(n_samples, n_features)
X = np.random.randn(n_samples, n_regressors)

# Define session boundaries
session_lengths = [250, 250, 250, 250]
indices = np.zeros((n_sessions, 2), dtype=int)
indices[0, 0] = 0
indices[0, 1] = session_lengths[0]
for j in range(1, n_sessions):
    indices[j, 0] = indices[j-1, 1]
    indices[j, 1] = indices[j, 0] + session_lengths[j]

# Initialize the HMM with configuration options
hmm = glhmm(
    K=5,                    # Number of states
    covtype='shareddiag',   # Covariance type: 'shareddiag', 'diag', 'sharedfull', 'full', 'identity'
    model_mean='state',     # Mean model: 'state', 'shared', 'no'
    model_beta='state',     # Beta model: 'state', 'shared', 'no'
    dirichlet_diag=10       # Dirichlet prior diagonal (controls state persistence)
)

# Training options
options = {
    'cyc': 100,             # Maximum training cycles
    'tol': 1e-4,            # Convergence tolerance
    'initrep': 5,           # Number of initialization repetitions
    'verbose': True,        # Print progress
    'deactivate_states': True  # Allow deactivating unused states
}

# Train the model
Gamma, Xi, free_energy = hmm.train(X=X, Y=Y, indices=indices, options=options)

# Gamma: State probability timeseries (n_samples, n_states)
# Xi: Joint state transition probabilities
# free_energy: Training free energy per iteration

print(f"Trained model with {hmm.get_active_K()} active states")
print(f"Final free energy: {free_energy[-1]:.2f}")
```

--------------------------------

### Display Session Indices (Python)

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/HMM-TDE_vs_HMM-MAR_example.ipynb

This snippet displays the generated index matrix, which indicates the start and end timepoints for each session in the concatenated brain activity data. It takes the idx_data matrix as input and outputs a visual representation of the session boundaries.

```python
# show the session indices
idx_data
```

--------------------------------

### Calculate State onsets

Source: https://context7.com/vidaurre/glhmm/llms.txt

Identifies the time points when states begin based on the Viterbi path and session indices.

```python
onsets = utils.get_state_onsets(vpath, indices)
```

--------------------------------

### Load Pre-trained Gaussian HMM

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/Prediction_tutorial.ipynb

Loads a pre-trained Gaussian Hidden Markov Model (HMM) from a pickle file. This HMM has been trained on all subjects' timeseries data using a Gaussian observation model with 6 states. It's important to consider potential data leakage if splitting data for training/testing.

```python
hmm = io.load_hmm('./data_prediction/hmm_pred.pkl')
```

--------------------------------

### Load data using pickle

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/HMM-TDE_vs_HMM-MAR_example.ipynb

This Python code snippet loads data from a pickle file named 'data_MEG_TDE.pkl'. It opens the file in binary read mode and uses the pickle.load function to deserialize the data. The loaded data is stored in the 'data_meg_tde' variable, and its length (number of sessions) is printed.

```python
# Loading the data

with open("./example_data/data_MEG_TDE.pkl", "rb") as f:
    data_meg_tde = pickle.load(f)

# Display data information
print(f"Number of sessions in data_meg: {len(data_meg_tde)}")
```

--------------------------------

### Visualize First 5 Timestamp Indices

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/HCP_Testing_across_subjects.ipynb

Displays the first 5 calculated timestamp indices, showing the start and end points for the initial subject segments in the concatenated timeseries. This helps in verifying the output of the `get_timestamp_indices` function.

```python
# Visualize the first 5 timepoints
idx_time[:5]
```

--------------------------------

### Get Beta Value in glhmm

Source: https://github.com/vidaurre/glhmm/blob/main/docs/_build/html/_modules/glhmm/glhmm.html

Retrieves the beta value for a specified state in the glhmm model. This function checks if the model has been trained and if it actually uses a beta parameter. It returns the 'Mu' value from the beta distribution if available.

```python
def get_beta(self,k=0):
        """Returns the beta of the specified state.

        Parameters:
        -----------
        k : int, optional
            The index of the state for which to retrieve the beta value. Default=0.

        Returns:
        --------
        float
            The beta value of the specified state.

        Raises:
        -------
        Exception
            If the model has not yet been trained.
        Exception
            If the model has no beta.
        """

        if not self.trained:
            raise Exception("The model has not yet been trained") 

        if self.hyperparameters["model_beta"] != 'no':
            raise Exception("The model has no beta")

        return self.beta[k]["Mu"]
```

--------------------------------

### Slice Matrix by Indices (Python)

Source: https://github.com/vidaurre/glhmm/blob/main/docs/_build/html/_modules/glhmm/auxiliary.html

Slices rows of an input matrix M based on provided indices, useful for segmenting data into trials or sessions. It takes the matrix and a 2D array of start and end indices as input, returning the concatenated sliced matrix.

```python
#!/usr/bin/env python3
# -*-

import numpy as np
import sys
import scipy.special
import scipy.io
import math

from numba import njit

def slice_matrix(M,indices):
    """Slices rows of input matrix M based on indices array along axis 0.

    Parameters:
    -----------
    M : array-like of shape (n_samples, n_parcels)
        The input matrix.
    indices : array-like of shape (n_sessions, 2)
        The indices that define the sections (i.e., trials/sessions) of the data to be processed.

    Returns:
    --------
    M_sliced : array-like of shape (n_total_samples, n_parcels)
        The sliced matrix.

    """
    N = indices.shape[0]
    T = indices[:,1] - indices[:,0]
    M_sliced = np.empty((np.sum(T),M.shape[1]))
    acc = 0
    for j in range(N):
        ind_1 = np.arange(indices[j,0],indices[j,1])
        ind_2 = np.arange(0,T[j]) + acc
        acc += T[j]
        M_sliced[ind_2,:] = M[ind_1,:]
    return M_sliced

```

--------------------------------

### Get State Visits (Python)

Source: https://github.com/vidaurre/glhmm/blob/main/docs/_build/html/_modules/glhmm/utils.html

Computes a list of visit durations and onsets for a specific state 'k' given a Viterbi path. It identifies contiguous sequences where the state is active and optionally filters visits shorter than a specified threshold.

```python
import numpy as np

def get_visits(vpath, k, threshold=0):
    """Computes a list of visits for state k, given a viterbi path (vpath).

    Parameters:
    ---------------
    vpath : array-like of shape (n_samples,)
        The viterbi path represents the most likely state sequence.
    k : int
        The state for which to compute the visits.
    threshold : int, optional, default=0
        A threshold value used to exclude visits with a duration below this value.

    Returns:
    ------------
    lengths : list of floats
        A list of visit durations for state k, where each duration is greater than the threshold.
    onsets : list of ints
        A list of onset time points for each visit.

    Notes:
    ------
    A visit to state k is defined as a contiguous sequence of time points in which state k is active.

    """

    lengths = []
    onsets = []
    T = vpath.shape[0]
    vpath_k = vpath[:, k]
    t = 0
    while t < T:
        t += np.where(vpath_k[t:] == 1)[0]
        if len(t) == 0:
            break
        t = t[0]
        onsets.append(t)
        tend = np.where(vpath_k[t:] == 0)[0]
        if len(tend) == 0:
            length_visit = len(vpath_k) - t
            if length_visit > threshold: lengths.append(float(length_visit))
            break
        tend = tend[0]
        length_visit = tend
        if length_visit > threshold: lengths.append(float(length_visit))
        t += tend
    return lengths, onsets
```

--------------------------------

### Initialize MAR-HMM Model

Source: https://github.com/vidaurre/glhmm/blob/main/docs/notebooks/HMM-TDE_vs_HMM-MAR_example.ipynb

Instantiates a MAR-HMM model with specified parameters: 'state' for model_beta and model_mean, 3 states (K=3), and 'full' covariance type. This configuration models states based on functional connectivity.

```python
# instantiate model
MAR_hmm = glhmm.glhmm(model_beta='state', model_mean='state', K=3, covtype='full')
```