### Starting the Interactive Shell

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Instructions on how to launch the CogniVal interactive shell or run a single command.

```APIDOC
## Starting the Interactive Shell

Launch the CogniVal interactive shell after installation.

### Command

```bash
# Start the interactive shell
cognival

# Run a single command without entering the shell
cognival list configs
```
```

--------------------------------

### Import custom embeddings

Source: https://github.com/ds3lab/cognival-cli/blob/master/README.md

Starts the assistant to import custom embeddings from a URL or local path.

```bash
import embeddings "<some URL or local path>"
```

--------------------------------

### Run All Experiments

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Execute all configured experiments. Use this command to start a comprehensive test run.

```python
run
```

--------------------------------

### EEG Cognitive Source Format Example

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Shows the format for EEG cognitive source data, where each line starts with a word followed by values for specified electrodes (e1, e2, etc.).

```text
# EEG cognitive source format (electrodes e1, e2, ...):
# word e1 e2 e3 e4 e5 ...
"""
word e1 e2 e3 e4 e5
his 0.539 0.435 0.523 0.505 0.466
the 0.612 0.398 0.487 0.521 0.445
"""
```

--------------------------------

### Generate PDF Report

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Generate a PDF report with experimental statistics and visualizations. Requires `wkhtmltopdf` to be installed.

```python
report pdf=True
```

--------------------------------

### Embedding input format example

Source: https://github.com/ds3lab/cognival-cli/blob/master/README.md

Embeddings must be provided as raw text with space-separated values.

```text
island 1.4738 0.097269 -0.87687 0.95299 -0.17249 0.10427 -1.1632 ...
```

--------------------------------

### Launch CogniVal Shell

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Commands to start the interactive shell or execute single commands directly.

```bash
# Start the interactive shell
cognival

# Run a single command without entering the shell
cognival list configs
```

--------------------------------

### Importing Word Embeddings

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Guide on importing default or custom word embeddings for evaluation.

```APIDOC
## Importing Word Embeddings

Import default or custom word embeddings for evaluation.

### Commands

```python
# Import a default embedding (e.g., GloVe 50-dimensional)
import embeddings glove.6B.50

# Import all default embeddings
import embeddings all

# Import custom embeddings from URL
import embeddings "https://example.org/my_embeddings.zip"

# Import custom embeddings from local path
import embeddings "/path/to/embeddings.txt"
```

### Prompts

The assistant will prompt for embedding name, dimensionality, format (binary/text), and chunking options.
```

--------------------------------

### Eye-Tracking Format Example

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Demonstrates the format for eye-tracking data, including columns for word and various fixation-related features.

```text
# Eye-tracking format (feature columns):
# word WORD_FIXATION_COUNT WORD_GAZE_DURATION WORD_FIRST_FIXATION_DURATION
"""
word WORD_FIXATION_COUNT WORD_GAZE_DURATION WORD_FIRST_FIXATION_DURATION
the 0.116 0.112 0.254
a 0.089 0.095 0.198
"""
```

--------------------------------

### fMRI Format Example

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Presents the format for fMRI data, where each line contains a word followed by voxel values (v0, v1, etc.).

```text
# fMRI format (voxels v0, v1, ...):
# word v0 v1 v2 v3 ...
"""
word v0 v1 v2 v3
beginning 0.358 0.432 0.794 0.123
"""
```

--------------------------------

### Word Embeddings Format Example

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Illustrates the expected space-separated format for word embeddings, with each line containing a word followed by its dimensional values.

```text
# Word embeddings format (space-separated, one word per line):
# word dim1 dim2 dim3 ...
"""
island 1.4738 0.097269 -0.87687 0.95299 -0.17249
ocean 0.8234 -0.12345 0.54321 -0.98765 0.11111
"""
```

--------------------------------

### Cognitive data source format examples

Source: https://github.com/ds3lab/cognival-cli/blob/master/README.md

Cognitive data sources must be space-separated text files with values scaled between 0 and 1.

```text
word e1 e2 e3 e4 e5 ...
his 0.5394774791900334 0.4356708610374691 0.523294558226597 0.5059544824545096 0.466957449316214 ...
```

```text
word v0 v1 v2 v3 ...
beginning 0.3585450775710978 0.43270347838578155 0.7947947579149615 ...
```

```text
word WORD_FIXATION_COUNT WORD_GAZE_DURATION WORD_FIRST_FIXATION_DURATION ...
the 0.1168531943034873 0.11272377054039184 0.25456297601240524 ...
```

--------------------------------

### CogniVal Configuration Dictionary Structure

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

An example of the configuration dictionary structure used in CogniVal, detailing paths, experiment settings, and configurations for cognitive data and word embeddings.

```python
config_dict = {
    "PATH": "/home/user/.cognival",
    "outputDir": "results/my_experiment",
    "folds": 5,
    "seed": 42,
    "run_id": 1,
    "cogDataConfig": {
        "eeg_zuco": {
            "dataset": "cognitive_sources/eeg/zuco.txt",
            "type": "multivariate_output",
            "modality": "eeg",
            "features": ["single"],
            "wordEmbSpecifics": {
                "glove.6B.50": {
                    "layers": [[128, 64], [256, 128]],
                    "activations": ["relu"],
                    "batch_size": [32],
                    "epochs": [50],
                    "cv_split": 5,
                    "validation_split": 0.1
                }
            }
        }
    },
    "wordEmbConfig": {
        "glove.6B.50": {
            "path": "embeddings/glove.6B/glove.6B.50d.txt",
            "dimensions": 50,
            "chunked": False,
            "random_embedding": "random-50-10"
        }
    },
    "randEmbConfig": {
        "random-50-10_1_2": {
            "path": "embeddings/random_multiseed/50_dim/10_seeds/random-50-10_1_2.txt"
        }
    }
}
```

--------------------------------

### Show Configuration Details

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Display properties and experiment details for configurations.

```python
# Show basic configuration properties
config show

# Show detailed experiment information
config show details=True

# Show details for a specific cognitive source
config show details=True cognitive-source=eeg_zuco

# Hide random baselines in output
config show details=True hide-random=True
```

--------------------------------

### List Configurations

Source: https://github.com/ds3lab/cognival-cli/blob/master/README.md

Lists all available configurations. The reference configuration is read-only and used for populating new configurations.

```bash
list configs
```

--------------------------------

### Creating and Opening Configurations

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Commands to create new experiment configurations or open existing ones for editing.

```APIDOC
## Creating and Opening Configurations

Create or edit experiment configurations.

### Commands

```python
# Create a new configuration
config open configuration=my_experiment

# Open existing configuration for editing
config open configuration=my_experiment edit=True

# Overwrite an existing configuration
config open configuration=my_experiment overwrite=True
```
```

--------------------------------

### Manage Configurations

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Create, edit, or overwrite experiment configurations.

```python
# Create a new configuration
config open configuration=my_experiment

# Open existing configuration for editing
config open configuration=my_experiment edit=True

# Overwrite an existing configuration
config open configuration=my_experiment overwrite=True
```

--------------------------------

### List Configurations

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

View available experiment configurations within the shell.

```python
# In the CogniVal shell
list configs
```

--------------------------------

### Showing Configuration Details

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

How to display configuration properties and experiment details, with options for verbosity and filtering.

```APIDOC
## Showing Configuration Details

Display configuration properties and experiment details.

### Commands

```python
# Show basic configuration properties
config show

# Show detailed experiment information
config show details=True

# Show details for a specific cognitive source
config show details=True cognitive-source=eeg_zuco

# Hide random baselines in output
config show details=True hide-random=True
```
```

--------------------------------

### Open Configuration

Source: https://github.com/ds3lab/cognival-cli/blob/master/README.md

Opens an existing configuration file or creates a new one. Use edit=True to set general parameters in the editor. Configurations can be overwritten.

```bash
config open configuration=demo [overwrite=False] [edit=False]
```

--------------------------------

### Configure Experiments

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Set up experiments by combining cognitive sources and embeddings.

```python
# Configure experiments for all cognitive sources and embeddings
config experiment cognitive-sources=[all] embeddings=[all]

# Configure specific combinations
config experiment cognitive-sources=[eeg_zuco, eeg_n400] embeddings=[glove.6B.50, word2vec]

# Include random baselines
config experiment cognitive-sources=[all] embeddings=[all] baselines=True

# Configure by modality
config experiment modalities=[eeg] embeddings=[all]

# Edit cognitive source parameters
config experiment cognitive-sources=[eeg_zuco] edit-cog-source-params=True

# Edit embeddings one by one
config experiment cognitive-sources=[all] embeddings=[all] single-edit=True
```

--------------------------------

### Listing Available Configurations

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

How to view all available experiment configurations within the CogniVal shell.

```APIDOC
## Listing Available Configurations

View all available experiment configurations.

### Command

```python
# In the CogniVal shell
list configs
```

### Description

Output displays a table with configuration names and their properties including paths, fold counts, and output directories.
```

--------------------------------

### Show Configuration Details

Source: https://github.com/ds3lab/cognival-cli/blob/master/README.md

Displays details of a configuration, including general properties, cognitive sources, and embeddings. Experiment details can be viewed for a single source or the entire configuration.

```bash
config show demo
```

```bash
config show details=True
```

```bash
config show details=True cognitive-source=eeg_zuco
```

```bash
config show [details=False] [cognitive-source=None] [hide-random=True]
```

--------------------------------

### Run experiments

Source: https://github.com/ds3lab/cognival-cli/blob/master/README.md

Executes experiments based on specified configuration parameters.

```bash
run [embeddings=[all]] [modalities=None] [cognitive-sources=[all]] [cognitive-features=None] [baselines=True]
```

--------------------------------

### Import Cognitive Sources

Source: https://github.com/ds3lab/cognival-cli/blob/master/README.md

Imports preprocessed CogniVal cognitive sources or custom sources. Custom sources must be placed in the path indicated by the assistant.

```bash
import cognitive-sources
```

```bash
import cognitive-sources source=yoursource
```

--------------------------------

### Configuring Experiments

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Steps to set up experiments by combining cognitive sources and embeddings, with various filtering options.

```APIDOC
## Configuring Experiments

Set up experiments combining cognitive sources and embeddings.

### Commands

```python
# Configure experiments for all cognitive sources and embeddings
config experiment cognitive-sources=[all] embeddings=[all]

# Configure specific combinations
config experiment cognitive-sources=[eeg_zuco, eeg_n400] embeddings=[glove.6B.50, word2vec]

# Include random baselines
config experiment cognitive-sources=[all] embeddings=[all] baselines=True

# Configure by modality
config experiment modalities=[eeg] embeddings=[all]

# Edit cognitive source parameters
config experiment cognitive-sources=[eeg_zuco] edit-cog-source-params=True

# Edit embeddings one by one
config experiment cognitive-sources=[all] embeddings=[all] single-edit=True
```
```

--------------------------------

### config open

Source: https://github.com/ds3lab/cognival-cli/blob/master/README.md

Opens or creates a configuration file for editing.

```APIDOC
## config open

### Description
Opens a configuration if it exists or creates an empty one. Allows editing of general parameters.

### Parameters
#### Query Parameters
- **configuration** (string) - Required - The name of the configuration to open.
- **overwrite** (boolean) - Optional - Whether to overwrite existing configuration (default: False).
- **edit** (boolean) - Optional - Whether to open the configuration in edit mode (default: False).
```

--------------------------------

### Run Experiments

Source: https://github.com/ds3lab/cognival-cli/blob/master/README.md

Executes experiments based on a provided configuration.

```APIDOC
## RUN run

### Description
Run all or a subset of experiments specified in a configuration.

### Parameters
#### Query Parameters
- **embeddings** (string) - Optional - List of embeddings to use (default: [all])
- **modalities** (string) - Optional - Modalities to use (default: None)
- **cognitive-sources** (string) - Optional - Cognitive sources to use (default: [all])
- **cognitive-features** (string) - Optional - Nested list of features for all cognitive-sources (default: None)
- **baselines** (boolean) - Optional - Whether to include random baselines (default: True)
```

--------------------------------

### config show

Source: https://github.com/ds3lab/cognival-cli/blob/master/README.md

Displays details of a configuration, cognitive sources, and experiments.

```APIDOC
## config show

### Description
Shows details of a configuration, associated cognitive sources, embeddings, and experiments.

### Parameters
#### Query Parameters
- **details** (boolean) - Optional - Whether to show verbose experiment details (default: False).
- **cognitive-source** (string) - Optional - Filter by a specific cognitive source.
- **hide-random** (boolean) - Optional - Whether to hide random baselines (default: True).
```

--------------------------------

### Import Cognitive Sources

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Download and import preprocessed cognitive data bundles or custom sources.

```python
# Import all CogniVal cognitive sources (EEG, fMRI, eye-tracking)
import cognitive-sources

# Import a custom cognitive source
import cognitive-sources source=my_custom_source
```

--------------------------------

### Importing CogniVal Cognitive Sources

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Instructions for downloading and importing CogniVal's preprocessed cognitive sources or custom ones.

```APIDOC
## Importing CogniVal Cognitive Sources

Download and import the preprocessed CogniVal cognitive sources bundle.

### Commands

```python
# Import all CogniVal cognitive sources (EEG, fMRI, eye-tracking)
import cognitive-sources

# Import a custom cognitive source
import cognitive-sources source=my_custom_source
```

### Note

Custom cognitive sources must conform to CogniVal format (space-separated, columns: word, feature columns named e[i]).
```

--------------------------------

### Import custom cognitive source via CLI

Source: https://github.com/ds3lab/cognival-cli/blob/master/README.md

Use this command to initiate the semi-automatic import process for a custom cognitive data source.

```bash
import cognitive-sources source=<name of the source>
```

--------------------------------

### Import Word Embeddings

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Load default or custom word embeddings for evaluation.

```python
# Import a default embedding (e.g., GloVe 50-dimensional)
import embeddings glove.6B.50

# Import all default embeddings
import embeddings all

# Import custom embeddings from URL
import embeddings "https://example.org/my_embeddings.zip"

# Import custom embeddings from local path
import embeddings "/path/to/embeddings.txt"
```

--------------------------------

### import random-baselines

Source: https://github.com/ds3lab/cognival-cli/blob/master/README.md

Associates embeddings with random baselines.

```APIDOC
## import random-baselines

### Description
Associates or re-associates embeddings with a set of random baselines for robustness testing.

### Parameters
#### Query Parameters
- **embeddings** (string) - Required - The embedding to associate.
- **num-baselines** (integer) - Optional - Number of folds to generate (default: 10).
- **seed-func** (string) - Optional - Seed function to use (default: exp_e_floored).
- **force** (boolean) - Optional - Whether to force re-association (default: False).
```

--------------------------------

### import cognitive-sources

Source: https://github.com/ds3lab/cognival-cli/blob/master/README.md

Imports preprocessed or custom cognitive sources.

```APIDOC
## import cognitive-sources

### Description
Imports the batch of preprocessed CogniVal cognitive sources or a custom source.

### Parameters
#### Query Parameters
- **source** (string) - Optional - Name of the custom source to import.
```

--------------------------------

### Run with Specific Cognitive Features

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Configure experiments to use particular cognitive features. This enables fine-grained analysis of specific cognitive signals.

```python
run cognitive-sources=[eye-tracking_geco] cognitive-features=["WORD_FIXATION_COUNT;WORD_GAZE_DURATION"]
```

--------------------------------

### Load and Prepare Data for Cross-Validation

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Load and prepare cognitive data along with word embeddings for model training using cross-validation. The `mode='proper'` is for real embeddings, while `mode='random'` is for baselines.

```python
from cognival.handlers.data_handler import data_handler

words_test, X_train, y_train, X_test, y_test = data_handler(
    mode='proper',           # 'proper' for real embeddings, 'random' for baselines
    config=config_dict,      # Configuration dictionary
    word_embedding='glove.6B.50',
    cognitive_data='eeg_zuco',
    feature='single',        # Feature name or 'single' for multivariate
    truncate_first_line=False
)
```

--------------------------------

### Generate Both Reports and Open HTML

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Create both HTML and PDF reports, and automatically open the HTML report in the default web browser. This offers comprehensive output and immediate viewing.

```python
report html=True pdf=True open-html=True
```

--------------------------------

### Run Specific Cognitive Sources

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Execute experiments with designated cognitive data sources. This helps in analyzing results from specific datasets.

```python
run cognitive-sources=[eeg_zuco, eeg_n400]
```

--------------------------------

### Run Cross-Validation and Prediction with Model Handler

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Execute the model training process using grid search cross-validation. The `config` dictionary specifies hyperparameters to tune, and the function returns word errors, grid search results, and mean squared errors.

```python
word_error, grids_result, mserrors = model_handler(
    config={
        'layers': [[128, 64], [256, 128]],
        'activations': ['relu', 'tanh'],
        'batch_size': [32, 64],
        'epochs': [50, 100],
        'cv_split': 5,
        'validation_split': 0.1
    },
    words_test=words_test,
    X_train=X_train,
    y_train=y_train,
    X_test=X_test,
    y_test=y_test
)
```

--------------------------------

### Generate HTML Report

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Create an HTML report containing statistics and visualizations from the experiments. This provides a web-viewable summary of results.

```python
report html=True
```

--------------------------------

### Import Random Baselines

Source: https://github.com/ds3lab/cognival-cli/blob/master/README.md

Associates or re-associates embeddings with random baselines. The 'num-baselines' parameter specifies the number of folds. Generation is parallelized.

```bash
import random-baselines embeddings=glove.6B.50 [num-baselines=10] [seed-func=exp_e_floored] [force=False]
```

--------------------------------

### Run Without Baselines

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Execute experiments excluding baseline comparisons. Note: This is not recommended for publication purposes.

```python
run baselines=False
```

--------------------------------

### Generating Random Baselines

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

How to create random baseline embeddings for significance testing, with options to regenerate.

```APIDOC
## Generating Random Baselines

Create random baseline embeddings for significance testing.

### Commands

```python
# Generate 10-fold random baselines for an embedding
import random-baselines embeddings=glove.6B.50 num-baselines=10

# Force regeneration of existing baselines
import random-baselines embeddings=glove.6B.50 num-baselines=10 force=True
```

### Note

Random baselines match the dimensionality of the reference embedding and use different seeds for robustness.
```

--------------------------------

### Generate Random Baselines

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Create random baseline embeddings for significance testing.

```python
# Generate 10-fold random baselines for an embedding
import random-baselines embeddings=glove.6B.50 num-baselines=10

# Force regeneration of existing baselines
import random-baselines embeddings=glove.6B.50 num-baselines=10 force=True
```

--------------------------------

### Generate Report for Specific Run and Modalities

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Create a report for a specific experiment run and selected modalities, with a custom alpha value for significance. This allows for highly targeted report generation.

```python
report run_id=1 modalities=[eeg] alpha=0.01
```

--------------------------------

### Delete Configuration Elements

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Remove experiments, cognitive sources, or entire configurations.

```python
# Delete specific experiments
config delete cognitive-sources=[eeg_zuco] embeddings=[glove.6B.50]

# Delete entire cognitive source and its experiments
config delete cognitive-sources=[eeg_zuco]

# Delete by modality
config delete modalities=[eeg]

# Delete entire configuration
config delete my_experiment
```

--------------------------------

### Generate reports

Source: https://github.com/ds3lab/cognival-cli/blob/master/README.md

Performs significance testing and aggregation, then generates HTML or PDF reports.

```bash
report [run_id=0] [modalities=[eye-tracking, eeg, fmri]] [alpha=0.01] [test=Wilcoxon] [html=True] [open-html=False] [pdf=False] [open-pdf=False]
```

--------------------------------

### Listing Embeddings and Cognitive Sources

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Commands to display available word embeddings and imported cognitive data sources.

```APIDOC
## Listing Embeddings and Cognitive Sources

Display available word embeddings and cognitive data sources.

### Commands

```python
# List all embeddings (default and custom)
list embeddings

# List imported cognitive sources with their features
list cognitive-sources
```
```

--------------------------------

### Edit Experiments in Configuration

Source: https://github.com/ds3lab/cognival-cli/blob/master/README.md

Edits experiments, allowing setting of grid search parameters for activations, batch size, epochs, layers, cross-validation folds, and validation split. Use 'all' to specify all cognitive sources or embeddings.

```bash
config experiment [baselines=False] [modalities=None] [cognitive-sources=[all]] [embeddings=[all]] [single-edit=False] [edit-cog-source-params=False]
```

--------------------------------

### Run Specific Modalities

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Run experiments focusing on particular data modalities. Useful for isolating and testing specific data types.

```python
run modalities=[eeg]
```

--------------------------------

### Import Embeddings

Source: https://github.com/ds3lab/cognival-cli/blob/master/README.md

Imports and preprocesses default or custom embeddings. Supports word2vec and BERT compliant formats. Random baselines can be directly associated.

```bash
import embeddings glove.6B.50
```

```bash
import embeddings http://example.org/example.zip
```

--------------------------------

### List Embeddings and Cognitive Sources

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Display imported word embeddings and cognitive data sources.

```python
# List all embeddings (default and custom)
list embeddings

# List imported cognitive sources with their features
list cognitive-sources
```

--------------------------------

### Run Single Experiment Evaluation

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Execute a complete evaluation for a single embedding-cognitive source pair using the `run_single` function. It returns the word embedding used, logging information, word error, and training history. Set `gpu_id=None` to use the CPU.

```python
from cognival.cog_evaluate import run_single

word_embedding, logging, word_error, history = run_single(
    mode='proper',
    config=config_dict,
    word_embedding='glove.6B.50',
    cognitive_data='eeg_zuco',
    cognitive_parent='eeg',
    modality='eeg',
    feature='single',
    truncate_first_line=False,
    gpu_id=None  # Set GPU ID or None for CPU
)
```

--------------------------------

### Run Specific Embeddings

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Execute experiments using specified word embeddings. This allows targeted testing of different embedding models.

```python
run embeddings=[glove.6B.50, word2vec]
```

--------------------------------

### Render Statistical Plots with Jinja2

Source: https://github.com/ds3lab/cognival-cli/blob/master/cognival/reporting/templates/cognival_report.html

Iterates through statistical plot data to render base64 encoded images in a grid layout.

```jinja2
{% for modality, fig_data in stats_plots%}

### {{ modality }}

![{{ modality }}](data:image/png;base64,{{ fig_data }})

{% endfor %}
```

```jinja2
{% for modality, fig_list in stats_over_time_plots.items()%}

### {{ modality }}

{% for fig_data in fig_list %}

![stats_over_time_plot_{{ modality }}](data:image/png;base64,{{ fig_data }})

{% endfor %}

{% endfor %}
```

```jinja2
{% if training_history_plots %}

{% for plot_name, fig_data in training_history_plots.items() %}

![{{plot_name}}](data:image/png;base64,{{ fig_data }})

{% endfor %}

{% endif %}
```

--------------------------------

### Delete Configuration Elements

Source: https://github.com/ds3lab/cognival-cli/blob/master/README.md

Deletes experiments, cognitive sources (along with their experiments), or entire configurations. Specify cognitive sources, modalities, or embeddings to delete specific elements.

```bash
config delete cognitive-sources=[eeg_zuco] embeddings=[glove.6B.50]
```

```bash
config delete cognitive-sources=[eeg_zuco]
```

```bash
config delete modalities=[eeg_zuco]
```

```bash
config delete demo
```

```bash
config delete [modalities=None] [cognitive-sources=None] [embeddings=None]
```

--------------------------------

### Deleting Configuration Elements

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Commands for removing experiments, cognitive sources, or entire configurations.

```APIDOC
## Deleting Configuration Elements

Remove experiments, cognitive sources, or entire configurations.

### Commands

```python
# Delete specific experiments
config delete cognitive-sources=[eeg_zuco] embeddings=[glove.6B.50]

# Delete entire cognitive source and its experiments
config delete cognitive-sources=[eeg_zuco]

# Delete by modality
config delete modalities=[eeg]

# Delete entire configuration
config delete my_experiment
```
```

--------------------------------

### Sort HTML Table with JavaScript

Source: https://github.com/ds3lab/cognival-cli/blob/master/cognival/reporting/templates/cognival_report.html

Provides client-side sorting functionality for HTML tables by column index.

```javascript
function sortTable(table_id, n) { var table, rows, switching, i, x, y, shouldSwitch, dir, switchcount = 0; table = document.getElementById(table_id); switching = true; // Set the sorting direction to ascending: dir = "asc"; /* Make a loop that will continue until no switching has been done: */ while (switching) { // Start by saying: no switching is done: switching = false; rows = table.rows; /* Loop through all table rows (except the first, which contains table headers): */ for (i = 1; i < (rows.length - 1); i++) { // Start by saying there should be no switching: shouldSwitch = false; /* Get the two elements you want to compare, one from current row and one from the next: */ x = rows[i].getElementsByTagName("TD")[n]; y = rows[i + 1].getElementsByTagName("TD")[n]; /* Check if the two rows should switch place, based on the direction, asc or desc: */ if (isNaN(x.innerHTML)) { if (dir == "asc") { if (x.innerHTML.toLowerCase() > y.innerHTML.toLowerCase()) { // If so, mark as a switch and break the loop: shouldSwitch = true; break; } } else if (dir == "desc") { if (x.innerHTML.toLowerCase() < y.innerHTML.toLowerCase()) { // If so, mark as a switch and break the loop: shouldSwitch = true; break; } } } else { if (dir == "asc") { if (Number(x.innerHTML) > Number(y.innerHTML)) { shouldSwitch = true; break; } } else if (dir == "desc") { if (Number(x.innerHTML) < Number(y.innerHTML)) { shouldSwitch = true; break; } } } } if (shouldSwitch) { /* If a switch has been marked, make the switch and mark that a switch has been done: */ rows[i].parentNode.insertBefore(rows[i + 1], rows[i]); switching = true; // Each time a switch is done, increase this count by 1: switchcount ++; } else { /* If no switching has been done AND the direction is "asc", set the direction to "desc" and run the while loop again. */ if (switchcount == 0 && dir == "asc") { dir = "desc"; switching = true; } } } }
```

--------------------------------

### Create a Keras Sequential Model

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Define a Keras Sequential model with specified hidden layer sizes and activation functions. The `input_dim` should match the embedding dimensions, and `output_dim` the number of cognitive features.

```python
from cognival.handlers.model_handler import model_handler, create_model

model = create_model(
    layers=[128, 64],      # Hidden layer sizes
    activation='relu',      # Activation function
    input_dim=50,          # Embedding dimensions
    output_dim=1           # Output dimensions (cognitive features)
)
```

--------------------------------

### config delete

Source: https://github.com/ds3lab/cognival-cli/blob/master/README.md

Deletes experiments, cognitive sources, or entire configurations.

```APIDOC
## config delete

### Description
Removes experiments, cognitive sources, or the entire configuration from the system.

### Parameters
#### Query Parameters
- **modalities** (list) - Optional - Modalities to delete.
- **cognitive-sources** (list) - Optional - Cognitive sources to delete.
- **embeddings** (list) - Optional - Embeddings to delete.
```

--------------------------------

### Aggregate results

Source: https://github.com/ds3lab/cognival-cli/blob/master/README.md

Aggregates significance test results to determine accepted hypotheses under Bonferroni correction.

```bash
aggregate [run_id=0] [modalities=[eye-tracking, eeg, fmri]] [test=Wilcoxon]
```

--------------------------------

### Test Significance Between Model and Baseline

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Compares a model's performance against a baseline using statistical tests like Wilcoxon. Requires baseline and model score files, and an alpha level for significance.

```python
significant, p_value, experiment_name = test_significance(
    baseline_file='baseline_scores_experiment.txt',
    model_file='embeddings_scores_experiment.txt',
    alpha=corrected_alpha,
    test='Wilcoxon'
)

if significant:
    print(f"{experiment_name}: significant (p={p_value:.3e})")
else:
    print(f"{experiment_name}: not significant (p={p_value:.3e})")
```

--------------------------------

### Generate Report

Source: https://github.com/ds3lab/cognival-cli/blob/master/README.md

Performs significance testing and aggregation to generate a report.

```APIDOC
## POST report

### Description
Perform significance testing and result aggregation, and generate a HTML or PDF report tabulating and plotting statistics.

### Parameters
#### Query Parameters
- **run_id** (integer) - Optional - 0 for last run or specific run ID (default: 0)
- **modalities** (list) - Optional - Modalities for which significance is determined (default: [eye-tracking, eeg, fmri])
- **alpha** (float) - Optional - Alpha for significance computation (default: 0.01)
- **test** (string) - Optional - Significance test (default: Wilcoxon)
- **html** (boolean) - Optional - Generate HTML report (default: True)
- **open-html** (boolean) - Optional - Open HTML report (default: False)
- **pdf** (boolean) - Optional - Generate PDF report (default: False)
- **open-pdf** (boolean) - Optional - Open PDF report (default: False)
```

--------------------------------

### Aggregate Results

Source: https://github.com/ds3lab/cognival-cli/blob/master/README.md

Aggregates significance test results for an experimental run.

```APIDOC
## POST aggregate

### Description
Aggregate the significance test results of an experimental run and output accepted hypotheses under Bonferroni correction.

### Parameters
#### Query Parameters
- **run_id** (integer) - Optional - 0 for last run or specific run ID (default: 0)
- **modalities** (list) - Optional - Modalities for which significance is determined (default: [eye-tracking, eeg, fmri])
- **test** (string) - Optional - Significance test (default: Wilcoxon)
```

--------------------------------

### Compute Significance with Custom Alpha

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Run significance testing using a user-defined alpha value for statistical significance. This allows for adjusting the threshold for significance.

```python
significance alpha=0.05
```

--------------------------------

### Render Aggregated Data Tables

Source: https://github.com/ds3lab/cognival-cli/blob/master/cognival/reporting/templates/cognival_report.html

Iterates through aggregated dataframes to generate HTML table rows with conditional formatting for significance values.

```jinja2
{% for column in df_agg.columns %} {% endfor %} {% for _, row in df_agg.iterrows() %} {% for key, value in row.to_dict().items() %} {% if value is number %} {% else %} {% if key == 'Significance' %} {% if sig_status(value) == 'all' %} {% elif sig_status(value) == 'some' %} {% else %} {% endif %} {% else %} {% endif %} {% endif %} {% endfor %} {% endfor %}

{{ column }}

{{ '{:.5f}'.format(value) }}

{{ value }}

{{ value }}

{{ value }}

{{ value }}
```

```jinja2
{% for column in df_details.columns %} {% endfor %} {% for _, row in df_details.iterrows() %} {% for key, value in row.to_dict().items() %} {% if key == 'significant' %} {% if sig_status(value) == 'all' %} {% elif sig_status(value) == 'some' %} {% else %} {% endif %} {% else %} {% endif %} {% endfor %} {% endfor %}

{{ column }}

{{ get_sig(value, average_multi_hypothesis) }}

{{ get_sig(value, average_multi_hypothesis) }}

{{ get_sig(value, average_multi_hypothesis) }}

{{ value }}
```

```jinja2
{% for column in df_random.columns %} {% endfor %} {% for _, row in df_random.iterrows() %} {% for key, value in row.to_dict().items() %} {% endfor %} {% endfor %}

{{ column }}

{{ value }}
```

--------------------------------

### Compute significance

Source: https://github.com/ds3lab/cognival-cli/blob/master/README.md

Calculates the significance of experimental results using the Wilcoxon rank-sum test.

```bash
significance [run_id=0] [modalities=[eye-tracking, eeg, fmri]] [alpha=0.01 test=Wilcoxon]
```

--------------------------------

### Significance Testing

Source: https://github.com/ds3lab/cognival-cli/blob/master/README.md

Computes the significance of results from an experimental run.

```APIDOC
## POST significance

### Description
Compute the significance of results of an experimental run. Requires random baselines to have been associated and evaluated.

### Parameters
#### Query Parameters
- **run_id** (integer) - Optional - 0 for last run or specific run ID (default: 0)
- **modalities** (list) - Optional - Modalities for which significance is determined (default: [eye-tracking, eeg, fmri])
- **alpha** (float) - Optional - Alpha for significance computation (default: 0.01)
- **test** (string) - Optional - Significance test (default: Wilcoxon)
```

--------------------------------

### Aggregate Results for Last Run

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Aggregate significance test results, applying Bonferroni correction, for the most recent experiment run. This provides a consolidated view of statistical significance.

```python
aggregate
```

--------------------------------

### Compute Significance for Last Run

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Perform statistical significance testing on the results of the most recent experiment run. This is a quick way to assess the significance of the last operation.

```python
significance
```

--------------------------------

### Aggregate Results for Specific Modalities

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Aggregate significance test results for selected data modalities. This helps in summarizing statistical findings across different data types.

```python
aggregate modalities=[eeg, eye-tracking]
```

--------------------------------

### Calculate Bonferroni-Corrected Alpha

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Compute the Bonferroni-corrected alpha value for multiple hypothesis testing. This helps in controlling the family-wise error rate.

```python
from cognival.significance_testing.testing_helpers import (
    bonferroni_correction,
    test_significance
)

num_hypotheses = 15
alpha = 0.01
corrected_alpha = bonferroni_correction(alpha, num_hypotheses)
# corrected_alpha = 0.01 / 15 = 0.000667
```

--------------------------------

### Compute Significance for Specific Modalities

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Perform significance testing on results from selected data modalities. This is useful for comparing the statistical significance across different data types.

```python
significance modalities=[eeg, fmri]
```

--------------------------------

### Chunk Large Embedding Files

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Splits large embedding files into a specified number of smaller, manageable chunks. Use this when embedding files exceed memory or processing limits. The `truncate_first_line` option can be used to skip header lines.

```python
from cognival.handlers.data_handler import chunk

# Chunk embeddings into 4 parts
chunk(
    input_path_name='/path/to/embeddings',
    output_path_name='/path/to/output',
    input_file_name='large_embeddings.txt',
    output_file_name='chunked_emb',
    number_of_chunks=4,
    truncate_first_line=True  # Skip header line
)

# Creates: chunked_emb_0.txt, chunked_emb_1.txt, chunked_emb_2.txt, chunked_emb_3.txt
```

--------------------------------

### Aggregate Results for Specific Run ID

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Aggregate significance test results for a specific experiment run identified by its ID. This allows for focused analysis of historical aggregation results.

```python
aggregate run_id=1
```

--------------------------------

### Compute Significance for Specific Run ID

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Calculate statistical significance for a particular experiment run identified by its ID. This allows for targeted analysis of historical results.

```python
significance run_id=1
```

--------------------------------

### Compute Significance with Specified Test Type

Source: https://context7.com/ds3lab/cognival-cli/llms.txt

Perform significance testing using a specific statistical test. The default test is Wilcoxon.

```python
significance test=Wilcoxon
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.