NicheCompass

https://nichecompass.readthedocs.io/en/latest/
Admin
NicheCompass is a package for end-to-end analysis of spatial multi-omics data, including spatial...
Tokens:82,346
Snippets:559
Trust Score:9.6
Update:3 months ago
Show doc for...
Tokens:
Raw
### Install PyTorch for CUDA 12.4 Example

Source: https://nichecompass.readthedocs.io/en/latest/installation.html

Example command to install PyTorch version 2.6.0 with CUDA 12.4 support.

```bash
uv pip install torch --index-url https://download.pytorch.org/whl/cu124
uv pip install pyg_lib torch_scatter torch_sparse -f https://data.pyg.org/whl/torch-2.6.0+cu124.html

```

--------------------------------

### Install NicheCompass with Optional Dependencies

Source: https://nichecompass.readthedocs.io/en/latest/installation.html

Installs NicheCompass along with all optional dependencies required for benchmarking, multimodal analysis, and tutorials.

```bash
uv pip install nichecompass[all]

```

--------------------------------

### Enable GPU Support for JAX (CUDA 12 Example)

Source: https://nichecompass.readthedocs.io/en/latest/installation.html

Example command to enable GPU support for JAX with CUDA 12.

```bash
uv pip install jax[cuda12]

```

--------------------------------

### Install NicheCompass via Pip

Source: https://nichecompass.readthedocs.io/en/latest/installation.html

Installs the NicheCompass package using the uv pip command.

```bash
uv pip install nichecompass

```

--------------------------------

### Install PyTorch with GPU Support

Source: https://nichecompass.readthedocs.io/en/latest/installation.html

Installs PyTorch and related libraries with GPU support. Replace `${TORCH}` and `${CUDA}` with your specific PyTorch and CUDA versions.

```bash
uv pip install torch==${TORCH}+${CUDA} --extra-index-url https://download.pytorch.org/whl/${CUDA}
uv pip install pyg_lib torch_scatter torch_sparse -f https://data.pyg.org/whl/torch-${TORCH}+${CUDA}.html

```

--------------------------------

### Enable GPU Support for JAX

Source: https://nichecompass.readthedocs.io/en/latest/installation.html

Installs JAX with GPU support. Replace `${CUDA}` with your major CUDA version (e.g., 12).

```bash
uv pip install jax[cuda${CUDA}]

```

--------------------------------

### Display Model Initialization Parameters

Source: https://nichecompass.readthedocs.io/en/latest/tutorials/notebooks/mouse_cns_sample_integration.html

This output shows the configuration parameters used when initializing the NicheCompass model, including loss functions, node label methods, and network architecture details. It's useful for verifying model setup.

```text
--- INITIALIZING NEW NETWORK MODULE: VARIATIONAL GENE PROGRAM GRAPH AUTOENCODER ---
LOSS -> include_edge_recon_loss: True, include_gene_expr_recon_loss: True, rna_recon_loss: nb
NODE LABEL METHOD -> one-hop-norm
ACTIVE GP THRESHOLD RATIO -> 0.01
LOG VARIATIONAL -> True
CATEGORICAL COVARIATES EMBEDDINGS INJECTION -> ['gene_expr_decoder']
ONE HOP GCN NORM RNA NODE LABEL AGGREGATOR
ENCODER -> n_input: 1022, n_cat_covariates_embed_input: 0, n_hidden: 297, n_latent: 197, n_addon_latent: 100, n_fc_layers: 1, n_layers: 1, conv_layer: gcnconv, n_attention_heads: 0, dropout_rate: 0.0, 
COSINE SIM GRAPH DECODER -> dropout_rate: 0.0
MASKED TARGET RNA DECODER -> n_prior_gp_input: 197, n_addon_gp_input: 100, n_cat_covariates_embed_input: 3, n_output: 1022
MASKED SOURCE RNA DECODER -> n_prior_gp_input: 197, n_addon_gp_input: 100, n_cat_covariates_embed_input: 3, n_output: 1022

```

--------------------------------

### Install Bedtools using Conda

Source: https://nichecompass.readthedocs.io/en/latest/installation.html

Installs bedtools version 2.30.0 from the bioconda channel using conda.

```bash
conda install bedtools=2.30.0 -c bioconda

```

--------------------------------

### Display Example OmniPath GP

Source: https://nichecompass.readthedocs.io/en/latest/tutorials/notebooks/mouse_cns_single_sample.html

Displays a random example Gene Program (GP) name and its associated genes from the OmniPath dictionary.

```python
# Display example OmniPath GP
omnipath_gp_names = list(omnipath_gp_dict.keys())
random.shuffle(omnipath_gp_names)
omnipath_gp_name = omnipath_gp_names[0]
print(f"{omnipath_gp_name}: {omnipath_gp_dict[omnipath_gp_name]}")

```

--------------------------------

### Display Example MEBOCOST GP

Source: https://nichecompass.readthedocs.io/en/latest/tutorials/notebooks/mouse_cns_single_sample.html

Presents a random Gene Program (GP) name and its genes from the MEBOCOST dictionary.

```python
# Display example MEBOCOST GP
mebocost_gp_names = list(mebocost_gp_dict.keys())
random.shuffle(mebocost_gp_names)
mebocost_gp_name = mebocost_gp_names[0]
print(f"{mebocost_gp_name}: {mebocost_gp_dict[mebocost_gp_name]}")

```

--------------------------------

### Display Example NicheNet GP

Source: https://nichecompass.readthedocs.io/en/latest/tutorials/notebooks/mouse_cns_single_sample.html

Shows a randomly selected Gene Program (GP) name and its genes from the NicheNet dictionary.

```python
# Display example NicheNet GP
nichenet_gp_names = list(nichenet_gp_dict.keys())
random.shuffle(nichenet_gp_names)
nichenet_gp_name = nichenet_gp_names[0]
print(f"{nichenet_gp_name}: {nichenet_gp_dict[nichenet_gp_name]}")

```

--------------------------------

### Display Example Active GPs

Source: https://nichecompass.readthedocs.io/en/latest/tutorials/notebooks/mouse_cns_single_sample.html

Retrieves and displays a summary of the active gene programs (GPs). This snippet shows the first few active GPs, useful for initial exploration.

```python
# Display example active GPs
gp_summary_df = model.get_gp_summary()
gp_summary_df[gp_summary_df["gp_active"] == True].head()
```

--------------------------------

### Create and Activate Python Virtual Environment

Source: https://nichecompass.readthedocs.io/en/latest/installation.html

Sets up a new virtual environment and activates it using the uv package manager. Replace `${/path/to/new/virtual/environment}` with your desired path.

```bash
python3 -m venv ${/path/to/new/virtual/environment}
source ${/path/to/new/virtual/environment}/bin/activate
pip install uv

```

--------------------------------

### Initialize Paths and Create Figure Directory

Source: https://nichecompass.readthedocs.io/en/latest/tutorials/notebooks/mouse_cns_spatial_reference_mapping.html

Sets up necessary folder paths for figures and model loading, and creates the figure directory if it doesn't exist. This is a prerequisite for subsequent analysis steps.

```python
load_timestamp = "22082024_004645"

figure_folder_path = f"{artifacts_folder_path}/spatial_reference_mapping/{load_timestamp}/figures"
model_folder_path = f"{artifacts_folder_path}/spatial_reference_mapping/{load_timestamp}/model" \
                     "/reference_query"

os.makedirs(figure_folder_path, exist_ok=True)

```

--------------------------------

### Initialize Model Training

Source: https://nichecompass.readthedocs.io/en/latest/tutorials/notebooks/mouse_cns_spatial_reference_mapping.html

Set up the model path for loading a pre-trained reference model. This step prepares for transfer learning with new query data.

```python
load_timestamp = "22082024_004645"
# load_timestamp = current_timestamp # uncomment if you trained the model in this notebook

model_folder_path = f"{artifacts_folder_path}/spatial_reference_mapping/{load_timestamp}/model"

```

--------------------------------

### Get Unique Query Samples

Source: https://nichecompass.readthedocs.io/en/latest/tutorials/notebooks/mouse_cns_spatial_reference_mapping.html

Retrieves a list of unique sample identifiers from the query data.

```python
samples_query = adata_query.obs[sample_key].unique().tolist()

```

--------------------------------

### Get Unique Samples

Source: https://nichecompass.readthedocs.io/en/latest/tutorials/notebooks/mouse_cns_sample_integration.html

Retrieves a list of unique sample identifiers from the AnnData object's observation metadata.

```python
samples = adata.obs[sample_key].unique().tolist()
```

--------------------------------

### load

Source: https://nichecompass.readthedocs.io/en/latest/api/generated/nichecompass.models.NicheCompass.html

Instantiates a model from saved output.

```APIDOC
## load

### Description
Instantiates a model from saved output.

### Method
N/A (Method of a class)

### Endpoint
N/A

### Parameters
#### Path Parameters
- **dir_path** (any) - Description not explicitly documented, likely the directory path where the model is saved.
#### Query Parameters
- **adata** (any, optional) - Description not explicitly documented.
- **adata_atac** (any, optional) - Description not explicitly documented.
- **...** (any, optional) - Additional parameters not explicitly documented.

### Response
None explicitly documented.
```

--------------------------------

### Download Sample Data

Source: https://nichecompass.readthedocs.io/en/latest/tutorials/notebooks/mouse_cns_single_sample.html

Downloads the 'starmap_plus_mouse_cns_batch1.h5ad' dataset from a Google Drive URL into the specified spatial omics data folder. This step is optional if data is already available.

```python
gdown.download("https://drive.google.com/uc?id=1MOjIyue7a-JDAcnAseqIljDyoO7KtH99", so_data_folder_path+"/starmap_plus_mouse_cns_batch1.h5ad")

```

--------------------------------

### Download Sample Data

Source: https://nichecompass.readthedocs.io/en/latest/tutorials/notebooks/mouse_cns_sample_integration.html

Downloads the starmap plus mouse CNS dataset files (batch1, batch2, batch3) from Google Drive using gdown. This is optional if data is already available.

```python
gdown.download("https://drive.google.com/uc?id=1MOjIyue7a-JDAcnAseqIljDyoO7KtH99", so_data_folder_path+'/starmap_plus_mouse_cns_batch1.h5ad')
gdown.download("https://drive.google.com/uc?id=1_RcLVuZcJiFw-iaB7saPX4ydR1X2CvaS", so_data_folder_path+'/starmap_plus_mouse_cns_batch2.h5ad')
gdown.download("https://drive.google.com/uc?id=1sIIHGZ55aYBbgCXCBvIrGxB7i7OgUuJ9", so_data_folder_path+'/starmap_plus_mouse_cns_batch3.h5ad')

```

--------------------------------

### Get Current Timestamp

Source: https://nichecompass.readthedocs.io/en/latest/tutorials/notebooks/mouse_cns_single_sample.html

Captures the current date and time to be used for timestamping saved artifacts, ensuring unique naming for outputs.

```python
# Get time of notebook execution for timestamping saved artifacts
now = datetime.now()
current_timestamp = now.strftime("%d%m%Y_%H%M%S")

```

--------------------------------

### Define Dataset and Model Parameters

Source: https://nichecompass.readthedocs.io/en/latest/tutorials/notebooks/mouse_cns_spatial_reference_mapping.html

Sets up parameters for the dataset, model architecture, and training process, including AnnData keys and regularization weights.

```python
### Dataset ###
dataset = "starmap_plus_mouse_cns"
species = "mouse"
reference_batches = ["batch1", "batch2"]
query_batches = ["batch3"]
spatial_key = "spatial"
n_neighbors = 4
mapping_entity_key = "mapping_entity"

### Model ###
# AnnData keys
counts_key = "counts"
adj_key = "spatial_connectivities"
cat_covariates_keys = ["batch"]
gp_names_key = "nichecompass_gp_names"
active_gp_names_key = "nichecompass_active_gp_names"
gp_targets_mask_key = "nichecompass_gp_targets"
gp_targets_categories_mask_key = "nichecompass_gp_targets_categories"
gp_sources_mask_key = "nichecompass_gp_sources"
gp_sources_categories_mask_key = "nichecompass_gp_sources_categories"
latent_key = "nichecompass_latent"

# Architecture
cat_covariates_embeds_injection = ["gene_expr_decoder"]
cat_covariates_embeds_nums = [3]
cat_covariates_no_edges = [True]
conv_layer_encoder = "gcnconv" # change to "gatv2conv" if enough compute and memory
active_gp_thresh_ratio = 0.01

# Trainer
n_epochs = 400
n_epochs_all_gps = 25
lr = 0.001
lambda_edge_recon = 500000.
lambda_gene_expr_recon = 300.
lambda_l1_masked = 0. # prior GP  regularization
lambda_l1_addon = 30. # de novo GP regularization
edge_batch_size = 4096 # increase if more memory available or decrease to save memoryle
n_sampled_neighbors = 4
use_cuda_if_available = True

### Analysis ###
cell_type_key = "Main_molecular_cell_type"
latent_leiden_resolution = 0.3
latent_cluster_key = f"latent_leiden_{str(latent_leiden_resolution)}"
sample_key = "batch"
spot_size = 0.2
differential_gp_test_results_key = "nichecompass_differential_gp_test_results"

```

--------------------------------

### Display Example CollecTRI GP

Source: https://nichecompass.readthedocs.io/en/latest/tutorials/notebooks/mouse_brain_multimodal.html

Illustrates a random Gene Program (GP) from the CollecTRI dictionary. Useful for inspecting the format of CollecTRI-derived GPs.

```python
# Display example CollecTRI GP
collectri_gp_names = list(collectri_gp_dict.keys())
random.shuffle(collectri_gp_names)
collectri_gp_name = collectri_gp_names[0]
print(f"{collectri_gp_name}: {collectri_gp_dict[collectri_gp_name]}")

```

--------------------------------

### Set up artifact paths for multimodal analysis

Source: https://nichecompass.readthedocs.io/en/latest/tutorials/notebooks/mouse_brain_multimodal.html

Configures the file paths for figures and the trained model based on a load timestamp. Ensures the figure directory exists.

```python
load_timestamp = "22082024_142839"
# load_timestamp = current_timestamp # uncomment if you trained the model in this notebook

figure_folder_path = f"{artifacts_folder_path}/multimodal/{load_timestamp}/figures"
model_folder_path = f"{artifacts_folder_path}/multimodal/{load_timestamp}/model"

os.makedirs(figure_folder_path, exist_ok=True)
```

--------------------------------

### Get Unique Reference Samples

Source: https://nichecompass.readthedocs.io/en/latest/tutorials/notebooks/mouse_cns_spatial_reference_mapping.html

Retrieves a list of unique sample identifiers from the reference AnnData object. This is a preparatory step for iterating through individual samples.

```python
samples_reference = adata_reference.obs[sample_key].unique().tolist()

```

--------------------------------

### Get unique samples from AnnData

Source: https://nichecompass.readthedocs.io/en/latest/tutorials/notebooks/mouse_brain_multimodal.html

Extracts a list of unique sample identifiers from the 'obs' DataFrame of the AnnData object within the loaded NicheCompass model.

```python
samples = model.adata.obs[sample_key].unique().tolist()
```

--------------------------------

### Create Output Directories

Source: https://nichecompass.readthedocs.io/en/latest/tutorials/notebooks/mouse_cns_single_sample.html

Ensures that the necessary directories for storing model artifacts, figures, and spatial omics data are created. If directories already exist, they are not recreated.

```python
os.makedirs(model_folder_path, exist_ok=True)
os.makedirs(figure_folder_path, exist_ok=True)
os.makedirs(so_data_folder_path, exist_ok=True)

```

--------------------------------

### Explore and Visualize Query Data by Sample

Source: https://nichecompass.readthedocs.io/en/latest/tutorials/notebooks/mouse_cns_spatial_reference_mapping.html

Iterates through each unique sample in the query data, prints summary statistics, and visualizes the spatial distribution of cell types.

```python
for sample in samples_query:
    adata_batch = adata_query[adata_query.obs[sample_key] == sample]
    
    print(f"Summary of sample {sample}:")
    print(f"Number of nodes (observations): {adata_batch.layers[counts_key].shape[0]}")
    print(f"Number of node features (genes): {adata_batch.layers[counts_key].shape[1]}")

    # Visualize cell-level annotated data in physical space
    sc.pl.spatial(adata_batch,
                  color=cell_type_key,
                  palette=cell_type_colors,
                  spot_size=spot_size)

```

--------------------------------

### Get Unique Samples from Model Data

Source: https://nichecompass.readthedocs.io/en/latest/tutorials/notebooks/mouse_cns_single_sample.html

Extracts a list of unique sample identifiers from the annotated data (adata) within the loaded NicheCompass model.

```python
samples = model.adata.obs[sample_key].unique().tolist()

```

--------------------------------

### Create Conda Environment from File

Source: https://nichecompass.readthedocs.io/en/latest/installation.html

Creates a conda environment using a provided YAML file, which includes external libraries for PyTorch 2.0.0 and CUDA 11.7.

```bash
conda env create -f environment.yaml

```

--------------------------------

### Get Active Gene Programs Summary

Source: https://nichecompass.readthedocs.io/en/latest/tutorials/notebooks/mouse_cns_spatial_reference_mapping.html

Retrieves and prints the total number of gene programs and the number of currently active gene programs. It also displays a summary of active GPs.

```python
# Check number of active GPs
active_gps = model.get_active_gps()
print(f"Number of total gene programs: {len(model.adata.uns[gp_names_key])}.")
print(f"Number of active gene programs: {len(active_gps)}.")


```

```python
# Display example active GPs
gp_summary_df = model.get_gp_summary()
gp_summary_df[gp_summary_df["gp_active"] == True].head()


```

--------------------------------

### Define Data and Artifact Paths

Source: https://nichecompass.readthedocs.io/en/latest/tutorials/notebooks/mouse_cns_sample_integration.html

Sets up the file paths for various data sources, including gene annotations, gene programs, spatial omics data, and network files, as well as output artifact directories.

```python
# Define paths
ga_data_folder_path = "../../../data/gene_annotations"
gp_data_folder_path = "../../../data/gene_programs"
so_data_folder_path = "../../../data/spatial_omics"
omnipath_lr_network_file_path = f"{gp_data_folder_path}/omnipath_lr_network.csv"
collectri_tf_network_file_path = f"{gp_data_folder_path}/collectri_tf_network_{species}.csv"
nichenet_lr_network_file_path = f"{gp_data_folder_path}/nichenet_lr_network_v2_{species}.csv"
nichenet_ligand_target_matrix_file_path = f"{gp_data_folder_path}/nichenet_ligand_target_matrix_v2_{species}.csv"
mebocost_enzyme_sensor_interactions_folder_path = f"{gp_data_folder_path}/metabolite_enzyme_sensor_gps"
gene_orthologs_mapping_file_path = f"{ga_data_folder_path}/human_mouse_gene_orthologs.csv"
artifacts_folder_path = f"../../../artifacts"
model_folder_path = f"{artifacts_folder_path}/sample_integration/{current_timestamp}/model"
figure_folder_path = f"{artifacts_folder_path}/sample_integration/{current_timestamp}/figures"

```

--------------------------------

### Initialize NicheCompass Model

Source: https://nichecompass.readthedocs.io/en/latest/tutorials/notebooks/mouse_cns_spatial_reference_mapping.html

Initializes the NicheCompass model with specified parameters. Ensure all required data keys and covariate information are correctly provided.

```python
# Initialize model
model = NicheCompass(adata_reference,
                     counts_key=counts_key,
                     adj_key=adj_key,
                     cat_covariates_embeds_injection=cat_covariates_embeds_injection,
                     cat_covariates_keys=cat_covariates_keys,
                     cat_covariates_no_edges=cat_covariates_no_edges,
                     cat_covariates_embeds_nums=cat_covariates_embeds_nums,
                     gp_names_key=gp_names_key,
                     active_gp_names_key=active_gp_names_key,
                     gp_targets_mask_key=gp_targets_mask_key,
                     gp_targets_categories_mask_key=gp_targets_categories_mask_key,
                     gp_sources_mask_key=gp_sources_mask_key,
                     gp_sources_categories_mask_key=gp_sources_categories_mask_key,
                     latent_key=latent_key,
                     conv_layer_encoder=conv_layer_encoder,
                     active_gp_thresh_ratio=active_gp_thresh_ratio)
```

--------------------------------

### Get Latent Representation

Source: https://nichecompass.readthedocs.io/en/latest/api/generated/nichecompass.modules.VGPGAE.html

Encodes input features into latent distribution parameters or latent features. Can optionally return only active gene programs and the distribution's mean and standard deviation.

```python
VGPGAE.get_latent_representation(_node_batch_ , _only_active_gps =True_, _return_mu_std =False_)
```

--------------------------------

### Create Data Directories

Source: https://nichecompass.readthedocs.io/en/latest/tutorials/notebooks/mouse_brain_multimodal.html

Ensures that the directories for storing the model, figures, and spatial omics data exist. Uses `os.makedirs` with `exist_ok=True` to prevent errors if directories already exist.

```python
os.makedirs(model_folder_path, exist_ok=True)
os.makedirs(figure_folder_path, exist_ok=True)
os.makedirs(so_data_folder_path, exist_ok=True)
```

--------------------------------

### Get Gene Program Weights

Source: https://nichecompass.readthedocs.io/en/latest/api/generated/nichecompass.modules.VGPGAE.html

Retrieves the gene program weights from the omics feature decoders. Returns a list of tensors, where each tensor contains the decoder weights for a specific omics modality.

```python
VGPGAE.get_gp_weights(_only_masked_features =False_, _gp_type ='all'_)
```

--------------------------------

### Get Active Gene Program Mask

Source: https://nichecompass.readthedocs.io/en/latest/api/generated/nichecompass.modules.VGPGAE.html

Determines a mask for active gene programs based on RNA decoder gene weights. Supports various aggregation modes for gene program weights and can optionally return the weights themselves.

```python
VGPGAE.get_active_gp_mask(_abs_gp_weights_agg_mode ='sum+nzmeans'_, _return_gp_weights =False_, _normalize_gp_weights_with_features_scale_factors =False_)
```

--------------------------------

### Initialize NicheCompass Model

Source: https://nichecompass.readthedocs.io/en/latest/tutorials/notebooks/mouse_cns_single_sample.html

Initializes the NicheCompass model with the provided AnnData object and various configuration keys. Ensure all specified keys exist in your AnnData object.

```python
model = NicheCompass(adata,
                     counts_key=counts_key,
                     adj_key=adj_key,
                     gp_names_key=gp_names_key,
                     active_gp_names_key=active_gp_names_key,
                     gp_targets_mask_key=gp_targets_mask_key,
                     gp_targets_categories_mask_key=gp_targets_categories_mask_key,
                     gp_sources_mask_key=gp_sources_mask_key,
                     gp_sources_categories_mask_key=gp_sources_categories_mask_key,
                     latent_key=latent_key,
                     conv_layer_encoder=conv_layer_encoder,
                     active_gp_thresh_ratio=active_gp_thresh_ratio)
```

--------------------------------

### Define Dataset and Model Parameters

Source: https://nichecompass.readthedocs.io/en/latest/tutorials/notebooks/mouse_cns_single_sample.html

Configures parameters for the dataset, including its name, species, and spatial key, as well as model-specific settings like AnnData keys, architecture choices, and trainer configurations.

```python
### Dataset ###
dataset = "starmap_plus_mouse_cns"
species = "mouse"
spatial_key = "spatial"
n_neighbors = 4

### Model ###
# AnnData Keys
counts_key = "counts"
adj_key = "spatial_connectivities"
gp_names_key = "nichecompass_gp_names"
active_gp_names_key = "nichecompass_active_gp_names"
gp_targets_mask_key = "nichecompass_gp_targets"
gp_targets_categories_mask_key = "nichecompass_gp_targets_categories"
gp_sources_mask_key = "nichecompass_gp_sources"
gp_sources_categories_mask_key = "nichecompass_gp_sources_categories"
latent_key = "nichecompass_latent"

# Architecture
conv_layer_encoder = "gcnconv" # change to "gatv2conv" if enough compute and memory
active_gp_thresh_ratio = 0.01

# Trainer
n_epochs = 400
n_epochs_all_gps = 25
lr = 0.001
lambda_edge_recon = 500000.
lambda_gene_expr_recon = 300.
lambda_l1_masked = 0. # prior GP  regularization
lambda_l1_addon = 30. # de novo GP regularization
edge_batch_size = 1024 # increase if more memory available or decrease to save memory
n_sampled_neighbors = 4
use_cuda_if_available = True

### Analysis ###
cell_type_key = "Main_molecular_cell_type"
latent_leiden_resolution = 0.4
latent_cluster_key = f"latent_leiden_{str(latent_leiden_resolution)}"
sample_key = "batch"
spot_size = 0.2
differential_gp_test_results_key = "nichecompass_differential_gp_test_results"

```

--------------------------------

### compute_cat_covariates_contrastive_loss

Source: https://nichecompass.readthedocs.io/en/latest/api/generated/nichecompass.modules.compute_cat_covariates_contrastive_loss.html

Computes categorical covariates contrastive weighted binary cross entropy loss with logits. It calculates the loss for each categorical covariate separately and sums them up. Positive and negative examples for contrastive learning are determined based on the provided ratios of edge reconstruction logits.

```APIDOC
## compute_cat_covariates_contrastive_loss

### Description
Compute categorical covariates contrastive weighted binary cross entropy loss with logits. The loss is computed for each categorical covariate separately and added up. Sampled edges with nodes from different categories whose edge reconstruction logits are among the top (´contrastive_logits_pos_ratio´ * 100)% logits are considered positive examples for a specific categorical covariate. Sampled edges with nodes from different categories whose edge reconstruction logits are among the bottom (´contrastive_logits_neg_ratio´ * 100)% logits are considered negative examples.

### Parameters
#### Positional Arguments
* **_edge_recon_logits_** (Tensor) – Predicted edge reconstruction logits for both positive and negative sampled edges (dim: 2 * edge_batch_size).
* **_edge_recon_labels_** (Tensor) – Edge ground truth labels for both positive and negative sampled edges (dim: 2 * edge_batch_size).

#### Keyword Arguments
* **_edge_same_cat_covariates_cat_** (Optional[List[Tensor]], default: None) – List of boolean tensors indicating whether the edge node pair has the same categorical covariate category for each categorical covariate respectively, and for both positive and negative sampled edges (dim of tensors: 2 * edge_batch_size).
* **_contrastive_logits_pos_ratio_** (float, default: 0.0) – Ratio for determining the logits threshold of positive contrastive examples of node pairs from different categories. The top (´contrastive_logits_pos_ratio´ * 100)% logits of node pairs from different categories serve as positive labels for the contrastive loss.
* **_contrastive_logits_neg_ratio_** (float, default: 0.0) – Ratio for determining the logits threshold of negative contrastive examples of node pairs from different categories. The bottom (´contrastive_logits_neg_ratio´ * 100)% logits of node pairs from different categories serve as negative labels for the contrastive loss.

### Returns
* **cat_covariates_contrastive_loss** (Tensor) – Categorical covariates contrastive binary cross entropy loss (calculated from logits for numerical stability in backpropagation, and summed up over all categorical covariates).
```

--------------------------------

### load

Source: https://nichecompass.readthedocs.io/en/latest/api/generated/nichecompass.models.NicheCompass.html

Class method to instantiate a NicheCompass model from saved output. Useful for transfer learning and de-novo gene program learning.

```APIDOC
## load

### Description
Instantiate a model from saved output. Can be used for transfer learning scenarios and to learn de-novo gene programs by adding add-on gene programs and freezing non add-on weights.

### Parameters
#### Path Parameters
- **dir_path** (str) - Required - Path to the directory where the model is saved.
- **adata** (Optional[AnnData]) - Optional - AnnData object.
- **adata_atac** (Optional[AnnData]) - Optional - AnnData object for ATAC data.
- **adata_file_name** (str) - Optional - File name for the adata object.
- **adata_atac_file_name** (str) - Optional - File name for the adata_atac object.
- **use_cuda** (bool) - Optional - Whether to use CUDA for computation.
- **n_addon_gps** (int) - Optional - Number of add-on gene programs.
- **gp_names_key** (str) - Optional - Key for gene program names.
- **genes_idx_key** (str) - Optional - Key for gene indices.
- **unfreeze_all_weights** (bool) - Optional - Whether to unfreeze all weights.
- **unfreeze_addon_gp_weights** (bool) - Optional - Whether to unfreeze add-on gene program weights.
- **unfreeze_cat_covariates_embedder_weights** (bool) - Optional - Whether to unfreeze categorical covariates embedder weights.
```

--------------------------------

### Import NicheCompass Library

Source: https://nichecompass.readthedocs.io/en/latest/api/index.html

Import the NicheCompass library for use in your Python project.

```python
import nichecompass

```

--------------------------------

### Explore Reference Data by Sample

Source: https://nichecompass.readthedocs.io/en/latest/tutorials/notebooks/mouse_cns_spatial_reference_mapping.html

Iterates through each unique sample in the reference data, prints summary statistics (number of nodes and features), and visualizes cell-level annotated data in physical space using spatial plots.

```python
for sample in samples_reference:
    adata_batch = adata_reference[adata_reference.obs[sample_key] == sample]
    
    print(f"Summary of sample {sample}:")
    print(f"Number of nodes (observations): {adata_batch.layers[counts_key].shape[0]}")
    print(f"Number of node features (genes): {adata_batch.layers[counts_key].shape[1]}")

    # Visualize cell-level annotated data in physical space
    sc.pl.spatial(adata_batch,
                  color=cell_type_key,
                  palette=cell_type_colors,
                  spot_size=spot_size)

```

--------------------------------

### Define Artifacts and Figure Paths

Source: https://nichecompass.readthedocs.io/en/latest/tutorials/notebooks/mouse_cns_single_sample.html

Sets up the directory paths for storing model artifacts and generated figures based on a load timestamp. Ensures the figure directory exists.

```python
load_timestamp = "21082024_235239"
# load_timestamp = current_timestamp # uncomment if you trained the model in this notebook

figure_folder_path = f"{artifacts_folder_path}/single_sample/{load_timestamp}/figures"
model_folder_path = f"{artifacts_folder_path}/single_sample/{load_timestamp}/model"

os.makedirs(figure_folder_path, exist_ok=True)

```

--------------------------------

### Model Initialization Summary

Source: https://nichecompass.readthedocs.io/en/latest/tutorials/notebooks/mouse_brain_multimodal.html

This output details the configuration of a new Variational Gene Program Graph Autoencoder module, including loss functions, normalization methods, and network architecture parameters.

```text
--- INITIALIZING NEW NETWORK MODULE: VARIATIONAL GENE PROGRAM GRAPH AUTOENCODER ---
LOSS -> include_edge_recon_loss: True, include_gene_expr_recon_loss: True, rna_recon_loss: nb, include_chrom_access_recon_loss: True, atac_recon_loss: nb 
NODE LABEL METHOD -> one-hop-norm
ACTIVE GP THRESHOLD RATIO -> 0.01
LOG VARIATIONAL -> True
ONE HOP GCN NORM RNA NODE LABEL AGGREGATOR
ONE HOP GCN NORM ATAC NODE LABEL AGGREGATOR
ENCODER -> n_input: 6069, n_cat_covariates_embed_input: 0, n_hidden: 1996, n_latent: 1896, n_addon_latent: 100, n_fc_layers: 1, n_layers: 1, conv_layer: gatv2conv, n_attention_heads: 4, dropout_rate: 0.0, 
COSINE SIM GRAPH DECODER -> dropout_rate: 0.0
MASKED TARGET RNA DECODER -> n_prior_gp_input: 1896, n_addon_gp_input: 100, n_cat_covariates_embed_input: 0, n_output: 2773
MASKED SOURCE RNA DECODER -> n_prior_gp_input: 1896, n_addon_gp_input: 100, n_cat_covariates_embed_input: 0, n_output: 2773
MASKED TARGET ATAC DECODER -> n_prior_gp_input: 1896, n_addon_gp_input: 100, n_cat_covariates_embed_input: 0, n_output: 3296
MASKED SOURCE ATAC DECODER -> n_prior_gp_input: 1896, n_addon_gp_input: 100, n_cat_covariates_embed_input: 0, n_output: 3296
```

--------------------------------

### Define Dataset and Model Parameters

Source: https://nichecompass.readthedocs.io/en/latest/tutorials/notebooks/mouse_brain_multimodal.html

Configures parameters for dataset loading, NicheCompass model architecture, training, and subsequent analysis, including gene/peak filtering thresholds and CUDA usage.

```python
### Dataset ###
dataset = "spatial_atac_rna_seq_mouse_brain"
species = "mouse"
spatial_key = "spatial"
n_neighbors = 4
n_sampled_neighbors = 4
filter_genes = True
n_svg = 3000
n_svp = 15000
filter_peaks = True
min_cell_peak_thresh_ratio = 0.005 # 0.05%
min_cell_gene_thresh_ratio = 0.005 # 0.05%

### Model ###
# AnnData keys
counts_key = "counts"
adj_key = "spatial_connectivities"
gp_names_key = "nichecompass_gp_names"
active_gp_names_key = "nichecompass_active_gp_names"
gp_targets_mask_key = "nichecompass_gp_targets"
gp_targets_categories_mask_key = "nichecompass_gp_targets_categories"
gp_sources_mask_key = "nichecompass_gp_sources"
gp_sources_categories_mask_key = "nichecompass_gp_sources_categories"
latent_key = "nichecompass_latent"

# Architecture
active_gp_thresh_ratio = 0.01
conv_layer_encoder = "gatv2conv" # change to "gcnconv" if not enough compute and memory

# Trainer
n_epochs = 400
n_epochs_all_gps = 25
lr = 0.001
lambda_edge_recon = 500000.
lambda_gene_expr_recon = 300.
lambda_chrom_access_recon = 300.
lambda_l1_masked = 0. # prior GP  regularization
lambda_l1_addon = 30. # de novo GP regularization
edge_batch_size = 256 # increase if more memory available or decrease to save memory
use_cuda_if_available = True

### Analysis ###
cell_type_key = "cell_type"
latent_leiden_resolution = 0.6
latent_cluster_key = f"latent_leiden_{str(latent_leiden_resolution)}"
sample_key = "batch"
spot_size = 30
differential_gp_test_results_key = "nichecompass_differential_gp_test_results"

```

--------------------------------

### Import Core Libraries

Source: https://nichecompass.readthedocs.io/en/latest/tutorials/notebooks/mouse_cns_sample_integration.html

Imports essential Python libraries for data analysis, machine learning, and visualization, including Anndata, Scanpy, NicheCompass, and others.

```python
import os
import random
import warnings
from datetime import datetime

import anndata as ad
import gdown
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import scanpy as sc
import scipy.sparse as sp
import seaborn as sns
import squidpy as sq
from matplotlib import gridspec
from sklearn.preprocessing import MinMaxScaler

from nichecompass.models import NicheCompass
from nichecompass.utils import (
    add_gps_from_gp_dict_to_adata,
    create_new_color_dict,
    compute_communication_gp_network,
    visualize_communication_gp_network,
    extract_gp_dict_from_mebocost_ms_interactions,
    extract_gp_dict_from_nichenet_lrt_interactions,
    extract_gp_dict_from_omnipath_lr_interactions,
    filter_and_combine_gp_dict_gps_v2,
    generate_enriched_gp_info_plots
)

```

--------------------------------

### Load Fine-tuned Query Model

Source: https://nichecompass.readthedocs.io/en/latest/tutorials/notebooks/mouse_cns_spatial_reference_mapping.html

Loads a fine-tuned NicheCompass query model with integrated reference and query data. Ensure the model folder path and adata file name are correctly specified.

```python
# Load query model with the integrated data
print("Retrieving query model...")
model = NicheCompass.load(
    dir_path=f"{model_folder_path}/query",
    adata=adata_reference_query,
    adata_file_name="adata.h5ad",
    gp_names_key=gp_names_key)

```

--------------------------------

### Configure Warnings and Display Options

Source: https://nichecompass.readthedocs.io/en/latest/tutorials/notebooks/mouse_brain_multimodal.html

Suppresses warnings and sets pandas to display all columns for better data inspection during analysis.

```python
warnings.filterwarnings("ignore")
pd.set_option("display.max_columns", None)

```

--------------------------------

### Trainer Class Initialization

Source: https://nichecompass.readthedocs.io/en/latest/api/generated/nichecompass.train.Trainer.html

Initializes the Trainer class with AnnData objects, a model instance, and various training configuration parameters.

```APIDOC
## class nichecompass.train.Trainer

_class _nichecompass.train.Trainer(_adata_ , _model_ , _adata_atac =None_, _counts_key ='counts'_, _adj_key ='spatial_connectivities'_, _cat_covariates_keys =None_, _gp_targets_mask_key ='nichecompass_gp_targets'_, _gp_sources_mask_key ='nichecompass_gp_sources'_, _edge_val_ratio =0.1_, _node_val_ratio =0.1_, _edge_batch_size =512_, _node_batch_size =None_, _n_sampled_neighbors =-1_, _use_early_stopping =True_, _reload_best_model =True_, _early_stopping_kwargs =None_, _use_cuda_if_available =True_, _seed =0_, _monitor =True_, _verbose =False_, _** kwargs_)

Trainer class. Encapsulates all logic for NicheCompass model training preparation and model training.

Parameters:
    
  * **adata** (`AnnData`) – AnnData object with counts stored in ´adata.layers[counts_key]´ or ´adata.X´ depending on ´counts_key´ and sparse adjacency matrix stored in ´adata.obsp[adj_key]´.
  * **adata_atac** (`Optional`[`AnnData`] (default: `None`)) – Additional optional AnnData object with paired spatial ATAC data.
  * **model** (`Module`) – An NicheCompass module model instance.
  * **counts_key** (`Optional`[`str`] (default: `'counts'`)) – Key under which the counts are stored in ´adata.layer´. If ´None´, uses ´adata.X´ as counts.
  * **adj_key** (`str` (default: `'spatial_connectivities'`)) – Key under which the sparse adjacency matrix is stored in ´adata.obsp´.
  * **cat_covariates_keys** (`Optional`[`List`[`str`]] (default: `None`)) – Keys under which the categorical covariates are stored in ´adata.obs´.
  * **gp_targets_mask_key** (`str` (default: `'nichecompass_gp_targets'`)) – Key under which the gene program targets mask is stored in ´model.adata.varm´. This mask will only be used if no ´gp_targets_mask´ is passed explicitly to the model.
  * **gp_sources_mask_key** (`str` (default: `'nichecompass_gp_sources'`)) – Key under which the gene program sources mask is stored in ´model.adata.varm´. This mask will only be used if no ´gp_sources_mask´ is passed explicitly to the model.
  * **edge_val_ratio** (`float` (default: `0.1`)) – Fraction of the data that is used as validation set on edge-level. The rest of the data will be used as training set on edge-level.
  * **node_val_ratio** (`float` (default: `0.1`)) – Fraction of the data that is used as validation set on node-level. The rest of the data will be used as training set on edge-level.
  * **edge_batch_size** (`int` (default: `512`)) – Batch size for the edge-level dataloaders.
  * **node_batch_size** (`Optional`[`int`] (default: `None`)) – Batch size for the node-level dataloaders.
  * **n_sampled_neighbors** (`int` (default: `-1`)) – Number of neighbors that are sampled during model training from the spatial neighborhood graph.
  * **use_early_stopping** (`bool` (default: `True`)) – If `True`, the EarlyStopping class is used to prevent overfitting.
  * **reload_best_model** (`bool` (default: `True`)) – If `True`, the best state of the model with respect to the early stopping criterion is reloaded at the end of training.
  * **early_stopping_kwargs** (`Optional`[`dict`] (default: `None`)) – Kwargs for the EarlyStopping class.
  * **use_cuda_if_available** (`bool` (default: `True`)) – If `True`, use cuda if available.
  * **seed** (`int` (default: `0`)) – Random seed to get reproducible results.
  * **monitor** (`bool` (default: `True`)) – If ´True´, the progress of training will be printed after each epoch.
  * **verbose** (`bool` (default: `False`)) – If ´True´, print out detailed training progress of individual losses.
```

--------------------------------

### NicheCompass.load

Source: https://nichecompass.readthedocs.io/en/latest/api/generated/nichecompass.models.NicheCompass.html

Loads a pre-trained NicheCompass model from a specified directory. It allows for flexible loading of AnnData objects and can utilize GPU acceleration if available.

```APIDOC
## NicheCompass.load

### Description
Loads a NicheCompass model from a directory, optionally loading associated AnnData objects and configuring GPU usage.

### Method
load

### Parameters
#### Path Parameters
* **dir_path** (str) – Path to saved outputs.

#### Optional Parameters
* **adata** (Optional[AnnData]) – AnnData organized in the same way as data used to train the model. If None, will check for and load adata saved with the model.
* **adata_atac** (Optional[AnnData]) – ATAC AnnData organized in the same way as data used to train the model. If None and adata_atac_file_name is not None, will check for and load adata_atac saved with the model.
* **adata_file_name** (str, default: 'adata.h5ad') – File name of the AnnData object to be loaded.
* **adata_atac_file_name** (Optional[str]) – File name of the ATAC AnnData object to be loaded.
* **use_cuda** (bool, default: False) – If True, load model on GPU.
* **n_addon_gps** (int, default: 0) – Number of (new) add-on gene programs to be added to the model’s architecture.
* **gp_names_key** (Optional[str]) – Key under which the gene program names are stored in adata.uns.
* **unfreeze_all_weights** (bool, default: False) – If True, unfreeze all weights.
* **unfreeze_addon_gp_weights** (bool, default: False) – If True, unfreeze addon gp weights.
* **unfreeze_cat_covariates_embedder_weights** (bool, default: False) – If True, unfreeze categorical covariates embedder weights.

### Returns
* **model** (Module) – Model with loaded state dictionaries and, if specified, frozen non add-on weights.
```

--------------------------------

### Trainer.train()

Source: https://nichecompass.readthedocs.io/en/latest/api/generated/nichecompass.train.Trainer.html

Train the NicheCompass model with various configurable parameters.

```APIDOC
## Trainer.train()

### Description
Train the NicheCompass model.

### Parameters
- **n_epochs** (int, default: `100`) - Number of epochs.
- **n_epochs_all_gps** (int, default: `25`) - Number of epochs during which all gene programs are used for model training. After that only active gene programs are retained.
- **n_epochs_no_edge_recon** (int, default: `0`) - Number of epochs without edge reconstruction loss for gene expression decoder pretraining.
- **n_epochs_no_cat_covariates_contrastive** (int, default: `5`) - Number of epochs for categorical covariates contrastive loss.
- **lr** (float, default: `0.001`) - Learning rate.
- **weight_decay** (float, default: `0.0`) - Weight decay (L2 penalty).
- **lambda_edge_recon** (Optional[float], default: `500000.0`) - Lambda (weighting factor) for the edge reconstruction loss. If >0, this will enforce gene programs to be meaningful for edge reconstruction and, hence, to preserve spatial colocalization information.
- **lambda_cat_covariates_contrastive** (Optional[float], default: `0.0`) - Lambda (weighting factor) for the categorical covariates contrastive loss. If >0, this will enforce observations with different categorical covariates categories with very similar latent representations to become more similar, and observations with different latent representations to become more different.
- **contrastive_logits_pos_ratio** (Optional[float], default: `0.125`) - Ratio for determining the logits threshold of positive contrastive examples of node pairs from different categorical covariates categories. The top (contrastive_logits_pos_ratio * 100)% logits of node pairs from different categorical covariates categories serve as positive labels for the contrastive loss.
- **contrastive_logits_neg_ratio** (Optional[float], default: `0.125`) - Ratio for determining the logits threshold of negative contrastive examples of node pairs from different categorical covariates categories. The bottom (contrastive_logits_neg_ratio * 100)% logits of node pairs from different categorical covariates categories serve as negative labels for the contrastive loss.
- **lambda_gene_expr_recon** (float, default: `100.0`) - Lambda (weighting factor) for the gene expression reconstruction loss. If >0, this will enforce interpretable gene programs that can be combined in a linear way to reconstruct gene expression.
- **lambda_chrom_access_recon** (float, default: `10.0`) - Lambda (weighting factor) for the chromatin accessibility reconstruction loss. If >0, this will enforce interpretable gene programs that can be combined in a linear way to reconstruct chromatin accessibility.
- **lambda_group_lasso** (float, default: `0.0`) - Lambda (weighting factor) for the group lasso regularization loss of gene programs. If >0, this will enforce sparsity of gene programs.
- **lambda_l1_masked** (float, default: `0.0`) - Lambda (weighting factor) for the L1 regularization loss of genes in masked gene programs. If >0, this will enforce sparsity of genes in masked gene programs.
- **l1_targets_mask** (Optional[Tensor], default: `None`) - Boolean gene program gene mask that is True for all gene program target genes to which the L1 regularization loss should be applied (dim: n_genes, n_gps).
- **l1_sources_mask** (Optional[Tensor], default: `None`) - Boolean gene program gene mask that is True for all gene program source genes to which the L1 regularization loss should be applied (dim: n_genes, n_gps).
- **lambda_l1_addon** (float, default: `0.0`) - Lambda (weighting factor) for the L1 regularization loss of genes in addon gene programs. If >0, this will enforce sparsity of genes in addon gene programs.
- **mlflow_experiment_id** (Optional[str], default: `None`) - ID of the mlflow experiment that will be used for tracking.

### Method
Not specified (likely internal or called by Trainer.train)

### Endpoint
Not applicable (SDK method)
```

--------------------------------

### Load Reference Model for Transfer Learning

Source: https://nichecompass.readthedocs.io/en/latest/tutorials/notebooks/mouse_cns_spatial_reference_mapping.html

Load a pre-trained reference model using NicheCompass. The model is initialized with query data, and only covariate embedding weights are unfrozen for fine-tuning.

```python
# Loading reference model with query data
print("Retrieving reference model...")
model = NicheCompass.load(
    dir_path=f"{model_folder_path}/reference",
    adata=adata_query,
    adata_file_name="adata.h5ad",
    gp_names_key=gp_names_key,
    unfreeze_all_weights=False,
    unfreeze_cat_covariates_embedder_weights=True)

```

--------------------------------

### save

Source: https://nichecompass.readthedocs.io/en/latest/api/generated/nichecompass.models.NicheCompass.html

Saves the model to disk.

```APIDOC
## save

### Description
Saves the model to disk. Note that the Trainer optimizer state is not saved.

### Method
N/A (Method of a class)

### Endpoint
N/A

### Parameters
#### Path Parameters
- **dir_path** (any) - Description not explicitly documented, likely the directory path to save the model.
#### Query Parameters
- **overwrite** (boolean, optional) - Description not explicitly documented.
- **save_adata** (boolean, optional) - Description not explicitly documented.
- **...** (any, optional) - Additional parameters not explicitly documented.

### Response
None explicitly documented.
```
Tokens:
Raw