### Install MultiVeloVAE via pip Source: https://github.com/welch-lab/multivelovae/blob/main/README.md Use this command to install the package from PyPI. ```bash pip install multivelovae ``` -------------------------------- ### Importing Libraries Source: https://github.com/welch-lab/multivelovae/blob/main/paper-notebooks/preprocess/example_prepro_hspc_starsolo.ipynb Initial setup of required data analysis and visualization libraries. ```python import csv import numpy as np import pandas as pd import scanpy as sc import scvelo as scv import matplotlib.pyplot as plt import multivelovae as vv ``` -------------------------------- ### Install MultiVeloVAE via pip Source: https://github.com/welch-lab/multivelovae/blob/main/docs/usage.md Use this command to install the package in your Python environment. ```console $ pip install multivelovae ``` -------------------------------- ### Initialize Environment and Imports Source: https://github.com/welch-lab/multivelovae/blob/main/paper-notebooks/mvv_mouse_brain_scRNA.ipynb Import necessary libraries for data analysis, visualization, and deep learning. ```python import numpy as np import scanpy as sc import scvelo as scv import torch import matplotlib.pyplot as plt import umap import os import multivelovae as vv from datetime import datetime import seaborn as sns import pandas as pd ``` -------------------------------- ### Import required libraries Source: https://github.com/welch-lab/multivelovae/blob/main/paper-notebooks/mvv_hspc_mix_integration.ipynb Initializes the environment with necessary data analysis and machine learning libraries. ```python import numpy as np import anndata as ad import scanpy as sc import scvelo as scv import torch from sklearn.decomposition import PCA import umap import os import multivelovae as vv from datetime import datetime ``` -------------------------------- ### Import required libraries Source: https://github.com/welch-lab/multivelovae/blob/main/paper-notebooks/mvv_hspc_macrophage_integration.ipynb Initializes the environment by importing necessary data science and multivelovae modules. ```python import numpy as np import scanpy as sc import scvelo as scv import torch import matplotlib.pyplot as plt import umap import os import multivelovae as vv from datetime import datetime import seaborn as sns import pandas as pd import plotnine as p9 ``` -------------------------------- ### Importing Libraries Source: https://github.com/welch-lab/multivelovae/blob/main/paper-notebooks/preprocess/example_prepro_hspc_macrophage_merge.ipynb Initial imports for data manipulation, single-cell analysis, and multivelovae integration. ```python import numpy as np import pandas as pd import scanpy as sc import scvelo as scv import multivelovae as vv ``` -------------------------------- ### Model initialization output Source: https://github.com/welch-lab/multivelovae/blob/main/paper-notebooks/mvv_EB_multiome.ipynb Console output indicating the configuration parameters used during model initialization. ```text Latent dimension set to 16. Learning rate set to 9.1e-4 based on data sparsity. Early stop threshold set to 1.6. Using Gaussian Prior. Initializing using the steady-state and dynamical models. ``` -------------------------------- ### Initialize and train MultiVeloVAE Source: https://github.com/welch-lab/multivelovae/blob/main/paper-notebooks/mvv_mouse_brain_scRNA.ipynb Configures the VAEChrom model with specific parameters, trains it, and saves the resulting model and AnnData object. ```python torch.manual_seed(2022) np.random.seed(2022) model = vv.VAEChrom(adata, device='cuda:0', plot_init=False, gene_plot=gene_plot, cluster_key="clusters", figure_path=figure_path, embed="umap") model.train(plot=False, gene_plot=gene_plot, figure_path=figure_path, embed="umap") model.save_model(model_path) model.save_anndata(data_path, file_name="out.h5ad") ``` -------------------------------- ### Initialize and train VAEChrom model Source: https://github.com/welch-lab/multivelovae/blob/main/paper-notebooks/mvv_hspc_macrophage_integration.ipynb Configures the VAEChrom model with specific batch and regression parameters, trains the model, and saves the output. ```python torch.manual_seed(2022) np.random.seed(2022) model = vv.VAEChrom(adata, adata_atac, batch_key='batch', ref_batch=0, batch_hvg_key='highly_variable', var_to_regress=['total_unspliced3'], device='cuda:0', plot_init=False, gene_plot=gene_plot, cluster_key="leiden", figure_path=figure_path, embed="umap") model.train(plot=False, gene_plot=gene_plot, figure_path=figure_path, embed="umap") model.save_model(model_path) model.save_anndata(data_path, file_name="out.h5ad") ``` -------------------------------- ### Prepare knockout datasets Source: https://github.com/welch-lab/multivelovae/blob/main/paper-notebooks/mvv_hspc_mix_integration.ipynb Create copies of the layers and set the target gene expression to zero for knockout simulation. ```python c1 = adata_atac.layers['Mc'].copy() u1 = adata.layers['Mu'].copy() s1 = adata.layers['Ms'].copy() c2 = adata_atac.layers['Mc'].copy() u2 = adata.layers['Mu'].copy() s2 = adata.layers['Ms'].copy() ``` ```python c1[:, np.where(adata.var_names == 'SPI1')[0][0]] = 0 u1[:, np.where(adata.var_names == 'SPI1')[0][0]] = 0 s1[:, np.where(adata.var_names == 'SPI1')[0][0]] = 0 c2[:, np.where(adata.var_names == 'GATA1')[0][0]] = 0 u2[:, np.where(adata.var_names == 'GATA1')[0][0]] = 0 s2[:, np.where(adata.var_names == 'GATA1')[0][0]] = 0 ``` ```python spi1_ko_set = model.prepare_dataset(c1, u1, s1, only_full=True) gata1_ko_set = model.prepare_dataset(c2, u2, s2, only_full=True) ``` -------------------------------- ### Placeholder for Seurat integration Source: https://github.com/welch-lab/multivelovae/blob/main/paper-notebooks/preprocess/example_prepro_hspc_starsolo.ipynb Indicates the step for running external Seurat WNN analysis. ```python ### Run seurat_wnn.R ### ``` -------------------------------- ### Configuring Analysis Settings Source: https://github.com/welch-lab/multivelovae/blob/main/paper-notebooks/preprocess/example_prepro_hspc_macrophage_merge.ipynb Sets global verbosity, display options, and figure parameters for scvelo and pandas. ```python scv.settings.verbosity = 3 scv.settings.presenter_view = True scv.set_figure_params('scvelo') pd.set_option('display.max_columns', 100) pd.set_option('display.max_rows', 200) np.set_printoptions(suppress=True) ``` -------------------------------- ### Reperform Preprocessing Steps Source: https://github.com/welch-lab/multivelovae/blob/main/paper-notebooks/preprocess/example_prepro_hspc_macrophage_merge.ipynb Applies log transformation, regression of cell cycle differences, scaling, and moment computation for RNA velocity analysis. ```python # Reperform several preprocessing steps sc.pp.log1p(adata_) sc.pp.regress_out(adata_, ['CC_difference']) sc.pp.scale(adata_) scv.pp.moments(adata_, n_pcs=30, n_neighbors=30) sc.pp.log1p(adata2_) sc.pp.regress_out(adata2_, ['CC_difference']) sc.pp.scale(adata2_) scv.pp.moments(adata2_, n_pcs=30, n_neighbors=30) ``` -------------------------------- ### Training logs Source: https://github.com/welch-lab/multivelovae/blob/main/paper-notebooks/mvv_EB_multiome.ipynb Detailed logs showing ELBO values and timing for training stages. ```text Output: 2613 out of 3138 = 83.3% genes have good ellipse fits. KS-test result: [1. 1. 1. 1. 1. 1. 1.] Assigning cluster 6 to repressive. Initial induction: 2532, repression: 606 out of 3138. -------------------------- Train a MultiVeloVAE ------------------------- ********* Creating Training and Validation Datasets ********* Total Number of Iterations Per Epoch: 12, test iteration: 22 ********* Finished. ********* ********* Stage 1 ********* Epoch 1: Train ELBO = -21489.391, Test ELBO = -250868.052 Total Time = 0 h : 0 m : 0 s Epoch 50: Train ELBO = 14445.737, Test ELBO = 14370.251 Total Time = 0 h : 0 m : 7 s Epoch 100: Train ELBO = 16401.478, Test ELBO = 16270.854 Total Time = 0 h : 0 m : 15 s Epoch 150: Train ELBO = 16905.745, Test ELBO = 16748.806 Total Time = 0 h : 0 m : 22 s Epoch 200: Train ELBO = 17068.374, Test ELBO = 16926.925 Total Time = 0 h : 0 m : 29 s Epoch 250: Train ELBO = 17170.108, Test ELBO = 17017.487 Total Time = 0 h : 0 m : 36 s Epoch 300: Train ELBO = 17220.520, Test ELBO = 17036.000 Total Time = 0 h : 0 m : 44 s Epoch 350: Train ELBO = 17260.815, Test ELBO = 17087.239 Total Time = 0 h : 0 m : 51 s Epoch 400: Train ELBO = 17289.872, Test ELBO = 17104.871 Total Time = 0 h : 0 m : 58 s Epoch 450: Train ELBO = 17318.591, Test ELBO = 17122.996 Total Time = 0 h : 1 m : 6 s Epoch 500: Train ELBO = 17335.009, Test ELBO = 17131.167 Total Time = 0 h : 1 m : 13 s Epoch 550: Train ELBO = 17353.692, Test ELBO = 17138.741 Total Time = 0 h : 1 m : 20 s Epoch 600: Train ELBO = 17363.631, Test ELBO = 17144.883 Total Time = 0 h : 1 m : 28 s Epoch 650: Train ELBO = 17378.854, Test ELBO = 17153.178 Total Time = 0 h : 1 m : 35 s ********* Stage 1: Early Stop Triggered at epoch 655. ********* ********* Retrieving best model from iteration 7723. ********* ********* Stage 2 ********* Cell-wise KNN estimation. Using 148 latent neighbors to select ancestors. ``` -------------------------------- ### Initialize MultiVeloVAE Model Source: https://github.com/welch-lab/multivelovae/blob/main/paper-notebooks/mvv_hspc_integration.ipynb Sets the model identifier key for subsequent operations. ```python key = "vae" ``` -------------------------------- ### Define output paths Source: https://github.com/welch-lab/multivelovae/blob/main/paper-notebooks/mvv_hspc_macrophage_integration.ipynb Sets base directory paths for saving models, figures, and data. ```python dataset = "8489-MV-1-9060-MV-3" model_path_base = f"checkpoints/{dataset}" figure_path_base = f"figures/{dataset}" data_path_base = f"data/{dataset}" ``` -------------------------------- ### Import MultiVeloVAE Source: https://github.com/welch-lab/multivelovae/blob/main/docs/api.md Standard import statement for accessing the library functionality. ```default import multivelovae as vv ``` -------------------------------- ### Visualizing QC Metrics Source: https://github.com/welch-lab/multivelovae/blob/main/paper-notebooks/preprocess/example_prepro_hspc_starsolo.ipynb Creating violin plots to visualize QC metrics across cells. ```python sc.pl.violin(adata_, ['n_genes_by_counts', 'total_counts', 'pct_counts_mt', 'pct_counts_rb', 'pct_counts_hla', 'pct_counts_hb', 'pct_counts_in_top_20_genes'], jitter=0.4, multi_panel=True) ``` -------------------------------- ### Set Output Paths Source: https://github.com/welch-lab/multivelovae/blob/main/paper-notebooks/mvv_mouse_brain_scRNA.ipynb Construct final paths for figures, models, and data using the current date. ```python figure_path = figure_path_base+'/'+date model_path = model_path_base+'/'+date data_path = data_path_base ``` -------------------------------- ### Data Preparation and Subsetting Source: https://github.com/welch-lab/multivelovae/blob/main/paper-notebooks/mvv_hspc_mix_integration.ipynb Assigns cluster labels and creates subsets for scRNA cells. ```python adata.obs['clusters'] = adata.obs['leiden'] ``` ```python # Subset to scRNA cells only for later use adata_sub = adata[adata.obs['batch']=='scRNA',:].copy() adata_atac_sub = adata_atac[adata.obs['batch']=='scRNA',:].copy() ``` -------------------------------- ### Perform Leiden clustering and visualize Source: https://github.com/welch-lab/multivelovae/blob/main/paper-notebooks/preprocess/example_prepro_EB_starsolo.ipynb Runs Leiden clustering at a specified resolution and visualizes the resulting clusters on a UMAP. ```python sc.tl.leiden(adata, resolution=0.7) sc.pl.umap(adata, color='leiden') ``` -------------------------------- ### Define Path Variables Source: https://github.com/welch-lab/multivelovae/blob/main/paper-notebooks/mvv_mouse_brain_scRNA.ipynb Set base paths for saving model checkpoints, figures, and data. ```python model_path_base = f"checkpoints/{dataset}" figure_path_base = f"figures/{dataset}" data_path_base = f"data/{dataset}" ``` -------------------------------- ### Training progress bar Source: https://github.com/welch-lab/multivelovae/blob/main/paper-notebooks/mvv_EB_multiome.ipynb Progress bar display during the initial training phase. ```text 0%| | 0/3138 [00:000].index[:10], color_by='leiden', by='velocity') ``` -------------------------------- ### Plot log-transformed mean distributions Source: https://github.com/welch-lab/multivelovae/blob/main/paper-notebooks/mvv_hspc_macrophage_integration.ipynb Visualizes the distribution of log-transformed mean values for C, U, and S metrics. ```python plt.hist(np.log(df_dd_mac['mean_c1']), bins=100, alpha=0.7); plt.hist(np.log(df_dd_mac['mean_u1']), bins=100, alpha=0.7); plt.hist(np.log(df_dd_mac['mean_s1']), bins=100, alpha=0.7); ``` -------------------------------- ### Visualize Expression Distributions Source: https://github.com/welch-lab/multivelovae/blob/main/paper-notebooks/mvv_hspc_macrophage_integration.ipynb Plots histograms of log-transformed mean expression values for different components. ```python plt.hist(np.log(df_dd_dc['mean_c1']), bins=100, alpha=0.7); plt.hist(np.log(df_dd_dc['mean_u1']), bins=100, alpha=0.7); plt.hist(np.log(df_dd_dc['mean_s1']), bins=100, alpha=0.7); ``` -------------------------------- ### Inspect AnnData Object Source: https://github.com/welch-lab/multivelovae/blob/main/paper-notebooks/mvv_mouse_brain_scRNA.ipynb Display the structure and contents of the loaded AnnData object. ```python adata ``` -------------------------------- ### Visualize latent time distribution Source: https://github.com/welch-lab/multivelovae/blob/main/paper-notebooks/mvv_hspc_macrophage_integration.ipynb Displays a violin plot of latent time across different batches. ```python # Latent time distribution of two datasets sns.violinplot(y="batch", x=f'{key}_time', data=adata.obs, orient='h'); ``` -------------------------------- ### Define comparison groups Source: https://github.com/welch-lab/multivelovae/blob/main/paper-notebooks/mvv_mouse_brain_multiome.ipynb Define lists of cell types for differential dynamics comparisons. ```python # Define comparison pairs a_list = [['V-SVZ'], ['Upper Layer'], ['Upper Layer', 'Deeper Layer'], ['Deeper Layer']] b_list = [['nIPC', 'Upper Layer'], ['V-SVZ', 'Deeper Layer'], ['V-SVZ'], ['V-SVZ', 'Upper Layer']] ``` -------------------------------- ### Plot phase portraits for top genes Source: https://github.com/welch-lab/multivelovae/blob/main/paper-notebooks/mvv_hspc_mix_integration.ipynb Visualizes phase portraits for the top 20 genes using scVelo scatter plots. ```python scv.pl.scatter(adata, basis=top_genes[:20], x=f'{key}_uhat', y=f'{key}_chat', color='batch', ncols=5) ``` ```python adata.layers['Mc'] = adata_atac.layers['Mc'] scv.pl.scatter(adata, basis=top_genes[:20], x='Mu', y='Mc', color='batch', ncols=5) ``` -------------------------------- ### Plot phase portraits Source: https://github.com/welch-lab/multivelovae/blob/main/paper-notebooks/mvv_EB_multiome.ipynb Generates phase portraits for the selected genes. ```python # Phase portraits of original data scv.pl.scatter(adata, basis=gene_plot, color='leiden') ``` -------------------------------- ### Loading Gene Expression Data Source: https://github.com/welch-lab/multivelovae/blob/main/paper-notebooks/preprocess/example_prepro_hspc_starsolo.ipynb Loading GEX data and ensuring unique variable names. ```python adata_ = sc.read_10x_mtx('filtered_feature_bc_matrix/', var_names='gene_symbols', cache=True, gex_only=True) adata_.var_names_make_unique() ``` -------------------------------- ### Visualize UMAP embedding Source: https://github.com/welch-lab/multivelovae/blob/main/paper-notebooks/mvv_EB_multiome.ipynb Creates a UMAP scatter plot colored by the leiden clustering. ```python os.makedirs(figure_path_base, exist_ok=True) scv.pl.scatter(adata, basis='umap', color='leiden') ``` -------------------------------- ### Save AnnData object to H5AD Source: https://github.com/welch-lab/multivelovae/blob/main/paper-notebooks/mvv_mouse_brain_scRNA.ipynb Persists the current AnnData object to a specified file path. Ensure the directory exists before executing. ```python adata.write_h5ad(data_path_base+"/final.h5ad") ```