### Install scperturb and scanpy
Source: https://github.com/sanderlab/scperturb/blob/master/revision/notebooks/get_obs.ipynb
Installs the scperturb and scanpy libraries. Use --quiet to suppress installation output.
```python
!pip install scanpy scperturb --quiet
```
--------------------------------
### Install Dependencies
Source: https://github.com/sanderlab/scperturb/blob/master/website/datavzrd/datavzrd_notebook.ipynb
Install the scanpy library in the current environment.
```python
!pip install --quiet scanpy
```
--------------------------------
### Load Example Dataset
Source: https://github.com/sanderlab/scperturb/blob/master/revision/unsolicited_review/notebooks/old/getExplanatoryPCs_py.ipynb
Loads an example dataset using scanpy. This is a prerequisite for using other functions in the library.
```python
adata = sc.read('../exampledataset/exampledataset.h5')
```
--------------------------------
### Import Libraries and Setup
Source: https://github.com/sanderlab/scperturb/blob/master/revision/notebooks/delete_me.ipynb
Imports core Python libraries for data manipulation, scientific computing, and visualization. Includes setup for scvelo, matplotlib, and custom utilities.
```python
import subprocess
import os
import sys
import matplotlib.backends.backend_pdf
import scanpy as sc
import matplotlib.pyplot as pl
import anndata as ad
import pandas as pd
import numpy as np
import seaborn as sns
import scvelo as scv
scv.settings.verbosity=1
from pathlib import Path
# Jupyter stuff
from tqdm.auto import tqdm
from IPython.display import clear_output
from IPython.core.display import display, HTML
display(HTML(""))
%matplotlib inline
# Custom functions
sys.path.insert(1, '../..')
%load_ext autoreload
%autoreload 2
from utils import *
# scperturb package
sys.path.insert(1, '../../package/src/')
from scperturb import *
from pathlib import Path
figure_path = Path('../../figures/')
```
--------------------------------
### Initialize Environment and Imports
Source: https://github.com/sanderlab/scperturb/blob/master/revision/unsolicited_review/notebooks/old/scANVI_run_notebook.ipynb
Setup necessary libraries and append the project root to the system path for module resolution.
```python
import scvi
import numpy as np
import pandas as pd
import scanpy as sc
import os
from sklearn.metrics import confusion_matrix
import seaborn as sns
import sys
sys.path.append(os.path.abspath(os.path.join(os.getcwd(), '../../')))
import scanvi
import importlib
```
--------------------------------
### Install and Import scperturb
Source: https://github.com/sanderlab/scperturb/blob/master/package/notebooks/e-distance.ipynb
Installs the scperturb library and reloads the Python environment to make its functions available. This is typically done after installing the library.
```python
# !pip install scperturb --upgrade
# from scperturb import *
import sys
sys.path.append(".."")
%reload_ext autoreload
%autoreload 2
from src.scperturb import *
```
--------------------------------
### Import Libraries and Setup
Source: https://github.com/sanderlab/scperturb/blob/master/revision/unsolicited_review/notebooks/old/getExplanatoryPCs_py.ipynb
Imports necessary libraries for data analysis and visualization, and configures display settings for notebooks.
```python
from tqdm.notebook import tqdm
import scanpy as sc
import matplotlib.pyplot as pl
import anndata as ad
import pandas as pd
import numpy as np
import seaborn as sns
from IPython.core.display import display, HTML
display(HTML(""))
%matplotlib inline
import scvelo as scv
scv.settings.verbosity=1
```
--------------------------------
### Install scperturb and Dependencies
Source: https://github.com/sanderlab/scperturb/blob/master/package/notebooks/e-distance.ipynb
Installs the scperturb library and its dependencies using pip. This is a prerequisite for using the library.
```python
!pip install scanpy scikit-misc --quiet
```
--------------------------------
### Install Required Packages
Source: https://github.com/sanderlab/scperturb/blob/master/revision/unsolicited_review/notebooks/old/perturbation_statistics.ipynb
Installs the scmmd and torch-two-sample packages from GitHub. These are necessary for advanced statistical comparisons, such as MMD calculations.
```bash
# !pip install git+https://github.com/calico/scmmd
# !pip install git+https://github.com/josipd/torch-two-sample
```
--------------------------------
### Install Dependencies
Source: https://github.com/sanderlab/scperturb/blob/master/dataset_processing/notebooks/XuCao2023.ipynb
Install the required Python packages for data analysis.
```python
!pip install chembl_webresource_client scanpy scperturb --quiet
```
--------------------------------
### Install Project Dependencies
Source: https://github.com/sanderlab/scperturb/blob/master/dataset_processing/notebooks/LotfollahiTheis2023.ipynb
Installs all necessary Python packages for the scPerturb project using pip. Ensure you are in a suitable environment before running.
```python
!pip install mygene statannotations scrublet scanpy scvelo decoupler matplotlib_venn goatools gseapy scperturb biomart PyComplexHeatmap statsmodels omnipath git+https://github.com/saezlab/pypath.git --quiet
```
--------------------------------
### Install Project Dependencies
Source: https://github.com/sanderlab/scperturb/blob/master/dataset_processing/notebooks/SunshineHein2023.ipynb
Installs all necessary Python packages for the scperturb project using pip. This command should be run in a compatible environment.
```python
!pip install mygene statannotations scrublet scanpy scvelo decoupler goatools gseapy scperturb chembl_webresource_client biomart PyComplexHeatmap statsmodels omnipath git+https://github.com/saezlab/pypath.git --quiet
```
--------------------------------
### Import Libraries and Setup
Source: https://github.com/sanderlab/scperturb/blob/master/revision/unsolicited_review/notebooks/old/Graph_label_entropy.ipynb
Imports necessary Python libraries for data analysis and sets up the environment for Jupyter notebooks. Includes custom utility functions.
```python
import subprocess
import os
import sys
import matplotlib.backends.backend_pdf
import scanpy as sc
import matplotlib.pyplot as pl
import anndata as ad
import pandas as pd
import numpy as np
import seaborn as sns
import scvelo as scv
scv.settings.verbosity=1
# Jupyter stuff
from tqdm.notebook import tqdm
from IPython.display import clear_output
from IPython.core.display import display, HTML
display(HTML(""))
%matplotlib inline
# Custom functions
sys.path.insert(1, '../..')
from utils import *
```
--------------------------------
### Partial KNN Classifier Setup
Source: https://github.com/sanderlab/scperturb/blob/master/revision/unsolicited_review/notebooks/old/Classifier_loss_similarity.ipynb
A truncated snippet showing the initialization of a KNN classifier pipeline.
```python
for selected_class in pd.unique(adata.obs.perturbation)[:10]:
if selected_class == 'control':
continue
mask = (adata.obs.perturbation == selected_class) | (adata.obs.perturbation == 'control')
# Define and split training and test data
from sklearn.model_selection import train_test_split
X = adata.obsm['X_pca'][mask] # data
y = list(adata.obs.perturbation[mask]) # labels
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, stratify=y) # stratify=labels makes sure same label ratio in splits
# Feature scaling
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
scaler.fit(X_train)
X_train = scaler.transform(X_train)
X_test = scaler.transform(X_test)
# select and train classifier (here KNN classifier)
from sklearn.neighbors import KNeighborsClassifier
```
--------------------------------
### Setup Figure and Load Icons for Plotting
Source: https://github.com/sanderlab/scperturb/blob/master/revision/unsolicited_review/notebooks/old/data_qc.ipynb
Initializes a Matplotlib figure with multiple subplots and loads various icon images for use in annotations. This setup is for creating complex visualizations with custom icons.
```python
from matplotlib.offsetbox import TextArea, DrawingArea, OffsetImage, AnnotationBbox
import matplotlib.image as mpimg
n = len(B.index)
m = len(B.columns)
# setup figure
# with sns.axes_style("whitegrid"):
fig, axs = pl.subplots(3, 1, figsize=[17, 10], sharex=True, dpi=300, gridspec_kw={'height_ratios': [1.5, 1, 1]})
# fig, ax = pl.subplots(figsize=[17,5], dpi=300)
text_options = {'fontsize': 6, 'ha': 'center', 'va': 'center'}
zoom_factor=0.6
mouse = OffsetImage(mpimg.imread('icons/mouse.png'), zoom=0.06*zoom_factor)
stem = OffsetImage(mpimg.imread('icons/stem.png'), zoom=0.04*zoom_factor)
human = OffsetImage(mpimg.imread('icons/person-standing.png'), zoom=0.08*zoom_factor)
primary = OffsetImage(mpimg.imread('icons/scalpel.png'), zoom=0.06*zoom_factor)
cell_line = OffsetImage(mpimg.imread('icons/petri-dish.png'), zoom=0.05*zoom_factor)
CRISPR = OffsetImage(mpimg.imread('icons/gene.png'), zoom=0.05*zoom_factor)
drug = OffsetImage(mpimg.imread('icons/pills.png'), zoom=0.05*zoom_factor)
True_ = OffsetImage(mpimg.imread('icons/check.png'), zoom=0.04*zoom_factor)
False_ = OffsetImage(mpimg.imread('icons/close.png'), zoom=0.035*zoom_factor)
```
--------------------------------
### Import Libraries and Setup Environment
Source: https://github.com/sanderlab/scperturb/blob/master/revision/notebooks/theory.ipynb
Imports necessary libraries and sets up the Python environment for the scperturb project, including custom paths and autoreload functionality.
```python
import subprocess
import os
import sys
import matplotlib.backends.backend_pdf
import scanpy as sc
import matplotlib.pyplot as pl
import anndata as ad
import pandas as pd
import numpy as np
import seaborn as sns
import scvelo as scv
scv.settings.verbosity=1
from pathlib import Path
# Jupyter stuff
from tqdm.auto import tqdm
from IPython.display import clear_output
from IPython.core.display import display, HTML
display(HTML(""))
%matplotlib inline
# Custom functions
sys.path.insert(1, '../..')
%load_ext autoreload
%autoreload 2
from utils import *
# scperturb package
sys.path.insert(1, '../../package/src/')
from scperturb import *
from pathlib import Path
figure_path = Path('../figures/')
```
--------------------------------
### Import Libraries and Setup Environment
Source: https://github.com/sanderlab/scperturb/blob/master/dataset_processing/notebooks/LotfollahiTheis2023.ipynb
Imports essential Python libraries for single-cell data analysis and sets up the plotting environment. Includes custom utility functions and scperturb package.
```python
import subprocess
import os
import sys
import matplotlib.backends.backend_pdf
import scanpy as sc
import matplotlib.pyplot as pl
import anndata as ad
import pandas as pd
import numpy as np
import seaborn as sns
import scvelo as scv
scv.settings.verbosity=1
from pathlib import Path
# Jupyter stuff
from tqdm.notebook import tqdm
from IPython.display import clear_output
from IPython.core.display import display, HTML
display(HTML(""))
%matplotlib inline
# Custom functions
sys.path.insert(1, '../')
from utils import *
# scperturb package
sys.path.insert(1, '../package/src/')
from scperturb import *
from pathlib import Path
figure_path = Path('../figures/')
```
--------------------------------
### Split String Example
Source: https://github.com/sanderlab/scperturb/blob/master/revision/unsolicited_review/notebooks/old/scANVI.ipynb
Demonstrates splitting a string by an underscore to extract the first part. This is useful for parsing dataset identifiers.
```python
stringthing = "a_b"
```
```python
stringthing.split("_")[0]
```
--------------------------------
### Run datavzrd Configuration
Source: https://github.com/sanderlab/scperturb/blob/master/website/datavzrd/README.md
Use this command to run datavzrd with a specified configuration file and output directory. Ensure datavzrd version 2.27.0 or later is installed.
```bash
datavzrd config.yaml --output test/
```
--------------------------------
### jQuery Animation Timer Start
Source: https://github.com/sanderlab/scperturb/blob/master/dataset_processing/notebooks/CuiHacohen2023.nb.html
The `at` function is used to get the current timestamp, typically for starting animation timers.
```javascript
function at(){return C.setTimeout(function(){Ze=void 0}),Ze=Date.now()}
```
--------------------------------
### Initialize Environment and Load Data Paths
Source: https://github.com/sanderlab/scperturb/blob/master/revision/unsolicited_review/notebooks/old/present_2_selected_datasets.ipynb
Sets up the data directory path and imports necessary utility functions.
```python
# sys.path.insert(1, '/extra/stefan/utils/scrnaseq_utils/')
from scrnaseq_util_functions import *
data_path = '/fast/work/users/peidlis_c/data/perturbation_resource_paper/'
```
```python
data_path = '/fast/work/users/peidlis_c/data/perturbation_resource_paper/'
```
--------------------------------
### Generate Guide IDs from Counts
Source: https://github.com/sanderlab/scperturb/blob/master/dataset_processing/notebooks/data_processing_JS.ipynb
Determines the most likely guide ID for each cell based on the maximum count in HTO and GDO data. 'control' is assigned if GDO starts with 'NT'.
```python
htoguides = list(hto.idxmax())
gdoguides = ["control" if x.startswith("NT") else x for x in gdo.idxmax()]
```
--------------------------------
### Initialize Environment and Paths
Source: https://github.com/sanderlab/scperturb/blob/master/notebooks/Fig2.ipynb
Sets up necessary imports, Jupyter notebook configurations, and file system paths for data processing.
```python
import subprocess
import os
import sys
import scanpy as sc
import matplotlib.pyplot as pl
import anndata as ad
import pandas as pd
import numpy as np
import seaborn as sns
from statsmodels.stats.multitest import multipletests
import scvelo as scv
scv.settings.verbosity=1
# Jupyter stuff
from tqdm.notebook import tqdm
from IPython.display import clear_output
from IPython.core.display import display, HTML
display(HTML(""))
%matplotlib inline
# Custom functions
sys.path.insert(1, '../')
from utils import *
# path with scPerturb data (replace accordingly)
data_path = '/fast/work/users/peidlis_c/data/perturbation_resource_paper/'
# temp path
SDIR = '/fast/scratch/users/peidlis_c/perturbation_resource_paper/'
# output from snakemake (tables)
table_path = '/fast/work/users/peidlis_c/projects/perturbation_resource_paper/single_cell_perturbation_data/code/notebooks/data_analysis/analysis_screens/tables/'
# path for figures
website_path = '../website/'
# path for supplemental figures and tables
supp_path = '../supplement/'
```
--------------------------------
### Get Chunk Boundaries
Source: https://github.com/sanderlab/scperturb/blob/master/dataset_processing/notebooks/JoungZhang2023.ipynb
Displays the start and end row indices for a specific chunk (determined by variable 'i' and 'chunk_size'). This is useful for verifying which part of the dataset is being processed or has been processed.
```python
i*chunk_size, (i+1)*chunk_size
```
--------------------------------
### Initialize Utility Environment
Source: https://github.com/sanderlab/scperturb/blob/master/revision/unsolicited_review/notebooks/old/data_qc_2.ipynb
Sets up the Python path and imports utility functions for single-cell RNA sequencing analysis.
```python
sys.path.insert(1, utils_path)
from scrnaseq_util_functions import *
```
--------------------------------
### Create Conda Environment for Reproducibility
Source: https://github.com/sanderlab/scperturb/blob/master/README.md
Create a new conda environment with all necessary packages to reproduce the figures and tables from the scPerturb paper. This requires having conda installed.
```bash
conda env create -f sc_env.yaml
```
--------------------------------
### Initialize Environment and Paths
Source: https://github.com/sanderlab/scperturb/blob/master/notebooks/Fig3-updated.ipynb
Sets up the Jupyter environment, imports necessary libraries, and loads configuration paths from a YAML file.
```python
import os
import matplotlib.pyplot as pl
import pandas as pd
import numpy as np
import seaborn as sns
# Jupyter stuff
from tqdm.notebook import tqdm
from IPython.display import clear_output
from IPython.core.display import display, HTML
display(HTML(""))
%matplotlib inline
# paths
import yaml
config = yaml.safe_load(open('../config.yaml', "r"))
data_path = config['DOWNDIR']
SDIR = config['DIR']
# output from snakemake (tables)
table_path = '/fast/work/users/peidlis_c/projects/perturbation_resource_paper/single_cell_perturbation_data/code/notebooks/data_analysis/analysis_screens/tables/'
# path for figures
figure_path = '../figures/'
# path for supplemental figures and tables
supp_path = '../supplement/'
dayfirst = True
```
--------------------------------
### Import Dependencies and Configure Environment
Source: https://github.com/sanderlab/scperturb/blob/master/revision/unsolicited_review/notebooks/old/singlecell_vs_pseudobulk.ipynb
Initializes necessary libraries and Jupyter notebook settings for scperturb analysis.
```python
import subprocess
import os
import sys
import matplotlib.backends.backend_pdf
import scanpy as sc
import matplotlib.pyplot as pl
import anndata as ad
import pandas as pd
import numpy as np
import scvelo as scv
scv.settings.verbosity=1
# Jupyter stuff
from tqdm.notebook import tqdm
from IPython.display import clear_output
from IPython.core.display import display, HTML
display(HTML(""))
%matplotlib inline
# Custom functions
sys.path.insert(1, '../..')
from utils import *
```
--------------------------------
### Example Data Processing Workflow
Source: https://github.com/sanderlab/scperturb/blob/master/dataset_processing/notebooks/LaraAstiasoHuntly2023.ipynb
Demonstrates the typical workflow of merging and harmonizing data for a 'leukemia' experiment. This involves calling `merge_data` followed by `harmonize_data`.
```python
adata = merge_data('leukemia')
bdata = harmonize_data(adata, 'leukemia')
```
--------------------------------
### jQuery AJAX Request Methods (GET, POST)
Source: https://github.com/sanderlab/scperturb/blob/master/dataset_processing/notebooks/CuiHacohen2023.nb.html
Convenience methods for making GET and POST requests. Use `get` for retrieving data and `post` for sending data. They simplify common AJAX calls.
```javascript
S.each(["get","post"],function(e,i){S[i]=function(e,t,n,r){return m(t)&&(r=r||n,n=t,t=void 0),S.ajax(S.extend({url:e,type:i,dataType:r,data:t,success:n},S.isPlainObject(e)&&e))}})
```
--------------------------------
### Initialize Environment and Imports
Source: https://github.com/sanderlab/scperturb/blob/master/revision/notebooks/bio_interpretation_cluster_perturbations.ipynb
Sets up the Python environment with necessary libraries for single-cell analysis and plotting.
```python
import subprocess
import os
import sys
import matplotlib.backends.backend_pdf
import scanpy as sc
import matplotlib.pyplot as pl
import anndata as ad
import pandas as pd
import numpy as np
import seaborn as sns
import scvelo as scv
scv.settings.verbosity=1
from pathlib import Path
# Jupyter stuff
from tqdm.notebook import tqdm
from IPython.display import clear_output
from IPython.core.display import display, HTML
display(HTML(""))
%matplotlib inline
# Custom functions
sys.path.insert(1, '../..')
%load_ext autoreload
%autoreload 2
from utils import *
# scperturb package
sys.path.insert(1, '../../package/src/')
from scperturb import *
from pathlib import Path
figure_path = Path('../figures/')
```
--------------------------------
### Initialize Environment and Imports
Source: https://github.com/sanderlab/scperturb/blob/master/revision/unsolicited_review/notebooks/old/DEG_profile.ipynb
Sets up the Python environment with necessary libraries for single-cell analysis and visualization.
```python
import subprocess
import os
import sys
import matplotlib.backends.backend_pdf
import scanpy as sc
import matplotlib.pyplot as pl
import anndata as ad
import pandas as pd
import numpy as np
import seaborn as sns
import scvelo as scv
scv.settings.verbosity=1
# Jupyter stuff
from tqdm.notebook import tqdm
from IPython.display import clear_output
from IPython.core.display import display, HTML
display(HTML(""))
%matplotlib inline
# Custom functions
sys.path.insert(1, '../..')
from utils import *
```
--------------------------------
### Initialize scperturb Analysis Environment
Source: https://github.com/sanderlab/scperturb/blob/master/revision/notebooks/Feature_Selection.ipynb
Sets up the Python environment with necessary libraries, custom paths, and Jupyter display settings for scperturb analysis.
```python
import subprocess
import os
import sys
import matplotlib.backends.backend_pdf
import scanpy as sc
import matplotlib.pyplot as pl
import anndata as ad
import pandas as pd
import numpy as np
import seaborn as sns
import scvelo as scv
scv.settings.verbosity=1
from pathlib import Path
# Jupyter stuff
from tqdm.auto import tqdm
from IPython.display import clear_output
from IPython.core.display import display, HTML
display(HTML(""))
%matplotlib inline
# Custom functions
sys.path.insert(1, '../..')
%load_ext autoreload
%autoreload 2
from utils import *
# scperturb package
sys.path.insert(1, '../../package/src/')
from scperturb import *
from pathlib import Path
figure_path = Path('../../figures/')
```
--------------------------------
### Map Targets to Guides
Source: https://github.com/sanderlab/scperturb/blob/master/revision/unsolicited_review/notebooks/old/Graph_label_entropy.ipynb
Creates a dictionary mapping each unique target gene to a list of its corresponding guides.
```python
# dict of all targets : [guides]
A = {}
for target in unique_targets:
A[target] = [x for x in sim.columns if target in x]
```
--------------------------------
### Initialize scperturb environment
Source: https://github.com/sanderlab/scperturb/blob/master/revision/unsolicited_review/notebooks/Edist_results.ipynb
Imports core libraries, configures scvelo verbosity, adjusts Jupyter notebook width, and adds the local utils directory to the system path.
```python
import subprocess
import os
import sys
import matplotlib.backends.backend_pdf
import scanpy as sc
import matplotlib.pyplot as pl
import anndata as ad
import pandas as pd
import numpy as np
import seaborn as sns
import scvelo as scv
scv.settings.verbosity=1
# Jupyter stuff
from tqdm.notebook import tqdm
from IPython.display import clear_output
from IPython.core.display import display, HTML
display(HTML(""))
# Custom functions
sys.path.insert(1, '../..')
from utils import *
```
--------------------------------
### Initialize McFarland H5 Files List
Source: https://github.com/sanderlab/scperturb/blob/master/dataset_processing/notebooks/quality_control_JS.ipynb
Finds all H5 files in the McFarlandTshemiak2020 directory and processes the first one for QC.
```python
mcfarlandh5 = glob.glob('/n/data1/hms/cellbio/sander/judy/resource_paper/McFarlandTshemiak2020/*.h5')
qc_h5(mcfarlandh5[0], writeadata=True)
```
--------------------------------
### Get and Set CSS Properties with jQuery
Source: https://github.com/sanderlab/scperturb/blob/master/dataset_processing/notebooks/CuiHacohen2023.nb.html
Use the .css() method to get or set CSS properties for elements. When setting, you can pass a single property-value pair or an object of multiple properties. When getting, it returns the computed value of the first element in the set.
```javascript
S.extend({cssHooks:{opacity:{get:function(e,t){if(t){var n=We(e,"opacity");return""===n?"1":n}}}},cssNumber:{animationIterationCount:!0,columnCount:!0,fillOpacity:!0,flexGrow:!0,flexShrink:!0,fontWeight:!0,gridArea:!0,gridColumn:!0,gridColumnEnd:!0,gridColumnStart:!0,gridRow:!0,gridRowEnd:!0,gridRowStart:!0,lineHeight:!0,opacity:!0,order:!0,orphans:!0,widows:!0,zIndex:!0,zoom:!0},cssProps:{},style:function(e,t,n,r){if(e&&3!==e.nodeType&&8!==e.nodeType&&e.style){var i,o,a,s=X(t),u=Xe.test(t),l=e.style;if(u||(t=ze(s)),a=S.cssHooks[t]||S.cssHooks[s],void 0===n)return a&&"get"in a&&void 0!==(i=a.get(e,!1,r))?i:l[t] ("string"===(o=typeof n)&&(i=te.exec(n))&&i[1]&&(n=se(e,t,i),o="number"),null!=n&&n==n&&("number"!==o||u||(n+=i&&i[3]||(S.cssNumber[s] ?"" :"px")),y.clearCloneStyle||""!==n||0!==t.indexOf("background")||(l[t]="inherit"),a&&"set"in a&&void 0==(n=a.set(e,n,r))||(u?l.setProperty(t,n):l[t]=n))}},css:function(e,t,n,r){var i,o,a,s=X(t);return Xe.test(t)||(t=ze(s)),(a=S.cssHooks[t]||S.cssHooks[s])&&"get"in a&&(i=a.get(e,!0,n)),void 0===i&&(i=We(e,t,r)),"normal"===i&&t in Ge&&(i=Ge[t]),""===n||n?(o=parseFloat(i),!0===n||isFinite(o)?o||0:i):i}}),S.each(["height","width"],function(e,u){S.cssHooks[u]={get:function(e,t,n){if(t)return!Ue.test(S.css(e,"display"))||e.getClientRects().length&&e.getBoundingClientRect().width?Je(e,u,n):Me(e,Ve,function(){return Je(e,u,n)})},set:function(e,t,n){var r,i=Re(e),o=!y.scrollboxSize()&&"absolute"===i.position,a=(o||n)&&"border-box"===S.css(e,"boxSizing",!1,i),s=n?Qe(e,u,n,a,i):0;return a&&o&&(s-=Math.ceil(e["offset"+u[0].toUpperCase()+u.slice(1)]-parseFloat(i[u])-Qe(e,u,"border",!1,i)-.5)),s&&(r=te.exec(t))&&"px"!==(r[3]||"px")&&(e.style[u]=t,t=S.css(e,u)),Ye(0,t,s)}}}),S.cssHooks.marginLeft=Fe(y.reliableMarginLeft,function(e,t){if(t)return(parseFloat(We(e,"marginLeft"))||e.getBoundingClientRect().left-Me(e,{marginLeft:0},function(){return e.getBoundingClientRect().left}))+"px"}),S.each({margin:"",padding:"",border:"Width"},function(i,o){S.cssHooks[i+o]={expand:function(e){for(var t=0,n={},r="string"==typeof e?e.split(" "):[e];t<4;t++)n[i+ne[t]+o]=r[t]||r[t-2]||r[0];return n}},"margin"!==i&&(S.cssHooks[i+o].set=Ye)}),S.fn.extend({css:function(e,t){return $(this,function(e,t,n){var r,i,o={},a=0;if(Array.isArray(t)){for(r=Re(e),i=t.length;a.container { width:95% !important; }"))
%matplotlib inline
# Custom functions
sys.path.insert(1, '../..')
from utils import *
# paths
at_home = False if '/fast/work/users/' in os.getcwd() else True
data_path = '/extra/stefan/data/perturbation_resource_paper/' if at_home else '/fast/work/users/peidlis_c/data/perturbation_resource_paper/'
signatures_path = '/home/peidli/utils/scrnaseq_signature_collection/' if at_home else '/fast/work/users/peidlis_c/utils/scrnaseq_signature_collection/'
utils_path = '/extra/stefan/utils/scrnaseq_utils/' if at_home else '/fast/work/users/peidlis_c/utils/single_cell_rna_seq/scrnaseq_utils/'
# Stefan's utils
sys.path.insert(1, utils_path)
from scrnaseq_util_functions import *
```
--------------------------------
### Calculate similarity metrics for guide targets
Source: https://github.com/sanderlab/scperturb/blob/master/revision/unsolicited_review/notebooks/old/Graph_label_entropy.ipynb
Computes average similarity scores for guides targeting the same gene compared to those targeting different genes.
```python
# dict of guides : [guides with at least one same target]
A = {}
for t in targets:
A[t] = list(sim.columns[[t in c for c in sim.columns]])
# Average similarity across guides with same target vs different target
tab = pd.DataFrame(index=targets, columns=['same', 'different'])
for t in targets:
a = A[t]
tab.loc[t, 'same'] = np.mean(sim.loc[a, a].values)
tab.loc[t, 'different'] = np.mean(sim.loc[~np.isin(sim.index, a), ~np.isin(sim.index, a)].values)
```
--------------------------------
### Calculate Average Similarity
Source: https://github.com/sanderlab/scperturb/blob/master/revision/unsolicited_review/notebooks/old/Graph_label_entropy.ipynb
Calculates the average similarity between guides targeting the same gene versus guides targeting different genes. Requires pandas and numpy.
```python
tab = pd.DataFrame(index=dups, columns=['same', 'different'])
for dup in dups:
g1, g2 = A[dup]
tab.loc[dup, 'same'] = sim.loc[g1, g2]
tab.loc[dup, 'different'] = np.mean(sim.loc[sim.index!=g1, sim.columns!=g2].values)
```
--------------------------------
### Install scperturbR R Package
Source: https://github.com/sanderlab/scperturb/blob/master/README.md
Install the scperturbR R package from CRAN. This package is designed to work with Seurat objects for analyzing single-cell perturbation data.
```r
install.packages('scperturbR')
```
--------------------------------
### Initialize scperturb environment
Source: https://github.com/sanderlab/scperturb/blob/master/revision/notebooks/parallel.ipynb
Imports core libraries and configures Jupyter notebook settings for analysis.
```python
import subprocess
import os
import sys
import matplotlib.backends.backend_pdf
import scanpy as sc
import matplotlib.pyplot as pl
import anndata as ad
import pandas as pd
import numpy as np
import seaborn as sns
import scvelo as scv
scv.settings.verbosity=1
from pathlib import Path
# Jupyter stuff
from tqdm.auto import tqdm
from IPython.display import clear_output
from IPython.core.display import display, HTML
display(HTML(""))
%matplotlib inline
# Custom functions
sys.path.insert(1, '../..')
%load_ext autoreload
%autoreload 2
from utils import *
```
--------------------------------
### Initialize scANVI Environment
Source: https://github.com/sanderlab/scperturb/blob/master/revision/unsolicited_review/notebooks/old/scANVI.ipynb
Imports necessary libraries and appends the project directory to the system path to enable module loading.
```python
import scvi
import numpy as np
import pandas as pd
import scanpy as sc
import os
from sklearn.metrics import confusion_matrix
import importlib
import sys
sys.path.append(os.path.abspath(os.path.join(os.getcwd(), '../../')))
import scanvi
```
--------------------------------
### Initialize Environment and Paths
Source: https://github.com/sanderlab/scperturb/blob/master/revision/unsolicited_review/notebooks/old/e-testing.ipynb
Configures the Python path and defines global variables for data directories and perturbation color mappings.
```python
sys.path.insert(1, utils_path)
from scrnaseq_util_functions import *
```
```python
colors_perturbation_types = {
'CRISPRi': 'tab:blue',
'CRISPRa': 'tab:red',
'CRISPR': 'tab:orange',
'drug': 'tab:green',
'cytokine': 'tab:olive'
}
SDIR = '/fast/scratch/users/peidlis_c/perturbation_resource_paper/'
table_path = '/fast/work/users/peidlis_c/projects/perturbation_resource_paper/single_cell_perturbation_data/code/notebooks/data_analysis/analysis_screens/tables/'
```
--------------------------------
### Install scperturb Python Package
Source: https://github.com/sanderlab/scperturb/blob/master/README.md
Install the scperturb Python package using pip. This package is used for computing E-distances and performing E-tests on single-cell perturbation data.
```bash
pip install scperturb
```
--------------------------------
### Map Duplicate Targets to Guides
Source: https://github.com/sanderlab/scperturb/blob/master/revision/unsolicited_review/notebooks/old/Graph_label_entropy.ipynb
Creates a dictionary where keys are duplicate target names and values are lists of corresponding guide identifiers found in the similarity matrix columns.
```python
# dict of duplicate targets : [guides]
A = {}
for dup in dups:
A[dup] = [x for x in sim.columns if dup+'_pDS' in x]
```
--------------------------------
### Calculate and Plot Guide Similarity
Source: https://github.com/sanderlab/scperturb/blob/master/revision/unsolicited_review/notebooks/old/aggregate_results.ipynb
Calculates and plots the average similarity between guides targeting the same gene versus different genes. Requires pandas and numpy for data manipulation and matplotlib for plotting.
```python
N = len(modes)
fig, axs = pl.subplots(1,N,figsize=[4*N, 3])
tabs = {}
for ax, mode in zip(axs, modes):
sim = pd.read_csv(f'./analysis_screens/tables/{mode}_AdamsonWeissman2016_GSM2406681_10X010_tables.csv', index_col=0)
# we do have dups! Are they more similar to each?
targets = [x.split('_')[0] for x in sim.columns]
# print(f'Found {len(sim) - len(pd.unique(targets))} duplicate targets.')
perts, cts = np.unique(targets, return_counts=True)
dups = perts[cts>1]
# dict of guides : [guides with at least one same target]
A = {}
for t in targets:
A[t] = list(sim.columns[[t in c for c in sim.columns]])
# Average similarity across guides with same target vs different target
# dict of duplicate targets : [guides]
A = {}
for dup in dups:
A[dup] = [x for x in sim.columns if dup+'_pDS' in x]
# Average similarity across guides with same target vs different target
tab = pd.DataFrame(index=dups, columns=['same', 'different'])
for dup in dups:
g1, g2 = A[dup]
tab.loc[dup, 'same'] = sim.loc[g1, g2]
tab.loc[dup, 'different'] = np.mean(sim.loc[sim.index!=g1, sim.columns!=g2].values)
tabs[mode] = tab
ax.bar([0,1], [np.mean(tab.same), np.mean(tab.different)], yerr=[np.std(tab.same), np.std(tab.different)])
ax.set_ylabel(f'Average similarity')
ax.set_xticks([0,1])
ax.set_xticklabels(['same target', 'different target'])
ax.set_title(mode)
pl.tight_layout()
pl.suptitle('Similarity between cells that got a guide with same and different target\n(AdamsonWeissman2016_GSM2406681_10X010)',
y=1.15, fontsize=14)
pl.show()
```
--------------------------------
### Calculate Guide Similarity
Source: https://github.com/sanderlab/scperturb/blob/master/revision/unsolicited_review/notebooks/old/aggregate_results.ipynb
Iterates through different similarity modes, reads similarity data, identifies duplicate targets, and calculates average similarity for guides targeting the same or different genes. Requires pandas, numpy, and matplotlib.pyplot.
```python
fig, axs = pl.subplots(1,5, figsize=[4*5, 3])
tabs = {}
SDIR = "/fast/scratch/users/peidlis_c/perturbation_resource_paper/"
for ax, mode in zip(axs, modes):
sim = pd.read_csv(f'./analysis_screens/tables/{mode}_NormanWeissman2019_filtered_tables.csv', index_col=0)
# we do have dups! Are they more similar to each?
targets = [y for x in sim.columns for y in x.split('_')]
# print(f'Found {len(sim) - len(pd.unique(targets))} duplicate targets.')
perts, cts = np.unique(targets, return_counts=True)
dups = perts[cts>1]
# dict of guides : [guides with at least one same target]
A = {}
for t in targets:
A[t] = list(sim.columns[[t in c for c in sim.columns]])
# Average similarity across guides with same target vs different target
tab = pd.DataFrame(index=perts, columns=['same', 'different'])
for t in targets:
a = A[t] # list of guides with overlapping targets
X_same = sim.loc[a, a].values # submatriks for this list onli
tab.loc[t, 'same'] = np.sum(X_same-np.diag(np.diag(X_same))) / (len(a)**2 - len(a)) # mean of these similarities without diagonal
X_different = sim.loc[~np.isin(sim.index, a), np.isin(sim.index, a)].values # submatriks for these with all other
# X_different = sim.loc[~np.isin(sim.index, a), ~np.isin(sim.index, a)].values # submatriks for all other among them
tab.loc[t, 'different'] = np.mean(X_different) # mean of these sims without diagonal
tabs[mode] = tab
```
--------------------------------
### Initialize scperturb environment
Source: https://github.com/sanderlab/scperturb/blob/master/revision/notebooks/parallel.ipynb
Sets up the path for the scperturb package and defines the figure directory.
```python
sys.path.insert(1, '../../package/src/')
from scperturb import *
from pathlib import Path
figure_path = Path('../../figures/')
```
--------------------------------
### Get and Set Form Element Values with jQuery
Source: https://github.com/sanderlab/scperturb/blob/master/dataset_processing/notebooks/CuiHacohen2023.nb.html
The `val()` method is primarily used to get the current value of the first element in the set of matched elements or to set the value of every matched element. It works with input, select, and textarea elements.
```javascript
S.fn.extend({val:function(n){var r,e,i,t=this[0];return arguments.length?(i=m(n),this.each(function(e){var t;1===this.nodeType&&(null==(t=i?n.call(this,e,S(this).val()):n)?t="":"number"==typeof t?t+="":Array.isArray(t)&&(t=S.map(t,function(e){return null==e?"":e+""})),(r=S.valHooks[this.type]||S.valHooks[this.nodeName.toLowerCase()])&&"set"in r&&void 0!==r.set(this,t,"value")||(this.value=t))})):t?(r=S.valHooks[t.type]||S.valHooks[t.nodeName.toLowerCase()])&&"get"in r&&void 0!==(e=r.get(t,"value"))?e:"string"==typeof(e=t.value)?e.replace(yt,""):null==e?"":e:void 0}})
```
--------------------------------
### Annotate mitochondrial and ribosomal genes
Source: https://github.com/sanderlab/scperturb/blob/master/dataset_processing/notebooks/ShifrutMarson2018.ipynb
Adds boolean columns 'mt' and 'ribo' to the AnnData object's variable DataFrame to identify mitochondrial and ribosomal genes, respectively. Genes starting with 'MT-' are marked as mitochondrial, and those starting with 'RPS' or 'RPL' are marked as ribosomal.
```python
adata.var['mt'] = adata.var_names.str.startswith('MT-') # annotate the group of mitochondrial genes as 'mt'
adata.var['ribo']= adata.var_names.str.startswith('RPS') | adata.var_names.str.startswith('RPL') # annotate the group of ribosomal genes as 'ribo'
```
--------------------------------
### Load Configuration
Source: https://github.com/sanderlab/scperturb/blob/master/revision/notebooks/get_obs.ipynb
Initializes global directory paths from a YAML configuration file.
```python
import yaml
with open('../../configuration/config.yaml', 'r') as file:
config = yaml.safe_load(file)
DOWNDIR = Path(config['DOWNDIR'])
TEMPDIR = Path(config['TEMPDIR'])
```
--------------------------------
### Inspect Guide IDs
Source: https://github.com/sanderlab/scperturb/blob/master/dataset_processing/notebooks/ShifrutMarson2018.ipynb
Examines the structure of the guide_id column.
```python
adata.obs['guide_id'].str.split('.')
```
--------------------------------
### Initialize scperturb Environment
Source: https://github.com/sanderlab/scperturb/blob/master/revision/notebooks/PBMC_comparison_batches.ipynb
Imports necessary libraries for single-cell analysis, distance calculations, and plotting.
```python
import scanpy as sc
from scperturb import edist, pairwise_pca_distances, equal_subsampling
import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from tqdm import tqdm
from muon import prot as pt
import matplotlib
import math
from sklearn.metrics import pairwise_distances
from statsmodels.stats.multitest import multipletests
import h5py
from scipy.stats import zscore
from scipy.cluster.hierarchy import distance, linkage, dendrogram
from scipy.cluster import hierarchy
import matplotlib as mpl
mpl.rcParams['figure.dpi'] = 300
```
--------------------------------
### Get AnnData Version
Source: https://github.com/sanderlab/scperturb/blob/master/dataset_processing/notebooks/JoungZhang2023.ipynb
Imports the anndata library and returns its version.
```python
import anndata as ad
ad.__version__
```
--------------------------------
### Configure Project Paths
Source: https://github.com/sanderlab/scperturb/blob/master/revision/unsolicited_review/notebooks/Evaluate_split.ipynb
Sets up data and utility paths dynamically based on whether the script is running on a local machine or a cluster. This ensures correct file access.
```python
# paths
at_home = False if '/fast/work/users/' in os.getcwd() else True
data_path = '/extra/stefan/data/perturbation_resource_paper/' if at_home else '/fast/work/users/peidlis_c/data/perturbation_resource_paper/'
signatures_path = '/home/peidli/utils/scrnaseq_signature_collection/' if at_home else '/fast/work/users/peidlis_c/utils/scrnaseq_signature_collection/'
utils_path = '/extra/stefan/utils/scrnaseq_utils/' if at_home else '/fast/work/users/peidlis_c/utils/single_cell_rna_seq/scrnaseq_utils/'
# # Stefan's utils
# sys.path.insert(1, utils_path)
# from scrnaseq_util_functions import *
```
--------------------------------
### Get Perturbations from Index
Source: https://github.com/sanderlab/scperturb/blob/master/revision/unsolicited_review/notebooks/old/Graph_label_entropy.ipynb
Extracts the perturbation names from the index of the similarity matrix.
```python
perturbations = sim.index
```
--------------------------------
### Initialize scperturb environment
Source: https://github.com/sanderlab/scperturb/blob/master/revision/notebooks/deprecated_correct_for_bias.ipynb
Imports necessary libraries, configures Jupyter display settings, and sets up paths for custom modules and figures.
```python
import subprocess
import os
import sys
import matplotlib.backends.backend_pdf
import scanpy as sc
import matplotlib.pyplot as pl
import anndata as ad
import pandas as pd
import numpy as np
import seaborn as sns
import scvelo as scv
scv.settings.verbosity=1
from pathlib import Path
# Jupyter stuff
from tqdm.notebook import tqdm
from IPython.display import clear_output
from IPython.core.display import display, HTML
display(HTML(""))
%matplotlib inline
# Custom functions
sys.path.insert(1, '../..')
%load_ext autoreload
%autoreload 2
from utils import *
# scperturb package
sys.path.insert(1, '../../package/src/')
from scperturb import *
from pathlib import Path
figure_path = Path('../../figures/')
```
--------------------------------
### Example console output
Source: https://github.com/sanderlab/scperturb/blob/master/package_r/notebooks/scperturbr.ipynb
Expected console output when verbose mode is enabled.
```text
[1] "Computing E-test statistics for each group."
```
--------------------------------
### Import Data Science Libraries
Source: https://github.com/sanderlab/scperturb/blob/master/revision/notebooks/check-overlaps-K562.ipynb
Initializes the environment with scanpy, numpy, pandas, and itertools.
```python
import scanpy as sc
import numpy as np
import pandas as pd
import itertools as it
```
--------------------------------
### Initialize Paths and Imports for scPerturb
Source: https://github.com/sanderlab/scperturb/blob/master/notebooks/Fig3.ipynb
Sets up necessary imports and defines paths for data, temporary storage, tables, figures, and supplemental materials. Ensure the 'data_path' is correctly configured for your environment.
```python
import os
import matplotlib.pyplot as pl
import pandas as pd
import numpy as np
import seaborn as sns
# Jupyter stuff
from tqdm.notebook import tqdm
from IPython.display import clear_output
from IPython.core.display import display, HTML
display(HTML(""))
%matplotlib inline
# path with scPerturb data (replace accordingly)
data_path = '/fast/work/users/peidlis_c/data/perturbation_resource_paper/'
# temp path
SDIR = '/fast/scratch/users/peidlis_c/perturbation_resource_paper/'
# output from snakemake (tables)
table_path = '/fast/work/users/peidlis_c/projects/perturbation_resource_paper/single_cell_perturbation_data/code/notebooks/data_analysis/analysis_screens/tables/'
# path for figures
figure_path = '../figures/'
# path for supplemental figures and tables
supp_path = '../supplement/'
```
--------------------------------
### Define Data Index
Source: https://github.com/sanderlab/scperturb/blob/master/dataset_processing/notebooks/data_processing_JS.ipynb
Sets the index for the dataset. No specific setup required.
```python
index = "FrangiehIzar2021"
```
--------------------------------
### Retrieve perturbation counts
Source: https://github.com/sanderlab/scperturb/blob/master/revision/notebooks/Complex analysis demo.ipynb
Get the raw count values for each perturbation present in the dataset.
```python
adata.obs['perturbation'].value_counts().values
```
--------------------------------
### Initialize Analysis Environment
Source: https://github.com/sanderlab/scperturb/blob/master/notebooks/Fig4-updated.ipynb
Imports necessary libraries, configures Jupyter display settings, and loads project paths from a YAML configuration file.
```python
import subprocess
import os
import sys
import scanpy as sc
import matplotlib.pyplot as pl
import anndata as ad
import pandas as pd
import numpy as np
import seaborn as sns
import scvelo as scv
scv.settings.verbosity=1
print('Scanpy version:', sc.__version__)
# Jupyter stuff
from tqdm.notebook import tqdm
from IPython.display import clear_output
from IPython.core.display import display, HTML
display(HTML(""))
%matplotlib inline
# Custom functions
sys.path.insert(1, '../')
from utils import *
# paths
import yaml
config = yaml.safe_load(open('../config.yaml', "r"))
data_path = config['DOWNDIR']
SDIR = config['DIR']
# output from snakemake (tables)
table_path = '/fast/work/users/peidlis_c/projects/perturbation_resource_paper/single_cell_perturbation_data/code/notebooks/data_analysis/analysis_screens/tables/'
# path for figures
figure_path = '../figures/'
# path for supplemental figures and tables
supp_path = '../supplement/'
```
--------------------------------
### Get Unique Datasets
Source: https://github.com/sanderlab/scperturb/blob/master/notebooks/add_chembl.ipynb
Extract a list of unique dataset names from the drugs dataframe.
```python
dsets = drugs['Dataset'].value_counts().index.values
```
--------------------------------
### Configure Data and Utility Paths
Source: https://github.com/sanderlab/scperturb/blob/master/revision/unsolicited_review/notebooks/old/data_qc_2.ipynb
Sets up environment-specific paths for data, signatures, and utility scripts based on whether the code is running on a local machine or a cluster. This ensures correct data access.
```python
# paths
at_home = False if '/fast/work/users/' in os.getcwd() else True
data_path = '/extra/stefan/data/perturbation_resource_paper/' if at_home else '/fast/work/users/peidlis_c/data/perturbation_resource_paper/'
signatures_path = '/home/peidli/utils/scrnaseq_signature_collection/' if at_home else '/fast/work/users/peidlis_c/utils/scrnaseq_signature_collection/'
utils_path = '/extra/stefan/utils/scrnaseq_utils/' if at_home else '/fast/work/users/peidlis_c/utils/single_cell_rna_seq/scrnaseq_utils/'
```
--------------------------------
### Inspect perturbation counts
Source: https://github.com/sanderlab/scperturb/blob/master/revision/unsolicited_review/notebooks/Evaluate_split.ipynb
Displays the counts of perturbations and guide IDs in the AnnData object.
```python
adata.obs.value_counts(['perturbation', 'guide_id'], sort=False)
```
--------------------------------
### Get Number of Perturbation Groups
Source: https://github.com/sanderlab/scperturb/blob/master/revision/unsolicited_review/notebooks/old/subsampling.ipynb
Calculates the number of unique perturbation groups present in the dataset.
```python
len(counts_per_group)
```
--------------------------------
### Load Required Libraries
Source: https://github.com/sanderlab/scperturb/blob/master/package_r/notebooks/scperturbr.ipynb
Initializes the environment with Seurat, dplyr, and rdist packages.
```R
library(Seurat)
library(dplyr)
library(rdist)
```