### Install CMake and lightgbm dependencies Source: https://github.com/theislab/ehrapy/blob/main/docs/installation.md Run these commands if you encounter a RuntimeError regarding CMake during installation. ```console conda install -c anaconda cmake conda install -c conda-forge lightgbm ``` -------------------------------- ### Enable pre-commit locally Source: https://github.com/theislab/ehrapy/blob/main/docs/contributing.md Run this command in the repository root to install pre-commit hooks. ```bash pre-commit install ``` -------------------------------- ### Sync Dependencies with uv Source: https://github.com/theislab/ehrapy/blob/main/docs/contributing.md Initialize and synchronize dependencies for a virtual environment using uv. This command installs all extras. ```bash uv sync --all-extras ``` -------------------------------- ### Install causal inference dependencies Source: https://github.com/theislab/ehrapy/blob/main/docs/installation.md Install the causal extra to enable causal inference functionality. ```console pip install ehrapy[causal] ``` -------------------------------- ### Create and Activate Pip Environment Source: https://github.com/theislab/ehrapy/blob/main/docs/contributing.md Manually create a virtual environment using Python's venv module, activate it, and install development dependencies with pip. ```bash python3 -m venv .venv source .venv/bin/activate pip install -e ".[dev,test,doc]" ``` -------------------------------- ### Define time-series example data Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/dtw_test_reference.ipynb Minimal single time-series sequences for testing. ```python x = [1, 3, 9, 2, 1] y = [2, 0, 0, 8, 7, 2] N = len(x) M = len(y) ``` -------------------------------- ### Install stable release via pip Source: https://github.com/theislab/ehrapy/blob/main/docs/installation.md Use this command to install the most recent stable version of ehrapy. ```console pip install ehrapy ``` -------------------------------- ### Import Libraries Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/clustermap_scanpy_expected.ipynb Imports the required libraries for plotting and data manipulation. Ensure these are installed before running. ```python import matplotlib.pyplot as plt import numpy as np import ehrapy as ep ``` -------------------------------- ### Import necessary libraries Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/dpt_timeseries_expected.ipynb Imports the required libraries for data handling, plotting, and Ehrapy functionalities. Ensure these are installed before running. ```python import ehrdata as ed\nimport matplotlib.pyplot as plt\nimport numpy as np\nfrom ehrdata.core.constants import FEATURE_TYPE_KEY, NUMERIC_TAG\n\nimport ehrapy as ep ``` -------------------------------- ### Configure Matplotlib for Ehrapy Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/rank_features_groups_expected.ipynb Sets up Matplotlib with specific configurations for figure size, DPI, and font settings, suitable for use with Ehrapy. Ensure Matplotlib is installed. ```python import matplotlib as mpl import matplotlib.pyplot as plt import ehrapy as ep mpl.use("Agg", force=True) mpl.rcParams.update( { "figure.figsize": (8, 6), "figure.dpi": 80, "savefig.dpi": 80, "savefig.facecolor": "white", "axes.facecolor": "white", "font.family": "DejaVu Sans", "font.sans-serif": ["DejaVu Sans"], "mathtext.fontset": "dejavusans", "text.antialiased": False, } ) ``` -------------------------------- ### Ehrapy Neighbors Implementation Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/dtw_test_reference.ipynb Illustrates how to use Ehrapy's `pp.neighbors` function to compute nearest neighbors for an EHRData object. This example uses the DTW metric and shows the resulting distances and connectivities stored in the `obsp` attribute. ```python edata = ed.EHRData(X=None, R=data) ep.pp.neighbors(edata, n_neighbors=3, metric="dtw") ``` ```python edata.obsp["distances"].toarray() ``` -------------------------------- ### Build and Open Documentation with uv Source: https://github.com/theislab/ehrapy/blob/main/docs/contributing.md Use these commands to build and view the documentation locally when using the uv environment. ```bash cd docs uv run sphinx-build -M html . _build -W (xdg-)open _build/html/index.html ``` -------------------------------- ### Build and Open Documentation with Pip Source: https://github.com/theislab/ehrapy/blob/main/docs/contributing.md Use these commands to build and view the documentation locally when using a standard pip virtual environment. ```bash source .venv/bin/activate cd docs sphinx-build -M html . _build -W (xdg-)open _build/html/index.html ``` -------------------------------- ### Initialize environment and load data Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/missing_values_barplot_expected.ipynb Sets the working directory and loads the MIMIC-II dataset for analysis. ```python current_notebook_dir = %pwd _TEST_IMAGE_PATH = f"{current_notebook_dir}/../plot/_images" adata = ep.dt.mimic_2(encoded=False) ``` -------------------------------- ### Build and Open Documentation with Hatch Source: https://github.com/theislab/ehrapy/blob/main/docs/contributing.md Use these commands to build and view the documentation locally when using the Hatch environment. ```bash hatch run docs:build hatch run docs:open ``` -------------------------------- ### Prepare sample data Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/matrix_scanpy_plot_expected.ipynb Sets the working directory and loads a subset of the MIMIC-II dataset for testing. ```python current_notebook_dir = %pwd _TEST_IMAGE_PATH = f"{current_notebook_dir}/../plot/_images" adata = ep.dt.mimic_2(encoded=True) adata_sample = adata[:200].copy() ``` -------------------------------- ### Initialize ehrapy environment Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/tsne_expected.ipynb Import necessary libraries and configure the notebook environment. ```python import matplotlib.pyplot as plt import numpy as np import ehrapy as ep ``` ```python %load_ext autoreload %autoreload 2 ``` -------------------------------- ### Save Plot to File Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/dpt_groups_pseudotime_expected.ipynb Saves the current figure to a PNG file. This is a commented-out example, uncomment to enable saving. ```python # plt.gcf().savefig(f"{_TEST_IMAGE_PATH}/dpt_groups_pseudotime_expected.png", dpi=80) ``` -------------------------------- ### Show Hatch Environments Source: https://github.com/theislab/ehrapy/blob/main/docs/contributing.md Display a list of all available environments for your project managed by Hatch. ```bash hatch env show -i ``` -------------------------------- ### Run Hatch Tests and Build Docs Source: https://github.com/theislab/ehrapy/blob/main/docs/contributing.md Use Hatch CLI commands to run tests or build documentation. These commands automatically resolve environments. ```bash hatch test ``` ```bash hatch run docs:build ``` -------------------------------- ### Initialize Ehrapy Environment Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/sankey_matplotlib_expected.ipynb Imports necessary libraries and configures the plotting backend. ```python import ehrdata as ed import holoviews as hv import matplotlib.pyplot as plt import numpy as np import pandas as pd from ehrdata.core.constants import DEFAULT_TEM_LAYER_NAME import ehrapy as ep ``` ```python hv.extension("matplotlib") ``` -------------------------------- ### Run tests with Hatch Source: https://github.com/theislab/ehrapy/blob/main/docs/contributing.md Execute tests using the Hatch build system. ```bash hatch test # test with the highest supported Python version # or hatch test --all # test with all supported Python versions ``` -------------------------------- ### Run tests with uv Source: https://github.com/theislab/ehrapy/blob/main/docs/contributing.md Execute tests using the uv package manager. ```bash uv run pytest ``` -------------------------------- ### Clone repository from source Source: https://github.com/theislab/ehrapy/blob/main/docs/installation.md Download the source code by cloning the public GitHub repository. ```console git clone git://github.com/theislab/ehrapy ``` -------------------------------- ### Initialize HoloViews Extension Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/sankey_bokeh_expected.ipynb Initializes the Bokeh extension for HoloViews. This is required for rendering plots in a notebook environment. ```python hv.extension("bokeh") ``` -------------------------------- ### Create Hatch Environment Source: https://github.com/theislab/ehrapy/blob/main/docs/contributing.md Create a specific development environment managed by Hatch. Ensure the environment name is valid. ```bash hatch env create hatch-test.py3.13-stable ``` -------------------------------- ### Initialize CohortTracker and Plot Bar Chart (Step 1) Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/cohort_tracker_test_create_expected_plots.ipynb Initializes CohortTracker with the dataset and performs the first tracking step. It then generates and saves a cohort bar plot with custom legend labels. ```python ct = ep.tl.CohortTracker(adata_mini) ct(adata_mini, label="First step", operations_done="Some operations") fig1, ax1 = ct.plot_cohort_barplot(show=False, legend_labels={"weight": "weight(kg)", "glucose": "glucose(mg/dL)"}) ct(adata_mini, label="Second step", operations_done="Some other operations") fig2, ax2 = ct.plot_cohort_barplot(show=False, legend_labels={"weight": "weight(kg)", "glucose": "glucose(mg/dL)"}) fig1.savefig( f"{_TEST_IMAGE_PATH}/cohorttracker_edata_mini_step1_vanilla_expected.png", dpi=80, ) fig2.savefig( f"{_TEST_IMAGE_PATH}/cohorttracker_edata_mini_step2_vanilla_expected.png", dpi=80, ) ``` -------------------------------- ### Generate Graph Layout Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/draw_graph_expected.ipynb Compute and visualize the graph layout using PAGA initialization. ```python ep.tl.draw_graph(adata, init_pos="paga") ep.pl.draw_graph(adata, color=["leiden_0_5", "icu_exp_flg"], legend_loc="on data", show=False) plt.gcf().set_size_inches(16, 6) plt.gcf().subplots_adjust(left=0.2, right=0.8, bottom=0.2, top=0.8) plt.gcf().savefig(f"{_TEST_IMAGE_PATH}/draw_graph2_expected.png", dpi=80) ``` -------------------------------- ### Prepare Data for Analysis Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/rank_features_groups_violin_expected.ipynb Generates a sample AnnData object and selects a subset of features for analysis. Ensure the necessary data is available for mimic_2 function. ```python current_notebook_dir = %pwd _TEST_IMAGE_PATH = f"{current_notebook_dir}/../plot/_images" adata = ep.dt.mimic_2(encoded=True) adata_sample = adata[ :200, [ "abg_count", "wbc_first", "hgb_first", "potassium_first", "tco2_first", "bun_first", "creatinine_first", "pco2_first", ], ].copy() ``` -------------------------------- ### Manage test environments with Hatch Source: https://github.com/theislab/ehrapy/blob/main/docs/contributing.md Commands to create and locate test environments for supported Python versions. ```bash hatch env create hatch-test # create test environments for all supported versions hatch env find hatch-test # list all possible test environment paths ``` -------------------------------- ### Define Test Image Path Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/diffmap_expected.ipynb Determines the current notebook directory and sets a path for saving test images. ```python current_notebook_dir = %pwd _TEST_IMAGE_PATH = f"{current_notebook_dir}/../plot/_images" ``` -------------------------------- ### Configure Numba CPU environment Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/embedding_expected.ipynb Sets the Numba CPU target to generic for compatibility. ```python !NUMBA_CPU_NAME=generic ``` -------------------------------- ### Run tests with Pip Source: https://github.com/theislab/ehrapy/blob/main/docs/contributing.md Activate the virtual environment and execute tests using pytest. ```bash source .venv/bin/activate pytest ``` -------------------------------- ### Class Documentation Template Source: https://github.com/theislab/ehrapy/blob/main/docs/_templates/autosummary/class.rst Template for generating class-level documentation including attribute and method summaries and detailed descriptions. ```APIDOC ## Class Documentation Template ### Description This template is used by Sphinx autodoc to generate documentation for ehrapy classes. It includes sections for attributes and methods. ### Structure - **Attributes Table**: Lists all class attributes. - **Methods Table**: Lists all class methods (excluding __init__). - **Attributes Documentation**: Detailed documentation for each attribute. - **Methods Documentation**: Detailed documentation for each method (excluding __init__). ``` -------------------------------- ### Plot Cohort Flowchart Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/cohort_tracker_test_create_expected_plots.ipynb Initializes CohortTracker, performs two tracking steps, and then generates and saves a cohort flowchart visualization. ```python ct = ep.tl.CohortTracker(adata_mini) ct(adata_mini, label="Base Cohort") ct(adata_mini, operations_done="Some processing") fig, ax = ct.plot_flowchart( show=False, ) fig.savefig( f"{_TEST_IMAGE_PATH}/cohorttracker_edata_mini_flowchart_expected.png", dpi=80, ) ``` -------------------------------- ### Set Test Image Path Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/dendogram_expected.ipynb Define the directory path for saving test images. This path is relative to the current notebook directory. ```python current_notebook_dir = %pwd _TEST_IMAGE_PATH = f"{current_notebook_dir}/../plot/_images" ``` -------------------------------- ### Execute PAGA analysis pipeline Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/paga_expected copy.ipynb Load MIMIC-II data, perform imputation, normalization, neighbor calculation, and PAGA clustering. ```python adata = ep.dt.mimic_2(encoded=True) ep.pp.knn_impute(adata) ep.pp.log_norm(adata, offset=1) ep.pp.neighbors(adata) ep.tl.leiden(adata, resolution=0.5, key_added="leiden_0_5") ep.tl.paga(adata, groups="leiden_0_5") plt.gcf().set_size_inches(16, 6) ep.pl.paga( adata, color=["leiden_0_5", "day_28_flg"], cmap=ep.pl.Colormaps.grey_red.value, title=["Leiden 0.5", "Died in less than 28 days"], show=False, ) plt.gcf().subplots_adjust(left=0.2, right=0.8, bottom=0.2, top=0.8) plt.gcf().savefig(f"{_TEST_IMAGE_PATH}/paga_expected.png", dpi=80) plt.close("all") ``` -------------------------------- ### Load Ehrapy and Initialize Autoreload Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/kaplain_meier_create_expected_plots.ipynb Loads the autoreload extension for interactive development and imports necessary libraries, including ehrapy. ```python %load_ext autoreload %autoreload 2 ``` ```python import numpy as np import ehrapy as ep current_notebook_dir = %pwd _TEST_IMAGE_PATH = f"{current_notebook_dir}/../plot/_images" mimic_2 = ep.dt.mimic_2(encoded=False) ``` -------------------------------- ### Load and Prepare Data for Plotting Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/catplot_create_expected_plots.ipynb Loads a dataset from a CSV file and prepares an AnnData object for plotting. Ensure the test data path is correctly set. ```python current_notebook_dir = %pwd _TEST_DATA_PATH = f"{current_notebook_dir}/../test_data/dataset1.csv" _TEST_IMAGE_PATH = f"{current_notebook_dir}/../plot/_images" adata_mini = ep.io.read_csv(_TEST_DATA_PATH, columns_obs_only=["glucose", "weight", "disease", "station"]) ``` -------------------------------- ### Import required libraries Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/dtw_test_reference.ipynb Standard imports for ehrdata, numpy, scipy, and ehrapy. ```python import ehrdata as ed import numpy as np import scipy import ehrapy as ep ``` -------------------------------- ### ehrapy Data Loading and Structure Source: https://context7.com/theislab/ehrapy/llms.txt Demonstrates how to load EHR data using the ehrdata package and inspect its structure. ```APIDOC ## Loading Data with ehrdata ### Description Load EHR datasets using the ehrdata package's built-in datasets or read from CSV/FHIR/H5AD files. ### Method N/A (Illustrative Example) ### Endpoint N/A ### Request Example ```python import ehrdata as ed import ehrapy as ep # Load the MIMIC-II demo dataset edata = ed.dt.mimic_2() # View the data structure print(edata) # EHRData object with n_obs × n_vars = 1776 × 36 # obs: 'day_icu_intime', 'day_icu_intime_num', 'icustay_id', ... # var: 'feature_type' # Access the data matrix print(edata.X.shape) # (1776, 36) # Access observation metadata print(edata.obs.head()) # Access variable metadata print(edata.var.head()) ``` ### Response #### Success Response (200) N/A (Illustrative Output) #### Response Example ``` EHRData object with n_obs × n_vars = 1776 × 36 obs: 'day_icu_intime', 'day_icu_intime_num', 'icustay_id', ... var: 'feature_type' (1776, 36) day_icu_intime day_icu_intime_num icustay_id ... hospital_expire_flag has_chartevents_data has_labevents_data 0 2147483647.0 -2147483647.0 200001.0 ... 0.0 1.0 1.0 1 2147483647.0 -2147483647.0 200002.0 ... 0.0 1.0 1.0 2 2147483647.0 -2147483647.0 200003.0 ... 0.0 1.0 1.0 3 2147483647.0 -2147483647.0 200004.0 ... 0.0 1.0 1.0 4 2147483647.0 -2147483647.0 200005.0 ... 0.0 1.0 1.0 [1776 rows x 36 columns] feature_type age numeric bun_first numeric creatinine_first numeric glucose_first numeric hematocrit_first numeric Name: 0, dtype: object ``` ``` -------------------------------- ### Initialize CohortTracker and Plot Bar Chart with Settings Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/cohort_tracker_test_create_expected_plots.ipynb Initializes CohortTracker and plots a bar chart, applying custom y-tick labels and legend labels. The plot is saved to a file. ```python ct = ep.tl.CohortTracker(adata_mini) ct(adata_mini, label="First step", operations_done="Some operations") fig1_use_settings, _ = ct.plot_cohort_barplot( show=False, yticks_labels={"weight": "wgt"}, legend_labels={"A": "Dis. A", "weight": "(kg)", "glucose": "glucose(mg/dL)"}, ) fig1_use_settings.savefig( f"{_TEST_IMAGE_PATH}/cohorttracker_edata_mini_step1_use_settings_expected.png", dpi=80, ) ``` -------------------------------- ### Find Hatch Environment Path Source: https://github.com/theislab/ehrapy/blob/main/docs/contributing.md Obtain the file system path to a specific Hatch environment. This is useful for IDE configuration. ```bash hatch env find hatch-test.py3.13-stable ``` -------------------------------- ### Print Dependency Versions Source: https://github.com/theislab/ehrapy/blob/main/docs/api/settings_index.md Prints the versions of all dependencies used in the current ehrapy environment for reproducibility. ```APIDOC ## ep.print_versions ### Description Outputs the versions of all dependencies to ensure a consistent runtime environment and facilitate result reproduction. ### Request Example ```python import ehrapy as ep ep.print_versions() ``` ``` -------------------------------- ### Import ehrapy and ehrdata Source: https://context7.com/theislab/ehrapy/llms.txt Standard import convention for accessing the framework API. ```python import ehrapy as ep import ehrdata as ed ``` -------------------------------- ### Load and Subset MIMIC-II Data Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/coxph_forestplot_create_expected.ipynb Loads the MIMIC-II dataset and selects specific columns for analysis. ```python adata = ep.dt.mimic_2(encoded=False) adata_subset = adata[:, ["mort_day_censored", "censor_flg", "gender_num", "afib_flg", "day_icu_intime_num"]] ``` -------------------------------- ### Execute data analysis pipeline Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/embedding_expected.ipynb Loads mimic_2 data, performs imputation, normalization, neighbor calculation, and UMAP visualization. ```python adata_full = ep.dt.mimic_2(columns_obs_only=["service_unit", "day_icu_intime"]) adata_full = adata_full[~np.isnan(adata_full.X).any(axis=1)].copy() adata = adata_full[:200, :].copy() del adata_full adata = ep.pp.encode(adata, autodetect=True) ep.pp.simple_impute(adata) ep.pp.log_norm(adata, offset=1) ep.pp.neighbors(adata, transformer="sklearn") ep.tl.umap(adata) ep.pl.embedding(adata, "umap", color="icu_exp_flg", size=(np.arange(adata.shape[0]) / 40) ** 4, show=False) plt.gcf().set_size_inches(16, 6) plt.gcf().subplots_adjust(left=0.2, right=0.8, bottom=0.2, top=0.8) plt.gcf().savefig(f"{_TEST_IMAGE_PATH}/embedding_expected.png", dpi=80) ``` -------------------------------- ### Import Libraries Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/pca_overview_expected.ipynb Imports necessary libraries for data manipulation and visualization. ```python import ehrdata as ed import matplotlib.pyplot as plt import numpy as np import ehrapy as ep ``` -------------------------------- ### Load and Copy Data Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/dpt_groups_pseudotime_expected.ipynb Loads a dataset using ehrapy's mimic function and creates a copy to ensure the original data remains unchanged during analysis. ```python adata = ep.dt.mimic_2(encoded=True).copy() ``` -------------------------------- ### Perform Data Preprocessing and Analysis Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/dpt_groups_pseudotime_expected.ipynb Applies several preprocessing and analysis steps to the AnnData object, including k-NN imputation, log normalization, neighbor graph calculation, Leiden clustering, and Diffusion Pseudotime (DPT) analysis. Requires `adata` to be initialized. ```python ep.pp.knn_impute(adata) ep.pp.log_norm(adata, offset=1) ep.pp.neighbors(adata, method="gauss") ep.tl.leiden(adata, resolution=0.5, key_added="leiden_0_5") ep.tl.diffmap(adata, n_comps=10) adata.uns["iroot"] = np.flatnonzero(adata.obs["leiden_0_5"] == "0")[0] ep.tl.dpt(adata, n_branchings=3) ep.pl.dpt_groups_pseudotime(adata, show=False) ``` -------------------------------- ### Visualize PCA Overview and Save Figures Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/pca_overview_expected.ipynb Generates a PCA overview plot, adjusts figure sizes, and saves the plots to a specified directory. This is useful for exploring the principal components and their relationship with categorical variables. ```python ep.pl.pca_overview(adata, components="1,2", color="service_unit", show=False) for id, fignum in enumerate(plt.get_fignums(), start=1): fig = plt.figure(fignum) if fignum == 2: fig.set_size_inches(12, 6) else: fig.set_size_inches(8, 6) fig.subplots_adjust(left=0.2, right=0.8, bottom=0.2, top=0.8) fig.savefig(f"{_TEST_IMAGE_PATH}/pca_overview_{id}_expected.png", dpi=80) ``` -------------------------------- ### Import Libraries for CohortTracker Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/cohort_tracker_test_create_expected_plots.ipynb Imports necessary libraries: ehrdata for data handling, matplotlib.pyplot for plotting, and ehrapy for cohort tracking functionalities. ```python import ehrdata as ed import matplotlib.pyplot as plt import ehrapy as ep ``` -------------------------------- ### Prepare Data for Survival Analysis Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/kaplain_meier_create_expected_plots.ipynb Prepares the dataset by modifying a censoring flag and splitting data into groups for survival analysis. ```python mimic_2[:, ["censor_flg"]].X = np.where(mimic_2[:, ["censor_flg"]].X == 0, 1, 0) groups = mimic_2[:, ["service_unit"]].X ada_ficu = mimic_2[groups == "FICU"] ada_micu = mimic_2[groups == "MICU"] kmf_1 = ep.tl.kaplan_meier(adata_ficu, duration_col="mort_day_censored", event_col="censor_flg", label="FICU") kmf_2 = ep.tl.kaplan_meier(adata_micu, duration_col="mort_day_censored", event_col="censor_flg", label="MICU") ``` -------------------------------- ### Visualize PCA Loadings Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/pca_loadings_expected.ipynb Generates and saves a plot of PCA loadings for the specified components. ```python ep.pl.pca_loadings(adata, components="1,2,3", show=False) plt.gcf().set_size_inches(12, 6) plt.gcf().subplots_adjust(left=0.2, right=0.8, bottom=0.2, top=0.8) plt.gcf().savefig(f"{_TEST_IMAGE_PATH}/pca_loadings_expected.png", dpi=80) ``` -------------------------------- ### Complete EHR Analysis Pipeline Source: https://context7.com/theislab/ehrapy/llms.txt Demonstrates a full end-to-end ehrapy workflow, including data loading, quality control, encoding, imputation, normalization, dimensionality reduction, clustering, visualization, differential analysis, and survival analysis. Requires ehrdata, ehrapy, and numpy. ```python import ehrdata as ed import ehrapy as ep import numpy as np # 1. Load data edata = ed.dt.mimic_2() print(f"Initial data: {edata.n_obs} patients, {edata.n_vars} features") # 2. Quality control obs_qc, var_qc = ep.pp.qc_metrics(edata) print(f"Features with >50% missing: {(var_qc['missing_values_pct'] > 50).sum()}") # 3. Encode categorical variables edata = ep.pp.encode(edata, autodetect=True, encodings="one-hot") # 4. Impute missing values ep.pp.simple_impute(edata, strategy="median") # 5. Normalize data ep.pp.scale_norm(edata) # 6. Dimensionality reduction ep.pp.pca(edata, n_comps=30) ep.pp.neighbors(edata, n_neighbors=15, n_pcs=30) ep.tl.umap(edata) # 7. Clustering ep.tl.leiden(edata, resolution=0.5) # 8. Visualization ep.pl.umap(edata, color=["leiden", "age", "gender_num"], ncols=3) # 9. Differential analysis ep.tl.rank_features_groups(edata, groupby="leiden") ep.pl.rank_features_groups(edata, n_features=5) # 10. Survival analysis edata_orig = ed.dt.mimic_2() # Reload for survival analysis edata_orig[:, ["censor_flg"]].X = np.where( edata_orig[:, ["censor_flg"]].X == 0, 1, 0 ) kmf = ep.tl.kaplan_meier( edata_orig, duration_col="mort_day_censored", event_col="censor_flg" ) ep.pl.kaplan_meier(kmf) print("Analysis complete!") ``` -------------------------------- ### Perform PAGA Analysis and Visualization Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/draw_graph_expected.ipynb Run imputation, normalization, neighbor calculation, and PAGA clustering, then save the resulting plot. ```python ep.pp.simple_impute(adata) ep.pp.log_norm(adata, offset=1) ep.pp.neighbors(adata) ep.tl.leiden(adata, resolution=0.5, key_added="leiden_0_5") ep.tl.paga(adata, groups="leiden_0_5") ep.pl.paga( adata, color=["leiden_0_5", "day_28_flg"], cmap=ep.pl.Colormaps.grey_red.value, title=["Leiden 0.5", "Died in less than 28 days"], show=False, ) plt.gcf().set_size_inches(16, 6) plt.gcf().subplots_adjust(left=0.2, right=0.8, bottom=0.2, top=0.8) plt.gcf().savefig(f"{_TEST_IMAGE_PATH}/draw_graph1_expected.png", dpi=80) ``` -------------------------------- ### Load and preprocess mimic_2 dataset Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/pca_variance_ratio_expected.ipynb Load the mimic_2 dataset, filter out rows with NaN values, and encode categorical features. ```python adata_full = ep.dt.mimic_2(columns_obs_only=["service_unit", "day_icu_intime"]) adata_full = adata_full[~np.isnan(adata_full.X).any(axis=1)].copy() adata = adata_full[:200, :].copy() del adata_full adata = ep.pp.encode(adata, autodetect=True) ``` -------------------------------- ### Integrate pre-commit.ci changes Source: https://github.com/theislab/ehrapy/blob/main/docs/contributing.md Use this command to pull changes made by pre-commit.ci into your local branch. ```bash git pull --rebase ``` -------------------------------- ### Plotting Multiple Panels in Ehrapy Source: https://context7.com/theislab/ehrapy/llms.txt Demonstrates how to create plots with multiple panels using ep.pl.scatter. Specify the data, basis, and columns for coloring, along with the number of columns for the layout. ```python ep.pl.scatter( edata, basis="pca", color=["age", "gender_num", "service_unit"], ncols=3 ) ``` -------------------------------- ### Load and Preprocess Data Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/diffmap_expected.ipynb Mimics a dataset, removes rows with NaN values, selects a subset, encodes categorical features, and handles potential feature type warnings. ```python adata_full = ep.dt.mimic_2(columns_obs_only=["service_unit", "day_icu_intime"]) ada_full = adata_full[~np.isnan(adata_full.X).any(axis=1)].copy() ada = adata_full[:200, :].copy() del adata_full ada = ep.pp.encode(adata, autodetect=True) ``` -------------------------------- ### Perform imputation, normalization, and neighbor graph construction Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/dpt_timeseries_expected.ipynb Applies k-NN imputation to numerical features, log-normalizes the data, and constructs a Gaussian kernel neighbor graph. These steps prepare the data for downstream analysis like clustering and pseudotime ordering. ```python ep.pp.knn_impute(edata, backend="scikit-learn", var_names=edata.var_names[edata.var[FEATURE_TYPE_KEY] == NUMERIC_TAG])\nep.pp.log_norm(edata, offset=1)\nep.pp.neighbors(edata, method="gauss")\nep.tl.leiden(edata, resolution=0.5, key_added="leiden_0_5")\nep.tl.diffmap(edata, n_comps=10)\n\nedata.uns["iroot"] = np.flatnonzero(edata.obs["leiden_0_5"] == "0")[0]\n\nep.tl.dpt(edata, n_branchings=2) ``` -------------------------------- ### Load and Preprocess Data Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/pca_overview_expected.ipynb Loads MIMIC-II data, filters out rows with NaN values, subsets the data, and encodes categorical features. ```python adata_full = ed.dt.mimic_2(columns_obs_only=["service_unit", "day_icu_intime"]) ada_full = adata_full[~np.isnan(adata_full.X).any(axis=1)].copy() ada = adata_full[:200, :].copy() del adata_full ada = ep.pp.encode(adata, autodetect=True) ``` -------------------------------- ### Load Diabetes Dataset Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/sankey_bokeh_expected.ipynb Loads the diabetes_130_fairlearn dataset, selecting specific columns and limiting the observations. Handles existing files by using them. ```python edata = ed.dt.diabetes_130_fairlearn(columns_obs_only=["gender", "race"])[:100] ``` -------------------------------- ### Download source tarball Source: https://github.com/theislab/ehrapy/blob/main/docs/installation.md Download the latest source code as a tarball using curl. ```console curl -OJL https://github.com/theislab/ehrapy/tarball/master ``` -------------------------------- ### Generate and save heatmap Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/heatmap_scanpy_plt_expected.ipynb Create a heatmap plot from the loaded data and save it to the specified image directory. ```python ep.pl.heatmap( adata_mini, var_names=["idx", "sys_bp_entry", "dia_bp_entry", "glucose", "weight", "in_days"], groupby="station", show=False, figsize=(5, 6), ) plt.gcf().savefig(f"{_TEST_IMAGE_PATH}/heatmap_scanpy_plt_expected.png", dpi=80) ``` -------------------------------- ### Generate and save dotplot Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/dotplot_scanpy_plt_expected.ipynb Creates a dotplot for specified variables grouped by service unit and saves the figure to the test image directory. ```python ep.pl.dotplot( adata, var_names=[ "abg_count", "wbc_first", "hgb_first", "potassium_first", "tco2_first", "bun_first", "creatinine_first", "pco2_first", ], groupby="service_unit", show=False, figsize=(5, 6), ) plt.gcf().savefig(f"{_TEST_IMAGE_PATH}/dotplot_scanpy_plt_expected.png", dpi=80) ``` -------------------------------- ### Load and Clean Data for Clustermap Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/clustermap_scanpy_expected.ipynb Loads sample data using ehrapy's mimic function, selects specific features, and cleans the data by removing rows with non-finite values. This prepares the data for visualization. ```python adata = ep.dt.mimic_2(encoded=True) ada_sample = adata[ :200, [ "abg_count", "wbc_first", "hgb_first", ], ].copy() mask = np.isfinite(adata_sample.X).all(axis=1) ada_clean = adata_sample[mask].copy() ``` -------------------------------- ### Track Cohort Changes with CohortTracker Source: https://context7.com/theislab/ehrapy/llms.txt Initialize and use CohortTracker to document and visualize changes in a cohort through filtering steps. Requires specifying columns and categorical features. Can plot cohort flow and characteristic bar plots. ```python import ehrdata as ed import ehrapy as ep edata = ed.dt.mimic_2() # Initialize cohort tracker tracker = ep.tl.CohortTracker( edata, columns=["gender_num", "age", "service_unit"], categorical=["gender_num", "service_unit"] ) # Track initial cohort tracker(edata, label="Initial Cohort") # Apply filtering and track edata_filtered = edata[edata.obs["age"] >= 18] tracker(edata_filtered, label="Adults Only", operations_done="Filtered age >= 18") # Further filtering edata_icu = edata_filtered[edata_filtered.obs["service_unit"].isin(["MICU", "SICU"])] tracker(edata_icu, label="ICU Patients", operations_done="Filtered to MICU/SICU") # Plot cohort flow tracker.plot_flowchart(title="Cohort Selection") # Plot cohort characteristics tracker.plot_cohort_barplot( subfigure_title=True, legend_labels={"1.0": "Male", "0.0": "Female"} ) # Access TableOne summaries print(tracker.tracked_tables[0].tableone) ``` -------------------------------- ### Load and Subset Data Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/ols_expected.ipynb Loads a dataset using Ehrapy's mimic function and subsets it for analysis. Ensure the dataset is compatible with Ehrapy's data structures. ```python adata_full = ep.dt.mimic_2(encoded=False) ata = adata_full[:200].copy() ``` -------------------------------- ### Rank and visualize feature groups Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/rank_features_groups_matrixplot_expected.ipynb Subsets the data, performs feature ranking, and generates a matrix plot saved to the test image directory. ```python adata_sample = adata[ :200, [ "abg_count", "wbc_first", "hgb_first", "potassium_first", "tco2_first", "bun_first", "creatinine_first", ], ].copy() ep.tl.rank_features_groups(adata_sample, groupby="service_unit") ep.pl.rank_features_groups_matrixplot(adata_sample, key="rank_features_groups", groupby="service_unit", show=False) plt.gcf().set_size_inches(8, 6) plt.gcf().subplots_adjust(left=0.2, right=0.8, bottom=0.2, top=0.8) plt.gcf().savefig(f"{_TEST_IMAGE_PATH}/rank_features_groups_matrixplot_scanpy_expected.png", dpi=80) ``` -------------------------------- ### Load Mimicked Data Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/missing_values_matrix_expected.ipynb Loads a mimicked dataset using ehrapy's data loading utility. Ensure the notebook directory is correctly set for relative pathing. ```python current_notebook_dir = %pwd _TEST_IMAGE_PATH = f"{current_notebook_dir}/../plot/_images" adata = ep.dt.mimic_2(encoded=True) ``` -------------------------------- ### Enable Autoreload for Notebooks Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/catplot_create_expected_plots.ipynb Enable autoreload to automatically reload modules before executing code, useful during development in notebooks. ```python %load_ext autoreload %autoreload 2 ``` -------------------------------- ### Generate tracksplot Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/tracksplot_expected.ipynb Creates a tracksplot visualization and saves the output to a file. ```python ep.pl.tracksplot( adata_sample, var_names=["age", "gender_num", "weight_first", "bmi", "sapsi_first", "day_icu_intime_num", "hour_icu_intime"], groupby="service_unit", show=False, ) plt.gcf().savefig(f"{_TEST_IMAGE_PATH}/tracksplot_scanpy_plt_expected.png", dpi=80) ``` -------------------------------- ### Compute neighbors and PAGA Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/paga_expected.ipynb Calculates neighborhood graph, performs Leiden clustering, and computes PAGA connectivity. ```python ep.pp.neighbors(adata, random_state=0) ep.tl.leiden(adata, resolution=0.5, key_added="leiden_0_5", random_state=0) ep.tl.paga(adata, groups="leiden_0_5") ``` -------------------------------- ### Visualize and save PCA plot Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/pca_scanpy_expected.ipynb Generate a PCA plot colored by service unit and save it to the specified test image path. ```python ep.pl.pca(adata, color="service_unit", show=False) plt.gcf().set_size_inches(8, 6) plt.gcf().subplots_adjust(left=0.2, right=0.8, bottom=0.2, top=0.8) plt.gcf().savefig(f"{_TEST_IMAGE_PATH}/pca_expected.png", dpi=80) ``` -------------------------------- ### Load and preprocess data Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/dpt_timeseries_expected.ipynb Loads a sample dataset, imputes missing gender values, infers feature types, and encodes categorical features. Handles potential NaN values in categorical features. ```python edata = ed.dt.mimic_2().copy()\n# replace NaN in gender_num with most frequent value to avoid NaN in categorical without introducing 3rd category\nep.pp.simple_impute(edata, var_names=["gender_num"], strategy="most_frequent")\ned.infer_feature_types(edata)\nedata = ep.pp.encode(edata, autodetect=True) ``` -------------------------------- ### Import Ehrapy library Source: https://github.com/theislab/ehrapy/blob/main/README.md Import the ehrapy library for use in your Python scripts. This is a common first step when using the library. ```python import ehrapy as ep ``` -------------------------------- ### Generate Time-based Sankey Diagram Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/sankey_matplotlib_expected.ipynb Creates a Sankey diagram representing state transitions over time using a blobs dataset. ```python edata = ed.dt.ehrdata_blobs(base_timepoints=5, n_variables=1, n_observations=5, random_state=59) edata.layers[DEFAULT_TEM_LAYER_NAME] = edata.layers[DEFAULT_TEM_LAYER_NAME].astype(int) ``` ```python sankey_time = ep.pl.sankey_diagram_time( edata, var_name="feature_0", layer=DEFAULT_TEM_LAYER_NAME, state_labels={-2: "no", -3: "mild", -4: "moderate", -5: "severe", -6: "critical"}, ) fig = hv.render(sankey_time, backend="matplotlib") fig.savefig(f"{_TEST_IMAGE_PATH}/sankey_time_expected.png", dpi=80) ``` ```python sankey_time ``` -------------------------------- ### Generate and save matrix plot Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/matrix_scanpy_plot_expected.ipynb Creates a matrix plot for specified variables grouped by service unit and saves the output as a PNG file. ```python ep.pl.matrixplot( adata_sample, var_names=[ "abg_count", "wbc_first", "hgb_first", "potassium_first", "tco2_first", "bun_first", "creatinine_first", "pco2_first", ], groupby="service_unit", show=False, ) plt.gcf().savefig(f"{_TEST_IMAGE_PATH}/matrix_plot_expected.png", dpi=80) ``` -------------------------------- ### Create Heatmap Visualization Source: https://context7.com/theislab/ehrapy/llms.txt Generate heatmaps to visualize feature values across observations or groups. Allows selection of specific features, grouping, and customization of colormaps and dendrograms. ```python import ehrdata as ed import ehrapy as ep edata = ed.dt.mimic_2() edata = ep.pp.encode(edata, autodetect=True) ep.pp.simple_impute(edata, strategy="median") ep.pp.scale_norm(edata) # Heatmap of selected features ep.pl.heatmap( edata, var_names=["age", "bun_first", "creatinine_first", "hgb_first"], groupby="service_unit", show_gene_labels=True ) # Customize heatmap ep.pl.heatmap( edata, var_names=edata.var_names[:10], groupby="gender_num", cmap="RdBu_r", dendrogram=True, swap_axes=True ) ``` -------------------------------- ### Load and Encode MIMIC-II Data Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/pca_loadings_expected.ipynb Loads a subset of the MIMIC-II dataset and performs automatic feature encoding. ```python adata_full = ed.dt.mimic_2(columns_obs_only=["service_unit", "day_icu_intime"]) adata_full = adata_full[~np.isnan(adata_full.X).any(axis=1)].copy() adata = adata_full[:200, :].copy() del adata_full adata = ep.pp.encode(adata, autodetect=True) ``` -------------------------------- ### Perform Leiden Clustering Source: https://context7.com/theislab/ehrapy/llms.txt Identify groups of similar observations using the Leiden algorithm. ```python import ehrdata as ed import ehrapy as ep edata = ed.dt.mimic_2() edata = ep.pp.encode(edata, autodetect=True) ep.pp.simple_impute(edata, strategy="median") ep.pp.neighbors(edata) # Compute Leiden clusters ep.tl.leiden(edata, resolution=0.5) # Access cluster assignments print(edata.obs["leiden"].value_counts()) # 0 523 # 1 412 # 2 356 # ... # Adjust resolution for more/fewer clusters ep.tl.leiden(edata, resolution=1.0, key_added="leiden_fine") ep.tl.leiden(edata, resolution=0.2, key_added="leiden_coarse") # Visualize clusters ep.tl.umap(edata) ep.pl.umap(edata, color="leiden") ``` -------------------------------- ### Mimic Data Creation Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/dendogram_expected.ipynb Generate a sample AnnData object using the `mimic_2` function from Ehrapy's data utilities. The `encoded=True` argument suggests that the data might be in a processed or encoded format. ```python adata = ep.dt.mimic_2(encoded=True) ``` -------------------------------- ### Import Libraries for Ehrapy Plotting Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/catplot_create_expected_plots.ipynb Import matplotlib.pyplot for plotting and ehrapy for data handling and visualization. ```python import matplotlib.pyplot as plt import ehrapy as ep ``` -------------------------------- ### Generate stacked violin plot Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/stacked_violin_scanpy_plt_expected.ipynb Create a stacked violin plot for specified variables grouped by service unit and save the output. ```python ep.pl.stacked_violin( adata_sample, var_names=["icu_los_day", "hospital_los_day", "age", "gender_num", "weight_first", "bmi"], groupby="service_unit", show=False, ) plt.gcf().savefig(f"{_TEST_IMAGE_PATH}/stacked_violin_scanpy_plt_expected.png", dpi=80) ``` -------------------------------- ### Ehrapy Tools - Normalized Complexity Profile Source: https://github.com/theislab/ehrapy/blob/main/docs/api/tools_index.md Tools for calculating the Normalized Complexity Profile. ```APIDOC ## Ehrapy Tools - Normalized Complexity Profile ### Description Tools for calculating the Normalized Complexity Profile. ### Available Functions - `ehrapy.tools.ncp` ``` -------------------------------- ### Visualize PCA variance ratio Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/pca_variance_ratio_expected.ipynb Plot the PCA variance ratio and save the figure to the specified test image path. ```python ep.pl.pca_variance_ratio(adata, show=False) plt.gcf().set_size_inches(8, 6) plt.gcf().subplots_adjust(left=0.2, right=0.8, bottom=0.2, top=0.8) plt.gcf().savefig(f"{_TEST_IMAGE_PATH}/pca_variance_ratio_expected.png", dpi=80) ``` -------------------------------- ### Generate and Save Clustermap Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/clustermap_scanpy_expected.ipynb Generates a clustermap from the cleaned data and saves the figure to a specified path with a given DPI. The `show=False` argument prevents the plot from displaying directly. ```python ep.pl.clustermap(adata_clean, show=False) plt.gcf().savefig(f"{_TEST_IMAGE_PATH}/clustermap_scanpy_expected.png", dpi=80) ``` -------------------------------- ### Generate Standard Sankey Diagram Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/sankey_matplotlib_expected.ipynb Loads a diabetes dataset and renders a Sankey diagram based on gender and race columns. ```python edata = ed.dt.diabetes_130_fairlearn(columns_obs_only=["gender", "race"])[:100] ``` ```python sankey = ep.pl.sankey_diagram(edata, columns=["gender", "race"]) fig = hv.render(sankey, backend="matplotlib") fig.savefig(f"{_TEST_IMAGE_PATH}/sankey_expected.png", dpi=80) ``` ```python sankey ``` -------------------------------- ### Track Cohort with Loose Category and Plot Bar Chart Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/cohort_tracker_test_create_expected_plots.ipynb Demonstrates tracking a cohort with a 'loose category' in the data and plotting a bar chart using a 'colorblind' palette. The plot is saved to a file. ```python adata_mini_loose_category = adata_mini.copy() ct = ep.tl.CohortTracker(adata_mini_loose_category) ct(adata_mini_loose_category, label="First step", operations_done="Some operations") adata_mini_loose_category = adata_mini_loose_category[adata_mini_loose_category.obs.disease == "A", :] ct(adata_mini_loose_category) fig_loose_category, _ = ct.plot_cohort_barplot( color_palette="colorblind", legend_labels={"weight": "weight(kg)", "glucose": "glucose(mg/dL)"}, show=False ) fig_loose_category.savefig( f"{_TEST_IMAGE_PATH}/cohorttracker_edata_mini_step2_loose_category_expected.png", dpi=80, ) ``` -------------------------------- ### Set Figure Parameters Source: https://github.com/theislab/ehrapy/blob/main/docs/api/settings_index.md Configures the global figure parameters for ehrapy visualizations. ```APIDOC ## ep.settings.set_figure_params ### Description Sets the global figure parameters for ehrapy, such as DPI. ### Parameters #### Arguments - **dpi** (int) - Optional - The dots per inch for figure resolution. ### Request Example ```python import ehrapy as ep ep.settings.set_figure_params(dpi=150) ``` ``` -------------------------------- ### Ehrapy Tools - Cohort Tracking Source: https://github.com/theislab/ehrapy/blob/main/docs/api/tools_index.md Tools for tracking cohorts. ```APIDOC ## Ehrapy Tools - Cohort Tracking ### Description Tools for tracking cohorts. ### Available Functions - `ehrapy.tools.CohortTracker` ``` -------------------------------- ### Load Test Data Source: https://github.com/theislab/ehrapy/blob/main/tests/_scripts/cohort_tracker_test_create_expected_plots.ipynb Loads a mini dataset from a CSV file for testing purposes. It specifies columns to be treated as observation-only data. ```python current_notebook_dir = %pwd _TEST_DATA_PATH = f"{current_notebook_dir}/ehrapy_data/dataset1.csv" _TEST_IMAGE_PATH = f"{current_notebook_dir}/../tools/_images" adata_mini = ed.io.read_csv(_TEST_DATA_PATH, columns_obs_only=["glucose", "weight", "disease", "station"]) ```