### Install diffxpy Source: https://context7.com/theislab/diffxpy/llms.txt Installs the diffxpy library using pip. This is the first step to using the library for differential expression analysis. ```python pip install diffxpy ``` -------------------------------- ### Configure NumPy/SciPy Threading via Shell Environment Source: https://github.com/theislab/diffxpy/blob/master/docs/parallelization.rst Limits multi-threading for NumPy and SciPy linear algebra operations by setting environment variables in the shell before starting the Python session. ```bash export MKL_NUM_THREADS=1 export NUMEXPR_NUM_THREADS=1 export OMP_NUM_THREADS=1 ``` -------------------------------- ### Fit Models and Access Parameters Source: https://context7.com/theislab/diffxpy/llms.txt Demonstrates how to fit a negative binomial model to AnnData objects and extract model parameters such as location coefficients, scale parameters, predicted means, and log-likelihoods. ```python import diffxpy.api as de import numpy as np import pandas as pd import anndata n_cells, n_genes = 200, 50 counts = np.random.negative_binomial(n=5, p=0.4, size=(n_cells, n_genes)) sample_description = pd.DataFrame({"condition": np.repeat(["A", "B"], n_cells // 2), "batch": np.tile(["1", "2"], n_cells // 2)}) adata = anndata.AnnData(X=counts, obs=sample_description) adata.var_names = [f"gene_{i}" for i in range(n_genes)] estimator = de.fit.model(data=adata, formula_loc="~ 1 + condition + batch", formula_scale="~ 1", noise_model="nb") a = estimator.model.a b = estimator.model.b mu = estimator.model.location ll = estimator.model.ll_byfeature ``` -------------------------------- ### Create Design Matrix from Annotations Source: https://context7.com/theislab/diffxpy/llms.txt Constructs a design matrix from sample metadata using patsy-style formulas, allowing for the inclusion of categorical factors and numeric covariates. ```python import diffxpy.api as de import pandas as pd sample_description = pd.DataFrame({"condition": ["A", "A", "B", "B", "C", "C"], "batch": ["1", "2", "1", "2", "1", "2"], "time": [0, 1, 2, 3, 4, 5]}) dmat = de.utils.design_matrix(sample_description=sample_description, formula="~ 1 + condition + batch") dmat_numeric = de.utils.design_matrix(sample_description=sample_description, formula="~ 1 + condition + time", as_numeric=["time"]) coef_names = de.utils.preview_coef_names(sample_description=sample_description, formula="~ 1 + condition * batch", as_numeric=[]) ``` -------------------------------- ### Cleaning P-values and Generating Summary Tables Source: https://context7.com/theislab/diffxpy/llms.txt Shows how to retrieve cleaned log-transformed p-values and q-values that handle zeros and NaNs, and how to extract a full summary table as a pandas DataFrame. ```python log10_pval = test_result.log10_pval_clean(log10_threshold=-30) log10_qval = test_result.log10_qval_clean(log10_threshold=-30) summary = test_result.summary() print(summary.columns.tolist()) ``` -------------------------------- ### de.utils.design_matrix - Create Design Matrix Source: https://context7.com/theislab/diffxpy/llms.txt Creates a design matrix from sample annotations using patsy formula syntax. ```APIDOC ## de.utils.design_matrix - Create Design Matrix ### Description Creates a design matrix from sample annotations using patsy formula syntax. ### Method `de.utils.design_matrix` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **sample_description** (pandas.DataFrame) - Required - DataFrame containing sample annotations. - **formula** (string) - Required - Patsy-style formula string. - **as_numeric** (list of strings) - Optional - List of column names in `sample_description` to be treated as numeric covariates. ### Request Example ```python import diffxpy.api as de import pandas as pd import numpy as np # Sample annotations sample_description = pd.DataFrame({ "condition": ["A", "A", "B", "B", "C", "C"], "batch": ["1", "2", "1", "2", "1", "2"], "time": [0, 1, 2, 3, 4, 5] # Numeric covariate }) # Create design matrix with categorical factors dmat = de.utils.design_matrix( sample_description=sample_description, formula="~ 1 + condition + batch" ) print("Design matrix shape:", dmat.shape) print("Column names:", dmat.design_info.column_names) # Treat 'time' as numeric (not categorical) dmat_numeric = de.utils.design_matrix( sample_description=sample_description, formula="~ 1 + condition + time", as_numeric=["time"] ) print("With numeric time:", dmat_numeric.design_info.column_names) ``` ### Response #### Success Response (200) - **dmat** (patsy.design_info.DesignInfo) - The created design matrix object. #### Response Example ```python # Preview coefficient names without building full matrix coef_names = de.utils.preview_coef_names( sample_description=sample_description, formula="~ 1 + condition * batch", as_numeric=[] ) print("Coefficient names:", coef_names) ``` ``` -------------------------------- ### Perform Gene Set Enrichment Analysis with Diffxpy Source: https://context7.com/theislab/diffxpy/llms.txt This snippet illustrates how to perform gene set enrichment analysis using diffxpy. It covers creating or loading gene sets, preparing differential expression scores (q-values), and running the enrichment test. The results can be summarized, filtered for significant sets, or detailed for specific pathways. ```python import diffxpy.api as de import numpy as np import pandas as pd import anndata # Create sample differential expression results n_genes = 500 gene_names = [f"GENE{i}" for i in range(n_genes)] # Simulated q-values (some significant) qvalues = np.random.uniform(0, 1, n_genes) qvalues[:50] = np.random.uniform(0, 0.05, 50) # 50 significant genes # Create gene sets (RefSets) # Option 1: From .gmt file # ref_sets = de.enrich.RefSets(fn="path/to/gene_sets.gmt", type="gmt") # Option 2: Create programmatically gene_set_data = [ ["PATHWAY_A", "http://source.com/A", "GENE1", "GENE5", "GENE10", "GENE15", "GENE20"], ["PATHWAY_B", "http://source.com/B", "GENE2", "GENE6", "GENE11", "GENE25", "GENE30"], ["PATHWAY_C", "http://source.com/C", "GENE3", "GENE7", "GENE12", "GENE35", "GENE40"], ] ref_sets = de.enrich.RefSets(sets=gene_set_data, type="gmt") # Add more gene sets manually ref_sets.add( id="CUSTOM_PATHWAY", source="manual", gene_ids=["GENE45", "GENE46", "GENE47", "GENE48", "GENE49"] ) # Run enrichment analysis enrichment_result = de.enrich.test( ref=ref_sets, scores=qvalues, # Use q-values as scores gene_ids=gene_names, # Gene identifiers threshold=0.05, # Significance threshold for scores capital=True, # Convert gene IDs to uppercase clean_ref=False # Whether to filter reference sets ) # Get enrichment results summary = enrichment_result.summary() print(summary) # Columns: set, pval, qval, intersection, reference, enquiry, background # Get significant gene sets significant_sets = enrichment_result.significant_set_ids(threshold=0.05) print(f"Significant pathways: {significant_sets}") # Get details for a specific set if len(ref_sets.sets) > 0: set_detail = enrichment_result.set_summary(ref_sets._ids[0]) print(set_detail) ``` -------------------------------- ### Manage Gene Sets with Diffxpy RefSets Class Source: https://context7.com/theislab/diffxpy/llms.txt This snippet demonstrates the usage of the `de.enrich.RefSets` class for managing gene sets. It shows how to create an empty `RefSets` object, add gene sets programmatically, load from a GMT file, search for sets, retrieve specific sets, subset based on keywords, and clean sets to match a background gene list. ```python import diffxpy.api as de import numpy as np # Create empty RefSets and add sets manually ref_sets = de.enrich.RefSets() # Add gene sets one by one ref_sets.add( id="APOPTOSIS", source="KEGG", gene_ids=["BCL2", "BAX", "CASP3", "CASP9", "CYCS", "TP53"] ) ref_sets.add( id="CELL_CYCLE", source="GO", gene_ids=["CDK1", "CDK2", "CCNA2", "CCNB1", "CCND1", "RB1"] ) ref_sets.add( id="INFLAMMATION", source="Reactome", gene_ids=["IL6", "IL1B", "TNF", "NFKB1", "STAT3", "CXCL8"] ) # Load from GMT file (MSigDB format) # ref_sets_msigdb = de.enrich.RefSets(fn="c2.cp.kegg.v7.5.symbols.gmt", type="gmt") # Search for gene sets by keyword matching_sets = ref_sets.grepv_sets(["APOP", "CYCLE"]) print(f"Matching sets: {matching_sets}") # Get a specific set apoptosis_set = ref_sets.get_set("APOPTOSIS") print(f"Genes in APOPTOSIS: {apoptosis_set.genes}") # Subset by keywords subset = ref_sets.subset_bykey(keys=["CELL"]) print(f"Filtered sets: {subset._ids}") # Clean reference sets to only include genes in your data background_genes = {"BCL2", "BAX", "CASP3", "CDK1", "CDK2", "IL6", "IL1B", "TP53"} ref_sets.clean(background_genes) ``` -------------------------------- ### Extracting Statistical Metrics from Test Results Source: https://context7.com/theislab/diffxpy/llms.txt Demonstrates how to access raw p-values, FDR-corrected q-values, gene identifiers, and mean expression levels from a diffxpy test result object. It also shows how to compute various log-fold change metrics. ```python pvalues = test_result.pval qvalues = test_result.qval gene_ids = test_result.gene_ids mean_expr = test_result.mean log2fc = test_result.log2_fold_change() log10fc = test_result.log10_fold_change() lnfc = test_result.log_fold_change() ``` -------------------------------- ### Configure TensorFlow Threading via Package Constants Source: https://github.com/theislab/diffxpy/blob/master/docs/parallelization.rst Adjusts TensorFlow parallelization settings after importing the package but before the first session is initialized. This allows dynamic configuration of inter-op and intra-op parallelism. ```python import diffxpy.api as de from batchglm.pkg_constants import TF_CONFIG_PROTO TF_CONFIG_PROTO.inter_op_parallelism_threads = 1 TF_CONFIG_PROTO.intra_op_parallelism_threads = 1 from batchglm.pkg_constants import TF_LOOP_PARALLEL_ITERATIONS TF_LOOP_PARALLEL_ITERATIONS = 1 ``` -------------------------------- ### Import Diffxpy API Source: https://github.com/theislab/diffxpy/blob/master/docs/api/index.rst Imports the high-level API of the diffxpy library for convenient access to its functionalities. This is the primary way to interact with diffxpy's core features. ```python import diffxpy.api as de ``` -------------------------------- ### Configure TensorFlow Threading via Environment Variables Source: https://github.com/theislab/diffxpy/blob/master/docs/parallelization.rst Sets environment variables to restrict TensorFlow to a single thread before importing diffxpy. This must be executed before the package is loaded to take effect. ```python import os os.environ.setdefault("TF_NUM_THREADS", "1") os.environ.setdefault("TF_LOOP_PARALLEL_ITERATIONS", "1") import diffxpy.api as de ``` -------------------------------- ### Generating Volcano Plots Source: https://context7.com/theislab/diffxpy/llms.txt Illustrates how to visualize differential expression results using a volcano plot, including options for significance thresholds, fold change cutoffs, and gene highlighting. ```python test_result.plot_volcano( corrected_pval=True, alpha=0.05, min_fc=1.5, size=20, highlight_ids=["gene_0", "gene_1"], highlight_col="red", save="my_volcano", suffix="_volcano.png", show=True ) ``` -------------------------------- ### Fit Generalized Linear Model in Diffxpy Source: https://context7.com/theislab/diffxpy/llms.txt This snippet shows how to use `de.fit.model` to fit a generalized linear model for each gene in the dataset. This function is useful for model diagnostics and residual analysis, as it fits the model without performing differential expression testing. It requires an AnnData object as input. ```python import diffxpy.api as de import numpy as np import pandas as pd import anndata ``` -------------------------------- ### Fit Models to Gene Expression in Diffxpy Source: https://github.com/theislab/diffxpy/blob/master/docs/api/index.rst Allows fitting statistical models to gene expression data without performing differential tests. This functionality can also be used to extract model fits or compute residuals, and can be partitioned across data subsets. ```python de.fit.model de.fit.residuals de.fit.partition ``` -------------------------------- ### Reference Gene Sets in Diffxpy Source: https://github.com/theislab/diffxpy/blob/master/docs/api/index.rst Provides functionality for managing reference gene sets used in enrichment analysis. This includes loading and creating annotated gene sets for comparison with differential expression results. ```python de.enrich.RefSets ``` -------------------------------- ### Perform Wald Test for Differential Expression in Diffxpy Source: https://context7.com/theislab/diffxpy/llms.txt This snippet demonstrates how to perform differential expression testing using the Wald test in diffxpy. It covers fitting a model with specified factors and then testing for the effect of a particular factor, including repeated measures if applicable. The results are summarized by counting significant genes. ```python import diffxpy.api as de import numpy as np import pandas as pd import anndata # Create sample data with multiple factors n_cells, n_genes = 300, 50 counts = np.random.negative_binomial(n=4, p=0.3, size=(n_cells, n_genes)) sample_description = pd.DataFrame({ "condition": np.repeat(["A", "B", "C"], n_cells // 3), "treatment": np.tile(["control", "drug"], n_cells // 2), "batch": np.tile(["1", "2", "3"], n_cells // 3) }) adata = anndata.AnnData(X=counts, obs=sample_description) adata.var_names = [f"gene_{i}" for i in range(n_genes)] # First, fit model and test condition effect initial_test = de.test.wald( data=adata, formula_loc="~ 1 + condition + treatment + batch", factor_loc_totest="condition", noise_model="nb" ) # Now test treatment effect using the same fitted model (no refitting needed) treatment_test = de.test.wald_repeated( det=initial_test, # Previous test result with fitted model factor_loc_totest="treatment" # New factor to test ) # Compare results print(f"Condition effect - significant genes: {sum(initial_test.qval < 0.05)}") print(f"Treatment effect - significant genes: {sum(treatment_test.qval < 0.05)}") ``` -------------------------------- ### de.test.two_sample - Unified Two-Sample Test Interface Source: https://context7.com/theislab/diffxpy/llms.txt Provides a unified interface for two-sample differential expression testing, supporting multiple statistical tests including t-test, Wilcoxon rank test, and model-based tests like Wald and LRT. ```APIDOC ## de.test.two_sample - Unified Two-Sample Test Interface ### Description A convenience wrapper that provides a unified interface for two-sample differential expression testing, supporting multiple statistical tests. ### Method `de.test.two_sample` ### Parameters #### Data - **data** (anndata.AnnData) - The AnnData object containing the gene expression data. - **grouping** (str or array-like) - Column name in `adata.obs` or an array with group labels for each cell. - **test** (str) - The statistical test to perform. Options include: "t-test", "rank" (Wilcoxon rank sum), "wald", "lrt". - **noise_model** (str, optional) - The noise model to use for model-based tests (e.g., "nb" for negative binomial). Required for "wald" and "lrt" tests. ### Request Example ```python import diffxpy.api as de import numpy as np import pandas as pd import anndata # Create sample data n_cells, n_genes = 250, 100 counts = np.random.negative_binomial(n=5, p=0.35, size=(n_cells, n_genes)) grouping = np.array(["control"] * 125 + ["treated"] * 125) sample_description = pd.DataFrame({"treatment": grouping}) adata = anndata.AnnData(X=counts, obs=sample_description) adata.var_names = [f"gene_{i}" for i in range(n_genes)] # Run with different test methods # Option 1: Welch's t-test (fast, non-parametric) test_ttest = de.test.two_sample( data=adata, grouping="treatment", test="t-test" ) # Option 2: Wilcoxon rank test (non-parametric) test_rank = de.test.two_sample( data=adata, grouping="treatment", test="rank" ) # Option 3: Wald test with negative binomial model (model-based) test_wald = de.test.two_sample( data=adata, grouping="treatment", test="wald", noise_model="nb" ) # Option 4: Likelihood ratio test (model-based) test_lrt = de.test.two_sample( data=adata, grouping="treatment", test="lrt", noise_model="nb" ) # Compare results print("T-test significant genes:", sum(test_ttest.qval < 0.05)) print("Rank test significant genes:", sum(test_rank.qval < 0.05)) print("Wald test significant genes:", sum(test_wald.qval < 0.05)) print("LRT significant genes:", sum(test_lrt.qval < 0.05)) ``` ### Response #### Success Response (200) Returns a test result object (e.g., `DiffxpyResult`) containing p-values, q-values, and other statistics depending on the chosen test method. #### Response Example ```json { "qval": [0.01, 0.05, 0.1, ...] } ``` ``` -------------------------------- ### Multiple Tests Per Gene in Diffxpy Source: https://github.com/theislab/diffxpy/blob/master/docs/api/index.rst Provides infrastructure for performing multiple differential expression tests per gene. This includes pairwise comparisons, tests of each group against the rest, and partitioning tests based on experimental covariates or cell types. ```python de.test.pairwise de.test.versus_rest de.test.partition ``` -------------------------------- ### Load AnnData for diffxpy Source: https://context7.com/theislab/diffxpy/llms.txt Loads single-cell data into an AnnData object, which is the primary data structure used by diffxpy and scanpy. The AnnData object should contain the count matrix (adata.X) and cell annotations (adata.obs). ```python import diffxpy.api as de import anndata import numpy as np import pandas as pd # Load single-cell data (cells x genes matrix with annotations) # adata.X contains count matrix, adata.obs contains cell metadata adata = anndata.read_h5ad("my_single_cell_data.h5ad") ``` -------------------------------- ### Unified Two-Sample Test Interface for Differential Expression Source: https://context7.com/theislab/diffxpy/llms.txt Provides a unified interface to perform various two-sample differential expression tests, including t-test, Wilcoxon rank test, Wald test, and Likelihood Ratio Test. It accepts an AnnData object, grouping, and specifies the desired test and noise model. Results can be compared across different test methods. ```python import diffxpy.api as de import numpy as np import pandas as pd import anndata # Create sample data n_cells, n_genes = 250, 100 counts = np.random.negative_binomial(n=5, p=0.35, size=(n_cells, n_genes)) grouping = np.array(["control"] * 125 + ["treated"] * 125) sample_description = pd.DataFrame({"treatment": grouping}) adata = anndata.AnnData(X=counts, obs=sample_description) adata.var_names = [f"gene_{i}" for i in range(n_genes)] # Run with different test methods # Option 1: Welch's t-test (fast, non-parametric) test_ttest = de.test.two_sample( data=adata, grouping="treatment", test="t-test" ) # Option 2: Wilcoxon rank test (non-parametric) test_rank = de.test.two_sample( data=adata, grouping="treatment", test="rank" ) # Option 3: Wald test with negative binomial model (model-based) test_wald = de.test.two_sample( data=adata, grouping="treatment", test="wald", noise_model="nb" ) # Option 4: Likelihood ratio test (model-based) test_lrt = de.test.two_sample( data=adata, grouping="treatment", test="lrt", noise_model="nb" ) # Compare results print("T-test significant genes:", sum(test_ttest.qval < 0.05)) print("Rank test significant genes:", sum(test_rank.qval < 0.05)) print("Wald test significant genes:", sum(test_wald.qval < 0.05)) print("LRT significant genes:", sum(test_lrt.qval < 0.05)) ``` -------------------------------- ### Execute Stratified Differential Expression Analysis Source: https://context7.com/theislab/diffxpy/llms.txt Partitions data by a factor to perform independent differential expression tests within each partition. This is ideal for batch-aware analysis or cell-type-specific comparisons. ```python import diffxpy.api as de import numpy as np import pandas as pd import anndata n_cells, n_genes = 400, 60 counts = np.random.negative_binomial(n=4, p=0.3, size=(n_cells, n_genes)) sample_description = pd.DataFrame({ "batch": np.tile(["batch_A", "batch_B"], n_cells // 2), "condition": np.repeat(["control", "treated"], n_cells // 2) }) adata = anndata.AnnData(X=counts, obs=sample_description) adata.var_names = [f"gene_{i}" for i in range(n_genes)] partition = de.test.partition(data=adata, parts="batch") test_result = partition.t_test(grouping="condition", is_logged=False) summary = test_result.summary() print(summary.head()) ``` -------------------------------- ### Result Object Methods - Summary and Visualization Source: https://context7.com/theislab/diffxpy/llms.txt All differential expression test results provide common methods for accessing results and visualization. ```APIDOC ## Result Object Methods - Summary and Visualization ### Description All differential expression test results provide common methods for accessing results and visualization. ### Method Methods available on the result object returned by differential expression testing functions (e.g., `de.test.t_test`). ### Parameters None (These are methods of a result object). ### Request Example ```python import diffxpy.api as de import numpy as np import pandas as pd import anndata # Run a differential expression test n_cells, n_genes = 200, 100 counts = np.random.negative_binomial(n=5, p=0.3, size=(n_cells, n_genes)) grouping = np.array(["control"] * 100 + ["treated"] * 100) adata = anndata.AnnData(X=counts) ata.var_names = [f"gene_{i}" for i in range(n_genes)] test_result = de.test.t_test( data=adata, grouping=grouping, is_logged=False ) # Example: Accessing p-values pvals = test_result.pvals() print(f"P-values shape: {pvals.shape}") # Example: Accessing log fold changes lfc = test_result.logfc() print(f"Log fold change shape: {lfc.shape}") # Example: Plotting results (requires matplotlib) # test_result.plot_volcano() # test_result.plot_ma() ``` ### Response #### Success Response (200) - **Methods** (object) - The result object provides methods to access various statistics (p-values, log fold changes, etc.) and generate plots. #### Response Example ```python # Accessing summary statistics summary = test_result.summary() print(summary) # Accessing specific statistics for a gene gene_stats = test_result.summary(gene_names=['gene_0']) print(gene_stats) ``` ``` -------------------------------- ### Compute Residuals for Batch Effect Correction Source: https://context7.com/theislab/diffxpy/llms.txt Shows how to compute residuals by regressing out nuisance variables like batch effects. These residuals are useful for downstream analysis such as PCA or clustering. ```python import diffxpy.api as de import numpy as np import pandas as pd import anndata n_cells, n_genes = 300, 80 counts = np.random.negative_binomial(n=4, p=0.35, size=(n_cells, n_genes)) sample_description = pd.DataFrame({"batch": np.repeat(["batch1", "batch2", "batch3"], n_cells // 3), "condition": np.tile(["ctrl", "treat"], n_cells // 2)}) adata = anndata.AnnData(X=counts, obs=sample_description) adata.var_names = [f"gene_{i}" for i in range(n_genes)] residuals = de.fit.residuals(data=adata, formula_loc="~ 1 + batch", formula_scale="~ 1", noise_model="nb") ``` -------------------------------- ### Gene Set Enrichment Analysis in Diffxpy Source: https://github.com/theislab/diffxpy/blob/master/docs/api/index.rst Enables gene set enrichment analysis downstream of differential expression results. It allows loading or creating reference gene set annotations and comparing them with diffxpy objects or differential expression test outcomes. ```python de.enrich.test ``` -------------------------------- ### Perform One-vs-Rest Differential Expression Analysis Source: https://context7.com/theislab/diffxpy/llms.txt Tests each group against all other groups combined to identify marker genes. This function requires an AnnData object and a grouping factor, returning statistical results including p-values and log fold changes. ```python import diffxpy.api as de import numpy as np import pandas as pd import anndata n_cells, n_genes = 400, 80 counts = np.random.negative_binomial(n=5, p=0.35, size=(n_cells, n_genes)) cell_types = np.array(["Stem"] * 100 + ["Progenitor"] * 100 + ["Differentiated"] * 100 + ["Terminal"] * 100) sample_description = pd.DataFrame({"stage": cell_types}) adata = anndata.AnnData(X=counts, obs=sample_description) adata.var_names = [f"gene_{i}" for i in range(n_genes)] test_result = de.test.versus_rest( data=adata, grouping="stage", test="wald", noise_model="nb", pval_correction="global" ) pvals = test_result.pval qvals = test_result.qval groups = test_result.groups for i, group in enumerate(groups): significant = np.sum(qvals[0, i, :] < 0.05) print(f"Significant markers for {group}: {significant}") ``` -------------------------------- ### Likelihood Ratio Test (LRT) for Differential Expression (Python) Source: https://context7.com/theislab/diffxpy/llms.txt Performs a Likelihood Ratio Test (LRT) to compare a full statistical model against a reduced model. This is useful for testing the significance of multiple coefficients simultaneously or when a nested model comparison is preferred. The function requires both full and reduced model formulas for the location and scale parameters, along with the AnnData object and noise model. It returns a test result object containing log-likelihood values and summary statistics. ```python import diffxpy.api as de import numpy as np import pandas as pd import anndata # Create sample data n_cells, n_genes = 300, 50 counts = np.random.negative_binomial(n=4, p=0.25, size=(n_cells, n_genes)) sample_description = pd.DataFrame({ "condition": np.repeat(["A", "B", "C"], n_cells // 3), "batch": np.tile(["batch1", "batch2", "batch3"], n_cells // 3) }) adata = anndata.AnnData(X=counts, obs=sample_description) adata.var_names = [f"gene_{i}" for i in range(n_genes)] # Run LRT: compare full model with condition effect vs reduced model without test_result = de.test.lrt( data=adata, full_formula_loc="~ 1 + condition + batch", # Full model reduced_formula_loc="~ 1 + batch", # Reduced model (no condition) full_formula_scale="~ 1", # Scale model (dispersion) reduced_formula_scale="~ 1", noise_model="nb" ) # Get differential expression results results = test_result.summary() print(f"Genes tested: {len(results)}") print(f"Significant genes (q < 0.05): {sum(results['qval'] < 0.05)}") # Access log-likelihood values ll = test_result.log_likelihood ``` -------------------------------- ### Single Gene Differential Expression Tests in Diffxpy Source: https://github.com/theislab/diffxpy/blob/master/docs/api/index.rst Performs standard differential expression tests where a single p-value is computed for each gene. Diffxpy supports likelihood ratio tests, Wald tests, t-tests, and Wilcoxon tests for this purpose. ```python de.test.two_sample de.test.wald de.test.lrt de.test.t_test de.test.rank_test ``` -------------------------------- ### Pairwise Group Comparisons for Differential Expression Source: https://context7.com/theislab/diffxpy/llms.txt Performs differential expression tests between all combinations of groups in a multi-group dataset. It supports efficient tests like 'z-test' and offers options for lazy computation and p-value correction. The output provides matrices for p-values, q-values, and log fold changes for all pairwise comparisons. ```python import diffxpy.api as de import numpy as np import pandas as pd import anndata # Create sample data with multiple groups n_cells, n_genes = 300, 50 counts = np.random.negative_binomial(n=4, p=0.3, size=(n_cells, n_genes)) # 4 cell types cell_types = np.array(["T_cell"] * 75 + ["B_cell"] * 75 + ["Monocyte"] * 75 + ["NK_cell"] * 75) sample_description = pd.DataFrame({"cell_type": cell_types}) adata = anndata.AnnData(X=counts, obs=sample_description) adata.var_names = [f"gene_{i}" for i in range(n_genes)] # Run pairwise comparisons using z-test (efficient, requires only one model fit) test_result = de.test.pairwise( data=adata, grouping="cell_type", test="z-test", # Efficient coefficient-based test lazy=True, # Compute p-values only when requested noise_model="nb", pval_correction="global" # Correct all p-values together ) # Access specific pairwise comparison pval_matrix = test_result.pval # genes x groups x groups qval_matrix = test_result.qval lfc_matrix = test_result.log_fold_change() # Get summary for specific comparison (T_cell vs B_cell) # Groups are indexed in order they appear groups = np.unique(cell_types) print(f"Groups: {groups}") # Access p-values for gene 0, comparing group 0 (B_cell) vs group 3 (T_cell) print(f"P-value gene_0, B_cell vs T_cell: {pval_matrix[0, 3, 0]:.4e}") ``` -------------------------------- ### Apply Multiple Testing Correction Source: https://context7.com/theislab/diffxpy/llms.txt Applies statistical correction methods to raw p-values to control for false discovery rates, supporting various methods like Benjamini-Hochberg and Bonferroni. ```python import diffxpy.api as de import numpy as np pvalues = np.array([0.001, 0.01, 0.03, 0.04, 0.05, 0.1, 0.2, 0.5, 0.8, 0.99]) qvalues_bh = de.stats.correct(pvalues, method="fdr_bh") qvalues_bonf = de.stats.correct(pvalues, method="bonferroni") ``` -------------------------------- ### Model Fitting Source: https://github.com/theislab/diffxpy/blob/master/docs/api/index.rst Fit models to gene expression data without conducting differential tests, or extract model fits from existing test results. ```APIDOC ## Model Fitting ### Description Allows fitting statistical models to gene expression data without performing Wald or likelihood ratio tests. Model fits can also be extracted from differential expression test output objects. Residuals can be computed directly, and fitting can be partitioned across data subsets. ### Methods - `diffxpy.api.fit.model`: To fit a statistical model to gene expression data. - `diffxpy.api.fit.residuals`: To compute residuals directly. - `diffxpy.api.fit.partition`: To distribute the fitting process across partitions of the data. ### Usage Example ```python import diffxpy.api as de # Example for fitting a model (replace with actual data and parameters) # model_fit = de.fit.model(counts, metadata, "group") # Example for computing residuals (replace with actual data and parameters) # residuals = de.fit.residuals(counts, metadata, "group") # Example for partitioned fitting (replace with actual data and parameters) # partitioned_fit = de.fit.partition(counts, metadata, "group", "condition") ``` ``` -------------------------------- ### Gene Set Enrichment Analysis Source: https://github.com/theislab/diffxpy/blob/master/docs/api/index.rst Perform gene set enrichment analysis using reference gene sets. ```APIDOC ## Gene Set Enrichment Analysis ### Description Provides infrastructure for gene set enrichment analysis downstream of differential expression analysis. Allows loading or creating reference gene sets and comparing them to diffxpy objects or differential expression results. ### Modules - `diffxpy.enrich.RefSet`: For managing reference gene sets. - `diffxpy.enrich.test`: For performing enrichment tests. ### Usage Example ```python import diffxpy.api as de # Example for loading reference gene sets (replace with actual path) # ref_sets = de.enrich.load_ref_sets("path/to/gene_sets.gmt") # Example for performing an enrichment test (replace with actual data and results) # enrichment_results = de.enrich.test(de_results, ref_sets) ``` ``` -------------------------------- ### de.stats.correct - Multiple Testing Correction Source: https://context7.com/theislab/diffxpy/llms.txt Applies multiple testing correction to p-values using methods available in statsmodels. ```APIDOC ## de.stats.correct - Multiple Testing Correction ### Description Applies multiple testing correction to p-values using methods available in statsmodels. ### Method `de.stats.correct` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **pvalues** (numpy.ndarray) - Required - A 1D numpy array of p-values to correct. - **method** (string) - Optional - The multiple testing correction method to use. Defaults to 'fdr_bh' (Benjamini-Hochberg). ### Request Example ```python import diffxpy.api as de import numpy as np # Raw p-values from some analysis pvalues = np.array([0.001, 0.01, 0.03, 0.04, 0.05, 0.1, 0.2, 0.5, 0.8, 0.99]) # Apply Benjamini-Hochberg FDR correction (default) qvalues_bh = de.stats.correct(pvalues, method="fdr_bh") print("BH-corrected q-values:", qvalues_bh) # Apply Bonferroni correction qvalues_bonf = de.stats.correct(pvalues, method="bonferroni") print("Bonferroni-corrected q-values:", qvalues_bonf) ``` ### Response #### Success Response (200) - **qvalues** (numpy.ndarray) - A 1D numpy array of corrected p-values (q-values). #### Response Example ```python # Available methods include: # - "fdr_bh": Benjamini-Hochberg (default) # - "fdr_by": Benjamini-Yekutieli # - "bonferroni": Bonferroni # - "holm": Holm-Bonferroni # - "hommel": Hommel # - "simes-hochberg": Simes-Hochberg # See statsmodels.stats.multitest.multipletests for full list ``` ``` -------------------------------- ### Analyze Differential Expression along Continuous Covariates Source: https://context7.com/theislab/diffxpy/llms.txt Tests for gene expression changes relative to a continuous variable, such as pseudotime, using spline basis functions. The function supports complex model formulas to account for additional covariates. ```python import diffxpy.api as de import numpy as np import pandas as pd import anndata n_cells, n_genes = 300, 40 counts = np.random.negative_binomial(n=5, p=0.4, size=(n_cells, n_genes)) pseudotime = np.linspace(0, 1, n_cells) + np.random.normal(0, 0.05, n_cells) pseudotime = np.clip(pseudotime, 0, 1) sample_description = pd.DataFrame({ "pseudotime": pseudotime, "batch": np.tile(["A", "B", "C"], n_cells // 3 + 1)[:n_cells] }) adata = anndata.AnnData(X=counts, obs=sample_description) adata.var_names = [f"gene_{i}" for i in range(n_genes)] test_result = de.test.continuous_1d( data=adata, continuous="pseudotime", factor_loc_totest="pseudotime", formula_loc="~ 1 + pseudotime + batch", df=5, spline_basis="bs", test="wald", noise_model="nb" ) results = test_result.summary() print(f"Genes significantly varying with pseudotime: {sum(results['qval'] < 0.05)}") ``` -------------------------------- ### Differential Expression Tests - Multiple Tests Source: https://github.com/theislab/diffxpy/blob/master/docs/api/index.rst Perform multiple differential expression tests per gene, including pairwise comparisons and comparisons against a reference. ```APIDOC ## Differential Expression Tests - Multiple Tests ### Description Perform multiple differential expression tests per gene. This includes pairwise comparisons between groups, comparing each group against the rest, and partitioning tests based on covariates. ### Methods - `diffxpy.api.test.pairwise`: For pairwise comparisons across more than two groups. - `diffxpy.api.test.versus_rest`: For t-tests of each group against the rest. - `diffxpy.api.test.partition`: For mapping differential tests across partitions of the data (e.g., by experimental covariate or cell type). ### Usage Example ```python import diffxpy.api as de # Example for pairwise tests (replace with actual data and parameters) # de.test.pairwise(counts, metadata, "group") # Example for versus_rest tests (replace with actual data and parameters) # de.test.versus_rest(counts, metadata, "group") # Example for partition tests (replace with actual data and parameters) # de.test.partition(counts, metadata, "group", "condition") ``` ``` -------------------------------- ### Wald Test for Differential Expression (Python) Source: https://context7.com/theislab/diffxpy/llms.txt Performs a Wald test for differential gene expression using a negative binomial generalized linear model. It tests if the coefficients for a specified factor (e.g., 'condition') are significantly different from zero. The function takes an AnnData object, a model formula, the factor to test, and the noise model as input. It returns a test result object from which p-values, q-values, and log2 fold changes can be accessed. ```python import diffxpy.api as de import numpy as np import pandas as pd import anndata # Create sample data: 200 cells x 100 genes n_cells, n_genes = 200, 100 counts = np.random.negative_binomial(n=5, p=0.3, size=(n_cells, n_genes)) # Create sample annotations with condition labels sample_description = pd.DataFrame({ "condition": np.repeat(["control", "treatment"], n_cells // 2), "batch": np.tile(["batch1", "batch2"], n_cells // 2) }) # Create AnnData object adata = anndata.AnnData(X=counts, obs=sample_description) adata.var_names = [f"gene_{i}" for i in range(n_genes)] # Run Wald test for differential expression between conditions test_result = de.test.wald( data=adata, formula_loc="~ 1 + condition + batch", # Model formula factor_loc_totest="condition", # Factor to test noise_model="nb" # Negative binomial noise model ) # Get results summary results = test_result.summary() print(results.head()) # Output columns: gene, pval, qval, log2fc, mean, zero_mean, grad, coef_mle, coef_sd # Get significant genes (q-value < 0.05) significant = results[results["qval"] < 0.05] print(f"Found {len(significant)} differentially expressed genes") # Access individual metrics pvalues = test_result.pval # Raw p-values qvalues = test_result.qval # FDR-corrected q-values log2fc = test_result.log2_fold_change() # Log2 fold changes ``` -------------------------------- ### Run t-test for Differential Expression Source: https://context7.com/theislab/diffxpy/llms.txt Performs a t-test to identify differentially expressed genes between groups. It takes an AnnData object, grouping information, and parameters for log-transformation and handling zero variance. Outputs a summary including p-values and log2 fold changes. ```python import diffxpy.api as de # Assuming 'adata' is an AnnData object with gene expression data and 'group' information test_result = de.test.t_test( data=adata, grouping="group", # Column name or array with group labels is_logged=False, # Whether data is already log-transformed is_sig_zerovar=True # Assign p=0 to genes with zero variance but different means ) # Get results results = test_result.summary() print(results[['gene', 'pval', 'qval', 'log2fc']].head(10)) # Filter significant results significant_genes = results[results['qval'] < 0.01]['gene'].values print(f"Significant genes: {significant_genes}") ``` -------------------------------- ### Run Wilcoxon Rank Sum Test for Differential Expression Source: https://context7.com/theislab/diffxpy/llms.txt Performs the Wilcoxon rank sum test (Mann-Whitney U test) for differential expression, suitable for non-normal distributions. Requires an AnnData object and grouping information. The output includes a summary sorted by p-value and log2 fold changes. ```python import diffxpy.api as de import numpy as np import pandas as pd import anndata # Create sample data n_cells, n_genes = 200, 60 counts = np.random.negative_binomial(n=3, p=0.3, size=(n_cells, n_genes)) grouping = np.array(["wildtype"] * 100 + ["knockout"] * 100) sample_description = pd.DataFrame({"genotype": grouping}) adata = anndata.AnnData(X=counts, obs=sample_description) adata.var_names = [f"gene_{i}" for i in range(n_genes)] # Run Wilcoxon rank sum test test_result = de.test.rank_test( data=adata, grouping="genotype", is_logged=False, is_sig_zerovar=True ) # Get summary with sorted results by p-value results = test_result.summary().sort_values('pval') print(results.head(10)) # Get log fold changes lfc = test_result.log2_fold_change() print(f"Max absolute log2 fold change: {np.nanmax(np.abs(lfc)):.3f}") ``` -------------------------------- ### Differential Expression Tests - Single Tests Source: https://github.com/theislab/diffxpy/blob/master/docs/api/index.rst Perform single differential expression tests per gene using various statistical methods. ```APIDOC ## Differential Expression Tests - Single Tests ### Description Run single differential expression tests for each gene. This involves computing a single p-value per gene using methods like likelihood ratio tests, Wald tests, t-tests, and Wilcoxon tests. ### Methods - `diffxpy.api.test.two_sample` - `diffxpy.api.test.wald` - `diffxpy.api.test.lrt` - `diffxpy.api.test.t_test` - `diffxpy.api.test.rank_test` ### Usage Example ```python import diffxpy.api as de # Assuming 'counts' is your gene expression data and 'metadata' contains sample information # For example: # counts = de.load_anndata("path/to/your/data.h5ad") # metadata = counts.obs # Example for a two-sample test (replace with actual data and parameters) # de.test.two_sample(counts, metadata, "group", "condition1", "condition2") ``` ```