### Create IntervalTree from Motifs Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/docs/html/_modules/garnet.html Generates an IntervalTree from a DataFrame containing motif start and end positions. ```python def IntervalTree_from_motifs(motifs): """ Arguments: motifs (dataframe): Must be a dataframe with motifStart and motifEnd columns Returns: IntervalTree: of motifs """ intervals = zip(motifs.motifStart.values, motifs.motifEnd.values, motifs.to_dict(orient='records')) tree = IntervalTree.from_tuples(intervals) return tree ``` -------------------------------- ### Import Libraries and Set Paths Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/example/GSLR.ipynb Imports necessary libraries and sets up paths for data and resources. This is typically done at the beginning of a script. ```python %pylab inline import sys import pickle import pandas as pd import networkx as nx # Use the following lines to test the basic functionality of the package if developing locally: import sys sys.path.insert(0, "/Users/alex/Documents/OmicsIntegrator2/src") from graph import * from gsslr import gsslr # import OmicsIntegrator as oi from matplotlib_venn import venn3, venn3_circles, venn2 repo_path = '/Users/alex/Documents/gslr/' data_path = repo_path + 'experiments/generated_data/3/' KEGG_path = repo_path + 'experiments/KEGG/KEGG_df.filtered.with_correlates.pickle' interactome_path = repo_path + 'experiments/algorithms/pcsf/inbiomap_temp.tsv' pathway_id = 'hsa04110' ``` -------------------------------- ### Get PCSF Vertex Count Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/example/Example_Usage.ipynb Retrieves the number of vertices in the identified PCSF subnetwork. ```python len(vertex_indices) ``` -------------------------------- ### Build and Upload to PyPI Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/DEPLOY.md This command cleans previous build artifacts, creates source and wheel distributions, and uploads them to PyPI using twine. ```bash rm -rf build dist OmicsIntegrator.egg-info && python setup.py sdist bdist_wheel && twine upload dist/* ``` -------------------------------- ### Get NetworkX Graph Nodes as DataFrame Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/docs/html/_modules/graph.html Converts a NetworkX graph's nodes and their associated attributes into a pandas DataFrame for easier manipulation and analysis. ```python def get_networkx_graph_as_dataframe_of_nodes(nxgraph): """ Arguments: nxgraph (networkx.Graph): any instance of networkx.Graph Returns: pd.DataFrame: nodes from the input graph and their attributes as a dataframe """ return pd.DataFrame.from_dict(dict(nxgraph.nodes(data=True)), orient='index') ``` -------------------------------- ### Prepare and Run PCSF Algorithm Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/docs/html/_modules/graph.html Prepares graph data and runs the PCSF algorithm to find an optimal subgraph. Handles dummy node creation and input validation. ```python all = list(range(len(self.nodes))) others = list(set(all) - set(self.terminals)) if self.params.dummy_mode == 'terminals': endpoints = self.terminals elif self.params.dummy_mode == 'other': endpoints = others elif self.params.dummy_mode == 'all': endpoints = all else: raise ValueError("Improper input to PCSF: dummy_mode must be one of 'terminals', 'other', or 'all'") dummy_edges, dummy_costs, root, dummy_prize = self._add_dummy_node(connected_to=endpoints) # `edges`: a 2D int64 array. Each row (of length 2) specifies an undirected edge in the input graph. The nodes are labeled 0 to n-1, where n is the number of nodes. edges = np.concatenate((self.edges, dummy_edges)) # `prizes`: the node prizes as a 1D float64 array. prizes = np.concatenate((self.prizes, dummy_prize)) # `costs`: the edge costs as a 1D float64 array. costs = np.concatenate((self.costs, dummy_costs)) # `root`: the root note for rooted PCST. For the unrooted variant, this parameter should be -1. # `num_clusters`: the number of connected components in the output. num_clusters = 1 # `pruning`: a string value indicating the pruning method. Possible values are `none`, `simple`, `gw`, and `strong` (all literals are case-insensitive). # `verbosity_level`: an integer indicating how much debug output the function should produce. if not self.params.skip_checks: self._check_validity_of_instance(edges, prizes, costs, root, num_clusters, pruning, verbosity_level) vertex_indices, edge_indices = pcst_fast(edges, prizes, costs, root, num_clusters, pruning, verbosity_level) # `vertex_indices`: indices of the vertices in the solution as a 1D int64 array. # `edge_indices`: indices of the edges in the output as a 1D int64 array. The list contains indices into the list of edges passed into the function. # Remove the dummy node and dummy edges for convenience vertex_indices = vertex_indices[vertex_indices != root] edge_indices = edge_indices[np.in1d(edge_indices, self.interactome_dataframe.index)] return vertex_indices, edge_indices ``` -------------------------------- ### Get NetworkX Graph Edges as DataFrame Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/docs/html/_modules/graph.html Converts a NetworkX graph's edges and their attributes into a pandas DataFrame using NetworkX's built-in function. ```python def get_networkx_graph_as_dataframe_of_edges(nxgraph): """ Arguments: nxgraph (networkx.Graph): any instance of networkx.Graph Returns: pd.DataFrame: edges from the input graph and their attributes as a dataframe """ return nx.to_pandas_edgelist(nxgraph, 'protein1', 'protein2') ``` -------------------------------- ### Perform Parameter Grid Search Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/example/OI2_pipeline.ipynb Initiates a grid search over specified parameters (Ws, Bs, Gs) using a prize file. This helps in exploring the parameter space for network generation. ```python results = graph.grid_search(prize_file, Ws, Bs, Gs) ``` -------------------------------- ### Load and Display Prize File Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/example/OI2_pipeline.ipynb Loads a prize file using pandas and displays the first few rows. Ensure the prize file is tab-separated. ```python pd.read_csv(prize_file, sep='\t').head() ``` -------------------------------- ### Create IntervalTree from Gene Reference Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/docs/html/_modules/garnet.html Generates an IntervalTree of genes from a reference DataFrame. Allows specifying upstream and downstream windows relative to transcription start or end sites. ```python def reference_to_intervaltree(reference, options={}): """ Arguments: reference (dataframe): Must be a dataframe with geneStart, geneEnd, and strand columns options (dict): {"upstream_window": int, "downstream_window": int, "tss": bool} Returns: IntervalTree: of genes from the reference """ upstream_window = options.get('upstream_window') or 2000 downstream_window = options.get('downstream_window') or 2000 window_ends_downstream_from_transcription_start_site_instead_of_transcription_end_site = options.get('tss') or False if window_ends_downstream_from_transcription_start_site_instead_of_transcription_end_site: starts = reference.apply(lambda x: x.geneStart - upstream_window if x.strand == '+' else x.geneEnd - upstream_window, axis=1) ends = reference.apply(lambda x: x.geneStart + downstream_window if x.strand == '+' else x.geneEnd + downstream_window, axis=1) else: starts = reference.apply(lambda x: x.geneStart - upstream_window, axis=1) ends = reference.apply(lambda x: x.geneEnd + downstream_window, axis=1) intervals = zip(starts.values, ends.values, reference.to_dict(orient='records')) tree = IntervalTree.from_tuples(intervals) return tree ``` -------------------------------- ### Initialize OmicsIntegrator Graph and Prepare Prizes Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/example/OI2_pipeline.ipynb Sets up the OmicsIntegrator graph object by loading an interactome file and preparing prize data from a specified file. Includes parameters for graph construction and prize processing. ```python interactome_file = "./OI2_pipeline_data/iRefIndex_v14_MIScore_interactome_C9.costs.txt" prize_file = "./OI2_pipeline_data/protein_TF_prizes.tsv" output_dir = "." Ws = [0.25, 0.5, 0.75, 1] Bs = [0.25, 0.5, 0.75, 1, 1.5, 2] Gs = [3, 3.5, 4, 4.5] Ws = [0.25, 0.5] Bs = [0.25, 0.5] Gs = [3, 3.5] params = { "noise": 0.1, "dummy_mode": "terminals", "exclude_terminals": False, "seed": 1 } graph = oi.Graph(interactome_file, params) graph.prepare_prizes(prize_file) ``` -------------------------------- ### Create IntervalTree from Peaks DataFrame Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/docs/html/_modules/garnet.html Constructs an IntervalTree from a pandas DataFrame containing peak start and end positions. Each interval in the tree will store the corresponding row data. ```python def IntervalTree_from_peaks(peaks): """ Arguments: peaks (dataframe): Must be a dataframe with peakStart and peakEnd columns Returns: IntervalTree: of peaks """ intervals = zip(peaks.peakStart.values, peaks.peakEnd.values, peaks.to_dict(orient='records')) tree = IntervalTree.from_tuples(intervals) return tree ``` -------------------------------- ### Determine Type of Peak Relative to Gene Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/docs/html/_modules/garnet.html Classifies the relationship between a peak and a gene based on their start and end positions and the gene's strand. Handles both positive and negative strands. ```python def type_of_peak(row): """ Arguments: row (pd.Series): A row of data from a dataframe with peak and gene information Returns: str: a name for the relationship between the peak and the gene: - upstream if the start of the peak is more than 2kb above the start of the gene - promoter if the start of the peak is above the start of the gene - downstream if the start of the peak is below the start of the gene """ if row['geneStrand'] == '+': if -2000 >= row['peakStart'] - row['geneStart']: return 'upstream' if -2000 < row['peakStart'] - row['geneStart'] < 0: return 'promoter' if 0 <= row['peakStart'] - row['geneStart']: return 'downstream' # a.k.a. row['peakStart'] < row['geneStart'] return 'intergenic' if row['geneStrand'] == '-': if 2000 <= row['peakEnd'] - row['geneEnd']: return 'upstream' if 2000 > row['peakEnd'] - row['geneEnd'] > 0: return 'promoter' if 0 >= row['peakEnd'] - row['geneEnd']: return 'downstream' # a.k.a. row['peakEnd'] < row['geneEnd'] return 'intergenic' ``` -------------------------------- ### output_networkx_graph_as_gpickle Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/docs/html/_modules/graph.html Outputs a NetworkX graph to a gpickle file, which is a binary format for saving and loading NetworkX graphs. The output directory and filename can be specified, and the directory will be created if it doesn't exist. Compressed files are supported. ```APIDOC ## output_networkx_graph_as_gpickle ### Description Outputs a NetworkX graph to a gpickle file. Supports compression via filename extensions (.gz, .bz2) and creates the output directory if it does not exist. ### Arguments - **nxgraph** (networkx.Graph) - The NetworkX graph object to save. - **output_dir** (str) - Optional. The directory where the graph will be saved. Defaults to the current directory. - **filename** (str) - Optional. The name of the output file. If it ends with .gz or .bz2, the file will be compressed. Defaults to "pcsf_results.gpickle". ### Returns - **Path** - The absolute path to the saved gpickle file. ``` -------------------------------- ### Get Level 2 Biological Process Terms Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/src/annotation/biological_process.ipynb Extracts a list of GO IDs for biological processes that are at depth 2 from a DataFrame. This is used to identify more specific biological categories for further analysis. ```python level2_terms = df[df.depth == 2].index.tolist() ``` -------------------------------- ### Load GO Biological Process Graph Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/src/annotation/biological_process.ipynb Loads the Gene Ontology biological process graph from a specified pickle file using NetworkX. This is the initial step to access GO data. ```python g = nx.read_gpickle('../../GONN/GO/GO_biological_process.pickle') g ``` -------------------------------- ### Get NetworkX Graph as DataFrame of Nodes Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/example/GSLR.ipynb This function retrieves a NetworkX graph and converts its node data into a pandas DataFrame. The DataFrame includes attributes such as betweenness, degree, location, and louvainClusters for each node. ```python get_networkx_graph_as_dataframe_of_nodes(class_networks[0]) ``` -------------------------------- ### Initialize Graph from Interactome File Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/example/Murat's multi-PCSF.ipynb Loads an interactome file and initializes a NetworkX graph. This is a foundational step for network-based analysis. ```Python %matplotlib inline import pickle import numpy as np import pandas as pd import networkx as nx import matplotlib.pyplot as plt import seaborn as sns from OmicsIntegrator import * interactome_file = "/Users/alex/Documents/OmicsIntegrator2/data/iref13_iref14_merged.cleaned.connected.tsv" graph = Graph(interactome_file, {}) ``` -------------------------------- ### Extract Top-Level Biological Processes Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/src/annotation/biological_process.ipynb Filters a DataFrame to select rows where the 'depth' is 1, extracting the names of top-level biological processes. This is useful for getting a high-level overview of the biological categories present in the data. ```python processes = df[df.depth == 1]['name'].to_frame() processes ``` -------------------------------- ### Prepare Prizes for the Graph Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/example/Example_Usage.ipynb Prepares the prize information for the graph by loading it from a specified file. ```python graph.prepare_prizes(prize_file) ``` -------------------------------- ### Extract Level 2 Compartment Names Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/src/annotation/cellular_compartment.ipynb Filters a DataFrame to select only rows where the 'depth' column is equal to 2, and extracts the 'name' column. This is used to get the names of cellular compartments at a specific hierarchical level (level 2). ```python compartments = df[df.depth == 2]['name'].to_frame() compartments ``` -------------------------------- ### Clustering Method Execution and Output Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/example/clustering_methods.ipynb Applies Louvain, Edge-Betweenness, and K-Clique clustering algorithms to a generated graph and prints the resulting cluster assignments for each method. ```python louvain_test = louvain_clustering(G) print('louvain_test cluster list:{}'.format(louvain_test)) betweenness_test = betweenness_clustering(G, 12) print('edge-betweenness cluster list:{}'.format(betweenness_test)) kclique_test = kclique_clustering(G, 2) print('k-clique cluster list:{}'.format(kclique_test)) ``` -------------------------------- ### Load GO Molecular Function Graph Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/src/annotation/molecular_function.ipynb Loads the GO molecular function graph from a pickle file using NetworkX. This is the initial step to access GO data. ```python g = nx.read_gpickle('../../GONN/GO/GO_molecular_function.pickle') g ``` -------------------------------- ### Load GO Cellular Component Graph Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/src/annotation/cellular_compartment.ipynb Loads the Gene Ontology cellular component graph from a pickle file using NetworkX. This is the initial step to access the ontology data. ```python g = nx.read_gpickle('../../GONN/GO/GO_cellular_component.pickle') g ``` -------------------------------- ### graph.Graph.__init__ Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/docs/source/index.md Initializes a Graph object by parsing an interactome file. It populates various graph attributes like dataframes, nodes, edges, and costs. ```APIDOC ## graph.Graph.__init__ ### Description Builds a representation of a graph from an interactome file. ### Method __init__ ### Parameters #### Path Parameters * **interactome_file** (str or FILE) - Required - tab-delimited text file containing edges in interactome and their weights formatted like “ProteinA ProteinB Cost” * **params** (dict) - Optional - params with which to run the program ### Returns None. Populates graph attributes such as interactome_dataframe, interactome_graph, nodes, edges, costs, edge_penalties, and node_degrees. ``` -------------------------------- ### Visualize Membership Heatmap (R) Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/example/OI2_pipeline.ipynb Generates a heatmap visualization of the membership data using R. Requires the 'membership_df' and 'node_attributes_df' to be available in the R environment. ```R plotHeatmap(membership_df, node_attributes_df, "Membership") ``` -------------------------------- ### Graph.prepare_prizes Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/docs/html/index.html Parses a prize file to add prizes and other attributes to the graph object. The prize file must contain column headers, with at least node name and prize columns. ```APIDOC ## Graph.prepare_prizes ### Description Parses a prize file and adds prizes and other attributes to the graph object. Requires the input file to contain headers, with at least node name and prize columns. ### Method prepare_prizes ### Parameters #### Path Parameters - **prize_file** (str or FILE) - Required - a filepath or file object containing a tsv with column headers. ``` -------------------------------- ### Map Terms to Genes with Evidence Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/src/annotation/biological_process.ipynb Builds a dictionary where each term is associated with a list of genes that map to it or its subterms, including their evidence codes. ```python terms_and_genes = {term: genes[genes.GO_ID.isin(subterms+[term])][['GeneSymbol', 'Evidence']].values.tolist() for term, subterms in terms_and_subterms.items()} ``` -------------------------------- ### Initialize Graph Object Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/example/GSLR.ipynb Creates a graph object using the Graph class, loading data from a specified interactome path. A dictionary with a large integer value is passed as a parameter. ```python graph = Graph(interactome_path, {'a':100000000}) ``` -------------------------------- ### Load or Parse Motifs File into IntervalTree Dictionary Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/docs/html/_modules/garnet.html Loads TF binding motifs from a file, attempting to unpickle it first. If unpickling fails, it parses the file and constructs IntervalTrees for each chromosome. Optionally saves the resulting IntervalTree dictionary to a pickle file. ```python def dict_of_IntervalTree_from_motifs_file(motifs_file, output_dir): """ Arguments: motifs_file (str or FILE): filepath or file object for the motifs file Returns: dict: dictionary of chromosome to IntervalTree of TF binding motifs """ logger.info("Checking if motifs file was generated by pickle...") motifs = try_to_load_as_pickled_object_or_None(motifs_file) if motifs: logger.info(' - Motifs file seems to have been generated by pickle, assuming IntervalTree format and proceeding...') return motifs logger.info(' - Motifs file does not seem to have been generated by pickle, proceeding to parse...') motifs = parse_motifs_file(motifs_file) motifs = group_by_chromosome(motifs) logger.info(' - Parse complete, constructing IntervalTrees...') motifs = {chrom: IntervalTree_from_motifs(chromosome_motifs) for chrom, chromosome_motifs in motifs.items()} if output_dir: logger.info(' - IntervalTree construction complete, saving pickle file for next time.') save_as_pickled_object(motifs, output_dir + 'motifs_IntervalTree_dictionary.pickle') return motifs ``` -------------------------------- ### Perform Parameter Sweep with Randomization Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/example/example.ipynb Conducts a parameter sweep by running graph randomization multiple times for different combinations of W, B, and G parameters. Saves the results to a pickle file. ```python robustness_reps = 100 specificity_reps = 100 results = graph.grid_randomization(prize_file, Ws, Bs, Gs, robustness_reps, specificity_reps) ``` -------------------------------- ### Visualize Robustness Heatmap (R) Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/example/OI2_pipeline.ipynb Generates a heatmap visualization for robustness data using R. Requires 'robustness_df' and 'node_attributes_df' to be available in the R environment. ```R plotHeatmap(robustness_df, node_attributes_df, "Robustness") ``` -------------------------------- ### Load Randomization Results Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/example/example.ipynb Loads the previously saved randomization results from a pickle file. This allows for resuming analysis without recomputing the entire sweep. ```python results = pickle.load(open("./OI2_pipeline_data/randomization_results.pkl", "rb")) ``` -------------------------------- ### Load Shortest Paths from Pickle Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/example/Murat's multi-PCSF.ipynb Deserializes the shortest paths from a pickle file. ```python shortest_paths = pickle.load(open('../data/shortest_paths.pickle', 'rb')) ``` -------------------------------- ### Evaluate PCSF Runs with Parameters Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/docs/html/_modules/graph.html A helper method to set hyperparameters and perform PCSF randomizations. It logs the run and returns parameter string along with the forest and augmented forest graphs. ```python def _eval_PCSF_runs(self, params): """ Convenience method which sets parameters and performs PCSF randomizations. Arguments: params (dict): dictionary with regular OI2 parameters _AND_ noisy_edge_reps and random_terminals_reps Returns: str: Parameter values in string format networkx.Graph: forest networkx.Graph: augmented_forest """ self._reset_hyperparameters(params=params) paramstring = "W_{:04.2f}_B_{:04.2f}_G_{:04.2f}".format(self.params.w, self.params.b, self.params.g) if params["noisy_edge_reps"] == params["random_terminals_reps"] == 0: logger.info("Single PCSF run for " + paramstring) else: logger.info("Randomizations for " + paramstring) forest, augmented_forest = self.randomizations(params["noisy_edge_reps"], params["random_terminals_reps"]) return paramstring, forest, augmented_forest ``` -------------------------------- ### Visualize Augmented Forest Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/example/Example_Usage.ipynb Draws the augmented forest using a spring layout for visualization. ```python nx.draw_spring(augmented_forest) ``` -------------------------------- ### graph.output_networkx_graph_as_pickle Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/docs/html/index.html Outputs a NetworkX graph as a pickle file. ```APIDOC ## graph.output_networkx_graph_as_pickle ### Parameters * **nxgraph** (_networkx.Graph_) – any instance of networkx.Graph * **output_dir** (_str_) – the directory in which to output the graph. * **filename** (_str_) – Filenames ending in .gz or .bz2 will be compressed. ### Returns the filepath which was outputted to ### Return type Path ``` -------------------------------- ### Load InBioMap Data Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/interactomes/PreProcess_InBioMap_For_OmicsIntegrator.ipynb Loads experimental and full InBioMap datasets from pickle files. ```python inbiomap_exp = pd.read_pickle("../../interactomes/InBioMap/inbiomap.9.12.2016.cleaned.namespace-mapped.exp.pickle") inbiomap_full = pd.read_pickle("../../interactomes/InBioMap/inbiomap.9.12.2016.cleaned.namespace-mapped.full.pickle") ``` -------------------------------- ### Run PCSF Algorithm Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/docs/html/_modules/graph.html Selects the Prize-Collecting Steiner Forest subgraph. It adds a dummy root node and then calls the pcst_fast function. The results can be interpreted using `output_forest_as_networkx`. ```python def pcsf(self, pruning="strong", verbosity_level=0): """ Select the subgraph which approximately optimizes the Prize-Collecting Steiner Forest objective. This function mostly defers to pcst_fast, but does one important pre-processing step: it adds a dummy node which will serve as the PCSF root and connects that dummy node to either terminals, non-terminals, or all other nodes with edges weighted by self.params.w. In order to interpret the results of this function, use `output_forest_as_networkx` with the results of this function. Arguments: pruning (str): a string value indicating the pruning method. Possible values are `'none'`, `'simple'`, `'gw'`, and `'strong'` (all literals are case-insensitive). ``` -------------------------------- ### graph.output_networkx_graph_as_interactive_html Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/docs/html/index.html Outputs a NetworkX graph as an interactive HTML file. ```APIDOC ## graph.output_networkx_graph_as_interactive_html ### Parameters * **nxgraph** (_networkx.Graph_) – any instance of networkx.Graph * **attribute_metadata** (_dict_) – Metadata for attributes to be displayed. * **output_dir** (_str_) – the directory in which to output the file * **filename** (_str_) – the filename of the output file ### Returns the filepath which was outputted to ### Return type Path ``` -------------------------------- ### Import Core Libraries Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/example/example.ipynb Imports essential Python libraries for data manipulation, network analysis, and plotting. Includes pandas, numpy, networkx, and matplotlib. ```python import pickle import numpy as np import pandas as pd import networkx as nx # import OmicsIntegrator as oi import warnings warnings.filterwarnings('ignore') ``` -------------------------------- ### Output NetworkX Graph as gpickle Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/docs/html/_modules/graph.html Saves a networkx graph to a gpickle file. Supports compression if the filename ends with .gz or .bz2. Ensures the output directory exists. ```python def output_networkx_graph_as_gpickle(nxgraph, output_dir=".", filename="pcsf_results.gpickle.gz"): """ Arguments: nxgraph (networkx.Graph): any instance of networkx.Graph output_dir (str): the directory in which to output the graph. filename (str): Filenames ending in .gz or .bz2 will be compressed. Returns: Path: the filepath which was outputted to """ path = Path(output_dir) path.mkdir(exist_ok=True, parents=True) path = path / filename nx.write_gpickle(nxgraph, open(path, 'wb')) return path.resolve() ``` -------------------------------- ### Map terms to their subterms using DFS Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/src/annotation/molecular_function.ipynb Constructs a dictionary where keys are level 2 GO terms and values are lists of all unique successor terms (subterms) found via a Depth First Search (DFS) on the GO graph. This helps in understanding the hierarchical relationships. ```python terms_and_subterms = {term: np.unique(flatten(list(nx.dfs_successors(g, term).values()))).tolist() for term, subterms in list(terms_and_subterms.items())} ``` -------------------------------- ### Initialize Omics Integrator Graph Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/example/example.ipynb Initializes the Omics Integrator Graph object with interactome and prize files. Sets parameters like noise, dummy mode, and seed. Prepares prize data for the graph. ```python interactome_file = "./OI2_pipeline_data/iRefIndex_v14_MIScore_interactome_C9.costs.txt" prize_file = "./OI2_pipeline_data/protein_TF_prizes.tsv" output_dir = "./OI2_pipeline_data/" Ws = [0.5, 1] Bs = [1, 2] Gs = [3, 4] params = { "noise": 0.1, "dummy_mode": "terminals", "exclude_terminals": False, "seed": 1 } graph = Graph(interactome_file, params) graph.prepare_prizes(prize_file) ``` -------------------------------- ### Try to Load Pickled Object Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/docs/html/_modules/garnet.html Attempts to load a Python object from a file using pickle, handling potential errors and supporting very large files by reading in chunks. Returns None if loading fails. ```python def try_to_load_as_pickled_object_or_None(filepath): """ This is a defensive way to write pickle.load, allowing for very large files on all platforms """ max_bytes = 2**31 - 1 try: input_size = os.path.getsize(filepath) bytes_in = bytearray(0) with open(filepath, 'rb') as f_in: for _ in range(0, input_size, max_bytes): bytes_in += f_in.read(max_bytes) obj = pickle.loads(bytes_in) except: return None return obj ``` -------------------------------- ### Display Experimental Data Head Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/interactomes/PreProcess_InBioMap_For_OmicsIntegrator.ipynb Shows the first few rows of the experimental InBioMap data to inspect its structure. ```python inbiomap_exp.head() ``` -------------------------------- ### Create IntervalTree from Reference DataFrame Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/docs/html/_modules/garnet.html Constructs an IntervalTree from a reference DataFrame containing gene information. Requires 'strand', 'geneStart', and 'geneEnd' columns. ```python def IntervalTree_from_reference(reference, options): """ Arguments: reference (dataframe): Must be a dataframe with `strand`, `geneStart`, and `geneEnd` columns ``` -------------------------------- ### Visualize Specificity Heatmap (R) Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/example/OI2_pipeline.ipynb Generates a heatmap visualization for specificity data using R. Requires 'specificity_df' and 'node_attributes_df' to be available in the R environment. ```R plotHeatmap(specificity_df, node_attributes_df, "Specificity") ``` -------------------------------- ### Import Libraries Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/src/annotation/final_annotation.ipynb Imports necessary libraries for data manipulation, network analysis, and plotting. ```python import numpy as np import pandas as pd import networkx as nx import matplotlib.pyplot as plt %matplotlib inline ``` -------------------------------- ### Import Libraries for Graph Clustering Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/example/clustering_methods.ipynb Imports necessary libraries for graph analysis and clustering, including NetworkX and the community detection library. ```python import networkx as nx import itertools import community #note: community installation requires running setup.py from https://github.com/taynaud/python-louvain/ ``` -------------------------------- ### Output NetworkX Graph as Interactive HTML Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/example/Example_Usage.ipynb Use this function to save a NetworkX graph as an interactive HTML file. Specify the graph object and the output directory. ```python oi.output_networkx_graph_as_interactive_html(augmented_forest, output_dir='/Users/alex/Desktop/') ``` -------------------------------- ### generate_basic_statistics Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/docs/source/index.md Summarizes the results of robust network randomization experiments. ```APIDOC ## generate_basic_statistics(robust_results) ### Description Summarizes robust network randomization results ### Parameters **robust_results** (*dict* *of* *networkx*) – {paramstring: robust network (networkx)} ### Returns summary of each robust network ### Return type pd.DataFrame ``` -------------------------------- ### Run PCSF Algorithm Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/example/Example_Usage.ipynb Executes the PCSF (Prize-Collecting Steiner Forest) algorithm on the graph to identify a subnetwork. ```python vertex_indices, edge_indices = graph.pcsf() ``` -------------------------------- ### Output NetworkX Graph as Pickle Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/docs/html/_modules/graph.html Saves a NetworkX graph object to a file in pickle format. Supports compression for filenames ending in .gz or .bz2. ```python def output_networkx_graph_as_pickle(nxgraph, output_dir=".", filename="pcsf_results.pickle"): """ Arguments: nxgraph (networkx.Graph): any instance of networkx.Graph output_dir (str): the directory in which to output the graph. filename (str): Filenames ending in .gz or .bz2 will be compressed. ``` -------------------------------- ### Output NetworkX Graph as Interactive HTML Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/docs/html/_modules/graph.html Generates an interactive HTML visualization of a networkx graph using the axial library. Supports inline data and scripts for self-contained output. ```python def output_networkx_graph_as_interactive_html(nxgraph, attribute_metadata=dict(), output_dir=".", filename="graph.html"): """ Arguments: nxgraph (networkx.Graph): any instance of networkx.Graph output_dir (str): the directory in which to output the file filename (str): the filename of the output file Returns: Path: the filepath which was outputted to """ return axial.graph(nxgraph, title='OmicsIntegrator2 Results', scripts_mode="inline", data_mode="inline", output_dir=output_dir, filename=filename) ``` -------------------------------- ### Create Gene to Compartment Mapping Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/src/annotation/cellular_compartment.ipynb Extracts gene and GO_ID pairs, then sets the gene as the index to create a mapping from genes to their associated compartment GO_IDs. Displays the first few rows. ```python gene_to_compartment_term = best_evidence.reset_index()[['gene', 'GO_ID']].set_index('gene') gene_to_compartment_term.head() ``` -------------------------------- ### Summarize Grid Search for Robustness and Specificity Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/example/OI2_pipeline.ipynb Summarizes grid search results to create dataframes for robustness and specificity. This is used to analyze the stability and unique contribution of nodes across different parameter sets. ```python robustness_df = oi.summarize_grid_search(results, "robustness", top_n=1000) specificity_df = oi.summarize_grid_search(results, "specificity", top_n=1000) ``` -------------------------------- ### Import Core Libraries Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/example/OI2_pipeline.ipynb Imports essential Python libraries for data manipulation, network analysis, and the OmicsIntegrator package. Also suppresses warnings. ```python import numpy as np import pandas as pd import networkx as nx import OmicsIntegrator as oi import warnings warnings.filterwarnings('ignore') ``` -------------------------------- ### Display First Rows of STRING Experimental Data Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/interactomes/PreProcess_STRING_For_OmicsIntegrator.ipynb Shows the first few rows of the STRING experimental interactome data to inspect its structure and content. ```python STRING_exp.head() ``` -------------------------------- ### graph.output_networkx_graph_as_interactive_html Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/docs/source/index.md Generates an interactive HTML file from a NetworkX graph, including specified attribute metadata. ```APIDOC ## graph.output_networkx_graph_as_interactive_html ### Description Generates an interactive HTML file from a NetworkX graph, including specified attribute metadata. ### Method N/A (Python function) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Parameters * **nxgraph** (*networkx.Graph*) – any instance of networkx.Graph * **attribute_metadata** (*dict*) – A dictionary mapping attribute names to their metadata. Defaults to an empty dictionary. * **output_dir** (*str*) – the directory in which to output the file. Defaults to the current directory. * **filename** (*str*) – the filename of the output file. Defaults to 'graph.html'. ### Returns * **filepath** (Path) – the filepath which was outputted to ``` -------------------------------- ### Load STRING Data Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/interactomes/PreProcess_STRING_For_OmicsIntegrator.ipynb Loads the STRING interactome data from pickle files. These files contain protein-protein interaction information. ```python STRING_exp = pd.read_pickle("../../interactomes/STRING/string.v10.5.cleaned.namespace-mapped.exp.pickle") STRING_full = pd.read_pickle("../../interactomes/STRING/string.v10.5.cleaned.namespace-mapped.full.pickle") ``` -------------------------------- ### Import necessary libraries Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/example/Example_Usage.ipynb Imports the required libraries for OmicsIntegrator, including numpy, pandas, and networkx. ```python %matplotlib inline import numpy as np import pandas as pd import networkx as nx import OmicsIntegrator as oi ``` -------------------------------- ### Graph Class Initialization Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/docs/html/index.html Initializes a Graph object from an interactome file and optional parameters. This populates the graph's internal data structures like dataframes, networkx graphs, node lists, edge lists, costs, and degrees. ```APIDOC ## Graph Class Initialization ### Description Builds a representation of a graph from an interactome file. Populates internal data structures including interactome_dataframe, interactome_graph, nodes, edges, costs, and edge_penalties. ### Method __init__ ### Parameters #### Path Parameters - **interactome_file** (str or FILE) - Required - tab-delimited text file containing edges in interactome and their weights formatted like “ProteinA ProteinB Cost” #### Request Body - **params** (dict) - Optional - params with which to run the program ``` -------------------------------- ### Load Libraries and Data Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/src/annotation/subcellular_location.ipynb Imports necessary libraries (numpy, pandas, matplotlib, mygene) and loads experimental data from a TSV file into a pandas DataFrame. This sets up the environment for data processing. ```python %load_ext autoreload %autoreload 2 import numpy as np import pandas as pd import matplotlib.pyplot as plt %matplotlib inline import mygene ``` ```python experiments_full = pd.read_csv('./human_compartment_experiments_full.tsv', sep=' ', header=None, names=['Ensembl','GeneSymbol','GO','location','evidence_type','evidence','confidence']) experiments_full.head() ``` -------------------------------- ### Save Shortest Paths to Pickle Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/example/Murat's multi-PCSF.ipynb Serializes the computed shortest paths to a pickle file for later use. ```python pickle.dump(shortest_paths, open('../data/shortest_paths.pickle', 'wb')) ``` -------------------------------- ### Output Robust Network as GraphML Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/example/OI2_pipeline.ipynb Use this snippet to save a robust network to a GraphML file. Specify the parameter set, output directory, and filename. The function returns the path to the generated file. ```python parameter_set = "W_0.25_B_0.50_G_3.00" oi.output_networkx_graph_as_graphml_for_cytoscape(results[parameter_set]["robust"], output_dir=output_dir, filename="{}.robust_network.graphml".format(parameter_set)) ``` -------------------------------- ### graph.output_networkx_graph_as_graphml_for_cytoscape Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/docs/html/index.html Outputs a NetworkX graph as a GraphML file, compressed for Cytoscape. ```APIDOC ## graph.output_networkx_graph_as_graphml_for_cytoscape ### Parameters * **nxgraph** (_networkx.Graph_) – any instance of networkx.Graph * **output_dir** (_str_) – the directory in which to output the graph. * **filename** (_str_) – Filenames ending in .gz or .bz2 will be compressed. ### Returns the filepath which was outputted to ### Return type Path ``` -------------------------------- ### Consolidate All Terms Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/src/annotation/biological_process.ipynb Generates a comprehensive list of all unique terms, including both the top-level terms and their subterms. ```python terms = [item for l in [subterms+[term] for term, subterms in list(terms_and_subterms.items())] for item in l] len(terms), len(np.unique(terms)), len(df) ``` -------------------------------- ### Prepare and Filter Matrix for Final Clustermap Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/example/Murat's multi-PCSF.ipynb Joins the normalized matrix with labels, removes rows with missing PAM50 mRNA information, sets the index, and drops specific columns. This prepares the data for a more refined clustermap visualization. ```python final_matrix = normalized_matrix.join(brca_labels).dropna(subset=['PAM50 mRNA']).set_index('PAM50 mRNA', append=True).swaplevel().sort_index(level=0) del final_matrix['263d3f-I'], final_matrix['blcdb9-I'], final_matrix['c4155b-C'] final_matrix.head() ``` -------------------------------- ### output_networkx_graph_as_graphml_for_cytoscape Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/docs/html/_modules/graph.html Outputs a NetworkX graph in GraphML format, suitable for use with Cytoscape. The output directory and filename can be specified, and the directory will be created if it doesn't exist. Compressed files are supported. ```APIDOC ## output_networkx_graph_as_graphml_for_cytoscape ### Description Outputs a NetworkX graph in GraphML format, optimized for Cytoscape. Supports compression via filename extensions (.gz, .bz2) and creates the output directory if it does not exist. ### Arguments - **nxgraph** (networkx.Graph) - The NetworkX graph object to save. - **output_dir** (str) - Optional. The directory where the graph will be saved. Defaults to the current directory. - **filename** (str) - Optional. The name of the output file. If it ends with .gz or .bz2, the file will be compressed. Defaults to "pcsf_results.graphml.gz". ### Returns - **Path** - The absolute path to the saved GraphML file. ``` -------------------------------- ### Load Dataset with Pandas Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/example/GSLR.ipynb Loads a dataset from a CSV file into a pandas DataFrame. The dataset is indexed by a pathway ID and columns represent gene symbols. ```python dataset = pd.read_csv(data_path + pathway_id + '_inbiomap_exp.csv', index_col=0) dataset.head() ``` -------------------------------- ### Graph Initialization Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/docs/html/_modules/graph.html Initializes a Graph object from an interactome file. It populates the graph's internal representation including dataframes, networkx graphs, node lists, edge lists, costs, and node degrees. This is the primary method for loading network data. ```python def __init__(self, interactome_file, params=dict()): """ Builds a representation of a graph from an interactome file. From the interactome_file, populates - `graph.interactome_dataframe` (pandas.DataFrame) - `graph.interactome_graph` (networkx.Graph) - `graph.nodes` (pandas.Index), - `graph.edges` (list of pairs), - `graph.costs` and `graph.edge_penalties` (lists, such that the ordering is the same as in graph.edges), - `graph.node_degrees` (list, such that the ordering is the same as in graph.nodes). Arguments: interactome_file (str or FILE): tab-delimited text file containing edges in interactome and their weights formatted like "ProteinA\tProteinB\tCost" params (dict): params with which to run the program """ self.interactome_dataframe = pd.read_csv(interactome_file, sep='\t') self.interactome_graph = nx.from_pandas_edgelist(self.interactome_dataframe, 'protein1', 'protein2', edge_attr=self.interactome_dataframe.columns[2:].tolist()) # Convert the interactome dataframe from string interactor IDs to integer interactor IDs. # Do so by selecting the protein1 and protein2 columns from the interactome dataframe, # then unstacking them, which (unintuitively) stacks them into one column, allowing us to use factorize. # Factorize builds two datastructures, a unique pd.Index which maps each ID string to an integer ID, # and the datastructure we passed in with string IDs replaced with those integer IDs. # We place those in self.nodes and self.edges respectively, but self.edges will need reshaping. (self.edges, self.nodes) = pd.factorize(self.interactome_dataframe[[ "protein1","protein2" ]].unstack()) # Here we do the inverse operation of "unstack" above, which gives us an interpretable edges datastructure self.edges = self.edges.reshape(self.interactome_dataframe[[ "protein1","protein2" ]].shape, order='F') self.edge_costs = self.interactome_dataframe['cost'].astype(float).values # Count the number of incident edges into each node. # The indices into this datastructure are the same as those in self.nodes which are the IDs in self.edges. self.node_degrees = np.bincount(self.edges.flatten()) # The rest of the setup work is occasionally repeated, so use another method to complete setup. ``` -------------------------------- ### Summarize Grid Search Results (Membership Mode) Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/docs/html/_modules/graph.html Summarizes grid search results into a membership matrix. Each row represents a gene, and columns represent parameter experiments. Entries are 0 or 1 indicating node presence. Used for analyzing parameter sensitivity. ```python def summarize_grid_search(results, mode, top_n=np.Infinity): """ Summarizes results of `grid_randomization` or `grid_search` into a matrix where each row is a gene and each column is a parameter run. If summarizing "membership", entries will be 0 or 1 indicating whether or not a node appeared in each experiment. If summarizing "robustness" or "specificity", entries indicate robustness or specificity values for each experiment. Arguments: results (list of tuples): Results of `grid_randomization` or `grid_search` of form `{'paramstring': { 'forest': object, 'augmented forest': object}} mode (str): Reported values "membership", "robustness", "specificity" top_n (int): Takes the top_n values of the summary dataframe. top_n=-1 sets no threshold Returns: pd.DataFrame: Columns correspond to each parameter experiment, indexed by nodes """ # Exclude any degenerate results results = {paramstring: graphs for paramstring, graphs in results.items() if graphs["augmented_forest"].number_of_nodes() > 0} if mode == "membership": # Summarize single-run parameter search ``` -------------------------------- ### output_networkx_graph_as_pickle Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/docs/html/_modules/graph.html Saves a NetworkX graph to a pickle file. ```APIDOC ## output_networkx_graph_as_pickle ### Description Saves a NetworkX graph to a pickle file. ### Arguments * **nxgraph** (networkx.Graph) - any instance of networkx.Graph * **output_dir** (str) - the directory in which to output the graph. * **filename** (str) - Filenames ending in .gz or .bz2 will be compressed. ``` -------------------------------- ### Load or Parse Peaks File into IntervalTree Dictionary Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/docs/html/_modules/garnet.html Loads peaks from a file, attempting to unpickle it first. If unpickling fails, it parses the file and constructs IntervalTrees for each chromosome. Optionally saves the resulting IntervalTree dictionary to a pickle file. ```python def dict_of_IntervalTree_from_peaks_file(peaks_file, output_dir): logger.info("Checking if peaks file was generated by pickle...") speaks = try_to_load_as_pickled_object_or_None(peaks_file) if peaks: logger.info(' - Peaks file seems to have been generated by pickle, assuming IntervalTree format and proceeding...') return peaks logger.info(' - Peaks file does not seem to have been generated by pickle, proceeding to parse...') speaks = parse_peaks_file(peaks_file) speaks = group_by_chromosome(peaks) logger.info(' - Parse complete, constructing IntervalTrees...') speaks = {chrom: IntervalTree_from_peaks(chromosome_peaks) for chrom, chromosome_peaks in peaks.items()} if output_dir: logger.info(' - IntervalTree construction complete, saving pickle file for next time.') save_as_pickled_object(peaks, output_dir + 'peaks_IntervalTree_dictionary.pickle') return peaks ``` -------------------------------- ### Output NetworkX Graph as GraphML for Cytoscape Source: https://github.com/fraenkel-lab/omicsintegrator2/blob/master/docs/html/_modules/graph.html Saves a networkx graph to a GraphML file, suitable for Cytoscape. Handles compression based on filename extension. Creates the output directory if it does not exist. ```python def output_networkx_graph_as_graphml_for_cytoscape(nxgraph, output_dir=".", filename="pcsf_results.graphml.gz"): """ Arguments: nxgraph (networkx.Graph): any instance of networkx.Graph output_dir (str): the directory in which to output the graph. filename (str): Filenames ending in .gz or .bz2 will be compressed. Returns: Path: the filepath which was outputted to """ path = Path(output_dir) path.mkdir(exist_ok=True, parents=True) path = path / filename nx.write_graphml(nxgraph, path) return path.resolve() ```