CPTAC

https://paynelab.github.io/cptac/
Admin
CPTAC is a Python package providing easy access to cancer proteogenomic data from the National...
Tokens:28,965
Snippets:345
Trust Score:9.7
Update:5 months ago
Show doc for...
Tokens:
Raw
### Import Standard Data Analysis Libraries

Source: https://paynelab.github.io/cptac/usecase01_omics.html

Imports essential Python libraries for data manipulation and visualization. Ensure these are installed in your environment.

```python
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
```

--------------------------------

### Install cptac Python Package

Source: https://paynelab.github.io/cptac

Install the cptac package using pip. This command installs the package and its dependencies.

```bash
pip install cptac

```

--------------------------------

### Get help for a CPTAC function

Source: https://paynelab.github.io/cptac/tutorial01_data_intro.html

Use Python's `help()` function to view detailed documentation for a specific CPTAC function, including its parameters, return values, and usage.

```python
help(en.join_omics_to_omics)
```

--------------------------------

### Install cptac Python Package with Conda

Source: https://paynelab.github.io/cptac

If using Anaconda, install pip into your environment first if it's not already present, then use pip to install the cptac package.

```bash
conda install -c anaconda pip
pip install cptac

```

--------------------------------

### T-test Result Example 1 (Output)

Source: https://paynelab.github.io/cptac/usecase03_clinical_and_acetylation.html

This is an example output from an independent samples t-test, showing the calculated statistic, p-value, and degrees of freedom (df).

```python
TtestResult(statistic=5.653510545875068, pvalue=1.520752350373057e-07, df=99.0)
```

--------------------------------

### Get Phosphoproteomics Data

Source: https://paynelab.github.io/cptac/tutorial04_multiindex.html

Retrieves the first 5 rows of phosphoproteomics data for a specified dataset. Ensure the cptac library is installed and imported.

```python
en.get_phosphoproteomics('umich').head()
```

--------------------------------

### T-test Result Example 2 (Output)

Source: https://paynelab.github.io/cptac/usecase03_clinical_and_acetylation.html

Another example output from an independent samples t-test, presenting the statistic, p-value, and degrees of freedom (df).

```python
TtestResult(statistic=-5.487492982044729, pvalue=3.1403534632193126e-07, df=99.0)
```

--------------------------------

### cptac Package Update Warning

Source: https://paynelab.github.io/cptac/tutorial05_updates.html

This is an example of the warning message displayed when the installed cptac package version is out-of-date. It advises running 'pip install --upgrade cptac' to update.

```text
cptac warning: Your version of cptac (1.5.1) is out-of-date. Latest is 1.5.0. Please run 'pip install --upgrade cptac' to update it. (C:\Users\sabme\anaconda3\lib\threading.py, line 910)

```

--------------------------------

### T-test Results Example

Source: https://paynelab.github.io/cptac/usecase07_trans_genetic_effect.html

Example output of the t-test comparison, showing proteins and their corresponding p-values. Note that Bonferroni correction is applied, making the significance threshold strict.

```text
Doing t-test comparisons

                   Comparison   P_Value
0      STAT2_umich_proteomics  0.000002
1     DLGAP5_umich_proteomics  0.000030
2   SLC4A1AP_umich_proteomics  0.000053
3      CMTR1_umich_proteomics  0.000126
4      TRMT6_umich_proteomics  0.000165
..                        ...       ...
72     RRP36_umich_proteomics  0.047035
73    TOM1L1_umich_proteomics  0.048238
74    KIF13B_umich_proteomics  0.048484
75    PARP14_umich_proteomics  0.048569
76      BDH2_umich_proteomics  0.049898

[77 rows x 2 columns]
```

--------------------------------

### Get Available Data Source Options

Source: https://paynelab.github.io/cptac/tutorial01_data_intro.html

Call the `cptac.get_source_options()` function to understand the data types and their sources available for CPTAC datasets. Note that some data types may have multiple sources, representing different computational pipelines.

```python
cptac.get_source_options()

```

--------------------------------

### Import Packages and Load Endometrial Data

Source: https://paynelab.github.io/cptac/usecase03_clinical_and_acetylation.html

Imports necessary Python libraries and loads the Endometrial (UCEC) dataset using the cptac package. Ensure all dependencies are installed via pip.

```python
import pandas as pd
import numpy as np
import scipy.stats
import statsmodels.stats.multitest
import matplotlib.pyplot as plt
import seaborn as sns
import math
import cptac
import cptac.utils as ut

en = cptac.Ucec()
```

--------------------------------

### Get Somatic Mutation Data

Source: https://paynelab.github.io/cptac/tutorial01_data_intro.html

Retrieves somatic mutation data for a cohort. The data is a direct import of a MAF file.

```python
somatic_mutations = en.get_somatic_mutation('harmonized')
somatic_mutations.head()
```

--------------------------------

### Join Ovarian Proteomics to Mutations

Source: https://paynelab.github.io/cptac/usecase04_mutations_and_omics.html

Load the Ovarian dataset and join proteomics data with somatic mutations for a specified gene. This is useful for investigating how mutations in a gene correlate with protein levels. Ensure the CPTAC library is installed and imported.

```python
ov = cptac.Ov()

source1 = 'umich'
source2 = 'washu'
somatic_mutations = ov.get_somatic_mutation(source2)

gene = "TP53"
ovarian_omics_mutations = ov.join_omics_to_mutations(
    omics_name="proteomics",
    omics_source=source1,
    mutations_genes=gene,
    mutations_source=source2,
    omics_genes=gene, 
    mutations_filter=[]
)

plt.figure(figsize=(11.7, 8.27))

boxplot = sns.boxplot(
    data=ovarian_omics_mutations, 
    x=gene + "_Mutation", 
    y=gene + "_" + source1 + "_proteomics", 
    order=["Wildtype_Normal","Wildtype_Tumor","Missense_Mutation","Nonsense_Mutation", "Frame_Shift_Ins","Frame_Shift_Del","Splice_Site"]
)
boxplot.set_title("TP53 gene mutation protein abundance")
boxplot = sns.stripplot(
    data=ovarian_omics_mutations, 
    x=gene + "_Mutation", 
    y=gene + "_" + source1 + "_proteomics",
    jitter=True, 
    color=".3",
    order=["Wildtype_Normal","Wildtype_Tumor","Missense_Mutation", "Nonsense_Mutation","Frame_Shift_Ins","Frame_Shift_Del","Splice_Site"]
)
boxplot.set(xlabel="Somatic Gene Mutation", ylabel="Proteomics")
boxplot.set_xticklabels(boxplot.get_xticklabels(),rotation=30)
plt.show()
```

--------------------------------

### Export DataFrame to TSV

Source: https://paynelab.github.io/cptac/tutorial03_joining_dataframes.html

Use the `to_csv` method to save a DataFrame to a file. Specify the `path_or_buf` for the output file and `sep` for the value separator. This example saves to a TSV file.

```python
survival_and_SHANK2.to_csv(path_or_buf="histologic_type_and_PTEN_mutation.tsv", sep='\t')
```

--------------------------------

### Get Available Cancer Information

Source: https://paynelab.github.io/cptac/tutorial01_data_intro.html

Call the `cptac.get_cancer_info()` function to view a list of available cancers and their abbreviations. This helps in identifying the specific cancer type you want to analyze.

```python
cptac.get_cancer_info()

```

--------------------------------

### Get Somatic Mutation Data and Identify Gene of Interest

Source: https://paynelab.github.io/cptac/usecase04_mutations_and_omics.html

Retrieves somatic mutation data for a specified cancer cohort (e.g., 'washu') and identifies the gene with the highest mutation count. This gene will be used for further analysis.

```python
somatic_mutations = en.get_somatic_mutation('washu')
gene = somatic_mutations["Gene"].value_counts().index[0]

```

--------------------------------

### Display Q-values Table Head

Source: https://paynelab.github.io/cptac/usecase08_outliers.html

Displays the first few rows of the Q-values DataFrame to inspect enrichment probabilities.

```python
qvalues.df.head()
```

--------------------------------

### Import BlackSheep and CPTAC Packages

Source: https://paynelab.github.io/cptac/usecase08_outliers.html

Imports the blacksheep and cptac packages and initializes the Brca object to access clinical and proteomic data.

```python
import blacksheep
import cptac

brca = cptac.Brca()
clinical = brca.get_clinical('mssm')
proteomics = brca.get_proteomics('umich')
```

--------------------------------

### Import CPTAC and load UCEC dataset

Source: https://paynelab.github.io/cptac/tutorial04_multiindex.html

Import the cptac library and initialize the Ucec dataset object. This is the first step for accessing cancer omics data.

```python
import cptac
en = cptac.Ucec()

```

--------------------------------

### Documentation for join_omics_to_omics method

Source: https://paynelab.github.io/cptac/tutorial01_data_intro.html

This is the detailed documentation for the `join_omics_to_omics` method, explaining its purpose, parameters, and return type. It is useful for understanding how to join omics dataframes.

```python
Help on method join_omics_to_omics in module cptac.cancers.cancer:

join_omics_to_omics(df1_name: str, df2_name: str, df1_source: str = None, df2_source: str = None, genes1: str = None, genes2: str = None, how: str = 'outer', quiet: bool = False, tissue_type: str = 'both') -> pandas.core.frame.DataFrame method of cptac.cancers.ucec.Ucec instance
    Take specified column(s) from one omics dataframe, and join to specified columns(s) from another omics dataframe. Intersection (inner join) of indices is used.
    
    Parameters:
    df1_name (str): Name of first omics dataframe to select columns from.
    df2_name (str): Name of second omics dataframe to select columns from.
    df1_source (str): Name of source for the first omics dataframe.
    df2_source (str): Name of source for the second omics dataframe.
    genes1 (str, or list or array-like of str, optional): Gene(s) for column(s) to select from df1_name. str if one key, list or array-like of str if multiple.
        Default of None will select entire dataframe.
    genes2 (str, or list or array-like of str, optional): Gene(s) for Column(s) to select from df2_name. str if one key, list or array-like of str if multiple.
        Default of None will select entire dataframe.
    how (str, optional): How to perform the join, acceptable values are from ['outer', 'inner', 'left', 'right']. Defaults to 'outer'.
    quiet (bool, optional): Whether to warn when inserting NaNs. Defaults to False.
    tissue_type (str): Acceptable values in ["tumor","normal","both"]. Specifies the desired tissue type for the dataframe. Defaults to "both".
    
    Returns:
    pandas.DataFrame: The selected columns from the two omics dataframes, joined into one dataframe.

```

--------------------------------

### List Available Data Sources for a Dataset

Source: https://paynelab.github.io/cptac/tutorial02_pandas.html

Display all available data sources for a loaded cancer dataset. This helps in identifying the specific data types and their sources that can be retrieved.

```python
en.list_data_sources()
```

--------------------------------

### List available data sources in the dataset

Source: https://paynelab.github.io/cptac/tutorial04_multiindex.html

Use the list_data_sources method to view all available data types and their corresponding sources within the loaded dataset. This helps in identifying relevant data for analysis.

```python
en.list_data_sources()

```

--------------------------------

### Get Unique Values for Ancestry Prediction

Source: https://paynelab.github.io/cptac/usecase06_derived_molecular.html

Retrieves and displays the unique values present in the 'washU_pred_ancestry' column of the derived molecular dataframe.

```python
der_molecular['washU_pred_ancestry'].unique()
```

--------------------------------

### Load Endometrial Cancer Data using CPTAC

Source: https://paynelab.github.io/cptac/usecase09_clinical_outcomes.html

Initializes the CPTAC Ucec (Endometrial Cancer) dataset and loads clinical, proteomics, and follow-up data. Note: A warning may appear if the CPTAC library is outdated.

```python
en = cptac.Ucec()
clinical = en.get_clinical('mssm')
proteomics = en.get_proteomics('umich')
follow_up = en.get_followup('mssm')
```

--------------------------------

### Import CPTAC Utilities

Source: https://paynelab.github.io/cptac/tutorial04_multiindex.html

Imports the cptac.utils submodule, commonly aliased as 'ut', which contains utility functions for data manipulation.

```python
import cptac.utils as ut
```

--------------------------------

### Filter Columns by Gene Name Prefix

Source: https://paynelab.github.io/cptac/tutorial04_multiindex.html

Selects columns from the proteomics DataFrame where the 'Name' level of the multi-index starts with 'A'. This is useful for filtering genes based on a common prefix.

```python
gene1_filter = prot.columns.get_level_values("Name").str.startswith("A") # Select all columns where the gene starts with "A". This will grab every column where the key "Name" starts with AA
gene1_data = prot.loc[:, gene1_filter]
gene1_data.head()
```

--------------------------------

### Import Python Packages

Source: https://paynelab.github.io/cptac/usecase06_derived_molecular.html

Imports necessary Python libraries and loads the Endometrial dataset from the cptac package.

```python
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
import cptac
en = cptac.Ucec() # Loading the Endometrial dataset
```

--------------------------------

### Import cptac and Load Ucec Dataset

Source: https://paynelab.github.io/cptac/tutorial03_joining_dataframes.html

Imports the cptac package and creates a Ucec (Endometrial carcinoma) data object. It also lists available data sources within the dataset.

```python
# Start by importing the cptac package
import cptac

# Create an endometrial data object, named 'en'
en = cptac.Ucec()

# List the available data sources
en.list_data_sources()

```

--------------------------------

### Import Data and Dependencies for Survival Analysis

Source: https://paynelab.github.io/cptac/usecase09_clinical_outcomes.html

Imports necessary libraries for data manipulation, statistical analysis, and visualization, including pandas, cptac, numpy, matplotlib, seaborn, scipy, and lifelines. Sets up the plotting environment.

```python
import pandas as pd
import cptac
import cptac.utils as ut
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import scipy
import lifelines
from lifelines import KaplanMeierFitter
from lifelines import CoxPHFitter
from lifelines.statistics import proportional_hazard_test
%matplotlib inline
```

--------------------------------

### Initialize and Fit CoxPHFitter Model

Source: https://paynelab.github.io/cptac/usecase09_clinical_outcomes.html

Initializes a CoxPHFitter object and fits it to the cleaned DataFrame. Requires specifying the duration and event columns for survival analysis.

```python
cph = CoxPHFitter()
cph.fit(df_clean, duration_col = "Days_Until_Last_Contact_Or_Death", 
        event_col = "Survival status (1, dead; 0, alive)")
```

--------------------------------

### Join Proteomics and Transcriptomics Data for SOX9

Source: https://paynelab.github.io/cptac/usecase01_omics.html

Use this function to join proteomics and transcriptomics data for a specific gene from the colon cancer dataset. Ensure the CPTAC library is installed and imported.

```python
co = cptac.Coad()
source1 = 'umich'
source2 = 'bcm'
colon_cross = co.join_omics_to_omics(df1_name="proteomics", df2_name="transcriptomics",
                                     df1_source=source1, df2_source=source2,
                                     genes1="SOX9", genes2="SOX9")

sns.set(style="darkgrid")
plot = sns.regplot(x=colon_cross.columns[0], y=colon_cross.columns[1], data=colon_cross)
plot.set(xlabel='Proteomics', ylabel='Transcriptomics', title='Proteomics vs. Transcriptomics for the SOX9 gene')
plt.show()
```

--------------------------------

### Compile List of Outlier Samples

Source: https://paynelab.github.io/cptac/usecase08_outliers.html

This code compiles a list of all sample column names from 'erbb2_outliers' where the value is 1.0, indicating an outlier. The resulting list is then printed.

```python
outlier_list = erbb2_outliers.columns[erbb2_outliers.isin([1.0]).all()].tolist()
print(outlier_list)
```

--------------------------------

### Load and Inspect Phosphoproteomics Data

Source: https://paynelab.github.io/cptac/tutorial04_multiindex.html

This snippet demonstrates loading phosphoproteomics data and inspecting its structure, which features multi-level column indices. It's useful for understanding complex datasets with hierarchical headers.

```python
import pandas as pd

df = pd.read_csv("../CPTAC-3.csv")
df.head()
```

--------------------------------

### Visualize BMI by Cancer History

Source: https://paynelab.github.io/cptac/usecase02_clinical_attributes.html

Create a KDE plot for BMI distribution based on cancer history, excluding 'Unknown' values, and a combined boxplot and stripplot to visualize BMI summary statistics. This helps compare BMI across different cancer histories.

```python
cancer_bmi = clinical_data[['history_of_cancer', 'bmi']]

sns.displot(cancer_bmi[cancer_bmi['history_of_cancer'] != "Unknown"], x="bmi", hue="history_of_cancer", kind="kde", common_norm=False)
plt.show()

diabetes_plot = sns.boxplot(data=cancer_bmi, x="history_of_cancer", y="bmi", showfliers=False)
diabetes_plot = sns.stripplot(data=cancer_bmi, x="history_of_cancer", y="bmi", jitter=True, color=".3")
plt.show()
```

--------------------------------

### Visualize BMI by Tumor Stage

Source: https://paynelab.github.io/cptac/usecase02_clinical_attributes.html

Generate a Kernel Density Estimate (KDE) plot to visualize BMI distribution across different pathological tumor stages and a boxplot with stripplot to show BMI summary statistics per stage. `showfliers=False` hides outliers in the boxplot.

```python
sns.displot(clinical_data, x="bmi", hue="tumor_stage_pathological", kind="kde", common_norm=False, bw_adjust=.7)
plt.show()

figoStrip = sns.boxplot(data=clinical_data, x="tumor_stage_pathological", y="bmi", showfliers=False)
figoStrip = sns.stripplot(data=clinical_data, x="tumor_stage_pathological", y="bmi", jitter=True, color=".3")
figoStrip.set(xlabel = "FIGO stage")
plt.show()
```

--------------------------------

### Import CPTAC and Load Endometrial Dataset

Source: https://paynelab.github.io/cptac/usecase01_omics.html

Imports the CPTAC package and loads the Endometrial (Ucec) dataset. This object provides access to endometrial cancer patient data.

```python
import cptac
import cptac.utils as ut

en = cptac.Ucec()
```

--------------------------------

### Import Libraries and Load Data

Source: https://paynelab.github.io/cptac/usecase05_enrichment_analysis.html

Imports essential Python libraries for data manipulation, visualization, and GSEA. Initializes the CPTAC endometrial dataset (Ucec).

```python
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from scipy import stats
import gseapy as gp
from gseapy.plot import barplot, dotplot

import cptac
en = cptac.Ucec()

```

--------------------------------

### Import Data Analysis Libraries

Source: https://paynelab.github.io/cptac/usecase02_clinical_attributes.html

Import pandas for data manipulation, matplotlib for plotting, and seaborn for enhanced visualizations. These are standard libraries for data analysis tasks.

```python
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
```

--------------------------------

### Import Python Packages

Source: https://paynelab.github.io/cptac/usecase08_outliers.html

Imports necessary libraries for data analysis, visualization, and specific CPTAC and Blacksheep functionalities.

```python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
```

--------------------------------

### Load and Display Transcriptomics Data Head

Source: https://paynelab.github.io/cptac/tutorial01_data_intro.html

Loads transcriptomics data for a specific sample type ('bcm') and displays its first few rows. This allows for inspection of transcriptomics data structure and content.

```python
transcriptomics = en.get_transcriptomics('bcm')
transcriptomics.head()

```

--------------------------------

### Import Data Science and CPTAC Packages

Source: https://paynelab.github.io/cptac/usecase04_mutations_and_omics.html

Import necessary libraries for data manipulation, visualization, and accessing cancer data. Initializes the Ucec dataset from the CPTAC package.

```python
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

import cptac
import cptac.utils as ut

en = cptac.Ucec()

```

--------------------------------

### Prepare Data for TP53 Trans-Effects Analysis (Python)

Source: https://paynelab.github.io/cptac/usecase07_trans_genetic_effect.html

Prepares a dataframe for analyzing trans-genetic effects of TP53. It joins omics data with mutation data and categorizes samples into 'Mutated' or 'Wildtype' based on TP53 mutation status.

```python
gene = "TP53"
print("\nGene: ", gene)

# Step 1: Create dataframe in order to do comparisons with wrap_ttest
# Note that here we do not pass any values for 'omics_genes', so it will use all available genes
protdf = en.join_omics_to_mutations(mutations_genes=[gene],
                                    mutations_source="washu",
                                    omics_name=omics,
                                    omics_source="umich")
protdf = protdf.loc[protdf['Sample_Status'] == 'Tumor']

protdf = protdf.loc[:,~protdf.columns.duplicated()]

for ind, row in protdf.iterrows():
    mutation_status = row["TP53_Mutation_Status_washu_somatic_mutation"]
    if mutation_status == 'Single_mutation' or mutation_status == 'Multiple_mutation':
        protdf.at[ind,'Gene Mutation Status'] = 'Mutated'
    else:
        protdf.at[ind,'Gene Mutation Status'] = 'Wildtype'
```

--------------------------------

### Load and Print Clinical Data Columns

Source: https://paynelab.github.io/cptac/usecase02_clinical_attributes.html

Loads clinical data for a specified cohort and prints a list of all available column names. This is useful for understanding the scope of clinical information.

```python
clinical_data = en.get_clinical('mssm')
print(clinical_data.columns.to_list()) # a ton of data columns
```

--------------------------------

### Access First 5 Rows by Index

Source: https://paynelab.github.io/cptac/tutorial02_pandas.html

Use `.iloc` with slicing to access a range of rows by their integer position. This is useful for previewing data.

```python
proteomics.iloc[0:5]
```

--------------------------------

### Display DataFrame Head (Post-Cleaning)

Source: https://paynelab.github.io/cptac/usecase09_clinical_outcomes.html

Shows the first few rows of the cleaned DataFrame. Useful for verifying the results of data cleaning operations.

```python
df_clean.head()
```

--------------------------------

### Load Multiple Cancer Datasets

Source: https://paynelab.github.io/cptac/tutorial02_pandas.html

Load multiple cancer datasets simultaneously by assigning each to a unique variable. Ensure that functions used are compatible with the intended dataset, as API calls are consistent across datasets.

```python
ov = cptac.Ov()
co = cptac.Coad()
```

--------------------------------

### Download CPTAC Breast Cancer Transcriptomics Data

Source: https://paynelab.github.io/cptac/tutorial01_data_intro.html

Instantiate the Brca dataset and download transcriptomics data from the 'broad' source. This is useful when you need to access data offline or want to ensure data is available before running analyses.

```python
br = cptac.Brca()
br.get_transcriptomics('broad')
```

--------------------------------

### Join specific columns from dataframes

Source: https://paynelab.github.io/cptac/tutorial03_joining_dataframes.html

Specify the exact columns to join from each dataframe by passing a list of column names. This is useful for reducing the size of the resulting dataframe and focusing on relevant data.

```python
clin_and_tran = en.multi_join({"mssm clinical": ["age", "Overall survival, days"], "bcm transcriptomics": ["ZYX", 'ZZEF1']})
clin_and_tran.head()
```

--------------------------------

### Import cptac Package

Source: https://paynelab.github.io/cptac/tutorial05_updates.html

Importing the cptac package automatically checks for the latest version. If an update is available, a warning will be displayed.

```python
import cptac

```

--------------------------------

### Display DataFrame Head

Source: https://paynelab.github.io/cptac/usecase09_clinical_outcomes.html

Displays the first few rows of a pandas DataFrame. Useful for a quick inspection of the data structure and content.

```python
print(df_genes)
```

--------------------------------

### Importing Python Packages for Pathway Analysis

Source: https://paynelab.github.io/cptac/usecase10_pathway_overlay.html

Imports essential Python libraries for data manipulation, statistical analysis, and visualization, including cptac, pandas, matplotlib, and scipy.

```python
import cptac
import pandas as pd
import cptac.utils as utils
import matplotlib.pyplot as plt
from scipy import stats
import statsmodels as statsmodels
from IPython.display import Image
```

--------------------------------

### Load and Display Clinical Data

Source: https://paynelab.github.io/cptac/tutorial01_data_intro.html

Loads the clinical data for the 'mssm' cohort and displays the first 5 rows. This is useful for an initial overview of the available clinical information.

```python
clinical = en.get_clinical('mssm')
clinical.head()
```

--------------------------------

### Calculate IQR and Extreme Outlier Bounds (Python)

Source: https://paynelab.github.io/cptac/usecase03_clinical_and_acetylation.html

Defines a function to compute the first quartile (Q1), third quartile (Q3), and interquartile range (IQR). It then calculates the lower and upper bounds for extreme outliers, defined as 3 times the IQR below Q1 and above Q3.

```python
def calculate_iqr_and_bounds_extreme(data):
    Q1 = data.quantile(0.25)
    Q3 = data.quantile(0.75)
    IQR = Q3 - Q1

    # Define bounds for extreme outliers
    lower_bound = Q1 - 3.0 * IQR
    upper_bound = Q3 + 3.0 * IQR

    return lower_bound, upper_bound
```

--------------------------------

### Import Libraries for CPTAC Analysis

Source: https://paynelab.github.io/cptac/usecase07_trans_genetic_effect.html

Imports essential Python libraries for data analysis, statistical computations, visualization, and accessing CPTAC data. Initializes the CPTAC Ucec dataset.

```python
import pandas as pd
import numpy as np
import scipy.stats
import matplotlib.pyplot as plt
import seaborn as sns
import cptac
import cptac.utils as ut

en = cptac.Ucec()
```

--------------------------------

### Load Acetylproteomics Data

Source: https://paynelab.github.io/cptac/usecase03_clinical_and_acetylation.html

Retrieves the acetylproteomics data for the Endometrial dataset. The dataframe has a multiindex including Gene Name, Acetylation Site, and Peptide Sequence.

```python
en.get_acetylproteomics('umich')
```

--------------------------------

### Prepare Column List for T-test

Source: https://paynelab.github.io/cptac/usecase07_trans_genetic_effect.html

Creates a list of columns from the formatted dataframe that will be used for t-test comparisons. Excludes the 'Gene Mutation Status' column from this list.

```python
col_list = list(protdf.columns)
col_list.remove('Gene Mutation Status')
```

--------------------------------

### Generate General Kaplan-Meier Plot

Source: https://paynelab.github.io/cptac/usecase09_clinical_outcomes.html

Creates a general Kaplan-Meier plot for overall survival using the KaplanMeierFitter from the lifelines package. Requires 'Days_Until_Last_Contact_Or_Death' as time and 'Survival status (1, dead; 0, alive)' as the event observed.

```python
time = focus_group['Days_Until_Last_Contact_Or_Death']
status = focus_group['Survival status (1, dead; 0, alive)']

kmf = KaplanMeierFitter()
kmf.fit(time, event_observed = status)
kmf.plot()
```

--------------------------------

### Combine Time-to-Event Columns

Source: https://paynelab.github.io/cptac/usecase09_clinical_outcomes.html

Combines 'number_of_days_from_date_of_collection_to_date_of_last_contact' and 'number_of_days_from_date_of_collection_to_date_of_death' into a single 'Days_Until_Last_Contact_Or_Death' column. This simplifies time-to-event calculations for survival analysis.

```python
cols = ['number_of_days_from_date_of_collection_to_date_of_last_contact', 'number_of_days_from_date_of_collection_to_date_of_death']

focus_group = focus_group.assign(Days_Until_Last_Contact_Or_Death=focus_group[cols].sum(1)).drop(cols, axis=1)
```

--------------------------------

### Create Data Subset for Analysis

Source: https://paynelab.github.io/cptac/usecase09_clinical_outcomes.html

Selects specific columns for survival analysis, including survival status, time-related metrics, tumor stage, and proteomics data for specified genes. Duplicates are removed from the resulting subset.

```python
#Determine columns to focus on, and create a subset to work with
columns_to_focus_on = ['Survival status (1, dead; 0, alive)',
                       'number_of_days_from_date_of_collection_to_date_of_last_contact', 
                       'number_of_days_from_date_of_collection_to_date_of_death',
                       'tumor_stage_pathological']

#This adds the protein data that we got from the clinical and proteomics join
#so that it will be present in our subset of data to work with
for i in range(len(omics_genes)):
    omics_genes[i] += '_umich_proteomics'
    columns_to_focus_on.append(omics_genes[i])

focus_group = clin_prot_follow[columns_to_focus_on].copy().drop_duplicates()

```

--------------------------------

### Join Metadata DataFrames

Source: https://paynelab.github.io/cptac/tutorial03_joining_dataframes.html

Use `multi_join` to combine multiple dataframes. Pass an empty string or list to select the entire dataframe for joining.

```python
clin_and_tran = en.multi_join({
    "mssm clinical": "",
    "bcm transcriptomics": '' # Note that by using an empty string or list as the value, we join the entire dataframe
})

clin_and_tran.head()
```

--------------------------------

### Join Metadata to Multiple Mutation Genes

Source: https://paynelab.github.io/cptac/tutorial03_joining_dataframes.html

Use `multi_join` to combine clinical data with mutation data for several genes. By default, it returns all mutations as lists.

```python
en.multi_join({"mssm clinical": ["age", "sex", "race"],
               "harmonized somatic_mutation": ["SHANK2", "PTEN", "TP53"]})

```

--------------------------------

### Rename and Merge Follow-up Data

Source: https://paynelab.github.io/cptac/usecase09_clinical_outcomes.html

Renames the 'PPID' column to 'Patient_ID' in the follow-up table and merges it with the clinical and protein data. Requires 'follow_up' and 'clinical_and_protein' dataframes to be available.

```python
follow_up = follow_up.rename({'PPID' : 'Patient_ID'}, axis='columns')
clin_prot_follow = pd.merge(clinical_and_protein, follow_up, on = "Patient_ID")

```

--------------------------------

### Overlay Proteosome Data on Reactome Pathway

Source: https://paynelab.github.io/cptac/usecase10_pathway_overlay.html

Use `reactome_pathway_overlay` to query Reactome and overlay proteosome data. Specify the pathway ID, expression data frame, and export options. Set `quality` to 10 for detailed zoom. Requires the `cptac` package and expression data in `dfObj['simplified_change']`.

```python
expression_vals, image_path = utils.reactome_pathway_overlay(
    'R-HSA-1236978', 
    df=dfObj['simplified_change'], 
    analysis_token=None, 
    open_browser=False,
    export_path="test.png", 
    image_format='png', 
    display_col_idx=None,                      
    diagram_colors='Modern', 
    overlay_colors='Standard', 
    quality=10)
Image(image_path)
```

--------------------------------

### Load Ovarian Dataset and Plot PTEN Correlation

Source: https://paynelab.github.io/cptac/usecase01_omics.html

Loads the Ovarian (Ov) cancer dataset and performs the same omics data comparison for the PTEN gene. This demonstrates the reusability of the analysis across different cancer types.

```python
ov = cptac.Ov()
ovarian_cross = ov.multi_join({"umich proteomics": "PTEN", "bcm transcriptomics": "PTEN"})

# Use cptac.utils.reduce_multiindex to get rid of a level we don't need from the column index
ovarian_cross = ut.reduce_multiindex(ovarian_cross, levels_to_drop="Database_ID")

sns.set(style="darkgrid")
plot = sns.regplot(x=ovarian_cross.columns[0], y=ovarian_cross.columns[1], data=ovarian_cross)
plot.set(xlabel='Proteomics', ylabel='Transcriptomics', title='Proteomics vs. Transcriptomics for the PTEN gene')
plt.show()
```

--------------------------------

### Load CPTAC Dataset

Source: https://paynelab.github.io/cptac/tutorial01_data_intro.html

Load a specific CPTAC dataset into a Python variable for further analysis. Use the dataset name in UpperCamelCase.

```python
en = cptac.Ucec()
```

--------------------------------

### View All Column Names in Pandas DataFrame

Source: https://paynelab.github.io/cptac/tutorial02_pandas.html

Append `.columns` to a DataFrame to view a list of all column names. Use `.columns.values` to see all names if there are many.

```python
proteomics.columns
```

--------------------------------

### Display Derived Molecular Columns

Source: https://paynelab.github.io/cptac/usecase06_derived_molecular.html

Prints the column names of the derived molecular dataframe to inspect available attributes.

```python
der_molecular.columns
```

--------------------------------

### Display Proteomics Data Head

Source: https://paynelab.github.io/cptac/tutorial01_data_intro.html

Displays the first few rows of the proteomics dataframe to show its structure and sample data. This is useful for initial data inspection.

```python
proteomics.head()

```

--------------------------------

### Retrieve Proteins Interacting with ARID1A

Source: https://paynelab.github.io/cptac/usecase07_trans_genetic_effect.html

Fetches a list of proteins known to directly interact with the specified gene (ARID1A) using the Bioplex process via the CPTAC utility function. Prints the list of interacting proteins.

```python
gene = "ARID1A"
omics = "proteomics"
interacting_proteins = ut.get_interacting_proteins_bioplex(gene)
print("Interacting Proteins:")
print(interacting_proteins)
```

--------------------------------

### Join and Plot Proteomics vs. Transcriptomics for RPL11

Source: https://paynelab.github.io/cptac/usecase01_omics.html

Applies the same joining and plotting procedure to the RPL11 gene. This visualizes the correlation between mRNA and protein abundance for RPL11 in the Endometrial dataset.

```python
gene = 'RPL11'
gene_cross = en.multi_join({"umich proteomics": gene, "bcm transcriptomics": gene});
# Use cptac.utils.reduce_multiindex to get rid of a level we don't need from the column index
gene_cross = ut.reduce_multiindex(gene_cross, levels_to_drop="Database_ID")
plot = sns.regplot(x=gene_cross.columns[0], y=gene_cross.columns[1], data=gene_cross, color="green")
plot.set(xlabel='Proteomics', ylabel='Transcriptomics',
         title='Proteomics vs. Transcriptomics for the ' + gene + ' gene')
plt.show()
```

--------------------------------

### Join metadata and omics dataframes

Source: https://paynelab.github.io/cptac/tutorial03_joining_dataframes.html

Use `multi_join` to combine dataframes. By default, it joins all common columns. Use `.head()` to display the first few rows of the resulting dataframe.

```python
clin_and_tran = en.multi_join({"mssm clinical":'', "bcm transcriptomics":''})
clin_and_tran.head()
```

--------------------------------

### Load Clinical Data and Filter by Sex using .loc

Source: https://paynelab.github.io/cptac/tutorial02_pandas.html

Loads clinical data and then filters it to select only patients recorded as 'Female'. This is useful for analyzing data specific to a particular gender.

```python
clinical = en.get_clinical("mssm")
female_clinical = clinical.loc[clinical["sex"] == "Female"]
```

--------------------------------

### Load omics data from specified sources

Source: https://paynelab.github.io/cptac/tutorial04_multiindex.html

Retrieve specific omics data (proteomics, phosphoproteomics, CNV) from their designated sources within the Ucec dataset. Note that some data loading may take time.

```python
#Some data may take several minutes to load
en.get_proteomics('umich')
en.get_phosphoproteomics('umich')
en.get_CNV('washu')

```

--------------------------------

### Analyze CTNNB1 Gene Mutation Impact on Protein Abundance

Source: https://paynelab.github.io/cptac/usecase04_mutations_and_omics.html

Repeats the process for a different gene (CTNNB1), joining its mutation and omics data, and then generating box and strip plots to visualize protein abundance changes. This demonstrates the reusability of the analysis pipeline for other genes.

```python
gene2 = 'CTNNB1'
omics_mutations2 = en.join_omics_to_mutations(
    omics_name="proteomics",
    omics_source='umich',
    mutations_genes=gene2,
    mutations_source='washu',
    omics_genes=gene2, 
    mutations_filter=[]
)

boxplot2 = sns.boxplot(
    data=omics_mutations2, 
    x=gene2 + "_Mutation",
    y=gene2 + "_umich_proteomics", 
    order=["Wildtype_Normal","Wildtype_Tumor","Missense_Mutation","In_Frame_Del"]
)
boxplot2.set_title("CTNNB1 gene mutation protein abundance")
boxplot2 = sns.stripplot(
    data=omics_mutations2, 
    x=gene2 + "_Mutation",
    y=gene2 + "_umich_proteomics",
    jitter=True, 
    color=".3", 
    order=["Wildtype_Normal","Wildtype_Tumor","Missense_Mutation","In_Frame_Del"]
)
boxplot2.set(xlabel="Somatic Gene Mutation",ylabel="Proteomics")
boxplot2.set_xticklabels(boxplot2.get_xticklabels(),rotation=30)
plt.show()

```

--------------------------------

### Join Omics Data to Mutations

Source: https://paynelab.github.io/cptac/usecase07_trans_genetic_effect.html

Use this method to create a dataframe of protein measurements for a gene and its interactors. It handles missing genes by raising a warning and filling with NaN.

```python
protdf = en.join_omics_to_mutations(mutations_genes=gene,
                                    mutations_source='washu',
                                    omics_name=omics,
                                    omics_source='umich',
                                    omics_genes=interacting_proteins)
protdf = protdf.loc[protdf['Sample_Status'] == 'Tumor']
protdf.head()
```

--------------------------------

### Join Metadata to Mutations

Source: https://paynelab.github.io/cptac/tutorial03_joining_dataframes.html

Join clinical metadata to mutation data. Specify the metadata source, desired columns, mutation source, target gene, and mutation filter. This function can generate warnings for unknown mutation types or samples with no mutations found.

```python
en.join_metadata_to_mutations(
    metadata_name="clinical",
    metadata_source="mssm",
    metadata_cols=["age", "sex", "race"],
    mutations_source="harmonized",
    mutations_genes="SHANK2",
    mutations_filter=["Missense_Mutation"])
```

--------------------------------

### Retrieve Clinical Data

Source: https://paynelab.github.io/cptac/tutorial03_joining_dataframes.html

Use the `get_clinical` function to fetch clinical data for a specified source. This is often a prerequisite for joining metadata with other datasets.

```python
en.get_clinical('mssm')
```

--------------------------------

### Simplify Mutation Status

Source: https://paynelab.github.io/cptac/usecase07_trans_genetic_effect.html

This code simplifies mutation status to 'Mutated' or 'Wildtype' and removes redundant columns. Ensure the 'ARID1A_Mutation_Status_washu_somatic_mutation' column exists for this operation.

```python
protdf = protdf.loc[:,~protdf.columns.duplicated()]
for ind, row in protdf.iterrows():
    mutation_status = row["ARID1A_Mutation_Status_washu_somatic_mutation"]
    if mutation_status == 'Single_mutation' or mutation_status == 'Multiple_mutation':
        protdf.at[ind,'Gene Mutation Status'] = 'Mutated'
    else:
        protdf.at[ind,'Gene Mutation Status'] = 'Wildtype'
protdf = protdf.drop(gene+"_Mutation",axis=1)
protdf = protdf.drop(gene+"_Location",axis=1)
protdf = protdf.drop(gene+"_Mutation_Status_washu_somatic_mutation", axis=1)
protdf = protdf.drop("Sample_Status",axis=1)
```

--------------------------------

### Export DataFrame to CSV

Source: https://paynelab.github.io/cptac/tutorial02_pandas.html

Use the `to_csv` method to save a dataframe to a file. Specify the output path and the desired separator.

```python
clinical = en.get_clinical('mssm')
clinical.to_csv(path_or_buf="clinical_df.tsv", sep='\t')
```

--------------------------------

### Access CoxPHFitter Confidence Intervals

Source: https://paynelab.github.io/cptac/usecase09_clinical_outcomes.html

Retrieves and displays the confidence intervals for the hazard ratios calculated by the CoxPHFitter model. This helps in assessing the uncertainty around the estimated effects of covariates.

```python
cph.confidence_intervals_

```

--------------------------------

### Plot Proteomics vs. Transcriptomics for A1BG

Source: https://paynelab.github.io/cptac/usecase01_omics.html

Generates a scatter plot to visualize the correlation between proteomics and transcriptomics data for the A1BG gene in the Endometrial dataset. A positive correlation is observed.

```python
sns.set(style="darkgrid")
plot = sns.regplot(x=A1BG_cross.columns[0], y=A1BG_cross.columns[1], 
                   data=A1BG_cross)
plot.set(xlabel='Proteomics', ylabel='Transcriptomics', 
         title='Proteomics vs. Transcriptomics for the A1BG gene')
plt.show()
```

--------------------------------

### Visualize ERBB2 Gene Outliers with Swarmplot

Source: https://paynelab.github.io/cptac/usecase08_outliers.html

Generates a swarmplot to visualize the distribution of ERBB2 protein levels across different histologic types, highlighting outliers and annotating specific samples.

```python
# Setup
plt.figure(figsize=(8, 8))
sns.set_palette('colorblind')
cols = {'Inflitrating Ductal Carcinoma': 0, 'Inflitrating Lobular Carcinoma':1, 'Other (specify)':2, 'other, specify':3, 'IDC and DCIS':4, 'Infiltrating Ductal and Lobular Carcinoma':5, 'Mucinous Carcinoma':6, 'Infiltrating Ductal and Mucinous Carcinoma':7, 'invasive ductal carcinoma with extensive high-grade DCIS with comedonecrosis and features of papillary carcinoma':8, 'Mixed ductal and lobular features':9, 'Triple Negative':10}

# Plotting
ax = sns.boxplot(data=combined_data, y='ERBB2_umich_proteomics', x='histologic_type', color='lightgray')
ax = sns.swarmplot(data=combined_data, y='ERBB2_umich_proteomics',x='histologic_type', hue='histologic_type')

# Rotate x-labels for clarity
plt.xticks(rotation=45, ha="right") # rotation by 45 degrees

# Add annotations for outliers
for sample in outlier_list:
    try:
        sample = sample.split("_")[0]
        ax.annotate(sample, (cols[combined_data.transpose()[sample].values[0]], combined_data.transpose()[sample].values[1]))
    except KeyError:
        continue

# Move the legend outside of the plot
ax.legend(loc='center left', bbox_to_anchor=(1, 0.5))

plt.tight_layout() # Adjust layout to prevent any overlaps
plt.show()
```

--------------------------------

### Loading Renal Cancer Proteomics Data

Source: https://paynelab.github.io/cptac/usecase10_pathway_overlay.html

Loads the Renal Cancer (Ccrcc) proteomics dataset using the cptac package. This step is necessary to access the protein expression data for analysis.

```python
ds = cptac.Ccrcc()
df = ds.get_proteomics('umich')
df.head()
```

--------------------------------

### Visualize GSEA results with barplot

Source: https://paynelab.github.io/cptac/usecase05_enrichment_analysis.html

Generates barplots to visualize the Gene Set Enrichment Analysis results. This helps in understanding the pathways significantly associated with the gene lists. Requires matplotlib for display.

```python
barplot(tumor_enr.res2d, title="Proteomics Tumor KEGG_2016")
plt.show()

barplot(normal_enr.res2d, title="Normal KEGG_2016")
plt.show()
```

--------------------------------

### Join Proteomics and Somatic Mutation Data

Source: https://paynelab.github.io/cptac/tutorial03_joining_dataframes.html

Use this function to join omics data with somatic mutation data for specified genes. It handles cases where samples may not have mutations for a gene, filling with 'Wildtype_Tumor' or 'Wildtype_Normal'.

```python
somatic_mutations = en.get_somatic_mutation('harmonized')
selected_prot_and_som_mut = en.join_omics_to_mutations(
    omics_name = "proteomics",
    mutations_genes = "SHANK2",
    omics_genes = ["ARF5", "M6PR"],
    omics_source = 'umich',
    mutations_source = 'harmonized')
selected_prot_and_som_mut.head(10)
```

--------------------------------

### Execute and Print T-test Results

Source: https://paynelab.github.io/cptac/usecase07_trans_genetic_effect.html

Calls the wrap_ttest utility function with the prepared dataframe and column list, then prints the results if they are not None. This function performs Bonferroni correction, leading to strict significance cutoffs.

```python
print("Doing t-test comparisons\n")
wrap_results = ut.wrap_ttest(protdf, 'Gene Mutation Status', col_list)

if wrap_results is not None:
        print(wrap_results)
        print("\n\n")
```

--------------------------------

### Filter Multiple Mutations with multi_join

Source: https://paynelab.github.io/cptac/tutorial03_joining_dataframes.html

Use multi_join to filter multiple mutations. Specify mutation types or locations to prioritize. An empty mutations_filter uses a default hierarchy.

```python
SHANK2_default_filter = en.multi_join({"umich proteomics": ["ARF5", "M6PR"],
                                     "harmonized somatic_mutation": "SHANK2"},
                                    mutations_filter=[])

SHANK2_simple_filter = en.multi_join({"umich proteomics": ["ARF5", "M6PR"],
                                    "harmonized somatic_mutation": "SHANK2"},
                                   mutations_filter=["Missense_Mutation"])

PTEN_complex_filter = en.multi_join({"umich proteomics": ["ARF5", "M6PR"],
                                    "harmonized somatic_mutation": "SHANK2"}, 
                                    mutations_filter=["p.R130Q", "Nonsense_Mutation"])

```

--------------------------------

### Join Clinical and Proteomics Data

Source: https://paynelab.github.io/cptac/usecase05_enrichment_analysis.html

Merges clinical metadata with proteomics data using `join_metadata_to_omics`. Focuses on 'type_of_analyzed_samples' for sample differentiation.

```python
tumorProt = en.join_metadata_to_omics(
    metadata_name="clinical",
    metadata_source='mssm',
    metadata_cols='type_of_analyzed_samples',
    omics_name="proteomics",
    omics_source='umich'
)

```

--------------------------------

### Retrieve Proteomics Dataframe

Source: https://paynelab.github.io/cptac/tutorial02_pandas.html

Access the proteomics data for Endometrial cancer (UCEC) generated by UMich. The `.head()` method is then used to display the first few rows of the resulting pandas DataFrame.

```python
#Let's get the proteomics data associated with Endometrial cancer; it was generated by the team at UMich
proteomics = en.get_proteomics('umich')
proteomics.head()
```
Tokens:
Raw