### Hail Initialization and Spark Output

Source: https://github.com/nhgritctran/phetk/blob/main/docs/tutorials/v0.2.1 APOE PheWAS - CLI.ipynb

This output shows the initialization of Hail, a library for large-scale genomic data analysis, and its integration with Apache Spark. It includes warnings about deprecated features and Spark UI availability, indicating the environment setup for the analysis.

```python
Running on Apache Spark version 3.5.3
SparkUI available at http://all-of-us-5797-m.us-central1-c.c.terra-vpc-sc-7883f2cd.internal:34783
Welcome to
     __  __     <>__
    / /_/ /__  __/ / 
   / __  / _ `/ / / 
  /_/ /_/_,_/_/_/   version 0.2.134-952ae203dbbe
LOGGING: writing to /home/jupyter/workspaces/phetkv8/hail-20250801-0323-0.2.134-952ae203dbbe.log
2025-08-01 03:24:18.414 Hail: WARN: entries(): Resulting entries table is sorted by '(row_key, col_key)'.
    To preserve row-major matrix table order, first unkey columns with 'key_cols_by()'
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.apache.spark.util.SizeEstimator$ (file:/usr/lib/spark/jars/spark-core_2.12-3.5.3.jar) to field java.lang.ref.Reference.referent
WARNING: Please consider reporting this to the maintainers of org.apache.spark.util.SizeEstimator$
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
```

--------------------------------

### Install PheTK using pip

Source: https://github.com/nhgritctran/phetk/blob/main/README.md

Installs the latest version of the PheTK library using pip. It also shows how to install a specific older version (0.1.47) and how to check the currently installed version.

```bash
pip install phetk --upgrade

```

```bash
pip install PheTK==0.1.47

```

```bash
pip show phetk | grep Version

```

--------------------------------

### Install and Show PheTK Version

Source: https://github.com/nhgritctran/phetk/blob/main/docs/tutorials/v0.2.1 APOE PheWAS - dsub.ipynb

Installs or upgrades the PheTK library and displays its current version. This is a prerequisite for using PheTK functionalities.

```python
!pip install phetk --upgrade
```

```python
!pip show PheTK | grep Version
```

--------------------------------

### Install PheTK using Docker

Source: https://github.com/nhgritctran/phetk/blob/main/README.md

Pulls the latest Docker image for the PheTK library from Docker Hub. This allows users to run PheTK in a containerized environment.

```bash
docker pull phetk/phetk:latest

```

--------------------------------

### Calculate Time to First Phecode Event (CLI)

Source: https://github.com/nhgritctran/phetk/blob/main/docs/phecode-module.md

Command-line interface for calculating the time from study start to the first phecode event for survival analysis. This adds a 'phecode_time_to_event' column. It requires paths to the phecode counts and cohort files, the study start date column name, and optionally the time unit and output file path.

```bash
phetk phecode add-phecode-time-to-event \
  --phecode_count_file_path "phecode_counts.tsv" \
  --cohort_file_path "cohort.tsv" \
  --study_start_date_col "study_start_date" \
  --time_unit "years" \
  --output_file_path "phecode_counts_with_time.tsv"
```

--------------------------------

### Display PheTK Package Version

Source: https://github.com/nhgritctran/phetk/blob/main/docs/tutorials/v0.2.1 APOE PheWAS - Logistic Regression.ipynb

Displays the currently installed version of the PheTK package. This command is useful for verifying the installation and checking compatibility with specific notebook requirements.

```bash
!pip show PheTK | grep Version
```

--------------------------------

### PheTK CLI - Cohort Creation

Source: https://github.com/nhgritctran/phetk/blob/main/docs/tutorials/v0.2.1 APOE PheWAS - CLI.ipynb

Command to create a cohort based on genotype information.

```APIDOC
## POST /cohort/by-genotype

### Description
Creates a cohort based on specified genotype information from a given platform.

### Method
POST

### Endpoint
/cohort/by-genotype

### Parameters
#### Query Parameters
- **--platform** (string) - Required - The genomic platform to use (e.g., 'aou').
- **--aou_db_version** (integer) - Required - The version of the All of Us database.
- **--chromosome_number** (integer) - Required - The chromosome number.
- **--genomic_position** (integer) - Required - The genomic position on the chromosome.
- **--ref_allele** (string) - Required - The reference allele.
- **--alt_allele** (string) - Required - The alternative allele.
- **--gt_dict** (string) - Required - A JSON string defining genotype mappings (e.g., '{"0": "0/0", "1":["0/1", "1/1"]}').
- **--output_file_path** (string) - Required - The path to save the output cohort file.

### Request Example
```bash
phetk cohort by-genotype \
--platform aou \
--aou_db_version 8 \
--chromosome_number 19 \
--genomic_position 44908684 \
--ref_allele T \
--alt_allele C \
--gt_dict '{"0": "0/0", "1":["0/1", "1/1"]}' \
--output_file_path "rs429358_cohort.tsv"
```

### Response
#### Success Response (200)
- **message** (string) - Confirmation message indicating cohort creation.

#### Response Example
```json
{
  "message": "Cohort created successfully."
}
```
```

--------------------------------

### Calculate Time to First Phecode Event (Python)

Source: https://github.com/nhgritctran/phetk/blob/main/docs/phecode-module.md

Calculates the time from study start to the first phecode event for survival analysis, adding a 'phecode_time_to_event' column. This static method requires paths to the phecode counts and cohort files, the study start date column name, and optionally the time unit and output file path.

```python
from phetk.phecode import Phecode

# Static method - no instance needed
Phecode.add_phecode_time_to_event(
    phecode_count_file_path="phecode_counts.tsv",
    cohort_file_path="cohort.tsv",
    study_start_date_col="study_start_date",
    time_unit="years",
    output_file_path="phecode_counts_with_time.tsv"
)
```

--------------------------------

### PheTK CLI - Generate Phecode Counts

Source: https://github.com/nhgritctran/phetk/blob/main/docs/tutorials/v0.2.1 APOE PheWAS - CLI.ipynb

Command to generate phecode counts from a specified platform and phecode version.

```APIDOC
## POST /phecode/count-phecode

### Description
Generates counts for phecodes based on the specified platform and phecode version.

### Method
POST

### Endpoint
/phecode/count-phecode

### Parameters
#### Query Parameters
- **--platform** (string) - Required - The genomic platform to use (e.g., 'aou').
- **--phecode_version** (string) - Required - The version of the phecode mapping to use (e.g., 'X').
- **--output_file_path** (string) - Required - The path to save the output phecode counts file.

### Request Example
```bash
phetk phecode count-phecode \
--platform aou \
--phecode_version X \
--output_file_path aou_phecode_counts.tsv
```

### Response
#### Success Response (200)
- **message** (string) - Confirmation message indicating phecode counts were generated.

#### Response Example
```json
{
  "message": "Phecode counts generated successfully."
}
```
```

--------------------------------

### Calculate Time to Event for Survival Analysis using Python

Source: https://context7.com/nhgritctran/phetk/llms.txt

Computes the time from the study start date to the first phecode event for each participant. This is crucial for survival analyses, such as Cox regression. It requires a phecode count file, a cohort file with study start dates, and specifies the time unit for the output.

```python
from phetk.phecode import Phecode

# Static method - no instance required
Phecode.add_phecode_time_to_event(
    phecode_count_file_path="phecode_counts.tsv",
    cohort_file_path="cohort.tsv",  # Must contain study_start_date column
    study_start_date_col="enrollment_date",
    time_unit="years",  # or "days"
    output_file_path="phecode_counts_with_time.tsv"
)
```

--------------------------------

### Run PheWAS Analysis with phetk

Source: https://github.com/nhgritctran/phetk/blob/main/docs/tutorials/v0.2.1 APOE PheWAS - CLI.ipynb

This command initiates a PheWAS analysis using the phetk tool. It requires specifying various parameters such as PheCode version, file paths for counts and cohorts, covariate columns, and analysis method. The output is a TSV file containing the PheWAS results.

```bash
echo "Running PheWAS..."
phetk phewas \
--phecode_version X \
--phecode_count_file_path aou_phecode_counts.tsv \
--cohort_file_path rs429358_cohort_with_covariates.tsv \
--sex_at_birth_col sex_at_birth \
--male_as_one true \
--covariate_cols age_at_last_ehr_event sex_at_birth pc1 pc2 pc3 pc4 pc5 \
--independent_variable_of_interest genotype \
--min_cases 50 \
--min_phecode_count 2 \
--method logit \
--output_file_path rs429358_phewas_results.tsv
```

--------------------------------

### PheTK CLI - Add Covariates to Cohort

Source: https://github.com/nhgritctran/phetk/blob/main/docs/tutorials/v0.2.1 APOE PheWAS - CLI.ipynb

Command to add covariates to an existing cohort file.

```APIDOC
## POST /cohort/add-covariates

### Description
Adds covariates such as age and sex to a previously created cohort file.

### Method
POST

### Endpoint
/cohort/add-covariates

### Parameters
#### Query Parameters
- **--cohort_file_path** (string) - Required - The path to the input cohort file.
- **--age_at_last_ehr_event** (boolean) - Optional - Whether to include age at last EHR event as a covariate (true/false).
- **--sex_at_birth** (boolean) - Optional - Whether to include sex at birth as a covariate (true/false).
- **--first_n_pcs** (integer) - Optional - The number of principal components to include as covariates.
- **--drop_nulls** (boolean) - Optional - Whether to drop rows with null values in covariates (true/false).
- **--output_file_path** (string) - Required - The path to save the output cohort file with added covariates.

### Request Example
```bash
phetk cohort add-covariates \
--cohort_file_path rs429358_cohort.tsv \
--age_at_last_ehr_event true \
--sex_at_birth true \
--first_n_pcs 5 \
--drop_nulls true \
--output_file_path rs429358_cohort_with_covariates.tsv
```

### Response
#### Success Response (200)
- **message** (string) - Confirmation message indicating covariates were added.

#### Response Example
```json
{
  "message": "Covariates added successfully."
}
```
```

--------------------------------

### Install and Upgrade PheTK Package

Source: https://github.com/nhgritctran/phetk/blob/main/docs/tutorials/v0.2.1 APOE PheWAS - Logistic Regression.ipynb

Installs or upgrades the PheTK Python package to the latest version. This is a prerequisite for using PheTK functionalities. It ensures that the most recent features and bug fixes are available.

```python
!pip install phetk --upgrade
```

--------------------------------

### PheWAS Pipeline: Cohort Creation (CLI)

Source: https://github.com/nhgritctran/phetk/blob/main/docs/tutorials/v0.2.1 APOE PheWAS - CLI.ipynb

Creates a genetic cohort based on specified genotype information using the PheTK CLI. It requires platform details, genomic coordinates, reference and alternative alleles, and a genotype dictionary. Outputs a TSV file containing the cohort data.

```bash
# exit script if a command fails
set -e

# add phetk path
export PATH=$PATH:/home/jupyter/.local/bin

# create cohort
echo "Creating cohort..."echo
phetk cohort by-genotype \
--platform aou \
--aou_db_version 8 \
--chromosome_number 19 \
--genomic_position 44908684 \
--ref_allele T \
--alt_allele C \
--gt_dict '{"0": "0/0", "1":["0/1", "1/1"]}' \
--output_file_path "rs429358_cohort.tsv"
```

--------------------------------

### PheWAS Pipeline: Generate Phecode Counts (CLI)

Source: https://github.com/nhgritctran/phetk/blob/main/docs/tutorials/v0.2.1 APOE PheWAS - CLI.ipynb

Generates phecode counts from a specified platform and phecode version using the PheTK CLI. This step is crucial for subsequent phewas analysis. Outputs a TSV file containing the phecode counts.

```bash
# generate phecode counts
echo "Generating phecode counts..."echo
phetk phecode count-phecode \
--platform aou \
--phecode_version X \
--output_file_path aou_phecode_counts.tsv
```

--------------------------------

### Import PheTK Core Classes

Source: https://github.com/nhgritctran/phetk/blob/main/docs/tutorials/v0.2.1 APOE PheWAS - CLI.ipynb

Imports essential classes from the PheTK library: Cohort, Phecode, and PheWAS. These classes are fundamental for building and running PheWAS analyses.

```python
from phetk.cohort import Cohort
from phetk.phecode import Phecode
from phetk.phewas import PheWAS
```

--------------------------------

### Add Phecode Time to Event

Source: https://github.com/nhgritctran/phetk/blob/main/docs/phecode-module.md

Calculates the time from study start to the first phecode event for survival analysis, adding a 'phecode_time_to_event' column.

```APIDOC
## POST /add_phecode_time_to_event

### Description
Calculate time from study start to first phecode event for survival analysis. Adds phecode_time_to_event column.

### Method
POST

### Endpoint
/add_phecode_time_to_event

### Parameters
#### Query Parameters
- **phecode_count_file_path** (str) - Required - Path to phecode counts CSV/TSV file
- **cohort_file_path** (str) - Required - Path to cohort file with study start dates
- **study_start_date_col** (str) - Required - Column name containing study start dates
- **time_unit** (str) - Optional - Time unit for calculations, "days" or "years" (default: "days")
- **output_file_path** (str) - Optional - Path for output file

### Request Example
```python
from phetk.phecode import Phecode

# Static method - no instance needed
Phecode.add_phecode_time_to_event(
    phecode_count_file_path="phecode_counts.tsv",
    cohort_file_path="cohort.tsv",
    study_start_date_col="study_start_date",
    time_unit="years",
    output_file_path="phecode_counts_with_time.tsv"
)
```

### Response
#### Success Response (200)
- **output_file_path** (str) - Path to the generated TSV file with time to event calculations.

#### Response Example
```json
{
  "output_file_path": "phecode_counts_with_time.tsv"
}
```
```

--------------------------------

### Run PheWAS Analysis with Cox Regression (CLI)

Source: https://github.com/nhgritctran/phetk/blob/main/docs/phewas-module.md

Executes a phenome-wide association study using Cox proportional hazards regression via the command line. Requires specifying phecode version, file paths for counts and cohort data, covariate columns, the independent variable of interest, sex column, and specific Cox regression parameters like start date, control observed time, and phecode observed time columns. Outputs results to a specified file.

```bash
phetk phewas \
  --phecode_version "X" \
  --cohort_file_path "cohort.tsv" \
  --phecode_count_file_path "phecode_counts.tsv" \
  --sex_at_birth_col "sex" \
  --covariate_cols age sex pc1 \
  --independent_variable_of_interest "exposure" \
  --min_cases 50 \
  --min_phecode_count 2 \
  --method "cox" \
  --cox_start_date_col "start_date" \
  --cox_control_observed_time_col "follow_up_time" \
  --cox_phecode_observed_time_col "time_to_event" \
  --output_file_path "cox_results.tsv"
```

--------------------------------

### PheWAS Pipeline: Add Covariates (CLI)

Source: https://github.com/nhgritctran/phetk/blob/main/docs/tutorials/v0.2.1 APOE PheWAS - CLI.ipynb

Adds covariates such as age, sex, and principal components to an existing cohort file using the PheTK CLI. It takes the cohort file path as input and outputs a new TSV file with added covariates, dropping any rows with null values.

```bash
# add covariates
echo "Adding covariates..."echo
phetk cohort add-covariates \
--cohort_file_path rs429358_cohort.tsv \
--age_at_last_ehr_event true \
--sex_at_birth true \
--first_n_pcs 5 \
--drop_nulls true \
--output_file_path rs429358_cohort_with_covariates.tsv
```

--------------------------------

### Run PheWAS Analysis with Cox Regression (Python)

Source: https://github.com/nhgritctran/phetk/blob/main/docs/phewas-module.md

Executes a phenome-wide association study using Cox proportional hazards regression. Requires specifying phecode version, file paths for counts and cohort data, covariate columns, the independent variable of interest, sex column, and specific Cox regression parameters like start date, control observed time, and phecode observed time columns. Outputs results to a specified file.

```python
phewas = PheWAS(
    phecode_version="X",
    phecode_count_file_path="phecode_counts.tsv",
    cohort_file_path="cohort.tsv",
    covariate_cols=["age", "sex", "pc1"],
    independent_variable_of_interest="exposure",
    sex_at_birth_col="sex",
    method="cox",
    cox_start_date_col="start_date",
    cox_control_observed_time_col="follow_up_time",
    cox_phecode_observed_time_col="time_to_event"
)
phewas.run()
```

--------------------------------

### Run PheTK 1-minute demo

Source: https://github.com/nhgritctran/phetk/blob/main/README.md

Executes a quick 1-minute PheWAS demo using the PheTK library. This can be run directly from the terminal or within a Jupyter Notebook environment.

```bash
phetk demo

```

```python
from phetk import demo

demo.run()

```

--------------------------------

### Instantiate PheWAS Class for dsub Execution

Source: https://github.com/nhgritctran/phetk/blob/main/docs/tutorials/v0.2.1 APOE PheWAS - dsub.ipynb

Initializes the PheWAS class with parameters for a PheWAS analysis intended to be run via dsub. It specifies input file paths within a Google Cloud bucket, covariate columns, and analysis method.

--------------------------------

### Run PheTK Demo - Python

Source: https://context7.com/nhgritctran/phetk/llms.txt

Executes an interactive demonstration of PheTK's capabilities using synthetic data. This demo generates mock cohort and phecode count files, then performs a complete PheWAS analysis. It can be run from the command line or directly within a Python script, with options to customize parameters like covariates and the independent variable of interest.

```python
from phetk import demo

# Execute interactive demo with default settings
demo.run()

# Run demo with custom parameters
demo.run(
    covariates_cols=("age", "sex", "pc1", "pc2", "pc3"),
    independent_variable_of_interest="independent_variable_of_interest",
    phecode_to_process=None,  # Process all phecodes
    verbose=False
)
```

--------------------------------

### Configure Phecode Platform (Python)

Source: https://github.com/nhgritctran/phetk/blob/main/docs/phecode-module.md

Initializes the Phecode class with a specified platform. Supports predefined platforms like 'aou' (All of Us) or a 'custom' platform where an ICD file path must be provided.

```python
phecode = Phecode(platform="aou")
```

```python
phecode = Phecode(platform="custom", icd_file_path="path/to/icd_data.tsv")
```

--------------------------------

### Run PheWAS Analysis with dsub

Source: https://github.com/nhgritctran/phetk/blob/main/docs/tutorials/v0.2.1 APOE PheWAS - dsub.ipynb

Initiates a PheWAS analysis using dsub, specifying the Docker image, machine type, and whether to display the dsub command. The machine type recommendation depends on the regression method used (logistic or Cox).

```python
phewas.run_dsub(
    docker_image="phetk/phetk:0.2.2",  # Replace the version number after ":" with the desired version available on dockerhub
    machine_type="c2d-standard-16",  # For logistic regression, highcpu or standard machine is recommended; for Cox regression, standard or highmem machine is recommended
    show_dsub_command=False
)
```

--------------------------------

### Prepare PheWAS Inputs using dsub Commands

Source: https://github.com/nhgritctran/phetk/blob/main/docs/tutorials/v0.2.1 APOE PheWAS - dsub.ipynb

This bash script prepares input files for PheWAS analysis by creating a cohort, adding covariates, and generating phecode counts using PheTK's command-line interface. All operations are directed to a Google Cloud bucket, and the script includes error handling.

```bash
# exit script if a command fails
set -e

# add phetk path if needed; this path is specific for this notebook environment.
export PATH=$PATH:/home/jupyter/.local/bin

# create cohort
echo "Creating cohort..."echo
phetk cohort by-genotype \
--platform aou \
--aou_db_version 8 \
--chromosome_number 19 \
--genomic_position=44908684 \
--ref_allele T \
--alt_allele C \
--gt_dict '{"0": "0/0", "1":["0/1", "1/1"]}' \
--output_file_path "${WORKSPACE_BUCKET}/test/data/dsub__rs429358_cohort.tsv"

# add covariates
echo "Adding covariates..."echo
phetk cohort add-covariates \
--cohort_file_path "${WORKSPACE_BUCKET}/test/data/dsub__rs429358_cohort.tsv" \
--age_at_last_ehr_event true \
--sex_at_birth true \
--first_n_pcs 5 \
--drop_nulls true \
--output_file_path "${WORKSPACE_BUCKET}/test/data/dsub__rs429358_cohort_with_covariates.tsv"

# generate phecode counts
echo "Generating phecode counts..."echo
phetk phecode count-phecode \
--platform aou \
--phecode_version X \
--output_file_path "${WORKSPACE_BUCKET}/test/data/dsub__aou_phecode_counts.tsv"

```

--------------------------------

### Run PheWAS on Google Cloud with dsub using phetk

Source: https://context7.com/nhgritctran/phetk/llms.txt

Execute PheWAS analysis on Google Cloud Platform using dsub for distributed computing with large datasets. This involves initializing the PheWAS object with cloud storage paths and then running the analysis via dsub, specifying docker image, job name, machine type, region, and other cloud-specific parameters. It also includes methods for monitoring job status, viewing logs, and canceling jobs.

```python
from phetk.phewas import PheWAS

# Initialize PheWAS with cloud storage paths
phewas = PheWAS(
    phecode_version="X",
    phecode_count_file_path="gs://your-bucket/phecode_counts.tsv",
    cohort_file_path="gs://your-bucket/cohort.tsv",
    covariate_cols=["age", "sex", "pc1", "pc2", "pc3"],
    independent_variable_of_interest="genotype",
    sex_at_birth_col="sex",
    min_cases=50,
    method="logit",
    output_file_path="gs://your-bucket/phewas_results.tsv"
)

# Run with dsub on Google Cloud
phewas.run_dsub(
    docker_image="phetk/phetk:latest",
    job_name="my-phewas-analysis",
    machine_type="c2d-highcpu-4",  # Use c2d-standard-4 for Cox regression
    region="us-central1",
    preemptible=True,  # Cost savings with preemptible instances
    disk_size=256,
    boot_disk_size=50
)

# Monitor job status
phewas.dsub.check_status()

# Stream job status updates
phewas.dsub.check_status(mode="stream", update_interval=30)

# View logs
phewas.dsub.view_log(log_type="stdout", n_lines=50)
phewas.dsub.view_log(log_type="stderr", n_lines=20)

# Cancel job if needed
phewas.dsub.kill()
```

--------------------------------

### Initialize Plot Object with Custom Color Palettes

Source: https://github.com/nhgritctran/phetk/blob/main/docs/plot-module.md

Demonstrates initializing the Plot object with different color palettes for visualizations. Supports default, colorblind-friendly, rainbow, and custom tuple-based color schemes. Used for customizing the visual appearance of plots.

```python
# Default palette
p = Plot("results.tsv")

# Colorblind-friendly palette
p = Plot("results.tsv", color_palette="colorblind")

# Rainbow palette
p = Plot("results.tsv", color_palette="rainbow")

# Custom colors
p = Plot("results.tsv", color_palette=("blue", "red", "green"))
```

--------------------------------

### Configure Custom Cohort (Python)

Source: https://github.com/nhgritctran/phetk/blob/main/docs/cohort-module.md

Initializes a Cohort object for custom platforms, such as those using Google BigQuery. This configuration requires specifying the platform as 'custom' and providing the Google BigQuery dataset ID.

```python
cohort = Cohort(platform="custom", gbq_dataset_id="your_dataset_id")
```

--------------------------------

### Variant Quality Control Metrics

Source: https://github.com/nhgritctran/phetk/blob/main/docs/tutorials/v0.2.1 APOE PheWAS - CLI.ipynb

This output displays detailed quality control metrics for a specific genetic variant (chr19:44908684). It includes information on alleles, filters, split status, genotype quality (GQ) statistics, call rate, number of called/not called/filtered variants, heterozygote frequency, and p-values for Hardy-Weinberg equilibrium and excess heterozygotes.

```text
Locus chr19:44908684 found!
+----------------+------------+----------+---------+-----------+
| locus          | alleles    | filters  | a_index | was_split |
+----------------+------------+----------+---------+-----------+
| locus<GRCh38>  | array<str> | set<str> |   int32 |      bool |
+----------------+------------+----------+---------+-----------+
| chr19:44908684 | ["T","C"]  | {}       |       1 |     False |
+----------------+------------+----------+---------+-----------+

+--------------------------+---------------------------+
| variant_qc.gq_stats.mean | variant_qc.gq_stats.stdev |
+--------------------------+---------------------------+
|                  float64 |                   float64 |
+--------------------------+---------------------------+
|                 4.30e+01 |                  8.93e+00 |
+--------------------------+---------------------------+

+-------------------------+-------------------------+----------------------+
| variant_qc.gq_stats.min | variant_qc.gq_stats.max | variant_qc.call_rate |
+-------------------------+-------------------------+----------------------+
|                 float64 |                 float64 |              float64 |
+-------------------------+-------------------------+----------------------+
|                6.00e+00 |                9.90e+01 |             1.00e+00 |
+-------------------------+-------------------------+----------------------+

+---------------------+-------------------------+-----------------------+
| variant_qc.n_called | variant_qc.n_not_called | variant_qc.n_filtered |
+---------------------+-------------------------+-----------------------+
|               int64 |                   int64 |                 int64 |
+---------------------+-------------------------+-----------------------+
|              414715 |                       0 |                   115 |
+---------------------+-------------------------+-----------------------+

+------------------+----------------------+-------------------------+
| variant_qc.n_het | variant_qc.n_non_ref | variant_qc.het_freq_hwe |
+------------------+----------------------+-------------------------+
|            int64 |                int64 |                 float64 |
+------------------+----------------------+-------------------------+
|           103189 |               112726 |                2.51e-01 |
+------------------+----------------------+-------------------------+

+------------------------+-------------------------------+--------------+
| variant_qc.p_value_hwe | variant_qc.p_value_excess_het | info.AC      |
+------------------------+-------------------------------+--------------+
|                float64 |                       float64 | array<int32> |

```

--------------------------------

### Import PheTK and Supporting Libraries

Source: https://github.com/nhgritctran/phetk/blob/main/docs/tutorials/v0.2.1 APOE PheWAS - dsub.ipynb

Imports necessary classes from the PheTK library (Cohort, Phecode, PheWAS, Plot) and the os module for environment variable access. These are fundamental for setting up and running analyses.

--------------------------------

### Calculate Age at First Phecode Event (CLI)

Source: https://github.com/nhgritctran/phetk/blob/main/docs/phecode-module.md

Command-line interface for calculating the age at the first phecode event for each participant. This adds an 'age_at_first_event' column to the phecode counts file. It requires the path to the phecode counts file and can optionally specify an output file path.

```bash
phetk phecode add-age-at-first-event \
  --phecode_count_file_path "phecode_counts.tsv" \
  --output_file_path "phecode_counts_with_age.tsv"
```

--------------------------------

### File Path Requirements for dsub

Source: https://github.com/nhgritctran/phetk/blob/main/docs/dsub-considerations.md

Demonstrates the correct and incorrect ways to specify file paths when using Google Cloud Storage with dsub. All input and output file paths must be GCS URLs (gs://). Local paths will not work.

```python
# Correct
phecode_count_file_path="gs://your-bucket/phecode_counts.tsv"
cohort_file_path="gs://your-bucket/cohort.tsv"
output_file_path="gs://your-bucket/results/phewas_results.tsv"

# Incorrect - local paths won't work
phecode_count_file_path="./local_file.tsv"

```

--------------------------------

### Run PheWAS Analysis with dsub

Source: https://github.com/nhgritctran/phetk/blob/main/docs/phewas-module.md

This Python code demonstrates how to instantiate a PheWAS object and run the analysis using dsub for distributed computing. It requires specifying Phecode versions, file paths for counts and cohorts, covariate columns, and output paths. The `run_dsub` method handles the execution on a cloud platform with specified machine types and regions.

```python
from phetk.phewas import PheWAS

# Create PheWAS instance
phewas = PheWAS(
    phecode_version="X",
    phecode_count_file_path="gs://your-bucket/phecode_counts.tsv",
    cohort_file_path="gs://your-bucket/cohort.tsv",
    covariate_cols=["age", "sex", "pc1", "pc2", "pc3"],
    independent_variable_of_interest="genotype",
    sex_at_birth_col="sex",
    min_cases=50,
    min_phecode_count=2,
    method="logit",
    output_file_path="gs://your-bucket/phewas_results.tsv"
)

# Run with dsub
phewas.run_dsub(
    docker_image="phetk/phetk:latest",
    job_name="my-phewas-job",
    machine_type="c2d-standard-4",
    region="us-central1",
    preemptible=True
)
```

--------------------------------

### Instantiate Plot Class

Source: https://github.com/nhgritctran/phetk/blob/main/docs/tutorials/v0.2.1 APOE PheWAS - dsub.ipynb

Creates an instance of the `Plot` class, initializing it with the path to the PheWAS results file. This object will be used for generating various plots.

```python
# instantiate class Plot with PheWAS results as input
p = Plot(f"{bucket}/test/results/dsub__rs429358_phewas_results.tsv")
```

--------------------------------

### Add Age at First Phecode Event using Python

Source: https://context7.com/nhgritctran/phetk/llms.txt

Calculates the age of participants at their first occurrence of each phecode event. This is useful for analyzing disease onset patterns and for age-adjusted statistical analyses. It requires a phecode count file as input and outputs a new file with an added 'age_at_first_event' column.

```python
from phetk.phecode import Phecode

phecode = Phecode(platform="aou")

# Calculate age at first phecode event
phecode.add_age_at_first_event(
    phecode_count_file_path="phecode_counts.tsv",
    output_file_path="phecode_counts_with_age.tsv"
)
```

--------------------------------

### Configure All of Us Cohort (Python)

Source: https://github.com/nhgritctran/phetk/blob/main/docs/cohort-module.md

Initializes a Cohort object for the 'All of Us' research program. This configuration specifies the platform as 'aou' and allows for setting the All of Us database version (CDR version) and an optional OMOP CDR string.

```python
cohort = Cohort(platform="aou", aou_db_version=8)
```

--------------------------------

### Initialize Plot Object with Filtering Options

Source: https://github.com/nhgritctran/phetk/blob/main/docs/plot-module.md

Shows how to initialize the Plot object with filtering options, specifically for `converged_only` results and custom Bonferroni thresholds. This allows users to control which data points are included in the analysis and adjust significance thresholds.

```python
# Only converged results (default)
p = Plot("results.tsv", converged_only=True)

# Include non-converged
p = Plot("results.tsv", converged_only=False)

# Custom Bonferroni threshold
p = Plot("results.tsv", bonferroni=1e-7)
```

--------------------------------

### View dsub Job Logs

Source: https://github.com/nhgritctran/phetk/blob/main/docs/tutorials/v0.2.1 APOE PheWAS - dsub.ipynb

Retrieves and displays the standard output (stdout) or standard error (stderr) logs for a dsub job. This is useful for debugging and understanding the job's execution details.

```python
# Users can check the output of the run as below:
phewas.dsub.view_log(log_type="stdout", n_lines=100)
```

```python
# If there was any error, the log below should contain some error details:
phewas.dsub.view_log(log_type="stderr", n_lines=100)
```

--------------------------------

### Run PheWAS Analysis using Python

Source: https://github.com/nhgritctran/phetk/blob/main/docs/tutorials/v0.2.1 APOE PheWAS - Logistic Regression.ipynb

Initiates a PheWAS analysis using generated cohort and phecode data. This method requires the PheWAS class and allows for adjustment of covariates, setting minimum case/control thresholds, and specifying the analysis method. It outputs summary statistics for each phecode.

```python
%%time
# instantiate class PheWAS and provide information for the PheWAS run
phewas = PheWAS(
    phecode_version="X",
    phecode_count_file_path="aou_phecode_counts.tsv",
    cohort_file_path="rs429358_cohort_with_covariates.tsv",
    sex_at_birth_col="sex_at_birth",
    male_as_one=True,
    covariate_cols=[
        "age_at_last_ehr_event", "sex_at_birth",
        "pc1", "pc2", "pc3", "pc4", "pc5"
    ],
    independent_variable_of_interest="genotype",
    min_cases=50,
    min_phecode_count=2,
    method="logit",
    output_file_path="rs429358_phewas_results.tsv"
)
```

--------------------------------

### Managing dsub Job Logs

Source: https://github.com/nhgritctran/phetk/blob/main/docs/dsub-considerations.md

Illustrates how to view logs for dsub jobs using the PheWAS utility. Users can retrieve standard output, standard error, or all logs, specifying the number of lines to display.

```python
# View recent stdout logs
phewas.view_log(log_type="stdout", n_lines=50)

# View error logs
phewas.view_log(log_type="stderr", n_lines=20)

# View all logs
phewas.view_log(log_type="all")

```

--------------------------------

### Run PheWAS Analysis with Logistic Regression (CLI)

Source: https://github.com/nhgritctran/phetk/blob/main/docs/phewas-module.md

Executes a phenome-wide association study using logistic regression via the command line. Requires specifying phecode version, file paths for counts and cohort data, covariate columns, the independent variable of interest, and sex column. Outputs results to a specified file.

```bash
phetk phewas \
  --phecode_version "X" \
  --cohort_file_path "cohort.tsv" \
  --phecode_count_file_path "phecode_counts.tsv" \
  --sex_at_birth_col "sex" \
  --covariate_cols age sex pc1 pc2 pc3 \
  --independent_variable_of_interest "genotype" \
  --min_cases 50 \
  --min_phecode_count 2 \
  --output_file_path "phewas_results.tsv"
```

--------------------------------

### Generate Cohort by Genotype (CLI)

Source: https://github.com/nhgritctran/phetk/blob/main/docs/cohort-module.md

Command-line interface for generating a genetic cohort based on genotype. This command takes variant information and genotype mappings as arguments to create a cohort file. The `--gt_dict` parameter accepts a JSON string for genotype mapping.

```bash
phetk cohort by-genotype \
  --chromosome_number 7 \
  --genomic_position 117559590 \
  --ref_allele "ATCT" \
  --alt_allele "A" \
  --gt_dict '{"0": "0/0", "1": ["0/1", "1/1"]}' \
  --output_file_path "cftr_cohort.tsv"
```

--------------------------------

### Retrieve Google Cloud Bucket Environment Variable

Source: https://github.com/nhgritctran/phetk/blob/main/docs/tutorials/v0.2.1 APOE PheWAS - dsub.ipynb

Retrieves the Google Cloud Storage bucket name from the WORKSPACE_BUCKET environment variable. This is crucial for specifying data and output locations in cloud-based workflows.

--------------------------------

### Calculate Age at First Phecode Event (Python)

Source: https://github.com/nhgritctran/phetk/blob/main/docs/phecode-module.md

Calculates the age at the first phecode event for each participant and adds an 'age_at_first_event' column to the phecode counts file. This function requires the path to the input phecode counts file and can optionally specify an output file path.

```python
from phetk.phecode import Phecode

phecode = Phecode(platform="aou")
phecode.add_age_at_first_event(
    phecode_count_file_path="phecode_counts.tsv",
    output_file_path="phecode_counts_with_age.tsv"
)
```

--------------------------------

### Import PheTK Modules

Source: https://github.com/nhgritctran/phetk/blob/main/docs/tutorials/v0.2.1 APOE PheWAS - Logistic Regression.ipynb

Imports essential classes from the PheTK library for conducting PheWAS. These include Cohort for data selection, Phecode for phecode mapping, PheWAS for analysis, and Plot for visualization.

```python
from phetk.cohort import Cohort
from phetk.phecode import Phecode
from phetk.phewas import PheWAS
from phetk.plot import Plot
```

--------------------------------

### Enable Phecode Exclusion

Source: https://github.com/nhgritctran/phetk/blob/main/docs/phewas-module.md

This Python snippet demonstrates how to enable phecode exclusion, which is applicable only for Phecode version 1.2 and above. Setting `use_exclusion` to `True` activates this feature.

```python
use_exclusion=True  # Apply phecode exclusion ranges
```

--------------------------------

### Common Customizations: Label Specific Results

Source: https://github.com/nhgritctran/phetk/blob/main/docs/plot-module.md

Illustrates common methods for labeling specific results in plots, such as labeling by p-value, positive beta effects, or a list of specific phecodes. These options enhance the interpretability of the generated visualizations.

```python
# Label top p-values
label_values="p_value"

# Label positive effects
label_values="positive_beta"

# Label specific phecodes
label_values=["185", "250.2", "401.1"]
```

--------------------------------

### Add Age at First Event

Source: https://github.com/nhgritctran/phetk/blob/main/docs/phecode-module.md

Calculates the age at the first phecode event for each participant and adds an 'age_at_first_event' column to the phecode counts.

```APIDOC
## POST /add_age_at_first_event

### Description
Calculate age at first phecode event for each participant. Adds age_at_first_event column to phecode counts.

### Method
POST

### Endpoint
/add_age_at_first_event

### Parameters
#### Query Parameters
- **phecode_count_file_path** (str) - Required - Path to phecode counts TSV file
- **output_file_path** (str) - Optional - Path for output file with age calculations

### Request Example
```python
from phetk.phecode import Phecode

phecode = Phecode(platform="aou")
phecode.add_age_at_first_event(
    phecode_count_file_path="phecode_counts.tsv",
    output_file_path="phecode_counts_with_age.tsv"
)
```

### Response
#### Success Response (200)
- **output_file_path** (str) - Path to the generated TSV file with age at first event.

#### Response Example
```json
{
  "output_file_path": "phecode_counts_with_age.tsv"
}
```
```

--------------------------------

### Create Manhattan Plot with phetk.plot.Plot

Source: https://context7.com/nhgritctran/phetk/llms.txt

Generate publication-ready Manhattan plots visualizing -log10(p-values) across phecode categories. The plots can include effect direction indicators, automatic labeling of top p-values or specific phecodes, and options for filtering by phecode categories. Various customization options for marker size, transparency, legend, and output format (PNG, PDF) are available.

```python
from phetk.plot import Plot

# Initialize plot from PheWAS results
p = Plot(
    phewas_result_file_path="phewas_results.tsv",
    converged_only=True,
    color_palette="default"  # or "colorblind", "rainbow", or custom tuple
)

# Create Manhattan plot with automatic labeling
p.manhattan(
    label_values="p_value",       # Label top p-values
    label_count=10,               # Number of labels
    label_size=8,
    marker_size_by_effect_size=True,
    marker_scale_factor=1.5,
    positive_marker_alpha=0.8,    # Transparency for positive effects
    negative_marker_alpha=0.5,    # Transparency for negative effects
    show_legend=True,
    dpi=300,
    save_plot=True,
    output_file_path="manhattan_plot.png"
)

# Label specific phecodes
p.manhattan(
    label_values=["GE_979.2", "250.2", "401.1"],
    label_count=5,
    phecode_categories=["respiratory", "endocrine/metabolic"],  # Filter categories
    save_plot=True,
    output_file_path="manhattan_filtered.pdf"
)

# Label by effect direction
p.manhattan(
    label_values="positive_beta",  # or "negative_beta"
    label_count=10,
    sort_by_significance=True,
    save_plot=True
)
```

--------------------------------

### Generate Cohort by Genotype using Hail

Source: https://github.com/nhgritctran/phetk/blob/main/docs/tutorials/v0.2.1 APOE PheWAS - Logistic Regression.ipynb

This Python code snippet utilizes the Hail library to generate a cohort based on specific genotype information. It requires chromosome number, genomic position, reference and alternative alleles, a dictionary mapping genotype counts to allele strings, and an output file path. The function processes this input to create a cohort file.

```python
import hail as hl

# Initialize Hail (if not already initialized)
# hl.init()

# Example usage of cohort.by_genotype
# Note: Ensure hl.init() has been called with appropriate configurations before this.
hl.cohort.by_genotype(
    chromosome_number=19,
    genomic_position=44908684,
    ref_allele="T",
    alt_allele="C",
    gt_dict={0: "0/0", 1:["0/1", "1/1"]},
    output_file_path="rs429358_cohort.tsv"
)
```

--------------------------------

### Configure Custom Phecode Mapping

Source: https://github.com/nhgritctran/phetk/blob/main/docs/phewas-module.md

This Python snippet illustrates how to use a custom phecode mapping file. Set `icd_version` to "custom" and provide the `phecode_map_file_path` to your custom mapping file.

```python
icd_version="custom"
phecode_map_file_path="path/to/custom_mapping.tsv"
```

--------------------------------

### Generate Cohort by Genotype (Python)

Source: https://github.com/nhgritctran/phetk/blob/main/docs/cohort-module.md

Generates a genetic cohort based on the genotype of a specific variant. It extracts genotype data from a Hail matrix table and creates a cohort file with person IDs and genotype labels. Key parameters include chromosome, position, alleles, genotype mapping, and reference genome.

```python
from phetk.cohort import Cohort

# Create cohort instance
cohort = Cohort(platform="aou", aou_db_version=8)

# Generate genotype-based cohort
cohort.by_genotype(
    chromosome_number=7,
    genomic_position=117559590,
    ref_allele="ATCT",
    alt_allele="A",
    gt_dict={0: "0/0", 1: ["0/1", "1/1"]},
    output_file_path="cftr_cohort.tsv"
)
```

--------------------------------

### Generate Phecode Counts from ICD Data (CLI)

Source: https://github.com/nhgritctran/phetk/blob/main/docs/phecode-module.md

Command-line interface for generating phecode counts from ICD code data. This command maps ICD codes to phecodes and aggregates counts per person-phecode combination. It requires specifying the platform, phecode version, ICD version, and optionally an output file path.

```bash
phetk phecode count-phecode \
  --platform "aou" \
  --phecode_version "X" \
  --icd_version "US" \
  --output_file_path "phecode_counts.tsv"
```