### Install OpenSpliceAI from Source Source: https://ccb.jhu.edu/openspliceai/content/installation.html Clone the OpenSpliceAI GitHub repository and install it locally using the setup script. This method is useful for development or when needing the latest unreleased code. ```bash git clone https://github.com/Kuanhao-Chao/OpenSpliceAI.git cd OpenSpliceAI python setup.py install ``` -------------------------------- ### Example: Calibrating a Trained Model Source: https://ccb.jhu.edu/openspliceai/content/openspliceai_calibrate.html Demonstrates how to calibrate a pre-trained model using a test dataset. This example specifies the pre-trained model, test dataset, flanking size, output directory, and random seed. ```bash openspliceai calibrate \ --pretrained-model model_best.pt \ --test-dataset dataset_test.h5 \ --flanking-size 10000 \ --output-dir ./calibration_results/ \ --random-seed 42 ``` -------------------------------- ### Initiate Transfer Learning Source: https://ccb.jhu.edu/openspliceai/content/quick_start_guide/quickstart_transfer.html Execute this command to start the transfer learning process. Ensure you have prepared your training and testing datasets and obtained a pre-trained model. ```bash openspliceai transfer \ --train-dataset dataset_train.h5 \ --test-dataset dataset_test.h5 \ --pretrained-model model_10000nt_rs10.pt \ --flanking-size 10000 \ --unfreeze-all \ --epochs 10 \ --early-stopping \ --project-name new_species_transfer \ --output-dir ./transfer_out/ ``` -------------------------------- ### Check OpenSpliceAI Installation Source: https://ccb.jhu.edu/openspliceai/content/installation.html Run this command after installation to verify that OpenSpliceAI is correctly set up and accessible in your environment. It should display usage information and version details. ```bash openspliceai -h ``` -------------------------------- ### Install OpenSpliceAI via pip Source: https://ccb.jhu.edu/openspliceai/content/installation.html Use this command to install OpenSpliceAI and its dependencies directly from PyPI. Pip handles dependency resolution automatically. ```bash pip install openspliceai ``` -------------------------------- ### Transfer Learning Example Source: https://ccb.jhu.edu/openspliceai/content/openspliceai_transfer.html Demonstrates how to fine-tune a pre-trained human model to a new species using specific datasets, model paths, and training configurations. This example specifies parameters like flanking size, unfreezing layers, learning rate scheduler, loss function, epochs, patience, and output directory. ```bash openspliceai transfer \ --train-dataset dataset_train.h5 \ --test-dataset dataset_test.h5 \ --pretrained-model OpenSpliceAI-MANE-best.pt \ --flanking-size 400 \ --unfreeze 2 \ --scheduler CosineAnnealingWarmRestarts \ --loss focal_loss \ --epochs 15 \ --patience 3 \ --early-stopping \ --project-name new_species_transfer \ --output-dir ./transfer_outputs/ ``` -------------------------------- ### Install OpenSpliceAI via Conda Source: https://ccb.jhu.edu/openspliceai/content/installation.html Install OpenSpliceAI and its dependencies in a sandboxed environment using Conda. This is the recommended method for managing complex dependencies. ```bash conda install -c bioconda openspliceai ``` -------------------------------- ### Install PyTorch (CPU/GPU) via pip Source: https://ccb.jhu.edu/openspliceai/content/installation.html Commands to install PyTorch, either the CPU-only version or the version with GPU support, using pip. These are necessary if you are not using Conda for PyTorch installation. ```bash # CPU-only version (Conda packages are no longer available): pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu # For GPU support (Conda packages are no longer available): pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu126 ``` -------------------------------- ### Calibrate OpenSpliceAI Model Source: https://ccb.jhu.edu/openspliceai/content/quick_start_guide/quickstart_calibrate.html Run this command to start the calibration process. Ensure you have a pre-trained model and a test dataset ready. The output directory will store calibration results and the optimized temperature parameter. ```bash openspliceai calibrate \ --pretrained-model model_10000nt_rs10.pt \ --test-dataset dataset_test.h5 \ --flanking-size 10000 \ --output-dir ./calibration_results/ ``` -------------------------------- ### Install mappy via pip Source: https://ccb.jhu.edu/openspliceai/content/installation.html Install the mappy package, which provides Python bindings for minimap2, using pip. This is an alternative to Conda installation. ```bash pip install mappy ``` -------------------------------- ### GPL Notice for Interactive Programs Source: https://ccb.jhu.edu/openspliceai/content/license.html Include this short notice in your program's output when it starts in interactive mode. It informs users about the program's free software status and warranty. ```text > Copyright (C) > This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. This is free software, and you are welcome to redistribute it under certain conditions; type `show c' for details. ``` -------------------------------- ### OpenSpliceAI Terminal Output Source: https://ccb.jhu.edu/openspliceai/content/installation.html This output shows the successful installation and basic usage of the OpenSpliceAI toolkit, including its version and available subcommands for various deep learning tasks. ```bash ==================================================================== Deep learning framework to train your own SpliceAI model ==================================================================== ██████╗ ██████╗ ███████╗███╗ ██╗███████╗██████╗ ██╗ ██╗ ██████╗███████╗ █████╗ ██╗ ██╔═══██╗██╔══██╗██╔════╝████╗ ██║██╔════╝██╔══██╗██║ ██║██╔════╝██╔════╝██╔══██╗██║ ██║ ██║██████╔╝█████╗ ██╔██╗ ██║███████╗██████╔╝██║ ██║██║ █████╗ ███████║██║ ██║ ██║██╔═══╝ ██╔══╝ ██║╚██╗██║╚════██║██╔═══╝ ██║ ██║██║ ██╔══╝ ██╔══██║██║ ╚██████╔╝██║ ███████╗██║ ╚████║███████║██║ ███████╗██║╚██████╗███████╗██║ ██║██║ ╚═════╝ ╚═╝ ╚══════╝╚═╝ ╚═══╝╚══════╝╚═╝ ╚══════╝╚═╝ ╚═════╝╚══════╝╚═╝ ╚═╝╚═╝ 0.0.1 usage: openspliceai [-h] {create-data,train,test,calibrate,transfer,predict,variant} ... OpenSpliceAI toolkit to help you retrain your own splice site predictor positional arguments: {create-data,train,test,calibrate,predict,transfer,variant} Subcommands: create-data, train, test, calibrate, predict, transfer, variant create-data Create dataset for your genome for SpliceAI model training train Train the SpliceAI model test Test the SpliceAI model calibrate Calibrate the SpliceAI model transfer transfer a pre-trained SpliceAI model on new data. predict Predict splice sites in a given sequence using the SpliceAI model variant Label genetic variations with their predicted effects on splicing. optional arguments: -h, --help show this help message and exit ``` -------------------------------- ### Example: Annotating All Variants in VCF Source: https://ccb.jhu.edu/openspliceai/content/openspliceai_variant.html Annotates all variants in a VCF file using Keras models, rounding scores to three decimal places. This command includes all scores by default. ```bash openspliceai variant \ --input-vcf sample_variants.vcf \ --ref-genome GRCh37.fa \ --annotation grch37.txt \ --model /path/to/keras_models/ \ --model-type keras \ --flanking-size 10000 \ --output-vcf annotated_samples.vcf --precision 3 ``` -------------------------------- ### Applying GPL to New Programs Source: https://ccb.jhu.edu/openspliceai/content/license.html Use this template to attach notices to your program, ensuring it is free software under the GPL. Include copyright information and a pointer to the full license. ```text > > Copyright (C) > This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. > This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. > You should have received a copy of the GNU General Public License along with this program. If not, see . ``` -------------------------------- ### Install mappy via Conda Source: https://ccb.jhu.edu/openspliceai/content/installation.html Install the mappy package, which provides Python bindings for minimap2, using Conda. This is useful for genomic alignment tasks. ```bash conda install -c bioconda mappy ``` -------------------------------- ### Create Mouse Training and Test Datasets Source: https://ccb.jhu.edu/openspliceai/content/train_your_own_model/train_mouse.html Use the `create-data` subcommand with mouse-specific genome and annotation files to generate HDF5 datasets for training and testing. Key options include filtering paralogs, setting identity and coverage thresholds, and using canonical transcripts. ```bash openspliceai create-data \ --genome-fasta mouse_genome.fna \ --annotation-gff mouse_annotation.gff \ --output-dir train_test_dataset_mouse/ \ --remove-paralogs \ --min-identity 0.8 \ --min-coverage 0.8 \ --parse-type canonical \ --write-fasta \ --split-method mouse \ --canonical-only ``` -------------------------------- ### Example OpenSpliceAI VCF Annotation Source: https://ccb.jhu.edu/openspliceai/content/openspliceai_variant.html An example of a VCF INFO field annotation generated by OpenSpliceAI. It details the alternate allele, gene symbol, delta scores for acceptor and donor gain/loss, and their corresponding positions. ```text A|MYGENE|0.27|0.00|0.09|0.02|3|-4|7|-2 ``` -------------------------------- ### OpenSpliceAI create-data Usage Source: https://ccb.jhu.edu/openspliceai/content/openspliceai_create-data.html This command outlines the available arguments for the `create-data` subcommand, including input file paths, output directory, and various processing options for generating training and testing datasets. ```bash usage: openspliceai create-data [-h] --annotation-gff ANNOTATION_GFF --genome-fasta GENOME_FASTA --output-dir OUTPUT_DIR [--parse-type {canonical,all_isoforms}] [--biotype {protein-coding,non-coding}] [--chr-split {train-test,test}] [--split-method {random,human}] [--split-ratio SPLIT_RATIO] [--canonical-only] [--flanking-size FLANKING_SIZE] [--verify-h5] [--remove-paralogs] [--min-identity MIN_IDENTITY] [--min-coverage MIN_COVERAGE] [--write-fasta] optional arguments: -h, --help show this help message and exit --annotation-gff ANNOTATION_GFF Path to the GFF file --genome-fasta GENOME_FASTA Path to the FASTA file --output-dir OUTPUT_DIR Output directory to save the data --parse-type {canonical,all_isoforms} Type of transcript processing --biotype {protein-coding,non-coding} Biotype of transcript processing --chr-split {train-test,test} Whether to obtain testing or both training and testing groups --split-method {random,human} Chromosome split method for training and testing dataset --split-ratio SPLIT_RATIO Ratio of training and testing dataset --canonical-only Flag to obtain only canonical splice site pairs --flanking-size FLANKING_SIZE Sum of flanking sequence lengths on each side of input (i.e. 40+40) --verify-h5 Verify the generated HDF5 file(s) --remove-paralogs Remove paralogous sequences between training and testing dataset --min-identity MIN_IDENTITY Minimum minimap2 alignment identity for paralog removal between training and testing dataset --min-coverage MIN_COVERAGE Minimum minimap2 alignment coverage for paralog removal between training and testing dataset --write-fasta Flag to write out sequences into fasta files ``` -------------------------------- ### Create Training and Testing HDF5 Datasets Source: https://ccb.jhu.edu/openspliceai/content/quick_start_guide/quickstart_create-data.html Use this command to generate HDF5 files for training and testing. Ensure you have the reference genome (FASTA) and annotation (GFF/GTF) files. This command includes options for removing paralogs, setting minimum identity and coverage thresholds, and specifying the split method. ```bash openspliceai create-data \ --remove-paralogs \ --min-identity 0.8 \ --min-coverage 0.8 \ --parse-type canonical \ --split-method human\ --canonical-only \ --genome-fasta GCF_000001405.40_GRCh38.p14_genomic_10_sample.fna \ --annotation-gff MANE.GRCh38.v1.3.refseq_genomic_10_sample.gff \ --output-dir train_test_dataset/ ``` -------------------------------- ### OpenSpliceAI Calibrate Usage Source: https://ccb.jhu.edu/openspliceai/content/openspliceai_calibrate.html Displays the help message and available arguments for the openspliceai calibrate subcommand. Use this to understand all configuration options. ```bash usage: openspliceai calibrate [-h] [--epochs EPOCHS] [--early-stopping] [--patience PATIENCE] --output-dir OUTPUT_DIR --project-name PROJECT_NAME [--exp-num EXP_NUM] [--flanking-size {80,400,2000,10000}] [--random-seed RANDOM_SEED] [--temperature-file TEMPERATURE_FILE] --pretrained-model PRETRAINED_MODEL --train-dataset TRAIN_DATASET --test-dataset TEST_DATASET [--loss {cross_entropy_loss,focal_loss}] optional arguments: -h, --help show this help message and exit --epochs EPOCHS, -n EPOCHS Number of epochs for training --early-stopping, -E Enable early stopping --patience PATIENCE, -P PATIENCE Number of epochs to wait before early stopping --output-dir OUTPUT_DIR, -o OUTPUT_DIR Output directory for model checkpoints and logs --project-name PROJECT_NAME, -p PROJECT_NAME Project name for the fine-tuning experiment --exp-num EXP_NUM, -e EXP_NUM Experiment number --flanking-size {80,400,2000,10000}, -f {80,400,2000,10000} Flanking sequence size --random-seed RANDOM_SEED, -r RANDOM_SEED Random seed for reproducibility --temperature-file TEMPERATURE_FILE, -T TEMPERATURE_FILE Path to the temperature file --pretrained-model PRETRAINED_MODEL, -m PRETRAINED_MODEL Path to the pre-trained model --train-dataset TRAIN_DATASET, -train TRAIN_DATASET Path to the training dataset --test-dataset TEST_DATASET, -test TEST_DATASET Path to the testing dataset --loss {cross_entropy_loss,focal_loss}, -l {cross_entropy_loss,focal_loss} Loss function for fine-tuning ``` -------------------------------- ### Create Training and Test Datasets with `create-data` Source: https://ccb.jhu.edu/openspliceai/content/train_your_own_model/train_human_mane.html Use the `create-data` subcommand to process genome FASTA and MANE GFF files into HDF5 datasets. This command one-hot encodes sequences and labels splice sites, with options to filter paralogs and use canonical transcripts. ```bash openspliceai create-data \ --genome-fasta GCF_000001405.40_GRCh38.p14_genomic.fna \ --annotation-gff MANE.GRCh38.v1.3.refseq_genomic.gff \ --output-dir train_test_dataset_MANE/ \ --remove-paralogs \ --min-identity 0.8 \ --min-coverage 0.8 \ --parse-type canonical \ --write-fasta \ --split-method human \ --canonical-only ``` -------------------------------- ### Example: Discovering Pathogenic Human Variants Source: https://ccb.jhu.edu/openspliceai/content/openspliceai_variant.html Annotates variants using PyTorch models, specifying a distance of 100 nucleotides and masking specific scores. This is useful for discovering novel or pathogenic variants. ```bash openspliceai variant \ --input-vcf input_variants.vcf \ --ref-genome GRCh38.fa \ --annotation grch38.txt \ --model /path/to/pytorch_models/ \ --model-type pytorch \ --flanking-size 400 \ --distance 100 \ --mask 1 \ --output-vcf annotated_variants.vcf ``` -------------------------------- ### Train OpenSpliceAI-MANE Model with `train` Source: https://ccb.jhu.edu/openspliceai/content/train_your_own_model/train_human_mane.html Train the OpenSpliceAI model using the prepared datasets. This command specifies the flanking sequence size, dataset paths, output directory, and model configuration, including the SpliceAI architecture and cross-entropy loss. ```bash openspliceai train \ --flanking-size 10000 \ --exp-num full_dataset \ --train-dataset train_test_dataset_MANE/dataset_train.h5 \ --test-dataset train_test_dataset_MANE/dataset_test.h5 \ --output-dir model_train_outdir/ \ --project-name OpenSpliceAI-MANE \ --random-seed 1 \ --model SpliceAI \ --loss cross_entropy_loss ``` -------------------------------- ### Generate Human MANE Dataset Source: https://ccb.jhu.edu/openspliceai/content/openspliceai_create-data.html Use this command to create training and testing datasets for the human MANE dataset. Ensure you have downloaded the reference genome and annotation files beforehand. ```bash openspliceai create-data \ --genome-fasta GCF_000001405.40_GRCh38.p14_genomic.fna \ --annotation-gff MANE.GRCh38.v1.3.refseq_genomic.gff \ --output-dir train_test_dataset_MANE/ ``` -------------------------------- ### OpenSpliceAI Transfer Command Usage Source: https://ccb.jhu.edu/openspliceai/content/openspliceai_transfer.html Displays the command-line arguments and options available for the `openspliceai transfer` subcommand. Use this to understand the parameters for fine-tuning. ```bash usage: openspliceai transfer [-h] [--epochs EPOCHS] [--scheduler {MultiStepLR,CosineAnnealingWarmRestarts}] [--early-stopping] [--patience PATIENCE] --output-dir OUTPUT_DIR --project-name PROJECT_NAME [--exp-num EXP_NUM] [--flanking-size {80,400,2000,10000}] [--random-seed RANDOM_SEED] --pretrained-model PRETRAINED_MODEL --train-dataset TRAIN_DATASET --test-dataset TEST_DATASET [--loss {cross_entropy_loss,focal_loss}] [--unfreeze-all] [--unfreeze UNFREEZE] optional arguments: -h, --help show this help message and exit --epochs EPOCHS, -n EPOCHS Number of epochs for training --scheduler {MultiStepLR,CosineAnnealingWarmRestarts}, -s {MultiStepLR,CosineAnnealingWarmRestarts} Learning rate scheduler --early-stopping, -E Enable early stopping --patience PATIENCE, -P PATIENCE Number of epochs to wait before early stopping --output-dir OUTPUT_DIR, -o OUTPUT_DIR Output directory for model checkpoints and logs --project-name PROJECT_NAME, -p PROJECT_NAME Project name for the fine-tuning experiment --exp-num EXP_NUM, -e EXP_NUM Experiment number --flanking-size {80,400,2000,10000}, -f {80,400,2000,10000} Flanking sequence size --random-seed RANDOM_SEED, -r RANDOM_SEED Random seed for reproducibility --pretrained-model PRETRAINED_MODEL, -m PRETRAINED_MODEL Path to the pre-trained model --train-dataset TRAIN_DATASET, -train TRAIN_DATASET Path to the training dataset --test-dataset TEST_DATASET, -test TEST_DATASET Path to the testing dataset --loss {cross_entropy_loss,focal_loss}, -l {cross_entropy_loss,focal_loss} Loss function for fine-tuning --unfreeze-all, -A Unfreeze all layers for fine-tuning --unfreeze UNFREEZE, -u UNFREEZE Number of layers to unfreeze for fine-tuning ``` -------------------------------- ### OpenSpliceAI Train Command Usage Source: https://ccb.jhu.edu/openspliceai/content/openspliceai_train.html This is the command-line usage for the 'openspliceai train' subcommand. It outlines all available optional arguments for configuring the training process, such as epochs, learning rate scheduler, early stopping, output directory, project name, dataset paths, and loss function. ```bash usage: openspliceai train [-h] [--epochs EPOCHS] [--scheduler {MultiStepLR,CosineAnnealingWarmRestarts}] [--early-stopping] [--patience PATIENCE] --output-dir OUTPUT_DIR --project-name PROJECT_NAME [--exp-num EXP_NUM] [--flanking-size {80,400,2000,10000}] [--random-seed RANDOM_SEED] --train-dataset TRAIN_DATASET --test-dataset TEST_DATASET [--loss {cross_entropy_loss,focal_loss}] [--model MODEL] optional arguments: -h, --help show this help message and exit --epochs EPOCHS, -n EPOCHS Number of epochs for training --scheduler {MultiStepLR,CosineAnnealingWarmRestarts}, -s {MultiStepLR,CosineAnnealingWarmRestarts} Learning rate scheduler --early-stopping, -E Enable early stopping --patience PATIENCE, -P PATIENCE Number of epochs to wait before early stopping --output-dir OUTPUT_DIR, -o OUTPUT_DIR Output directory to save the data --project-name PROJECT_NAME, -p PROJECT_NAME Project name for the train experiment --exp-num EXP_NUM, -e EXP_NUM Experiment number --flanking-size {80,400,2000,10000}, -f {80,400,2000,10000} Flanking sequence size --random-seed RANDOM_SEED, -r RANDOM_SEED Random seed for reproducibility --train-dataset TRAIN_DATASET, -train TRAIN_DATASET Path to the training dataset --test-dataset TEST_DATASET, -test TEST_DATASET Path to the testing dataset --loss {cross_entropy_loss,focal_loss}, -l {cross_entropy_loss,focal_loss} Loss function for training --model MODEL, -m MODEL ``` -------------------------------- ### Calibrate OpenSpliceAI Model Source: https://ccb.jhu.edu/openspliceai/content/train_your_own_model/train_human_mane.html Run this command to calibrate the trained model's output probabilities. It optimizes a temperature parameter using specified training and testing datasets, loading the best model checkpoint from training. Calibrated outputs and diagnostic plots are saved to the specified directory. ```bash openspliceai calibrate \ --flanking-size 10000 \ --train-dataset train_test_dataset_MANE/dataset_train.h5 \ --test-dataset train_test_dataset_MANE/dataset_test.h5 \ --output-dir model_calibrate_outdir/ \ --project-name OpenSpliceAI-MANE-calibrate \ --random-seed 1 \ --pretrained-model model_train_outdir/model_best.pt \ --loss cross_entropy_loss ``` -------------------------------- ### Calibrate Mouse Model Source: https://ccb.jhu.edu/openspliceai/content/train_your_own_model/train_mouse.html Run the calibration step using the `calibrate` subcommand to refine the model’s output probabilities. This command loads the best model checkpoint, optimizes the temperature parameter, and saves calibration outputs. ```bash openspliceai calibrate \ --flanking-size 10000 \ --train-dataset train_test_dataset_mouse/dataset_train.h5 \ --test-dataset train_test_dataset_mouse/dataset_test.h5 \ --output-dir model_calibrate_outdir_mouse/ \ --project-name OpenSpliceAI-Mouse-calibrate \ --random-seed 42 \ --pretrained-model model_train_outdir_mouse/model_best.pt \ --loss cross_entropy_loss ```