### Locatable XSV Configuration Example Source: https://github.com/broadinstitute/gatk/blob/master/docs/funcotator/forum_info/forum_post_tutorial.md Configuration for a locatable XSV file, including source details, type, delimiter, and specific columns for contig, start, and end positions. ```properties name = Oreganno version = 20160119 src_file = oreganno.tsv origin_location = http://www.oreganno.org/dump/ORegAnno_Combined_2016.01.19.tsv preprocessing_script = getOreganno.py isB37DataSource = false type = locatableXSV xsv_delimiter = xsv_permissive_cols = true contig_column = 1 start_column = 2 end_column = 3 ``` -------------------------------- ### Example Funcotator WDL JSON Configuration Source: https://github.com/broadinstitute/gatk/blob/master/scripts/funcotator_wdl/README.md This JSON file provides a template for configuring the Funcotator WDL script. It includes essential parameters for specifying the GATK Docker image, reference genome files, data sources, and input/output variant files. This example is a starting point and may require modification for specific use cases. ```json { "Funcotator.gatk_docker": "broadinstitute/gatk:latest", "Funcotator.ref_fasta": "gs://broad-references/hg38/v0/Homo_sapiens_assembly38.fasta", "Funcotator.ref_fasta_index": "gs://broad-references/hg38/v0/Homo_sapiens_assembly38.fasta.fai", "Funcotator.ref_dict": "gs://broad-references/hg38/v0/Homo_sapiens_assembly38.dict", "Funcotator.reference_version": "hg38", "Funcotator.output_format": "VCF", "Funcotator.compress": "false", "Funcotator.use_gnomad": "false", "Funcotator.data_sources_tar_gz": "gs://gcp-public-data--broad-references/funcotator/funcotator_dataSources.v1.6.20190124s.tar.gz", "Funcotator.variant_vcf_to_funcotate": "variants.vcf", "Funcotator.variant_vcf_to_funcotate_index": "variants.vcf.idx", "Funcotator.output_file_base_name": "variants.funcotated" } ``` -------------------------------- ### Cloud Data Source Configuration Example Source: https://github.com/broadinstitute/gatk/blob/master/docs/funcotator/forum_info/forum_post_tutorial.md Example of configuring a Funcotator data source where the source file is located on the cloud, specified by a URL in the 'src_file' property. ```properties ... src_file = gs://broad-references/hg19/v0/1000G_phase1.snps.high_confidence.b37.vcf.gz ... ``` -------------------------------- ### Simple XSV Configuration File Example Source: https://github.com/broadinstitute/gatk/blob/master/docs/funcotator/forum_info/forum_post_tutorial.md Example of a configuration file for a simple XSV data source. This file defines metadata such as name, version, source file, and data type. ```properties name = Familial_Cancer_Genes version = 20110905 src_file = Familial_Cancer_Genes.no_dupes.tsv origin_location = oncotator_v1_ds_April052016.tar.gz preprocessing_script = UNKNOWN # Whether this data source is for the b37 reference. # Required and defaults to false. isB37DataSource = false # Supported types: # simpleXSV -- Arbitrary separated value table (e.g. CSV), keyed off Gene Name OR Transcript ID # locatableXSV -- Arbitrary separated value table (e.g. CSV), keyed off a genome location # gencode -- Custom datasource class for GENCODE # cosmic -- Custom datasource class for COSMIC # vcf -- Custom datasource class for Variant Call Format (VCF) files type = simpleXSV # Required field for GENCODE files. # Path to the FASTA file from which to load the sequences for GENCODE transcripts: gencode_fasta_path = # Required field for GENCODE files. # NCBI build version (either hg19 or hg38): ncbi_build_version = ``` -------------------------------- ### Install PAT Dependencies Source: https://github.com/broadinstitute/gatk/wiki/Running-Intel-PAT-on-Dataproc Install necessary dependencies for Intel PAT on each machine in the cluster. These commands should be run after SSHing into each node. ```bash $ apt-get install perf-tools-unstable ``` ```bash $ apt-get install gawk ``` ```bash $ apt-get install sysstat ``` -------------------------------- ### Install Intel PAT Source: https://github.com/broadinstitute/gatk/wiki/Running-Intel-PAT-on-Dataproc Execute this command on the master node after configuring the PAT config file to install PAT across the cluster. ```bash $ ./pat install ``` -------------------------------- ### Install Temurin JDK 17 with Homebrew Source: https://github.com/broadinstitute/gatk/wiki/Switching-to-Java-17 Use Homebrew to tap the cask versions and install Temurin JDK 17. This is the recommended JDK for GATK. ```bash brew tap homebrew/cask-versions ``` ```bash brew install --cask temurin17 ``` -------------------------------- ### Simple XSV Configuration Example Source: https://github.com/broadinstitute/gatk/blob/master/docs/funcotator/forum_info/forum_post_tutorial.md Configuration for a simple XSV file, specifying the key, key column, delimiter, and permissive column matching. ```properties xsv_key = GENE_NAME xsv_key_column = 2 xsv_delimiter = xsv_permissive_cols = true ``` -------------------------------- ### Run a Standard GATK4 Tool Source: https://github.com/broadinstitute/gatk/blob/master/README.md Example of running a non-Spark GATK4 tool, `PrintReads`, with input and output BAM files specified. ```bash ./gatk PrintReads -I input.bam -O output.bam ``` -------------------------------- ### Get GATK Help Source: https://github.com/broadinstitute/gatk/blob/master/README.md Access general help information for running the GATK by invoking the --help flag. ```bash ./gatk --help ``` -------------------------------- ### Run a Spark-based GATK4 Tool Source: https://github.com/broadinstitute/gatk/blob/master/README.md Example of running a Spark-enabled GATK4 tool, `PrintReadsSpark`, with input and output BAM files specified. ```bash ./gatk PrintReadsSpark -I input.bam -O output.bam ``` -------------------------------- ### Install Miniconda Source: https://github.com/broadinstitute/gatk/blob/master/README.md Install Miniconda to a specified location on your PATH. This command assumes you have downloaded the installer script. ```bash bash Miniconda3-py310_23.10.0-1-[YOUR_OS].sh -p /opt/miniconda -b ``` -------------------------------- ### Test GATK4 Docker Image Source: https://github.com/broadinstitute/gatk/wiki/How-to-release-GATK4 Verify your Docker setup by pulling the latest GATK4 image and running it interactively. This ensures Docker is installed correctly and can access GATK4 images. ```bash docker pull broadinstitute/gatk:latest docker run -it broadinstitute/gatk:latest ``` -------------------------------- ### Install Git LFS Source: https://github.com/broadinstitute/gatk/blob/master/README.md Install Git Large File Storage (LFS) to download large files required for GATK. Run this command after downloading Git LFS. ```bash git lfs install ``` -------------------------------- ### Command Line Program Annotation Example Source: https://github.com/broadinstitute/gatk/wiki/GATK4-Documentation-Generation Example of Javadoc annotations for a GATK command-line program. `@CommandLineProgramProperties` provides metadata, and `@DocumentedFeature` tags the class for inclusion in documentation. ```java @CommandLineProgramProperties( summary = "Prints reads from the input SAM/BAM/CRAM file to the SAM/BAM/CRAM file.", oneLineSummary = "Print reads in the SAM/BAM/CRAM file", programGroup = ReadProgramGroup.class ) @DocumentedFeature public final class PrintReads extends ReadWalker {... ``` -------------------------------- ### Get Help for a Specific GATK4 Tool Source: https://github.com/broadinstitute/gatk/blob/master/README.md Use this command to display detailed help and usage information for any specific GATK4 tool. ```bash ./gatk ToolName --help ``` -------------------------------- ### Reference Context Example Source: https://github.com/broadinstitute/gatk/blob/master/docs/funcotator/forum_info/forum_post_annotation_specifications.md Illustrates the reference coding sequence in a window around a variant, showing strand-correctness. The window size defaults to 10 bases. ```text [REF ALLELE] | v GAACCCACGTCGGTGAGGGCC |________| |________| v v 10 bases 10 bases (window size) (window size) ``` ```text [REF ALLELE] | v CACGAAAGTCGTTGCGGATCT |________| |________| v v 10 bases 10 bases (window size) (window size) ``` -------------------------------- ### Perform a Full GATK4 Build Source: https://github.com/broadinstitute/gatk/blob/master/README.md Run this command to create a complete, standalone GATK distribution zip archive. This includes the launcher, local and Spark jars, and documentation. The resulting archive is found in the `build/` directory. ```bash ./gradlew bundle ``` ```bash ./gradlew ``` -------------------------------- ### Prepare Directory on gsaweb Server Source: https://github.com/broadinstitute/gatk/wiki/Forum-How-to-update-gatk-tools-doc-to-default-to-the-latest-version Navigate to the tool docs directory on the gsaweb server and create a new directory for the specific version of the documentation you are deploying. ```bash cd /local/htdocs/gatk/documentation/tooldocs/ mkdir 4.0.12.0 ``` -------------------------------- ### Dockerfile for GATK Development Environment Source: https://github.com/broadinstitute/gatk/wiki/How-to-set-up-and-use-cloud-dev-environment This Dockerfile sets up a Java 8 JDK environment, installs essential tools like Python, pip, R, GCC, Git, and Google Cloud SDK. It also configures crcmod for gsutil and installs Git LFS. Finally, it updates Google Cloud SDK components and installs the ggplot2 R package. ```dockerfile #Use a jdk, not a JRE because we want to compile things FROM java:8-jdk MAINTAINER DSDE ENV TERM=xterm-256color # Install python2, python3, and R, etc RUN apt-get update && \ apt-get upgrade -y && \ apt-get install -y python && \ apt-get install -y python3-pip && \ apt-get install -y r-base wget curl unzip gcc python-dev python-setuptools emacs git less lynx hdfview #Make sure we're using crcmod in gsutil RUN easy_install -U pip RUN pip install -U crcmod # Install GIT LFS RUN wget https://github.com/github/git-lfs/releases/download/v1.2.0/git-lfs-linux-386-1.2.0.tar.gz \ && tar -zxvf git-lfs-linux-386-1.2.0.tar.gz \ && cd git-lfs-1.2.0 && ./install.sh && cd .. RUN wget https://dl.google.com/dl/cloudsdk/release/google-cloud-sdk.zip \ && unzip google-cloud-sdk.zip \ && rm google-cloud-sdk.zip RUN google-cloud-sdk/install.sh --usage-reporting=true --path-update=true --bash-completion=true --rc-path=/.bashrc --disable-installation-options VOLUME ["/root/.config"] ENV PATH /google-cloud-sdk/bin:$PATH RUN yes | gcloud components update RUN yes | gcloud components update preview WORKDIR /usr/gitc # Install ggplot2 RUN echo 'install.packages(c("ggplot2"), repos="http://cran.us.r-project.org", dependencies=TRUE)' > /tmp/packages.R && Rscript /tmp/packages.R ``` -------------------------------- ### Build GATK4 Docker Image and Push to Docker Hub Source: https://github.com/broadinstitute/gatk/blob/master/scripts/docker/README.md Use this script to create the official GATK4 Docker image and push it to Docker Hub. Requires appropriate access. Allow approximately one hour for completion, including unit tests. ```bash # REPLACE VALUE OF GITHUB_TAG WITH DESIRED VERSION export GITHUB_TAG=1.0.0.0-alpha1.2.1 # REPLACE with the directory where you'd like to clone the repo export STAGING_DIR=~/tmp/tmp_build_docker_image/ sudo bash build_docker.sh -e ${GITHUB_TAG} -p -d ${STAGING_DIR} ``` -------------------------------- ### Oncotator Default Configuration Example Source: https://github.com/broadinstitute/gatk/blob/master/scripts/mutect2_wdl/README.md Provides an example INI-style configuration file for Oncotator, specifying manual annotations and overrides for TCGA MAF generation. This helps prevent '__UNKNOWN__' values. ```ini [manual_annotations] override:NCBI_Build=37,Strand=+,status=Somatic,phase=Phase_I,sequencer=Illumina,Tumor_Validation_Allele1=,Tumor_Validation_Allele2=,Match_Norm_Validation_Allele1=,Match_Norm_Validation_Allele2=,Verification_Status=,Validation_Status=,Validation_Method=,Score=,BAM_file=,Match_Norm_Seq_Allele1=,Match_Norm_Seq_Allele2= ``` -------------------------------- ### Run GATK Tool Documentation Generation Source: https://github.com/broadinstitute/gatk/wiki/GATK4-Documentation-Generation Execute this Gradle task to generate HTML and JSON documentation files for GATK tools. The output is placed in the 'gatkDoc' directory within the build directory. ```bash ./gradlew gatkDoc ``` -------------------------------- ### Fast Build for Local Development Source: https://github.com/broadinstitute/gatk/blob/master/README.md Use this command for a quick build that allows running GATK tools only from within your git clone, locally. It's optimized for developers who need rapid iteration. ```bash ./gradlew installDist ``` -------------------------------- ### Utility Class Annotation Example Source: https://github.com/broadinstitute/gatk/wiki/GATK4-Documentation-Generation Example of Javadoc annotations for a GATK utility class. `@DocumentedFeature` is used here to both tag the class for documentation and provide group information, as it's not a command-line program. ```java @DocumentedFeature( groupName=HelpConstants.DOC_CAT_READFILTERS, groupSummary=HelpConstants.DOC_CAT_READFILTERS_SUMMARY) public final class ReadLengthReadFilter extends ReadFilter implements Serializable{... ``` -------------------------------- ### Gencode Annotation: Insertion Example Source: https://github.com/broadinstitute/gatk/blob/master/docs/funcotator/forum_info/forum_post_annotation_specifications.md Represents an insertion of bases between two positions in a transcript. ```text c.2018_2019insAA ``` -------------------------------- ### Navigate to GATK Directory Source: https://github.com/broadinstitute/gatk/blob/master/docs/funcotator/forum_info/forum_post_tutorial.md Change the current directory to your GATK installation folder. This is a prerequisite for running GATK commands. ```bash cd ~/gatk ``` -------------------------------- ### Gencode Annotation: Single Base Deletion Example Source: https://github.com/broadinstitute/gatk/blob/master/docs/funcotator/forum_info/forum_post_annotation_specifications.md Represents the deletion of a single base at a specific position in a transcript. ```text c2018delT ``` -------------------------------- ### Generate GATK Documentation Locally Source: https://github.com/broadinstitute/gatk/wiki/Forum-How-to-update-gatk-tools-doc-to-default-to-the-latest-version Navigate to the GATK workspace, pull the latest changes, checkout a specific version, and generate the documentation using Gradle. This is a prerequisite for deploying the updated docs. ```bash cd ~/Workspace/gatk/ git pull git checkout _versionnumber_ ./gradlew gatkDoc ``` -------------------------------- ### Build GATK Base Docker Image (Local) Source: https://github.com/broadinstitute/gatk/blob/master/scripts/docker/gatkbase/README.md Builds the GATK base image on your local machine. Provide the new version number as an argument. ```bash build_docker_base_locally.sh 3.0.0rc1 ``` -------------------------------- ### Gencode Annotation: Multiple Base Deletion Example Source: https://github.com/broadinstitute/gatk/blob/master/docs/funcotator/forum_info/forum_post_annotation_specifications.md Represents the deletion of multiple bases within a specified range in a transcript. ```text c2018_2022delTTCAG ``` -------------------------------- ### Configure Gradle Credentials Source: https://github.com/broadinstitute/gatk/wiki/How-to-release-GATK4 Add Sonatype and GPG key credentials to your `~/.gradle/gradle.properties` file. Ensure `signing.keyId`, `signing.password`, `signing.secretKeyRingFile`, `sonatypeUsername`, and `sonatypePassword` are correctly set. ```properties signing.keyId=ID_FOR_YOUR_PUBLIC_KEY signing.password=PASSPHRASE_FOR_YOUR_KEYPAIR signing.secretKeyRingFile=PATH_TO_YOUR_SECRET_KEYRING_FILE sonatypeUsername=YOUR_SONATYPE_TOKEN_USERNAME sonatypePassword=YOUR_SONATYPE_TOKEN_PASSWORD ``` -------------------------------- ### Create BWA-MEM Index Image Source: https://github.com/broadinstitute/gatk/blob/master/docs/mutect/mutect.tex Generate a BWA-MEM index image required by FilterAlignmentArtifacts. This is typically a one-time setup step. ```bash gatk BwaMemIndexCreator -I reference.fasta -O reference.fasta.img ``` -------------------------------- ### Generate Documentation in PHP Format Source: https://github.com/broadinstitute/gatk/wiki/GATK4-Documentation-Generation Add the '-PphpDoc' flag to the Gradle command to generate documentation in PHP format, suitable for PHP-enabled web servers. ```bash ./gradlew gatkDoc -PphpDoc ``` -------------------------------- ### Copy Documentation to gsaweb Server Source: https://github.com/broadinstitute/gatk/wiki/Forum-How-to-update-gatk-tools-doc-to-default-to-the-latest-version Securely copy all generated documentation files from your local machine to the designated directory on the gsaweb server. ```bash scp * bgandham@gsaweb:/local/htdocs/gatk/documentation/tooldocs/4.0.12.0/ ``` -------------------------------- ### Generate PHP Documentation Locally Source: https://github.com/broadinstitute/gatk/wiki/Forum-How-to-update-gatk-tools-doc-to-default-to-the-latest-version Generate PHP documentation for GATK tools using Gradle. The output will be placed in the 'gatkdoc' folder. ```bash ./gradlew -PphpDoc gatkDoc ``` -------------------------------- ### Gencode Annotation: Intronic Variant Example Source: https://github.com/broadinstitute/gatk/blob/master/docs/funcotator/forum_info/forum_post_annotation_specifications.md Represents a variant occurring in an intron, indicating the offset from the exon boundary and the change in bases. ```text c.e81-4TAA>A ``` -------------------------------- ### Build gatkbase Docker Image Locally Source: https://github.com/broadinstitute/gatk/blob/master/scripts/docker/README.md Execute the build script to create the gatkbase Docker image on your local machine. Replace with the desired version tag. ```bash build_docker_base_locally.sh ``` -------------------------------- ### Gencode Annotation: SNP Example Source: https://github.com/broadinstitute/gatk/blob/master/docs/funcotator/forum_info/forum_post_annotation_specifications.md Represents a single nucleotide polymorphism (SNP) where one base is changed to another at a specific position. ```text c.1507T>G ``` -------------------------------- ### Configure Docker Environment for VM Source: https://github.com/broadinstitute/gatk/wiki/How-to-set-up-and-use-cloud-dev-environment After creating a virtual host (e.g., 'gatkbuild'), run this command to configure your current shell session to communicate with the Docker daemon running on that VM. This command needs to be re-run if you open a new shell. ```bash > bash bash-3.2$ eval "$(docker-machine env gatkbuild)" ``` -------------------------------- ### Build GATK4 Release Bundle Source: https://github.com/broadinstitute/gatk/wiki/How-to-release-GATK4 Clean the project, build the release bundle, and create a zip file for distribution. The release flag enables specific build configurations. ```bash ./gradlew clean bundle -Drelease=true ```