### Install cuteSV Source: https://github.com/tjianghit/cutesv/wiki/Diploid-assembly-based-SV-detection-using-cuteSV This command sequence clones the cuteSV repository from GitHub and installs it using Python's setup script. This is a prerequisite for running the SV calling functionalities provided by the project. ```sh git clone https://github.com/tjiangHIT/cuteSV.git cd cuteSV && python3 setup install && cd .. ``` -------------------------------- ### Decompress and Organize BED files for Simulation Source: https://github.com/tjianghit/cutesv/blob/master/simulation/README.md This section details the initial preparation steps, including decompressing gzipped BED files and organizing them into a working directory. It also includes cloning and installing the VISOR tool, which is essential for subsequent simulation steps. Dependencies include gzip and git. ```shell gzip -d sim_del.bed.gz gzip -d sim_ins.bed.gz gzip -d sim_inv.bed.gz gzip -d sim_dup.bed.gz gzip -d sim_tra.bed.gz mkdir workdir && cd workdir mv ../sim_*.bed ../LASeR.bed ./ git clone https://github.com/tjiangHIT/VISOR.git cd VISOR && python3 setup.py install && cd .. ``` -------------------------------- ### Install Benchmark Tools Environment Source: https://github.com/tjianghit/cutesv/blob/master/src/documentation/README.md Sets up a conda environment named 'test_fc' with Python 3.10 and installs Sniffles (v2.0.7), cuteSV (v2.1.0), svjedi (v1.1.6), truvari (v3.5.0), samtools, and tabix. ```sh conda create -n test_fc python=3.10 conda activate test_fc conda install -c bioconda sniffles==2.0.7 cuteSV==2.1.0 svjedi==1.1.6 truvari==3.5.0 samtools tabix ``` -------------------------------- ### Install Sniffles1 Environment Source: https://github.com/tjianghit/cutesv/blob/master/src/documentation/README.md Creates a conda environment named 'sniffles1_env' with Python 3.10 and installs Sniffles version 1.0.12 from the bioconda channel. ```sh conda create -n sniffles1_env python=3.10 conda activate sniffles1_env conda install -c bioconda sniffles==1.0.12 ``` -------------------------------- ### Align Contigs using Minimap2 Source: https://github.com/tjianghit/cutesv/wiki/Diploid-assembly-based-SV-detection-using-cuteSV This command utilizes minimap2 to align the prepared assembly contigs against the reference human genome. It uses specific parameters for assembly-to-assembly (asm5) alignment and generates SAM output files. The installation of minimap2 via conda is also included. ```sh conda install minimap2 -c bioconda minimap2 --paf-no-hit -a -x asm5 --cs -r 2k -t 8 ref/human_hs37d5.fasta assemblies/HG002_wenger.fasta \ > alns/HG002_wenger.sam minimap2 --paf-no-hit -a -x asm5 --cs -r 2k -t 8 ref/human_hs37d5.fasta assemblies/HG002_garg.fasta \ > alns/HG002_garg.sam ``` -------------------------------- ### Process PacBio HiFi Reads and Align Source: https://github.com/tjianghit/cutesv/blob/master/src/documentation/README.md Downloads PacBio HiFi alignment BAM, converts it to FASTA, aligns the FASTA to the reference using pbmm2, and then uses samtools to call variants and index the resulting BAM file. ```sh curl -s https://downloads.pacbcloud.com/public/revio/2022Q4/HG002-rep1/analysis/HG002.m84011_220902_175841_s1.GRCh38.bam > alns/HG002.m84011_220902_175841_s1.GRCh38.bam samtools fasta -@ 16 alns/HG002.m84011_220902_175841_s1.GRCh38.bam > alns/HG002.FASTA pbmm2 align --num-threads 16 --preset CCS --sample HG002 --log-level INFO --sort --unmapped -c 0 -y 70 ref/human_hs37d5.fasta alns/HG002.fasta alns/HG002_origin.bam samtools calmd -b alns/HG002_origin.bam ref/human_hs37d5.fasta > alns/HG002_all.bam samtools index alns/HG002_all.bam ``` -------------------------------- ### Prepare SVJedi Output for Truvari Comparison Source: https://github.com/tjianghit/cutesv/blob/master/src/documentation/README.md Prepares the SVJedi output VCF for comparison by sorting and filtering. It includes steps to extract headers, sort variants by chromosome and position, filter out homozygous reference genotypes, compress the sorted VCF with bgzip, and index it with tabix. ```shell grep '#' tools/svjedi.call.vcf > tools/svjedi.sort.vcf grep -v '#' tools/svjedi.call.vcf | sort -k 1,1 -k 2,2n | awk -F '\t' '{split($10,X,":"); if(X[1]!="0/0"&&X[1]!="./.") print $0}' >> tools/svjedi.sort.vcf bgzip -c tools/svjedi.sort.vcf > tools/svjedi.sort.vcf.gz tabix tools/svjedi.sort.vcf.gz ``` -------------------------------- ### Download HG002 Ground Truth Data Source: https://github.com/tjianghit/cutesv/blob/master/src/documentation/README.md Downloads the HG002 high-confidence variant BED and VCF.gz files from the Genome in a Bottle (GiaB) FTP server. These files serve as the ground truth for benchmarking. ```sh FTPDIR=https://ftp-trace.ncbi.nlm.nih.gov/giab/ftp/data/AshkenazimTrio/analysis curl -s ${FTPDIR}/NIST_SVs_Integration_v0.6/HG002_SVs_Tier1_v0.6.bed > giab/HG002_SVs_Tier1_v0.6.bed curl -s ${FTPDIR}/NIST_SVs_Integration_v0.6/HG002_SVs_Tier1_v0.6.vcf.gz > giab/HG002_SVs_Tier1_v0.6.vcf.gz ``` -------------------------------- ### Prepare Sniffles1 Output for Truvari Source: https://github.com/tjianghit/cutesv/blob/master/src/documentation/README.md Sorts and filters the Sniffles1 output VCF to prepare it for comparison with truvari. It includes header lines, adds a STRANDBIAS filter, and keeps only high-quality variants before compressing and indexing. ```sh grep '#' tools/sniffles1.call.vcf > tools/sniffles1.sort.vcf sed -i 'N;96 a ##FILTER=' tools/sniffles1.sort.vcf grep -v '#' tools/sniffles1.call.vcf | sort -k 1,1 -k 2,2n | awk -F '\t' '{split($10,X,":"); if(X[1]!="0/0"&&X[1]!="./.") print $0}' >> tools/sniffles1.sort.vcf bgzip -c tools/sniffles1.sort.vcf > tools/sniffles1.sort.vcf.gz tabix tools/sniffles1.sort.vcf.gz ``` -------------------------------- ### Prepare cuteSV2 Output for Truvari Source: https://github.com/tjianghit/cutesv/blob/master/src/documentation/README.md Sorts and filters the cuteSV2 output VCF to prepare it for comparison with truvari. It includes header lines, keeps only high-quality variants, and then compresses and indexes the sorted VCF file. ```sh grep '#' tools/cutesv.call.vcf > tools/cutesv.sort.vcf grep -v '#' tools/cutesv.call.vcf | sort -k 1,1 -k 2,2n | awk -F '\t' '{split($10,X,":"); if(X[1]!="0/0"&&X[1]!="./.") print $0}' >> tools/cutesv.sort.vcf bgzip -c tools/cutesv.sort.vcf > tools/cutesv.sort.vcf.gz tabix tools/cutesv.sort.vcf.gz ``` -------------------------------- ### Prepare Reference Genome Source: https://github.com/tjianghit/cutesv/blob/master/src/documentation/README.md Downloads the hg19 reference genome (hs37d5.fa.gz), unzips it, and then modifies the FASTA file to replace all non-ACGT characters with 'N'. This ensures consistency for variant calling. ```sh curl -s https://ftp-trace.ncbi.nih.gov/1000genomes/ftp/technical/reference/phase2_reference_assembly_sequence/hs37d5.fa.gz > ref/human_hs37d5.fasta.gz gunzip ref/human_hs37d5.fasta.gz sed -i '/^[^>]/ y/BDEFHIJKLMNOPQRSUVWXYZbdefhijklmnopqrsuvwxyz/NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN/' ref/human_hs37d5.fasta ``` -------------------------------- ### Create Directory Structure Source: https://github.com/tjianghit/cutesv/wiki/Diploid-assembly-based-SV-detection-using-cuteSV This script creates the necessary directory structure for storing assembly data, references, alignments, and variant calls. It is a foundational step for organizing project files. ```sh mkdir -p assemblies ref alns calls ``` -------------------------------- ### Prepare Sniffles2 Output for Truvari Source: https://github.com/tjianghit/cutesv/blob/master/src/documentation/README.md Formats the Sniffles2 output VCF for truvari comparison. This includes adding specific FORMAT fields (DV, GQ, DR), sorting, filtering for high-quality variants, and then compressing and indexing the results. ```sh grep '#' tools/sniffles2.call.vcf > tools/sniffles2.sort.vcf sed -i 'N;104 a ##FORMAT=' tools/sniffles2.sort.vcf sed -i 'N;104 a ##FORMAT=' tools/sniffles2.sort.vcf sed -i 'N;104 a ##FORMAT=' tools/sniffles2.sort.vcf grep -v '#' tools/sniffles2.call.vcf | sort -k 1,1 -k 2,2n | awk -F '\t' '{split($10,X,":"); if(X[1]!="0/0"&&X[1]!="./.") print $0}' >> tools/sniffles2.sort.vcf bgzip -c tools/sniffles2.sort.vcf > tools/sniffles2.sort.vcf.gz tabix tools/sniffles2.sort.vcf.gz ``` -------------------------------- ### VISOR LASeR: Generate Simulated Alignments Source: https://github.com/tjianghit/cutesv/blob/master/simulation/README.md This series of commands uses the VISOR LASeR module to generate simulated sequencing alignments. It takes a reference genome, a modified donor genome, and a BED file defining insertion points as input. The simulation is performed at varying coverage levels (5x, 10x, 20x, 30x) and outputs data files for each configuration. Key parameters include coverage (-c) and threads (--threads). ```shell # Simulated alignments for deletion variations VISOR LASeR -g reference_genome.fasta -s donor_genome_del -bed LASeR.bed -o data_del_5x -c 5 --threads 16 --noaddtag VISOR LASeR -g reference_genome.fasta -s donor_genome_del -bed LASeR.bed -o data_del_10x -c 10 --threads 16 --noaddtag VISOR LASeR -g reference_genome.fasta -s donor_genome_del -bed LASeR.bed -o data_del_20x -c 20 --threads 16 --noaddtag VISOR LASeR -g reference_genome.fasta -s donor_genome_del -bed LASeR.bed -o data_del_30x -c 30 --threads 16 --noaddtag # Simulated alignments for insertion variations VISOR LASeR -g reference_genome.fasta -s donor_genome_ins -bed LASeR.bed -o data_ins_5x -c 5 --threads 16 --noaddtag VISOR LASeR -g reference_genome.fasta -s donor_genome_ins -bed LASeR.bed -o data_ins_10x -c 10 --threads 16 --noaddtag VISOR LASeR -g reference_genome.fasta -s donor_genome_ins -bed LASeR.bed -o data_ins_20x -c 20 --threads 16 --noaddtag VISOR LASeR -g reference_genome.fasta -s donor_genome_ins -bed LASeR.bed -o data_ins_30x -c 30 --threads 16 --noaddtag # Simulated alignments for inversion variations VISOR LASeR -g reference_genome.fasta -s donor_genome_inv -bed LASeR.bed -o data_inv_5x -c 5 --threads 16 --noaddtag VISOR LASeR -g reference_genome.fasta -s donor_genome_inv -bed LASeR.bed -o data_inv_10x -c 10 --threads 16 --noaddtag VISOR LASeR -g reference_genome.fasta -s donor_genome_inv -bed LASeR.bed -o data_inv_20x -c 20 --threads 16 --noaddtag VISOR LASeR -g reference_genome.fasta -s donor_genome_inv -bed LASeR.bed -o data_inv_30x -c 30 --threads 16 --noaddtag # Simulated alignments for duplication variations VISOR LASeR -g reference_genome.fasta -s donor_genome_dup -bed LASeR.bed -o data_dup_5x -c 5 --threads 16 --noaddtag VISOR LASeR -g reference_genome.fasta -s donor_genome_dup -bed LASeR.bed -o data_dup_10x -c 10 --threads 16 --noaddtag VISOR LASeR -g reference_genome.fasta -s donor_genome_dup -bed LASeR.bed -o data_dup_20x -c 20 --threads 16 --noaddtag VISOR LASeR -g reference_genome.fasta -s donor_genome_dup -bed LASeR.bed -o data_dup_30x -c 30 --threads 16 --noaddtag # Simulated alignments for translocation variations VISOR LASeR -g reference_genome.fasta -s donor_genome_tra -bed LASeR.bed -o data_tra_5x -c 5 --threads 16 --noaddtag VISOR LASeR -g reference_genome.fasta -s donor_genome_tra -bed LASeR.bed -o data_tra_10x -c 10 --threads 16 --noaddtag VISOR LASeR -g reference_genome.fasta -s donor_genome_tra -bed LASeR.bed -o data_tra_20x -c 20 --threads 16 --noaddtag VISOR LASeR -g reference_genome.fasta -s donor_genome_tra -bed LASeR.bed -o data_tra_30x -c 30 --threads 16 --noaddtag ``` -------------------------------- ### Run SVJedi for SV Calling Source: https://github.com/tjianghit/cutesv/blob/master/src/documentation/README.md Executes the SVJedi tool (v1.1.6) to call structural variants from a VCF file, using a reference genome and an alignment input. Outputs the results to a VCF file. ```shell python3 svjedi.py -v giab/HG002_SVs_Tier1_v0.6.vcf -r ref/human_hs37d5.fasta -i alns/HG002.fasta -o tools/svjedi.call.vcf ``` -------------------------------- ### Run Sniffles1 (v1.0.12) Source: https://github.com/tjianghit/cutesv/blob/master/src/documentation/README.md Executes Sniffles version 1.0.12 using the aligned BAM file and the filtered ground truth VCF. The output is a VCF file containing the called variants. The environment is activated before running and deactivated afterward. ```sh conda activate sniffles1_env sniffles -m alns/HG002_all.bam -v tools/sniffles1.call.vcf --Ivcf giab/HG002_SVs_Tier1_v0.6.vcf conda deactivate ``` -------------------------------- ### Compare SVs using Truvari Source: https://github.com/tjianghit/cutesv/blob/master/src/documentation/README.md Compares structural variant calls from multiple tools (sniffles1, sniffles2, cutesv, svjedi) against a NIST ground truth VCF. Uses Truvari to benchmark the results, specifying input VCFs, a BED file for regions of interest, output directory, and various performance parameters. ```shell for i in {sniffles1, sniffles2, cutesv, svjedi} do truvari bench -b giab/HG002_SVs_Tier1_v0.6.filter.vcf.gz -c tools/$i.sort.vcf.gz --includebed giab/HG002_SVs_Tier1_v0.6.bed -o cmp -p 0 -r 2 -P 1 --sizemax 1000000 done ``` -------------------------------- ### Run Sniffles2 (v2.0.7) Source: https://github.com/tjianghit/cutesv/blob/master/src/documentation/README.md Runs Sniffles version 2.0.7 for structural variant calling, using the aligned BAM and the ground truth VCF. The output is saved to 'tools/sniffles2.call.vcf'. ```sh sniffles --input alns/HG002_all.bam --vcf tools/sniffles2.call.vcf --genotype-vcf giab/HG002_SVs_Tier1_v0.6.vcf ``` -------------------------------- ### Run cuteSV for Diploid SV Calling Source: https://github.com/tjianghit/cutesv/wiki/Diploid-assembly-based-SV-detection-using-cuteSV This is the core command for performing diploid-assembly-based SV calling using cuteSV. It takes the sorted BAM alignment file, the reference genome, and an output VCF file as input. Various parameters are set to control the SV calling process, including genotype calling, read ID reporting, and bias/merging thresholds. ```sh cuteSV alns/HG002_wenger.bam ref/human_hs37d5.fasta calls/HG002_wenger.vcf ./ \ -s 1 --genotype --report_readid -p -1 -mi 500 -md 500 \ --max_cluster_bias_INS 1000 \ --diff_ratio_merging_INS 0.9 \ --max_cluster_bias_DEL 1000 \ --diff_ratio_merging_DEL 0.5 python3 cuteSV/src/cuteSV/diploid_calling.py calls/HG002_wenger.vcf calls/HG002_wenger_final.vcf cuteSV alns/HG002_garg.bam ref/human_hs37d5.fasta calls/HG002_garg.vcf ./ \ -s 1 --genotype --report_readid -p -1 -mi 500 -md 500 \ --max_cluster_bias_INS 1000 \ --diff_ratio_merging_INS 0.9 \ --max_cluster_bias_DEL 1000 \ --diff_ratio_merging_DEL 0.5 python3 cuteSV/src/cuteSV/diploid_calling.py calls/HG002_garg.vcf calls/HG002_garg_final.vcf ``` -------------------------------- ### Sort and Index Alignments with Samtools Source: https://github.com/tjianghit/cutesv/wiki/Diploid-assembly-based-SV-detection-using-cuteSV This section demonstrates the use of samtools to sort the generated SAM alignment files into BAM format and then index them. Sorting and indexing are crucial preprocessing steps for many downstream bioinformatics tools, including cuteSV. ```sh conda install samtools -c bioconda samtools sort -@ 8 -o alns/HG002_wenger.bam alns/HG002_wenger.sam samtools index -@ 8 alns/HG002_wenger.bam samtools sort -@ 8 -o alns/HG002_garg.bam alns/HG002_garg.sam samtools index -@ 8 alns/HG002_garg.bam ``` -------------------------------- ### Run cuteSV2 (v2.1.0) Source: https://github.com/tjianghit/cutesv/blob/master/src/documentation/README.md Executes cuteSV version 2.1.0 for structural variant calling. It takes the aligned BAM, reference FASTA, and ground truth VCF as input, specifying various parameters for cluster bias and merging, and outputs results to 'tools/cutesv.call.vcf'. ```sh cuteSV alns/HG002_all.bam ref/human_hs37d5.fasta tools/cutesv.call.vcf ./ --max_cluster_bias_INS 1000 --diff_ratio_merging_INS 0.9 --max_cluster_bias_DEL 1000 --diff_ratio_merging_DEL 0.5 -Ivcf giab/HG002_SVs_Tier1_v0.6.vcf -q 10 -L -1 ``` -------------------------------- ### Download Human Genome Assembly Datasets Source: https://github.com/tjianghit/cutesv/wiki/Diploid-assembly-based-SV-detection-using-cuteSV This set of commands downloads two distinct human genome assembly datasets (NA24385) from different sources (Wenger et al. and Garg et al.). These datasets are crucial for generating haplotype-resolved genome assemblies required for phased SV detection. They are downloaded using curl and saved into the 'assemblies' directory. ```sh echo "Download maternal haplotype from Wenger." curl-s https://downloads.pacbcloud.com/public/publications/2019-HG002-CCS/asm/HG002_canu_maternal.fasta \ > assemblies/HG002_canu_maternal.fasta echo "Download paternal haplotype from Wenger." curl-s https://downloads.pacbcloud.com/public/publications/2019-HG002-CCS/asm/HG002_canu_paternal.fasta \ > assemblies/HG002_canu_paternal.fasta echo "Download haplotype H1 from Garg." curl-s ftp://ftp.dfci.harvard.edu/pub/hli/whdenovo/asm/NA24385-denovo-H1.fa.gz \ > assemblies/NA24385-denovo-H1.fa.gz echo "Download haplotype H2 from Garg." curl-s ftp://ftp.dfci.harvard.edu/pub/hli/whdenovo/asm/NA24385-denovo-H2.fa.gz \ > assemblies/NA24385-denovo-H2.fa.gz ``` -------------------------------- ### Download and Prepare Reference Genome Source: https://github.com/tjianghit/cutesv/wiki/Diploid-assembly-based-SV-detection-using-cuteSV This script downloads the human reference genome (hg19, hs37d5) with decoys, decompresses it, and then modifies non-ACGT characters to 'N' using sed. This preprocessing ensures the reference genome is suitable for downstream alignment tasks. ```sh curl -s ftp://ftp-trace.ncbi.nih.gov/1000genomes/ftp/technical/reference/phase2_reference_assembly_sequence/hs37d5.fa.gz > ref/human_hs37d5.fasta.gz gunzip ref/human_hs37d5.fasta.gz sed -i '/^[^>]/ y/BDEFHIJKLMNOPQRSUVWXYZbdefhijklmnopqrsuvwxyz/NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN/' \ ref/human_hs37d5.fasta ``` -------------------------------- ### Add Haplotype Tags to Assemblies Source: https://github.com/tjianghit/cutesv/wiki/Diploid-assembly-based-SV-detection-using-cuteSV This script concatenates fasta files from different haplotypes (maternal/paternal or H1/H2) and adds unique prefixes ('cutesvh1_' and 'cutesvh2_') to their sequence headers. This tagging is essential for distinguishing between haplotypes during alignment and SV calling. ```sh cat <(cat assemblies/HG002_canu_maternal.fasta | sed 's/>/>cutesvh1_/g') \ <(cat assemblies/HG002_canu_paternal.fasta | sed 's/>/>cutesvh2_/g') \ > assemblies/HG002_wenger.fasta cat <(zcat assemblies/NA24385-denovo-H1.fa.gz | sed 's/>/>cutesvh1_/g') \ <(zcat assemblies/NA24385-denovo-H2.fa.gz | sed 's/>/>cutesvh2_/g') \ > assemblies/HG002_garg.fasta ``` -------------------------------- ### Downsample BAM Alignments using Samtools Source: https://github.com/tjianghit/cutesv/blob/master/src/documentation/README.md Downsamples a BAM alignment file to specific percentages (66%, 33%, 17%) using samtools view. The process involves piping the output of one samtools view command (with subsampling) to another to convert to BAM format. ```shell samtools view -h -s 0.66 alns/HG002_all.bam | samtools view -bS > alns/HG002_20x.bam samtools view -h -s 0.33 alns/HG002_all.bam | samtools view -bS > alns/HG002_10x.bam samtools view -h -s 0.17 alns/HG002_all.bam | samtools view -bS > alns/HG002_5x.bam ``` -------------------------------- ### Filter Target SVs Source: https://github.com/tjianghit/cutesv/blob/master/src/documentation/README.md Filters the HG002 ground truth VCF file to include only high-confidence structural variants. It decompresses the VCF, extracts header lines, filters variants with '0/0' or './.' genotypes, and then re-compresses and indexes the filtered VCF. ```sh gzip -d giab/HG002_SVs_Tier1_v0.6.vcf.gz grep '#' giab/HG002_SVs_Tier1_v0.6.vcf > giab/HG002_SVs_Tier1_v0.6.filter.vcf grep -v '#' giab/HG002_SVs_Tier1_v0.6.vcf | awk -F '\t' '{split($10,X,":"); if(X[1]!="0/0"&&X[1]!="./.") print $0}' >> giab/HG002_SVs_Tier1_v0.6.filter.vcf bgzip -c giab/HG002_SVs_Tier1_v0.6.filter.vcf > giab/HG002_SVs_Tier1_v0.6.filter.vcf.gz tabix giab/HG002_SVs_Tier1_v0.6.filter.vcf.gz ``` -------------------------------- ### VISOR HACk: Genome Modification for Structural Variations Source: https://github.com/tjianghit/cutesv/blob/master/simulation/README.md This snippet demonstrates how to use the VISOR HACk module to modify a reference genome by introducing specific structural variations (deletions, insertions, inversions, duplications, translocations). It requires a reference genome FASTA file and BED files defining the variations. The output is a modified donor genome. ```shell VISOR HACk -g reference_genome.fasta -bed sim_del.bed -o donor_genome_del VISOR HACk -g reference_genome.fasta -bed sim_ins.bed -o donor_genome_ins VISOR HACk -g reference_genome.fasta -bed sim_inv.bed -o donor_genome_inv VISOR HACk -g reference_genome.fasta -bed sim_dup.bed -o donor_genome_dup VISOR HACk -g reference_genome.fasta -bed sim_tra.bed -o donor_genome_tra ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.