### Install Rust Toolchain with Rustup Source: https://github.com/fulcrumgenomics/fgumi/blob/main/docs/DEVELOPING.md Installs the Rust toolchain using rustup. Ensure curl is installed before running. ```bash curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh ``` -------------------------------- ### Build fgumi with Features Source: https://github.com/fulcrumgenomics/fgumi/blob/main/CLAUDE.md Example of how to build the fgumi project with specific features enabled, such as 'compare' and 'simulate'. ```bash cargo build --release --features compare,simulate ``` -------------------------------- ### Install Git Hooks Source: https://github.com/fulcrumgenomics/fgumi/blob/main/CONTRIBUTING.md Run this script after cloning to install pre-commit hooks for code quality checks. ```bash ./scripts/install-hooks.sh ``` -------------------------------- ### Install fgumi with Cargo Source: https://github.com/fulcrumgenomics/fgumi/blob/main/README.md Installs the fgumi tool using the Cargo package manager. Ensure Rust and Cargo are installed. ```bash cargo install fgumi ``` -------------------------------- ### Paired-End with Sample Index Source: https://github.com/fulcrumgenomics/fgumi/blob/main/docs/src/guide/read-structures.md Example of a paired-end sequencing run with an 8bp sample index. ```text +T, 8B, +T ``` -------------------------------- ### Example High-Depth Benchmarking Data Generation Source: https://github.com/fulcrumgenomics/fgumi/blob/main/docs/simulate-cli.md Generates data with many UMIs at few positions to benchmark the MIH optimization in `fgumi group`. This uses position distribution options. ```bash fgumi simulate high-depth-benchmarking \ --output high_depth_benchmarking.bam \ --reference hg38.fa \ --num-reads 100000 \ --num-umis 100 \ --positions 10 ``` -------------------------------- ### Streaming Alignment Workflow with fgumi Source: https://github.com/fulcrumgenomics/fgumi/blob/main/docs/src/guide/migration-from-fgbio.md This example demonstrates a Unix pipe-based streaming pipeline for the alignment workflow, replacing multiple separate fgbio/picard steps with a single pass. `fgumi zipper` can accept SAM piped from the aligner or a BAM file via `--input`. ```bash fgumi fastq --input unaligned.bam \ | bwa mem -p -K 150000000 -Y ref.fa - \ | fgumi zipper --unmapped unaligned.bam \ | fgumi sort --output sorted.bam --order template-coordinate ``` -------------------------------- ### Example Includelist Format Source: https://github.com/fulcrumgenomics/fgumi/blob/main/docs/simulate-cli.md The includelist is a plain text file with one UMI per line, sorted alphabetically. This format is used by `fgumi simulate correct-reads`. ```text AAAACCCC AAAACCCT AAAACCGG ... ``` -------------------------------- ### Vertical Key-Value Metrics Example (Simplex/Duplex) Source: https://github.com/fulcrumgenomics/fgumi/blob/main/docs/src/guide/working-with-metrics.md The `simplex` and `duplex` commands use a three-column format with one metric per row, compatible with fgbio's `CallMolecularConsensusReads` output. ```text key value description raw_reads_considered 50000 Total raw reads considered from input file raw_reads_used 41800 Total count of raw reads used in consensus reads consensus_reads_emitted 12000 Total number of consensus reads (R1+R2=2) emitted ``` -------------------------------- ### Get fgumi Command Help Source: https://github.com/fulcrumgenomics/fgumi/blob/main/README.md Displays detailed usage information for any fgumi command. Replace `` with the specific tool name (e.g., extract, correct). ```bash fgumi --help ``` -------------------------------- ### Typical fgumi Grouping Pipeline Source: https://context7.com/fulcrumgenomics/fgumi/llms.txt An example of the command-line interface (CLI) commands for a typical UMI processing pipeline using fgumi. This sequence includes sorting, grouping, deduplication, and filtering steps. ```bash # Typical group pipeline (CLI): # fgumi sort -i aligned.bam -o tc_sorted.bam --order template-coordinate # fgumi group -i tc_sorted.bam -o grouped.bam --strategy paired --edits 1 # fgumi duplex -i grouped.bam -o duplex.bam --min-reads 3,2,1 # fgumi sort -i duplex.bam -o unmapped.bam --order queryname # bwa mem ref.fa unmapped.bam | samtools view -bS > mapped.bam # fgumi zipper -i mapped.bam -u unmapped.bam -r ref.fa -o zipped.bam # fgumi sort -i zipped.bam -o coord.bam --order coordinate # fgumi filter -i coord.bam -o filtered.bam --min-reads 3 --max-base-error-rate 0.1 ``` -------------------------------- ### Development/Testing Configuration Source: https://github.com/fulcrumgenomics/fgumi/blob/main/docs/src/guide/performance-tuning.md Enable fast iteration with minimal resource usage by running single-threaded, using minimal memory, and fast compression for quick turnaround times. ```bash fgumi filter \ --queue-memory 256 \ --compression-level 1 \ --input small_test.bam \ --output test_output.bam ``` -------------------------------- ### Install fgumi with Bioconda Source: https://github.com/fulcrumgenomics/fgumi/blob/main/docs/src/index.md Install fgumi using the Bioconda package manager, suitable for bioinformatics environments. ```bash conda install -c bioconda fgumi ``` -------------------------------- ### Build and Test Commands for fgumi Source: https://github.com/fulcrumgenomics/fgumi/blob/main/CLAUDE.md Standard commands for building the project, running tests with various configurations, checking formatting, linting, and executing benchmarks. ```bash # Build cargo build --release ``` ```bash # Run all tests (recommended - uses nextest) cargo ci-test ``` ```bash # Run tests with test-utils feature cargo t ``` ```bash # Check formatting cargo ci-fmt ``` ```bash # Run linting cargo ci-lint ``` ```bash # Run all CI checks cargo ci-test && cargo ci-fmt && cargo ci-lint ``` ```bash # Run a single test cargo nextest run test_name ``` ```bash # Run benchmarks cargo bench ``` -------------------------------- ### RawQuerynameLexKey::name Source: https://github.com/fulcrumgenomics/fgumi/blob/main/crates/fgumi-sort/public-api.txt Gets the name slice from a RawQuerynameLexKey. ```APIDOC ## RawQuerynameLexKey::name ### Description Returns a byte slice representing the name stored within the `RawQuerynameLexKey`. ### Signature `pub fn fgumi_sort::keys::RawQuerynameLexKey::name(&self) -> &[u8]` ``` -------------------------------- ### RawQuerynameKey::init Source: https://github.com/fulcrumgenomics/fgumi/blob/main/crates/fgumi-sort/public-api.txt Initializes a RawQuerynameKey at a given pointer. ```APIDOC ## RawQuerynameKey::init ### Description Initializes a `RawQuerynameKey` at a given memory location using the provided initialization data. Returns the pointer to the initialized memory. ### Safety This function is unsafe because the caller must ensure that the memory pointed to by the returned pointer is valid and that the initialization data is correct. ### Signature `pub unsafe fn fgumi_sort::keys::RawQuerynameKey::init(init: ::Init) -> usize` ``` -------------------------------- ### RawQuerynameLexKey::new Source: https://github.com/fulcrumgenomics/fgumi/blob/main/crates/fgumi-sort/public-api.txt Creates a new RawQuerynameLexKey. ```APIDOC ## RawQuerynameLexKey::new ### Description Constructs a new `RawQuerynameLexKey` with the given name (as a `Vec`) and flags (as a `u16`). ### Signature `pub fn fgumi_sort::keys::RawQuerynameLexKey::new(name: alloc::vec::Vec, flags: u16) -> Self` ``` -------------------------------- ### Simple Paired-End Read Structure Source: https://github.com/fulcrumgenomics/fgumi/blob/main/docs/src/guide/read-structures.md Example of a simple paired-end sequencing run with two reads of 150bp each and no indices. ```text +T, +T ``` -------------------------------- ### Filter Stats TSV Example (No Header) Source: https://github.com/fulcrumgenomics/fgumi/blob/main/docs/src/guide/working-with-metrics.md The `filter --stats` command produces a two-column key-value format without a header row. ```text total_reads 10000 passed_reads 8542 pass_rate 0.8542 ``` -------------------------------- ### Clone and Build fgumi from Source Source: https://github.com/fulcrumgenomics/fgumi/blob/main/README.md Clones the fgumi repository and builds the release version. This is useful for development or when a pre-built binary is not available. ```bash git clone https://github.com/fulcrumgenomics/fgumi ``` ```bash cd fgumi cargo build --release ``` -------------------------------- ### Rust FFI Call to mimalloc Source: https://github.com/fulcrumgenomics/fgumi/blob/main/CLAUDE.md Example of an approved non-stdlib FFI call to the mimalloc library for memory statistics. This is isolated to a specific submodule and is safe to invoke concurrently. ```rust #![allow(unsafe_code)] // ... FFI wrappers for mimalloc functions like force_mi_collect, process_rss_bytes, print_mi_stats ``` -------------------------------- ### Fast Local SSD Configuration Source: https://github.com/fulcrumgenomics/fgumi/blob/main/docs/src/guide/performance-tuning.md Optimize for fast I/O on local SSDs by allocating high memory for pipeline buffers and using minimal compression, as I/O is not expected to be the bottleneck. ```bash fgumi filter \ --threads 8 \ --queue-memory 2GB \ --compression-level 1 \ --input dataset.bam \ --output filtered.bam ``` -------------------------------- ### Horizontal TSV Metrics Example Source: https://github.com/fulcrumgenomics/fgumi/blob/main/docs/src/guide/working-with-metrics.md This format is used by commands like `dedup`, `codec`, `duplex-metrics`, `simplex-metrics`, and `group`. It includes a header row followed by a single data row. ```text total_templates unique_templates duplicate_templates duplicate_rate 25000 18750 6250 0.25 ``` -------------------------------- ### Paired-End with Inline UMI in R1 Source: https://github.com/fulcrumgenomics/fgumi/blob/main/docs/src/guide/read-structures.md Example of a paired-end sequencing run where the first read (R1) includes a 6bp inline Molecular Barcode (UMI) followed by template bases. ```text 6M+T, 8B, +T ``` -------------------------------- ### Run Code Formatting Check Source: https://github.com/fulcrumgenomics/fgumi/blob/main/CONTRIBUTING.md Execute this command to check if the code adheres to the project's formatting standards. It will fail if formatting differs. ```bash cargo ci-fmt ``` -------------------------------- ### Rust FFI Call to mach2 Source: https://github.com/fulcrumgenomics/fgumi/blob/main/CLAUDE.md Example of an approved non-stdlib FFI call to the mach2 library on macOS for reading physical memory footprint (RSS). This is isolated to a specific submodule. ```rust #![allow(unsafe_code)] // ... FFI call to task_info(TASK_VM_INFO) for phys_footprint ``` -------------------------------- ### Generate FASTQ Reads with fgumi simulate Source: https://github.com/fulcrumgenomics/fgumi/blob/main/docs/simulate-cli.md Generates paired-end FASTQ files with UMI sequences for input to fgumi extract. Requires a reference FASTA file and specifies output R1/R2 FASTQ.gz files. Options control molecule count, read length, UMI length, read structure, quality model, family size, insert size, and methylation. ```bash fgumi simulate fastq-reads \ --r1 output_R1.fastq.gz \ --r2 output_R2.fastq.gz \ [OPTIONS] ``` ```bash fgumi simulate fastq-reads \ -1 output_R1.fastq.gz \ -2 output_R2.fastq.gz \ -r Homo_sapiens.GRCh38.fasta \ --num-molecules 10000 \ --read-length 150 \ --umi-length 12 \ --family-size-mean 5.0 \ --family-size-stddev 3.0 \ --insert-size-mean 400 \ --insert-size-stddev 100 ``` -------------------------------- ### Example of CIGAR Alignment Differences Source: https://github.com/fulcrumgenomics/fgumi/blob/main/docs/src/guide/duplex-consensus-calling.md Illustrates how an insertion in one read can cause it to be out of phase with others, leading to disagreements. This highlights the need for CIGAR filtering in duplex consensus calling. ```text 1: ACGTGACTGACTAGCTTTTTTT-AGACTAGCTACTACT 2: ACGTGACTGACTAGCTTTTTTT-AGACTAGCTACTACT 3: ACGTGACTGACTAGCTTTTTTTT-GACTAGCTACTACT ``` -------------------------------- ### Run All Tests Source: https://github.com/fulcrumgenomics/fgumi/blob/main/CONTRIBUTING.md Execute this command to run the entire test suite for the project. This is also part of the CI checks. ```bash cargo ci-test ``` -------------------------------- ### Raw Read Strand Assignment Example Source: https://github.com/fulcrumgenomics/fgumi/blob/main/docs/src/guide/tracking-reads.md Illustrates how raw reads are assigned to top ('/A') or bottom ('/B') strands based on their 5' unclipped positions relative to sequencing order. ```text x: R1-----------------> <-------------------R2 y: R2-----------------> <-------------------R1 z: R1-----------------> <-----------------R2 ``` -------------------------------- ### OS Hints Source: https://github.com/fulcrumgenomics/fgumi/blob/main/crates/fgumi-bam-io/public-api.txt Functions for providing operating system-level hints for file access patterns. ```APIDOC ## advise_sequential ### Description Advises the OS to expect sequential access to a file. ### Signature `pub fn fgumi_bam_io::os_hints::advise_sequential(file: &std::fs::File)` ## advise_willneed_raw ### Description Advises the OS that a specific range of a file descriptor will be needed. ### Signature `pub fn fgumi_bam_io::os_hints::advise_willneed_raw(fd: i32, offset: i64, len: i64)` ## hint_fd ### Description Retrieves a file descriptor for a given file, if available. ### Signature `pub fn fgumi_bam_io::os_hints::hint_fd(_file: &std::fs::File) -> core::option::Option` ``` -------------------------------- ### Create BAM Reader with Options Source: https://github.com/fulcrumgenomics/fgumi/blob/main/crates/fgumi-bam-io/public-api.txt Creates a BAM reader with automatic compression detection, a specified number of threads, and custom pipeline reader options. It also returns the SAM header. ```APIDOC ## create_bam_reader_with_opts ### Description Creates a BAM reader with automatic compression detection, using the specified number of threads and custom pipeline reader options. It returns the reader and the SAM header. ### Function Signature ```rust pub fn create_bam_reader_with_opts>(path: P, threads: usize, opts: fgumi_bam_io::reader::PipelineReaderOpts) -> anyhow::Result<(fgumi_bam_io::reader::BamReaderAuto, noodles_sam::header::Header)> ``` ### Parameters - `path` (P): A type that can be converted into a reference to a `std::path::Path`, representing the path to the BAM file. - `threads` (usize): The number of threads to use for reading the BAM file. - `opts` (fgumi_bam_io::reader::PipelineReaderOpts): Custom options for the pipeline reader. ``` -------------------------------- ### RawQuerynameLexKey Methods Source: https://github.com/fulcrumgenomics/fgumi/blob/main/crates/fgumi-sort/public-api.txt Details the `RawQuerynameLexKey` struct, including its constructor, methods for accessing its data, and implementations for common traits like `Clone`, `Eq`, `Ord`, `PartialEq`, `PartialOrd`, `Default`, `Debug`, `StructuralPartialEq`, and `RawSortKey`. ```APIDOC ## RawQuerynameLexKey ### Constructor - `new(name: alloc::vec::Vec, flags: u16) -> Self` ### Methods - `name(&self) -> &[u8]` - `clone(&self) -> fgumi_sort::keys::RawQuerynameLexKey` - `cmp(&self, other: &Self) -> core::cmp::Ordering` - `eq(&self, other: &fgumi_sort::keys::RawQuerynameLexKey) -> bool` - `partial_cmp(&self, other: &Self) -> core::option::Option` - `default() -> fgumi_sort::keys::RawQuerynameLexKey` - `fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result` - `extract(bam: &[u8], _ctx: &fgumi_sort::keys::SortContext) -> Self` - `extract_from_record(bam: &[u8]) -> Self` - `read_from(reader: &mut R) -> std::io::error::Result` - `write_to(&self, writer: &mut W) -> std::io::error::Result<()>` ### Traits Implemented - `core::clone::Clone` - `core::cmp::Eq` - `core::cmp::Ord` - `core::cmp::PartialEq` - `core::cmp::PartialOrd` - `core::default::Default` - `core::fmt::Debug` - `core::marker::StructuralPartialEq` - `fgumi_sort::keys::RawSortKey` ### Constants - `EMBEDDED_IN_RECORD: bool` - `SERIALIZED_SIZE: core::option::Option` ``` -------------------------------- ### Configure and Execute Simplex Consensus Calling Source: https://context7.com/fulcrumgenomics/fgumi/llms.txt Instantiate and execute the `Simplex` command with various options for input/output, read filtering, consensus calling parameters, threading, and statistics. Ensure necessary imports are included. The `execute` method takes a command-line string as an argument. ```rust use fgumi_lib::commands::simplex::Simplex; use fgumi_lib::commands::command::Command; use fgumi_lib::commands::common::{ BamIoOptions, CompressionOptions, ConsensusCallingOptions, OverlappingConsensusOptions, ReadGroupOptions, RejectsOptions, StatsOptions, ThreadingOptions, SchedulerOptions, QueueMemoryOptions, }; use std::path::PathBuf; let simplex = Simplex { io: BamIoOptions { input: PathBuf::from("grouped.bam"), output: PathBuf::from("consensus.bam"), async_reader: false, }, min_reads: 2, // minimum raw reads to produce a consensus max_reads: Some(100), // optionally downsample large families consensus: ConsensusCallingOptions { error_rate_pre_umi: 45, // Q45 = ~3.2e-5 error rate before UMI ligation error_rate_post_umi: 40, // Q40 = ~1e-4 error rate after UMI ligation min_input_base_quality: 10, // ignore input bases below Q10 output_per_base_tags: true, // emit cd/ce per-base depth/error arrays min_consensus_base_quality: 0, // mask bases below this quality to 'N' trim: false, }, overlapping: OverlappingConsensusOptions { consensus_call_overlapping_bases: true, // correct overlapping paired-end bases }, read_group: ReadGroupOptions { read_group_id: "RG001".to_string(), read_name_prefix: Some("CONSENSUS".to_string()), }, threading: ThreadingOptions::new(8), // enables 7-step parallel pipeline rejects_opts: RejectsOptions { rejects: Some(PathBuf::from("rejects.bam")) }, stats_opts: StatsOptions { stats: Some(PathBuf::from("simplex_stats.txt")) }, compression: CompressionOptions { compression_level: 1 }, scheduler_opts: SchedulerOptions::default(), queue_memory: QueueMemoryOptions::default(), methylation_mode: None, // or Some(MethylationModeArg::EmSeq) / Some(MethylationModeArg::Taps) reference: None, // required when methylation_mode is set }; simplex.execute("fgumi simplex -i grouped.bam -o consensus.bam -M 2")?; // Output tags on consensus reads: // cD (int) = max depth of raw reads at any position // cM (int) = min depth of raw reads at any position // cE (float) = fraction of raw reads disagreeing with consensus // cd (short[]) = per-base read depth (if --output-per-base-tags) // ce (short[]) = per-base raw read errors (if --output-per-base-tags) ``` -------------------------------- ### Sort BAM Files with Different Orders Source: https://github.com/fulcrumgenomics/fgumi/blob/main/crates/fgumi-sort/README.md Demonstrates how to sort BAM files using coordinate, queryname (natural), and template-coordinate orders. Configure threads and output compression as needed. ```rust use fgumi_sort::{RawExternalSorter, SortOrder}; use fgumi_sort::keys::QuerynameComparator; use std::path::Path; // Coordinate sort (for IGV, variant callers, etc.) RawExternalSorter::new(SortOrder::Coordinate) .threads(4) .output_compression(6) .sort(Path::new("input.bam"), Path::new("sorted.bam")) .expect("sort failed"); // Queryname sort — natural ordering (read1 < read2 < read10) RawExternalSorter::new(SortOrder::Queryname(QuerynameComparator::Natural)) .threads(4) .sort(Path::new("input.bam"), Path::new("by-name.bam")) .expect("sort failed"); // Template-coordinate sort (required before fgumi group) RawExternalSorter::new(SortOrder::TemplateCoordinate) .threads(4) .sort(Path::new("input.bam"), Path::new("template-coord.bam")) .expect("sort failed"); ``` -------------------------------- ### Enable dhat-heap Feature Source: https://github.com/fulcrumgenomics/fgumi/blob/main/docs/DEVELOPING.md Enables heap profiling via the `dhat` crate by replacing the default allocator. ```bash cargo build --features dhat-heap ``` -------------------------------- ### RawQuerynameKey Methods Source: https://github.com/fulcrumgenomics/fgumi/blob/main/crates/fgumi-sort/public-api.txt Provides documentation for the `RawQuerynameKey` type, including its implementations for various traits like `Equivalent`, `Into`, `TryFrom`, `ToOwned`, `Any`, `Borrow`, `BorrowMut`, `CloneToUninit`, `From`, and `Pointable`. ```APIDOC ## RawQuerynameKey ### Methods - `equivalent(&self, key: &K) -> bool` where `K: core::borrow::Borrow` - `into(self) -> U` where `U: core::convert::From` - `try_from(value: U) -> core::result::Result>::Error>` where `U: core::convert::Into` - `try_into(self) -> core::result::Result>::Error>` where `U: core::convert::TryFrom` - `clone_into(&self, target: &mut T)` where `T: core::clone::Clone` - `to_owned(&self) -> T` where `T: core::clone::Clone` - `type_id(&self) -> core::any::TypeId` - `borrow(&self) -> &T` where `T: ?core::marker::Sized` - `borrow_mut(&mut self) -> &mut T` where `T: ?core::marker::Sized` - `clone_to_uninit(&self, dest: *mut u8)` where `T: core::clone::Clone` - `from(t: T) -> T` - `deref<'a>(ptr: usize) -> &'a T` (unsafe) - `deref_mut<'a>(ptr: usize) -> &'a mut T` (unsafe) - `drop(ptr: usize)` (unsafe) - `init(init: ::Init) -> usize` (unsafe) ### Associated Types - `Error = core::convert::Infallible` for `TryFrom` - `Error = >::Error` for `TryInto` - `Owned = T` for `ToOwned` - `Init = T` for `crossbeam_epoch::atomic::Pointable` ### Constants - `ALIGN: usize` ``` -------------------------------- ### fgumi simulate fastq-reads Source: https://github.com/fulcrumgenomics/fgumi/blob/main/docs/simulate-cli.md Generates simulated FASTQ files with specified read structures and UMI lengths. This command is useful for creating synthetic data for testing and benchmarking. ```APIDOC ## fgumi simulate fastq-reads ### Description Generates simulated FASTQ files with specified read structures and UMI lengths. ### Usage ```bash fgumi simulate fastq-reads \ --r1 \ --r2 \ --truth \ --reference \ [OPTIONS] ``` ### Required Arguments | Option | Type | Description | |--------|------|-------------| | `--r1` | PATH | Output FASTQ file for read 1 | | `--r2` | PATH | Output FASTQ file for read 2 | | `--truth` | PATH | Output truth TSV file (for validation) | | `--reference` | PATH | Reference FASTA file (sequences sampled from here) | ### Simulation Options | Option | Type | Default | Description | |--------|------|---------|-------------| | `-n, --num-molecules` | INT | 1000 | Number of unique molecules to simulate | | `-l, --read-length` | INT | 150 | Length of each read in bases | | `-u, --umi-length` | INT | 8 | Length of UMI sequence in bases | | `--seed` | INT | (random) | Random seed for reproducibility | ### Read Structure Options | Option | Type | Description | |--------|------|-------------| | `--read-structure-r1` | STRING | Read structure for R1 (e.g., "8M142T") | | `--read-structure-r2` | STRING | Read structure for R2 (e.g., "150T") | ### Quality Model Options | Option | Type | Default | Description | |--------|------|---------|-------------| | `--warmup-bases` | INT | 10 | Number of bases before peak quality | | `--warmup-quality` | INT | 25 | Starting quality during warmup | | `--peak-quality` | INT | 37 | Peak quality score | | `--decay-start` | INT | 100 | Position where decay begins | | `--decay-rate` | FLOAT | 0.08 | Quality drop per base | | `--quality-noise` | FLOAT | 2.0 | Quality noise std dev | | `--r2-quality-offset` | INT | -2 | R2 quality offset | ### Family Size Options | Option | Type | Default | Description | |--------|------|---------|-------------| | `--min-family-size` | INT | 1 | Minimum UMI family size | | `--max-family-size` | INT | 1000 | Maximum UMI family size | | `--family-size-sigma` | FLOAT | 50.0 | Standard deviation for family size distribution | ### Insert Size Options | Option | Type | Default | Description | |--------|------|---------|-------------| | `--mean-insert-size` | INT | 300 | Mean insert size | | `--insert-size-sigma` | FLOAT | 50.0 | Standard deviation for insert size distribution | ### Methylation Options | Option | Type | Default | Description | |--------|------|---------|-------------| | `--methylation-rate` | FLOAT | 0.0 | Rate of methylation (0.0 to 1.0) | | `--methylation-noise` | FLOAT | 0.0 | Noise in methylation detection | ### Example ```bash # Generate 10,000 molecules with 8bp UMIs fgumi simulate fastq-reads \ --r1 sim_R1.fastq.gz \ --r2 sim_R2.fastq.gz \ --truth sim_truth.tsv \ --reference hg38.fa \ --num-molecules 10000 \ --umi-length 8 \ --read-structure-r1 "8M142T" \ --read-structure-r2 "150T" \ --seed 42 ``` ``` -------------------------------- ### Build fgumi with Optional Features Source: https://github.com/fulcrumgenomics/fgumi/blob/main/README.md Builds the release version of fgumi with specific features enabled. Use this to include additional developer tools or data simulation capabilities. ```bash cargo build --release --features ``` -------------------------------- ### Create BAM Reader with Options Source: https://github.com/fulcrumgenomics/fgumi/blob/main/crates/fgumi-bam-io/public-api.txt Creates a BAM reader with specified threads and pipeline options. ```APIDOC ## create_bam_reader_with_opts> (path: P, threads: usize, opts: PipelineReaderOpts) -> Result<(BamReaderAuto, Header)> ### Description Creates a BAM reader with the specified number of threads and pipeline options. ### Parameters #### Path Parameters - **path** (P): The path to the BAM file. - **threads** (usize): The number of threads to use for reading. - **opts** (PipelineReaderOpts): Options for configuring the pipeline reader. ### Returns - Result<(BamReaderAuto, Header)>: A tuple containing the BAM reader and the header, or an error. ``` -------------------------------- ### Network Storage Configuration Source: https://github.com/fulcrumgenomics/fgumi/blob/main/docs/src/guide/performance-tuning.md Minimize network I/O and hide latency by using `--async-reader`, moderate threading, conservative memory, and maximum compression to reduce data transfer over the network. ```bash fgumi filter \ --async-reader \ --threads 4 \ --queue-memory 512 \ --compression-level 9 \ --input dataset.bam \ --output filtered.bam ``` -------------------------------- ### RawBamRecordReader::new Source: https://github.com/fulcrumgenomics/fgumi/blob/main/crates/fgumi-sort/public-api.txt Creates a new RawBamRecordReader directly from a reader. This is the primary constructor for reading BAM data. ```APIDOC ## RawBamRecordReader::new ### Description Creates a new `RawBamRecordReader` instance. This is the primary constructor for initiating BAM record reading from a given reader. ### Method `new` ### Parameters - `reader` (R): The input reader implementing `std::io::Read`. ### Returns - `std::io::error::Result`: A `Result` containing the `RawBamRecordReader` on success or an `std::io::Error` on failure. ``` -------------------------------- ### Create Raw BAM Reader with Options Source: https://github.com/fulcrumgenomics/fgumi/blob/main/crates/fgumi-bam-io/public-api.txt Creates a raw BAM reader with custom pipeline reader options and a specified number of threads. It returns the raw reader and the SAM header. ```APIDOC ## create_raw_bam_reader_with_opts ### Description Creates a raw BAM reader with custom pipeline reader options and the specified number of threads. It returns the raw reader and the SAM header. ### Function Signature ```rust pub fn create_raw_bam_reader_with_opts>(path: P, threads: usize, opts: fgumi_bam_io::reader::PipelineReaderOpts) -> anyhow::Result<(fgumi_bam_io::reader::RawBamReaderAuto, noodles_sam::header::Header)> ``` ### Parameters - `path` (P): A type that can be converted into a reference to a `std::path::Path`, representing the path to the BAM file. - `threads` (usize): The number of threads to use for reading the BAM file. - `opts` (fgumi_bam_io::reader::PipelineReaderOpts): Custom options for the pipeline reader. ``` -------------------------------- ### fgumi Memory Management Options Source: https://github.com/fulcrumgenomics/fgumi/blob/main/docs/src/guide/performance-tuning.md Configure pipeline queue memory using human-readable formats or specify a fixed total budget. ```bash fgumi filter --queue-memory 768 --queue-memory-per-thread true ``` ```bash fgumi filter --queue-memory 2GB ``` ```bash fgumi filter --queue-memory 1024MiB ``` ```bash fgumi filter --queue-memory 4096 --queue-memory-per-thread false ``` -------------------------------- ### RawQuerynameLexKey Writing Source: https://github.com/fulcrumgenomics/fgumi/blob/main/crates/fgumi-sort/public-api.txt Provides methods for writing RawQuerynameLexKey instances to a writer. ```APIDOC ## write_to ### Description Writes the RawQuerynameLexKey to a given writer. ### Signature `pub fn fgumi_sort::keys::RawQuerynameLexKey::write_to(&self, writer: &mut W) -> std::io::error::Result<()>` ``` -------------------------------- ### Downsample BAM by UMI Family with `fgumi downsample` Source: https://context7.com/fulcrumgenomics/fgumi/llms.txt Initializes and executes the UMI downsampling command. Requires input and output BAM files, a fraction to keep, and optional parameters for reproducibility and validation. ```rust use fgumi_lib::commands::downsample::Downsample; use fgumi_lib::commands::command::Command; use fgumi_lib::commands::common::{BamIoOptions, CompressionOptions}; use std::path::PathBuf; let downsample = Downsample { io: BamIoOptions { input: PathBuf::from("grouped.bam"), output: PathBuf::from("downsampled.bam"), async_reader: false, }, fraction: 0.1, // keep 10% of UMI families rejects: Some(PathBuf::from("rejected.bam")), seed: Some(42), // reproducible random sampling validate_mi_order: true, // error if MI tags not consecutive histogram_kept: None, histogram_rejected: None, compression: CompressionOptions::default(), }; downsample.execute("fgumi downsample -i grouped.bam -o downsampled.bam -f 0.1 --seed 42")?; ``` -------------------------------- ### Create Raw BAM Reader with Options Source: https://github.com/fulcrumgenomics/fgumi/blob/main/crates/fgumi-bam-io/public-api.txt Creates a raw BAM reader with custom pipeline options and a specified number of threads. ```APIDOC ## create_raw_bam_reader_with_opts ### Description Creates a raw BAM reader with automatic detection of compression format, using the specified number of threads and custom pipeline reader options. ### Signature ```rust pub fn create_raw_bam_reader_with_opts>(path: P, threads: usize, opts: fgumi_bam_io::reader::PipelineReaderOpts) -> anyhow::Result<(fgumi_bam_io::reader::RawBamReaderAuto, noodles_sam::header::Header)> ``` ### Parameters - `path`: The path to the BAM file. - `threads`: The number of threads to use for reading. - `opts`: Custom options for the pipeline reader. ``` -------------------------------- ### LibraryLookup Initialization and Management Source: https://github.com/fulcrumgenomics/fgumi/blob/main/crates/fgumi-sort/public-api.txt Provides functions for initializing, dereferencing, and dropping library lookup structures. ```APIDOC ## LibraryLookup Initialization and Management ### `fgumi_sort::external::LibraryLookup::Init` Type alias for initialization parameters. ### `fgumi_sort::external::LibraryLookup::ALIGN` Constant representing alignment information. ### `fgumi_sort::external::LibraryLookup::deref` #### Description Safely dereferences a pointer to a library lookup structure. #### Signature `pub unsafe fn deref<'a>(ptr: usize) -> &'a T` ### `fgumi_sort::external::LibraryLookup::deref_mut` #### Description Safely dereferences a pointer to a library lookup structure for mutable access. #### Signature `pub unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T` ### `fgumi_sort::external::LibraryLookup::drop` #### Description Drops a library lookup structure at the given pointer. #### Signature `pub unsafe fn drop(ptr: usize)` ### `fgumi_sort::external::LibraryLookup::init` #### Description Initializes a new library lookup structure. #### Signature `pub unsafe fn init(init: ::Init) -> usize` ``` -------------------------------- ### Compare BAM files with full mode Source: https://github.com/fulcrumgenomics/fgumi/blob/main/docs/compare-cli.md Use this command to compare BAM files with MI tags, including grouping and full content comparison. This is recommended for `group` output where MI tags are present and order is deterministic. ```bash fgumi compare bams grouped1.bam grouped2.bam --mode full ``` -------------------------------- ### Enable Asynchronous I/O Reading Source: https://github.com/fulcrumgenomics/fgumi/blob/main/docs/src/guide/performance-tuning.md Use `--async-reader` to improve I/O performance in environments where OS readahead cannot be tuned, such as containers or network mounts. It spawns a dedicated I/O thread to read data ahead of processing. ```bash fgumi group \ --async-reader \ --threads 8 \ --input reads.bam \ --output grouped.bam ``` -------------------------------- ### Align, Filter, and Sort (High-Throughput) Source: https://github.com/fulcrumgenomics/fgumi/blob/main/docs/src/guide/best-practices.md Combine alignment, filtering, and sorting into a single pipeline for high-throughput processing. Note that this pipeline requires `fgumi zipper` and cannot be fully merged with consensus calling due to random access needs. ```bash fgumi fastq --input consensus.bam \ | bwa mem -t 16 -p -K 150000000 -Y ref.fa - \ | fgumi zipper --unmapped consensus.bam --reference ref.fa \ | fgumi filter --input /dev/stdin --ref ref.fa --min-reads 3 \ | fgumi sort --input /dev/stdin --output filtered.bam --order coordinate --threads 4 ``` -------------------------------- ### Generate Grouped Reads with fgumi simulate Source: https://github.com/fulcrumgenomics/fgumi/blob/main/docs/simulate-cli.md Generates template-coordinate sorted BAM files with MI tags for input to fgumi simplex, duplex, or codec commands. Requires a reference FASTA file and specifies the output BAM file. Options control the number of molecules, read length, UMI length, and insert size distribution. ```bash fgumi simulate grouped-reads \ --output grouped_reads.bam \ -r Homo_sapiens.GRCh38.fasta \ [OPTIONS] ``` -------------------------------- ### Simulate FASTQ Reads with UMIs Source: https://github.com/fulcrumgenomics/fgumi/blob/main/docs/simulate-cli.md Generates simulated FASTQ files with specified UMI lengths and read structures. Use this to create synthetic data for testing read processing pipelines. ```bash fgumi simulate fastq-reads \ --r1 sim_R1.fastq.gz \ --r2 sim_R2.fastq.gz \ --truth sim_truth.tsv \ --reference hg38.fa \ --num-molecules 10000 \ --umi-length 8 \ --read-structure-r1 "8M142T" \ --read-structure-r2 "150T" \ --seed 42 ``` -------------------------------- ### Generate Mapped Reads with fgumi simulate Source: https://github.com/fulcrumgenomics/fgumi/blob/main/docs/simulate-cli.md Generates template-coordinate sorted BAM files for input to the fgumi group command. Requires a reference FASTA file and specifies the output BAM file. Options control the number of molecules, read length, UMI length, and insert size distribution. ```bash fgumi simulate mapped-reads \ --output mapped_reads.bam \ -r Homo_sapiens.GRCh38.fasta \ [OPTIONS] ``` -------------------------------- ### create_indexing_bam_writer Source: https://github.com/fulcrumgenomics/fgumi/blob/main/crates/fgumi-bam-io/public-api.txt Creates a BAM writer that also generates an index file (BAI) as records are written. It takes the path, header, compression level, and number of threads. ```APIDOC ## create_indexing_bam_writer ### Description Creates a BAM writer that also generates an index file (BAI) as records are written. ### Parameters - `path`: `P: core::convert::AsRef` - The path to the BAM file to be created. - `header`: `&noodles_sam::header::Header` - The header for the BAM file. - `compression_level`: `u32` - The compression level to use. - `threads`: `usize` - The number of threads to use for writing. ### Returns - `anyhow::Result` - An `IndexingBamWriter` instance on success, or an error on failure. ``` -------------------------------- ### RawQuerynameLexKey Source: https://github.com/fulcrumgenomics/fgumi/blob/main/crates/fgumi-sort/public-api.txt RawQuerynameLexKey provides methods for comparing, checking equivalence, and converting keys for lexicographical sorting. ```APIDOC ## RawQuerynameLexKey ### Description Represents a key used for lexicographical sorting. It implements various traits for comparison, equivalence checking, and conversion. ### Methods - `compare(&self, key: &K) -> core::cmp::Ordering` Compares this key with another key. - `equivalent(&self, key: &K) -> bool` Checks if this key is equivalent to another key. - `into(self) -> U` Converts the key into another type `U` if `U` implements `From`. - `try_from(value: U) -> core::result::Result>::Error>` Attempts to convert a value `U` into this key type. - `try_into(self) -> core::result::Result>::Error>` Attempts to convert this key into another type `U`. - `clone_into(&self, target: &mut T)` Clones the key into a mutable target. - `to_owned(&self) -> T` Creates an owned version of the key. - `type_id(&self) -> core::any::TypeId` Returns the type ID of the key. - `borrow(&self) -> &T` Borrows the key as a reference to type `T`. - `borrow_mut(&mut self) -> &mut T` Borrows the key mutably as a reference to type `T`. - `clone_to_uninit(&self, dest: *mut u8)` Unsafely clones the key into an uninitialized memory location. - `from(t: T) -> T` Creates a key from a value `T`. ### Associated Types - `Error` (for `try_from`): `core::convert::Infallible` - `Error` (for `try_into`): `>::Error` - `Owned`: `T` ### Constants - `ALIGN`: `usize` ### Safety - `deref(ptr: usize) -> &'a T` - `deref_mut(ptr: usize) -> &'a mut T` - `drop(ptr: usize)` - `init(init: ::Init) -> usize` These methods are unsafe and require careful handling of pointers and initialization. ``` -------------------------------- ### Enable Tests in Pre-Commit Hook Source: https://github.com/fulcrumgenomics/fgumi/blob/main/CONTRIBUTING.md Set this environment variable to 1 to enable running tests as part of the pre-commit hook. Use sparingly. ```bash FGUMI_PRECOMMIT_TEST=1 git commit -m "message" ``` -------------------------------- ### RawQuerynameKey Methods Source: https://github.com/fulcrumgenomics/fgumi/blob/main/crates/fgumi-sort/public-api.txt Details the methods available for the `RawQuerynameKey` struct, including creation, accessors, comparison, and I/O operations. ```APIDOC ## RawQuerynameKey::name ### Description Returns the query name as a byte slice. ### Signature `pub fn fgumi_sort::keys::RawQuerynameKey::name(&self) -> &[u8]` ``` ```APIDOC ## RawQuerynameKey::new ### Description Creates a new `RawQuerynameKey` with the given name and flags. ### Signature `pub fn fgumi_sort::keys::RawQuerynameKey::new(name: alloc::vec::Vec, flags: u16) -> Self` ``` ```APIDOC ## RawQuerynameKey::clone ### Description Creates a clone of the `RawQuerynameKey`. ### Signature `pub fn fgumi_sort::keys::RawQuerynameKey::clone(&self) -> fgumi_sort::keys::RawQuerynameKey` ``` ```APIDOC ## RawQuerynameKey::cmp ### Description Compares this `RawQuerynameKey` with another for ordering. ### Signature `pub fn fgumi_sort::keys::RawQuerynameKey::cmp(&self, other: &Self) -> core::cmp::Ordering` ``` ```APIDOC ## RawQuerynameKey::eq ### Description Checks if this `RawQuerynameKey` is equal to another. ### Signature `pub fn fgumi_sort::keys::RawQuerynameKey::eq(&self, other: &Self) -> bool` ``` ```APIDOC ## RawQuerynameKey::partial_cmp ### Description Partially compares this `RawQuerynameKey` with another for ordering. ### Signature `pub fn fgumi_sort::keys::RawQuerynameKey::partial_cmp(&self, other: &Self) -> core::option::Option` ``` ```APIDOC ## RawQuerynameKey::default ### Description Returns the default `RawQuerynameKey`. ### Signature `pub fn fgumi_sort::keys::RawQuerynameKey::default() -> fgumi_sort::keys::RawQuerynameKey` ``` ```APIDOC ## RawQuerynameKey::fmt ### Description Formats the `RawQuerynameKey` for debugging output. ### Signature `pub fn fgumi_sort::keys::RawQuerynameKey::fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result` ``` ```APIDOC ## RawQuerynameKey::extract ### Description Extracts a `RawQuerynameKey` from BAM data with a sort context. ### Signature `pub fn fgumi_sort::keys::RawQuerynameKey::extract(bam: &[u8], _ctx: &fgumi_sort::keys::SortContext) -> Self` ``` ```APIDOC ## RawQuerynameKey::extract_from_record ### Description Extracts a `RawQuerynameKey` directly from a record in BAM data. ### Signature `pub fn fgumi_sort::keys::RawQuerynameKey::extract_from_record(bam: &[u8]) -> Self` ``` ```APIDOC ## RawQuerynameKey::read_from ### Description Reads a `RawQuerynameKey` from a given reader. ### Signature `pub fn fgumi_sort::keys::RawQuerynameKey::read_from(reader: &mut R) -> std::io::error::Result` ``` ```APIDOC ## RawQuerynameKey::write_to ### Description Writes the `RawQuerynameKey` to a given writer. ### Signature `pub fn fgumi_sort::keys::RawQuerynameKey::write_to(&self, writer: &mut W) -> std::io::error::Result<()>` ``` ```APIDOC ## RawQuerynameKey::compare ### Description Compares the `RawQuerynameKey` with a given key. ### Signature `pub fn fgumi_sort::keys::RawQuerynameKey::compare(&self, key: &K) -> core::cmp::Ordering` ``` ```APIDOC ## RawQuerynameKey::equivalent ### Description Checks if the `RawQuerynameKey` is equivalent to a given key. ### Signature `pub fn fgumi_sort::keys::RawQuerynameKey::equivalent(&self, key: &K) -> bool` ``` -------------------------------- ### RawQuerynameLexKey Cloning Source: https://github.com/fulcrumgenomics/fgumi/blob/main/crates/fgumi-sort/public-api.txt Provides methods for cloning RawQuerynameLexKey instances. ```APIDOC ## clone ### Description Creates a deep copy of the RawQuerynameLexKey. ### Signature `pub fn fgumi_sort::keys::RawQuerynameLexKey::clone(&self) -> fgumi_sort::keys::RawQuerynameLexKey` ``` ```APIDOC ## clone_into ### Description Clones the RawQuerynameLexKey into a mutable target. ### Signature `pub fn fgumi_sort::keys::RawQuerynameLexKey::clone_into(&self, target: &mut T)` ``` ```APIDOC ## to_owned ### Description Converts the RawQuerynameLexKey into an owned type. ### Signature `pub fn fgumi_sort::keys::RawQuerynameLexKey::to_owned(&self) -> T` ```