### start_char Method Source: https://docs.rs/needletail/0.7.3/needletail/parser/enum.Format.html Returns the starting character associated with each format. ```APIDOC ## impl Format ### pub fn start_char(&self) -> char Returns the starting character for the format. ``` -------------------------------- ### Get Start Character for Format Source: https://docs.rs/needletail/0.7.3/needletail/parser/enum.Format.html Returns the characteristic starting character for the specified sequence format. This can be useful for format detection or validation. ```rust pub fn start_char(&self) -> char ``` -------------------------------- ### Get Sequence Start Position Source: https://docs.rs/needletail/0.7.3/needletail/parser/struct.SequenceRecord.html Returns the line and byte position of the start of the sequence. ```rust pub fn position(&self) -> &Position ``` -------------------------------- ### Get Start Line Number Source: https://docs.rs/needletail/0.7.3/needletail/parser/struct.SequenceRecord.html Returns the line number in the source file where the sequence begins. ```rust pub fn start_line_number(&self) -> u64 ``` -------------------------------- ### Create FastqReader from File Path Source: https://docs.rs/needletail/0.7.3/needletail/parser/struct.FastqReader.html Creates a FastqReader directly from a file path. This is a convenient way to start reading a FASTQ file without manually opening it. ```rust use needletail::parser::{FastxReader, FastqReader}; let mut reader = FastqReader::from_path("seqs.fastq").unwrap(); // (... do something with the reader) ``` -------------------------------- ### Get Record Format Source: https://docs.rs/needletail/0.7.3/needletail/parser/struct.SequenceRecord.html Returns the format (FASTA or FASTQ) of the SequenceRecord. ```rust pub fn format(&self) -> Format ``` -------------------------------- ### Parse FASTA File and Analyze Sequences Source: https://docs.rs/needletail/0.7.3/index.html This example demonstrates how to parse a FASTA file using `needletail`, count the total number of bases, and find specific k-mers like 'AAAA'. It utilizes `parse_fastx_file` for reading and `Sequence` methods for normalization and k-mer analysis. Ensure the file path is valid and the record parsing is handled. ```rust extern crate needletail; use needletail::{parse_fastx_file, Sequence, FastxReader}; fn main() { let filename = "tests/data/28S.fasta"; let mut n_bases = 0; let mut n_valid_kmers = 0; let mut reader = parse_fastx_file(&filename).expect("valid path/file"); while let Some(record) = reader.next() { let seqrec = record.expect("invalid record"); // keep track of the total number of bases n_bases += seqrec.num_bases(); // normalize to make sure all the bases are consistently capitalized and // that we remove the newlines since this is FASTA let norm_seq = seqrec.normalize(false); // we make a reverse complemented copy of the sequence first for // `canonical_kmers` to draw the complemented sequences from. let rc = norm_seq.reverse_complement(); // now we keep track of the number of AAAAs (or TTTTs via // canonicalization) in the file; note we also get the position (i.0; // in the event there were `N`-containing kmers that were skipped) // and whether the sequence was complemented (i.2) in addition to // the canonical kmer (i.1) for (_, kmer, _) in norm_seq.canonical_kmers(4, &rc) { if kmer == b"AAAA" { n_valid_kmers += 1; } } } println!("There are {} bases in your file.", n_bases); println!("There are {} AAAAs in your file.", n_valid_kmers); } ``` -------------------------------- ### Test: Start Line Number Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/record.rs.html Verifies that `start_line_number` correctly reports the line number for consecutive records in a FASTQ input. ```rust #[test] fn test_start_line_number() { let mut reader = parse_fastx_reader(seq(b"@test\nACGT\n+\nIIII\n@test2\nACGT\n+\nIIII")).unwrap(); let rec = reader.next().unwrap().unwrap(); assert_eq!(rec.start_line_number(), 1); let rec = reader.next().unwrap().unwrap(); assert_eq!(rec.start_line_number(), 5); } ``` -------------------------------- ### Initialize FASTQ Reader Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/mod.rs.html Instantiates a FASTQ reader for a given `std::io::Read` object. This is used internally when the input stream starts with '@'. ```rust Ok(Box::new(FastqReader::new(reader))) ``` -------------------------------- ### BufferPosition::all - Get the entire record content Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/fastq.rs.html Returns a slice of the buffer containing the entire current record, from the start position to the end position. ```rust &buffer[self.start..self.end] ``` -------------------------------- ### BufferPosition::new - Needletail Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/fasta.rs.html Initializes a new BufferPosition with a starting index and an empty sequence position vector. ```rust pub(crate) fn new(start: usize) -> Self { BufferPosition { start, seq_pos: Vec::new() } } ``` -------------------------------- ### Get Raw Sequence Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/record.rs.html Returns the raw sequence data. For FASTA, this may include newlines. ```rust pub fn raw_seq(&self) -> &[u8] { match self.buf_pos { BufferPositionKind::Fasta(bp) => bp.raw_seq(self.buffer), BufferPositionKind::Fastq(bp) => bp.seq(self.buffer), } } ``` -------------------------------- ### SequenceRecord Methods Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/record.rs.html Provides access to sequence record metadata such as start line number, position, and line ending. ```rust pub fn start_line_number(&self) -> u64 { self.position.line } ``` ```rust pub fn position(&self) -> &Position { self.position } ``` ```rust pub fn line_ending(&self) -> LineEnding { self.line_ending } ``` -------------------------------- ### Initialize FASTA Reader Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/mod.rs.html Instantiates a FASTA reader for a given `std::io::Read` object. This is used internally when the input stream starts with '>'. ```rust Ok(Box::new(FastaReader::new(reader))) ``` -------------------------------- ### Format enum for FASTA/FASTQ Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/utils.rs.html Distinguishes between FASTA and FASTQ formats. Provides the starting character for each format. ```rust /// FASTA or FASTQ? #[derive(Debug, Copy, Clone, PartialEq, Eq)] pub enum Format { Fasta, Fastq, } impl Format { pub fn start_char(&self) -> char { match self { Self::Fasta => '>', Self::Fastq => '@', } } } ``` -------------------------------- ### Fastq Record Parsing Test Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/fastq.rs.html Tests the parsing of a Fastq record from a byte slice, verifying the start line number of subsequent records. ```rust let test = b"@NCBI actually has files like this\nACGTACGATCGTACGTAGCTGCTAGCTAGCATGCATGACACACACGTACGATCGTACGTAGCTGCTAGCTAGCATGCATGACACAC\n+\n00000000000000000000000000000000000000000000000000000000000000000000000000000000000000\n@NCBI actually has files like this\n\n+\n\n@NCBI actually has files like this\nACGTACGATCGTACGTAGCTGCTAGCTAGCATGCATGACACACACGTACGATCGTACGTAGCTGCTAGCTAGCATGCATGACACAC\n+\n00000000000000000000000000000000000000000000000000000000000000000000000000000000000000"; let mut reader = Reader::new(seq(test)); let rec = reader.next().unwrap(); assert!(rec.is_ok()); let r = rec.unwrap(); assert_eq!(r.start_line_number(), 1); let rec = reader.next().unwrap(); assert!(rec.is_ok()); let r = rec.unwrap(); assert_eq!(r.start_line_number(), 5); let rec = reader.next().unwrap(); assert!(rec.is_ok()); let r = rec.unwrap(); assert_eq!(r.start_line_number(), 9); ``` -------------------------------- ### Initialize BitNuclKmer Iterator Source: https://docs.rs/needletail/0.7.3/src/needletail/bitkmer.rs.html Constructor for the `BitNuclKmer` iterator. Initializes the kmer and starting position based on the provided slice, kmer size, and canonical flag. ```rust pub fn new(slice: &'a [u8], k: u8, canonical: bool) -> BitNuclKmer<'a> { let mut kmer = (0u64, k); let mut start_pos = 0; update_position(&mut start_pos, &mut kmer, slice, true); BitNuclKmer { start_pos, cur_kmer: kmer, buffer: slice, canonical, } } ``` -------------------------------- ### ParseError Constructor: new_invalid_start Source: https://docs.rs/needletail/0.7.3/src/needletail/errors.rs.html Creates a `ParseError` for an invalid start byte encountered when parsing a FASTA or FASTQ file. It specifies the expected and found characters and the position. ```rust pub fn new_invalid_start(byte_found: u8, position: ErrorPosition, format: Format) -> Self { let msg = format!( "Expected '{}' but found '{}'", format.start_char(), (byte_found as char).escape_default() ); Self { kind: ParseErrorKind::InvalidStart, msg, position, format: Some(format), } } ``` -------------------------------- ### Get Full Record Data Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/record.rs.html Returns the entire record's data, excluding any trailing newline. ```rust pub fn all(&self) -> &[u8] { match self.buf_pos { BufferPositionKind::Fasta(bp) => bp.all(self.buffer), BufferPositionKind::Fastq(bp) => bp.all(self.buffer), } } ``` -------------------------------- ### Create ParseError: Invalid Start Byte Source: https://docs.rs/needletail/0.7.3/needletail/errors/struct.ParseError.html Constructs a ParseError when an invalid byte is found at the beginning of a record. Requires the byte found, its position, and the file format. ```rust pub fn new_invalid_start( byte_found: u8, position: ErrorPosition, format: Format, ) -> Self ``` -------------------------------- ### Test Handling of Extra Non-Empty Newlines at End Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/fastq.rs.html Verifies that trailing non-empty newlines that could be mistaken for the start of a new record result in an `InvalidStart` error. ```rust #[test] fn test_extra_non_empty_newlines_at_end_are_not_ok() { let mut reader = Reader::new(seq(b"@test\nAGCT\n+test\n~~a!\n\n@TEST\nA\n+TEST\n~")); let rec = reader.next().unwrap(); assert!(rec.is_ok()); let rec2 = reader.next().unwrap(); let e = rec2.unwrap_err(); assert_eq!(e.kind, ParseErrorKind::InvalidStart); } ``` -------------------------------- ### Get Full Sequence with Line Endings Source: https://docs.rs/needletail/0.7.3/needletail/parser/struct.SequenceRecord.html Returns the entire sequence, including line endings, as a byte slice. Does not include a trailing newline. ```rust pub fn all(&self) -> &[u8] ``` -------------------------------- ### Internal Buffer Management in FastaReader Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/fasta.rs.html These methods manage the internal buffer used for reading FASTA files. `grow` increases buffer capacity, while `make_room` shifts incomplete data to the start for processing. ```rust fn grow(&mut self) { let cap = self.buf_reader.capacity(); let new_size = grow_to(cap); let additional = new_size - cap; self.buf_reader.reserve(additional); } ``` ```rust fn make_room(&mut self) { let consumed = self.buf_pos.start; self.buf_reader.consume(consumed); self.buf_reader.make_room(); self.buf_pos.start = 0; self.search_pos -= consumed; for s in &mut self.buf_pos.seq_pos { *s -= consumed; } } ``` -------------------------------- ### FASTA Reader Initialization and Basic Usage Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/fasta.rs.html Demonstrates initializing the FASTA reader with a byte slice and iterating through records. It also shows how to detect the line ending type used in the FASTA file. ```rust fn seq(s: &[u8]) -> Cursor<&[u8]> { Cursor::new(s) } #[test] fn test_basic() { let mut reader = Reader::new(seq(b">test\nACGT\n>test2\nTGCA\n")); assert!(reader.line_ending().is_none()); let rec = reader.next().unwrap(); assert!(rec.is_ok()); let r = rec.unwrap(); assert_eq!(r.id(), b"test"); assert_eq!(r.raw_seq(), b"ACGT"); assert_eq!(r.all(), b">test\nACGT"); assert_eq!(reader.line_ending().unwrap(), LineEnding::Unix); let rec = reader.next().unwrap(); assert!(rec.is_ok()); let r = rec.unwrap(); assert_eq!(r.id(), b"test2"); assert_eq!(r.raw_seq(), b"TGCA"); assert!(reader.next().is_none()); } ``` -------------------------------- ### Searching for Record Delimiters in FastaReader Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/fasta.rs.html The `_find` method iterates through the buffer to locate newline characters and checks for the start of a new record ('>'). It updates the search position and returns true if a complete record is identified. ```rust fn _find(&mut self) -> bool { let bufsize = self.get_buf().len(); for pos in Memchr::new(b'\n', &self.buf_reader.buffer()[self.search_pos..]) { let pos = self.search_pos + pos; let next_line_start = pos + 1; if next_line_start == bufsize { // cannot check next byte -> treat as incomplete self.search_pos = pos; // make sure last byte is re-searched next time return false; } self.buf_pos.seq_pos.push(pos); if self.get_buf()[next_line_start] == b'>' { // complete record was found self.search_pos = next_line_start; return true; } } // record end not found self.search_pos = bufsize; false } ``` -------------------------------- ### BufferPosition::id - Get the record ID Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/fastq.rs.html Extracts and trims the record identifier from the buffer. Assumes the ID is located between the start of the record and the sequence start. ```rust trim_cr(&buffer[self.start + 1..self.seq - 1]) ``` -------------------------------- ### FastaReader::new Source: https://docs.rs/needletail/0.7.3/needletail/parser/struct.FastaReader.html Creates a new FastaReader with the default buffer size of 64 KiB. ```APIDOC ## FastaReader::new ### Description Creates a new reader with the default buffer size of 64 KiB. ### Signature ```rust pub fn new(reader: R) -> Self ``` ### Example ```rust use needletail::parser::{FastaReader, FastxReader}; let fasta = b">id\nSEQUENCE"; let mut reader = FastaReader::new(&fasta[..]); let record = reader.next().unwrap().unwrap(); assert_eq!(record.id(), b"id") ``` ``` -------------------------------- ### Parse FASTX File Entry Point Source: https://docs.rs/needletail/0.7.3/needletail/parser/fn.parse_fastx_file.html Use this function as the main entry point when reading from a file. It is a shortcut for calling `parse_fastx_reader` with a file object. ```rust pub fn parse_fastx_file>( path: P, ) -> Result, ParseError> ``` -------------------------------- ### BufferPosition::num_bases - Get the number of bases in the sequence Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/fastq.rs.html Calculates the number of bases in the sequence by getting the length of the trimmed sequence data. ```rust self.seq(buffer).len() ``` -------------------------------- ### Create FastqReader with Custom Capacity Source: https://docs.rs/needletail/0.7.3/needletail/parser/struct.FastqReader.html Creates a new FastqReader with a specified buffer capacity. The minimum allowed capacity is 3. This allows for tuning buffer performance based on expected file size or read patterns. ```rust pub fn with_capacity(reader: R, capacity: usize) -> Self ``` -------------------------------- ### Fastq Parser: Validate Record Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/fastq.rs.html Verifies the integrity of a FASTQ record. It checks if the record starts with '@', the separator line starts with '+', and if the sequence and quality lines have the same length. Errors are returned if validation fails. ```rust fn validate(&mut self) -> Result<(), ParseError> { let start_byte = self.get_buf()[self.buf_pos.start]; if start_byte != b'@' { self.finished = true; return Err(ParseError::new_invalid_start( start_byte, self.get_error_pos(0, false), Format::Fastq, )); } let sep_byte = self.get_buf()[self.buf_pos.sep]; if sep_byte != b'+' { self.finished = true; return Err(ParseError::new_invalid_separator( sep_byte, self.get_error_pos(2, true), )); } let buf = self.get_buf(); // We assume we only have ASCII in sequence and quality let seq_len = self.buf_pos.seq(buf).len(); let qual_len = self.buf_pos.qual(buf).len(); // TODO: we don't do that every time because it's a ~90% performance penalty. // TODO: mention it on the README // And we can further validate quality chars // and the vast majority of files don't have this issue // let qual_len = self // .buf_pos // .qual(&buf) // .iter() // .filter(|c| *c >= &b'!' && *c <= &b'~') // .count(); if seq_len != qual_len { self.finished = true; return Err(ParseError::new_unequal_length( seq_len, qual_len, self.get_error_pos(0, true), )); } Ok(()) } ``` -------------------------------- ### Create FastaReader from Path Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/fasta.rs.html Use `from_path` to create a FastaReader instance from a given file path. This function handles file opening and initialization. ```rust use needletail::parser::{FastaReader, FastxReader}; let mut reader = FastaReader::from_path("seqs.fasta").unwrap(); ``` -------------------------------- ### Get Record ID Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/record.rs.html Retrieves the identifier for the sequence record. ```rust pub fn id(&self) -> &[u8] { match self.buf_pos { BufferPositionKind::Fasta(bp) => bp.id(self.buffer), BufferPositionKind::Fastq(bp) => bp.id(self.buffer), } } ``` -------------------------------- ### Create FastqReader with Default Buffer Source: https://docs.rs/needletail/0.7.3/needletail/parser/struct.FastqReader.html Creates a new FastqReader with the default buffer size of 64 KiB. Use this when you have a FASTQ file and want to read it directly without compression. ```rust use needletail::parser::{FastqReader, FastxReader}; let fastq = b"@id\nACGT\n+\nIIII"; let mut reader = FastqReader::new(&fastq[..]); let record = reader.next().unwrap().unwrap(); assert_eq!(record.id(), b"id") ``` -------------------------------- ### Get Record ID Source: https://docs.rs/needletail/0.7.3/needletail/parser/struct.SequenceRecord.html Returns the identifier of the SequenceRecord as a byte slice. ```rust pub fn id(&self) -> &[u8] ``` -------------------------------- ### FastqReader::new Source: https://docs.rs/needletail/0.7.3/needletail/parser/struct.FastqReader.html Creates a new FastqReader with the default buffer size of 64 KiB. This is suitable for uncompressed FASTQ files. ```APIDOC ## FastqReader::new ### Description Creates a new reader with the default buffer size of 64 KiB. ### Method `FastqReader::new(reader: R) -> Self` ### Parameters * `reader`: An object implementing the `Read` trait. ### Example ```rust use needletail::parser::{FastqReader, FastxReader}; let fastq = b"@id\nACGT\n+\nIIII"; let mut reader = FastqReader::new(&fastq[..]); let record = reader.next().unwrap().unwrap(); assert_eq!(record.id(), b"id") ``` ``` -------------------------------- ### Get Number of Bases Source: https://docs.rs/needletail/0.7.3/needletail/parser/struct.SequenceRecord.html Efficiently returns the total number of bases in the sequence. ```rust pub fn num_bases(&self) -> usize ``` -------------------------------- ### Create FastaReader from file path Source: https://docs.rs/needletail/0.7.3/needletail/parser/struct.FastaReader.html Create a FastaReader directly from a file path. This method handles opening and reading from the specified file. It returns a Result, so error handling is required. ```rust use needletail::parser::{FastaReader, FastxReader}; let mut reader = FastaReader::from_path("seqs.fasta").unwrap(); // (... do something with the reader) ``` -------------------------------- ### CloneToUninit Implementation (Nightly) Source: https://docs.rs/needletail/0.7.3/needletail/errors/struct.ErrorPosition.html Nightly-only experimental API for performing copy-assignment from self to an uninitialized destination. ```rust unsafe fn clone_to_uninit(&self, dest: *mut u8) ``` -------------------------------- ### Get Cleaned Sequence Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/record.rs.html Returns the sequence data, with newlines removed for FASTA format. ```rust pub fn seq(&self) -> Cow<'_, [u8]> { match self.buf_pos { BufferPositionKind::Fasta(bp) => bp.seq(self.buffer), BufferPositionKind::Fastq(bp) => bp.seq(self.buffer).into(), } } ``` -------------------------------- ### Parse FASTA/FASTQ and Analyze Sequences Source: https://docs.rs/needletail/0.7.3/src/needletail/lib.rs.html Demonstrates parsing a FASTA file, counting total bases, and identifying specific k-mers (e.g., 'AAAA'). Requires the `needletail` crate and uses `parse_fastx_file` for reading. Normalizes sequences and calculates reverse complements for k-mer analysis. ```rust #![crate_name = "needletail"] //! Needletail is a crate to quickly and easily parse FASTA and FASTQ sequences out of //! streams/files and manipulate and analyse that data. //! //! A contrived example of how to use it: //! ``` //! extern crate needletail; //! use needletail::{parse_fastx_file, Sequence, FastxReader}; //! //! fn main() { //! let filename = "tests/data/28S.fasta"; //! //! let mut n_bases = 0; //! let mut n_valid_kmers = 0; //! let mut reader = parse_fastx_file(&filename).expect("valid path/file"); //! while let Some(record) = reader.next() { //! let seqrec = record.expect("invalid record"); //! // keep track of the total number of bases //! n_bases += seqrec.num_bases(); //! // normalize to make sure all the bases are consistently capitalized and //! // that we remove the newlines since this is FASTA //! let norm_seq = seqrec.normalize(false); //! // we make a reverse complemented copy of the sequence first for //! // `canonical_kmers` to draw the complemented sequences from. //! let rc = norm_seq.reverse_complement(); //! // now we keep track of the number of AAAAs (or TTTTs via //! // canonicalization) in the file; note we also get the position (i.0; //! // in the event there were `N`-containing kmers that were skipped) //! // and whether the sequence was complemented (i.2) in addition to //! // the canonical kmer (i.1) //! for (_, kmer, _) in norm_seq.canonical_kmers(4, &rc) { //! if kmer == b"AAAA" { //! n_valid_kmers += 1; //! } //! } //! } //! println!("There are {} bases in your file.", n_bases); //! println!("There are {} AAAAs in your file.", n_valid_kmers); //! } //! ``` ``` -------------------------------- ### Create FASTQ SequenceRecord Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/record.rs.html Constructs a new SequenceRecord for FASTQ format. Uses Unix line endings by default if not specified. ```rust pub(crate) fn new_fastq( buffer: &'a [u8], buf_pos: &'a FastqBufferPosition, position: &'a Position, line_ending: Option, ) -> Self { Self { buffer, position, buf_pos: BufferPositionKind::Fastq(buf_pos), line_ending: line_ending.unwrap_or(LineEnding::Unix), } } ``` -------------------------------- ### BufferPosition::reset - Needletail Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/fasta.rs.html Resets the BufferPosition to a new starting index, clearing previous sequence positions. ```rust #[inline] fn reset(&mut self, start: usize) { self.seq_pos.clear(); self.start = start; } ``` -------------------------------- ### FastqReader::with_capacity Source: https://docs.rs/needletail/0.7.3/needletail/parser/struct.FastqReader.html Creates a new FastqReader with a specified buffer capacity. The minimum allowed capacity is 3. ```APIDOC ## FastqReader::with_capacity ### Description Creates a new reader with a given buffer capacity. The minimum allowed capacity is 3. ### Method `FastqReader::with_capacity(reader: R, capacity: usize) -> Self` ### Parameters * `reader`: An object implementing the `Read` trait. * `capacity`: The desired buffer capacity for the reader (minimum 3). ``` -------------------------------- ### FastqReader::from_path Source: https://docs.rs/needletail/0.7.3/needletail/parser/struct.FastqReader.html Creates a FastqReader directly from a file path. This method assumes the file is uncompressed FASTQ. ```APIDOC ## FastqReader::from_path ### Description Creates a reader from a file path. This method is part of the `Reader` implementation. ### Method `FastqReader::from_path>(path: P) -> Result` ### Parameters * `path`: A type that can be converted into a `Path` (e.g., a string slice or `PathBuf`). ### Example ```rust use needletail::parser::{FastxReader, FastqReader}; let mut reader = FastqReader::from_path("seqs.fastq").unwrap(); // (... do something with the reader) ``` ``` -------------------------------- ### FastaReader::with_capacity Source: https://docs.rs/needletail/0.7.3/needletail/parser/struct.FastaReader.html Creates a new FastaReader with a specified buffer capacity. The minimum allowed capacity is 3. ```APIDOC ## FastaReader::with_capacity ### Description Creates a new reader with a given buffer capacity. The minimum allowed capacity is 3. ### Signature ```rust pub fn with_capacity(reader: R, capacity: usize) -> Self ``` ``` -------------------------------- ### Get Sequence as Byte Slice Source: https://docs.rs/needletail/0.7.3/needletail/parser/struct.SequenceRecord.html Returns the sequence as a byte slice. This is part of the `Sequence` trait implementation. ```rust fn sequence(&'a self) -> &'a [u8] ``` -------------------------------- ### Get Quality Scores Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/record.rs.html Retrieves the raw Phred-encoded quality scores for FASTQ records. Returns None for FASTA. ```rust pub fn qual(&self) -> Option<&[u8]> { match self.buf_pos { BufferPositionKind::Fasta(_) => None, BufferPositionKind::Fastq(bp) => Some(bp.qual(self.buffer)), } } ``` -------------------------------- ### Get Line Ending Type Source: https://docs.rs/needletail/0.7.3/needletail/parser/struct.SequenceRecord.html Identifies the type of line ending used by the record (e.g., LF, CRLF). ```rust pub fn line_ending(&self) -> LineEnding ``` -------------------------------- ### FastqReader::with_capacity - Create a reader with custom buffer size Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/fastq.rs.html Creates a new reader with a specified buffer capacity. The minimum allowed capacity is 3. Use this to optimize memory usage or performance for specific file sizes. ```rust assert!(capacity >= 3); Self { buf_reader: buffer_redux::BufReader::with_capacity(capacity, reader), buf_pos: BufferPosition::default(), search_pos: SearchPosition::Id, position: Position::new(1, 0), finished: false, line_ending: None, } ``` -------------------------------- ### BitNuclKmer::new Source: https://docs.rs/needletail/0.7.3/needletail/bitkmer/struct.BitNuclKmer.html Constructs a new BitNuclKmer. It takes a slice of bytes, the k-mer size, and a boolean indicating whether to use canonical representation. ```APIDOC ## BitNuclKmer::new ### Description Constructs a new `BitNuclKmer`. ### Signature ```rust pub fn new(slice: &'a [u8], k: u8, canonical: bool) -> BitNuclKmer<'a> ``` ### Parameters * `slice` (&'a [u8]): The input byte slice to process. * `k` (u8): The size of the k-mers to generate. * `canonical` (bool): If true, only canonical k-mers are considered. ``` -------------------------------- ### Get Cleaned Sequence Data Source: https://docs.rs/needletail/0.7.3/needletail/parser/struct.SequenceRecord.html Returns the sequence data with newlines removed. For FASTQ, this is the same as `raw_seq`. For FASTA, it removes `\r\n`. ```rust pub fn seq(&self) -> Cow<'_, [u8]> ``` -------------------------------- ### Fastq Invalid Separator Test Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/fastq.rs.html Tests the parser's error handling for Fastq files where the separator line does not start with '+'. ```rust let mut reader = Reader::from_path("tests/data/random_tsv.fq").unwrap(); let rec = reader.next().unwrap(); assert!(rec.is_ok()); let rec2 = reader.next().unwrap(); let e = rec2.unwrap_err(); assert_eq!(e.kind, ParseErrorKind::InvalidSeparator); ``` -------------------------------- ### Create ParseError: Empty File Source: https://docs.rs/needletail/0.7.3/needletail/errors/struct.ParseError.html Constructs a ParseError specifically for an empty input file. ```rust pub fn new_empty_file() -> Self ``` -------------------------------- ### Create FASTA SequenceRecord Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/record.rs.html Constructs a new SequenceRecord for FASTA format. Uses Unix line endings by default if not specified. ```rust pub(crate) fn new_fasta( buffer: &'a [u8], buf_pos: &'a FastaBufferPosition, position: &'a Position, line_ending: Option, ) -> Self { Self { buffer, position, buf_pos: BufferPositionKind::Fasta(buf_pos), line_ending: line_ending.unwrap_or(LineEnding::Unix), } } ``` -------------------------------- ### Get Reverse Complement of Sequence Source: https://docs.rs/needletail/0.7.3/needletail/parser/struct.SequenceRecord.html Computes and returns the reverse complement of the sequence as a `Vec`. This is part of the `Sequence` trait implementation. ```rust fn reverse_complement(&'a self) -> Vec ``` -------------------------------- ### Read First Byte from Compressed Stream Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/mod.rs.html After initializing a compression decoder (e.g., Gzip, Bzip2, XZ, Zstd), this code reads the very first byte from the decompressed stream to determine the FASTA/FASTQ format. It handles potential `UnexpectedEof` errors, mapping them to `ParseError::new_empty_file()`. ```rust let mut first = [0; 1]; gz_reader .read_exact(&mut first) .map_err(|e| match e.kind() { io::ErrorKind::UnexpectedEof => ParseError::new_empty_file(), _ => e.into(), })?; let r = Cursor::new(first).chain(gz_reader); get_fastx_reader(r, first[0]) ``` ```rust let mut first = [0; 1]; bz_reader .read_exact(&mut first) .map_err(|e| match e.kind() { io::ErrorKind::UnexpectedEof => ParseError::new_empty_file(), _ => e.into(), })?; let r = Cursor::new(first).chain(bz_reader); get_fastx_reader(r, first[0]) ``` ```rust let mut first = [0; 1]; xz_reader .read_exact(&mut first) .map_err(|e| match e.kind() { io::ErrorKind::UnexpectedEof => ParseError::new_empty_file(), _ => e.into(), })?; let r = Cursor::new(first).chain(xz_reader); get_fastx_reader(r, first[0]) ``` -------------------------------- ### FastaReader::new Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/fasta.rs.html Creates a new FASTA reader with the default buffer size (64 KiB). This is suitable for reading FASTA files directly. For compressed files or unknown formats, `parse_fastx_file` is recommended. ```APIDOC ## FastaReader::new ### Description Creates a new reader with the default buffer size of 64 KiB. ### Method `FastaReader::new(reader: R)` ### Parameters - `reader`: An object implementing `std::io::Read`. ### Example ```rust use needletail::parser::{FastaReader, FastxReader}; let fasta = b">id\nSEQUENCE"; let mut reader = FastaReader::new(&fasta[..]); let record = reader.next().unwrap().unwrap(); assert_eq!(record.id(), b"id") ``` ``` -------------------------------- ### FastqReader Source: https://docs.rs/needletail/0.7.3/needletail/parser/index.html Direct parser for FASTQ files. Use this only if the file is confirmed to be FASTQ and uncompressed. For general use, `parse_fastx_file` is recommended. ```APIDOC ## Struct FastqReader Parser for FASTQ files. Only use this directly if you know your file is FASTQ and that it is not compressed as it does not handle decompression. If you are unsure, it’s better to use `parse_fastx_file`. ``` -------------------------------- ### grow_to Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/utils.rs.html Determines the next buffer size, doubling the current size until it reaches 8 MiB, after which it increases in fixed 8 MiB increments. This implements a dynamic buffer growth strategy. ```APIDOC ## grow_to ### Description Calculates the next buffer size based on a dynamic growth strategy. ### Function Signature `pub(crate) fn grow_to(current_size: usize) -> usize` ### Parameters * `current_size` (*usize*) - The current size of the buffer. ### Returns * (*usize*) - The recommended next buffer size. ``` -------------------------------- ### Get Raw Sequence Data Source: https://docs.rs/needletail/0.7.3/needletail/parser/struct.SequenceRecord.html Returns the raw sequence data, including any newlines, as a byte slice. This is particularly relevant for FASTA format. ```rust pub fn raw_seq(&self) -> &[u8] ``` -------------------------------- ### partition Source: https://docs.rs/needletail/0.7.3/needletail/kmer/struct.Kmers.html Consumes an iterator, creating two collections from it. ```APIDOC ## fn partition(self, f: F) -> (B, B) ### Description Consumes an iterator, creating two collections from it. ### Method `partition` ### Parameters #### Path Parameters - **f** (F) - Required - A closure that returns `true` for elements to be placed in the first collection, and `false` for the second. ``` -------------------------------- ### FastaReader::from_path Source: https://docs.rs/needletail/0.7.3/needletail/parser/struct.FastaReader.html Creates a FastaReader from a file path. This implementation is specific to `File` readers. ```APIDOC ## FastaReader::from_path ### Description Creates a reader from a file path. ### Signature ```rust pub fn from_path>(path: P) -> Result ``` ### Example ```rust use needletail::parser::{FastaReader, FastxReader}; let mut reader = FastaReader::from_path("seqs.fasta").unwrap(); // (... do something with the reader) ``` ``` -------------------------------- ### FASTA Reader with Wrapped Sequences Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/fasta.rs.html Illustrates how the FASTA reader handles sequences that span multiple lines. It verifies that wrapped sequences are correctly concatenated and that the total number of bases is accurate. ```rust fn seq(s: &[u8]) -> Cursor<&[u8]> { Cursor::new(s) } #[test] fn test_wrapped_fasta() { let mut reader = Reader::new(seq(b">test\nACGT\nACGT\n>test2\nTGCA\nTG")); let rec = reader.next().unwrap(); assert!(rec.is_ok()); let r = rec.unwrap(); assert_eq!(r.id(), b"test"); assert_eq!(r.raw_seq(), b"ACGT\nACGT"); assert_eq!(r.num_bases(), 8); assert_eq!(reader.line_ending().unwrap(), LineEnding::Unix); let rec = reader.next().unwrap(); assert!(rec.is_ok()); let r = rec.unwrap(); assert_eq!(r.id(), b"test2"); assert_eq!(r.raw_seq(), b"TGCA\nTG"); assert_eq!(r.num_bases(), 6); assert!(reader.next().is_none()); } ``` -------------------------------- ### Create FastaReader from byte slice Source: https://docs.rs/needletail/0.7.3/needletail/parser/struct.FastaReader.html Instantiate a FastaReader with a byte slice. This is useful for in-memory FASTA data. Ensure the data is valid FASTA format. ```rust use needletail::parser::{FastaReader, FastxReader}; let fasta = b">id\nSEQUENCE"; let mut reader = FastaReader::new(&fasta[..]); let record = reader.next().unwrap().unwrap(); assert_eq!(record.id(), b"id") ``` -------------------------------- ### Parse FASTX Reader with Compression Handling Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/mod.rs.html Handles different compression formats (Bzip2, Gzip, XZ, Zstd) when parsing FASTX data from a reader. It detects the compression type based on magic bytes and decodes accordingly. ```rust match first_two_bytes { [0x1f, 0x8b] => { let mut gz_reader = GzDecoder::new(new_reader); let mut first = [0; 1]; gz_reader.read_exact(&mut first).map_err(|e| match e.kind() { io::ErrorKind::UnexpectedEof => ParseError::new_empty_file(), _ => e.into(), })?; let r = Cursor::new(first).chain(gz_reader); get_fastx_reader(r, first[0]) } #[cfg(feature = "bzip2")] [0x42, 0x5a] => { let mut bz_reader = BzDecoder::new(new_reader); let mut first = [0; 1]; bz_reader.read_exact(&mut first).map_err(|e| match e.kind() { io::ErrorKind::UnexpectedEof => ParseError::new_empty_file(), _ => e.into(), })?; let r = Cursor::new(first).chain(bz_reader); get_fastx_reader(r, first[0]) } #[cfg(feature = "xz2")] [0xfd, 0x37] => { let mut xz_reader = XzDecoder::new(new_reader); let mut first = [0; 1]; xz_reader.read_exact(&mut first).map_err(|e| match e.kind() { io::ErrorKind::UnexpectedEof => ParseError::new_empty_file(), _ => e.into(), })?; let r = Cursor::new(first).chain(xz_reader); get_fastx_reader(r, first[0]) } #[cfg(feature = "zstd")] ZST_MAGIC => { let mut zst_reader = ZstdDecoder::new(new_reader)?; let mut first = [0; 1]; zst_reader .read_exact(&mut first) .map_err(|e| match e.kind() { io::ErrorKind::UnexpectedEof => ParseError::new_empty_file(), _ => e.into(), })?; let r = Cursor::new(first).chain(zst_reader); get_fastx_reader(r, first[0]) } _ => get_fastx_reader(new_reader, first_two_bytes[0]), } } ``` -------------------------------- ### Parse FASTX from Standard Input Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/mod.rs.html Provides a convenient entry point for parsing FASTX data directly from standard input. It's a shortcut for calling `parse_fastx_reader` with `stdin()`. ```rust pub fn parse_fastx_stdin() -> Result, ParseError> { let stdin = stdin(); parse_fastx_reader(stdin) } ``` -------------------------------- ### Get Quality Scores Source: https://docs.rs/needletail/0.7.3/needletail/parser/struct.SequenceRecord.html Returns the Phred-encoded quality scores as an Option of a byte slice. This is always `Some` for FASTQ records, even if empty, and `None` for FASTA records. ```rust pub fn qual(&self) -> Option<&[u8]> ``` -------------------------------- ### Fastq Parser: Get Error Position Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/fastq.rs.html Retrieves the error position for a FASTQ parsing error, including the line number and optionally the sequence ID if available and requested. ```rust fn get_error_pos(&self, line_offset: u64, parse_id: bool) -> ErrorPosition { let id = if parse_id && self.buf_pos.seq - self.buf_pos.start > 1 { let id = self .buf_pos .id(self.get_buf()) .split(|b| *b == b' ') .next() .unwrap(); Some(String::from_utf8_lossy(id).into()) } else { None }; ErrorPosition { line: self.position.line() + line_offset, id, } } ``` -------------------------------- ### BufferPosition::seq - Get the sequence data Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/fastq.rs.html Extracts and trims the sequence data from the buffer. Assumes the sequence is located between the ID line and the quality score separator line. ```rust trim_cr(&buffer[self.seq..self.sep - 1]) ``` -------------------------------- ### Test Simple FASTQ Parsing with Different Line Endings Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/fastq.rs.html Tests the `Reader`'s ability to parse FASTQ records with both Unix (\n) and Windows (\r\n) line endings. It asserts the correctness of parsed IDs, sequences, quality scores, and line endings. ```rust #[test] fn test_simple_fastq() { // Test both line endings let sequences = vec![ ( "@test\nAGCT\n+test\n~~a!\n@test2\nTGCA\n+test\nWUI9", LineEnding::Unix, ), ( "@test\r\nAGCT\r\n+test\r\n~~a!\r\n@test2\r\nTGCA\r\n+test\r\nWUI9", LineEnding::Windows, ), ]; for (sequence, line_ending) in sequences { let mut i = 0; let mut reader = Reader::new(seq(sequence.as_bytes())); while let Some(record) = reader.next() { let rec = record.unwrap(); match i { 0 => { assert_eq!(&rec.id(), b"test"); assert_eq!(&rec.raw_seq(), b"AGCT"); assert_eq!(&rec.qual().unwrap(), b"~~a!"); assert_eq!(reader.line_ending().unwrap(), line_ending); } 1 => { assert_eq!(&rec.id(), b"test2"); assert_eq!(&rec.raw_seq(), b"TGCA"); assert_eq!(&rec.qual().unwrap(), b"WUI9"); assert_eq!(reader.line_ending().unwrap(), line_ending); } _ => unreachable!("Too many records"), } i += 1; } assert_eq!(i, 2); } } ``` -------------------------------- ### PartialEq Implementation for LineEnding Source: https://docs.rs/needletail/0.7.3/needletail/parser/enum.LineEnding.html Allows comparison of LineEnding variants for equality. Use this to check if two line ending values are the same. ```rust fn eq(&self, other: &LineEnding) -> bool ``` -------------------------------- ### BitNuclKmer Iterator Next Item Source: https://docs.rs/needletail/0.7.3/src/needletail/bitkmer.rs.html Implements the `Iterator` trait for `BitNuclKmer`. Returns the next kmer along with its starting position and a boolean indicating if it was reverse complemented (if `canonical` is true). ```rust fn next(&mut self) -> Option<(usize, BitKmer, bool)> { if !update_position(&mut self.start_pos, &mut self.cur_kmer, self.buffer, false) { return None; } self.start_pos += 1; if self.canonical { let (kmer, was_rc) = canonical(self.cur_kmer); Some((self.start_pos - 1, kmer, was_rc)) } else { Some((self.start_pos - 1, self.cur_kmer, false)) } } ``` -------------------------------- ### BufferPosition::qual - Get the quality scores Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/fastq.rs.html Extracts and trims the quality scores from the buffer. Assumes the quality scores are located between the quality score separator line and the end of the record. ```rust trim_cr(&buffer[self.qual..self.end]) ``` -------------------------------- ### take Source: https://docs.rs/needletail/0.7.3/needletail/kmer/struct.CanonicalKmers.html Creates an iterator that yields the first `n` elements. If the underlying iterator ends before yielding `n` elements, this iterator will yield fewer than `n` elements. ```APIDOC ## fn take(self, n: usize) -> Take where Self: Sized, Creates an iterator that yields the first `n` elements, or fewer if the underlying iterator ends sooner. ``` -------------------------------- ### FastxReader trait definition Source: https://docs.rs/needletail/0.7.3/src/needletail/parser/utils.rs.html Defines the core iterator-like interface for FASTA and FASTQ readers. Provides methods to get the next record, current position, and line ending style. ```rust /// The main trait, iterator-like, that the FASTA and FASTQ readers implement pub trait FastxReader: Send { /// Gets the next record in the stream. /// This imitates the Iterator API but does not support any iterator functions. /// This returns None once we reached the EOF. fn next(&mut self) -> Option, ParseError>>; /// Returns the current line/byte in the stream we are reading from fn position(&self) -> &Position; /// Returns whether the current stream uses Windows or Unix style line endings /// It is `None` only before calling `next`, once `next` has been called it will always /// return a line ending. fn line_ending(&self) -> Option; } ``` -------------------------------- ### Kmers Iterator step_by() Method Source: https://docs.rs/needletail/0.7.3/needletail/kmer/struct.Kmers.html Creates a new iterator that yields elements from the original iterator, but steps by the specified amount at each iteration. Requires the iterator to be Sized. ```rust fn step_by(self, step: usize) -> StepBy where Self: Sized, ``` -------------------------------- ### ParseErrorKind Enum Source: https://docs.rs/needletail/0.7.3/src/needletail/errors.rs.html Enumerates the different types of parsing errors that can occur, such as I/O errors, unknown formats, invalid record starts, unequal lengths in FASTQ files, and unexpected end of input. ```rust pub enum ParseErrorKind { /// An error happened during file/stream input/output Io, /// The file didn't start with `@` or `>` and we didn't know what to expect yet UnknownFormat, /// Invalid start byte of record encountered (expected `@` in FASTQ and `>` in FASTA) InvalidStart, /// The separator line in a FASTQ file is not valid (no `+`) InvalidSeparator, /// Sequence and quality lengths are not equal (in a FASTQ file only) UnequalLengths, /// Truncated record found UnexpectedEnd, /// The file appears to be empty EmptyFile, } ``` -------------------------------- ### parse_fastx_file Source: https://docs.rs/needletail/0.7.3/needletail/parser/index.html The main entry point for parsing FASTA/FASTQ files. This function handles file reading and automatically detects the format. ```APIDOC ## Function parse_fastx_file The main entry point of needletail if you’re reading from a file. Shortcut to calling `parse_fastx_reader` with a file ```