### Usage Guide and Examples Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/DOCUMENTATION_MANIFEST.txt Provides practical guidance and examples for using the Daachorse library in various scenarios. ```APIDOC ## Usage Guide and Examples ### Description This section serves as a practical guide to using the Daachorse library, offering examples for common use cases and advanced patterns. It covers construction, search strategies, streaming input, serialization, and configuration tuning. ### Key Topics - Quick start examples. - Basic and CJK pattern matching. - Four construction patterns explained. - Four search strategy examples. - Streaming input handling. - Serialization for saving and loading automata. - Error handling patterns. - Four advanced usage patterns. - Configuration tuning for performance. - Decision guide for byte-wise vs. character-wise matching. - Memory and performance checking. - Five edge case examples. ``` -------------------------------- ### Quick Start: DoubleArrayAhoCorasick Example Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/README.md Demonstrates basic usage of DoubleArrayAhoCorasick for pattern matching. Requires the daachorse crate and Rust 1.77+. ```rust use daachorse::DoubleArrayAhoCorasick; fn main() -> daachorse::Result<()> { let patterns = vec!["apple", "application", "apply"]; let pma = DoubleArrayAhoCorasick::new(patterns)?; for m in pma.find_iter("apple application apply") { println!("Pattern {}: {}..{}", m.value(), m.start(), m.end()); } Ok(()) } ``` -------------------------------- ### Inline Method for Start Index Calculation Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/09-implementation-notes.md This method calculates the starting index of a pattern. It is marked with `#[inline(always)]` to ensure it's compiled inline, reducing call overhead for this trivial operation. ```rust #[inline(always)] pub fn start(&self) -> usize { self.end - self.length } ``` -------------------------------- ### Standard MatchKind Example Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/04-charwise-builder-and-match.md Demonstrates the Standard match kind, which reports the first matching pattern at each search position. This is the default behavior. ```rust let patterns = vec!["ab", "a", "abcd"]; let pma = DoubleArrayAhoCorasickBuilder::new() .match_kind(MatchKind::Standard) // Default .build(&patterns)?; let mut it = pma.find_iter("abcd"); let m = it.next().unwrap(); assert_eq!((0, 2, 1), (m.start(), m.end(), m.value())); // "ab" (first match) ``` -------------------------------- ### Serialization Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/00-api-overview.md Provides examples for saving (serializing) an automaton to bytes and loading (deserializing) it back, including an unsafe option for unchecked deserialization. ```APIDOC ## Serialization ### Save ```rust let bytes = pma.serialize(); fs::write("automaton.bin", &bytes)?; ``` ### Load ```rust let (pma, _) = DoubleArrayAhoCorasick::deserialize(&bytes)?; ``` ### Load unchecked (unsafe) ```rust let (pma, _) = unsafe { DoubleArrayAhoCorasick::deserialize_unchecked(&bytes) }; ``` ``` -------------------------------- ### Standard MatchKind Example Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/00-api-overview.md Demonstrates the default Standard Aho-Corasick behavior for non-overlapping matches. Useful for general pattern searching. ```rust let patterns = vec!["a", "ab", "ba"]; let pma = DoubleArrayAhoCorasick::new(patterns)?; let matches: Vec<_> = pma.find_iter("aba").collect(); // [(0,1,0), (2,3,1)] - first match at each position ``` -------------------------------- ### Create New CharwiseDoubleArrayAhoCorasickBuilder Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/04-charwise-builder-and-match.md Instantiate a new builder with default settings to begin constructing an automaton. This is the starting point for configuring custom matching behaviors. ```rust use daachorse::CharwiseDoubleArrayAhoCorasickBuilder; let builder = CharwiseDoubleArrayAhoCorasickBuilder::new(); let patterns = vec!["全世界", "世界", "に"]; let pma = builder.build(patterns)?; ``` -------------------------------- ### LeftmostFirst Match Example Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/01-double-array-aho-corasick.md Demonstrates finding the earliest registered matching pattern from the beginning of the text using `MatchKind::LeftmostFirst`. Requires `daachorse` and `MatchKind` imports. ```rust use daachorse::{DoubleArrayAhoCorasickBuilder, MatchKind}; let patterns = vec!["ab", "a", "abcd"]; let pma = DoubleArrayAhoCorasickBuilder::new() .match_kind(MatchKind::LeftmostFirst) .build(&patterns)?; let mut it = pma.leftmost_find_iter("abcd"); let m = it.next().unwrap(); assert_eq!((0, 2, 0), (m.start(), m.end(), m.value())); // "ab" (registered first) assert_eq!(None, it.next()); ``` -------------------------------- ### Get Match Start Position Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/04-charwise-builder-and-match.md Retrieves the starting byte position of a match within the haystack. Note that for character-wise automata, this is a byte offset, not a character index. ```rust let patterns = vec!["bcd", "ab", "a"]; let pma = DoubleArrayAhoCorasick::new(patterns)?; let mut it = pma.find_iter("abcd"); let m = it.next().unwrap(); assert_eq!(0, m.start()); ``` -------------------------------- ### Create a new DoubleArrayAhoCorasickBuilder Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/02-double-array-aho-corasick-builder.md Instantiate a builder with default settings for constructing a DoubleArrayAhoCorasick automaton. This is the starting point for configuring custom automata. ```rust use daachorse::DoubleArrayAhoCorasickBuilder; let builder = DoubleArrayAhoCorasickBuilder::new(); let patterns = vec!["bcd", "ab", "a"]; let pma = builder.build(patterns)?; ``` -------------------------------- ### Charwise Aho-Corasick Byte Position Semantics Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/03-charwise-double-array-aho-corasick.md Demonstrates how match results for CharwiseDoubleArrayAhoCorasick use byte offsets in UTF-8 haystacks. The example shows finding a pattern and asserting the start and end byte positions of the match. ```rust let pma = CharwiseDoubleArrayAhoCorasick::new(vec!["全世界"])?; let m = pma.find_iter("全世界").next().unwrap(); assert_eq!(m.start(), 0); // Byte offset 0 assert_eq!(m.end(), 9); // Byte offset 9 (3 chars × 3 bytes each) ``` -------------------------------- ### serialized_bytes Implementation for Empty Type Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/06-serialization-and-traits.md Example implementation of `serialized_bytes` for a type that serializes to zero bytes. This is useful for types that do not require storage. ```rust fn serialized_bytes() -> usize { 0 // No bytes } ``` -------------------------------- ### serialized_bytes Implementation for u32 Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/06-serialization-and-traits.md Example implementation of `serialized_bytes` for `u32`. It returns the size of a `u32` in bytes, which is typically 4. ```rust fn serialized_bytes() -> usize { core::mem::size_of::() // 4 bytes } ``` -------------------------------- ### LeftmostFirst MatchKind Example Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/00-api-overview.md Shows the LeftmostFirst greedy matching strategy, which selects the earliest registered pattern among longest matches. Useful for priority-based matching. ```rust let patterns = vec!["ab", "a", "abcd"]; let pma = DoubleArrayAhoCorasickBuilder::new() .match_kind(MatchKind::LeftmostFirst) .build(&patterns)?; let matches: Vec<_> = pma.leftmost_find_iter("abcd").collect(); // [(0,2,0), (2,3,1)] - matches "ab" (registered first), then "c" ``` -------------------------------- ### FindIterator Example Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/07-iterators.md Iterates over non-overlapping matches in a haystack. Reports the first match at each position and advances to the end of the found pattern. Efficient for linear-time traversal. ```rust use daachorse::DoubleArrayAhoCorasick; let patterns = vec!["bcd", "ab", "a"]; let pma = DoubleArrayAhoCorasick::new(patterns)?; for m in pma.find_iter("abcd") { println!("{}..{}: {}", m.start(), m.end(), m.value()); } // Output: // 0..1: 2 // 1..4: 0 ``` -------------------------------- ### LeftmostLongest Match Example Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/01-double-array-aho-corasick.md Demonstrates finding the longest matching pattern from the beginning of the text using `MatchKind::LeftmostLongest`. Requires `daachorse` and `MatchKind` imports. ```rust use daachorse::{DoubleArrayAhoCorasickBuilder, MatchKind}; let patterns = vec!["ab", "a", "abcd"]; let pma = DoubleArrayAhoCorasickBuilder::new() .match_kind(MatchKind::LeftmostLongest) .build(&patterns)?; let mut it = pma.leftmost_find_iter("abcd"); let m = it.next().unwrap(); assert_eq!((0, 4, 2), (m.start(), m.end(), m.value())); // "abcd" (longest) assert_eq!(None, it.next()); ``` -------------------------------- ### Standard (Non-Overlapping) Search Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/08-usage-guide-and-examples.md Use `find_iter` for standard matching, which reports the first matching pattern at each position in the text. This is suitable when you only need one match per starting point. ```rust use daachorse::DoubleArrayAhoCorasick; let patterns = vec!["a", "aa", "aaa"]; let pma = DoubleArrayAhoCorasick::new(patterns)?; let matches: Vec<_> = pma.find_iter("aaaa").collect(); // Output: [(0,1,0), (1,2,0), (2,3,0), (3,4,0)] // Matches first occurrence at each position ``` -------------------------------- ### Unsafe Deserialization Example Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/06-serialization-and-traits.md Demonstrates deserializing data using `deserialize_unchecked`. This method skips validation for maximum performance and should only be used with trusted data sources. ```rust let pma = DoubleArrayAhoCorasick::::new(vec!["a", "b"])?; let bytes = pma.serialize(); // Safe: data from serialize() let (restored, _) = unsafe { DoubleArrayAhoCorasick::::deserialize_unchecked(&bytes) }; ``` -------------------------------- ### LeftmostFirst Match Iterator Example Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/07-iterators.md Shows how to find the earliest-registered pattern at each leftmost position using `leftmost_find_iter` with `MatchKind::LeftmostFirst`. Patterns are built using `DoubleArrayAhoCorasickBuilder`. ```rust use daachorse::{DoubleArrayAhoCorasickBuilder, MatchKind}; let patterns = vec!["ab", "a", "abcd"]; let pma = DoubleArrayAhoCorasickBuilder::new() .match_kind(MatchKind::LeftmostFirst) .build(&patterns)?; let matches: Vec<_> = pma.leftmost_find_iter("abcd").collect(); assert_eq!(2, matches.len()); assert_eq!((0, 2, 0), (matches[0].start(), matches[0].end(), matches[0].value())); // "ab" (index 0) assert_eq!((2, 4, 1), (matches[1].start(), matches[1].end(), matches[1].value())); // "a" (index 1) ``` -------------------------------- ### FindOverlappingIterator Example Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/07-iterators.md Iterates over all overlapping matches ending at each position. Matches are ordered by descending length, with duplicates reported in registration order. Advances per match, not per position. ```rust use daachorse::DoubleArrayAhoCorasick; let patterns = vec!["bcd", "ab", "a"]; let pma = DoubleArrayAhoCorasick::new(patterns)?; let matches: Vec<_> = pma.find_overlapping_iter("abcd").collect(); assert_eq!(3, matches.len()); // Position 1 (after "a"): assert_eq!((0, 1, 2), (matches[0].start(), matches[0].end(), matches[0].value())); // "a" // Position 2 (after "ab"): assert_eq!((0, 2, 1), (matches[1].start(), matches[1].end(), matches[1].value())); // "ab" // Position 4 (after "bcd"): assert_eq!((1, 4, 0), (matches[2].start(), matches[2].end(), matches[2].value())); // "bcd" ``` -------------------------------- ### deserialize_from_slice Implementation for u32 Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/06-serialization-and-traits.md Example implementation of `deserialize_from_slice` for `u32`. It attempts to read 4 bytes from the source slice and convert them into a `u32` value, returning the deserialized value and the remaining slice. ```rust fn deserialize_from_slice(src: &[u8]) -> Result<(Self, &[u8])> { let (&bytes, rest) = src.split_first_chunk() .ok_or(DaachorseError::invalid_automaton())?; Ok((u32::from_le_bytes(bytes), rest)) } ``` -------------------------------- ### Handle Construction Errors with Match Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/08-usage-guide-and-examples.md Handle potential errors during automaton construction using a `match` statement. This example demonstrates matching specific `DaachorseError` variants like `InvalidConversion` and `AutomatonScale`. ```rust use daachorse::{DoubleArrayAhoCorasick, DaachorseError}; let result = DoubleArrayAhoCorasick::::new((0..300).map(|i| format!("{}", i))); match result { Ok(pma) => println!("Built automaton"), Err(DaachorseError::InvalidConversion(e)) => { eprintln!("Value type too small: {}", e); } Err(DaachorseError::AutomatonScale(e)) => { eprintln!("Automaton too large: {}", e); } Err(e) => eprintln!("Other error: {}", e), } ``` -------------------------------- ### Match Result Properties Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/00-api-overview.md Access properties of a match result to get its start and end byte offsets, and the associated value. ```rust let m: Match; m.start() // usize - beginning byte offset m.end() // usize - ending byte offset (exclusive) m.value() // V - associated value ``` -------------------------------- ### Builder Pattern Configuration Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/DOCUMENTATION_MANIFEST.txt General documentation on the builder pattern used for configuration. ```APIDOC ## Builder Pattern Configuration This section covers the general methods and configurations available through the builder pattern for constructing automata. ``` -------------------------------- ### Get Match End Position Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/04-charwise-builder-and-match.md Retrieves the ending byte position (exclusive) of a match within the haystack. This is calculated as `start + pattern.len()`. ```rust let patterns = vec!["bcd", "ab", "a"]; let pma = DoubleArrayAhoCorasick::new(patterns)?; let mut it = pma.find_iter("abcd"); let m = it.next().unwrap(); assert_eq!(1, m.end()); ``` -------------------------------- ### Tuning Options Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/DOCUMENTATION_MANIFEST.txt Documentation on tuning parameters like num_free_blocks. ```APIDOC ## Tuning Options - `num_free_blocks`: A parameter that can be tuned to optimize automaton performance. ``` -------------------------------- ### LeftmostLongest MatchKind Example Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/04-charwise-builder-and-match.md Illustrates the LeftmostLongest match kind, which reports the longest pattern among those starting at the same position. Requires using `leftmost_find_iter`. ```rust use daachorse::{DoubleArrayAhoCorasickBuilder, MatchKind}; let patterns = vec!["ab", "a", "abcd"]; let pma = DoubleArrayAhoCorasickBuilder::new() .match_kind(MatchKind::LeftmostLongest) .build(&patterns)?; let mut it = pma.leftmost_find_iter("abcd"); let m = it.next().unwrap(); assert_eq!((0, 4, 2), (m.start(), m.end(), m.value())); // "abcd" (longest) ``` -------------------------------- ### Double-Array Conversion Algorithm Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/09-implementation-notes.md Illustrates the core logic for finding a suitable base value during the double-array conversion phase, ensuring no collisions with outgoing transitions. ```pseudocode for each NFA state s: labels = outgoing transition labels from s base = find_first_base(labels) for each label c: da_index = base XOR c da[da_index].check = c da[da_index].next_state = ... ``` -------------------------------- ### Map Iterator Results Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/07-iterators.md Transform the elements yielded by an iterator using `map()`. This example transforms each match into a tuple of its start and end positions. ```rust let positions: Vec<_> = pma.find_iter("abcd") .map(|m| (m.start(), m.end())) .collect(); ``` -------------------------------- ### Match Implementations (Copy, Clone, Debug, Eq, Hash, PartialEq) Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/04-charwise-builder-and-match.md Demonstrates that the Match struct implements common traits like Copy, Clone, Debug, Eq, Hash, and PartialEq, allowing for easy comparison and debugging. ```rust let patterns = vec!["test"]; let pma = DoubleArrayAhoCorasick::new(patterns)?; let match1 = pma.find_iter("test").next().unwrap(); let match2 = match1; // Copy assert_eq!(match1, match2); // Eq/PartialEq assert_eq!(format!("{:?}", match1), "Match { ... }"); // Debug ``` -------------------------------- ### LeftmostLongest Match Iterator Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/03-charwise-double-array-aho-corasick.md Use `leftmost_find_iter` with `MatchKind::LeftmostLongest` to find the longest possible match starting from the earliest position in the haystack. This example demonstrates finding the longest match for patterns 'ab' and 'abc' in the string 'abc'. ```rust use daachorse::{CharwiseDoubleArrayAhoCorasickBuilder, MatchKind}; let patterns = vec!["ab", "abc"]; let pma = CharwiseDoubleArrayAhoCorasickBuilder::new() .match_kind(MatchKind::LeftmostLongest) .build(&patterns)?; let mut it = pma.leftmost_find_iter("abc"); let m = it.next().unwrap(); assert_eq!((0, 3, 1), (m.start(), m.end(), m.value())); // "abc" (longest) ``` -------------------------------- ### Constructors Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/README.md Provides methods for creating new DoubleArrayAhoCorasick and CharwiseDoubleArrayAhoCorasick instances, including options for custom builders. ```APIDOC ## Constructors ### Byte-wise Construction - `DoubleArrayAhoCorasick::new(patterns)`: Creates a new automaton from a collection of patterns. - `DoubleArrayAhoCorasick::with_values(patvals)`: Creates a new automaton with associated values for each pattern. ### Builder Pattern (Byte-wise) - `DoubleArrayAhoCorasickBuilder::new()`: Initializes a builder. - `.match_kind(MatchKind)`: Sets the matching strategy. - `.num_free_blocks(u32)`: Configures the number of free blocks. - `.build(patterns)`: Constructs the automaton from patterns using the builder configuration. ### Character-wise Construction - `CharwiseDoubleArrayAhoCorasick::new(patterns)`: Creates a new character-wise automaton. - `CharwiseDoubleArrayAhoCorasick::with_values(patvals)`: Creates a new character-wise automaton with associated values. ### Builder Pattern (Character-wise) - `CharwiseDoubleArrayAhoCorasickBuilder::new()`: Initializes a character-wise builder. - `.match_kind(MatchKind)`: Sets the matching strategy. - `.build(patterns)`: Constructs the character-wise automaton. ``` -------------------------------- ### FindStepper and FindOverlappingStepper Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/DOCUMENTATION_MANIFEST.txt Documentation for the stateful stepper consumers. ```APIDOC ## Stateful Stepper Consumers - `FindStepper`: A stateful consumer for iterating through matches. - `FindOverlappingStepper`: A stateful consumer for iterating through overlapping matches. ``` -------------------------------- ### find_iter_from_iter Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/01-double-array-aho-corasick.md Performs a non-overlapping search starting from a byte iterator, utilizing the same algorithm as `find_iter`. ```APIDOC ## find_iter_from_iter(haystack) -> FindIterator ### Description Non-overlapping search from a byte iterator (same algorithm as `find_iter`). ### Method `find_iter_from_iter` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Parameters - **haystack** (`P: Iterator`) - Required - Iterator of u8 bytes ### Return `FindIterator<'_, P, V>` ### Panics If `MatchKind` is not `Standard` ``` -------------------------------- ### find_overlapping_no_suffix_iter_from_iter Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/01-double-array-aho-corasick.md Provides an iterator for finding overlapping matches without considering suffixes, starting from a byte iterator. ```APIDOC ## `find_overlapping_no_suffix_iter_from_iter(haystack)` ### Description Overlapping no-suffix search from a byte iterator. ### Method `find_overlapping_no_suffix_iter_from_iter` ### Parameters #### Path Parameters - **haystack** (`P: Iterator`) - Required - Iterator of u8 bytes ### Return `FindOverlappingNoSuffixIterator<'_, P, V>` ### Panics If `MatchKind` is not `Standard` ``` -------------------------------- ### Keyword Highlighting with Daachorse Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/08-usage-guide-and-examples.md Demonstrates a practical application of Daachorse for keyword highlighting by iterating through matches and reconstructing the text with added markers. ```rust use daachorse::DoubleArrayAhoCorasick; let keywords = vec!["Rust", "programming", "language"]; let pma = DoubleArrayAhoCorasick::new(keywords)?; let text = "Rust is a programming language"; let mut last = 0; let mut result = String::new(); for m in pma.find_iter(text) { result.push_str(&text[last..m.start()]); result.push_str("**"); result.push_str(&text[m.start()..m.end()]); result.push_str("**"); last = m.end(); } result.push_str(&text[last..]); println!("{}", result); // Output: **Rust** is a **programming** **language** ``` -------------------------------- ### Match Result Extraction Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/README.md Extracts start, end, and value from a match result within a `match` expression. ```rust match m { _ => { let start: usize = m.start(); let end: usize = m.end(); let value: V = m.value(); } } ``` -------------------------------- ### CharwiseDoubleArrayAhoCorasickBuilder Configuration Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/DOCUMENTATION_MANIFEST.txt Configuration methods for building a CharwiseDoubleArrayAhoCorasick automaton. ```APIDOC ## CharwiseDoubleArrayAhoCorasickBuilder Configuration Methods for configuring and building a `CharwiseDoubleArrayAhoCorasick` automaton. ``` -------------------------------- ### Get Number of States Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/03-charwise-double-array-aho-corasick.md Retrieve the total number of states in the automaton. Useful for understanding the complexity of the pattern set. ```rust use daachorse::CharwiseDoubleArrayAhoCorasick; let patterns = vec!["全世界", "世界", "に"]; let pma = CharwiseDoubleArrayAhoCorasick::::new(patterns)?; println!("States: {}", pma.num_states()); ``` -------------------------------- ### MatchKind Options Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/README.md Enumerates the different matching strategies available for the automaton. ```APIDOC ## MatchKind Options | Variant | Associated Method | Behavior | |-------------------|---------------------------|-------------------------------------------| | `Standard` | `find_iter()` | Standard Aho-Corasick matching behavior. | | `LeftmostLongest` | `leftmost_find_iter()` | Finds the longest match at each position. | | `LeftmostFirst` | `leftmost_find_iter()` | Finds the earliest-registered match at each position. | ``` -------------------------------- ### Handle Empty Pattern in Daachorse Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/08-usage-guide-and-examples.md Shows how to handle an empty pattern. An empty pattern matches at every boundary. ```rust use daachorse::DoubleArrayAhoCorasick; let pma = DoubleArrayAhoCorasick::new(vec!["", "a"])?; // Empty pattern matches at every boundary ``` -------------------------------- ### Build Automaton with Result Handling Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/05-errors-and-types.md Demonstrates building a DoubleArrayAhoCorasick automaton and handling the Result, printing success or error messages. ```rust use daachorse::{DoubleArrayAhoCorasick, Result}; fn build_automaton(patterns: Vec<&str>) -> Result> { DoubleArrayAhoCorasick::new(patterns) } match build_automaton(vec!["a", "b"]) { Ok(pma) => println!("Success"), Err(e) => eprintln!("Error: {}", e), } ``` -------------------------------- ### Use Byte-Wise Matching for ASCII Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/08-usage-guide-and-examples.md For ASCII strings, use `DoubleArrayAhoCorasick` for fast and compact matching. This is the default and recommended approach for simple character sets. ```rust use daachorse::DoubleArrayAhoCorasick; let patterns = vec!["hello", "world"]; let pma = DoubleArrayAhoCorasick::new(patterns)?; ``` -------------------------------- ### serialize_to_vec Implementation for u32 Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/06-serialization-and-traits.md Example implementation of `serialize_to_vec` for `u32`. It appends the little-endian byte representation of the `u32` value to the destination vector. ```rust fn serialize_to_vec(&self, dst: &mut Vec) { dst.extend_from_slice(&self.to_le_bytes()); } ``` -------------------------------- ### Handle Empty Haystack in Daachorse Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/08-usage-guide-and-examples.md Demonstrates how to handle an empty search string. No matches will be found. ```rust use daachorse::DoubleArrayAhoCorasick; let pma = DoubleArrayAhoCorasick::new(vec!["a"])?; let matches: Vec<_> = pma.find_iter("").collect(); assert_eq!(0, matches.len()); // No matches ``` -------------------------------- ### Fold Iterator Results Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/07-iterators.md Use `fold()` to accumulate a single value from an iterator. This example counts the total number of matches found. ```rust let count = pma.find_iter("abcd").fold(0, |acc, _| acc + 1); ``` -------------------------------- ### Double-Array Trie State Transition Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/09-implementation-notes.md Illustrates the logic for transitioning between states in a double-array trie using base and check arrays. This is crucial for efficient character-based lookups. ```pseudocode next_state = base[s] XOR c if check[next_state] == c: // Valid transition else: // Invalid, follow failure link ``` -------------------------------- ### Extracting Matched Text Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/00-api-overview.md Use the start and end offsets from a match result to slice the original haystack and retrieve the matched text. ```rust let haystack = "hello world"; let m = pma.find_iter(haystack).next().unwrap(); let text = &haystack[m.start()..m.end()]; ``` -------------------------------- ### CJK Pattern Matching with CharwiseDoubleArrayAhoCorasick Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/08-usage-guide-and-examples.md Illustrates pattern matching for CJK characters using `CharwiseDoubleArrayAhoCorasick`. This is useful for languages where characters can span multiple bytes. ```rust use daachorse::CharwiseDoubleArrayAhoCorasick; fn main() -> daachorse::Result<()> { let patterns = vec!["全世界", "世界", "に"]; let pma = CharwiseDoubleArrayAhoCorasick::new(patterns)?; for m in pma.find_iter("全世界中に") { println!("{}..{}: value={}", m.start(), m.end(), m.value()); } Ok(()) } // Output: // 0..9: value=0 (全世界 in bytes) // 12..15: value=2 (に in bytes) ``` -------------------------------- ### Collect All Matches Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/04-charwise-builder-and-match.md Demonstrates collecting all overlapping matches found in the haystack using `find_overlapping_iter` and asserting their start, end, and value properties. ```rust let patterns = vec!["bcd", "ab", "a"]; let pma = DoubleArrayAhoCorasick::new(patterns)?; let matches: Vec<_> = pma.find_overlapping_iter("abcd") .collect(); assert_eq!(3, matches.len()); assert_eq!((0, 1, 2), (matches[0].start(), matches[0].end(), matches[0].value())); ``` -------------------------------- ### Matching Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/00-api-overview.md Illustrates different methods for finding patterns within a haystack string, including standard non-overlapping, overlapping, and leftmost-first matching strategies, as well as a byte-by-byte stepper. ```APIDOC ## Matching ### Non-overlapping standard ```rust for m in pma.find_iter(haystack) { } ``` ### Overlapping ```rust for m in pma.find_overlapping_iter(haystack) { } ``` ### Overlapping, longest only ```rust for m in pma.find_overlapping_no_suffix_iter(haystack) { } ``` ### Leftmost (requires LeftmostLongest or LeftmostFirst MatchKind) ```rust for m in pma.leftmost_find_iter(haystack) { } ``` ### Byte-by-byte ```rust let mut stepper = pma.find_stepper(); stepper.consume(byte); if let Some(m) = stepper.matches() { } ``` ``` -------------------------------- ### Get Match Value (Custom Value) Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/04-charwise-builder-and-match.md Retrieves the custom value associated with the matched pattern. This is used when constructing the automaton with `with_values()`. ```rust let patvals = vec![("bcd", 100), ("ab", 200), ("a", 300)]; let pma = DoubleArrayAhoCorasick::with_values(patvals)?; let mut it = pma.find_iter("abcd"); let m = it.next().unwrap(); assert_eq!(300, m.value()); // Custom value for "a" ``` -------------------------------- ### build Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/02-double-array-aho-corasick-builder.md Builds an automaton from patterns, automatically assigning identifiers (0, 1, 2, ...). ```APIDOC ## build(self, patterns: I) -> Result> Builds an automaton from patterns, automatically assigning identifiers (0, 1, 2, ...). ### Signature ```rust pub fn build(self, patterns: I) -> Result> where I: IntoIterator, P: AsRef<[u8]>, V: Copy + TryFrom ``` ### Parameters #### Path Parameters - **patterns** (I: IntoIterator) - Required - Iterator of byte patterns ### Return `Result, DaachorseError>` ### Errors - `DaachorseError::InvalidConversion` — Pattern index cannot be converted to `V` - `DaachorseError::AutomatonScale` — Pattern count exceeds 2^24-1 - `DaachorseError::AutomatonScale` — Resulting automaton exceeds capacity ### Value Assignment The value `i` is automatically associated with `patterns[i]`. ### Example ```rust use daachorse::DoubleArrayAhoCorasickBuilder; let builder = DoubleArrayAhoCorasickBuilder::new(); let patterns = vec!["bcd", "ab", "a"]; let pma = builder.build(patterns)?; let mut it = pma.find_iter("abcd"); let m = it.next().unwrap(); assert_eq!((0, 1, 2), (m.start(), m.end(), m.value())); // value=2 for patterns[2]="a" ``` ``` -------------------------------- ### DoubleArrayAhoCorasickBuilder with Default Settings Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/02-double-array-aho-corasick-builder.md Builds a DoubleArrayAhoCorasick automaton using the default configuration, which includes MatchKind::Standard and 16 free blocks. This is a quick way to create an automaton when custom settings are not required. ```rust use daachorse::DoubleArrayAhoCorasickBuilder; // Default has: // - MatchKind::Standard // - num_free_blocks = 16 let pma = DoubleArrayAhoCorasickBuilder::new() .build(vec!["pattern1", "pattern2"])?; ``` -------------------------------- ### Trigger AutomatonScale Error Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/05-errors-and-types.md Shows an example that would trigger an AutomatonScale error if the number of patterns exceeded the maximum allowed limit (2^24-1). ```rust use daachorse::DoubleArrayAhoCorasick; // This would fail if you somehow provided > 2^24 patterns let huge_count = (0..16_777_216).map(|i| format!("{}", i)).collect::>(); let result = DoubleArrayAhoCorasick::new( huge_count.iter().map(|s| s.as_str()) ); // Returns Err(DaachorseError::AutomatonScale(...)) ``` -------------------------------- ### Basic Pattern Matching with DoubleArrayAhoCorasick Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/08-usage-guide-and-examples.md Demonstrates basic pattern matching using `DoubleArrayAhoCorasick`. Ensure patterns are provided as a vector of strings and the input text is iterated over for matches. ```rust use daachorse::DoubleArrayAhoCorasick; fn main() -> daachorse::Result<()> { let patterns = vec!["a", "ab", "ba", "bab", "bb"]; let pma = DoubleArrayAhoCorasick::new(patterns)?; for m in pma.find_iter("abab") { println!("{}..{}: value={}", m.start(), m.end(), m.value()); } Ok(()) } // Output: // 0..1: value=0 // 2..4: value=3 ``` -------------------------------- ### Find Non-Overlapping Occurrences (LeftmostFirst) Source: https://github.com/daac-tools/daachorse/blob/main/README.md Use `leftmost_find_iter()` with `MatchKind::LeftmostFirst` to find the earliest registered pattern among those starting from the search position. ```rust use daachorse::{DoubleArrayAhoCorasickBuilder, MatchKind}; let patterns = vec!["ab", "a", "abcd"]; let pma = DoubleArrayAhoCorasickBuilder::new() .match_kind(MatchKind::LeftmostFirst) .build(&patterns) .unwrap(); let mut it = pma.leftmost_find_iter("abcd"); let m = it.next().unwrap(); assert_eq!((0, 2, 0), (m.start(), m.end(), m.value())); assert_eq!(None, it.next()); ``` -------------------------------- ### CharwiseDoubleArrayAhoCorasickBuilder::new Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/04-charwise-builder-and-match.md Creates a new builder with default settings for constructing a CharwiseDoubleArrayAhoCorasick automaton. ```APIDOC ## CharwiseDoubleArrayAhoCorasickBuilder::new() ### Description Creates a new builder with default settings. ### Signature ```rust pub fn new() -> Self ``` ### Default Configuration: | Setting | | Value | |--|--| | match_kind | `MatchKind::Standard` | ### Example: ```rust use daachorse::CharwiseDoubleArrayAhoCorasickBuilder; let builder = CharwiseDoubleArrayAhoCorasickBuilder::new(); let patterns = vec!["全世界", "世界", "に"]; let pma = builder.build(patterns)?; ``` ``` -------------------------------- ### Build Automaton from Patterns Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/02-double-array-aho-corasick-builder.md Use this method to build an automaton when you only have patterns and want to assign sequential integer values (0, 1, 2, ...) to them. Ensure the pattern index can be converted to the specified value type `V`. ```rust use daachorse::DoubleArrayAhoCorasickBuilder; let builder = DoubleArrayAhoCorasickBuilder::new(); let patterns = vec!["bcd", "ab", "a"]; let pma = builder.build(patterns)?; let mut it = pma.find_iter("abcd"); let m = it.next().unwrap(); assert_eq!((0, 1, 2), (m.start(), m.end(), m.value())); // value=2 for patterns[2]="a" ``` -------------------------------- ### CharwiseDoubleArrayAhoCorasick Reference Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/README.md Reference for the `CharwiseDoubleArrayAhoCorasick` class, optimized for multibyte characters. ```APIDOC ## CharwiseDoubleArrayAhoCorasick Reference ### Description Reference for the `CharwiseDoubleArrayAhoCorasick` class, optimized for multibyte characters. It offers the same interface as the byte-wise version but is tailored for Unicode and CJK text. ### Interface - Construction: `new()`, `with_values()` - Searching: `find_iter()`, `find_overlapping_iter()`, `leftmost_find_iter()` - Serialization: `serialize()`, `deserialize()` ### Performance Optimized for multibyte character processing, offering different performance characteristics compared to byte-wise matching. ``` -------------------------------- ### Filter Iterator Results Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/07-iterators.md Apply a filter to an iterator to select only the matches that satisfy a specific condition. The example filters for matches where the length is greater than 2. ```rust let long_matches: Vec<_> = pma.find_iter("abcd") .filter(|m| m.end() - m.start() > 2) .collect(); ``` -------------------------------- ### DoubleArrayAhoCorasickBuilder Configuration Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/DOCUMENTATION_MANIFEST.txt Configuration methods for building a DoubleArrayAhoCorasick automaton. ```APIDOC ## DoubleArrayAhoCorasickBuilder Configuration Methods for configuring and building a `DoubleArrayAhoCorasick` automaton using the builder pattern. ### Methods - `new()`: Creates a new `DoubleArrayAhoCorasickBuilder`. - `add_pattern(pattern, value)`: Adds a pattern with an associated value to the builder. - `num_free_blocks(num)`: Sets the number of free blocks for tuning. - `build()`: Constructs the `DoubleArrayAhoCorasick` automaton from the configured patterns and settings. ``` -------------------------------- ### Displaying Errors in Daachorse Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/05-errors-and-types.md All error types implement the `Display` trait for human-readable output. This example shows how to catch and print an error during automaton construction. ```rust use daachorse::DoubleArrayAhoCorasick; let result = DoubleArrayAhoCorasick::::new((0..300).map(|i| format!("p{}", i))); if let Err(e) = result { println!("Error: {}", e); // Output: "InvalidConversionError: index cannot be converted to u8" } ``` -------------------------------- ### Process Input Byte-by-Byte Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/08-usage-guide-and-examples.md Use `find_stepper` for efficient byte-by-byte processing of input streams. This method allows for incremental matching without loading the entire input into memory. ```rust use daachorse::DoubleArrayAhoCorasick; let patterns = vec!["abc", "bcd"]; let pma = DoubleArrayAhoCorasick::new(patterns)?; let mut stepper = pma.find_stepper(); let haystack = b"abcd"; for &byte in haystack { stepper.consume(byte); if let Some(m) = stepper.matches() { println!("Found match at {}..{}", m.start(), m.end()); } } ``` -------------------------------- ### Construction Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/00-api-overview.md Demonstrates various ways to construct Double Array Aho Corasick automata, including byte-wise and character-wise approaches, with and without custom values, and with configurable matching options. ```APIDOC ## Construction ### Byte-wise with auto-indexed values ```rust let pma = DoubleArrayAhoCorasick::new(patterns)?; ``` ### Byte-wise with custom values ```rust let pma = DoubleArrayAhoCorasick::with_values(patvals)?; ``` ### Character-wise with auto-indexed values ```rust let pma = CharwiseDoubleArrayAhoCorasick::new(patterns)?; ``` ### Character-wise with custom values ```rust let pma = CharwiseDoubleArrayAhoCorasick::with_values(patvals)?; ``` ### With configuration ```rust let pma = DoubleArrayAhoCorasickBuilder::new() .match_kind(MatchKind::LeftmostLongest) .build(patterns)?; ``` ``` -------------------------------- ### SerializableVec Trait Definition Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/06-serialization-and-traits.md Defines the SerializableVec trait for serializing vectors of Serializable types. It includes methods for serialization, deserialization, and getting the byte size. ```rust pub trait SerializableVec: Sized { fn serialize_to_vec(&self, dst: &mut Vec); fn deserialize_from_slice(src: &[u8]) -> Result<(Self, &[u8]>) ; fn serialized_bytes(&self) -> usize; } ``` -------------------------------- ### Serializable Implementation for Option Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/06-serialization-and-traits.md Defines serialization for Option. None is encoded as 0u32, and Some(x) is encoded as x. This implementation relies on the u32 serialization. ```rust impl Serializable for Option { fn serialize_to_vec(&self, dst: &mut Vec) { self.map_or(0, NonZeroU32::get).serialize_to_vec(dst); } fn deserialize_from_slice(src: &[u8]) -> Result<(Self, &[u8])> { let (x, src) = u32::deserialize_from_slice(src)?; Ok((NonZeroU32::new(x), src)) } fn serialized_bytes() -> usize { u32::serialized_bytes() } } ``` -------------------------------- ### Build Faster Automata on Multibyte Characters Source: https://github.com/daac-tools/daachorse/blob/main/README.md Use `CharwiseDoubleArrayAhoCorasick` for faster matching on multibyte characters by using Unicode code point values for transitions instead of byte values. ```rust use daachorse::CharwiseDoubleArrayAhoCorasick; let patterns = vec!["全世界", "世界", "に"]; let pma = CharwiseDoubleArrayAhoCorasick::new(patterns).unwrap(); let mut it = pma.find_iter("全世界中に"); let m = it.next().unwrap(); assert_eq!((0, 9, 0), (m.start(), m.end(), m.value())); let m = it.next().unwrap(); assert_eq!((12, 15, 2), (m.start(), m.end(), m.value())); assert_eq!(None, it.next()); ``` -------------------------------- ### CodeMapper Initialization Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/09-implementation-notes.md Initializes a CodeMapper for bidirectional mapping between Unicode code points and compressed indices. This is used for character-wise processing. ```rust let mapper = CodeMapper::new(patterns); // Builds bidirectional mapping: // - char → index in double-array // - index → char (for reverse operations) ``` -------------------------------- ### LeftmostLongest Match Iterator Example Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/07-iterators.md Illustrates finding the longest pattern at each leftmost position using `leftmost_find_iter` with `MatchKind::LeftmostLongest`. Patterns are built using `DoubleArrayAhoCorasickBuilder`. ```rust use daachorse::{DoubleArrayAhoCorasickBuilder, MatchKind}; let patterns = vec!["ab", "a", "abcd"]; let pma = DoubleArrayAhoCorasickBuilder::new() .match_kind(MatchKind::LeftmostLongest) .build(&patterns)?; let matches: Vec<_> = pma.leftmost_find_iter("abcd").collect(); assert_eq!(1, matches.len()); assert_eq!((0, 4, 2), (matches[0].start(), matches[0].end(), matches[0].value())); // "abcd" ``` -------------------------------- ### Find Overlapping No Suffix Iterator Example Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/07-iterators.md Demonstrates how to use `find_overlapping_no_suffix_iter` to find the longest non-suffix patterns at each position in a text. Requires building a DoubleArrayAhoCorasick instance. ```rust use daachorse::DoubleArrayAhoCorasick; let patterns = vec!["bcd", "cd", "abc"]; let pma = DoubleArrayAhoCorasick::new(patterns)?; let matches: Vec<_> = pma.find_overlapping_no_suffix_iter("abcd").collect(); assert_eq!(2, matches.len()); assert_eq!((0, 3, 2), (matches[0].start(), matches[0].end(), matches[0].value())); // "abc" (longest) assert_eq!((1, 4, 0), (matches[1].start(), matches[1].end(), matches[1].value())); // "bcd" (longest) ``` -------------------------------- ### Configure builder with custom number of free blocks Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/02-double-array-aho-corasick-builder.md Adjust the number of trailing free blocks used during construction to balance construction time and memory efficiency. A smaller number speeds up construction but may use more memory, while a larger number improves memory efficiency at the cost of slower construction. Panics if n < 1. ```rust use daachorse::DoubleArrayAhoCorasickBuilder; let builder = DoubleArrayAhoCorasickBuilder::new() .num_free_blocks(8); let patterns = vec!["pattern1", "pattern2"]; let pma = builder.build(patterns)?; ``` -------------------------------- ### Extract Matched Text with Daachorse Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/08-usage-guide-and-examples.md Illustrates how to extract the exact substring from the input text that corresponds to a found pattern using match start and end indices. ```rust use daachorse::DoubleArrayAhoCorasick; let patterns = vec!["hello", "world"]; let pma = DoubleArrayAhoCorasick::new(patterns)?; let text = "hello world"; for m in pma.find_iter(text) { let matched_text = &text[m.start()..m.end()]; println!("Matched: '{}'", matched_text); } ``` -------------------------------- ### Overlapping Match Algorithm Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/09-implementation-notes.md Details how to find all overlapping matches, including suffix matches, by traversing failure links after a state transition. ```pseudocode state = root for each byte b in haystack: state = next_state(state, b) // Report all outputs (suffix matches) cur = state while output[cur]: report_output[cur] cur = fail[cur] // Continue matching ``` -------------------------------- ### Overlapping Search Source: https://github.com/daac-tools/daachorse/blob/main/_autodocs/08-usage-guide-and-examples.md Employ `find_overlapping_iter` to report all patterns that match at each position, including overlapping ones. This strategy is useful when multiple patterns can start at the same text position. ```rust use daachorse::DoubleArrayAhoCorasick; let patterns = vec!["a", "aa", "aaa"]; let pma = DoubleArrayAhoCorasick::new(patterns)?; let matches: Vec<_> = pma.find_overlapping_iter("aaaa").collect(); // Output: matches at each ending position, longest first // Position 1: "a" // Position 2: "aa", "a" // Position 3: "aaa", "aa", "a" // Position 4: "aaa", "aa", "a" ```