### Rust StreamCDC File Chunking Example

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/v2016/struct.StreamCDC

Provides a practical Rust example demonstrating how to use the `StreamCDC` struct to process a file. It shows file opening, `StreamCDC` instantiation with specific chunk size parameters, and iterating over the generated chunks to print their offsets and lengths.

```Rust
let source = File::open("test/fixtures/SekienAkashita.jpg").unwrap();
let chunker = StreamCDC::new(source, 4096, 16384, 65535);
for result in chunker {
    let chunk = result.unwrap();
    println!("offset={} length={}", chunk.offset, chunk.length);
}
```

--------------------------------

### Rust FastCDC File Chunking Example

Source: https://docs.rs/fastcdc/3.2.1/src/fastcdc/v2020/mod.rs

Demonstrates how to use the FastCDC library to read a file into memory, create a FastCDC chunker instance with specified min/avg/max sizes, and then iterate over the generated data chunks, printing their offset and size. This example uses `v2020::FastCDC`.

```Rust
use std::fs;
use fastcdc::v2020;
let contents = fs::read("test/fixtures/SekienAkashita.jpg").unwrap();
let chunker = v2020::FastCDC::new(&contents, 8192, 16384, 65535);
for entry in chunker {
    println!("offset={} size={}", entry.offset, entry.length);
}
```

--------------------------------

### Example: Chunking a File with FastCDC in Rust

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/ronomon/struct.FastCDC

This Rust example demonstrates how to use the "fastcdc" crate to read a file into memory and split its contents into chunks. The "FastCDC::new" method is used to configure chunk sizes between 16KB and 64KB, preferring 32KB. The example then iterates through the generated chunks, printing their offset and length, showcasing basic usage of the "Iterator" trait.

```Rust
let contents = std::fs::read("test/fixtures/SekienAkashita.jpg").unwrap();
let chunker = fastcdc::ronomon::FastCDC::new(&contents, 16384, 32768, 65536);
for entry in chunker {
    println!("offset={} size={}", entry.offset, entry.length);
}
```

--------------------------------

### Example: Chunking a File with Rust StreamCDC

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/v2020/struct.StreamCDC

This Rust example demonstrates how to use the `StreamCDC` struct to perform content-defined chunking on a file. It opens a JPEG image, initializes the chunker with specified minimum, average, and maximum chunk sizes, and then iterates through the resulting data chunks, printing their offset and length.

```Rust
let source = File::open("test/fixtures/SekienAkashita.jpg").unwrap();
let chunker = StreamCDC::new(source, 4096, 16384, 65535);
for result in chunker {
    let chunk = result.unwrap();
    println!("offset={} length={}", chunk.offset, chunk.length);
}
```

--------------------------------

### Example: AsyncStreamCDC Usage for File Chunking

Source: https://docs.rs/fastcdc/3.2.1/src/fastcdc/v2020/async_stream_cdc.rs

This Rust example demonstrates how to use `AsyncStreamCDC` to process a file asynchronously, chunking its content based on the FastCDC algorithm. It shows how to create an `AsyncStreamCDC` instance, convert it into an asynchronous stream, collect the resulting chunks, and print their offsets and lengths. The example is compatible with both `futures` and `tokio` runtimes via feature flags.

```Rust
# use std::fs::File;
# use fastcdc::v2020::AsyncStreamCDC;
# #[cfg(all(feature = "futures", not(feature = "tokio")))]
# use futures::stream::StreamExt;
# #[cfg(all(feature = "tokio", not(feature = "futures")))]
# use tokio_stream::StreamExt;

async fn run() {
    let source = std::fs::read("test/fixtures/SekienAkashita.jpg").unwrap();
    let mut chunker = AsyncStreamCDC::new(source.as_ref(), 4096, 16384, 65535);
    let stream = chunker.as_stream();

    let chunks = stream.collect::<Vec<_>>().await;

    for result in chunks {
        let chunk = result.unwrap();
        println!("offset={} length={}", chunk.offset, chunk.length);
    }
}
```

--------------------------------

### FastCDC Pre-3.0 Usage Example

Source: https://docs.rs/fastcdc/3.2.1/src/fastcdc/lib.rs

Demonstrates the syntax for initializing the FastCDC chunker before version 3.0 of the crate, where the `FastCDC` struct was directly accessible under the `fastcdc` module.

```Rust
use fastcdc::ronomon as fastcdc;
use std::fs;
let contents = fs::read("test/fixtures/SekienAkashita.jpg").unwrap();
let chunker = fastcdc::FastCDC::new(&contents, 8192, 16384, 32768);
```

--------------------------------

### FastCDC Post-3.0 Usage Example

Source: https://docs.rs/fastcdc/3.2.1/src/fastcdc/lib.rs

Shows the updated syntax for initializing the FastCDC chunker after version 3.0, requiring explicit access to the `ronomon` module (e.g., `fastcdc::ronomon::FastCDC`) for the same implementation.

```Rust
use std::fs;
let contents = fs::read("test/fixtures/SekienAkashita.jpg").unwrap();
let chunker = fastcdc::ronomon::FastCDC::new(&contents, 8192, 16384, 32768);
```

--------------------------------

### Rust FastCDC File Chunking Example

Source: https://docs.rs/fastcdc/3.2.1/src/fastcdc/v2016/mod.rs

An example demonstrating how to use the `FastCDC` chunker in Rust to process a file. It reads file contents, initializes `FastCDC` with specified min, avg, and max chunk sizes, and then iterates over the generated `Chunk` entries, printing their offset and length.

```Rust
# use std::fs;
# use fastcdc::v2016::FastCDC;
let contents = fs::read("test/fixtures/SekienAkashita.jpg").unwrap();
let chunker = FastCDC::new(&contents, 8192, 16384, 65535);
for entry in chunker {
    println!("offset={} length={}", entry.offset, entry.length);
}
```

--------------------------------

### Basic FastCDC v2020 Chunking

Source: https://docs.rs/fastcdc/3.2.1/src/fastcdc/lib.rs

Illustrates a fundamental example of using the `v2020` implementation of FastCDC to process a file. It initializes the chunker with content and chunk size parameters, then iterates through the generated chunks, printing their offset and length.

```Rust
use std::fs;
use fastcdc::v2020;
let contents = fs::read("test/fixtures/SekienAkashita.jpg").unwrap();
let chunker = v2020::FastCDC::new(&contents, 4096, 16384, 65535);
for entry in chunker {
    println!("offset={} size={}", entry.offset, entry.length);
}
```

--------------------------------

### Example: Chunk a File with FastCDC in Rust

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/v2020/struct.FastCDC

Demonstrates how to use the `FastCDC` struct to read a file into memory and split it into chunks. It initializes a `FastCDC` instance with specified minimum, average, and maximum chunk sizes, then iterates over the generated chunks, printing their offset and length.

```Rust
use std::fs;
use fastcdc::v2020;
let contents = fs::read("test/fixtures/SekienAkashita.jpg").unwrap();
let chunker = v2020::FastCDC::new(&contents, 8192, 16384, 65535);
for entry in chunker {
    println!("offset={} size={}", entry.offset, entry.length);
}
```

--------------------------------

### Example: Chunking a File with FastCDC in Rust

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/v2016/struct.FastCDC

Demonstrates reading a file into memory and splitting it into chunks using the FastCDC chunker. It shows how to instantiate FastCDC with source data and specified minimum, average, and maximum chunk size parameters, then iterate over the generated chunks, printing their offset and length.

```Rust
let contents = fs::read("test/fixtures/SekienAkashita.jpg").unwrap();
let chunker = FastCDC::new(&contents, 8192, 16384, 65535);
for entry in chunker {
    println!("offset={} length={}", entry.offset, entry.length);
}
```

--------------------------------

### Basic File Chunking using FastCDC v2020

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/index

This example demonstrates how to use the `fastcdc::v2020` module to perform content-defined chunking on a file. It reads a file's contents and then iterates through the `FastCDC` chunker to print the offset and length of each generated chunk. This uses normalized chunking level 1 as described in the 2020 paper.

```Rust
use std::fs;
use fastcdc::v2020;
let contents = fs::read("test/fixtures/SekienAkashita.jpg").unwrap();
let chunker = v2020::FastCDC::new(&contents, 4096, 16384, 65535);
for entry in chunker {
    println!("offset={} size={}", entry.offset, entry.length);
}
```

--------------------------------

### Example Usage of AsyncStreamCDC for File Chunking in Rust

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/v2020/struct.AsyncStreamCDC

Demonstrates how to use `AsyncStreamCDC` to read a JPEG file, chunk its contents, and iterate over the resulting chunks. It initializes the chunker with source data and size parameters, then collects and prints chunk offsets and lengths.

```Rust
async fn run() {
    let source = std::fs::read("test/fixtures/SekienAkashita.jpg").unwrap();
    let mut chunker = AsyncStreamCDC::new(source.as_ref(), 4096, 16384, 65535);
    let stream = chunker.as_stream();

    let chunks = stream.collect::<Vec<_>>().await;

    for result in chunks {
        let chunk = result.unwrap();
        println!("offset={} length={}", chunk.offset, chunk.length);
    }
}
```

--------------------------------

### Rust FastCDC File Chunking Example

Source: https://docs.rs/fastcdc/3.2.1/src/fastcdc/ronomon/mod.rs

Demonstrates how to use the `FastCDC` chunker to process a file. It reads file contents into memory, initializes the chunker with preferred chunk sizes (16KB min, 32KB avg, 64KB max), and then iterates over the generated chunks, printing their offset and length.

```Rust
let contents = std::fs::read("test/fixtures/SekienAkashita.jpg").unwrap();
let chunker = fastcdc::ronomon::FastCDC::new(&contents, 16384, 32768, 65536);
for entry in chunker {
    println!("offset={} size={}", entry.offset, entry.length);
}
```

--------------------------------

### Example Usage of StreamCDC for File Chunking (Rust)

Source: https://docs.rs/fastcdc/3.2.1/src/fastcdc/v2020/mod.rs

This code snippet demonstrates how to use the `StreamCDC` struct to perform content-defined chunking on a file. It shows opening a file, creating a `StreamCDC` instance with specified min, avg, and max chunk sizes, and then iterating over the resulting chunks to print their offset and length.

```Rust
# use std::fs::File;
# use fastcdc::v2020::StreamCDC;
let source = File::open("test/fixtures/SekienAkashita.jpg").unwrap();
let chunker = StreamCDC::new(source, 4096, 16384, 65535);
for result in chunker {
    let chunk = result.unwrap();
    println!("offset={} length={}", chunk.offset, chunk.length);
}
```

--------------------------------

### Customizing FastCDC Chunking Normalization Level

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/index

This example shows how to apply a specific normalization level to the FastCDC chunking process using `FastCDC::with_level`. It demonstrates setting `Normalization::Level3` and highlights how changing min/max chunk sizes can be relevant with higher normalization levels to achieve desired chunk distributions, as generated chunks tend to be closer to the expected size.

```Rust
use std::fs;
use fastcdc::v2020::{FastCDC, Normalization};
let contents = fs::read("test/fixtures/SekienAkashita.jpg").unwrap();
let chunker = FastCDC::with_level(&contents, 8192, 16384, 32768, Normalization::Level3);
for entry in chunker {
    println!("offset={} size={}", entry.offset, entry.length);
}
```

--------------------------------

### step_by Iterator Method

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/ronomon/struct.FastCDC

Creates a new iterator that steps by a given amount at each iteration, starting from the same point as the original iterator. This method is useful for processing elements at regular intervals.

```APIDOC
fn step_by(self, step: usize) -> StepBy<Self> where Self: Sized,
```

--------------------------------

### Iterating Over Chunks with StreamCDC in Rust

Source: https://docs.rs/fastcdc/3.2.1/src/fastcdc/v2016/mod.rs

Demonstrates how to initialize a `StreamCDC` instance with a file source and iterate over the generated `ChunkData` results, printing their offset and length. This example uses the `v2016::StreamCDC` implementation.

```Rust
# use std::fs::File;
# use fastcdc::v2016::StreamCDC;
let source = File::open("test/fixtures/SekienAkashita.jpg").unwrap();
let chunker = StreamCDC::new(source, 4096, 16384, 65535);
for result in chunker {
    let chunk = result.unwrap();
    println!("offset={} length={}", chunk.offset, chunk.length);
}
```

--------------------------------

### Rust Iterator Trait: enumerate method

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/v2020/struct.StreamCDC

Creates an iterator which gives the current iteration count along with the next value. The count starts at 0.

```APIDOC
fn enumerate(self) -> Enumerate<Self> where Self: Sized,
```

--------------------------------

### Implement Any for T

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/v2016/enum.Normalization

Documents the blanket implementation of the `Any` trait for any type `T` that is `'static` and not `Sized`. This implementation allows runtime introspection to get the `TypeId` of an instance.

```APIDOC
impl<T> Any for T where T: 'static + ?Sized
  fn type_id(&self) -> TypeId
    Gets the TypeId of self.
```

--------------------------------

### Get Nth Iterator Element in Rust

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/v2016/struct.StreamCDC

Returns the `n`th element of the iterator, consuming all elements before it. If the iterator has fewer than `n+1` elements, it returns `None`.

```APIDOC
fn nth(&mut self, n: usize) -> Option<Self::Item>
```

--------------------------------

### Get TypeId for Any Trait Implementation

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/v2020/struct.Chunk

Retrieves the TypeId of the current instance (self) when the Any trait is implemented. This method is crucial for runtime type identification and dynamic downcasting in Rust.

```Rust
fn type_id(&self) -> TypeId
```

--------------------------------

### Get Index of Element in Rust Iterator (position)

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/ronomon/struct.FastCDC

Searches for an element in an iterator that satisfies a given predicate, returning the index of the first matching element if found.

```APIDOC
fn position<P>(
    &mut self,
    predicate: P
) -> Option<usize>
where
    Self: Sized,
    P: FnMut(Self::Item) -> bool,
```

--------------------------------

### FastCDC Struct API Reference

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/v2020/struct.FastCDC

API documentation for the `FastCDC` struct, detailing its constructor methods, core functionality for data chunking, and trait implementations.

```APIDOC
Struct: FastCDC<'a>
  Description: The FastCDC chunker implementation from 2020.

  Methods:
    new(source: &'a [u8], min_size: u32, avg_size: u32, max_size: u32) -> Self
      Description: Construct a FastCDC that will process the given slice of bytes. Uses chunk size normalization level 1 by default.
      Parameters:
        source: &'a [u8] - A slice of bytes to be processed.
        min_size: u32 - The minimum desired chunk size.
        avg_size: u32 - The average desired chunk size.
        max_size: u32 - The maximum desired chunk size.
      Returns: Self (a new FastCDC instance)

    with_level(source: &'a [u8], min_size: u32, avg_size: u32, max_size: u32, level: Normalization) -> Self
      Description: Create a new FastCDC with the given normalization level.
      Parameters:
        source: &'a [u8] - A slice of bytes to be processed.
        min_size: u32 - The minimum desired chunk size.
        avg_size: u32 - The average desired chunk size.
        max_size: u32 - The maximum desired chunk size.
        level: Normalization - The Normalization level to apply.
      Returns: Self (a new FastCDC instance)

    with_level_and_seed(source: &'a [u8], min_size: u32, avg_size: u32, max_size: u32, level: Normalization, seed: u64) -> Self
      Description: Create a new FastCDC with the given normalization level and seed to be XOR’d with the values in the gear tables.
      Parameters:
        source: &'a [u8] - A slice of bytes to be processed.
        min_size: u32 - The minimum desired chunk size.
        avg_size: u32 - The average desired chunk size.
        max_size: u32 - The maximum desired chunk size.
        level: Normalization - The Normalization level to apply.
        seed: u64 - A u64 seed to XOR with gear table values.
      Returns: Self (a new FastCDC instance)

    cut(&self, start: usize, remaining: usize) -> (u64, usize)
      Description: Find the next cut point in the data. The returned 2-tuple consists of the 64-bit hash (fingerprint) and the byte offset of the end of the chunk. Special case: if remaining bytes < min_size, returns hash 0 and cut point is end of source data.
      Parameters:
        start: usize - The position from which to start processing the source data.
        remaining: usize - The number of bytes left to be processed.
      Returns: (u64, usize) - A tuple containing the 64-bit hash and the byte offset of the end of the chunk.

  Trait Implementations:
    impl<'a> Clone for FastCDC<'a>
      Description: Implements the Clone trait for FastCDC, allowing instances to be duplicated.
```

--------------------------------

### ChunkData Struct Fields Definition

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/v2016/struct.ChunkData

Defines the fields of the `ChunkData` struct, which encapsulates information about a data chunk, including its hash, starting offset, length, and the raw byte data itself.

```APIDOC
struct ChunkData {
  hash: u64, // The gear hash value as of the end of the chunk.
  offset: u64, // Starting byte position within the source.
  length: usize, // Length of the chunk in bytes.
  data: Vec<u8> // Source bytes contained in this chunk.
}
```

--------------------------------

### FastCDC Struct API Documentation

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/v2016/struct.FastCDC

Detailed API documentation for the FastCDC struct, including its constructor methods (`new`, `with_level`) and the `cut` method for finding chunk boundaries. It also covers the `Clone` trait implementation for FastCDC instances.

```APIDOC
impl<'a> FastCDC<'a>:
  Methods:
    new(source: &'a [[u8]], min_size: u32, avg_size: u32, max_size: u32) -> Self
      Description: Construct a `FastCDC` that will process the given slice of bytes. Uses chunk size normalization level 1 by default.
    with_level(source: &'a [[u8]], min_size: u32, avg_size: u32, max_size: u32, level: Normalization) -> Self
      Description: Create a new `FastCDC` with the given normalization level.
    cut(&self, start: usize, remaining: usize) -> (u64, usize)
      Description: Find the next cut point in the data, where `start` is the position from which to start processing the source data, and `remaining` are the number of bytes left to be processed. The returned 2-tuple consists of the 64-bit hash (fingerprint) and the byte offset of the end of the chunk. There is a special case in which the remaining bytes are less than the minimum chunk size, at which point this function returns a hash of 0 and the cut point is the end of the source data.

Trait Implementations:
  impl<'a> Clone for FastCDC<'a>:
    Methods:
      clone(&self) -> FastCDC<'a>
        Description: Returns a duplicate of the value.
      clone_from(&mut self, source: &Self)
        Description: Performs copy-assignment from `source`.
```

--------------------------------

### Rust Iterator `step_by` Method

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/v2020/struct.StreamCDC

This method creates a new iterator that yields elements from the original iterator, but only every `step`th element. The new iterator starts at the same point as the original.

```APIDOC
Iterator.step_by:
  Signature: fn step_by(self, step: usize) -> StepBy<Self> where Self: Sized
  Description: Creates an iterator starting at the same point, but stepping by the given amount at each iteration.
  Parameters:
    step: usize - The amount to step by at each iteration.
```

--------------------------------

### Rust Iterator Trait: skip method

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/v2020/struct.StreamCDC

Creates a new iterator that advances past the first `n` elements of the original iterator. This method is useful for processing a sequence starting from a specific offset.

```APIDOC
fn skip(self, n: usize) -> Skip<Self> where Self: Sized,
```

--------------------------------

### FastCDC Struct API Reference

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/v2020/struct.FastCDC

Comprehensive API documentation for the `FastCDC` struct, detailing its methods and associated types inherited from the `Clone`, `Debug`, and `Iterator` traits. This includes information on method signatures, parameters, return types, and a brief description of their functionality.

```APIDOC
FastCDC<'a> struct:
  Implements Clone trait:
    fn clone(&self) -> FastCDC<'a>
      Description: Returns a duplicate of the value.
    fn clone_from(&mut self, source: &Self)
      Description: Performs copy-assignment from `source`.
  Implements Debug trait:
    fn fmt(&self, f: &mut Formatter<'_>) -> Result
      Description: Formats the value using the given formatter.
  Implements Iterator trait:
    type Item = Chunk
      Description: The type of the elements being iterated over.
    fn next(&mut self) -> Option<Chunk>
      Description: Advances the iterator and returns the next value.
    fn size_hint(&self) -> (usize, Option<usize>)
      Description: Returns the bounds on the remaining length of the iterator.
    fn next_chunk<const N: usize>(&mut self) -> Result<Self::Item; N], IntoIter<Self::Item, N>>
      Parameters:
        N: const usize
      Returns: Result<Self::Item; N], IntoIter<Self::Item, N>>
      Description: Advances the iterator and returns an array containing the next `N` values. This is a nightly-only experimental API (`iter_next_chunk`).
```

--------------------------------

### Rust Iterator Trait: enumerate method

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/v2016/struct.StreamCDC

Creates an iterator which gives the current iteration count as well as the next value. The yielded items are tuples of `(index, value)`, where `index` starts at 0.

```Rust
fn enumerate(self) -> Enumerate<Self> where Self: Sized,
```

--------------------------------

### Create Stepped Iterator in Rust

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/v2016/struct.StreamCDC

Creates a new iterator that starts at the same point as the original but advances by the given `step` amount at each iteration. This method is available on any type that implements the `Iterator` trait and is `Sized`.

```APIDOC
fn step_by(self, step: usize) -> StepBy<Self>
where Self: Sized
```

--------------------------------

### List StreamCDC Methods and Trait Implementations

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/v2020/struct.StreamCDC

This section outlines the key methods available for the `StreamCDC` struct, including constructors like `new`, `with_level`, and `with_level_and_seed`. It also enumerates the various traits implemented by `StreamCDC`, such as `Iterator`, `Send`, `Sync`, and common blanket implementations, indicating its capabilities and how it interacts with the Rust ecosystem.

```APIDOC
StreamCDC
Methods:
- new
- with_level
- with_level_and_seed
Trait Implementations:
- Iterator
Auto Trait Implementations:
- Freeze
- RefUnwindSafe
- Send
- Sync
- Unpin
- UnwindSafe
Blanket Implementations:
- Any
- Borrow<T>
- BorrowMut<T>
- From<T>
- Into<U>
- IntoIterator
- TryFrom<U>
- TryInto<U>
```

--------------------------------

### StreamCDC Available Methods and Trait Implementations

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/v2016/struct.StreamCDC

Lists the primary methods and various trait implementations available for the `StreamCDC` struct, providing an overview of its capabilities and how it interacts with Rust's type system.

```APIDOC
Methods:
  - new
  - with_level

Trait Implementations:
  - Iterator

Auto Trait Implementations:
  - Freeze
  - RefUnwindSafe
  - Send
  - Sync
  - Unpin
  - UnwindSafe

Blanket Implementations:
  - Any
  - Borrow<T>
  - BorrowMut<T>
  - From<T>
  - Into<U>
  - IntoIterator
  - TryFrom<U>
  - TryInto<U>
```

--------------------------------

### Rust FastCDC Chunk Struct Definition

Source: https://docs.rs/fastcdc/3.2.1/src/fastcdc/v2016/mod.rs

Defines the `Chunk` struct, representing a data chunk returned by the FastCDC iterator. It contains the `hash` value, `offset` (starting position), and `length` of the chunk.

```Rust
#[derive(Debug, Clone, Copy, Eq, PartialEq, Hash)]
pub struct Chunk {
    /// The gear hash value as of the end of the chunk.
    pub hash: u64,
    /// Starting byte position within the source.
    pub offset: usize,
    /// Length of the chunk in bytes.
    pub length: usize,
}
```

--------------------------------

### FastCDC Struct API Reference

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/v2020/struct.FastCDC

Detailed API documentation for the `FastCDC` struct within the `fastcdc` Rust crate. This includes a list of its public methods and all implemented traits, providing a comprehensive overview of its functionality and behavior.

```APIDOC
FastCDC:
  Methods:
    - cut()
    - new()
    - with_level()
    - with_level_and_seed()
  Trait Implementations:
    - Clone
    - Debug
    - Eq
    - Iterator
    - PartialEq
    - StructuralPartialEq
  Auto Trait Implementations:
    - Freeze
    - RefUnwindSafe
    - Send
    - Sync
    - Unpin
    - UnwindSafe
  Blanket Implementations:
    - Any
    - Borrow<T>
    - BorrowMut<T>
    - CloneToUninit
    - From<T>
    - Into<U>
    - IntoIterator
    - ToOwned
    - TryFrom<U>
    - TryInto<U>
```

--------------------------------

### Rust Iterator: fold Method

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/ronomon/struct.FastCDC

Reduces an iterator to a single value by applying a function to each element and an accumulator. It starts with an initial value and iteratively combines it with each element, returning the final accumulated result.

```Rust
fn fold<B, F>(self, init: B, f: F) -> B where Self: Sized, F: FnMut(B, Self::Item) -> B
```

--------------------------------

### Get Minimum Element by Key (Rust Iterator)

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/v2020/struct.StreamCDC

Returns the element from the iterator that yields the minimum value based on a key extracted by a provided function. The key must implement the Ord trait for ordering.

```APIDOC
fn min_by_key<B, F>(self, f: F) -> Option<Self::Item> where B: Ord, Self: Sized, F: FnMut(&Self::Item) -> B,
```

--------------------------------

### AsyncStreamCDC API Reference

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/v2020/struct.AsyncStreamCDC

API documentation for the `AsyncStreamCDC` struct, detailing its constructor methods and the `as_stream` method for generating a stream of chunks. It includes parameter types and return values for each method.

```APIDOC
impl<R: AsyncRead + Unpin> AsyncStreamCDC<R>:
  pub fn new(source: R, min_size: u32, avg_size: u32, max_size: u32) -> Self
    Description: Constructs an AsyncStreamCDC instance. Uses chunk size normalization level 1 by default.
  pub fn with_level(source: R, min_size: u32, avg_size: u32, max_size: u32, level: Normalization) -> Self
    Description: Creates a new AsyncStreamCDC with the given normalization level.
  pub fn with_level_and_seed(source: R, min_size: u32, avg_size: u32, max_size: u32, level: Normalization, seed: u64) -> Self
    Description: Creates a new AsyncStreamCDC with the given normalization level and seed to be XOR’d with the values in the gear tables.
  pub fn as_stream(&mut self) -> impl Stream<Item = Result<ChunkData, Error>> + '_ 
    Description: Produces an async Stream of ChunkData.
```

--------------------------------

### API: AsyncStreamCDC Struct and Constructors

Source: https://docs.rs/fastcdc/3.2.1/src/fastcdc/v2020/async_stream_cdc.rs

Detailed API documentation for the `AsyncStreamCDC` struct, which provides an async-streamable FastCDC chunker. This includes its internal fields, type parameters, and public constructor methods for creating instances with various chunk sizing, normalization levels, and optional seeding.

```APIDOC
AsyncStreamCDC<R>:
  Description: An async-streamable version of the FastCDC chunker implementation from 2020 with streaming support.
  Type Parameters:
    R: AsyncRead + Unpin
  Fields:
    buffer: Vec<u8> - Buffer of data from source for finding cut points.
    capacity: usize - Maximum capacity of the buffer (always `max_size`).
    length: usize - Number of relevant bytes in the `buffer`.
    source: R - Source from which data is read into `buffer`.
    processed: u64 - Number of bytes read from the source so far.
    eof: bool - True when the source produces no more data.
    min_size: usize - Minimum chunk size.
    avg_size: usize - Average chunk size.
    max_size: usize - Maximum chunk size.
    mask_s: u64
    mask_l: u64
    mask_s_ls: u64
    mask_l_ls: u64
    gear: Box<[u64; 256]>
    gear_ls: Box<[u64; 256]>
  Methods:
    new(source: R, min_size: u32, avg_size: u32, max_size: u32) -> Self
      Description: Construct a new AsyncStreamCDC that will process bytes from the given source. Uses chunk size normalization level 1 by default.
      Parameters:
        source: R - The async reader source.
        min_size: u32 - Minimum chunk size.
        avg_size: u32 - Average chunk size.
        max_size: u32 - Maximum chunk size.
    with_level(source: R, min_size: u32, avg_size: u32, max_size: u32, level: Normalization) -> Self
      Description: Create a new AsyncStreamCDC with the given normalization level.
      Parameters:
        source: R - The async reader source.
        min_size: u32 - Minimum chunk size.
        avg_size: u32 - Average chunk size.
        max_size: u32 - Maximum chunk size.
        level: Normalization - The normalization level to use.
    with_level_and_seed(source: R, min_size: u32, avg_size: u32, max_size: u32, level: Normalization, seed: u64) -> Self
      Description: Create a new AsyncStreamCDC with the given normalization level and seed to be XOR'd with the values in the gear tables.
      Parameters:
        source: R - The async reader source.
        min_size: u32 - Minimum chunk size.
        avg_size: u32 - Average chunk size.
        max_size: u32 - Maximum chunk size.
        level: Normalization - The normalization level to use.
        seed: u64 - The seed to XOR with gear table values.
```

--------------------------------

### Document count method of Rust Iterator Trait

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/ronomon/struct.FastCDC

This method consumes the iterator, iterating through all its elements and returning the total number of iterations. It's a convenient way to get the length of an iterator if it's finite.

```APIDOC
fn count(self) -> usize
  Description: Consumes the iterator, counting the number of iterations and returning it.
  Parameters:
    self: The iterator instance (consumed).
  Returns:
    usize: The total number of elements in the iterator.
  Constraints:
    Self: Sized
```

--------------------------------

### fastcdc Crate API Reference (v3.2.1)

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/all

Comprehensive API reference for the fastcdc Rust crate, version 3.2.1, detailing all public structs, enums, functions, and constants available within its ronomon, v2016, and v2020 modules. This includes types for chunking data, error handling, normalization, and utility functions.

```APIDOC
Crate: fastcdc 3.2.1

Structs:
  ronomon::Chunk
  ronomon::FastCDC
  v2016::Chunk
  v2016::ChunkData
  v2016::FastCDC
  v2016::StreamCDC
  v2020::AsyncStreamCDC
  v2020::Chunk
  v2020::ChunkData
  v2020::FastCDC
  v2020::StreamCDC

Enums:
  v2016::Error
  v2016::Normalization
  v2020::Error
  v2020::Normalization

Functions:
  v2020::cut
  v2020::cut_gear
  v2020::get_gear_with_seed
  v2020::logarithm2

Constants:
  ronomon::AVERAGE_MAX
  ronomon::AVERAGE_MIN
  ronomon::MAXIMUM_MAX
  ronomon::MAXIMUM_MIN
  ronomon::MINIMUM_MAX
  ronomon::MINIMUM_MIN
  v2016::AVERAGE_MAX
  v2016::AVERAGE_MIN
  v2016::MAXIMUM_MAX
  v2016::MAXIMUM_MIN
  v2016::MINIMUM_MAX
  v2016::MINIMUM_MIN
  v2020::AVERAGE_MAX
  v2020::AVERAGE_MIN
  v2020::MASKS
  v2020::MAXIMUM_MAX
  v2020::MAXIMUM_MIN
  v2020::MINIMUM_MAX
  v2020::MINIMUM_MIN
```

--------------------------------

### Rust FastCDC Chunk Struct Definition

Source: https://docs.rs/fastcdc/3.2.1/src/fastcdc/v2020/mod.rs

Represents a data chunk returned by the FastCDC iterator. Each chunk contains its calculated gear hash value, its starting byte position (offset) within the source data, and its length in bytes.

```APIDOC
#[derive(Debug, Clone, Copy, Eq, PartialEq, Hash)]
pub struct Chunk {
    /// The gear hash value as of the end of the chunk.
    pub hash: u64,
    /// Starting byte position within the source.
    pub offset: usize,
    /// Length of the chunk in bytes.
    pub length: usize,
}
```

--------------------------------

### Rust FastCDC Chunk Structure

Source: https://docs.rs/fastcdc/3.2.1/src/fastcdc/ronomon/mod.rs

Defines the `Chunk` struct, which represents a single data chunk returned by the FastCDC iterator. It includes the hash of the chunk, its starting offset in the original content, and its length in bytes.

```APIDOC
Chunk:
  hash: u32
  offset: usize
  length: usize
```

--------------------------------

### Rust TryInto Trait API Documentation

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/v2016/struct.ChunkData

Detailed API documentation for the `TryInto` trait in Rust, which enables fallible conversions between types. It includes the trait's generic parameters, its associated error type, and the `try_into` method signature and purpose.

```APIDOC
Trait: TryInto<U> for T
  Description: Implements fallible conversion from type T to type U.
  Constraints: U must implement TryFrom<T>

  Associated Type: Error
    Type: <U as TryFrom<T>>::Error
    Description: The type returned in the event of a conversion error.

  Method: try_into
    Signature: fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>
    Description: Performs the conversion from self (type T) to type U.
    Parameters:
      self: The value of type T to be converted.
    Returns:
      Result<U, <U as TryFrom<T>>::Error>: Ok(U) on successful conversion, Err(Error) on failure.
```

--------------------------------

### Rust Iterator Trait: enumerate Method

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/v2016/struct.FastCDC

The `enumerate` method creates an iterator that yields pairs of `(index, element)`, where `index` is the current iteration count starting from zero, and `element` is the next value from the original iterator.

```APIDOC
Method: enumerate
Signature: fn enumerate(self) -> Enumerate<Self> where Self: Sized
Description: Creates an iterator which gives the current iteration count as well as the next value.
Parameters:
  - self: Self - The iterator instance.
Returns: Enumerate<Self> - A new iterator yielding `(usize, Self::Item)` pairs.
```

--------------------------------

### FastCDC Struct Constructors

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/ronomon/struct.FastCDC

Documents the primary methods for initializing and configuring the `FastCDC` struct, which is used for content-defined chunking of byte data. It includes constructors for single-block processing (`new`) and multi-block processing with end-of-file indication (`with_eof`).

```APIDOC
impl<'a> FastCDC<'a>
  new(source: &'a [u8], min_size: usize, avg_size: usize, max_size: usize) -> Self
    source: The slice of bytes to process.
    min_size: The preferred minimum chunk size.
    avg_size: The desired "normal size" of the chunks.
    max_size: The preferred maximum chunk size.
  with_eof(source: &'a [u8], min_size: usize, avg_size: usize, max_size: usize, eof: bool) -> Self
    source: The slice of bytes to process.
    min_size: The preferred minimum chunk size.
    avg_size: The desired "normal size" of the chunks.
    max_size: The preferred maximum chunk size.
    eof: Indicates if the source contains the final block of data (true) or if more data will follow (false).
```

--------------------------------

### FastCDC v2016 Module API Documentation

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/v2016/index

This section details the API of the `v2016` module within the `fastcdc` crate, which implements the 2016 FastCDC algorithm. It describes the core structs and enums used for content-defined chunking, including how `FastCDC` and `StreamCDC` can be utilized to process data and yield chunks.

```APIDOC
Module: v2016
  Description: This module implements the canonical FastCDC algorithm as described in the paper by Wen Xia, et al., in 2016. The algorithm incorporates a simplified hash judgement using the fast Gear hash, sub-minimum chunk cut-point skipping, and normalized chunking to produce chunks of a more consistent length.
  Usage Notes:
    - The `FastCDC` struct can be used by invoking `cut()` or as an `Iterator` yielding `Chunk` structs.
    - Attempting to use both `cut()` and `Iterator` on the same `FastCDC` instance will yield incorrect results.
    - The `cut()` function returns the 64-bit hash of the chunk, which is also available in the `hash` field of the `Chunk` struct.
    - The `StreamCDC` implementation reads data from a `Read` into an internal buffer of `max_size` and produces `ChunkData` values from its `Iterator`.

  Structs:
    Chunk: Represents a chunk returned from the FastCDC iterator.
    ChunkData: Represents a chunk returned from the StreamCDC iterator.
    FastCDC: The FastCDC chunker implementation from 2016.
    StreamCDC: The FastCDC chunker implementation from 2016 with streaming support.

  Enums:
    Error: The error type returned from the `StreamCDC` iterator.
    Normalization: The level for the normalized chunking used by FastCDC and StreamCDC.
```

--------------------------------

### Get Minimum Element by Custom Comparison (Rust Iterator)

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/v2020/struct.StreamCDC

Returns the element from the iterator that yields the minimum value according to a provided comparison function. This method requires a custom comparison logic to determine the minimum element.

```APIDOC
fn min_by<F>(self, compare: F) -> Option<Self::Item> where Self: Sized, F: FnMut(&Self::Item, &Self::Item) -> Ordering,
```

--------------------------------

### Rust: CloneToUninit Trait Method - clone_to_uninit

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/ronomon/struct.FastCDC

Performs copy-assignment from `self` to `dest`. This is a nightly-only experimental API.

```APIDOC
unsafe fn clone_to_uninit(&self, dest: *mut u8)
```

--------------------------------

### Get Maximum Element by Custom Comparison (Rust Iterator)

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/v2020/struct.StreamCDC

Returns the element from the iterator that yields the maximum value according to a provided comparison function. This method requires a custom comparison logic to determine the maximum element.

```APIDOC
fn max_by<F>(self, compare: F) -> Option<Self::Item> where Self: Sized, F: FnMut(&Self::Item, &Self::Item) -> Ordering,
```

--------------------------------

### APIDOC: FastCDC Chunker Implementation

Source: https://docs.rs/fastcdc/3.2.1/src/fastcdc/v2016/mod.rs

Documentation for the `FastCDC` struct, detailing its role as the 2016 FastCDC chunker implementation. It explains how to construct and iterate over it, and provides guidance on setting minimum and maximum chunk sizes, highlighting their impact on the algorithm.

```APIDOC
The FastCDC chunker implementation from 2016.

Use `new` to construct an instance, and then iterate over the `Chunk`s via
the `Iterator` trait.

This example reads a file into memory and splits it into chunks that are
roughly 16 KB in size. The minimum and maximum sizes are the absolute limit
on the returned chunk sizes. With this algorithm, it is helpful to be more
lenient on the maximum chunk size as the results are highly dependent on the
input data. Changing the minimum chunk size will affect the results as the
algorithm may find different cut points given it uses the minimum as a
starting point (cut-point skipping).
```

--------------------------------

### Get Last Iterator Element in Rust

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/v2016/struct.StreamCDC

Consumes the iterator, returning the last element produced by the iterator wrapped in an `Option`. Returns `None` if the iterator is empty. This method is available on any type that implements the `Iterator` trait and is `Sized`.

```APIDOC
fn last(self) -> Option<Self::Item>
where Self: Sized
```

--------------------------------

### CloneToUninit Trait Blanket Implementation for Cloneable Type T

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/v2020/struct.FastCDC

Documents the blanket implementation of the `CloneToUninit` trait for any type `T` that implements `Clone`. This is a nightly-only experimental API designed for efficient cloning into uninitialized memory regions.

```APIDOC
Trait Implementation:
  impl<T> CloneToUninit for T
  where
    T: Clone
```

--------------------------------

### Get minimum element by key (Rust Iterator)

Source: https://docs.rs/fastcdc/3.2.1/fastcdc/v2016/struct.FastCDC

Returns the element from an iterator that yields the minimum value based on a key extracted by a provided function. The extracted key must implement the `Ord` trait for ordering.

```APIDOC
fn min_by_key<B, F>(self, f: F) -> Option<Self::Item> where B: Ord, Self: Sized, F: FnMut(&Self::Item) -> B,
```