### Rust gzp Parallel Compression Usage Example

Source: https://docs.rs/gzp/latest/gzp/all.html/1.0.1/gzp/par/compress/index

This Rust example demonstrates how to use `ParCompress` from the `gzp` crate to perform parallel Gzip compression. It initializes a `ParCompress` instance with a `Gzip` encoder and writes multiple lines of data, then finalizes the compression.

```Rust
use std::{env, fs::File, io::Write};

use gzp::{par::compress::{ParCompress, ParCompressBuilder}, deflate::Gzip, ZWriter};

let mut writer = vec![];
let mut parz: ParCompress<Gzip> = ParCompressBuilder::new().from_writer(writer);
parz.write_all(b"This is a first test line\n").unwrap();
parz.write_all(b"This is a second test line\n").unwrap();
parz.finish().unwrap();
```

--------------------------------

### Rust: Parallel Zlib Compression Example

Source: https://docs.rs/gzp/latest/gzp/all.html/1.0.1/src/gzp/deflate.rs

Demonstrates how to use `ParCompress` with the `Zlib` format to compress data in parallel. This example initializes a writer, writes multiple lines of data, and then finalizes the compression stream.

```Rust
# #[cfg(feature = "any_zlib")] {
use std::{env, fs::File, io::Write};

use gzp::{deflate::Zlib, par::compress::{ParCompress, ParCompressBuilder}, ZWriter};

let mut writer = vec![];
let mut parz: ParCompress<Zlib> = ParCompressBuilder::new().from_writer(writer);
parz.write_all(b"This is a first test line\n").unwrap();
parz.write_all(b"This is a second test line\n").unwrap();
parz.finish().unwrap();
# }
```

--------------------------------

### Rust: Compress Data in Parallel using gzp::par::compress

Source: https://docs.rs/gzp/latest/gzp/all.html/1.0.1/src/gzp/par/compress.rs

This Rust example demonstrates how to use `ParCompress` from the `gzp` library to compress data in parallel. It initializes a `ParCompress` instance with a `Gzip` format, writes multiple lines of data, and then finalizes the compression. This snippet showcases basic setup and data writing for parallel compression.

```Rust
# #[cfg(feature = "deflate")] {
use std::{env, fs::File, io::Write};

use gzp::{par::compress::{ParCompress, ParCompressBuilder}, deflate::Gzip, ZWriter};

let mut writer = vec![];
let mut parz: ParCompress<Gzip> = ParCompressBuilder::new().from_writer(writer);
parz.write_all(b"This is a first test line\n").unwrap();
parz.write_all(b"This is a second test line\n").unwrap();
parz.finish().unwrap();
# }
```

--------------------------------

### Rust BGZF Data Compression and Decompression Example

Source: https://docs.rs/gzp/latest/gzp/all.html/1.0.1/src/gzp/bgzf.rs

This Rust code snippet demonstrates the end-to-end process of compressing data using `BgzfSyncWriter` and then decompressing it with `BgzfSyncReader`. It takes an input byte slice, writes it to a BGZF stream, reads the compressed output, and finally decompresses it, asserting that the original and decompressed data are identical. This example requires the `flate2` or a similar BGZF-compatible crate for `BgzfSyncWriter` and `BgzfSyncReader`.

```Rust
        // Compress input to output
        let mut bgzf = BgzfSyncWriter::new(out_writer, Compression::new(3));
        bgzf.write_all(input).unwrap();
        bgzf.flush().unwrap();
        drop(bgzf);
        // dbg!(output_file);
        // dbg!(orig_file);
        // std::process::exit(1);

        // Read output back in
        let mut reader = BufReader::new(File::open(output_file).unwrap());
        let mut result = vec![];
        reader.read_to_end(&mut result).unwrap();

        // Decompress it
        let mut decoder = BgzfSyncReader::new(&result[..]);
        // let mut gz = MultiGzDecoder::new(&result[..]);
        let mut bytes = vec![];
        decoder.read_to_end(&mut bytes).unwrap();

        // Assert decompressed output is equal to input
        assert_eq!(input.to_vec(), bytes);
```

--------------------------------

### Rust Gzip Compression and Decompression Example

Source: https://docs.rs/gzp/latest/gzp/all.html/latest/src/gzp/deflate.rs

This Rust snippet demonstrates a full cycle of Gzip compression and decompression. It writes input bytes to a temporary file using `SyncZBuilder` for compression, then reads the compressed data back and decompresses it using `GzDecoder`. The example verifies that the decompressed output matches the original input. This code relies on `std::io::{BufReader, BufWriter, Read, Write}`, `std::fs::File`, `tempfile::tempdir`, and types like `Gzip`, `SyncZBuilder`, `GzDecoder` from a compression library (e.g., `flate2`).

```Rust
    #[test]
    fn test_simple_gzip() {
        let dir = tempdir().unwrap();

        // Create output file
        let output_file = dir.path().join("output.txt");
        let out_writer = BufWriter::new(File::create(&output_file).unwrap());

        // Define input bytes
        let input = b"
        This is a longer test than normal to come up with a bunch of text.
        We'll read just a few lines at a time.
        ";

        // Compress input to output
        let mut z = SyncZBuilder::<Gzip, _>::new().from_writer(out_writer);
        z.write_all(input).unwrap();
        drop(z);

        // Read output back in
        let mut reader = BufReader::new(File::open(output_file).unwrap());
        let mut result = vec![];
        reader.read_to_end(&mut result).unwrap();

        // Decompress it
        let mut gz = GzDecoder::new(&result[..]);
        let mut bytes = vec![];
        gz.read_to_end(&mut bytes).unwrap();

        // Assert decompressed output is equal to input
        assert_eq!(input.to_vec(), bytes);
    }
```

--------------------------------

### Rust: Parallel Zlib Compression Example

Source: https://docs.rs/gzp/latest/gzp/all.html/latest/src/gzp/deflate.rs

Demonstrates how to use `ParCompress` with `Zlib` to compress data in parallel. This example writes two lines of text to a `Vec<u8>` and then finishes the compression stream. Requires the `any_zlib` feature.

```Rust
# #[cfg(feature = "any_zlib")] {
use std::{env, fs::File, io::Write};

use gzp::{deflate::Zlib, par::compress::{ParCompress, ParCompressBuilder}, ZWriter};

let mut writer = vec![];
let mut parz: ParCompress<Zlib> = ParCompressBuilder::new().from_writer(writer);
parz.write_all(b"This is a first test line\n").unwrap();
parz.write_all(b"This is a second test line\n").unwrap();
parz.finish().unwrap();
# }
```

--------------------------------

### Rust Regression Test Setup for Compression with Byte Array Input

Source: https://docs.rs/gzp/latest/gzp/all.html/1.0.1/src/gzp/deflate.rs

This snippet sets up a regression test in Rust, preparing an output file and defining a specific byte array as input for a compression operation. The provided code segment focuses on the initial setup and input definition, with the compression logic expected to follow.

```Rust
    #[test]
    fn test_regression() {
        let dir = tempdir().unwrap();

        // Create output file
        let output_file = dir.path().join("output.txt");
        let out_writer = BufWriter::new(File::create(&output_file).unwrap());

        // Define input bytes that is 206 bytes long
        // let input = b"The quick brown fox jumped over the moon\n";
        let input = [
            132, 19, 107, 159, 69, 217, 180, 131, 224, 49, 143, 41, 194, 30, 151, 22, 55, 30, 42,
            139, 219, 62, 123, 44, 148, 144, 88, 233, 199, 126, 110, 65, 6, 87, 51, 215, 17, 253,
            22, 63, 110, 1, 100, 202, 44, 138, 187, 226, 50, 50, 218, 24, 193, 218, 43, 172, 69,
            71, 8, 164, 5, 186, 189, 215, 151, 170, 243, 235, 219, 103, 1, 0, 102, 80, 179, 95,
            247, 26, 168, 147, 139, 245, 177, 253, 94, 82, 146, 133, 103, 223, 96, 34, 128, 237,
            143, 182, 48, 201, 201, 92, 29, 172, 137, 70, 227, 98, 181, 246, 80, 21, 106, 175, 246,
            41, 229, 187, 87, 65, 79, 63, 115, 66, 143, 251, 41, 251, 214, 7, 64, 196, 27, 180, 42,
            132, 116, 211, 148, 44, 177, 137, 91, 119, 245, 156, 78, 24, 253, 69, 38, 52, 152, 115,
            123, 94, 162, 72, 186, 239, 136, 179, 11, 180, 78, 54, 217, 120, 173, 141, 114, 174,
            220, 160, 223, 184, 114, 73, 148, 120, 43, 25, 21, 62, 62, 244, 85, 87, 19, 174, 182,
            227, 228, 70, 153, 5, 92, 51, 161, 9, 140, 199, 244, 241, 151, 236, 81, 211,
        ];

        // Compress input to output
```

--------------------------------

### Rust Zlib Compression and Decompression Examples

Source: https://docs.rs/gzp/latest/gzp/all.html/latest/src/gzp/deflate.rs

These Rust snippets illustrate Zlib compression and decompression, showcasing both parallel and synchronous approaches. The first example uses `ParCompressBuilder` for parallel compression, while the second uses `SyncZBuilder` for synchronous compression, both writing to a temporary file. `ZlibDecoder` is then used to decompress the data. Both examples assert that the decompressed output matches the original input. These snippets require the `any_zlib` feature to be enabled and rely on `std::io::{BufReader, BufWriter, Read, Write}`, `std::fs::File`, `tempfile::tempdir`, and types like `Zlib`, `ParCompressBuilder`, `SyncZBuilder`, `ZlibDecoder` from a compression library (e.g., `flate2`).

```Rust
    #[test]
    #[cfg(feature = "any_zlib")]
    fn test_simple_zlib() {
        let dir = tempdir().unwrap();

        // Create output file
        let output_file = dir.path().join("output.txt");
        let out_writer = BufWriter::new(File::create(&output_file).unwrap());

        // Define input bytes
        let input = b"\
        This is a longer test than normal to come up with a bunch of text.\n\
        We'll read just a few lines at a time.\n\
        ";

        // Compress input to output
        let mut par_gz: ParCompress<Zlib> = ParCompressBuilder::new().from_writer(out_writer);
        par_gz.write_all(input).unwrap();
        par_gz.finish().unwrap();

        // Read output back in
        let mut reader = BufReader::new(File::open(output_file).unwrap());
        let mut result = vec![];
        reader.read_to_end(&mut result).unwrap();

        // Decompress it
        let mut gz = ZlibDecoder::new(&result[..]);
        let mut bytes = vec![];
        gz.read_to_end(&mut bytes).unwrap();

        // Assert decompressed output is equal to input
        assert_eq!(input.to_vec(), bytes);
    }
```

```Rust
    #[test]
    #[cfg(feature = "any_zlib")]
    fn test_simple_zlib_sync() {
        let dir = tempdir().unwrap();

        // Create output file
        let output_file = dir.path().join("output.txt");
        let out_writer = BufWriter::new(File::create(&output_file).unwrap());

        // Define input bytes
        let input = b"\
        This is a longer test than normal to come up with a bunch of text.\n\
        We'll read just a few lines at a time.\n\
        ";

        // Compress input to output
        let mut z = SyncZBuilder::<Zlib, _>::new().from_writer(out_writer);
        z.write_all(input).unwrap();
        z.finish().unwrap();

        // Read output back in
        let mut reader = BufReader::new(File::open(output_file).unwrap());
        let mut result = vec![];
        reader.read_to_end(&mut result).unwrap();

        // Decompress it
        let mut gz = ZlibDecoder::new(&result[..]);
        let mut bytes = vec![];
        gz.read_to_end(&mut bytes).unwrap();

        // Assert decompressed output is equal to input
        assert_eq!(input.to_vec(), bytes);
    }
```

--------------------------------

### Synchronous Compress and Decompress with Rust Gzip (Drop)

Source: https://docs.rs/gzp/latest/gzp/all.html/1.0.1/src/gzp/deflate.rs

This example begins to demonstrate synchronous compression using `SyncZBuilder` with `Gzip`, focusing on the effect of dropping the compressor. The snippet provided is incomplete, but it sets up the initial file and input for a compression test.

```Rust
#[test]
fn test_simple_sync_drop() {
    let dir = tempdir().unwrap();

    // Create output file
    let output_file = dir.path().join("output.txt");
    let out_writer = BufWriter::new(File::create(&output_file).unwrap());
```

--------------------------------

### Example: Configuring Parallel Gzip Compression in Rust

Source: https://docs.rs/gzp/latest/gzp/all.html/latest/src/gzp/par/compress.rs

This Rust code snippet demonstrates how to initialize and use `ParCompressBuilder` to perform parallel Gzip compression. It sets up a `ParCompress` instance, writes two lines of byte data, and then finalizes the compression process, writing the compressed output to a `Vec<u8>` buffer. This example requires the `deflate` feature to be enabled.

```Rust
# #[cfg(feature = "deflate")] {
use std::{env, fs::File, io::Write};

use gzp::{par::compress::{ParCompress, ParCompressBuilder}, deflate::Gzip, ZWriter};

let mut writer = vec![];
let mut parz: ParCompress<Gzip> = ParCompressBuilder::new().from_writer(writer);
parz.write_all(b"This is a first test line\n").unwrap();
parz.write_all(b"This is a second test line\n").unwrap();
parz.finish().unwrap();
# }
```

--------------------------------

### Compress and Decompress with Rust Gzip

Source: https://docs.rs/gzp/latest/gzp/all.html/1.0.1/src/gzp/deflate.rs

This example shows how to compress data using `ParCompress` with `Gzip` and then decompress it using `GzDecoder`. It writes the compressed data to a temporary file and verifies that the decompressed output matches the original input. This demonstrates a standard parallel compression and decompression workflow.

```Rust
#[test]
fn test_simple() {
    let dir = tempdir().unwrap();

    // Create output file
    let output_file = dir.path().join("output.txt");
    let out_writer = BufWriter::new(File::create(&output_file).unwrap());

    // Define input bytes
    let input = b"
    This is a longer test than normal to come up with a bunch of text.
    We'll read just a few lines at a time.
    ";

    // Compress input to output
    let mut par_gz: ParCompress<Gzip> = ParCompressBuilder::new().from_writer(out_writer);
    par_gz.write_all(input).unwrap();
    par_gz.finish().unwrap();

    // Read output back in
    let mut reader = BufReader::new(File::open(output_file).unwrap());
    let mut result = vec![];
    reader.read_to_end(&mut result).unwrap();

    // Decompress it
    let mut gz = GzDecoder::new(&result[..]);
    let mut bytes = vec![];
    gz.read_to_end(&mut bytes).unwrap();

    // Assert decompressed output is equal to input
    assert_eq!(input.to_vec(), bytes);
}
```

--------------------------------

### Rust Gzp: Create ParCompress Object from Writer

Source: https://docs.rs/gzp/latest/gzp/all.html/1.0.1/gzp/par/compress/struct.ParCompressBuilder

This method finalizes the `ParCompressBuilder` configuration and creates a `ParCompress` object, ready for parallel compression. It takes a writer implementing `Write` and `Send` traits, which will receive the compressed output. This is the entry point to start the compression process.

```Rust
pub fn from_writer<W: Write + Send + 'static>(self, writer: W) -> ParCompress<F>
```

```APIDOC
Method: from_writer
Description: Create a configured ParCompress object from the builder.
Signature: pub fn from_writer<W: Write + Send + 'static>(self, writer: W) -> ParCompress<F>
Parameters:
  - writer: W (A writer implementing the Write and Send traits, where compressed data will be written.)
Returns:
  - ParCompress<F> (A new ParCompress object configured with the builder's settings.)
```

--------------------------------

### Rust: Compress and Decompress Data with BGZF

Source: https://docs.rs/gzp/latest/gzp/all.html/latest/src/gzp/bgzf.rs

This snippet shows a complete workflow for compressing a byte array to a file using `BgzfSyncWriter` and then decompressing it back using `BgzfSyncReader`. It includes file creation, writing, reading, and assertion of data integrity. This example requires the `bgzf` crate.

```Rust
        // Compress input to output
        let mut bgzf = BgzfSyncWriter::new(out_writer, Compression::new(3));
        bgzf.write_all(input).unwrap();
        bgzf.flush().unwrap();
        drop(bgzf);
        // dbg!(output_file);
        // dbg!(orig_file);
        // std::process::exit(1);

        // Read output back in
        let mut reader = BufReader::new(File::open(output_file).unwrap());
        let mut result = vec![];
        reader.read_to_end(&mut result).unwrap();

        // Decompress it
        let mut decoder = BgzfSyncReader::new(&result[..]);
        // let mut gz = MultiGzDecoder::new(&result[..]);
        let mut bytes = vec![];
        decoder.read_to_end(&mut bytes).unwrap();

        // Assert decompressed output is equal to input
        assert_eq!(input.to_vec(), bytes);
```

--------------------------------

### Rust ZBuilder Struct and API

Source: https://docs.rs/gzp/latest/gzp/all.html/1.0.1/src/gzp/lib.rs

The `ZBuilder` struct provides a flexible way to configure and create `ZWriter` instances for parallel compression. It allows customization of various parameters before constructing the final writer.

**Fields:**
- `num_threads`: `usize` - Number of threads to use for compression.
- `pin_threads`: `Option<usize>` - Optional physical CPU to start pinning threads at.
- `compression_level`: `Compression` - The desired compression level.
- `buffer_size`: `usize` - The buffer size to use for compression.
- `writer`: `PhantomData<W>` - Placeholder for the writer type.
- `format`: `PhantomData<F>` - Placeholder for the format spec type.

**Methods:**
- `new()`: Creates a new `ZBuilder` with default settings. Returns `Self`.
- `compression_level(compression_level: Compression)`: Sets the compression level. Returns `Self`.
- `num_threads(num_threads: usize)`: Sets the number of threads to use for compression. Returns `Self`.
- `pin_threads(pin_threads: Option<usize>)`: Configures whether or not to pin compression threads and which physical CPU to start pinning at. Returns `Self`.
- `buffer_size(buffer_size: usize)`: Sets the buffer size to use. Returns `Self`.
- `from_writer(self, writer: W)`: Creates a `Box<dyn ZWriter>` trait object from the configured builder and an input writer. Returns `Box<dyn ZWriter>`.

```Rust
/// Unified builder that returns a trait object
pub struct ZBuilder<F, W>
where
    F: FormatSpec + SyncWriter<W>,
    W: Write + Send + 'static,
{
    num_threads: usize,
    pin_threads: Option<usize>,
    compression_level: Compression,
    buffer_size: usize,
    writer: PhantomData<W>,
    format: PhantomData<F>,
}

impl<F, W> ZBuilder<F, W>
where
    F: FormatSpec + SyncWriter<W>,
    W: Write + Send + 'static,
{
    pub fn new() -> Self {
        Self {
            num_threads: num_cpus::get(),
            pin_threads: None,
            compression_level: Compression::new(3),
            buffer_size: F::DEFAULT_BUFSIZE,
            writer: PhantomData,
            format: PhantomData,
        }
    }

    pub fn compression_level(mut self, compression_level: Compression) -> Self {
        self.compression_level = compression_level;
        self
    }

    /// Number of threads to use for compression
    pub fn num_threads(mut self, num_threads: usize) -> Self {
        self.num_threads = num_threads;
        self
    }

    /// Whether or not to pin compression threads and which physical CPU to start pinning at.
    pub fn pin_threads(mut self, pin_threads: Option<usize>) -> Self {
        if core_affinity::get_core_ids().is_none() {
            warn!("Pinning threads is not supported on your platform. Please see core_affinity_rs. No threads will be pinned, but everything will work.");
            self.pin_threads = None;
        } else {
            self.pin_threads = pin_threads;
        }
        self
    }

    /// Buffer size to use (the effect of this may vary depending on `F`),
    /// check the documentation on the `F` type you are creating to see if
    /// there are restrictions on the buffer size.
    pub fn buffer_size(mut self, buffer_size: usize) -> Self {
        self.buffer_size = buffer_size;
        self
    }

    /// Create a [`ZWriter`] trait object from a writer.
    #[allow(clippy::missing_panics_doc)]
    pub fn from_writer(self, writer: W) -> Box<dyn ZWriter>
    where
        SyncZ<<F as SyncWriter<W>>::OutputWriter>: ZWriter + Send,
    {
        if self.num_threads > 1 {
            Box::new(
                ParCompressBuilder::<F>::new()
                    .compression_level(self.compression_level)
                    .num_threads(self.num_threads)
                    .unwrap()
                    .buffer_size(self.buffer_size)
                    .unwrap()
                    .pin_threads(self.pin_threads)
                    .from_writer(writer),
            )
        } else {
            Box::new(
                SyncZBuilder::<F, W>::new()
                    .compression_level(self.compression_level)
                    .from_writer(writer),
            )
        }
    }
}
```

--------------------------------

### Rust: Configure and Build SyncZ Single-threaded Compressor

Source: https://docs.rs/gzp/latest/gzp/all.html/1.0.1/src/gzp/syncz.rs

The `SyncZBuilder` struct provides a fluent API to configure and instantiate a `SyncZ` single-threaded compressor. It allows setting the compression level and creating a `SyncZ` instance from a given writer. This builder ensures proper setup before compression operations.

```APIDOC
struct SyncZBuilder<F, W>
  where F: FormatSpec + SyncWriter<W>, W: Write
  Fields:
    compression_level: Compression
    format: PhantomData<F>
    phantom: PhantomData<W>

  Methods:
    new(): Self
      Description: Creates a new builder with default compression level (3).
      Returns: Self

    compression_level(compression_level: Compression): Self
      Description: Sets the compression level for the builder.
      Parameters:
        compression_level: Compression - The desired compression level.
      Returns: Self

    from_writer(writer: W): SyncZ<F::OutputWriter>
      Description: Creates a SyncZ instance from a writer with the configured compression level.
      Parameters:
        writer: W - The underlying writer to compress data into.
      Returns: SyncZ<F::OutputWriter> - A new SyncZ compressor.

  Implementations:
    impl Default for SyncZBuilder<F, W>
      default(): Self
        Description: Returns a new SyncZBuilder with default settings.
        Returns: Self
```

```Rust
pub struct SyncZBuilder<F, W>
where
    F: FormatSpec + SyncWriter<W>,
    W: Write,
{
    compression_level: Compression,
    format: PhantomData<F>,
    phantom: PhantomData<W>,
}

impl<F, W> SyncZBuilder<F, W>
where
    F: FormatSpec + SyncWriter<W>,
    W: Write,
{
    pub fn new() -> Self {
        Self {
            compression_level: Compression::new(3),
            format: PhantomData,
            phantom: PhantomData,
        }
    }

    pub fn compression_level(mut self, compression_level: Compression) -> Self {
        self.compression_level = compression_level;
        self
    }

    pub fn from_writer(self, writer: W) -> SyncZ<F::OutputWriter> {
        SyncZ {
            inner: Some(F::sync_writer(writer, self.compression_level)),
        }
    }
}

impl<F, W> Default for SyncZBuilder<F, W>
where
    F: FormatSpec + SyncWriter<W>,
    W: Write,
{
    fn default() -> Self {
        Self::new()
    }
}
```

--------------------------------

### Rust: Create ParCompress Builder Instance

Source: https://docs.rs/gzp/latest/gzp/all.html/latest/gzp/par/compress/struct.ParCompress

Provides a static method to create a `ParCompressBuilder` instance. This builder allows for configuring various parameters of the `ParCompress` runtime before instantiation, enabling flexible setup for parallel compression tasks and custom compression behavior.

```Rust
pub fn builder() -> ParCompressBuilder<F>
```

--------------------------------

### Rust ZBuilder Default Implementation

Source: https://docs.rs/gzp/latest/gzp/all.html/latest/src/gzp/lib.rs

Provides a default implementation for the `ZBuilder` struct, allowing it to be created using `ZBuilder::default()`. This simplifies instantiation for users who wish to start with the standard configuration without explicitly calling `new()`.

```Rust
impl<F, W> Default for ZBuilder<F, W>
where
    F: FormatSpec + SyncWriter<W>,
    W: Write + Send + 'static,
{
    fn default() -> Self {
        Self::new()
    }
}
```

--------------------------------

### Rust ZBuilder::new Constructor for Default Configuration

Source: https://docs.rs/gzp/latest/gzp/all.html/latest/src/gzp/lib.rs

Creates a new `ZBuilder` instance with default configuration values. This includes setting the number of threads to the CPU count, a default compression level, and a buffer size based on the format specification, providing a convenient starting point for builder customization.

```Rust
    pub fn new() -> Self {
        Self {
            num_threads: num_cpus::get(),
            pin_threads: None,
            compression_level: Compression::new(3),
            buffer_size: F::DEFAULT_BUFSIZE,
            writer: PhantomData,
            format: PhantomData,
        }
    }
```

--------------------------------

### API Documentation for gzp::Pair Struct

Source: https://docs.rs/gzp/latest/gzp/all.html/1.0.1/gzp/struct.Pair

This section provides a detailed API reference for the `gzp::Pair` struct, outlining its structure, purpose, and all implemented traits and methods. It serves as a comprehensive guide for developers interacting with header and footer information in `gzp`'s compression logic.

```APIDOC
Struct: Pair
  Description: A `Pair` is used to represent header or footer information.
  Source: ../src/gzp/lib.rs.html#315-320
  Definition: pub struct Pair { /* private fields */ }

  Trait Implementations:
    Trait: Debug (core::fmt::Debug)
      Source: ../src/gzp/lib.rs.html#314
      Method: fmt
        Signature: fn fmt(&self, f: &mut Formatter<'_>) -> Result
        Parameters:
          - name: self
            type: &Self
            description: The instance of the Pair struct.
          - name: f
            type: &mut Formatter<'_>
            description: The formatter to use.
        Return:
          type: Result (core::fmt::Result)
          description: The formatting result.
        Description: Formats the value using the given formatter.

    Trait: Freeze (core::marker::Freeze)
    Trait: RefUnwindSafe (core::panic::unwind_safe::RefUnwindSafe)
    Trait: Send (core::marker::Send)
    Trait: Sync (core::marker::Sync)
    Trait: Unpin (core::marker::Unpin)
    Trait: UnwindSafe (core::marker::UnwindSafe)

  Blanket Implementations:
    Trait: Any
    Trait: Borrow<T>
    Trait: BorrowMut<T>
    Trait: From<T>
    Trait: Into<U>
    Trait: TryFrom<U>
    Trait: TryInto<U>
```

--------------------------------

### Initialize ParDecompressBuilder with Default Settings in Rust

Source: https://docs.rs/gzp/latest/gzp/all.html/latest/src/gzp/par/decompress.rs

This method creates a new instance of ParDecompressBuilder with default configuration. It sets the buffer size to BUFSIZE, the number of threads to the number of available CPUs, and initializes the format spec. This provides a convenient starting point for building a decompressor.

```Rust
impl<F> ParDecompressBuilder<F>
where
    F: BlockFormatSpec,
{
    pub fn new() -> Self {
        Self {
            buffer_size: BUFSIZE,
            num_threads: num_cpus::get(),
            format: F::new(),
            pin_threads: None,
        }
    }
}
```

--------------------------------

### Rust: Perform Single-Threaded Compression with gzp

Source: https://docs.rs/gzp/latest/gzp/all.html/1.0.1/gzp/index

This example illustrates single-threaded compression using `gzp`'s `SyncZBuilder`. It sets up a synchronous writer with `Gzip` encoding, writes data sequentially, and then completes the compression process. This is suitable for scenarios where parallel processing is not required or desired.

```Rust
use std::{env, fs::File, io::Write};

use gzp::{deflate::Gzip, syncz::SyncZBuilder, ZWriter};

let mut writer = vec![];
let mut parz = SyncZBuilder::<Gzip, _>::new().from_writer(writer);
parz.write_all(b"This is a first test line\n").unwrap();
parz.write_all(b"This is a second test line\n").unwrap();
parz.finish().unwrap();
```

--------------------------------

### Rust: Perform Single-Threaded Gzip Compression

Source: https://docs.rs/gzp/latest/gzp/all.html/latest/gzp/index

This example illustrates single-threaded Gzip compression using `gzp`'s `SyncZBuilder`. It sets up a synchronous writer, writes data sequentially, and then completes the compression. This is suitable for scenarios where parallel processing is not required or desired.

```Rust
use std::{env, fs::File, io::Write};

use gzp::{deflate::Gzip, syncz::SyncZBuilder, ZWriter};

let mut writer = vec![];
let mut parz = SyncZBuilder::<Gzip, _>::new().from_writer(writer);
parz.write_all(b"This is a first test line\n").unwrap();
parz.write_all(b"This is a second test line\n").unwrap();
parz.finish().unwrap();
```

--------------------------------

### Compress and Decompress with Rust Gzip (Drop)

Source: https://docs.rs/gzp/latest/gzp/all.html/1.0.1/src/gzp/deflate.rs

This example illustrates compressing data with `ParCompress` and `Gzip`, explicitly dropping the compressor to ensure all buffered data is flushed to the output. It then decompresses the data using `GzDecoder` and asserts equality with the original input, demonstrating proper resource management.

```Rust
#[test]
fn test_simple_drop() {
    let dir = tempdir().unwrap();

    // Create output file
    let output_file = dir.path().join("output.txt");
    let out_writer = BufWriter::new(File::create(&output_file).unwrap());

    // Define input bytes
    let input = b"
    This is a longer test than normal to come up with a bunch of text.
    We'll read just a few lines at a time.
    ";

    // Compress input to output
    let mut par_gz: ParCompress<Gzip> = ParCompressBuilder::new().from_writer(out_writer);
    par_gz.write_all(input).unwrap();
    drop(par_gz);

    // Read output back in
    let mut reader = BufReader::new(File::open(output_file).unwrap());
    let mut result = vec![];
    reader.read_to_end(&mut result).unwrap();

    // Decompress it
    let mut gz = GzDecoder::new(&result[..]);
    let mut bytes = vec![];
    gz.read_to_end(&mut bytes).unwrap();

    // Assert decompressed output is equal to input
    assert_eq!(input.to_vec(), bytes);
}
```

--------------------------------

### Compress and Decompress with Rust Mgzip

Source: https://docs.rs/gzp/latest/gzp/all.html/1.0.1/src/gzp/deflate.rs

This example demonstrates how to compress data using `ParCompress` with `Mgzip` and then decompress it using `MultiGzDecoder`. It writes the compressed data to a temporary file and asserts that the decompressed output matches the original input. Key dependencies include `tempdir` for temporary file handling and `std::io` for buffered file operations.

```Rust
use crate::{ZBuilder, ZWriter, BUFSIZE, DICT_SIZE};

use super::*;

#[test]
fn test_simple_mgzip() {
    let dir = tempdir().unwrap();

    // Create output file
    let output_file = dir.path().join("output.txt");
    let out_writer = BufWriter::new(File::create(&output_file).unwrap());

    // Define input bytes
    let input = b"
    This is a longer test than normal to come up with a bunch of text.
    We'll read just a few lines at a time.
    ";

    // Compress input to output
    let mut par_gz: ParCompress<Mgzip> = ParCompressBuilder::new().from_writer(out_writer);
    par_gz.write_all(input).unwrap();
    par_gz.finish().unwrap();

    // Read output back in
    let mut reader = BufReader::new(File::open(output_file).unwrap());
    let mut result = vec![];
    reader.read_to_end(&mut result).unwrap();

    // Decompress it
    let mut gz = MultiGzDecoder::new(&result[..]);
    let mut bytes = vec![];
    gz.read_to_end(&mut bytes).unwrap();

    // Assert decompressed output is equal to input
    assert_eq!(input.to_vec(), bytes);
}
```

--------------------------------

### Rust BGZF Compression Test Setup

Source: https://docs.rs/gzp/latest/gzp/all.html/latest/src/gzp/deflate.rs

This test verifies the compression functionality of the BGZF format. It generates random input data and parameters for BGZF compression, including buffer size, number of threads, write size, and compression level. The data is compressed and written to a temporary file. The snippet sets up the compression process, aiming to ensure that the BGZF compression process completes successfully and produces valid compressed output.

```Rust
        #[test]
        #[ignore]
        fn test_all_bgzf_compress(
            input in prop::collection::vec(0..u8::MAX, 1..(DICT_SIZE * 10)),
            buf_size in DICT_SIZE..BGZF_BLOCK_SIZE,
            num_threads in 0..num_cpus::get(),
            write_size in 1..(4*BGZF_BLOCK_SIZE),
            mut comp_level in 1..9_u32
        ) {
            let dir = tempdir().unwrap();

            // Create output file
            let output_file = dir.path().join("output.txt");
            let out_writer = BufWriter::new(File::create(&output_file).unwrap());

            // This is a bit of a hack, other libs are not compressing the input data well
            // and the resulting block size is larger than it should be for small compression
```

--------------------------------

### Synchronous Compress and Decompress with Rust Gzip

Source: https://docs.rs/gzp/latest/gzp/all.html/1.0.1/src/gzp/deflate.rs

This example demonstrates synchronous compression using `SyncZBuilder` with `Gzip` and subsequent decompression with `GzDecoder`. It writes the compressed data to a temporary file and verifies that the decompressed output matches the original input, showcasing a non-parallel compression approach.

```Rust
#[test]
fn test_simple_sync() {
    let dir = tempdir().unwrap();

    // Create output file
    let output_file = dir.path().join("output.txt");
    let out_writer = BufWriter::new(File::create(&output_file).unwrap());

    // Define input bytes
    let input = b"
    This is a longer test than normal to come up with a bunch of text.
    We'll read just a few lines at a time.
    ";

    // Compress input to output
    let mut z = SyncZBuilder::<Gzip, _>::new().from_writer(out_writer);
    z.write_all(input).unwrap();
    z.finish().unwrap();

    // Read output back in
    let mut reader = BufReader::new(File::open(output_file).unwrap());
    let mut result = vec![];
    reader.read_to_end(&mut result).unwrap();

    // Decompress it
    let mut gz = GzDecoder::new(&result[..]);
    let mut bytes = vec![];
    gz.read_to_end(&mut bytes).unwrap();

    // Assert decompressed output is equal to input
    assert_eq!(input.to_vec(), bytes);
}
```

--------------------------------

### gzp ZBuilder Struct API Reference

Source: https://docs.rs/gzp/latest/gzp/all.html/1.0.1/gzp/struct.ZBuilder

Detailed API documentation for the `ZBuilder` struct's public methods, used to configure and build compression/decompression writers. This includes methods for initialization, setting compression level, buffer size, and thread management.

```APIDOC
Class: ZBuilder<F, W>
Description: Unified builder that returns a trait object for compression/decompression.
Generics:
  - F: FormatSpec + SyncWriter<W>
  - W: Write + Send + 'static

Methods:
  - Name: new
    Signature: pub fn new() -> Self
    Description: Creates a new `ZBuilder` instance.
    Parameters: []
    Returns: Self

  - Name: compression_level
    Signature: pub fn compression_level(self, compression_level: Compression) -> Self
    Description: Sets the compression level for the builder.
    Parameters:
      - Name: compression_level
        Type: Compression
        Description: The desired compression level.
    Returns: Self

  - Name: buffer_size
    Signature: pub fn buffer_size(self, buffer_size: usize) -> Self
    Description: Sets the internal buffer size for the builder.
    Parameters:
      - Name: buffer_size
        Type: usize
        Description: The buffer size in bytes.
    Returns: Self

  - Name: from_writer
    Signature: pub fn from_writer(writer: W) -> Self
    Description: Creates a `ZBuilder` instance from an existing writer.
    Parameters:
      - Name: writer
        Type: W
        Description: The underlying writer.
    Returns: Self

  - Name: num_threads
    Signature: pub fn num_threads(self, num_threads: usize) -> Self
    Description: Sets the number of threads to use for compression/decompression.
    Parameters:
      - Name: num_threads
        Type: usize
        Description: The number of threads.
    Returns: Self

  - Name: pin_threads
    Signature: pub fn pin_threads(self, pin_threads: bool) -> Self
    Description: Configures whether to pin threads to CPU cores.
    Parameters:
      - Name: pin_threads
        Type: bool
        Description: True to pin threads, false otherwise.
    Returns: Self
```

--------------------------------

### Rust Example: Parallel Zlib Compression with gzp

Source: https://docs.rs/gzp/latest/gzp/all.html/latest/gzp/deflate/index

This Rust code snippet demonstrates how to use the `gzp::par::compress::ParCompress` builder with `gzp::deflate::Zlib` to perform parallel Zlib compression. It writes multiple lines of data to an in-memory buffer and finalizes the compression, showcasing basic usage of the parallel compressor.

```Rust
use std::{env, fs::File, io::Write};

use gzp::{deflate::Zlib, par::compress::{ParCompress, ParCompressBuilder}, ZWriter};

let mut writer = vec![];
let mut parz: ParCompress<Zlib> = ParCompressBuilder::new().from_writer(writer);
parz.write_all(b"This is a first test line\n").unwrap();
parz.write_all(b"This is a second test line\n").unwrap();
parz.finish().unwrap();
```

--------------------------------

### Rust ParCompressBuilder: Create New Instance

Source: https://docs.rs/gzp/latest/gzp/all.html/latest/gzp/par/compress/struct.ParCompressBuilder

Initializes a new `ParCompressBuilder` object. This method serves as the entry point for configuring parallel compression settings before building the compressor.

```Rust
pub fn new() -> Self
// Description: Create a new ParCompressBuilder object.
// Returns:
//   - Self: A new ParCompressBuilder instance.
```

--------------------------------

### Rust Zlib Compression and Decompression with ParCompress

Source: https://docs.rs/gzp/latest/gzp/all.html/1.0.1/src/gzp/deflate.rs

This Rust example illustrates compressing and decompressing data using the Zlib algorithm with ParCompress for parallel compression. It writes input bytes to a temporary file, reads the compressed data, and then uses ZlibDecoder to decompress it. This test requires the "any_zlib" feature to be enabled.

```Rust
    #[test]
    #[cfg(feature = "any_zlib")]
    fn test_simple_zlib() {
        let dir = tempdir().unwrap();

        // Create output file
        let output_file = dir.path().join("output.txt");
        let out_writer = BufWriter::new(File::create(&output_file).unwrap());

        // Define input bytes
        let input = b"\
        This is a longer test than normal to come up with a bunch of text.\n\
        We'll read just a few lines at a time.\n\
        ";

        // Compress input to output
        let mut par_gz: ParCompress<Zlib> = ParCompressBuilder::new().from_writer(out_writer);
        par_gz.write_all(input).unwrap();
        par_gz.finish().unwrap();

        // Read output back in
        let mut reader = BufReader::new(File::open(output_file).unwrap());
        let mut result = vec![];
        reader.read_to_end(&mut result).unwrap();

        // Decompress it
        let mut gz = ZlibDecoder::new(&result[..]);
        let mut bytes = vec![];
        gz.read_to_end(&mut bytes).unwrap();

        // Assert decompressed output is equal to input
        assert_eq!(input.to_vec(), bytes);
    }
```

--------------------------------

### Rust ZBuilder::new Method

Source: https://docs.rs/gzp/latest/gzp/all.html/latest/gzp/struct.ZBuilder

Constructs a new `ZBuilder` instance with default settings. This method serves as the entry point for configuring a compression builder.

```APIDOC
pub fn new() -> Self
```

--------------------------------

### Rust: Perform Single-Threaded Gzip Compression with SyncZ

Source: https://docs.rs/gzp/latest/gzp/all.html/latest/src/gzp/lib.rs

This Rust example illustrates how to perform single-threaded Gzip compression using `gzp`'s `SyncZBuilder`. It sets up a `SyncZ` writer, writes data sequentially, and then finalizes the compression. This method is suitable for scenarios where parallel overhead is undesirable or for smaller data volumes.

```Rust
use std::{env, fs::File, io::Write};

use gzp::{deflate::Gzip, syncz::SyncZBuilder, ZWriter};

let mut writer = vec![];
let mut parz = SyncZBuilder::<Gzip, _>::new().from_writer(writer);
parz.write_all(b"This is a first test line\n").unwrap();
parz.write_all(b"This is a second test line\n").unwrap();
parz.finish().unwrap();
```

--------------------------------

### Rust: Create ParCompressBuilder Instance

Source: https://docs.rs/gzp/latest/gzp/all.html/1.0.1/src/gzp/par/compress.rs

This method provides a convenient way to obtain a `ParCompressBuilder` instance, which is used to configure and construct a `ParCompress` runtime. It acts as an entry point for setting up parallel compression.

```APIDOC
{
  "method_name": "builder",
  "signature": "pub fn builder() -> ParCompressBuilder<F>",
  "description": "Creates a builder to configure the `ParCompress` runtime.",
  "parameters": [],
  "returns": {
    "type": "ParCompressBuilder<F>",
    "description": "A new `ParCompressBuilder` instance."
  }
}
```

```Rust
pub fn builder() -> ParCompressBuilder<F> {
    ParCompressBuilder::new()
}
```

--------------------------------

### API Documentation for `gzp::FormatSpec` Trait

Source: https://docs.rs/gzp/latest/gzp/all.html/latest/gzp/trait.FormatSpec

Detailed API reference for the `FormatSpec` trait in the `gzp` crate. This documentation outlines its associated constants, types, and both required and provided methods, including their parameters, return types, and descriptions. It serves as a comprehensive guide for implementing or interacting with custom compression formats.

```APIDOC
Trait: FormatSpec
	Description: Defines how to write the header and footer for each format.

	Associated Constants:
		DEFAULT_BUFSIZE:
			Type: usize
			Value: 131072
			Description: The default buffersize to use for this format.

	Required Associated Types:
		C:
			Description: A type that implements Check + Send + 'static.
		Compressor:
			Description: The compressor type for this format.

	Required Methods:
		new():
			Returns: Self
			Description: Creates a new instance of the format specification.
		needs_dict(&self):
			Parameters:
				- name: self
				  type: &Self
			Returns: bool
			Description: Indicates whether the format requires a dictionary.
		create_compressor(&self, compression_level: Compression):
			Parameters:
				- name: self
				  type: &Self
				- name: compression_level
				  type: Compression
			Returns: Result<Self::Compressor, GzpError>
			Description: Creates a compressor instance for the specified compression level.
		encode(&self, input: &[u8], encoder: &mut Self::Compressor, compression_level: Compression, dict: Option<&Bytes>, is_last: bool):
			Parameters:
				- name: self
				  type: &Self
				- name: input
				  type: &[u8]
				- name: encoder
				  type: &mut Self::Compressor
				- name: compression_level
				  type: Compression
				- name: dict
				  type: Option<&Bytes>
				- name: is_last
				  type: bool
			Returns: Result<Vec<u8>, GzpError>
			Description: Encodes the input data using the provided encoder and compression parameters.
		header(&self, compression_level: Compression):
			Parameters:
				- name: self
				  type: &Self
				- name: compression_level
				  type: Compression
			Returns: Vec<u8>
			Description: Generates the header bytes for the specified compression level.
		footer(&self, check: &Self::C):
			Parameters:
				- name: self
				  type: &Self
				- name: check
				  type: &Self::C
			Returns: Vec<u8>
			Description: Generates the footer bytes using the provided check.

	Provided Methods:
		create_check():
			Returns: Self::C
			Description: Creates a new check instance for the format.
		to_bytes(&self, pairs: &[Pair]):
			Parameters:
				- name: self
				  type: &Self
				- name: pairs
				  type: &[Pair]
			Returns: Vec<u8>
			Description: Converts a slice of pairs into a vector of bytes.
```

--------------------------------

### Rust ParCompress: Parallel Gzip Compression Example

Source: https://docs.rs/gzp/latest/gzp/all.html/latest/gzp/par/compress/index

This Rust example demonstrates using `ParCompress` with `Gzip` for parallel data compression. It initializes a `ParCompress` instance from a `ParCompressBuilder`, writes byte slices, and then finalizes the compression process. The snippet showcases basic usage for in-memory compression.

```Rust
use std::{env, fs::File, io::Write};

use gzp::{par::compress::{ParCompress, ParCompressBuilder}, deflate::Gzip, ZWriter};

let mut writer = vec![];
let mut parz: ParCompress<Gzip> = ParCompressBuilder::new().from_writer(writer);
parz.write_all(b"This is a first test line\n").unwrap();
parz.write_all(b"This is a second test line\n").unwrap();
parz.finish().unwrap();
```