Gzp

https://github.com/sstadick/gzp
Admin
A Rust crate providing multi-threaded encoding and decoding for Gzip, Zlib, Raw Deflate, Snappy,...
Tokens:876
Snippets:4
Trust Score:9.7
License:Unlicense
Update:1 year ago
Show doc for...
Tokens:
Raw
### Stream Compress Data with Snappy in Rust

Source: https://github.com/sstadick/gzp/blob/main/README.md

This Rust example demonstrates streaming compression using `gzp` with the Snappy format. Similar to the Mgzip example, it reads data from standard input in chunks and writes them to a `ParZ` instance configured for Snappy, outputting compressed data to standard output. This highlights `gzp`'s flexibility in supporting various compression algorithms for high-throughput scenarios.

```Rust
use gzp::{parz::{ParZ, ParZBuilder}, snap::Snap};
use std::io::{Read, Write};

fn main() {
    let chunksize = 64 * (1 << 10) * 2;

    let stdout = std::io::stdout();
    let mut writer: ParZ<Snap> = ParZBuilder::new().from_writer(stdout);

    let stdin = std::io::stdin();
    let mut stdin = stdin.lock();

    let mut buffer = Vec::with_capacity(chunksize);
    loop {
        let mut limit = (&mut stdin).take(chunksize as u64);
        limit.read_to_end(&mut buffer).unwrap();
        if buffer.is_empty() {
            break;
        }
        writer.write_all(&buffer).unwrap();
        buffer.clear();
    }
    writer.finish().unwrap();
}
```

--------------------------------

### Compress Data with gzp's ZBuilder in Rust

Source: https://github.com/sstadick/gzp/blob/main/README.md

This Rust example demonstrates basic Gzip compression using `gzp`'s `ZBuilder`. It initializes a `ZBuilder` with an in-memory `Vec` as the underlying writer, writes multiple lines of data, and then finalizes the compression. This snippet illustrates how to use `gzp` for efficient, multi-threaded in-memory compression.

```Rust
use std::{env, fs::File, io::Write};

use gzp::{deflate::Gzip, ZBuilder, ZWriter};

fn main() {
    let mut writer = vec![];
    // ZBuilder will return a trait object that transparent over `ParZ` or `SyncZ`
    let mut parz = ZBuilder::<Gzip, _>::new()
        .num_threads(0)
        .from_writer(writer);
    parz.write_all(b"This is a first test line\n").unwrap();
    parz.write_all(b"This is a second test line\n").unwrap();
    parz.finish().unwrap();
}
```

--------------------------------

### Stream Compress Data with Mgzip in Rust

Source: https://github.com/sstadick/gzp/blob/main/README.md

This Rust example shows how to stream compress data from standard input to standard output using `gzp`'s multi-threaded Mgzip compressor. It reads data in configurable chunks, writes them to a `ParCompress` instance, and flushes the compressed output. This approach is highly efficient for processing large files or continuous data streams.

```Rust
use gzp::{ZWriter,
    deflate::Mgzip,
    par::{compress::{ParCompress, ParCompressBuilder}}
};
use std::io::{Read, Write};

fn main() {
    let chunksize = 64 * (1 << 10) * 2;

    let stdout = std::io::stdout();
    let mut writer: ParCompress<Mgzip> = ParCompressBuilder::new().from_writer(stdout);

    let stdin = std::io::stdin();
    let mut stdin = stdin.lock();

    let mut buffer = Vec::with_capacity(chunksize);
    loop {
        let mut limit = (&mut stdin).take(chunksize as u64);
        limit.read_to_end(&mut buffer).unwrap();
        if buffer.is_empty() {
            break;
        }
        writer.write_all(&buffer).unwrap();
        buffer.clear();
    }
    writer.finish().unwrap();
}
```

--------------------------------

### Configure gzp Dependencies in Cargo.toml

Source: https://github.com/sstadick/gzp/blob/main/README.md

This snippet demonstrates how to add `gzp` as a dependency in your `Cargo.toml` file. It shows configurations for default features (using libdeflate), a Rust-only backend (disabling Zlib), and Snappy-only compression, allowing users to select specific compression backends based on their needs.

```TOML
[dependencies]
gzp = { version = "*" }
```

```TOML
[dependencies]
gzp = { version = "*", default-features = false, features = ["deflate_rust"] }
```

```TOML
[dependencies]
gzp = { version = "*", default-features = false, features = ["snap_default"] }
```
Tokens:
Raw