Archive Reader

https://github.com/yaxincheng/archive-reader
Admin
Archive Reader is a Rust library that provides a rustic interface over libarchive's partial read...
Tokens:3,863
Snippets:13
Trust Score:7.5
License:MIT
Update:11 months ago
Show doc for...
Tokens:
Raw
### Compiling archive-reader Project

Source: https://github.com/yaxincheng/archive-reader/blob/master/README.md

This code snippet shows the shell commands required to clone the archive-reader repository and build the project using Cargo. It assumes Git and Cargo are installed.

```shell
cd SOME_DIR
git clone git@github.com:YaxinCheng/archive-reader.git
cd archive-reader
cargo build --release
```

--------------------------------

### Reading Files from Archives using ArchiveReader in Rust

Source: https://github.com/yaxincheng/archive-reader/blob/master/README.md

This example demonstrates how to open an archive, list its file names, and read the content of the first file using the `archive-reader` crate. It requires the `archive-reader` dependency to be added to your project.

```rust
use archive_reader::Archive;
use archive_reader::error::Result;

fn main() -> Result<()> {
    let mut archive = Archive::open("some_archive.zip");
    let file_names = archive
                        .block_size(1024 * 1024)
                        .list_file_names()? 
                        .collect::<Result<Vec<_>>>()?;
    let mut content = vec![];
    let _ = archive.read_file(&file_names[0], &mut content)?;
    println!("content={:?}", content);
    Ok(())
}
```

--------------------------------

### Archive Opening and Configuration in Rust

Source: https://context7.com/yaxincheng/archive-reader/llms.txt

Demonstrates how to open archives using the Archive Reader library in Rust. It shows setting default configurations, customizing block sizes, adding passwords for encrypted archives, and specifying custom decoders for file names. It also covers resetting configurations and accessing archive paths.

```rust
use archive_reader::Archive;
use archive_reader::error::Result;

fn main() -> Result<()> {
    // Open archive with default settings (1MB block size, UTF-8 decoding)
    let mut archive = Archive::open("documents.zip");

    // Configure archive with custom block size and password
    archive
        .block_size(2 * 1024 * 1024)  // 2MB blocks for faster reading
        .try_password("secret123")     // Add password for encrypted archives
        .try_password("backup_pw");    // Can add multiple passwords to try

    // Custom decoder for non-UTF8 file names
    fn gb18030_decoder(bytes: &[u8]) -> Option<std::borrow::Cow<'_, str>> {
        encoding_rs::GB18030.decode(bytes).0.into()
    }

    let mut chinese_archive = Archive::open("chinese_files.zip");
    chinese_archive.decoder(gb18030_decoder);

    // Reset to defaults if needed
    archive.reset_block_size().reset_decoder();

    // Access archive path
    println!("Archive path: {}", archive.path().display());

    Ok(())
}

```

--------------------------------

### Use Entry API to Access Archive Content in Rust

Source: https://context7.com/yaxincheng/archive-reader/llms.txt

Demonstrates the Entry API for direct interaction with individual archive entries. It shows how to read file names, extract entire file contents, and read data block by block, including zero-allocation block reading with the 'lending_iter' feature.

```rust
use archive_reader::{Archive, Entry};
use archive_reader::error::Result;

fn main() -> Result<()> {
    let archive = Archive::open("documents.zip");

    // Process entries individually
    archive.entries(|entry: Entry| {
        let file_name = entry.file_name()?;
        println!("Processing: {}", file_name);

        // Read entire file content
        let mut buffer = Vec::new();
        let bytes_read = entry.read_file(&mut buffer)?;
        println!("  Size: {} bytes", bytes_read);

        // Or read block by block
        let blocks = entry.read_file_by_block();
        for (i, block_result) in blocks.enumerate() {
            let block = block_result?;
            println!("  Block {}: {} bytes", i, block.len());

            // Process block data here
            if i >= 2 { break; } // First 3 blocks only
        }

        Ok(())
    })?;

    // With lending_iter feature
    #[cfg(feature = "lending_iter")]
    {
        use archive_reader::LendingIterator;

        let mut entries = archive.entries()?;
        while let Some(Ok(entry)) = entries.next() {
            let name = entry.file_name()?;

            // Zero-allocation block reading
            let mut blocks = entry.read_file_by_block();
            let mut total = 0;

            while let Some(Ok(block)) = LendingIterator::next(&mut blocks) {
                total += block.len();
            }

            println!("{}: {} bytes", name, total);
        }
    }

    Ok(())
}
```

--------------------------------

### Iterate Archive Entries with Lending Iterator in Rust

Source: https://context7.com/yaxincheng/archive-reader/llms.txt

Demonstrates how to process archive entries efficiently using the lending iterator pattern provided by the 'lending_iter' feature. This method allows for zero-copy iteration over entry data and blocks.

```rust
// Cargo.toml: archive-reader = { version = "0.4", features = ["lending_iter"] }

use archive_reader::{Archive, LendingIterator};
use archive_reader::error::Result;

fn main() -> Result<()> {
    let archive = Archive::open("backup.tar.gz");

    // With lending_iter feature: get an iterator of entries
    let mut entries = archive.entries()?;

    let mut total_files = 0;
    let mut total_dirs = 0;

    while let Some(entry_result) = entries.next() {
        let entry = entry_result?;
        let name = entry.file_name()?;

        if name.ends_with('/') {
            total_dirs += 1;
            println!("Directory: {}", name);
        } else {
            total_files += 1;
            println!("File: {}", name);
        }
    }

    println!("\nSummary: {} files, {} directories", total_files, total_dirs);

    // Example: Stream blocks from an entry
    let mut entries2 = archive.entries()?;
    if let Some(Ok(entry)) = entries2.next() {
        let name = entry.file_name()?;
        println!("Reading: {}", name);

        let mut blocks = entry.read_file_by_block();
        let mut byte_count = 0;

        while let Some(block_result) = LendingIterator::next(&mut blocks) {
            let block = block_result?;
            byte_count += block.len();
        }

        println!("Total size: {} bytes", byte_count);
    }

    Ok(())
}
```

--------------------------------

### Iterate archive entries using closures in Rust

Source: https://context7.com/yaxincheng/archive-reader/llms.txt

This Rust code shows how to process each entry within an archive using a closure, allowing custom logic for each file. It demonstrates calculating file sizes and conditionally extracting specific file types. Dependencies: 'archive-reader'.

```rust
use archive_reader::{Archive, Entry};
use archive_reader::error::Result;
use std::collections::HashMap;

fn main() -> Result<()> {
    let archive = Archive::open("website.zip");

    let mut file_sizes = HashMap::new();

    // Process each entry with a closure (without lending_iter feature)
    archive.entries(|entry: Entry| {
        let name = entry.file_name()?.to_string();

        // Read content to count size
        let mut content = Vec::new();
        let size = entry.read_file(&mut content)?;

        file_sizes.insert(name.clone(), size);
        println!("Processed: {} ({} bytes)", name, size);

        Ok(())
    })?;

    // Find largest file
    let largest = file_sizes.iter()
        .max_by_key(|(_, &size)| size)
        .map(|(name, size)| (name.as_str(), size));

    if let Some((name, size)) = largest {
        println!("\nLargest file: {} ({} bytes)", name, size);
    }

    // Example: Extract only text files
    let archive2 = Archive::open("mixed_content.zip");
    archive2.entries(|entry| {
        let name = entry.file_name()?;
        if name.ends_with(".txt") {
            let output_path = format!("extracted/{}", name);
            std::fs::create_dir_all(std::path::Path::new(&output_path).parent().unwrap())?;
            let mut file = std::fs::File::create(output_path)?;
            entry.read_file(&mut file)?;
        }
        Ok(())
    })?;

    Ok(())
}
```

--------------------------------

### Listing Archive Contents in Rust

Source: https://context7.com/yaxincheng/archive-reader/llms.txt

Illustrates how to list all files and directories within an archive using the Archive Reader library in Rust. It shows collecting file names into a vector and also streaming them for memory efficiency, including basic error handling for individual entry reads.

```rust
use archive_reader::Archive;
use archive_reader::error::Result;

fn main() -> Result<()> {
    let archive = Archive::open("project.tar.gz");

    // Get all file names as a vector
    let file_names = archive
        .list_file_names()? // Collects into Result<Vec<String>>
        .collect::<Result<Vec<String>>>()?;

    println!("Archive contains {} entries:", file_names.len());
    for (i, name) in file_names.iter().enumerate() {
        println!("  [{}] {}", i, name);
    }

    // Example output:
    // Archive contains 5 entries:
    //   [0] project/
    //   [1] project/src/
    //   [2] project/src/main.rs
    //   [3] project/Cargo.toml
    //   [4] project/README.md

    // Stream file names without collecting (memory efficient)
    let archive2 = Archive::open("large_archive.zip");
    for result in archive2.list_file_names()? {
        match result {
            Ok(name) => println!("Found: {}", name),
            Err(e) => eprintln!("Error reading entry: {}", e),
        }
    }

    Ok(())
}

```

--------------------------------

### Memory-efficient block streaming with Rust lending_iter

Source: https://context7.com/yaxincheng/archive-reader/llms.txt

This Rust code utilizes the 'lending_iter' feature for memory-efficient streaming of archive file blocks, avoiding heap allocations by borrowing blocks directly. It demonstrates calculating a checksum and extracting files with zero copy. Dependencies: 'archive-reader' with 'lending_iter' feature enabled.

```rust
// Cargo.toml: archive-reader = { version = "0.4", features = ["lending_iter"] }

use archive_reader::{Archive, LendingIterator};
use archive_reader::error::Result;
use std::io::Write;

fn main() -> Result<()> {
    let archive = Archive::open("large_dataset.tar.gz");

    // With lending_iter feature: blocks are borrowed, not boxed
    let mut blocks = archive.read_file_by_block("data.csv")?;

    let mut checksum: u64 = 0;
    while let Some(block_result) = blocks.next() {
        let block: &[u8] = block_result?;
        // Block is borrowed from internal buffer - zero copy!
        checksum = checksum.wrapping_add(block.iter().map(|&b| b as u64).sum::<u64>());
    }

    println!("Checksum: {}", checksum);

    // Extract with zero allocations
    let mut output = std::fs::File::create("output.csv")?;
    let mut blocks2 = archive.read_file_by_block("data.csv")?;

    while let Some(block_result) = LendingIterator::next(&mut blocks2) {
        output.write_all(block_result?)?;
    }

    Ok(())
}
```

--------------------------------

### Handle Archive Errors in Rust

Source: https://context7.com/yaxincheng/archive-reader/llms.txt

Illustrates how to handle various error types that can occur when working with archives, including IO errors, invalid paths, extraction failures, and password-related issues. This ensures robust application behavior.

```rust
use archive_reader::{Archive, Error};
use archive_reader::error::Result;

fn main() -> Result<()> {
    let archive = Archive::open("test.zip");

    // Handle different error types
    let file_names = match archive.list_file_names() {
        Ok(names) => names.collect::<Result<Vec<_>>>()?,
        Err(Error::Io(io_err)) => {
            eprintln!("IO error: {}", io_err);
            return Err(Error::Io(io_err));
        },
        Err(Error::PathNotUtf8) => {
            eprintln!("Archive path is not valid UTF-8");
            return Err(Error::PathNotUtf8);
        },
        Err(Error::Extraction(msg)) => {
            eprintln!("Archive extraction failed: {}", msg);
            return Err(Error::Extraction(msg));
        },
        Err(e) => return Err(e),
    };

    // Safe file reading with error handling
    for name in &file_names {
        let mut content = Vec::new();
        match archive.read_file(name, &mut content) {
            Ok(size) => println!("Read {} ({} bytes)", name, size),
            Err(Error::Encoding) => {
                eprintln!("Could not decode filename: {}", name);
            },
            Err(e) => {
                eprintln!("Error reading {}: {}", name, e);
            }
        }
    }

    // Password-protected archive errors
    let mut secure = Archive::open("encrypted.zip");
    secure.try_password("wrong_password");

    if let Err(e) = secure.list_file_names() {
        eprintln!("Failed to decrypt archive: {}", e);
        // Try another password
    }

    Ok(())
}
```

--------------------------------

### Reading Complete Files from Archive in Rust

Source: https://context7.com/yaxincheng/archive-reader/llms.txt

Shows how to extract the content of a specific file from an archive using the Archive Reader library in Rust. It covers reading file content into a byte vector, extracting directly to a file on disk, handling password-protected archives, and managing errors for missing files.

```rust
use archive_reader::Archive;
use archive_reader::error::Result;
use std::fs::File;
use std::io::Write;

fn main() -> Result<()> {
    let archive = Archive::open("data.zip");

    // Read file content into a Vec<u8>
    let mut content = Vec::new();
    let bytes_read = archive.read_file("data/config.json", &mut content)?;
    println!("Read {} bytes", bytes_read);

    // Convert to string if it's text content
    let config_text = String::from_utf8_lossy(&content);
    println!("Config: {}", config_text);

    // Extract directly to a file on disk
    let mut output_file = File::create("extracted_config.json")?;
    let size = archive.read_file("data/config.json", &mut output_file)?;
    println!("Extracted {} bytes to disk", size);

    // Handle password-protected archives
    let mut secure_archive = Archive::open("encrypted.7z");
    secure_archive.try_password("mypassword");

    let mut secret_data = Vec::new();
    match secure_archive.read_file("secret.txt", &mut secret_data) {
        Ok(size) => println!("Successfully read {} bytes", size),
        Err(e) => eprintln!("Failed to decrypt: {}", e),
    }

    // Handle missing files
    match archive.read_file("nonexistent.txt", &mut content) {
        Ok(_) => println!("File found"),
        Err(e) => println!("File not found: {}", e),
    }

    Ok(())
}

```

--------------------------------

### Stream large files block-by-block with Rust

Source: https://context7.com/yaxincheng/archive-reader/llms.txt

This Rust code demonstrates how to stream large files from an archive in chunks, preventing the entire file from being loaded into memory. It shows how to iterate over blocks, write them to a new file, and perform basic analysis on the content. Dependencies: 'archive-reader'.

```rust
use archive_reader::Archive;
use archive_reader::error::Result;
use std::fs::File;
use std::io::Write;

fn main() -> Result<()> {
    let archive = Archive::open("videos.zip");

    // Stream large file block by block (without lending_iter feature)
    let blocks = archive.read_file_by_block("movie.mp4")?;

    let mut output = File::create("extracted_movie.mp4")?;
    let mut total_bytes = 0;

    for block_result in blocks {
        let block: Box<[u8]> = block_result?;
        total_bytes += block.len();
        output.write_all(&block)?;

        // Progress indicator
        if total_bytes % (10 * 1024 * 1024) == 0 {
            println!("Processed {} MB", total_bytes / (1024 * 1024));
        }
    }

    println!("Total extracted: {} bytes", total_bytes);

    // Example: Process blocks with custom logic
    let archive2 = Archive::open("logs.tar.gz");
    let blocks = archive2.read_file_by_block("app.log")?;

    let mut line_count = 0;
    for block_result in blocks {
        let block = block_result?;
        let text = String::from_utf8_lossy(&block);
        line_count += text.lines().count();
    }
    println!("Log file contains {} lines", line_count);

    Ok(())
}
```
Tokens:
Raw