### Set Up Basic Evicter for Cache Management in Rust Source: https://context7.com/kahing/catfs/llms.txt This Rust example illustrates the basic setup of the Evicter for automatic cache management. It opens the cache directory and creates an Evicter instance configured to maintain a minimum of 10GB of free space, then starts its background eviction thread. ```rust use catfs::evicter::Evicter; use catfs::catfs::flags::DiskSpace; use catfs::catfs::rlibc; fn main() -> Result<(), Box> { // Open cache directory let cache_dir = rlibc::open(&"/var/cache/catfs", rlibc::O_RDONLY, 0)?; // Create evicter that maintains at least 10GB free let mut evicter = Evicter::new(cache_dir, &DiskSpace::Bytes(10 * 1024 * 1024 * 1024)); // Start background eviction thread (scans every 60 seconds) evicter.run(); // Evicter will automatically: // - Monitor free space on cache filesystem // - When free space drops below high watermark, evict files // - Evict oldest files first (LRU-based) // - Keep 25% most recently accessed files as "hot" // - Weight eviction by file size and age Ok(()) } ``` -------------------------------- ### Mount CatFS Programmatically in Rust Source: https://context7.com/kahing/catfs/llms.txt This example demonstrates how to mount and run the CatFS filesystem programmatically using Rust. It initializes the filesystem, sets up an evicter for cache management, and configures FUSE session options before running. ```rust use catfs::CatFS; use catfs::pcatfs::PCatFS; use catfs::evicter::Evicter; use catfs::catfs::flags::DiskSpace; use std::path::Path; use std::ffi::OsStr; fn main() -> Result<(), Box> { let source = Path::new("/mnt/remote"); let cache = Path::new("/var/cache/catfs"); let mountpoint = Path::new("/mnt/local"); // Initialize filesystem let fs = CatFS::new(&source, &cache)?; let fs = PCatFS::new(fs); // Get cache directory file descriptor for evicter let cache_dir_fd = fs.get_cache_dir()?; // Create evicter to maintain 5% free space let mut evicter = Evicter::new(cache_dir_fd, &DiskSpace::Percent(5.0)); evicter.run(); // Start background eviction thread // Mount options let options: Vec<&OsStr> = vec![ OsStr::new("-o"), OsStr::new("atomic_o_trunc"), OsStr::new("-o"), OsStr::new("default_permissions"), ]; // Create FUSE session and run let mut session = fuse::Session::new(fs, mountpoint, &options)?; session.run()?; Ok(()) } ``` -------------------------------- ### Cache S3 Buckets with CatFS and goofys Source: https://context7.com/kahing/catfs/llms.txt Explains how to cache S3 buckets that are mounted locally using goofys. This setup allows for file-like access to S3 data with the performance benefits of local caching provided by CatFS. ```bash # Mount S3 bucket with goofys goofys mybucket /mnt/s3 ``` -------------------------------- ### Set Custom UID/GID for Catfs (Bash) Source: https://context7.com/kahing/catfs/llms.txt Allows running the Catfs filesystem process with specific user and group IDs. This is typically done after starting as root and requires the --uid and --gid options. The process drops privileges after mounting, enhancing security. ```bash # Run as specific user/group (requires starting as root) catfs --uid=1000 --gid=1000 /mnt/s3 /home/user/cache /home/user/files # This drops privileges after mounting ``` -------------------------------- ### Read Cached File with CatFS Source: https://context7.com/kahing/catfs/llms.txt Reads a file from a CatFS mounted path. It automatically checks for a valid cache, otherwise it opens the source and starts read-ahead. Reads are served from the cache as data becomes available. ```rust use catfs::CatFS; use std::path::Path; use std::fs::File; use std::io::Read; fn read_cached_file() -> Result> { // After mounting catfs at /mnt/local // Reading a file automatically: // 1. Checks if valid cache exists // 2. If not, opens source and starts read-ahead (background copy to cache) // 3. Serves reads from cache as data becomes available let mut file = File::open("/mnt/local/documents/report.txt")?; let mut contents = String::new(); file.read_to_string(&mut contents)?; // Subsequent reads of the same file come directly from cache // Cache validity is tracked via user.catfs.src_chksum xattr Ok(contents) } ``` -------------------------------- ### Validate Cached File Integrity in Rust Source: https://context7.com/kahing/catfs/llms.txt This Rust example shows how to check if a cached file is still valid by comparing it against its source. It uses the `Handle::validate_cache` function, which checks the `user.catfs.src_chksum` extended attribute against the source file's metadata. ```rust use catfs::catfs::file::Handle; use catfs::catfs::rlibc; use std::path::Path; fn check_cache_validity() -> Result> { let src_dir = rlibc::open(&"/mnt/remote", rlibc::O_RDONLY, 0)?; let cache_dir = rlibc::open(&"/var/cache/catfs", rlibc::O_RDONLY, 0)?; let file_path = Path::new("documents/report.pdf"); // Validate cache integrity // Checks user.catfs.src_chksum xattr against source file metadata let is_valid = Handle::validate_cache( src_dir, cache_dir, &file_path, false, // cache_valid_if_present: trust existing cache true, // check_only: don't delete invalid cache )?; if is_valid { println!("Cache is valid, will read from cache"); } else { println!("Cache invalid or missing, will fetch from source"); } rlibc::close(src_dir)?; rlibc::close(cache_dir)?; Ok(is_valid) } ``` -------------------------------- ### Configure Catfs in Fstab (Bash) Source: https://context7.com/kahing/catfs/llms.txt Demonstrates how to configure Catfs mounts in the /etc/fstab file for automatic mounting at boot. The format specifies the Catfs source and cache paths, the mount point, the filesystem type ('fuse'), and any desired options. ```bash # /etc/fstab entry format: # catfs## fuse 0 0 # Example fstab entry: catfs#/mnt/s3#/var/cache/s3 /data fuse allow_other,--uid=1001,--gid=1001,--free=1% 0 0 # After adding to fstab, mount with: mount /data ``` -------------------------------- ### Initialize CatFS Filesystem (Rust) Source: https://context7.com/kahing/catfs/llms.txt Initializes a new CatFS instance in Rust. It takes the source directory and cache directory paths as arguments. The `CatFS::new` function returns a `Result` which, upon success, provides a CatFS instance ready to handle filesystem operations. ```rust use catfs::CatFS; use std::path::Path; fn main() -> Result<(), Box> { let source_dir = Path::new("/mnt/remote"); let cache_dir = Path::new("/var/cache/catfs"); // Create new CatFS instance let fs = CatFS::new(&source_dir, &cache_dir)?; // CatFS is now ready to handle filesystem operations // Source files at /mnt/remote will be cached to /var/cache/catfs Ok(()) } ``` -------------------------------- ### Create File with CatFS Source: https://context7.com/kahing/catfs/llms.txt Creates a new file that writes through to both the source and cache directories. It handles opening directories, creating the file with specified flags and mode, writing data, and flushing changes. ```rust use catfs::catfs::file::Handle; use catfs::catfs::rlibc; use std::path::Path; fn create_file() -> Result<(), Box> { let src_dir = rlibc::open(&"/mnt/remote", rlibc::O_RDONLY, 0)?; let cache_dir = rlibc::open(&"/var/cache/catfs", rlibc::O_RDONLY, 0)?; let file_path = Path::new("newfile.txt"); let flags = rlibc::O_WRONLY | rlibc::O_CREAT | rlibc::O_EXCL; let mode = 0o644; // Create file in both source and cache let mut handle = Handle::create(src_dir, cache_dir, &file_path, flags, mode)?; // Write data (goes to both source and cache) let data = b"Hello, World!"; let bytes_written = handle.write(0, data)?; println!("Wrote {} bytes", bytes_written); // Flush ensures data is persisted handle.flush()?; rlibc::close(src_dir)?; rlibc::close(cache_dir)?; Ok(()) } ``` -------------------------------- ### Mount Catfs Filesystem (Bash) Source: https://context7.com/kahing/catfs/llms.txt Mounts the Catfs caching filesystem. It requires the source directory, cache directory, and the desired mount point as arguments. Files accessed at the mount point will be served from the cache or fetched from the source and stored in the cache. ```bash # Mount catfs with source directory, cache directory, and mountpoint catfs /mnt/remote /var/cache/catfs /mnt/local # Files in /mnt/remote will be accessible at /mnt/local # Cached copies stored in /var/cache/catfs ``` -------------------------------- ### Write File with Catfs Source: https://context7.com/kahing/catfs/llms.txt Demonstrates writing a file to the catfs mount. Writes are buffered locally and uploaded to the remote storage upon file close, handling S3 quirks like ENOTSUP for random writes. ```bash echo "data" > /mnt/files/newfile.txt ``` -------------------------------- ### Cache Remote SSH-Mounted Filesystems with CatFS Source: https://context7.com/kahing/catfs/llms.txt Demonstrates how to use CatFS to cache remote filesystems mounted via sshfs. It involves first mounting the remote filesystem using sshfs, then overlaying CatFS for caching, and finally accessing files through the CatFS mount point for improved performance. ```bash # First, mount remote filesystem via sshfs sshfs user@remote:/data /mnt/sshfs -o reconnect,ServerAliveInterval=15 # Then overlay catfs for caching catfs /mnt/sshfs /var/cache/sshfs /mnt/data --free=10% # Access files through /mnt/data # First access fetches from remote, subsequent reads from local cache ls /mnt/data/ cat /mnt/data/largefile.bin # Cached after first read # Benchmark shows significant speedup for repeated reads # (see bench/bench.catfs_vs_sshfs.png in repository) ``` -------------------------------- ### Wrap CatFS with PCatFS for Parallelism (Rust) Source: https://context7.com/kahing/catfs/llms.txt Wraps an existing CatFS instance with PCatFS to enable parallel request handling using a thread pool. The `PCatFS::new` function takes a CatFS instance and creates a parallel filesystem wrapper. This improves performance by allowing concurrent operations. ```rust use catfs::CatFS; use catfs::pcatfs::PCatFS; use std::path::Path; fn main() -> Result<(), Box> { let source = Path::new("/mnt/nfs"); let cache = Path::new("/tmp/nfs_cache"); // Create base CatFS let fs = CatFS::new(&source, &cache)?; // Wrap with parallel thread pool (100 worker threads) let parallel_fs = PCatFS::new(fs); // PCatFS implements the fuse::Filesystem trait // All operations are dispatched to thread pool workers Ok(()) } ``` -------------------------------- ### Configure Catfs Free Space (Bash) Source: https://context7.com/kahing/catfs/llms.txt Controls the amount of free space maintained in the cache directory using the --free option. This can be specified in absolute units (K, M, G, T) or as a percentage of the cache partition. This helps prevent the cache from consuming all available disk space. ```bash # Ensure at least 10GB free space on cache partition catfs --free=10G /mnt/remote /var/cache/catfs /mnt/local # Ensure at least 5% of cache partition stays free catfs --free=5% /mnt/remote /var/cache/catfs /mnt/local # Supported units: K (KB), M (MB), G (GB), T (TB), or % for percentage ``` -------------------------------- ### Add Catfs Caching Layer Source: https://context7.com/kahing/catfs/llms.txt This command mounts the S3 bucket to a local cache directory and a local mount point. The `--free=5%` option ensures that 5% of the disk space is kept free. ```bash catfs /mnt/s3 /var/cache/s3 /mnt/files --free=5% ``` -------------------------------- ### Directory Operations with CatFS Source: https://context7.com/kahing/catfs/llms.txt Performs standard directory operations on a CatFS mounted path. This includes creating directories, listing contents (which are read from the source and cached), removing directories, renaming files, and removing files. ```rust use std::fs; use std::path::Path; fn directory_operations() -> Result<(), Box> { let base = Path::new("/mnt/local"); // Create directory (created on source, cached path created on demand) fs::create_dir(base.join("newdir"))?; // List directory contents (reads from source, caches entries) for entry in fs::read_dir(base.join("documents"))? { let entry = entry?; println!("{:?}", entry.file_name()); } // Remove empty directory fs::remove_dir(base.join("emptydir"))?; // Rename (renames on both source and cache) fs::rename( base.join("oldname.txt"), base.join("newname.txt") )?; // Remove file (removes from both source and cache) fs::remove_file(base.join("obsolete.txt"))?; Ok(()) } ``` -------------------------------- ### Run Catfs in Foreground Mode (Bash) Source: https://context7.com/kahing/catfs/llms.txt Runs the Catfs filesystem in foreground mode, which is useful for debugging and monitoring. It enables debug logging when the RUST_LOG environment variable is set to 'debug'. The output shows filesystem operations like lookups and reads. ```bash # Run in foreground with debug logging RUST_LOG=debug catfs -f /mnt/sshfs /tmp/cache /mnt/cached # Output shows operations like: # 2024-01-15 10:30:45 DEBUG - <-- lookup "myfile.txt" = 0x0000000000000002, RegularFile refcnt *1 # 2024-01-15 10:30:45 DEBUG - read ahead "myfile.txt" # 2024-01-15 10:30:46 DEBUG - <-- open "myfile.txt" = 1 ``` -------------------------------- ### Pass FUSE Mount Options to Catfs (Bash) Source: https://context7.com/kahing/catfs/llms.txt Enables passing additional FUSE mount options to the Catfs filesystem using the -o flag. This allows customization of mount behavior, such as allowing other users to access the mount point with 'allow_other' or setting the maximum read size. ```bash # Allow other users to access the mount catfs -o allow_other /mnt/remote /tmp/cache /mnt/local # Multiple options catfs -o allow_other,max_read=131072 /mnt/remote /tmp/cache /mnt/local ``` -------------------------------- ### Write File with CatFS Source: https://context7.com/kahing/catfs/llms.txt Writes data to a CatFS mounted path using write-through caching. This process marks the cache as dirty, writes data to both source and cache simultaneously, and restores the pristine status on close or flush. ```rust use std::fs::OpenOptions; use std::io::Write; fn write_file() -> Result<(), Box> { // Writing to a catfs-mounted path: // 1. Marks cache as dirty (removes pristine xattr) // 2. Writes data to both source and cache simultaneously // 3. On close/flush, restores pristine status let mut file = OpenOptions::new() .write(true) .create(true) .open("/mnt/local/output.txt")?; file.write_all(b"Data written through to source and cache")?; // Flush triggers write-through completion // If source doesn't support random writes (like goofys/S3), // catfs detects ENOTSUP and falls back to full file copy on close Ok(()) } ``` -------------------------------- ### Trigger Manual Cache Eviction Pass in Rust Source: https://context7.com/kahing/catfs/llms.txt This Rust code snippet demonstrates how to manually trigger a single eviction pass for the CatFS cache. It creates an Evicter instance with a percentage-based disk space threshold and then calls `loop_once()` to perform the eviction if necessary. ```rust use catfs::evicter::Evicter; use catfs::catfs::flags::DiskSpace; use catfs::catfs::rlibc; fn evict_cache() -> Result<(), Box> { let cache_dir = rlibc::open(&"/tmp/cache", rlibc::O_RDONLY, 0)?; // Create evicter with percentage-based threshold let evicter = Evicter::new(cache_dir, &DiskSpace::Percent(10.0)); // Run single eviction pass // This will evict files if free space < 10% // Target is 10% * 1.1 = 11% (low watermark) evicter.loop_once()?; rlibc::close(cache_dir)?; Ok(()) } ``` -------------------------------- ### Modify Existing File with CatFS Source: https://context7.com/kahing/catfs/llms.txt Modifies an existing file at a specific offset within a CatFS mounted path. CatFS ensures read-ahead completes before writing and marks the file as dirty until closed. Full file flushes may occur if the source doesn't support random writes. ```rust use std::fs::OpenOptions; use std::os::unix::fs::FileExt; fn modify_file() -> Result<(), Box> { // Open for writing without truncating let file = OpenOptions::new() .write(true) .open("/mnt/local/config.txt")?; // Write at specific offset // catfs waits for read-ahead to reach this offset before writing let data = b"modified"; file.write_at(data, 100)?; // The file is marked dirty until close // On close, full file may be flushed if source doesn't support // random writes Ok(()) } ``` -------------------------------- ### Parse DiskSpace Enum Values in Rust Source: https://context7.com/kahing/catfs/llms.txt This Rust code snippet shows how to parse DiskSpace enum values, which are used for cache management. It demonstrates parsing both percentage and absolute byte values with various units (G, M, K, T) and raw bytes. ```rust use catfs::catfs::flags::DiskSpace; use std::str::FromStr; fn main() { // Parse percentage value let space_percent = DiskSpace::from_str("25%").unwrap(); assert_eq!(space_percent, DiskSpace::Percent(25.0)); // Parse absolute byte values with units let space_gb = DiskSpace::from_str("25G").unwrap(); assert_eq!(space_gb, DiskSpace::Bytes(25 * 1024 * 1024 * 1024)); let space_mb = DiskSpace::from_str("512M").unwrap(); assert_eq!(space_mb, DiskSpace::Bytes(512 * 1024 * 1024)); let space_kb = DiskSpace::from_str("1024K").unwrap(); assert_eq!(space_kb, DiskSpace::Bytes(1024 * 1024)); let space_tb = DiskSpace::from_str("1T").unwrap(); assert_eq!(space_tb, DiskSpace::Bytes(1024 * 1024 * 1024 * 1024)); // Raw bytes (no unit) let space_bytes = DiskSpace::from_str("4096").unwrap(); assert_eq!(space_bytes, DiskSpace::Bytes(4096)); } ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.