### Canopydb: Create Database and Basic CRUD Operations in Rust Source: https://context7.com/arthurprs/canopydb/llms.txt Demonstrates how to create a new Canopydb database, begin write and read transactions, perform basic CRUD operations (insert, get, delete) on keys within a tree (keyspace), and commit changes. It also shows how to verify data persistence after committing. ```rust use canopydb::Database; use tempfile::tempdir; fn main() -> Result<(), canopydb::Error> { // Create a new database in a temporary directory let dir = tempdir().unwrap(); let db = Database::new(dir.path())?; // Begin a write transaction let tx = db.begin_write()?; // Create or get a tree (keyspace) within the transaction let mut tree = tx.get_or_create_tree(b"my_tree")?; // Insert key-value pairs tree.insert(b"key1", b"value1")?; tree.insert(b"key2", b"value2")?; tree.insert(b"key3", b"value3")?; // Read within the same write transaction let value = tree.get(b"key1")?; assert_eq!(value.as_deref(), Some(&b"value1"[..])); // Delete a key let deleted = tree.delete(b"key2")?; assert!(deleted); // Drop tree reference before commit drop(tree); // Commit the transaction to persist changes let tx_id = tx.commit()?; println!("Committed transaction: {}", tx_id); // Begin a read-only transaction let rx = db.begin_read()?; let tree = rx.get_tree(b"my_tree")?.unwrap(); // Verify persisted data assert_eq!(tree.get(b"key1")?.as_deref(), Some(&b"value1"[..])); assert_eq!(tree.get(b"key2")?, None); // Was deleted assert_eq!(tree.get(b"key3")?.as_deref(), Some(&b"value3"[..])); Ok(()) } ``` -------------------------------- ### Configure CanopyDB Tree Options for Large Value Compression Source: https://context7.com/arthurprs/canopydb/llms.txt Illustrates how to set tree-specific options in CanopyDB, focusing on enabling compression for large values. This example demonstrates storing and retrieving a large value that exceeds the specified compression threshold. It requires the 'canopydb' and 'tempfile' crates. ```rust use canopydb::{Database, TreeOptions}; use tempfile::tempdir; fn main() -> Result<(), canopydb::Error> { let dir = tempdir().unwrap(); let db = Database::new(dir.path())?; // Configure tree with compression for large values let mut options = TreeOptions::default(); options.compress_overflow_values = Some(8 * 1024); // Compress values > 8KB let tx = db.begin_write()?; let mut tree = tx.get_or_create_tree_with(b"documents", options)?; // Store a large compressible value (JSON-like) let large_value = r#"{ "title": "Lorem Ipsum", "content": "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum. "#.repeat(100); // ~30KB of text tree.insert(b"doc:1", large_value.as_bytes())?; // Store smaller values (won't be compressed) tree.insert(b"doc:2", b"Small document content")?; drop(tree); tx.commit()?; // Read back the data let rx = db.begin_read()?; let tree = rx.get_tree(b"documents")?.unwrap(); let doc1 = tree.get(b"doc:1")?.unwrap(); let doc2 = tree.get(b"doc:2")?.unwrap(); println!("Document 1 size: {} bytes", doc1.len()); println!("Document 2 size: {} bytes", doc2.len()); println!("Total documents: {}", tree.len()); Ok(()) } ``` -------------------------------- ### Optimize Storage with Fixed-Length Keys and Values (Rust) Source: https://context7.com/arthurprs/canopydb/llms.txt Illustrates how to configure CanopyDB trees for fixed-length keys and values, optimizing storage for structured numeric data. This example shows storing and querying integer pairs using big-endian for keys and native-endian for values. ```rust use canopydb::{Database, TreeOptions}; use tempfile::tempdir; fn main() -> Result<(), canopydb::Error> { let dir = tempdir().unwrap(); let db = Database::new(dir.path())?; // Configure tree for fixed-length key-value pairs let mut options = TreeOptions::default(); options.fixed_key_len = 8; // 8-byte keys (u64) options.fixed_value_len = 8; // 8-byte values (u64) let tx = db.begin_write()?; let mut tree = tx.get_or_create_tree_with(b"integers", options)?; // Store integer pairs (use big-endian for proper sorting) for i in 0u64..100 { let key = i.to_be_bytes(); // Big-endian for sorted iteration let value = (i * i).to_ne_bytes(); // Native-endian for values tree.insert(&key, &value)?; } drop(tree); tx.commit()?; // Query the data let rx = db.begin_read()?; let tree = rx.get_tree(b"integers")?.unwrap(); // Get a specific value let key = 7u64.to_be_bytes(); let value = tree.get(&key)? .map(|v| u64::from_ne_bytes(v.as_ref().try_into().unwrap())); println!("7^2 = {:?}", value); // Should be 49 // Range query: keys 10..20 let start = 10u64.to_be_bytes(); let end = 20u64.to_be_bytes(); println!("\nSquares from 10 to 19:"); for result in tree.range(&start[..]..&end[..])? { let (k, v) = result?; let key = u64::from_be_bytes(k.as_ref().try_into().unwrap()); let value = u64::from_ne_bytes(v.as_ref().try_into().unwrap()); println!(" {}^2 = {}", key, value); } // Tree statistics println!("\nTree height: {}", tree.height()); println!("Total entries: {}", tree.len()); Ok(()) } ``` -------------------------------- ### Concurrent Write Transactions with Retry Logic in Rust Source: https://github.com/arthurprs/canopydb/blob/master/README.md Demonstrates how to perform concurrent write transactions with retry logic in Rust using CanopyDB. This example shows multiple threads attempting to concurrently update a key-value pair, handling write conflicts by retrying the transaction. ```rust const THREADS: usize = 4; const INC_PER_THREAD: usize = 5_000; let _dir = tempfile::tempdir().unwrap(); let db = canopydb::Database::new(_dir.path()).unwrap(); std::thread::scope(|scope| { for _thread in 0..THREADS { scope.spawn(|| { let mut successes = 0; while successes < INC_PER_THREAD { // This write txn will run concurrently with other concurrent transactions. // But they may conflict at commit time. let tx = db.begin_write_concurrent().unwrap(); let mut tree = tx.get_or_create_tree(b"tree").unwrap(); // Each thread will independently increment a counter INC_PER_THREAD times let prev = if let Some(value) = tree.get(b"key").unwrap() { usize::from_ne_bytes(value.as_ref().try_into().unwrap()) } else { 0 }; tree.insert(b"key", &(prev + 1).to_ne_bytes()).unwrap(); drop(tree); match tx.commit() { Ok(_tx_id) => { successes += 1; } Err(canopydb::Error::WriteConflict) => { // Conflict, retry... } Err(e) => panic!("Commit error: {e}"), } } }); } }); let rx = db.begin_read().unwrap(); let tree = rx.get_tree(b"tree").unwrap().unwrap(); let value = tree.get(b"key").unwrap().unwrap(); let value_usize = usize::from_ne_bytes(value.as_ref().try_into().unwrap()); println!("Final value: {value_usize}"); assert_eq!(THREADS * INC_PER_THREAD, value_usize); ``` -------------------------------- ### Managing Multiple Named Trees in a Database Transaction (Rust) Source: https://context7.com/arthurprs/canopydb/llms.txt Illustrates how to organize data into multiple logical keyspaces using named trees within a single database transaction. This example shows creating, reading, renaming, and deleting trees. It demonstrates efficient data management by grouping related data under distinct tree names. ```rust use canopydb::Database; use tempfile::tempdir; fn main() -> Result<(), canopydb::Error> { let dir = tempdir().unwrap(); let db = Database::new(dir.path())?; // Create multiple trees in a single transaction let tx = db.begin_write()?; // Users tree let mut users = tx.get_or_create_tree(b"users")?; users.insert(b"1", b"Alice")?; users.insert(b"2", b"Bob")?; drop(users); // Orders tree let mut orders = tx.get_or_create_tree(b"orders")?; orders.insert(b"order:1001", b"user:1,product:A,qty:2")?; orders.insert(b"order:1002", b"user:2,product:B,qty:1")?; drop(orders); // Products tree let mut products = tx.get_or_create_tree(b"products")?; products.insert(b"A", b"Widget")?; products.insert(b"B", b"Gadget")?; drop(products); tx.commit()?; // Read from multiple trees let rx = db.begin_read()?; // List all trees in the database let tree_names = rx.list_trees()?; println!("Trees in database:"); for name in &tree_names { println!(" - {:?}", String::from_utf8_lossy(name)); } // Access each tree let users = rx.get_tree(b"users")?.unwrap(); let orders = rx.get_tree(b"orders")?.unwrap(); let products = rx.get_tree(b"products")?.unwrap(); println!("\nUsers: {} entries", users.len()); println!("Orders: {} entries", orders.len()); println!("Products: {} entries", products.len()); // Rename and delete trees let tx = db.begin_write()?; tx.rename_tree(b"users", b"customers")?; tx.delete_tree(b"products")?; tx.commit()?; let rx = db.begin_read()?; assert!(rx.get_tree(b"customers")?.is_some()); assert!(rx.get_tree(b"users")?.is_none()); assert!(rx.get_tree(b"products")?.is_none()); Ok(()) } ``` -------------------------------- ### Concurrent Write Transactions with Retry Logic in Rust Source: https://context7.com/arthurprs/canopydb/llms.txt Demonstrates how to perform concurrent write transactions using optimistic concurrency control and automatic retry logic for handling write conflicts. This is crucial for high-throughput workloads where multiple threads might attempt to modify the same data simultaneously. The example initializes a counter, spawns multiple threads to increment it concurrently, and verifies the final count. ```rust use canopydb::{Database, Error}; use tempfile::tempdir; use std::thread; const THREADS: usize = 4; const INCREMENTS_PER_THREAD: usize = 1000; fn main() -> Result<(), Error> { let dir = tempdir().unwrap(); let db = Database::new(dir.path())?; // Initialize counter let tx = db.begin_write()?; let mut tree = tx.get_or_create_tree(b"counters")?; tree.insert(b"counter", &0u64.to_ne_bytes())?; drop(tree); tx.commit()?; // Spawn multiple threads performing concurrent increments thread::scope(|scope| { for thread_id in 0..THREADS { scope.spawn(|| { let mut successes = 0; let mut conflicts = 0; while successes < INCREMENTS_PER_THREAD { // Begin a concurrent write transaction let tx = db.begin_write_concurrent().unwrap(); let mut tree = tx.get_or_create_tree(b"counters").unwrap(); // Read current value let current = tree.get(b"counter").unwrap() .map(|v| u64::from_ne_bytes(v.as_ref().try_into().unwrap())) .unwrap_or(0); // Increment and write back tree.insert(b"counter", &(current + 1).to_ne_bytes()).unwrap(); drop(tree); // Attempt to commit - may conflict with other transactions match tx.commit() { Ok(_) => successes += 1, Err(Error::WriteConflict) => { conflicts += 1; // Retry the transaction } Err(e) => panic!("Unexpected error: {}", e), } } println!("Thread {}: {} successes, {} conflicts", thread_id, successes, conflicts); }); } }); // Verify final count let rx = db.begin_read()?; let tree = rx.get_tree(b"counters")?.unwrap(); let final_value = tree.get(b"counter")? .map(|v| u64::from_ne_bytes(v.as_ref().try_into().unwrap())) .unwrap(); assert_eq!(final_value, (THREADS * INCREMENTS_PER_THREAD) as u64); println!("Final counter value: {}", final_value); Ok(()) } ``` -------------------------------- ### Configure CanopyDB Database Options Source: https://context7.com/arthurprs/canopydb/llms.txt Demonstrates how to configure environment and database options for CanopyDB. This includes setting page cache size, WAL limits, commit behavior, and checkpointing parameters. It requires the 'canopydb' and 'tempfile' crates. ```rust use canopydb::{Database, EnvOptions, DbOptions}; use std::time::Duration; use tempfile::tempdir; fn main() -> Result<(), canopydb::Error> { let dir = tempdir().unwrap(); // Environment options let mut env_opts = EnvOptions::new(dir.path()); env_opts.page_cache_size = 256 * 1024 * 1024; // 256MB page cache env_opts.use_checksums = true; // Enable page checksums env_opts.max_wal_size = 64 * 1024 * 1024; // 64MB max WAL size env_opts.wal_background_sync_interval = Some(Duration::from_millis(500)); env_opts.file_lock_timeout = Duration::from_secs(10); // Database options let mut db_opts = DbOptions::default(); db_opts.use_wal = true; // Enable Write-Ahead Log db_opts.default_commit_sync = false; // Async commits by default db_opts.checkpoint_interval = Duration::from_secs(30); db_opts.checkpoint_target_size = 32 * 1024 * 1024; // 32MB checkpoint target db_opts.write_txn_memory_limit = 64 * 1024 * 1024; // 64MB uncommitted limit let db = Database::with_options(env_opts, db_opts)?; // Perform operations let tx = db.begin_write()?; let mut tree = tx.get_or_create_tree(b"data")?; for i in 0..10000 { let key = format!("key:{:06}", i); let value = format!("value:{}", i); tree.insert(key.as_bytes(), value.as_bytes())?; } drop(tree); // Commit with explicit sync for durability tx.commit_with(true)?; // sync=true forces fsync // Manual checkpoint to persist in-memory state to disk db.checkpoint()?; // Sync to ensure WAL is durable db.sync()?; println!("Database configured and checkpointed successfully"); Ok(()) } ``` -------------------------------- ### Create Environment with Multiple Databases and Atomic Commits (Rust) Source: https://context7.com/arthurprs/canopydb/llms.txt Demonstrates how to set up an environment with multiple databases in CanopyDB, perform independent transactions, and execute atomic cross-database commits. This is useful for maintaining data consistency across related datasets. ```rust use canopydb::{Environment, EnvOptions, DbOptions}; use tempfile::tempdir; fn main() -> Result<(), canopydb::Error> { let dir = tempdir().unwrap(); // Configure environment options let mut env_options = EnvOptions::new(dir.path()); env_options.page_cache_size = 512 * 1024 * 1024; // 512MB shared cache env_options.max_wal_size = 128 * 1024 * 1024; // 128MB WAL limit let env = Environment::with_options(env_options)?; // Create multiple databases in the environment let db1 = env.get_or_create_database("accounts")?; let db2 = env.get_or_create_database("transactions")?; // Independent transactions on each database let tx1 = db1.begin_write()?; let mut accounts = tx1.get_or_create_tree(b"data")?; accounts.insert(b"acc:1", b"balance:1000")?; accounts.insert(b"acc:2", b"balance:500")?; drop(accounts); tx1.commit()?; let tx2 = db2.begin_write()?; let mut txns = tx2.get_or_create_tree(b"data")?; txns.insert(b"txn:1", b"from:1,to:2,amount:100")?; drop(txns); tx2.commit()?; // Atomic cross-database commit (both or neither) let tx1 = db1.begin_write()?; let tx2 = db2.begin_write()?; // Update account balances let mut accounts = tx1.get_or_create_tree(b"data")?; accounts.insert(b"acc:1", b"balance:900")?; accounts.insert(b"acc:2", b"balance:600")?; drop(accounts); // Record the transaction let mut txns = tx2.get_or_create_tree(b"data")?; txns.insert(b"txn:2", b"from:1,to:2,amount:100")?; drop(txns); // Commit both databases atomically env.group_commit([tx1, tx2], true)?; // sync=true for durability println!("Cross-database atomic commit successful!"); // Verify both databases let rx1 = db1.begin_read()?; let rx2 = db2.begin_read()?; let acc = rx1.get_tree(b"data")?.unwrap(); let txn = rx2.get_tree(b"data")?.unwrap(); println!("Accounts: {} entries", acc.len()); println!("Transactions: {} entries", txn.len()); Ok(()) } ``` -------------------------------- ### Canopydb: Range Queries and Iteration in Rust Source: https://context7.com/arthurprs/canopydb/llms.txt Illustrates how to perform various data retrieval operations using Canopydb's ordered map API, including full tree iteration, range queries based on key bounds, prefix scans, and retrieving the first/last entries. It also shows efficient keys-only iteration. ```rust use canopydb::Database; use tempfile::tempdir; fn main() -> Result<(), canopydb::Error> { let dir = tempdir().unwrap(); let db = Database::new(dir.path())?; // Populate the database let tx = db.begin_write()?; let mut tree = tx.get_or_create_tree(b"users")?; tree.insert(b"user:alice", b"Alice Smith")?; tree.insert(b"user:bob", b"Bob Jones")?; tree.insert(b"user:charlie", b"Charlie Brown")?; tree.insert(b"admin:root", b"Root Admin")?; drop(tree); tx.commit()?; let rx = db.begin_read()?; let tree = rx.get_tree(b"users")?.unwrap(); // Full iteration (sorted by key) println!("All entries:"); for result in tree.iter()? { let (key, value) = result?; println!(" {:?} => {:?}", String::from_utf8_lossy(&key), String::from_utf8_lossy(&value)); } // Range query: keys from "user:bob" onwards println!("\nFrom 'user:bob' onwards:"); for result in tree.range(&b"user:bob"[..]..)? { let (key, value) = result?; println!(" {:?} => {:?}", String::from_utf8_lossy(&key), String::from_utf8_lossy(&value)); } // Prefix scan: all keys starting with "user:" println!("\nAll users (prefix 'user:'):"); for result in tree.prefix(&b"user:")? { let (key, value) = result?; println!(" {:?} => {:?}", String::from_utf8_lossy(&key), String::from_utf8_lossy(&value)); } // Get first and last entries let first = tree.first()?; let last = tree.last()?; println!("\nFirst: {:?}", first.map(|(k, _)| String::from_utf8_lossy(&k).to_string())); println!("Last: {:?}", last.map(|(k, _)| String::from_utf8_lossy(&k).to_string())); // Keys-only iteration (more efficient when values not needed) println!("\nAll keys:"); for result in tree.keys()? { let key = result?; println!(" {:?}", String::from_utf8_lossy(&key)); } Ok(()) } ``` -------------------------------- ### Atomic Cross-Database Commits in Rust Source: https://github.com/arthurprs/canopydb/blob/master/README.md Illustrates how to perform atomic commits across multiple databases within the same CanopyDB environment using Rust. This is useful for ensuring data consistency between different databases. ```rust let sample_data = [ (&b"baz"[..], &b"qux"[..]), (&b"foo"[..], &b"bar"[..]), (&b"qux"[..], &b"quux"[..]), ]; let _dir = tempfile::tempdir().unwrap(); let mut options = canopydb::EnvOptions::new(_dir.path()); // all databases in the same environment will share this 1GB cache options.page_cache_size = 1024 * 1024 * 1024; let env = canopydb::Environment::new(_dir.path()).unwrap(); let db1 = env.get_or_create_database("db1").unwrap(); let db2 = env.get_or_create_database("db2").unwrap(); // Each database unique write transaction is independent. let tx1 = db1.begin_write().unwrap(); let tx2 = db2.begin_write().unwrap(); let mut tree = tx1.get_or_create_tree(b"my_tree").unwrap(); tree.insert(b"foo", b"bar").unwrap(); drop(tree); tx1.commit().unwrap(); tx2.rollback().unwrap(); // Write transactions for databases in the same environment // can be committed together atomically. // This allows stablishing a consistent state between them. let tx1 = db1.begin_write().unwrap(); let tx2 = db2.begin_write().unwrap(); // Use tx1 and tx2 here.. env.group_commit([tx1, tx2], false).unwrap(); ``` -------------------------------- ### Compact CanopyDB Database to Reclaim Space Source: https://context7.com/arthurprs/canopydb/llms.txt Illustrates how to compact a CanopyDB database to reclaim disk space after significant data deletion. This process defragments the database file, making it smaller. It's typically performed after large-scale deletions to optimize storage. ```rust use canopydb::Database; use tempfile::tempdir; fn main() -> Result<(), canopydb::Error> { let dir = tempdir().unwrap(); let db = Database::new(dir.path())?; // Insert a lot of data let tx = db.begin_write()?; let mut tree = tx.get_or_create_tree(b"data")?; for i in 0..50000 { let key = format!("key:{:08}", i); let value = format!("value:{:0100}", i); // 100+ byte values tree.insert(key.as_bytes(), value.as_bytes())?; } drop(tree); tx.commit()?; // Force checkpoint to write data to disk db.checkpoint()?; // Delete most of the data let tx = db.begin_write()?; let mut tree = tx.get_or_create_tree(b"data")?; // Keep only keys 40000-50000 tree.delete_range(&b"key:00000000"[..]..&b"key:00040000"[..])?; drop(tree); tx.commit()?; db.checkpoint()?; println!("Before compaction - check file size in: {:?}", dir.path()); // Compact the database to reclaim space // This defragments and shrinks the database file db.compact()?; println!("After compaction - file should be smaller"); // Verify data integrity let rx = db.begin_read()?; let tree = rx.get_tree(b"data")?.unwrap(); println!("Remaining entries: {}", tree.len()); Ok(()) } ``` -------------------------------- ### Delete Key Ranges from CanopyDB Tree Source: https://context7.com/arthurprs/canopydb/llms.txt Demonstrates how to efficiently delete a range of keys from a specified tree within a CanopyDB database. This operation is useful for managing time-series data or cleaning up specific data segments. It requires a mutable transaction and a reference to the tree. ```rust use canopydb::Database; use tempfile::tempdir; fn main() -> Result<(), canopydb::Error> { let dir = tempdir().unwrap(); let db = Database::new(dir.path())?; // Populate with time-series data let tx = db.begin_write()?; let mut tree = tx.get_or_create_tree(b"events")?; for day in 1..=31 { for hour in 0..24 { let key = format!("2024-01-{:02}T{:02}:00", day, hour); let value = format!("event_data_{}", day * 100 + hour); tree.insert(key.as_bytes(), value.as_bytes())?; } } drop(tree); tx.commit()?; println!("Initial count:"); let rx = db.begin_read()?; println!(" {} events", rx.get_tree(b"events")?.unwrap().len()); drop(rx); // Delete a range of keys (e.g., first week of January) let tx = db.begin_write()?; let mut tree = tx.get_or_create_tree(b"events")?; // Delete all events from Jan 1-7 let start = b"2024-01-01"; let end = b"2024-01-08"; // Exclusive tree.delete_range(&start[..]..&end[..])?; drop(tree); tx.commit()?; println!("\nAfter deleting first week:"); let rx = db.begin_read()?; let tree = rx.get_tree(b"events")?.unwrap(); println!(" {} events remaining", tree.len()); // Verify first remaining event let first = tree.first()?; if let Some((key, _)) = first { println!(" First event: {}", String::from_utf8_lossy(&key)); } // Clear entire tree let tx = db.begin_write()?; let mut tree = tx.get_or_create_tree(b"events")?; tree.clear()?; drop(tree); tx.commit()?; let rx = db.begin_read()?; println!("\nAfter clear: {} events", rx.get_tree(b"events")?.unwrap().len()); Ok(()) } ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.