SurrealKV

https://docs.rs/surrealkv/0.21.0
Admin
SurrealKV is a high-performance Log-Structured Merge (LSM) tree implementation in Rust providing...
Tokens:374,373
Snippets:2,131
Trust Score:9.1
Update:6 months ago
Show doc for...
Tokens:
Raw
### CompactionIterator usage example

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/iter.rs.html

Demonstrates how to iterate over the CompactionIterator using a for loop.

```text
let compaction_iter = CompactionIterator::new(...);
for result in compaction_iter {
    let (key, value) = result?;
    // Write to output SSTable
}
```

--------------------------------

### Test Case Initialization

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/task.rs.html

Example of initializing a test case for the task manager.

```rust
#[test(tokio::test(flavor = "multi_thread"))]
	async fn test_wake_up_memtable() {
		let opts = Arc::new(Options::default());
```

--------------------------------

### Tree Creation and Initialization

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/lsm.rs.html

Details on how to create and initialize a new LSM Tree, including directory structure setup.

```APIDOC
## POST /api/tree/new

### Description
Creates a new LSM tree with the specified options and initializes its directory structure.

### Method
POST

### Endpoint
/api/tree/new

### Parameters
#### Request Body
- **opts** (object) - Required - Options for configuring the LSM tree.
  - **path** (string) - Required - The base path for the LSM tree.
  - **enable_vlog** (boolean) - Optional - Enables the vlog feature.
  - **enable_versioning** (boolean) - Optional - Enables versioning.

### Request Example
```json
{
  "opts": {
    "path": "/data/lsm_tree",
    "enable_vlog": true,
    "enable_versioning": false
  }
}
```

### Response
#### Success Response (200)
- **tree_id** (string) - Identifier for the newly created LSM tree.

#### Response Example
```json
{
  "tree_id": "tree_abc123"
}
```
```

--------------------------------

### Prefix Compression Example

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/sstable/block.rs.html

Demonstrates how prefix compression works with a sequence of keys and a specified restart interval, showing the resulting entries.

```text
Key sequence: "apple", "apricot", "banana"
Restart interval: 2

Entry 0 (restart point):
  shared=0, unshared=5, key_suffix="apple"
  Full key: "" + "apple" = "apple"

Entry 1:
  shared=2, unshared=5, key_suffix="ricot"
  Full key: "ap" + "ricot" = "apricot"

Entry 2 (new restart point because interval=2):
  shared=0, unshared=6, key_suffix="banana"
  Full key: "" + "banana" = "banana"
```

--------------------------------

### Iterate Keys in a Range

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/transaction.rs.html

Use this method to get keys and values within a specified range at the current timestamp. The iterator includes the start key but excludes the end key. Ensure you call `seek_first()` before iterating with `next()`.

```rust
let mut iter = tx.range(b"a", b"z")?;
iter.seek_first()?;
while iter.valid() {
    let key = iter.key().user_key();
    let ts = iter.key().timestamp();
    let value = iter.value()?;
    iter.next()?;
}

// Or seek to a specific key:
iter.seek(b"foo")?  // Position at first version of "foo"
```

--------------------------------

### Excluded Bound Seek Example

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/sstable/table.rs.html

Example of seeking with an Excluded bound.

```text
Example: Excluded("apple") with data [("apple",5), ("apple",3), ("banana",7)]

seek(("apple", 0)):
  ("apple", 5) < ("apple", 0)  → skip (5 > 0 descending)
  ("apple", 3) < ("apple", 0)  → skip
  ("banana", 7) >= ("apple", 0) → land here ✓

Check: user_key == "apple"? No, it's "banana" → done
Result: ("banana", 7) - first entry with user_key > "apple"
```

--------------------------------

### Initialize BPlusTree

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/bplustree/tree.rs.html

Initializes a new BPlusTree, setting up the file, header, and cache. It also handles the creation of the initial root node and writes the header to disk if the storage size is zero.

```rust
let mut tree = BPlusTree {
			file,
			header,
			cache,
			compare,
			durability: Durability::Manual,
		};

		// Initialize storage if it's a new tree
		if storage_size == 0 {
			let header_bytes = tree.header.serialize();
			let mut buffer = vec![0u8; PAGE_SIZE];
			buffer[..header_bytes.len()].copy_from_slice(&header_bytes);

			tree.file.write_at(0, &buffer)?;

			// Create initial root node
			let root = LeafNode::new(tree.header.root_offset);
			tree.write_node(&NodeType::Leaf(root))?;
			tree.file.sync_data()?;
		} else {
			// Read root node into cache
			tree.read_node(tree.header.root_offset)?;
		}

		Ok(tree)
```

--------------------------------

### Included Bound Seek Example

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/sstable/table.rs.html

Example of seeking with an Included bound.

```text
Example: Included("banana") with data [("apple",5), ("banana",7), ("banana",2)]

seek(("banana", MAX_SEQ)):
  ("apple", 5) < ("banana", MAX_SEQ)  → skip
  ("banana", 7) >= ("banana", MAX_SEQ) → land here ✓

Result: ("banana", 7) - first entry with user_key >= "banana"
```

--------------------------------

### Initialize VLogFile

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/vlog.rs.html

Creates a new VLogFile instance with the specified ID and path.

```rust
fn new(_id: u32, path: PathBuf, _size: u64) -> Self {
		Self {
			path,
		}
	}
```

--------------------------------

### Get Type ID

Source: https://docs.rs/surrealkv/0.21.0/surrealkv/struct.InternalKeyComparator.html

Gets the `TypeId` of the implementing type. This is part of the `Any` trait implementation.

```rust
fn type_id(&self) -> TypeId>
```

--------------------------------

### Set the starting sequence number for the batch

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/batch.rs.html

Allows modification of the batch's starting sequence number. This is useful for re-assigning sequence numbers if a batch is being reused or re-processed.

```rust
pub(crate) fn set_starting_seq_num(&mut self, seq_num: u64) {
		self.starting_seq_num = seq_num;
	}
```

--------------------------------

### Implement Strategy Initialization

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/compaction/leveled.rs.html

Provides methods to initialize the compaction strategy from existing configuration options.

```rust
36impl Default for Strategy {
37	fn default() -> Self {
38		let opts = Options::default();
39		Self {
40			level0_file_num_trigger: opts.level0_max_files,
41			max_bytes_for_level: opts.max_bytes_for_level,
42			level_multiplier: opts.level_multiplier,
43			compaction_priority: CompactionPriority::default(),
44		}
45	}
46}
47
48impl Strategy {
49	/// Create a Strategy from Options
50	pub(crate) fn from_options(opts: Arc<Options>) -> Self {
51		Self {
52			level0_file_num_trigger: opts.level0_max_files,
53			max_bytes_for_level: opts.max_bytes_for_level,
54			level_multiplier: opts.level_multiplier,
55			compaction_priority: CompactionPriority::default(),
56		}
57	}
58
59	/// Create a Strategy from Options with a specific compaction priority
60	#[cfg(test)]
61	pub(crate) fn from_options_with_priority(
62		opts: Arc<Options>,
63		priority: CompactionPriority,
64	) -> Self {
65		Self {
66			level0_file_num_trigger: opts.level0_max_files,
67			max_bytes_for_level: opts.max_bytes_for_level,
68			level_multiplier: opts.level_multiplier,
69			compaction_priority: priority,
70		}
71	}
```

--------------------------------

### Initialize New BPlusTree and Verify Header

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/bplustree/tree.rs.html

Creates a new BPlusTree on disk and verifies that the header information is correctly initialized by reading it directly from the file.

```rust
let temp_file = NamedTempFile::new().unwrap();
let path = temp_file.path();

// Create new BPlusTree
let _tree = BPlusTree::disk(path, Arc::new(TestComparator)).unwrap();

// Read header directly from file
let mut file = File::open(path).unwrap();
let mut buffer = [0u8; 48];
file.read_exact(&mut buffer).unwrap();

let header = Header::deserialize(&buffer).unwrap();

assert_eq!(header.magic, MAGIC);
assert_eq!(header.version, 1);
assert_eq!(header.root_offset, PAGE_SIZE as u64);
assert_eq!(header.free_page_count, 0);
assert_eq!(header.trunk_page_head, 0);
assert_eq!(header.total_pages, 2);
```

--------------------------------

### Get Immutable Raw Pointer to Vec Buffer

Source: https://docs.rs/surrealkv/0.21.0/surrealkv/type.Value.html

Use `as_ptr` to get a raw pointer to the vector's buffer. Ensure the vector outlives the pointer and do not mutate the memory through this pointer.

```rust
let x = vec![1, 2, 4];
let x_ptr = x.as_ptr();

unsafe {
    for i in 0..x.len() {
        assert_eq!(*x_ptr.add(i), 1 << i);
    }
}
```

```rust
unsafe {
    let mut v = vec![0, 1, 2];
    let ptr1 = v.as_ptr();
    let _ = ptr1.read();
    let ptr2 = v.as_mut_ptr().offset(2);
    ptr2.write(2);
    // Notably, the write to `ptr2` did *not* invalidate `ptr1`
    // because it mutated a different element:
    let _ = ptr1.read();
}
```

--------------------------------

### Get Mutable Raw Pointer to Vec Buffer

Source: https://docs.rs/surrealkv/0.21.0/surrealkv/type.Value.html

Use `as_mut_ptr` to get a mutable raw pointer to the vector's buffer. The caller must ensure the vector outlives the pointer and handle potential reallocations.

```rust
// Allocate vector big enough for 4 elements.
let size = 4;
let mut x: Vec<i32> = Vec::with_capacity(size);
let x_ptr = x.as_mut_ptr();

// Initialize elements via raw pointer writes, then set length.
unsafe {
    for i in 0..size {
        *x_ptr.add(i) = i as i32;
    }
    x.set_len(size);
}
assert_eq!(&*x, &[0, 1, 2, 3]);
```

```rust
unsafe {
    let mut v = vec![0];
    let ptr1 = v.as_mut_ptr();
    ptr1.write(1);
    let ptr2 = v.as_mut_ptr();
    ptr2.write(2);
    // Notably, the write to `ptr2` did *not* invalidate `ptr1`:
    ptr1.write(3);
}
```

```rust
use std::mem::{ManuallyDrop, MaybeUninit};

let mut v = ManuallyDrop::new(vec![0, 1, 2]);
let ptr = v.as_mut_ptr();
let capacity = v.capacity();
let slice_ptr: *mut [MaybeUninit<i32>] =
    std::ptr::slice_from_raw_parts_mut(ptr.cast(), capacity);
drop(unsafe { Box::from_raw(slice_ptr) });
```

--------------------------------

### Test Internal Key Separator with Misordered User Keys

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/comparator.rs.html

Tests the `separator` method when the user keys are misordered (start key is lexicographically greater than the limit key). In this scenario, the method should return the start key unchanged.

```rust
#[test]
fn test_internal_key_short_separator_misordered() {
	let cmp = InternalKeyComparator::new(Arc::new(BytewiseComparator::default()));

	// When user keys are misordered (start > limit), return unchanged
	assert_eq!(
		ikey(b"foo", 100, InternalKeyKind::Set),
		cmp.separator(
				&ikey(b"foo", 100, InternalKeyKind::Set),
				&ikey(b"bar", 99, InternalKeyKind::Set)
			)
	);
}
```

--------------------------------

### VLog Initialization

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/vlog.rs.html

Creates a new VLog instance and triggers the pre-filling of existing file handles.

```rust
pub(crate) fn new(opts: Arc<Options>) -> Result<Self> {
		let vlog = Self {
			path: opts.vlog_dir(),
			max_file_size: opts.vlog_max_file_size,
			checksum_level: opts.vlog_checksum_verification,
			next_file_id: AtomicU32::new(1),
			active_writer_id: AtomicU32::new(0),
			writer: RwLock::new(None),
			files_map: RwLock::new(HashMap::new()),
			file_handles: RwLock::new(HashMap::new()),
			opts,
		};

		// PRE-FILL ALL EXISTING FILE HANDLES ON STARTUP
		vlog.prefill_file_handles()?;

		Ok(vlog)
	}
```

--------------------------------

### Read Overflow Chain Data

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/bplustree/tree.rs.html

Reads data sequentially from a chain of overflow pages starting from a given page offset. Returns an empty Bytes if the starting page is 0. Returns an error if an invalid node type is encountered in the chain.

```rust
let mut result = BytesMut::new();
let mut current_offset = first_page;

while current_offset != 0 {
	let node = self.read_node(current_offset)?;
	match node.as_ref() {
		NodeType::Overflow(overflow) => {
			result.extend_from_slice(&overflow.data);
			current_offset = overflow.next_overflow;
		}
		_ => {
			return Err(BPlusTreeError::InvalidOverflowChain(current_offset));
		}
	}
}

Ok(result.freeze())
```

--------------------------------

### Core Initialization

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/lsm.rs.html

Initializes a new LSM tree with background task management. Sets up the database path and creates the inner core structure.

```APIDOC
## new

### Description
Creates a new LSM tree with background task management.

### Method
`pub(crate) fn new(opts: Arc<Options>) -> Result<Self>`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
None

### Response
#### Success Response (200)
- `Self`: An instance of the Core struct representing the initialized LSM tree.

#### Response Example
```json
{
  "example": "Core { ... }"
}
```

### Error Handling
- `Result<Self>`: Returns Ok(Self) on success or an Err on failure during initialization.
```

--------------------------------

### get_at

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/transaction.rs.html

Gets a value for a key at a specific timestamp.

```APIDOC
## get_at

### Description
Gets a value for a key at a specific timestamp. Requires versioned queries to be enabled.

### Parameters
#### Request Body
- **key** (IntoBytes) - Required - The key to retrieve.
- **timestamp** (u64) - Required - The timestamp for the versioned read.

### Response
#### Success Response (200)
- **value** (Option<Value>) - The value associated with the key if it exists.
```

--------------------------------

### Create New Options Instance

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/lib.rs.html

Initializes a new `Options` instance using the default configuration.

```rust
pub fn new() -> Self {
		Self::default()
	}
```

--------------------------------

### Initialize VLog Writer

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/vlog.rs.html

Sets up the next file ID and initializes the active VLog writer based on existing files.

```rust
if let Some(highest_file_id) = max_file_id {
			// Found existing files, set next_file_id to one past the highest existing file
			// ID
			self.next_file_id.store(highest_file_id + 1, Ordering::SeqCst);

			// Set the last file up as the active writer
			let writer = VLogWriter::new(
				&max_file_path.unwrap(),
				highest_file_id,
				self.opts.vlog_max_file_size,
				CompressionType::None as u8,
			)?;
			self.active_writer_id.store(highest_file_id, Ordering::SeqCst);
			*self.writer.write() = Some(writer);
		}
```

--------------------------------

### Get Length of Vec

Source: https://docs.rs/surrealkv/0.21.0/surrealkv/type.Value.html

Returns the number of elements currently in the vector.

```rust
let a = vec![1, 2, 3];
assert_eq!(a.len(), 3);

```

--------------------------------

### DiskBPlusTree Initialization and Configuration

Source: https://docs.rs/surrealkv/0.21.0/surrealkv/bplustree/tree/type.DiskBPlusTree.html

Methods for initializing and configuring a DiskBPlusTree.

```APIDOC
## DiskBPlusTree Initialization and Configuration

### `disk` Method

#### Description

Creates a new `DiskBPlusTree` instance associated with a specific file path.

#### Method

`pub fn disk<P: AsRef<Path>>(path: P, compare: Arc<dyn Comparator>) -> Result<Self, BPlusTreeError>`

#### Parameters

- **path** (P: AsRef<Path>) - Required - The file path for the BPlusTree.
- **compare** (Arc<dyn Comparator>) - Required - The comparator function for ordering keys.

#### Response

- **Success Response (200)**: `Result<Self, BPlusTreeError>` - A `DiskBPlusTree` instance on success, or a `BPlusTreeError` on failure.

### `with_file` Method

#### Description

Initializes a `BPlusTree` with a given file and comparator. This is a generic method that `DiskBPlusTree` can utilize.

#### Method

`pub fn with_file(file: F, compare: Arc<dyn Comparator>) -> Result<Self, BPlusTreeError>`

#### Parameters

- **file** (F: VfsFile) - Required - The file object to use for storage.
- **compare** (Arc<dyn Comparator>) - Required - The comparator function for ordering keys.

#### Response

- **Success Response (200)**: `Result<Self, BPlusTreeError>` - A `BPlusTree` instance on success, or a `BPlusTreeError` on failure.

### `set_durability` Method

#### Description

Sets the durability level for the `BPlusTree`.

#### Method

`pub fn set_durability(&mut self, durability: Durability)`

#### Parameters

- **durability** (Durability) - Required - The durability setting to apply.
```

--------------------------------

### Initialize VLogWriter

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/vlog.rs.html

Creates a new VLogWriter, handling file creation and optional header writing.

```rust
pub(crate) fn new(
		path: &Path,
		file_id: u32,
		max_file_size: u64,
		compression: u8,
	) -> Result<Self> {
		let file_exists = path.exists();
		let file = OpenOptions::new().create(true).append(true).open(path)?;

		// Clone the fd before BufWriter consumes ownership. This cloned fd
		// is used to perform fsync outside the write lock.
		let sync_fd = Arc::new(file.try_clone()?);

		let current_offset = file.metadata()?.len();
		let mut writer = BufWriter::new(file);

		// If this is a new file, write the header
		if !file_exists || current_offset == 0 {
			let header = VLogFileHeader::new(file_id, max_file_size, compression);
			let header_bytes = header.encode();
			writer.write_all(&header_bytes)?;
			writer.flush()?;
		}

		// Update offset to account for header
		let current_offset = if !file_exists || current_offset == 0 {
			VLogFileHeader::SIZE as u64
		} else {
			current_offset
		};

		Ok(Self {
			writer,
			sync_fd,
			current_offset,
			file_id,
			bytes_written: current_offset,
		})
	}
```

--------------------------------

### Get Entry Count

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/sstable/block.rs.html

Returns the number of entries stored in the block.

```rust
478	pub(crate) fn entries(&self) -> usize {
479		self.num_entries
480	}
```

--------------------------------

### Initialize and Write Trunk Pages

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/bplustree/tree.rs.html

Initializes trunk pages and writes them to the BPlusTree. This setup is used before performing page allocation tests.

```rust
let mut trunk1 = TrunkPage::new(trunk1_offset);
trunk1.add_free_page(4 * PAGE_SIZE as u64); // Free page 4
trunk1.next_trunk = trunk2_offset;
bloop.write_trunk_page(&trunk1).unwrap();

// Create trunk2 with one free page, pointing to trunk3
let mut trunk2 = TrunkPage::new(trunk2_offset);
trunk2.add_free_page(5 * PAGE_SIZE as u64); // Free page 5
trunk2.next_trunk = trunk3_offset;
bloop.write_trunk_page(&trunk2).unwrap();

// Create trunk3 with one free page
let mut trunk3 = TrunkPage::new(trunk3_offset);
trunk3.add_free_page(6 * PAGE_SIZE as u64); // Free page 6
bloop.write_trunk_page(&trunk3).unwrap();
```

--------------------------------

### Initialize or Load LevelManifest

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/levels/mod.rs.html

Handles manifest creation if the file is missing or loads an existing one from disk.

```rust
if manifest_file_path.exists() {
			// Load existing manifest from file
			return Self::load_from_file(&manifest_file_path, opts);
		}

		// If no manifest exists, create a new one

		// Initialize levels with default values
		let levels = Self::initialize_levels(opts.level_count);

		// Start with next_table_id = 1 (0 is reserved)
		let next_table_id = Arc::new(AtomicU64::new(1));

		let manifest = Self {
			path: manifest_file_path,
			levels,
			hidden_set: HashSet::with_capacity(10),
			next_table_id,
			manifest_format_version: MANIFEST_FORMAT_VERSION_V1,
			snapshots: Vec::new(),
			log_number: 0,
			last_sequence: 0,
		};

		// Write levels to disk with the counter
		write_manifest_to_disk(&manifest)?;

		Ok(manifest)
```

--------------------------------

### Slice Extraction

Source: https://docs.rs/surrealkv/0.21.0/surrealkv/type.Value.html

Methods to get immutable and mutable slices of the entire vector.

```APIDOC
## GET /vector/as_slice

### Description
Extracts an immutable slice containing the entire vector. Equivalent to `&s[..]`.

### Method
GET

### Endpoint
/vector/as_slice

### Parameters
#### Query Parameters
- **vector_ref** (Vec<T>) - Required - A reference to the vector.

### Response
#### Success Response (200)
- **&[T]** - An immutable slice of the vector.

### Request Example
```json
{
  "vector_ref": [1, 2, 3, 5, 8]
}
```

### Response Example
```json
{
  "slice": [1, 2, 3, 5, 8]
}
```

## GET /vector/as_mut_slice

### Description
Extracts a mutable slice of the entire vector. Equivalent to `&mut s[..]`.

### Method
GET

### Endpoint
/vector/as_mut_slice

### Parameters
#### Query Parameters
- **vector_mut_ref** (Vec<T>) - Required - A mutable reference to the vector.

### Response
#### Success Response (200)
- **&mut [T]** - A mutable slice of the vector.

### Request Example
```json
{
  "vector_mut_ref": [0, 0, 0]
}
```

### Response Example
```json
{
  "mutable_slice": [0, 0, 0]
}
```
```

--------------------------------

### Get Restart Point

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/sstable/block.rs.html

Retrieves the offset of a specific restart point by its index.

```APIDOC
## BlockIterator::get_restart_point

### Description
Gets a restart point offset by index.

### Method
`get_restart_point`

### Endpoint
N/A (Method on `BlockIterator` struct)

### Parameters
#### Path Parameters
N/A

#### Query Parameters
N/A

#### Path Parameters
- **index** (usize) - Required - The index of the restart point to retrieve.

### Request Example
```rust
let offset = block_iterator.get_restart_point(5);
```

### Response
#### Success Response (usize)
- **usize** - The offset of the specified restart point.
```

--------------------------------

### SurrealKV Configuration Methods

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/lib.rs.html

Methods for configuring the SurrealKV database instance via the Options builder.

```APIDOC
## Configuration Methods

### Description
These methods are used to configure the database instance before building. They follow a builder pattern, returning the modified instance.

### Parameters
- **with_l0_no_compression** () - Configures L0 to use no compression.
- **with_path** (PathBuf) - Sets the database storage path.
- **with_level_count** (u8) - Sets the number of LSM levels.
- **with_max_memtable_size** (usize) - Sets the maximum size of a memtable.
- **with_block_cache_capacity** (u64) - Sets the unified block cache capacity in bytes.
- **with_index_partition_size** (usize) - Configures the partitioned index size.
- **with_vlog_max_file_size** (u64) - Sets the maximum size for VLog files.
- **with_vlog_checksum_verification** (VLogChecksumLevel) - Sets the VLog checksum verification level.
- **with_enable_vlog** (bool) - Enables or disables the Value Log (VLog).
- **with_vlog_value_threshold** (usize) - Sets the threshold in bytes for storing values in VLog vs SSTables.
- **with_versioning** (bool, u64) - Enables versioned queries and sets retention time in nanoseconds.
- **with_versioned_index** (bool) - Enables the B+tree versioned index.
- **with_flush_on_close** (bool) - Determines if memtables are flushed during shutdown.
- **with_wal_recovery_mode** (WalRecoveryMode) - Sets the WAL recovery behavior.
- **with_memtable_stall_threshold** (usize) - Sets the threshold for write stalls based on immutable memtables.
- **with_l0_stall_threshold** (usize) - Sets the threshold for write stalls based on L0 file count.
```

--------------------------------

### Get Memtable Stall Limit

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/stall.rs.html

Retrieves the configured memtable stall limit.

```rust
self.thresholds.memtable_limit
```

--------------------------------

### Manage Directory Structure

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/lsm.rs.html

Creates the necessary subdirectories for SSTables, WAL, manifests, and optional VLog or versioning components.

```rust
fn create_directory_structure(opts: &Options) -> Result<()> {
		// Create base directory
		create_dir_all(&opts.path)?;

		// Create all subdirectories
		create_dir_all(opts.sstable_dir())?;
		create_dir_all(opts.wal_dir())?;
		create_dir_all(opts.manifest_dir())?;

		// Create VLog directories
		if opts.enable_vlog {
			create_dir_all(opts.vlog_dir())?;
		}

		if opts.enable_versioning {
			create_dir_all(opts.versioned_index_dir())?;
		}

		Ok(())
	}
```

--------------------------------

### Disable compression in TreeBuilder

Source: https://docs.rs/surrealkv/0.21.0/surrealkv/struct.TreeBuilder.html

Example of configuring a TreeBuilder to disable data block compression.

```rust
use surrealkv::TreeBuilder;

let tree = TreeBuilder::new()
    .with_path("./data".into())
    .without_compression()
    .build()
    .unwrap();
```

--------------------------------

### Initialize and Open WAL Instance

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/wal/manager.rs.html

Opens or creates a new WAL instance, ensuring directory preparation and stale file cleanup.

```rust
pub(crate) fn open(dir: &Path, opts: Options) -> Result<Self> {
		// Ensure the options are valid
		opts.validate()?;

		// Ensure the directory exists with proper permissions
		Self::prepare_directory(dir, &opts)?;

		// Clean up any stale .wal.repair files from previous crashed repair attempts
		Self::cleanup_stale_repair_files(dir)?;

		// Determine the active log number
		let active_log_number = Self::calculate_active_log_number(dir)?;

		// Create the active Writer and sync fd
		let (active_writer, sync_fd) = Self::create_writer(dir, active_log_number, &opts)?;

		Ok(Self {
			active_writer,
			sync_fd,
			active_log_number,
			dir: dir.to_path_buf(),
			opts,
			closed: false,
		})
	}
```

```rust
pub(crate) fn open_with_min_log_number(
		dir: &Path,
		min_log_number: u64,
		opts: Options,
	) -> Result<Self> {
		// Ensure the options are valid
		opts.validate()?;

		// Ensure the directory exists with proper permissions
		Self::prepare_directory(dir, &opts)?;

		// Clean up any stale .wal.repair files from previous crashed repair attempts
		Self::cleanup_stale_repair_files(dir)?;

		// Use max of min_log_number and highest existing segment on disk.
		// This prevents writing to an old segment that could be skipped on recovery.
		//
		// Example scenario this fixes:
		// 1. Crash with segments #5, #6, #7 on disk
		// 2. Recovery replays all three into memtable
		// 3. Without this fix, new writes would go to segment #5
		// 4. If memtable flushes (log_number = 6), then crash again
		// 5. Segment #5 would be skipped, losing the new writes
		let highest_on_disk = Self::calculate_active_log_number(dir).unwrap_or(0);
		let active_log_number = std::cmp::max(min_log_number, highest_on_disk);

		if active_log_number > min_log_number {
			log::info!(
				"WAL: advancing from min_log_number {} to {} (highest existing segment)",
				min_log_number,
				active_log_number
			);
		}

		// Create the active Writer and sync fd at the calculated log number
		let (active_writer, sync_fd) = Self::create_writer(dir, active_log_number, &opts)?;

		Ok(Self {
			active_writer,
			sync_fd,
			active_log_number,
			dir: dir.to_path_buf(),
			opts,
			closed: false,
		})
	}
```

--------------------------------

### GET /capacity

Source: https://docs.rs/surrealkv/0.21.0/surrealkv/type.Value.html

Returns the total number of elements the vector can hold without reallocating.

```APIDOC
## GET /capacity

### Description
Returns the total number of elements the vector can hold without reallocating.

### Method
GET

### Response
#### Success Response (200)
- **capacity** (usize) - The number of elements the vector can hold.
```

--------------------------------

### Type Identification

Source: https://docs.rs/surrealkv/0.21.0/surrealkv/enum.Error.html

Provides a method to get the unique TypeId of a static, sized type.

```APIDOC
## GET /api/type_id

### Description
Gets the `TypeId` of `self`.

### Method
GET

### Endpoint
/api/type_id

### Parameters
#### Query Parameters
- **self** (T) - Required - The object to get the TypeId from.

### Response
#### Success Response (200)
- **type_id** (TypeId) - The unique identifier for the type.
```

--------------------------------

### Initialize SurrealKV Core

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/lsm.rs.html

Initializes the core SurrealKV structure, including setting up stall controllers, task managers, commit environments, and pipelines. It replays the Write-Ahead Log (WAL) and recovers the memtable if necessary. This process ensures data consistency and prepares the database for operation.

```rust
let wal_seq_num_opt = Self::replay_wal_with_repair(
			&wal_path,
			min_wal_number,
			"Database startup",
			opts.wal_recovery_mode,
			opts.max_memtable_size,
			|memtable, wal_number| {
				// Flush intermediate memtable to SST during recovery
				let table_id = inner.level_manifest.read()?.next_table_id();
				inner.flush_immutable_to_sst(Arc::clone(&memtable), table_id, wal_number)?;
				log::info!(
					"Recovery: flushed memtable to SST table_id={}, wal_number={}",
					table_id,
					wal_number
				);
				Ok(())
			},
		)?;
```

```rust
if let Some(memtable) = recovered_memtable {
			let mut active_memtable = inner.active_memtable.write()?;
			*active_memtable = memtable;
		}
```

```rust
{
			let active_memtable = inner.active_memtable.read()?;
			let current_wal_number = inner.wal.read().get_active_log_number();
			active_memtable.set_wal_number(current_wal_number);
		}
```

```rust
let max_seq_num = match wal_seq_num_opt {
			Some(wal_seq) => {
				let effective = std::cmp::max(manifest_last_seq, wal_seq);
				log::debug!(
					"WAL replayed: manifest_last_seq={}, wal_seq={}, using max={}",
					manifest_last_seq,
					wal_seq,
					effective
				);
				effective
			}
			None => {
				log::debug!("WAL skipped or empty, using manifest_last_seq={}", manifest_last_seq);
				manifest_last_seq
			}
		};
```

```rust
commit_pipeline.set_seq_num(max_seq_num);
```

```rust
inner.cleanup_orphaned_sst_files()?;
```

```rust
inner.cleanup_orphaned_vlog_files()?;
```

```rust
task_manager.wake_up_level();
```

```rust
let core = Self {
			inner: Arc::clone(&inner),
			commit_pipeline: Arc::clone(&commit_pipeline),
			task_manager: Mutex::new(Some(task_manager)),
			write_stall,
		};

		log::info!("=== LSM tree initialization complete ===");

		Ok(core)
```

--------------------------------

### Options Configuration

Source: https://docs.rs/surrealkv/0.21.0/surrealkv/struct.Options.html

The `Options` struct allows for detailed configuration of SurrealKV's behavior. You can customize aspects like block size, compression, versioning, and WAL recovery.

```APIDOC
## Struct Options

### Fields
- `block_size` (usize)
- `block_restart_interval` (usize)
- `filter_policy` (Option<Arc<dyn FilterPolicy>>)
- `comparator` (Arc<dyn Comparator>)
- `compression_per_level` (Vec<CompressionType>)
- `path` (PathBuf)
- `level_count` (u8)
- `max_memtable_size` (usize)
- `index_partition_size` (usize)
- `vlog_max_file_size` (u64)
- `vlog_checksum_verification` (VLogChecksumLevel)
- `enable_vlog` (bool) - If true, disables `VLog` creation entirely
- `vlog_value_threshold` (usize) - If value size is less than this, it will be stored inline in `SSTable`
- `enable_versioning` (bool) - If true, enables versioned queries with timestamp tracking
- `enable_versioned_index` (bool) - If true, creates a B+tree index for timestamp-based queries. Requires `enable_versioning` to be true. When false, versioned queries will scan the LSM tree directly.
- `versioned_history_retention_ns` (u64) - History retention period in nanoseconds (0 means no retention limit). Default: 0 (no retention limit)
- `flush_on_close` (bool) - If true, flush active memtable to SSTable during shutdown. If false, skip flush for faster shutdown. DEFAULT: false
- `wal_recovery_mode` (WalRecoveryMode) - Controls behavior when WAL corruption is detected during recovery. Default: TolerateCorruptedWithRepair (attempt repair and continue)
- `level0_max_files` (usize) - Number of L0 files that trigger compaction. Default: 4
- `max_bytes_for_level` (u64) - Base size for level 1 in bytes. Default: 256MB
- `level_multiplier` (f64) - Multiplier for calculating max bytes for each level. Level N max bytes = base * multiplier^(N-1). Default: 10.0
- `memtable_stall_threshold` (usize) - Number of immutable memtables that triggers write stall. When immutable memtable count >= this threshold, writes block until flushes complete. Default: 2
- `l0_stall_threshold` (usize) - Number of L0 files that triggers write stall. When L0 file count >= this threshold, writes block until compactions complete. Should be >= `level0_max_files` (compaction trigger). Default: 12 (3x `level0_max_files`)

### impl Options

#### pub fn new() -> Self

#### pub const fn with_block_size(self, value: usize) -> Self

#### pub const fn with_block_restart_interval(self, value: usize) -> Self

#### pub fn with_filter_policy(self, value: Option<Arc<dyn FilterPolicy>>) -> Self

#### pub fn with_comparator(self, value: Arc<dyn Comparator>) -> Self

#### pub fn without_compression(self) -> Self
Disables compression for data blocks in SSTables. This clears the `compression_per_level` vector, causing all levels to default to `CompressionType::None`.

##### Example
```rust
use surrealkv::Options;

let opts = Options::new().without_compression();
```

#### pub fn with_compression_per_level(self, levels: Vec<CompressionType>) -> Self
Sets compression per level. Vector index corresponds to level number. If vector is shorter than level count, last compression type is used for higher levels. If vector is empty, global compression setting is used.

##### Example
```rust
use surrealkv::{Options, CompressionType};

let opts = Options::new()
    .with_compression_per_level(vec![
        CompressionType::None,        // L0: no compression
        CompressionType::SnappyCompression, // L1+: Snappy compression
    ]);
```

#### pub fn with_l0_no_compression(self) -> Self
Convenience method: no compression on L0, Snappy compression on other levels. Equivalent to `with_compression_per_level(vec![CompressionType::None, CompressionType::SnappyCompression])`.

##### Example
```rust
use surrealkv::Options;

let opts = Options::new().with_l0_no_compression();
```

#### pub fn with_path(self, value: PathBuf) -> Self

#### pub const fn with_level_count(self, value: u8) -> Self

#### pub const fn with_max_memtable_size(self, value: usize) -> Self
```

--------------------------------

### Get Active Log Number

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/wal/manager.rs.html

Retrieves the current active log number for the WAL.

```APIDOC
## GET /wal/log_number

### Description
Returns the current log number of the active Write-Ahead Log file.

### Method
GET

### Endpoint
/wal/log_number

### Response
#### Success Response (200)
- **log_number** (u64) - The current active log number.

#### Response Example
```json
{
  "log_number": 12345
}
```
```

--------------------------------

### Get WAL Directory Path

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/wal/manager.rs.html

Retrieves the directory path where WAL files are stored.

```APIDOC
## GET /wal/dir

### Description
Returns the absolute path to the directory where the Write-Ahead Log files are stored.

### Method
GET

### Endpoint
/wal/dir

### Response
#### Success Response (200)
- **path** (string) - The directory path for WAL files.

#### Response Example
```json
{
  "path": "/var/lib/surrealkv/wal"
}
```
```

--------------------------------

### VLog File Initialization

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/vlog.rs.html

Initializes VLog files by checking for existing files, setting the next file ID, and preparing the active writer.

```APIDOC
## Initialize VLog Files

### Description
This function initializes the VLog files. It checks for existing files, determines the next available file ID, and sets up the active writer for new entries. If existing files are found, it uses the highest existing file ID to set the `next_file_id` and prepares the last file as the active writer.

### Method
Internal function (not directly exposed as an API endpoint)

### Endpoint
N/A

### Parameters
None

### Request Body
None

### Request Example
None

### Response
#### Success Response (Ok(()))
- Indicates successful initialization of VLog files.

#### Response Example
None

### Error Handling
- Returns `Err(Error::Io)` if there's an issue opening VLog files.
- Returns `Err(Error::Other)` for other I/O related errors during initialization.
```

--------------------------------

### Prefill File Handles

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/vlog.rs.html

Scans the VLog directory to populate the file handle cache and register existing files.

```rust
fn prefill_file_handles(&self) -> Result<()> {
		let entries = match std::fs::read_dir(&self.path) {
			Ok(entries) => entries,
			Err(_) => return Ok(()), // Directory doesn't exist yet
		};

		let mut max_file_id: Option<u32> = None;
		let mut max_file_path: Option<PathBuf> = None;
		let mut file_handles = self.file_handles.write();
		let mut files_map = self.files_map.write();

		for entry in entries {
			let entry = entry?;
			let file_name = entry.file_name();
			let file_name_str = file_name.to_string_lossy();

			// Look for VLog files
			if let Some(file_id) = self.opts.extract_vlog_file_id(&file_name_str) {
				let file_path = entry.path();
				let file_size = entry.metadata()?.len();

				// Track the file with maximum ID for active writer setup
				if max_file_id.is_none_or(|current_max| file_id >= current_max) {
					max_file_id = Some(file_id);
					max_file_path = Some(file_path.clone());
				}

				// Pre-open the file handle and validate header
				match File::open(&file_path) {
					Ok(file) => {
						// Validate file header for existing files
						if file_size > 0 {
							let mut header_data = vec![0u8; VLogFileHeader::SIZE];
							vfs::File::read_at(&file, 0, &mut header_data)?;
							let header = VLogFileHeader::decode(&header_data)?;
							header.validate(file_id)?;
						}

						let handle = Arc::new(file);
						file_handles.insert(file_id, handle);

						// Also register in files_map
						files_map.insert(
							file_id,
							Arc::new(VLogFile::new(file_id, file_path, file_size)),
						);
					}

```

--------------------------------

### Get IOError Kind

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/wal/mod.rs.html

Accessor method to retrieve the io::ErrorKind from an IOError instance.

```rust
pub(crate) fn kind(&self) -> io::ErrorKind {
		self.kind
	}
```

--------------------------------

### Example: Versioning with Retention

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/iter.rs.html

Illustrates data retention based on versioning and a time-based retention policy. Older versions are kept if they fall within the retention period, otherwise they are dropped.

```text
Input:
  ("key1", seq=100, PUT, "v3", timestamp=now)
  ("key1", seq=50,  PUT, "v2", timestamp=now-30min)
  ("key1", seq=20,  PUT, "v1", timestamp=now-2hours)

With versioning=true, retention=1hour, no snapshots:
  - seq=100: age=0, KEEP (latest)
  - seq=50:  age=30min < 1hour, KEEP (within retention)
  - seq=20:  age=2hours > 1hour, DROP (expired)
  
Output: [seq=100, seq=50]
delete_list: [seq=20]
```

--------------------------------

### Get sequence number from InternalKeyRef

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/lib.rs.html

Retrieves the sequence number from the InternalKeyRef by decoding its trailer.

```rust
pub fn seq_num(&self) -> u64 {
		InternalKey::seq_num_from_encoded(self.encoded)
	}
```

--------------------------------

### Open WAL File with Options

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/wal/manager.rs.html

Opens a Write-Ahead Log (WAL) file with specified read, write, create, and append options. Includes platform-specific file mode settings for Unix.

```rust
fn open_wal_file(file_path: &Path, opts: &Options) -> Result<File> {
    let mut open_options = OpenOptions::new();
    open_options.read(true).write(true).create(true).append(true);

    #[cfg(unix)]
    {
        use std::os::unix::fs::OpenOptionsExt;
        if let Some(file_mode) = opts.file_mode {
            open_options.mode(file_mode);
        }
    }

    Ok(open_options.open(file_path)?)
}
```

--------------------------------

### Get sequence number from InternalKey

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/lib.rs.html

Extracts the sequence number from an InternalKey by converting its trailer.

```rust
pub(crate) fn seq_num(&self) -> u64 {
		trailer_to_seq_num(self.trailer)
	}
```

--------------------------------

### Skiplist::new

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/memtable/skiplist.rs.html

Initializes a new Skiplist instance with a provided arena and comparison function, setting up head and tail sentinels.

```APIDOC
## Skiplist::new

### Description
Constructs a new Skiplist, allocating head and tail sentinel nodes and linking them across all levels.

### Parameters
- **arena** (Arc<Arena>) - Required - The memory arena for node allocation.
- **cmp** (Compare) - Required - The comparison logic for keys.
```

--------------------------------

### Create Memtable Iterator

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/memtable/mod.rs.html

Creates a new iterator for the memtable, starting from the first element.

```rust
pub(crate) fn iter(&self) -> MemTableIterator<'_> {
		self.range(None, None)
	}
```

--------------------------------

### Prepare Directory Permissions

Source: https://docs.rs/surrealkv/0.21.0/src/surrealkv/wal/manager.rs.html

Ensures a directory exists and sets its permissions. This function is primarily for Unix-based systems, as Windows handles directory permissions differently.

```rust
fn prepare_directory(dir: &Path, opts: &Options) -> Result<()> {
    // Directory should already be created by Tree::new()
    // Set permissions on Unix only; Windows NTFS uses ACLs and
    // set_permissions on directories can fail with ERROR_ACCESS_DENIED.
    #[cfg(unix)]
    if let Ok(metadata) = fs::metadata(dir) {
        let mut permissions = metadata.permissions();
        use std::os::unix::fs::PermissionsExt;
        permissions.set_mode(opts.dir_mode.unwrap_or(0o750));
        fs::set_permissions(dir, permissions)?;
    }

    Ok(())
}
```

--------------------------------

### unwrap_err() - Get Err value

Source: https://docs.rs/surrealkv/0.21.0/surrealkv/type.Result.html

Returns the contained `Err` value. Panics if the `Result` is an `Ok`.

```APIDOC
## GET /api/result/unwrap_err

### Description
Returns the contained `Err` value. Panics if the `Result` is an `Ok`.

### Method
GET

### Endpoint
/api/result/unwrap_err

### Parameters
#### Query Parameters
- **value** (Result<T, E>) - Required - The Result to unwrap.

### Request Example
```json
{
  "value": "Err(\"emergency failure\")"
}
```

### Response
#### Success Response (200)
- **error** (E) - The error value contained within the `Err` variant.

#### Response Example
```json
{
  "error": "emergency failure"
}
```

#### Error Response (Panic)
Panics if the input is an `Ok` variant.
```
Tokens:
Raw