hnsw_rs

https://docs.rs/hnsw_rs/latest/
Admin
hnsw_rs is a Rust implementation of approximate nearest neighbor search using Hierarchical Navigable...
Tokens:529,001
Snippets:3,473
Trust Score:9.7
Update:2 days ago
Show doc for...
Tokens:
Raw
### Box::from_raw Usage Examples

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnsw/type.PointDistance.html?search=

Examples demonstrating how to reconstruct a Box from a raw pointer, either from a previous conversion or manual allocation.

```rust
let x = Box::new(5);
let ptr = Box::into_raw(x);
let x = unsafe { Box::from_raw(ptr) };
```

```rust
use std::alloc::{alloc, Layout};

unsafe {
    let ptr = alloc(Layout::new::<i32>()) as *mut i32;
    // In general .write is required to avoid attempting to destruct
    // the (uninitialized) previous contents of `ptr`, though for this
    // simple example `*ptr = 5` would have worked as well.
    ptr.write(5);
    let x = Box::from_raw(ptr);
}
```

--------------------------------

### Box::into_raw Usage Examples

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnsw/type.PointDistance.html?search=

Examples demonstrating how to consume a Box into a raw pointer and perform cleanup.

```rust
let x = Box::new(String::from("Hello"));
let ptr = Box::into_raw(x);
let x = unsafe { Box::from_raw(ptr) };
```

```rust
use std::alloc::{dealloc, Layout};
use std::ptr;

let x = Box::new(String::from("Hello"));
let ptr = Box::into_raw(x);
unsafe {
    ptr::drop_in_place(ptr);
    dealloc(ptr as *mut u8, Layout::new::<String>());
}
```

```rust
let x = Box::new(String::from("Hello"));
let ptr = Box::into_raw(x);
unsafe {
    drop(Box::from_raw(ptr));
}
```

--------------------------------

### Box::from_non_null Usage Examples

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnsw/type.PointDistance.html?search=

Examples demonstrating how to reconstruct a Box from a NonNull pointer.

```rust
let x = Box::new(5);
let non_null = Box::into_non_null(x);
let x = unsafe { Box::from_non_null(non_null) };
```

```rust
use std::alloc::{alloc, Layout};
use std::ptr::NonNull;

unsafe {
    let non_null = NonNull::new(alloc(Layout::new::<i32>()).cast::<i32>())
        .expect("alloc should have successfully allocated memory");
    // In general .write is required to avoid attempting to destruct
    // the (uninitialized) previous contents of `non_null`.
    non_null.write(5);
    let x = Box::from_non_null(non_null);
}
```

--------------------------------

### Box::take Example

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnsw/type.PointDistance.html

Demonstrates taking a value out of a box while retaining the allocation for reuse.

```rust
let c = Box::new(5);

// take the value out of the box
let (value, uninit) = Box::take(c);
assert_eq!(value, 5);

// reuse the box for a second value
let c = Box::write(uninit, 6);
assert_eq!(*c, 6);
```

--------------------------------

### Box::into_raw Drop Example

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnsw/type.PointDistance.html

Manual cleanup using drop and Box::from_raw.

```rust
let x = Box::new(String::from("Hello"));
let ptr = Box::into_raw(x);
unsafe {
    drop(Box::from_raw(ptr));
}
```

--------------------------------

### Box::pin_in Example

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnsw/type.PointDistance.html

Demonstrates pinning a value in memory using a specific allocator.

```rust
#![feature(allocator_ext)]
use std::alloc::System;

let x = Box::pin_in(1, System);
```

--------------------------------

### Box::new_zeroed_in Example

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnsw/type.PointDistance.html

Demonstrates constructing a box with memory filled with zero bytes.

```rust
#![feature(allocator_ext)]

use std::alloc::System;

let zero = Box::<u32, _>::new_zeroed_in(System);
let zero = unsafe { zero.assume_init() };

assert_eq!(*zero, 0)
```

--------------------------------

### DumpInit::new

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnswio/struct.DumpInit.html

Creates a new DumpInit instance with the specified directory, default basename, and overwrite flag.

```APIDOC
## DumpInit::new

### Description
Creates a new DumpInit instance with the specified directory, default basename, and overwrite flag.

### Method
new

### Signature
pub fn new(dir: &Path, basename_default: &str, overwrite: bool) -> Self

### Parameters
- **dir** (&Path) - Required - The directory where the dump will be stored.
- **basename_default** (&str) - Required - The default basename for the dump file.
- **overwrite** (bool) - Required - Whether to overwrite an existing dump file.

### Returns
Self - A new DumpInit instance.
```

--------------------------------

### DumpInit::new

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnswio/struct.DumpInit.html?search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E

Creates a new DumpInit instance with the specified directory, default basename, and overwrite flag.

```APIDOC
## DumpInit::new

### Description
Creates a new DumpInit instance with the given directory, default basename, and overwrite flag.

### Method
new

### Signature
pub fn new(dir: &Path, basename_default: &str, overwrite: bool) -> Self

### Parameters
- **dir** (&Path) - Required - The directory where the dump will be stored.
- **basename_default** (&str) - Required - The default basename for the dump file.
- **overwrite** (bool) - Required - Whether to overwrite an existing dump file.

### Returns
- **Self** - A new DumpInit instance.
```

--------------------------------

### DumpInit::new

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnswio/struct.DumpInit.html?search=u32+-%3E+bool

Creates a new DumpInit instance with the specified directory, default basename, and overwrite flag.

```APIDOC
## DumpInit::new

### Description
Creates a new DumpInit instance.

### Method
new

### Signature
pub fn new(dir: &Path, basename_default: &str, overwrite: bool) -> Self

### Parameters
- **dir** (&Path) - Required - The directory path for the dump.
- **basename_default** (&str) - Required - The default basename for the dump.
- **overwrite** (bool) - Required - Whether to overwrite existing dumps.

### Returns
Self - A new DumpInit instance.
```

--------------------------------

### Manually create Box from scratch using System allocator

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnsw/type.PointDistance.html

Shows how to manually allocate memory and construct a Box using the system allocator.

```rust
#![feature(slice_ptr_get)]

use std::alloc::{Allocator, Layout, System};

unsafe {
    let ptr = System.allocate(Layout::new::<i32>())?.as_mut_ptr() as *mut i32;
    // In general .write is required to avoid attempting to destruct
    // the (uninitialized) previous contents of `ptr`, though for this
    // simple example `*ptr = 5` would have worked as well.
    ptr.write(5);
    let x = Box::from_raw_in(ptr, System);
}
```

--------------------------------

### try_get_i8

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnsw/type.PointDistance.html?search=

Gets a signed 8 bit integer from self.

```APIDOC
## try_get_i8

### Description
Gets a signed 8 bit integer from `self`.

### Method
fn try_get_i8(&mut self) -> Result<i8, TryGetError>

### Parameters
- `self` (mutable reference) - Required - The byte buffer to read from.

### Returns
Result<i8, TryGetError> - The signed 8-bit integer read from the buffer, or a TryGetError if there were not enough bytes.
```

--------------------------------

### try_get_u8

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnsw/type.PointDistance.html?search=

Gets an unsigned 8 bit integer from self.

```APIDOC
## try_get_u8

### Description
Gets an unsigned 8 bit integer from `self`.

### Method
fn try_get_u8(&mut self) -> Result<u8, TryGetError>

### Parameters
- `self` (mutable reference) - Required - The byte buffer to read from.

### Returns
Result<u8, TryGetError> - The unsigned 8-bit integer read from the buffer, or a TryGetError if there were not enough bytes.
```

--------------------------------

### BufReadExt::try_get_i8

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnsw/type.PointDistance.html?search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E

Gets a signed 8 bit integer from self.

```APIDOC
## BufReadExt::try_get_i8

### Description
Gets a signed 8 bit integer from `self`.

### Method
fn try_get_i8(&mut self) -> Result<i8, TryGetError>

### Parameters
None

### Returns
Result<i8, TryGetError> - The signed 8-bit integer read from the buffer, or a TryGetError if there were not enough bytes.
```

--------------------------------

### DumpInit::new

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnswio/struct.DumpInit.html?search=std%3A%3Avec

Creates a new DumpInit instance with the specified directory path, default basename, and overwrite flag. This is the constructor for the dump initialization object.

```APIDOC
## DumpInit::new

### Description
Creates a new DumpInit instance with the specified directory path, default basename, and overwrite flag. This is the constructor for the dump initialization object.

### Method
new

### Signature
pub fn new(dir: &Path, basename_default: &str, overwrite: bool) -> Self

### Parameters
- **dir** (&Path) - Required - The directory path where the dump will be stored.
- **basename_default** (&str) - Required - The default basename for the dump file.
- **overwrite** (bool) - Required - Whether to overwrite an existing dump file.

### Returns
- **Self** - A new DumpInit instance.

```

--------------------------------

### Box::from_raw Manual Allocation Example

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnsw/type.PointDistance.html

Manually create a Box from scratch by using the global allocator.

```rust
use std::alloc::{alloc, Layout};

unsafe {
    let ptr = alloc(Layout::new::<i32>()) as *mut i32;
    // In general .write is required to avoid attempting to destruct
    // the (uninitialized) previous contents of `ptr`, though for this
    // simple example `*ptr = 5` would have worked as well.
    ptr.write(5);
    let x = Box::from_raw(ptr);
}
```

--------------------------------

### BufReadExt::try_get_u8

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnsw/type.PointDistance.html?search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E

Gets an unsigned 8 bit integer from self.

```APIDOC
## BufReadExt::try_get_u8

### Description
Gets an unsigned 8 bit integer from `self`.

### Method
fn try_get_u8(&mut self) -> Result<u8, TryGetError>

### Parameters
None

### Returns
Result<u8, TryGetError> - The unsigned 8-bit integer read from the buffer, or a TryGetError if there were not enough bytes.
```

--------------------------------

### Box::into_inner Example

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnsw/type.PointDistance.html

Demonstrates consuming a box to return the wrapped value.

```rust
#![feature(box_into_inner)]

let c = Box::new(5);

assert_eq!(Box::into_inner(c), 5);
```

--------------------------------

### Get layer iterator

Source: https://docs.rs/hnsw_rs/latest/src/hnsw_rs/hnsw.rs.html

Returns an iterator over the points stored in a specific layer.

```rust
pub fn get_layer_iterator<'a>(&'a self, layer: usize) -> IterPointLayer<'a, 'b, T> {
        IterPointLayer::new(self, layer)
    }
```

--------------------------------

### Dump HNSW to file

Source: https://docs.rs/hnsw_rs/latest/src/hnsw_rs/hnswio.rs.html

Demonstrates initializing a dump with DumpInit and performing a file dump of the HNSW structure.

```rust
let dump_init = DumpInit::new(directory.path(), fname, false);
info!("will use basename : {}", dump_init.get_basename());
let res = hnsw.file_dump(directory.path(), dump_init.get_basename());
if res.is_err() {
    error!("hnsw.file_dump failed");
    std::panic!("hnsw.file_dump failed");
}
```

--------------------------------

### get_int_ne

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnsw/type.PointDistance.html?search=u32+-%3E+bool

Gets a signed n-byte integer from the buffer in native-endian byte order.

```APIDOC
## get_int_ne

### Description
Gets a signed n-byte integer from `self` in native-endian byte order.

### Method
fn get_int_ne(&mut self, nbytes: usize) -> i64

### Parameters
- `self` (mutable reference) - The buffer to read from.
- `nbytes` (usize) - The number of bytes to read.

### Returns
- `i64` - The signed n-byte integer read from the buffer.
```

--------------------------------

### DumpInit::new

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnswio/struct.DumpInit.html?search=

Creates a new DumpInit instance with the specified directory path, default basename, and overwrite flag. This is the constructor for the dump initialization struct.

```APIDOC
## DumpInit::new

### Description
Creates a new DumpInit instance with the specified directory path, default basename, and overwrite flag.

### Method
Constructor

### Signature
pub fn new(dir: &Path, basename_default: &str, overwrite: bool) -> Self

### Parameters
- **dir** (&Path) - Required - The directory path where the dump will be stored.
- **basename_default** (&str) - Required - The default basename for the dump file.
- **overwrite** (bool) - Required - Whether to overwrite an existing dump file.

### Returns
- **Self** - A new DumpInit instance.
```

--------------------------------

### get_int_le

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnsw/type.PointDistance.html?search=u32+-%3E+bool

Gets a signed n-byte integer from the buffer in little-endian byte order.

```APIDOC
## get_int_le

### Description
Gets a signed n-byte integer from `self` in little-endian byte order.

### Method
fn get_int_le(&mut self, nbytes: usize) -> i64

### Parameters
- `self` (mutable reference) - The buffer to read from.
- `nbytes` (usize) - The number of bytes to read.

### Returns
- `i64` - The signed n-byte integer read from the buffer.
```

--------------------------------

### Box::try_new_uninit_in Example

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnsw/type.PointDistance.html

Demonstrates constructing a box with uninitialized contents using a specific allocator and deferred initialization.

```rust
#![feature(allocator_ext)]

use std::alloc::System;

let mut five = Box::<u32, _>::try_new_uninit_in(System)?;
// Deferred initialization:
five.write(5);
let five = unsafe { five.assume_init() };

assert_eq!(*five, 5);
```

--------------------------------

### get_int

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnsw/type.PointDistance.html?search=u32+-%3E+bool

Gets a signed n-byte integer from the buffer in big-endian byte order.

```APIDOC
## get_int

### Description
Gets a signed n-byte integer from `self` in big-endian byte order.

### Method
fn get_int(&mut self, nbytes: usize) -> i64

### Parameters
- `self` (mutable reference) - The buffer to read from.
- `nbytes` (usize) - The number of bytes to read.

### Returns
- `i64` - The signed n-byte integer read from the buffer.
```

--------------------------------

### Box::from_non_null Manual Allocation Example

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnsw/type.PointDistance.html

Manually create a Box from scratch by using the global allocator with NonNull.

```rust
use std::alloc::{alloc, Layout};
use std::ptr::NonNull;

unsafe {
    let non_null = NonNull::new(alloc(Layout::new::<i32>()).cast::<i32>())
        .expect("alloc should have successfully allocated memory");
    // In general .write is required to avoid attempting to destruct
    // the (uninitialized) previous contents of `non_null`.
    non_null.write(5);
    let x = Box::from_non_null(non_null);
}
```

--------------------------------

### get_uint_ne

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnsw/type.PointDistance.html?search=u32+-%3E+bool

Gets an unsigned n-byte integer from the buffer in native-endian byte order.

```APIDOC
## get_uint_ne

### Description
Gets an unsigned n-byte integer from `self` in native-endian byte order.

### Method
fn get_uint_ne(&mut self, nbytes: usize) -> u64

### Parameters
- `self` (mutable reference) - The buffer to read from.
- `nbytes` (usize) - The number of bytes to read.

### Returns
- `u64` - The unsigned n-byte integer read from the buffer.
```

--------------------------------

### new

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnswio/struct.HnswIo.html

Creates a new HnswIo instance with the given directory and basename, using default ReloadOptions.

```APIDOC
## new

### Description
Creates a new HnswIo instance with the given directory and basename, using default ReloadOptions.

### Method
new

### Signature
pub fn new(directory: &Path, basename: &str) -> Self

### Parameters
- **directory** (&Path) - Required - Directory containing the dumped files.
- **basename** (&str) - Required - Used to build $basename.hnsw.data and $basename.hnsw.graph.

### Returns
Self - A new HnswIo instance.
```

--------------------------------

### get_uint_le

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnsw/type.PointDistance.html?search=u32+-%3E+bool

Gets an unsigned n-byte integer from the buffer in little-endian byte order.

```APIDOC
## get_uint_le

### Description
Gets an unsigned n-byte integer from `self` in little-endian byte order.

### Method
fn get_uint_le(&mut self, nbytes: usize) -> u64

### Parameters
- `self` (mutable reference) - The buffer to read from.
- `nbytes` (usize) - The number of bytes to read.

### Returns
- `u64` - The unsigned n-byte integer read from the buffer.
```

--------------------------------

### get_uint

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnsw/type.PointDistance.html?search=u32+-%3E+bool

Gets an unsigned n-byte integer from the buffer in big-endian byte order.

```APIDOC
## get_uint

### Description
Gets an unsigned n-byte integer from `self` in big-endian byte order.

### Method
fn get_uint(&mut self, nbytes: usize) -> u64

### Parameters
- `self` (mutable reference) - The buffer to read from.
- `nbytes` (usize) - The number of bytes to read.

### Returns
- `u64` - The unsigned n-byte integer read from the buffer.
```

--------------------------------

### Load Entry Point and Construct PointIndexation

Source: https://docs.rs/hnsw_rs/latest/src/hnsw_rs/hnswio.rs.html

This snippet reads the entry point's origin ID, layer, and rank from the graph file, then constructs a PointIndexation structure with the loaded points and entry point. It includes logging for the loaded entry point and a caveat comment about incrementing the graph.

```Rust
        //
        // get id of entry_point
        // load entry point
        info!(
            "end of layer loading, allocating PointIndexation, nb points loaded {:?}",
            nb_points_loaded
        );
        //
        let mut it_slice = [0u8; std::mem::size_of::<DataId>()];
        graph_in.read_exact(&mut it_slice)?;
        let origin_id = DataId::from_ne_bytes(it_slice);
        //
        let mut it_slice = [0u8; ::std::mem::size_of::<u8>()];
        graph_in.read_exact(&mut it_slice)?;
        let layer = u8::from_ne_bytes(it_slice);
        //
        let mut it_slice = [0u8; std::mem::size_of::<i32>()];
        graph_in.read_exact(&mut it_slice)?;
        let rank_in_l = i32::from_ne_bytes(it_slice);
        //
        info!(
            "found entry point, origin_id {:?} , layer {:?}, rank in layer {:?} ",
            origin_id, layer, rank_in_l
        );
        let entry_point = Arc::clone(&points_by_layer[layer as usize][rank_in_l as usize]);
        info!(
            " loaded entry point, origin_id {:} p_id {:?}",
            entry_point.get_origin_id(),
            entry_point.get_point_id()
        );
        //
        let point_indexation = PointIndexation {
            max_nb_connection: descr.max_nb_connection as usize,
            max_layer: NB_LAYER_MAX as usize,
            points_by_layer: Arc::new(RwLock::new(points_by_layer)),
            layer_g: LayerGenerator::new_with_scale(
                descr.max_nb_connection as usize,
                descr.level_scale,
                NB_LAYER_MAX as usize,
            ),
            nb_point: Arc::new(RwLock::new(nb_points_loaded)), // CAVEAT , we should increase , the whole thing is to be able to increment graph ?
            entry_point: Arc::new(RwLock::new(Some(entry_point))),
        };
        //
        debug!("Exiting load_pointIndexation");
        Ok(point_indexation)
```

--------------------------------

### get_i128

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnsw/type.PointDistance.html?search=u32+-%3E+bool

Gets a signed 128-bit integer from the buffer in big-endian byte order.

```APIDOC
## get_i128

### Description
Gets a signed 128 bit integer from `self` in big-endian byte order.

### Method
fn get_i128(&mut self) -> i128

### Parameters
- `self` (mutable reference) - The buffer to read from.

### Returns
- `i128` - The signed 128-bit integer read from the buffer.
```

--------------------------------

### DumpInit::new Initialization

Source: https://docs.rs/hnsw_rs/latest/src/hnsw_rs/hnswio.rs.html

Initializes a new DumpInit instance, handling unique filename generation if overwrite is disabled and opening the necessary files.

```rust
pub fn new(dir: &Path, basename_default: &str, overwrite: bool) -> Self {
        // if we cannot overwrite data files (in case of mmap in particular)
        // we will ensure we have a unique basename
        let basename = match overwrite {
            true => basename_default.to_string(),
            false => {
                // we check
                let mut dataname = basename_default.to_string();
                dataname.push_str(".hnsw.data");
                let mut datapath = PathBuf::from(dir);
                datapath.push(dataname);
                let exist_res = std::fs::metadata(datapath.as_os_str());
                if exist_res.is_ok() {
                    loop {
                        let mut unique_basename;
                        let mut dataname: String;
                        let id: usize = rand::random_range(0..10000);
                        let strid: String = id.to_string();
                        unique_basename = basename_default.to_string();
                        unique_basename.push('-');
                        unique_basename.push_str(&strid);
                        dataname = unique_basename.clone();
                        dataname.push_str(".hnsw.data");
                        let mut datapath = PathBuf::from(dir);
                        datapath.push(dataname);
                        let exist_res = std::fs::metadata(datapath.as_os_str());
                        if exist_res.is_err() {
                            break unique_basename;
                        }
                    }
                } else {
                    basename_default.to_string()
                }
            }
        };
        //
        debug!(
            "DumpInit : dumping in dir {:#?} with (unique) basename : {}",
            dir, basename
        );
        //
        let mut graphname = basename.clone();
        graphname.push_str(".hnsw.graph");
        let mut graphpath = PathBuf::from(dir);
        graphpath.push(graphname);
        let graphfileres = OpenOptions::new()
            .create(true)
            .truncate(true)
            .write(true)
            .open(&graphpath);
        if graphfileres.is_err() {
            println!(
                "DumpInit::new : could not open file {:?}",
                graphpath.as_os_str()
            );
            std::panic::panic_any("HnswIo::init : could not open file".to_string());
        }
        let graphfile = graphfileres.unwrap();
        //  same thing for data file
        let mut dataname = basename.clone();
        dataname.push_str(".hnsw.data");
        let mut datapath = PathBuf::from(dir);
        datapath.push(dataname);
        let datafileres = OpenOptions::new()
            .create(true)
            .truncate(true)
            .write(true)
            .open(&datapath);
        if datafileres.is_err() {
            println!(
                "DumpInit::init : could not open file {:?}",
                datapath.as_os_str()
            );
            std::panic::panic_any("HnswIo::init : could not open file".to_string());
        }
        let datafile = datafileres.unwrap();
        //
        let graph_out = BufWriter::new(graphfile);
        let data_out = BufWriter::new(datafile);
        //
        DumpInit {
            basename,
            data_out,
            graph_out,
        }
    }
```

--------------------------------

### get_int_ne

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnsw/type.PointDistance.html?search=

Gets a signed n-byte integer from self in native-endian byte order.

```APIDOC
## get_int_ne

### Description
Gets a signed n-byte integer from `self` in native-endian byte order.

### Method
fn get_int_ne(&mut self, nbytes: usize) -> i64

### Parameters
- `self` (mutable reference) - Required - The byte buffer to read from.
- `nbytes` (usize) - Required - The number of bytes to read.

### Returns
i64 - The signed n-byte integer read from the buffer.
```

--------------------------------

### DumpInit::new implementation

Source: https://docs.rs/hnsw_rs/latest/src/hnsw_rs/hnswio.rs.html?search=

Creates a new DumpInit, generating a unique basename if overwrite is false and the default data file exists. Opens graph and data files with create, truncate, and write options, panicking if file opening fails.

```rust
impl DumpInit {
    // This structure will check existence of dumps of same name and generate a unique filename if necessary according to overwrite flag
    pub fn new(dir: &Path, basename_default: &str, overwrite: bool) -> Self {
        // if we cannot overwrite data files (in case of mmap in particular)
        // we will ensure we have a unique basename
        let basename = match overwrite {
            true => basename_default.to_string(),
            false => {
                // we check
                let mut dataname = basename_default.to_string();
                dataname.push_str(".hnsw.data");
                let mut datapath = PathBuf::from(dir);
                datapath.push(dataname);
                let exist_res = std::fs::metadata(datapath.as_os_str());
                if exist_res.is_ok() {
                    loop {
                        let mut unique_basename;
                        let mut dataname: String;
                        let id: usize = rand::random_range(0..10000);
                        let strid: String = id.to_string();
                        unique_basename = basename_default.to_string();
                        unique_basename.push('-');
                        unique_basename.push_str(&strid);
                        dataname = unique_basename.clone();
                        dataname.push_str(".hnsw.data");
                        let mut datapath = PathBuf::from(dir);
                        datapath.push(dataname);
                        let exist_res = std::fs::metadata(datapath.as_os_str());
                        if exist_res.is_err() {
                            break unique_basename;
                        }
                    }
                } else {
                    basename_default.to_string()
                }
            }
        };
        //
        debug!(
            "DumpInit : dumping in dir {:#?} with (unique) basename : {}",
            dir, basename
        );
        //
        let mut graphname = basename.clone();
        graphname.push_str(".hnsw.graph");
        let mut graphpath = PathBuf::from(dir);
        graphpath.push(graphname);
        let graphfileres = OpenOptions::new()
            .create(true)
            .truncate(true)
            .write(true)
            .open(&graphpath);
        if graphfileres.is_err() {
            println!(
                "DumpInit::new : could not open file {:?}",
                graphpath.as_os_str()
            );
            std::panic::panic_any("HnswIo::init : could not open file".to_string());
        }
        let graphfile = graphfileres.unwrap();
        //  same thing for data file
        let mut dataname = basename.clone();
        dataname.push_str(".hnsw.data");
        let mut datapath = PathBuf::from(dir);
        datapath.push(dataname);
        let datafileres = OpenOptions::new()
            .create(true)
            .truncate(true)
            .write(true)
            .open(&datapath);
        if datafileres.is_err() {
            println!(
                "DumpInit::init : could not open file {:?}",
                datapath.as_os_str()
            );
            std::panic::panic_any("HnswIo::init : could not open file".to_string());
        }
        let datafile = datafileres.unwrap();
        //
        let graph_out = BufWriter::new(graphfile);
        let data_out = BufWriter::new(datafile);
        //
        DumpInit {
            basename,
            data_out,
            graph_out,
        }
    }
```

--------------------------------

### get_int_le

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnsw/type.PointDistance.html?search=

Gets a signed n-byte integer from self in little-endian byte order.

```APIDOC
## get_int_le

### Description
Gets a signed n-byte integer from `self` in little-endian byte order.

### Method
fn get_int_le(&mut self, nbytes: usize) -> i64

### Parameters
- `self` (mutable reference) - Required - The byte buffer to read from.
- `nbytes` (usize) - Required - The number of bytes to read.

### Returns
i64 - The signed n-byte integer read from the buffer.
```

--------------------------------

### get_int

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnsw/type.PointDistance.html?search=

Gets a signed n-byte integer from self in big-endian byte order.

```APIDOC
## get_int

### Description
Gets a signed n-byte integer from `self` in big-endian byte order.

### Method
fn get_int(&mut self, nbytes: usize) -> i64

### Parameters
- `self` (mutable reference) - Required - The byte buffer to read from.
- `nbytes` (usize) - Required - The number of bytes to read.

### Returns
i64 - The signed n-byte integer read from the buffer.
```

--------------------------------

### HnswIo::new

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnswio/struct.HnswIo.html

Creates a new HnswIo instance with a directory path and a base name for the dump files.

```APIDOC
## HnswIo::new

### Description
Creates a new HnswIo instance for reloading a previous dump. The directory and base name are used to locate the dump files.

### Method
Constructor

### Signature
pub fn new(directory: &Path, base_name: &str) -> HnswIo

### Parameters
- **directory** (&Path) - Required - The directory where the dump files are located.
- **base_name** (&str) - Required - The base name of the dump files.

### Example
```rust
let directory = Path::new(".");
let mut reloader = HnswIo::new(directory, "mmapreloadtest");
```
```

--------------------------------

### Dump and reload HNSW with DistL1

Source: https://docs.rs/hnsw_rs/latest/src/hnsw_rs/hnswio.rs.html

Demonstrates creating an HNSW index, dumping it to a temporary directory, and reloading it using HnswIo.

```rust
#[test]
fn test_dump_reload_1() {
    println!("\n\n test_dump_reload_1");
    log_init_test();
    // generate a random test
    let mut rng = rand::rng();
    let unif = Uniform::<f32>::new(0., 1.).unwrap();
    // 1000 vectors of size 10 f32
    let nbcolumn = 1000;
    let nbrow = 10;
    let mut xsi;
    let mut data = Vec::with_capacity(nbcolumn);
    for j in 0..nbcolumn {
        data.push(Vec::with_capacity(nbrow));
        for _ in 0..nbrow {
            xsi = unif.sample(&mut rng);
            data[j].push(xsi);
        }
    }
    // define hnsw
    let ef_construct = 25;
    let nb_connection = 10;
    let hnsw = Hnsw::<f32, dist::DistL1>::new(
        nb_connection,
        nbcolumn,
        16,
        ef_construct,
        dist::DistL1 {},
    );
    for (i, d) in data.iter().enumerate() {
        hnsw.insert((d, i));
    }
    // some loggin info
    hnsw.dump_layer_info();
    // dump in a file.  Must take care of name as tests runs in // !!!
    let fname = "dumpreloadtest1";
    let directory = tempfile::tempdir().unwrap();
    let _res = hnsw.file_dump(directory.path(), fname);
    //
    // reload
    debug!("\n\n test_dump_reload_1 hnsw reload");
    // we will need a procedural macro to get from distance name to its instanciation.
    // from now on we test with DistL1
    let mut reloader = HnswIo::new(directory.path(), fname);
    let hnsw_loaded: Hnsw<f32, DistL1> = reloader.load_hnsw::<f32, DistL1>().unwrap();
    // test equality
    check_graph_equality(&hnsw_loaded, &hnsw);
} // end of test_dump_reload
```

--------------------------------

### get_uint_ne

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnsw/type.PointDistance.html?search=

Gets an unsigned n-byte integer from self in native-endian byte order.

```APIDOC
## get_uint_ne

### Description
Gets an unsigned n-byte integer from `self` in native-endian byte order.

### Method
fn get_uint_ne(&mut self, nbytes: usize) -> u64

### Parameters
- `self` (mutable reference) - Required - The byte buffer to read from.
- `nbytes` (usize) - Required - The number of bytes to read.

### Returns
u64 - The unsigned n-byte integer read from the buffer.
```

--------------------------------

### get_uint_le

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnsw/type.PointDistance.html?search=

Gets an unsigned n-byte integer from self in little-endian byte order.

```APIDOC
## get_uint_le

### Description
Gets an unsigned n-byte integer from `self` in little-endian byte order.

### Method
fn get_uint_le(&mut self, nbytes: usize) -> u64

### Parameters
- `self` (mutable reference) - Required - The byte buffer to read from.
- `nbytes` (usize) - Required - The number of bytes to read.

### Returns
u64 - The unsigned n-byte integer read from the buffer.
```

--------------------------------

### Library Module Initialization and Logging

Source: https://docs.rs/hnsw_rs/latest/src/hnsw_rs/lib.rs.html

The main library file configures feature flags, defines module structure, and initializes the logging facility using lazy_static.

```rust
1#![cfg_attr(feature = "stdsimd", feature(portable_simd))]
2//
3// for logging (debug mostly, switched at compile time in cargo.toml)
4use env_logger::Builder;
5
6use lazy_static::lazy_static;
7
8pub mod api;
9pub mod datamap;
10pub mod filter;
11pub mod flatten;
12pub mod hnsw;
13pub mod hnswio;
14pub mod libext;
15pub mod prelude;
16
17// we impose our version of anndists
18pub use anndists;
19
20lazy_static! {
21    static ref LOG: u64 = init_log();
22}
23
24// install a logger facility
25#[allow(unused)]
26fn init_log() -> u64 {
27    Builder::from_default_env().init();
28    println!("\n ************** initializing logger *****************\n");
29    1
30}
```

--------------------------------

### get_uint

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnsw/type.PointDistance.html?search=

Gets an unsigned n-byte integer from self in big-endian byte order.

```APIDOC
## get_uint

### Description
Gets an unsigned n-byte integer from `self` in big-endian byte order.

### Method
fn get_uint(&mut self, nbytes: usize) -> u64

### Parameters
- `self` (mutable reference) - Required - The byte buffer to read from.
- `nbytes` (usize) - Required - The number of bytes to read.

### Returns
u64 - The unsigned n-byte integer read from the buffer.
```

--------------------------------

### Box::try_map Example

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnsw/type.PointDistance.html?search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E

Attempts to map the value in a box, requiring the smart_pointer_try_map feature.

```rust
#![feature(smart_pointer_try_map)]
```

--------------------------------

### DumpInit Helper Methods

Source: https://docs.rs/hnsw_rs/latest/src/hnsw_rs/hnswio.rs.html

Provides methods to retrieve the basename and flush the buffered writers.

```rust
pub fn get_basename(&self) -> &String {
        &self.basename
    }

    pub fn flush(&mut self) -> Result<()> {
        self.data_out.flush()?;
        self.graph_out.flush()?;
        Ok(())
    }
```

--------------------------------

### Box try_map usage

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnsw/type.PointDistance.html

Example of using Box::try_map to transform a boxed value.

```rust
let b = Box::new(7);
let new = Box::try_map(b, u32::try_from).unwrap();
assert_eq!(*new, 7);
```

--------------------------------

### Box::new_uninit

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnsw/type.PointDistance.html?search=

Constructs a new box with uninitialized contents.

```APIDOC
## Box::new_uninit

### Description
Constructs a new box with uninitialized contents.

### Method
pub fn new_uninit() -> Box<MaybeUninit<T>>

### Return Value
- **Box<MaybeUninit<T>>** - A box with uninitialized contents.

### Examples
```rust
let mut five = Box::<u32>::new_uninit();
// Deferred initialization:
five.write(5);
let five = unsafe { five.assume_init() };

assert_eq!(*five, 5)
```
```

--------------------------------

### Box::map Example

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnsw/type.PointDistance.html

Demonstrates mapping the value inside a box while reusing the allocation.

```rust
let b = Box::new(7);
let new = Box::map(b, |i| i + 7);
assert_eq!(*new, 14);
```

--------------------------------

### Reload HNSW with mmap

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnswio/struct.HnswIo.html

Demonstrates how to initialize HnswIo, configure it for memory mapping, and load an HNSW graph from a directory.

```rust
    let directory = Path::new(".");
    let mut reloader = HnswIo::new(directory, "mmapreloadtest");
    let options = ReloadOptions::default().set_mmap(true);
    reloader.set_options(options);
    let hnsw_loaded : Hnsw<f32,DistL1>= reloader.load_hnsw::<f32, DistL1>().unwrap();
```

--------------------------------

### Partial HNSW graph initialization

Source: https://docs.rs/hnsw_rs/latest/src/hnsw_rs/hnswio.rs.html

Initial setup code for a test case focusing on graph-only operations.

```rust
#[test]
fn test_dump_reload_graph_only() {
    println!("\n\n test_dump_reload_graph_only");
    log_init_test();
    // generate a random test
    let mut rng = rand::rng();
    let unif = Uniform::<f32>::new(0., 1.).unwrap();
    // 1000 vectors of size 10 f32
    let nbcolumn = 1000;
    let nbrow = 10;
    let mut xsi;
    let mut data = Vec::with_capacity(nbcolumn);
    for j in 0..nbcolumn {
        data.push(Vec::with_capacity(nbrow));
        for _ in 0..nbrow {
            xsi = unif.sample(&mut rng);
            data[j].push(xsi);
        }
    }
    // define hnsw
    let ef_construct = 25;
    let nb_connection = 10;
    let hnsw = Hnsw::<f32, dist::DistL1>::new(
        nb_connection,
        nbcolumn,
        16,
        ef_construct,
```

--------------------------------

### Box::try_new_zeroed_in Example

Source: https://docs.rs/hnsw_rs/latest/hnsw_rs/hnsw/type.PointDistance.html

Demonstrates constructing a box with zeroed memory, returning an error if allocation fails.

```rust
#![feature(allocator_ext)]

use std::alloc::System;

let zero = Box::<u32, _>::try_new_zeroed_in(System)?;
let zero = unsafe { zero.assume_init() };

assert_eq!(*zero, 0);
```

--------------------------------

### DumpInit Initialization

Source: https://docs.rs/hnsw_rs/latest/src/hnsw_rs/hnswio.rs.html?search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E

Initializes the DumpInit structure, handling unique filename generation if overwriting is disabled.

```rust
impl DumpInit {
    // This structure will check existence of dumps of same name and generate a unique filename if necessary according to overwrite flag
    pub fn new(dir: &Path, basename_default: &str, overwrite: bool) -> Self {
        // if we cannot overwrite data files (in case of mmap in particular)
        // we will ensure we have a unique basename
        let basename = match overwrite {
            true => basename_default.to_string(),
            false => {
                // we check
                let mut dataname = basename_default.to_string();
                dataname.push_str(".hnsw.data");
                let mut datapath = PathBuf::from(dir);
                datapath.push(dataname);
                let exist_res = std::fs::metadata(datapath.as_os_str());
                if exist_res.is_ok() {
                    loop {
                        let mut unique_basename;
                        let mut dataname: String;
                        let id: usize = rand::random_range(0..10000);
                        let strid: String = id.to_string();
                        unique_basename = basename_default.to_string();
                        unique_basename.push('-');
                        unique_basename.push_str(&strid);
                        dataname = unique_basename.clone();
                        dataname.push_str(".hnsw.data");
                        let mut datapath = PathBuf::from(dir);
                        datapath.push(dataname);
                        let exist_res = std::fs::metadata(datapath.as_os_str());
                        if exist_res.is_err() {
                            break unique_basename;
                        }
                    }
                } else {
                    basename_default.to_string()
                }
            }
        };
        //
        debug!(
            "DumpInit : dumping in dir {:#?} with (unique) basename : {}",
            dir, basename
        );
        //
        let mut graphname = basename.clone();
        graphname.push_str(".hnsw.graph");
        let mut graphpath = PathBuf::from(dir);
        graphpath.push(graphname);
        let graphfileres = OpenOptions::new()
            .create(true)
            .truncate(true)
            .write(true)
            .open(&graphpath);
        if graphfileres.is_err() {
            println!(
                "DumpInit::new : could not open file {:?}",
                graphpath.as_os_str()
            );
            std::panic::panic_any("HnswIo::init : could not open file".to_string());
        }
        let graphfile = graphfileres.unwrap();
        //  same thing for data file
        let mut dataname = basename.clone();
        dataname.push_str(".hnsw.data");
        let mut datapath = PathBuf::from(dir);
        datapath.push(dataname);
        let datafileres = OpenOptions::new()
            .create(true)
            .truncate(true)
            .write(true)
            .open(&datapath);
        if datafileres.is_err() {
            println!(
                "DumpInit::init : could not open file {:?}",
                datapath.as_os_str()
            );
            std::panic::panic_any("HnswIo::init : could not open file".to_string());
        }
        let datafile = datafileres.unwrap();
        //
        let graph_out = BufWriter::new(graphfile);
        let data_out = BufWriter::new(datafile);
        //
        DumpInit {
            basename,
            data_out,
            graph_out,
        }
    }

    /// returns the basename used for the dump. May be it has been made unique to void overwriting a previous or mmapped dump
    pub fn get_basename(&self) -> &String {
        &self.basename
    }

    pub fn flush(&mut self) -> Result<()> {
        self.data_out.flush()?;
        self.graph_out.flush()?;
        Ok(())
    }
}
```
Tokens:
Raw