Hyperloglockless

https://docs.rs/hyperloglockless/0.5.0
Admin
Hyperloglockless is a high-performance Rust library for accurate and space-efficient cardinality...
Tokens:29,706
Snippets:237
Trust Score:9.1
Update:6 months ago
Show doc for...
Tokens:
Raw
### HyperLogLog Test Suite Setup

Source: https://docs.rs/hyperloglockless/0.5.0/src/hyperloglockless/lib.rs.html

Macro for setting up test modules for HyperLogLog implementations, including tests for cloning and low error rates. This is conditional on not using the 'loom' feature and being in a test configuration.

```rust
macro_rules! impl_tests {
    ($modname:ident, $name:ident, $seed:literal) => {
        #[allow(unused_mut)]
        #[cfg(not(feature = "loom"))]
        #[cfg(test)]
        mod $modname {
            use super::*;
            #[test]
            fn test_clone() {
                let mut hll = $name::seeded(4, $seed);
                hll.insert_all(1..10);
                let mut cloned = hll.clone();
                assert_eq!(hll, cloned);
                for x in 0..=1000 {
                    cloned.insert(&x);
                }
                assert!(hll != cloned);
            }

            #[test]
            fn test_low_error() {
                for p in 4..=18 {
                    low_error(p);
                }
            }

            fn low_error(precision: u8) {
                let thresh = error_for_precision(precision) * 1.3; // within 30% of the expected error

                let mut counted = 0;
                let mut total_err = 0f64;
                let mut total_diff = 0f64;

                for seed in 1..=4 {
                    let mut hll = $name::seeded(precision, seed);
                    let mut rng = fastrand::Rng::with_seed(643340961);
                    for x in 1..10_000_000 {
                        let hash = rng.u64(..);
                        hll.insert_hash(hash);
                        if x % 10 == 0 {
                            let real = x as f64;

```

--------------------------------

### Create HyperLogLogPlus with Default Hasher

Source: https://docs.rs/hyperloglockless/0.5.0/hyperloglockless/struct.HyperLogLogPlus.html

Initializes a new HyperLogLogPlus instance using the default hasher and a random seed. It starts with a sparse representation and may switch to dense mode based on memory usage.

```rust
pub fn new(precision: u8) -> Self
```

--------------------------------

### Get Precision of HyperLogLogPlus

Source: https://docs.rs/hyperloglockless/0.5.0/hyperloglockless/struct.HyperLogLogPlus.html

Returns the precision value that was used to initialize the HyperLogLogPlus instance.

```rust
pub fn precision(&self) -> u8
```

--------------------------------

### Initialization Methods

Source: https://docs.rs/hyperloglockless/0.5.0/src/hyperloglockless/sparse.rs.html

Methods for creating new instances of HyperLogLogPlus with different seeding and hasher configurations.

```APIDOC
## Initialization Methods

### `new(precision: u8) -> Self`

Returns a new `Self` using the default hasher with a random seed. The structure is initialized to use a sparse representation, which may later switch to a dense representation if memory usage dictates.

### `seeded(precision: u8, seed: u128) -> Self`

Returns a new `Self` using the default hasher seeded with the provided `seed`. Similar to `new`, it starts with a sparse representation and can transition to a dense one.

### `with_hasher(precision: u8, hasher: S) -> Self`

Returns a new `Self` using the provided `hasher`. This method initializes the structure with a sparse representation, which can dynamically switch to a dense representation based on memory usage.
```

--------------------------------

### Initialize and Use HyperLogLog

Source: https://docs.rs/hyperloglockless/0.5.0/hyperloglockless/struct.HyperLogLog.html

Demonstrates basic usage of HyperLogLog: creating an instance, inserting elements, and retrieving the approximate count. Requires importing the HyperLogLog struct.

```rust
use hyperloglockless::HyperLogLog;

let mut hll = HyperLogLog::new(16);
hll.insert("42");
hll.insert("🦀");

let count = hll.count();
```

--------------------------------

### Get Number of Registers

Source: https://docs.rs/hyperloglockless/0.5.0/hyperloglockless/struct.AtomicHyperLogLog.html

Retrieve the total number of registers used by the AtomicHyperLogLog instance.

```rust
pub fn len(&self) -> usize
```

--------------------------------

### Initialize and use HyperLogLog

Source: https://docs.rs/hyperloglockless/0.5.0/index.html

Create a standard HyperLogLog instance, insert elements, and retrieve the estimated cardinality.

```rust
use hyperloglockless::HyperLogLog;

let precision = hyperloglockless::precision_for_error(0.01); // 1% error
assert_eq!(precision, 14);

let mut hll = HyperLogLog::new(precision);
hll.insert(&'🦀');
hll.insert_all('a'..='z');

let count = hll.count(); // ~27
assert_eq!(hll.len(), 1 << precision); // 16384 bytes
```

--------------------------------

### Get Type ID

Source: https://docs.rs/hyperloglockless/0.5.0/hyperloglockless/struct.AtomicHyperLogLog.html

Retrieve the `TypeId` of a given type, useful for runtime type information.

```rust
fn type_id(&self) -> TypeId
```

--------------------------------

### Get Raw Approximate Count

Source: https://docs.rs/hyperloglockless/0.5.0/hyperloglockless/struct.AtomicHyperLogLog.html

Retrieve the approximate number of distinct elements as a floating-point number.

```rust
pub fn raw_count(&self) -> f64
```

--------------------------------

### HyperLogLogPlus Initialization

Source: https://docs.rs/hyperloglockless/0.5.0/hyperloglockless/struct.HyperLogLogPlus.html

Methods to create a new instance of the HyperLogLogPlus structure with varying configuration options.

```APIDOC
## new(precision: u8)

### Description
Returns a new HyperLogLogPlus instance using the default hasher with a random seed.

### Parameters
- **precision** (u8) - Required - The precision parameter determining the number of registers.

## seeded(precision: u8, seed: u128)

### Description
Returns a new HyperLogLogPlus instance using the default hasher seeded with a specific value.

### Parameters
- **precision** (u8) - Required - The precision parameter.
- **seed** (u128) - Required - The seed for the hasher.

## with_hasher(precision: u8, hasher: S)

### Description
Returns a new HyperLogLogPlus instance using a custom provided hasher.

### Parameters
- **precision** (u8) - Required - The precision parameter.
- **hasher** (S) - Required - The custom hasher implementation.
```

--------------------------------

### Get Approximate Element Count

Source: https://docs.rs/hyperloglockless/0.5.0/hyperloglockless/struct.AtomicHyperLogLog.html

Obtain the estimated number of distinct elements inserted into the AtomicHyperLogLog.

```rust
pub fn count(&self) -> usize
```

--------------------------------

### Initialize HyperLogLog with custom hasher

Source: https://docs.rs/hyperloglockless/0.5.0/index.html

Configure a HyperLogLog instance using a specific hasher implementation.

```rust
use hyperloglockless::HyperLogLog;
use foldhash::fast::RandomState;

let hll = HyperLogLog::with_hasher(14, RandomState::default());
```

--------------------------------

### Get Approximate Raw Count

Source: https://docs.rs/hyperloglockless/0.5.0/src/hyperloglockless/lib.rs.html

Returns the approximate number of elements in the HyperLogLog. This method uses internal state for calculation.

```rust
/// Returns the approximate number of elements in `self`.
    #[inline]
    pub fn raw_count(&self) -> f64 {
        let zeros = self.zeros.load(Relaxed);
        let sum = self.sum.load(Relaxed);
        self.raw_count_inner(zeros, sum)
    }
```

--------------------------------

### Get Raw Approximate Element Count

Source: https://docs.rs/hyperloglockless/0.5.0/hyperloglockless/struct.HyperLogLogPlus.html

Returns the raw, floating-point approximation of the number of unique elements. This method requires mutable access.

```rust
pub fn raw_count(&mut self) -> f64
```

--------------------------------

### Constructor: new

Source: https://docs.rs/hyperloglockless/0.5.0/src/hyperloglockless/lib.rs.html

Constructs a new HyperLogLog or AtomicHyperLogLog instance with a specified precision using the default hasher.

```APIDOC
## Constructor: `new`

### Description
Returns a new [`Self`] with `1 << precision` registers (1 byte each) using the default hasher with a random seed.

### Method
`new(precision: u8) -> Self<DefaultHasher>`

### Parameters
- **precision** (u8): The precision of the HyperLogLog structure, determining the number of registers.

### Returns
A new instance of `HyperLogLog` or `AtomicHyperLogLog` using `DefaultHasher`.
```

--------------------------------

### Initialize HyperLogLogPlus

Source: https://docs.rs/hyperloglockless/0.5.0/src/hyperloglockless/sparse.rs.html

Methods to create a new instance with default or custom hashers and seeds.

```rust
pub fn new(precision: u8) -> Self {
        Self::with_hasher(precision, DefaultHasher::default())
    }
```

```rust
pub fn seeded(precision: u8, seed: u128) -> Self {
        Self::with_hasher(precision, DefaultHasher::seeded(&seed.to_be_bytes()))
    }
```

```rust
pub fn with_hasher(precision: u8, hasher: S) -> Self {
        crate::validate_precision(precision);
        Self {
            sparse: Some(SparseLogLog::new(precision)),
            dense: None,
            hasher,
        }
    }
```

--------------------------------

### Get Approximate Element Count

Source: https://docs.rs/hyperloglockless/0.5.0/hyperloglockless/struct.HyperLogLogPlus.html

Returns the estimated number of unique elements inserted into the HyperLogLogPlus. This method requires mutable access as it might trigger internal state changes.

```rust
pub fn count(&mut self) -> usize
```

--------------------------------

### Get approximate error for precision

Source: https://docs.rs/hyperloglockless/0.5.0/hyperloglockless/fn.error_for_precision.html

Returns the approximate error of `count` and `raw_count` given the precision of a `HyperLogLog` or `AtomicHyperLogLog`. Use this to understand the expected accuracy of cardinality estimates based on the chosen precision.

```rust
pub fn error_for_precision(precision: u8) -> f64
```

--------------------------------

### Construct HyperLogLog from parts

Source: https://docs.rs/hyperloglockless/0.5.0/src/hyperloglockless/lib.rs.html

Demonstrates how to serialize and deserialize a HyperLogLog instance using its internal components.

```rust
use hyperloglockless::HyperLogLog;

let mut before = HyperLogLog::seeded(16, 42);
before.extend(1000..=2000);
let (x, y, z, w) = before.parts();
let after = HyperLogLog::from_parts(x.into(), y.clone(), z, w);
assert_eq!(before, after);
```

--------------------------------

### Initialize and Use AtomicHyperLogLog

Source: https://docs.rs/hyperloglockless/0.5.0/hyperloglockless/struct.AtomicHyperLogLog.html

Create a new AtomicHyperLogLog with a specified precision and insert elements. The count can then be retrieved.

```rust
use hyperloglockless::AtomicHyperLogLog;

let hll = AtomicHyperLogLog::new(16);
hll.insert("42");
hll.insert("🦀");

let count = hll.count();
```

--------------------------------

### Buf Initialization and Length

Source: https://docs.rs/hyperloglockless/0.5.0/src/hyperloglockless/buf.rs.html

Provides methods to create a new Buf instance with a specified capacity and maximum length, and to retrieve the current length of the buffer.

```rust
14    #[inline]
15    pub fn new(cap: usize, max_len: usize) -> Self {
16        let buf: Vec<_> = repeat(0).take(cap).collect();
17        Self {
18            buf,
19            len: 0,
20            max_len,
21        }
22    }
23
24    #[inline]
25    pub fn len(&self) -> usize {
26        self.len
27    }
```

--------------------------------

### Create HyperLogLogPlus with Seeded Hasher

Source: https://docs.rs/hyperloglockless/0.5.0/hyperloglockless/struct.HyperLogLogPlus.html

Initializes a new HyperLogLogPlus instance with a specific seed for the default hasher. The data structure begins in sparse mode and can transition to dense mode.

```rust
pub fn seeded(precision: u8, seed: u128) -> Self
```

--------------------------------

### Construct HyperLogLog with Default Hasher and Seed

Source: https://docs.rs/hyperloglockless/0.5.0/hyperloglockless/struct.HyperLogLog.html

Initializes a HyperLogLog with the default hasher, seeded with a specific value. This is useful for reproducible results.

```rust
use hyperloglockless::HyperLogLog;

let mut hll = HyperLogLog::seeded(16, 42);
hll.insert("another test");
```

--------------------------------

### Initialize SparseLogLog

Source: https://docs.rs/hyperloglockless/0.5.0/src/hyperloglockless/sparse.rs.html

Creates a new `SparseLogLog` with a specified precision. It initializes an empty vector for new hashes and a default `DiffVec` for storing indexed hashes.

```rust
pub fn new(precision: u8) -> Self {
    Self {
        new: Vec::new(),
        indexes: Default::default(),
        precision,
    }
}
```

--------------------------------

### AtomicHyperLogLog Methods

Source: https://docs.rs/hyperloglockless/0.5.0/hyperloglockless/struct.AtomicHyperLogLog.html

Core methods for interacting with the AtomicHyperLogLog instance.

```APIDOC
### impl<S: BuildHasher> AtomicHyperLogLog<S>

#### pub fn len(&self) -> usize

Returns the number registers in `self`.

#### pub fn count(&self) -> usize

Returns the approximate number of elements in `self`.

#### pub fn iter(&self) -> impl Iterator<Item = u8> + '_ 

Returns an iterator over the value of each register.

#### pub fn insert<T: Hash + ?Sized>(&self, value: &T)

Inserts the item into the HyperLogLog.

#### pub fn insert_hash(&self, hash: u64)

Inserts the hash of an item into the HyperLogLog.

#### pub fn insert_all<T: Hash, I: IntoIterator<Item = T>>(&self, iter: I)

Inserts all the items in `iter` into the `self`. Immutable version of `Self::extend`.

#### pub fn union(&self, other: &Self) -> Result<(), Error>

Merges another HyperLogLog into `self`, updating the count. Returns `Err(Error::IncompatibleLength)` if the two HyperLogLogs have different length (`Self::len`).

#### pub fn raw_count(&self) -> f64

Returns the approximate number of elements in `self`.

#### pub fn parts<'a>(&'a self) -> (&'a [AtomicU8], &'a S, usize, f64)

Low level method to expose de/serializable parts of `self`.

#### pub fn from_parts( registers: Box<[AtomicU8]>, hasher: S, zeros: usize, sum: f64, ) -> Self

Low level method to construct `Self` de/serializable parts.
```

--------------------------------

### HyperLogLog Struct and Constructors

Source: https://docs.rs/hyperloglockless/0.5.0/hyperloglockless/struct.HyperLogLog.html

Details on the HyperLogLog struct and its various constructors.

```APIDOC
## Struct HyperLogLog

```
pub struct HyperLogLog<S = DefaultHasher> { /* private fields */ }
```

HyperLogLog is a data structure for the “count-distinct problem”, approximating the number of distinct elements in a multiset.

### `new(precision: u8) -> HyperLogLog<DefaultHasher>`

Returns a new `Self` with `1 << precision` registers (1 byte each) using the default hasher with a random seed.

### `seeded(precision: u8, seed: u128) -> HyperLogLog<DefaultHasher>`

Returns a new `Self` with `1 << precision` registers (1 byte each) using the default hasher seeded with `seed`.

### `with_hasher(precision: u8, hasher: S) -> Self`

Returns a new `HyperLogLog` with `1 << precision` registers (1 byte each) using the provided hasher.
```

--------------------------------

### AtomicHyperLogLog Constructors

Source: https://docs.rs/hyperloglockless/0.5.0/hyperloglockless/struct.AtomicHyperLogLog.html

Methods for creating new instances of AtomicHyperLogLog.

```APIDOC
### impl<S: BuildHasher> AtomicHyperLogLog<S>

#### pub fn with_hasher(precision: u8, hasher: S) -> Self

Returns a new `AtomicHyperLogLog` with `1 << precision` registers (1 byte each) using the provided hasher.

### impl AtomicHyperLogLog

#### pub fn new(precision: u8) -> AtomicHyperLogLog<DefaultHasher>

Returns a new `Self` with `1 << precision` registers (1 byte each) using the default hasher with a random seed.

#### pub fn seeded(precision: u8, seed: u128) -> AtomicHyperLogLog<DefaultHasher>

Returns a new `Self` with `1 << precision` registers (1 byte each) using the default hasher seeded with `seed`.
```

--------------------------------

### Create AtomicHyperLogLog with Default Hasher

Source: https://docs.rs/hyperloglockless/0.5.0/hyperloglockless/struct.AtomicHyperLogLog.html

Construct a new AtomicHyperLogLog using the default hasher with a random seed.

```rust
pub fn new(precision: u8) -> AtomicHyperLogLog<DefaultHasher>
```

--------------------------------

### Create AtomicHyperLogLog with Seeded Hasher

Source: https://docs.rs/hyperloglockless/0.5.0/hyperloglockless/struct.AtomicHyperLogLog.html

Initialize an AtomicHyperLogLog with a specific precision and a custom seed for the default hasher.

```rust
pub fn seeded(precision: u8, seed: u128) -> AtomicHyperLogLog<DefaultHasher>
```

--------------------------------

### HyperLogLog Methods

Source: https://docs.rs/hyperloglockless/0.5.0/hyperloglockless/struct.HyperLogLog.html

Core methods for interacting with the HyperLogLog data structure.

```APIDOC
## HyperLogLog Methods

### `len(&self) -> usize`

Returns the number registers in `self`.

### `count(&self) -> usize`

Returns the approximate number of elements in `self`.

### `raw_count(&self) -> f64`

Returns the approximate number of elements in `self`.

### `insert<T: Hash + ?Sized>(&mut self, value: &T)`

Inserts the item into the HyperLogLog.

### `insert_hash(&mut self, hash: u64)`

Inserts the hash of an item into the HyperLogLog.

### `insert_all<T: Hash, I: IntoIterator<Item = T>>(&mut self, iter: I)`

Inserts all the items in `iter` into the `self`.

### `union(&mut self, other: &Self) -> Result<(), Error>`

Merges another HyperLogLog into `self`, updating the count. Returns `Err(Error::IncompatibleLength)` if the two HyperLogLogs have different length (`Self::len`).
This does not verify that the HLLs use the same hasher or seed. If they are different then `self` will be “corrupted”.

### `iter(&self) -> impl Iterator<Item = u8> + '_`

Returns an iterator over the value of each register.

### `parts<'a>(&'a self) -> (&'a [u8], &'a S, usize, f64)`

Low level method to expose de/serializable parts of `self`.

### `from_parts(registers: Box<[u8]>, hasher: S, zeros: usize, sum: f64) -> Self`

Low level method to construct `Self` de/serializable parts.

##### §Example
```
use hyperloglockless::HyperLogLog;

let mut before = HyperLogLog::seeded(16, 42);
before.extend(1000..=2000);
let (x, y, z, w) = before.parts();
let after = HyperLogLog::from_parts(x.into(), y.clone(), z, w);
assert_eq!(before, after);
```
```

--------------------------------

### Create AtomicHyperLogLog with Custom Hasher

Source: https://docs.rs/hyperloglockless/0.5.0/hyperloglockless/struct.AtomicHyperLogLog.html

Instantiate an AtomicHyperLogLog using a provided BuildHasher implementation.

```rust
pub fn with_hasher(precision: u8, hasher: S) -> Self
```

--------------------------------

### SparseLogLog to HyperLogLog Conversion

Source: https://docs.rs/hyperloglockless/0.5.0/src/hyperloglockless/sparse.rs.html

Implementation of the From trait to convert a sparse representation into a dense HyperLogLog structure.

```rust
impl From<SparseLogLog> for HyperLogLog {
    fn from(mut sparse: SparseLogLog) -> Self {
        sparse.flush();
        let registers = sparse.indexes.into_iter();
        let mut hll = HyperLogLog::new(sparse.precision);
        for encoded in registers {
            let (rank, register) = decode_hash(encoded, sparse.precision);
            hll.update(rank as u8, register);
        }
        hll
    }
}
```

--------------------------------

### Initialize AtomicF64

Source: https://docs.rs/hyperloglockless/0.5.0/src/hyperloglockless/atomic_f64.rs.html

Creates a new AtomicF64 from an f64 value. This is useful for initializing atomic floating-point numbers.

```rust
pub fn new(float: f64) -> Self {
    Self(AtomicU64::new(u64::from_ne_bytes(float.to_ne_bytes())))
}
```

--------------------------------

### Serialize and Deserialize HyperLogLog

Source: https://docs.rs/hyperloglockless/0.5.0/src/hyperloglockless/lib.rs.html

Demonstrates compatibility between HyperLogLog and AtomicHyperLogLog using serde_cbor for serialization.

```rust
let mut non = HyperLogLog::seeded(precision, 42);
            non.insert_all(0..=1000);
            let atomic = AtomicHyperLogLog::seeded(precision, 42);
            atomic.insert_all(0..=1000);

            let non_bytes = serde_cbor::to_vec(&non).unwrap();
            let atomic_bytes = serde_cbor::to_vec(&atomic).unwrap();
            assert_eq!(non_bytes, atomic_bytes);

            let non_from_atomic: HyperLogLog = serde_cbor::from_slice(&atomic_bytes).unwrap();
            let atomic_from_non: AtomicHyperLogLog = serde_cbor::from_slice(&non_bytes).unwrap();
            assert_eq!(non_from_atomic, non);
            assert_eq!(atomic_from_non, atomic);
```

--------------------------------

### PartialEq Implementation for HyperLogLogPlus

Source: https://docs.rs/hyperloglockless/0.5.0/src/hyperloglockless/sparse.rs.html

Implements the equality comparison for HyperLogLogPlus instances. Checks if the sparse and dense representations are equal.

```rust
self.sparse == other.sparse && self.dense == other.dense
```

--------------------------------

### HyperLogLog with Custom Hasher

Source: https://docs.rs/hyperloglockless/0.5.0/hyperloglockless/index.html

Instantiate a HyperLogLog with a specific hasher, such as `foldhash::fast::RandomState`, for custom hashing needs.

```rust
use hyperloglockless::HyperLogLog;
use foldhash::fast::RandomState;

let hll = HyperLogLog::with_hasher(14, RandomState::default());

```

--------------------------------

### Trait Implementations

Source: https://docs.rs/hyperloglockless/0.5.0/hyperloglockless/struct.HyperLogLog.html

Overview of standard and custom trait implementations for types within the crate.

```APIDOC
## Trait Implementations

### Borrow<T>
- **fn borrow(&self) -> &T**: Immutably borrows from an owned value.

### BorrowMut<T>
- **fn borrow_mut(&mut self) -> &mut T**: Mutably borrows from an owned value.

### CloneToUninit
- **unsafe fn clone_to_uninit(&self, dest: *mut u8)**: Performs copy-assignment from self to dest. (Nightly-only experimental API).

### ToOwned
- **type Owned = T**: The resulting type after obtaining ownership.
- **fn to_owned(&self) -> T**: Creates owned data from borrowed data.
- **fn clone_into(&self, target: &mut T)**: Uses borrowed data to replace owned data.

### VZip<V>
- **fn vzip(self) -> V**: Zips the value into a multi-lane structure where V: MultiLane<T>.
```

--------------------------------

### Construct HyperLogLog with Custom Hasher

Source: https://docs.rs/hyperloglockless/0.5.0/hyperloglockless/struct.HyperLogLog.html

Creates a new HyperLogLog instance using a specific hasher implementation. The precision determines the number of registers (1 << precision).

```rust
use hyperloglockless::HyperLogLog;
use std::collections::hash_map::RandomState;

let hasher = RandomState::new();
let mut hll = HyperLogLog::with_hasher(16, hasher);
hll.insert("test");
```

--------------------------------

### Utility Methods

Source: https://docs.rs/hyperloglockless/0.5.0/src/hyperloglockless/sparse.rs.html

Methods for retrieving information about the HyperLogLogPlus structure and performing operations like union.

```APIDOC
## Utility Methods

### `count(&mut self) -> usize`

Returns the approximate number of unique elements inserted into the HyperLogLogPlus. This method returns a `usize`.

### `raw_count(&mut self) -> f64`

Returns the approximate number of unique elements as a floating-point number (`f64`). This provides a more precise count before potential conversion to `usize`.

### `is_sparse(&self) -> bool`

Returns `true` if the HyperLogLogPlus is currently using its internal sparse representation, and `false` if it is using the dense (classic HyperLogLog) representation.

### `precision(&self) -> u8`

Returns the precision parameter configured for this HyperLogLogPlus instance.

### `union(&mut self, other: &Self) -> Result<(), Error>`

Merges another HyperLogLogPlus structure (`other`) into the current one (`self`). This operation updates the count based on the elements in both structures. It returns an `Err(Error::IncompatibleLength)` if the `other` HyperLogLogPlus has a different precision. Note that this method does not verify if the hashers or seeds are compatible; incompatible hashers may lead to corrupted results.
```

--------------------------------

### HyperLogLog Insertion Methods

Source: https://docs.rs/hyperloglockless/0.5.0/src/hyperloglockless/lib.rs.html

Methods for inserting elements into the HyperLogLog.

```APIDOC
## HyperLogLog Insertion Methods

### Description
Methods for inserting elements into the HyperLogLog, either individually by value or hash, or in bulk from an iterator.

### Methods

#### `insert(value: &T)`
- **Description**: Inserts an item into the HyperLogLog.
- **Method**: `POST` (conceptual)
- **Endpoint**: N/A (Instance method)
- **Parameters**:
  - **value** (T) - Required - The item to insert.

#### `insert_hash(hash: u64)`
- **Description**: Inserts the hash of an item into the HyperLogLog.
- **Method**: `POST` (conceptual)
- **Endpoint**: N/A (Instance method)
- **Parameters**:
  - **hash** (u64) - Required - The hash of the item to insert.

#### `insert_all<T: Hash, I: IntoIterator<Item = T>>(iter: I)`
- **Description**: Inserts all the items in `iter` into the `self`.
- **Method**: `POST` (conceptual)
- **Endpoint**: N/A (Instance method)
- **Parameters**:
  - **iter** (I) - Required - An iterator providing items to insert.

### Request Example (Conceptual)
```json
{
  "value": "example_item"
}
```

### Response Example (Conceptual)
```json
{
  "status": "inserted"
}
```
```

--------------------------------

### Constructor: with_hasher

Source: https://docs.rs/hyperloglockless/0.5.0/src/hyperloglockless/lib.rs.html

Constructs a new HyperLogLog or AtomicHyperLogLog instance with a specified precision and a custom hasher.

```APIDOC
## Constructor: `with_hasher`

### Description
Returns a new `HyperLogLog` or `AtomicHyperLogLog` with `1 << precision` registers (1 byte each) using the provided hasher.

### Method
`with_hasher(precision: u8, hasher: S) -> Self`

### Parameters
- **precision** (u8): The precision of the HyperLogLog structure, determining the number of registers.
- **hasher** (S): An instance of a type implementing `BuildHasher`.

### Returns
A new instance of `HyperLogLog` or `AtomicHyperLogLog`.
```

--------------------------------

### Initialize DiffVec with size

Source: https://docs.rs/hyperloglockless/0.5.0/src/hyperloglockless/sparse.rs.html

Creates a new `DiffVec` with a specified initial size and maximum capacity for its internal buffer. This is used for efficient storage of differences.

```rust
pub fn with_size(size: usize, max_size: usize) -> Self {
    Self {
        encoded: Buf::new(size, max_size),
        last: 0,
        len: 0,
    }
}
```

--------------------------------

### HyperLogLog Core Methods

Source: https://docs.rs/hyperloglockless/0.5.0/src/hyperloglockless/lib.rs.html

Provides methods for managing and querying the HyperLogLog data structure.

```APIDOC
## HyperLogLog Core Methods

### Description
Provides methods for managing and querying the HyperLogLog data structure, including getting the number of registers, the approximate count of elements, and iterating over register values.

### Methods

#### `len()`
- **Description**: Returns the number of registers in the HyperLogLog.
- **Method**: `GET` (conceptual)
- **Endpoint**: N/A (Instance method)

#### `count()`
- **Description**: Returns the approximate number of elements in the HyperLogLog.
- **Method**: `GET` (conceptual)
- **Endpoint**: N/A (Instance method)

#### `iter()`
- **Description**: Returns an iterator over the value of each register.
- **Method**: `GET` (conceptual)
- **Endpoint**: N/A (Instance method)

#### `raw_count()`
- **Description**: Returns the approximate number of elements in `self`.
- **Method**: `GET` (conceptual)
- **Endpoint**: N/A (Instance method)

### Response Example (Conceptual)
```json
{
  "registers_count": 16384, 
  "approximate_count": 15000
}
```
```

--------------------------------

### Constructor: seeded

Source: https://docs.rs/hyperloglockless/0.5.0/src/hyperloglockless/lib.rs.html

Constructs a new HyperLogLog or AtomicHyperLogLog instance with a specified precision and a seed for the default hasher.

```APIDOC
## Constructor: `seeded`

### Description
Returns a new [`Self`] with `1 << precision` registers (1 byte each) using the default hasher seeded with `seed`.

### Method
`seeded(precision: u8, seed: u128) -> Self<DefaultHasher>`

### Parameters
- **precision** (u8): The precision of the HyperLogLog structure, determining the number of registers.
- **seed** (u128): The seed value for the `DefaultHasher`.

### Returns
A new instance of `HyperLogLog` or `AtomicHyperLogLog` using `DefaultHasher` with the specified seed.
```

--------------------------------

### Add hyperloglockless to Dependencies

Source: https://docs.rs/hyperloglockless/0.5.0/hyperloglockless/index.html

Include the hyperloglockless crate in your project's Cargo.toml file to use its features.

```toml
[dependencies]
hyperloglockless = "0.5.0"

```

--------------------------------

### Reconstruct HyperLogLog from Parts

Source: https://docs.rs/hyperloglockless/0.5.0/hyperloglockless/struct.HyperLogLog.html

Constructs a HyperLogLog instance from its low-level components: registers, hasher, zero count, and sum. This is useful for serialization and deserialization.

```rust
use hyperloglockless::HyperLogLog;

let mut before = HyperLogLog::seeded(16, 42);
before.extend(1000..=2000);
let (registers, hasher, zeros, sum) = before.parts();
let after = HyperLogLog::from_parts(registers.into(), hasher.clone(), zeros, sum);
assert_eq!(before, after);
```

--------------------------------

### Insertion Methods

Source: https://docs.rs/hyperloglockless/0.5.0/src/hyperloglockless/sparse.rs.html

Methods for adding elements to the HyperLogLogPlus structure.

```APIDOC
## Insertion Methods

### `insert_hash(&mut self, hash: u64)`

Inserts a pre-computed hash value into the HyperLogLogPlus. The internal representation may switch from sparse to dense if the sparse mode exceeds the memory usage of the dense mode.

### `insert<T: Hash + ?Sized>(&mut self, value: &T)`

Inserts a value into the HyperLogLogPlus by first hashing it. This method also handles the potential switch from sparse to dense representation.

### `insert_all<T: Hash, I: IntoIterator<Item = T>>(&mut self, iter: I)`

Inserts all elements from an iterator into the HyperLogLogPlus. This is a convenience method that repeatedly calls `insert` for each item in the iterator.
```

--------------------------------

### Pushing Data to Buf

Source: https://docs.rs/hyperloglockless/0.5.0/src/hyperloglockless/buf.rs.html

Appends a u64 value to the buffer. If the buffer is full, it first calls the resize method. The u64 value is converted to little-endian bytes before being copied into the buffer. The length of the buffer is then incremented.

```rust
36    #[inline]
37    pub fn push(&mut self, x: u64, num: usize) {
38        let l = self.len();
39        if l + 8 > self.buf.len() {
40            self.resize();
41        }
42        self.buf[l..l + 8].copy_from_slice(&x.to_le_bytes());
43        self.len += num;
44    }
```

--------------------------------

### Extend AtomicHyperLogLog with One Item (Nightly)

Source: https://docs.rs/hyperloglockless/0.5.0/hyperloglockless/struct.AtomicHyperLogLog.html

Add a single item to the AtomicHyperLogLog. This is an experimental API available only on nightly Rust.

```rust
fn extend_one(&mut self, item: A)
```

--------------------------------

### SparseLogLog Internal Management Methods

Source: https://docs.rs/hyperloglockless/0.5.0/src/hyperloglockless/sparse.rs.html

Internal methods for managing the sparse representation, including size calculation, flushing, and merging logic.

```rust
#[inline]
    fn hll_size_bytes(&self) -> usize {
        (1 << self.precision) as usize
    }

    #[inline]
    fn should_flush(&self) -> bool {
        let dense_hll_size = self.hll_size_bytes() << 2;
        let new_size = self.new.len();
        new_size * Self::NEW_SIZE_FACTOR > dense_hll_size
    }

    #[inline]
    pub fn insert_hash(&mut self, hash: u64) {
        let encoded = encode_hash(hash);
        if self.new.len() == self.new.capacity() {
            let dense_hll_size = ((1 << self.precision) as usize) << 2;
            let max_len =
                crate::math::ceil(dense_hll_size as f64 / Self::NEW_SIZE_FACTOR as f64) as usize;
            let new_cap = core::cmp::min(1 + (3 * self.new.len()) >> 1, max_len);
            self.new.reserve_exact(new_cap - self.new.len());
        }
        self.new.push(encoded);
        if self.should_flush() {
            self.flush()
        }
    }

    #[inline]
    pub(crate) fn flush_inner(&mut self, mut other: impl ExactSizeIterator<Item = u32>) {
        // TODO: empiraclly derive the size from the precision
        let size = self.indexes.size() + (other.len() * 3);
        let max_size = self.indexes.size() + (other.len() * 5) + 8;
        let mut buf = DiffVec::with_size(size, max_size);
        let binding = core::mem::take(&mut self.indexes);
        let mut this = binding.into_iter();
        let (mut new_hash, mut old_hash) = (other.next(), this.next());
        while new_hash.is_some() && old_hash.is_some() {
            let new_hash_ = new_hash.unwrap();
            let old_hash_ = old_hash.unwrap();
            if new_hash_ == old_hash_ {
                buf.push(new_hash_);
                new_hash = other.next();
                old_hash = this.next();
            } else if new_hash_ > old_hash_ {
                buf.push(old_hash_);
                old_hash = this.next();
            } else {
                buf.push(new_hash_);
                new_hash = other.next();
            }
        }
        while let Some(h) = new_hash {
            buf.push(h);
            new_hash = other.next();
        }
        while let Some(h) = old_hash {
            buf.push(h);
            old_hash = this.next();
        }
        self.indexes = buf;
        self.indexes.encoded.shrink_to_fit();
    }

    #[inline]
    pub(crate) fn flush(&mut self) {
        if self.new.is_empty() {
            return;
        }
        self.new.sort_unstable();
        let new = core::mem::take(&mut self.new);
        self.flush_inner(new.into_iter());
    }

    #[inline]
    pub(crate) fn union(&mut self, other: &Self) -> Result<(), Error> {
        if self.precision != other.precision {
            return Err(Error::IncompatibleLength);
        }
        let mut other_new = other.new.clone();
        other_new.sort_unstable();
        self.flush_inner(other_new.into_iter());
        self.flush_inner(other.indexes.into_iter());
        Ok(())
    }

    #[inline]
    pub fn count(&mut self) -> f64 {
        self.flush();
        correction(self.indexes.len())
    }

    #[inline]
    pub fn full(&self) -> bool {
        self.indexes.size() > self.hll_size_bytes()
    }
```

--------------------------------

### Buf Struct API

Source: https://docs.rs/hyperloglockless/0.5.0/src/hyperloglockless/buf.rs.html

Documentation for the Buf struct, including its fields and methods for managing a byte buffer.

```APIDOC
## Buf Struct

### Description
Represents a buffer with a current length and a maximum capacity, used for storing byte data.

### Fields
- **buf** (Vec<u8>) - The underlying vector storing the byte data.
- **len** (usize) - The current number of valid bytes in the buffer.
- **max_len** (usize) - The maximum allowed capacity of the buffer.

### Methods

#### `new(cap: usize, max_len: usize) -> Self`

Creates a new `Buf` instance with a specified initial capacity and maximum length.

#### `len(&self) -> usize`

Returns the current length of the data in the buffer.

#### `push(&mut self, x: u64, num: usize)`

Pushes a `u64` value into the buffer. If the buffer is full, it resizes.

#### `read_u64(&self, i: usize) -> u64`

Reads a `u64` value from the buffer at the specified index `i`.

#### `shrink_to_fit(&mut self)`

Shrinks the capacity of the buffer to its current length, optimizing memory usage.
```

--------------------------------

### HyperLogLog Serialization/Deserialization

Source: https://docs.rs/hyperloglockless/0.5.0/src/hyperloglockless/lib.rs.html

Methods for accessing and constructing HyperLogLog from its internal parts.

```APIDOC
## HyperLogLog Serialization/Deserialization

### Description
Provides low-level methods to access the internal components of a HyperLogLog for serialization and to construct a HyperLogLog from these components.

### Methods

#### `parts()`
- **Description**: Returns a tuple containing the de/serializable parts of the HyperLogLog: registers, hasher, zero count, and sum.
- **Method**: `GET` (conceptual)
- **Endpoint**: N/A (Instance method)
- **Returns**: `(&[u8], &S, usize, f64)` - A tuple of the HyperLogLog's internal state.

#### `from_parts(registers: Box<[u8]>, hasher: S, zeros: usize, sum: f64)`
- **Description**: Constructs a HyperLogLog from its de/serializable parts.
- **Method**: `POST` (conceptual)
- **Endpoint**: N/A (Static method)
- **Parameters**:
  - **registers** (Box<[u8]>) - Required - The byte slice representing the registers.
  - **hasher** (S) - Required - The build hasher used by the HyperLogLog.
  - **zeros** (usize) - Required - The count of zero registers.
  - **sum** (f64) - Required - The sum used in count estimation.
- **Returns**: `Self` - A new HyperLogLog instance.

### Request Example (`from_parts`)
```json
{
  "registers": [0, 1, 2, ...],
  "hasher": { ... }, // Representation of the hasher state
  "zeros": 100,
  "sum": 1234.56
}
```

### Response Example (`parts`)
```json
{
  "registers": [0, 1, 2, ...],
  "hasher": { ... }, // Representation of the hasher state
  "zeros": 100,
  "sum": 1234.56
}
```

### Example Usage (`from_parts`)
```rust
use hyperloglockless::HyperLogLog;

let mut before = HyperLogLog::seeded(16, 42);
before.extend(1000..=2000);
let (registers, hasher, zeros, sum) = before.parts();
let after = HyperLogLog::from_parts(registers.into(), hasher.clone(), zeros, sum);
assert_eq!(before, after);
```
```

--------------------------------

### Sparse HyperLogLog Representation

Source: https://docs.rs/hyperloglockless/0.5.0/hyperloglockless/index.html

Use `HyperLogLogPlus` for a memory-efficient sparse representation that automatically transitions to a dense representation when necessary. It offers improved speed and accuracy.

```rust
use hyperloglockless::HyperLogLogPlus;

let mut hll = HyperLogLogPlus::new(14);
hll.insert(&'🦀');
hll.insert_all('a'..='z');

```

--------------------------------

### Trait Implementations

Source: https://docs.rs/hyperloglockless/0.5.0/hyperloglockless/struct.HyperLogLog.html

Implementations of standard Rust traits for HyperLogLog.

```APIDOC
## Trait Implementations

### `impl<S: Clone> Clone for HyperLogLog<S>`

#### `clone(&self) -> HyperLogLog<S>`

Returns a duplicate of the value.

#### `clone_from(&mut self, source: &Self)`

Performs copy-assignment from `source`.

### `impl<S: Debug> Debug for HyperLogLog<S>`

#### `fmt(&self, f: &mut Formatter<'_>) -> Result`

Formats the value using the given formatter.

### `impl<T: Hash, S: BuildHasher> Extend<T> for HyperLogLog<S>`

#### `extend<I: IntoIterator<Item = T>>(&mut self, iter: I)`

Extends a collection with the contents of an iterator.

#### `extend_one(&mut self, item: A)`

🔬This is a nightly-only experimental API. (`extend_one`)
Extends a collection with exactly one element.

#### `extend_reserve(&mut self, additional: usize)`

🔬This is a nightly-only experimental API. (`extend_one`)
Reserves capacity in a collection for the given number of additional elements.

### `impl<S: BuildHasher> PartialEq for HyperLogLog<S>`

#### `eq(&self, other: &Self) -> bool`

Tests for `self` and `other` values to be equal, and is used by `==`.

#### `ne(&self, other: &Rhs) -> bool`

Tests for `!=`.

### `impl<S: BuildHasher> Eq for HyperLogLog<S>`

### Auto Trait Implementations

### `impl<S> Freeze for HyperLogLog<S>`
where S: Freeze,

### `impl<S> RefUnwindSafe for HyperLogLog<S>`
where S: RefUnwindSafe,

### `impl<S> Send for HyperLogLog<S>`
where S: Send,

### `impl<S> Sync for HyperLogLog<S>`
where S: Sync,

### `impl<S> Unpin for HyperLogLog<S>`
where S: Unpin,

### `impl<S> UnwindSafe for HyperLogLog<S>`
where S: UnwindSafe,

### Blanket Implementations

### `impl<T> Any for T`
where T: 'static + ?Sized,

#### `type_id(&self) -> TypeId`

Gets the `TypeId` of `self`.
```

--------------------------------

### Use HyperLogLogPlus for sparse representation

Source: https://docs.rs/hyperloglockless/0.5.0/index.html

Employ the sparse representation which automatically transitions to dense mode as memory usage grows.

```rust
use hyperloglockless::HyperLogLogPlus;

let mut hll = HyperLogLogPlus::new(14);
hll.insert(&'🦀');
hll.insert_all('a'..='z');
```

--------------------------------

### Query and Merge HyperLogLogPlus

Source: https://docs.rs/hyperloglockless/0.5.0/src/hyperloglockless/sparse.rs.html

Methods to retrieve cardinality estimates, check internal representation, and merge instances.

```rust
pub fn count(&mut self) -> usize {
        self.raw_count() as usize
    }
```

```rust
pub fn raw_count(&mut self) -> f64 {
        match self.sparse.as_mut() {
            Some(s) => s.count() as f64,
            _ => self.dense.as_ref().unwrap().raw_count(),
        }
    }
```

```rust
pub fn is_sparse(&self) -> bool {
        self.sparse.is_some()
    }
```

```rust
pub fn precision(&self) -> u8 {
        match self.sparse.as_ref() {
            Some(s) => s.precision,
            _ => self.dense.as_ref().unwrap().precision as u8,
        }
    }
```

```rust
pub fn union(&mut self, other: &Self) -> Result<(), Error> {
        if self.precision() != other.precision() {
            return Err(Error::IncompatibleLength);
        }
        match (self.is_sparse(), other.is_sparse()) {
            (true, true) => self
                .sparse
                .as_mut()
                .unwrap()
                .union(other.sparse.as_ref().unwrap()),
            (false, false) => self
                .dense
                .as_mut()
                .unwrap()
        }
```

--------------------------------

### Clone AtomicHyperLogLog

Source: https://docs.rs/hyperloglockless/0.5.0/hyperloglockless/struct.AtomicHyperLogLog.html

Create a deep copy of an AtomicHyperLogLog instance.

```rust
fn clone(&self) -> Self
```

--------------------------------

### Construct from Low-Level Parts

Source: https://docs.rs/hyperloglockless/0.5.0/hyperloglockless/struct.AtomicHyperLogLog.html

Create an AtomicHyperLogLog instance from its constituent parts, typically used after deserialization.

```rust
pub fn from_parts( registers: Box<[AtomicU8]>, hasher: S, zeros: usize, sum: f64, ) -> Self
```
Tokens:
Raw