### Install maturin and build from source Source: https://github.com/moritzmucha/fastdigest/blob/main/README.md Install the maturin build tool and then build and install fastdigest from source. This requires Rust to be installed. ```bash pip install maturin ``` ```bash maturin build --release pip install target/wheels/fastdigest-0.12.0-.whl ``` -------------------------------- ### Install fastdigest using pip Source: https://github.com/moritzmucha/fastdigest/blob/main/README.md Install the fastdigest package from PyPI using pip. This is the recommended method for most users. ```bash pip install fastdigest ``` -------------------------------- ### Run Benchmark Script Source: https://github.com/moritzmucha/fastdigest/blob/main/README.md This command executes the benchmark script to compare the performance of fastdigest against other t-digest implementations. Ensure fastdigest and optionally tdigest and pytdigest are installed. ```bash python benchmark.py ``` -------------------------------- ### Get Sum of Ingested Values Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Returns the sum of all values that have been ingested into the TDigest. ```python from fastdigest import TDigest digest = TDigest.from_values(range(11)) print(f"Sum: {digest.sum()}") ``` -------------------------------- ### Get Total Mass of Ingested Values Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Returns the total amount of values ingested into the TDigest. This is equivalent to the number of values if no weighted updates were used. ```python from fastdigest import TDigest digest = TDigest.from_values(range(11)) print(f"Mass: {digest.mass()}") ``` -------------------------------- ### Get Maximum Ingested Value Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Returns the highest value that has been ingested into the TDigest. ```python from fastdigest import TDigest digest = TDigest.from_values(range(11)) print(f"Maximum: {digest.max()}") ``` -------------------------------- ### Get Minimum Ingested Value Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Returns the lowest value that has been ingested into the TDigest. ```python from fastdigest import TDigest digest = TDigest.from_values(range(11)) print(f"Minimum: {digest.min()}") ``` -------------------------------- ### Estimate quantiles and CDF Source: https://github.com/moritzmucha/fastdigest/blob/main/README.md Use the quantile method to estimate the value at a given rank (q) and the cdf method to find the rank of a given value. ```python digest = TDigest.from_values(range(1001)) print("99th percentile:", digest.quantile(0.99)) print("cdf(990) =", digest.cdf(990)) ``` -------------------------------- ### Initialize TDigest instances Source: https://github.com/moritzmucha/fastdigest/blob/main/README.md Create a new TDigest instance or initialize one directly from a sequence of values. ```python from fastdigest import TDigest digest = TDigest() digest = TDigest.from_values([2.71, 3.14, 1.42]) ``` -------------------------------- ### self.update(x, w=None) Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Updates a digest in-place with a single value `x`, with an optional weight `w`. This method is efficient for small, ad-hoc updates. ```APIDOC ## self.update(x, w=None) ### Description Updates a digest in-place with a single value `x`, with optional weight `w`. ### Method `update(x, w=None)` ### Parameters * **x** (float) - Required - The value to add to the digest. * **w** (float) - Optional - The weight of the value (default is None, implying a weight of 1). ### Request Example ```python from fastdigest import TDigest digest = TDigest.from_values([1, 2, 3, 4, 5, 6]) digest.update(7) digest.update(42, w=5.0) print(f"{digest}: {digest.n_values} values, combined weight of {digest.mass()}") ``` ### Response None. The digest is updated in-place. ``` -------------------------------- ### Estimate Quantiles using quantile_vec() Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Estimates values at multiple quantiles (between 0 and 1) efficiently. Returns results as a list. ```python from fastdigest import TDigest digest = TDigest.from_values(range(41)) results = digest.quantile_vec([0.25, 0.5, 0.75]) print(results) ``` -------------------------------- ### Initialize TDigest Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Creates a new TDigest instance. The max_centroids parameter controls compression and precision. ```python from fastdigest import TDigest digest = TDigest() print(digest) ``` -------------------------------- ### Create TDigest from Bytes Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Restores a TDigest instance from serialized binary data. Ensure the file is opened in binary read mode. ```python with open("digest.bin", "rb") as f: restored = TDigest.from_bytes(f.read()) print(f"{restored}: {len(restored)} centroids from {restored.n_values} values") ``` -------------------------------- ### self.median() Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Estimates the median value. This is an alias for `quantile(0.5)`. ```APIDOC ## self.median() ### Description Estimates the median value. ### Method `median()` ### Parameters None ### Request Example ```python digest = TDigest.from_values(normally_distributed_data) print(f"Median: {digest.median():.3f}") ``` ### Response * **float** - The estimated median value. ``` -------------------------------- ### Estimate Median using median() Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Estimates the median value of the distribution. This is an alias for quantile(0.5). ```python from fastdigest import TDigest # Assuming normally_distributed_data is pre-defined digest = TDigest.from_values(normally_distributed_data) print(f"Median: {digest.median():.3f}") ``` -------------------------------- ### From Dictionary Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Creates a new TDigest instance from a dictionary representation. ```APIDOC ## TDigest.from_dict(tdigest_dict) ### Description Creates a new TDigest instance from the `tdigest_dict`. Static method. ### Method `from_dict` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **tdigest_dict** (dict) - A dictionary representing a TDigest. ``` -------------------------------- ### self.quantile(q) Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Estimates the value at a given quantile `q` (a float between 0 and 1). This is the inverse function of `cdf(x)`. ```APIDOC ## self.quantile(q) ### Description Estimates the value at a specified quantile `q`. The quantile `q` must be a float between 0 and 1, inclusive. This function is the inverse of the cumulative distribution function (`cdf`). ### Method Instance method ### Parameters #### Quantile - **q** (float) - Required - The quantile to query, must be between 0.0 and 1.0. ### Request Example ```python from fastdigest import TDigest import numpy as np data = np.random.normal(0, 1, 10000) digest = TDigest.from_values(data) median_value = digest.quantile(0.5) percentile_99 = digest.quantile(0.99) print(f"Median: {median_value:.3f}") print(f"99th percentile: {percentile_99:.3f}") ``` ### Response #### Success Response - **quantile_value** (float) - The estimated value at the specified quantile `q`. #### Response Example ``` Median: 0.001 99th percentile: 2.274 ``` ``` -------------------------------- ### Serialize TDigest to Binary Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Returns a serialized binary representation of the TDigest. This method is significantly faster and more memory-efficient than converting to a dictionary. ```python digest = TDigest.from_values(range(101), max_centroids=3) with open("digest.bin", "wb") as f: f.write(digest.to_bytes()) ``` -------------------------------- ### TDigest.from_values(x, w=None) Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Creates a TDigest directly from a sequence of numeric values `x`. Optional weights `w` can be provided as a sequence or a scalar. ```APIDOC ## TDigest.from_values(x, w=None) ### Description Creates a TDigest instance directly from a sequence of numeric values `x`. Optional weights `w` can be provided as a sequence of the same length as `x`, or as a scalar value to be applied to all elements. ### Method Static method ### Parameters #### Input Values - **x** (sequence of numbers) - Required - The sequence of numeric values to initialize the TDigest with. - **w** (sequence of numbers or scalar) - Optional - The weights for the values in `x`. If a scalar, it's applied to all values. Defaults to `None` (all weights are 1). ### Request Example ```python import numpy as np from fastdigest import TDigest digest_list = TDigest.from_values([2.71, 3.14, 1.42]) digest_array = TDigest.from_values(np.random.random(1000)) digest_weighted = TDigest.from_values([1, 2, 3], w=[1, 2, 3]) digest_scalar_weight = TDigest.from_values([1, 2, 3], w=2.0) ``` ### Response #### Success Response - **TDigest instance**: A new TDigest object initialized with the provided values and weights. ``` -------------------------------- ### Create TDigest from Values Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Creates a TDigest directly from a sequence of numeric values, optionally with weights. Supports lists, tuples, ranges, NumPy arrays, and scalar weights. ```python import numpy as np from fastdigest import TDigest digest = TDigest.from_values([2.71, 3.14, 1.42]) # from list digest = TDigest.from_values((42,)) # from tuple digest = TDigest.from_values(range(101)) # from range digest = TDigest.from_values([1, 2], w=[1, 2]) # weighted individually digest = TDigest.from_values([1, 2], w=2.0) # weighted with scalar data = np.random.random(10_000) digest = TDigest.from_values(data) # from NumPy array print(f"{digest}: {len(digest)} centroids from {digest.n_values} values") ``` -------------------------------- ### self.quantile_vec(q) Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Estimates the values at the quantiles `q` (between 0 and 1). This method takes a sequence of quantiles and returns a list of corresponding values. ```APIDOC ## self.quantile_vec(q) ### Description Estimates the values at the quantiles `q` (between 0 and 1). ### Method `quantile_vec(q)` ### Parameters * **q** (list of floats) - Required - A list of quantiles (values between 0 and 1). ### Request Example ```python from fastdigest import TDigest digest = TDigest.from_values(range(41)) results = digest.quantile_vec([0.25, 0.5, 0.75]) print(results) ``` ### Response * **list of floats** - A list of estimated values corresponding to the input quantiles. ``` -------------------------------- ### self.min() Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Returns the minimum value that has been ingested into the TDigest. ```APIDOC ## self.min() ### Description Returns the minimum value among all the values that have been ingested into the TDigest. ### Method Instance method ### Parameters None ### Request Example ```python from fastdigest import TDigest digest = TDigest.from_values(range(11)) print(f"Minimum: {digest.min()}") ``` ### Response #### Success Response - **minimum** (float) - The smallest ingested value. #### Response Example ``` Minimum: 0.0 ``` ``` -------------------------------- ### Convert TDigest to Dictionary and Back Source: https://github.com/moritzmucha/fastdigest/blob/main/README.md Demonstrates how to serialize a TDigest object to a dictionary using `to_dict()` and then deserialize it back into a TDigest instance using `from_dict()`. This is useful for saving and loading digest states. ```python from fastdigest import TDigest import json digest = TDigest.from_values(range(101)) td_dict = digest.to_dict() print(json.dumps(td_dict, indent=2)) restored = TDigest.from_dict(td_dict) ``` -------------------------------- ### To Bytes Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Returns a serialized binary representation of the TDigest. ```APIDOC ## self.to_bytes() ### Description Returns a serialized binary representation of the TDigest. ### Method `to_bytes` ### Parameters None ### Response #### Success Response (200) - (bytes) - A binary representation of the TDigest. ``` -------------------------------- ### self.sum() Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Returns the sum of all ingested values, considering their weights. If values were ingested without weights, this is the arithmetic sum. ```APIDOC ## self.sum() ### Description Returns the sum of all ingested values, taking into account any weights applied during ingestion. If values were ingested without weights, this represents the simple arithmetic sum of those values. ### Method Instance method ### Parameters None ### Request Example ```python from fastdigest import TDigest digest = TDigest.from_values(range(11)) print(f"Sum: {digest.sum()}") ``` ### Response #### Success Response - **sum** (float) - The sum of all ingested values. #### Response Example ``` Sum: 55.0 ``` ``` -------------------------------- ### TDigest.copy Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Returns a shallow copy of the TDigest instance. ```APIDOC ## self.copy() ### Description Returns a copy of the instance. ### Method Instance method. ### Parameters None ### Request Example ```python from fastdigest import TDigest digest = TDigest.from_values(range(101)) copy_digest = digest.copy() print(f"Original digest: {digest}") print(f"Copied digest: {copy_digest}") ``` ### Response #### Success Response (200) * **TDigest instance** - A new TDigest object that is a copy of the original instance. #### Response Example ``` Original digest: TDigest(max_centroids=3): 3 centroids from 101 values Copied digest: TDigest(max_centroids=3): 3 centroids from 101 values ``` ``` -------------------------------- ### Estimate Percentiles using percentile() Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Estimates the value at a specified percentile. This is an alias for the quantile() method. ```python from fastdigest import TDigest # Assuming normally_distributed_data is pre-defined digest = TDigest.from_values(normally_distributed_data) print(f" Median: {digest.percentile(50):.3f}") print(f"99th percentile: {digest.percentile(99):.3f}") ``` -------------------------------- ### Batch Update Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Updates a digest in-place by merging a sequence of many values `x` at once. The optional weights `w` can be either a sequence of the same length as `x`, or a scalar that will be used as the weight for the entire batch. ```APIDOC ## self.batch_update(x, w=None) ### Description Updates a digest in-place by merging a sequence of many values `x` at once. The optional weights `w` can be either a sequence of the same length as `x`, or a scalar that will be used as the weight for the entire batch. ### Method `batch_update` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **x** (sequence) - The sequence of values to update. - **w** (scalar or sequence, optional) - The weights for the values in `x`. ``` -------------------------------- ### self.var() Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Estimates the population variance of the distribution. ```APIDOC ## self.var() ### Description Estimates the population variance of the distribution. ### Method `var()` ### Parameters None ### Request Example ```python import numpy as np normally_distributed_data = np.random.normal(0, 1, 10_000) digest = TDigest.from_values(normally_distributed_data) print(f"Variance: {digest.var():.3f}") ``` ### Response * **float** - The estimated population variance. ``` -------------------------------- ### To Dictionary Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Returns a dictionary representation of the TDigest. ```APIDOC ## self.to_dict() ### Description Returns a dict representation of the TDigest. ### Method `to_dict` ### Parameters None ### Response #### Success Response (200) - **max_centroids** (int) - The maximum number of centroids. - **mass** (float) - The total mass (sum of weights). - **sum** (float) - The sum of all values. - **min** (float) - The minimum value. - **max** (float) - The maximum value. - **n_values** (int) - The total number of values. - **centroids** (list of dict) - A list of centroid objects, each with 'm' (mean) and 'c' (count/weight). ``` -------------------------------- ### Estimate Population Variance using var() Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Estimates the population variance of the distribution. Requires numpy for data generation. ```python import numpy as np from fastdigest import TDigest normally_distributed_data = np.random.normal(0, 1, 10_000) digest = TDigest.from_values(normally_distributed_data) print(f"Variance: {digest.var():.3f}") ``` -------------------------------- ### self.mean() Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Calculates the arithmetic mean of the distribution as `sum() / mass()`. ```APIDOC ## self.mean() ### Description Calculates the arithmetic mean of the distribution. ### Method `mean()` ### Parameters None ### Request Example ```python digest = TDigest.from_values(range(11)) print(f"Mean value: {digest.mean()}") ``` ### Response * **float** - The calculated arithmetic mean. ``` -------------------------------- ### Estimate Interquartile Range using iqr() Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Estimates the interquartile range (IQR) of the distribution. This is an alias for quantile(0.75) - quantile(0.25). ```python from fastdigest import TDigest # Assuming normally_distributed_data is pre-defined digest = TDigest.from_values(normally_distributed_data) print(f"IQR: {digest.iqr():.3f}") ``` -------------------------------- ### Serialize TDigest to Dictionary Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Returns a dictionary representation of the TDigest, including its configuration and centroid data. This format is useful for inspection and can be used to reconstruct the TDigest. ```python import json from fastdigest import TDigest digest = TDigest.from_values(range(101), max_centroids=3) tdigest_dict = digest.to_dict() print(json.dumps(tdigest_dict, indent=2)) ``` -------------------------------- ### Calculate Arithmetic Mean using mean() Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Calculates the arithmetic mean of the distribution using sum() / mass(). ```python from fastdigest import TDigest digest = TDigest.from_values(range(11)) print(f"Mean value: {digest.mean()}") ``` -------------------------------- ### TDigest() Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Creates a new TDigest instance. The `max_centroids` parameter controls compression, balancing memory footprint, speed, and precision. A value of 0 disables compression. ```APIDOC ## TDigest() ### Description Creates a new TDigest instance. The `max_centroids` parameter controls how large the data structure is allowed to grow, affecting compression, memory footprint, and precision. Setting `max_centroids` to 0 disables compression. ### Parameters #### Initialization Parameters - **max_centroids** (int) - Optional - The maximum number of centroids to use for compression. Defaults to 1000. ### Request Example ```python from fastdigest import TDigest digest = TDigest(max_centroids=500) print(digest) ``` ### Response #### Success Response - **TDigest instance**: A new TDigest object is returned. #### Response Example ``` TDigest(max_centroids=500): 0 centroids from 0 values ``` ``` -------------------------------- ### self.is_normal() Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Performs a Kolmogorov-Smirnov test to determine if the ingested data follows a normal distribution. An optional `alpha` argument can adjust the significance threshold (default is 0.05). ```APIDOC ## self.is_normal(alpha=0.05) ### Description Performs a Kolmogorov-Smirnov test to determine if the ingested data follows a normal distribution. ### Method `is_normal(alpha=0.05)` ### Parameters * **alpha** (float) - Optional - The significance threshold for the test (default is 0.05). ### Request Example ```python import numpy as np normally_distributed_data = np.random.normal(0, 1, 10_000) normal_digest = TDigest.from_values(normally_distributed_data) skewed_data = np.random.standard_gamma(5, 10_000) skewed_digest = TDigest.from_values(skewed_data) print(normal_digest.is_normal()) print(skewed_digest.is_normal()) ``` ### Response * **bool** - True if the data appears to follow a normal distribution, False otherwise. ``` -------------------------------- ### self.iqr() Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Estimates the interquartile range (IQR). This is an alias for `quantile(0.75) - quantile(0.25)`. ```APIDOC ## self.iqr() ### Description Estimates the interquartile range (IQR). ### Method `iqr()` ### Parameters None ### Request Example ```python digest = TDigest.from_values(normally_distributed_data) print(f"IQR: {digest.iqr():.3f}") ``` ### Response * **float** - The estimated interquartile range. ``` -------------------------------- ### self.probability(x1, x2) Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Estimates the probability of finding a value in the interval [`x1`, `x2`]. This is an alias for `cdf(x2) - cdf(x1)`. ```APIDOC ## self.probability(x1, x2) ### Description Estimates the probability of finding a value in the interval [`x1`, `x2`]. ### Method `probability(x1, x2)` ### Parameters * **x1** (float) - Required - The lower bound of the interval. * **x2** (float) - Required - The upper bound of the interval. ### Request Example ```python digest = TDigest.from_values(normally_distributed_data) prob = digest.probability(-2.0, 2.0) print(f"Probability of value between ±2: {100 * prob:.1f}%") ``` ### Response * **float** - The estimated probability of a value falling within the specified interval. ``` -------------------------------- ### self.std() Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Estimates the standard deviation of the distribution. This is an alias for `var() ** 0.5`. ```APIDOC ## self.std() ### Description Estimates the standard deviation of the distribution. ### Method `std()` ### Parameters None ### Request Example ```python digest = TDigest.from_values(normally_distributed_data) print(f"Standard deviation: {digest.std():.3f}") ``` ### Response * **float** - The estimated standard deviation. ``` -------------------------------- ### self.percentile(p) Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Estimates the value at the `p`th percentile. This is an alias for `quantile(p/100)`. ```APIDOC ## self.percentile(p) ### Description Estimates the value at the `p`th percentile. ### Method `percentile(p)` ### Parameters * **p** (float) - Required - The percentile to estimate (e.g., 50 for median, 99 for 99th percentile). ### Request Example ```python digest = TDigest.from_values(normally_distributed_data) print(f"99th percentile: {digest.percentile(99):.3f}") ``` ### Response * **float** - The estimated value at the `p`th percentile. ``` -------------------------------- ### Estimate Probability in an Interval using probability() Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Estimates the probability of finding a value within a specified interval [x1, x2]. This is an alias for cdf(x2) - cdf(x1). ```python from fastdigest import TDigest # Assuming normally_distributed_data is pre-defined digest = TDigest.from_values(normally_distributed_data) prob = digest.probability(-2.0, 2.0) prob_pct = 100 * prob print(f"Probability of value between ±2: {prob_pct:.1f}%") ``` -------------------------------- ### TDigest.from_bytes Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Creates a new TDigest instance from serialized binary data. This is a static method. ```APIDOC ## TDigest.from_bytes(data) ### Description Creates a new TDigest instance from the serialized binary `data`. ### Method Static method. ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body * **data** (bytes) - Required - The binary data representing a TDigest. ### Request Example ```python with open("digest.bin", "rb") as f: restored = TDigest.from_bytes(f.read()) print(f"{restored}: {len(restored)} centroids from {restored.n_values} values") ``` ### Response #### Success Response (200) * **TDigest instance** - A new TDigest object initialized from the provided data. #### Response Example ``` TDigest(max_centroids=3): 3 centroids from 101 values ``` ``` -------------------------------- ### Batch Update TDigest with Values Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Updates a TDigest in-place by merging a sequence of many values at once. Supports optional weights for individual values or the entire batch. Can handle numpy arrays and empty sequences. ```python import numpy as np digest = TDigest() digest.batch_update([1, 2, 3, 4, 5, 6]) digest.batch_update(np.arange(7, 11)) # using numpy array digest.batch_update([1, 2], w=[1, 2]) # weighted individually digest.batch_update([1, 2], w=2.0) # weighted with scalar digest.batch_update([5]) # can also just be one value ... digest.batch_update([]) # ... or empty print(f"{digest}: {digest.n_values} values, combined weight of {digest.mass()}") ``` -------------------------------- ### Calculate Trimmed Mean using trimmed_mean() Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Estimates the truncated mean between two specified quantiles (q1, q2). Useful for mitigating the effect of outliers. ```python from fastdigest import TDigest data = list(range(11)) data[-1] = 100_000 # extreme outlier digest = TDigest.from_values(data) mean = digest.mean() trimmed_mean = digest.trimmed_mean(0.1, 0.9) print(f" Mean: {mean}") print(f"Trimmed mean: {trimmed_mean}") ``` -------------------------------- ### Estimate Quantile Value Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Estimates the value at a given quantile `q` (0 to 1). This is the inverse function of `cdf(x)`. Use `quantile_vec(q)` for vectorized operations. ```python from fastdigest import TDigest import numpy as np normally_distributed_data = np.random.normal(0, 1, 10_000) digest = TDigest.from_values(normally_distributed_data) print(f" Median: {digest.quantile(0.5):.3f}") print(f"99th percentile: {digest.quantile(0.99):.3f}") ``` -------------------------------- ### self.mass() Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Returns the total mass (sum of weights) of all values ingested into the TDigest. If no weighted updates were used, this is equivalent to the number of values. ```APIDOC ## self.mass() ### Description Returns the total mass of all values ingested into the TDigest. This is equivalent to the sum of weights of all ingested values. If no weights were explicitly provided during updates, it represents the total count of ingested values. ### Method Instance method ### Parameters None ### Request Example ```python from fastdigest import TDigest digest = TDigest.from_values(range(11)) print(f"Mass: {digest.mass()}") ``` ### Response #### Success Response - **mass** (float) - The total mass of the ingested values. #### Response Example ``` Mass: 11.0 ``` ``` -------------------------------- ### Update TDigest with a Single Value using update() Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Updates a TDigest instance in-place with a single value 'x' and an optional weight 'w'. This method is optimized for small, ad-hoc updates. ```python from fastdigest import TDigest digest = TDigest.from_values([1, 2, 3, 4, 5, 6]) digest.update(7) digest.update(42, w=5.0) print(f"{digest}: {digest.n_values} values, combined weight of {digest.mass()}") ``` -------------------------------- ### Calculate mean and trimmed mean Source: https://github.com/moritzmucha/fastdigest/blob/main/README.md Compute the arithmetic mean or the trimmed mean between specified quantiles. Useful for handling outliers. ```python data = list(range(11)) # numbers 1-10 data[-1] = 100_000 # extreme outlier digest = TDigest.from_values(data) print(f" Mean: {digest.mean()}") print(f"Trimmed mean: {digest.trimmed_mean(0.1, 0.9)}") ``` -------------------------------- ### self.trimmed_mean(q1, q2) Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Estimates the truncated mean between the two quantiles `q1` and `q2`. ```APIDOC ## self.trimmed_mean(q1, q2) ### Description Estimates the truncated mean between the two quantiles `q1` and `q2`. ### Method `trimmed_mean(q1, q2)` ### Parameters * **q1** (float) - Required - The lower quantile (between 0 and 1). * **q2** (float) - Required - The upper quantile (between 0 and 1). ### Request Example ```python data = list(range(11)) data[-1] = 100_000 # extreme outlier digest = TDigest.from_values(data) trimmed_mean = digest.trimmed_mean(0.1, 0.9) print(f"Trimmed mean: {trimmed_mean}") ``` ### Response * **float** - The estimated trimmed mean. ``` -------------------------------- ### Estimate Cumulative Probability using cdf() Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Estimates the relative rank (cumulative probability) of a given value. This is the inverse function of quantile(q). ```python from fastdigest import TDigest # Assuming normally_distributed_data is pre-defined digest = TDigest.from_values(normally_distributed_data) print(f"cdf(0.0) = {digest.cdf(0.0):.3f}") print(f"cdf(1.0) = {digest.cdf(1.0):.3f}") ``` -------------------------------- ### Estimate Median Absolute Deviation using mad() Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Estimates the median absolute deviation (MAD) of the distribution. ```python from fastdigest import TDigest digest = TDigest.from_values(range(101)) print(f"MAD: {digest.mad()}") ``` -------------------------------- ### TDigest.is_empty Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Checks if the TDigest instance is empty, meaning no data has been ingested yet. ```APIDOC ## self.is_empty() ### Description Returns `True` if no data has been ingested yet. ### Method Instance method. ### Parameters None ### Request Example ```python from fastdigest import TDigest empty_digest = TDigest() print(f"Is empty digest empty? {empty_digest.is_empty()}") digest = TDigest.from_values(range(101)) print(f"Is populated digest empty? {digest.is_empty()}") ``` ### Response #### Success Response (200) * **bool** - `True` if the digest is empty, `False` otherwise. #### Response Example ``` Is empty digest empty? True Is populated digest empty? False ``` ``` -------------------------------- ### self.max() Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Returns the maximum value that has been ingested into the TDigest. ```APIDOC ## self.max() ### Description Returns the maximum value among all the values that have been ingested into the TDigest. ### Method Instance method ### Parameters None ### Request Example ```python from fastdigest import TDigest digest = TDigest.from_values(range(11)) print(f"Maximum: {digest.max()}") ``` ### Response #### Success Response - **maximum** (float) - The largest ingested value. #### Response Example ``` Maximum: 10.0 ``` ``` -------------------------------- ### Update TDigest with single and batch values Source: https://github.com/moritzmucha/fastdigest/blob/main/README.md Add values to a TDigest using batch_update for sequences or update for individual values. Supports weighted updates. ```python digest = TDigest() digest.batch_update([0, 1, 2]) digest.update(3) ``` ```python digest.update(4, 0.4) # single weighted value digest.batch_update([5, 6, 7], w=1.5) # whole-batch weight digest.batch_update([8, 9], [1.0, 2.0]) # per-value weights ``` -------------------------------- ### self.cdf(x) Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Estimates the relative rank (cumulative probability) of the value `x`. This is the inverse function of `quantile(q)`. ```APIDOC ## self.cdf(x) ### Description Estimates the relative rank (cumulative probability) of the value `x`. ### Method `cdf(x)` ### Parameters * **x** (float) - Required - The value for which to estimate the cumulative probability. ### Request Example ```python digest = TDigest.from_values(normally_distributed_data) print(f"cdf(0.0) = {digest.cdf(0.0):.3f}") ``` ### Response * **float** - The estimated cumulative probability of the value `x`. ``` -------------------------------- ### Estimate Standard Deviation using std() Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Estimates the standard deviation of the distribution. This is an alias for var() ** 0.5. ```python import numpy as np from fastdigest import TDigest normally_distributed_data = np.random.normal(0, 1, 10_000) digest = TDigest.from_values(normally_distributed_data) print(f"Standard deviation: {digest.std():.3f}") ``` -------------------------------- ### Test for Normal Distribution using is_normal() Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Performs a Kolmogorov-Smirnov test to determine if the ingested data follows a normal distribution. The significance threshold can be adjusted with the 'alpha' argument. ```python import numpy as np from fastdigest import TDigest normally_distributed_data = np.random.normal(0, 1, 10_000) normal_digest = TDigest.from_values(normally_distributed_data) skewed_data = np.random.standard_gamma(5, 10_000) skewed_digest = TDigest.from_values(skewed_data) print(normal_digest.is_normal()) print(skewed_digest.is_normal()) ``` -------------------------------- ### self.mad() Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Estimates the median absolute deviation (MAD) of the distribution. ```APIDOC ## self.mad() ### Description Estimates the median absolute deviation (MAD) of the distribution. ### Method `mad()` ### Parameters None ### Request Example ```python digest = TDigest.from_values(range(101)) print(f"MAD: {digest.mad()}") ``` ### Response * **float** - The estimated median absolute deviation. ``` -------------------------------- ### Estimate Cumulative Probabilities using cdf_vec() Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Estimates the relative ranks (cumulative probabilities) for a sequence of values efficiently. Returns results as a list. ```python from fastdigest import TDigest digest = TDigest.from_values(range(41)) results = digest.cdf_vec([10, 20, 30]) print(results) ``` -------------------------------- ### TDigest.equals Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Compares two TDigest instances for equality. Returns True if they have identical centroids, properties, and max_centroids. ```APIDOC ## self.equals(other) ### Description Returns `True` if both TDigests have identical centroids, properties and `max_centroids`, otherwise `False`. ### Method Instance method. ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body * **other** (TDigest) - Required - The other TDigest instance to compare against. ### Request Example ```python from fastdigest import TDigest digest = TDigest.from_values(range(101)) restored = TDigest.from_dict(digest.to_dict()) print(f"digest == restored: {digest.equals(restored)}") ``` ### Response #### Success Response (200) * **bool** - `True` if the digests are equal, `False` otherwise. #### Response Example ``` digest == restored: True ``` ### Raises * **TypeError** - If `other` is not a TDigest. ``` -------------------------------- ### self.cdf_vec(x) Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Estimates the relative ranks (cumulative probabilities) of the values `x`. This method takes a sequence of values and returns a list of their cumulative probabilities. ```APIDOC ## self.cdf_vec(x) ### Description Estimates the relative ranks (cumulative probabilities) of the values `x`. ### Method `cdf_vec(x)` ### Parameters * **x** (list of floats) - Required - A list of values for which to estimate cumulative probabilities. ### Request Example ```python digest = TDigest.from_values(range(41)) results = digest.cdf_vec([10, 20, 30]) print(results) ``` ### Response * **list of floats** - A list of estimated cumulative probabilities for the input values. ``` -------------------------------- ### Deserialize TDigest from Dictionary Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Creates a new TDigest instance from a dictionary representation. This allows for restoring a TDigest from its serialized form. ```python restored = TDigest.from_dict(tdigest_dict) print(f"{restored}: {len(restored)} centroids from {restored.n_values} values") ``` -------------------------------- ### Merge Digests Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Creates a new TDigest instance by merging two existing digests. ```APIDOC ## self.merge(other) ### Description Creates a new TDigest instance from two digests. Alias: [`+` operator](#magic-methods--operators) ### Method `merge` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **other** (TDigest) - The other TDigest instance to merge with. ``` -------------------------------- ### Compare TDigests for Equality Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Checks if two TDigest instances have identical centroids, properties, and max_centroids. This method is strict and raises a TypeError for non-TDigest types. ```python from fastdigest import TDigest digest = TDigest.from_values(range(101)) restored = TDigest.from_dict(digest.to_dict()) print(f"digest == restored: {digest.equals(restored)}") ``` -------------------------------- ### Merge Multiple TDigests Efficiently Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Creates a new TDigest instance by efficiently merging an iterable of TDigests in a single operation. An optional `max_centroids` argument can be provided; otherwise, the largest `max_centroids` from the input digests is used. ```python from fastdigest import merge_all # create a list of 10 digests from (non-overlapping) ranges partial_digests = [] for i in range(10): partial_data = range(i * 10, (i+1) * 10) digest = TDigest.from_values(partial_data, max_centroids=30) partial_digests.append(digest) # merge all digests and create a new instance merged = merge_all(partial_digests) print(f"{merged}: {len(merged)} centroids from {merged.n_values} values") ``` -------------------------------- ### Merge All Digests Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Creates a new TDigest instance from an iterable of digests that are efficiently merged in a single operation. ```APIDOC ## merge_all(digests) ### Description Creates a new TDigest instance from an iterable of `digests` that are efficiently merged in a single operation. Module-level function. ### Method `merge_all` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **digests** (iterable) - An iterable of TDigest instances to merge. - **max_centroids** (int, optional) - The maximum number of centroids for the new TDigest. If None, the largest `max_centroids` of the input digests is used. ``` -------------------------------- ### In-place Merge Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Updates a digest in-place with the centroids from another TDigest. ```APIDOC ## self.merge_inplace(other) ### Description Updates a digest in-place with the centroids from an `other` TDigest. Alias: [`+=` operator](#magic-methods--operators) ### Method `merge_inplace` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **other** (TDigest) - The other TDigest instance to merge into the current one. ``` -------------------------------- ### Merge TDigest objects Source: https://github.com/moritzmucha/fastdigest/blob/main/README.md Combine TDigest instances using the '+' operator for a new instance or '+=' for in-place merging. The merge_all function merges an iterable of digests. ```python digest1 = TDigest.from_values(range(20)) digest2 = TDigest.from_values(range(20, 51)) digest3 = TDigest.from_values(range(51, 101)) digest1 += digest2 merged_new = digest1 + digest3 ``` ```python from fastdigest import TDigest, merge_all digests = [TDigest.from_values(range(i, i+10)) for i in range(0, 100, 10)] merged = merge_all(digests) ``` -------------------------------- ### Merge TDigests In-Place Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Updates a TDigest in-place by merging centroids from another TDigest. The `max_centroids` parameter of the calling TDigest remains unchanged. This is an alias for the '+=' operator. ```python digest = TDigest.from_values(range(50), max_centroids=30) tmp_digest = TDigest.from_values(range(50, 101)) digest += tmp_digest # alias for: digest.merge_inplace(tmp_digest) print(f"{digest}: {len(digest)} centroids from {digest.n_values} values") ``` -------------------------------- ### Merge Two TDigests Source: https://github.com/moritzmucha/fastdigest/blob/main/API.md Creates a new TDigest instance by merging two existing TDigests. The resulting digest uses the larger of the two `max_centroids` parameters. This is an alias for the '+' operator. ```python from fastdigest import TDigest digest1 = TDigest.from_values(range(50), max_centroids=1000) digest2 = TDigest.from_values(range(50, 101), max_centroids=3) merged = digest1 + digest2 # alias for digest1.merge(digest2) print(f"{merged}: {len(merged)} centroids from {merged.n_values} values") ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.