### Import libraries for soft clustering example

Source: https://hdbscan.readthedocs.io/en/latest/soft_clustering.html

Imports sklearn datasets, TSNE, matplotlib, seaborn, and numpy for the soft clustering example.

```python
from sklearn import datasets
from sklearn.manifold import TSNE
import matplotlib.pyplot as plt
import seaborn as sns
import numpy as np
```

--------------------------------

### Import libraries and set up plotting for HDBSCAN example

Source: https://hdbscan.readthedocs.io/en/latest/how_hdbscan_works.html

Loads numpy, matplotlib, seaborn, and sklearn.datasets, and configures plotting style and plot keyword arguments for the HDBSCAN demonstration.

```python
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import sklearn.datasets as data
%matplotlib inline
sns.set_context('poster')
sns.set_style('white')
sns.set_color_codes()
plot_kwds = {'alpha' : 0.5, 's' : 80, 'linewidths':0}
```

--------------------------------

### Generate sample clustering data with sklearn

Source: https://hdbscan.readthedocs.io/en/latest/how_hdbscan_works.html

Creates a dataset of 100 points combining two moons and two blobs, then plots the data. This provides a small, illustrative dataset for the HDBSCAN example.

```python
moons, _ = data.make_moons(n_samples=50, noise=0.05)
blobs, _ = data.make_blobs(n_samples=50, centers=[(-0.75,2.25), (1.0, 2.0)], cluster_std=0.25)
test_data = np.vstack([moons, blobs])
plt.scatter(test_data.T[0], test_data.T[1], color='b', **plot_kwds)
```

--------------------------------

### Example outlier scores output

Source: https://hdbscan.readthedocs.io/en/latest/outlier_detection.html

Sample output of the outlier_scores_ array.

```python
array([ 0.14791852,  0.14116731,  0.09171929, ...,  0.62050534,
        0.56749298,  0.20681685])
```

--------------------------------

### Run Mean Shift clustering with bandwidth 0.175

Source: https://hdbscan.readthedocs.io/en/latest/comparing_clustering_algorithms.html

Example call to the Mean Shift clustering algorithm using the sklearn implementation. The bandwidth is set to 0.175 and cluster_all is set to False to allow noise points. This configuration was found to produce a reasonable clustering on the sample data, though it still leaves dense regions as noise and can split natural clusters.

```python
plot_clusters(data, cluster.MeanShift, (0.175,), {'cluster_all':False})
```

--------------------------------

### Import matplotlib for plotting Approximation Graph

Source: https://hdbscan.readthedocs.io/en/latest/_modules/hdbscan/plots.html

Attempts to import matplotlib.pyplot and matplotlib.collections. Raises an ImportError if matplotlib is not installed, instructing the user to install it to plot the Approximation Graph.

```python
try:
    import matplotlib.pyplot as plt
    import matplotlib.collections as mc
except ImportError:
    raise ImportError(
        "You must install the matplotlib library to plot the Approximation Graph."
    )
```

--------------------------------

### Output of cluster labels for manhattan metric

Source: https://hdbscan.readthedocs.io/en/latest/basic_hdbscan.html

Shows the resulting labels array from the previous example, with values mostly 1 and some 0.

```text
array([1, 1, 1, ..., 1, 1, 0])
```

--------------------------------

### Run HDBSCAN with leaf cluster selection

Source: https://hdbscan.readthedocs.io/en/latest/how_to_detect_branches.html

Runs HDBSCAN with cluster_selection_method='leaf' to get more detailed leaf clusters, then plots the labels.

```python
leaf_clusterer = HDBSCAN(min_cluster_size=15, cluster_selection_method='leaf').fit(data)
plot(leaf_clusterer.labels_)
```

--------------------------------

### Example output of cluster labels

Source: https://hdbscan.readthedocs.io/en/latest/basic_hdbscan.html

Shows the array of cluster labels returned by the HDBSCAN fit, with values like 1 and 2. The note explains that the result appears different only due to a different labelling order for the clusters.

```python
array([1, 1, 1, ..., 1, 1, 2])
```

--------------------------------

### Get branch exemplars property

Source: https://hdbscan.readthedocs.io/en/latest/_modules/hdbscan/branches.html

Computes and returns branch exemplars for each cluster. Raises AttributeError if branch exemplars are not available with precomputed distances or if no branches were detected. Uses condensed trees, selected branch IDs, and raw data to extract exemplar points.

```python
        if self._branch_exemplars is not None:
            return self._branch_exemplars
        if self._clusterer._raw_data is None:
            raise AttributeError(
                "Branch exemplars not available with precomputed " "distances."
            )
        if self._condensed_trees is None:
            raise AttributeError("No branches detected; try running fit first.")

        num_clusters = len(self._condensed_trees)
        branch_cluster_trees = [
            branch_tree[branch_tree["child_size"] > 1]
            for branch_tree in self._condensed_trees
        ]
        selected_branch_ids = [
            sorted(branch_tree._select_clusters())
            for branch_tree in self.condensed_trees_
        ]

        self._branch_exemplars = [None] * num_clusters

        for i, points in enumerate(self.cluster_points_):
            selected_branches = selected_branch_ids[i]
            if len(selected_branches) <= (1 if self.label_sides_as_branches else 2):
                continue

            self._branch_exemplars[i] = []
            raw_condensed_tree = self._condensed_trees[i]
            # Group this tree by parent once instead of re-scanning it per leaf
            # (see hdbscan.prediction._group_by_parent).
            group_max_lambda, group_rows = _group_by_parent(raw_condensed_tree)

            for branch in selected_branches:
                _branch_exemplars = np.array([], dtype=np.intp)
                for leaf in recurse_leaf_dfs(branch_cluster_trees[i], np.intp(branch)):
                    leaf_rows = group_rows[leaf]
                    leaf_max_lambda = group_max_lambda[leaf]
                    candidates = raw_condensed_tree["child"][
                        leaf_rows[raw_condensed_tree["lambda_val"][leaf_rows]
                                  == leaf_max_lambda]
                    ]
                    _branch_exemplars = np.hstack([_branch_exemplars, candidates])
                ids = points[_branch_exemplars]
                self._branch_exemplars[i].append(self._clusterer._raw_data[ids, :])

        return self._branch_exemplars
```

--------------------------------

### Fit BranchDetector with custom cluster labels (valid)

Source: https://hdbscan.readthedocs.io/en/latest/how_to_detect_branches.html

Shows how to pass custom cluster labels to the BranchDetector's fit() method, e.g., for DBSCAN labels. The example merges cluster 3 into cluster 2. Custom clusters must be within-cluster-path-connected in HDBSCAN's MST.

```python
# Valid option
custom_labels = clusterer.labels_.copy()
custom_labels[clusterer.labels_ == 3] = 2
branch_detector.fit(clusterer, custom_labels)
plot(branch_detector.labels_)
```

--------------------------------

### to_pandas method: export points and edges as DataFrames

Source: https://hdbscan.readthedocs.io/en/latest/_modules/hdbscan/plots.html

Converts internal point and edge data to pandas DataFrames. Requires pandas to be installed; raises ImportError otherwise. Returns two DataFrames: points with 8 columns and edges with 5 columns.

```python
try:
    from pandas import DataFrame
except ImportError:
    raise ImportError(
        "You must have pandas installed to export pandas DataFrames"
    )

points = DataFrame(self._points)
edges = DataFrame(self._edges)
return points, edges
```

--------------------------------

### Get condensed trees property

Source: https://hdbscan.readthedocs.io/en/latest/_modules/hdbscan/branches.html

Returns a list of CondensedTree objects built from stored condensed trees and branch labels. Raises AttributeError if no condensed trees were generated, instructing to run fit first.

```python
        if self._condensed_trees is None:
            raise AttributeError(
                "No condensed trees were generated; try running fit first."
            )

        return [
            CondensedTree(tree, self.branch_labels_[points])
            for tree, points in zip(
                self._condensed_trees, 
                self.cluster_points_,
            )
        ]
```

--------------------------------

### hdbscan(X, min_cluster_size=5, min_samples=None, alpha=1.0, cluster_selection_epsilon=0.0, cluster_selection_persistence=0.0, max_cluster_size=0, metric='minkowski', p=2, leaf_size=40, algorithm='best', memory=Memory(None, verbose=0), approx_min_span_tree=True, gen_min_span_tree=False, core_dist_n_jobs=4, cluster_selection_method='eom', allow_single_cluster=False, match_reference_implementation=False, cluster_selection_epsilon_max=float('inf'), **kwargs)

Source: https://hdbscan.readthedocs.io/en/latest/_modules/hdbscan/hdbscan_.html

Perform HDBSCAN clustering from a vector array or distance matrix. This is the main entry point for clustering data with HDBSCAN.

```APIDOC
## hdbscan(X, min_cluster_size=5, min_samples=None, alpha=1.0, cluster_selection_epsilon=0.0, cluster_selection_persistence=0.0, max_cluster_size=0, metric='minkowski', p=2, leaf_size=40, algorithm='best', memory=Memory(None, verbose=0), approx_min_span_tree=True, gen_min_span_tree=False, core_dist_n_jobs=4, cluster_selection_method='eom', allow_single_cluster=False, match_reference_implementation=False, cluster_selection_epsilon_max=float('inf'), **kwargs)

### Description
Perform HDBSCAN clustering from a vector array or distance matrix.

### Parameters
- **X** (array or sparse (CSR) matrix of shape (n_samples, n_features), or array of shape (n_samples, n_samples)) - Required - A feature array, or array of distances between samples if `metric='precomputed'`.
- **min_cluster_size** (int, optional, default=5) - The minimum number of samples in a group for that group to be considered a cluster; groupings smaller than this size will be left as noise.
- **min_samples** (int, optional, default=None) - The number of samples in a neighborhood for a point to be considered as a core point. This includes the point itself. Defaults to the min_cluster_size.
- **cluster_selection_epsilon** (float, optional, default=0.0) - A distance threshold. Clusters below this value will be merged. Note that this should not be used if we want to predict the cluster labels for new points in future (e.g. using approximate_predict), as the approximate_predict function is not aware of this argument. This is the minimum epsilon allowed.
- **cluster_selection_persistence** (float, optional, default=0.0) - A persistence threshold. Clusters with a persistence lower than this value will be merged. Note that this should not be used if we want to predict the cluster labels for new points in future (e.g. using approximate_predict), as the approximate_predict function is not aware of this argument.
- **alpha** (float, optional, default=1.0) - A distance scaling parameter as used in robust single linkage.
- **max_cluster_size** (int, optional, default=0) - A limit to the size of clusters returned by the eom algorithm. Has no effect when using leaf clustering (where clusters are usually small regardless) and can also be overridden in rare cases by a high value for cluster_selection_epsilon. Note that this should not be used if we want to predict the cluster labels for new points in future (e.g. using approximate_predict), as the approximate_predict function is not aware of this argument.
- **metric** (string or callable, optional, default='minkowski') - The metric to use when calculating distance between instances in a feature array. If metric is a string or callable, it must be one of the options allowed by metrics.pairwise.pairwise_distances for its metric parameter. If metric is "precomputed", X is assumed to be a distance matrix and must be square.
- **p** (int, optional, default=2) - p value to use if using the minkowski metric.
- **leaf_size** (int, optional, default=40) - Leaf size for trees responsible for fast nearest neighbor queries.

### Returns
- Clustering result object (not explicitly documented in source).
```

--------------------------------

### Import make_blobs and pandas

Source: https://hdbscan.readthedocs.io/en/latest/basic_hdbscan.html

Imports the necessary libraries for generating sample data and handling dataframes.

```python
from sklearn.datasets import make_blobs
import pandas as pd
```

--------------------------------

### outlier_scores_

Source: https://hdbscan.readthedocs.io/en/latest/_modules/hdbscan/hdbscan_.html

Property to get or set the outlier scores. If not set, computes them from the condensed tree if available.

```APIDOC
## outlier_scores_

### Description
Property to get or set the outlier scores. If not set, computes them from the condensed tree if available.

### Method
outlier_scores_ (getter/setter)

### Returns
- **outlier_scores** - The outlier scores.

### Errors
- Raises `AttributeError` if no condensed tree was generated; try running fit first.
```

--------------------------------

### HDBSCAN __init__ method with parameters

Source: https://hdbscan.readthedocs.io/en/latest/_modules/hdbscan/hdbscan_.html

Constructor for the HDBSCAN class, initializing all clustering parameters and internal state variables. Sets default values for min_cluster_size, metric, algorithm, and other options.

```python
    def __init__(
        self,
        min_cluster_size=5,
        min_samples=None,
        cluster_selection_epsilon=0.0,
        cluster_selection_persistence=0.0,
        max_cluster_size=0,
        metric="euclidean",
        alpha=1.0,
        p=None,
        algorithm="best",
        leaf_size=40,
        memory=Memory(None, verbose=0),
        approx_min_span_tree=True,
        gen_min_span_tree=False,
        core_dist_n_jobs=4,
        cluster_selection_method="eom",
        allow_single_cluster=False,
        prediction_data=False,
        branch_detection_data=False,
        match_reference_implementation=False,
        cluster_selection_epsilon_max=float('inf'),
        **kwargs
    ):
        self.min_cluster_size = min_cluster_size
        self.min_samples = min_samples
        self.alpha = alpha
        self.max_cluster_size = max_cluster_size
        self.cluster_selection_epsilon = cluster_selection_epsilon
        self.cluster_selection_persistence = cluster_selection_persistence
        self.metric = metric
        self.p = p
        self.algorithm = algorithm
        self.leaf_size = leaf_size
        self.memory = memory
        self.approx_min_span_tree = approx_min_span_tree
        self.gen_min_span_tree = gen_min_span_tree
        self.core_dist_n_jobs = core_dist_n_jobs
        self.cluster_selection_method = cluster_selection_method
        self.allow_single_cluster = allow_single_cluster
        self.match_reference_implementation = match_reference_implementation
        self.prediction_data = prediction_data
        self.branch_detection_data = branch_detection_data
        self.cluster_selection_epsilon_max = cluster_selection_epsilon_max

        self._metric_kwargs = kwargs

        self._condensed_tree = None
        self._single_linkage_tree = None
        self._min_spanning_tree = None
        self._raw_data = None
        self._outlier_scores = None
        self._prediction_data = None
        self._finite_index = None
        self._branch_detection_data = None
        self._relative_validity = None
```

--------------------------------

### prediction_data_

Source: https://hdbscan.readthedocs.io/en/latest/_modules/hdbscan/hdbscan_.html

Property to get or set the prediction data. Raises an AttributeError if no prediction data was generated.

```APIDOC
## prediction_data_

### Description
Property to get or set the prediction data. Raises an AttributeError if no prediction data was generated.

### Method
prediction_data_ (getter/setter)

### Returns
- **prediction_data** - The prediction data.

### Errors
- Raises `AttributeError` if no prediction data was generated.
```

--------------------------------

### BranchDetector.__init__

Source: https://hdbscan.readthedocs.io/en/latest/_modules/hdbscan/branches.html

Initializes the BranchDetector with parameters for branch detection and cluster selection.

```APIDOC
## BranchDetector.__init__

### Description
Initializes the BranchDetector with parameters for branch detection and cluster selection.

### Method
Constructor

### Parameters
- **branch_detection_method** (str, optional, default='full') - Method for branch detection. Options: 'core' or 'full'.
- **label_sides_as_branches** (bool, optional, default=False) - Whether to label sides as branches.
- **min_cluster_size** (int, optional, default=None) - Minimum size of clusters. If None, uses the default from HDBSCAN.
- **max_cluster_size** (int, optional, default=None) - Maximum size of clusters. If None, no limit.
- **allow_single_cluster** (bool, optional, default=None) - Whether to allow a single cluster.
- **cluster_selection_method** (str, optional, default=None) - Method for cluster selection (e.g., 'eom' or 'leaf').
- **cluster_selection_epsilon** (float, optional, default=0.0) - Epsilon for cluster selection.
- **cluster_selection_persistence** (float, optional, default=0.0) - Persistence for cluster selection.

### Returns
None
```

--------------------------------

### condensed_tree_

Source: https://hdbscan.readthedocs.io/en/latest/_modules/hdbscan/hdbscan_.html

Property to get or set the condensed tree. Returns a CondensedTree object wrapping the internal tree and labels.

```APIDOC
## condensed_tree_

### Description
Property to get or set the condensed tree. Returns a CondensedTree object wrapping the internal tree and labels.

### Method
condensed_tree_ (getter/setter)

### Returns
- **condensed_tree** (CondensedTree) - The condensed tree.

### Errors
- Raises `AttributeError` if no condensed tree was generated; try running fit first.
```

--------------------------------

### RobustSingleLinkage.__init__

Source: https://hdbscan.readthedocs.io/en/latest/_modules/hdbscan/robust_single_linkage_.html

Constructor for the RobustSingleLinkage class. Initializes the clustering parameters.

```APIDOC
## RobustSingleLinkage.__init__

### Description
Initializes the RobustSingleLinkage clustering object with the specified parameters.

### Method
Constructor

### Parameters
- **cut** (float, optional) - Default: 0.4 - The cut parameter for the clustering.
- **k** (int, optional) - Default: 5 - The number of nearest neighbors to consider.
- **alpha** (float, optional) - Default: 1.4142135623730951 - The alpha parameter for the clustering.
- **gamma** (int, optional) - Default: 5 - The gamma parameter for the clustering.
- **metric** (string, optional) - Default: 'euclidean' - The distance metric to use.
- **algorithm** (string, optional) - Default: 'best' - Which algorithm to use; options include 'small', 'small_kdtree', 'large_kdtree', 'large_kdtree_fastcluster'.
- **core_dist_n_jobs** (int, optional) - Default: 4 - Number of parallel jobs for core distance computations.
- **metric_params** (dict, optional) - Default: {} - Additional parameters for the metric.

### Returns
None
```

--------------------------------

### branch_detection_data_

Source: https://hdbscan.readthedocs.io/en/latest/_modules/hdbscan/hdbscan_.html

Property to get or set the branch detection data. Raises an AttributeError if no branch detection data was generated.

```APIDOC
## branch_detection_data_

### Description
Property to get or set the branch detection data. Raises an AttributeError if no branch detection data was generated.

### Method
branch_detection_data_ (getter/setter)

### Returns
- **branch_detection_data** - The branch detection data.

### Errors
- Raises `AttributeError` if no branch detection data was generated.
```

--------------------------------

### Initialize _points structured array with labels and probabilities

Source: https://hdbscan.readthedocs.io/en/latest/_modules/hdbscan/plots.html

Builds a NumPy structured array for storing point data, including id, label, probability, lens values, and optional cluster and sub-cluster labels/probabilities. Uses np.empty with a dtype list and assigns values from input arrays.

```python
label_name = f"{self.sub_cluster_name}_label"
probability_name = f"{self.sub_cluster_name}_probability"
dtypes = [
        ("id", np.intp),
        ("label", np.intp),
        ("probability", np.float64),
        (self.lens_name, np.float64)
]
if cluster_labels is not None:
    dtypes += [("cluster_label", np.intp)]
if cluster_probabilities is not None:
    dtypes += [("cluster_probability", np.float64)]
if sub_cluster_labels is not None:
    dtypes += [(label_name, np.intp)]
if sub_cluster_probabilities is not None:
    dtypes += [(probability_name, np.float64)]
self._points = np.empty(self.point_mask.sum(), dtype=dtypes)
self._points["id"] = np.arange(self.point_mask.sum())
self._points["label"] = labels[self.point_mask]
self._points["probability"] = probabilities[self.point_mask]
self._points[self.lens_name] = lens_values[self.point_mask]
if cluster_labels is not None:
    self._points["cluster_label"] = cluster_labels[self.point_mask]
if cluster_probabilities is not None:
    self._points["cluster_probability"] = cluster_probabilities[self.point_mask]
if sub_cluster_labels is not None:
    self._points[label_name] = sub_cluster_labels[self.point_mask]
if sub_cluster_probabilities is not None:
    self._points[probability_name] = sub_cluster_probabilities[self.point_mask]
```

--------------------------------

### Benchmark six clustering implementations on large datasets

Source: https://hdbscan.readthedocs.io/en/latest/performance_and_scalability.html

Sets up dataset sizes from 4,000 to 60,000 points and benchmarks HDBSCAN (boruvka_kdtree), K-Means, DBSCAN, fastcluster linkage_vector, scipy K-Means, and scipy single linkage using the benchmark_algorithm helper with a 90-second max time and one sample per size.

```python
large_dataset_sizes = np.arange(1,16) * 4000

hdbscan_boruvka = hdbscan.HDBSCAN(algorithm='boruvka_kdtree')
large_hdbscan_boruvka_data = benchmark_algorithm(large_dataset_sizes,
                                         hdbscan_boruvka.fit, (), {},
                                                 max_time=90, sample_size=1)

k_means = sklearn.cluster.KMeans(10)
large_k_means_data = benchmark_algorithm(large_dataset_sizes,
                                         k_means.fit, (), {},
                                         max_time=90, sample_size=1)

dbscan = sklearn.cluster.DBSCAN(eps=1.25, min_samples=5)
large_dbscan_data = benchmark_algorithm(large_dataset_sizes,
                                        dbscan.fit, (), {},
                                        max_time=90, sample_size=1)

large_fastclust_data = benchmark_algorithm(large_dataset_sizes,
                                           fastcluster.linkage_vector, (), {},
                                           max_time=90, sample_size=1)

large_scipy_k_means_data = benchmark_algorithm(large_dataset_sizes,
                                               scipy.cluster.vq.kmeans, (10,), {},
                                               max_time=90, sample_size=1)

large_scipy_single_data = benchmark_algorithm(large_dataset_sizes,
                                              scipy.cluster.hierarchy.single, (), {},
                                              max_time=90, sample_size=1)
```

--------------------------------

### Run HDBSCAN clustering with plot_clusters

Source: https://hdbscan.readthedocs.io/en/latest/comparing_clustering_algorithms.html

Call the plot_clusters function with HDBSCAN and the min_cluster_size parameter set to 15. This is the example usage shown on the page.

```python
plot_clusters(data, hdbscan.HDBSCAN, (), {'min_cluster_size':15})
```

--------------------------------

### Convert single linkage tree to pandas DataFrame

Source: https://hdbscan.readthedocs.io/en/latest/_modules/hdbscan/plots.html

Returns a pandas DataFrame with columns 'parent', 'left_child', 'right_child', 'distance', and 'size'. Requires pandas to be installed.

```python
try:
    from pandas import DataFrame, Series
except ImportError:
    raise ImportError('You must have pandas installed to export pandas DataFrames')

max_node = 2 * self._linkage.shape[0]
num_points = max_node - (self._linkage.shape[0] - 1)

parent_array = np.arange(num_points, max_node + 1)

result = DataFrame({
    'parent': parent_array,
    'left_child': self._linkage.T[0],
    'right_child': self._linkage.T[1],
    'distance': self._linkage.T[2],
    'size': self._linkage.T[3]
})[['parent', 'left_child', 'right_child', 'distance', 'size']]

return result
```

--------------------------------

### Generate sample blobs data

Source: https://hdbscan.readthedocs.io/en/latest/basic_hdbscan.html

Generates a dataset with 2000 samples and 10 features using make_blobs.

```python
blobs, labels = make_blobs(n_samples=2000, n_features=10)
```

--------------------------------

### Run HDBSCAN with branch_detection_data and fit BranchDetector

Source: https://hdbscan.readthedocs.io/en/latest/how_to_detect_branches.html

Runs HDBSCAN with branch_detection_data=True to cache internal structures, then fits a BranchDetector with min_cluster_size=15 and plots the branch labels.

```python
clusterer = HDBSCAN(min_cluster_size=15, branch_detection_data=True).fit(data)
branch_detector = BranchDetector(min_cluster_size=15).fit(clusterer)
plot(branch_detector.labels_)
```

--------------------------------

### Get linkage trees property

Source: https://hdbscan.readthedocs.io/en/latest/_modules/hdbscan/branches.html

Returns a list of SingleLinkageTree objects, or None for missing trees. Raises AttributeError if no linkage trees were generated, instructing to run fit first.

```python
        if self._linkage_trees is None:
            raise AttributeError(
                "No linkage trees were generated; try running fit first."
            )
        return [
            SingleLinkageTree(tree) if tree is not None else None
            for tree in self._linkage_trees
        ]
```

--------------------------------

### Get indices of finite rows

Source: https://hdbscan.readthedocs.io/en/latest/_modules/hdbscan/hdbscan_.html

Returns the indices of rows that contain only finite values in a sparse matrix or dense ndarray. For sparse matrices, it converts to LIL format and checks each row.

```python
def get_finite_row_indices(matrix):
    """Returns the indices of the purely finite rows of a sparse matrix or dense ndarray"""
    if issparse(matrix):
        row_indices = np.array(
            [i for i, row in enumerate(matrix.tolil().data) if np.all(np.isfinite(row))]
        )
    else:
        row_indices = np.where(np.isfinite(matrix).sum(axis=1) == matrix.shape[1])[0]
    return row_indices
```

--------------------------------

### Generate dataset sizes with numpy

Source: https://hdbscan.readthedocs.io/en/latest/performance_and_scalability.html

Creates an array of dataset sizes from 500 to 32000 with increasing step sizes: 500 for the first five, 1000 for the next four, and 2000 for the remaining thirteen.

```python
dataset_sizes = np.hstack([np.arange(1, 6) * 500, np.arange(3,7) * 1000, np.arange(4,17) * 2000])
```

--------------------------------

### to_networkx method: build NetworkX graph with node and edge attributes

Source: https://hdbscan.readthedocs.io/en/latest/_modules/hdbscan/plots.html

Constructs a NetworkX Graph from internal data. Requires networkx; raises ImportError if missing. Adds edges with weight, mutual_reachability, component, and lens attributes, then sets node attributes for FLASC features and raw data features.

```python
try:
    import networkx as nx
except ImportError:
    raise ImportError(
        "You must have networkx installed to export networkx graphs"
    )

g = nx.Graph()
# Add edges
for row in self._edges:
    attrs = dict(
        weight=1 / (1 + row["mutual_reachability"]),
        mutual_reachability=row["mutual_reachability"],
        component=row["component"],
    )
    attrs[self.lens_name] = row[self.lens_name]
    g.add_edge(
        row["parent"],
        row["child"],
        **attrs,
    )

# Add FLASC features
for attr in self._points.dtype.names[1:]:
    nx.set_node_attributes(g, dict(self._points[["id", attr]]), attr)

# Add raw data features
if self._raw_data is not None:
    if feature_names is None:
        feature_names = [f"feature {i}" for i in range(self._raw_data.shape[1])]
    for idx, name in enumerate(feature_names):
        nx.set_node_attributes(
            g,
            dict(zip(self._points["id"], self._raw_data[:, idx])),
            name,
        )

return g
```

--------------------------------

### Get approximation graph property

Source: https://hdbscan.readthedocs.io/en/latest/_modules/hdbscan/branches.html

Returns an ApproximationGraph built from stored approximation graphs and cluster metadata. Raises AttributeError if no approximation graph was generated, instructing to run fit first.

```python
        if self._approximation_graphs is None:
            raise AttributeError(
                "No approximation graph was generated; try running fit first."
            )
        return ApproximationGraph(
            self._approximation_graphs,
            self.labels_,
            self.probabilities_,
            self.centralities_,
            self.cluster_labels_,
            self.cluster_probabilities_,
            self.branch_labels_,
            self.branch_probabilities_,
            lens_name="centrality",
            sub_cluster_name="branch",
            raw_data=self._clusterer._raw_data,
        )
```

--------------------------------

### Generate data with branches and run HDBSCAN

Source: https://hdbscan.readthedocs.io/en/latest/how_to_detect_branches.html

Creates noisy line segments between three clusters, loads data, and runs HDBSCAN with min_cluster_size=15, then plots the resulting labels.

```python
# Control points for line segments that merge three clusters
p0 = (0.13, -0.26)
p1 = (0.24, -0.12)
p2 = (0.32, 0.1)

# Noisy points along lines between three clusters
segments = [
    np.column_stack(
        (np.linspace(p_start[0], p_end[0], 100), np.linspace(p_start[1], p_end[1], 100))
    )
    + np.random.normal(size=(100, 2), scale=0.01)
    for p_start, p_end in [(p0, p1), (p1, p2)]
]

# Original data with new segments
data = np.load("./clusterable_data.npy")
data = np.concatenate((data, *segments))

# HDBSCAN clusters
clusterer = HDBSCAN(min_cluster_size=15).fit(data)
plot(clusterer.labels_)
```

--------------------------------

### Plot DBSCAN clustering with eps=0.025

Source: https://hdbscan.readthedocs.io/en/latest/comparing_clustering_algorithms.html

Applies DBSCAN clustering to the test dataset with epsilon set to 0.025 and plots the results. The epsilon value was chosen by experimentation, as getting the parameters right can be hard.

```python
plot_clusters(data, cluster.DBSCAN, (), {'eps':0.025})
```

--------------------------------

### HDBSCAN.__init__

Source: https://hdbscan.readthedocs.io/en/latest/_modules/hdbscan/hdbscan_.html

Constructor for the HDBSCAN clustering object. Initializes all clustering parameters including minimum cluster size, metric, algorithm, and various options for cluster selection and prediction data.

```APIDOC
## HDBSCAN.__init__

### Description
Initializes the HDBSCAN clustering object with configurable parameters for clustering behavior, metric, algorithm, and data handling.

### Method
Constructor

### Signature
```python
HDBSCAN(
    min_cluster_size=5,
    min_samples=None,
    cluster_selection_epsilon=0.0,
    cluster_selection_persistence=0.0,
    max_cluster_size=0,
    metric="euclidean",
    alpha=1.0,
    p=None,
    algorithm="best",
    leaf_size=40,
    memory=Memory(None, verbose=0),
    approx_min_span_tree=True,
    gen_min_span_tree=False,
    core_dist_n_jobs=4,
    cluster_selection_method="eom",
    allow_single_cluster=False,
    prediction_data=False,
    branch_detection_data=False,
    match_reference_implementation=False,
    cluster_selection_epsilon_max=float('inf'),
    **kwargs
)
```

### Parameters
- **min_cluster_size** (int) - Optional - Minimum size of clusters.
- **min_samples** (int or None) - Optional - Number of samples in a neighborhood for core point calculation.
- **cluster_selection_epsilon** (float) - Optional - Distance threshold for cluster selection.
- **cluster_selection_persistence** (float) - Optional - Persistence threshold for cluster selection.
- **max_cluster_size** (int) - Optional - Maximum size of clusters.
- **metric** (str) - Optional - Distance metric to use.
- **alpha** (float) - Optional - Scaling factor for distance.
- **p** (int or None) - Optional - Power parameter for Minkowski metric.
- **algorithm** (str) - Optional - Algorithm to use for clustering.
- **leaf_size** (int) - Optional - Leaf size for tree algorithms.
- **memory** (Memory) - Optional - Memory caching object.
- **approx_min_span_tree** (bool) - Optional - Whether to approximate the minimum spanning tree.
- **gen_min_span_tree** (bool) - Optional - Whether to generate the minimum spanning tree.
- **core_dist_n_jobs** (int) - Optional - Number of parallel jobs for core distance computation.
- **cluster_selection_method** (str) - Optional - Method for selecting clusters (e.g., 'eom').
- **allow_single_cluster** (bool) - Optional - Whether to allow a single cluster.
- **prediction_data** (bool) - Optional - Whether to generate prediction data.
- **branch_detection_data** (bool) - Optional - Whether to generate branch detection data.
- **match_reference_implementation** (bool) - Optional - Whether to match the reference implementation.
- **cluster_selection_epsilon_max** (float) - Optional - Maximum epsilon for cluster selection.
- **kwargs** - Optional - Additional metric keyword arguments.

### Returns
- **self** (HDBSCAN) - The initialized HDBSCAN object.
```

--------------------------------

### Convert single linkage tree to networkx DiGraph

Source: https://hdbscan.readthedocs.io/en/latest/_modules/hdbscan/plots.html

Returns a NetworkX DiGraph representing the single linkage tree. Edge weights are distance values, and nodes have a 'size' attribute. Requires networkx to be installed.

```python
try:
    from networkx import DiGraph, set_node_attributes
except ImportError:
    raise ImportError('You must have networkx installed to export networkx graphs')

max_node = 2 * self._linkage.shape[0]
num_points = max_node - (self._linkage.shape[0] - 1)

result = DiGraph()
for parent, row in enumerate(self._linkage, num_points):
    result.add_edge(parent, row[0], weight=row[2])
    result.add_edge(parent, row[1], weight=row[2])

return result
```

--------------------------------

### Generate timing predictions and build datasize table

Source: https://hdbscan.readthedocs.io/en/latest/performance_and_scalability.html

Applies get_timing_series to pre-existing timing datasets for various clustering algorithms, then builds a table of maximum dataset sizes for interactive, coffee, lunch, and overnight time frames. The scipy KMeans timings are commented out because noise made the model unrealistic.

```python
ap_timings = get_timing_series(ap_data)
spectral_timings = get_timing_series(spectral_data)
agg_timings = get_timing_series(agg_data)
debacl_timings = get_timing_series(debacl_data)
fastclust_timings = get_timing_series(large_fastclust_data.ix[:10,:].copy())
scipy_single_timings = get_timing_series(large_scipy_single_data.ix[:10,:].copy())
hdbscan_boruvka = get_timing_series(huge_hdbscan_data, quadratic=True)
#scipy_k_means_timings = get_timing_series(huge_scipy_k_means_data, quadratic=False)
dbscan_timings = get_timing_series(huge_dbscan_data, quadratic=True)
k_means_timings = get_timing_series(huge_k_means_data, quadratic=False)

timing_data = pd.concat([ap_timings, spectral_timings, agg_timings, debacl_timings,
                            scipy_single_timings, fastclust_timings, hdbscan_boruvka,
                            dbscan_timings, k_means_timings
                           ], axis=1)
timing_data.columns=['AffinityPropagation', 'Spectral', 'Agglomerative',
                                       'DeBaCl', 'ScipySingleLinkage', 'Fastcluster',
                                       'HDBSCAN', 'DBSCAN', 'SKLearn KMeans'
                                      ]
def get_size(series, max_time):
    return series.index[series < max_time].max()

datasize_table = pd.concat([
                            timing_data.apply(get_size, max_time=30),
                            timing_data.apply(get_size, max_time=300),
                            timing_data.apply(get_size, max_time=3600),
                            timing_data.apply(get_size, max_time=8*3600)
                            ], axis=1)
datasize_table.columns=('Interactive', 'Get Coffee', 'Over Lunch', 'Overnight')
datasize_table
```

--------------------------------

### Plot cluster exemplars over data

Source: https://hdbscan.readthedocs.io/en/latest/soft_clustering_explanation.html

Visualizes the exemplar points for each cluster by plotting all data in gray and overlaying exemplars with cluster colors. Uses the clusterer's condensed tree and the exemplars function.

```python
tree = clusterer.condensed_tree_
plt.scatter(data.T[0], data.T[1], c='grey', **plot_kwds)
for i, c in enumerate(tree._select_clusters()):
    c_exemplars = exemplars(c, tree)
    plt.scatter(data.T[0][c_exemplars], data.T[1][c_exemplars], c=palette[i], **plot_kwds)
```

--------------------------------

### Display HDBSCAN clusterer parameters

Source: https://hdbscan.readthedocs.io/en/latest/how_hdbscan_works.html

Shows the default parameters of the fitted HDBSCAN clusterer, including algorithm, metric, and min_cluster_size.

```python
HDBSCAN(algorithm='best', alpha=1.0, approx_min_span_tree=True,
    gen_min_span_tree=True, leaf_size=40, memory=Memory(None),
    metric='euclidean', min_cluster_size=5, min_samples=None, p=None)
```

--------------------------------

### Fit HDBSCAN clusterer with minimum cluster size

Source: https://hdbscan.readthedocs.io/en/latest/how_hdbscan_works.html

Creates an HDBSCAN clusterer with min_cluster_size=5 and gen_min_span_tree=True, then fits it to the test data.

```python
clusterer = hdbscan.HDBSCAN(min_cluster_size=5, gen_min_span_tree=True)
clusterer.fit(test_data)
```

--------------------------------

### Convert condensed tree to pandas DataFrame

Source: https://hdbscan.readthedocs.io/en/latest/advanced_hdbscan.html

Convert the condensed tree to a pandas DataFrame using `to_pandas()`. Each row corresponds to an edge, with columns `parent`, `child`, `lambda_val`, and `child_size`. Filter rows with `child_size > 1` to get only clusters.

```python
clusterer.condensed_tree_.to_pandas().head()

```

```output
| | parent | child | lambda_val | child_size |
| --- | --- | --- | --- | --- |
| 0 | 2309 | 2048 | 5.016526 | 1 |
| 1 | 2309 | 2006 | 5.076503 | 1 |
| 2 | 2309 | 2024 | 5.279133 | 1 |
| 3 | 2309 | 2050 | 5.347332 | 1 |
| 4 | 2309 | 1992 | 5.381930 | 1 |

```

```python
tree = clusterer.condensed_tree_.to_pandas()
cluster_tree = tree[tree.child_size > 1]

```

--------------------------------

### Fit HDBSCAN with prediction_data=True

Source: https://hdbscan.readthedocs.io/en/latest/prediction_tutorial.html

Loads clusterable data and fits an HDBSCAN model with prediction_data=True to enable fast approximate predictions later. Also sets up a color palette and plots the clustered data.

```python
data = np.load('clusterable_data.npy')
clusterer = hdbscan.HDBSCAN(min_cluster_size=15, prediction_data=True).fit(data)
pal = sns.color_palette('deep', 8)
colors = [sns.desaturate(pal[col], sat) for col, sat in zip(clusterer.labels_,
                                                            clusterer.probabilities_)]
plt.scatter(data.T[0], data.T[1], c=colors, **plot_kwds);
```

--------------------------------

### Benchmark ten clustering algorithms

Source: https://hdbscan.readthedocs.io/en/latest/performance_and_scalability.html

Runs the benchmark_algorithm function on ten clustering implementations. This step is very expensive and can take over an hour to complete. Some algorithms use sample_size to limit the number of runs.

```python
k_means = sklearn.cluster.KMeans(10)
k_means_data = benchmark_algorithm(dataset_sizes, k_means.fit, (), {})

dbscan = sklearn.cluster.DBSCAN(eps=1.25)
dbscan_data = benchmark_algorithm(dataset_sizes, dbscan.fit, (), {})

scipy_k_means_data = benchmark_algorithm(dataset_sizes,
                                         scipy.cluster.vq.kmeans, (10,), {})

scipy_single_data = benchmark_algorithm(dataset_sizes,
                                        scipy.cluster.hierarchy.single, (), {})

fastclust_data = benchmark_algorithm(dataset_sizes,
                                     fastcluster.linkage_vector, (), {})

hdbscan_ = hdbscan.HDBSCAN()
hdbscan_data = benchmark_algorithm(dataset_sizes, hdbscan_.fit, (), {})

debacl_data = benchmark_algorithm(dataset_sizes,
                                  debacl.geom_tree.geomTree, (5, 5), {'verbose':False})

agglomerative = sklearn.cluster.AgglomerativeClustering(10)
agg_data = benchmark_algorithm(dataset_sizes,
                               agglomerative.fit, (), {}, sample_size=4)

spectral = sklearn.cluster.SpectralClustering(10)
spectral_data = benchmark_algorithm(dataset_sizes,
                                    spectral.fit, (), {}, sample_size=6)

affinity_prop = sklearn.cluster.AffinityPropagation()
ap_data = benchmark_algorithm(dataset_sizes,
                              affinity_prop.fit, (), {}, sample_size=3)
```

--------------------------------

### _select_clusters method implementation

Source: https://hdbscan.readthedocs.io/en/latest/_modules/hdbscan/plots.html

Internal method that computes the lowest cluster segment id for each cluster label using the raw tree and labels. It groups points by label and returns the minimum parent segment for each cluster, excluding noise if labels start at -1.

```python
    def _select_clusters(self):
        points_tree = self._raw_tree[self._raw_tree["child_size"] == 1]
        
        # Find lowest cluster segment id for each cluster label
        segments = points_tree['parent']
        labels = self._labels[points_tree['child']]
        order = np.argsort(labels)
        groups = np.split(segments[order], np.flatnonzero(np.diff(labels[order]) != 0) + 1)        
        return [
            groups[label].min() for label in range(int(labels[order[0]] == -1), len(groups))
        ]
        
```

--------------------------------

### Branch detection and segmentation workflow

Source: https://hdbscan.readthedocs.io/en/latest/_modules/hdbscan/branches.html

Main workflow that detects branches, segments them, and updates labels. Uses memory.cache to cache results and handles non-finite data by remapping indices.

```python
    # Detect branches
    (
        points,
        centralities,
        linkage_trees,
        approximation_graphs,
    ) = memory.cache(compute_branch_linkage, ignore=["thread_pool"])(
        cluster_labels,
        cluster_probabilities,
        clusterer._min_spanning_tree,
        clusterer.branch_detection_data_.tree,
        clusterer.branch_detection_data_.neighbors,
        clusterer.branch_detection_data_.core_distances,
        clusterer.branch_detection_data_.dist_metric,
        num_clusters,
        thread_pool,
        run_core=run_core,
        overridden_labels=overridden_labels,
    )
    (
        branch_labels,
        branch_probabilities,
        branch_persistences,
        condensed_trees,
        linkage_trees,
    ) = memory.cache(compute_branch_segmentation, ignore=["thread_pool"])(
        linkage_trees,
        thread_pool,
        min_cluster_size=min_cluster_size,
        allow_single_cluster=allow_single_cluster,
        cluster_selection_method=cluster_selection_method,
        cluster_selection_epsilon=cluster_selection_epsilon,
        cluster_selection_persistence=cluster_selection_persistence,
        max_cluster_size=max_cluster_size,
    )
    (
        labels,
        probabilities,
        branch_labels,
        branch_probabilities,
        centralities,
    ) = memory.cache(update_labelling)(
        cluster_probabilities,
        linkage_trees,
        points,
        centralities,
        branch_labels,
        branch_probabilities,
        branch_persistences,
        label_sides_as_branches=label_sides_as_branches,
    )

    # Maintain data indices for non-finite data
    if not clusterer.branch_detection_data_.all_finite:
        internal_to_raw = clusterer.branch_detection_data_.internal_to_raw
        _remap_point_lists(points, internal_to_raw)
        _remap_edge_lists(approximation_graphs, internal_to_raw)

        num_points = len(clusterer.labels_)
        labels = _remap_labels(labels, finite_index, num_points)
        probabilities = _remap_probabilities(probabilities, finite_index, num_points)
        cluster_labels = _remap_labels(cluster_labels, finite_index, num_points)
        cluster_probabilities = _remap_probabilities(
            cluster_probabilities, finite_index, num_points
        )
        branch_labels = _remap_labels(branch_labels, finite_index, num_points, 0)
        branch_probabilities = _remap_probabilities(
            branch_probabilities, finite_index, num_points
        )
        centralities = _remap_probabilities(centralities, finite_index, num_points)

    return (
        labels,
        probabilities,
        cluster_labels,
        cluster_probabilities,
        branch_labels,
        branch_probabilities,
        branch_persistences,
        approximation_graphs,
        condensed_trees,
        linkage_trees,
        centralities,
        points,
    )
```

--------------------------------

### Fit HDBSCAN with alpha=1.3 and visualize clusters

Source: https://hdbscan.readthedocs.io/en/latest/parameter_selection.html

Adjusts the alpha parameter to 1.3 to make clustering more conservative. Note that adjusting alpha recomputes the hard computation of the single linkage tree. The example fits the model on data and plots the clusters with colors based on cluster labels and probabilities.

```python
clusterer = hdbscan.HDBSCAN(min_cluster_size=60, min_samples=15, alpha=1.3).fit(data)
color_palette = sns.color_palette('Paired', 12)
cluster_colors = [color_palette[x] if x >= 0
                  else (0.5, 0.5, 0.5)
                  for x in clusterer.labels_]
cluster_member_colors = [sns.desaturate(x, p) for x, p in
                         zip(cluster_colors, clusterer.probabilities_)]
plt.scatter(*projection.T, s=50, linewidth=0, c=cluster_member_colors, alpha=0.25)
```

--------------------------------

### List supported metrics via hdbscan.dist_metrics.METRIC_MAPPING

Source: https://hdbscan.readthedocs.io/en/latest/basic_hdbscan.html

Displays the dictionary mapping metric names to their distance classes, inherited from sklearn.

```python
hdbscan.dist_metrics.METRIC_MAPPING
```

--------------------------------

### Display clusterer parameters

Source: https://hdbscan.readthedocs.io/en/latest/basic_hdbscan.html

Shows the default parameters of the fitted HDBSCAN clusterer.

```python
HDBSCAN(algorithm='best', alpha=1.0, approx_min_span_tree=True,
    gen_min_span_tree=False, leaf_size=40, memory=Memory(None),
    metric='euclidean', min_cluster_size=5, min_samples=None, p=None)
```

--------------------------------

### Import hdbscan library

Source: https://hdbscan.readthedocs.io/en/latest/how_hdbscan_works.html

Imports the hdbscan library to use the HDBSCAN clustering algorithm.

```python
import hdbscan
```

--------------------------------

### Load digits dataset and project with t-SNE

Source: https://hdbscan.readthedocs.io/en/latest/parameter_selection.html

Loads the digits dataset from sklearn and projects it into two dimensions using t-SNE for visualization. This is the initial step before clustering.

```python
digits = datasets.load_digits()
data = digits.data
projection = TSNE().fit_transform(data)
plt.scatter(*projection.T, **plot_kwds)
```