### vat(data, return_odm=False, figure_size=(10, 10)) Source: https://pyclustertend.readthedocs.io/en/latest/pyclustertend.html Performs VAT visualization. ```APIDOC ## vat(data, return_odm=False, figure_size=(10, 10)) ### Description VAT (Visual Assessment of Tendency) allows assessing cluster tendency through a map based on the dissimilarity matrix. ### Parameters - **data** (matrix) - Required - numpy array - **return_odm** (boolean) - Optional - Whether to return the Ordered Dissimilarity Matrix (default: False) - **figure_size** (tuple) - Optional - Size of the VAT (default: (10, 10)) ### Returns - **ODM** (matrix) - The ordered dissimilarity matrix plotted. ``` -------------------------------- ### assess_tendency_by_mean_metric_score(dataset, n_cluster=10, random_state=None) Source: https://pyclustertend.readthedocs.io/en/latest/_modules/pyclustertend/metric.html Assess the clusterability of a dataset using KMeans and the mean of silhouette, calinski-harabasz, and davies-bouldin scores. ```APIDOC ## assess_tendency_by_mean_metric_score ### Description Assess the clusterability of a dataset using KMeans algorithm and the silhouette, calinski, and davies bouldin scores. The best cluster number is the mean of the result of the three methods. ### Parameters - **dataset** (numpy array, DataFrame) - The input dataset. - **n_cluster** (int) - The maximum number of clusters to consider. - **random_state** (int) - Optional, default to None. ### Returns - **n_clusters** (float) - The mean of the best number of cluster scores calculated across the three methods. ``` -------------------------------- ### assess_tendency_by_metric(dataset, metric='silhouette', n_cluster=10, random_state=None) Source: https://pyclustertend.readthedocs.io/en/latest/_modules/pyclustertend/metric.html Assess the clusterability of a dataset using the KMeans algorithm and a specified metric score. Returns the optimal number of clusters and the scores for each cluster count. ```APIDOC ## assess_tendency_by_metric ### Description Assess the clusterability of a dataset using KMeans algorithm and a metric score. The best cluster number is determined by the highest (or lowest for davies_bouldin) score. ### Parameters - **dataset** (numpy array, DataFrame) - The input dataset. - **metric** (string) - The method to assess cluster quality ('silhouette', 'calinski_harabasz', 'davies_bouldin'), default to 'silhouette'. - **n_cluster** (int) - The maximum number of clusters to consider. - **random_state** (int) - Optional, default to None. ### Returns - **(n_clusters, value)** (tuple) - n_clusters is the number of clusters that best scored on the chosen metric. value is the array of scores for each number of clusters. ``` -------------------------------- ### ivat(data, return_odm=False, figure_size=(10, 10)) Source: https://pyclustertend.readthedocs.io/en/latest/pyclustertend.html Performs iVAT visualization. ```APIDOC ## ivat(data, return_odm=False, figure_size=(10, 10)) ### Description iVAT returns a visualization based on the VAT, which is more reliable and easier to interpret. ### Parameters - **data** (matrix) - Required - numpy array - **return_odm** (boolean) - Optional - Whether to return the Ordered Dissimilarity Matrix (default: False) - **figure_size** (tuple) - Optional - Size of the VAT (default: (10, 10)) ### Returns - **D_prim** (matrix) - The iVAT ordered dissimilarity matrix. ``` -------------------------------- ### Visualize Cluster Tendency with VAT Source: https://pyclustertend.readthedocs.io/en/latest/_modules/pyclustertend/visual_assessment_of_tendency.html Generates a VAT plot based on the dissimilarity matrix of the input data. ```python def vat(data, return_odm=False, figure_size=(10, 10)): """VAT means Visual assesement of tendency. basically, it allow to asses cluster tendency through a map based on the dissimiliraty matrix. Parameters ---------- data : matrix numpy array return_odm : return the Ordered Dissimalirity Matrix boolean (default to False) figure_size : size of the VAT. tuple (default to (10,10)) Return ------- ODM : matrix the ordered dissimalarity matrix plotted. """ ordered_dissimilarity_matrix = compute_ordered_dissimilarity_matrix(data) _, ax = plt.subplots(figsize=figure_size) ax.imshow(ordered_dissimilarity_matrix, cmap='gray', vmin=0, vmax=np.max(ordered_dissimilarity_matrix)) if return_odm is True: return ordered_dissimilarity_matrix ``` -------------------------------- ### vat(data, return_odm=False, figure_size=(10, 10)) Source: https://pyclustertend.readthedocs.io/en/latest/_modules/pyclustertend/visual_assessment_of_tendency.html Performs Visual Assessment of Tendency (VAT) to assess cluster tendency through a map based on the dissimilarity matrix. ```APIDOC ## vat(data, return_odm=False, figure_size=(10, 10)) ### Description Allows assessment of cluster tendency through a map based on the dissimilarity matrix. ### Parameters - **data** (matrix) - Required - numpy array - **return_odm** (boolean) - Optional - Whether to return the Ordered Dissimilarity Matrix (default: False) - **figure_size** (tuple) - Optional - Size of the VAT plot (default: (10, 10)) ### Returns - **ODM** (matrix) - The ordered dissimilarity matrix plotted. ``` -------------------------------- ### compute_ordered_dissimilarity_matrix(X) Source: https://pyclustertend.readthedocs.io/en/latest/pyclustertend.html Computes the ordered dissimilarity matrix used by VAT. ```APIDOC ## compute_ordered_dissimilarity_matrix(X) ### Description Computes the ordered dissimilarity matrix used by visual assessment of tendency. ### Parameters - **X** (matrix) - Required - numpy array ### Returns - **ODM** (matrix) - The ordered dissimilarity matrix. ``` -------------------------------- ### Compute Ordered Dissimilarity Matrix Source: https://pyclustertend.readthedocs.io/en/latest/_modules/pyclustertend/visual_assessment_of_tendency.html Calculates the reordered dissimilarity matrix required for VAT analysis. ```python def compute_ordered_dissimilarity_matrix(X): """The ordered dissimilarity matrix is used by visual assesement of tendency. It is a just a a reordering of the dissimilarity matrix. Parameters ---------- X : matrix numpy array Return ------- ODM : matrix the ordered dissimalarity matrix . """ # Step 1 : observation_path = [] matrix_of_pairwise_distance = pairwise_distances(X) list_of_int = np.zeros(matrix_of_pairwise_distance.shape[0], dtype="int") index_of_maximum_value = np.argmax(matrix_of_pairwise_distance) column_index_of_maximum_value = index_of_maximum_value // matrix_of_pairwise_distance.shape[1] list_of_int[0] = column_index_of_maximum_value observation_path.append(column_index_of_maximum_value) K = np.linspace(0, matrix_of_pairwise_distance.shape[0] - 1, matrix_of_pairwise_distance.shape[0], dtype="int") J = np.delete(K, column_index_of_maximum_value) # Step 2 : for r in range(1, matrix_of_pairwise_distance.shape[0]): p, q = (-1, -1) mini = np.max(matrix_of_pairwise_distance) for candidate_p in observation_path: for candidate_j in J: if matrix_of_pairwise_distance[candidate_p, candidate_j] < mini: p = candidate_p q = candidate_j mini = matrix_of_pairwise_distance[p, q] list_of_int[r] = q observation_path.append(q) ind_q = np.where(np.array(J) == q)[0][0] J = np.delete(J, ind_q) # Step 3 ordered_matrix = np.zeros(matrix_of_pairwise_distance.shape) for column_index_of_maximum_value in range(ordered_matrix.shape[0]): for j in range(ordered_matrix.shape[1]): ordered_matrix[column_index_of_maximum_value, j] = matrix_of_pairwise_distance[ list_of_int[column_index_of_maximum_value], list_of_int[j]] # Step 4 : return ordered_matrix ``` -------------------------------- ### Assess cluster tendency using a specific metric Source: https://pyclustertend.readthedocs.io/en/latest/_modules/pyclustertend/metric.html Evaluates dataset clusterability by finding the optimal number of clusters based on a chosen metric score. ```python >>> from sklearn import datasets >>> from pyclustertend import assess_tendency_by_metric >>> from sklearn.preprocessing import scale >>> X = scale(datasets.load_boston().data) >>> assess_tendency_by_metric(X, n_cluster=10) (2, array([0.36011769, 0.25740335, 0.28098046, 0.28781574, 0.26746932, 0.26975514, 0.27155699, 0.28883395, 0.29028124])) ``` -------------------------------- ### compute_ivat_ordered_dissimilarity_matrix(X) Source: https://pyclustertend.readthedocs.io/en/latest/pyclustertend.html Computes the ordered dissimilarity matrix used by iVAT. ```APIDOC ## compute_ivat_ordered_dissimilarity_matrix(X) ### Description Computes the ordered dissimilarity matrix used by iVAT, which is a reordering of the dissimilarity matrix. ### Parameters - **X** (matrix) - Required - numpy array ### Returns - **D_prim** (matrix) - The ordered dissimilarity matrix. ``` -------------------------------- ### Visualize Cluster Tendency with iVAT Source: https://pyclustertend.readthedocs.io/en/latest/_modules/pyclustertend/visual_assessment_of_tendency.html Generates an iVAT plot, which is generally more reliable and easier to interpret than standard VAT. ```python def ivat(data, return_odm=False, figure_size=(10, 10)): """iVat return a visualisation based on the Vat but more reliable and easier to interpret. Parameters ---------- data : matrix numpy array return_odm : return the Ordered Dissimalirity Matrix boolean (default to False) figure_size : size of the VAT. tuple (default to (10,10)) Return ------- D_prim : matrix the ivat ordered dissimalarity matrix. """ ordered_matrix = compute_ivat_ordered_dissimilarity_matrix(data) _, ax = plt.subplots(figsize=figure_size) ax.imshow(ordered_matrix, cmap='gray', vmin=0, vmax=np.max(ordered_matrix)) if return_odm is True: return ordered_matrix ``` -------------------------------- ### ivat(data, return_odm=False, figure_size=(10, 10)) Source: https://pyclustertend.readthedocs.io/en/latest/_modules/pyclustertend/visual_assessment_of_tendency.html Performs Improved Visual Assessment of Tendency (iVAT) which provides a more reliable and easier to interpret visualization than standard VAT. ```APIDOC ## ivat(data, return_odm=False, figure_size=(10, 10)) ### Description Returns a visualization based on the VAT but more reliable and easier to interpret. ### Parameters - **data** (matrix) - Required - numpy array - **return_odm** (boolean) - Optional - Whether to return the Ordered Dissimilarity Matrix (default: False) - **figure_size** (tuple) - Optional - Size of the VAT plot (default: (10, 10)) ### Returns - **D_prim** (matrix) - The iVAT ordered dissimilarity matrix. ``` -------------------------------- ### Hopkins Function Implementation Source: https://pyclustertend.readthedocs.io/en/latest/_modules/pyclustertend/hopkins.html The source code implementation of the hopkins function. ```python from sklearn.neighbors import BallTree import numpy as np import pandas as pd [docs]def hopkins(data_frame, sampling_size): """Assess the clusterability of a dataset. A score between 0 and 1, a score around 0.5 express no clusterability and a score tending to 0 express a high cluster tendency. Parameters ---------- data_frame : numpy array The input dataset sampling_size : int The sampling size which is used to evaluate the number of DataFrame. Returns --------------------- score : float The hopkins score of the dataset (between 0 and 1) Examples -------- >>> from sklearn import datasets >>> from pyclustertend import hopkins >>> X = datasets.load_iris().data >>> hopkins(X,150) 0.16 """ if type(data_frame) == np.ndarray: data_frame = pd.DataFrame(data_frame) # Sample n observations from D : P if sampling_size > data_frame.shape[0]: raise Exception( 'The number of sample of sample is bigger than the shape of D') data_frame_sample = data_frame.sample(n=sampling_size) # Get the distance to their neirest neighbors in D : X tree = BallTree(data_frame, leaf_size=2) dist, _ = tree.query(data_frame_sample, k=2) data_frame_sample_distances_to_nearest_neighbours = dist[:, 1] # Randomly simulate n points with the same variation as in D : Q. max_data_frame = data_frame.max() min_data_frame = data_frame.min() uniformly_selected_values_0 = np.random.uniform(min_data_frame[0], max_data_frame[0], sampling_size) uniformly_selected_values_1 = np.random.uniform(min_data_frame[1], max_data_frame[1], sampling_size) uniformly_selected_observations = np.column_stack((uniformly_selected_values_0, uniformly_selected_values_1)) if len(max_data_frame) >= 2: for i in range(2, len(max_data_frame)): uniformly_selected_values_i = np.random.uniform(min_data_frame[i], max_data_frame[i], sampling_size) to_stack = (uniformly_selected_observations, uniformly_selected_values_i) uniformly_selected_observations = np.column_stack(to_stack) uniformly_selected_observations_df = pd.DataFrame(uniformly_selected_observations) # Get the distance to their neirest neighbors in D : Y tree = BallTree(data_frame, leaf_size=2) dist, _ = tree.query(uniformly_selected_observations_df, k=1) uniformly_df_distances_to_nearest_neighbours = dist # return the hopkins score x = sum(data_frame_sample_distances_to_nearest_neighbours) y = sum(uniformly_df_distances_to_nearest_neighbours) if x + y == 0: raise Exception('The denominator of the hopkins statistics is null') return x / (x + y)[0] ``` -------------------------------- ### Assess cluster tendency using mean metric scores Source: https://pyclustertend.readthedocs.io/en/latest/_modules/pyclustertend/metric.html Calculates the mean of the optimal cluster counts determined by silhouette, Calinski-Harabasz, and Davies-Bouldin scores. ```python >>> from sklearn import datasets >>> from pyclustertend import assess_tendency_by_mean_metric_score >>> from sklearn.preprocessing import scale >>> X = scale(datasets.load_boston().data) >>> assess_tendency_by_mean_metric_score(X,10) 2.6666666666666665 ``` -------------------------------- ### Calculate Hopkins Statistic Source: https://pyclustertend.readthedocs.io/en/latest/_modules/pyclustertend/hopkins.html Assess the clusterability of a dataset using the hopkins function. ```python from sklearn import datasets from pyclustertend import hopkins X = datasets.load_iris().data hopkins(X,150) ``` -------------------------------- ### hopkins(data_frame, sampling_size) Source: https://pyclustertend.readthedocs.io/en/latest/_modules/pyclustertend/hopkins.html Calculates the Hopkins score for a given dataset to determine its clusterability. ```APIDOC ## hopkins(data_frame, sampling_size) ### Description Assess the clusterability of a dataset. A score between 0 and 1 is returned, where a score around 0.5 expresses no clusterability and a score tending to 0 expresses a high cluster tendency. ### Parameters - **data_frame** (numpy array) - The input dataset. - **sampling_size** (int) - The sampling size used to evaluate the number of observations. ### Returns - **score** (float) - The hopkins score of the dataset (between 0 and 1). ### Example ```python from sklearn import datasets from pyclustertend import hopkins X = datasets.load_iris().data hopkins(X, 150) # Returns: 0.16 ``` ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.