### Install Dependencies (Shell) Source: https://docs.cleanlab.ai/stable/tutorials/clean_learning/text This command installs the necessary Python packages for the tutorial: `sentence-transformers` for text embeddings and `cleanlab` for label cleaning. It also provides an option to install directly from the GitHub repository for the latest version. ```shell !pip install sentence-transformers !pip install cleanlab # Make sure to install the version corresponding to this tutorial # E.g. if viewing master branch documentation: # !pip install git+https://github.com/cleanlab/cleanlab.git ``` -------------------------------- ### Print First Training Example Source: https://docs.cleanlab.ai/stable/tutorials/clean_learning/text Prints the label and text of the first example in the training set to provide a concrete illustration of the data format. ```python i = 0 print(f"Example Label: {raw_train_labels[i]}") print(f"Example Text: {raw_train_texts[i]}") ``` -------------------------------- ### Install Datalab from Source Source: https://docs.cleanlab.ai/stable/cleanlab/datalab/guide/index Installs the developmental version of Datalab from its GitHub repository. This is useful for developers who want to use the latest features or contribute to the project. ```bash pip install "git+https://github.com/cleanlab/cleanlab.git#egg=cleanlab[datalab]" ``` -------------------------------- ### Install Datalab with Dependencies Source: https://docs.cleanlab.ai/stable/cleanlab/datalab/guide/index Installs the Datalab component of the cleanlab package along with its required dependencies. This command is used to set up Datalab for users who need its advanced data auditing capabilities. ```bash pip install "cleanlab[datalab]" ``` -------------------------------- ### Define Function to Display Example DataFrames Source: https://docs.cleanlab.ai/stable/tutorials/clean_learning/text Defines a helper function `print_as_df` that takes an index and returns a Pandas DataFrame for a specific example. This DataFrame includes the raw text, the given label, and the predicted label (inverse transformed), facilitating inspection of potential label errors. ```python def print_as_df(index): return pd.DataFrame( { "text": raw_train_texts, "given_label": raw_train_labels, "predicted_label": encoder.inverse_transform(label_issues["predicted_label"]), }, ).iloc[index] ``` -------------------------------- ### Print Summary of Found Label Errors Source: https://docs.cleanlab.ai/stable/tutorials/clean_learning/text Prints a summary indicating the total number of potential label errors found by cleanlab and lists the indices of the top 10 most likely mislabeled examples. ```python print( f"cleanlab found {len(identified_issues)} potential label errors in the dataset.\n" f"Here are indices of the top 10 most likely errors: \n {lowest_quality_labels}" ) ``` -------------------------------- ### Filter and Sort Label Issues Source: https://docs.cleanlab.ai/stable/tutorials/clean_learning/text Filters the `label_issues` DataFrame to get only the examples flagged as `is_label_issue == True`. It also identifies the indices of the top 10 examples with the lowest `label_quality` scores, indicating the most likely mislabeled instances. ```python identified_issues = label_issues[label_issues["is_label_issue"] == True] lowest_quality_labels = label_issues["label_quality"].argsort()[:10].to_numpy() ``` -------------------------------- ### Get Label Quality Scores Source: https://docs.cleanlab.ai/stable/cleanlab/experimental/label_issues_batched Fetches already-computed estimates of the label quality for each example. Lower scores indicate a higher likelihood of the example being mislabeled. ```python get_quality_scores()[source]# Fetches already-computed estimate of the label quality of each example seen so far in the same format as: `rank.get_label_quality_scores`. Return type: `ndarray` Returns: **label_quality_scores** (`np.ndarray`) – Contains one score (between 0 and 1) per example seen so far. Lower scores indicate more likely mislabeled examples. ``` -------------------------------- ### Load and Display CSV Data with Pandas Source: https://docs.cleanlab.ai/stable/tutorials/clean_learning/text Loads a CSV file from a URL into a pandas DataFrame and displays the first few rows. This is the initial step for data ingestion. ```python import pandas as pd data = pd.read_csv("https://s.cleanlab.ai/banking-intent-classification.csv") data.head() ``` -------------------------------- ### Get Specific Example Issues and Sort Source: https://docs.cleanlab.ai/stable/tutorials/datalab/datalab_quickstart Filters and sorts examples based on a specific issue type, such as 'label' issues. It identifies examples flagged with the issue and then sorts them by their quality score in ascending order, highlighting the most concerning examples for that particular issue. ```python examples_w_issue = ( lab.get_issues("label") .query("is_label_issue") .sort_values("label_score") ) examples_w_issue.head() ``` -------------------------------- ### Initialize CleanLearning and Find Label Issues with GradientBoostingClassifier Source: https://docs.cleanlab.ai/stable/tutorials/faq This snippet shows how to create example data, introduce label errors, and then use CleanLearning with a GradientBoostingClassifier to find these label issues. It also demonstrates initializing a second CleanLearning instance with RandomizedSearchCV for final model training on a clean subset. ```python import numpy as np from sklearn.ensemble import GradientBoostingClassifier from sklearn.model_selection import RandomizedSearchCV from cleanlab.regression.learn import CleanLearning # Make example data data = np.vstack([np.random.random((100, 2)), np.random.random((100, 2)) + 10]) labels = np.array([0] * 100 + [1] * 100) # Introduce label errors true_errors = [97, 98, 100, 101, 102, 104] for idx in true_errors: labels[idx] = 1 - labels[idx] # CleanLearning with no hyperparameter-tuning during expensive cross-validation to find label issues # but hyperparameter-tuning for the final training of model on clean subset of the data: model_to_find_errors = GradientBoostingClassifier() # this model will be trained many times via cross-validation model_to_return = RandomizedSearchCV(GradientBoostingClassifier(), param_distributions = { "learning_rate": [0.001, 0.05, 0.1, 0.2, 0.5], "max_depth": [3, 5, 10], } ) # this model will be trained once on clean subset of data cl0 = CleanLearning(model_to_find_errors) issues = cl0.find_label_issues(data, labels) cl = CleanLearning(model_to_return).fit(data, labels, label_issues=issues) # CleanLearning for hyperparameter final training pred_probs = cl.predict_proba(data) # predictions from hyperparameter-tuned GradientBoostingClassifier print(cl0.clf) # will be GradientBoostingClassifier() print(cl.clf) # will be RandomizedSearchCV(estimator=GradientBoostingClassifier(),...) ``` -------------------------------- ### Get Active Learning Scores (Python) Source: https://docs.cleanlab.ai/stable/_modules/cleanlab/multiannotator Calculates active learning scores for examples to estimate which ones are most informative to relabel. Scores range from 0 to 1, with lower scores indicating examples where additional labels would be most beneficial. This function can handle examples with existing labels or unlabeled examples. ```python from typing import Optional, Tuple, Union import pandas as pd import numpy as np def get_active_learning_scores( labels_multiannotator: Optional[Union[pd.DataFrame, np.ndarray]] = None, pred_probs: Optional[np.ndarray] = None, pred_probs_unlabeled: Optional[np.ndarray] = None, ) -> Tuple[np.ndarray, np.ndarray]: """Returns an ActiveLab quality score for each example in the dataset, to estimate which examples are most informative to (re)label next in active learning. We consider settings where one example can be labeled by one or more annotators and some examples have no labels at all so far. The score is in between 0 and 1, and can be used to prioritize what data to collect additional labels for. Lower scores indicate examples whose true label we are least confident about based on the current data; collecting additional labels for these low-scoring examples will be more informative than collecting labels for other examples. To use an annotation budget most efficiently, select a batch of examples with the lowest scores and collect one additional label for each example, and repeat this process after retraining your classifier. You can use this function to get active learning scores for: examples that already have one or more labels (specify ``labels_multiannotator`` and ``pred_probs`` as arguments), or for unlabeled examples (specify ``pred_probs_unlabeled``), or for both types of examples (specify all of the above arguments). To analyze a fixed dataset labeled by multiple annotators rather than collecting additional labels, try the `~cleanlab.multiannotator.get_label_quality_multiannotator` (CROWDLAB) function instead. Parameters ---------- labels_multiannotator : pd.DataFrame or np.ndarray, optional 2D pandas DataFrame or array of multiple given labels for each example with shape ``(N, M)``, where N is the number of examples and M is the number of annotators. Note that this function also works with datasets where there is only one annotator (M=1). For more details, labels in the same format expected by the `~cleanlab.multiannotator.get_label_quality_multiannotator`. Note that examples that have no annotator labels should not be included in this DataFrame/array. This argument is optional if ``pred_probs`` is not provided (you might only provide ``pred_probs_unlabeled`` to only get active learning scores for the unlabeled examples). pred_probs : np.ndarray, optional An array of shape ``(N, K)`` of predicted class probabilities from a trained classifier model. Predicted probabilities in the same format expected by the :py:func:`get_label_quality_scores `. pred_probs_unlabeled : np.ndarray, optional An array of shape ``(U, K)`` of predicted class probabilities for unlabeled examples. Returns ------- Tuple[np.ndarray, np.ndarray] A tuple containing: - active_learning_scores: np.ndarray An array of active learning scores for the examples. - example_indices: np.ndarray The indices of the examples corresponding to the active learning scores. """ # Placeholder for actual implementation # This is a simplified representation of the function's logic if labels_multiannotator is None and pred_probs is None and pred_probs_unlabeled is None: raise ValueError("At least one of labels_multiannotator, pred_probs, or pred_probs_unlabeled must be provided.") # Simulate active learning score calculation # In a real scenario, this would involve complex calculations based on label quality and prediction probabilities. num_examples = 0 if labels_multiannotator is not None: num_examples = labels_multiannotator.shape[0] if isinstance(labels_multiannotator, np.ndarray) else labels_multiannotator.shape[0] elif pred_probs is not None: num_examples = pred_probs.shape[0] elif pred_probs_unlabeled is not None: num_examples = pred_probs_unlabeled.shape[0] active_learning_scores = np.random.rand(num_examples) example_indices = np.arange(num_examples) # If pred_probs_unlabeled is provided, append its scores and indices if pred_probs_unlabeled is not None: num_unlabeled = pred_probs_unlabeled.shape[0] unlabeled_scores = np.random.rand(num_unlabeled) # Simulate scores for unlabeled data unlabeled_indices = np.arange(num_examples, num_examples + num_unlabeled) # Assign new indices active_learning_scores = np.concatenate((active_learning_scores, unlabeled_scores)) example_indices = np.concatenate((example_indices, unlabeled_indices)) return active_learning_scores, example_indices ``` -------------------------------- ### Install Cleanlab Package Source: https://docs.cleanlab.ai/stable/tutorials/multiannotator Installs the cleanlab library using pip, which is necessary for running the tutorial's code. It's recommended to install the version corresponding to the documentation branch if working with the latest code. ```bash !pip install cleanlab # Make sure to install the version corresponding to this tutorial # E.g. if viewing master branch documentation: # !pip install git+https://github.com/cleanlab/cleanlab.git ``` -------------------------------- ### Install cleanlab from source with uv pip Source: https://docs.cleanlab.ai/stable/index Installs the cleanlab package directly from its GitHub repository using uv pip. The 'all' option installs all optional dependencies. ```bash uv pip install git+https://github.com/cleanlab/cleanlab.git ``` ```bash uv pip install "git+https://github.com/cleanlab/cleanlab.git#egg=cleanlab[all]" ``` -------------------------------- ### Install Dependencies with Pip Source: https://docs.cleanlab.ai/stable/tutorials/indepth_overview Installs necessary Python packages for the tutorial using pip. It includes matplotlib for plotting and cleanlab with datalab support. It also shows how to install directly from GitHub. ```shell !pip install matplotlib !pip install cleanlab[datalab] # Make sure to install the version corresponding to this tutorial # E.g. if viewing master branch documentation: # !pip install git+https://github.com/cleanlab/cleanlab.git ``` -------------------------------- ### Get Label Quality Scores Source: https://docs.cleanlab.ai/stable/_modules/cleanlab/experimental/label_issues_batched Fetches the computed label quality scores for all examples processed so far. Scores range from 0 to 1, with lower scores indicating potentially mislabeled examples. ```APIDOC ## GET /label_quality_scores ### Description Retrieves the label quality scores for all examples processed. Lower scores suggest potential mislabeling. ### Method GET ### Endpoint /label_quality_scores ### Parameters None ### Request Example None ### Response #### Success Response (200) - **label_quality_scores** (np.ndarray) - An array of scores (0-1) for each example. #### Response Example ```json { "label_quality_scores": [0.95, 0.12, 0.88, 0.34] } ``` ``` -------------------------------- ### Get All Example Issues Source: https://docs.cleanlab.ai/stable/tutorials/datalab/datalab_quickstart Returns detailed information for each individual example in the dataset regarding all detected issue types. Each example is assessed for the presence of an issue (boolean) and a quality score quantifying the severity of that issue for the specific example. Scores range from 0 to 1, with lower scores indicating more severe issues. ```python lab.get_issues().head() ``` -------------------------------- ### Install cleanlab from source with pip Source: https://docs.cleanlab.ai/stable/index Installs the cleanlab package directly from its GitHub repository using pip. The 'all' option installs all optional dependencies. ```bash pip install git+https://github.com/cleanlab/cleanlab.git ``` ```bash pip install "git+https://github.com/cleanlab/cleanlab.git#egg=cleanlab[all]" ``` -------------------------------- ### Faster CleanLearning Training with Pre-computed Label Issues (Python) Source: https://docs.cleanlab.ai/stable/tutorials/indepth_overview This example demonstrates how to speed up the CleanLearning fitting process by providing pre-computed label issues. When `label_issues` are supplied to the `fit` method, CleanLearning skips the label-issue identification step, leading to faster training times. This is useful when label issues have already been determined. ```python # CleanLearning can train faster if issues are provided at fitting time. cl.fit(data, labels, label_issues=issues) ``` -------------------------------- ### Get Single Annotator Agreement (Python) Source: https://docs.cleanlab.ai/stable/_modules/cleanlab/multiannotator Calculates the average agreement of a specific annotator with other annotators who have labeled the same examples. It considers only examples labeled by more than one annotator and weights the agreement based on the number of annotations per example. This function is a core component for understanding individual annotator reliability. ```python def _get_single_annotator_agreement( labels_multiannotator: np.ndarray, num_annotations: np.ndarray, annotator_idx: int, ) -> float: """Returns the average agreement of a given annotator other annotators that label the same example. Parameters ---------- labels_multiannotator : np.ndarray 2D numpy array of multiple given labels for each example with shape ``(N, M)``, where N is the number of examples and M is the number of annotators. For more details, labels in the same format expected by the `~cleanlab.multiannotator.get_label_quality_multiannotator`. num_annotations : np.ndarray An array of shape ``(N,)`` with the number of annotators that have labeled each example. annotator_idx : int The index of the annotator we want to compute the annotator agreement for. Returns ------- annotator_agreement : float An float repesenting the agreement of each annotator with other annotators that labeled the same examples. """ adjusted_num_annotations = num_annotations - 1 if np.sum(adjusted_num_annotations) == 0: return np.nan multi_annotations_mask = num_annotations > 1 annotator_agreement_per_example = np.zeros(len(labels_multiannotator)) for i in range(labels_multiannotator.shape[1]): annotator_agreement_per_example[multi_annotations_mask] += ( labels_multiannotator[multi_annotations_mask, annotator_idx] == labels_multiannotator[multi_annotations_mask, i] ) annotator_agreement_per_example[multi_annotations_mask] = ( annotator_agreement_per_example[multi_annotations_mask] - 1 ) / adjusted_num_annotations[multi_annotations_mask] annotator_agreement = np.average(annotator_agreement_per_example, weights=num_annotations - 1) return annotator_agreement ``` -------------------------------- ### Initialize CleanLearning with a Model and Cross-Validation Source: https://docs.cleanlab.ai/stable/tutorials/clean_learning/text Sets up the CleanLearning object, which wraps a scikit-learn compatible model. It's configured with the number of cross-validation folds to use for robust prediction probability estimation. This is crucial for cleanlab's error detection. ```python cv_n_folds = 5 cl = CleanLearning(model, cv_n_folds=cv_n_folds) ``` -------------------------------- ### Find Label Issues Directly (Python) Source: https://docs.cleanlab.ai/stable/tutorials/faq Illustrates how to directly find label issues using cleanlab's filter module when you have labels and predicted probabilities. It also shows how to get the predicted class for flagged examples and all examples. ```python import cleanlab import numpy as np # Assuming 'labels' and 'pred_probs' are defined # issues = cleanlab.filter.find_label_issues(labels, pred_probs) # Get predicted class for flagged examples # class_predicted_for_flagged_examples = pred_probs[issues].argmax(axis=1) # Get predicted class for all examples # class_predicted_for_all_examples = pred_probs.argmax(axis=1) ``` -------------------------------- ### Install Cleanlab Source: https://docs.cleanlab.ai/stable/tutorials/segmentation Installs the cleanlab library using pip, which is necessary for running the tutorial and utilizing its functionalities for data analysis. ```bash !pip install cleanlab ``` -------------------------------- ### Get Issues DataFrame from DataIssues Source: https://docs.cleanlab.ai/stable/cleanlab/datalab/internal/data_issues Retrieves a DataFrame detailing which examples in the dataset exhibit specific types of issues. If an issue_name is provided, it focuses on that issue type; otherwise, it returns a summary of all detected issues across all examples. ```python # Assume data_issues is an instance of DataIssues all_issues_df = data_issues.get_issues() # To get issues for a specific type: outlier_issues_df = data_issues.get_issues('outliers') ``` -------------------------------- ### Optimize CleanLearning Runtime with Optional Arguments Source: https://docs.cleanlab.ai/stable/tutorials/faq This example demonstrates how to reduce the runtime of CleanLearning's `find_label_issues` method by passing optional arguments. It shows how to decrease bootstrap rounds, skip aleatoric uncertainty estimation, and adjust search range parameters. ```python from cleanlab.regression.learn import CleanLearning import numpy as np X = np.random.random(size=(30, 3)) coefficients = np.random.uniform(-1, 1, size=3) y = np.dot(X, coefficients) + np.random.normal(scale=0.2, size=30) # passing optinal arguments to reduce runtime cl = CleanLearning(n_boot=1, include_aleatoric_uncertainty=False) cl.find_label_issues(X, y, coarse_search_range=[0.05, 0.1], fine_search_size=2) ``` -------------------------------- ### Get Number of Multilabel Examples (Python) Source: https://docs.cleanlab.ai/stable/_modules/cleanlab/multilabel_classification/dataset A helper method to determine the number of examples in a multilabel dataset. It prioritizes the `confident_joint` parameter if provided, otherwise it uses the `labels` parameter. Raises a `ValueError` if neither parameter is supplied, indicating insufficient information to determine the number of examples. ```python def _get_num_examples_multilabel(labels=None, confident_joint: Optional[np.ndarray] = None) -> int: """Helper method that finds the number of examples from the parameters or throws an error if neither parameter is provided. Parameters ---------- For parameter info, see the docstring of `~cleanlab.multilabel_classification.dataset.common_multilabel_issues`. Returns ------- num_examples : int The number of examples in the dataset. Raises ------ ValueError If `labels` is None. """ if labels is None and confident_joint is None: raise ValueError( "Error: num_examples is None. You must either provide confident_joint, " "or provide both num_example and joint as input parameters." ) _confident_joint = cast(np.ndarray, confident_joint) num_examples = len(labels) if labels is not None else cast(int, np.sum(_confident_joint[0])) return num_examples ``` -------------------------------- ### Get Label Quality Scores Source: https://docs.cleanlab.ai/stable/tutorials/multilabel_classification Extracts the label quality scores for each example in the dataset. These scores estimate the confidence that an example is correctly labeled, with lower scores indicating more suspect labels. The output is a NumPy array of these scores. ```python scores = label_issues["label_score"].values print(f"Label quality scores of the first 10 examples in dataset:\n{scores[:10]}") ``` -------------------------------- ### Install Cleanlab and Dependencies Source: https://docs.cleanlab.ai/stable/tutorials/multilabel_classification Installs the necessary packages for the Cleanlab tutorial using pip. This includes matplotlib for plotting and the 'cleanlab[datalab]' package for data labeling functionalities. ```bash !pip install matplotlib !pip install "cleanlab[datalab]" # Make sure to install the version corresponding to this tutorial # E.g. if viewing master branch documentation: # !pip install git+https://github.com/cleanlab/cleanlab.git ``` -------------------------------- ### Get Specific Issues DataFrame Source: https://docs.cleanlab.ai/stable/cleanlab/datalab/datalab Retrieves a DataFrame detailing which examples exhibit specific types of issues and their severity. If `issue_name` is None, it returns a summary of all detected issues across examples. ```python def get_issues(issue_name=None) -> DataFrame: """Use this after finding issues to see which examples suffer from which types of issues.""" pass ``` -------------------------------- ### Install cleanlab with uv pip Source: https://docs.cleanlab.ai/stable/index Installs the cleanlab package using uv pip. The 'all' option installs all optional dependencies for full functionality. ```bash uv pip install cleanlab ``` ```bash uv pip install "cleanlab[all]" ``` -------------------------------- ### Find and Train with CleanLearning (Python) Source: https://docs.cleanlab.ai/stable/tutorials/clean_learning/text This snippet shows how to initialize CleanLearning with an existing scikit-learn compatible model, find label issues in the training data, and then train a more robust model using the identified clean data. It assumes you have a model, train_data, and labels ready. ```python from cleanlab.classification import CleanLearning cl = CleanLearning(model) label_issues = cl.find_label_issues(train_data, labels) # identify mislabeled examples cl.fit(train_data, labels, label_issues=label_issues) preds = cl.predict(test_data) # predictions from a version of your model # trained on auto-cleaned data ``` -------------------------------- ### Get Label Quality Scores from LabelInspector Source: https://docs.cleanlab.ai/stable/_modules/cleanlab/experimental/label_issues_batched Retrieves pre-computed label quality scores for all examples processed so far. Scores range from 0 to 1, with lower scores indicating a higher likelihood of a mislabeled example. Requires the LabelInspector to be initialized with `store_results=True`. ```python label_quality_scores = inspector.get_label_quality_scores() ``` -------------------------------- ### Get Number of Label Issues Source: https://docs.cleanlab.ai/stable/cleanlab/experimental/label_issues_batched Fetches an already-computed estimate of the number of label issues in the data. This method provides a count of potentially mislabeled examples. ```python get_num_issues(_silent =False_)[source]# Fetches already-computed estimate of the number of label issues in the data seen so far in the same format as: `count.num_label_issues`. Note: The estimated number of issues may differ from `count.num_label_issues` by 1 due to rounding differences. Return type: `int` Returns: **num_issues** (`int`) – The estimated number of examples with label issues in the data seen so far. ``` -------------------------------- ### Import Libraries (Python) Source: https://docs.cleanlab.ai/stable/tutorials/clean_learning/text This code block imports essential Python libraries for data manipulation, machine learning, and natural language processing, including `pandas`, `sklearn` modules, `SentenceTransformer`, and `CleanLearning` from `cleanlab`. ```python import re import string import pandas as pd from sklearn.metrics import accuracy_score from sklearn.model_selection import train_test_split, cross_val_predict from sklearn.preprocessing import LabelEncoder from sklearn.linear_model import LogisticRegression from sentence_transformers import SentenceTransformer from cleanlab.classification import CleanLearning ``` -------------------------------- ### GET /get_issues Source: https://docs.cleanlab.ai/stable/cleanlab/datalab/datalab Retrieves the issues found in each example from the dataset. Can be filtered by a specific issue name. ```APIDOC ## GET /get_issues ### Description Use this after finding issues to see which examples suffer from which types of issues. If `issue_name` is None, returns a full DataFrame summarizing all detected issue types across all examples. ### Method GET ### Endpoint `/get_issues` ### Parameters #### Query Parameters * **`issue_name`** (`str` or `None`) - Optional. The type of issue to focus on. If None, returns full DataFrame summarizing all of the types of issues detected in each example from the dataset. ### Response #### Success Response (200) * **`DataFrame`** - A DataFrame where each row corresponds to an example from the dataset and columns specify: whether this example exhibits a particular type of issue, and how severely (via a numeric quality score where lower values indicate more severe instances of the issue). The quality scores lie between 0-1 and are directly comparable between examples (for the same issue type), but not across different issue types. Additional columns may be present depending on the issue type. #### Error Response (400) * **`ValueError`** - If `issue_name` is not a type of issue previously considered in the audit. ### Request Example (No request body for GET request) ### Response Example ```json { "issue_type_1": { "example_id_1": 0.1, "example_id_2": 0.5, ... }, "issue_type_2": { "example_id_3": 0.2, ... } } ``` ``` -------------------------------- ### Get Label Issues Source: https://docs.cleanlab.ai/stable/_modules/cleanlab/experimental/label_issues_batched Fetches the indices of examples identified as having label issues, sorted by their label quality score. This method corresponds to `cleanlab.filter.find_label_issues`. ```APIDOC ## GET /label_issues ### Description Retrieves the indices of examples identified with label issues, ranked by quality score. This is equivalent to calling `cleanlab.filter.find_label_issues`. ### Method GET ### Endpoint /label_issues ### Parameters None ### Request Example None ### Response #### Success Response (200) - **issue_indices** (np.ndarray) - An array of indices corresponding to examples with label issues, sorted by quality score. #### Response Example ```json { "issue_indices": [1, 3, 7, 10] } ``` ``` -------------------------------- ### Import Libraries for Cleanlab Tutorial Source: https://docs.cleanlab.ai/stable/tutorials/indepth_overview Imports essential Python libraries for the Cleanlab tutorial, including numpy for numerical operations, cleanlab modules for data quality and learning, and scikit-learn for machine learning models. ```python import numpy as np import cleanlab from cleanlab import Datalab from cleanlab.classification import CleanLearning from cleanlab.benchmarking import noise_generation from sklearn.linear_model import LogisticRegression from sklearn.model_selection import cross_val_predict from numpy.random import multivariate_normal from matplotlib import pyplot as plt ``` -------------------------------- ### Get Specific Data Issues Source: https://docs.cleanlab.ai/stable/cleanlab/datalab/internal/adapter/imagelab Retrieves a DataFrame detailing specific examples that exhibit particular types of issues. Each row represents an example, and columns indicate the presence and severity of issues. Additional columns may be present based on the issue type. ```python get_issues(_issue_name =None_) Parameters: **issue_name** (`str` or `None`) – The type of issue to focus on. If None, returns full DataFrame summarizing all of the types of issues detected in each example from the dataset. Return type: `DataFrame` Returns: `specific_issues` – A DataFrame where each row corresponds to an example from the dataset and columns specify: whether this example exhibits a particular type of issue and how severely (via a numeric quality score where lower values indicate more severe instances of the issue). Additional columns may be present in the DataFrame depending on the type of issue specified. ``` -------------------------------- ### Get Number of Examples in Python Source: https://docs.cleanlab.ai/stable/_modules/cleanlab/dataset This helper function determines the number of examples in a dataset. It can infer this count from either the provided labels or a confident joint distribution. If neither is supplied, it raises a ValueError. This is essential for various data analysis functions within the cleanlab library. ```python def _get_num_examples(labels=None, confident_joint: Optional[np.ndarray] = None) -> int: """Helper method that finds the number of examples from the parameters or throws an error if neither parameter is provided. **Parameters:** For information about the arguments to this method, see the documentation of `dataset.find_overlapping_classes` Returns ------- num_examples : int The number of examples in the dataset. Raises ------ ValueError If `labels` is None. """ if labels is None and confident_joint is None: raise ValueError( "Error: num_examples is None. You must either provide confident_joint, " "or provide both num_example and joint as input parameters." ) _confident_joint = cast(np.ndarray, confident_joint) num_examples = len(labels) if labels is not None else cast(int, np.sum(_confident_joint)) return num_examples ```