### Quickstart: Method Training Source: https://github.com/dssg/aequitas/blob/master/README.md This section provides code examples for training different types of Fair ML methods: pre-processing, in-processing, and post-processing. ```APIDOC ## Method Training Quickstarts ### Description These snippets illustrate how to train and apply pre-processing, in-processing, and post-processing Fair ML methods using the Aequitas toolkit. They assume the existence of an `aequitas.flow.Dataset` object with train, validation, and test splits. ### Method N/A (Code examples, not API endpoints) ### Endpoint N/A ### Pre-processing Methods Example #### Description Example of using a pre-processing method like `PrevalenceSampling` to transform the training data. #### Request Example ```python from aequitas.flow.methods.preprocessing import PrevalenceSampling sampler = PrevalenceSampling() sampler.fit(dataset.train.X, dataset.train.y, dataset.train.s) X_sample, y_sample, s_sample = sampler.transform(dataset.train.X, dataset.train.y, dataset.train.s) ``` ### In-processing Methods Example #### Description Example of training an in-processing method like `FairGBM` on the pre-processed data. #### Request Example ```python from aequitas.flow.methods.inprocessing import FairGBM model = FairGBM() model.fit(X_sample, y_sample, s_sample) # Using pre-processed data scores_val = model.predict_proba(dataset.validation.X, dataset.validation.y, dataset.validation.s) scores_test = model.predict_proba(dataset.test.X, dataset.test.y, dataset.test.s) ``` ### Post-processing Methods Example #### Description Example of applying a post-processing method like `BalancedGroupThreshold` to adjust scores based on validation results. #### Request Example ```python from aequitas.flow.methods.postprocessing import BalancedGroupThreshold threshold = BalancedGroupThreshold("top_pct", 0.1, "fpr") threshold.fit(dataset.validation.X, scores_val, dataset.validation.y, dataset.validation.s) corrected_scores = threshold.transform(dataset.test.X, scores_test, dataset.test.s) ``` ### Response #### Success Response (Implicit) - **Transformed Data**: Modified feature sets (`X_sample`, `y_sample`, `s_sample`). - **Model Scores**: Probability scores from the in-processing model (`scores_val`, `scores_test`). - **Corrected Scores**: Adjusted scores after post-processing (`corrected_scores`). #### Response Example (No direct JSON response, outputs are data transformations and scores.) ``` -------------------------------- ### Install and Build Documentation with nbsphinx Source: https://github.com/dssg/aequitas/blob/master/docs/readme.md Installs nbsphinx and the Sphinx Read the Docs theme, then builds the documentation. Requires pandoc to be installed separately. This command initiates the documentation build process. ```bash python3 -m pip install nbsphinx python3 -m pip install sphinx_rtd_theme cd aequitas python3 -m sphinx docs/source docs ``` -------------------------------- ### Install Aequitas Python Library from Source Source: https://github.com/dssg/aequitas/blob/master/docs/_sources/installation.ipynb.txt Installs the Aequitas Python library by running the setup.py script. This method is useful for development or when you need the latest unreleased version. Ensure Python 3 is installed. ```bash python setup.py install ``` -------------------------------- ### Install and Configure Aequitas Environment Source: https://github.com/dssg/aequitas/blob/master/docs/source/examples/aequitas_flow_model_audit_and_correct.ipynb Installs the Aequitas library from the repository and initializes the logging environment to prevent duplicate logs in interactive notebooks. ```python !pip install git+https://github.com/dssg/aequitas.git@release-fixes &> /dev/null from aequitas.flow.utils.logging import clean_handlers clean_handlers() ``` -------------------------------- ### Install Aequitas using pip Source: https://github.com/dssg/aequitas/blob/master/README.md This snippet shows how to install the Aequitas library using pip, either from PyPI or directly from the GitHub repository. Ensure you have pip installed to use these commands. ```bash pip install aequitas ``` ```bash pip install git+https://github.com/dssg/aequitas.git ``` -------------------------------- ### Setup Virtual Environment with uv Source: https://github.com/dssg/aequitas/blob/master/DEVELOPMENT.md Sets up and instantiates a Python virtual environment using Astral's uv tool. This ensures project dependencies are managed correctly. ```bash cd aequitas uv run python # Instantiate a python shell with the venv. ``` -------------------------------- ### Install Aequitas Fairflow Source: https://github.com/dssg/aequitas/blob/master/src/aequitas/flow/README.md Install the Aequitas package via pip to access Fairflow functionality. ```bash pip install "aequitas>=0.43.0" ``` -------------------------------- ### Get Aequitas Version Source: https://github.com/dssg/aequitas/blob/master/viz_testing_notebook.ipynb Retrieves and displays the installed version of the aequitas library using pkg_resources. ```python pkg_resources.get_distribution("aequitas").version ``` -------------------------------- ### Install Aequitas Python Library using Pip Source: https://github.com/dssg/aequitas/blob/master/docs/_sources/installation.ipynb.txt Installs the Aequitas Python library directly from its GitHub repository using pip. This is a convenient way to install the library if it's available on PyPI or directly from a VCS. Requires pip and Python 3. ```bash pip install git+https://github.com/dssg/aequitas.git ``` -------------------------------- ### Set Up Live Preview with sphinx-autobuild Source: https://github.com/dssg/aequitas/blob/master/docs/readme.md Installs sphinx-autobuild for live previewing of documentation changes. This tool monitors source files and automatically rebuilds and reloads the browser when changes are detected. ```bash python3 -m pip install sphinx-autobuild --user python3 -m sphinx_autobuild docs/source docs ``` -------------------------------- ### Get Pandas Version Source: https://github.com/dssg/aequitas/blob/master/viz_testing_notebook.ipynb Retrieves and displays the installed version of the pandas library. ```python pd.__version__ ``` -------------------------------- ### Quickstart: Bias Reduction Experiment Source: https://github.com/dssg/aequitas/blob/master/README.md This snippet demonstrates how to perform a bias reduction experiment using the DefaultExperiment class. It requires a dataset with label, sensitive attribute, and features. ```APIDOC ## DefaultExperiment: Bias Reduction Experiment ### Description This endpoint demonstrates how to run a bias reduction experiment using the `DefaultExperiment` class from the Aequitas toolkit. It requires a pandas DataFrame as input, along with the names of the target and sensitive features. The experiment size can be configured. ### Method N/A (This is a code example, not an API endpoint) ### Endpoint N/A ### Parameters #### Request Body (Implicit) - **dataset** (pandas.DataFrame) - The input dataset containing features, a label column, and a sensitive attribute column. - **target_feature** (string) - The name of the column representing the target variable. - **sensitive_feature** (string) - The name of the column representing the sensitive attribute. - **experiment_size** (string) - Defines the scale of the experiment. Options: `test`, `small`, `medium`, `large`. ### Request Example ```python from aequitas.flow import DefaultExperiment # Assuming 'dataset' is a pandas DataFrame with appropriate columns experiment = DefaultExperiment.from_pandas(dataset, target_feature="label_value", sensitive_feature="attr", experiment_size="small") experiment.run() experiment.plot_pareto() ``` ### Response #### Success Response (Implicit) - **Plot**: A Pareto plot visualizing the results of the bias reduction experiment. #### Response Example (A plot image is generated, not a JSON response) ![Pareto Plot Example](https://raw.githubusercontent.com/dssg/aequitas/master/docs/_images/pareto_example.png) ``` -------------------------------- ### Initialize Aequitas Environment Source: https://github.com/dssg/aequitas/blob/master/docs/examples/Untitled.html Imports the essential libraries required for data manipulation, visualization, and Aequitas-specific preprocessing. This is the standard boilerplate for starting an Aequitas analysis workflow. ```python import pandas as pd import seaborn as sns from aequitas.preprocessing import preprocess_input_df ``` -------------------------------- ### Initialize Aequitas and Data Environment Source: https://github.com/dssg/aequitas/blob/master/docs/source/examples/compas_demo.ipynb Imports necessary Aequitas modules and data science libraries to perform bias auditing. This setup is required before running any fairness or disparity calculations. ```python import pandas as pd import seaborn as sns from aequitas.group import Group from aequitas.bias import Bias from aequitas.fairness import Fairness from aequitas.plotting import Plot %matplotlib inline ``` -------------------------------- ### Launch Aequitas Web Server Source: https://github.com/dssg/aequitas/blob/master/docs/_sources/30_seconds_webapp.ipynb.txt Starts the Aequitas web application server locally using the Python module execution command. The server typically listens on port 5000 by default. ```bash python -m serve ``` -------------------------------- ### Visualize Parity for Multiple Metrics - Python Source: https://github.com/dssg/aequitas/blob/master/docs/examples/compas_demo.html Generates treemaps to visualize disparities between attribute groups for multiple user-specified disparity metrics. This example focuses on 'pprev_disparity' and 'ppr_disparity'. ```python r_tm = aqp.plot_fairness_disparity_all(fdf, metrics=['pprev_disparity', 'ppr_disparity'], significance_alpha=0.05) ``` -------------------------------- ### Visualize Parity for Multiple Attributes - Python Source: https://github.com/dssg/aequitas/blob/master/docs/examples/compas_demo.html Generates treemaps to visualize disparities between attribute groups for multiple user-specified attributes. This example visualizes disparities for 'sex' and 'age_cat' attributes using default disparity metrics. ```python n_tm = aqp.plot_fairness_disparity_all(fdf, attributes=['sex', 'age_cat'], significance_alpha=0.05) ``` -------------------------------- ### Configure and Run Aequitas Experiment Source: https://context7.com/dssg/aequitas/llms.txt Demonstrates how to define an experiment configuration dictionary including datasets, methods, and optimization parameters, then execute the experiment and visualize results. ```python config = { "datasets": [{ "MyDataset": { "classpath": "aequitas.flow.datasets.GenericDataset", "threshold": {"type": "fixed", "value": 0.5}, "args": { "df": df, "label_column": "label", "sensitive_column": "sensitive", "categorical_columns": ["cat_feature"] } } }], "methods": [{ "FairGBM_Method": { "inprocessing": { "model": { "classpath": "aequitas.flow.methods.inprocessing.FairGBM", "args": {"n_estimators": [50, 100, 200]} } } } }], "optimization": { "sampler": "TPESampler", "sampler_args": {"seed": 42}, "n_trials": 20, "n_jobs": 1 } } experiment = Experiment(config=config) experiment.run() experiment.plot_pareto(dataset="MyDataset") ``` -------------------------------- ### Run Fair ML Experiments with DefaultExperiment Source: https://context7.com/dssg/aequitas/llms.txt Provides a quick entry point for running fair ML experiments. It automatically handles preprocessing and inprocessing methods with default hyperparameter configurations. ```python from aequitas.flow import DefaultExperiment experiment = DefaultExperiment( df=df, label_column="label", sensitive_column="sensitive", categorical_columns=["feature2"], threshold_type="fixed", score_threshold=0.5, dataset_name="MyDataset", methods="all", experiment_size="small", use_baseline=True ) experiment.run() experiment.plot_pareto(dataset="MyDataset", fairness_metric="Predictive Equality", performance_metric="TPR", split="test") ``` -------------------------------- ### GET /fairness/overall Source: https://github.com/dssg/aequitas/blob/master/docs/source/examples/compas_demo.ipynb Provides a high-level boolean assessment of model fairness. ```APIDOC ## GET /fairness/overall ### Description Returns a dictionary summarizing the overall, supervised, and unsupervised fairness status across all attributes. ### Method GET ### Endpoint /fairness/overall ### Parameters #### Request Body - **fdf** (DataFrame) - Required - The dataframe returned from the get_group_value_fairness or get_group_attribute_fairness methods. ### Response #### Success Response (200) - **fairness_summary** (Dictionary) - Contains keys 'Unsupervised Fairness', 'Supervised Fairness', and 'Overall Fairness' with boolean values. ``` -------------------------------- ### POST /experiment/run Source: https://context7.com/dssg/aequitas/llms.txt Executes a Fair ML experiment using either default settings or custom configurations. ```APIDOC ## POST /experiment/run ### Description Runs an automated Fair ML experiment to train models and evaluate them against fairness metrics. ### Method POST ### Endpoint /experiment/run ### Parameters #### Request Body - **df** (DataFrame) - Required - Dataset for the experiment. - **label_column** (string) - Required - Target variable column. - **sensitive_column** (string) - Required - Sensitive attribute column. - **methods** (string/list) - Optional - Fairness methods to apply (e.g., "all"). - **experiment_size** (string) - Optional - Scale of the experiment (e.g., "small", "medium"). ### Request Example { "label_column": "label", "sensitive_column": "sensitive", "methods": "all" } ### Response #### Success Response (200) - **status** (string) - Completion status of the experiment. #### Response Example { "status": "success" } ``` -------------------------------- ### GET /fairness/group_attribute Source: https://github.com/dssg/aequitas/blob/master/docs/source/examples/compas_demo.ipynb Aggregates fairness parity determinations at the attribute level. ```APIDOC ## GET /fairness/group_attribute ### Description Summarizes the fairness parity results from group-level calculations into attribute-level boolean indicators. ### Method GET ### Endpoint /fairness/group_attribute ### Parameters #### Request Body - **fdf** (DataFrame) - Required - The dataframe returned from the get_group_value_fairness method. ### Response #### Success Response (200) - **attribute_fairness_df** (DataFrame) - A dataframe where rows represent attributes and columns represent boolean parity status for various metrics. ``` -------------------------------- ### Run Bias Reduction Experiment with DefaultExperiment Source: https://github.com/dssg/aequitas/blob/master/README.md Demonstrates how to initialize and run a bias reduction experiment using the DefaultExperiment class. It requires a dataset with target and sensitive features, and allows configuration of experiment size and methods. ```python from aequitas.flow import DefaultExperiment experiment = DefaultExperiment.from_pandas(dataset, target_feature="label_value", sensitive_feature="attr", experiment_size="small") experiment.run() experiment.plot_pareto() ``` -------------------------------- ### GET /fairness/parities Source: https://github.com/dssg/aequitas/blob/master/docs/_modules/src/aequitas/fairness.html Retrieves a list of all parity determinations present in the provided dataframe. ```APIDOC ## GET /fairness/parities ### Description View a list of all parity determinations available in the provided dataframe. ### Method GET ### Endpoint /fairness/parities ### Parameters #### Query Parameters - **df** (DataFrame) - Required - The dataframe containing parity columns. ### Response #### Success Response (200) - **parities** (List) - A list of strings representing the parity determinations found. ### Response Example { "parities": ["TPR Parity", "FPR Parity", "Statistical Parity"] } ``` -------------------------------- ### Aequitas Initialization and Data Loading Source: https://github.com/dssg/aequitas/blob/master/viz_testing_notebook.ipynb This snippet shows how to load the Aequitas library, check its version, and load a sample dataset. ```APIDOC ## Aequitas Initialization and Data Loading ### Description This section demonstrates the initial setup for using the Aequitas library, including loading necessary modules and preparing a sample dataset for analysis. ### Method N/A (Script Execution) ### Endpoint N/A ### Parameters N/A ### Request Example ```python import pandas as pd from aequitas.group import Group from aequitas.bias import Bias from aequitas import plot import pkg_resources # Load and display the first few rows of the dataset df = pd.read_csv("./examples/data/compas_for_aequitas.csv") df.head() ``` ### Response #### Success Response (200) N/A (Output includes DataFrame head and version information) #### Response Example ``` பட்டா age_cat race sex ... label_value score label 0 1 18 - 25 Black M ... 0 0.5 0 1 2 25 - 45 Black M ... 0 0.9 1 2 3 25 - 45 Black F ... 0 0.7 0 3 4 25 - 45 Black M ... 0 0.4 0 4 5 25 - 45 Black M ... 0 0.6 0 [5 rows x 14 columns] ``` ``` '0.41.0' ``` ``` -------------------------------- ### GET /plot_fairness_disparity Source: https://github.com/dssg/aequitas/blob/master/docs/examples/compas_demo.html Visualizes disparity metrics for a specific attribute relative to a reference group. ```APIDOC ## GET /plot_fairness_disparity ### Description Displays disparity values between groups for a specified metric and attribute, colored by fairness determination. ### Method GET ### Endpoint /plot_fairness_disparity ### Parameters #### Query Parameters - **fdf** (DataFrame) - Required - The fairness dataframe. - **group_metric** (string) - Required - The disparity metric to plot (e.g., 'fdr'). - **attribute_name** (string) - Required - The attribute to analyze. - **min_group_size** (float) - Optional - Filter groups by minimum sample size percentage. - **significance_alpha** (float) - Optional - Statistical significance threshold. ### Request Example { "fdf": "dataframe_object", "group_metric": "fdr", "attribute_name": "race", "min_group_size": 0.01 } ### Response #### Success Response (200) - **plot** (object) - A visualization object representing the disparity metrics. ``` -------------------------------- ### Initialize Sphinx Documentation Options Source: https://github.com/dssg/aequitas/blob/master/docs/examples/Untitled.html This JavaScript snippet initializes the Sphinx documentation environment, setting the root URL, version, and language configuration for the documentation site. ```javascript var DOCUMENTATION_OPTIONS = { URL_ROOT:'../', VERSION:'', LANGUAGE:'None', COLLAPSE_INDEX:false, FILE_SUFFIX:'.html', HAS_SOURCE: true, SOURCELINK_SUFFIX: '.txt' }; jQuery(function () { SphinxRtdTheme.Navigation.enable(true); }); ``` -------------------------------- ### GET /plot_fairness_group Source: https://github.com/dssg/aequitas/blob/master/docs/examples/compas_demo.html Visualizes absolute group metrics for a specific metric across all attributes. ```APIDOC ## GET /plot_fairness_group ### Description Displays absolute group metric values across each attribute, color-coded by fairness determination (True/False). ### Method GET ### Endpoint /plot_fairness_group ### Parameters #### Query Parameters - **fdf** (DataFrame) - Required - The fairness dataframe returned from the Fairness() class. - **group_metric** (string) - Required - The specific metric to visualize (e.g., 'ppr'). ### Request Example { "fdf": "dataframe_object", "group_metric": "ppr" } ### Response #### Success Response (200) - **plot** (object) - A visualization object representing the group metrics. ``` -------------------------------- ### GET /bias/list_significance Source: https://github.com/dssg/aequitas/blob/master/docs/_modules/src/aequitas/bias.html Retrieves a list of all calculated significance metrics present in the provided dataframe. ```APIDOC ## GET /bias/list_significance ### Description Returns a list of column names from the dataframe that contain '_significance', indicating calculated statistical significance metrics. ### Method GET ### Endpoint /bias/list_significance ### Parameters #### Request Body - **df** (DataFrame) - Required - The dataframe containing calculated bias metrics. ### Request Example { "df": "" } ### Response #### Success Response (200) - **columns** (list) - A list of strings representing the names of significance metric columns. #### Response Example ["precision_significance", "tpr_significance"] ``` -------------------------------- ### GET /bias/list_disparities Source: https://github.com/dssg/aequitas/blob/master/docs/_modules/src/aequitas/bias.html Retrieves a list of all calculated disparity metrics present in the provided dataframe. ```APIDOC ## GET /bias/list_disparities ### Description Returns a list of column names from the dataframe that contain '_disparity', indicating calculated disparity metrics. ### Method GET ### Endpoint /bias/list_disparities ### Parameters #### Request Body - **df** (DataFrame) - Required - The dataframe containing calculated bias metrics. ### Request Example { "df": "" } ### Response #### Success Response (200) - **columns** (list) - A list of strings representing the names of disparity metric columns. #### Response Example ["precision_disparity", "tpr_disparity"] ``` -------------------------------- ### Load Model and Dataset for Fairness Audit Source: https://github.com/dssg/aequitas/blob/master/docs/source/examples/aequitas_flow_model_audit_and_correct.ipynb Demonstrates how to load a serialized model using pickle and instantiate a dataset object from the Aequitas library, including splitting data for validation and testing. ```python import pickle with open("examples/experiment_results/lgbm_baf_sample.pickle", "rb") as f: model = pickle.load(f) from aequitas.flow.datasets import BankAccountFraud dataset = BankAccountFraud("Sample") dataset.load_data() dataset.create_splits() validation = dataset.validation test = dataset.test ``` -------------------------------- ### GET /_plot_multiple Source: https://github.com/dssg/aequitas/blob/master/docs/_modules/src/aequitas/plotting.html Generates a grid of bar charts for multiple absolute metrics or disparity measures. ```APIDOC ## GET /_plot_multiple ### Description Plots multiple bar charts of absolute metrics or disparity measures in a grid layout. ### Method GET ### Parameters #### Query Parameters - **data_table** (DataFrame) - Required - Output from get_crosstabs, get_disparity, or get_fairness. - **plot_fcn** (callable) - Required - The single-metric plotting function to use. - **metrics** (list) - Optional - List of metrics to plot. Defaults to common metrics like pprev, ppr, fdr, etc. - **fillzeros** (boolean) - Optional - Whether to fill null values with zeros. Default: True. - **ncols** (int) - Optional - Number of columns in the subplot grid. Default: 3. ``` -------------------------------- ### Execute Fair ML Pipeline Source: https://github.com/dssg/aequitas/blob/master/src/aequitas/flow/README.md Demonstrates the sequence of pre-processing, in-processing, and post-processing steps to train a fair model and generate final scores. ```python from aequitas.fairflow.methods.preprocessing import PrevalenceSampling from aequitas.fairflow.methods.inprocessing import FairGBM from aequitas.fairflow.methods.postprocessing import GroupThreshold sampling = PrevalenceSampling() sampling.fit(*splits["train"]) sampled_data = sampling.transform(*splits["train"]) model = FairGBM() model.fit(*sampled_data) preds = model.predict_proba(*splits["validation"]) threshold = GroupThreshold() threshold.fit(*splits["validation"], preds) final_scores = threshold.transform(*splits["validation"], preds) ``` -------------------------------- ### GET /bias/list_absolute_metrics Source: https://github.com/dssg/aequitas/blob/master/docs/_modules/src/aequitas/bias.html Retrieves a list of all calculated absolute bias metrics present in the provided dataframe. ```APIDOC ## GET /bias/list_absolute_metrics ### Description Returns a list of column names that represent absolute bias metrics by intersecting the input group metrics with the columns available in the dataframe. ### Method GET ### Endpoint /bias/list_absolute_metrics ### Parameters #### Request Body - **df** (DataFrame) - Required - The dataframe containing calculated bias metrics. ### Request Example { "df": "" } ### Response #### Success Response (200) - **columns** (list) - A list of strings representing the names of absolute metric columns. #### Response Example ["binary_fpr", "binary_tpr", "binary_ppr"] ``` -------------------------------- ### GET /fairness/group_value Source: https://github.com/dssg/aequitas/blob/master/docs/source/examples/compas_demo.ipynb Calculates fairness parity metrics for specific group values based on disparity data. ```APIDOC ## GET /fairness/group_value ### Description Calculates parity metrics for individual group values by applying fairness thresholds to disparity data. ### Method GET ### Endpoint /fairness/group_value ### Parameters #### Request Body - **disparity_df** (DataFrame) - Required - The dataframe returned from the Bias() class get_disparity method. ### Response #### Success Response (200) - **fairness_df** (DataFrame) - A dataframe containing additional columns indicating parity status for each metric. ``` -------------------------------- ### Integrate FairlearnClassifier with Reductions Source: https://context7.com/dssg/aequitas/llms.txt Shows how to use FairlearnClassifier for integrating Fairlearn's reduction methods (ExponentiatedGradient and GridSearch) into the training pipeline. It covers setting up the classifier with different estimators, constraints, and parameters for both reduction types. ```python import pandas as pd import numpy as np from aequitas.flow.methods.inprocessing import FairlearnClassifier # Prepare data np.random.seed(42) n_samples = 500 X_train = pd.DataFrame({ "feature1": np.random.randn(n_samples), "feature2": np.random.randn(n_samples) }) y_train = pd.Series(np.random.randint(0, 2, n_samples)) s_train = pd.Series(np.random.choice([0, 1], n_samples)) # Create classifier with ExponentiatedGradient reduction model = FairlearnClassifier( reduction="fairlearn.reductions.ExponentiatedGradient", estimator="sklearn.linear_model.LogisticRegression", constraint="fairlearn.reductions.EqualizedOdds", # Model parameters prefixed with 'model__' model__C=1.0, model__max_iter=1000, # Constraint parameters prefixed with 'constraint__' constraint__difference_bound=0.01 ) # Or with GridSearch reduction model_grid = FairlearnClassifier( reduction="fairlearn.reductions.GridSearch", estimator="lightgbm.LGBMClassifier", constraint="fairlearn.reductions.DemographicParity", model__n_estimators=50, model__learning_rate=0.1 ) # Train with sensitive features model.fit(X_train, y_train, s_train) # Predict probabilities scores = model.predict_proba(X_train, s_train) print(f"Predictions: {scores[:5].values}") ``` -------------------------------- ### GET /plot_fairness_disparity Source: https://github.com/dssg/aequitas/blob/master/docs/_modules/src/aequitas/plotting.html Visualizes disparity metrics for a specific model, highlighting fairness across different population groups. ```APIDOC ## GET /plot_fairness_disparity ### Description Generates a plot of disparity metrics colored based on calculated disparity values for a specific attribute. This method requires a fairness table containing exactly one model_id. ### Method GET ### Parameters #### Query Parameters - **fairness_table** (DataFrame) - Required - The disparity table containing model data. - **group_metric** (string) - Required - The metric to plot (must exist in the disparity_table). - **attribute_name** (string) - Required - The attribute to plot the group_metric across. - **ax** (matplotlib.axes.Axes) - Optional - A matplotlib Axis object. - **fig** (matplotlib.figure.Figure) - Optional - A matplotlib Figure object. - **title** (boolean) - Optional - Whether to include a title. Default: True. - **min_group_size** (float) - Optional - Minimum proportion of total group size for inclusion. - **significance_alpha** (float) - Optional - Statistical significance level. Default: 0.05. ### Response #### Success Response (200) - **ax** (matplotlib.axes.Axes) - The generated Matplotlib axis object. ``` -------------------------------- ### Initialize Google Analytics and Tooltips Source: https://github.com/dssg/aequitas/blob/master/src/aequitas_webapp/templates/base.html This snippet includes the Google Analytics tracking configuration and the jQuery document ready function to initialize Bootstrap tooltips across the application. ```javascript window.dataLayer = window.dataLayer || []; function gtag() { dataLayer.push(arguments); } gtag('js', new Date()); gtag('config', 'UA-118975934-1'); $(document).ready(function () { $('[data-toggle="tooltip"]').tooltip(); }); ``` -------------------------------- ### Get Overall Fairness Source: https://github.com/dssg/aequitas/blob/master/docs/examples/compas_demo.html Provides a consolidated boolean assessment of fairness across all attributes for unsupervised, supervised, and overall fairness metrics. ```APIDOC ## GET /dssg/aequitas/fairness/overall ### Description Calculates an overall fairness assessment based on the results from `get_group_value_fairness` or `get_group_attribute_fairness`. ### Method GET ### Endpoint /dssg/aequitas/fairness/overall ### Parameters #### Query Parameters - **dataframe** (dataframe) - Required - The input dataframe containing fairness metrics. ### Request Example ```json { "dataframe": "[dataframe content]" } ``` ### Response #### Success Response (200) - **dict** (dict) - A dictionary containing boolean assessments for 'Unsupervised Fairness', 'Supervised Fairness', and 'Overall Fairness'. #### Response Example ```json { "dict": { "Unsupervised Fairness": false, "Supervised Fairness": false, "Overall Fairness": false } } ``` ``` -------------------------------- ### GET /group/crosstabs Source: https://github.com/dssg/aequitas/blob/master/docs/examples/Untitled.html Generates a crosstab of fairness metrics (such as TPR, TNR, FPR, etc.) for a provided dataframe, grouped by specified attributes. ```APIDOC ## GET /group/crosstabs ### Description Calculates fairness metrics for a given dataset by grouping data based on categorical attributes. This is the primary method for performing a bias audit on model predictions. ### Method GET ### Endpoint /group/crosstabs ### Parameters #### Request Body - **df** (DataFrame) - Required - The input pandas DataFrame containing entity_id, score, label_value, and protected attributes. ### Request Example { "df": "[pandas.DataFrame]" } ### Response #### Success Response (200) - **attribute_name** (string) - The name of the attribute used for grouping. - **attribute_value** (string) - The specific value within the attribute category. - **tpr** (float) - True Positive Rate. - **fpr** (float) - False Positive Rate. - **tnr** (float) - True Negative Rate. - **pprev** (float) - Predicted Prevalence. #### Response Example { "attribute_name": "race", "attribute_value": "African-American", "tpr": 0.720147, "fpr": 0.448468 } ``` -------------------------------- ### Get Group Attribute Fairness Source: https://github.com/dssg/aequitas/blob/master/docs/examples/compas_demo.html Provides a summarized view of fairness calculations at the attribute level, extracting only the parity determinations from the group value fairness results. ```APIDOC ## GET /dssg/aequitas/fairness/group_attribute ### Description Retrieves attribute-level fairness determinations, summarizing the parity results calculated by `get_group_value_fairness`. ### Method GET ### Endpoint /dssg/aequitas/fairness/group_attribute ### Parameters #### Query Parameters - **dataframe** (dataframe) - Required - The input dataframe containing fairness metrics. ### Request Example ```json { "dataframe": "[dataframe content]" } ``` ### Response #### Success Response (200) - **dataframe** (dataframe) - A dataframe showing fairness parity results for each attribute. #### Response Example ```json { "dataframe": "[attribute level fairness dataframe]" } ``` ``` -------------------------------- ### Load and Split Datasets Source: https://github.com/dssg/aequitas/blob/master/src/aequitas/flow/README.md Load a dataset from a file path and create training, validation, and testing splits using the GenericDataset interface. ```python from aequitas.fairflow.datasets import GenericDataset dataset = GenericDataset(path="dataset.csv") dataset.load_data() splits = dataset.create_splits() ``` -------------------------------- ### Input Data Preprocessing Source: https://github.com/dssg/aequitas/blob/master/docs/input_data.html Demonstrates how to preprocess input data for the Aequitas Python package using `preprocess_input_df`. ```APIDOC ## Input Data Preprocessing ### Description This section explains how to preprocess input data for the Aequitas Python package. The `preprocess_input_df()` function can be used to handle input data similarly to the CLI. Alternatively, continuous attribute columns must be discretized before being passed to `Group().get_crosstabs()`. ### Method Python Package Function Call ### Endpoint N/A (Python Package) ### Parameters #### Path Parameters N/A #### Query Parameters N/A #### Request Body - **input_data** (*dict*) - Required - The input data to be preprocessed. ### Request Example ```python from Aequitas.preprocessing import preprocess_input_df # Assume *input_data* is a dictionary containing your data df, _ = preprocess_input_df(*input_data) ``` ### Response #### Success Response (200) - **df** (*pandas.DataFrame*) - The preprocessed DataFrame. - **_** (*object*) - An object containing additional information (often ignored). #### Response Example ```json { "df": "[pandas DataFrame representation]", "_": "[additional info object]" } ``` ``` -------------------------------- ### Configure Advanced Experiments with Experiment Class Source: https://context7.com/dssg/aequitas/llms.txt Demonstrates fine-grained control over experiments using YAML configuration files. This is suitable for complex workflows requiring specific artifact saving and custom settings. ```python from pathlib import Path from aequitas.flow import Experiment experiment = Experiment( config_file=Path("experiment_config.yaml"), save_artifacts=True, save_folder=Path("./artifacts"), artifacts=("results", "methods", "predictions"), name="my_experiment" ) ``` -------------------------------- ### Initialize Sphinx Read the Docs Navigation Source: https://github.com/dssg/aequitas/blob/master/docs/examples/index.html Initializes the navigation menu for the Sphinx documentation theme using jQuery. This ensures the sidebar and mobile menus are functional for the user. ```javascript jQuery(function () { SphinxRtdTheme.Navigation.enable(true); }); ``` -------------------------------- ### Train Fair ML Methods (Pre-, In-, and Post-processing) Source: https://github.com/dssg/aequitas/blob/master/README.md Shows the workflow for training different types of bias mitigation algorithms. It includes sampling data for pre-processing, training a FairGBM model for in-processing, and applying thresholding for post-processing. ```python # Pre-processing from aequitas.flow.methods.preprocessing import PrevalenceSampling sampler = PrevalenceSampling() sampler.fit(dataset.train.X, dataset.train.y, dataset.train.s) X_sample, y_sample, s_sample = sampler.transform(dataset.train.X, dataset.train.y, dataset.train.s) # In-processing from aequitas.flow.methods.inprocessing import FairGBM model = FairGBM() model.fit(X_sample, y_sample, s_sample) scores_val = model.predict_proba(dataset.validation.X, dataset.validation.y, dataset.validation.s) scores_test = model.predict_proba(dataset.test.X, dataset.test.y, dataset.test.s) # Post-processing from aequitas.flow.methods.postprocessing import BalancedGroupThreshold threshold = BalancedGroupThreshold("top_pct", 0.1, "fpr") threshold.fit(dataset.validation.X, scores_val, dataset.validation.y, dataset.validation.s) corrected_scores = threshold.transform(dataset.test.X, scores_test, dataset.test.s) ``` -------------------------------- ### Get Supported Fairness Measures Source: https://github.com/dssg/aequitas/blob/master/docs/_modules/src/aequitas/fairness.html Determines the fairness measures supported based on the columns present in the input DataFrame. If 'label_value' is not found, it defaults to 'Statistical Parity' and 'Impact Parity'. ```APIDOC ## GET /api/fairness/measures/supported ### Description Determine fairness measures supported based on columns in data frame. ### Method GET ### Endpoint /api/fairness/measures/supported ### Parameters #### Query Parameters - **input_df** (DataFrame) - Required - The input DataFrame containing the data to analyze. ### Request Example ```json { "input_df": "" } ``` ### Response #### Success Response (200) - **fair_measures_supported** (list) - A list of supported fairness measures. #### Response Example ```json { "fair_measures_supported": [ "Statistical Parity", "Impact Parity" ] } ``` ``` -------------------------------- ### Run Aequitas Fairflow Benchmark Source: https://github.com/dssg/aequitas/blob/master/src/aequitas/flow/README.md Instantiate the Orchestrator or Benchmark class to execute an experiment based on configuration files and save results to a specified directory. ```python from aequitas.fairflow import Orchestrator # Instantiate the benchmark and define the folder where to save results benchmark = Benchmark(config_path, save_folder=Path("results")) benchmark.run() ``` -------------------------------- ### Perform Bias Auditing with the Audit Class Source: https://context7.com/dssg/aequitas/llms.txt Demonstrates how to initialize an Audit object with a DataFrame containing predictions, labels, and sensitive attributes. It covers running the audit process and accessing confusion matrices, fairness metrics, and disparity ratios. ```python import pandas as pd from aequitas import Audit df = pd.DataFrame({ "score": [1, 0, 1, 1, 0, 0, 1, 0, 1, 0], "label_value": [1, 0, 1, 0, 0, 1, 1, 0, 1, 0], "race": ["A", "A", "A", "B", "B", "B", "C", "C", "C", "C"], "gender": ["M", "F", "M", "F", "M", "F", "M", "F", "M", "F"] }) audit = Audit( df=df, score_column="score", label_column="label_value", sensitive_attribute_column=["race", "gender"], reference_groups="maj" ) audit.audit() print(audit.confusion_matrix) print(audit.metrics) print(audit.disparities) performance = audit.performance() print(performance) ``` -------------------------------- ### Plot Multiple Fairness Disparities Source: https://github.com/dssg/aequitas/blob/master/docs/api/aequitas.html Plots multiple fairness disparity metrics at once from a fairness object table. This is useful for getting a comprehensive overview of fairness across different attributes and metrics. ```APIDOC ## POST /dssg/aequitas/plot_disparity_all ### Description Plot multiple metrics at once from a fairness object table. ### Method POST ### Endpoint /dssg/aequitas/plot_disparity_all ### Parameters #### Request Body - **data_table** (object) - Required - Output of group.get_crosstabs, bias.get_disparity, or fairness.get_fairness functions. - **attributes** (list) - Optional - Which attribute(s) to plot metrics for. If this value is null, will plot metrics against all attributes. - **metrics** (list or string) - Optional - Which metric(s) to plot, or ‘all.’ If this value is null, will plot Predicted Prevalence Disparity (pprev_disparity), Predicted Positive Rate Disparity (ppr_disparity), False Discovery Rate Disparity (fdr_disparity), False Omission Rate Disparity (for_disparity), False Positive Rate Disparity (fpr_disparity), False Negative Rate Disparity (fnr_disparity). - **fillzeros** (boolean) - Optional - Whether to fill null values with zeros. Default is True. - **title** (boolean) - Optional - Whether to display a title on each plot. Default is True. - **label_dict** (object) - Optional - Optional dictionary of label replacements. Default is None. - **show_figure** (boolean) - Optional - Whether to show figure (plt.show()). Default is True. - **min_group_size** (float) - Optional - Minimum proportion of total group size (all data) a population group must meet in order to be included in metric visualization. - **significance_alpha** (float) - Optional - Statistical significance level. Used to determine visual representation of significance (number of asterisks on treemap). ### Response #### Success Response (200) - **figure** (object) - A Matplotlib figure ### Response Example { "figure": "
" } ``` -------------------------------- ### Train FairGBM Model with Fairness Constraints Source: https://context7.com/dssg/aequitas/llms.txt Demonstrates how to create and train a FairGBM model with specified fairness constraints, such as False Negative Rate parity. The model is fitted using training data and sensitive attributes, and probability predictions are generated for validation data. ```python from aequitas.flow.models import FairGBM # Create and train FairGBM model model = FairGBM( n_estimators=100, multiplier_learning_rate=0.1, # Fairness constraint learning rate constraint_type="FNR" # Constraint on False Negative Rate parity ) # Fit with sensitive attribute as constraint group model.fit(X_train, y_train, s_train) # Get probability predictions scores = model.predict_proba(X_val, s_val) print(f"Predictions shape: {scores.shape}") print(f"Score range: [{scores.min():.3f}, {scores.max():.3f}]") ``` -------------------------------- ### Initialize Fairness Dependency Configuration Source: https://github.com/dssg/aequitas/blob/master/docs/_modules/src/aequitas/fairness.html Sets the default high-level fairness dependencies if not provided. It maps unsupervised and supervised fairness categories to their respective parity measures. ```python if not high_level_fairness_depend: self.high_level_fairness_depend = { 'Unsupervised Fairness': ['Statistical Parity', 'Impact Parity'], 'Supervised Fairness': ['TypeI Parity', 'TypeII Parity'] } else: self.high_level_fairness_depend = high_level_fairness_depend ``` -------------------------------- ### Python: Preprocess Input Data with Aequitas Source: https://github.com/dssg/aequitas/blob/master/docs/_sources/input_data.ipynb.txt This snippet shows how to use the `preprocess_input_df` function from the Aequitas library to prepare input data for the Python package, ensuring it matches CLI input norms. It handles the initial loading and basic preprocessing of the dataframe. ```python from Aequitas.preprocessing import preprocess_input_df() # *input_data* matches CLI input data norms. df, _ = preprocess_input_df(*input_data*) ``` -------------------------------- ### Import Aequitas and Pandas Libraries Source: https://github.com/dssg/aequitas/blob/master/viz_testing_notebook.ipynb Imports necessary libraries for data manipulation (pandas) and bias analysis (aequitas). It also imports plotting utilities and a package for checking installed distribution versions. ```python import pandas as pd from aequitas.group import Group from aequitas.bias import Bias from aequitas import plot import pkg_resources ```