### Example of pip installation output Source: https://libr-ai.github.io/fairlib/tutorial/Structured_Inputs.html This output shows the process of installing fairlib and its dependencies, including PyYAML, torch, transformers, and others. It indicates which packages are already satisfied and which are being downloaded and collected. ```text Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/ Collecting fairlib Downloading fairlib-0.0.9-py3-none-any.whl (85 kB) |████████████████████████████████| 85 kB 2.6 MB/s Requirement already satisfied: PyYAML in /usr/local/lib/python3.7/dist-packages (from fairlib) (3.13) Requirement already satisfied: torch in /usr/local/lib/python3.7/dist-packages (from fairlib) (1.12.0+cu113) Collecting transformers Downloading transformers-4.20.1-py3-none-any.whl (4.4 MB) |████████████████████████████████| 4.4 MB 22.6 MB/s Requirement already satisfied: seaborn in /usr/local/lib/python3.7/dist-packages (from fairlib) (0.11.2) Requirement already satisfied: scikit-learn in /usr/local/lib/python3.7/dist-packages (from fairlib) (1.0.2) Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from fairlib) (4.64.0) Requirement already satisfied: docopt in /usr/local/lib/python3.7/dist-packages (from fairlib) (0.6.2) Collecting sacremoses Downloading sacremoses-0.0.53.tar.gz (880 kB) |████████████████████████████████| 880 kB 62.2 MB/s Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from fairlib) (1.21.6) Requirement already satisfied: pandas in /usr/local/lib/python3.7/dist-packages (from fairlib) (1.3.5) Requirement already satisfied: matplotlib in /usr/local/lib/python3.7/dist-packages (from fairlib) (3.2.2) Collecting pickle5 Downloading pickle5-0.0.12-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (256 kB) |████████████████████████████████| 256 kB 47.9 MB/s Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->fairlib) (3.0.9) Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->fairlib) (1.4.4) Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->fairlib) (2.8.2) Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib->fairlib) (0.11.0) Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from kiwisolver>=1.0.1->matplotlib->fairlib) (4.1.1) Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.1->matplotlib->fairlib) (1.15.0) Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.7/dist-packages (from pandas->fairlib) (2022.1) Requirement already satisfied: regex in /usr/local/lib/python3.7/dist-packages (from sacremoses->fairlib) (2022.6.2) Requirement already satisfied: click in /usr/local/lib/python3.7/dist-packages (from sacremoses->fairlib) (7.1.2) Requirement already satisfied: joblib in /usr/local/lib/python3.7/dist-packages (from sacremoses->fairlib) (1.1.0) Requirement already satisfied: scipy>=1.1.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn->fairlib) (1.7.3) Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn->fairlib) (3.1.0) Collecting huggingface-hub<1.0,>=0.1.0 Downloading huggingface_hub-0.8.1-py3-none-any.whl (101 kB) |████████████████████████████████| 101 kB 11.5 MB/s Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.7/dist-packages (from transformers->fairlib) (21.3) Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from transformers->fairlib) (2.23.0) Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/dist-packages (from transformers->fairlib) (4.12.0) Requirement already satisfied: filelock in /usr/local/lib/python3.7/dist-packages (from transformers->fairlib) (3.7.1) Collecting PyYAML Downloading PyYAML-6.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (596 kB) |████████████████████████████████| 596 kB 11.8 MB/s Collecting tokenizers!=0.11.3,<0.13,>=0.11.1 Downloading tokenizers-0.12.1-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (6.6 MB) |████████████████████████████████| 6.6 MB 33.9 MB/s Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata->transformers->fairlib) (3.8.1) Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->transformers->fairlib) (2.10) Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->transformers->fairlib) (3.0.4) Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->transformers->fairlib) (1.24.3) ``` -------------------------------- ### Install _fairlib_ with pip Source: https://libr-ai.github.io/fairlib/README.html Installs the _fairlib_ package using pip. It is recommended to start within a new virtual environment. ```bash conda create -n fairlib python=3.7 conda activate fairlib pip install fairlib ``` -------------------------------- ### Install _fairlib_ for Development Source: https://libr-ai.github.io/fairlib/README.html Sets up a development environment by cloning the repository and installing _fairlib_ in development mode. ```bash git clone https://github.com/HanXudong/fairlib.git ~/fairlib cd ~/fairlib python setup.py develop ``` -------------------------------- ### fairlib Installation Details Source: https://libr-ai.github.io/fairlib/tutorial_customized_evaluation.html This output shows the successful installation of the fairlib package and its dependencies, including versions of numpy, scikit-learn, and transformers. ```text Collecting fairlib Downloading fairlib-0.0.3-py3-none-any.whl (63 kB) [K |████████████████████████████████| 63 kB 2.0 MB/s [?25hRequirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from fairlib) (1.21.6) Requirement already satisfied: scikit-learn in /usr/local/lib/python3.7/dist-packages (from fairlib) (1.0.2) Requirement already satisfied: seaborn in /usr/local/lib/python3.7/dist-packages (from fairlib) (0.11.2) Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from fairlib) (4.64.0) Requirement already satisfied: docopt in /usr/local/lib/python3.7/dist-packages (from fairlib) (0.6.2) Requirement already satisfied: pandas in /usr/local/lib/python3.7/dist-packages (from fairlib) (1.3.5) Requirement already satisfied: PyYAML in /usr/local/lib/python3.7/dist-packages (from fairlib) (3.13) Collecting pickle5 Downloading pickle5-0.0.12-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (256 kB) [K |████████████████████████████████| 256 kB 29.1 MB/s [?25hRequirement already satisfied: matplotlib in /usr/local/lib/python3.7/dist-packages (from fairlib) (3.2.2) Collecting transformers Downloading transformers-4.18.0-py3-none-any.whl (4.0 MB) [K |████████████████████████████████| 4.0 MB 55.2 MB/s [?25hRequirement already satisfied: torch in /usr/local/lib/python3.7/dist-packages (from fairlib) (1.11.0+cu113) Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->fairlib) (2.8.2) Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->fairlib) (1.4.2) Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib->fairlib) (0.11.0) Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->fairlib) (3.0.8) Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from kiwisolver>=1.0.1->matplotlib->fairlib) (4.2.0) Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.1->matplotlib->fairlib) (1.15.0) Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.7/dist-packages (from pandas->fairlib) (2022.1) Requirement already satisfied: scipy>=1.1.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn->fairlib) (1.4.1) Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn->fairlib) (3.1.0) Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn->fairlib) (1.1.0) Collecting sacremoses Downloading sacremoses-0.0.53.tar.gz (880 kB) [K |████████████████████████████████| 880 kB 51.9 MB/s [?25hRequirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from transformers->fairlib) (2.23.0) Requirement already satisfied: filelock in /usr/local/lib/python3.7/dist-packages (from transformers->fairlib) (3.6.0) Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/dist-packages (from transformers->fairlib) (4.11.3) Collecting huggingface-hub<1.0,>=0.1.0 Downloading huggingface_hub-0.5.1-py3-none-any.whl (77 kB) [K |████████████████████████████████| 77 kB 6.9 MB/s [?25hCollecting PyYAML Downloading PyYAML-6.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (596 kB) [K |████████████████████████████████| 596 kB 60.7 MB/s [?25hRequirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.7/dist-packages (from transformers->fairlib) (2019.12.20) Collecting tokenizers!=0.11.3,<0.13,>=0.11.1 Downloading tokenizers-0.12.1-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (6.6 MB) [K |████████████████████████████████| 6.6 MB 54.8 MB/s [?25hRequirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.7/dist-packages (from transformers->fairlib) (21.3) Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata->transformers->fairlib) (3.8.0) Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->transformers->fairlib) (1.24.3) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->transformers->fairlib) (2021.10.8) Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->transformers->fairlib) (2.10) Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->transformers->fairlib) (3.0.4) Requirement already satisfied: click in /usr/local/lib/python3.7/dist-packages (from sacremoses->transformers->fairlib) (7.1.2) ``` -------------------------------- ### Fairlib Installation Dependencies Source: https://libr-ai.github.io/fairlib/tutorial/evaluation.html This output shows the successful installation of Fairlib and lists all its dependencies, including numpy, scikit-learn, seaborn, and others, along with their installed versions. It also indicates which packages were already satisfied. ```text Collecting fairlib Downloading fairlib-0.0.3-py3-none-any.whl (63 kB) |████████████████████████████████| 63 kB 2.0 MB/s Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from fairlib) (1.21.6) Requirement already satisfied: scikit-learn in /usr/local/lib/python3.7/dist-packages (from fairlib) (1.0.2) Requirement already satisfied: seaborn in /usr/local/lib/python3.7/dist-packages (from fairlib) (0.11.2) Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from fairlib) (4.64.0) Requirement already satisfied: docopt in /usr/local/lib/python3.7/dist-packages (from fairlib) (0.6.2) Requirement already satisfied: pandas in /usr/local/lib/python3.7/dist-packages (from fairlib) (1.3.5) Requirement already satisfied: PyYAML in /usr/local/lib/python3.7/dist-packages (from fairlib) (3.13) Collecting pickle5 Downloading pickle5-0.0.12-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (256 kB) |████████████████████████████████| 256 kB 29.1 MB/s Requirement already satisfied: matplotlib in /usr/local/lib/python3.7/dist-packages (from fairlib) (3.2.2) Collecting transformers Downloading transformers-4.18.0-py3-none-any.whl (4.0 MB) |████████████████████████████████| 4.0 MB 55.2 MB/s Requirement already satisfied: torch in /usr/local/lib/python3.7/dist-packages (from fairlib) (1.11.0+cu113) Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->fairlib) (2.8.2) Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->fairlib) (1.4.2) Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib->fairlib) (0.11.0) Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->fairlib) (3.0.8) Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from kiwisolver>=1.0.1->matplotlib->fairlib) (4.2.0) Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.1->matplotlib->fairlib) (1.15.0) Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.7/dist-packages (from pandas->fairlib) (2022.1) Requirement already satisfied: scipy>=1.1.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn->fairlib) (1.4.1) Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn->fairlib) (3.1.0) Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn->fairlib) (1.1.0) Collecting sacremoses Downloading sacremoses-0.0.53.tar.gz (880 kB) |████████████████████████████████| 880 kB 51.9 MB/s Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from transformers->fairlib) (2.23.0) Requirement already satisfied: filelock in /usr/local/lib/python3.7/dist-packages (from transformers->fairlib) (3.6.0) Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/dist-packages (from transformers->fairlib) (4.11.3) Collecting huggingface-hub<1.0,>=0.1.0 Downloading huggingface_hub-0.5.1-py3-none-any.whl (77 kB) |████████████████████████████████| 77 kB 6.9 MB/s Collecting PyYAML Downloading PyYAML-6.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (596 kB) |████████████████████████████████| 596 kB 60.7 MB/s Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.7/dist-packages (from transformers->fairlib) (2019.12.20) Collecting tokenizers!=0.11.3,<0.13,>=0.11.1 Downloading tokenizers-0.12.1-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (6.6 MB) |████████████████████████████████| 6.6 MB 54.8 MB/s Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.7/dist-packages (from transformers->fairlib) (21.3) Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata->transformers->fairlib) (3.8.0) Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->transformers->fairlib) (1.24.3) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->transformers->fairlib) (2021.10.8) Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->transformers->fairlib) (2.10) Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->transformers->fairlib) (3.0.4) Requirement already satisfied: click in /usr/local/lib/python3.7/dist-packages (from sacremoses->transformers->fairlib) (7.1.2) Building wheels for collected packages: sacremoses ``` -------------------------------- ### Setup Plotting Environment Source: https://libr-ai.github.io/fairlib/tutorial/interactive_plots.html Set up the plotting environment using Matplotlib and Altair. The `%matplotlib inline` magic command is used for displaying plots directly in the notebook. Ensure necessary libraries like altair and vega_datasets are installed. ```python # Notice: make sure the enable widget when using interactive plots # %matplotlib widget %matplotlib inline import altair as alt from vega_datasets import data ``` -------------------------------- ### DAdv Example Command Source: https://libr-ai.github.io/fairlib/reference_component_hyperparameter_tuning.html Example command to run DAdv with specific hyperparameter settings. Use --adv_num_subDiscriminator to set the number of subdiscriminators and --adv_diverse_lambda to control the strength of the difference loss. ```bash python main.py --adv_debiasing --adv_num_subDiscriminator 3 --adv_diverse_lambda 10 ``` -------------------------------- ### Import and Use _fairlib_ Package Source: https://libr-ai.github.io/fairlib/README.html Demonstrates how to import and use _fairlib_ as a Python package. This involves loading options from a configuration file, initializing a model, and starting the training process with debiasing. ```python from fairlib.base_options import options from src import networks config_file = 'opt.yaml' # Get options state = options.get_state(conf_file=config_file) # Init the model model = networks.get_main_model(state) # Training with debiasing model.train_self() ``` -------------------------------- ### Import Fairlib Library Source: https://libr-ai.github.io/fairlib/tutorial/interactive_plots.html Import the Fairlib library to access its analysis and plotting functionalities. Ensure Fairlib is installed in your environment. ```python import fairlib ``` -------------------------------- ### Example LATEX Table Output Source: https://libr-ai.github.io/fairlib/tutorial/fairlib_demo.html This is an example of the LATEX table output generated by the `to_latex()` method. It includes various performance and fairness metrics for different models. ```latex \begin{tabular}{lrrrrrrrrrlll} \toprule Models & test_performance mean & test_performance std & test_fairness mean & test_fairness std & dev_performance mean & dev_performance std & dev_fairness mean & dev_fairness std & DTO & epoch list & opt_dir list & is_pareto \\ \midrule GDEO & 0.752763 & 0.004999 & 0.892255 & 0.007860 & 0.749350 & 0.003494 & 0.912672 & 0.002766 & 0.269694 & [2, 12, 5, 9, 10] & [G:\\Experimental_results\\GroupDifference\\Moji\\G... & True \\ BTFairBatch & 0.746837 & 0.003407 & 0.899351 & 0.004936 & 0.743975 & 0.004236 & 0.919254 & 0.004731 & 0.272437 & [8, 6, 9, 5, 7] & [G:\\Experimental_results\\FairBatch\\Moji\\BTInitF... & False \\ Vanilla & 0.722981 & 0.004576 & 0.611870 & 0.014356 & 0.726650 & 0.003673 & 0.632302 & 0.013370 & 0.476849 & [2, 11, 2, 5, 2] & [G:\\Experimental_results\\vanilla\\Moji\\0\\opt.yam... & True \\ BTEO & 0.753927 & 0.001433 & 0.877469 & 0.003756 & 0.746325 & 0.000998 & 0.896874 & 0.005401 & 0.274892 & [8, 6, 9, 4, 5] & [G:\\Experimental_results\\GatedBT\\Moji\\GatedBT_R... & True \\ GatedDAdv & 0.750163 & 0.006945 & 0.908679 & 0.021678 & 0.745600 & 0.004828 & 0.928670 & 0.022488 & 0.266004 & [24, 13, 19, 4, 3] & [G:\\Experimental_results\\hypertune3\\Moji\\hypert... & False \\ FairBatch & 0.751488 & 0.005772 & 0.904373 & 0.008213 & 0.746050 & 0.003896 & 0.914526 & 0.006020 & 0.266276 & [9, 9, 6, 6, 5] & [G:\\Experimental_results\\FairBatch\\Moji\\FairBat... & True \\ GatedAdv & 0.753113 & 0.005196 & 0.890065 & 0.013302 & 0.748975 & 0.003805 & 0.910838 & 0.010314 & 0.270257 & [11, 4, 4, 13, 12] & [G:\\Experimental_results\\hypertune2\\Moji\\hypert... & False \\ DelayedCLS_Adv & 0.761015 & 0.003081 & 0.882425 & 0.015918 & 0.751675 & 0.003481 & 0.899346 & 0.011417 & 0.266341 & [13, 1, 3, 13, 1] & [/data/cephfs/punim1421/Fair_NLP_Classification... & True \\ GDMean & 0.752163 & 0.002130 & 0.901389 & 0.003916 & 0.749050 & 0.001368 & 0.922430 & 0.005829 & 0.266735 & [11, 12, 7, 2, 2] & [G:\\Experimental_results\\GroupDifference\\Moji\\G... & True \\ GatedBTEO & 0.762106 & 0.002592 & 0.900764 & 0.014701 & 0.759775 & 0.003798 & 0.909445 & 0.006631 & 0.257762 & [3, 1, 3, 1, 11] & [G:\\Experimental_results\\GatedBT\\Moji\\GatedBT_R... & True \\ BTGatedAdv & 0.735459 & 0.028830 & 0.866150 & 0.028232 & 0.730150 & 0.024594 & 0.886862 & 0.030537 & 0.296476 & [5, 19, 19, 6, 1] & [G:\\Experimental_results\\hypertune2\\Moji\\hypert... & True \\ Adv & 0.756414 & 0.007271 & 0.893286 & 0.005623 & 0.747425 & 0.004549 & 0.912125 & 0.008507 & 0.265936 & [9, 14, 18, 5, 16] & [G:\\Experimental_results\\hypertune\\Moji\\hypertu... & True \\ OldFairBatch & 0.750638 & 0.006012 & 0.905537 & 0.005046 & 0.744525 & 0.004995 & 0.917734 & 0.004761 & 0.266655 & [5, 7, 8, 13, 7] & [G:\\Experimental_results\\Original_FairBatch\\Moj... & True \\ ``` -------------------------------- ### Main Function for Model Training and Post-processing Source: https://libr-ai.github.io/fairlib/tutorial_customized_methods.html The main function initializes options, gets the model state, trains the model, and then conditionally runs post-processing methods like INLP if specified in the state. ```python def main(): options = BaseOptions() state = options.get_state() # Init the model model = networks.get_main_model(state) # state.opt.main_model = model logging.info('Model Initialized!') model.train_self() logging.info('Model Trained!') if state.INLP: logging.info('Run INLP') from src.networks.INLP import get_INLP_trade_offs get_INLP_trade_offs(model, state) logging.info('Finished!') ``` -------------------------------- ### Custom Dataset for Image Loading Source: https://libr-ai.github.io/fairlib/tutorial/Image_Inputs.html Create a custom dataset by inheriting from BaseDataset. This example defines a transform for image normalization and loads data from a specified file. ```python class CustomizedDataset(dataloaders.utils.BaseDataset): transform=transforms.Compose([ transforms.ToTensor(), transforms.Normalize((0.1307, 0.1307, 0.), (0.3081, 0.3081, 0.3081))]) def load_data(self): self.data_dir = os.path.join(self.args.data_dir, "colored_MNIST_{}.pt".format(self.split)) data = torch.load(self.data_dir) self.X = [self.transform(_img) for _img in data[0]] self.y = data[1] self.protected_label = data[2] ``` -------------------------------- ### Model Training Output Log Source: https://libr-ai.github.io/fairlib/tutorial/fairlib_demo.html Example log output from the `train_self` method, showing epoch progress, loss, data time, training time, and evaluation metrics like accuracy and F-score. ```log 2022-07-21 06:53:10 [INFO ] Epoch: 0 [ 0/ 99998 ( 0%)] Loss: 0.6907 Data Time: 0.06s Train Time: 0.18s 2022-07-21 06:53:14 [INFO ] Epoch: 0 [ 51200/ 99998 (51%)] Loss: 0.4075 Data Time: 0.44s Train Time: 3.34s 2022-07-21 06:53:18 [INFO ] Evaluation at Epoch 0 2022-07-21 06:53:18 [INFO ] Validation accuracy: 71.44 macro_fscore: 71.35 micro_fscore: 71.44 TPR_GAP: 39.59 FPR_GAP: 39.59 PPR_GAP: 38.67 2022-07-21 06:53:18 [INFO ] Test accuracy: 71.21 macro_fscore: 71.12 micro_fscore: 71.21 TPR_GAP: 39.42 FPR_GAP: 39.42 PPR_GAP: 38.49 2022-07-21 06:53:18 [INFO ] Epoch: 1 [ 0/ 99998 ( 0%)] Loss: 0.4195 Data Time: 0.01s Train Time: 0.07s 2022-07-21 06:53:22 [INFO ] Epoch: 1 [ 51200/ 99998 (51%)] Loss: 0.3780 Data Time: 0.42s Train Time: 3.29s 2022-07-21 06:53:26 [INFO ] Evaluation at Epoch 1 2022-07-21 06:53:26 [INFO ] Validation accuracy: 72.10 macro_fscore: 72.06 micro_fscore: 72.10 TPR_GAP: 37.69 FPR_GAP: 37.69 PPR_GAP: 37.00 2022-07-21 06:53:26 [INFO ] Test accuracy: 71.72 macro_fscore: 71.68 micro_fscore: 71.72 TPR_GAP: 37.11 FPR_GAP: 37.11 PPR_GAP: 36.41 2022-07-21 06:53:27 [INFO ] Epoch: 2 [ 0/ 99998 ( 0%)] Loss: 0.3608 Data Time: 0.01s Train Time: 0.07s 2022-07-21 06:53:30 [INFO ] Epoch: 2 [ 51200/ 99998 (51%)] Loss: 0.3567 Data Time: 0.42s Train Time: 3.29s 2022-07-21 06:53:34 [INFO ] Epochs since last improvement: 1 2022-07-21 06:53:34 [INFO ] Evaluation at Epoch 2 2022-07-21 06:53:34 [INFO ] Validation accuracy: 71.53 macro_fscore: 71.44 micro_fscore: 71.53 TPR_GAP: 39.40 FPR_GAP: 39.40 PPR_GAP: 38.40 2022-07-21 06:53:34 [INFO ] Test accuracy: 71.38 macro_fscore: 71.28 micro_fscore: 71.38 TPR_GAP: 39.07 FPR_GAP: 39.07 PPR_GAP: 37.99 2022-07-21 06:53:35 [INFO ] Epoch: 3 [ 0/ 99998 ( 0%)] Loss: 0.3584 Data Time: 0.01s Train Time: 0.07s 2022-07-21 06:53:38 [INFO ] Epoch: 3 [ 51200/ 99998 (51%)] Loss: 0.3430 Data Time: 0.42s Train Time: 3.28s 2022-07-21 06:53:42 [INFO ] Epochs since last improvement: 2 2022-07-21 06:53:42 [INFO ] Evaluation at Epoch 3 2022-07-21 06:53:43 [INFO ] Validation accuracy: 71.08 macro_fscore: 70.73 micro_fscore: 71.08 TPR_GAP: 41.08 FPR_GAP: 41.08 PPR_GAP: 38.75 2022-07-21 06:53:43 [INFO ] Test accuracy: 70.87 macro_fscore: 70.49 micro_fscore: 70.87 TPR_GAP: 40.67 FPR_GAP: 40.67 PPR_GAP: 38.42 2022-07-21 06:53:43 [INFO ] Epoch: 4 [ 0/ 99998 ( 0%)] Loss: 0.3578 Data Time: 0.02s Train Time: 0.07s 2022-07-21 06:53:46 [INFO ] Epoch: 4 [ 51200/ 99998 (51%)] Loss: 0.3881 Data Time: 0.44s Train Time: 3.36s 2022-07-21 06:53:50 [INFO ] Epochs since last improvement: 3 2022-07-21 06:53:50 [INFO ] Evaluation at Epoch 4 2022-07-21 06:53:51 [INFO ] Validation accuracy: 72.06 macro_fscore: 71.96 micro_fscore: 72.06 TPR_GAP: 38.04 FPR_GAP: 38.04 PPR_GAP: 36.87 2022-07-21 06:53:51 [INFO ] Test accuracy: 71.72 macro_fscore: 71.61 micro_fscore: 71.72 TPR_GAP: 38.09 FPR_GAP: 38.09 PPR_GAP: 36.96 2022-07-21 06:53:51 [INFO ] Epoch: 5 [ 0/ 99998 ( 0%)] Loss: 0.4008 Data Time: 0.02s Train Time: 0.07s 2022-07-21 06:53:54 [INFO ] Epoch: 5 [ 51200/ 99998 (51%)] Loss: 0.3393 Data Time: 0.43s Train Time: 3.29s 2022-07-21 06:53:58 [INFO ] Epochs since last improvement: 4 2022-07-21 06:53:58 [INFO ] Evaluation at Epoch 5 2022-07-21 06:53:59 [INFO ] Validation accuracy: 71.44 macro_fscore: 71.41 micro_fscore: 71.44 TPR_GAP: 40.50 FPR_GAP: 40.50 PPR_GAP: 39.97 2022-07-21 06:53:59 [INFO ] Test accuracy: 71.13 macro_fscore: 71.11 micro_fscore: 71.13 TPR_GAP: 39.69 FPR_GAP: 39.69 PPR_GAP: 39.19 2022-07-21 06:53:59 [INFO ] Epoch: 6 [ 0/ 99998 ( 0%)] Loss: 0.3890 Data Time: 0.01s Train Time: 0.07s 2022-07-21 06:54:03 [INFO ] Epoch: 6 [ 51200/ 99998 (51%)] Loss: 0.3668 Data Time: 0.44s Train Time: 3.32s 2022-07-21 06:54:07 [INFO ] Epochs since last improvement: 5 2022-07-21 06:54:07 [INFO ] Evaluation at Epoch 6 2022-07-21 06:54:07 [INFO ] Validation accuracy: 71.96 macro_fscore: 71.92 micro_fscore: 71.96 TPR_GAP: 37.52 FPR_GAP: 37.52 PPR_GAP: 36.82 2022-07-21 06:54:07 [INFO ] Test accuracy: 72.04 macro_fscore: 72.00 micro_fscore: 72.04 TPR_GAP: 36.75 FPR_GAP: 36.75 PPR_GAP: 36.01 ``` -------------------------------- ### Configure Debiasing Arguments in Fairlib Source: https://libr-ai.github.io/fairlib/tutorial/fairlib_demo.html Specify arguments for debiasing methods, including experiment ID, adversarial debiasing flag, and Balanced Training (BT) parameters like 'Downsampling' and 'EO' objective. This setup is essential for enabling advanced bias mitigation techniques. ```python debiasing_args = args.copy() # Update the experiment name debiasi_args["exp_id"] = "BT_Adv" # Perform adversarial training if True debiasi_args["adv_debiasing"] = True # Specify the hyperparameters for Balanced Training debiasi_args["BT"] = "Downsampling" debiasi_args["BTObj"] = "EO" debias_options = fairlib.BaseOptions() debias_state = debias_options.get_state(args=debiasing_args, silence=True) fairlib.utils.seed_everything(2022) debias_model = fairlib.networks.get_main_model(debias_state) ``` -------------------------------- ### Initialize Fairlib Options and State Source: https://libr-ai.github.io/fairlib/tutorial/evaluation.html Set up experiment-specific arguments by merging shared options and initializing the Fairlib BaseOptions. This prepares the environment for training. ```python args = { "dataset":Shared_options["dataset"], "data_dir":Shared_options["data_dir"], "device_id":Shared_options["device_id"], # Give a name to the exp, which will be used in the path "exp_id":"vanilla", } # Init the argument options = fairlib.BaseOptions() state = options.get_state(args=args, silence=True) fairlib.utils.seed_everything(2022) ``` -------------------------------- ### Configure Experiment Options Source: https://libr-ai.github.io/fairlib/tutorial/fairlib_demo.html Initialize experiment arguments and state, including dataset, data directory, device ID, and experiment ID. This sets up the configuration for training. ```python args = { # The name of the dataset, corresponding dataloader will be used, "dataset": "Moji", # Specifiy the path to the input data "data_dir": "data/deepmoji", # Device for computing, -1 is the cpu; non-negative numbers indicate GPU id. "device_id": -1, # Give a name to the exp, which will be used in the path "exp_id":"vanilla", } # Init the argument options = fairlib.BaseOptions() state = options.get_state(args=args, silence=True) ``` -------------------------------- ### Reproduce Experiments with Configuration File Source: https://libr-ai.github.io/fairlib/tutorial_usage.html Use a configuration file to reproduce experiments by specifying the path to the opt.yaml file. ```python python fairlib --conf_file PATH_TO_EXP/opt.yaml ``` -------------------------------- ### Initialize Fairlib Options and Seed Source: https://libr-ai.github.io/fairlib/tutorial_customized_evaluation.html Set up experiment-specific arguments by inheriting from shared options, then initialize the fairlib BaseOptions and seed the random number generator for reproducibility. ```python args = { "dataset":Shared_options["dataset"], "data_dir":Shared_options["data_dir"], "device_id":Shared_options["device_id"], # Give a name to the exp, which will be used in the path "exp_id":"vanilla", } # Init the argument options = fairlib.BaseOptions() state = options.get_state(args=args, silence=True) fairlib.utils.seed_everything(2022) ``` -------------------------------- ### Get data distribution after sampling Source: https://libr-ai.github.io/fairlib/tutorial/manipulate_data_distribution.html Calculate the data distribution of the resampled data to verify the effect of generalized sampling. ```python balanced_g_distribution = get_data_distribution( y_data = y[balanced_g_indices], g_data = g[balanced_g_indices]) ``` -------------------------------- ### Enable Inline Plotting Source: https://libr-ai.github.io/fairlib/tutorial/fairlib_demo.html This command enables matplotlib plots to be displayed directly within the notebook or script. It's a standard setup for data visualization. ```python %matplotlib inline ``` -------------------------------- ### Run INLP with main.py Source: https://libr-ai.github.io/fairlib/reference_component_hyperparameter_tuning.html This command initiates the INLP process by running the main script with the --INLP flag. It serves as the entry point for applying the INLP method. ```python python main.py --INLP ```