CoChat
Sign Up
Cited by 261Ethics and Social Impacts of AIPrivacy-Preserving Technologies in DataExplainable Artificial Intelligence (XAI)

A survey on datasets for fairness‐aware machine learning

Open access
Le Quy, Tai, Roy, Arjun, Iosifidis, Vasileios, Zhang, Wenbin, Ntoutsi, Eirini·2022·WIREs Data Mining and Knowledge Discovery
DOI 10.1002/widm.1452
Open this paper in CoChat — free accountOpen PDF

Abstract

As decision‐making increasingly relies on machine learning (ML) and (big) data, the issue of fairness in data‐driven artificial intelligence systems is receiving increasing attention from both research and industry. A large variety of fairness‐aware ML solutions have been proposed which involve fairness‐related interventions in the data, learning algorithms, and/or model outputs. However, a vital part of proposing new approaches is evaluating them empirically on benchmark datasets that represent realistic and diverse settings. Therefore, in this paper, we overview real‐world datasets used for fairness‐aware ML. We focus on tabular data as the most common data representation for fairness‐aware ML. We start our analysis by identifying relationships between the different attributes, particularly with respect to protected attributes and class attribute, using a Bayesian network. For a deeper understanding of bias in the datasets, we investigate interesting relationships using exploratory analysis. This article is categorized under: Commercial, Legal, and Ethical Issues > Fairness in Data Mining Fundamental Concepts of Data and Knowledge > Data Concepts Technologies > Data Preprocessing

AI snapshot

Written from the paper’s full text.

TL;DR

This survey reviews real-world tabular datasets used for fairness-aware machine learning, organizes them by application domain, and analyzes attribute dependencies with Bayesian networks. It also benchmarks logistic regression across 15 datasets using predictive and fairness metrics, finding substantial variation in both bias and performance.

Key findings

  • The survey covers financial, criminological, healthcare/social, and educational datasets; prevalent protected attributes are sex/gender, race, age, and marriage.
  • Bayesian-network analysis shows direct or indirect dependencies between protected attributes and class labels in many datasets, indicating dataset-level bias; examples include Adult, KDD Census-Income, Dutch census, COMPAS, Communities and Crime, and Law School.
  • In the logistic-regression evaluation, 10/15 datasets had absolute statistical parity below 0.1; Diabetes, Credit card clients, and OULAD had SP = 0.0, while COMPAS recid. had the worst equalized odds (0.6452), followed by Law School (0.5043) and Communities and Crime (0.4507).
  • Ricci was an exception with accuracy 1.0, equalized odds 0.0, and ABROCA 0.0; the worst ABROCA cases were German credit and COMPAS.

Methods

  • Dataset collection used Google Scholar with “fairness datasets” plus terms such as “bias,” “discrimination,” and “public”; included datasets had to be public via repositories such as UCI or Kaggle and used in at least three fairness-related papers.
  • For each dataset, the authors learned a Bayesian network with the class attribute constrained as a leaf node, discretizing continuous variables, and used the graph plus exploratory analysis to examine relationships involving protected attributes.
  • Quantitative evaluation used logistic regression with a 70%:30% train/test split, reporting accuracy, balanced accuracy, TPR/TNR by protected/non-protected group, statistical parity, equalized odds, and ABROCA.

Limitations

  • Stated: the experimental evaluation is limited to logistic regression and does not compare pre-, in-, and post-processing fairness-aware methods; for each dataset only one popular protected attribute is demonstrated.
  • Stated: the survey focuses on tabular data, notes that other data types such as text and images are also used in fairness-aware ML, and reports that surveyed datasets are old on average (about 20 years; oldest 48 years, newest 7 years).
  • Evident from the text: Bayesian-network structure depends on discretization and preprocessing choices, and the dataset inclusion criteria may exclude relevant datasets that are not public or have fewer than three fairness-related paper uses.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1002/widm.1452

Cites

136 of 136 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 90 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    In the Adult dataset, in the high education segment and high-income class, the number of males is at least five times higher than that of females.

    “in the high education segment and for the high-income class, the number of males is at least five times higher than that of females”
    Quote in the paperSubstantial claim
  • Findingcitation

    The Adult dataset is one of the most popular datasets for fairness-aware classification studies.

    “The adult dataset (Kohavi, 1996) (also known as “Census Income” dataset 6 ) is one of the most popular datasets for fairness-aware classification studies (Appendix A).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Kohavi, 1996

      Not checked in the free run.

  • Findingown data

    The Adult dataset has missing values in 3,620 (7.41%) records.

    “Missing values exist in 3,620 (7.41%) records.”
    Quote in the paperNot a substantial claim
  • Findingown data

    In the KDD Census-Income dataset, the number of men with high income is approximately five times more than the number of women.

    “the number of men with high income is approximately five times more than the number of women.”
    Quote in the paperNot a substantial claim
  • Findingreasoning

    After cleaning, the KDD Census-Income dataset has 284,556 instances.

    “The result is a cleaned dataset with 284,556 instances.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim. 111
  • May combine two claims. 62
  • The quote only partly states this claim. 2

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The KDD Census-Income dataset class attribute is income ≤50K, >50K.

    “The class attribute is income ≤ 50K, > 50K f gindicating whether an individual makes less or more than 50K.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    The KDD Census-Income dataset contains 299,285 instances with 41 attributes, 32 categorical, 7 numerical, and 2 binary.

    “The dataset contains 299,285 instances with 41 attributes, 32 of which are categorical, 7 are numerical and 2 are binary attributes.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The positive class in the KDD Census-Income dataset is >50K.

    “The positive class is “>50K.””
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    In the Adult dataset, the male:female ratio is 32,650:16,192 (66.9%:33.1%).

    “The ratio of male:female is 32,650:16,192 (66.9%:33.1%).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    The positive class in the Adult dataset is >50K.

    “The positive class is “>50K.””
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    In the KDD Census-Income dataset, the male:female ratio is 136,447:148,109 (48%:52%).

    “the male:female ratio is 136,447:148,109 (48%:52%).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    In the Adult dataset, the white:non-white ratio is 38,903:6,319 (86%:14%).

    “the white:non-white ratio is 38,903:6,319 (86%:14%).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In the Adult dataset Bayesian network, there is a direct dependency between income and education and between sex and education.

    “there is a direct dependency between income and education as well as between sex and education.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    In the Adult dataset, the [25-60] age group ratio is 35,066:10,156 (77.5%:22.5%).

    “the ratio is 35,066:10,156 (77.5%:22.5%).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    In the Adult dataset Bayesian network, the class label income is a leaf node with no outgoing edges.

    “The class label income is the leaf node, that is, there are no outgoing edges.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In the Adult dataset, unmarried people have an income higher than 50K when they are older than 45 years, while people in the Ownchild group can have a high income when they are young.

    “Unmarried people have an income higher than 50 K when they are older than 45 years, while people in the Ownchild group can have a high income when they are young.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    The Adult dataset contains 48,842 instances, each described by 15 attributes: 6 numerical, 7 categorical, and 2 binary.

    “The dataset consists of 48,842 instances, each described via 15 attributes, of which 6 are numerical, 7 are categorical and 2 are binary attributes.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Kohavi, 1996

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The KDD Census-Income dataset is very imbalanced with an IR 1:15.30 (positive:negative).

    “The dataset is very imbalanced with an IR 1 : 15:30 (positive:negative).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In the KDD Census-Income dataset, women tend to do part-time jobs, with the number of weeks worked per year less than 26.

    “women tend to do part-time jobs, that is, the number of weeks worked per year is less than 26.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The Adult dataset class attribute is income ≤50K, >50K.

    “The class attribute is income ≤ 50K, > 50K f gindicating whether an individual makes less or more than 50K.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    In the KDD Census-Income dataset, the white:non-white ratio is 239,081:29,239 (86%:14%).

    “the white:non-white ratio is 239,081:29,239 (86%:14%).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The Adult dataset is imbalanced with an imbalance ratio (IR) 1:3.03 (positive:negative).

    “The dataset is imbalanced with an imbalance ratio (IR) 1 : 3:03 (positive:negative).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingnone given

    The German credit dataset contains only 1000 instances without any missing values.

    “The dataset contains only 1000 instances without any missing values.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The percentage of women identified as bad customers is 35.2% while that of men is only 27.7%.

    “The percentage of women identified as bad customers is 35.2% while that of men is only 27.7%.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    Each sample in the German credit dataset is described by 13 categorical, 7 numerical and 1 binary attributes.

    “Each sample is described by 13 categorical, 7 numerical and 1 binary attributes.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The number of widows is 1.7 times higher than the number of widowers in terms of high income.

    “Interestingly, the number of widows is 1.7 times higher than the number of widowers in terms of high income.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Most people have a high income when they are over 40 years old.

    “Regarding the age effect, most people have a high income when they are over 40 years old.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The German credit dataset is dominated by male instances, with a male:female ratio of 690:310 (69%:31%).

    “The dataset is dominated by male instances, the ratio of male:female is 690:310 (69%:31%).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The attribute personal-status-and-sex contains information of marital status and the gender of people.

    “Attribute personal-status-andsex contains information of marital status and the gender of people.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Only 3.2% of the non-white group have an income above 50K, compared to 6.7% for the white group.

    “Only 3.2% of the non-white group have an income above 50K, compared to 6.7% for the white group.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    42.1% of young people are recognized as bad customers while this proportion in old people is 27.2%.

    “There are 42.1% of young people are recognized as bad customers while this proportion in old people is 27.2%.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Males comprise the majority of the high-income group, especially in the Married-civilian spouse present segment where the number of males is 5 times higher than that of females.

    “As shown in this figure, males comprise the majority of the high-income group, especially for certain population segments like the Married-civilian spouse present segment where the number of males is 5 times higher than that of females.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The German credit dataset is imbalanced with an IR 2.33:1 (positive:negative).

    “The dataset is imbalanced with an IR 2:33 : 1 (positive:negative).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    89.5% of people having tertiary level are working in high-level jobs while around 80% of people with lower secondary degrees are doing low-level work.

    “For example, 89.5% of people having tertiary level are working in high-level jobs while around 80% of people with lower secondary degrees are doing low-level work.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The bank marketing dataset is dominated by people from 25 to 60 years old, with a ratio of 43,214:1997 (95.6%:4.4%).

    “the dataset is dominated by people from 25 to 60 years old, the ratio of 25-60:<25 or >60 is 43,214:1997 (95.6%:4.4%).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The class attribute for the Dutch census dataset is occupation {0,1} demonstrating if an individual has a prestigious profession or not.

    “The class attribute is occupation 0, 1 f g demonstrating if an individual has a prestigious profession or not.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    People with high education are doing prestigious jobs and vice versa.

    “In addition, people with high education are doing prestigious jobs and vice versa, as depicted in Figure 10.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The married group is the majority in the bank marketing dataset, with a ratio of married:non-married of 27,214:17,997 (60.2%:39.8%).

    “married group is the majority with the ratio of married:non-married is 27,214:17,997 (60.2%: 39.8%).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    University is the biggest education group in the credit card clients dataset with 14,030 (46.8%) clients.

    “university is the biggest group with 14,030 (46.8%) clients.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    Age has a direct effect on many attributes in the Dutch census Bayesian network.

    “Interestingly, age has a direct effect on many attributes.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    There is no missing value in the credit card clients dataset.

    “There is no missing value in the dataset.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    62.6% of males (18,860 out of 30,147) have a high-level occupation, while this proportion in the female group is only 32.7%.

    “In fact, 62.6% of males (18,860 out of 30,147) have a high-level occupation, while this proportion on females group is only 32.7%.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    The bank marketing dataset comprises 45,211 samples, each with 6 categorical, 4 binary and 7 numerical attributes.

    “The dataset comprises 45,211 samples, each with 6 categorical, 4 binary and 7 numerical attributes, as summarized in Table 6.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingnone given

    The Dutch census dataset includes 60,420 samples where each sample is described by 12 attributes.

    “The dataset includes 60,420 samples 11 where each sample is described by 12 attributes.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The single group is the majority in the credit card clients dataset, with a ratio of married:single:others of 13,659:15,964:377 (45.5%:53.2%:1.3%).

    “single group is the majority with the ratio of married:single:others is 13,659:15,964:377 (45.5%:53.2%:1.3%).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The credit card clients dataset includes 30,000 customers described by 8 categorical, 14 numerical and 2 binary attributes.

    “The dataset includes 30,000 customers described by 8 categorical, 14 numerical and 2 binary attributes, as depicted in Table 7.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    88.3% of people having no checking account are identified as good customers while half of the customers having a balance less than 0 DM in their checking account are classified as bad customers.

    “As we can see, a very high proportion of people, that is, 88.3%, having no checking account is identified as the good customers while half of the customers having a balance less than 0 DM (Deutsche Mark) in their checking account are classified as the bad customers.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The bank marketing dataset is imbalanced with an IR 1:7.55 (positive:negative).

    “The dataset is imbalanced with an IR 1 : 7:55 (positive:negative).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The credit card clients dataset is dominated by females, with a male:female ratio of 11,888:18,112 (39.6%:60.4%).

    “the dataset is dominated by females, the ratio of male:female is 11,888:18,112 (39.6%:60.4%).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingreasoning

    The cleaned datasets used in the analysis contain 6,172 (COMPAS recid.) and 4,020 (COMPAS viol.recid.) records.

    “The cleaned datasets used in our analysis contain 6,172 (COMPAS recid.)and 4,020 (COMPAS viol.recid.)records.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In the recidivist group, the number of black criminals is four times and two times more than that of white criminals with a high recidivism score and medium recidivism score, respectively.

    “In the recidivist group, the number of black criminals is four times and two times more than that of white criminals with a high recidivism score and medium recidivism score, respectively.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    In the COMPAS Bayesian network, the class label two_year_recid = {0,1} is assigned as a leaf node.

    “The class label two_year_recid = {0, 1} is assigned as a leaf node.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingnone given

    The Communities and Crime dataset contains only 1994 samples; each instance is described by 127 attributes (4 categorical and 123 numerical attributes).

    “The dataset contains only 1994 samples; each instance is described by 127 attributes (4 categorical and 123 numerical attributes).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    The COMPAS recid. and COMPAS viol.recid. datasets contain 7,214 and 4,743 samples, respectively.

    “COMPAS recid.and COMPAS viol.recid.datasets contain 7,214 and 4,743 samples, respectively.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    In the credit card clients Bayesian network, the protected attributes (sex, education, marriage) are conditionally dependent on each other.

    “Interestingly, the protected attributes (sex, education, marriage) are conditionally dependent on each other.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In the credit card clients Bayesian network, the class label default payment is directly conditionally dependent on the repayment status in July 2005 (attribute pay_3), and the given credit (attribute limit_bal) and indirectly dependent on the amount of bill statements.

    “The class label default payment is directly conditionally dependent on the repayment status in July 2005 (attribute pay_3), and the given credit (attribute limit_bal) and indirectly dependent on the amount of bill statements (the attributes with a prefix bill_amt).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The majority of recidivists are under the age of 30.

    “The majority of recidivists are under the age of 30.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Most financial datasets are imbalanced, with the only exception of the Dutch census dataset which is almost balanced.

    “Moreover, most of the datasets in this category are imbalanced, with the only exception of the Dutch census dataset which is almost balanced (see Table 1).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The score_text attribute has many ingoing and outgoing edges in the COMPAS Bayesian network.

    “Interestingly, score_text attribute (defines the category of the recidivism score) has many ingoing and outgoing edges as depicted in Figure 17.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingnone given

    The ratio male:female is 4,997:1,175 (81%:19%) in the COMPAS recid. subset, while this ratio in the COMPAS viol.recid. subset is 3,179:841 (79.1%:20.9%).

    “The ratio male:female is 4,997:1,175 (81%:19%) in the COMPAS recid.subset, while this ratio in the COMPAS viol.recid.subset is 3,179:841 (79.1%:20.9%).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). The quote only partly states this claim.

    Findingnone given

    In terms of class imbalance, financial datasets demonstrate different imbalance ratios.

    “In terms of class imbalance, datasets demonstrate different imbalance ratios.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    There are 6,395 rows (88.6%) containing missing values in the COMPAS recid. subset while this number in the COMPAS viol.recid. subset is 3,748 (79%).

    “There are 6,395 rows (88.6%) containing missing values in the COMPAS recid.subset while this number in the COMPAS viol.revid.subset is 3,748 (79%).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The recidivism rate in black defendants is higher than that of white defendants in both COMPAS subsets.

    “The recidivism rate in the black defendants is higher than that of the white defendants in both subsets.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingnone given

    In the COMPAS recid. subset, the black:white ratio is 3,175:2,103 (51.4%:34%) computed on the total number of defendants, while this ratio in the COMPAS viol.recid. subset is 1,918:1,459 (47.7%:36.3%).

    “In the COM-PAS recid.subset, the black:white ratio is 3,175:2,103 (51.4%:34%) (computed on the total number of defendants), while this ratio in the COMPAS viol.recid.subset is 1,918:1,459 (47.7%:36.3%).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    If a defendant has a long history of convictions, his probability of recidivism is higher, especially when the number of convictions is more than 27 times, the recidivism probability is almost 100%.

    “If a defendant has a long history of convictions, his probability of recidivism is higher, especially when the number of convictions is more than 27 times, the recidivism probability is almost 100%.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    The COMPAS recid. subset is fairly balanced with an IR 1:1.20 (positive:negative) while the COMPAS viol.recid. subset is imbalanced with an IR 1:5.17.

    “The COMPAS recid.subset is fairly balanced with an IR 1 : 1:20 (positive:negative) while the COMPAS viol.recid.subset is imbalanced with an IR 1 : 5:17.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The ratio of males with the default payment is 24.2% while that of females is only 20.8%.

    “In particular, the ratio of males with the default payment is 24.2% while that of females is only 20.8%.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The credit card clients dataset is imbalanced with an IR 1:3.52 (positive:negative).

    “This is an imbalanced dataset with an IR 1 : 3:52 (positive: negative).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The attribute number_outpatient also has an impact on eight other features in the diabetes Bayesian network.

    “The attribute number_outpatient also has an impact on eight other features.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The diabetes dataset is imbalanced with an IR 1:3.13 (positive:negative).

    “The dataset is imbalanced with an IR 1 : 3:13 (positive:negative).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Race in the Ricci dataset contains three values (black, white, and hispanic).

    “Race contains three values (black, white, and hispanic).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    In terms of class imbalance, healthcare and social datasets are less imbalanced than datasets in other domains, although the Diabetes dataset is still imbalanced.

    “In terms of class imbalance, these datasets are less imbalanced than datasets in other domains, although the Diabetes dataset is still imbalanced.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    Race and gender are considered as protected attributes in healthcare and social datasets.

    “Race and gender are considered as protected attributes.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The ratio of male:female in the diabetes dataset is 20,653:25,062 (45.2%:54.8%).

    “The ratio of male:female is 20,653:25,062 (45.2%:54.8%).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    In the Ricci dataset, only attribute race can be used as a protected attribute.

    “In this dataset, only attribute race can be used as a protected attribute.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The student performance dataset contains information of 395 (Mathematics subject) and 649 (Portuguese subject) students described by 33 attributes (4 categorical, 13 binary and 16 numerical attributes).

    “The dataset contains information of 395 (Mathematics subject) and 649 (Portuguese subject) students described by 33 attributes (4 categorical, 13 binary and 16 numerical attributes).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    92.4% of the communities are dominated by black people where the percentage of kids in family housing with two parents is less than 50%, while only 55.6% of the communities are dominated by black people where the percentage of kids in family housing with two parents is greater than 50%.

    “Looking into the dataset, we discover that 92.4% of the communities are dominated by black people, where the percentage of kids in family housing with two parents less than 50%), while only 55.6% of the communities are dominated by black people, where the percentage of kids in family housing with two parents greater than 50%.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    The datasets in healthcare and society domains were only surveyed in the United States.

    “In summary, the datasets in healthcare and society domains were only surveyed in the United States.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    Race and sex are considered as protected attributes in criminological datasets, with race being the most prevalent.

    “Race and sex are considered as protected attributes, with race being the most prevalent protected attribute.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingreasoning

    The clean version of the diabetes dataset contains 45,715 records.

    “The clean version of the dataset contains 45,715 records.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    The ratio of males or females who have to readmit hospital in less than 30 days is approximately 24%.

    “The ratio of males or females who have to readmit hospital in less than 30 days is approximately 24%.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    The Ricci dataset consists of 118 samples, where each sample is characterized by 6 attributes (3 numerical and 3 binary attributes).

    “The dataset consists of 118 samples, where each sample is characterized by 6 attributes (3 numerical and 3 binary attributes), as presented in Table 11.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    In the Communities and Crime Bayesian network, the percentage of kids born to never married (PctIlleg) and the percentage of kids in family housing with two parents (PctKids2Par) have a direct impact on the class label and the race.

    “In which, the percentage of kids born to never married (PctIlleg) and the percentage of kids in family housing with two parents (PctKids2Par) have a direct impact on the class label and the race.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    The criminological datasets were only surveyed in the United States.

    “In summary, the criminological datasets were only surveyed in the United States.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    In the Student performance dataset, female students are the majority, with male:female ratios of 208:187 (52.7%:47.3%) for Mathematics and 383:266 (59%:41%) for Portuguese.

    “• sex = {male, female}: the dataset is dominated by female students.The ratios of male:female are 208:187 (52.7%:47.3%)and 383:266 (59%:41%) for the Mathematics subject and Portuguese subject, respectively.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The diabetes dataset contains 101,766 patients described by 50 attributes (10 numerical, 7 binary and 33 categorical).

    “The dataset contains 101,766 patients described by 50 attributes (10 numerical, 7 binary and 33 categorical).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    Historical bias with respect to race has been detected in the criminological data.

    “Historical bias with respect to race has been detected in the data, but comprises a challenge for ML models.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In the cleaned OULAD dataset, the pass:fail ratio is 14,655:6,907 (68%:32%), i.e., the dataset is imbalanced with an imbalance ratio of 2.12:1.

    “The ratio of pass:fail is 14,655:6,907 (68%:32%).In other words, the dataset is imbalanced with the IR is 2.12:1 (positive:negative).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In the OULAD dataset, the final result attribute is directly conditionally dependent on the highest education level and the number of previous attempts, while gender has a more negligible effect on the outcome.

    “The final result attribute is directly conditionally dependent on the highest education level (highest_education) and the number times the student has attempted the module (num_of_prev_attempts) attributes, while gender has a more negligible effect on the outcome.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The student performance dataset is imbalanced with imbalance ratios 1:2.04 (Mathematics subject) and 1:5.09 (Portuguese subject).

    “The dataset is imbalanced with imbalance ratios 1:2.04 (Mathematics subject) and 1:5.09 (Portuguese subject).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    In the Diabetes dataset, male is the majority group with a male:female ratio of 11,568:9,994 (56.6%:46.4%).

    “Male is the majority group with the ratio male:female is 11,568:9,994 (56.6%:46.4%).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingreasoning

    The ratio of white:non-white in the Ricci dataset is 68:50 (57.6%:42.4%).

    “The ratio of white:non-white is 68:50 (57.6%:42.4%).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The Ricci dataset is almost balanced with an IR 1:1.11 (positive:negative).

    “The dataset is almost balanced with an IR 1 : 1:11 (positive:negative).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    In the Law school dataset, male is the majority group with a male:female ratio of 11,675:9,123 (56.1%:43.9%).

    “• male = {1, 0}.Male is the majority group.The ratio of male (1):female (0) is 11,675:9,123 (56.1%:43.9%).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In the ACS PUMS data, many US states have low gender differences in the positive class (income over 50K$) in 2011, but the gender differences increase over the years, as seen in 2019.

    “Many states have low gender differences (depicted in green) in 2011.However, the gender differences increase over the years, as seen in 2019.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Regarding balanced accuracy, the Student-Mathematics dataset shows the best result, followed by Student-Portuguese and Dutch census.

    “Regarding balanced accuracy, the Student-Mathematics is the dataset showing the best result of the predictive model, followed by the Student-Portuguese and the Dutch census datasets.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In the ACS PUMS data, the number of people with an income above 50K$ gradually increases in both sexes, but the growth rate in the male group is slightly higher than that in the female group.

    “In relation to the population size, which is shown in Figure 35, the number of people with an income above 50K$ gradually increases in both sexes.However, the growth rate in the male group is slightly higher than that in the female group.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The best value of the ABROCA is seen in the Ricci dataset, followed by the OULAD and KDD Census-Income datasets.

    “The best value of the ABROCA is seen in the Ricci dataset, followed by the OULAD and the KDD Census-Income datasets.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The Diabetes, Credit card clients, and OULAD datasets have the best value (0.0) of SP, while the Bank marketing dataset has the worst value.

    “The Diabetes, Credit card clients, and OULAD datasets have the best value (0.0) of SP while the Bank marketing dataset has the worst value.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    In the Law school dataset, non-white is the minority group with a white:non-white ratio of 17,491:3307 (84%:16%).

    “Non-white is the minority group with the ratio white:non-white is 17,491:3307 (84%:16%).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In the Law school dataset, 92.1% of white students (16,114/17,491) pass the bar exam, while this ratio in non-white students is only 72.3%.

    “We discover that 92.1% of white students (16,114/17,491) pass the bar exam, while this ratio in non-white students is only 72.3%.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    In the educational datasets surveyed, gender is the most popular protected attribute, followed by age and race.

    “Gender is the most popular protected attribute, followed by age and race.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The Law school dataset is imbalanced with an imbalance ratio of 8.07:1 (positive:negative).

    “The dataset is imbalanced with an imbalance ratio 8.07:1 (positive:negative).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    All educational datasets surveyed are imbalanced with very different imbalance ratios.

    “All datasets are imbalanced with very different imbalance ratios in terms of class imbalance.”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    In the ACS PUMS data for California (2007-2019), there is a lower representation of the protected groups (female, non-white) over the years.

    “It is easy to observe a lower representation of the protected groups (female, non-white) over the years.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Findingnone given

    Bias is observed in the educational datasets with respect to protected attributes, that is, race and sex.

    “The bias is observed in the datasets w.r.t protected attributes, that is, race, sex; hence, fairness-aware algorithms need to take into account these attributes to achieve fairness in education.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The results from data analysis and experiments show that bias may appear in the data itself and/or in the outcome of predictive models.

    “The results from data analysis and experiments show that the bias may appear in the data itself and/or in the outcome of predictive models.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The surveyed datasets were all collected quite a long time in the past with an average age of about 20 years.

    “The surveyed datasets were all collected quite a long time in the past with an average age of about 20 years.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    By generating the BN, we discover the relationship between attributes showing their conditional dependence.

    “Second, by generating the BN, we discover the relationship between attributes showing their conditional dependence.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    The oldest dataset was obtained 48 years ago, while the newest dataset was identified from 7 years ago.

    “The oldest dataset was obtained 48 years ago, while the newest dataset was identified from 7 years ago.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The KDD Census-Income dataset was collected from Current Population Surveys implemented by the US Census Bureau from 1994 to 1995.

    “The KDD Census-Income 8 dataset (Dheeru & Karra Taniskidou, 2017) was collected from Current Population Surveys implemented by the US Census Bureau from 1994 to 1995.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Dheeru & Karra Taniskidou, 2017

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The "credit rating" can be used as a proxy for "safe driving".

    “the “credit rating” can be used as a proxy for “safe driving” (Warner & Sloan, 2021).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Warner & Sloan, 2021

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    More than 20 measures of fairness have been introduced in the computer science research area.

    “Only in the computer science research area, more than 20 measures of fairness have been introduced thus far (Verma & Rubin, 2018;Žliobaitė, 2017).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Verma & Rubin, 2018

      Not checked in the free run.

    • cites Žliobaitė, 2017

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The German credit dataset consists of samples of bank account holders.

    “The German credit 10 dataset (Dheeru & Karra Taniskidou, 2017) consists of samples of bank account holders.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Dheeru & Karra Taniskidou, 2017

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Statistical parity is one of the earliest and most popular discrimination measures in the fairness-aware ML literature.

    “statistical parity (Dwork et al., 2012) is one of the earliest and most popular discrimination measures in the fairness-aware ML literature.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Dwork et al., 2012

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Equalized odds considers the ground truth of both positive- and negative-class instances and has become one of the most promising fairness notions.

    “Equalized odds introduced by (Hardt et al., 2016) countered this problem by considering the ground truth of both positive-and negative-class instances and grew to be one of the most promising fairness notion”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Hardt et al., 2016

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Search algorithms in job search websites exhibit gender-bias by displaying higher-paying jobs to male applicants compared to female ones.

    “it has been found that such algorithms exhibit gender-bias as they display higher-paying jobs to male applicants compared to female ones (Datta et al., 2015;Simonite, 2015).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Datta et al., 2015

      Not checked in the free run.

    • cites Simonite, 2015

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Statistical parity does not require compliance to ground truth labels, and hence in many ML scenarios might not be ideal.

    “one main disadvantage of SP is that it does not require compliance to the ground truth labels; hence, in many ML scenarios might not be ideal (Hardt et al., 2016).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Hardt et al., 2016

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The German credit dataset is frequently employed in fairness-aware learning research.

    “The dataset is frequently employed in fairness-aware learning researches (Appendix A).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Appendix A

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The German credit dataset is used for risk assessment prediction, to determine whether it is risky to grant credit to a person or not.

    “The dataset is used for risk assessment prediction, that is, to determine whether it is risky to grant credit to a person or not.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Dheeru & Karra Taniskidou, 2017

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Equalized odds does not consider any formal strategy such as slicing analysis to identify prevalent biases.

    “Later Gardner et al. (2019)argued that equalized odds does not consider any formal strategy such as slicing analysis to identify the prevalent biases”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Gardner et al., 2019

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    ABROCA tackles the analysis issue of equalized odds and is an illustratively efficient method of representing the divergence of values of a protected attribute.

    “ABROCA measure introduced by (Gardner et al., 2019) tackles such an analysis issue and is argued to be an illustratively efficient method of representing the divergence of values of a protected attribute.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Gardner et al., 2019

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Researchers have used the Dutch dataset to formulate a binary classification task to predict a person's occupation categorized as high-level or low-level profession.

    “Researchers (Appendix A) have used Dutch dataset to formulate a binary classification task to predict a person's occupation which can be categorized as high-level (prestigious) or low-level profession.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Appendix A

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The bank marketing dataset is related to direct marketing campaigns of a Portuguese banking institution from 2008 to 2013.

    “The bank marketing 12 dataset (Moro et al., 2014) is related to the direct marketing campaigns of a Portuguese banking institution from 2008 to 2013.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Moro et al., 2014

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The Dutch census dataset represents aggregated groups of people in the Netherlands for the year 2001.

    “The Dutch census dataset (Van der Laan, 2000) represented aggregated groups of people in the Netherlands for the year 2001.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Van der Laan, 2000

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The credit card clients dataset investigated customers' default payments in Taiwan in October 2005.

    “The credit card clients 13 dataset (Yeh & Lien, 2009) investigated the customers' default payments in Taiwan in October 2005.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Yeh & Lien, 2009

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    With respect to collection time, the financial datasets are pretty old, especially Adult and KDD census datasets.

    “With respect to the collection time, the datasets are pretty old, esp.Adult and KDD census datasets.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationown data

    The financial datasets are very diverse as they were collected from several diverse locations and at very different time points.

    “In general, the financial datasets are very diverse as they were collected from several diverse locations (from the United States to Taiwan) and at very different time points (from 1994 to 2013).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The Communities and Crime dataset contains socioeconomic data from 46 states of the United States in 1990.

    “The Communities and Crime 16 dataset (Dheeru & Karra Taniskidou, 2017) was a small dataset containing the socioeconomic data from 46 states of the United States in 1990 (the US Census).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Dheeru & Karra Taniskidou, 2017

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    Due to dependencies among protected attributes, ensuring fairness for one protected attribute may positively affect fairness for other protected attributes.

    “Due to these dependencies, ensuring fairness for one protected attribute may positively affect fairness for other protected attributes.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The COMPAS dataset was released by ProPublica in 2016 based on Broward County data collected from January 2013 to December 2014.

    “The COMPAS (Angwin et al., 2016) is a recent dataset, compared to the rest of the datasets in our survey, which was released by ProPublica 14 in 2016 based on the Broward County data (collected from January 2013 to December 2014).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Angwin et al., 2016

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The Law school dataset contains law school admission records.

    “The dataset contains the law school admission records.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Wightman, 1998

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The OULAD dataset contains information of students and their activities in the virtual learning environment for seven courses.

    “The dataset contains information of students and their activities in the virtual learning environment (VLE) for seven courses.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Kuzilek et al., 2017

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The Ricci dataset was generated by the Ricci v. DeStefano case, investigating the results of a promotion exam within a fire department in November 2003 and December 2003.

    “The Ricci 20 dataset was generated by the Ricci v.DeStefano case (Supreme Court of the United States, 2009), in which they investigated the results of a promotion exam within a fire department in November 2003 and December 2003.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Supreme Court of the United States, 2009

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The prediction task for the Law school dataset is to predict whether a candidate would pass the bar exam or predict a student's first-year average grade.

    “The prediction task is to predict whether a candidate would pass the bar exam or predict a student's first-year average grade.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Wightman, 1998

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The OULAD dataset was collected from the OU analysis project of The Open University (England) in 2013-2014.

    “The Open University Learning Analytics (OULAD) dataset 22 was collected from the OU analysis project (Kuzilek et al., 2017) of The Open University (England) in 2013-2014.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Kuzilek et al., 2017

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The Law school dataset was conducted by a Law School Admission Council survey across 163 law schools in the United States in 1991.

    “The Law school 23 dataset (Wightman, 1998) was conducted by a Law School Admission Council (LSAC) survey across 163 law schools in the United States in 1991.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Wightman, 1998

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    In Kearns et al. (2019), romantic (relationship) and dalc, walc (alcohol consumption) are considered as protected attributes.

    “Especially, in the research (Kearns et al., 2019), they consider atttributes romatic (relationship) and dalc, walc (alcohol consumption) as the protected attributes.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Kearns et al., 2019

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    In the Diabetes dataset, gender is considered as the protected attribute in the literature.

    “Protected attributes Attribute gender = {male, female} is considered as the protected attribute, in the literature.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    A possible prediction task of the Adult reconstruction dataset is to decide whether an individual earns annually more than 50,000 US dollars.

    “A possible prediction task of the Adult reconstruction dataset is to decide whether an individual earns annually more than 50,000 US dollars.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Ding et al., 2021

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The datasets are ordered based on the sensitive attribute of their particular task at hand before sequential processing to further simulate potential concept and fairness drifts in online settings.

    “In addition, the datasets are ordered based on the sensitive attribute of their particular task at hand before sequential processing to further simulating the potential concept and fairness drifts in the online settings (W.Zhang et al., 2021).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites W. Zhang et al., 2021

      Not checked in the free run.

  • May combine two claims.

    Premisereasoning

    The works (D'Amour et al., 2020; Tu et al., 2020) study the long-term effects of a currently fair decision-making system and therefore require data that capture the decision of the classifier continuously through time and change the underlying population accordingly.

    “For example, the works (D' Amour et al., 2020;Tu et al., 2020) study the long-term effects of a currently fair decision-making system and therefore require data that capture the decision of the classifier continuously through time and change the underlying population accordingly.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites D'Amour et al., 2020

      Not checked in the free run.

    • cites Tu et al., 2020

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    It is typical for ML evaluation to employ synthetic data which allow for evaluation under different learning complexity scenarios.

    “Apart from using real-world datasets, it is typical for ML evaluation (Ntoutsi et al., 2019) to also employ synthetic data which allow for evaluation under different learning complexity scenarios.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Ntoutsi et al., 2019

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Sequential decision-making requires the operating model to process each new individual on the fly and without the need for storage and reprocessing.

    “Often times, the processing also needs to be on the fly and without the need for storage and reprocessing (W.Zhang et al., 2021).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites W. Zhang et al., 2021

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    In the Law school dataset, race and male are considered as protected attributes in the literature.

    “Protected attributes In the literature, race (Bechavod & Ligett, 2017;Chzhen et al., 2020;Kearns et al., 2019;Kusner et al., 2017;Lahoti et al., 2020;Ruoss et al., 2020;Russell et al., 2017;Yang et al., 2020) and male (Berk et al., 2017;Kearns et al., 2019;Kusner et al., 2017;Lahoti et al., 2020;Yang et al., 2020) are considered as the protected attributes.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Bechavod & Ligett, 2017

      Not checked in the free run.

    • cites Chzhen et al., 2020

      Not checked in the free run.

    • cites Kearns et al., 2019

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The synthetic instances in Iosifidis & Ntoutsi (2018) are generated via SMOTE by interpolating between original instances.

    “The synthetic instances are generated via synthetic minority over-sampling technique (SMOTE) (Chawla et al., 2002) by interpolating between original instances.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Chawla et al., 2002

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The Adult reconstruction dataset consists of 49,531 instances with 14 attributes, in which 13 of 15 attributes of the Adult dataset are matched.

    “The dataset consists of 49,531 instances with 14 attributes, in which 13 of 15 attributes of the Adult dataset (see Section 3.1.1)are matched.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Ding et al., 2021

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The exclusion of censorship information could lead to important information loss and introduce substantial bias.

    “As the exclusion of censorship information could lead to important information loss and introduce substantial bias (Wang et al., 2021), more attention on the censorship of fairness datasets is warranted.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Wang et al., 2021

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The experiments in Naumann & Ntoutsi (2021) were conducted on the Adult dataset; however, the fairness of the decisions was not investigated.

    “The experiments were conducted on the Adult dataset; however, the fairness of the decisions was not investigated.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Naumann & Ntoutsi, 2021

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisenone given

    The datasets are collected in the US and European countries where the data protection laws are in place.

    “The datasets are collected in the US and European countries where the data protection laws are in place.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    The definition of fairness is different across domains.

    “The definition of fairness, of course, is different across domains.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisereasoning

    There is no universal fairness notion for every problem.

    “Therefore, it is crucial and necessary to select or define the appropriate fairness notions for each problem in each domain because there is no universal fairness notion for every problem.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given

    The newer the data, the more up-to-date with the trends of the modern society, so the analysis and application of fairness-aware algorithms on the new datasets will reflect the manifestations of the social behaviors more realistic.

    “Of course, the newer the data, the more up-to-date with the trends of the modern society, so the analysis and application of fairness-aware algorithms on the new datasets will reflect the manifestations of the social behaviors more realistic.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Sequential-friendly datasets are still scarce, albeit their significance for the development of fair sequential models.

    “However, sequential-friendly datasets, due to their magnified requirements, are still in scarce, albeit their significance for the development of fair sequential models which are widely applicable in many real-world applications (W.Zhang et al., 2019).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites W. Zhang et al., 2019

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Premisenone given

    Fairness-aware ML has attracted many recently in various domains from criminology, healthcare, finance to education.

    “To conclude, fairness-aware ML has attracted many recently in various domains from criminology, healthcare, finance to education.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    It is not easy to evaluate the efficiency of fairness-aware algorithms because they must be based on such fairness notions.

    “It is not easy to evaluate the efficiency of fairness-aware algorithms because they must be based on such fairness notions.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationnone given

    The old datasets are of reference value in comparing and contrasting the movement and variation of fairness in the same or different domains.

    “On the other hand, the old datasets are of reference value in comparing and contrasting the movement and variation of fairness in the same or different domains.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Premisenone given

    Explainability tools can help debugging ML models and uncover biased decision-making.

    “Explainability tools can help debugging ML models and uncover biased decision-making.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The new Adult dataset(s) might be suitable for sequential learning as they contain temporal information (year of data collection).

    “The new Adult dataset(s) (see Section 5.1) might be suitable for sequential learning as they contain temporal information (year of data collection).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationnone given

    If a dataset contains bias and discrimination, it would be difficult for fairness-aware algorithms to find the trade-off between fairness requirement and performance.

    “It is understandable that if a dataset contains bias and discrimination, it would be difficult for fairness-aware algorithms to find the trade-off between fairness requirement and performance.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Premisenone given

    Bias and discrimination are common problems of almost all domains in reality.

    “Third, bias and discrimination are the common problems of almost all domains in reality.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationnone given

    The selection of one or more protected attributes for the experiment depends on many factors such as domain, problem, and the purpose of the experiment.

    “The selection of one or more protected attributes for the experiment depends on many factors such as domain, problem, and the purpose of the experiment.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    For sequential decision-making, the notion of sequential counterfactuals seems prominent as it takes into account longer-term consequences of feature-value changes.

    “For sequential decision-making, the notion of sequential counterfactuals (Naumann & Ntoutsi, 2021) seems prominent as it takes into account longer-term consequences of feature-value changes.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Naumann & Ntoutsi, 2021

      Not checked in the free run.

158 of 158

  • The paper relies on this without support.

    Tabular data is the most common data representation for fairness-aware ML.

    “We focus on tabular data as the most common data representation for fairness-aware ML.”

  • The paper relies on this without support.

    An overview of the datasets used for fairness-aware ML evaluation is still missing.

    “However, an overview of the datasets used for fairness-aware ML evaluation is still missing.”

  • The paper relies on this without support.

    Data are a vital part of ML and benchmark datasets are a decisive factor for the success of AI research.

    “As data are a vital part of ML and benchmark datasets a decisive factor for the success of AI research, 1 we believe our survey is serving to fill a gap in the extant research.”

  • The paper relies on this without support.

    Fairness depends on context.

    “Fairness depends on context; thus, a large variety of fairness measures exists.”

  • The paper relies on this without support.

    The datasets surveyed are the most popular datasets used in fairness-aware ML researches.

    “This paper reviews the most popular datasets used in fairness-aware ML researches.”

  • The paper relies on this without support.

    Logistic regression is a suitable model to characterize the difficulty of the fairness-aware learning task across datasets.

    “However, in order to characterize the different datasets in terms of the difficulty of the fairness-aware learning task, in this section, we present a short fairness-versus-predictive performance evaluation 24 using a popular classification method (namely, logistic regression).”
6 of 6

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • A significant portion of publications rely on a small set of common datasets that are either already prepared or do not require significant data preparation.

    “It is not present in all papers, of course, yet a signiicant portion of publications rely on a small set of common datasets that are either already prepared or do not require signiicant data preparation (see e.g.[189]).”

    From Fairness in machine learning: A survey (2024) · cites this paper as [189]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • KDD Census-Income race counts do not cover the cleaned dataset

    Minor

    The stated white and non-white counts sum to 268,320, which is 16,236 fewer than the stated 284,556 cleaned instances. Because these are the two values of the encoded race attribute, their counts should cover that dataset.

    Reported 284556 · recomputed 268320

  • OULAD male percentage is too high

    Minor

    The stated 11,568 male students out of 21,562 cleaned instances represent about 53.7%, not 56.6%.

    Reported 56.6 · recomputed 53.64994

  • Mathematics dataset has two different sample counts

    Minor

    Table 1 lists 649 Student-Mathematics instances, whereas the dataset description says there are 395 Mathematics students.

  • Portuguese dataset has two different imbalance ratios

    Minor

    Table 1 gives the Student-Portuguese class ratio as 1:5.49, but the dataset description gives its imbalance ratio as 1:5.09.

  • Student dataset is said to be female-dominated despite larger male counts

    Minor

    The stated male counts exceed the female counts in both subjects, contradicting the claim that female students dominate the dataset.

5 of 5

Cites

Flagged first, then by citations

Research works printed but not linked: 43

  1. Agarwal A., 2018, International Conference on Machine Learning, 60
  2. Calmon F. P., 2017, Advances in Neural Information Processing Systems 30, 3995
  3. Grgić‐Hlaća N., 2018, Proceedings of the AAAI Conference on Artificial Intelligence, 51
  4. Huang L., 2019, Proceedings of the 33rd International Conference on Neural Information Processing Systems, 7589
  5. Le Quy T., 2021, Proceedings of the 14th International Conference on Educational Data Mining (EDM21), 407
  6. Mahabadi S., 2020, International Conference on Machine Learning, 6586
  7. Quadrianto N., 2017, Advances in Neural Information Processing Systems, 677
  8. Riazy S., 2019, Predictive algorithms in learning analytics and their fairness, DELFI, 2019, 223
  9. Schelter S., 2020, Proceedings of the 23rd International Conference on Extending Database Technology, EDBT 2020, 395
  10. Zemel R., 2013, International Conference on Machine Learning, 325
  11. Abraham S. S., 2020, 23rd International Conference on Extending Database Technology (EDBT), 287
  12. Backurs A., 2019, International Conference on Machine Learning, 405
  13. Bera S. K., 2019, Proceedings of the 33rd International Conference on Neural Information Processing Systems, 4955
  14. Buolamwini J., 2018, Conference on fairness, accountability and transparency, 77
  15. Chierichetti F., 2017, Proceedings of the 31st International Conference on Neural Information Processing Systems, 5036
  16. Cortez P., 2008, Proceedings of 5th FUture BUsiness TEChnology Conference (FUBUTEC 2008), 5
  17. Deepak P., 2020, International Conference on Web Information Systems Engineering, 447
  18. Ding F., 2021, Advances in Neural Information Processing Systems
  19. Faliagka E., 2012, Proceeding of the International Conference on Internet and Web Applications and Services, 215
  20. Hardt M., 2016, Proceedings of the 30th International Conference on Neural Information Processing Systems, 3323
  21. Hébert‐Johnson U., 2018, International Conference on Machine Learning, 1939
  22. Heidari H., 2018, Proceedings of the 32nd International Conference on Neural Information Processing Systems, 1273
  23. Husmeier D., 2006, Probabilistic modeling in bioinformatics and medical informatics
  24. Iosifidis V., 2018, Proceedings of the International Workshop on Bias in Information, Algorithms, and Systems co‐located with 13th International Conference on Transforming Digital Worlds (iConference 2018), 24
  25. Kearns M., 2018, International Conference on Machine Learning, 2564
  26. Kohavi R., 1996, Proceedings of the 2nd International Conference on Knowledge Discovery and Data mining, 202
  27. Kusner M., 2017, Proceedings of the 31st International Conference on Neural Information Processing Systems, 4069
  28. Lahoti P., 2020, 34th Conference on Neural Information Processing Systems
  29. Liu L. T., 2018, International Conference on Machine Learning, 3150
  30. Martinez N., 2020, International Conference on Machine Learning, 6755
  31. Ntoutsi E., 2019, CEUR Workshop Proceedings 2436 (2019), 4
  32. Ruoss A., 2020, Proceedings of the 33rd International Conference on Neural Information Processing Systems (NeurIPS 2020), 7584
  33. Sarah Flood R. R. S. R., 2020, Integrated public use microdata series, current population survey: Version 8.0 [dataset]
  34. Simonite T., 2015, Probing the dark side of google's ad‐targeting system
  35. Tu R., 2020, Thirty‐fourth Conference on Neural Information Processing Systems
  36. Van der Laan P.(2000). The 2001 census in The Netherlands. In Conference the Census of Population.
  37. Wang X., 2021, Survival Prediction‐Algorithms, Challenges and Applications, 171
  38. Wen M., 2021, International Conference on Artificial Intelligence and Statistics, 1144
  39. Wightman L. F., 1998, LSAC national longitudinal bar passage study
  40. Yang F., 2020, Advances in neural information processing systems, 4067
  41. Yeom S., 2018, Proceedings of the 32nd International Conference on Neural Information Processing Systems, 4573
  42. Zafar M. B., 2017, Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, 962
  43. Ziko I. M., 2021, Proceedings of the AAAI Conference on Artificial Intelligence, 11202

Cited by

Sorted by citations

Related

Authors
Le Quy, T., Roy, A., Iosifidis, V., Zhang, W., & Ntoutsi, E.
Affiliations
  • L3S Research Center Leibniz University Hannover Hannover Germany
  • Leibniz University Hannover (DE)
  • Carnegie Mellon University (US)
  • Freie Universität Berlin (DE)
ORCID
Venue
WIREs Data Mining and Knowledge Discovery
Publisher
Wiley
Published
2022-03-03
Volume
12
Issue
3
ISSN
  • 1942-4787 (ISSN-L)
  • 1942-4795
DOI
10.1002/widm.1452
OpenAlex
W3202598875
Cited by
261
References
136
Citations by year
2026: 36 · 2025: 87 · 2024: 68 · 2023: 57 · 2022: 13
FWCI
65.13
Type
journal article
Language
en
Open access
Licence
Locations
  • Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • Refubium (Universitätsbibliothek der Freien Universität Berlin) (submittedVersion, cc-by)
  • Institutional Repository of Leibniz Universität Hannover (Leibniz Universität Hannover) (submittedVersion, cc-by-nc-nd)
  • arXiv (Cornell University) (cc-by)
  • Leibniz Universität Hannover (cc-by)
  • Universitätsbibliothek der FU Berlin Hochschulschriftenstelle u. Dokumentenserver (cc-by)
Funders
Volkswagen Foundation
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
Keywords
algorithmic fairness, benchmark dataset, tabular data, protected characteristics, Bayesian networks, data preprocessing, class attributes

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review