Fairness Score and process standardization: framework for fairness certification in artificial intelligence systems
Open accessThe DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
Decisions made by various artificial intelligence (AI) systems greatly influence our day-to-day lives. With the increasing use of AI systems, it becomes crucial to know that they are fair, identify the underlying biases in their decision-making, and create a standardized framework to ascertain their fairness. Biases in AI systems lead to unintended ethical, social and even legal issues. In this paper, we propose a novel and versatile Fairness Score and Bias Index for measuring the fairness of a supervised learning AI system. We also propose a standard operating procedure (SOP) for issuing Fairness Certification for such data-driven applications. Fairness Score and audit process standardization will ensure quality, reduce ambiguity, enable comparison and improve the trustworthiness of the AI systems. It will also provide a framework to operationalize the concept of fairness and facilitate the commercial deployment of such systems. Furthermore, a Fairness Certificate issued by a designated third-party auditing agency following the standardized process would boost the conviction of the organizations in the AI systems that they intend to deploy. The Bias Index proposed in this paper reveals comparative bias amongst the various protected attributes within the dataset, and the Fairness Score provides overall fairness with an ideal value of 1. To substantiate the proposed framework, we iteratively train a model on biased and unbiased data using multiple labelled datasets and check that the Fairness Score, Bias Indexes and the proposed process correctly identify the biases and judge the fairness.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137
A Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2019) · arXiv · doi:10.48550/arxiv.1908.09635 · Cited by 5,812
A Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2022) · ACM Computing Surveys · doi:10.1145/3457607 · Cited by 5,812
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501
Barocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048 · cited 1×
“Unwanted biases in training data, due to either prejudice in labels or under-/ over-sampling, can be checked using statistical tests on datasets or models [Barocas and Selbst, 2016, Bellamy et al., 2018].”
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · cited 4×
“Mathematically, it can be written as: 6) where, y ∈ { 0 , 1 } It tries to ensure high accuracy in all demographics [Hardt et al., 2016].”
InfluentialBellamy, R. K. E., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilovic, A., et al. (2018) · arXiv · doi:10.48550/arxiv.1810.01943 · Cited by 1,581
Friedman, B., & Nissenbaum, H. (1996) · ACM Transactions on Information Systems · doi:10.1145/230538.230561 · Cited by 1,203 · cited 1×
“Reference [23] also describes similar causes of biases.”
Algorithmic Decision Making and the Cost of Fairness
Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098095 · Cited by 1,149
Fairness definitions explained
Verma, S., & Rubin, J. (2018) · Proceedings of the International Workshop on Software Fairness · doi:10.1145/3194770.3194776 · Cited by 1,124 · cited 3×
“…individual and group fairness, fairness definitions can also be classified using the statistical approach used: definitions based on the predicted outcome; definitions based on predicted and actual outcomes; and definitions based on predicted probabilities and actual outcomes [Verma and Rubin, 2018].”
Research works printed but not linked: 18
- Yapo, A., Weiss, J.W.: Ethical implications of bias in machine learning. In: HICSS (2018) · DOI · Paper page
- Curionin, A.: Trusted AI for Business. https://docbox.etsi.org/Workshop/2019/201904_SUMMIT_AI/S04_AI_CHALLENGES_OPPORT/TRUSTED_AI_BUSINESS_IBM_CURIONI.pdfAccessed 14 Aug 2020
- Kodiyan, A.A.: An overview of ethical issues in using AI systems in hiring with a case study of Amazon’s AI based hiring tool. Researchgate Preprint (2019)
- Wadsworth, C., Vera, F., Piech, C.: Achieving fairness through adversarial learning: an application to recidivism prediction (2018). arXiv:1807.00199 · arXiv
- Leavy, S., O’Sullivan, B., Siapera, E.: Data, power and bias in artificial intelligence (2020). arXiv:2008.07341 · arXiv
- Zheng, K., Gao, J., Ngiam, K.Y., Ooi, B.C., Yip, W.L.J.: Resolving the bias in electronic medical records. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 2171–2180 (2017) · DOI
- Segal, S., Adi, Y., Pinkas, B., Baum, C., Ganesh, C., Keshet, J.: Fairness in the eyes of the data: certifying machine-learning models. In: Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, pp. 926–935 (2021) · DOI
- Tommasi, T., Bucci, S., Caputo, B., Asinari, P.: Towards fairness certification in artificial intelligence (2021). arXiv:2106.02498 · arXiv
- Schubert, C., Hütt, M.T.: Economy-on-demand and the fairness of algorithms. Eur. Labour Law J. 10(1), 3–16 (2019) · doi:10.1177/2031952519829082
- Chouldechova, A., Roth, A.: The frontiers of fairness in machine learning (2018). arXiv:1810.08810 · arXiv
- Zliobaite, I.: Fairness-aware machine learning: a perspective (2017). arXiv:1708.00754 · arXiv
- Aggarwal, A., Lohia, P., Nagar, S., Dey, K., Saha, D.: Black box fairness testing of machine learning models. In: Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pp. 625–635 (2019) · DOI · Paper page
- Hertweck, C., Heitz, C., Loi, M.: On the moral justification of statistical parity. In: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pp. 747–757 (2021) · DOI · Paper page
- Singh, H., Singh, R., Mhasawade, V., Chunara, R.: Fairness violations and mitigation under covariate shift. In: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pp. 3–13 (2021) · DOI · Paper page
- Hinnefeld, J.H., Cooman, P., Mammo, N., Deese, R.: Evaluating fairness metrics in the presence of dataset bias (2018). arXiv:1809.09245 · arXiv
- Lohia, P.K., Ramamurthy, K.N., Bhide, M., Saha, D., Varshney, K.R., Puri, R.: Bias mitigation post-processing for individual and group fairness. In: Icassp 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (icassp), pp. 2847–2851. IEEE (2019) · DOI · Paper page
- Castelnovo, A., Crupi, R., Greco, G., Regoli, D.: The zoo of fairness metrics in machine learning (2021). arXiv:2106.00467 · DOI
- Pandit, S., Gupta, S., et al.: A comparative study on distance measuring approaches for clustering. Int. J. Res. Comput. Sci. 2(1), 29–31 (2011) · doi:10.7815/ijorcs.21.2011.011
Cited by
La nueva realidad de la educación ante los avances de la inteligencia artificial generativa
García Peñalvo, F. J., Llorens-Largo, F., & Vidal, J. (2023) · RIED-Revista Iberoamericana de Educación a Distancia · doi:10.5944/ried.27.1.37716 · Cited by 238
A step toward building a unified framework for managing AI bias
Rana, S. A., Azizul, Z. H., & Awan, A. A. (2023) · PeerJ Computer Science · doi:10.7717/peerj-cs.1630 · Cited by 23
A federated architecture for sector-led AI governance: lessons from India
Agarwal, A., & Nene, M. J. (2026) · Transforming Government: People, Process and Policy · doi:10.1108/tg-09-2025-0310
“Current scholarship con’irms that key governance functions, such as incident reporting in critical infrastructure and fairness certi’ication for the startup ecosystem, lack the necessary institutional mechanisms for effective implementation [Agarwal and Nene, 2024a, Agarwal et al., 2023].”
Self-citation
Metadata
- Authors
- Agarwal, A., Agarwal, H., & Agarwal, N.
- Venue
- AI and Ethics
- Published
- 2022-03-07
- Volume
- 3
- Issue
- 1
- Pages
- 267-279
- ISSN
- 2730-5953 (ISSN-L)
- 2730-5961
- OpenAlex
- W4221080962
- Cited by
- 48
- References
- 29
- FWCI
- 10.80
- Type
- journal article
- Language
- en
- Open access
- green
- Licence
- www.springernature.com/gp/researchers/text-and-data-mining (tdm)
- www.springernature.com/gp/researchers/text-and-data-mining (vor)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Fairness Score and process standardization: framework for fairness certification in artificial intelligence systems. https://app.cochat.ai/papers/10.1007/s43681-022-00147-7
- The paper
- Agarwal, A., Agarwal, H., & Agarwal, N. (2023). Fairness Score and process standardization: framework for fairness certification in artificial intelligence systems. AI and Ethics. https://doi.org/10.1007/s43681-022-00147-7
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.