The DOI matches its OpenAlex record; 5 of the 5 references it cites were found in a registry or an index.
On Fairness and Calibration
Open accessOverview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159 · cited 8×
“We know from recent work that, except in the most constrained cases, it is impossible to achieve calibration while also satisfying Equalized Odds [8, 26].”
BackgroundInfluentialKusner, M. J., Loftus, J. R., Russell, C., & Silva, R. (2017) · arXiv · doi:10.48550/arxiv.1703.06856 · Cited by 995 · cited 1×
“Fairness has also been considered in online learning [21, 24], unsupervised learning [5], and causal inference [25, 27].”
BackgroundInherent Trade-Offs in the Fair Determination of Risk Scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2016) · arXiv · doi:10.48550/arxiv.1609.05807 · Cited by 941
Self-citationFairness Constraints: Mechanisms for Fair Classification
Zafar, M. B., Valera, I., Rodriguez, M. G., & Gummadi, K. P. (2015) · arXiv · doi:10.48550/arxiv.1507.05259 · Cited by 645
Cited by
A Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2022) · ACM Computing Surveys · doi:10.1145/3457607 · Cited by 5,812
Explainable AI: a review of machine learning interpretability methods
Linardatos, P., Papastefanopoulos, V., & Kotsiantis, S. (2020) · Entropy · doi:10.3390/e23010018 · Cited by 2,922
“[97], building on from [92], studied the problem of producing calibrated probability scores, the end goal of many machine learning applications, while, at the same time, ensuring fair decisions across different demographic segments.”
Bellamy, R. K. E., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilovic, A., et al. (2018) · arXiv · doi:10.48550/arxiv.1810.01943 · Cited by 1,581
Fairness definitions explained
Verma, S., & Rubin, J. (2018) · Proceedings of the International Workshop on Software Fairness · doi:10.1145/3194770.3194776 · Cited by 1,124
Fairness in Criminal Justice Risk Assessments: The State of the Art
Berk, R., Heidari, H., Jabbari, S., Kearns, M., & Roth, A. (2021) · Sociological Methods & Research · doi:10.1177/0049124118782533 · Cited by 988
“Arguably, the trade-off that has gotten the most attention is between conditional use accuracy equality and the false positive and false negative rates (Angwin et al. 2016; Chouldechova 2017; Dieterich et al. 2016; Kleinberg et al. 2016; Pleiss et al. 2017).”
BackgroundAI Fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias
Bellamy, R. K. E., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilović, A., et al. (2019) · IBM Journal of Research and Development · doi:10.1147/jrd.2019.2942287 · Cited by 884
Lekadir, K., Frangi, A. F., Porras, A. R., Glocker, B., Cintas, C., Langlotz, C. P., Weicken, E., Asselbergs, F. W., Prior, F., Collins, G. S., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081554 · Cited by 957
Trustworthy AI: From Principles to Practices
Li, B., Qi, P., Liu, B., Di, S., Liu, J., Pei, J., Yi, J., & Zhou, B. (2023) · ACM Computing Surveys · doi:10.1145/3555803 · Cited by 612
Fairness in machine learning: A survey
Caton, S., & Haas, C. (2024) · ACM Computing Surveys · doi:10.1145/3616865 · Cited by 646
Mitigating Gender Bias in Natural Language Processing: Literature Review
Sun, T., Gaut, A., Tang, S., Huang, Y., ElSherief, M., Zhao, J., Mirza, D., Belding, E., Chang, K.-W., & Wang, W. Y. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1159 · Cited by 492
Explainable Deep Learning: A Field Guide for the Uninitiated
Ras, G., Xie, N., Van Gerven, M., & Doran, D. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13200 · Cited by 446
The Measure and Mismeasure of Fairness
Corbett-Davies, S., Gaebler, J. D., Nilforoshan, H., Shroff, R., & Goel, S. (2018) · arXiv · doi:10.48550/arxiv.1808.00023 · Cited by 444
Fairness in Recommendation Ranking through Pairwise Comparisons
Beutel, A., Chen, J., Doshi, T., Qian, H., Wei, L., Wu, Y., Heldt, L., Zhao, Z., Hong, L., Chi, E. H., et al. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330745 · Cited by 362
Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty
Bhatt, U., Antorán, J., Zhang, Y., Liao, Q. V., Sattigeri, P., Fogliato, R., Melançon, G., Krishnan, R., Stanley, J., Tickoo, O., et al. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462571 · Cited by 300
Oneto, L., & Chiappa, S. (2020) · Studies in Computational Intelligence · doi:10.1007/978-3-030-43883-8_7 · Cited by 291
Illuminating the dark spaces of healthcare with ambient intelligence
Haque, A., Milstein, A., & Fei-Fei, L. (2020) · Nature · doi:10.1038/s41586-020-2669-y · Cited by 274
“One solution is to statistically calibrate the algorithm, resulting in consistent error rates across specified attributes 139 .”
MethodTimmons, A. C., Duong, J. B., Simo Fiallo, N., Lee, T., Vo, H. P. Q., Ahle, M. W., Comer, J. S., Brewer, L. C., Frazier, S. L., & Chaspari, T. (2023) · Perspectives on Psychological Science · doi:10.1177/17456916221134490 · Cited by 263
“Researchers should also ensure that the proportion of negative and positive outcomes or the likelihood of positive outcomes is the same for both groups (Hardt et al., 2016; Pleiss et al., 2017), as well as ensuring that favorable outcomes do not solely occur for the majority group (Kamiran et al.”
BackgroundMethodTrustworthy ai: A computational perspective
Liu, H., Wang, Y., Fan, W., Liu, X., Li, Y., Jain, S., Liu, Y., Jain, A., & Tang, J. (2023) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3546872 · Cited by 261
Classification with Fairness Constraints: A Meta-Algorithm with Provable Guarantees
Celis, L. E., Huang, L., Keswani, V., & Vishnoi, N. K. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287586 · Cited by 231
“Another motivating example of false discovery parity is mentioned in [55].”
BackgroundInfluentialClassifier calibration: a survey on how to assess and improve predicted class probabilities
Silva Filho, T., Song, H., Perello-Nieto, M., Santos-Rodriguez, R., Kull, M., & Flach, P. (2023) · Machine Learning · doi:10.1007/s10994-023-06336-7 · Cited by 228
Fairness in credit scoring: Assessment, implementation and profit implications
Kozodoi, N., Jacob, J., & Lessmann, S. (2022) · European Journal of Operational Research · doi:10.1016/j.ejor.2021.06.023 · Cited by 198
Multiaccuracy: Black-Box Post-Processing for Fairness in Classification
Kim, M. P., Ghorbani, A., & Zou, J. (2019) · Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3306618.3314287 · Cited by 193
An empirical characterization of fair machine learning for clinical risk prediction
Pfohl, S. R., Foryciarz, A., & Shah, N. H. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103621 · Cited by 164
Identifying and Correcting Label Bias in Machine Learning
Jiang, H., & Nachum, O. (2019) · arXiv · doi:10.48550/arxiv.1901.04966 · Cited by 152
Bogina, V., Hartman, A., Kuflik, T., & Shulner-Tal, A. (2022) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-021-00248-0 · Cited by 142
“post calibrated equalized odds (Pleiss et al. 2017) Finding probabilities for changing output labels and calibrated equalized odds optimization.”
MethodThe Landscape and Gaps in Open Source Fairness Toolkits
Lee, M. S. A., & Singh, J. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445261 · Cited by 129
Fairness in Machine Learning: A Survey
Caton, S., & Haas, C. (2020) · arXiv · doi:10.48550/arxiv.2010.04053 · Cited by 646
Inherent Trade-Offs in Algorithmic Fairness
Kleinberg, J. (2018) · Abstracts of the 2018 ACM International Conference on Measurement and Modeling of Computer Systems · doi:10.1145/3219617.3219634 · Cited by 125
Self-citationPutting Fairness Principles into Practice: Challenges, Metrics, and Improvements
Beutel, A., Chen, J., Doshi, T., Qian, H., Woodruff, A., Luu, C., Kreitmann, P., Bischof, J., & Chi, E. H. (2019) · Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3306618.3314234 · Cited by 111
In Pursuit of Interpretable, Fair and Accurate Machine Learning for Criminal Recidivism Prediction
Wang, C., Han, B., Patel, B., & Rudin, C. (2023) · Journal of Quantitative Criminology · doi:10.1007/s10940-022-09545-w · Cited by 101
Mitigating face recognition bias via group adaptive classifier
Gong, S., Liu, X., & Jain, A. K. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00342 · Cited by 92
Smith, C. E., Yu, B., Srivastava, A., Halfaker, A., Terveen, L., & Zhu, H. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376783 · Cited by 77
Perrone, V., Donini, M., Zafar, M. B., Schmucker, R., Kenthapadi, K., & Archambeau, C. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462629 · Cited by 66
Taking Advantage of Multitask Learning for Fair Classification
Oneto, L., Doninini, M., Elders, A., & Pontil, M. (2019) · Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3306618.3314255 · Cited by 64
Establishing and evaluating trustworthy AI: overview and research challenges
Kowald, D., Scher, S., Pammer-Schindler, V., Müllner, P., Waxnegger, K., Demelius, L., Fessl, A., Toller, M., Mendoza Estrada, I. G., Šimić, I., et al. (2024) · Frontiers in Big Data · doi:10.3389/fdata.2024.1467222 · Cited by 59
Silva: Interactively Assessing Machine Learning Fairness Using Causality
Yan, J. N., Gu, Z., Lin, H., & Rzeszotarski, J. M. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376447 · Cited by 54
Fairness metrics and bias mitigation strategies for rating predictions
Ashokan, A., & Haas, C. (2021) · Information Processing & Management · doi:10.1016/j.ipm.2021.102646 · Cited by 47
Towards Unbiased and Accurate Deferral to Multiple Experts
Keswani, V., Lease, M., & Kenthapadi, K. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462516 · Cited by 45
“[62] Geoff Pleiss, Manish Raghavan, Felix Wu, Jon Kleinberg, and Kilian Q Weinberger.”
BackgroundFAIR: Fair adversarial instance re-weighting
Petrović, A., Nikolić, M., Radovanović, S., Delibašić, B., & Jovanović, M. (2022) · Neurocomputing · doi:10.1016/j.neucom.2021.12.082 · Cited by 40
Marrying Fairness and Explainability in Supervised Learning
Grabowicz, P. A., Perello, N., & Mishra, A. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533236 · Cited by 37
Redefining profit metrics for boosting student retention in higher education
Maldonado, S., Miranda, J., Olaya, D., Vásquez, J., & Verbeke, W. (2021) · Decision Support Systems · doi:10.1016/j.dss.2021.113493 · Cited by 36
Lewicki, K., Lee, M. S. A., Cobbe, J., & Singh, J. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581463 · Cited by 30
DeAR: Debiasing Vision-Language Models with Additive Residuals
Seth, A., Hemani, M., & Agarwal, C. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00659 · Cited by 34
The Fairness of Credit Scoring Models
Hurlin, C., Perignon, C., & Saurin, S. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3785882 · Cited by 26
Algorithmic decision making methods for fair credit scoring
Moldovan, D. (2023) · IEEE Access · doi:10.1109/access.2023.3286018 · Cited by 28
“[40], the method builds upon the work of Hardt et al.”
MethodA transformer-based deep learning approach for fairly predicting post-liver transplant risk factors
Li, C., Jiang, X., & Zhang, K. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104545 · Cited by 25
End-To-End Bias Mitigation: Removing Gender Bias in Deep Learning
Feldman, T., & Peake, A. (2021) · arXiv · doi:10.48550/arxiv.2104.02532 · Cited by 19
A Fair Individualized Polysocial Risk Score for Identifying Increased Social Risk in Type 2 Diabetes
Huang, Y., Guo, J., Donahoo, W. T., Lee, Y. A., Fan, Z., Lu, Y., Chen, W.-H., Tang, H., Bilello, L., Saguil, A. A., et al. (2024) · Nature Communications · doi:10.1038/s41467-024-52960-9 · Cited by 14
Recovering from Biased Data: Can Fairness Constraints Improve Accuracy?
Blum, A., & Stangl, K. (2019) · arXiv · doi:10.48550/arxiv.1912.01094 · Cited by 13
Fair Machine Learning in Healthcare: A Survey
Feng, Q., Du, M., Zou, N., & Hu, X. (2025) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3361836 · Cited by 12
Beyond Individual and Group Fairness
Awasthi, P., Cortes, C., Mansour, Y., & Mohri, M. (2020) · arXiv · doi:10.48550/arxiv.2008.09490 · Cited by 8
Metadata
- Authors
- Pleiss, G., Raghavan, M., Wu, F., Kleinberg, J. M., & Weinberger, K. Q.
- Venue
- arXiv (Cornell University)
- Publisher
- Cornell University
- Published
- 2017-09-06
- Volume
- 30
- Pages
- 5680-5689
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1709.02012
- OpenAlex
- W2963178340
- Cited by
- 263
- References
- 5
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.