Net benefit, calibration, threshold selection, and training objectives for algorithmic fairness in healthcare
Open accessThe DOI matches its Crossref record; 82 of the 82 references it cites were found in a registry or an index.
Abstract
A growing body of work uses the paradigm of algorithmic fairness to frame the development of techniques to anticipate and proactively mitigate the introduction or exacerbation of health inequities that may follow from the use of model-guided decision-making. We evaluate the interplay between measures of model performance, fairness, and the expected utility of decision-making to offer practical recommendations for the operationalization of algorithmic fairness principles for the development and evaluation of predictive models in healthcare. We conduct an empirical case-study via development of models to estimate the ten-year risk of atherosclerotic cardiovascular disease to inform statin initiation in accordance with clinical practice guidelines. We demonstrate that approaches that incorporate fairness considerations into the model training objective typically do not improve model performance or confer greater net benefit for any of the studied patient populations compared to the use of standard learning paradigms followed by threshold selection concordant with patient preferences, evidence of intervention effectiveness, and model calibration. These results hold when the measured outcomes are not subject to differential measurement error across patient populations and threshold selection is unconstrained, regardless of whether differences in model performance metrics, such as in true and false positive error rates, are present. In closing, we argue for focusing model development efforts on developing calibrated models that predict outcomes well for all patient populations while emphasizing that such efforts are complementary to transparent reporting, participatory design, and reasoning about the impact of model-informed interventions in context.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Model threshold adjustments can deliver improved responsiveness to user-entered data, such as a patient’s or clinician’s preferences, recognizing individual opinions or beliefs surrounding a given prediction task73.”
From Bias recognition and mitigation strategies in artificial intelligence healthcare applications (2025) · cites this paper as [73]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
Proceedings of the 24th international conference on Machine learning
(2007) · doi:10.1145/1273496 · Cited by 11,630
Stone, N. J., Robinson, J. G., Lichtenstein, A. H., Bairey Merz, C. N., Blum, C. B., Eckel, R. H., Goldberg, A. C., Gordon, D., Levy, D., Lloyd-Jones, D. M., et al. (2014) · Circulation · doi:10.1161/01.cir.0000437738.63853.7a · Cited by 7,081
Decision curve analysis: a novel method for evaluating prediction models
Vickers, A. J., & Elkin, E. B. (2006) · Medical Decision Making · doi:10.1177/0272989x06295361 · Cited by 6,341 · cited 5×
“[34] Andrew J Vickers and Elena B Elkin.”
BackgroundMethodInfluentialStructural racism and health inequities in the USA: evidence and interventions
Bailey, Z. D., Krieger, N., Agénor, M., Graves, J., Linos, N., & Bassett, M. T. (2017) · The Lancet · doi:10.1016/s0140-6736(17)30569-x · Cited by 5,203 · cited 1×
“When those differences in outcome incidence, which may or may not be mismeasured or observable, are a result of population-level differences in disease burden across patient subgroups as a result of structural disparities [107, 108], we argue that the appropriate response is to endeavor to understand both the cause of…”
BackgroundCausal Inference for Statistics, Social, and Biomedical Sciences: An Introduction
Imbens, G. W., & Rubin, D. B. (2015) · doi:10.1017/cbo9781139025751 · Cited by 4,210
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681 · cited 3×
“Furthermore, the narrow scope of statistical algorithmic fairness assessments does not, by default, consider the context, values, and norms under-pinning problem formulation, data collection, and measurement, nor the benefits and harms of the downstream model-guided intervention [60, 68].”
BackgroundInfluentialFair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159
2019 ACC/AHA Guideline on the Primary Prevention of Cardiovascular Disease
Arnett, D. K., Blumenthal, R. S., Albert, M. A., Buroker, A. B., Goldberger, Z. D., Hahn, E. J., Himmelfarb, C. D., Khera, A., Lloyd-Jones, D., McEvoy, J. W., et al. (2019) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2019.03.010 · Cited by 1,896 · cited 5×
“Clinical practice guidelines for the primary prevention of cardiovascular disease recommend the use of estimates of ten-year atherosclerotic cardiovascular disease (ASCVD) risk to inform the initiation of cholesterol-lowering statin therapy [3, 35, 38, 54, 87].”
BackgroundMethodInfluentialGrundy, S. M., Stone, N. J., Bailey, A. L., Beam, C., Birtcher, K. K., Blumenthal, R. S., Braun, L. T., de Ferranti, S., Faiella-Tommasino, J., Forman, D. E., et al. (2019) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2018.11.002 · Cited by 1,751
Hidden in Plain Sight — Reconsidering the Use of Race Correction in Clinical Algorithms
Vyas, D. A., Eisenstein, L. G., & Jones, D. S. (2020) · New England Journal of Medicine · doi:10.1056/nejmms2004740 · Cited by 1,731
How Structural Racism Works — Racist Policies as a Root Cause of U.S. Racial Health Inequities
Bailey, Z. D., Feldman, J. M., & Bassett, M. T. (2021) · New England Journal of Medicine · doi:10.1056/nejmms2025396 · Cited by 1,598
George, H., Duke Jon, D., Shah Nigam, H., Reich Christian, G., Vojtech, H., Schuemie Martijn, J., Suchard Marc, A., Woong, P. R., Kei, W. I. C., Rijnbeek Peter, R., et al. (2015) · Studies in Health Technology and Informatics · doi:10.3233/978-1-61499-564-7-574 · Cited by 1,542 · cited 1×
“[95] George Hripcsak, Jon D Duke, Nigam H Shah, Christian G Reich, Vojtech Huser, Martijn J Schuemie, Marc A Suchard, Rae Woong Park, Ian Chi Kei Wong, Peter R Rijnbeek, Johan Van Der Lei, Nicole Pratt, G Niklas Norén, Yu-Chuan Chuan Li, Paul E Stang, David Madigan, and Patrick B Ryan.”
BackgroundSelf-citationEnsuring fairness in machine learning to advance health equity.
Rajkomar, A., Hardt, M., Howell, M. D., Corrado, G., & Chin, M. H. (2018) · Annals of Internal Medicine · doi:10.7326/m18-1990 · Cited by 1,341 · cited 2×
“31, 428* Statin (ATC) C10AA0[1-8] Type 1 diabetes (OMOP) 201254, 40484648, 201254, 435216 Gestational diabetes (OMOP) 4058243 Type 2 diabetes (OMOP) 443238, 201820, 442793 (exclude all type 1 and gestational concepts) Chronic kidney disease (OMOP) 192279, 192359, 193253, 194385, 195314, 201313, 261071, 4103224, 426336…”
A Reductions Approach to Fair Classification
Agarwal, A., Beygelzimer, A., Dudík, M., Langford, J., & Wallach, H. (2018) · arXiv · doi:10.48550/arxiv.1803.02453 · Cited by 1,288
Vickers, A. J., Van Calster, B., & Steyerberg, E. W. (2016) · BMJ · doi:10.1136/bmj.i6 · Cited by 1,163
Algorithmic Decision Making and the Cost of Fairness
Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098095 · Cited by 1,149
Unified Methods for Censored Longitudinal Data and Causality
van der Laan, M. J., & Robins, J. M. (2003) · Springer Series in Statistics · doi:10.1007/978-0-387-21700-0 · Cited by 959
Jacobson, T. A., Ito, M. K., Maki, K. C., Orringer, C. E., Bays, H. E., Jones, P. H., McKenney, J. M., Grundy, S. M., Gill, E. A., Wild, R. A., et al. (2015) · Journal of Clinical Lipidology · doi:10.1016/j.jacl.2015.02.003 · Cited by 773
Robust Solutions of Optimization Problems Affected by Uncertain Probabilities
Ben-Tal, A., den Hertog, D., De Waegenaere, A., Melenberg, B., & Rennen, G. (2013) · Management Science · doi:10.1287/mnsc.1120.1641 · Cited by 748
Recovery of Information and Adjustment for Dependent Censoring Using Surrogate Markers
Robins, J. M., & Rotnitzky, A. (1992) · AIDS Epidemiology · doi:10.1007/978-1-4757-1229-2_14 · Cited by 641
Application of New Cholesterol Guidelines to a Population-Based Sample
Pencina, M. J., Navar-Boggan, A. M., D'Agostino, R. B., Williams, K., Neely, B., Sniderman, A. D., & Peterson, E. D. (2014) · New England Journal of Medicine · doi:10.1056/nejmoa1315665 · Cited by 614
Lloyd-Jones, D. M., Braun, L. T., Ndumele, C. E., Smith, S. C., Sperling, L. S., Virani, S. S., & Blumenthal, R. S. (2019) · Circulation · doi:10.1161/cir.0000000000000638 · Cited by 554
Validation of a common data model for active safety surveillance research
Overhage, J. M., Ryan, P. B., Reich, C. G., Hartzema, A. G., & Stang, P. E. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000376 · Cited by 548
DeFilippis, A. P., Young, R., Carrubba, C. J., McEvoy, J. W., Budoff, M. J., Blumenthal, R. S., Kronmal, R. A., McClelland, R. L., Nasir, K., & Blaha, M. J. (2015) · Annals of Internal Medicine · doi:10.7326/m14-1281 · Cited by 515
Building Classifiers with Independency Constraints
Calders, T., Kamiran, F., & Pechenizkiy, M. (2009) · 2009 IEEE International Conference on Data Mining Workshops · doi:10.1109/icdmw.2009.83 · Cited by 468 · cited 2×
“Common instantiations of metric parity include equalized odds (Ŷ ⊥ A | Y or S ⊥ A | Y ) [42], which requires both the true positive rates and the false positive rates to be equal across subgroups, demographic parity (Ŷ ⊥ A or S ⊥ A) [43], which requires the rate at which patients are classified as belonging to the pos…”
BackgroundMethodThe Measure and Mismeasure of Fairness
Corbett-Davies, S., Gaebler, J. D., Nilforoshan, H., Shroff, R., & Goel, S. (2018) · arXiv · doi:10.48550/arxiv.1808.00023 · Cited by 444
Evaluating Prediction Rules for t -Year Survivors With Censored Regression Models
Uno, H., Cai, T., Tian, L., & Wei, L. J. (2007) · Journal of the American Statistical Association · doi:10.1198/016214507000000149 · Cited by 413 · cited 1×
“To address systematic censoring of ten-year ASCVD outcomes, we adopt an inverse probability of censoring weighting (IPCW) approach during model training and evaluation [14, 44, 61, 78, 91, 93].”
MethodFairness in Recommendation Ranking through Pairwise Comparisons
Beutel, A., Chen, J., Doshi, T., Qian, H., Wei, L., Wu, Y., Heldt, L., Zhao, Z., Hong, L., Chi, E. H., et al. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330745 · Cited by 362
Jacobs, A. Z., & Wallach, H. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445901 · Cited by 362 · cited 1×
“For our analysis, we assume that the observed outcomes are not subject to unobserved differential measurement error across patient subgroups [45, 64], that the choice of decision threshold used to allocate a clinical intervention on the basis of the output of a predictive model is not constrained by resource or operat…”
BackgroundSuchard, M. A., Schuemie, M. J., Krumholz, H. M., You, S. C., Chen, R., Pratt, N., Reich, C. G., Duke, J., Madigan, D., Hripcsak, G., et al. (2019) · The Lancet · doi:10.1016/s0140-6736(19)32317-7 · Cited by 359
Austin, P. C., & Steyerberg, E. W. (2019) · Statistics in Medicine · doi:10.1002/sim.8281 · Cited by 304
Rana, J. S., Tabada, G. H., Solomon, M. D., Lo, J. C., Jaffe, M. G., Sung, S. H., Ballantyne, C. M., & Go, A. S. (2016) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2016.02.055 · Cited by 299
Ethical limitations of algorithmic fairness solutions in health care machine learning
McCradden, M. D., Joshi, S., Mazwi, M., & Anderson, J. A. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30065-0 · Cited by 291
Problem Formulation and Fairness
Passi, S., & Barocas, S. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287567 · Cited by 283 · cited 2×
“Furthermore, the narrow scope of statistical algorithmic fairness assessments does not, by default, consider the context, values, and norms underpinning problem formulation, data collection, and measurement, nor the benefits and harms of the downstream model-guided intervention [60, 68].”
BackgroundCheXclusion: Fairness gaps in deep chest X-ray classifiers
Seyyed-Kalantari, L., Liu, G., McDermott, M., Chen, I. Y., & Ghassemi, M. (2020) · Biocomputing 2021 · doi:10.1142/9789811232701_0022 · Cited by 275 · cited 1×
“Aiming to detect and mitigate such harms, recent works leverage the algorithmic fairness paradigm [10] to define evaluation criteria and model development procedures that quantify and constrain the magnitude of statistical differences in model behavior or performance across patient subgroups [8, 9, 19, 22, 67, 70, 74,…”
BackgroundYadlowsky, S., Hayward, R. A., Sussman, J. B., McClelland, R. L., Min, Y.-I., & Basu, S. (2018) · Annals of Internal Medicine · doi:10.7326/m17-3011 · Cited by 271
Classification with Fairness Constraints: A Meta-Algorithm with Provable Guarantees
Celis, L. E., Huang, L., Keswani, V., & Vishnoi, N. K. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287586 · Cited by 231 · cited 3×
“We focus our efforts on “in-processing” approaches [2, 17, 26, 27, 105] rather than on pre- [43, 55, 56, 84, 106] or post-processing [10, 39] (e.”
MethodA scalable discrete-time survival model for neural networks
Gensheimer, M. F., & Narasimhan, B. (2019) · PeerJ · doi:10.7717/peerj.6257 · Cited by 229
Machine Learning Outperforms ACC/AHA CVD Risk Calculator in MESA
Kakadiaris, I. A., Vrigkas, M., Yen, A. A., Kuznetsova, T., Budoff, M., & Naghavi, M. (2018) · Journal of the American Heart Association · doi:10.1161/jaha.118.009476 · Cited by 227 · cited 1×
“[81] Ioannis A Kakadiaris, Michalis Vrigkas, Albert A Yen, Tatiana Kuznetsova, Matthew Budoff, and Morteza Naghavi.”
BackgroundThree myths about risk thresholds for prediction models
on behalf of the Topic Group ‘Evaluating diagnostic tests and prediction models’ of the STRATOS initiative, Wynants, L., van Smeden, M., McLernon, D. J., Timmerman, D., Steyerberg, E. W., & Van Calster, B. (2019) · BMC Medicine · doi:10.1186/s12916-019-1425-3 · Cited by 213
Don’t ask if artificial intelligence is good or fair, ask how it shifts power
Kalluri, P. (2020) · Nature · doi:10.1038/d41586-020-02003-2 · Cited by 213
Reps, J. M., Schuemie, M. J., Suchard, M. A., Ryan, P. B., & Rijnbeek, P. R. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy032 · Cited by 205
Managing Dyslipidemia in Chronic Kidney Disease
Harper, C. R., & Jacobson, T. A. (2008) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2008.03.025 · Cited by 204
Modeling Discrete Time-to-Event Data
Tutz, G., & Schmid, M. (2016) · Springer Series in Statistics · doi:10.1007/978-3-319-28158-2 · Cited by 185
Learning Non-Discriminatory Predictors
Woodworth, B., Gunasekar, S., Ohannessian, M. I., & Srebro, N. (2017) · arXiv · doi:10.48550/arxiv.1702.06081 · Cited by 182
The Variational Fair Autoencoder
Louizos, C., Swersky, K., Li, Y., Welling, M., & Zemel, R. S. (2015) · UvA-DARE (University of Amsterdam) · doi:10.48550/arxiv.1511.00830 · Cited by 178
DeFilippis, A. P., Young, R., McEvoy, J. W., Michos, E. D., Sandfort, V., Kronmal, R. A., McClelland, R. L., & Blaha, M. J. (2016) · European Heart Journal · doi:10.1093/eurheartj/ehw301 · Cited by 169
An empirical characterization of fair machine learning for clinical risk prediction
Pfohl, S. R., Foryciarz, A., & Shah, N. H. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103621 · Cited by 164
Self-citationCalibration of the Pooled Cohort Equations for Atherosclerotic Cardiovascular Disease: An Update
Cook, N. R., & Ridker, P. M. (2016) · Annals of Internal Medicine · doi:10.7326/m16-1739 · Cited by 155
The problem of infra-marginality in outcome tests for discrimination
Simoiu, C., Corbett-Davies, S., & Goel, S. (2017) · The Annals of Applied Statistics · doi:10.1214/17-aoas1058 · Cited by 154 · cited 2×
“[32] Camelia Simoiu, Sam Corbett-Davies, and Sharad Goel.”
BackgroundMethodJung, K. J., Jang, Y., Oh, D. J., Oh, B., Lee, S. H., Park, S., Seung, K. B., Kim, H., Yun, Y. D., Choi, S. H., et al. (2015) · Atherosclerosis · doi:10.1016/j.atherosclerosis.2015.07.033 · Cited by 140
Sox, H. C., Higgins, M. C., & Owens, D. K. (2013) · Wiley eBooks · doi:10.1002/9781118341544 · Cited by 136
Algorithmic Fairness from a Non-ideal Perspective
Fazelpour, S., & Lipton, Z. C. (2020) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3375627.3375828 · Cited by 119
Ward, A., Sarraju, A., Chung, S., Li, J., Harrington, R., Heidenreich, P., Palaniappan, L., Scheinker, D., & Rodriguez, F. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-00331-1 · Cited by 112 · cited 1×
“…these issues include the development of new risk estimators from large, diverse observational cohorts using modern machine learning methods [49, 70, 98, 104, 107], revisions to guidelines to encourage follow-up testing when the benefits of statin therapy are unclear and shared patient-clinician…”
BackgroundVickers, A. J., Kattan, M. W., & Sargent, D. J. (2007) · Trials · doi:10.1186/1745-6215-8-14 · Cited by 105
Equity in essence: a call for operationalising fairness in machine learning for healthcare
Wawira Gichoya, J., McCoy, L. G., Celi, L. A., & Ghassemi, M. (2021) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2020-100289 · Cited by 104
Zhao, Y., Wood, E. P., Mirin, N., Vedanthan, R., Cook, S., & Chunara, R. (2020) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2021.04.016 · Cited by 101
Blanche, P., Dartigues, J., & Jacqmin‐Gadda, H. (2013) · Biometrical Journal · doi:10.1002/bimj.201200045 · Cited by 94
Training Well-Generalizing Classifiers for Fairness Metrics and Other Data-Dependent ConstraintsPreprint
Cotter, A., Gupta, M. R., Jiang, H., Srebro, N., Sridharan, K., Wang, S., Woodworth, B., & You, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.00028 · Cited by 84
Tree-based multivariate regression and density estimation with right-censored data
Molinaro, A. M., Dudoit, S., & van der Laan, M. J. (2004) · Journal of Multivariate Analysis · doi:10.1016/j.jmva.2004.02.003 · Cited by 83
Mora, S., Wenger, N. K., Cook, N. R., Liu, J., Howard, B. V., Limacher, M. C., Liu, S., Margolis, K. L., Martin, L. W., Paynter, N. P., et al. (2018) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2018.2875 · Cited by 83
Pairwise Fairness for Ranking and Regression
Narasimhan, H., Cotter, A., Gupta, M., & Wang, S. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5970 · Cited by 82
A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle
Suresh, H., & Guttag, J. (2021) · Equity and Access in Algorithms, Mechanisms, and Optimization · doi:10.1145/3465416.3483305 · Cited by 82
A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle
Suresh, H., & Guttag, J. V. (2019) · arXiv · doi:10.48550/arxiv.1901.10002 · Cited by 82 · cited 1×
“Finally, it is important to recognize that the algorithmic fairness paradigm, inclusive of the approach considered in this work, is insufficient to characterize the extent to which machine learning systems may exacerbate health disparities or promote health equity [12, 18, 31, 36, 59, 71, 89].”
BackgroundCosby, K. (2017) · Diagnosis · doi:10.1201/9781315116334-2 · Cited by 80
A framework for making predictive models useful in practice
Jung, K., Kashyap, S., Avati, A., Harman, S., Shaw, H., Li, R., Smith, M., Shum, K., Javitz, J., Vetteth, Y., et al. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa318 · Cited by 78
Self-citationMinimax group fairness: algorithms and experiments
Diana, E., Gill, W., Kearns, M., Kenthapadi, K., & Roth, A. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462523 · Cited by 74 · cited 12×
“We compare pooled and stratified unconstrained empirical risk minimization (ERM) to regularized fairness objectives and distributionally robust optimization (DRO) objectives that aim to minimize differences in or improve the worst-case area under the receiver operating characteristic curve (AUC) or log-loss across sub…”
BackgroundMethodInfluentialAddressing bias in prediction models by improving subpopulation calibration
Barda, N., Yona, G., Rothblum, G. N., Greenland, P., Leibowitz, M., Balicer, R., Bachmat, E., & Dagan, N. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa283 · Cited by 59
Management of cardiovascular risk in patients with rheumatoid arthritis: evidence and expert opinion
van den Oever, I. A., van Sijl, A. M., & Nurmohamed, M. T. (2013) · Therapeutic Advances in Musculoskeletal Disease · doi:10.1177/1759720x13491025 · Cited by 58
Machine Learning, Health Disparities, and Causal Reasoning
Goodman, S. N., Goel, S., & Cullen, M. R. (2018) · Annals of Internal Medicine · doi:10.7326/m18-3297 · Cited by 57
Fair regression for health care spending
Zink, A., & Rose, S. (2020) · Biometrics · doi:10.1111/biom.13206 · Cited by 56
Fairness On The Ground: Applying Algorithmic Fairness Approaches to Production SystemsPreprint
Bakalar, C., Barreto, R. D. C., Bergman, S., Bogen, M., Chern, B., Corbett‐Davies, S., Hall, M. A., Kloumann, I. M., Lam, M. S., Candela, J. Q., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.06172 · Cited by 49
Diabetes, Kidney Disease, and Cardiovascular Outcomes in the Jackson Heart Study.
Afkarian, M., Katz, R., Bansal, N., Correa, A., Kestenbaum, B., Himmelfarb, J., de Boer, I., & Young, B. (2016) · American Society of Nephrology. Clinical Journal · doi:10.2215/cjn.13111215 · Cited by 46
Pfohl, S. R., Zhang, H., Xu, Y., Foryciarz, A., Ghassemi, M., & Shah, N. H. (2022) · Scientific Reports · doi:10.1038/s41598-022-07167-7 · Cited by 43
Self-citationForyciarz, A., Pfohl, S. R., Patel, B., & Shah, N. (2022) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2021-100460 · Cited by 36
Self-citation(1992) · Birkhäuser Boston eBooks · doi:10.1007/978-1-4757-1229-2 · Cited by 32
Reading Race: AI Recognises Patient's Racial Identity In Medical Images
Banerjee, I., Bhimireddy, A. R., Burns, J. L., Celi, L. A., Chen, L.-C., Correa, R., Dullerud, N., Ghassemi, M., Huang, S.-C., Kuo, P., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.10356 · Cited by 29
Cotter, A., Jiang, H., Wang, S., Narayan, T., Gupta, M. R., You, S., & Sridharan, K. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.04198 · Cited by 28
Fairness and Transparency in the Age of the Algorithm
Olhede, S., & Rodrigues, R. (2017) · Significance · doi:10.1111/j.1740-9713.2017.01012.x · Cited by 25
Foryciarz, A., Pfohl, S., Patel, B., et al. (2021) · medRxiv · doi:10.1101/2021.11.08.21266076 · Cited by 2
Self-citation
Research works printed but not linked: 31
- Ruha Benjamin. 2019. Assessing risk, automating racism. Science 366, 6464 (2019), 421–422.
- Diana Cagliero, Natalie Deuitch, Nigam Shah, and Danton Char. 2021. Evaluating ethical concerns with machine learning to guide advance care planning. In 2021 Western Medical Research Conference, Vol. 69. BMJ Publishing Group Limited, 103–296.
- Irene Y Chen, Emma Pierson, Sherri Rose, Shalmali Joshi, Kadija Ferryman, and Marzyeh Ghassemi. 2020. Ethical Machine Learning in Healthcare. Annual Review of Biomedical Data Science 4 (2020).
- Irene Y Chen, Peter Szolovits, and Marzyeh Ghassemi. 2019. Can AI help reduce disparities in general medical and mental health care?AMA journal of ethics 21, 2 (2019), 167–179.
- 10.5555/3295222.3295224 · doi:10.5555/3295222.3295224
- R Yates Coley, Eric Johnson, Gregory E Simon, Maricela Cruz, and Susan M Shortreed. 2021. Racial/ethnic disparities in the performance of prediction models for death by suicide after mental health visits. JAMA psychiatry (2021).
- Timnit Gebru Jamie Morgenstern Briana Vecchione Jennifer Wortman Vaughan Hanna Wallach Hal Daumé III and Kate Crawford. 2018. Datasheets for datasets. arXiv preprint arXiv:1803.09010(2018). · arXiv
- David C Goff, Donald M Lloyd-Jones, Glen Bennett, Sean Coady, Ralph B D’agostino, Raymond Gibbons, Philip Greenland, Daniel T Lackland, Daniel Levy, Christopher J O’donnell, 2014. 2013 ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Journal of the American College of Cardiology 63, 25 Part B(2014), 2935–2959.
- Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. 2012. A kernel two-sample test. Journal of Machine Learning Research 13, Mar (2012), 723–773.
- Weihua Hu, Gang Niu, Issei Sato, and Masashi Sugiyama. 2018. Does distributionally robust supervised learning give robust classifiers?. In International Conference on Machine Learning. PMLR, 2029–2037.
- Christina Ilvento. 2020. Metric Learning for Individual Fairness. In 1st Symposium on Foundations of Responsible Computing (FORC 2020). Schloss Dagstuhl-Leibniz-Zentrum für Informatik.
- Guido W Imbens and Donald B Rubin. 2015. Causal inference in statistics, social, and biomedical sciences. Cambridge University Press.
- Håvard Kvamme and Ørnulf Borgan. 2019. Continuous and discrete-time survival prediction with neural networks. arXiv preprint arXiv:1910.06724(2019). · arXiv
- Lydia T Liu, Max Simchowitz, and Moritz Hardt. 2019. The Implicit Fairness Criterion of Unconstrained Learning. In Proceedings of the 36th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 97), Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR, Long Beach, California, USA, 4051–4060. http://proceedings.mlr.press/v97/liu19f.html
- David Madras, Elliot Creager, Toniann Pitassi, and Richard Zemel. 2018. Learning Adversarially Fair and Transferable Representations. Proceedings of the 35th International Conference on Machine Learning 80 (feb 2018), 3384–3393. arxiv:1802.06309http://proceedings.mlr.press/v80/madras18a.htmlhttp://arxiv.org/abs/1802.06309
- Natalia Martinez, Martin Bertran, and Guillermo Sapiro. 2020. Minimax Pareto fairness: A multi objective perspective. In International Conference on Machine Learning. PMLR, 6755–6764.
- Ziad Obermeyer, Brian Powers, Christine Vogeli, and Sendhil Mullainathan. 2019. Dissecting racial bias in an algorithm used to manage the health of populations. Science 366, 6464 (2019), 447–453.
- Gulsen Ozen, Murat Sunbul, Pamir Atagunduz, Haner Direskeneli, Kursat Tigen, and Nevsun Inanc. 2016. The 2013 ACC/AHA 10-year atherosclerotic cardiovascular disease risk index is better than SCORE and QRisk II in rheumatoid arthritis: is it enough?Rheumatology 55, 3 (2016), 513–522.
- Yoonyoung Park, Jianying Hu, Moninder Singh, Issa Sylla, Irene Dankwa-Mullan, Eileen Koski, and Amar K Das. 2021. Comparison of Methods to Reduce Bias From Clinical Prediction Models of Postpartum Depression. JAMA network open 4, 4 (2021), e213909–e213909.
- Stephen Pfohl, Ben Marafino, Adrien Coulet, Fatima Rodriguez, Latha Palaniappan, and Nigam H. Shah. 2019. Creating Fair Models of Atherosclerotic Cardiovascular Disease Risk. In AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society. arxiv:1809.04663 · arXiv
- Geoff Pleiss Manish Raghavan Felix Wu Jon Kleinberg and Kilian Q. Weinberger. 2017. On fairness and calibration. In Advances in Neural Information Processing Systems. 5680–5689. arxiv:1709.02012 · arXiv
- 10.1093/jamia · doi:10.1093/jamia
- Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. 2020. Distributionally Robust Neural Networks. In International Conference on Learning Representations.
- Mark P Sendak, Michael Gao, Nathan Brajer, and Suresh Balu. 2020. Presenting machine learning model information to clinical end users with model facts labels. NPJ digital medicine 3, 1 (2020), 1–4.
- Laleh Seyyed-Kalantari, Haoran Zhang, Matthew McDermott, Irene Y Chen, and Marzyeh Ghassemi. 2021. Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations. Nature medicine (2021), 1–7.
- Jiaming Song, Pratyusha Kalluri, Aditya Grover, Shengjia Zhao, and Stefano Ermon. 2019. Learning controllable fair representations. In The 22nd International Conference on Artificial Intelligence and Statistics. PMLR, 2164–2173.
- Handrean Soran, Jonathan D Schofield, and Paul N Durrington. 2015. Cholesterol, not just cardiovascular risk, is important in deciding who should receive statin treatment. European heart journal 36, 43 (2015), 2975–2983.
- Harini Suresh and John V. Guttag. 2019. A Framework for Understanding Unintended Consequences of Machine Learning. www.aaai.orghttp://arxiv.org/abs/1901.10002
- Robert C. Williamson and Aditya Krishna Menon. 2019. Fairness risk measures. In 36th International Conference on Machine Learning, ICML 2019, Vol. 2019-June. International Machine Learning Society (IMLS), 11763–11774. arxiv:1901.08665https://arxiv.org/abs/1901.08665v1
- Steve Yadlowsky, Sanjay Basu, and Lu Tian. 2019. A calibration metric for risk scores with survival data. In Machine Learning for Healthcare Conference. 424–450.
- Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, Krishna P Gummadi, Manuel Gomez Rogriguez, and Krishna P Gummadi. 2017. Fairness Constraints: Mechanisms for Fair Classification. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics(Proceedings of Machine Learning Research, Vol. 54), Aarti Singhand Jerry Zhu (Eds.). PMLR, Fort Lauderdale, FL, USA, 962–970.
Cited by
Bias recognition and mitigation strategies in artificial intelligence healthcare applications
Hasanzadeh, F., Josephson, C. B., Waters, G., Adedinsewo, D., Azizi, Z., & White, J. A. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01503-7 · Cited by 320
The limits of fair medical imaging AI in real-world generalization
Yang, Y., Zhang, H., Gichoya, J. W., Katabi, D., & Ghassemi, M. (2024) · Nature Medicine · doi:10.1038/s41591-024-03113-4 · Cited by 285
“…that differences in calibration between groups is a significant source of disparity, as consistent under-estimation or over-estimation of risk for a particular group could lead to under-treatment or over-treatment for that group at a fixed operating threshold relative to the true risk 76 .”
APPRAISE-AI Tool for Quantitative Evaluation of AI Studies for Clinical Decision Support
Kwong, J. C. C., Khondker, A., Lajkosz, K., McDermott, M. B. A., Frigola, X. B., McCradden, M. D., Mamdani, M., Kulkarni, G. S., & Johnson, A. E. W. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.35377 · Cited by 142
Human visual explanations mitigate bias in AI-based assessment of surgeon skills
Kiyasseh, D., Laca, J., Haque, T. F., Otiato, M., Miles, B. J., Wagner, C., Donoho, D. A., Trinh, Q.-D., Anandkumar, A., & Hung, A. J. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00766-2 · Cited by 53
de Hond, A., van Buchem, M. M., Fanconi, C., Roy, M., Blayney, D. W., Kant, I. M. J., Steyerberg, E. W., & Hernandez‐Boussard, T. (2024) · JMIR Medical Informatics · doi:10.2196/51925 · Cited by 33
“A next step could be to apply bias mitigation techniques for in-or postprocessing during model development, like threshold selection and recalibration within specific groups [55].”
Metadata
- Authors
- Pfohl, S., Xu, Y., Foryciarz, A., Ignatiadis, N., Genkins, J., & Shah, N.
- Affiliations
- Stanford University (US)
- Venue
- 2022 ACM Conference on Fairness Accountability and Transparency
- Publisher
- ACM
- Published
- 2022-06-20
- Pages
- 1039-1052
- OpenAlex
- W4283162970
- Cited by
- 30
- References
- 69
- Citations by year
- 2026: 7 · 2025: 8 · 2024: 6 · 2023: 8 · 2022: 1
- FWCI
- 5.71
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Heart, Lung, and Blood Institute (R01 HL144555)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- algorithmic fairness, health care, health disparities, net benefit, model calibration, threshold selection, training objectives, statin initiation, prediction model, model evaluation metrics, measurement error
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.