A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle
Open accessAbstract
As machine learning (ML) increasingly affects people and society, awareness of its potential unwanted consequences has also grown. To anticipate, prevent, and mitigate undesirable downstream consequences, it is critical that we understand when and how harm might be introduced throughout the ML life cycle. In this paper, we provide a framework that identifies seven distinct potential sources of downstream harm in machine learning, spanning data collection, development, and deployment. In doing so, we aim to facilitate more productive and precise communication around these issues, as well as more direct, application-grounded ways to mitigate them.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
One of the main causes of undesirable learned patterns lies in biased training data.
“Yet, one of the main causes of undesirable learned patterns lies in biased training data55,56.”
From The METRIC-framework for assessing data quality for trustworthy AI in medicine: A systematic review (2024) · cites this paper as [55]
The Introduction (page unavailable) acknowledges that biased data can contribute to undesirable ML behavior, and §3.2 describes how an unrepresentative development sample can impair a model. The paper does not rank biased training data as a main cause; it also identifies sources of harm in modeling, evaluation, and deployment. (LLM, conf=0.85)
Experimental — accuracy not yet measuredAddressing safety and ethical concerns, including algorithmic bias, data privacy and security, and transparency, is a critical part of implementing AI in healthcare.
“More critically, it involves addressing safety and ethical concerns, including algorithmic bias, data privacy and security, and transparency16,17.”
From Establishing organizational AI governance in healthcare: a case study in Canada (2025) · cites this paper as [17]
The Introduction and §3.8 describe harms from ML systems and identify bias in medical applications as a concern. Section 3.5.1 discusses privacy, including a trade-off between differential privacy and performance for underrepresented data. The paper does not substantively address data security or transparency as concerns in implementing AI in healthcare, so it supports only part of the attributed list. (LLM, conf=0.86)
Experimental — accuracy not yet measuredTraining data used to train an LLM may be drawn from a nonrepresentative sample of the population, which can cause the model to fail to generalize well to some social groups.
“Training Data: The data used to train an LLM may be drawn from a nonrepresentative sample of the population, which can cause the model to fail to generalize well to some social groups.”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Suresh and Guttag 2021
Section 3.2 (Representation Bias; page unavailable) says that an underrepresentative development sample can cause a model to generalize poorly to a subset of its use population. This supports the claimed mechanism, but the paper discusses machine-learning models generally rather than establishing it specifically for LLMs. (LLM, conf=0.84)
Experimental — accuracy not yet measuredTraining data may omit important contexts, and proxies used as labels may incorrectly measure the actual outcome of interest.
“The data may omit important contexts, and proxies used as labels ( e.g. , sentiment) may incorrectly measure the actual outcome of interest ( e.g. , representational harms).”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Suresh and Guttag 2021
Section 3.2 (Representation Bias) explains that development data may fail to represent the population or setting in which a model is used. Section 3.3 (Measurement Bias) explains that labels can be poor proxies for the constructs they are meant to measure. The paper does not connect the citing sentence’s specific example of sentiment labels to measuring representational harms. (LLM, conf=0.82)
Experimental — accuracy not yet measuredStandard benchmarks may misrepresent some sub-populations and single average performance metrics can hide poor performance on a minority group.
“Standard benchmarks may misrepresent some sub-populations and single average performance metrics can hide poor performance on a minority group ([69]).”
From Fairness and explainability in automatic decision-making systems. A challenge for computer science and law (2023) · cites this paper as [69]
The aggregation of data may obscure distinct social groups that should be treated differently.
“The aggregation of data may also obscure distinct social groups that should be treated differently, causing the model”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Suresh and Guttag 2021
Bias emerges at points within the model development and deployment process, which may exacerbate preexisting historical biases.
“Another way of understanding social bias in LLMs is to examine at which points within the model development and deployment process the bias emerges, which may exacerbate preexisting historical biases. This has been thoroughly explored by Mehrabi et al. (2021), Shah, Schwartz, and Hovy (2020), and Suresh and Guttag (2021), and we summarize these pathways here:”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Suresh and Guttag 2021
If a particular medication is primarily tested on a specific population (e.g., predominantly White males), there may be limited evidence on how it affects other populations, leading to biased healthcare decisions for those groups.
“For example, if a particular medication is primarily tested on a specific population (e.g., predominantly White males), there may be limited evidence on how it affects other populations, leading to biased healthcare decisions for those groups [71].”
From Ethical Framework for Harnessing the Power of AI in Healthcare and Beyond (2024) · cites this paper as [71]
In healthcare, historical bias can be seen in the underrepresentation of certain demographic groups in clinical trials or medical research.
“In healthcare, historical bias can be seen in the underrepresentation of certain demographic groups in clinical trials or medical research.”
From Ethical Framework for Harnessing the Power of AI in Healthcare and Beyond (2024) · cites this paper as [71]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865 · see citing works
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 24,190 · see citing works
A Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2022) · ACM Computing Surveys · doi:10.1145/3457607 · Cited by 5,812 · see citing works
Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments
Huang, G. B., Mattar, M., Berg, T. L., & Miller, E. G. (2008) · HAL · Cited by 4,445 · see citing works
(2018) · Cited by 3,510 · see citing works
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501 · see citing works
Barocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048 · see citing works
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681 · see citing works
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · see citing works
Hoffman, K. M., Trawalter, S., Axt, J. R., & Oliver, M. N. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1516047113 · Cited by 2,486 · see citing works
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159 · see citing works
Certifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827 · see citing works
Fairness and Abstraction in Sociotechnical Systems
Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287598 · Cited by 1,631 · see citing works
Impact of weight bias and stigma on quality of care and outcomes for patients with obesity
Phelan, S. M., Burgess, D. J., Yeazel, M. W., Hellerstedt, W. L., Griffin, J. M., & van Ryn, M. (2015) · Obesity Reviews · doi:10.1111/obr.12266 · Cited by 1,361 · see citing works
Friedman, B., & Nissenbaum, H. (1996) · ACM Transactions on Information Systems · doi:10.1145/230538.230561 · Cited by 1,203 · cited 1× · see citing works
“Similarly, Friedman and Nissenbaum [21]’s “preexisting bias” might map to our “historical bias” (Section 3.1) when it describes existing societal stereotypes that are reflected in datasets.”
BackgroundInfluentialAlgorithmic Decision Making and the Cost of Fairness
Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098095 · Cited by 1,149 · see citing works
(2013) · Cited by 1,126 · see citing works
The accuracy, fairness, and limits of predicting recidivism
Dressel, J., & Farid, H. (2018) · Science Advances · doi:10.1126/sciadv.aao5580 · Cited by 1,071 · see citing works
Word embeddings quantify 100 years of gender and ethnic stereotypes
Garg, N., Schiebinger, L., Jurafsky, D., & Zou, J. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1720347115 · Cited by 1,065 · see citing works
Bias in data‐driven artificial intelligence systems—An introductory survey
Ntoutsi, E., Fafalios, P., Gadiraju, U., Iosifidis, V., Nejdl, W., Vidal, M., Ruggieri, S., Turini, F., Papadopoulos, S., Krasanakis, E., et al. (2020) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1356 · Cited by 1,043 · see citing works
Kusner, M. J., Loftus, J. R., Russell, C., & Silva, R. (2017) · arXiv · doi:10.48550/arxiv.1703.06856 · Cited by 995 · see citing works
Inherent trade-offs in the fair determination of risk scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2017) · Schloss Dagstuhl – Leibniz-Zentrum für Informatik · doi:10.4230/lipics.itcs.2017.43 · Cited by 943 · see citing works
Mitigating bias in algorithmic hiring: evaluating claims and practices
Raghavan, M., Barocas, S., Kleinberg, J., & Levy, K. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372828 · Cited by 845 · see citing works
Discrimination in online ad delivery
Sweeney, L. (2013) · Communications of the ACM · doi:10.1145/2447976.2447990 · Cited by 783 · see citing works
Zafar, M. B., Valera, I., Gomez Rodriguez, M., & Gummadi, K. P. (2017) · Proceedings of the 26th International Conference on World Wide Web · doi:10.1145/3038912.3052660 · Cited by 632 · see citing works
The Girl Who Cried Pain: A Bias against Women in the Treatment of Pain
Hoffmann, D. E., & Tarzian, A. J. (2001) · Journal of Law, Medicine & Ethics · doi:10.1111/j.1748-720x.2001.tb00037.x · Cited by 622 · see citing works
A comparative study of fairness-enhancing interventions in machine learning
Friedler, S. A., Scheidegger, C., Venkatasubramanian, S., Choudhary, S., Hamilton, E. P., & Roth, D. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287589 · Cited by 601 · see citing works
Kärkkäinen, K., & Joo, J. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00159 · Cited by 546 · see citing works
Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI
Jacovi, A., Marasović, A., Miller, T., & Goldberg, Y. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445923 · Cited by 545 · see citing works
Algorithmic Bias in Autonomous Systems
Danks, D., & London, A. J. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/654 · Cited by 472 · see citing works
The Measure and Mismeasure of Fairness
Corbett-Davies, S., Gaebler, J. D., Nilforoshan, H., Shroff, R., & Goel, S. (2018) · arXiv · doi:10.48550/arxiv.1808.00023 · Cited by 444 · see citing works
Avoiding Discrimination through Causal Reasoning
Kilbertus, N., Rojas-Carulla, M., Parascandolo, G., Hardt, M., Janzing, D., & Schölkopf, B. (2017) · arXiv · doi:10.48550/arxiv.1706.02744 · Cited by 358 · see citing works
Defining racial and ethnic disparities in pain management
Mossey, J. M. (2011) · Clinical Orthopaedics & Related Research · doi:10.1007/s11999-011-1770-9 · Cited by 353 · see citing works
Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting
De-Arteaga, M., Romanov, A., Wallach, H., Chayes, J., Borgs, C., Chouldechova, A., Geyik, S., Kenthapadi, K., & Kalai, A. T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287572 · Cited by 337 · see citing works
Herman, W. H., & Cohen, R. M. (2012) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2011-1894 · Cited by 287 · see citing works
Kleinberg, J., Ludwig, J., Mullainathan, S., & Rambachan, A. (2018) · AEA Papers and Proceedings · doi:10.1257/pandp.20181018 · Cited by 278 · see citing works
k-NN as an implementation of situation testing for discrimination discovery and prevention
Luong, B. T., Ruggieri, S., & Turini, F. (2011) · Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2020408.2020488 · Cited by 263 · see citing works
On the (im)possibility of fairness
Friedler, S. A., Scheidegger, C., & Venkatasubramanian, S. (2016) · arXiv · doi:10.48550/arxiv.1609.07236 · Cited by 247 · see citing works
Differential Privacy Has Disparate Impact on Model Accuracy
Bagdasaryan, E., & Shmatikov, V. (2019) · arXiv · doi:10.48550/arxiv.1905.12101 · Cited by 219 · see citing works
A Convex Framework for Fair Regression
Berk, R., Heidari, H., Jabbari, S., Joseph, M., Kearns, M., Morgenstern, J., Neel, S., & Roth, A. (2017) · arXiv · doi:10.48550/arxiv.1706.02409 · Cited by 198 · see citing works
Assessing Risk Assessment in Action
Stevenson, M. T. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.3016088 · Cited by 195 · see citing works
Why is my classifier discriminatory?
Chen, I., Johansson, F. D., & Sontag, D. (2018) · arXiv · doi:10.48550/arxiv.1805.12002 · Cited by 185 · see citing works
Moving beyond “algorithmic bias is a data problem”
Hooker, S. (2021) · Patterns · doi:10.1016/j.patter.2021.100241 · Cited by 181 · see citing works
Mitigating Bias in Algorithmic Hiring: Evaluating Claims and Practices
Raghavan, M., Barocas, S., Kleinberg, J., & Levy, K. (2019) · arXiv · Cited by 176 · see citing works
Shankar, S., Halpern, Y., Breck, E. J., Atwood, J. R., Wilson, J., & Sculley, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.08536 · Cited by 175 · see citing works
An other-race effect for face recognition algorithms
Phillips, P. J., Jiang, F., Narvekar, A., Ayyad, J., & O’Toole, A. J. (2011) · ACM Transactions on Applied Perception · doi:10.1145/1870076.1870082 · Cited by 173 · see citing works
On Formalizing Fairness in Prediction with Machine LearningPreprint
Gajane, P., & Pechenizkiy, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.03184 · Cited by 152 · see citing works
Calderone, K. L. (1990) · Sex Roles · doi:10.1007/bf00289259 · Cited by 141 · see citing works
Johndrow, J. E., & Lum, K. (2019) · The Annals of Applied Statistics · doi:10.1214/18-aoas1201 · Cited by 141 · see citing works
Characterising Bias in Compressed ModelsPreprint
Hooker, S., Moorosi, N., Clark, G., Samy Bengio, A., & Denton, E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.03058 · Cited by 132 · see citing works
Bringing the People Back In: Contesting Benchmark Machine Learning Datasets
Denton, R., Hanna, A., Amironesei, R., Smart, A., Nicole, H., & Scheuerman, M. K. (2020) · arXiv · doi:10.48550/arxiv.2007.07399 · Cited by 100 · see citing works
Bridging Machine Learning and Mechanism Design towards Algorithmic Fairness
Finocchiaro, J., Maio, R., Monachou, F., Patro, G. K., Raghavan, M., Stoica, A.-A., & Tsirtsis, S. (2021) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3442188.3445912 · Cited by 83 · see citing works
Frey, W. R., Patton, D. U., Gaskell, M. B., & McGregor, K. A. (2018) · Social Science Computer Review · doi:10.1177/0894439318788314 · Cited by 79 · see citing works
InclusiveFaceNet: Improving Face Attribute Detection with Race and Gender DiversityPreprint
Ryu, H. J., Adam, H., & Mitchell, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.00193 · Cited by 78 · see citing works
(2018) · Cited by 78 · see citing works
The Myth in the Methodology: Towards a Recontextualization of Fairness in Machine Learning
Green, B., & Hu, L. J. (2018) · International Conference on Machine Learning · Cited by 66 · see citing works
Taking Advantage of Multitask Learning for Fair Classification
Oneto, L., Doninini, M., Elders, A., & Pontil, M. (2019) · Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3306618.3314255 · Cited by 64 · see citing works
Learning Tasks for Multitask Learning: Heterogenous Patient Populations in the ICU
Suresh, H., Gong, J. J., & Guttag, J. V. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3219930 · Cited by 54 · see citing works
Self-citationContextual Analysis of Social Media
Patton, D. U., Frey, W. R., McGregor, K. A., Lee, F.-T., McKeown, K. R., & Moss, E. (2020) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3375627.3375841 · Cited by 50 · see citing works
Algorithms, Platforms, and Ethnic Bias: An Integrative EssayPreprint
Selena, S., & Kenney, M. F. (2018) · SSRN Electronic Journal · Cited by 47 · see citing works
Probably Approximately Metric-Fair LearningPreprint
Rothblum, G. N., & Yona, G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.03242 · Cited by 40 · see citing works
“Should This Loan be Approved or Denied?”: A Large Dataset with Class Assignment Guidelines
Li, M., Mickel, A. E., & Taylor, S. A. (2018) · Journal of Statistics Education · doi:10.1080/10691898.2018.1434342 · Cited by 33 · see citing works
Nabi, R., & Shpitser, I. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11553 · Cited by 20 · see citing works
The Inclusive Images Competition
Atwood, J. R., Halpern, Y., Baljekar, P. N., Breck, E. J., Sculley, D., Ostyakov, P., Nikolenko, S., Ivanov, I., Solovyev, R. A., Wang, W., et al. (2019) · The Springer series on challenges in machine learning · doi:10.1007/978-3-030-29135-8_6 · Cited by 12 · see citing works
Defining and Designing Fair Algorithms
Corbett‐Davies, S., Goel, S., Morgenstern, J., et al. (2018) · ACM Conference on Economics and Computation (EC) · doi:10.1145/3219166.3277556 · Cited by 7 · see citing works
Flagged first, then by citations
Research works printed but not linked: 3
- Erin Collins. 2018. Punishing Risk. Geo. LJ 107 (2018), 57.
- Cynthia Dwork, Nicole Immorlica, Adam Tauman Kalai, and Max Leiserson. 2018. Decoupled Classifiers for Group-Fair and Efficient Machine Learning. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency (Proceedings of Machine Learning Research, Vol. 81), Sorelle A. Friedler and Christo Wilson (Eds.). PMLR, New York, NY, USA, 119-133. http://proceedings.mlr.press/ v81/dwork18a.html
- Steven L Salzberg. 1997. On comparing classifiers: Pitfalls to avoid and a recom- mended approach. Data mining and knowledge discovery 1, 3 (1997), 317-328.
Cited by
Sorted by citations
Díaz-Rodríguez, N., Del Ser, J., Coeckelbergh, M., López de Prado, M., Herrera-Viedma, E., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101896 · Cited by 800 · see citing works
Bias and Fairness in Large Language Models: A Survey
Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., & Ahmed, N. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00524 · Cited by 665 · see citing works
Towards a standard for identifying and managing bias in artificial intelligence
Schwartz, R., Vassilev, A., Greene, K., Perine, L., Burt, A., & Hall, P. (2022) · doi:10.6028/nist.sp.1270 · Cited by 548 · see citing works
“When individuals feel that they are not being fairly judged when applying for jobs [2, 3, 5, 7, 54–57] or loans [58–60] it can reduce public trust in AI technology [61, 62].”
BackgroundFairness in machine learning: A survey
Caton, S., & Haas, C. (2024) · ACM Computing Surveys · doi:10.1145/3616865 · Cited by 646 · see citing works
Machine learning in concrete science: applications, challenges, and best practices
Li, Z., Yoon, J., Zhang, R., Rajabipour, F., Srubar III, W. V., Dabo, I., & Radlińska, A. (2022) · npj Computational Materials · doi:10.1038/s41524-022-00810-x · Cited by 388 · see citing works
Haltaufderheide, J., & Ranisch, R. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01157-x · Cited by 396 · see citing works
Artificial Intelligence in Education: Implications for Policymakers, Researchers, and Practitioners
Ifenthaler, D., Majumdar, R., Gorissen, P., Judge, M., Mishra, S., Raffaghelli, J., & Shimada, A. (2024) · Technology, Knowledge and Learning · doi:10.1007/s10758-024-09747-0 · Cited by 238 · see citing works
The METRIC-framework for assessing data quality for trustworthy AI in medicine: A systematic review
Schwabe, D., Becker, K., Seyferth, M., Klaß, A., & Schaeffter, T. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01196-4 · Cited by 169 · see citing works
Aderibigbe, A. O., Ohenhen, P. E., Nwaobia, N. K., Gidiagba, J. O., & Ani, E. C. (2023) · Computer Science & IT Research Journal · doi:10.51594/csitrj.v4i3.629 · Cited by 172 · see citing works
Koçak, B., Ponsiglione, A., Stanzione, A., Bluethgen, C., Santinha, J., Ugga, L., Huisman, M., Klontzas, M. E., Cannella, R., & Cuocolo, R. (2024) · Diagnostic and Interventional Radiology · doi:10.4274/dir.2024.242854 · Cited by 168 · see citing works
“69,70 These issues warrant thoughtful data collection and processing to ensure that datasets are representative of the diversity of the population or phenomena they are intended to model.”
Ethical Framework for Harnessing the Power of AI in Healthcare and Beyond
Nasir, S., Khan, R. A., & Bai, S. (2024) · IEEE Access · doi:10.1109/access.2024.3369912 · Cited by 152 · see citing works
Drukker, K., Chen, W., Gichoya, J. W., Gruszauskas, N. P., Kalpathy-Cramer, J., Koyejo, S., Myers, K. J., Sá, R. C., Sahiner, B., Whitney, H. M., et al. (2023) · Journal of Medical Imaging · doi:10.1117/1.jmi.10.6.061104 · Cited by 149 · see citing works
Fairness in Machine Learning: A Survey
Caton, S., & Haas, C. (2020) · arXiv · doi:10.48550/arxiv.2010.04053 · Cited by 646 · see citing works
Zhang, R., Li, H., Meng, H., Zhan, J., Gan, H., & Lee, Y.-C. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713429 · Cited by 123 · see citing works
Economou-Zavlanos, N. J., Bessias, S., Cary, M. P., Bedoya, A. D., Goldstein, B. A., Jelovsek, J. E., O’Brien, C. L., Walden, N., Elmore, M., Parrish, A. B., et al. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad221 · Cited by 57 · see citing works
Kim, J. Y., Hasan, A., Kellogg, K. C., Ratliff, W., Murray, S. G., Suresh, H., Valladares, A., Shaw, K., Tobey, D., Vidal, D. E., et al. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000390 · Cited by 61 · see citing works
Self-citationEstablishing organizational AI governance in healthcare: a case study in Canada
Kim, J. Y., Hasan, A., Kueper, J., Tang, T., Hayes, C., Fine, B., Balu, S., & Sendak, M. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01909-3 · Cited by 42 · see citing works
Mccradden, M., Odusi, O., Joshi, S., Akrout, I., Ndlovu, K., Glocker, B., Maicas, G., Liu, X., Mazwi, M., Garnett, T., et al. (2023) · 2023 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3593013.3594096 · Cited by 33 · see citing works
Warr, M., & Heath, M. K. (2025) · Journal of Teacher Education · doi:10.1177/00224871251325073 · Cited by 35 · see citing works
Directional Bias Amplification
Wang, A., & Russakovsky, O. (2021) · arXiv · doi:10.48550/arxiv.2102.12594 · Cited by 30 · see citing works
Kirat, T., Tambou, O., Do, V., & Tsoukiàs, A. (2023) · EURO Journal on Decision Processes · doi:10.1016/j.ejdp.2023.100036 · Cited by 24 · see citing works
Beyond Behaviorist Representational Harms: A Plan for Measurement and Mitigation
Chien, J., & Danks, D. (2024) · The 2024 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3630106.3658946 · Cited by 22 · see citing works
“…Image Cropping, Story Generation, Image Tagging, Knowledge Distillation Reification of Social Groups or Use of Es-sentialist Categories [7, 65, 111, 119, 123, 128, 135] Image Captioning, Image Cropping, Image Tag-ging Inaccurate/Skewed Representations [44, 82, 119, 128, 135, 137] Misinformation,…”
BackgroundA Classification of Feedback Loops and Their Relation to Biases in Automated Decision-Making Systems
Pagan, N., Baumann, J., Elokda, E., De Pasquale, G., Bolognani, S., & Hannák, A. (2023) · arXiv · doi:10.48550/arxiv.2305.06055 · Cited by 4 · see citing works
EU Administrative Decision-Making Delegated to Machines – Legal Challenges and Issues
Hubková, P. (2024) · AUC IURIDICA · doi:10.14712/23366478.2024.25 · Cited by 4 · see citing works
- Authors
- Suresh, H., & Guttag, J.
- Affiliations
- Massachusetts Institute of Technology (US)
- Venue
- Equity and Access in Algorithms, Mechanisms, and Optimization
- Publisher
- ACM
- Published
- 2021-10-05
- Pages
- 1-9
- OpenAlex
- W3172100757
- Cited by
- 82
- References
- 66
- Citations by year
- 2026: 1 · 2025: 8 · 2024: 12 · 2023: 15 · 2022: 8 · 2021: 23 · 2020: 13 · 2019: 2
- FWCI
- 13.13
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO) (publishedVersion, cc-by-nc)
- arXiv (Cornell University) (submittedVersion)
- DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc)
- arXiv (Cornell University)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Keywords
- machine learning lifecycle, harm mitigation, data collection, model deployment, societal impact of AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
