Who Is Included in Human Perceptions of AI?: Trust and Perceived Fairness around Healthcare AI and Cultural Mistrust
Abstract
Emerging research suggests that people trust algorithmic decisions less than human decisions. However, different populations, particularly in marginalized communities, may have different levels of trust in human decision-makers. Do people who mistrust human decision-makers perceive human decisions to be more trustworthy and fairer than algorithmic decisions? Or do they trust algorithmic decisions as much as or more than human decisions? We examine the role of mistrust in human systems in people’s perceptions of algorithmic decisions. We focus on healthcare Artificial Intelligence (AI), group-based medical mistrust, and Black people in the United States. We conducted a between-subjects online experiment to examine people’s perceptions of skin cancer screening decisions made by an AI versus a human physician depending on their medical mistrust, and we conducted interviews to understand how to cultivate trust in healthcare AI. Our findings highlight that research around human experiences of AI should consider critical differences in social groups.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The study is subject to representativeness limitations.
“Thus, we acknowledge that our study is subject to representativeness limitations [61].”
From Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability (2023) · cites this paper as [61]
Cites
Qualitative research and evaluation methods
Patton, M. Q. (1980) · Cited by 35,637 · see citing works
Basics of Qualitative Research
Ozanne, J. L., Strauss, A., & Corbin, J. M. (1992) · Journal of Marketing Research · doi:10.2307/3172751 · Cited by 17,852 · see citing works
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362 · see citing works
TRUST AND DISTRUST IN ORGANIZATIONS: Emerging Perspectives, Enduring Questions
Kramer, R. M. (1999) · Annual Review of Psychology · doi:10.1146/annurev.psych.50.1.569 · Cited by 3,265 · see citing works
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232 · see citing works
(1997) · Penn Carey Law Legal Scholarship Repository (University of Pennsylvania) · Cited by 2,736 · see citing works
Algorithm appreciation: People prefer algorithmic to human judgment
Logg, J. M., Minson, J. A., & Moore, D. A. (2019) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2018.12.005 · Cited by 2,061 · see citing works
Resistance to Medical Artificial Intelligence
Longoni, C., Bonezzi, A., & Morewedge, C. K. (2019) · Journal of Consumer Research · doi:10.1093/jcr/ucz013 · Cited by 2,015 · see citing works
Race after technology. Abolitionist tools for the new Jim Code
Merz, S. (2020) · Ethnic and Racial Studies · doi:10.1080/01419870.2020.1715454 · Cited by 1,889 · see citing works
Task-Dependent Algorithm Aversion
Castelo, N., Bos, M. W., & Lehmann, D. R. (2019) · Springer eBooks · doi:10.1007/978-3-031-88067-4_21 · Cited by 1,596 · see citing works
Review of “Race After Technology: Abolitionist Tools for the New Jim Code”
Jipguep-Akhtar, M. C. (2019) · Social Forces · doi:10.1093/sf/soz162 · Cited by 1,482 · see citing works
Lee, M. K. (2018) · Big Data & Society · doi:10.1177/2053951718756684 · Cited by 1,338 · see citing works
Self-citationWorking with Machines: The Impact of Algorithmic and Data-Driven Management on Human Workers
Lee, M. K., Kusbit, D., Metsky, E., & Dabbish, L. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702548 · Cited by 823 · see citing works
Self-citationThe threat of algocracy: reality, resistance and accommodation
Danaher, J. (2016) · Philosophy & Technology · doi:10.1007/s13347-015-0211-1 · Cited by 519 · see citing works
African Americans and their distrust of the health care system: healthcare for diverse populations.
Kennedy, B. R., Mathis, C. C., & Woods, A. K. (2007) · PubMed · Cited by 504 · see citing works
Veale, M., Van Kleek, M., & Binns, R. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174014 · Cited by 490 · cited 1× · see citing works
“healthcare, education, criminal justice systems, organizational management and public assistance [8, 9, 25, 40], AI systems are automating or augmenting decisions that human experts used to make.”
BackgroundThompson, H. S., Valdimarsdottir, H. B., Winkel, G. H., Jandorf, L. H., & Redd, W. H. (2003) · Preventive Medicine · doi:10.1016/j.ypmed.2003.09.041 · Cited by 488 · see citing works
Patient-physician trust: an exploratory study.
Thom, D. H., & Campbell, B. (1997) · PubMed · Cited by 401 · see citing works
Algorithmic governance: Developing a research agenda through the power of collective intelligence
Danaher, J., Hogan, M. J., Noone, C., Kennedy, R., Behan, A., De Paor, A., Felzmann, H., Haklay, M., Khoo, S.-M., Morison, J., et al. (2017) · Big Data & Society · doi:10.1177/2053951717726554 · Cited by 369 · cited 1× · see citing works
“healthcare, education, criminal justice systems, organizational management and public assistance [8, 9, 25, 40], AI systems are automating or augmenting decisions that human experts used to make.”
BackgroundAn Inventory to Measure Cultural Mistrust among Blacks.
Terrell, F., & Terrell, S. L. (1981) · The Western journal of black studies · Cited by 324 · see citing works
Arnett, M. J., Thorpe, R. J., Gaskin, D. J., Bowie, J. V., & LaVeist, T. A. (2016) · Journal of Urban Health · doi:10.1007/s11524-016-0054-9 · Cited by 278 · see citing works
Sigall, H., & Mills, J. R. (1998) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0203_5 · Cited by 224 · see citing works
Highly automated job interviews: Acceptance under the influence of stakes
Langer, M., König, C. J., & Papathanasiou, M. (2019) · International Journal of Selection and Assessment · doi:10.1111/ijsa.12246 · Cited by 203 · see citing works
A Qualitative Exploration of Perceptions of Algorithmic Fairness
Woodruff, A., Fox, S. E., Rousso-Schindler, S., & Warshaw, J. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174230 · Cited by 190 · see citing works
Lee, M. K., & Baykal, S. (2017) · Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing · doi:10.1145/2998181.2998230 · Cited by 182 · see citing works
Self-citationMedicalizing Blackness: Making Racial Difference in the Atlantic World, 1780-1840
Cornelius, A. (2018) · Civil War Book Review · doi:10.31390/cwbr.20.3.24 · Cited by 176 · see citing works
Black and blue: the origins and consequences of medical racism
(2012) · Choice Reviews Online · doi:10.5860/choice.50-2098 · Cited by 175 · see citing works
Validation of the Group-Based Medical Mistrust Scale Among Urban Black Men
Shelton, R. C., Winkel, G. H., Davis, S. N., Roberts, N. A., Valdimarsdottir, H. B., Hall, S. J., & Thompson, H. S. (2010) · Journal of General Internal Medicine · doi:10.1007/s11606-010-1288-y · Cited by 142 · see citing works
Research in the crowdsourcing age: a case study
Hitlin, P. (2016) · Cited by 119 · see citing works
Trust in black America: race, discrimination, and politics
Nunnally, S. C. (2012) · Choice Reviews Online · doi:10.5860/choice.50-0557 · Cited by 96 · see citing works
Impacts on Trust of Healthcare AI
LaRosa, E., & Danks, D. J. (2018) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3278721.3278771 · Cited by 92 · see citing works
Racial differences in attitudes toward innovative medical technology
Groeneveld, P. W., Sonnad, S. S., Lee, A. K., Asch, D. A., & Shea, J. (2006) · Journal of General Internal Medicine · doi:10.1111/j.1525-1497.2006.00453.x · Cited by 72 · see citing works
"They Just Don't Get It": Towards Social Technologies for Coping with Interpersonal Racism
To, A., Sweeney, W., Hammer, J., & Kaufman, G. F. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3392828 · Cited by 63 · see citing works
Algorithms of Oppression: How Search Engines Reinforce Racism
Hankwitz, M. B. (2022) · Leonardo · doi:10.1162/leon_r_02331 · Cited by 59 · see citing works
Medicalizing Blackness: Making Racial Difference in the Atlantic World
Klepp, S. E. (2018) · Journal of American History · doi:10.1093/jahist/jay317 · Cited by 47 · see citing works
Fondacaro, M. R., Frogner, B. K., & Moos, R. H. (2005) · Social Justice Research · doi:10.1007/s11211-005-3393-3 · Cited by 37 · see citing works
Beard, K. S., & Brown, K. M. (2008) · International Journal of Qualitative Studies in Education · doi:10.1080/09518390802297763 · Cited by 28 · see citing works
Do I Look Like a Criminal? Examining how Race Presentation Impacts Human Judgement of Recidivism
Mallari, K., Inkpen, K., Johns, P., Tan, S., Ramesh, D., & Kamar, E. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376257 · Cited by 24 · see citing works
Wheldon, C. W., Kolar, S., Hernandez, N., et al. (2017) · Journal of Health Care for the Poor and Underserved · doi:10.1353/hpu.2017.0011 · Cited by 14 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Trust in AI and Its Role in the Acceptance of AI Technologies
Choung, H., David, P., & Ross, A. (2023) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2022.2050543 · Cited by 963 · see citing works
“Past studies have examined trust in various applications of AI, including algorithmic journalism (Shin, 2020c), healthcare AI (Lee & Rich, 2021), and AI used for hiring and work evaluations (Lee, 2018).”
BackgroundAlgorithmic fairness in artificial intelligence for medicine and healthcare
Chen, R. J., Wang, J. J., Williamson, D. F. K., Chen, T. Y., Lipkova, J., Lu, M. Y., Sahai, S., & Mahmood, F. (2023) · Nature Biomedical Engineering · doi:10.1038/s41551-023-01056-8 · Cited by 750 · see citing works
From promise to practice: towards the realisation of AI-informed mental health care
Koutsouleris, N., Hauser, T. U., Skvortsova, V., & De Choudhury, M. (2022) · The Lancet Digital Health · doi:10.1016/s2589-7500(22)00153-4 · Cited by 251 · see citing works
Zhang, S., Yu, J., Xu, X., Yin, C., Lu, Y., Yao, B., Tory, M., Padilla, L. M., Caterino, J., Zhang, P., et al. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642343 · Cited by 99 · see citing works
”Because AI is 100% right and safe”: User Attitudes and Sources of AI Authority in India
Kapania, S., Siy, O., Clapper, G., SP, A. M., & Sambasivan, N. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517533 · Cited by 90 · see citing works
Will AI Console Me when I Lose my Pet? Understanding Perceptions of AI-Mediated Email Writing
Liu, Y., Mittal, A., Yang, D., & Bruckman, A. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517731 · Cited by 92 · see citing works
Yurrita, M., Draws, T., Balayn, A., Murray-Rust, D., Tintarev, N., & Bozzon, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581161 · Cited by 73 · see citing works
“Thus, we acknowledge that our study is subject to representativeness limitations [61].”
BackgroundSchoeffer, J., Kuehl, N., & Machowski, Y. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533218 · Cited by 67 · see citing works
Langer, M., Hunsicker, T., Feldkamp, T., König, C. J., & Grgić-Hlača, N. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517527 · Cited by 55 · see citing works
Dangol, A., Newman, M., Wolfe, R., Lee, J. H., Kientz, J. A., Yip, J., & Pitt, C. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3661515 · Cited by 42 · see citing works
“Additionally, due to the lack of cultural understanding of AI systems there is concern that potentially harmful cultural representations found online and elsewhere may be further amplified via the use of AI integration into other technologies by novice users (or malicious actors) [51].”
Trusting AI: does uncertainty visualization affect decision-making?
Reyes, J., Batmaz, A. U., & Kersten-Oertel, M. (2025) · Frontiers in Computer Science · doi:10.3389/fcomp.2025.1464348 · Cited by 27 · see citing works
A Human-Centric Perspective on Fairness and Transparency in Algorithmic Decision-Making
Schoeffer, J. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3503811 · Cited by 9 · see citing works
Perceptions of the Fairness Impacts of Multiplicity in Machine Learning
Meyer, A. P., Kim, Y.-S., D'Antoni, L., & Albarghouthi, A. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713524 · Cited by 1 · see citing works
- Authors
- Lee, M. K., & Rich, K.
- Affiliations
- The University of Texas at Austin (US)
- ORCID
- Lee, Min Kyung
- Venue
- Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2021-05-06
- Pages
- 1-14
- OpenAlex
- W3163582547
- Cited by
- 145
- References
- 39
- Citations by year
- 2026: 17 · 2025: 38 · 2024: 35 · 2023: 30 · 2022: 23 · 2021: 2
- FWCI
- 20.38
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Funders
- National Science Foundation (IIS-1939606)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- trust in AI, perceived fairness, artificial intelligence in healthcare, medical mistrust, Black people, skin cancer screening, algorithmic decision-making, cultural mistrust, marginalized communities
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
