"The human body is a black box": supporting clinical decision-making with deep learning
Open accessThe DOI matches its Crossref record; 53 of the 53 references it cites were found in a registry or an index.
Abstract
Machine learning technologies are increasingly developed for use in healthcare. While research communities have focused on creating state-of-the-art models, there has been less focus on real world implementation and the associated challenges to fairness, transparency, and accountability that come from actual, situated use. Serious questions remain underexamined regarding how to ethically build models, interpret and explain model output, recognize and account for biases, and minimize disruptions to professional expertise and work cultures. We address this gap in the literature and provide a detailed case study covering the development, implementation, and evaluation of Sepsis Watch, a machine learning-driven tool that assists hospital clinicians in the early diagnosis and treatment of sepsis. Sepsis is a severe infection that can lead to organ failure or death if not treated in time and is the leading cause of inpatient deaths in US hospitals. We, the team that developed and evaluated the tool, discuss our conceptualization of the tool not as a model deployed in the world but instead as a socio-technical system requiring integration into existing social and professional contexts. Rather than focusing solely on model interpretability to ensure fair and accountable machine learning, we point toward four key values and practices that should be considered when developing machine learning to support clinical decision-making: rigorously define the problem in context, build relationships with stakeholders, respect professional discretion, and create ongoing feedback loops with stakeholders. Our work has significant implications for future research regarding mechanisms of institutional accountability and considerations for responsibly designing machine learning systems. Our work underscores the limits of model interpretability as a solution to ensure transparency, accuracy, and accountability in practice. Instead, our work demonstrates other means and goals to achieve FATML values in design and in practice.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“there has been increasing recent research efforts [8, 15, 22, 41, 58, 65] that highlight the importance of involving stakeholders to understand their practices and needs [15, 41, 65] and socio-environmental factors [8] for the design and evaluation of a system.”
From Understanding the Effect of Counterfactual Explanations on Trust and Reliance on AI for Human-AI Collaborative Clinical Decision Making (2023) · cites this paper as [58]
In the citing paper
“Despite a growing body of literature on both physician perspectives of sepsis CDS implementation [26-28] and sepsis machine learning algorithms, more research is needed to understand clinician views on black box machine learning models that do not explain their predictions in a way that humans can understand [29,30].”
From Integrating a Machine Learning System Into Clinical Workflows: Qualitative Study (2020) · cites this paper as [30]
In the citing paper
“although the field of explainable AI can contribute to trustworthy AI, it has its limits [83].”
Citations
Cites
Rogers, E. M., Singhal, A., & Quinlan, M. M. (2019) · An Integrated Approach to Communication Theory and Research · doi:10.4324/9780203710753-35 · Cited by 37,322
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561
High-performance medicine: the convergence of human and artificial intelligence
Topol, E. J. (2019) · Nature Medicine · doi:10.1038/s41591-018-0300-7 · Cited by 10,450 · cited 2×
“challenges in healthcare [45, 59].”
BackgroundTo Err is Human: Building a Safer Health System
Hendee, W. R. (2001) · Journal of Vascular and Interventional Radiology · doi:10.1016/s1051-0443(01)70072-3 · Cited by 10,103
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · cited 1×
“At the same time, concerns have been raised about the role machine learning technologies might play in exacerbating systemic inequities in healthcare, undercutting patient privacy, increasing surveillance of vulnerable populations, and “dehumanizing” medicine [17, 38, 41, 60].”
Backgroundboyd, D., & Crawford, K. (2012) · Information, Communication & Society · doi:10.1080/1369118x.2012.678878 · Cited by 5,479
Lipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427 · cited 1×
“Although there are emerging methods to improve explainability of RNNs, end users cannot contemplate the entire model and cannot reliably understand the relationships between model inputs and outputs [13, 32].”
MethodExplanation in Artificial Intelligence: Insights from the Social Sciences
Miller, T. (2017) · arXiv · doi:10.48550/arxiv.1706.07269 · Cited by 5,338
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147
Research works printed but not linked: 14
- Futoma , J. , Hariharan , S. and Heller , K . 2017. Learning to Detect Sepsis with a Multitask Gaussian Process RNN Classifier . International Conference on Machine Learning. (Jun. 2017 ), 1--9. Futoma, J., Hariharan, S. and Heller, K. 2017. Learning to Detect Sepsis with a Multitask Gaussian Process RNN Classifier. International Conference on Machine Learning. (Jun. 2017), 1--9.
- Gawande , A. 2010. The Velluvial Matrix. The New Yorker. (Jan . 2010 ). Gawande, A. 2010. The Velluvial Matrix. The New Yorker. (Jan. 2010).
- Green , B. and Chen , Y . 2019 . Disparate Interactions. ( New York, New York, USA , 2019 ), 90--99. Green, B. and Chen, Y. 2019. Disparate Interactions. (New York, New York, USA, 2019), 90--99.
- Kluttz , D.N. , Kohli , N. and Mulligan , D.K . 2020 . Shaping Our Tools: Contestability as a Means to Promote Responsible Algorithmic Decision Making in the Professions, in After the Dihital Tornado: Networks, Algorithms, Humanity . Combridge University Press . (2020). Kluttz, D.N., Kohli, N. and Mulligan, D.K. 2020. Shaping Our Tools: Contestability as a Means to Promote Responsible Algorithmic Decision Making in the Professions, in After the Dihital Tornado: Networks, Algorithms, Humanity. Combridge University Press. (2020).
- Lin , A.L. , Sendak , M.P. , Bedoya , A.D. , Clement , M.E. , Futoma , J. , Nichols , M. , Gao , M. , Heller , K. and O Brien , C. 2018. What Is Sepsis: Investigating the Heterogeneity of Patient Populations Captured by Different Sepsis Definitions. American Journal of Respiratory and Critical Care Medicine. (May 2018 ). Lin, A.L., Sendak, M.P., Bedoya, A.D., Clement, M.E., Futoma, J., Nichols, M., Gao, M., Heller, K. and OBrien, C. 2018. What Is Sepsis: Investigating the Heterogeneity of Patient Populations Captured by Different Sepsis Definitions. American Journal of Respiratory and Critical Care Medicine. (May 2018).
- Lipton , Z.C. 2017 . The Doctor Just Won't Accept That . Interpretable ML Symposium NeurIPS 2017 . (Nov. 2017). Lipton, Z.C. 2017. The Doctor Just Won't Accept That. Interpretable ML Symposium NeurIPS 2017. (Nov. 2017).
- Lipton , Z.C. 2016 . The Mythos of Model Interpretability. ICML Workshop on Human Interpretability. (2016). Lipton, Z.C. 2016. The Mythos of Model Interpretability. ICML Workshop on Human Interpretability. (2016).
- Nestor , B. , McDermott , M.B. , Chauhau , G. , Naumann , T. , Hughes , M.C. , Goldenberg , A. and Ghassemi , M . 2018. Rethinking Clinical Prediction: Why Machine learning Must Consider Year of Care and Feature Aggregation . Machine Learning for Health MLH Workshop at NeurIPS. (Nov. 2018 ). Nestor, B., McDermott, M.B., Chauhau, G., Naumann, T., Hughes, M.C., Goldenberg, A. and Ghassemi, M. 2018. Rethinking Clinical Prediction: Why Machine learning Must Consider Year of Care and Feature Aggregation. Machine Learning for Health MLH Workshop at NeurIPS. (Nov. 2018).
- Rhee , C. 2018 . Compliance With the National SEP-1 Quality Measure and Association With Sepsis Outcomes . Journal of the American Medical Association. 46 , 10 (Oct. 2018), 1585--1591. Rhee, C. et al. 2018. Compliance With the National SEP-1 Quality Measure and Association With Sepsis Outcomes. Journal of the American Medical Association. 46, 10 (Oct. 2018), 1585--1591.
- Saria , S. and Subbaswamy , A . 2019 . Tutorial: Safe and Reliable Machine Learning. ACM Conference on Fairness, Accountability, and Transparency. (Apr. 2019 ), 1--4. Saria, S. and Subbaswamy, A. 2019. Tutorial: Safe and Reliable Machine Learning. ACM Conference on Fairness, Accountability, and Transparency. (Apr. 2019), 1--4.
- Select Committee on Artificial Intelligence 2019. The National Artificial Intelligence Research and Development Strategic Plan: 2019 Update . National Science Technology Council . ( Jun. 2019 ). Select Committee on Artificial Intelligence 2019. The National Artificial Intelligence Research and Development Strategic Plan: 2019 Update. National Science Technology Council. (Jun. 2019).
- Sendak , M.P. et al . 2019 . Sepsis Watch : A Real-World Integration of Deep Learning into Routine Clinical Care. JMIR Preprints . (Jun. 2019). Sendak, M.P. et al. 2019. Sepsis Watch: A Real-World Integration of Deep Learning into Routine Clinical Care. JMIR Preprints. (Jun. 2019).
- Sendak , M.P. , Gao , M. , Nichols , M. , Lin , A. and Balu , S . 2019 . Machine Learning in Health Care: A Critical Appraisal of Challenges and Opportunities. eGEMs Generating Evidence and Methods to Improve Patient Outcomes. 7, 1 (Jan. 2019), 1--4. Sendak, M.P., Gao, M., Nichols, M., Lin, A. and Balu, S. 2019. Machine Learning in Health Care: A Critical Appraisal of Challenges and Opportunities. eGEMs Generating Evidence and Methods to Improve Patient Outcomes. 7, 1 (Jan. 2019), 1--4.
- Topol , E.J. Deep Medicine (p. 18). Basic Books . March 2019 . Topol, E.J. Deep Medicine (p. 18). Basic Books. March 2019.
Cited by
Items 101–200 of 321
"It depends": Configuring AI to Improve Clinical Usefulness Across Contexts
Zając, H. D., Ribeiro, J., Ingala, S., Gentile, S., Wanjohi, R., Gitau, S. N., Carlsen, J. F., Nielsen, M. B., & Andersen, T. O. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3660707 · Cited by 25
Ghosh, S., Abushukair, H. M., Ganesan, A., Pan, C., Naqash, A. R., & Lü, K. (2024) · PLoS ONE · doi:10.1371/journal.pone.0311510 · Cited by 24
Fogliato, R., Chouldechova, A., & Lipton, Z. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479572 · Cited by 23
Should AI-Enabled Medical Devices be Explainable?
Matulionytė, R., Nolan, P., Magrabi, F., & Beheshti, A. (2022) · International Journal of Law and Information Technology · doi:10.1093/ijlit/eaac015 · Cited by 23
Pruinelli, L., Balakrishnan, K., Ma, S., Li, Z., Wall, A. E., Lai, J. C., Schold, J. D., Pruett, T. L., & Simon, G. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-02890-3 · Cited by 23
Zhou, Z., Li, Z., Zhang, Y., & Sun, L. (2022) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2022.2093773 · Cited by 22
Waldock, W. J., Guni, A., Darzi, A. W., & Ashrafian, H. (2026) · PLOS Digital Health · doi:10.1371/journal.pdig.0001310 · Cited by 23
McCradden, M. D., Thai, K., Assadi, A., Tonekaboni, S., Stedman, I., Joshi, S. D., Zhang, M., Chevalier, F., & Goldenberg, A. (2025) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2024-112919 · Cited by 21
Thieme, A., Rajamohan, A. M., Cooper, B., Groombridge, H., Simister, R. J., Wong, B., Woznitza, N., Pinnock, M. A., Wetscherek, M. T., Morrison, C. P., et al. (2025) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3716500 · Cited by 20
Trust in Machine Learning Driven Clinical Decision Support Tools Among Otolaryngologists
Chen, H. Y., Ma, X., Rives, H., Serpedin, A., Yao, P., & Rameau, A. (2024) · The Laryngoscope · doi:10.1002/lary.31260 · Cited by 20
Lee, M. H., Siewiorek, D. P., Smailagic, A., Bernardino, A., & Bermúdez i Badia, S. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511112 · Cited by 19
“In addition, researchers discuss the necessity of evaluating a system in socio-technical contexts [6, 21], and creating ongoing feedback loops with stakeholders [54].”
BackgroundKaleidoscope: Semantically-grounded, context-specific ML model evaluation
Suresh, H., Shanmugam, D., Chen, T., Bryan, A. G., D’Amour, A., Guttag, J. V., & Satyanarayan, A. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581482 · Cited by 19
Verma, A. A., Trbovich, P., Mamdani, M., & Shojania, K. G. (2023) · BMJ Quality & Safety · doi:10.1136/bmjqs-2022-015713 · Cited by 19
Benjamin, J. J., Kinkeldey, C., Müller-Birn, C., Korjakow, T., & Herbst, E.-M. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3492858 · Cited by 19
He, G., Hemmer, P., Vössing, M., Schemmer, M., & Gadiraju, U. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.10909 · Cited by 20
A scoping review of silent trials for medical artificial intelligence
Tikhomirov, L., Semmler, C., Prizant, N., Bhasin, S., Kenyon, G., van der Vegt, A., Erdman, L., Kurian, N. C., Thompson, H., Palmer, L. J., et al. (2026) · Nature Health · doi:10.1038/s44360-025-00048-z · Cited by 17
Self-citationDissecting users' needs for search result explanations
Juneja, P., Zhang, W., Smith, A., Lamba, H., Tetreault, J., & Jaimes, A. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642059 · Cited by 17
U-Fair: Uncertainty-based Multimodal Multitask Learning for Fairer Depression DetectionPreprint
Cheong, J., Bangar, A., Kalkan, S., & Güneş, H. (2024) · arXiv (Cornell University) · doi:10.17863/cam.114195 · Cited by 17
Early Warning Systems for Critical Illness Outside the Intensive Care Unit
Henry, K. E., & Giannini, H. M. (2024) · Critical Care Clinics · doi:10.1016/j.ccc.2024.03.007 · Cited by 16
Sandhu, S., Sendak, M., Ratliff, W., Knechtle, W. S., Fulkerson, W. J., & Balu, S. (2023) · Patterns · doi:10.1016/j.patter.2023.100710 · Cited by 16
Self-citationTrust in Clinical AI: Expanding the Unit of Analysis
Browne, J., Bakker, S., Yu, B., Lloyd, P. A., & Allouch, S. B. (2022) · Frontiers in artificial intelligence and applications · doi:10.3233/faia220192 · Cited by 15
Chen, F., Chen, H., Yu, T., Wang, R., Wang, Y., Zhang, X., Li, J., Liu, K., Hai, D., Bao, X., et al. (2026) · MedComm · doi:10.1002/mco2.70597 · Cited by 14
Virtual Reality (VR) and Augmented Reality (AR) in Medicine: A Review of Clinical Applications
Nwokedi, C. N., Soyege, O. S., Balogun, O. D., Mustapha, A. Y., Tomoh, B. O., Mbata, A. O., & Iguma, D. R. (2024) · International Journal of Scientific Research in Science Engineering and Technology · doi:10.32628/ijserset242435 · Cited by 14
Current knowledge and availability of machine learning across the spectrum of trauma science
Gauss, T., Perkins, Z., & Tjardes, T. (2023) · Current Opinion in Critical Care · doi:10.1097/mcc.0000000000001104 · Cited by 13
Interdisciplinary expertise to advance human-centered explainable AI
Bennett, C. R., Cole-Lewis, H. J., Farquhar, S. A., Hammel, N., Babenko, B., Lang, O., Fleck, M. S., Traynis, I., Lau, C. T., Liu, Y., et al. (2026) · Social Science & Medicine · doi:10.1016/j.socscimed.2026.119036 · Cited by 13
Sassi, Z., Eickmann, S., Roller, R., Osmanodja, B., Burchardt, A., Samhammer, D., Dabrock, P., Möller, S., Budde, K., & Herrmann, A. (2024) · BMJ Open · doi:10.1136/bmjopen-2023-081318 · Cited by 13
AI Fairness in PracticePreprint
Leslie, D., Rincón, C., Briggs, M., Perini, A., Jayadeva, S., Borda, A., Bennett, S., Burr, C., Aitken, M., Katell, M. A., et al. (2023) · arXiv (Cornell University) · doi:10.5281/zenodo.10680526 · Cited by 16
Sun, L., Li, Z., Zhang, Y., Liu, Y., Lou, S., & Zhou, Z. (2021) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3411763.3451819 · Cited by 12
Wilson, B., Natali, C., Roach, M., Scott, D., Rahat, A., Rawlinson, D. R., & Cabitza, F. (2025) · Computer Supported Cooperative Work (CSCW) · doi:10.1007/s10606-025-09514-4 · Cited by 12
Schaffernak, I., Cecil, J., Kleine, A., & Lermer, E. (2025) · BMC Health Services Research · doi:10.1186/s12913-025-13620-w · Cited by 12
New Innovation Models in Medical AIPreprint
Price, W. N., II, Sachs, R. E., & Eisenberg, R. S. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3783879 · Cited by 12
A Conditional Companion: Lived Experiences of People with Mental Health Disorders Using LLMs
Purohit, A. K., & Heuer, H. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791763 · Cited by 12
Took a Pic and Got Declined, Vexed and Perplexed: Facial Recognition in Algorithmic Management
Watkins, E. A. (2020) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3406865.3418383 · Cited by 11
The Social and Ethical Component in AI Systems Design and Management
Mantelero, A. (2022) · Information technology and law series/Information technology & law series · doi:10.1007/978-94-6265-531-7_3 · Cited by 11
"A cold, technical decision-maker": Can AI provide explainability, negotiability, and humanity?Preprint
Woodruff, A., Anderson, Y. A., Armstrong, K. J., Gkiza, M., Jennings, J., Moessner, C., Viégas, F. B., WATTENBERG, M. P., Webb, A. L., Wrede, F., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.00874 · Cited by 11
Tempo: Helping Data Scientists and Domain Experts Collaboratively Specify Predictive Modeling Tasks
Sivaraman, V., Vaishampayan, A., Li, X., Buck, B. R., Ma, Z., Boyce, R. D., & Perer, A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713664 · Cited by 11
La décision de l'algorithme : étude de droit privé sur les relations entre l'humain et la machine
Huttner, L. (2022) · theses.fr (ABES) · doi:10.70675/dd82241az1485z46f1zb4d2z9c73eceaef7b · Cited by 11
Thieme, A., Rajamohan, A., Cooper, B., Groombridge, H., Simister, R., Wong, B., Woznitza, N., Pinnock, M. A., Wetscherek, M. T., Morrison, C., et al. (2024) · arXiv · doi:10.48550/arxiv.2405.05299 · Cited by 10
Sun, L., Li, Z., Zhou, Z., Lou, S., Li, W., & Zhang, Y. (2023) · Artificial intelligence for engineering design analysis and manufacturing · doi:10.1017/s0890060422000270 · Cited by 10
Usmani, U. A., Usmani, A. Y., & Usmani, M. U. (2023) · International Conference on Computing, Communication, and Intelligent Systems (ICCCIS) · doi:10.1109/icccis60361.2023.10425285 · Cited by 10
Lin, A. L., Parrish, A. B., Cary, M. P., Silcox, C., Balu, S., Jelovsek, J. E., O’Brien, C. L., Pencina, M. J., Poon, E. G., & Economou-Zavlanos, N. (2025) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-103123-094601 · Cited by 10
Self-citationAnalysis of Ethical Development for Public Policies in the Acquisition of AI-Based Systems
Tabares-Soto, R., Bernal-Salcedo, J., García-Arias, Z. N., Ortega-Bolaños, R., Hermosilla, M. P., Arteaga-Arteaga, H. B., & Ruz, G. A. (2022) · Advances in human and social aspects of technology book series · doi:10.4018/978-1-6684-5892-1.ch010 · Cited by 10
Escudero-Arnanz, Ó., Martínez-Agüero, S., Martín-Palomeque, P., Marques, A. G., Mora-Jiménez, I., Álvarez-Rodríguez, J., & Soguero-Ruíz, C. (2025) · Health Information Science and Systems · doi:10.1007/s13755-025-00351-9 · Cited by 10
Lau, R. P., Kim, T. H., & Rao, J. (2021) · Frontiers in Medicine · doi:10.3389/fmed.2021.689954 · Cited by 10
Ihejirika, C. J. (2025) · International Journal of Research Publication and Reviews · doi:10.55248/gengpi.6.0225.0905 · Cited by 10
Navigating Healthcare Insights: A Bird’s Eye View of Explainability with Knowledge Graphs
Garg, S., Parikh, S. B., & Garg, S. (2023) · IEEE International Conference on Artificial Intelligence and Knowledge Engineering (AIKE) · doi:10.1109/aike59827.2023.00016 · Cited by 9
'Poker with Play Money': Exploring Psychotherapist Training with Virtual Patients
Baseman, C. M., Hasan, M., Swinger, N., Rauch, S. A. M., Hoque, M. E., & Arriaga, R. I. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757450 · Cited by 9
Internet of Things and Machine Learning for Smart Healthcare
da Costa, C. A., Zeiser, F. A., Righi, R. D. R., Antunes, R. S., Alegretti, A. P., Bertoni, A. P. S., Ramos, G. D. O., de Mello, B. H., Vanin, F. N. D. S., Bertoletti, O. A., et al. (2024) · Studies in computational intelligence · doi:10.1007/978-981-97-5624-7_3 · Cited by 10
Human performance evaluation of a pediatric artificial intelligence sepsis model
Kandaswamy, S., Muthu, N., Braykov, N. P., Carter, R., Blanco, R., Bui, T. T., Orenstein, E. W., & Mai, M. V. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf106 · Cited by 9
Temporal Clustering of Critical Illness Events on Medical Wards
Doshi, S., Shin, S., Lapointe‐Shaw, L., Fowler, R., Fralick, M., Kwan, J. L., Shojania, K. G., Tang, T., Razak, F. A., & Verma, A. A. (2023) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2023.2629 · Cited by 9
Leadership and Ethical Decision-Making in Healthcare Management
Hasan, F., Thakkar, A., Sahoo, A. K., Goyal, W., Seth, K., Jayakumar, S. S., & Sikchi, S. R. (2024) · Health Leadership and Quality of Life · doi:10.56294/hl2024.379 · Cited by 9
Morley, J. (2023) · Oxford University Research Archive (ORA) (University of Oxford) · doi:10.5287/ora-jxprn9ry5 · Cited by 9
Sivaraman, V., Mason, E. P., Li, M. E., Tong, J., King, A. J., Kahn, J. M., & Perer, A. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790953 · Cited by 9
Yao, B., Zhao, M., Sun, Y., Cao, W., Yin, C., Intille, S. S., Xu, X., Zhang, P., Yang, J. G., & Wang, D. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.03732 · Cited by 9
Lee, M. H., Choo, S., & Thilarajah, S. D. O. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.16424 · Cited by 9
Machine learning, healthcare resource allocation, and patient consent
Webb, J. (2024) · The New Bioethics · doi:10.1080/20502877.2024.2416858 · Cited by 9
Moral Values in Medical AI: A Scoping ReviewPreprint
Victor, G., Barbu, A., & Bélisle‐Pipon, J. (2024) · Research Square · doi:10.21203/rs.3.rs-4391239/v1 · Cited by 8
van der Vegt, A. H., Campbell, V. K., Webb, R. T., Venkatesh, B., Lane, P. J., Wilks, K. M., McPhail, S., Rice, M., Isaacs, T., Abdel‐Hafez, A., et al. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf192 · Cited by 8
Islam, S., Bentahar, J., Cohen, R. A., & Rjoub, G. (2025) · Information Sciences · doi:10.1016/j.ins.2025.122114 · Cited by 8
Algorithms in the Stacks: Investigating automated, for-profit diversity audits in public libraries
Walsh, M., Rey, C. F., Ge, C., Nowak, T., & Tomkins, S. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732140 · Cited by 7
Designing Equitable Health Care Outreach Programs From Machine Learning Patient Risk Scores
Hane, C. A., & Wasserman, M. (2022) · Medical Care Research and Review · doi:10.1177/10775587221098831 · Cited by 7
AI and the iterable epistopics of risk
Crabtree, A., McGarry, G., & Urquhart, L. (2024) · AI & Society · doi:10.1007/s00146-024-02021-y · Cited by 7
Bias, Accuracy, and Trust: Gender-Diverse Perspectives on Large Language ModelsPreprint
Gaba, A., Wall, E., Babu, T. R., Brun, Y., Hall, K. W., & Bearfield, C. X. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.21898 · Cited by 7
Pallamala, G., Jenkins, J., Sherin, P. K., Nyathi, A., & George, J. (2026) · Oxford Open Digital Health · doi:10.1093/oodh/oqag004 · Cited by 6
Noselli, M., van der Vegt, A. H., Cordeil, M., Campbell, V. K., Wang, A. P., Shetty, A., & Scott, I. A. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3572494 · Cited by 6
What should AI explain to you? Visions of older adults regarding explainable AI
van Leersum, C. M. (2026) · The Gerontologist · doi:10.1093/geront/gnaf308 · Cited by 6
Some Characteristics and Arguments in Favor of a Science of Machine Behavior Analysis
Lanovaz, M. J. (2022) · Perspectives on Behavior Science · doi:10.1007/s40614-022-00332-3 · Cited by 6
The Objective Function: Science and Society in the Age of Machine IntelligencePreprint
Moss, E. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2209.10418 · Cited by 6
Adam, A. A., & Alfred, R. (2024) · Lecture notes in electrical engineering · doi:10.1007/978-981-97-2977-7_27 · Cited by 6
Fang, S., Hong, S., Li, Q., Li, P., Coats, T., Zou, B., & Kong, G. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105680 · Cited by 5
Repairing Innovation: A Study of Integrating AI in Clinical Care
Elish, M. C., & Watkins, E. A. (2020) · doi:10.69985/vcvu5761 · Cited by 5
Self-citationRaszke, P., Giebel, G. D., Wasem, J., Adamzik, M., Nowak, H., Palmowski, L., Heinz, P., Timmesfeld, N., Tokic, M. C., Brunkhorst, F. M., et al. (2026) · Journal of Medical Internet Research · doi:10.2196/76772 · Cited by 6
Muñoz, J., Fernández-Araujo, N. J., Ruíz-Cacho, R., & Muñoz-Visedo, J. (2026) · Journal of Critical Care · doi:10.1016/j.jcrc.2026.155600 · Cited by 5
Testing the Black Box: Institutional Investors, Risk Disclosure, and Ethical AI
Sanders, T. (2020) · Philosophy & Technology · doi:10.1007/s13347-020-00409-4 · Cited by 5
Sivaraman, V., Kwak, Y., Kuza, C. C., Yang, Q., Adamson, K. M., Suda, K. J., Tang, L., Gellad, W. F., & Perer, A. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711072 · Cited by 5
Pellegrino, A., & Stasi, A. (2024) · Journal of Infrastructure Policy and Development · doi:10.24294/jipd7560 · Cited by 5
RelAItionship Building: Analyzing Recruitment Strategies for Participatory AI
Kim, E. S., Balloli, V., Karimian, B., Bondi, E., & Fish, B. (2025) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1609/aies.v8i2.36643 · Cited by 5
Leveraging imperfection with MEDLEY: a multi-model approach harnessing bias in medical AI
Abtahi, F., Astaraki, M., & Seoane, F. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1701665 · Cited by 5
Conceptualising Contestability: Perspectives on Contesting Algorithmic Decisions
Lyons, H., Velloso, E., & Miller, T. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449180 · Cited by 4
A Survey on Human-Centered Evaluation of Explainable AI Methods in Clinical Decision Support Systems
Gambetti, A., Han, Q., Shen, H., & Soares, C. (2025) · arXiv · doi:10.48550/arxiv.2502.09849 · Cited by 4
Ali, M. B. G., & Smiti, A. (2026) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2026.102730 · Cited by 4
Counterfactual Explanations via Locally-guided Sequential Algorithmic Recourse
Small, E. A., Clark, J. N., McWilliams, C., Ambler, M. T., Sokol, K., Chan, J., Salim, F. D., & Santos‐Rodríguez, R. (2026) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3770065 · Cited by 5
Yamin, M., Basahel, S. B., Bajaba, S. M., Abusurrah, M. M., & Lydia, E. L. (2023) · Computer Systems Science and Engineering · doi:10.32604/csse.2023.036455 · Cited by 4
Explainable Deep Learning Models for Healthcare Decision Support
(2023) · International Journal of Advances in Computer Science and Technology · doi:10.30534/ijacst/2023/0112102023 · Cited by 4
Diversity as Ethical Infrastructure: Reimagining AI Governance for Justice and Accountability
Iseko, A. (2025) · International Journal of Science Technology and Society · doi:10.11648/j.ijsts.20251305.13 · Cited by 4
Li, X., Gao, X., Wang, C., Wang, Q., Li, B., & Wan, K. (2024) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2024.3390426 · Cited by 4
Davidson, A., Ray, J. M., Patel, A. K., Levites, Y. S., Rashidi, P., & Bihorac, A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.08814 · Cited by 4
Trust Formation in Healthcare AI: An Exploration of Older Adults’ Perspectives
Çelik, Ö., Kulla, M., & Stypińska, J. (2025) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1609/aies.v8i1.36566 · Cited by 4
Medical AI, Categories of Value Conflict, and Conflict Bypasses
Victor, G., & Bélisle‐Pipon, J. (2024) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1609/aies.v7i1.31740 · Cited by 4
Shapiro, R. B., Cashore, L. Z., & Yuan, W. (2023) · Journal of Computer Science and Research (JoCoSiR) · doi:10.65126/jocosir.v1i3.18 · Cited by 4
Applications of Artificial Intelligence in Cancer Diagnosis and Treatment
Xie, Y., Wang, Z., Zeng, Z., Xin, T., Yuan, S., Qin, F., & Chen, Z. (2026) · Cancer Medicine · doi:10.1002/cam4.72143 · Cited by 4
Boyce, L., & Prybutok, V. R. (2026) · Healthcare · doi:10.3390/healthcare14060806 · Cited by 3
Advances in Closed-Loop Artificial Intelligence for Healthcare
Das, D., Adams, S. D., Corva, D. M., Bucknall, T. K., & Kouzani, A. Z. (2026) · Electronics · doi:10.3390/electronics15071396 · Cited by 3
How patients distinguish between clinical and administrative predictive models in health care
Nong, P., Adler-Milstein, J., & Platt, J. (2024) · The American Journal of Managed Care · doi:10.37765/ajmc.2024.89484 · Cited by 3
The fundamentals of AI ethics in medical imaging
Amann, J., Bürger, V. K., Livne, M., Bui, C. K. T., & Madai, V. I. (2025) · Elsevier eBooks · doi:10.1016/b978-0-44-323761-4.00010-9 · Cited by 3
Costa, J. B., Silva‐Correia, J., Reis, R. L., & Oliveira, J. M. (2021) · Journal of 3D Printing in Medicine · doi:10.2217/3dp-2021-0007 · Cited by 3
Neurosurgery, Explainable AI, and Legal Liability
Matulionytė, R., Molina, E. S., & Ieva, A. D. (2024) · Advances in experimental medicine and biology · doi:10.1007/978-3-031-64892-2_34 · Cited by 3
Looking for clinician involvement under the wrong lamp post: The need for collaboration measures
Sendak, M., Gao, M. C., Ratliff, W., Nichols, M., Bedoya, A., O’Brien, C. L., & Balu, S. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab129 · Cited by 3
Self-citationHow to Approach and Interpret Studies on AI in Gastroenterology
Carleton, N. M., & Thakkar, S. J. (2020) · Gastroenterology · doi:10.1053/j.gastro.2020.04.001 · Cited by 3
Brandl, L., Jansen-Kosterink, S. M., Brodbeck, J., Jacinto, S., Mooser, B., & Heylen, D. (2024) · JMIR Formative Research · doi:10.2196/63262 · Cited by 3
Metadata
- Authors
- Sendak, M., Elish, M. C., Gao, M., Futoma, J., Ratliff, W., Nichols, M., Bedoya, A., Balu, S., & O'Brien, C.
- Affiliations
- Duke University (US)
- Data & Society Research Institute (US)
- Harvard University (US)
- Duke School of Medicine
- ORCID
- Venue
- Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency
- Publisher
- ACM
- Published
- 2020-01-27
- Pages
- 99-109
- OpenAlex
- W3002972902
- Cited by
- 196
- References
- 49
- Citations by year
- 2026: 37 · 2025: 22 · 2024: 33 · 2023: 26 · 2022: 30 · 2021: 32 · 2020: 16
- FWCI
- 9.92
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- clinical decision support, socio-technical systems, model interpretability, stakeholder engagement, professional discretion, institutional accountability, responsible AI design, healthcare implementation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.