Designing AI for Trust and Collaboration in Time-Constrained Medical Decisions: A Sociotechnical Lens
Open accessThe DOI matches its Crossref record; 79 of the 79 references it cites were found in a registry or an index.
Abstract
Major depressive disorder is a debilitating disease affecting 264 million people worldwide. While many antidepressant medications are available, few clinical guidelines support choosing among them. Decision support tools (DSTs) embodying machine learning models may help improve the treatment selection process, but often fail in clinical practice due to poor system integration.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“a recent study found that clinicians simply did not have time to engage with feature-based explanations for individual patient predictions and instead requested evidence that the tool had been clinically validated [37].”
From Explanations Can Reduce Overreliance on AI Systems During Decision-Making (2023) · cites this paper as [37]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The qualitative content analysis process
Elo, S., & Kyngäs, H. (2008) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.2007.04569.x · Cited by 22,617
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · cited 1×
“Our results therefore align with recent discussions of the potential problems of using explainable AI in clinical settings [52].”
ResultDissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · cited 1×
“We have seen several examples in recent years of AI predictions leading to biased or unfair behaviors [20, 45].”
BackgroundRush, A. J., Trivedi, M. H., Wisniewski, S. R., Nierenberg, A. A., Stewart, J. W., Warden, D., Niederehe, G., Thase, M. E., Lavori, P. W., Lebowitz, B. D., et al. (2006) · American Journal of Psychiatry · doi:10.1176/ajp.2006.163.11.1905 · Cited by 5,643
Explanation in Artificial Intelligence: Insights from the Social Sciences
Miller, T. (2017) · arXiv · doi:10.48550/arxiv.1706.07269 · Cited by 5,338
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Charles, C., Gafni, A., & Whelan, T. (1997) · Social Science & Medicine · doi:10.1016/s0277-9536(96)00221-3 · Cited by 4,137 · cited 1×
“While AI research continues to focus on improving information accuracy for clinicians, modern clinical care aspires toward shared decisionmaking with patients and clinicians working together to make decisions [12, 39].”
BackgroundGuidelines for Human-AI Interaction
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300233 · Cited by 2,439 · cited 1×
“HCI research has helped to advance the field of explainable AI, examining how people interact with AI tools and designing tools to help end-users understand the inner workings of machine learning models [3, 8, 10, 24, 46, 73].”
BackgroundThe practical implementation of artificial intelligence technologies in medicine
He, J., Baxter, S. L., Xu, J., Xu, J., Zhou, X., & Zhang, K. (2019) · Nature Medicine · doi:10.1038/s41591-018-0307-0 · Cited by 2,246 · cited 1×
“child welfare system [54], and medicine [22, 59, 68].”
Self-citationQualitative Interviewing and Grounded Theory Analysis
Charmaz, K., & Belgrave, L. L. (2012) · The SAGE Handbook of Interview Research: The Complexity of the Craft · doi:10.4135/9781452218403.n25 · Cited by 1,997
Research works printed but not linked: 3
- [2] Alex Albright. 2019. If You Give a Judge a Risk Score: Evidence from Kentucky Bail Decisions. The John M. Olin Center for Law, Economics, and Business Fellows' Discussion Paper Series (2019), 1-44. -
- [25] Michael C Hughes, Gabriel Hope, Leah Weiner, Thomas H Mccoy, Roy H Perlis, Erik Sudderth, and Finale Doshi-velez. 2018. Semi-Supervised PredictionConstrained Topic Models. In Proceedings of the 21st International Conference on Artificial Intelligence and Statistics (AISTATS) , Vol. 84. -
- [50] Melanie F. Pradier, Javier Zazo, Sonali Parbhoo, Roy H. Perlis, Maurizio Zazzi, and Finale Doshi-Velez. 2021. Preferential Mixture-of-Experts: Interpretable Models that Rely on Human Expertise As Much As Possible. In AMIA 2021 Virtual Informatics Summit . -
Cited by
Items 201–239 of 239
Mastrianni, A. (2023) · Designing Interactive Systems Conference · doi:10.1145/3563703.3593064
Real-world readiness of explainable AI in clinical decision support: a systematic reviewPreprint
Milbeck, S., Pervan, M., Corriveau, B., Hama, L., Kringos, D. S., & Groene, O. (2026) · Research Square · doi:10.21203/rs.3.rs-10052279/v1
Bokenfohr, M., Shachak, A., Maslej, M. M., Sockalingam, S., Hilton, M., Dickie, E. W., Quilty, L. C., Makhlouf, L., Lankarany, M., Glatard, T., et al. (2026) · doi:10.2196/preprints.111993
Privacy and security challenges of explainable artificial intelligence in healthcare
Kandan, M., Mutharasu, M., Ramesh, S., Nagarajan, G., & Tyagi, A. K. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-44111-0.00015-9
Bawah, F. U., Abdulai, J., Baxter, W., & Porat, T. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798827
Otify, A., & Nabney, I. T. (2026) · Frontiers in Digital Health · doi:10.3389/fdgth.2026.1878569
Pakianathan, P. V. S., Islambouli, R., Branco, D., Rosa, G. B., Pinto, R., Schmidt, A., Guerreiro, T., & Smeddinck, J. D. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.18479
Ng, C. Y. J., Kon, W. X., Goh, W. W. B., & Zhong, L. L. D. (2026) · Integrative Medicine Research · doi:10.1016/j.imr.2026.101413
Jung, J.-Y., Yang, W., & Kim, J.-J. (2026) · JMIR Medical Informatics · doi:10.2196/92831
Technoskepticism or Justified Caution? The Future of Human-Centered AI in Mental Health Care
Swinger, N., Moran, L. H., Abdullah, S., Wiese, C. W., Lee, U., Tseng, Y., Sherrill, A. M., & Arriaga, R. I. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3716286
Smart Medical Devices: Engineering for the Future of Healthcare
Younus, A. S., Ismail, A. A., Khairi, A.-H. A., Ahmed, Y. J., & Hussain, A. A. (2025) · Bright Sky Publications eBooks · doi:10.62906/bs.book.336
Alotaibi, A., & Sas, C. (2026) · JMIR Mental Health · doi:10.2196/91884
Towards accessible and ethical AI tools for disability services: a stakeholder-informed study
Ždanovič, N., Bezzina, L., Mollinetti, M. A. F., & Takeda, S. (2026) · International Conference on Advances in Artificial Intelligence and Machine Learning (AAIML) · doi:10.1109/aaiml67890.2026.11498118
Kim, S., Morrison, K., Tan, N., Turner, K., Zhu, H., & Eslami, M. (2026) · Designing Interactive Systems Conference · doi:10.1145/3800645.3813100
INSIGHTS: Demonstration-Based Summaries of Time Series PredictorsPreprint
Eini-Porat, B., Gutman, R., Shalit, U., & Amir, O. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.18849
Studies on Explainable Machine Learning Based on Integer Linear Optimization
金森, 憲. (2022) · Institutional Repositories DataBase (IRDB) · doi:10.14943/doctoral.k15071
Ruixi, L., Wang, J., Pan, X., Hao, X., Kendrick, K., Yang, M., Yao, D., Becker, B., & Jiang, X. (2026) · Psychoradiology · doi:10.1093/psyrad/kkag035
Zakreuskaya, A. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3707612
Daily Encounters with AI: An Inquiry into Users’ Sensemaking
Alizadeh, F. (2025) · Recherche und Kataloge (Universitätsbibliothek Siegen) · doi:10.25819/ubsi/10803
Li, Z. "., Freeman, B., Raman, A., Rajkumar, K. A., Fang, X., Srivastava, R., Feiner, S., & Thakoor, K. A. (2026) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3830398.3830722
Shi, H., Wang, K., Doherty, S., & DiFranzo, D. J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.27849
Toward A Trustworthy And Responsible Artificial Intelligence
Afroogh, S. (2023) · Scholars Archive - University at Albany (University at Albany, State University of New York) · doi:10.54014/87mn-dwne
Advancing Collaborative Health Monitoring Amidst Infrastructural Constraints
Jo, E. (2025) · eScholarship (California Digital Library)
Schäfer, U., Saakyan, W., Norden, M., Kuruc, F., Sörries, P., Dziobek, I., Müller-Birn, C., & Drimalla, H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.22005
Quisenberry, W. L., & Burrell, D. N. (2026) · RAIS Journal for Social Sciences · doi:10.5281/zenodo.19657414
Baumgartner, R. (2026) · VU Research Portal · doi:10.15496/publikation-122074
Vasu, R., Doneva, S. E., & Ineichen, B. V. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.39808
Li, B. (2025) · TSpace (University of Toronto)
Towards a Framework for Trust in Clinical AI: Expanding the Unit of Analysis
Browne, J., Bakker, S., Yu, B., Raijmakers, J., Pluyter, J. R., Ruijs, N., & Deckers, E. (2022) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.7217962
Sin, J. (2024) · TSpace (University of Toronto)
Computing and Communicating Uncertainty for Responsible Human-AI Decision-making
Prabhudesai, S. (2026) · Deep Blue (University of Michigan) · doi:10.7302/dspace/30708
Participation and Transparency in AI System Design and Integration
Delgado, F. A. (2024) · eCommons (Cornell University) · doi:10.7298/92jd-y273
Jesso, S. T., Kelliher, A., Sanghavi, H., Martin, T., & Parker, S. H. (2022) · doi:10.3389/fpsyg.2022.830345.s001
Aprendizado de máquina aplicado à análise macroergonômica: uma revisão sistemática
de Oliveira, A. S., da Fonte, D. P. G., Pessa, S. L. R., & de Oliveira, E. C. (2024) · Anais ... Encontro Nacional de Engenharia de Produção/Anais do Encontro Nacional de Engenharia de Produção · doi:10.14488/enegep2024_tn_wpg_418_2057_47208
Hammad, O. (2023) · CU Scholar (University of Colorado Boulder)
Mansi, G., Kim, J., Dombrowski, L., Riedl, M. O., & Karusala, N. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2611.14989
Expérience Parentale dans les Soins Pédiatriques : Une Analyse Qualitative des Interactions
Makengo, L. N. (2025) · Revue Congolaise des Sciences & Technologies · doi:10.59228/rcst.025.v4.i2.157
Learning Human Proxy Functions to Optimize Machine Learning Systems for Sociotechnical Context
Lage, I. (2023) · Digital Access to Scholarship at Harvard (DASH) (Harvard University)
Artificial Intelligence In Transfusion Medicine: A Contemporary Review
Singh, N. (2026) · Journal of Clinical Transplantation · doi:10.52338/joct.2026.5361
Metadata
- Authors
- Jacobs, M., He, J., F. Pradier, M., Lam, B., Ahn, A. C., McCoy, T. H., Perlis, R. H., Doshi-Velez, F., & Gajos, K. Z.
- Affiliations
- Northwestern University (US)
- Harvard University (US)
- Center for Research on Computation and Society Harvard University, United States
- Beth Israel Deaconess Medical Center (US)
- Massachusetts General Hospital (US)
- SEAS Harvard University, United States
- Venue
- Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2021-05-06
- Pages
- 1-14
- OpenAlex
- W3127032480
- Cited by
- 177
- References
- 79
- Citations by year
- 2026: 30 · 2025: 43 · 2024: 38 · 2023: 33 · 2022: 29 · 2021: 4
- FWCI
- 23.31
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- major depressive disorder, antidepressant selection, clinical decision support systems, machine learning in medicine, trust in AI, socio-technical systems, system integration
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.