Designing AI for Trust and Collaboration in Time-Constrained Medical Decisions: A Sociotechnical Lens
Open accessThe DOI matches its Crossref record; 78 of the 79 references it cites were found in a registry or an index.
Abstract
Major depressive disorder is a debilitating disease affecting 264 million people worldwide. While many antidepressant medications are available, few clinical guidelines support choosing among them. Decision support tools (DSTs) embodying machine learning models may help improve the treatment selection process, but often fail in clinical practice due to poor system integration.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“a recent study found that clinicians simply did not have time to engage with feature-based explanations for individual patient predictions and instead requested evidence that the tool had been clinically validated [37].”
From Explanations Can Reduce Overreliance on AI Systems During Decision-Making (2023) · cites this paper as [37]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The qualitative content analysis process
Elo, S., & Kyngäs, H. (2008) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.2007.04569.x · Cited by 22,617
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · cited 1×
“Our results therefore align with recent discussions of the potential problems of using explainable AI in clinical settings [52].”
ResultDissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · cited 1×
“We have seen several examples in recent years of AI predictions leading to biased or unfair behaviors [20, 45].”
BackgroundRush, A. J., Trivedi, M. H., Wisniewski, S. R., Nierenberg, A. A., Stewart, J. W., Warden, D., Niederehe, G., Thase, M. E., Lavori, P. W., Lebowitz, B. D., et al. (2006) · American Journal of Psychiatry · doi:10.1176/ajp.2006.163.11.1905 · Cited by 5,643
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Charles, C., Gafni, A., & Whelan, T. (1997) · Social Science & Medicine · doi:10.1016/s0277-9536(96)00221-3 · Cited by 4,137 · cited 1×
“While AI research continues to focus on improving information accuracy for clinicians, modern clinical care aspires toward shared decisionmaking with patients and clinicians working together to make decisions [12, 39].”
BackgroundGuidelines for Human-AI Interaction
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300233 · Cited by 2,439 · cited 1×
“HCI research has helped to advance the field of explainable AI, examining how people interact with AI tools and designing tools to help end-users understand the inner workings of machine learning models [3, 8, 10, 24, 46, 73].”
BackgroundThe practical implementation of artificial intelligence technologies in medicine
He, J., Baxter, S. L., Xu, J., Xu, J., Zhou, X., & Zhang, K. (2019) · Nature Medicine · doi:10.1038/s41591-018-0307-0 · Cited by 2,246 · cited 1×
“child welfare system [54], and medicine [22, 59, 68].”
Self-citationQualitative Interviewing and Grounded Theory Analysis
Charmaz, K., & Belgrave, L. L. (2012) · The SAGE Handbook of Interview Research: The Complexity of the Craft · doi:10.4135/9781452218403.n25 · Cited by 1,997
Hibbard, J. H., & Greene, J. (2013) · Health Affairs · doi:10.1377/hlthaff.2012.1061 · Cited by 1,787 · cited 1×
“Studies have connected increased patient activation to improved healthcare experiences and health outcomes [23].”
BackgroundFairness and Abstraction in Sociotechnical Systems
Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287598 · Cited by 1,631
An integrative model of shared decision making in medical encounters
Makoul, G., & Clayman, M. L. (2006) · Patient Education and Counseling · doi:10.1016/j.pec.2005.06.010 · Cited by 1,621
Do no harm: a roadmap for responsible machine learning for health care
Wiens, J., Saria, S., Sendak, M., Ghassemi, M., Liu, V. X., Doshi-Velez, F., Jung, K., Heller, K., Kale, D., Saeed, M., et al. (2019) · Nature Medicine · doi:10.1038/s41591-019-0548-6 · Cited by 1,266
Self-citationKennedy, S. H., Lam, R. W., McIntyre, R. S., Tourjman, S. V., Bhat, V., Blier, P., Hasnain, M., Jollant, F., Levitt, A. J., MacQueen, G. M., et al. (2016) · The Canadian Journal of Psychiatry · doi:10.1177/0706743716659417 · Cited by 1,181
Irving, G., Neves, A. L., Dambha-Miller, H., Oishi, A., Tagashira, H., Verho, A., & Holden, J. (2017) · BMJ Open · doi:10.1136/bmjopen-2017-017902 · Cited by 1,083 · cited 1×
“However, primary care appointments last an average of only 20 minutes in the United States, and internationally primary care appointments can be as short as a few minutes [28].”
BackgroundMachine learning in mental health: a scoping review of methods and applications
Shatte, A. B. R., Hutchinson, D. M., & Teague, S. J. (2019) · Psychological Medicine · doi:10.1017/s0033291719000151 · Cited by 1,047 · cited 1×
“studies focused on creating ML models to support MDD treatment decisions [25, 49, 57].”
BackgroundDesigning Theory-Driven User-Centric Explainable AI
Wang, D., Yang, Q., Abdul, A., & Lim, B. Y. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300831 · Cited by 906
Rush, A. J. (2006) · American Journal of Psychiatry · doi:10.1176/appi.ajp.163.11.1905 · Cited by 870
Patient care information systems and health care work: a sociotechnical approach
Berg, M. (1999) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(99)00011-8 · Cited by 786 · cited 1×
“Therefore, motivated by Berg’s sociotechnical approach, which highlights the importance of empirical research of healthcare practices in which the technology will be used [6], we consider the social, technical, and organizational issues that must be considered in the design of DSTs for MDD treatment selection.”
Medication-related clinical decision support in computerized provider order entry systems: a review
Kuperman, G. J., Bobb, A., Payne, T. H., Avery, A. J., Gandhi, T. K., Burns, G., Classen, D. C., & Bates, D. W. (2007) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2170 · Cited by 763 · cited 1×
“For example, studies in medical informatics found that when DST alerts arise at inopportune times (from clinicians’ vantage), the alerts become ignored or overridden [1, 35, 43].”
A roadmap for national action on clinical decision support
Osheroff, J. A., Teich, J. M., Middleton, B., Steen, E. B., Wright, A., & Detmer, D. E. (2007) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2334 · Cited by 707
A Survey of Methods for Explaining Black Box Models
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2019) · ACM Computing Surveys · doi:10.1145/3236009 · Cited by 629 · cited 1×
“(3) Stability and dropout feature importance (figure 1B): Awidely used approach for explaining ML predictions that shows relevant features and their contributions to the prediction [21].”
BackgroundBeede, E., Baylor, E., Hersch, F., Iurchenko, A., Wilcox, L., Ruamviboonsuk, P., & Vardoulakis, L. M. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376718 · Cited by 614 · cited 2×
“also recently showed how environmental factors in a clinical setting influenced the usability of a deep learning system [5].”
BackgroundFava, M., & Kendler, K. S. (2000) · Neuron · doi:10.1016/s0896-6273(00)00112-4 · Cited by 603
Grand challenges in clinical decision support
Sittig, D. F., Wright, A., Osheroff, J. A., Middleton, B., Teich, J. M., Ash, J. S., Campbell, E., & Bates, D. W. (2008) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2007.09.003 · Cited by 582
Understanding the Effect of Accuracy on Trust in Machine Learning Models
Yin, M., Wortman Vaughan, J., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300509 · Cited by 527 · cited 1×
“HCI research has helped to advance the field of explainable AI, examining how people interact with AI tools and designing tools to help end-users understand the inner workings of machine learning models [3, 8, 10, 24, 46, 73].”
BackgroundBeyond Accuracy: The Role of Mental Models in Human-AI Team Performance
Bansal, G., Nushi, B., Kamar, E., Lasecki, W. S., Weld, D. S., & Horvitz, E. (2019) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v7i1.5285 · Cited by 466 · cited 1×
“One issue is that recent AI work has focused on improving the accuracy of the model, rather than the needs of the intended user [56], and improving model accuracy does not always correlate with overall performance once implemented in the real-world [4, 5].”
BackgroundThe Role of Explanations on Trust and Reliance in Clinical Decision Support Systems
Bussone, A., Stumpf, S., & O'Sullivan, D. (2015) · 2015 International Conference on Healthcare Informatics · doi:10.1109/ichi.2015.26 · Cited by 441 · cited 1×
“We have also seen an increased use of co-design activities that give end-users a greater voice in how these systems should function [9, 36, 64, 71].”
BackgroundHuman-Centered Tools for Coping with Imperfect Algorithms During Medical Decision-Making
Cai, C. J., Reif, E., Hegde, N., Hipp, J., Kim, B., Smilkov, D., Wattenberg, M., Viegas, F., Corrado, G. S., Stumpe, M. C., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300234 · Cited by 405
The Principles and Limits of Algorithm-in-the-Loop Decision Making
Green, B., & Chen, Y. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359152 · Cited by 392 · cited 1×
“We have seen several examples in recent years of AI predictions leading to biased or unfair behaviors [20, 45].”
BackgroundPersonalized Medicine for Depression: Can We Match Patients With Treatments?
Simon, G. E., & Perlis, R. H. (2010) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2010.09111680 · Cited by 365 · cited 2×
“The frequency of ineffective drug trials has resulted in the psychiatry community calling for more information on which treatments will be most effective for an individual patient [58].”
BackgroundSelf-citationReasons For Physicians Not Adopting Clinical Decision Support Systems: Critical Analysis
Khairat, S., Marc, D., Crosby, W., & Al Sanousi, A. (2018) · JMIR Medical Informatics · doi:10.2196/medinform.8912 · Cited by 335
What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Use
Tonekaboni, S., Joshi, S., McCradden, M. D., & Goldenberg, A. (2019) · arXiv · doi:10.48550/arxiv.1905.05134 · Cited by 318
Proxy tasks and subjective measures can be misleading in evaluating explainable AI systems
Buçinca, Z., Lin, P., Gajos, K. Z., & Glassman, E. L. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377498 · Cited by 307 · cited 1×
“HCI research has helped to advance the field of explainable AI, examining how people interact with AI tools and designing tools to help end-users understand the inner workings of machine learning models [3, 8, 10, 24, 46, 73].”
BackgroundSelf-citationOh, C., Song, J., Choi, J., Kim, S., Lee, S., & Suh, B. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174223 · Cited by 301 · cited 1×
“HCI research has helped to advance the field of explainable AI, examining how people interact with AI tools and designing tools to help end-users understand the inner workings of machine learning models [3, 8, 10, 24, 46, 73].”
BackgroundExplaining models: an empirical study of how explanations impact fairness judgment
Dodge, J., Liao, Q. V., Zhang, Y., Bellamy, R. K. E., & Dugan, C. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302310 · Cited by 286
Clinical Decision Support: a 25 Year Retrospective and a 25 Year Vision
Sittig, D. F., Wright, A., & Middleton, B. (2016) · Yearbook of Medical Informatics · doi:10.15265/iys-2016-s034 · Cited by 277
Human-Centered Explainable AI: Towards a Reflective Sociotechnical Approach
Ehsan, U., & Riedl, M. O. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-60117-1_33 · Cited by 274
ICT in Health Care: Sociotechnical Approaches
Berg, M., Aarts, J., & van der Lei, J. (2003) · Methods of Information in Medicine · doi:10.1055/s-0038-1634221 · Cited by 259
Jacobs, M., Pradier, M. F., McCoy, T. H., Perlis, R. H., Doshi-Velez, F., & Gajos, K. Z. (2021) · Translational Psychiatry · doi:10.1038/s41398-021-01224-x · Cited by 247
Self-citationYang, Q., Steinfeld, A., & Zimmerman, J. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300468 · Cited by 247
Gamut: A Design Probe to Understand How Data Scientists Understand Machine Learning Models
Hohman, F., Head, A., Caruana, R., DeLine, R., & Drucker, S. M. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300809 · Cited by 225
Overrides of medication-related clinical decision support alerts in outpatients
Nanji, K. C., Slight, S. P., Seger, D. L., Cho, I., Fiskio, J. M., Redden, L. M., Volk, L. A., & Bates, D. W. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-001813 · Cited by 219
Health information overload among health consumers: A scoping review
Khaleel, I., Wimmer, B. C., Peterson, G. M., Zaidi, S. T. R., Roehrer, E., Cummings, E., & Lee, K. (2020) · Patient Education and Counseling · doi:10.1016/j.pec.2019.08.008 · Cited by 205
Integrating Mental Health and Primary Care
Thielke, S., Vannoy, S., & Unützer, J. (2007) · Primary Care: Clinics in Office Practice · doi:10.1016/j.pop.2007.05.007 · Cited by 201
The Science of Prognosis in Psychiatry: A Review
Fusar-Poli, P., Hijazi, Z., Stahl, D., & Steyerberg, E. W. (2018) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2018.2530 · Cited by 197
Genetic and epigenetic heterogeneity in acute myeloid leukemia
Li, S., Mason, C. E., & Melnick, A. (2016) · Current Opinion in Genetics & Development · doi:10.1016/j.gde.2016.03.011 · Cited by 193
Assessing Risk Assessment in Action
Stevenson, M. (2018) · Minnesota Law Review · doi:10.24926/265535.958 · Cited by 193 · cited 1×
“In recent years, we have seen increased interest in embedding AI tools in a variety of contexts, such as the justice system [2, 62], the U.”
BackgroundIncorporating ideas from computer-supported cooperative work
Pratt, W., Reddy, M. C., McDonald, D. W., Tarczy-Hornoch, P., & Gennari, J. H. (2004) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2004.04.001 · Cited by 181
Drug interaction alert override rates in the Meaningful Use era: No evidence of progress
Bryant, A. D., Fletcher, G. S., & Payne, T. H. (2014) · Applied Clinical Informatics · doi:10.4338/aci-2013-12-ra-0103 · Cited by 172 · cited 1×
“For example, studies in medical informatics found that when DST alerts arise at inopportune times (from clinicians’ vantage), the alerts become ignored or overridden [1, 35, 43].”
BackgroundSolomon, M., Wagner, S. L., & Goes, J. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.1924 · Cited by 172
Conflicting health information: a critical research need
Carpenter, D. M., Geryk, L. L., Chen, A. T., Nagler, R. H., Dieckmann, N. F., & Han, P. K. J. (2016) · Health Expectations · doi:10.1111/hex.12438 · Cited by 167
Van de Velde, S., Heselmans, A., Delvaux, N., Brandt, L., Marco-Ruiz, L., Spitaels, D., Cloetens, H., Kortteisto, T., Roshanov, P., Kunnamo, I., et al. (2018) · Implementation Science · doi:10.1186/s13012-018-0790-1 · Cited by 156 · cited 1×
“A recent study also found that including patient-facing DSTs can improve clinicians’ adherence to recommended practices when compared to DSTs that were only clinicianfacing [66].”
Gifford, W. A., Squires, J. E., Angus, D. E., Ashley, L. A., Brosseau, L., Craik, J. M., Domecq, M.-C., Egan, M., Holyoke, P., Juergensen, L., et al. (2018) · Implementation Science · doi:10.1186/s13012-018-0817-7 · Cited by 148
Xie, Y., Chen, M., Kao, D., Gao, G., & Chen, X. '. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376807 · Cited by 141
Hutton, C., & Gunn, J. (2007) · BMC Health Services Research · doi:10.1186/1472-6963-7-71 · Cited by 129
MacQueen, G., Santaguida, P., Keshavarz, H., Jaworska, N., Levine, M., Beyene, J., & Raina, P. (2017) · The Canadian Journal of Psychiatry · doi:10.1177/0706743716664885 · Cited by 123
Wolf, N. J., & Hopko, D. R. (2008) · Clinical Psychology Review · doi:10.1016/j.cpr.2007.04.004 · Cited by 116 · cited 3×
“The majority of mental health care is initiated in primary care settings, yet the extent of training primary care providers (PCP) receive in managing MDD can vary widely [63, 69], and contemporary treatment guidelines provide little support in choosing among them [31].”
BackgroundA Human-Centered Review of Algorithms used within the U.S. Child Welfare System
Saxena, D., Badillo-Urquiola, K., Wisniewski, P. J., & Guha, S. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376229 · Cited by 116
Karhade, A. V., Ogink, P. T., Thio, Q. C. B. S., Cha, T. D., Gormley, W. B., Hershman, S. H., Smith, T. R., Mao, J., Schoenfeld, A. J., Bono, C. M., et al. (2019) · The Spine Journal · doi:10.1016/j.spinee.2019.06.002 · Cited by 115
Barriers and Facilitators to Clinical Decision Support Systems Adoption: A Systematic Review
Devaraj, S., Sharma, S. K., Fausto, D. J., Viernes, S., & Kharrazi, H. (2014) · Journal of Business Administration Research · doi:10.5430/jbar.v3n2p36 · Cited by 97
Sitapati, A., Kim, H., Berkovich, B., Marmor, R., Singh, S., El‐Kareh, R., Clay, B., & Ohno‐Machado, L. (2017) · WIREs Systems Biology and Medicine · doi:10.1002/wsbm.1378 · Cited by 81
Abandoning personalization to get to precision in the pharmacotherapy of depression
Perlis, R. H. (2016) · World Psychiatry · doi:10.1002/wps.20345 · Cited by 77
Self-citationTreatment strategies to improve and sustain remission in major depressive disorder
Madhukar H., T., & Ella J., D. (2008) · Dialogues in Clinical Neuroscience · doi:10.31887/dcns.2008.10.4/mhtrivedi · Cited by 75
Treatment strategies to improve and sustain remission in major depressive disorder.
Trivedi, M. H., & Daly, E. (2008) · Europe PMC (PubMed Central) · Cited by 70
HawkEye - Deploying a Design Fiction Probe
Noortman, R., Schulte, B. F., Marshall, P., Bakker, S., & Cox, A. L. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300652 · Cited by 70
Yang, Q., Steinfeld, A. M., & Zimmerman, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.09612 · Cited by 68
Luo, Y., Liu, P., & Choe, E. K. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300822 · Cited by 65
A Guide to Using Asynchronous Remote Communities (ARC) for Researching Distributed Populations
MacLeod, H., Jelen, B., Prabhakar, A. S., Oehlberg, L., Siek, K. A., & Connelly, K. (2017) · EAI Endorsed Transactions on Pervasive Health and Technology · doi:10.4108/eai.18-7-2017.152898 · Cited by 62
Acute Care Patient Portal Intervention: Portal Use and Patient Activation
Schnock, K. O., Snyder, J. E., Fuller, T. E., Duckworth, M., Grant, M., Yoon, C. S., Lipsitz, S. R., Dalal, A. K., Bates, D. W., & Dykes, P. C. (2019) · Journal of Medical Internet Research · doi:10.2196/13336 · Cited by 44
Menke, A. (2018) · Pharmacogenomics and Personalized Medicine · doi:10.2147/pgpm.s146110 · Cited by 41 · cited 1×
“The trial and error involved in identifying effective treatment for a given individual has prompted calls for more integration of evidence-based medicine in the treatment of mental health disorders [40, 48, 58].”
BackgroundThe Epidemiology of Major Depressive Disorder
Rovner, B. W., & Casten, R. J. (2003) · Evidence-Based Eye Care · doi:10.1097/00132578-200310000-00002 · Cited by 40
Predicting treatment dropout after antidepressant initiation
Pradier, M. F., McCoy Jr, T. H., Hughes, M., Perlis, R. H., & Doshi-Velez, F. (2020) · Translational Psychiatry · doi:10.1038/s41398-020-0716-y · Cited by 37
Self-citationHughes, M. C., Pradier, M. F., Ross, A. S., McCoy, T. H., Perlis, R. H., & Doshi‐Velez, F. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.5308 · Cited by 26
Self-citationSemi-Supervised Prediction-Constrained Topic Models
Hughes, M. C., Hope, G., Weiner, L., McCoy, T. H., Perlis, R. H., Sudderth, E. B., & Doshi‐Velez, F. (2018) · International Conference on Artificial Intelligence and Statistics · Cited by 17
Self-citationNegrão, A. B., & Gold, P. W. (2007) · Elsevier eBooks · doi:10.1016/b978-012373947-6.00245-2 · Cited by 10
Review of Medical Decision Support Tools: Emerging Opportunity for Interaction Design
Yang, Q., Zimmerman, J., & Steinfeld, A. (2015) · doi:10.13140/rg.2.1.1441.3284 · Cited by 6
Aspek-aspek penting dalam kajian minyak pati (Important aspects in essential oils research)
Omar, M. N. (1983) · HRB National Drugs Library (Health Research Board) · doi:10.1001/jamanetworkopen
Research works printed but not linked: 3
- Alex Albright. 2019. If You Give a Judge a Risk Score: Evidence from Kentucky Bail Decisions. The John M. Olin Center for Law, Economics, and Business Fellows’ Discussion Paper Series (2019), 1–44.
- 10.1267/meth03040297 · doi:10.1267/meth03040297
- Melanie F. Pradier, Javier Zazo, Sonali Parbhoo, Roy H. Perlis, Maurizio Zazzi, and Finale Doshi-Velez. 2021. Preferential Mixture-of-Experts: Interpretable Models that Rely on Human Expertise As Much As Possible. In AMIA 2021 Virtual Informatics Summit.
Cited by
Trust in AI: progress, challenges, and future directions
Afroogh, S., Akbari, A., Malone, E., Kargar, M., & Alambeigi, H. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-024-04044-8 · Cited by 507
Explanations Can Reduce Overreliance on AI Systems During Decision-Making
Vasconcelos, H., Jörke, M., Grunde-McLaughlin, M., Gerstenberg, T., Bernstein, M. S., & Krishna, R. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579605 · Cited by 402
“Ecological Validity: Selecting maze solving trade s off some ecological validity compared to other tasks, as few human-XAI tasks outside of the lab will include solving mazes, but a rich literature in cognitive psychology and cognitive science involves solving puzzles of this sort (e.g., [2, 60]).”
BackgroundInfluentialChen, H., Gomez, C., Huang, C.-M., & Unberath, M. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00699-2 · Cited by 300
“Formative user research is one possible strategy that can help designers to understand end users’ needs and background knowledge [14, 15], but other approaches exist [16].”
BackgroundMethodThe Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice
Delgado, F., Yang, S., Madaio, M., & Yang, Q. (2023) · Equity and Access in Algorithms, Mechanisms, and Optimization · doi:10.1145/3617694.3623261 · Cited by 256
Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations
Chen, V., Liao, Q. V., Wortman Vaughan, J., & Bansal, G. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610219 · Cited by 220
“Different from a small but growing number of qualitative studies that aim to understand the experience of decision-makers retrospectively [14, 41, 43, 60, 71], we adopt a think-aloud protocol to investigate the real-time process as participants engage with AI output [39, 77], and use the data to understand why two typ…”
Method
Metadata
- Authors
- Jacobs, M., He, J., F. Pradier, M., Lam, B., Ahn, A. C., McCoy, T. H., Perlis, R. H., Doshi-Velez, F., & Gajos, K. Z.
- Affiliations
- Northwestern University (US)
- Harvard University (US)
- Center for Research on Computation and Society Harvard University, United States
- Beth Israel Deaconess Medical Center (US)
- Massachusetts General Hospital (US)
- SEAS Harvard University, United States
- Venue
- Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2021-05-06
- Pages
- 1-14
- OpenAlex
- W3127032480
- Cited by
- 177
- References
- 79
- Citations by year
- 2026: 30 · 2025: 43 · 2024: 38 · 2023: 33 · 2022: 29 · 2021: 4
- FWCI
- 23.31
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- major depressive disorder, antidepressant selection, clinical decision support systems, machine learning in medicine, trust in AI, socio-technical systems, system integration
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.