Investigating the Heart Pump Implant Decision Process: Opportunities for Decision Support Tools to Help
Open accessAbstract
Clinical decision support tools (DSTs) are computational systems that aid healthcare decision-making. While effective in labs, almost all these systems failed when they moved into clinical practice. Healthcare researchers speculated it is most likely due to a lack of user-centered HCI considerations in the design of these systems. This paper describes a field study investigating how clinicians make a heart pump implant decision with a focus on how to best integrate an intelligent DST into their work process. Our findings reveal a lack of perceived need for and trust of machine intelligence, as well as many barriers to computer use at the point of clinical decision-making. These findings suggest an alternative perspective to the traditional use models, in which clinicians engage with DSTs at the point of making a decision. We identify situations across patients' healthcare trajectories when decision supports would help, and we discuss new forms it might take in these situations.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Yang et al. found that users saw little need for an ML-powered tool that performed well in laboratory tests, lacked trust in its output, and faced environmental barriers that made it difficult to use.
“They found that users saw little need for the tool, lacked trust in its output, and faced environmental barriers that made it difficult to use.”
From Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty (2021) · cites this paper as [132]
The Discussion (page unavailable) reports that VAD clinicians perceived little need for an intelligent decision-support tool, lacked trust in its ability to help with difficult cases, and faced barriers to computer use in the clinical environment. But the paper describes a field study of potential tool use, not users testing the output of a particular ML-powered tool that had performed well in laboratory tests; laboratory success is discussed for DSTs generally. (LLM, conf=0.85)
Experimental — accuracy not yet measuredAirdropped decision support systems often fail to embrace the richness of the clinical context.
“Only recently have HCI researchers in these settings begun to investigate institutional contexts behind these failures [83], noting primarily that airdropped decision support systems often fail to embrace the richness of the clinical context.”
From Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making (2018) · cites this paper as [83]
Organisational conditions are often missed in design contexts.
“As observed in studies of clinical decision tools, which often fail to be adopted [83], organisational conditions are often missed in design contexts.”
From Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making (2018) · cites this paper as [83]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Kawamoto, K., Houlihan, C. A., Balas, E. A., & Lobach, D. F. (2005) · BMJ · doi:10.1136/bmj.38398.500764.8f · Cited by 2,703 · see citing works
The Seattle Heart Failure Model
Levy, W. C., Mozaffarian, D., Linker, D. T., Sutradhar, S., Anker, S. D., Cropp, A. B., Anand, I. S., Maggioni, A. P., Burton, P. B. J., Sullivan, M. D., et al. (2006) · Circulation · doi:10.1161/circulationaha.105.584102 · Cited by 2,024 · see citing works
Ash, J. S. (2003) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1471 · Cited by 1,789 · see citing works
An integrative model of shared decision making in medical encounters
Makoul, G. T., & Clayman, M. L. (2005) · Patient Education and Counseling · doi:10.1016/j.pec.2005.06.010 · Cited by 1,639 · see citing works
Feldman, D. S., Pamboukian, S. V., Teuteberg, J. J., Birks, E. J., Lietz, K., Moore, S. A., Morgan, J. A., Arabía, F. A., Bauman, M. E., Buchholz, H. W., et al. (2013) · The Journal of Heart and Lung Transplantation · doi:10.1016/j.healun.2012.09.013 · Cited by 1,369 · see citing works
Service Blueprinting: A Practical Technique for Service Innovation
Bitner, M. J., Ostrom, A. L., & Morgan, F. N. (2008) · California Management Review · doi:10.2307/41166446 · Cited by 1,199 · see citing works
Benza, R. L., Miller, D. P., Barst, R. J., Badesch, D. B., Frost, A., & McGoon, M. D. (2012) · CHEST Journal · doi:10.1378/chest.11-1460 · Cited by 1,107 · see citing works
Predictive data mining in clinical medicine: Current issues and guidelines
BELLAZZI, R., & ZUPAN, B. (2008) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2006.11.006 · Cited by 823 · see citing works
Decision Making in Advanced Heart Failure
Allen, L. A., Stevenson, L. W., Grady, K. L., Goldstein, N. E., Matlock, D. D., Arnold, R. M., Cook, N. R., Felker, G. M., Francis, G. S., Hauptman, P. J., et al. (2012) · Circulation · doi:10.1161/cir.0b013e31824f2173 · Cited by 818 · see citing works
Medication-related clinical decision support in computerized provider order entry systems: a review
Kuperman, G. J., Bobb, A., Payne, T. H., Avery, A. J., Gandhi, T. K., Burns, G., Classen, D. C., & Bates, D. W. (2007) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2170 · Cited by 763 · see citing works
Gravel, K., Légaré, F., & Graham, I. D. (2006) · Implementation Science · doi:10.1186/1748-5908-1-16 · Cited by 755 · see citing works
Shared decision making in clinical medicine: past research and future directions
Frosch, D. L., & Kaplan, R. M. (1999) · American Journal of Preventive Medicine · doi:10.1016/s0749-3797(99)00097-5 · Cited by 721 · see citing works
Bensing, J. M. (2000) · Patient Education and Counseling · doi:10.1016/s0738-3991(99)00087-7 · Cited by 703 · see citing works
Coiera, E. W. (2003) · Health Information Management · doi:10.4324/9780203563298-15 · Cited by 608 · see citing works
Grand challenges in clinical decision support
Sittig, D. F., Wright, A., Osheroff, J. A., Middleton, B., Teich, J. M., Ash, J. S., Campbell, E., & Bates, D. W. (2008) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2007.09.003 · Cited by 582 · see citing works
The Centers for Medicare and Medicaid Services
Iglehart, J. K. (2001) · New England Journal of Medicine · doi:10.1056/nejm200112273452617 · Cited by 550 · see citing works
Jaspers, M. W. M., Smeulers, M., Vermeulen, H., & Peute, L. W. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000094 · Cited by 525 · see citing works
Elwyn, G. J., Scholl, I., Tietbohl, C. K., Mann, M. K., Edwards, A., Clay, C. F., Légaré, F., van der Weijden, T., Lewis, C. L., Wexler, R., et al. (2013) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-13-s2-s14 · Cited by 515 · see citing works
Effects of Computerized Physician Order Entry on Prescribing Practices
Teich, J. M., Merchia, P., Schmiz, J. L., Kuperman, G. J., Spurr, C. D., & Bates, D. W. (2000) · Archives of Internal Medicine · doi:10.1001/archinte.160.18.2741 · Cited by 475 · see citing works
Computer Technology and Clinical Work: Still Waiting for Godot
Wears, R. L., & Berg, M. (2005) · JAMA · doi:10.1001/jama.293.10.1261 · Cited by 448 · see citing works
Toward The ‘Tipping Point’: Decision Aids And Informed Patient Choice
O'Connor, A. M. C., Wennberg, J. E., Légaré, F., Llewellyn‐Thomas, H. A., Moulton, B. W., Sepucha, K., Sodano, A. G., & King, J. S. (2007) · Health Affairs · doi:10.1377/hlthaff.26.3.716 · Cited by 444 · see citing works
Evidence-Based Practice: An Alternative to Authority-Based Practice
Gambrill, E. D. (1999) · Families in Society The Journal of Contemporary Social Services · doi:10.1606/1044-3894.1214 · Cited by 440 · see citing works
Decision Making in Health and Medicine: Integrating Evidence and Values
Hunink, M. G. M., Weinstein, M. C., Wittenberg, E., Drummond, M. F., Pliskin, J. S., Wong, J. B., & Glasziou, P. P. (2014) · doi:10.1017/cbo9781139506779 · Cited by 425 · see citing works
Commentary: Prognostic models: clinically useful or quickly forgotten?
Wyatt, J. C., & Altman, D. G. (1995) · BMJ · doi:10.1136/bmj.311.7019.1539 · Cited by 367 · see citing works
Checklists to Reduce Diagnostic Errors
Ely, J. W., Graber, M. L., & Croskerry, P. (2011) · Academic Medicine · doi:10.1097/acm.0b013e31820824cd · Cited by 366 · see citing works
Diagnosing groups: Charting the flow of information in medical decision-making teams.
Larson, J. R., Christensen, C., Abbott, A. S., & Franz, T. M. (1996) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.71.2.315 · Cited by 331 · see citing works
Predicting Survival in Patients Receiving Continuous Flow Left Ventricular Assist Devices
Cowger, J. A., Sundareswaran, K. S., Rogers, J. G., Park, S. J., Pagani, F. D., Bhat, G., Jaski, B. E., Farrar, D. J., & Slaughter, M. S. (2012) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2012.09.055 · Cited by 316 · see citing works
Diagnosing groups: Charting the flow of information in medical decision-making teams.
Larson, J. R., Christensen, C., Abbott, A. S., & Franz, T. M. (1996) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.71.2.315 · Cited by 288 · see citing works
Coiera, E. W. (2003) · Routledge health management series · doi:10.4324/9780203563298.ch6 · Cited by 257 · see citing works
Guidelines for prevention of hospital acquired infections
Patil, V. P. B., Todi, S. K., Myatra, S. N., & Samaddar, D. P. (2014) · Indian Journal of Critical Care Medicine · doi:10.4103/0972-5229.128705 · Cited by 247 · see citing works
Decision Making in Health and Medicine: Integrating Evidence and Values
Ulahannan, T. J. (2002) · Journal of the Royal Society of Medicine · doi:10.1177/014107680209500220 · Cited by 232 · see citing works
Niazkhani, Z., Pirnejad, H., Berg, M., & Aarts, J. (2009) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2419 · Cited by 203 · see citing works
Tait, A. R., Voepel‐Lewis, T., Zikmund‐Fisher, B. J., & Fagerlin, A. (2010) · Journal of Health Communication · doi:10.1080/10810730.2010.492560 · Cited by 199 · see citing works
Clinical Decision Support Systems
(2016) · Health informatics · doi:10.1007/978-3-319-31913-1 · Cited by 196 · see citing works
Medical hegemony in decision‐making – a barrier to interdisciplinary working in intensive care?
Coombs, M. A., & Ersser, S. J. (2004) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.2004.02984.x · Cited by 194 · see citing works
Why do patients derogate physicians who use a computer-based diagnostic support system?
Shaffer, V. A., Probst, C. A., Merkle, E. C., Arkes, H. R., & Medow, M. A. (2013) · Medical Decision Making · doi:10.1177/0272989x12453501 · Cited by 115 · see citing works
Barriers and Facilitators to Clinical Decision Support Systems Adoption: A Systematic Review
Devaraj, S., SHARMA, S. K., Fausto, D. J., Viernes, S., & Kharrazi, H. H. K. (2014) · Journal of Business Administration Research · doi:10.5430/jbar.v3n2p36 · Cited by 98 · see citing works
(2006) · PubMed · Cited by 89 · see citing works
Different formats for communicating surgical risks to patients and the effect on choice of treatment
Timmermans, D. R. M., Molewijk, B. C., Stiggelbout, A. M., & Kievit, J. (2003) · Patient Education and Counseling · doi:10.1016/s0738-3991(03)00238-6 · Cited by 89 · see citing works
Baysari, M. T., Westbrook, J. I., Richardson, K. L., & Day, R. O. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000135 · Cited by 70 · see citing works
Collaborative Medical Decision Making
Christensen, C., & Larson, J. R. (1993) · Medical Decision Making · doi:10.1177/0272989x9301300410 · Cited by 65 · see citing works
From Care Plans to Care Coordination
Amir, O., Grosz, B. J., Gajos, K. Z., Swenson, S. M., & Sanders, L. (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702320 · Cited by 65 · see citing works
(2021) · Springer eBooks · doi:10.1007/978-3-030-58721-5 · Cited by 59 · see citing works
Leslie, S. J., Hartswood, M., Meurig, C., McKee, S. P., Slack, R., Procter, R., & Denvir, M. A. (2005) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2005.02.002 · Cited by 49 · see citing works
Medical secretaries' care of records
Bossen, C., Jensen, L. G., & Witt, F. (2012) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2145204.2145341 · Cited by 36 · see citing works
Uy, V., May, S. G., Tietbohl, C. K., & Frosch, D. L. (2012) · Health Expectations · doi:10.1111/j.1369-7625.2011.00760.x · Cited by 34 · see citing works
On Science, Antiscience, and the Client's Right to Effective Treatment
Thyer, B. A., & Myers, L. L. (1999) · Social Work · doi:10.1093/sw/44.5.501 · Cited by 16 · see citing works
The meaning of design in healthcare
Taneva, S., Sara, W., Julian, G., Peter, R., Emily, N., & Cafazzo, J. A. (2014) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/2559206.2579407 · Cited by 14 · see citing works
Physician perceptions of clinical reminders
Valenta, A. L., Browning, M. M., Weddle, T. E., Stevenson, G., Boyd, A. D., & Hynes, D. M. (2010) · International Health Informatics Symposium · doi:10.1145/1882992.1883101 · Cited by 14 · see citing works
Faiola, A. J., Srinivas, P., Karanam, Y., Chartash, D., & Doebbeling, B. N. (2014) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/2559206.2581332 · Cited by 6 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- [17]Faiola Anthony Srinivas Preethi Karanam Yamini Chartash David Doebbeling Bradley 2014 VizCom: a novel workflow model for ICU clinical decision support CHI’14 Extended Abstracts on Human Factors in Computing Systems ACM 1705 1710
- [48]Yang Qian Zimmerman John Steinfeld Aaron 2015 Review of Medical Decision Support Tools: Emerging Opportunity for Interaction Design IASDR 2015 Interplay Proceedings
Cited by
Sorted by citations
Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need?
Holstein, K., Wortman Vaughan, J., Daumé, H., Dudik, M., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300830 · Cited by 777 · see citing works
Re-examining Whether, Why, and How Human-AI Interaction Is Uniquely Difficult to Design
Yang, Q., Steinfeld, A., Rosé, C., & Zimmerman, J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376301 · Cited by 622 · see citing works
“First, we draw on our many years of experience in designing a wide range of AI systems, from simple adaptive user interfaces [58, 12], to large-scale crowd-sourced transportation systems [59]; from clinical decision supports [57, 55] to natural language productivity tools [52, 60, 16].”
BackgroundSelf-citationExpanding Explainability: Towards Social Transparency in AI systems
Ehsan, U., Liao, Q. V., Muller, M., Riedl, M. O., & Weisz, J. D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445188 · Cited by 538 · see citing works
Veale, M., Van Kleek, M., & Binns, R. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174014 · Cited by 490 · see citing works
“ose bought arXiv:1802.01029v1 [cs.CY] 3 Feb 2018 from vendors, and are facing immediate, value laden challenges [14]. As observed in studies of clinical decision tools, which often fail to be adopted [83], organisational conditions are often missed in design contexts. Indeed fields studying technologies in practice hav…”
BackgroundHuman-Centered Tools for Coping with Imperfect Algorithms During Medical Decision-Making
Cai, C. J., Reif, E., Hegde, N., Hipp, J., Kim, B., Smilkov, D., Wattenberg, M., Viegas, F., Corrado, G. S., Stumpe, M. C., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300234 · Cited by 405 · see citing works
Yang, Q., Steinfeld, A., & Zimmerman, J. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300468 · Cited by 247 · see citing works
Self-citationHow to Evaluate Trust in AI-Assisted Decision Making? A Survey of Empirical Methodologies
Vereschak, O., Bailly, G., & Caramiaux, B. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3476068 · Cited by 229 · see citing works
Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty
Bhatt, U., Antorán, J., Zhang, Y., Liao, Q. V., Sattigeri, P., Fogliato, R., Melançon, G., Krishnan, R., Stanley, J., Tickoo, O., et al. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462571 · Cited by 225 · see citing works
Intelligent tutors as teachers' aides
Holstein, K., McLaren, B. M., & Aleven, V. (2017) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3027385.3027451 · Cited by 179 · see citing works
Kawakami, A., Sivaraman, V., Cheng, H.-F., Stapleton, L., Cheng, Y., Qing, D., Perer, A., Wu, Z. S., Zhu, H., & Holstein, K. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517439 · Cited by 143 · see citing works
Wang, L., Zhang, Z., Wang, D., Cao, W., Zhou, X., Zhang, P., Liu, J., Fan, X., & Tian, F. (2023) · Frontiers in Computer Science · doi:10.3389/fcomp.2023.1187299 · Cited by 132 · see citing works
Modeling adoption of intelligent agents in medical imaging
Calisto, F. M., Nunes, N., & Nascimento, J. C. (2022) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102922 · Cited by 111 · see citing works
Designing Contestability: Interaction Design, Machine Learning, and Mental Health
Hirsch, T., Merced, K., Narayanan, S., Imel, Z. E., & Atkins, D. C. (2017) · Proceedings of the 2017 Conference on Designing Interactive Systems · doi:10.1145/3064663.3064703 · Cited by 110 · see citing works
Zhang, S., Yu, J., Xu, X., Yin, C., Lu, Y., Yao, B., Tory, M., Padilla, L. M., Caterino, J., Zhang, P., et al. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642343 · Cited by 99 · see citing works
Calisto, F. M., Santiago, C., Nunes, N., & Nascimento, J. C. (2021) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2021.102607 · Cited by 96 · see citing works
Charting the Sociotechnical Gap in Explainable AI: A Framework to Address the Gap in XAI
Ehsan, U., Saha, K., Choudhury, M. D., & Riedl, M. O. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579467 · Cited by 92 · see citing works
Improving fairness in machine learning systems: What do industry practitioners need?Preprint
Holstein, K., Vaughan, J. W., Daumé, H., Dudík, M., & Wallach, H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.05239 · Cited by 81 · see citing works
Yang, Q., Steinfeld, A. M., & Zimmerman, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.09612 · Cited by 80 · see citing works
Self-citationCreating Design Resources to Scaffold the Ideation of AI Concepts
Yildirim, N., Oh, C., Sayar, D., Brand, K., Challa, S., Turri, V., Walton, N., Wong, A. E., Forlizzi, J., McCann, J., et al. (2023) · Designing Interactive Systems Conference · doi:10.1145/3563657.3596058 · Cited by 77 · see citing works
Self-citationHarnessing Biomedical Literature to Calibrate Clinicians’ Trust in AI Decision Support Systems
Yang, Q., Hao, Y., Quan, K., Yang, S., Zhao, Y., Kuleshov, V., & Wang, F. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581393 · Cited by 75 · see citing works
Self-citationZając, H. D., Li, D., Dai, X., Carlsen, J. F., Kensing, F., & Andersen, T. O. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3582430 · Cited by 76 · see citing works
Veale, M., Van Kleek, M., & Binns, R. (2018) · doi:10.31235/osf.io/8kvf4 · Cited by 60 · see citing works
Lessons Learned from Designing an AI-Enabled Diagnosis Tool for Pathologists
Gu, H., Huang, J., Hung, L., & Chen, X. '. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449084 · Cited by 56 · see citing works
Jesso, S. T., Kelliher, A., Sanghavi, H. K., Martin, T., & Parker, S. H. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.830345 · Cited by 59 · see citing works
Gu, H., Liang, Y., Xu, Y., Williams, C. K., Magaki, S., Khanlou, N., Vinters, H. V., Chen, Z., Ni, S., Yang, C., et al. (2022) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3577011 · Cited by 56 · see citing works
Trustworthy and ethical AI-enabled cardiovascular care: a rapid review
Mooghali, M., Stroud, A. M., Yoo, D. W., Barry, B. A., Grimshaw, A. A., Ross, J. S., Zhu, X., & Miller, J. E. (2024) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02653-6 · Cited by 78 · see citing works
Evaluating the Promise of Human-Algorithm Collaborations in Everyday Work Practices
Wolf, C., & Blomberg, J. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359245 · Cited by 48 · see citing works
Bach, A. K. P., Nørgaard, T. M., Brok, J. C., & van Berkel, N. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581513 · Cited by 47 · see citing works
Why and How to Approach User Experience in Safety-Critical Domains: The Example of Health Care
Grundgeiger, T., Hurtienne, J., & Happel, O. (2020) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720819887575 · Cited by 45 · see citing works
"It's hard to argue with a computer"
Hirsch, T., Soma, C. S., Merced, K., Kuo, P. B., Dembe, A., Caperton, D. D., Atkins, D. C., & Imel, Z. E. (2018) · Designing Interactive Systems Conference · doi:10.1145/3196709.3196776 · Cited by 42 · see citing works
Where is the Human?: Bridging the Gap Between AI and HCI
Inkpen, K., Chancellor, S., De Choudhury, M., Veale, M., & Baumer, E. P. S. (2019) · Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290607.3299002 · Cited by 54 · see citing works
Street-Level Realities of Data Practices in Homeless Services Provision
Karusala, N., Wilson, J. H., Vayanos, P., & Rice, E. R. W. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359286 · Cited by 40 · see citing works
Holstein, K., De‐Arteaga, M., Tumati, L., & Cheng, Y. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579628 · Cited by 39 · see citing works
Yang, Q. (2018) · National Conference on Artificial Intelligence · Cited by 38 · see citing works
Self-citationHwang, J., Lee, T., Lee, H., & Byun, S. (2021) · Journal of Medical Internet Research · doi:10.2196/28659 · Cited by 39 · see citing works
van Berkel, N., Bellio, M., Skov, M. B., & BLANDFORD, A. E. (2022) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3571815 · Cited by 37 · see citing works
Gu, H., Yang, C., Haeri, M., Wang, J., Tang, S., Yan, W., He, S., Williams, C. K., Magaki, S., & Chen, X. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580694 · Cited by 36 · see citing works
Sketching AI Concepts with Capabilities and Examples: AI Innovation in the Intensive Care Unit
Yildirim, N., Zlotnikov, S., Sayar, D., Kahn, J. M., Bukowski, L. A., Amin, S. S., Riman, K. A., Davis, B. S., Minturn, J. S., King, A. J., et al. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641896 · Cited by 35 · see citing works
Self-citation"You Have to Piece the Puzzle Together"
Kaltenhauser, A., Rheinstädter, V., Butz, A. M., & Wallach, D. (2020) · Designing Interactive Systems Conference · doi:10.1145/3357236.3395436 · Cited by 34 · see citing works
Capoccia, M., Marconi, S., Singh, S. S. A., Pisanelli, D. M., & Lazzari, C. D. (2018) · BioMedical Engineering OnLine · doi:10.1186/s12938-018-0491-7 · Cited by 33 · see citing works
The Role of Design in Creating Machine-Learning-Enhanced User Experience.
Yang, Q. (2017) · National Conference on Artificial Intelligence · Cited by 32 · see citing works
Self-citationAI-Moderated Decision-Making: Capturing and Balancing Anchoring Bias in Sequential Decision Tasks
Echterhoff, J. M., Yarmand, M., & McAuley, J. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517443 · Cited by 31 · see citing works
Digital Fabrication Tools at Work
Yildirim, N., McCann, J., & Zimmerman, J. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376621 · Cited by 30 · see citing works
Self-citationSeeing Beyond Expert Blind Spots: Online Learning Design for Scale and Quality
Wang, X., Rosé, C. P., & Koedinger, K. R. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445045 · Cited by 23 · see citing works
Blanes-Selva, V., Cuesta, S. A., Doñate-Martínez, A., Mesquita, F. P., & Garcia-Gomez, J. M. (2023) · Digital Health · doi:10.1177/20552076221150735 · Cited by 22 · see citing works
Mooghali, M., Stroud, A. M., Yoo, D. W., Barry, B. A., Grimshaw, A. A., Ross, J. S., Zhu, X., & Miller, J. E. (2023) · medRxiv · doi:10.1101/2023.10.02.23296447 · Cited by 22 · see citing works
Potential for participatory big data ethics and algorithm design
Whitman, M., Hsiang, C.-Y., & Roark, K. (2018) · Participatory Design Conference · doi:10.1145/3210604.3210644 · Cited by 20 · see citing works
Overcoming Algorithm Aversion: A Comparison between Process and Outcome Control
Cheng, L., & Chouldechova, A. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581253 · Cited by 24 · see citing works
Yildirim, N., Zlotnikov, S., Venkat, A., Chawla, G., Kim, J., Bukowski, L. A., Kahn, J. M., McCann, J., & Zimmerman, J. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641982 · Cited by 19 · see citing works
Self-citationBeyond the Buzzwords: On the Perspective of AI in UX and Vice Versa
Wallach, D., Flohr, L. A., & Kaltenhauser, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-50334-5_10 · Cited by 17 · see citing works
Jo, E., Ryu, M., Kenderova, G., So, S., Shapiro, B. B., Papoutsaki, A., & Epstein, D. A. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502206 · Cited by 15 · see citing works
Lee, J.-J., Yap, C., & Roto, V. (2022) · International Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502128 · Cited by 17 · see citing works
Beyond Recommendations: From Backward to Forward AI Support of Pilots' Decision-Making Process
Zhang, Z. T., Feger, S. S., Dullenkopf, L., Liao, R., Süsslin, L. E., Liu, Y., & Butz, A. M. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3687024 · Cited by 16 · see citing works
Designing Interactive Alerts to Improve Recognition of Critical Events in Medical Emergencies
Mastrianni, A., Sarcevic, A., Chung, L., Zakeri, I., Alberto, E. C., Milestone, Z. P., Marsic, I., & Burd, R. S. (2021) · Designing Interactive Systems Conference · doi:10.1145/3461778.3462051 · Cited by 15 · see citing works
Transitioning Cognitive Aids into Decision Support Platforms: Requirements and Design Guidelines
Mastrianni, A., Sarcevic, A., Hu, A., Almengor, L., Tempel, P. E., Gao, S., & Burd, R. S. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3582431 · Cited by 11 · see citing works
Using Sensor-Captured Patient-Generated Data to Support Clinical Decision-making in PTSD Therapy
Evans, H. I., Ryu, M., Hsieh, T., Zhou, J., Xu, K., Akers, K. W., Sherrill, A. M., & Arriaga, R. I. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637426 · Cited by 12 · see citing works
"It depends": Configuring AI to Improve Clinical Usefulness Across Contexts
Zając, H. D., Ribeiro, J., Ingala, S., Gentile, S., Wanjohi, R., Gitau, S. N., Carlsen, J. F., Nielsen, M. B., & Andersen, T. O. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3660707 · Cited by 14 · see citing works
Lazzari, B. D., Badagliacca, R., Filomena, D., Papa, S., Vizza, C. D., Capoccia, M., & Lazzari, C. D. (2022) · Bioengineering · doi:10.3390/bioengineering9080383 · Cited by 12 · see citing works
Wolf, S., Grundgeiger, T., Zähringer, R., Shishkova, L., Maas, F., Dilling, C., & Happel, O. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713372 · Cited by 10 · see citing works
Hohm, A., Happel, O., Hurtienne, J., & Grundgeiger, T. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579590 · Cited by 11 · see citing works
Ghosh, P., Posner, K. L., Hyland, S. L., Cleve, W. V., Bristow, M., Long, D. R., Palla, K., Nair, B. G., Fong, C. T., Pauldine, R. W., et al. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3589953 · Cited by 11 · see citing works
Zając, H. D., Andersen, T. O., Kwasa, E., Wanjohi, R., Onyinkwa, M. K., Mwaniki, E. K., Gitau, S. N., Yaseen, S. S., Carlsen, J. F., Fraccaro, M., et al. (2025) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3715115 · Cited by 12 · see citing works
Thieme, A., Rajamohan, A. M., Cooper, B., Groombridge, H., Simister, R. J., Wong, B., Woznitza, N., Pinnock, M. A., Wetscherek, M. T., Morrison, C. P., et al. (2025) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3716500 · Cited by 12 · see citing works
Ambe, A. H., Salisbury, I. S., Grundgeiger, T., Bodnar, D., Rothwell, S., Brown, N. J., Sanderson, P. M., & Matthews, B. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713756 · Cited by 9 · see citing works
General Roadmap and Core Steps for the Development of AI Tools in Digital Pathology
Makhlouf, Y., Salto‐Tellez, M., James, J. A., O’Reilly, P. G., & Maxwell, P. (2022) · Diagnostics · doi:10.3390/diagnostics12051272 · Cited by 9 · see citing works
Yang, Q. (2021) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3411763.3457783 · Cited by 8 · see citing works
Self-citationHow HCI Interprets Service Design: A Systematic Literature Review
Yap, C. E. L., Lee, J.-J., & Roto, V. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-85616-8_16 · Cited by 8 · see citing works
Birriel, B., Alonso, W., Kitko, L. A., & Hupcey, J. E. (2019) · Heart & Lung · doi:10.1016/j.hrtlng.2019.03.002 · Cited by 8 · see citing works
Fong, A., Komolafe, T., Adams, K., Cohen, A., Howe, J. L., & Ratwani, R. M. (2019) · Applied Clinical Informatics · doi:10.1055/s-0039-1693427 · Cited by 8 · see citing works
König, P. D. (2020) · Statistics Politics and Policy · doi:10.1515/spp-2019-0006 · Cited by 8 · see citing works
Gomez-Beldarrain, G., Verma, H., Kim, E., & Bozzon, A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713978 · Cited by 8 · see citing works
Sun, L., Li, Z., Zhou, Z., Lou, S., Li, W., & Zhang, Y. (2023) · Artificial intelligence for engineering design analysis and manufacturing · doi:10.1017/s0890060422000270 · Cited by 7 · see citing works
Mastrianni, A., Kim, M. S., Sullivan, T. M., Sippel, G. J., Burd, R. S., Gajos, K. Z., & Sarcevic, A. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757512 · Cited by 6 · see citing works
“Effective clinical decision-support systems can be challenging to develop, especially for complex social-technical environments [29, 35, 36, 39, 74–76 , 81].”
Reinmund, T., Kunze, L., & Jirotka, M. D. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642247 · Cited by 5 · see citing works
Resilience Through Appropriation: Pilots’ View on Complex Decision Support
Zhang, Z. T., Storath, C., Liu, Y., & Butz, A. M. (2023) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3581641.3584056 · Cited by 5 · see citing works
A Visual Decision Support Tool for Appendectomy Care
Clarkson, E. C., Zutty, J., & Raval, M. V. (2018) · Journal of Medical Systems · doi:10.1007/s10916-018-0906-9 · Cited by 5 · see citing works
Designing positive experience for nurses in intensive care
Zumbruch, D., Kaltenhauser, A., & Knobel, M. (2020) · Mensch und Computer (MuC) · doi:10.1145/3404983.3409999 · Cited by 4 · see citing works
Reynolds, T. L., Zhang, J., Zheng, K., & Chen, Y. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555086 · Cited by 4 · see citing works
Mastrianni, A., Kmetz-Cutrone, P., Chang, K., Stein, J. Y., & Sarcevic, A. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757457 · Cited by 3 · see citing works
Mastrianni, A., Sarcevic, A., Cui, H., Krentsa, M. A., Sullivan, T. M., Zakeri, I., Marsic, I., & Burd, R. S. (2023) · Designing Interactive Systems Conference · doi:10.1145/3563657.3595979 · Cited by 3 · see citing works
Gomez-Beldarrain, G., Verma, H., Kim, E., & Bozzon, A. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3650964 · Cited by 3 · see citing works
Antweiler, D., & Fuchs, G. (2025) · Computers & Graphics · doi:10.1016/j.cag.2025.104212 · Cited by 3 · see citing works
Medical Nanorobots: The Biocompatibility Constraints of Clinical Translation
Burton, S. D. (2022) · Biosystems & biorobotics · doi:10.1007/978-3-031-04305-5_35 · Cited by 2 · see citing works
Carmichael, J. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3638182 · Cited by 2 · see citing works
Burton, S. D. (2022) · Social and cultural studies of robots and AI · doi:10.1007/978-3-030-88615-8_8 · Cited by 2 · see citing works
Designing Data Science Software for Social Care Organisations
Reinmund, T. (2023) · CHI Extended Abstracts · doi:10.1145/3544549.3577062 · Cited by 1 · see citing works
The role of service design in designing and developing AI applications: Scoping review
Tan, Y. E. I., & Lee, J.-J. (2023) · International Association of Societies of Design Research Congress (IASDR) · doi:10.21606/iasdr.2023.758 · Cited by 1 · see citing works
On the role of explanations in machine learning prediction of nonalcoholic steatohepatitis
Babakov, N., Rezgova, E., Reiter, E., & Bugarín, A. (2025) · Progress in Artificial Intelligence · doi:10.1007/s13748-025-00387-8 · Cited by 1 · see citing works
Interactive Social Agent Behaviors of Politeness and Compromise in Collaborative Decision-making
Jeong, J., & Hoffman, G. (2025) · ACM International Conference on Intelligent Virtual Agents (IVA) · doi:10.1145/3717511.3747073 · Cited by 1 · see citing works
Understanding Movement Disorder Specialists’ Workflow for Designing Digital Health Tools
Ikhile, J. A., & Pandey, V. (2026) · ACM Transactions on Computing for Healthcare · doi:10.1145/3803795 · Cited by 1 · see citing works
Perceive, Prioritize, Act: Eliciting Alert Design Requirements for Fast-Paced Medical Work
Zhang, Z., Agbugba, V. F., Billings, S., Bai, E., Luo, X., & Adelgais, K. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2682917 · Cited by 1 · see citing works
Mastrianni, A., Cui, H., & Sarcevic, A. (2022) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3491101.3519669 · Cited by 1 · see citing works
Two Case Studies of Experience Prototyping Machine Learning Systems in the WildPreprint
Yang, Q. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.09137 · Cited by 1 · see citing works
Self-citationSupporting Information Integration in Human-AI Augmentation via Reflection on UnobservablesPreprint
De‐Arteaga, M., & Holstein, K. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4884604 · Cited by 1 · see citing works
MCS Candidate Selection Criteria
Amarelli, C., Buonocore, M., Maiello, C., Montalto, A., & Wieselthaler, G. M. (2017) · Springer eBooks · doi:10.1007/978-3-319-43383-7_4
Lohmueller, L. C. (2018) · D-Scholarship@Pitt (University of Pittsburgh)
Shastri, K. (2019) · UWSpace (University of Waterloo)
Information Seeking and Sensemaking in Emergency Medical Service through Simulation Video Review.
Zhang, Z., Joy, K., Bhadani, A. S., Joshi, T. D., Adelgais, K., & Ozkaynak, M. (2023) · PubMed Central
Bias Delayed is Bias Denied? Assessing the Effect of Reporting Delays on Disparity Assessments
Gosciak, J., Balagopalan, A., Ouyang, D., Koenecke, A., Ghassemi, M., & Ho, D. E. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732123
User Centred Development of a Data Visualisation Dashboard for a Neonatal Intensive Care Unit
Lee, K. H., Yoo, S., Tran, U., Kay, J., McEwan, A. L., Sarsenbayeva, Z., Tracy, M. B., & Withana, A. I. (2024) · Australian Conference on Human-Computer Interaction (OzCHI) · doi:10.1145/3726986.3726989
- Authors
- Yang, Q., Zimmerman, J., Steinfeld, A., Carey, L., & Antaki, J. F.
- Affiliations
- Carnegie Mellon University (US)
- University of Pittsburgh (US)
- Venue
- Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2016-05-07
- Volume
- 2016
- Pages
- 4477-4488
- PMC
- PMC5101017
- OpenAlex
- W2397156396
- Cited by
- 105
- References
- 50
- Citations by year
- 2026: 5 · 2025: 12 · 2024: 12 · 2023: 20 · 2022: 15 · 2021: 10 · 2020: 7 · 2019: 11 · 2018: 9 · 2017: 4
- FWCI
- 11.12
- Type
- conference paper
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- NIH National Heart Lung and Blood Institute (NHLBI) (1R01HL122639-01A1)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Technology and Human Factors in Education and Health
- Keywords
- clinical decision support systems, clinical decision-making, user-centered design, human-computer interaction in healthcare, barriers to computer use, clinical workflow integration, care pathways
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review