“Brilliant AI Doctor” in Rural Clinics: Challenges in AI-Powered Clinical Decision Support System Deployment
Open accessThe DOI matches its Crossref record; 103 of the 104 references it cites were found in a registry or an index.
Abstract
Artificial intelligence (AI) technology has been increasingly used in the implementation of advanced Clinical Decision Support Systems (CDSS). Research demonstrated the potential usefulness of AI-powered CDSS (AI-CDSS) in clinical decision making scenarios. However, post-adoption user perception and experience remain understudied, especially in developing countries. Through observations and interviews with 22 clinicians from 6 rural clinics in China, this paper reports the various tensions between the design of an AI-CDSS system (“Brilliant Doctor”) and the rural clinical context, such as the misalignment with local context and workflow, the technical limitations and usability barriers, as well as issues related to transparency and trustworthiness of AI-CDSS. Despite these tensions, all participants expressed positive attitudes toward the future of AI-CDSS, especially acting as “a doctor’s AI assistant” to realize a Human-AI Collaboration future in clinical settings. Finally we draw on our findings to discuss implications for designing AI-CDSS interventions for rural clinical contexts in developing countries.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362
Artificial intelligence in healthcare: past, present and future
Jiang, F., Jiang, Y., Zhi, H., Dong, Y., Li, H., Ma, S., Wang, Y., Dong, Q., Shen, H., & Wang, Y. (2017) · Stroke and Vascular Neurology · doi:10.1136/svn-2017-000101 · Cited by 4,837 · cited 1×
“, [10, 10, 11, 14, 38, 71, 76, 91, 95, 96]), very little is published on the usage of AI-CDSS in practice [9, 32].”
BackgroundHansen, L. K., & Salamon, P. (1990) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.58871 · Cited by 4,260
Knowledge Graph Embedding by Translating on Hyperplanes
Wang, Z., Zhang, J., Feng, J., & Chen, Z. (2014) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v28i1.8870 · Cited by 3,915 · cited 1×
“In recent years, with the advent of artificial intelligence techniques, such as neural networkmodels [27] and knowledge graph [85], there has been an explosion in the development of AI technologies to improve CDSS performance and accuracy [56].”
BackgroundEuropean Union Regulations on Algorithmic Decision Making and a “Right to Explanation”
Goodman, B., & Flaxman, S. (2017) · AI Magazine · doi:10.1609/aimag.v38i3.2741 · Cited by 2,131 · cited 1×
“Health policies and initiatives should be enacted to address the accountability issue of AI-CDSS systems [26].”
BackgroundMachine learning for medical imaging
Erickson, B. J., Korfiatis, P., Akkus, Z., & Kline, T. L. (2017) · RadioGraphics · doi:10.1148/rg.2017160130 · Cited by 1,672
Principles of mixed-initiative user interfaces
Horvitz, E. (1999) · Proceedings of the SIGCHI conference on Human factors in computing systems the CHI is the limit - CHI '99 · doi:10.1145/302979.303030 · Cited by 1,490 · cited 1×
“This new clinician-AI collaboration paradigm will extend the current human-in-the-loop AI design philosophy [15] and the mixed-initiative design approach [28].”
MethodShared decision making: really putting patients at the centre of healthcare
Stiggelbout, A. M., Weijden, T. V. D., Wit, M. P. T. D., Frosch, D., Legare, F., Montori, V. M., Trevena, L., & Elwyn, G. (2012) · BMJ · doi:10.1136/bmj.e256 · Cited by 1,014 · cited 1×
“Emerging work in the field of shared decision making highlights the importance of considering personal and contextual factors when determining care plans between healthcare providers and patients [73].”
BackgroundClinical decision support in the era of artificial intelligence
Shortliffe, E. H., & Sepúlveda, M. J. (2018) · JAMA · doi:10.1001/jama.2018.17163 · Cited by 920 · cited 3×
“This research gap is critical because researchers need an in-depth understanding of how to appropriately implement AI-CDSS to increase user acceptance [22] and system uptake [69].”
BackgroundFriedberg, M., Chen, P., Van Busum, K., Aunon, F., Pham, C., Caloyeras, J., Mattke, S., Pitchforth, E., Quigley, D., Brook, R., et al. (2014) · RAND Health Quarterly · doi:10.7249/rhq-03-04-01 · Cited by 707 · cited 1×
“Prior work has suggested that the perceived threat of a technology to professional autonomy and the degree to which clinicians consider the use of a particular system decreases their control over decision making are salient factors that could directly affect clinicians’ willingness to use that technology [24, 64, 80].”
BackgroundA roadmap for national action on clinical decision support
Osheroff, J. A., Teich, J. M., Middleton, B., Steen, E. B., Wright, A., & Detmer, D. E. (2007) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2334 · Cited by 707
Deep neural network improves fracture detection by clinicians
Lindsey, R., Daluiski, A., Chopra, S., Lachapelle, A., Mozer, M., Sicular, S., Hanel, D., Gardner, M., Gupta, A., Hotchkiss, R., et al. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1806905115 · Cited by 640 · cited 1×
“These new AI methods, combined with the availability of large clinical datasets, revolutionize CDSS interventions from rule-based approaches toward a more “big data” and numeric-based approach [16, 41, 78].”
BackgroundFails, J. A., & Olsen, D. R. (2003) · Proceedings of the 8th international conference on Intelligent user interfaces · doi:10.1145/604045.604056 · Cited by 633 · cited 1×
“One recent research thread, Interactive Machine Learning [21], is a promising solution to the “black box” issue of AI systems.”
BackgroundThe coming of age of artificial intelligence in medicine
Patel, V. L., Shortliffe, E. H., Stefanelli, M., Szolovits, P., Berthold, M. R., Bellazzi, R., & Abu-Hanna, A. (2009) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2008.07.017 · Cited by 625 · cited 1×
“In recent years, with the advent of artificial intelligence techniques, such as neural networkmodels [27] and knowledge graph [85], there has been an explosion in the development of AI technologies to improve CDSS performance and accuracy [56].”
BackgroundBeede, E., Baylor, E., Hersch, F., Iurchenko, A., Wilcox, L., Ruamviboonsuk, P., & Vardoulakis, L. M. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376718 · Cited by 614 · cited 1×
“In this work, we join the effort of ongoing work [5, 10, 55, 74, 91] to design effective, interpretable, user-centered, and context-aware AI-CDSS systems.”
MethodGrand challenges in clinical decision support
Sittig, D. F., Wright, A., Osheroff, J. A., Middleton, B., Teich, J. M., Ash, J. S., Campbell, E., & Bates, D. W. (2008) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2007.09.003 · Cited by 582 · cited 1×
“In addition, they lack the appropriate capability to interpret ambiguous patient cases or unseen scenarios [37, 67, 72].”
BackgroundThe Chinese Health System At A Crossroads
Yip, W., & Hsiao, W. C. (2008) · Health Affairs · doi:10.1377/hlthaff.27.2.460 · Cited by 486 · cited 1×
“As the most populated country in the world, hundreds of millions of Chinese still live in rural areas and they often have low income and limited access to healthcare resources [42, 92].”
BackgroundCoiera, E. (2015) · CRC Press eBooks · doi:10.1201/b13617 · Cited by 415 · cited 2×
“When it comes to the AI era, these social and cultural factors can influence the success and outcome of AI-CDSS introduction in developing countries [48]; therefore, these factors need to be carefully examined to facilitate seamless implementation of AI-CDSS in such settings [13].”
BackgroundHuman-Centered Tools for Coping with Imperfect Algorithms During Medical Decision-Making
Cai, C. J., Reif, E., Hegde, N., Hipp, J., Kim, B., Smilkov, D., Wattenberg, M., Viegas, F., Corrado, G. S., Stumpe, M. C., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300234 · Cited by 405 · cited 6×
“In a similar vein, a group of Google researchers recently built an experimental visualization system to help physicians understand the AI interpretations of breast cancer imaging data [10].”
BackgroundMethodInfluentialIBM Watson, heal thyself: How IBM overpromised and underdelivered on AI health care
Strickland, E. (2019) · IEEE Spectrum · doi:10.1109/mspec.2019.8678513 · Cited by 402 · cited 2×
“However, given the increasing complexity of the constantly developing AI technologies, the implementation of AI-CDSS may have unique and unexpected barriers that need careful and thorough investigation, which otherwise could easily leave the new AI-CDSS system unused (just as the old ones), which is one of the biggest…”
BackgroundShiffman, R. N., Dixon, J., Brandt, C., Essaihi, A., Hsiao, A., Michel, G., & O'Connell, R. (2005) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-5-23 · Cited by 389
Magrabi, F., Ammenwerth, E., McNair, J. B., De Keizer, N. F., Hyppönen, H., Nykänen, P., Rigby, M., Scott, P. J., Vehko, T., Wong, Z. S.-Y., et al. (2019) · Yearbook of Medical Informatics · doi:10.1055/s-0039-1677903 · Cited by 347 · cited 1×
“Given these barriers, there is a general consensus that social-technical context must be considered when developing and implementing AI-CDSS [58, 75], and evaluation of these systems in real-world clinical context is essential for identifying barriers that inhibit successful implementation [43, 48].”
BackgroundHuman-AI Collaboration in Data Science: Exploring Data Scientists' Perceptions of Automated AI
Wang, D., Weisz, J. D., Muller, M., Ram, P., Geyer, W., Dugan, C., Tausczik, Y., Samulowitz, H., & Gray, A. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359313 · Cited by 342 · cited 2×
“This workshop discussion echos the broad Human-AI Collaboration research agenda happening in various application domains: education [90], data science [81, 83], and workplace [40, 65].”
BackgroundSelf-citationMoja, L., Kwag, K. H., Lytras, T., Bertizzolo, L., Brandt, L., Pecoraro, V., Rigon, G., Vaona, A., Ruggiero, F., Mangia, M., et al. (2014) · American Journal of Public Health · doi:10.2105/ajph.2014.302164 · Cited by 339 · cited 1×
“This application of AI falls under the category of a clinical decision support system (CDSS), which takes patient data as input and generates patient-specific advice [47].”
MethodGong, B., Nugent, J. P., Guest, W., Parker, W., Chang, P. J., Khosa, F., & Nicolaou, S. (2019) · Academic Radiology · doi:10.1016/j.acra.2018.10.007 · Cited by 325 · cited 2×
“But recent studies documented medical professionals’ concerns about the possibility of AI replacing their jobs as AI becomes more powerful and takes more initiatives [25].”
BackgroundPhysician acceptance of information technologies: Role of perceived threat to professional autonomy
Walter, Z., & Lopez, M. S. (2008) · Decision Support Systems · doi:10.1016/j.dss.2008.06.004 · Cited by 323 · cited 2×
“Prior work has suggested that the perceived threat of a technology to professional autonomy and the degree to which clinicians consider the use of a particular system decreases their control over decision making are salient factors that could directly affect clinicians’ willingness to use that technology [24, 64, 80].”
BackgroundPerformance of Four Computer-Based Diagnostic Systems
Berner, E. S., Webster, G. D., Shugerman, A. A., Jackson, J. R., Algina, J., Baker, A. L., Ball, E. V., Cobbs, C. G., Dennis, V. W., Frenkel, E. P., et al. (1994) · New England Journal of Medicine · doi:10.1056/nejm199406233302506 · Cited by 319 · cited 1×
“These challenges still persist to the present day in many CDSS systems, leading to non-optimal performance and low adoption rate [7].”
BackgroundComputerized clinical decision support for prescribing: provision does not guarantee uptake
Moxey, A., Robertson, J., Newby, D., Hains, I., Williamson, M., & Pearson, S.-A. (2010) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m3170 · Cited by 305 · cited 3×
“Consistent with the findings reported by [50], we noticed that recommendations generated by AI-CDSS are sometimes dismissed.”
BackgroundResultRomano, M. J., & Stafford, R. S. (2011) · Archives of Internal Medicine · doi:10.1001/archinternmed.2010.527 · Cited by 297 · cited 1×
“shows that CDSS is being used only in 17% of the patient visits [61].”
MethodImplementing electronic medical record systems in developing countries
Fraser, H., Biondich, P., Moodley, D., Choi, S., Mamlin, B., & Szolovits, P. (2005) · Journal of Innovation in Health Informatics · doi:10.14236/jhi.v13i2.585 · Cited by 296 · cited 2×
“For example, CDSS depends on EHR to gather relevant patient data, but the implementation of EHR in developing countries is limited [23].”
BackgroundMethodExplaining Decision-Making Algorithms through UI: Strategies to Help Non-Expert Stakeholders
Cheng, H.-F., Wang, R., Zhang, Z., O'Connell, F., Gray, T., Harper, F. M., & Zhu, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300789 · Cited by 292 · cited 1×
“More recently, researchers have begun emphasizing the importance of incorporating local stakeholders’ concerns and organizational contexts in developing acceptable and successful AI systems [12, 98].”
BackgroundSelf-citationDXplain: An Evolving Diagnostic Decision-Support System
Barnett, G. O. (1987) · JAMA · doi:10.1001/jama.1987.03400010071030 · Cited by 280 · cited 1×
“Early CDSS systems such as QMR [45], DXplain [4], and Meditel [87] are all rule-based or heuristic-based systems that heavily relied on human experts to manually craft knowledge-bases and heuristics [6, 88].”
BackgroundClinical Decision Support: a 25 Year Retrospective and a 25 Year Vision
Sittig, D. F., Wright, A., & Middleton, B. (2016) · Yearbook of Medical Informatics · doi:10.15265/iys-2016-s034 · Cited by 277 · cited 4×
“For example, they generally have limited number of disease states or conditions and suffer from the difficulty of maintaining the rule knowledge base in an up-to-date format [44].”
BackgroundInfluentialExploring Barriers and Facilitators to the Use of Computerized Clinical Reminders
Saleem, J. J., Patterson, E. S., Militello, L., Render, M. L., Orshansky, G., & Asch, S. M. (2005) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1777 · Cited by 250 · cited 1×
“frequent or false alarms, poor human interface design, interference with established workflow, time pressures, and inadequate training [57, 62, 63, 68, 97].”
BackgroundYang, Q., Steinfeld, A., & Zimmerman, J. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300468 · Cited by 247 · cited 4×
“In this work, we join the effort of ongoing work [5, 10, 55, 74, 91] to design effective, interpretable, user-centered, and context-aware AI-CDSS systems.”
BackgroundMethodResultInfluential中華人民共和国国家統計局 (1998) · 中国统计出版社 eBooks · Cited by 224
Rousseau, N., McColl, E., Newton, J., Grimshaw, J., & Eccles, M. (2003) · BMJ · doi:10.1136/bmj.326.7384.314 · Cited by 221
Value-Sensitive Algorithm Design: Method, Case Study, and Lessons
Zhu, H., Yu, B., Halfaker, A., & Terveen, L. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274463 · Cited by 220
Self-citationRural and Urban Disparity in Health Services Utilization in China
Liu, M., Zhang, Q., Lu, M., Kwon, C.-S., & Quan, H. (2007) · Medical Care · doi:10.1097/mlr.0b013e3180618b9a · Cited by 207 · cited 1×
“As the most populated country in the world, hundreds of millions of Chinese still live in rural areas and they often have low income and limited access to healthcare resources [42, 92].”
BackgroundJudson, T. J., Odisho, A. Y., Neinstein, A. B., Chao, J., Williams, A., Miller, C., Moriarty, T., Gleason, N., Intinarelli, G., & Gonzales, R. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa051 · Cited by 195 · cited 1×
“We believe that it will be helpful to deploy patient-facing, AI-powered self-serve triage systems, a technology that has been gaining momentum since the outbreak of COVID-19 pandemic [33].”
BackgroundThe INTERNIST-1/QUICK MEDICAL REFERENCE project--status report.
Miller, R. A., McNeil, M., Challinor, S. M., Masarie, F. E., & Myers, J. D. (1986) · PubMed Central · Cited by 191
Xu, Y., Wang, D., Collins, P., Lee, H., & Warschauer, M. (2021) · Computers & Education · doi:10.1016/j.compedu.2020.104059 · Cited by 175
Self-citationThe impact of Health Information Technology bundles on Hospital performance: An econometric study
Sharma, L., Chandrasekaran, A., Boyer, K. K., & McDermott, C. M. (2015) · Journal of Operations Management · doi:10.1016/j.jom.2015.10.001 · Cited by 168 · cited 1×
“For example, in 2009, the Health Information Technology for Economic and Clinical Health act was passed with $27 billion funding in the United States [66].”
BackgroundClinical decision support capabilities of commercially-available clinical information systems
Wright, A., Sittig, D. F., Ash, J. S., Sharma, S., Pang, J. E., & Middleton, B. (2009) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m3111 · Cited by 156 · cited 1×
“Early CDSS systems such as QMR [45], DXplain [4], and Meditel [87] are all rule-based or heuristic-based systems that heavily relied on human experts to manually craft knowledge-bases and heuristics [6, 88].”
BackgroundHealth information technology in primary health care in developing countries: a literature review.
Tomasi, E., Facchini, L. A., & Maia, M. D. F. S. (2004) · Europe PMC (PubMed Central) · Cited by 155
Utilization of Self-Diagnosis Health Chatbots in Real-World Settings: Case Study
Fan, X., Chao, D., Zhang, Z., Wang, D., Li, X., & Tian, F. (2021) · Journal of Medical Internet Research · doi:10.2196/19928 · Cited by 152 · cited 2×
“This research gap is critical because researchers need an in-depth understanding of how to appropriately implement AI-CDSS to increase user acceptance [22] and system uptake [69].”
BackgroundSelf-citationIdentifying barriers to the effective use of clinical reminders: Bootstrapping multiple methods
Patterson, E. S., Doebbeling, B. N., Fung, C. H., Militello, L. G., Anders, S. H., & Asch, S. M. (2004) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2004.11.015 · Cited by 145 · cited 1×
“frequent or false alarms, poor human interface design, interference with established workflow, time pressures, and inadequate training [57, 62, 63, 68, 97].”
BackgroundRomero-Brufau, S., Wyatt, K. D., Boyum, P., Mickelson, M., Moore, M., & Cognetta-Rieke, C. (2020) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2019.104072 · Cited by 143
Hsiao, C., & Hing, E. (2012) · PubMed · Cited by 135
Zheng, K., Padman, R., Johnson, M. P., & Diamond, H. S. (2005) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2005.03.007 · Cited by 129 · cited 1×
“frequent or false alarms, poor human interface design, interference with established workflow, time pressures, and inadequate training [57, 62, 63, 68, 97].”
BackgroundFace Value? Exploring the Effects of Embodiment for a Group Facilitation Agent
Shamekhi, A., Liao, Q. V., Wang, D., Bellamy, R. K. E., & Erickson, T. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173965 · Cited by 124 · cited 1×
“This workshop discussion echos the broad Human-AI Collaboration research agenda happening in various application domains: education [90], data science [81, 83], and workplace [40, 65].”
BackgroundSelf-citationAlden, D. L., Friend, J. M., Schapira, M. M., & Stiggelbout, A. M. (2014) · Social Science & Medicine · doi:10.1016/j.socscimed.2014.01.002 · Cited by 122 · cited 1×
“The AI-CDSS should therefore follow the guidelines of shared decision-making framework and take patients’ social, cultural, and personal context into consideration in order to generate more personalized recommendations [2].”
BackgroundKong, G., Xu, D.-L., & Yang, J.-B. (2008) · International Journal of Computational Intelligence Systems · doi:10.1080/18756891.2008.9727613 · Cited by 121 · cited 1×
“In addition, they lack the appropriate capability to interpret ambiguous patient cases or unseen scenarios [37, 67, 72].”
BackgroundDrozdal, J., Weisz, J., Wang, D., Dass, G., Yao, B., Zhao, C., Muller, M., Ju, L., & Su, H. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377501 · Cited by 119 · cited 2×
“, [18, 51, 86]), allowing users to monitor, interpret, and intervene the machine learning process so that they can establish trust toward the algorithm outputs.”
BackgroundSelf-citationLiao, Q. V., Mas-ud Hussain, M., Chandar, P., Davis, M., Khazaeni, Y., Crasso, M. P., Wang, D., Muller, M., Shami, N. S., & Geyer, W. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173577 · Cited by 116
Self-citationMosquera-Lopez, C., Agaian, S., Velez-Hoyos, A., & Thompson, I. (2015) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2014.2340401 · Cited by 115 · cited 1×
“CDSS interventions have been playing an important role in enhancing medical decision-making [54], such as assisting clinicians in diagnosing patients [49], suggesting medication and treatment options [93], and flagging potential adverse drug reactions and allergies [50].”
BackgroundShort, D., Frischer, M., & Bashford, J. (2004) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2004.02.001 · Cited by 110
Calendar.help: Designing a Workflow-Based Scheduling Agent with Humans in the Loop
Cranshaw, J., Elwany, E., Newman, T., Kocielnik, R., Yu, B., Soni, S., Teevan, J., & Monroy-Hernández, A. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025780 · Cited by 106 · cited 1×
“This new clinician-AI collaboration paradigm will extend the current human-in-the-loop AI design philosophy [15] and the mixed-initiative design approach [28].”
MethodAsh, J. S., Sittig, D. F., Guappone, K. P., Dykstra, R. H., Richardson, J., Wright, A., Carpenter, J., McMullen, C., Shapiro, M., Bunce, A., et al. (2012) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-12-6 · Cited by 103 · cited 1×
“This trend has been driven by the premise that these systems can help improve clinician performance and enhance the quality of healthcare [3].”
BackgroundSpeech recognition for clinical documentation from 1990 to 2018: a systematic review
Blackley, S. V., Huynh, J., Wang, L., Korach, Z., & Zhou, L. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy179 · Cited by 97 · cited 1×
“We propose that the AI-CDSS system can leverage the latest speech-recognition techniques to develop a voice-based user interface for hands-free data collection [8].”
BackgroundSelf-citationCoyle, D., & Doherty, G. (2009) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1519013 · Cited by 82 · cited 1×
“, [10, 10, 11, 14, 38, 71, 76, 91, 95, 96]), very little is published on the usage of AI-CDSS in practice [9, 32].”
BackgroundAutoDS: Towards Human-Centered Automation of Data Science
Wang, D., Andres, J., Weisz, J. D., Oduor, E., & Dugan, C. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445526 · Cited by 75 · cited 1×
“This workshop discussion echos the broad Human-AI Collaboration research agenda happening in various application domains: education [90], data science [81, 83], and workplace [40, 65].”
BackgroundSelf-citationFails, J. A., & Olsen, D. R. (2003) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/604050.604056 · Cited by 74
Identifying Challenges and Opportunities in Human-AI Collaboration in Healthcare
Park, S. Y., Kuo, P.-Y., Barbarin, A., Kaziunas, E., Chow, A., Singh, K., Wilcox, L., & Lasecki, W. S. (2019) · Companion Publication of the 2019 Conference on Computer Supported Cooperative Work and Social Computing · doi:10.1145/3311957.3359433 · Cited by 72 · cited 2×
“In this work, we join the effort of ongoing work [5, 10, 55, 74, 91] to design effective, interpretable, user-centered, and context-aware AI-CDSS systems.”
BackgroundMethodShared decision making: really putting patients at the centre of healthcare
(2012) · BMJ · doi:10.1136/bmj.e863 · Cited by 72
Sambasivan, M., Esmaeilzadeh, P., Kumar, N., & Nezakati, H. (2012) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-12-142 · Cited by 70 · cited 14×
“The uptake of advanced HITs is not great in rural areas of developed countries, but what about the situations in the global south?”
BackgroundMethodInfluentialYang, Q., Steinfeld, A. M., & Zimmerman, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.09612 · Cited by 68 · cited 4×
“In this work, we join the effort of ongoing work [5, 10, 55, 74, 91] to design effective, interpretable, user-centered, and context-aware AI-CDSS systems.”
BackgroundMethodResultInfluentialCalendar.Help: Designing a Workflow-Based Scheduling Agent with Humans in the LoopPreprint
Cranshaw, J., Kocielnik, R., Yu, B., & Teevan, J. B. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.2940295 · Cited by 67
CASS: Towards Building a Social-Support Chatbot for Online Health Community
Wang, L., Wang, D., Tian, F., Peng, Z., Fan, X., Zhang, Z., Yu, M., Ma, X., & Wang, H. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449083 · Cited by 65 · cited 2×
“Critical medical information is discussed and shared between patients and clinicians using these personal chat tools [84].”
BackgroundSelf-citationKussul, E. M., Makeyev, O., Baydyk, T., & Reyes, D. C. (2010) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2010.5596574 · Cited by 63 · cited 1×
“In recent years, with the advent of artificial intelligence techniques, such as neural networkmodels [27] and knowledge graph [85], there has been an explosion in the development of AI technologies to improve CDSS performance and accuracy [56].”
BackgroundWeidele, D. K. I., Weisz, J. D., Oduor, E., Muller, M., Andres, J., Gray, A., & Wang, D. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377538 · Cited by 57 · cited 3×
“, [18, 51, 86]), allowing users to monitor, interpret, and intervene the machine learning process so that they can establish trust toward the algorithm outputs.”
BackgroundSelf-citationMore Text Please! Understanding and Supporting the Use of Visualization for Clinical Text Overview
Sultanum, N., Brudno, M., Wigdor, D., & Chevalier, F. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173996 · Cited by 56 · cited 1×
“, [10, 10, 11, 14, 38, 71, 76, 91, 95, 96]), very little is published on the usage of AI-CDSS in practice [9, 32].”
BackgroundMoja, L., Liberati, E. G., Galuppo, L., Gorli, M., Maraldi, M., Nanni, O., Rigon, G., Ruggieri, P., Ruggiero, F. M., Scaratti, G., et al. (2014) · Implementation Science · doi:10.1186/s13012-014-0105-0 · Cited by 54 · cited 3×
“When it comes to the AI era, these social and cultural factors can influence the success and outcome of AI-CDSS introduction in developing countries [48]; therefore, these factors need to be carefully examined to facilitate seamless implementation of AI-CDSS in such settings [13].”
BackgroundEHR adoption across China's tertiary hospitals: A cross-sectional observational study
Shu, T., Liu, H., Goss, F. R., Yang, W. Y. L., Zhou, L., Bates, D. W., & Liang, M. (2013) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2013.08.008 · Cited by 54 · cited 2×
“[70] investigated EHR adoption in 848 tertiary hospitals in China, and found that most hospitals have only adopted a very basic EHR system and thus, the patient records need to be printed out to be shared across hospital departments.”
BackgroundMethodZamora, A., de Bobadilla, F. F., Carrión, C., Vázquez, G., Paluzié, G., Elosúa, R., Vilaseca, M., Martín-Urda, A., Rivera, A., Plana, N., et al. (2013) · Atherosclerosis · doi:10.1016/j.atherosclerosis.2013.09.029 · Cited by 49
A new era of oncology through artificial intelligence
Curioni-Fontecedro, A. (2017) · ESMO Open · doi:10.1136/esmoopen-2017-000198 · Cited by 48 · cited 1×
“These new AI methods, combined with the availability of large clinical datasets, revolutionize CDSS interventions from rule-based approaches toward a more “big data” and numeric-based approach [16, 41, 78].”
BackgroundKing, J., Furukawa, M. F., & Buntin, M. B. (2013) · Health Services Research · doi:10.1111/1475-6773.12078 · Cited by 46 · cited 11×
“The uptake of advanced HITs is not great in rural areas of developed countries, but what about the situations in the global south?”
BackgroundInfluentialAging village doctors in five counties in rural China: situation and implications
Xu, H., Zhang, W., Gu, L., Qu, Z., Sa, Z., Zhang, X., & Tian, D. (2014) · Human Resources for Health · doi:10.1186/1478-4491-12-36 · Cited by 40 · cited 1×
“China’s healthcare service has a strict hierarchical infrastructure where hospitals and clinics are categorized into three tiers according to their size, location, specialties, and so on [1, 17, 89].”
BackgroundKong, G., Xu, D.-L., & Yang, J.-B. (2008) · International Journal of Computational Intelligence Systems · doi:10.2991/ijcis.2008.1.2.6 · Cited by 40
Who are portal users vs. early e-Visit adopters? A preliminary analysis.
Jung, C., Padman, R., Shevchik, G. J., & Paone, S. (2011) · PubMed Central · Cited by 39
Tian, F., Fan, X., Fan, J., Zhu, Y., Gao, J., Wang, D., Bi, X., & Wang, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300313 · Cited by 38 · cited 1×
“These new AI methods, combined with the availability of large clinical datasets, revolutionize CDSS interventions from rule-based approaches toward a more “big data” and numeric-based approach [16, 41, 78].”
BackgroundSelf-citationPlease Call the Specialism: Using WeChat to Support Patient Care in China
Wang, D., Kale, S. D., & O’Neill, J. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376274 · Cited by 36 · cited 1×
“[35] described how WhatsApp is adopted in an Indian hospital to facilitate the organizational communication by nurses while [82] reported a case of a Chinese clinic using WeChat as its patient-facing communication channel.”
BackgroundSelf-citationKong, G., XU, D.-L., & YANG, J.-B. (2008) · International Journal of Computational Intelligence Systems · doi:10.2991/jnmp.2008.1.2.6 · Cited by 36
Making Chat at Home in the Hospital: Exploring Chat Use by Nurses
Karusala, N., Wang, D., & O’Neill, J. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376166 · Cited by 35 · cited 1×
“[35] described how WhatsApp is adopted in an Indian hospital to facilitate the organizational communication by nurses while [82] reported a case of a Chinese clinic using WeChat as its patient-facing communication channel.”
BackgroundSelf-citationWho is afraid of the system? Doctors’ attitude towards diagnostic systems
Ridderikhoff, J., & VANHERK, B. (1999) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(98)00145-2 · Cited by 33 · cited 1×
“Despite consistent findings demonstrating the potential of CDSS to support evidence-based practice and improve patient outcomes, the literature has shown that AI-CDSS applications may face difficulties when integrating into the real-world practice [60].”
BackgroundZhang, Y., Han, T., Ren, Z., Umetani, N., Tong, X., Liu, Y., Shiratori, T., & Cao, X. (2013) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2501988.2502015 · Cited by 32
Is my doctor listening to me? impact of health it systems on patient-provider interaction
Chen, Y., Cheng, K., Tang, C., Siek, K. A., & Bardram, J. E. (2013) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/2468356.2468791 · Cited by 30 · cited 1×
“, [10, 10, 11, 14, 38, 71, 76, 91, 95, 96]), very little is published on the usage of AI-CDSS in practice [9, 32].”
BackgroundPeiris, D., Usherwood, T. P., Weeramanthri, T. S., Cass, A., & Patel, A. A. (2011) · Sociology of Health & Illness · doi:10.1111/j.1467-9566.2011.01361.x · Cited by 28 · cited 1×
“Given these barriers, there is a general consensus that social-technical context must be considered when developing and implementing AI-CDSS [58, 75], and evaluation of these systems in real-world clinical context is essential for identifying barriers that inhibit successful implementation [43, 48].”
BackgroundMedical Shared Decision Making with a Virtual Agent
Zhang, Z., & Bickmore, T. (2018) · Proceedings of the 18th International Conference on Intelligent Virtual Agents · doi:10.1145/3267851.3267883 · Cited by 28 · cited 1×
“, [10, 10, 11, 14, 38, 71, 76, 91, 95, 96]), very little is published on the usage of AI-CDSS in practice [9, 32].”
BackgroundSelf-citationNavigators for medicine: evolution of online point-of-care evidence-based services
Moja, L., & Banzi, R. (2010) · International Journal of Clinical Practice · doi:10.1111/j.1742-1241.2010.02441.x · Cited by 25 · cited 1×
“This new generation of CDSS can replicate the natural flow of clinical reasoning at the point-of-care so that patientspecific recommendations are generated to augment clinician decision-making [46].”
BackgroundZhang, H., Han, B. T., & Tang, Z. (2017) · Health Policy and Technology · doi:10.1016/j.hlpt.2017.01.002 · Cited by 23 · cited 10×
“The uptake of advanced HITs is not great in rural areas of developed countries, but what about the situations in the global south?”
BackgroundInfluentialConversational Agents: Acting on the Wave of Research and Development
Jacques, R., Følstad, A., Gerber, E. M., Grudin, J. T., Luger, E. A., Monroy‐Hernández, A., & Wang, D. (2019) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3290607.3299034 · Cited by 23 · cited 1×
“Both the “black box” concern and the “threat to professional au-tonomy” concern highlight the emerging need of re-considering the relationship between the agency of human users and the agency of the more advanced, yet less transparent, AI technologies (e.g., [31, 34, 55, 83]).”
BackgroundSelf-citationReading Face, Reading Health: Exploring Face Reading Technologies for Everyday Health
Ding, X., Jiang, Y., Qin, X., Chen, Y., Zhang, W., & Qi, L. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300435 · Cited by 22 · cited 1×
“China’s healthcare service has a strict hierarchical infrastructure where hospitals and clinics are categorized into three tiers according to their size, location, specialties, and so on [1, 17, 89].”
BackgroundCASS: Towards Building a Social-Support Chatbot for Online Health CommunityPreprint
Wang, L., Wang, D., Tian, F., Peng, Z., Fan, X., Zhang, Z., Ma, S., Yu, M., Ma, X., & Wang, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.01583 · Cited by 22 · cited 2×
“Critical medical information is discussed and shared between patients and clinicians using these personal chat tools [84].”
BackgroundSelf-citationDrozdal, J., Weisz, J. D., Wang, D., Dass, G., Yao, B., Zhao, C., Muller, M. J., Lin, J., & Su, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.06509 · Cited by 21 · cited 2×
“, [18, 51, 86]), allowing users to monitor, interpret, and intervene the machine learning process so that they can establish trust toward the algorithm outputs.”
BackgroundSelf-citationModel LineUpper: Supporting Interactive Model Comparison at Multiple Levels for AutoML
Narkar, S., Zhang, Y., Liao, Q. V., Wang, D., & Weisz, J. D. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450658 · Cited by 21 · cited 1×
“, [18, 51, 86]), allowing users to monitor, interpret, and intervene the machine learning process so that they can establish trust toward the algorithm outputs.”
BackgroundSelf-citationSingh, A., Newhouse, N., Gibbs, J., BLANDFORD, A. E., Chen, Y., Briggs, P., Mentis, H. M., Sellen, K., & Bardram, J. E. (2017) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3027063.3049287 · Cited by 18 · cited 1×
“, [10, 10, 11, 14, 38, 71, 76, 91, 95, 96]), very little is published on the usage of AI-CDSS in practice [9, 32].”
BackgroundLi, D., Chao, J., Kong, J., Cao, G., Lv, M., & Zhang, M. (2019) · Health Informatics Journal · doi:10.1177/1460458219889794 · Cited by 16 · cited 2×
“The clinicians in these rural clinics also lack training opportunities to learn how to use sophisticated HITs, which further limits the adoption of AI-CDSS systems [39].”
BackgroundRe-Evaluating Professional Autonomy in Health Care
ten Have, H. (2000) · Metamedicine · doi:10.1023/a:1009933624853 · Cited by 13 · cited 1×
“This is related to the “professional autonomy” concept—the clinicians need guarantee to freely operate their professional judgement and decision-making in patient care without any interference [77].”
BackgroundKulp, L., & Sarcevic, A. (2018) · Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3170427.3188571 · Cited by 10 · cited 1×
“, [10, 10, 11, 14, 38, 71, 76, 91, 95, 96]), very little is published on the usage of AI-CDSS in practice [9, 32].”
BackgroundRehnmark, F., Bluethmann, W. J., Rochlis, J., Huber, E. L., & Ambrose, R. O. (2003) · NASA Technical Reports Server (NASA) · Cited by 8
Useful Interactions Between Human and Robotic Agents Performing a Cooperative Assembly Task
Rehnmark, F., Bluethmann, W. J., Huber, E. L., Rochlis, J., & Ambrose, R. O. (2003) · AIAA SPACE Conference & Exposition · doi:10.2514/6.2003-6273 · Cited by 5
Meditel: Computer Assisted Diagnosis.
(1990) · Annals of Internal Medicine · doi:10.7326/0003-4819-112-9-719_1 · Cited by 3 · cited 1×
“Early CDSS systems such as QMR [45], DXplain [4], and Meditel [87] are all rule-based or heuristic-based systems that heavily relied on human experts to manually craft knowledge-bases and heuristics [6, 88].”
Research works printed but not linked: 4
- MIT Technology Review Insights. 2019. AI in health care: Capacity capability and a future of active health in Asia.https://mit-insights.ai/ai-in-health-care/.
- Farah Magrabi, Elske Ammenwerth, Jytte Brender McNair, Nicolet F De Keizer, Hannele Hyppönen, Pirkko Nykänen, Michael Rigby, Philip J Scott, Tuulikki Vehko, Zoie Shui-Yee Wong, 2019. Artificial intelligence in clinical decision support: challenges for evaluating AI and practical implications. Yearbook of medical informatics 28, 01 (2019), 128–134.
- Peter Stone, R Brooks, E Brynjolfsson, R Calo, O Etzioni, G Hager, and A Teller. 2016. One hundred year study on artificial intelligence. Artificial Intelligence and Life in 2030 (2016).
- 10.5555/2893873.2894046 · doi:10.5555/2893873.2894046
Cited by
Challenges in deploying machine learning: A survey of case studies
Paleyes, A., Urma, R.-G., & Lawrence, N. D. (2023) · ACM Computing Surveys · doi:10.1145/3533378 · Cited by 638
“One of the main reasons quoted was the UX design that did not take into account particularities of the environment (screen sizes, interaction with other software in the clinic) where it was installed, often resulting in an unusable software [147].”
BackgroundCiecierski-Holmes, T., Singh, R., Axt, M., Brenner, S., & Barteit, S. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00700-y · Cited by 343
“…radiation treatment plans 17,21 , but also some discordance between clinical decision support systems and the local treatment options available 20,22 . i n d i c a t e t h a t t h e c i t e d a rt i c l e r e p o rt e d a m i x o f s tr e ng t h s a n d w e a k n e ss e s f o r t h a t p a rt i…”
BackgroundMethodInfluentialTun, H. M., Rahman, H. A., Naing, L., & Malik, O. A. (2025) · Journal of Medical Internet Research · doi:10.2196/69678 · Cited by 210
Shaik, T., Tao, X., Li, L., Xie, H., & Velásquez, J. D. (2024) · Information Fusion · doi:10.1016/j.inffus.2023.102040 · Cited by 196
“Integration of multimodal medical data fusion into Clinical Decision Support Systems (CDSS) enhances clinical decision-making and improves patient outcomes [155], [156].”
BackgroundShi, Y., Gao, T., Jiao, X., & Cao, N. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610217 · Cited by 170
“Regarding the other actor of the interaction, human, researchers have focused on how different user groups interact with AI, such as children [82, 179], clinicians [176, 195], artists [21], and data scientists [63, 177].”
Background
Metadata
- Authors
- Wang, D., Wang, L., Zhang, Z., Wang, D., Zhu, H., Gao, Y., Fan, X., & Tian, F.
- Affiliations
- IBM (United States) (US)
- Chinese Academy of Sciences (CN)
- Pace University (US)
- Carnegie Mellon University (US)
- State University of New York (US)
- Venue
- Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2021-05-06
- Pages
- 1-18
- OpenAlex
- W3120083395
- Cited by
- 209
- References
- 104
- Citations by year
- 2026: 40 · 2025: 55 · 2024: 43 · 2023: 43 · 2022: 22 · 2021: 6
- FWCI
- 25.55
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Technology and Human Factors in Education and Health
- Keywords
- rural clinics, human-AI collaboration, clinician perceptions, usability barriers, rural health care
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.