A Human-AI Collaborative Approach for Clinical Decision Making on Rehabilitation Assessment
Open accessThe DOI matches its Crossref record; 63 of the 63 references it cites were found in a registry or an index.
Abstract
Advances in artificial intelligence (AI) have made it increasingly applicable to supplement expert’s decision-making in the form of a decision support system on various tasks. For instance, an AI-based system can provide therapists quantitative analysis on patient’s status to improve practices of rehabilitation assessment. However, there is limited knowledge on the potential of these systems. In this paper, we present the development and evaluation of an interactive AI-based system that supports collaborative decision making with therapists for rehabilitation assessment. This system automatically identifies salient features of assessment to generate patient-specific analysis for therapists, and tunes with their feedback. In two evaluations with therapists, we found that our system supports therapists significantly higher agreement on assessment (0.71 average F1-score) than a traditional system without analysis (0.66 average F1-score, p < 0.05). After tuning with therapist’s feedback, our system significantly improves its performance from 0.8377 to 0.9116 average F1-scores (p < 0.01). This work discusses the potential of a human-AI collaborative system to support more accurate decision making while learning from each other’s strengths.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“researchers have explored the feasibility of human-AI collaborative decision-making, in which an AI model provides humans new data-driven insights on a task for achieving complementary performance, outperforming neither of the AI or the human alone [5, 30, 42, 43].”
From Understanding the Effect of Counterfactual Explanations on Trust and Reliance on AI for Human-AI Collaborative Clinical Decision Making (2023) · cites this paper as [42]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“In particular, humans can review AI explanations to understand how the AI models generate outputs [61] and identify whether an AI output is right or not [14, 42].”
From Understanding the Effect of Counterfactual Explanations on Trust and Reliance on AI for Human-AI Collaborative Clinical Decision Making (2023) · cites this paper as [42]
The cited source says what the claim attributes to it.
In the citing paper
“Many researchers have discussed that the explainability [61] of a system is critical for humanAI collaborative decision-making [14, 42].”
From Understanding the Effect of Counterfactual Explanations on Trust and Reliance on AI for Human-AI Collaborative Clinical Decision Making (2023) · cites this paper as [42]
The cited source says what the claim attributes to it.
In the citing paper
“researchers have discussed the importance of an AI explanation to communicate an AI output to a user [5, 20, 29] and realize human-AI collaboration [48] for a decision-making task [15, 42].”
From Understanding the Effect of Counterfactual Explanations on Trust and Reliance on AI for Human-AI Collaborative Clinical Decision Making (2023) · cites this paper as [42]
The cited source says what the claim attributes to it.
In the citing paper
“AI explanations have been considered as an important communication medium to realize effective human-AI collaborative decision-making tasks [37, 42].”
From Understanding the Effect of Counterfactual Explanations on Trust and Reliance on AI for Human-AI Collaborative Clinical Decision Making (2023) · cites this paper as [42]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · cited 2×
“As the application of soft target updates may lead to slow learning, we apply an experience replay [41] for sampling efficiency.”
MethodDevelopment of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research
(1988) · Advances in Psychology · doi:10.1016/s0166-4115(08)62386-9 · Cited by 15,003 · cited 2×
“• Usefulness: “[System - Condition X] is useful to understand and assess patient’s performance” • Richness: “[System - Condition X] generates new insights on patient’s performance” • Trust: “I can trust information from [System - Condition X]” • Workload: participants answered the “effort” and “frustra-tion” dimension…”
(2017) · Cited by 11,625
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782 · cited 1×
“Instead of directly updating the weight of the target network, we apply soft target updates [38]: θ ′ ← ρθ + (1− ρ)θ ′, where θ ≤ 1.”
Parkinson's disease: clinical features and diagnosis
Jankovic, J. (2008) · Journal of Neurology, Neurosurgery & Psychiatry · doi:10.1136/jnnp.2007.131045 · Cited by 5,720
Multimodal Machine Learning: A Survey and Taxonomy
Baltrusaitis, T., Ahuja, C., & Morency, L.-P. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2798607 · Cited by 5,018
Deep Reinforcement Learning with Double Q-Learning
Van Hasselt, H., Guez, A., & Silver, D. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10295 · Cited by 3,318 · cited 3×
“With this dataset, we applied reinforcement learning [37, 57] to identify the most salient features of assessment and learn a machine learning model to predict the quality of motion on patient’s exercises using leave-one-patient-out cross-validation.”
BackgroundMethodTowards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121 · cited 1×
“Explainability [8, 15, 16, 40] and interactive machine learning [1, 11, 32, 37] have been actively explored by researchers to create a better machine learning model with improved transparency and user acceptance.”
BackgroundGuidelines for Human-AI Interaction
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300233 · Cited by 2,439 · cited 5×
“To follow this practice, “social norms” [2], the interface includes comparison between the affected and unaffected side of a patient to present salient features and trajectory trends of three major joints (e.g. shoulder, elbow, and wrist) for upper-limb exercises [35].”
BackgroundMethodInfluential
Research works printed but not linked: 6
- 6. Mark T Bayley, Amanda Hurdowar, Carol L Richards, Nicol Korner-Bitensky, Sharon Wood-Dauphinee, Janice J Eng, Marilyn McKay-Lyons, Edward Harrison, Robert Teasell, Margaret Harrison, et al. 2012. Barriers to implementation of stroke rehabilitation evidence: findings from a multi-site pilot project. Disability and rehabilitation 34, 19 (2012), 1633-1638.
- 8. Or Biran and Courtenay Cotton. 2017. Explanation and justification in machine learning: A survey. In IJCAI-17 workshop on explainable AI (XAI), Vol. 8. 1.
- 20. Isabelle Guyon, Jason Weston, Stephen Barnhill, and Vladimir Vapnik. 2002. Gene selection for cancer classification using support vector machines. Machine learning 46, 1-3 (2002), 389-422.
- 43. Margit Alt Murphy, Carin Willén, and Katharina S Sunnerhagen. 2011. Kinematic variables quantifying upper-extremity performance after stroke during reaching and drinking from a glass. Neurorehabilitation and neural repair 25, 1 (2011), 71-80.
- 54. Jiliang Tang, Salem Alelyani, and Huan Liu. 2014. Feature selection for classifica- tion: A review. Data classification: Algorithms and applications (2014), 37.
- 55. Eric J Topol. 2019. High-performance medicine: the convergence of human and artificial intelligence. Nature medicine 25, 1 (2019), 44-56.
Cited by
When combinations of humans and AI are useful: A systematic review and meta-analysis
Vaccaro, M., Almaatouq, A., & Malone, T. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-02024-1 · Cited by 570
Chen, J., Yi, C., Okegbile, S. D., Cai, J., & Shen, X. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3308717 · Cited by 254
“A novel rehabilitation assessment paradigm was proposed in [264], where a domain expert and”
BackgroundWhat is Human-Centered about Human-Centered AI? A Map of the Research Landscape
Capel, T., & Brereton, M. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580959 · Cited by 228
“Medical and health research included medical imaging and diagnosis [26, 51, 95, 117, 134, 291], clinical assessment of Multiple Sclerosis [196], care within intensive care units [148], post-operative behavior [206], assessing cognitive health [321], identifying depression through social media [239], emotion recognitio…”
BackgroundMethodGomez, C., Cho, S. M., Ke, S., Huang, C.-M., & Unberath, M. (2025) · Frontiers in Computer Science · doi:10.3389/fcomp.2024.1521066 · Cited by 209
“We then analyzed the application domains where these interactions were studied.”
Lai, V., Chen, C., Smith-Renner, A., Liao, Q. V., & Tan, C. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594087 · Cited by 204
“Sample size (per condition) less than equals to 10 [17, 67], less than equals to 50 [5, 9, 13, 34, 35, 46, 55, 56, 58, 70, 87], less than equals to 100 [41, 61, 62], less than equals to 1000 [18, 29].”
BackgroundInfluentialHuman-AI interaction research agenda: A user-centered perspective
Tingting, J., Sun, Z., Fu, S., & Lv, Y. (2024) · Data and Information Management · doi:10.1016/j.dim.2024.100078 · Cited by 218
The Typing Cure: Experiences with Large Language Model Chatbots for Mental Health Support
Song, I., Pendse, S. R., Kumar, N., & Choudhury, M. D. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757430 · Cited by 198
“Through our analysis of engagements with AI chatbots for mental health support, we hope to foster understanding of how AI chatbots are being used by people in need, and discuss the broader sociocultural and contextual considerations motivating the use of this form of support.”
Joksimovic, S., Ifenthaler, D., Marrone, R., De Laat, M., & Siemens, G. (2023) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2023.100138 · Cited by 152
Wang, L., Zhang, Z., Wang, D., Cao, W., Zhou, X., Zhang, P., Liu, J., Fan, X., & Tian, F. (2023) · Frontiers in Computer Science · doi:10.3389/fcomp.2023.1187299 · Cited by 104
“As shown in Figure 2A, of the 20 studies, eleven were conducted in the United States (Stevens et al., 2012; Yang et al., 2016, 2019; Cai et al., 2019a,b; Chiang et al., 2020; Kumar et al., 2020; Romero-Brufau et al., 2020; Sendak et al., 2020; Jacobs et al., 2021; Lee et al., 2021), two each in China (Jin et al., 2020…”
BackgroundMethodResultInfluentialMa, S., Lei, Y., Wang, X., Zheng, C., Shi, C., Yin, M., & Ma, X. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581058 · Cited by 143
Sivaraman, V., Bukowski, L. A., Levin, J., Kahn, J. M., & Perer, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581075 · Cited by 141
“These issues have motivated studies that aim to evaluate and improve the humanAI collaborative system in health care [35, 40, 53], often drawing on insights from interpretable AI [52, 67, 85].”
BackgroundBreastScreening-AI: Evaluating medical intelligent agents for human-AI interactions
Calisto, F. M., Santiago, C., Nunes, N., & Nascimento, J. C. (2022) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2022.102285 · Cited by 125
Bao, Y., Gong, W., & Yang, K. (2023) · Systems · doi:10.3390/systems11090442 · Cited by 80
“Then, machine learning algorithms can help with the quantitative analysis [19].”
InfluentialKnowing About Knowing: An Illusion of Human Competence Can Hinder Appropriate Reliance on AI Systems
He, G., Kuiper, L., & Gadiraju, U. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581025 · Cited by 107
“, financial [33], medical domains [48]).”
Zhang, S., Yu, J., Xu, X., Yin, C., Lu, Y., Yao, B., Tory, M., Padilla, L. M., Caterino, J., Zhang, P., et al. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642343 · Cited by 99
“For instance, Yang et al. [95] introduce a system that auto-generates slides containing machine prognostics to aid clinicians in decision making.”
BackgroundWan, Q., Hu, S., Zhang, Y., Wang, P., Wen, B., & Lu, Z. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637361 · Cited by 88
Lee, M. H., & Chew, C. J. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610218 · Cited by 78
“For instance, a deep learning-based system has been used in clinics to bring new data-driven insights to assist the diagnosis of cancer [15], the detection of diabetic eye disease [8], or the assessment of physical stroke rehabilitation assessment [42].”
BackgroundMethodInfluentialSelf-citationMahmud, B. U., Hong, G. Y., & Fong, B. (2023) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3542698 · Cited by 100
A multifaceted vision of the human-AI collaboration: a comprehensive review
Puerta-Beldarrain, M., Gómez-Carmona, O., Sánchez-Corcuera, R., Casado-Mansilla, D., López-de-Ipiña, D., & Chen, L. (2025) · IEEE Access · doi:10.1109/access.2025.3536095 · Cited by 73
“For instance, Lee et al. developed a feature selection system that provides quantitative analysis of patients’ conditions to enhance rehabilitation assessment practices [198].”
Interactive Concept Bottleneck Models
Chauhan, K., Tiwari, R., Freyberg, J., Shenoy, P., & Dvijotham, K. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i5.25736 · Cited by 128
Liu, Y., Liu, C., Zheng, J., Xu, C., & Wang, D. (2025) · Journal of Medical Internet Research · doi:10.2196/73374 · Cited by 81
“These issues highlight the necessity of designing AI systems that align not only with technical and operational requirements but also with core ethical principles, such as transparency, fairness, and trustworthiness in health care.”
Capable but Amoral? Comparing AI and Human Expert Collaboration in Ethical Decision Making
Tolmeijer, S., Christen, M., Kandul, S., Kneer, M., & Bernstein, A. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517732 · Cited by 70
“co-creation of art [80] to HR decisions [109] and clinical decision making [78, 142].”
BackgroundHerrera, F. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103133 · Cited by 79
Lee, M. H., Siewiorek, D. P., Smailagic, A., Bernardino, A. J. M., & Badia, S. B. I. (2022) · International Journal of Social Robotics · doi:10.1007/s12369-022-00883-0 · Cited by 74
“In addition to the focus of this study, AI and robotic systems to assist post-stroke survivor’s self-directed rehabilitation therapy, it is important to explore how to make AI and robotic systems more acceptable in the perspective of therapists to administrate post-stroke survivor’s rehabilitation [24].”
BackgroundSelf-citationThe Typing Cure: Experiences with Large Language Model Chatbots for Mental Health Support
Song, I., Pendse, S. R., Kumar, N., & De Choudhury, M. (2024) · arXiv · doi:10.48550/arxiv.2401.14362 · Cited by 64
“Through our analysis of engagements with AI chatbots for mental health support, we hope to foster understanding of how AI chatbots are being used by people in need, and discuss the broader sociocultural and contextual considerations motivating the use of this form of support.”
Li, J., Yang, Y., Liao, Q. V., Zhang, J., & Lee, Y. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713336 · Cited by 88
Gu, H., Liang, Y., Xu, Y., Williams, C. K., Magaki, S., Khanlou, N., Vinters, H., Chen, Z., Ni, S., Yang, C., et al. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3577011 · Cited by 54
Al-Ansari, N., Al‐Thani, D., & Al-Mansoori, R. S. (2024) · Human Behavior and Emerging Technologies · doi:10.1155/2024/4628855 · Cited by 71
Advancing Patient-Centered Shared Decision-Making with AI Systems for Older Adult Cancer Patients
Hao, Y., Liu, Z., Riter, R. N., & Kalantari, S. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642353 · Cited by 62
Generative AI in Knowledge Work: Design Implications for Data Navigation and Decision-Making
Yun, B., Feng, D., Chen, A. S., Nikzad, A., & Salehi, N. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713337 · Cited by 38
“In the medical field, AI has been applied to rehabilitation assessment [41] and sepsis treatment [71].”
Ma, S., Chen, Q., Wang, X., Zheng, C., Peng, Z., Yin, M., & Ma, X. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713423 · Cited by 48
Bach, A. K. P., Nørgaard, T. M., Brok, J. C., & van Berkel, N. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581513 · Cited by 47
Artificial Intelligence in rehabilitation: A narrative review on advancing patient care
Alshami, A., Nashwan, A. J., AlDardour, A., & Qusini, A. (2025) · Rehabilitación · doi:10.1016/j.rh.2025.100911 · Cited by 62
Gu, H., Yang, C., Haeri, M., Wang, J., Tang, S., Yan, W., He, S., Williams, C. K., Magaki, S., & Chen, X. '. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580694 · Cited by 31
“Fortunately, recent HCI-AI-Health works have demonstrated prototypes and designs to close the gap between medical professionals and AI, which has facilitated human-AI communication and was viable to improve doctors’ works in various medical application domains, such as general medicine [44, 66, 84], radiology [16, 17]…”
BackgroundHolstein, K., De-Arteaga, M., Tumati, L., & Cheng, Y. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579628 · Cited by 38
“AI-based decision support tools (ADS) are being used to augment human decision-making across a growing range of domains—from predictive risk models used to augment decision-making in public services [11, 25, 27, 34, 41], to AI-based teacher support tools used in K-12 education [2, 7, 13, 23, 40], to clinical decision…”
BackgroundMethodCalisto, F. M., Abrantes, J. M., Santiago, C., Nunes, N. J., & Nascimento, J. C. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103444 · Cited by 40
A Missing Piece in the Puzzle: Considering the Role of Task Complexity in Human-AI Decision Making
Salimzadeh, S., He, G., & Gadiraju, U. (2023) · Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization · doi:10.1145/3565472.3592959 · Cited by 42
“Stroke Rehabilitation Assessment (25,60,0)[59, 60] Medical Image Retrieval (14,6,1023)[19]”
BackgroundSalimzadeh, S., He, G., & Gadiraju, U. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641905 · Cited by 41
“This preference could be motivated by ethical and legal concerns [68, 74, 104], as well as a desire for individual agency and accountability [54, 70, 81, 117].”
BackgroundGöndöcs, D., Horváth, S., & Dörfler, V. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103622 · Cited by 52
Public Health Calls for/with AI: An Ethnographic Perspective
Ismail, A., Thakkar, D., Madhiwalla, N., & Kumar, N. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610203 · Cited by 47
“Others have examined AI for decision-making with healthcare providers in rural clinics and rehabilitation assessment [54, 91].”
BackgroundEnhancing AI-Human Collaborative Decision-Making in Industry 4.0 Management Practices
Alam, S., & Khan, M. F. (2024) · IEEE Access · doi:10.1109/access.2024.3449415 · Cited by 49
Chen, H., Cohen, E., Wilson, D. A., & Alfred, M. C. (2024) · JMIR Human Factors · doi:10.2196/53378 · Cited by 44
Understanding Human-AI Collaboration in Music Therapy Through Co-Design with Therapists
Sun, J., Yang, J., Zhou, G., Jin, Y., & Gong, J. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642764 · Cited by 33
Who is Helping Whom? Student Concerns about AI-Teacher Collaboration in Higher Education Classrooms
Han, B., Coghlan, S., Buchanan, G., & McKay, D. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711104 · Cited by 31
“It has been revealed that human-AI collaborative systems can enhance decision-making accuracy [80] while human and AI can learn from each other’s strengths: therapists report greater con fi dence in their decisions when supported by AI’s analysis and feedback, while their comments on AI analysis…”
Explainable AI decision support improves accuracy during telehealth strep throat screening
Gomez, C., Smith, B.-L., Zayas, A., Unberath, M., & Canares, T. (2024) · Communications Medicine · doi:10.1038/s43856-024-00568-x · Cited by 30
Human-AI teaming in healthcare: 1 + 1 > 2?
Liu, P., Zhang, J., Chen, S., & Chen, S. (2025) · npj Artificial Intelligence · doi:10.1038/s44387-025-00052-4 · Cited by 40
“Lee et al. 59 asked their radiologists to evaluate breast ultrasonography images using the sequential and simultaneous modes, which took place 4 weeks apart for washout.”
Do Expressions Change Decisions? Exploring the Impact of AI's Explanation Tone on Decision-Making
Okoso, A., Yang, M., & Baba, Y. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713744 · Cited by 32
“Prediction-related information includes uncertainty of the predictions [25, 39], local feature importance [1, 14], rule-based explanations [38, 40], and example-based explanations [8, 11].”
Effective Generative AI: The Human-Algorithm CentaurPreprint
Saghafian, S. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4594780 · Cited by 31
Cultural Considerations in AI Systems for the Global South: A Systematic Review
Anuyah, O., Wan, R., Adejoro, C. O., YEH, T. P., Metoyer, R., & Badillo-Urquiola, K. A. (2023) · African Human Computer Interaction Conference (AfriCHI) · doi:10.1145/3628096.3629046 · Cited by 32
Faas, C., Uth, R., Sterz, S., Langer, M., & Feit, A. M. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812265 · Cited by 27
Cóias, A. R., Lee, M. H., & Bernardino, A. J. M. (2022) · Journal of NeuroEngineering and Rehabilitation · doi:10.1186/s12984-022-01053-z · Cited by 24
Self-citationBuschmeyer, K., Hatfield, S., & Zenner, J. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1249322 · Cited by 28
Wang, C., Feng, Y., Zhong, L., Zhu, S.-Y., Zhang, C., Zheng, S.-Q., Liang, C., Wang, Y.-T., He, C.-J., Yu, C., et al. (2023) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3643552 · Cited by 13
“[37] proposed a more comprehensive set of features to measure the performance of stroke patients performing three upper limb exercises.”
BackgroundDesigning for Control in Nurse-AI Collaboration During Emergency Medical Calls
Berge, A., Guribye, F., Fotland, S.-L. S., Fonnes, G., Johansen, I. H., & Trattner, C. (2023) · Proceedings of the 2023 ACM Designing Interactive Systems Conference · doi:10.1145/3563657.3596110 · Cited by 20
“For domain experts, decision support should provide information that is useful throughout the decision process instead of just recommending an outcome [41, 61].”
BackgroundLee, M. H., Siewiorek, D. P., Smailagic, A., Bernardino, A., & Bermúdez i Badia, S. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511112 · Cited by 19
Self-citationRetroLens: A Human-AI Collaborative System for Multi-step Retrosynthetic Route Planning
Shi, C., Hu, Y., Wang, S., Ma, S., Zheng, C., Ma, X., & Luo, Q. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581469 · Cited by 19
Jo, E., Ryu, M., Kenderova, G., So, S., Shapiro, B. B., Papoutsaki, A., & Epstein, D. A. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502206 · Cited by 22
“Clinical decision support tools have increasingly been proposed as promising ways to assist providers with computational support in various medical domains such as detection or diagnosis of a disease [8, 13, 14, 29], patient assessment [49, 50], and prognosis prediction [77, 78].”
BackgroundInfluentialHuman-AI Collaboration in Healthcare: A Scoping Review
Strong, J., Rogers, H., Sun, E., Todsen, A. L., Ede, J., Lumley, C., Yeung, N., Higham, H., & Noble, J. A. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02918-6 · Cited by 22
“…26 reported performance-style metrics for the AI, clinician, or team (used either to characterise intermediate outputs or decision qualitywhere a single “ correct ” answer was limited) such as AUC/accuracy, F1 agreement, intraclass correlation, or segmentation similarity (e.g., Dice/ HD) 32,35,36 .”
Colorbo: Envisioned Mandala Coloringthrough Human-AI Collaboration
Kim, E., Hong, J., Lee, H., & Ko, M. (2022) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3490099.3511135 · Cited by 25
“In addition, AI can collaborate with humans in business tasks, such as decision-making in the medical domain, or in planning their work tasks [16, 31].”
BackgroundFerguson, S. A., Aoyagui, P. A., Rizvi, R., Kim, Y., & Kuzminykh, A. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3687056 · Cited by 21
Zhu, Z., Tan, Y., Yamashita, N., Lee, Y., & Zhang, R. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713116 · Cited by 22
“…these imperfect AI agents might be better suited for scenarios where the outcome of the human-AI collaboration is not critical, such as scenarios outside of human-AI clinical decision-making [34], because a reduction in capability in these scenarios might impair outcomes and lead to adverse efects.”
Zhao, Z., Zhang, W., Chen, X., Song, F., Gunasegaram, J. R., Huang, W., Shi, D., He, M., & Liu, N. (2024) · Journal of Medical Internet Research · doi:10.2196/54047 · Cited by 18
“Their versatility extends to comprehensive patient assessments [14], precise disease diagnostics [15], informed treatment proposals [16], meticulous medical writing [17], innovative teaching methodologies [18], and robust question answering (QA) systems [19], embodying a multifaceted impact on the…”
Exploring the impact of explainable AI and cognitive capabilities on users’ decisions
Cau, F. M., & Spano, L. D. (2025) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-025-09438-0 · Cited by 20
“…processes (Herzog and Wörndl 2019; Zehrung et al. 2021; Musto et al. 2021; Liao et al. 2022; Viswanathan et al. 2022; Grace et al. 2022) to high-stakes scenarios like healthcare diagnostics (Cai et al. 2019b; Lee et al. 2020, 2021; Beede et al. 2020; Fogliato et al. 2022a; Panigutti et al. 2022).”
Lee, J., Jung, K., Newman, E., Chow, E., & Chen, Y. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713244 · Cited by 18
“Research on AI-integrated systems has grown, focusing on assisting clinicians in decision-making through the implementation of classical ML (e.g., predictive modeling) and deep learning (e.g., neural networks) [25, 111, 132].”
Salimzadeh, S., & Gadiraju, U. (2024) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3627043.3659567 · Cited by 19
Breckner, K., Neumayr, T., Mara, M., Streit, M., & Augstein, M. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2482742 · Cited by 16
Transitioning Cognitive Aids into Decision Support Platforms: Requirements and Design Guidelines
Mastrianni, A., Sarcevic, A., Hu, A., Almengor, L., Tempel, P. E., Gao, S., & Burd, R. S. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3582431 · Cited by 14
“Some studies found HCI issues, such as delayed alerts activation [63] and data entry burden [40].”
BackgroundInfluentialTeikari, P., & Pietrusz, A. (2021) · SportRxiv (OSF Preprints) · doi:10.31236/osf.io/w734a · Cited by 17
Agrawal, V., Kandul, S., Kneer, M., & Christen, M. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.15452 · Cited by 14
“, 2021) and processing of clinical data (Lee et al., 2021; Yang et al., 2019) to writing essays and co-creation of art (Li et al.”
BackgroundSwinger, N., Baseman, C. M., Ryu, M., Abdullah, S., Wiese, C. W., Sherrill, A. M., & Arriaga, R. I. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710917 · Cited by 15
Human–AI collaboration in operations and supply chain management: a systematic literature review
Kumar, N., & Kumar, R. (2025) · Management Review Quarterly · doi:10.1007/s11301-025-00575-9 · Cited by 19
AI-CDSS Design Guidelines and Practice Verification
He, X., Xi, Z., Ding, H., Liu, Y., & Zhu, H. (2023) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2023.2235882 · Cited by 14
“…intelligent systems have focused on three application areas: emergency management of the stroke process (Bayona et al., 2020; Wu et al., 2021), remote consultation with stroke specialists (Sakai et al., 2020), and prognostic rehabilitation of stroke patients (Kim et al., 2020; Lee et al., 2021).”
Nikolaev, V. A., & Nikolaev, A. A. (2024) · Life · doi:10.3390/life14091059 · Cited by 12
“Also, we are aware of the usability study limitations because of the small sample size and narrow range of questions.”
BackgroundWolf, S., Grundgeiger, T., Zähringer, R., Shishkova, L., Maas, F., Dilling, C., & Happel, O. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713372 · Cited by 13
“First, HCI researchers have shown the principal benefits of CDSSs, such as improved decision-making performance when working with a CDSS compared to not working with a CDSS or performance differences when working with a CDSS that provides different interaction forms (e.g., [15, 61, 70, 71, 90]).”
Schäfer, U., Sipos, L., & Müller-Birn, C. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757451 · Cited by 13
Antar, A. D., Molaei, S., Chen, Y.-Y., Lee, M. L., & Banović, N. (2024) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3654777.3676323 · Cited by 11
Self-citationUnderstanding the Effects of AI-Assisted Critical Thinking on Human-AI Decision Making
Tian, H. Y., Amin, H., & Yin, M. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790785 · Cited by 14
Trust in Clinical AI: Expanding the Unit of Analysis
Browne, J., Bakker, S., Yu, B., Lloyd, P. A., & Allouch, S. B. (2022) · Frontiers in artificial intelligence and applications · doi:10.3233/faia220192 · Cited by 16
Video Visualization and Visual Analytics: A Task-Based and Application- Driven Investigation
Xia, W., Sun, G., Li, T., Chang, B., Tang, J., Zhang, G., & Liang, R. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3423402 · Cited by 10
Esmaeilpour, D., Sivakumar, P. M., Khosravi, A., Zarepour, A., Taj, M. B., Zare, E. N., Sillanpää, M. E. T., & Karimi-Maleh, H. (2026) · Materials Today Bio · doi:10.1016/j.mtbio.2026.102981 · Cited by 14
“As a result, AI models may exhibit inflated predictive accuracy while remaining poorly equipped to identify suboptimal or non-viable nano-carrier designs [166].”
A Systematic Review of User-Centred Evaluation of Explainable AI in HealthcarePreprint
Donoso-Guzmán, I., Kacafírková, K. S., Szymanski, M., Jacobs, A., Parra, D., & Verbert, K. (2025) · VUBIR (Vrije Universiteit Brussel) · doi:10.48550/arxiv.2506.13904 · Cited by 12
Sauerzopf, L., Panduro, C. G. C., Luft, A. R., Kühnis, B., Gavagnin, E., Unger, T., Easthope, C. A., Schönhammer, J. G., Degenfellner, J., & Spiess, M. R. (2024) · Journal of NeuroEngineering and Rehabilitation · doi:10.1186/s12984-024-01506-7 · Cited by 9
“However, due to the complexity of developing accurate and sensitive observer-based scoring assessments and the lack of consensus regarding the use of marker-based motion capture systems, further research is required to construct reliable, feasible and affordable measurement tools [26, 29, 47, 51].”
Towards Uncertainty Aware Task Delegation and Human-AI Collaborative Decision-Making
Lee, M. H., & Tok, M. Z. Y. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732155 · Cited by 14
Self-citationNova, F. F., Pfafman, R., Kardys, K., Kerrigan, C., Guha, S., & Pater, J. A. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555605 · Cited by 8
“Formal clinical decision support systems have been considered as a promising way to provide medical practitioners with computational information about a patient’s condition in order to aid them in making better decisions in various health domains [99] (e.”
BackgroundInfluentialYang, D., Murr, C., Li, X., Yoo, S., Blitzblau, R. C., McDuff, S. G. R., Stephens, S. J., Wu, J., Wu, Q., & Sheng, Y. (2024) · Physics in Medicine and Biology · doi:10.1088/1361-6560/ad8e29 · Cited by 7
“Recently, approaches have been made by incorporating human feedback into model training loop to improve the clinical AI model performance and shown promising results (Dikici et al 2020, Lee et al 2021).”
Treating differently or equally: A study exploring attitudes towards AI moral advisors
Liu, Y., & Wang, T. Y. (2025) · Technology in Society · doi:10.1016/j.techsoc.2025.102862 · Cited by 9
Martinez, H. B., Cisek, K. K., García‐Rudolph, A., Kelleher, J. D., & Hines, A. (2022) · Frontiers in Neurology · doi:10.3389/fneur.2022.886477 · Cited by 7
Exploring the Impact of User and System Factors on Human-AI Interactions in Head-Worn Displays
Lu, F., Xu, Y. L., Xu, X., Jones, B., & Malamed, L. (2023) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar59233.2023.00025 · Cited by 11
Effective Generative AI: The Human-Algorithm CentaurPreprint
Saghafian, S. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4587250 · Cited by 7
Adaptive Human-Agent Teaming: A Review of Empirical Studies from the Process Dynamics PerspectivePreprint
Wang, M., Wu, J., Ma, S., Li, N., Zhang, P., Gu, N., & Lu, T. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.10918 · Cited by 8
“IML applications range from engaging novices in model development [109] to leveraging expert knowledge in critical domains like healthcare [14, 86].”
InfluentialMastrianni, A., Kim, M. S., Sullivan, T. M., Sippel, G. J., Burd, R. S., Gajos, K. Z., & Sarcevic, A. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757512 · Cited by 6
“During resuscitations, a team of providers stabilizes and evaluates the patient before determining a treatment plan and transporting the patient to their next destination in the hospital.”
Wilson, B., Natali, C., Roach, M., Scott, D., Rahat, A., Rawlinson, D. R., & Cabitza, F. (2025) · Computer Supported Cooperative Work (CSCW) · doi:10.1007/s10606-025-09514-4 · Cited by 12
Zhao, M., Yong, Z., Guan, R., Chang, K.-W., Haimovich, A. D., Ouchi, K., Bickmore, T., Zhang, Z., Yao, B., Wang, D., et al. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791301 · Cited by 8
“This aligns with and extends Human–AI collaboration frameworks [52] by emphasizing that in ED SICs, AI must serve as a partner in reconstructing actionable understanding, not merely a predictor of risk.”
Hashmi, F. S., Soroya, S. H., & Mahmood, K. (2024) · Journal of Information Science · doi:10.1177/01655515241227871 · Cited by 7
“They take proactive steps to interact with the information they find, applying it to their own health decisions and actions [18].”
Design patterns of human-AI interfaces in healthcare
Sheng, R., Shi, C., Lotfi, S., Liu, S., Perer, A., Qu, H., & Cheng, F. (2026) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2026.103737 · Cited by 10
“Many systems have provided model performance to help users better understand the effectiveness of an AI model [44, 46].”
FingHV: Efficient Sharing and Fine-Grained Scheduling of Virtualized HPU Resources
Wang, H., Yu, Z., Ren, Z., Zhang, Y., Liu, J., Wang, L., & Guo, B. (2025) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2024.3518569 · Cited by 6
Li, J., Kwong, P. W.-H., Lua, E. K., Chan, M. Y. L., Choo, A., & Donnelly, C. (2022) · Topics in Stroke Rehabilitation · doi:10.1080/10749357.2022.2127669 · Cited by 7
“As a recent study 39 suggests, domain experts and machine learning systems could learn from each other and build collective intelligence through interaction.”
BackgroundAI for Humans and Humans for AI: Towards Cultures of Participation in the Digital Age
Pagano, A., Mørch, A. I., Barletta, V. S., & Andersen, R. (2023) · Interaction design & architecture(s)/ID&A Interaction design & architecture(s) · doi:10.55612/s-5002-059-001psi · Cited by 8
Thomas, I., Oh, S. Y., & Szafir, D. A. (2024) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3640543.3645207 · Cited by 8
Cóias, A. R., Lee, M. H., Bernardino, A. J. M., Smailagic, A., Mateus, M. P. D. N., Fernandes, D., & Trapola, S. (2025) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2025.3602548 · Cited by 5
“As AI use expands, the associated risks grow too, mainly in critical decision-making areas like healthcare [32], [33], [34].”
Self-citation
Metadata
- Authors
- Lee, M. H., Siewiorek, D. P., Smailagic, A., Bernardino, A., & Bermúdez i Badia, S. B.
- Affiliations
- Carnegie Mellon University (US)
- University of Lisbon (PT)
- Universidade da Madeira / Madeira-ITI, NOVA-LINCS, Portugal
- Venue
- Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2021-05-06
- Pages
- 1-14
- OpenAlex
- W3161926791
- Cited by
- 137
- References
- 62
- Citations by year
- 2026: 14 · 2025: 47 · 2024: 29 · 2023: 31 · 2022: 15 · 2021: 1
- FWCI
- 17.03
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Singapore Management University Institutional Knowledge (InK) (Singapore Management University) (submittedVersion, other-oa)
- Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT) (submittedVersion, cc-by)
- Funders
- ERDF through the LISBOA 2020 and the FCT (LISBOA-01-0247-FEDER-045948)
- National Science Foundation (CNS-1518865)
- FCT LARSyS - Plurianual funding 2020-2023 (UIDB/50009/2020)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Technology and Human Factors in Education and Health
- Keywords
- human-AI collaboration, clinical decision support, rehabilitation assessment, therapist feedback, patient-specific analysis, adaptive decision support system
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.