Designing for Control in Nurse-AI Collaboration During Emergency Medical Calls
Open accessThe DOI matches its Crossref record; 73 of the 73 references it cites were found in a registry or an index.
Abstract
AI-powered symptom checkers are automating the work of telephone triage nurses in assessing patient urgency. Yet, these systems exclude several vulnerable patient groups and overlook telenurses’ competent interaction with their patients. This study, conducted in collaboration with telenurses, examines how AI can support their clinical assessment and was carried out in four phases: 1) interviews that revealed telenurses’ challenge of juggling decision-support and documentation interfaces, 2) a co-design workshop that conceptualized continuous nurse-AI interaction, 3) development of a prototype that suggested questions for nurses to ask callers, and 4) a role-play workshop that demonstrated nurse-AI interaction in practice. The study addresses how we can design for control in human-AI collaboration in order to enhance, rather than replace, human decision-making processes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Medical AI employs expert-supervised reactive agency, where clinicians maintain final decision-making authority ( reactive ).”
From Exploring Collaboration Patterns and Strategies in Human-AI Co-creation through the Lens of Agency: A Scoping Review of the Top-tier HCI Literature (2025) · cites this paper as [18]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Societally, the integration of autonomous AI into co-creation could significantly restructure creative industries and labor markets [18, 60, 118], raising critical questions about the future of creative work, including intellectual property [83], fair compensation [82], and evolving skill requirements [202].”
From Exploring Collaboration Patterns and Strategies in Human-AI Co-creation through the Lens of Agency: A Scoping Review of the Top-tier HCI Literature (2025) · cites this paper as [18]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
Sorting Things Out: Classification and Its Consequences
Bowker, G. C., & Star, S. L. (1999) · doi:10.7551/mitpress/6352.001.0001 · Cited by 6,188
Springer Science+Business Media
Технология (2013) · Betascript Publishing eBooks · Cited by 3,087
Speculative design: crafting the speculation
Auger, J. (2013) · Digital Creativity · doi:10.1080/14626268.2013.767276 · Cited by 732
Re-examining Whether, Why, and How Human-AI Interaction Is Uniquely Difficult to Design
Yang, Q., Steinfeld, A., Rosé, C., & Zimmerman, J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376301 · Cited by 622
Human Performance Consequences of Stages and Levels of Automation: An Integrated Meta-Analysis
Onnasch, L., Wickens, C. D., Li, H., & Manzey, D. (2014) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720813501549 · Cited by 524
Toward human-centered AI: a perspective from human-computer interaction
Xu, W. (2019) · Interactions · doi:10.1145/3328485 · Cited by 454
Human-Centered Tools for Coping with Imperfect Algorithms During Medical Decision-Making
Cai, C. J., Reif, E., Hegde, N., Hipp, J., Kim, B., Smilkov, D., Wattenberg, M., Viegas, F., Corrado, G. S., Stumpe, M. C., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300234 · Cited by 405
Artificial intelligence and the implementation challenge
Shaw, J., Rudzicz, F., Jamieson, T., & Goldfarb, A. (2019) · Journal of Medical Internet Research · doi:10.2196/13659 · Cited by 389
Xu, W., Dainoff, M. J., Ge, L., & Gao, Z. (2023) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2022.2041900 · Cited by 363
Fernandes, M., Vieira, S. M., Leite, F., Palos, C., Finkelstein, S., & Sousa, J. M. C. (2020) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2019.101762 · Cited by 312
Yang, Q., Steinfeld, A., & Zimmerman, J. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300468 · Cited by 247
Rosenbloom, S. T., Miller, R. A., Johnson, K. B., Elkin, P. L., & Brown, S. H. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1957 · Cited by 229
Deep features-based speech emotion recognition for smart affective services
Badshah, A. M., Rahim, N., Ullah, N., Ahmad, J., Muhammad, K., Lee, M. Y., Kwon, S., & Baik, S. W. (2019) · Multimedia Tools and Applications · doi:10.1007/s11042-017-5292-7 · Cited by 182
Safety of patient-facing digital symptom checkers
Fraser, H., Coiera, E., & Wong, D. (2018) · The Lancet · doi:10.1016/s0140-6736(18)32819-8 · Cited by 180
Eliciting Tech Futures Among Black Young Adults: A Case Study of Remote Speculative Co-Design
Harrington, C., & Dillahunt, T. R. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445723 · Cited by 169
Blomberg, S. N., Christensen, H. C., Lippert, F., Ersbøll, A. K., Torp-Petersen, C., Sayre, M. R., Kudenchuk, P. J., & Folke, F. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.32320 · Cited by 138
A Human-AI Collaborative Approach for Clinical Decision Making on Rehabilitation Assessment
Lee, M. H., Siewiorek, D. P., Smailagic, A., Bernardino, A., & Bermúdez i Badia, S. B. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445472 · Cited by 137
Liable, but not in control? Ensuring meaningful human agency in automated decision-making systems
Wagner, B. (2019) · Policy & Internet · doi:10.1002/poi3.198 · Cited by 136 · cited 1×
“When designing automation tools for decision-making, it is essential to consider and explicitly address how to retain “meaningful human agency” in the work processes [72].”
Coiera, E., Kocaballi, B., Halamka, J., & Laranjo, L. (2018) · npj Digital Medicine · doi:10.1038/s41746-018-0066-9 · Cited by 132
Artificial intelligence in emergency medicine: A scoping review
Kirubarajan, A., Taher, A., Khan, S., & Masood, S. (2020) · JACEP Open · doi:10.1002/emp2.12277 · Cited by 127
Murdoch, J., Barnes, R., Pooler, J., Lattimer, V., Fletcher, E., & Campbell, J. L. (2014) · Social Science & Medicine · doi:10.1016/j.socscimed.2014.12.013 · Cited by 116
A comprehensive model of the process of telephone nursing
Greenberg, M. E. (2009) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.2009.05132.x · Cited by 116
Machine learning methods applied to triage in emergency services: A systematic review
Sánchez-Salmerón, R., Gómez-Urquiza, J. L., Albendín-García, L., Correa-Rodríguez, M., Martos-Cabrera, M. B., Velando-Soriano, A., & Suleiman-Martos, N. (2022) · International Emergency Nursing · doi:10.1016/j.ienj.2021.101109 · Cited by 115
Salwei, M. E., & Carayon, P. (2022) · Journal of Cognitive Engineering and Decision Making · doi:10.1177/15553434221097357 · Cited by 112
Kocaballi, A. B., Ijaz, K., Laranjo, L., Quiroz, J. C., Rezazadegan, D., Tong, H. L., Willcock, S., Berkovsky, S., & Coiera, E. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa131 · Cited by 112
Ernesäter, A., Holmström, I. K., & Engström, M. (2009) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.2009.04966.x · Cited by 98
Morse, K. E., Ostberg, N. P., Jones, V. G., & Chan, A. S. (2020) · Journal of Medical Internet Research · doi:10.2196/20549 · Cited by 85
Byrsell, F., Claesson, A., Ringh, M., Svensson, L. A., Jonsson, M. N., Nordberg, P., Forsberg, S., Hollenberg, J., & Nord, A. (2021) · Resuscitation · doi:10.1016/j.resuscitation.2021.02.041 · Cited by 80
Rees, P., Edwards, A., Powell, C., Hibbert, P. D., Williams, H., Makeham, M., Carter, B., Luff, D. F., Parry, G. J., Avery, A. J., et al. (2017) · PLoS Medicine · doi:10.1371/journal.pmed.1002217 · Cited by 78
Lin, S. (2022) · The Journal of the American Board of Family Medicine · doi:10.3122/jabfm.2022.01.210226 · Cited by 77
Marshall, M., de Silva, D., Cruickshank, L., Shand, J., Wei, L., & Anderson, J. (2017) · BMJ Quality & Safety · doi:10.1136/bmjqs-2016-006143 · Cited by 71
Predictors of Patient Satisfaction With Telephone Nursing Services
Moscato, S. R., VALANIS, B. G., Gullion, C. M., Tanner, C., Shapiro, S., & Izumi, S. (2007) · Clinical Nursing Research · doi:10.1177/1054773806298507 · Cited by 69
Beyond guidelines: discretionary practice in face‐to‐face triage nursing
Johannessen, L. E. F. (2017) · Sociology of Health & Illness · doi:10.1111/1467-9566.12578 · Cited by 67
Does AI make PD obsolete?: exploring challenges from artificial intelligence to participatory design
Bratteteig, T., & Verne, G. (2018) · Proceedings of the 15th Participatory Design Conference: Short Papers, Situated Actions, Workshops and Tutorial - Volume 2 · doi:10.1145/3210604.3210646 · Cited by 66
Aboueid, S., Meyer, S. B., Wallace, J. R., Mahajan, S., & Chaurasia, A. K. (2020) · JMIR Public Health and Surveillance · doi:10.2196/22637 · Cited by 65
Disappearing expertise in clinical automation: Barcode medication administration and nurse autonomy
Hong, J. Y., Ivory, C. H., VanHouten, C., Simpson, C., & Novak, L. L. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa135 · Cited by 65
Therapy Decision Support Based on Recommender System Methods
Gräßer, F., Beckert, S., Küster, D., Schmitt, J. M., Abraham, S., Malberg, H., & Zaunseder, S. (2017) · Journal of Healthcare Engineering · doi:10.1155/2017/8659460 · Cited by 62
The Discourse of Medicine: Dialectics of Medical Interviews.
West, C., & Mishler, E. G. (1986) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2070073 · Cited by 60
Speculative Design as Thought Experiment
Barendregt, L., & Vaage, N. S. (2021) · She ji · doi:10.1016/j.sheji.2021.06.001 · Cited by 57
The implications of improvisational acting and role-playing on design methodologies
Medler, B., & Magerko, B. S. (2010) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753398 · Cited by 54
Johannessen, L. E. F. (2018) · Social Science & Medicine · doi:10.1016/j.socscimed.2018.02.004 · Cited by 54
Coiera, E., & Liu, S. (2022) · Cell Reports Medicine · doi:10.1016/j.xcrm.2022.100860 · Cited by 48
Control Matters in Elder Care Technology: Evidence and Direction for Designing It In
Berridge, C., Zhou, Y., Lazar, A., Porwal, A., Mattek, N., Gothard, S., & Kaye, J. (2022) · Designing Interactive Systems Conference · doi:10.1145/3532106.3533471 · Cited by 46
Taking part: role-play in the design of therapeutic systems
Matthews, M., Gay, G., & Doherty, G. (2014) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557103 · Cited by 42
Name that room: room identification using acoustic features in a recording
Peters, N., Lei, H., & Friedland, G. (2012) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2393347.2396326 · Cited by 40
Automated deception detection of 911 call transcripts
Burns, M. B., & Moffitt, K. C. (2014) · Security Informatics · doi:10.1186/s13388-014-0008-2 · Cited by 39
Greenbaum, N. R., Jernite, Y., Halpern, Y., Calder, S., Nathanson, L. A., Sontag, D. A., & Horng, S. (2019) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2019.103981 · Cited by 38
Triage accuracy of online symptom checkers for Accident and Emergency Department patients
Yu, S. W. Y., Ma, A., Tsang, V. H. M., Chung, L. S. W., Leung, S.-C., & Leung, L.-P. (2020) · Hong Kong Journal of Emergency Medicine · doi:10.1177/1024907919842486 · Cited by 36
Kudina, O., & de Boer, B. (2021) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.13535 · Cited by 34
Kujala, S., & Hörhammer, I. (2022) · Journal of Medical Internet Research · doi:10.2196/33505 · Cited by 32
Deschamps-Berger, T., Lamel, L. F., & Devillers, L. Y. (2022) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3536220.3558038 · Cited by 32
ICT-Enabled Grievance Redressal in Central India: A Comparative Analysis
Marathe, M., O’Neill, J., Pain, P., & Thies, W. (2016) · International Conference on Information and Communication Technologies and Development (ICTD) · doi:10.1145/2909609.2909653 · Cited by 29
Chin, K.-C., Hsieh, T.-C., Chiang, W., Chien, Y.-C., Sun, J.-T., Lin, H.-Y., Hsieh, M., Yang, C.-W., Chen, A. Y., & Huei‐Ming, M. (2021) · Resuscitation · doi:10.1016/j.resuscitation.2021.08.032 · Cited by 29
The discourse of distress: a narrative analysis of emergency calls to 911
Imbens-Bailey, A. L., & McCabe, A. (2000) · Language & Communication · doi:10.1016/s0271-5309(99)00025-7 · Cited by 27
Tollinton, L., Metcalf, A. M., & Velupillai, S. U. (2020) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2020.104179 · Cited by 27
Interactional misalignment in the UK NHS 111 healthcare telephone triage service
Morgan, J. I., & Muskett, T. (2019) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2019.104030 · Cited by 22
Eriksson, I., Ek, K., Jansson, S., Sjöström, U., & Larsson, M. (2019) · Nursing Open · doi:10.1002/nop2.264 · Cited by 20
Son, G., Kwon, S., & Park, N. (2019) · Symmetry · doi:10.3390/sym11040525 · Cited by 20
Symbolic and Statistical Learning Approaches to Speech Summarization: A Scoping Review
Rezazadegan, D., Berkovsky, S., Quiroz, J. C., Kocaballi, A. B., Wang, Y., Laranjo, L., & Coiera, E. W. (2021) · Computer Speech & Language · doi:10.1016/j.csl.2021.101305 · Cited by 19
Salmi, S., Mérelle, S. Y. M., Gilissen, R., & Brinkman, W. (2020) · Journal of Medical Internet Research · doi:10.2196/21690 · Cited by 17
Anthony, T., Mishra, A. K., Stassen, W., & Son, J. (2021) · Healthcare · doi:10.3390/healthcare9091107 · Cited by 17
Commentary: human-centred AI: the new zeitgeist
Rogers, Y. (2022) · Human-Computer Interaction · doi:10.1080/07370024.2021.1976643 · Cited by 16
What does it mean to provide decision support to a responsible and competent expert?
Richard, A., Mayag, B., Talbot, F., Tsoukiàs, A., & Meinard, Y. (2020) · EURO Journal on Decision Processes · doi:10.1007/s40070-020-00116-7 · Cited by 15
Ahmad, J., Sajjad, M., Rho, S., Kwon, S., Lee, M. Y., & Baik, S. W. (2016) · Multimedia Tools and Applications · doi:10.1007/s11042-016-4041-7 · Cited by 14
Fear emotion classification in speech by acoustic and behavioral cues
Yoon, S.-A., Son, G., & Kwon, S. (2018) · Multimedia Tools and Applications · doi:10.1007/s11042-018-6329-2 · Cited by 11
A call triage support system for emergency medical service using multiple random forests
Yasuda, T., Yamada, Y., Hamatsu, F., & Hamagami, T. (2017) · IEEJ Transactions on Electrical and Electronic Engineering · doi:10.1002/tee.22567 · Cited by 9
Tavi, L. (2018) · International Journal of Speech Technology · doi:10.1007/s10772-018-09574-6 · Cited by 9
Automatic Speech Summarisation: A Scoping ReviewPreprint
Rezazadegan, D., Berkovsky, S., Quiroz, J. C., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.11897 · Cited by 9
Using natural language processing in facilitating pre-hospital telephone triage of emergency calls
Gormley, K. J., Lockhart, K., & Isaac, J. (2022) · British Paramedic Journal · doi:10.29045/14784726.2022.09.7.2.31 · Cited by 7
Margaret McCartney: General practice can’t just exclude sick people
McCartney, M. (2017) · BMJ · doi:10.1136/bmj.j5190 · Cited by 5
High Accuracy of Call Triage Decision by Bayesian Network
Yunoki, S., Hamagami, T., Oshige, K., Kawakami, C., & Suzuki, N. (2013) · Electronics and Communications in Japan · doi:10.1002/ecj.11439 · Cited by 4
Ding, J., Freeman, M., & Hasija, S. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3806478 · Cited by 3
Research works printed but not linked: 6
- 10.7551/mitpress · doi:10.7551/mitpress
- 10.1155/2017 · doi:10.1155/2017
- 10.1093/jamia · doi:10.1093/jamia
- 10.1093/jamia · doi:10.1093/jamia
- Carlo Reverberi, Tommaso Rigon, Aldo Solari, Cesare Hassan, Paolo Cherubini, and Andrea Cherubini. 2022. Experimental evidence of effective human–AI collaboration in medical decision-making. Scientific reports 12, 1 (2022), 14952.
- Yue You and Xinning Gui. 2021. Self-Diagnosis through AI-enabled Chatbot-based Symptom Checkers: User Experiences and Design Considerations. AMIA Annual Symposium Proceedings 2020 (Jan 2021), 1354–1363.
Cited by
Zhang, S., Wang, H., & Yi, X. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757594 · Cited by 36
"It depends": Configuring AI to Improve Clinical Usefulness Across Contexts
Zając, H. D., Ribeiro, J., Ingala, S., Gentile, S., Wanjohi, R., Gitau, S. N., Carlsen, J. F., Nielsen, M. B., & Andersen, T. O. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3660707 · Cited by 23
Zając, H. D., Andersen, T. O., Kwasa, E., Wanjohi, R., Onyinkwa, M. K., Mwaniki, E. K., Gitau, S. N., Yaseen, S. S., Carlsen, J. F., Fraccaro, M., et al. (2025) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3715115 · Cited by 22
Islam, S., Shahriyar, R., Agarwala, A., Zaman, M., Ahamed, S. A., Rahman, R., Chowdhury, M. H., Sarker, F., & Mamun, K. A. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-02864-5 · Cited by 19
How Can I Signal You To Trust Me: Investigating AI Trust Signalling in Clinical Self-Assessments
Kollerup, N. K., Wester, J., Skov, M. B., & van Berkel, N. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3661612 · Cited by 15
Metadata
- Authors
- Berge, A., Guribye, F., Fotland, S.-L. S., Fonnes, G., Johansen, I. H., & Trattner, C.
- Affiliations
- NORCE Research AS (NO)
- University of Bergen (NO)
- ORCID
- Venue
- Proceedings of the 2023 ACM Designing Interactive Systems Conference
- Publisher
- ACM
- Published
- 2023-07-10
- Pages
- 1339-1352
- OpenAlex
- W4383682600
- Semantic Scholar
- f6b1d9184594c3a0b150d00c14e79c47d42ba1d1
- Cited by
- 20
- References
- 68
- Citations by year
- 2026: 7 · 2025: 8 · 2024: 5
- FWCI
- 3.61
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Designing Interactive Systems Conference (publishedVersion, cc-by)
- NORA - Norwegian Open Research Archives (submittedVersion, cc-by)
- Funders
- Norges Forskningsråd (310468)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Telemedicine and Telehealth Implementation
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Keywords
- telephone triage, emergency medical calls, symptom checkers, human-AI interaction, clinical decision support, vulnerable patients, co-design, role play
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.