The DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.
The Effect of Workload and Task Priority on Multitasking Performance and Reliance on Level 1 Explainable AI (XAI) Use
Abstract
Objective: This study investigates the effects of workload and task priority on multitasking performance and reliance on Level 1 Explainable Artificial Intelligence (XAI) systems in high-stakes decision environments. Background: Operators in critical settings manage multiple tasks under varying levels of workload and priority, potentially leading to performance degradation. XAI offers opportunities to support decision making by providing insights into AI’s reasoning, yet its adoption and effectiveness in multitasking scenarios remain underexplored. Method: Thirty participants engaged in a simulated multitasking environment, involving UAV command and control tasks, with the assistance of a Level 1 (i.e., basic perceptual information) XAI system on one of the tasks. The study utilized a within-subjects experimental design, manipulating workload (low, medium, and high) and AI-supported-task priority (low and high) across six conditions. Participants’ accuracy, use of automatic rerouting, AI miss detection, false alert identification, and use of AI explanations were measured and analyzed across the different experimental conditions. Results: Workload significantly hindered performance on the AI-assisted task and increased reliance on the AI system especially when the AI-assisted task was given low priority. The use of AI explanations was significantly affected by task priority only. Conclusion: An increase in workload led to proper offloading by relying on the AI’s alerts, but it also led to a lower rate of alert verification despite the alert feature’s high false alert rate. Application: The findings from the present work help inform AI system designers on how to design their systems for high-stakes environments such that reliance on AI is properly calibrated.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Generative adversarial networks
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2020) · Communications of the ACM · doi:10.1145/3422622 · Cited by 18,384
Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research
(1988) · Advances in Psychology · doi:10.1016/s0166-4115(08)62386-9 · Cited by 15,003
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,881
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 1×
“As trust is often de fi ned as the belief that an agent will help achieve an individual ’ s goals in situations characterized by uncertainty and vulnerability (Lee & See, 2004), this over-reliance could have been due to over-trust.”
Key challenges for delivering clinical impact with artificial intelligence
Kelly, C. J., Karthikesalingam, A., Suleyman, M., Corrado, G., & King, D. (2019) · BMC Medicine · doi:10.1186/s12916-019-1426-2 · Cited by 3,136
Human trust in artificial intelligence: Review of empirical research
Glikson, E., & Woolley, A. W. (2020) · Academy of Management Annals · doi:10.5465/annals.2018.0057 · Cited by 2,921
Trust in Automation: Integrating Empirical Evidence on Factors That Influence Trust
Hoff, K. A., & Bashir, M. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814547570 · Cited by 2,817
Complacency and Bias in Human Use of Automation: An Attentional Integration
Parasuraman, R., & Manzey, D. H. (2010) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720810376055 · Cited by 1,710 · cited 1×
“…and limitations that can result in under-reliance , where operators choose to exclude the AI from their work fl ow, or overreliance , where operators blindly rely on its decisions despite its susceptibility to errors (Buçinca et al., 2021; Parasuraman & Manzey, 2010; Vasconcelos et al., 2023).”
DARPA's Explainable Artificial Intelligence Program
Gunning, D., & Aha, D. W. (2019) · AI Magazine · doi:10.1609/aimag.v40i2.2850 · Cited by 1,360
Buçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021) · arXiv · doi:10.48550/arxiv.2102.09692 · Cited by 1,124 · cited 1×
“…and limitations that can result in under-reliance , where operators choose to exclude the AI from their work fl ow, or overreliance , where operators blindly rely on its decisions despite its susceptibility to errors (Buçinca et al., 2021; Parasuraman & Manzey, 2010; Vasconcelos et al., 2023).”
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems
Mohseni, S., Zarei, N., & Ragan, E. D. (2021) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3387166 · Cited by 785
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems
Mohseni, S., Zarei, N., & Ragan, E. D. (2018) · arXiv · doi:10.48550/arxiv.1811.11839 · Cited by 783
Threaded cognition: An integrated theory of concurrent multitasking.
Salvucci, D. D., & Taatgen, N. A. (2008) · Psychological Review · doi:10.1037/0033-295x.115.1.101 · Cited by 667
Human–Agent Teaming for Multirobot Control: A Review of Human Factors Issues
Chen, J. Y. C., & Barnes, M. J. (2014) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2013.2293535 · Cited by 522
Elwyn, G., Scholl, I., Tietbohl, C., Mann, M., Edwards, A. G., Clay, C., Légaré, F., Weijden, T. V. D., Lewis, C. L., Wexler, R. M., et al. (2013) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-13-s2-s14 · Cited by 502
Explanations Can Reduce Overreliance on AI Systems During Decision-Making
Vasconcelos, H., Jörke, M., Grunde-McLaughlin, M., Gerstenberg, T., Bernstein, M. S., & Krishna, R. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579605 · Cited by 402 · cited 6×
“Here, an explanation is de fi ned as a description of how the AI decision was an outcome of a given context (Vasconcelos et al., 2023).”
InfluentialSituation awareness-based agent transparency and human-autonomy teaming effectiveness
Chen, J. Y. C., Lakhmani, S. G., Stowers, K., Selkowitz, A. R., Wright, J. L., & Barnes, M. (2018) · Theoretical Issues in Ergonomics Science · doi:10.1080/1463922x.2017.1315750 · Cited by 351
Automation and Human Performance: Theory and Applications
(2018) · Routledge eBooks · doi:10.1201/9781315137957 · Cited by 307
Ng, K. K. H., Chen, C.-H., Lee, C. K. M., Jiao, J. (., & Yang, Z.-X. (2021) · Advanced Engineering Informatics · doi:10.1016/j.aei.2021.101246 · Cited by 261
Twenty years of load theory—Where are we now, and where should we go next?
Murphy, G., Groeger, J. A., & Greene, C. M. (2016) · Psychonomic Bulletin & Review · doi:10.3758/s13423-015-0982-5 · Cited by 258
Translational cognition for decision support in critical care environments: A review
Patel, V. L., Zhang, J., Yoskowitz, N. A., Green, R., & Sayan, O. R. (2008) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2008.01.013 · Cited by 145
Can we do better explanations? A proposal of user-centered explainable AI
Ribera, M., & Lapedriza, À. (2019) · IUI Workshops · Cited by 133
XAI Systems Evaluation: A Review of Human and Computer-Centred Methods
Lopes, P., Silva, E., Braga, C., Oliveira, T., & Rosado, L. (2022) · Applied Sciences · doi:10.3390/app12199423 · Cited by 112 · cited 1×
“However, XAI cannot be viewed in the same manner as AI as its goals, expectations, and evaluation methodologies vary by discipline and application (Lopes et al., 2022).”
Why black box machine learning should be avoided for high-stakes decisions, in brief
Rudin, C. D. (2022) · Nature Reviews Methods Primers · doi:10.1038/s43586-022-00172-0 · Cited by 112
The Last Mile: Where Artificial Intelligence Meets Reality
Coiera, E. (2019) · Journal of Medical Internet Research · doi:10.2196/16323 · Cited by 102
Vigilant Spirit Control Station (VSCS): The Face of COUNTER
Feitshans, G., Rowe, A., Davis, J., Holland, M., & Berger, L. (2008) · AIAA Guidance, Navigation, and Control Conference and Exhibit · doi:10.2514/6.2008-6309 · Cited by 83
Automation trust and attention allocation in multitasking workspace
Karpinsky, N. D., Chancey, E. T., Palmer, D. B., & Yamani, Y. (2018) · Applied Ergonomics · doi:10.1016/j.apergo.2018.03.008 · Cited by 74
Yau, K. A., Lee, H.-J., Chong, Y.-W., Ling, M. H., Syed, A. R., Wu, C., & Goh, H. G. (2021) · IEEE Access · doi:10.1109/access.2021.3115494 · Cited by 74
Time Sharing Between Robotics and Process Control
Wickens, C. D., Gutzwiller, R. S., Vieane, A. Z., Clegg, B. A., Sebok, A. L., & Janes, J. (2016) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720815622761 · Cited by 63
Choudhury, A. (2022) · JMIR Human Factors · doi:10.2196/35421 · Cited by 56
Automation trust increases under high-workload multitasking scenarios involving risk
Sato, T., Yamani, Y., Liechty, M., & Chancey, E. T. (2019) · Cognition Technology & Work · doi:10.1007/s10111-019-00580-5 · Cited by 52
He, X., Hong, Y., Xi, Z., & Zhang, Y. (2022) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2022.2095093 · Cited by 51
Ouma, J. A., Chappelle, W. L., & Salinas, A. (2011) · Cited by 34
Onnasch, L., Ruff, S., & Manzey, D. (2014) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2014.05.001 · Cited by 34
15. Workload and Automation Reliability in Unmanned Air Vehicles
Wickens, C., Dixon, S. R., & Ambinder, M. S. (2006) · Advances in human performance and cognitive engineering research · doi:10.1016/s1479-3601(05)07015-3 · Cited by 23
Does It MultiMatch? What Scanpath Comparison Tells us About Task Performance in Teams
Iskandarani, M. E., Atweh, J. A., McGarry, S. P. D., Riggs, S. L., & Moacdieh, N. M. (2023) · Journal of Cognitive Engineering and Decision Making · doi:10.1177/15553434231171484 · Cited by 19
Self-citationTransitions Between Low and High Levels of Mental Workload can Improve Multitasking Performance
Devlin, S. P., Moacdieh, N. M., Wickens, C. D., & Riggs, S. L. (2020) · IISE Transactions on Occupational Ergonomics and Human Factors · doi:10.1080/24725838.2020.1770898 · Cited by 17
Self-citationSanneman, L., Tucker, M., & Shah, J. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3659032 · Cited by 17
Information Systems and Neuroscience
(2021) · Lecture notes in information systems and organisation · doi:10.1007/978-3-030-88900-5 · Cited by 16
Impact Of Explainable AI On Cognitive Load: Insights From An Empirical StudyPreprint
Herm, L.-V. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.08861 · Cited by 16
Zhang, M. Y., & Yang, X. J. (2017) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931213601932 · Cited by 12
Guznov, S., Nelson, A., Lyons, J., & Dycus, D. (2015) · Communications in Computer and Information Science · doi:10.1007/978-3-319-21383-5_103 · Cited by 11
Sato, T., Islam, S., Still, J. D., Scerbo, M. W., & Yamani, Y. (2022) · Cognition Technology & Work · doi:10.1007/s10111-022-00717-z · Cited by 11
Trust in Clinical AI: Expanding the Unit of Analysis
Browne, J., Bakker, S., Yu, B., Lloyd, P. A., & Allouch, S. B. (2022) · Frontiers in artificial intelligence and applications · doi:10.3233/faia220192 · Cited by 10
Trusting AI: Integrating Artificial Intelligence into the Army's Professional Expert Knowledge
Pfaff, C. A., Lowrance, C. J., Washburn, B. M., Carey, B. A., & Evans, C. V. (2023) · University of North Texas Digital Library (University of North Texas) · Cited by 3
Research works printed but not linked: 3
- 10.1145/3449287 · doi:10.1145/3449287 · Paper page
- Shattuck L. G. (2015). Transitioning to autonomy: A human systems integration perspective. In Transitioning to autonomy: Changes in the role of humans in air transportation (pp. 10–12). NACC.
- Warm J. S., Dember W. N., Hancock P. A. (2018). Vigilance and workload in automated systems. In Automation and human performance (pp. 183–200). CRC Press.
Cited by
Exploring automation bias in human–AI collaboration: a review and implications for explainable AI
Romeo, G., & Conti, D. (2026) · AI & SOCIETY · doi:10.1007/s00146-025-02422-7 · Cited by 209
“Occurrences of overreliance may also be understood as strategies to reduce cognitive load, rather than as mere manifestations of misplaced trust (Alami et al. 2025).”
Influential
Metadata
- Authors
- Alami, J., El Iskandarani, M., & Riggs, S. L.
- Venue
- Human Factors: The Journal of the Human Factors and Ergonomics Society
- Publisher
- SAGE Publications
- Published
- 2025-03-12
- Volume
- 67
- Issue
- 9
- Pages
- 897-915
- ISSN
- 0018-7208
- 1547-8181
- Cited by
- 9
- References
- 45
- Language
- en
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Funders
- Division of Information and Intelligent Systems (2008680)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
