Automation Error Type and Methods of Communicating Automation Reliability Affect Trust and Performance: An Empirical Study in the Cyber Domain
The DOI matches its Crossref record; 59 of the 60 references it cites were found in a registry or an index.
Abstract
Antiphishing aid systems, among other automated systems, are not perfectly reliable. Automated systems can make errors, thereby resulting in false alarms or misses. An automated system's capabilities need to be communicated to the users to maintain proper user trust. System capabilities can be learned through an explicit description or from experience. Using a phishing-detection system as a testbed in this article, we systematically varied automation error type and the method of communicating system reliability in a factorial design and measured their effects on human performance and trust in the automation. Participants were asked to classify emails as legitimate or phishing with assistance from the phishing-detection system. The results from 510 participants suggest that learning through experience with feedback improved trust calibration for both objective and subjective trust measures in most conditions. Moreover, false alarms lowered trust more than misses for both unreliable and reliable systems, and false alarms turned out to be beneficial for proper trust calibration when using unreliable systems. Design implications of the results include using feedback whenever possible and choosing false alarms over misses for unreliable systems.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Many argued that such behavioral measures displayed a more objective trust measure than subjective self-reports (see, e.g., [14]).”
From Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions (2023) · cites this paper as [14]
In the citing paper
“For example, Chen et al. [14] found that false alarms reduced perceived trustworthiness more than misses.”
From Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions (2023) · cites this paper as [14]
In the citing paper
“Yet, others found positive efects of feedback [14,15,94].”
From Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions (2023) · cites this paper as [14]
Citations
Cites
(1966) · Medical Entomology and Zoology · Cited by 10,293
Signal detection theory and psychophysics
Ingleby, J. D. (1967) · Journal of Sound and Vibration · doi:10.1016/0022-460x(67)90197-6 · Cited by 8,509
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605
Humans and automation: Use, misuse, disuse, abuse
Parasuraman, R., & Riley, V. (1997) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872097778543886 · Cited by 4,554
A model for types and levels of human interaction with automation
Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/3468.844354 · Cited by 4,195
Trust in Automation: Integrating Empirical Evidence on Factors That Influence Trust
Hoff, K. A., & Bashir, M. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814547570 · Cited by 2,817
Dhamija, R., Tygar, J. D., & Hearst, M. (2006) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1124772.1124861 · Cited by 1,376
Do multiple outcome measures require p-value adjustment?
Feise, R. J. (2002) · BMC Medical Research Methodology · doi:10.1186/1471-2288-2-8 · Cited by 1,299
The role of trust in automation reliance
Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G., & Beck, H. P. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00038-7 · Cited by 1,276
The viability of crowdsourcing for survey research
Behrend, T. S., Sharek, D. J., Meade, A. W., & Wiebe, E. N. (2011) · Behavior Research Methods · doi:10.3758/s13428-011-0081-0 · Cited by 1,018
Research works printed but not linked: 1
- parrish, 2009, A personality based model for determining susceptibility to phishing attacks, Little Rock University of Arkansas, 285
Cited by
Mehrotra, S., Degachi, C., Vereschak, O., Jonker, C. M., & Tielman, M. L. (2024) · ACM Journal on Responsible Computing · doi:10.1145/3696449 · Cited by 144
“; Our set of future research directions results from a structured summary of our analysis based on the implications of the approaches (definitions, methods, tasks and measures) adopted by the authors to foster appropriate trust in human-AI interaction.”
Wischnewski, M., Krämer, N., & Müller, E. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581197 · Cited by 146
“Yet, others found positive efects of feedback [14, 15, 94].”
BackgroundPoornikoo, M., Gyldensten, W., Vesin, B., & Øvergård, K. I. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2399439 · Cited by 36
“…findings from the experimental study on Maritime Autonomous Surface Ships (MASS) align with the established research (eg, Chavaillaz et al., 2016; Chen et al., 2021; Hussein et al., 2020; Lee & Moray, 1994; Moray et al., 2000; Yang et al., 2016) suggesting that trust in automation is closely…”
Automation Bias and Complacency in Security Operation Centers
Tilbury, J., & Flowerday, S. (2024) · Computers · doi:10.3390/computers13070165 · Cited by 34
“Ten of the 13 represent core papers, as they clearly identify the implications that automation has on security analysts, ranging from bias, complacency, mis-and disuse, human-machine trust, and trust intentions [3,6,14,28,36,37,39,56,57].”
BackgroundMethodInfluentialHumans and Automation: Augmenting Security Operation Centers
Tilbury, J., & Flowerday, S. (2024) · Journal of Cybersecurity and Privacy · doi:10.3390/jcp4030020 · Cited by 30
“The prevalence of automated cybersecurity tools should provide ample motivation toward studying human-automation interaction, yet research evaluating trust in these tools appears limited [9].”
BackgroundSiddiqui, M., Kahandawa, G. C., & Hewawasam, H. S. (2023) · IEEE International Conference on Mechatronics (ICM) · doi:10.1109/icm54990.2023.10101971 · Cited by 28
“These systems can also make an error which can impact the reliability of the system [13].”
BackgroundHow Humans Comply With a (Potentially) Faulty Robot: Effects of Multidimensional Transparency
Simon, L., Guerin, C., Rauffet, P., Chauvin, C., & Martin, É. (2023) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2023.3273773 · Cited by 21
“In Chen et al. [11], compliance is measured in terms of the agreement rate with the decision support system.”
BackgroundZhou, Y., Cui, X., Qu, W., & Ge, Y. (2022) · Applied Ergonomics · doi:10.1016/j.apergo.2022.103754 · Cited by 22
Calibrating Reliance on Automated Advice: Transparency and Trust Calibration Feedback
Tatasciore, M., & Loft, S. (2025) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2025.2487861 · Cited by 20
“In contrast to most prior studies (e.g., Chen et al., 2021; Lu & Sarter, 2019), trust calibration feedback in the current study provided information about both human and automation performance (Okamura & Yamada, 2020).”
Følstad, A., Law, E. L.-C., & van As, N. (2024) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3690383 · Cited by 17
Sun, X., Ma, R., Wei, S., Cesar, P. S., Bosch, J. A., & Ali, A. E. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103714 · Cited by 17
“Traditional research on trust perception has heavily relied on self-reported assessments; however, many studies [31, 76] suggest behavioral and physiological signals may add a relevant layer of information.”
Dynamic calibration of trust and trustworthiness in AI-enabled systems
Liebherr, M., Enkel, E., Law, E. L., Mousavi, M. R., Sammartino, M., & Sieberg, P. M. (2026) · International Journal on Software Tools for Technology Transfer · doi:10.1007/s10009-026-00840-6 · Cited by 17
“Additionally, the nature of the error leading to reliability changes could significantly impact trust calibration; for example, false alarms were observed to diminish trust more than misses [21].”
Radensky, M., Séguin, J. A., Lim, J. S., Olson, K., & Geiger, R. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594043 · Cited by 14
Continuous Error Timing in Automation: The Peak-End Effect on Human-Automation Trust
Wang, K., Lü, J., Ruan, S., & Qi, Y. (2023) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2023.2223954 · Cited by 13
“Most previous studies have focused mainly on the overall reliability and ignored the effect of local reliability variation on trust, especially when the overall reliability is the same (Chavaillaz et al., 2016; Chen et al., 2021; Dzindolet et al., 2003; Sanchez et al., 2004).”
BackgroundXu, Z., Wang, G., Zhai, S., & Liu, P. (2025) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2023.2176986 · Cited by 12
“In the literatures of human – automation interaction (HAI), human – computer interaction, and human factors engineering, dozens of studies offer evidence of the negative influences of unreliable and imperfect automation and of automation failures (e.g., Chen et al., 2021; Wickens & Dixon, 2007).”
Wang, K., Hou, W., Ma, H., & Hong, L. (2024) · Sensors · doi:10.3390/s24247946 · Cited by 13
Phishing in Social Media: Investigating Training Techniques on Instagram Shop
Garcia, K. R., Ammons, J. G., Xu, X., & Chen, J. (2023) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/21695067231192588 · Cited by 11
“In their earliest form, phishing attacks often occurred primarily via emails with the intent of luring users to fictitious or malicious websites (Chen et al., 2018, 2021; Dunham, 2004; Kirda & Kruegel, 2006; Williams et al., 2017).”
BackgroundSelf-citationBuilding And Measuring Trust In Human-Machine Systems
Dizaji, L. G., & Hu, Y. (2021) · IEEE International Conference on Autonomous Systems (ICAS) · doi:10.1109/icas49788.2021.9551131 · Cited by 8
Sun, X., Wei, S., Bosch, J. A., Echizen, I., Sugawara, S., & Ali, A. E. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798613 · Cited by 8
“Several prior works in explainable AI (XAI) [1] show that AI’s explanations can influence users’ trust and decision-making [24, 32], reliance [26], and mental models [4], which lead to decision biases [4] and increase the acceptance of incorrect AI advice [7].”
Zhang, Y. “., Chen, J., Sun, L. Q., Hu, B., Politowicz, M. S., & Chancey, E. T. (2024) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/10711813241274652 · Cited by 7
Self-citationCan a conventional email phishing nudge help fight SMiShing attacks?
Edwards, M. E., Chen, J., & Still, J. D. (2025) · Computer Standards & Interfaces · doi:10.1016/j.csi.2025.104031 · Cited by 6
Self-citationSimon, L., Rauffet, P., Guerin, C., & Seguin, C. (2022) · AHFE international · doi:10.54941/ahfe1001602 · Cited by 5
Bahnsen, K. L., Dischinger, E., & Grundgeiger, T. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713685 · Cited by 4
“The fact that we did not include false positive cues (i.e., AR cues that pointed towards a wrong re-sumption point) probably contributed to the boosted performance in cue trials, as false alarms harm trust in automation more than misses [14].”
Greenlee, E. T., Funke, G. J., Hess, L. J., & Vidulich, M. A. (2022) · Applied Ergonomics · doi:10.1016/j.apergo.2021.103677 · Cited by 4
Simon, L., Rauffet, P., & Guerin, C. (2026) · Cognition Technology & Work · doi:10.1007/s10111-026-00890-5 · Cited by 2
Exploring Team Error Management in Human Autonomy Teams During Automation Errors
Zhang, R., Gorman, J. C., & Cooke, N. J. (2024) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/10711813241276486 · Cited by 1
“How automation errors are communicated plays a vital role in user performance (Chen et al., 2021).”
Human-Centered Artificial Intelligence in Cybersecurity
Chen, J., Garcia, K. R., Zhang, Y. “., Dudley, L. E., & Warren, A. D. (2025) · Springer eBooks · doi:10.1007/978-981-97-8440-0_99-1 · Cited by 1
Self-citationHuman-Centered Artificial Intelligence in Cybersecurity
Chen, J., Garcia, K. R., Zhang, Y. “., Dudley, L. E., & Warren, A. D. (2026) · Springer eBooks · doi:10.1007/978-981-95-3658-0_99 · Cited by 1
Self-citationMisses or False Alarms? How Error Type Affects Confidence In Automation Reliability Estimates
Guzman-Bonilla, E., & Patton, C. E. (2024) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/10711813241275093 · Cited by 1
“In the phishing domain, Chen et al. (2021) found that false alarms lowered trust behaviors more than misses, but there was no difference in subjective trust measures.”
Situation Awareness and Trust Methods for Improving Human-Robot Team Performance
Ali, A. (2025) · Deep Blue (University of Michigan) · doi:10.7302/25612 · Cited by 1
Dudley, L. E., Chen, J. M., DeLucia, P. R., & Liu, H. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6482026 · Cited by 1
Self-citationUsing anthropomorphism, transparency, and feedback to improve phishing detection aids
Mishler, S., Parker, C., & Chen, J. (2024) · Theoretical Issues in Ergonomics Science · doi:10.1080/1463922x.2024.2373457
“How the user perceives these characteristics of the automated aid can affect the user’s trust and, subsequently, their use of the aid (Chen et al. 2018, Chen, Mishler, and Hu 2021; Schaefer et al. 2016).”
InfluentialSelf-citationExamining the Effectiveness of Speech and Earcon Alerts for Aiding Phishing Email Detection
Zhang, Y. “., Berger, L., & Chen, J. (2024) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/10711813241275518
“Based on correction suggestions, 0.5 was added to the number of hits, and 1 was added to the number of signals and noise to accommodate either hit rate or false alarm rate being 0 or 1 (Chen et al., 2021; Hautus, 1995).”
InfluentialSelf-citationMao, T., Garcia, K. R., & Chen, J. (2024) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/10711813241276493
“Different error types can alter drivers’ perception and interaction with automated systems, such as perceived trustworthiness (Guznov et al., 2016), trust level (Chen et al., 2021), and their compliance-reliance behaviors (Azevedo-Sa et al., 2020b).”
Self-citationInvestigating Training and Priming to Combat Phishing on Instagram Shop
Garcia, K. R., & Chen, J. (2024) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/10711813241261384
“Numerous methods to combat email phishing efficiency have been tested, such as training or educating the user, using automated aids, and priming the user to think about their security (Baki & Verma, 2022; Chen et al., 2018, 2021; Chong et al., 2018; Kumaraguru et al., 2007; Yang et al., 2015).”
Self-citationHuman Perception of AI Capabilities at Classifying Perturbed Roadway Signs
Garcia, K. R., Chen, J., Xiao, Y., Mishler, S., Wang, C., & Hu, B. (2025) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2025.3573173
“It is important for users to understand the capabilities and limitations associated with automated systems, which are prone to making errors [27], [28].”
Self-citationScibelli, M., Papaux, K. G., Valenti, J., & Kush, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.11979
Siddiqui, M. (2023) · FedUni ResearchOnline (Federation University Australia)
Sharma, V., Nalla, U. C., Surage, L., Agasam, N., Glaser, S., Ghimire, R., Doppalapudi, C., Olujembola, O., & Carstens, D. S. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6301155
DPMLF: A Novel Phishing Detection Model Integrating URL Character and HTML Semantic Deep Features
Ma, Z.-J., Liu, X., Meng, L., & Zhu, E. (2026) · 電腦學刊 · doi:10.63367/199115992026023701006
Ryu, K., Ghang, H. (., Chon, S., Lee, K., & Kang, Y. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798817
Sun, X., Pan, T., Wang, Y., Wei, S., Bosch, J. A., Echizen, I., Ali, A. E., & Sugawara, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.25489
Chiantera, E., Arciulo, L., & Nocera, F. D. (2026) · Safety · doi:10.3390/safety12040087
“Experience, feedback, false positives, and false negatives can foster both proportionate trust and forms of distrust or overreliance [5,6,14].”
Ouyang, L., Jahng, J., & Woo, S. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2705163
When Trust Bends but Does Not Break: Trust During Algorithm-Supported Decision-Making
Sandia National Laboratories (SNL-NM), A., Sanchez, D. N., (NNSA), U. N. N. S. A., Winokur, E. J., Lancaster, C., Villarreal, R. T., Bradley, S., Schutte, P., Ingram, J. B., Bruner, T., et al. (2026) · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · doi:10.2172/17315896
Shin, J., Carley, L. R., & Carley, K. M. (2026) · SIMULATION · doi:10.1177/00375497261490937
Metadata
- Authors
- Chen, J., Mishler, S., & Hu, B.
- Affiliations
- Old Dominion University (US)
- Venue
- IEEE Transactions on Human-Machine Systems
- Published
- 2021-10
- Volume
- 51
- Issue
- 5
- Pages
- 463-473
- ISSN
- 2168-2291 (ISSN-L)
- 2168-2305
- OpenAlex
- W3130493744
- Cited by
- 40
- References
- 60
- Citations by year
- 2026: 10 · 2025: 6 · 2024: 13 · 2023: 7 · 2022: 3 · 2021: 1
- FWCI
- 5.88
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Human-Machine Systems (publishedVersion)
- Funders
- National Science Foundation (1760347)
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Keywords
- trust in automation, false alarms, Miss, phishing detection, trust calibration, feedback learning, human performance, email classification, human-automation interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.