Automation Error Type and Methods of Communicating Automation Reliability Affect Trust and Performance: An Empirical Study in the Cyber Domain
The DOI matches its Crossref record; 59 of the 60 references it cites were found in a registry or an index.
Abstract
Antiphishing aid systems, among other automated systems, are not perfectly reliable. Automated systems can make errors, thereby resulting in false alarms or misses. An automated system's capabilities need to be communicated to the users to maintain proper user trust. System capabilities can be learned through an explicit description or from experience. Using a phishing-detection system as a testbed in this article, we systematically varied automation error type and the method of communicating system reliability in a factorial design and measured their effects on human performance and trust in the automation. Participants were asked to classify emails as legitimate or phishing with assistance from the phishing-detection system. The results from 510 participants suggest that learning through experience with feedback improved trust calibration for both objective and subjective trust measures in most conditions. Moreover, false alarms lowered trust more than misses for both unreliable and reliable systems, and false alarms turned out to be beneficial for proper trust calibration when using unreliable systems. Design implications of the results include using feedback whenever possible and choosing false alarms over misses for unreliable systems.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Many argued that such behavioral measures displayed a more objective trust measure than subjective self-reports (see, e.g., [14]).”
From Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions (2023) · cites this paper as [14]
In the citing paper
“For example, Chen et al. [14] found that false alarms reduced perceived trustworthiness more than misses.”
From Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions (2023) · cites this paper as [14]
In the citing paper
“Yet, others found positive efects of feedback [14,15,94].”
From Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions (2023) · cites this paper as [14]
Citations
Cites
(1966) · Medical Entomology and Zoology · Cited by 10,293
Signal detection theory and psychophysics
Ingleby, J. D. (1967) · Journal of Sound and Vibration · doi:10.1016/0022-460x(67)90197-6 · Cited by 8,509
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605
Humans and automation: Use, misuse, disuse, abuse
Parasuraman, R., & Riley, V. (1997) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872097778543886 · Cited by 4,554
A model for types and levels of human interaction with automation
Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/3468.844354 · Cited by 4,195
Trust in Automation: Integrating Empirical Evidence on Factors That Influence Trust
Hoff, K. A., & Bashir, M. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814547570 · Cited by 2,817
Dhamija, R., Tygar, J. D., & Hearst, M. (2006) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1124772.1124861 · Cited by 1,376
Do multiple outcome measures require p-value adjustment?
Feise, R. J. (2002) · BMC Medical Research Methodology · doi:10.1186/1471-2288-2-8 · Cited by 1,299
The role of trust in automation reliance
Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G., & Beck, H. P. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00038-7 · Cited by 1,276
The viability of crowdsourcing for survey research
Behrend, T. S., Sharek, D. J., Meade, A. W., & Wiebe, E. N. (2011) · Behavior Research Methods · doi:10.3758/s13428-011-0081-0 · Cited by 1,018
Research works printed but not linked: 1
- parrish, 2009, A personality based model for determining susceptibility to phishing attacks, Little Rock University of Arkansas, 285
Cited by
Mehrotra, S., Degachi, C., Vereschak, O., Jonker, C. M., & Tielman, M. L. (2024) · ACM Journal on Responsible Computing · doi:10.1145/3696449 · Cited by 256
“; Our set of future research directions results from a structured summary of our analysis based on the implications of the approaches (definitions, methods, tasks and measures) adopted by the authors to foster appropriate trust in human-AI interaction.”
Wischnewski, M., Krämer, N., & Müller, E. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581197 · Cited by 146
“Yet, others found positive efects of feedback [14, 15, 94].”
BackgroundPoornikoo, M., Gyldensten, W., Vesin, B., & Øvergård, K. I. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2399439 · Cited by 36
“…findings from the experimental study on Maritime Autonomous Surface Ships (MASS) align with the established research (eg, Chavaillaz et al., 2016; Chen et al., 2021; Hussein et al., 2020; Lee & Moray, 1994; Moray et al., 2000; Yang et al., 2016) suggesting that trust in automation is closely…”
Automation Bias and Complacency in Security Operation Centers
Tilbury, J., & Flowerday, S. (2024) · Computers · doi:10.3390/computers13070165 · Cited by 34
“Ten of the 13 represent core papers, as they clearly identify the implications that automation has on security analysts, ranging from bias, complacency, mis-and disuse, human-machine trust, and trust intentions [3,6,14,28,36,37,39,56,57].”
BackgroundMethodInfluentialHumans and Automation: Augmenting Security Operation Centers
Tilbury, J., & Flowerday, S. (2024) · Journal of Cybersecurity and Privacy · doi:10.3390/jcp4030020 · Cited by 30
“The prevalence of automated cybersecurity tools should provide ample motivation toward studying human-automation interaction, yet research evaluating trust in these tools appears limited [9].”
Background
Metadata
- Authors
- Chen, J., Mishler, S., & Hu, B.
- Affiliations
- Old Dominion University (US)
- Venue
- IEEE Transactions on Human-Machine Systems
- Published
- 2021-10
- Volume
- 51
- Issue
- 5
- Pages
- 463-473
- ISSN
- 2168-2291 (ISSN-L)
- 2168-2305
- OpenAlex
- W3130493744
- Cited by
- 40
- References
- 60
- Citations by year
- 2026: 10 · 2025: 6 · 2024: 13 · 2023: 7 · 2022: 3 · 2021: 1
- FWCI
- 5.88
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Human-Machine Systems (publishedVersion)
- Funders
- National Science Foundation (1760347)
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Keywords
- trust in automation, false alarms, Miss, phishing detection, trust calibration, feedback learning, human performance, email classification, human-automation interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Automation Error Type and Methods of Communicating Automation Reliability Affect Trust and Performance: An Empirical Study in the Cyber Domain. https://app.cochat.ai/papers/10.1109/thms.2021.3051137
- The paper
- Chen, J., Mishler, S., & Hu, B. (2021). Automation Error Type and Methods of Communicating Automation Reliability Affect Trust and Performance: An Empirical Study in the Cyber Domain. IEEE Transactions on Human-Machine Systems. https://doi.org/10.1109/thms.2021.3051137
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.