The DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Automated diagnostic aids: The effects of aid reliability on users' trust and reliance
Abstract
We examined the effects that different levels of, and changes in, automation reliability have on users' trust of automated diagnostics aids. Participants were presented with a series of testing trials in which they diagnosed the validity of a system failure using only information provided to them by an automated diagnostic aid. The initial reliability of the aid was either 60, 80 or 100% reliable. However, for participants initially provided with the 60%-reliable aid, the accuracy of the aid increased to 80% half way through testing, whereas for those initially provided the 100%-reliable aid, the aid's reliability was reduced to 80%. Aid accuracy remained at 80% throughout testing for participants in the 80%-reliability group. Both subjective measures (i.e. perceived reliability of the aids and subjective confidence ratings) and objective measures of performance (concurrence with the aid's diagnosis and decision times) were examined. Results indicated that users of automated diagnostic aids were sensitive to different levels of aid reliabilities, as well as to subsequent changes in initial aid reliabilities. However, objective performance measures were related to, but not perfectly calibrated with, subjective measures of confidence and reliability estimates. These findings highlight the need to distinguish between automation trust as a psychological construct that can be assessed only through subjective measures and automation reliance that can only be defined in terms of performance data. A conceptual framework for understanding the relationship between trust and reliance is presented.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“A third way to measure the trust calibration was to measure over-, under-, and appropriate trust as the behavioral deviation (reliance) from the ideal behavior [27,50,55,104,110,132,137].”
From Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions (2023) · cites this paper as [132]
In the citing paper
“Decreases in reliability from perfect to 80% were perceived as less reliable than a constant reliability of 80%, whereas increases in reliability did not beneft as much, and reliability perceptions remained lower than the increase.”
From Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions (2023) · cites this paper as [133]
In the citing paper
“One study found, for example, that users reacted rather slowly to changes in reliability and that such changes depended on the direction of reliability change [133].”
From Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions (2023) · cites this paper as [133]
In the citing paper
“Testing diferent explanations, Wang and colleagues [132] found that feature importance and feature contribution explanations supported the users, whereas the efects of nearest neighbors and counterfactual explanations were inconclusive.”
From Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions (2023) · cites this paper as [132]
Citations
Cites
Reason, J. (1990) · doi:10.1017/cbo9781139062367 · Cited by 5,108
Humans and automation: Use, misuse, disuse, abuse
Parasuraman, R., & Riley, V. (1997) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872097778543886 · Cited by 4,554
The psychology of everyday things
Weld, D. S. (1989) · Artificial Intelligence · doi:10.1016/0004-3702(89)90083-0 · Cited by 2,113
Engineering Psychology and Human Performance
Alluisi, E. A., & Morgan, B. B. (1976) · Annual Review of Psychology · doi:10.1146/annurev.ps.27.020176.001513 · Cited by 1,204
Trust, self-confidence, and operators' adaptation to automation
Lee, J. D., & Moray, N. (1994) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1994.1007 · Cited by 932
Performance Consequences of Automation-Induced 'Complacency'
Parasuraman, R., Molloy, R., & Singh, I. L. (1993) · The International Journal of Aviation Psychology · doi:10.1207/s15327108ijap0301_1 · Cited by 875
Incentive contrast: A review of behavioral changes following shifts in reward
Flaherty, C. F. (1982) · Animal Learning & Behavior · doi:10.3758/bf03212282 · Cited by 317
Human Factors Analysis of Postaccident Data: Applying Theoretical Taxonomies of Human Error
Weigmann, D. A., & Shappell, S. A. (1997) · International Journal of Aviation Psychology · doi:10.1207/s15327108ijap0701_4 · Cited by 161
Predicting Misuse and Disuse of Combat Identification Systems
Dzindolet, M. T., Pierce, L. G., Beck, H. P., Dawe, L. A., & Anderson, B. W. (2001) · Military Psychology · doi:10.1207/s15327876mp1303_2 · Cited by 139
Misuse and Disuse of Automated AIDS
Dzindolet, M. T., Pierce, L. G., Beck, H. P., & Dawe, L. A. (1999) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193129904300345 · Cited by 34
Research works printed but not linked: 6
- DZINDOLET M. T., 2001, Human Factors
- GEMPLER K. S., 1998, Display of Predictor Reliability on a Cockpit Display Traffic Information
- KIRK R. E., 1982, Experimental Design
- LEE J. D., 1992, Ergonomics, 22, 671
- SHAPPELL S. A., 1996, Space and Environmental Medicine, 67, 65
- WIEGMANN D. A., 2001, Human Factors
Cited by
Sorted by citations
Items 101–176 of 176
Steinke, F., Ingenhoff, A., & Fritsch, T. (2014) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2014.903789 · Cited by 9
The Effects of Alarm System Errors on Dependence: Moderated Mediation of Trust With and Without Risk
Chancey, E. T. (2016) · ODU Digital Commons (Old Dominion University) · doi:10.25777/34gr-x929 · Cited by 10
How Does Acknowledging Users’ Preferences Impact AI’s Ability to Make Conflicting Recommendations?
Marti, D., Budathoki, A., Ding, Y., Lucas, G., & Nelson, D. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2426035 · Cited by 9
System Wide Trust: A Possible Contagion Effect
Mehta, R., & Rice, S. C. (2016) · Collegiate Aviation Review International · doi:10.22488/okstate.18.100465 · Cited by 8
Mental Models, Trust, and Reliance: Exploring the Effect of Human Perceptions on Automation Use
Cassidy, A. (2009) · Calhoun: The Naval Postgraduate School Institutional Archive (Naval Postgraduate School) · Cited by 8
Building Decision Support Systems for Acceptance
Riedel, R., Fransoo, J. C., Wiers, V. C. S., Fischer, K., Cegarra, J., & Jentsch, D. (2010) · Springer eBooks · doi:10.1007/978-3-642-13382-4_11 · Cited by 8
Alarcon, G. M., Gamble, R., Ryan, T. J., Walter, C., Jessup, S. A., Wood, D. W., & Capiola, A. (2018) · Applied Ergonomics · doi:10.1016/j.apergo.2018.02.027 · Cited by 7
Experimental Investigation of Trust in Anthropomorphic Agents as Task Partners
Maehigashi, A., Tsumura, T., & Yamada, S. (2022) · International Conference on Human-Agent Interaction (HAI) · doi:10.1145/3527188.3563921 · Cited by 10
In human–machine trust, humans rely on a simple averaging strategy
Love, J., Gronau, Q. F., Palmer, G., Eidels, A., & Brown, S. D. (2024) · Cognitive Research Principles and Implications · doi:10.1186/s41235-024-00583-5 · Cited by 15
Effects of extended lay-off periods on performance and operator trust under adaptable automation
Chavaillaz, A., Wastell, D. G., & Sauer, J. (2015) · Applied Ergonomics · doi:10.1016/j.apergo.2015.10.006 · Cited by 6
Safety Warnings for Automation
Lehto, M. R., Lesch, M. F., & Horrey, W. J. (2009) · Springer handbooks · doi:10.1007/978-3-540-78831-7_39 · Cited by 6
Lu, Y., & Sarter, N. (2019) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1071181319631345 · Cited by 5
Some cues are more equal than others: Cue plausibility for false alarms in baggage screening.
Chavaillaz, A., Schwaninger, A., Michel, S., & Sauer, J. (2019) · Applied Ergonomics · doi:10.1016/j.apergo.2019.102916 · Cited by 4
Effects of Beep-Sound Timings on Trust Dynamics in Human-Robot Interaction
Maehigashi, A., Tsumura, T., & Yamada, S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-24670-8_57 · Cited by 5
How Terminology Affects Users’ Responses to System Failures
Candrian, C., & Scherer, A. (2023) · Hum. Factors · doi:10.1177/00187208231202572 · Cited by 5
Driver trust and reliance on a navigation system: Effect of graphical display
Barthou, A., Kemeny, A. A., Reymond, G., Merienne, F., & Berthoz, A. (2010) · SAM, the Arts et Métiers ParisTech open access repository (Paris Institute of Technology) · Cited by 5
Agents and Decision Support Systems
Tolk, A., Madhavan, P., Tweedale, J. W., & Jain, L. C. (2009) · Wiley eBooks · doi:10.1002/9783527627783.ch14 · Cited by 5
Zhang, H.-S., Zhou, R., Cheng, C., Huang, S.-H., Cheng, M., & Yang, C.-T. (2025) · Cognitive Research · doi:10.1186/s41235-025-00659-w · Cited by 3
Self-citationExperimental Investigation of Human Acceptance of AI Suggestions with Heatmap and Pointing-based XAI
Maehigashi, A., Fukuchi, Y., & Yamada, S. (2023) · International Conference on Human-Agent Interaction (HAI) · doi:10.1145/3623809.3623834 · Cited by 4
Wiczorek, R., & Zirk, A. (2019) · Int. J. Hum. Comput. Stud. · doi:10.1016/j.ijhcs.2018.12.006 · Cited by 3
Mishler, S., Jeffcoat, C., & Chen, J. (2019) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1071181319631351 · Cited by 4
Aiding Human Reliance Decision Making Using Computational Models of Trust
van Maanen, P.-P., Klos, T., & Kees Van Dongen, A. (2007) · IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT) · doi:10.1109/wi-iatw.2007.108 · Cited by 4
Mayer, A. K. (2008) · SMARTech Repository (Georgia Institute of Technology) · Cited by 4
Tan, C., Xu, X., & Liang Ma, A. (2026) · International Journal of Industrial Ergonomics · doi:10.1016/j.ergon.2026.103924 · Cited by 5
Effect of Wait Time on Trust and Reliance in Human-Robot Interaction
Maehigashi, A., & Yamada, S. (2024) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/3610978.3640611 · Cited by 3
Bahnsen, K. L., Dischinger, E., & Grundgeiger, T. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713685 · Cited by 4
I Spy Differences in System Reliability
Patton, C. E., Warden, A. C., & Pharmer, R. L. (2023) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/21695067231192718 · Cited by 4
Towards neuro-symbolic reinforcement learning for trustworthy human-autonomy teaming
Gurung, P., Jiang, L., & Rawat, D. B. (2024) · Assurance and Security for AI-enabled Systems · doi:10.1117/12.3014232 · Cited by 3
Operator Aid-Switching in a Case of Redundant Automation with Single-Aid Failure
Sandry, J., Schwark, J. D., Hunt, G., Geels, K., & Rice, S. C. (2011) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1071181311551297 · Cited by 3
Trust in automated systems the effect of automation level on trust calibration
Walliser, J. C. (2011) · Calhoun: The Naval Postgraduate School Institutional Archive (Naval Postgraduate School) · Cited by 3
Holland, C. W., & Neyedli, H. F. (2025) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/10711813251367370 · Cited by 2
Alarcon, G. M., Jessup, S. A., Meyers, S. K., Willis, S. M., Harris, K. N., Johnson, D., Noblick, J., & Bennette, W. D. (2026) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2026.3651060 · Cited by 2
The Influence of Exposure and Error Type on Estimates of Automation Reliability
Patton, C. E., Warden, A. C., Guzman-Bonilla, E., & Pharmer, R. L. (2026) · Hum. Factors · doi:10.1177/00187208261435308 · Cited by 2
Trust in automation: A survey of neurophysiological perspectives on measurement and modelling
Jui, J. J., Hettiarachchi, I. T., & Bhatti, A. (2026) · Neurocomputing · doi:10.1016/j.neucom.2026.133505 · Cited by 3
Perception of Trust in Automation
Madhavan, P. (2015) · Cambridge University Press eBooks · doi:10.1017/cbo9780511973017.031 · Cited by 2
Designing for Human-Centred Decision Support Systems in PHM
McDonnell, D., Balfe, N., Al‐Dahidi, S., & O’Donnell, G. E. (2014) · PHM Society European Conference · doi:10.36001/phme.2014.v2i1.1558 · Cited by 2
Effects of Presenting Multiple Types of AI Explanations for Visual Task
Maehigashi, A., Fukuchi, Y., & Yamada, S. (2024) · International Conference on Human-Agent Interaction (HAI) · doi:10.1145/3687272.3688328 · Cited by 2
Sanchez, J., Fisk, A. D., & Rogers, W. A. (2006) · 2006 9th International Conference on Control, Automation, Robotics and Vision · doi:10.1109/icarcv.2006.345296 · Cited by 2
Effects of levels of automation on air traffic controller situation awareness and performance
Sethumadhavan, A. (2009) · ThinkTech (Texas Tech University) · Cited by 2
Safety Warnings for Automation
Lehto, M. R., & Nanda, G. (2023) · Springer handbooks · doi:10.1007/978-3-030-96729-1_29 · Cited by 2
Fault Diagnosis: Human Performance in the Digital and Automation Context
Lyu, X., She, M., Pan, D., Wu, X., Chen, K., & Li, Z. (2022) · Automation, collaboration, and e-services · doi:10.1007/978-3-031-10780-1_14 · Cited by 1
Trust Transfer in Autonomous Vehicles: The Role of Warm Brand Image Amid Automation Failure
Qin, S., Li, C., & Qi, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-60731-8_9 · Cited by 1
Zijlstra, M., & Payne, S. J. (2025) · Behaviour and Information Technology · doi:10.1080/0144929x.2025.2478277 · Cited by 1
Operators’ Adaption to Unreliability of Alarm Systems: A Performance and Eye-Tracking Analysis
Onnasch, L., Ruff, S., & Manzey, D. (2012) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1071181312561059 · Cited by 1
Aiding Human Reliance Decision Making Using Computational Models of Trust
van Maanen, P.-P., Klos, T., & Kees Van Dongen, A. (2007) · Data Archiving and Networked Services (DANS) · doi:10.5555/1339264.1339729 · Cited by 1
Steinke, F., Fritsch, T., & Silbermann, L. (2012) · International Conference on eHealth, Telemedicine, and Social Medicine · Cited by 1
An Examination of the Social, Cognitive, and Motivational Factors that Affect Automation Reliance
Dzindolet, M. T., Pierce, L. G., & Beck, H. P. (2017) · CRC Press eBooks · doi:10.1201/9781315587387-23 · Cited by 1
The Rise, Fall, and Repair of Trust for Automated Driving Systems
Mishler, S., & Chen, J. (2020) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1071181320641485 · Cited by 1
Trustworthiness Perceptions of Machine Learning Algorithms: The Influence of Confidence Intervals
Alarcon, G. M., Jessup, S. A., Meyers, S. K., Johnson, D., & Bennette, W. D. (2024) · IEEE International Conference on Human-Machine Systems (ICHMS) · doi:10.1109/ichms59971.2024.10555722 · Cited by 1
Jessup, S. A., Alarcon, G. M., Willis, S. M., & Lee, M. A. (2024) · Applied Ergonomics · doi:10.1016/j.apergo.2024.104363 · Cited by 1
Study on Individual Driving Style Based on Driving Behavior Characteristics
Ren, Y., Li, X., & Zheng, X. (2018) · COTA International Conference of Transportation Professionals (CICTP) · doi:10.1061/9780784481523.175 · Cited by 1
Closed-Loop Adaptive Decision Support Based on Automated Trust Assessment
van Maanen, P.-P., Klos, T., & Kees Van Dongen, A. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-73216-7_30 · Cited by 1
Madhavan, P., & Wiegmann, D. A. (2005) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign) · Cited by 1
Self-citationStudent Pilot Situational Awareness: The Effects of Trust in Technology
Kunii, Y. (2006) · Scholarly Commons (Embry–Riddle Aeronautical University) · Cited by 1
Expected Reliability of Everyday- and Ambient Assisted Living Technologies
Steinke, F., Fritsch, T., Hertzer, A., Tautz, H., & Zickwolf, S. (2013) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2013.040603
A Process Model of Trust in Automation
Zúñiga, J., McCurry, M., & Trafton, J. G. (2014) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931214581174 · Cited by 3
Assessing Human-Automation System Safety, Efficiency, and Performance
Oglesby, J. M., Stowers, K. L., Leyva, K., Dietz, A. S., Sonesh, S. C., Burke, S., & Eduardo Salas, A. (2014) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931214581240
Swanson, L. A. (2011) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign)
Williams, D., & Noyes, J. (2007)
Mohamed, S. H. (2015) · DR-NTU (Nanyang Technological University) · doi:10.32657/10356/62933
A Predictive Model of Driver Response in an Autonomous Environment
Samuel, S., Horrey, W. J., & Fisher, D. L. (2015) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931215591361
Application of neuroergonomics in the industrial design of mining equipment.
Mach, Q. H. (2015) · Lu Zone Ul (Laurentian University)
A Reliance Model for Automated Combat Identification Systems: Implications for Trust in Automation
Hollands, J. G., & Neyedli, H. (2018) · CRC Press eBooks · doi:10.1201/9781315549637-9 · Cited by 1
Fowler, M., Sasangohar, F., & Brydia, B. (2020) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1071181320641339
(2003) · CRC Press eBooks · doi:10.1201/9780203485088.bmatt
(2011) · doi:10.1201/b11371-21
Additional file 1 of How do humans learn about the reliability of automation?
Strickland, L. J. G., Farrell, S., Wilson, M. K., Hutchinson, J., & Loft, S. D. (2024) · Figshare · doi:10.6084/m9.figshare.26681521
Implicit neural measures of trust in artificial intelligence
Feldmann‐Wüstefeld, T., & Wiese, E. (2026) · Computers in Human Behavior Artificial Humans · doi:10.1016/j.chbah.2026.100274
Implicit neural measures of trust in artificial intelligencePreprint
Feldmann‐Wüstefeld, T. (2025) · OSF Preprints (OSF Preprints) · doi:10.17605/osf.io/ndbhg
Zijlstra, M., & Payne, S. (2026) · Int. J. Hum. Comput. Stud. · doi:10.1016/j.ijhcs.2026.103881
Tan, C., & Liang Ma, A. (2026) · International Journal of Industrial Ergonomics · doi:10.1016/j.ergon.2026.104059
Accuracy is not enough: The roles of accuracy, reliability and personality in the trust process
Alarcon, G. M., Ryan, T. J., Capiola, A., & Willis, S. M. (2026) · Applied Ergonomics · doi:10.1016/j.apergo.2026.104916
Metadata
- Authors
- Wiegmann, D. A., Rich, A., & Zhang, H.
- Venue
- Theoretical Issues in Ergonomics Science
- Publisher
- Informa UK Limited
- Published
- 2001-01
- Volume
- 2
- Issue
- 4
- Pages
- 352-367
- ISSN
- 1463-922X (ISSN-L)
- 1464-536X
- OpenAlex
- W2014615989
- Cited by
- 176
- References
- 11
- Citations by year
- 2026: 10 · 2025: 9 · 2024: 16 · 2023: 10 · 2022: 10 · 2021: 4 · 2020: 7 · 2019: 9 · 2018: 11 · 2017: 3 · 2016: 8 · 2015: 12 · 2014: 10 · 2013: 4 · 2012: 5
- FWCI
- 2.20
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Theoretical Issues in Ergonomics Science (publishedVersion)
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- trust in automation, reliance on automation, perceived reliability, subjective confidence, diagnostic accuracy, human-automation interaction, conceptual framework
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
