The DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Automated diagnostic aids: The effects of aid reliability on users' trust and reliance
Abstract
We examined the effects that different levels of, and changes in, automation reliability have on users' trust of automated diagnostics aids. Participants were presented with a series of testing trials in which they diagnosed the validity of a system failure using only information provided to them by an automated diagnostic aid. The initial reliability of the aid was either 60, 80 or 100% reliable. However, for participants initially provided with the 60%-reliable aid, the accuracy of the aid increased to 80% half way through testing, whereas for those initially provided the 100%-reliable aid, the aid's reliability was reduced to 80%. Aid accuracy remained at 80% throughout testing for participants in the 80%-reliability group. Both subjective measures (i.e. perceived reliability of the aids and subjective confidence ratings) and objective measures of performance (concurrence with the aid's diagnosis and decision times) were examined. Results indicated that users of automated diagnostic aids were sensitive to different levels of aid reliabilities, as well as to subsequent changes in initial aid reliabilities. However, objective performance measures were related to, but not perfectly calibrated with, subjective measures of confidence and reliability estimates. These findings highlight the need to distinguish between automation trust as a psychological construct that can be assessed only through subjective measures and automation reliance that can only be defined in terms of performance data. A conceptual framework for understanding the relationship between trust and reliance is presented.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“A third way to measure the trust calibration was to measure over-, under-, and appropriate trust as the behavioral deviation (reliance) from the ideal behavior [27,50,55,104,110,132,137].”
From Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions (2023) · cites this paper as [132]
In the citing paper
“Decreases in reliability from perfect to 80% were perceived as less reliable than a constant reliability of 80%, whereas increases in reliability did not beneft as much, and reliability perceptions remained lower than the increase.”
From Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions (2023) · cites this paper as [133]
In the citing paper
“One study found, for example, that users reacted rather slowly to changes in reliability and that such changes depended on the direction of reliability change [133].”
From Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions (2023) · cites this paper as [133]
In the citing paper
“Testing diferent explanations, Wang and colleagues [132] found that feature importance and feature contribution explanations supported the users, whereas the efects of nearest neighbors and counterfactual explanations were inconclusive.”
From Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions (2023) · cites this paper as [132]
Citations
Cites
Reason, J. (1990) · doi:10.1017/cbo9781139062367 · Cited by 5,108
Humans and automation: Use, misuse, disuse, abuse
Parasuraman, R., & Riley, V. (1997) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872097778543886 · Cited by 4,554
The psychology of everyday things
Weld, D. S. (1989) · Artificial Intelligence · doi:10.1016/0004-3702(89)90083-0 · Cited by 2,113
Engineering Psychology and Human Performance
Alluisi, E. A., & Morgan, B. B. (1976) · Annual Review of Psychology · doi:10.1146/annurev.ps.27.020176.001513 · Cited by 1,204
Trust, self-confidence, and operators' adaptation to automation
Lee, J. D., & Moray, N. (1994) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1994.1007 · Cited by 932
Performance Consequences of Automation-Induced 'Complacency'
Parasuraman, R., Molloy, R., & Singh, I. L. (1993) · The International Journal of Aviation Psychology · doi:10.1207/s15327108ijap0301_1 · Cited by 875
Incentive contrast: A review of behavioral changes following shifts in reward
Flaherty, C. F. (1982) · Animal Learning & Behavior · doi:10.3758/bf03212282 · Cited by 317
Human Factors Analysis of Postaccident Data: Applying Theoretical Taxonomies of Human Error
Weigmann, D. A., & Shappell, S. A. (1997) · International Journal of Aviation Psychology · doi:10.1207/s15327108ijap0701_4 · Cited by 161
Predicting Misuse and Disuse of Combat Identification Systems
Dzindolet, M. T., Pierce, L. G., Beck, H. P., Dawe, L. A., & Anderson, B. W. (2001) · Military Psychology · doi:10.1207/s15327876mp1303_2 · Cited by 139
Misuse and Disuse of Automated AIDS
Dzindolet, M. T., Pierce, L. G., Beck, H. P., & Dawe, L. A. (1999) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193129904300345 · Cited by 34
Research works printed but not linked: 6
- DZINDOLET M. T., 2001, Human Factors
- GEMPLER K. S., 1998, Display of Predictor Reliability on a Cockpit Display Traffic Information
- KIRK R. E., 1982, Experimental Design
- LEE J. D., 1992, Ergonomics, 22, 671
- SHAPPELL S. A., 1996, Space and Environmental Medicine, 67, 65
- WIEGMANN D. A., 2001, Human Factors
Cited by
Sorted by citations
Parasuraman, R., Sheridan, T., & Wickens, C. (2008) · Journal of Cognitive Engineering and Decision Making · doi:10.1518/155534308x284417 · Cited by 729
Similarities and differences between human–human and human–automation trust: an integrative review
Madhavan, P., & Wiegmann, D. A. (2007) · Theoretical Issues in Ergonomics Science · doi:10.1080/14639220500337708 · Cited by 577
Self-citationMerritt, S. M., & Ilgen, D. R. (2008) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872008x288574 · Cited by 527
The Role of Explanations on Trust and Reliance in Clinical Decision Support Systems
Bussone, A., Stumpf, S., & O'Sullivan, D. (2015) · 2015 International Conference on Healthcare Informatics · doi:10.1109/ichi.2015.26 · Cited by 441
Measurement of trust in automation: A narrative review and reference guide
Kohn, S. C., de Visser, E. J., Wiese, E., Lee, Y.-C., & Shaw, T. H. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.604977 · Cited by 315
Metadata
- Authors
- Wiegmann, D. A., Rich, A., & Zhang, H.
- Venue
- Theoretical Issues in Ergonomics Science
- Publisher
- Informa UK Limited
- Published
- 2001-01
- Volume
- 2
- Issue
- 4
- Pages
- 352-367
- ISSN
- 1463-922X (ISSN-L)
- 1464-536X
- OpenAlex
- W2014615989
- Cited by
- 176
- References
- 11
- Citations by year
- 2026: 10 · 2025: 9 · 2024: 16 · 2023: 10 · 2022: 10 · 2021: 4 · 2020: 7 · 2019: 9 · 2018: 11 · 2017: 3 · 2016: 8 · 2015: 12 · 2014: 10 · 2013: 4 · 2012: 5
- FWCI
- 2.20
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Theoretical Issues in Ergonomics Science (publishedVersion)
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- trust in automation, reliance on automation, perceived reliability, subjective confidence, diagnostic accuracy, human-automation interaction, conceptual framework
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
