Individual Differences in the Calibration of Trust in Automation
Abstract
Objective The objective was to determine whether operators with an expectancy that automation is trustworthy are better at calibrating their trust to changes in the capabilities of automation, and if so, why. Background Studies suggest that individual differences in automation expectancy may be able to account for why changes in the capabilities of automation lead to a substantial change in trust for some, yet only a small change for others. Method In a baggage screening task, 225 participants searched for weapons in 200 X-ray images of luggage. Participants were assisted by an automated decision aid exhibiting different levels of reliability. Measures of expectancy that automation is trustworthy were used in conjunction with subjective measures of trust and perceived reliability to identify individual differences in trust calibration. Results Operators with high expectancy that automation is trustworthy were more sensitive to changes (both increases and decreases) in automation reliability. This difference was eliminated by manipulating the causal attribution of automation errors. Conclusion Attributing the cause of automation errors to factors external to the automation fosters an understanding of tasks and situations in which automation differs in reliability and may lead to more appropriate trust. Application The development of interventions can lead to calibrated trust in automation.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Trust has become a central variable to explain both resistance to use automated systems (disuse) and overreliance on automated systems (misuse).
“Trust has been recognized as a central variable to explain the resistance to using automated systems (under-trust) and the overreliance on automated systems (over-trust).”
From Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions (2023) · cites this paper as [106]
Pop et al. found individuals with high expectations were more sensitive to changes in reliability and calibrated trust more appropriately for reliability increases but not decreases compared to lower expectations.
“Pop and colleagues [107] found that individuals who held high expectations towards systems were more sensitive to changes in reliability and calibrated their trust levels more appropriately for reliability increases but not decreases as compared to individuals who had lower expectations.”
From Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions (2023) · cites this paper as [107]
The effect of a mental model of AI accuracy is unclear, as some studies found that any accuracy score increases trust, others found high accuracy scores increased trust and low accuracy scores decreased trust, while another found that these effects are mediated by an individual's innate likelihood of trusting automation.
“The effect of a mental model of AI accuracy is unclear, as some studies found that any accuracy score increases trust [55], others found high accuracy scores increased trust and low accuracy scores decreased trust [96,97], while another found that these effects are mediated by an individual's innate likelihood of trusting automation [77].”
From Explanations Can Reduce Overreliance on AI Systems During Decision-Making (2023) · cites this paper as [77]
Some studies use reliance as a proxy measure of trust, while others subjectively measure an individual's trust in the system and innate disposition to trust.
“Some studies use reliance as a proxy measure of trust [4,15,55,97], while others subjectively measure an individual's trust in the system and innate disposition to trust [8-10, 42, 51, 77].”
From Explanations Can Reduce Overreliance on AI Systems During Decision-Making (2023) · cites this paper as [77]
Cites
An integrative model of organizational trust
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995) · The Academy of Management Review · doi:10.2307/258792 · Cited by 10,736 · see citing works
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 14× · see citing works
“A large body of research has identified that one of the critical variables that may influence decisions about automation use is an operator’s trust in the automation (see Lee & See, 2004 for a review).”
BackgroundMethodResultInfluentialHumans and automation: Use, misuse, disuse, abuse
Parasuraman, R., & Riley, V. (1997) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872097778543886 · Cited by 4,554 · see citing works
Foundations for an Empirically Determined Scale of Trust in Automated Systems
Jian, J.-Y., Bisantz, A. M., & Drury, C. G. (2000) · International Journal of Cognitive Ergonomics · doi:10.1207/s15327566ijce0401_04 · Cited by 1,799 · see citing works
The role of trust in automation reliance
Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G., & Beck, H. P. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00038-7 · Cited by 1,276 · see citing works
Merritt, S. M., & Ilgen, D. R. (2008) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872008x288574 · Cited by 527 · cited 23× · see citing works
“The parallel findings of Merritt and Ilgen (2008) and Yamagishi et al. (1999) suggest that individual differences in expectancy may be able to account for why changes in the capabilities of the automation in previous studies led to a substantial change in trust”
BackgroundMethodResultInfluentialThe benefits of imperfect diagnostic automation: a synthesis of the literature
Wickens, C. D., & Dixon, S. R. (2007) · Theoretical Issues in Ergonomics Science · doi:10.1080/14639220500370105 · Cited by 423 · cited 1× · see citing works
“For example, Wickens and Dixon (2007) found that in some cases operators relied on automated aids even when it resulted in worse diagnostic monitoring than when the operator had not used automation at all.”
BackgroundAutomated diagnostic aids: The effects of aid reliability on users' trust and reliance
Wiegmann, D. A., Rich, A., & Zhang, H. (2001) · Theoretical Issues in Ergonomics Science · doi:10.1080/14639220110110306 · Cited by 176 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Zhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372852 · Cited by 895 · see citing works
Metrics for explainable AI: challenges and prospects
Hoffman, R. R., Mueller, S. T., Klein, G., & Litman, J. (2018) · arXiv · doi:10.48550/arxiv.1812.04608 · Cited by 518 · see citing works
Introduction matters: Manipulating trust in automation and reliance in automated driving
Körber, M., Baseler, E., & Bengler, K. (2018) · Applied Ergonomics · doi:10.1016/j.apergo.2017.07.006 · Cited by 418 · see citing works
Explanations Can Reduce Overreliance on AI Systems During Decision-Making
Vasconcelos, H., Jörke, M., Grunde-McLaughlin, M., Gerstenberg, T., Bernstein, M. S., & Krishna, R. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579605 · Cited by 402 · see citing works
Hoffman, R. R., Mueller, S. T., Klein, G., & Litman, J. (2023) · Frontiers in Computer Science · doi:10.3389/fcomp.2023.1096257 · Cited by 272 · see citing works
Buçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021) · arXiv · doi:10.48550/arxiv.2102.09692 · Cited by 1,124 · see citing works
“Thus, numerous studies have highlighted the importance of calibrated trust in improving AI-assisted decision-making [7, 9, 31, 44, 51, 53, 67].”
BackgroundChen, Y., Zahedi, F. M., Abbasi, A., & Dobolyi, D. (2021) · Information & Management · doi:10.1016/j.im.2020.103394 · Cited by 53 · see citing works
Psychological Traits and Appropriate Reliance: Factors Shaping Trust in AI
Küper, A., & Krämer, N. (2024) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2024.2348216 · Cited by 49 · see citing works
The relationship of trust and dependence
Patton, C. E., & Wickens, C. D. (2024) · Ergonomics · doi:10.1080/00140139.2024.2342436 · Cited by 44 · see citing works
Having the Right Attitude: How Attitude Impacts Trust Repair in Human—Robot Interaction
Esterwood, C., & Robert, L. P. (2022) · 2022 17th ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri53351.2022.9889535 · Cited by 36 · see citing works
Huang, H., & Hou, G. (2025) · Frontiers in Computer Science · doi:10.3389/fcomp.2025.1659594 · Cited by 5 · see citing works
- Authors
- Pop, V. L., Shrewsbury, A., & Durso, F. T.
- Affiliations
- Georgia Institute of Technology (US)
- ORCID
- Durso, Francis T.
- Venue
- Human Factors: The Journal of the Human Factors and Ergonomics Society
- Publisher
- SAGE Publications
- Published
- 2014-12-29
- Volume
- 57
- Issue
- 4
- Pages
- 545-556
- ISSN
- 0018-7208 (ISSN-L)
- 1547-8181
- OpenAlex
- W2130087774
- Cited by
- 109
- References
- 8
- Citations by year
- 2026: 15 · 2025: 12 · 2024: 21 · 2023: 10 · 2022: 10 · 2021: 8 · 2020: 10 · 2019: 6 · 2018: 7 · 2017: 5 · 2016: 2 · 2015: 2 · 2014: 1
- FWCI
- 4.90
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Human Factors The Journal of the Human Factors and Ergonomics Society (publishedVersion)
- PubMed (publishedVersion)
- SMARTech Repository (Georgia Institute of Technology) (submittedVersion)
- Funders
- Georgia Institute of Technology
- Topics
- Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Risk and Safety Analysis
- Keywords
- trust in automation, trust calibration, individual differences, automation reliability, automated decision aids, baggage screening, X-ray inspection, human-automation interaction
- MeSH terms
- Adolescent, Adult, Automation, Female, Humans, Individuality, Male, Security Measures, Reproducibility of Results, Trust, Young Adult, Air Travel
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
