A Meta-Analysis of Factors Influencing the Development of Trust in Automation: Implications for Understanding Autonomy in Future Systems
Open accessAbstract
Objective: We used meta-analysis to assess research concerning human trust in automation to understand the foundation upon which future autonomous systems can be built. Background: Trust is increasingly important in the growing need for synergistic human–machine teaming. Thus, we expand on our previous meta-analytic foundation in the field of human–robot interaction to include all of automation interaction. Method: We used meta-analysis to assess trust in automation. Thirty studies provided 164 pairwise effect sizes, and 16 studies provided 63 correlational effect sizes. Results: The overall effect size of all factors on trust development was ḡ = +0.48, and the correlational effect was [Formula: see text] = +0.34, each of which represented medium effects. Moderator effects were observed for the human-related ( ḡ = +0.49; [Formula: see text] = +0.16) and automation-related ( ḡ = +0.53; [Formula: see text] = +0.41) factors. Moderator effects specific to environmental factors proved insufficient in number to calculate at this time. Conclusion: Findings provide a quantitative representation of factors influencing the development of trust in automation as well as identify additional areas of needed empirical research. Application: This work has important implications to the enhancement of current and future human–automation interaction, especially in high-risk or extreme performance environments.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Provider impressions can differ from their actual use.
“In order to better understand factors impacting the adoption of CDS, previous studies of deployed systems have used post-hoc surveys or interviews to gauge provider impressions 28,44,53 ; however, provider impressions can differ from their actual use 4,18,27 .”
From Factors driving provider adoption of the TREWS machine learning-based early warning system and its effects on sepsis treatment timing (2022) · cites this paper as [27]
Cites
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225 · see citing works
An integrative model of organizational trust
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995) · The Academy of Management Review · doi:10.2307/258792 · Cited by 10,736 · see citing works
Affect and Cognition Based Trust as Foundations for Interpersonal Cooperation in Organizations
McAllister, D. J. (1995) · Academy of Management Journal · doi:10.2307/256727 · Cited by 7,346 · see citing works
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 5× · see citing works
“The cognitive factors have been suggested to include (a) an understanding of how the automation works, (b) self-perceived ability to use that automation, and (c) expectancy concerning the automation (Ashleigh & Stanton, 2001; Lee & See, 2004; Madhavan & Wiegmann, 2007).”
BackgroundInfluentialA model for types and levels of human interaction with automation
Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/3468.844354 · Cited by 4,195 · cited 1× · see citing works
“The LOA relates to the amount of control the automation is designed to exercise (Parasuraman et al., 2000).”
BackgroundMachines and Mindlessness: Social Responses to Computers
Nass, C., & Moon, Y. (2000) · Journal of Social Issues · doi:10.1111/0022-4537.00153 · Cited by 4,022 · cited 1× · see citing works
“Aesthetic appearance has been shown to be related to both automation likeability and use (Li & Yeh, 2010; Nass & Moon, 2000).”
BackgroundA Meta-Analysis of Factors Affecting Trust in Human-Robot Interaction
Hancock, P. A., Billings, D. R., Schaefer, K. E., Chen, J. Y. C., de Visser, E. J., & Parasuraman, R. (2011) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720811417254 · Cited by 2,051 · cited 10× · see citing works
“Yet Hancock, Billings, Schaefer, Chen, et al. (2011) also identified numerous gaps in the literature, especially those related to the human operator and environmental factors.”
BackgroundMethodInfluentialSelf-citationThe role of trust in automation reliance
Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G., & Beck, H. P. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00038-7 · Cited by 1,276 · see citing works
MUIR, B. M., & MORAY, N. (1996) · Ergonomics · doi:10.1080/00140139608964474 · Cited by 993 · see citing works
MUIR, B. M. (1994) · Ergonomics · doi:10.1080/00140139408964957 · Cited by 873 · cited 2× · see citing works
“If experience with a machine provides predictable outcomes, then operators start to and then continue to trust the system (Cahour & Forzy, 2009; Muir, 1994).”
BackgroundModeling Web Site Design Across Cultures: Relationships to Trust, Satisfaction, and E-Loyalty
Cyr, D. (2008) · Journal of Management Information Systems · doi:10.2753/mis0742-1222240402 · Cited by 766 · see citing works
Merritt, S. M., & Ilgen, D. R. (2008) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872008x288574 · Cited by 527 · cited 4× · see citing works
“Many studies affirm that trust is something that builds over time, and past experiences with technology—specifically, automated systems—have a direct impact on such trust development (Merritt & Ilgen, 2008; Rajaonah, Anceaux, & Vienne, 2006; Sullivan, Tsimhoni, & Bogard, 2008).”
BackgroundInfluentialIntelligent agent transparency in human–agent teaming for multi-UxV management
Mercado, J. E., Rupp, M. A., Chen, J. Y. C., Barnes, M. J., Barber, D., & Procci, K. (2016) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720815621206 · Cited by 403 · see citing works
Self-citationde Vries, P., Midden, C., & Bouwhuis, D. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00039-9 · Cited by 264 · see citing works
Looije, R., Neerincx, M. A., & Cnossen, F. (2010) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2009.08.007 · Cited by 243 · see citing works
Impact of robot failures and feedback on real-time trust
Desai, M., Kaniarasu, P., Medvedev, M., Steinfeld, A., & Yanco, H. (2013) · 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2013.6483596 · Cited by 234 · see citing works
Imposing limits on autonomous systems
Hancock, P. A. (2017) · Ergonomics · doi:10.1080/00140139.2016.1190035 · Cited by 162 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Initial validation of the general attitudes towards Artificial Intelligence Scale
Schepman, A., & Rodway, P. (2020) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2020.100014 · Cited by 785 · see citing works
Trustworthy artificial intelligence
Thiebes, S., Lins, S., & Sunyaev, A. (2021) · Electronic Markets · doi:10.1007/s12525-020-00441-4 · Cited by 595 · see citing works
Introduction matters: Manipulating trust in automation and reliance in automated driving
Körber, M., Baseler, E., & Bengler, K. (2018) · Applied Ergonomics · doi:10.1016/j.apergo.2017.07.006 · Cited by 418 · see citing works
Trust and reliance on AI — An experimental study on the extent and costs of overreliance on AI
Klingbeil, A., Grützner, C., & Schreck, P. (2024) · Computers in Human Behavior · doi:10.1016/j.chb.2024.108352 · Cited by 450 · see citing works
Trust in Artificial Intelligence: Meta-Analytic Findings
Kaplan, A. D., Kessler, T. T., Brill, J. C., & Hancock, P. A. (2023) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208211013988 · Cited by 397 · see citing works
Self-citationSystematic Review: Trust-Building Factors and Implications for Conversational Agent Design
Rheu, M., Shin, J. Y., Peng, W., & Huh-Yoo, J. (2021) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2020.1807710 · Cited by 360 · see citing works
Grassini, S. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1191628 · Cited by 330 · see citing works
On the future of transportation in an era of automated and autonomous vehicles
Hancock, P. A., Nourbakhsh, I., & Stewart, J. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1805770115 · Cited by 312 · see citing works
Self-citationTrusting Automation: Designing for Responsivity and Resilience
Chiou, E. K., & Lee, J. D. (2023) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208211009995 · Cited by 301 · see citing works
Gkintoni, E., Vantaraki, F., Skoulidi, C., Anastassopoulos, P., & Vantarakis, A. (2024) · Behavioral Sciences · doi:10.3390/bs14020102 · Cited by 137 · see citing works
Henry, K. E., Adams, R., Parent, C., Soleimani, H., Sridharan, A., Johnson, L., Hager, D. N., Cosgrove, S. E., Markowski, A., Klein, E. Y., et al. (2022) · Nature Medicine · doi:10.1038/s41591-022-01895-z · Cited by 127 · see citing works
Wang, X., & Qiu, X. (2024) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2024.103777 · Cited by 118 · see citing works
Dikmen, M., & Burns, C. (2022) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102792 · Cited by 116 · see citing works
Kim, T., & Song, H. (2021) · Telematics and Informatics · doi:10.1016/j.tele.2021.101595 · Cited by 101 · see citing works
Effective human–AI work design for collaborative decision-making
Jain, R., Garg, N., & Khera, S. N. (2023) · Kybernetes · doi:10.1108/k-04-2022-0548 · Cited by 85 · see citing works
Sebok, A., & Wickens, C. D. (2017) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720816665201 · Cited by 75 · see citing works
Second Chance for a First Impression? Trust Development in Intelligent System Interaction
Tolmeijer, S., Gadiraju, U., Ghantasala, R., Gupta, A., & Bernstein, A. (2021) · Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization · doi:10.1145/3450613.3456817 · Cited by 67 · see citing works
The Impact of Training on Human–Autonomy Team Communications and Trust Calibration
Johnson, C. J., Demir, M., McNeese, N. J., Gorman, J. C., Wolff, A. T., & Cooke, N. J. (2023) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208211047323 · Cited by 68 · see citing works
Lacroux, A., & Martin-Lacroux, C. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.895997 · Cited by 67 · see citing works
The Next Generation of Medical Decision Support: A Roadmap Toward Transparent Expert Companions
Bruckert, S., Finzel, B., & Schmid, U. (2020) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2020.507973 · Cited by 66 · see citing works
Chen, Y., Zahedi, F. M., Abbasi, A., & Dobolyi, D. (2021) · Information & Management · doi:10.1016/j.im.2020.103394 · Cited by 53 · see citing works
Gutzwiller, R. S., Chiou, E. K., Craig, S. D., Lewis, C. M., Lematta, G. J., & Hsiung, C.-P. (2019) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1071181319631201 · Cited by 54 · see citing works
Chen, J., Mishler, S., Hu, B., Li, N., & Proctor, R. W. (2018) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2018.05.010 · Cited by 50 · see citing works
Self-citationPsychological Traits and Appropriate Reliance: Factors Shaping Trust in AI
Küper, A., & Krämer, N. (2024) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2024.2348216 · Cited by 49 · see citing works
Daronnat, S., Azzopardi, L., Halvey, M., & Dubiel, M. (2021) · Frontiers in Robotics and AI · doi:10.3389/frobt.2021.642201 · Cited by 45 · see citing works
McGrath, M. J., Duenser, A., Lacey, J., & Paris, C. (2025) · Computers in Human Behavior: Artificial Humans · doi:10.1016/j.chbah.2025.100200 · Cited by 51 · see citing works
Two Routes to Trust Calibration: Effects of Reliability and Brand Information on Trust in Automation
Kraus, J. M., Forster, Y., Hergeth, S., & Baumann, M. (2019) · International Journal of Mobile Human Computer Interaction · doi:10.4018/ijmhci.2019070101 · Cited by 42 · see citing works
Chen, J., Mishler, S., & Hu, B. (2021) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2021.3051137 · Cited by 40 · see citing works
Self-citationRezaeian, O., Bayrak, A. E., & Asan, O. (2026) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2025.2539458 · Cited by 39 · see citing works
Rezaeian, O., Bayrak, A. E., & Asan, O. (2025) · arXiv · doi:10.48550/arxiv.2501.16693 · Cited by 39 · see citing works
The relationship of trust and dependence
Patton, C. E., & Wickens, C. D. (2024) · Ergonomics · doi:10.1080/00140139.2024.2342436 · Cited by 44 · see citing works
Determinants of Laypersons’ Trust in Medical Decision Aids: Randomized Controlled Trial
Kopka, M., Schmieding, M. L., Rieger, T., Roesler, E., Balzer, F., & Feufel, M. A. (2022) · JMIR Human Factors · doi:10.2196/35219 · Cited by 37 · see citing works
How to Achieve Explainability and Transparency in Human AI Interaction
Hois, J., Theofanou-Fuelbier, D., & Junk, A. J. (2019) · Communications in Computer and Information Science · doi:10.1007/978-3-030-23528-4_25 · Cited by 31 · see citing works
Wardle, C., Urbani, S., & Wang, E. (2025) · Journal of Medical Internet Research · doi:10.2196/79961 · Cited by 43 · see citing works
Kox, E. S., Siegling, L. B., & Kerstholt, J. H. (2022) · International Journal of Social Robotics · doi:10.1007/s12369-022-00871-4 · Cited by 22 · see citing works
How Do We Assess the Trustworthiness of AI? Introducing the Trustworthiness Assessment Model (TrAM)
Schlicker, N., Baum, K., Uhde, A., Sterz, S., Hirsch, M. C., & Langer, M. (2022) · doi:10.31234/osf.io/qhwvx · Cited by 79 · see citing works
Guglielmucci, F., & Di Basilio, D. (2025) · JMIR Human Factors · doi:10.2196/70698 · Cited by 10 · see citing works
- Authors
- Schaefer, K. E., Chen, J. Y. C., Szalma, J. L., & Hancock, P. A.
- Affiliations
- DEVCOM Army Research Laboratory (US)
- University of Central Florida (US)
- Corresponding author
- Schaefer, Kristin E.
- Venue
- Human Factors: The Journal of the Human Factors and Ergonomics Society
- Publisher
- SAGE Publications
- Published
- 2016-03-22
- Volume
- 58
- Issue
- 3
- Pages
- 377-400
- ISSN
- 0018-7208 (ISSN-L)
- 1547-8181
- OpenAlex
- W2332079078
- Cited by
- 768
- References
- 17
- Citations by year
- 2026: 131 · 2025: 108 · 2024: 92 · 2023: 72 · 2022: 86 · 2021: 88 · 2020: 77 · 2019: 54 · 2018: 30 · 2017: 18 · 2016: 12
- FWCI
- 5.66
- Type
- journal article
- Language
- en
- Open access
- Locations
- Human Factors The Journal of the Human Factors and Ergonomics Society (publishedVersion)
- PubMed (publishedVersion)
- Journal of International Crisis and Risk Communication Research (submittedVersion)
- Funders
- University of Central Florida
- Army Research Laboratory
- Topics
- Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Risk and Safety Analysis
- Keywords
- trust in automation, human-automation interaction, human-robot interaction, human-machine teaming, autonomous systems, high-risk environments
- MeSH terms
- Automation, Humans, Man-Machine Systems, Trust
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
