Automated diagnostic aids: The effects of aid reliability on users' trust and reliance
Abstract
We examined the effects that different levels of, and changes in, automation reliability have on users' trust of automated diagnostics aids. Participants were presented with a series of testing trials in which they diagnosed the validity of a system failure using only information provided to them by an automated diagnostic aid. The initial reliability of the aid was either 60, 80 or 100% reliable. However, for participants initially provided with the 60%-reliable aid, the accuracy of the aid increased to 80% half way through testing, whereas for those initially provided the 100%-reliable aid, the aid's reliability was reduced to 80%. Aid accuracy remained at 80% throughout testing for participants in the 80%-reliability group. Both subjective measures (i.e. perceived reliability of the aids and subjective confidence ratings) and objective measures of performance (concurrence with the aid's diagnosis and decision times) were examined. Results indicated that users of automated diagnostic aids were sensitive to different levels of aid reliabilities, as well as to subsequent changes in initial aid reliabilities. However, objective performance measures were related to, but not perfectly calibrated with, subjective measures of confidence and reliability estimates. These findings highlight the need to distinguish between automation trust as a psychological construct that can be assessed only through subjective measures and automation reliance that can only be defined in terms of performance data. A conceptual framework for understanding the relationship between trust and reliance is presented.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A third way to measure trust calibration was to measure over-, under-, and appropriate trust as behavioral deviation (reliance) from ideal behavior.
“A third way to measure the trust calibration was to measure over-, under-, and appropriate trust as the behavioral deviation (reliance) from the ideal behavior [27,50,55,104,110,132,137].”
From Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions (2023) · cites this paper as [132]
Decreases in reliability from perfect to 80% were perceived as less reliable than a constant reliability of 80%, whereas increases in reliability did not benefit as much, and reliability perceptions remained lower than the increase.
“Decreases in reliability from perfect to 80% were perceived as less reliable than a constant reliability of 80%, whereas increases in reliability did not beneft as much, and reliability perceptions remained lower than the increase.”
From Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions (2023) · cites this paper as [133]
One study found users reacted rather slowly to changes in reliability and that such changes depended on the direction of reliability change.
“One study found, for example, that users reacted rather slowly to changes in reliability and that such changes depended on the direction of reliability change [133].”
From Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions (2023) · cites this paper as [133]
Wang et al. found feature importance and feature contribution explanations supported users, whereas effects of nearest neighbors and counterfactual explanations were inconclusive.
“Testing diferent explanations, Wang and colleagues [132] found that feature importance and feature contribution explanations supported the users, whereas the efects of nearest neighbors and counterfactual explanations were inconclusive.”
From Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions (2023) · cites this paper as [132]
Cites
Reason, J. (1990) · doi:10.1017/cbo9781139062367 · Cited by 5,108 · see citing works
Humans and automation: Use, misuse, disuse, abuse
Parasuraman, R., & Riley, V. (1997) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872097778543886 · Cited by 4,554 · see citing works
The psychology of everyday things
Weld, D. S. (1989) · Artificial Intelligence · doi:10.1016/0004-3702(89)90083-0 · Cited by 2,118 · see citing works
Engineering Psychology and Human Performance
Alluisi, E. A., & Morgan, B. B. (1976) · Annual Review of Psychology · doi:10.1146/annurev.ps.27.020176.001513 · Cited by 1,204 · see citing works
Trust, self-confidence, and operators' adaptation to automation
Lee, J. D., & Moray, N. (1994) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1994.1007 · Cited by 932 · see citing works
Performance Consequences of Automation-Induced 'Complacency'
Parasuraman, R., Molloy, R., & Singh, I. L. (1993) · The International Journal of Aviation Psychology · doi:10.1207/s15327108ijap0301_1 · Cited by 875 · see citing works
Incentive contrast: A review of behavioral changes following shifts in reward
Flaherty, C. F. (1982) · Animal Learning & Behavior · doi:10.3758/bf03212282 · Cited by 317 · see citing works
Human Factors Analysis of Postaccident Data: Applying Theoretical Taxonomies of Human Error
Weigmann, D. A., & Shappell, S. A. (1997) · International Journal of Aviation Psychology · doi:10.1207/s15327108ijap0701_4 · Cited by 161 · see citing works
Predicting Misuse and Disuse of Combat Identification Systems
Dzindolet, M. T., Pierce, L. G., Beck, H. P., Dawe, L. A., & Anderson, B. W. (2001) · Military Psychology · doi:10.1207/s15327876mp1303_2 · Cited by 139 · see citing works
Misuse and Disuse of Automated AIDS
Dzindolet, M. T., Pierce, L. G., Beck, H. P., & Dawe, L. A. (1999) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193129904300345 · Cited by 34 · see citing works
Wiegmann, D. A., & Cristina, F. J. (2000) · Theoretical Issues in Ergonomics Science · doi:10.1080/14639220050171306 · Cited by 10 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Parasuraman, R., Sheridan, T., & Wickens, C. (2008) · Journal of Cognitive Engineering and Decision Making · doi:10.1518/155534308x284417 · Cited by 729 · see citing works
Similarities and differences between human–human and human–automation trust: an integrative review
Madhavan, P., & Wiegmann, D. A. (2007) · Theoretical Issues in Ergonomics Science · doi:10.1080/14639220500337708 · Cited by 577 · see citing works
Self-citationMerritt, S. M., & Ilgen, D. R. (2008) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872008x288574 · Cited by 527 · see citing works
The Role of Explanations on Trust and Reliance in Clinical Decision Support Systems
Bussone, A., Stumpf, S., & O'Sullivan, D. (2015) · 2015 International Conference on Healthcare Informatics · doi:10.1109/ichi.2015.26 · Cited by 441 · see citing works
Measurement of Trust in Automation: A Narrative Review and Reference Guide
Kohn, S. C., de Visser, E. J., Wiese, E., Lee, Y., & Shaw, T. H. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.604977 · Cited by 337 · see citing works
Kraus, J., Scholz, D., Stiegemeier, D., & Baumann, M. (2020) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720819853686 · Cited by 276 · see citing works
Effects of Imperfect Automation on Decision Making in a Simulated Command and Control Task
Rovira, E., McGarry, K., & Parasuraman, R. (2007) · Hum. Factors · doi:10.1518/001872007779598082 · Cited by 297 · see citing works
Automation failures on tasks easily performed by operators undermine trust in automated aids
Madhavan, P., Wiegmann, D. A., & Lacson, F. C. (2006) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872006777724408 · Cited by 268 · see citing works
Self-citationEffects of Anthropomorphism and Accountability on Trust in Human Robot Interaction
Natarajan, M., & Gombolay, M. (2020) · Proceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/3319502.3374839 · Cited by 248 · see citing works
Pak, R., Fink, N., Price, M. M., Bass, B., & Sturre, L. (2012) · Ergonomics · doi:10.1080/00140139.2012.691554 · Cited by 183 · see citing works
Williams, D. J., & Noyes, J. M. (2007) · Theoretical Issues in Ergonomics Science · doi:10.1080/14639220500484419 · Cited by 179 · see citing works
Keep the driver in control: Automating automobiles of the future
Banks, V. A., & Stanton, N. A. (2015) · Applied Ergonomics · doi:10.1016/j.apergo.2015.06.020 · Cited by 147 · see citing works
Wischnewski, M., Krämer, N., & Müller, E. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581197 · Cited by 146 · see citing works
“One study found, for example, that users reacted rather slowly to changes in reliability and that such changes depended on the direction of reliability change [133].”
BackgroundInfluentialMeasuring Individual Differences in the Perfect Automation Schema
Merritt, S. M., Unnerstall, J. L., Lee, D., & Huber, K. (2015) · Hum. Factors · doi:10.1177/0018720815581247 · Cited by 119 · see citing works
System reliability, performance and trust in adaptable automation.
Chavaillaz, A., Wastell, D., & Sauer, J. S. (2016) · Applied Ergonomics · doi:10.1016/j.apergo.2015.07.012 · Cited by 102 · see citing works
Dexter, F., Willemsen‐Dunlap, A., & Lee, J. D. (2007) · Anesthesia & Analgesia · doi:10.1213/01.ane.0000268539.85847.c9 · Cited by 103 · see citing works
Situation awareness and driving performance in a simulated navigation task
Ma, R., & KABER, D. B. (2007) · Ergonomics · doi:10.1080/00140130701318913 · Cited by 102 · see citing works
Individual Differences in the Calibration of Trust in Automation
Pop, V. L., Shrewsbury, A., & Durso, F. T. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814564422 · Cited by 109 · see citing works
Understanding reliance on automation: effects of error type, error distribution, age and experience
Sánchez, J., Rogers, W. A., Fisk, A. D., & Rovira, E. M. (2011) · Theoretical Issues in Ergonomics Science · doi:10.1080/1463922x.2011.611269 · Cited by 95 · see citing works
Age differences in trust and reliance of a medication management system
Ho, G., Wheatley, D., & Scialfa, C. T. (2005) · Interacting with Computers · doi:10.1016/j.intcom.2005.09.007 · Cited by 90 · see citing works
System-Wide versus Component-Specific Trust Using Multiple Aids
Keller, D., & Rice, S. C. (2009) · The Journal of general psychology · doi:10.1080/00221300903266713 · Cited by 102 · see citing works
Merritt, S. M., Lee, D., Unnerstall, J. L., & Huber, K. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814561675 · Cited by 83 · see citing works
Building appropriate trust in human-‐robot teams
Ososky, S., Schuster, D. M., Phillips, E. K., & Jentsch, F. G. (2013) · Journal of International Crisis and Risk Communication Research · Cited by 79 · see citing works
A framework for explaining reliance on decision aids
van Dongen, K., & van Maanen, P. (2013) · Int. J. Hum. Comput. Stud. · doi:10.1016/j.ijhcs.2012.10.018 · Cited by 77 · see citing works
Sauer, J., Chavaillaz, A., & Wastell, D. (2016) · Ergonomics · doi:10.1080/00140139.2015.1094577 · Cited by 71 · see citing works
Geels-Blair, K., Rice, S. C., & Schwark, J. D. (2013) · International Journal of Aviation Psychology · doi:10.1080/10508414.2013.799355 · Cited by 69 · see citing works
Understanding Is Key: An Analysis of Factors Pertaining to Trust in a Real-World Automation System
Balfe, N., Sharples, S. C., & Wilson, J. R. (2018) · Hum. Factors · doi:10.1177/0018720818761256 · Cited by 69 · see citing works
The Role of Trust as a Mediator Between System Characteristics and Response Behaviors
Chancey, E. T., Bliss, J. P., Proaps, A. B., & Madhavan, P. (2015) · Hum. Factors · doi:10.1177/0018720815582261 · Cited by 61 · see citing works
Lucas, G., Boberg, J., Traum, D. R., Artstein, R., Gratch, J. S., Gainer, A., Johnson, E., Leuski, A., & Nakano, M. (2018) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/3171221.3171258 · Cited by 60 · see citing works
Under Pressure: Examining Social Conformity With Computer and Robot Groups
Hertz, N., & Wiese, E. (2018) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720818788473 · Cited by 53 · see citing works
Regulating for trust: Can law establish trust in artificial intelligence?
Tamò‐Larrieux, A., Guitton, C., Mayer, S., & Lutz, C. (2023) · Regulation & Governance · doi:10.1111/rego.12568 · Cited by 51 · see citing works
Using System-Wide Trust Theory to Make Predictions About Dependence on Four Diagnostic Aids
Rice, S. C., & Geels, K. (2010) · The Journal of general psychology · doi:10.1080/00221309.2010.499397 · Cited by 49 · see citing works
Automation Usage Decisions: Controlling Intent and Appraisal Errors in a Target Detection Task
Beck, H. P., Dzindolet, M. T., & Pierce, L. G. (2007) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872007x200076 · Cited by 48 · see citing works
Development of design principles for automated systems in transport control
Balfe, N., Wilson, J. R., Sharples, S. C., & Clarke, T. (2012) · Ergonomics · doi:10.1080/00140139.2011.636456 · Cited by 47 · see citing works
Huegli, D., Merks, S., & Schwaninger, A. (2020) · Applied Ergonomics · doi:10.1016/j.apergo.2020.103094 · Cited by 38 · see citing works
On the impact of different types of errors on trust in human-robot interaction
Flook, R., Shrinah, A., Wijnen, L., Eder, K., Melhuish, C., & Lemaignan, S. (2019) · Interaction Studies Social Behaviour and Communication in Biological and Artificial Systems · doi:10.1075/is.18067.flo · Cited by 41 · see citing works
Type of Automation Failure: The Effects on Trust and Reliance in Automation
Johnson, J. D., Sanchez, J., Fisk, A. D., & Rogers, W. A. (2004) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120404801807 · Cited by 41 · see citing works
Machine Agency in Human-Machine Networks; Impacts and Trust Implications
Engen, V., Pickering, B., & Walland, P. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-39513-5_9 · Cited by 37 · see citing works
Modelling and Measuring Trust in Human–Robot Collaboration
Loizaga, E., Bastida, L., Sillaurren, S., Moya, A., & Toledo, N. (2024) · Applied Sciences · doi:10.3390/app14051919 · Cited by 33 · see citing works
Detecting Human Trust Calibration in Automation: A Convolutional Neural Network Approach
Choo, S., & Nam, C. (2022) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2021.3137015 · Cited by 34 · see citing works
Madhavan, P., & Wiegmann, D. A. (2005) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1518/0018720054679489 · Cited by 32 · see citing works
Self-citationThe Perception of Automation Reliability and Acceptance of Automated Advice
Hutchinson, J., Strickland, L., Farrell, S., & Loft, S. (2023) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208211062985 · Cited by 32 · see citing works
Onnasch, L., Ruff, S., & Manzey, D. (2014) · Int. J. Hum. Comput. Stud. · doi:10.1016/j.ijhcs.2014.05.001 · Cited by 27 · see citing works
Effects of in-vehicle navigation assistance and performance on driver trust and vehicle control
Ma, R., & KABER, D. B. (2007) · International Journal of Industrial Ergonomics · doi:10.1016/j.ergon.2007.04.005 · Cited by 30 · see citing works
Chavaillaz, A., Schwaninger, A., Michel, S., & Sauer, J. (2018) · Ergonomics · doi:10.1080/00140139.2018.1481231 · Cited by 25 · see citing works
Moderators Of Trust And Reliance Across Multiple Decision Aids
Ross, J. M. (2008) · Journal of International Crisis and Risk Communication Research · Cited by 29 · see citing works
Practical Guidance for Evaluating Calibrated Trust
McDermott, P. L., & ten Brink, R. (2019) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1071181319631379 · Cited by 28 · see citing works
Strickland, L. J. G., Boag, R. J., Heathcote, A., Bowden, V. K., & Loft, S. D. (2023) · Journal of Experimental Psychology Applied · doi:10.1037/xap0000463 · Cited by 27 · see citing works
Transparent Automated Advice to Mitigate the Impact of Variation in Automation Reliability
Gegoff, I., Tatasciore, M., Bowden, V. K., McCarley, J. S., & Loft, S. D. (2023) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208231196738 · Cited by 28 · see citing works
Liu, W., & Wang, Y. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2313921 · Cited by 27 · see citing works
Near-Perfect Automation: Investigating Performance, Trust, and Visual Attention Allocation
Foroughi, C. K., Devlin, S. P., Pak, R., Brown, N. L., Sibley, C. M., & Coyne, J. T. (2021) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208211032889 · Cited by 26 · see citing works
FACTORS THAT AFFECT TRUST AND RELIANCE ON AN AUTOMATED AID
Sánchez, J. (2006) · UPT. Syiah Kuala University Library (Syiah Kuala University) · Cited by 24 · see citing works
Trusted Autonomy Under Uncertainty
Smithson, M. J. (2018) · Studies in systems, decision and control · doi:10.1007/978-3-319-64816-3_10 · Cited by 23 · see citing works
Spain, R. D., & Bliss, J. P. (2008) · Ergonomics · doi:10.1080/00140130802120234 · Cited by 23 · see citing works
Effects of expertise for automatic train operations
Brandenburger, N., & Jipp, M. (2017) · Cognition, Technology & Work · doi:10.1007/s10111-017-0434-2 · Cited by 25 · see citing works
Automation Type and Reliability Impact on Visual Automation Monitoring and Human Performance
Avril, E., Cegarra, J., Wioland, L., & Navarro, J. (2021) · International journal of human computer interactions · doi:10.1080/10447318.2021.1925435 · Cited by 21 · see citing works
Banks, V. A., & Stanton, N. A. (2016) · Ergonomics · doi:10.1080/00140139.2016.1146344 · Cited by 21 · see citing works
Continuous Calibration of Trust in Automated Systems
Merritt, S. M., Huber, K., LaChapell-Unnerstall, J., & Lee, D. (2014) · doi:10.21236/ada606748 · Cited by 20 · see citing works
Culture, Errors, and Rapport-building Dialogue in Social Agents
Lucas, G., Boberg, J., Traum, D. R., Artstein, R., Gratch, J. S., Gainer, A., Johnson, E., Leuski, A., & Nakano, M. (2018) · ACM International Conference on Intelligent Virtual Agents (IVA) · doi:10.1145/3267851.3267887 · Cited by 20 · see citing works
Kim, J. Y., Lester, C., & Yang, X. J. (2025) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208251326795 · Cited by 21 · see citing works
Avril, E. (2022) · Ergonomics · doi:10.1080/00140139.2022.2069870 · Cited by 17 · see citing works
Rovira, E., Cross, A., Leitch, E., & Bonaceto, C. (2014) · Hum. Factors · doi:10.1177/0018720813519675 · Cited by 26 · see citing works
Generational differences in trust in digital assistants
Noah, B., & Sethumadhavan, A. (2019) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1071181319631029 · Cited by 16 · see citing works
Neural dynamics of trust in human-robot interaction
Nam, C. S., Eskander, E., & Choo, S. (2020) · Elsevier eBooks · doi:10.1016/b978-0-12-819472-0.00020-4 · Cited by 18 · see citing works
Lee, J., Yamani, Y., Long, S. K., Unverricht, J. R., & Itoh, M. (2021) · Ergonomics · doi:10.1080/00140139.2021.1909752 · Cited by 14 · see citing works
Følstad, A., Law, E. L.-C., & van As, N. (2024) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3690383 · Cited by 17 · see citing works
Trust with increasing and decreasing reliability
Rittenberg, B. S. P., Holland, C. W., Barnhart, G. E., Gaudreau, S. M., & Neyedli, H. F. (2024) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208241228636 · Cited by 19 · see citing works
Mixing It Up: How Mixed Groups of Humans and Machines Modulate Conformity
Hertz, N., Shaw, T. H., de Visser, E. J., & Wiese, E. (2019) · Journal of Cognitive Engineering and Decision Making · doi:10.1177/1555343419869465 · Cited by 17 · see citing works
What Are the Predictors of System-Wide Trust Loss in Transportation Automation?
Rice, S. C., Winter, S. R., Deaton, J. E., & Cremer, I. (2016) · Journal of Aviation Technology and Engineering · doi:10.7771/2159-6670.1120 · Cited by 18 · see citing works
Type of automation failure: The effects on trust and reliance in automation
Johnson, J. D., Sánchez, J., Fisk, A. D., & Rogers, W. A. (2004) · PsycEXTRA Dataset · doi:10.1037/e577212012-007 · Cited by 17 · see citing works
Operator adaptation to changes in system reliability under adaptable automation
Chavaillaz, A., & Sauer, J. (2017) · Ergonomics · doi:10.1080/00140139.2016.1261187 · Cited by 15 · see citing works
Retention of a Time Pressure Heuristic in a Target Identification Task
Rice, S. C., Keller, D., Trafimow, D., & Sandry, J. (2010) · The Journal of general psychology · doi:10.1080/00221309.2010.484447 · Cited by 16 · see citing works
Adaptive Automation: Building Flexibility into Human-Machine Systems☆
Dzindolet, M. T., Beck, H. P., & PIERCE, L. (2005) · Advances in human performance and cognitive engineering research · doi:10.1016/s1479-3601(05)06007-8 · Cited by 15 · see citing works
Radensky, M., Séguin, J. A., Lim, J. S., Olson, K., & Geiger, R. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594043 · Cited by 14 · see citing works
“However, whereas confidence communicates the predicted accuracy of an individual output, reliability communicates the system’s overall predicted performance [12, 32, 52].”
BackgroundWeber, R. O., Johs, A. J., Goel, P., & Marques-Silva, J. P. (2024) · AI Magazine · doi:10.1002/aaai.12184 · Cited by 16 · see citing works
Keeping the human in the loop: are autonomous decisions inevitable?
Pöhler, J., Flegel, N., Mentler, T., & Laerhoven, K. V. (2025) · i-com · doi:10.1515/icom-2024-0068 · Cited by 15 · see citing works
Huegli, D., Merks, S., & Schwaninger, A. (2022) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2022.2107775 · Cited by 13 · see citing works
Beemer, B. A., & Gregg, D. G. (2010) · Decision Support Systems · doi:10.1016/j.dss.2010.04.007 · Cited by 13 · see citing works
Conceptual Model of Human-Automation Interaction
Sánchez, J. (2009) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120905301850 · Cited by 13 · see citing works
Chancey, E. T., Bliss, J. P., Liechty, M. M., & Proaps, A. B. (2015) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931215591141 · Cited by 7 · see citing works
Automation Failures on Tasks Easily Performed by Operators Undermines Trust in Automated Aids
Madhavan, P., Wiegmann, D. A., & Lacson, F. C. (2003) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120304700318 · Cited by 13 · see citing works
Self-citationA Recent Review on Subjective and Objective Assessment of Trust in Human Autonomy Teaming
Jui, J. J., Hettiarachchi, I. T., Bhatti, A., Farghaly, M. R. M., & Creighton, D. (2025) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2025.3589981 · Cited by 13 · see citing works
Human behavioral response to fluctuating automation reliability
Hutchinson, J., Strickland, L. J. G., Farrell, S., & Loft, S. D. (2022) · Applied Ergonomics · doi:10.1016/j.apergo.2022.103835 · Cited by 13 · see citing works
Under-Reliance on the Decision Aid: A Difference in Calibration and Attribution between Self and Aid
Kees Van Dongen, A., & van Maanen, P.-P. (2006) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120605000304 · Cited by 12 · see citing works
The Role of Social Dialogue and Errors in Robots
Lucas, G., Boberg, J., Traum, D. R., Artstein, R., Gratch, J., Gainer, A., Johnson, E., Leuski, A., & Nakano, M. (2017) · International Conference on Human-Agent Interaction (HAI) · doi:10.1145/3125739.3132617 · Cited by 12 · see citing works
Human Interactions Framework for Remote Ship Operations
Kari, R., Gaspar, H. M., Gausdal, A. H., & Morshedi, M. (2018) · Mediterranean Conference on Control and Automation (MED) · doi:10.1109/med.2018.8442624 · Cited by 12 · see citing works
Supporting operator reliance on automation through continuous feedback
Seppelt, B. D. (2018) · doi:10.17077/etd.90t9uqnh · Cited by 11 · see citing works
Dynamic calibration of trust and trustworthiness in AI-enabled systems
Liebherr, M., Enkel, E., Law, E. L., Mousavi, M. R., Sammartino, M., & Sieberg, P. M. (2026) · International Journal on Software Tools for Technology Transfer · doi:10.1007/s10009-026-00840-6 · Cited by 13 · see citing works
Kaduk, S. I., Roberts, A. P. J., & Stanton, N. A. (2020) · Theoretical Issues in Ergonomics Science · doi:10.1080/1463922x.2020.1758829 · Cited by 11 · see citing works
Garcia, K. R., Mishler, S., Xiao, Y., Wang, C., Hu, B., Still, J. D., & Chen, J. (2022) · Journal of Cognitive Engineering and Decision Making · doi:10.1177/15553434221117001 · Cited by 11 · see citing works
The Role of Trust as a Mediator between Signaling System Reliability and Response Behaviors
Chancey, E. T., Proaps, A. B., & Bliss, J. P. (2013) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931213571063 · Cited by 14 · see citing works
Sauer, J., & Chavaillaz, A. (2017) · Applied Ergonomics · doi:10.1016/j.apergo.2016.08.007 · Cited by 12 · see citing works
Adjusting Amount of AI Explanation for Visual Tasks
Maehigashi, A., Fukuchi, Y., & Yamada, S. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3650802 · Cited by 10 · see citing works
The Influence of Perceived Task Difficulty and Importance on Automation Use
Schwark, J. D., Dolgov, I., Graves, W. J., & Hor, D. (2010) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193121005401931 · Cited by 11 · see citing works
Transport and mining machines operators' behavioral attitudes in safety climate context
Spasojević-Brkić, V. K., Veljković, Z. A., Brkić, A., Misita, M., Perišić, M., & Papić, N. (2022) · Istrazivanja i projektovanja za privredu · doi:10.5937/jaes0-37669 · Cited by 10 · see citing works
Factors influencing trust in Ambient Assisted Living Technology: A scenario-based analysis
Steinke, F., Bading, N., Fritsch, T., & Simonsen, S. (2014) · Gerontechnology · doi:10.4017/gt.2013.12.2.002.00 · Cited by 11 · see citing works
How do humans learn about the reliability of automation?
Strickland, L., Farrell, S., Wilson, M. K., Hutchinson, J., & Loft, S. (2024) · Cognitive Research · doi:10.1186/s41235-024-00533-1 · Cited by 14 · see citing works
Explaining AI weaknesses improves human–AI performance in a dynamic control task
Rieger, T., Schindler, H., Onnasch, L., & Roesler, E. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103505 · Cited by 10 · see citing works
Detecting Human Trust Calibration in Automation: A Deep Learning Approach
Choo, S., Sanders, N. E., Kim, N., Kim, W., Nam, C., & Fitts, E. P. (2019) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1071181319631298 · Cited by 9 · see citing works
Steinke, F., Ingenhoff, A., & Fritsch, T. (2014) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2014.903789 · Cited by 9 · see citing works
- Authors
- Wiegmann, D. A., Rich, A., & Zhang, H.
- Venue
- Theoretical Issues in Ergonomics Science
- Publisher
- Informa UK Limited
- Published
- 2001-01
- Volume
- 2
- Issue
- 4
- Pages
- 352-367
- ISSN
- 1463-922X (ISSN-L)
- 1464-536X
- OpenAlex
- W2014615989
- Cited by
- 176
- References
- 11
- Citations by year
- 2026: 10 · 2025: 9 · 2024: 16 · 2023: 10 · 2022: 10 · 2021: 4 · 2020: 7 · 2019: 9 · 2018: 11 · 2017: 3 · 2016: 8 · 2015: 12 · 2014: 10 · 2013: 4 · 2012: 5
- FWCI
- 2.20
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Theoretical Issues in Ergonomics Science (publishedVersion)
- Topics
- Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- trust in automation, reliance on automation, perceived reliability, subjective confidence, diagnostic accuracy, human-automation interaction, conceptual framework
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
