Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions
Open accessPublished version of a preprint
The DOI matches its Crossref record; 156 of the 156 references it cites were found in a registry or an index; 28 of the 145 sources its claims cite were checked for support.
Abstract
Trust has been recognized as a central variable to explain the resistance to using automated systems (under-trust) and the overreliance on automated systems (over-trust). To achieve appropriate reliance, users’ trust should be calibrated to reflect a system’s capabilities. Studies from various disciplines have examined different interventions to attain such trust calibration. Based on a literature body of 1000+ papers, we identified 96 relevant publications which aimed to calibrate users’ trust in automated systems. To provide an in-depth overview of the state-of-the-art, we reviewed and summarized measurements of the trust calibration, interventions, and results of these efforts. For the numerous promising calibration interventions, we extract common design choices and structure these into four dimensions of trust calibration interventions to guide future studies. Our findings indicate that the measurement of the trust calibration often limits the interpretation of the effects of different interventions. We suggest future directions for this problem.
Overview
CoChat Summary
## TL;DR
This is a scoping review of empirical human-subject studies on trust calibration in automated systems, based on over 1000 papers screened down to 96 relevant publications. The paper finds that many interventions can influence trust, but measurement choices often make it hard to tell whether trust is actually calibrated to system capability.
## Key findings
- The search screened a literature body of 1000+ papers and included 96 empirical studies; among these, 40 were online studies and 61 were lab studies.
- Sample sizes ranged from 9 to 1224 participants in online studies (M = 232, SD = 224) and from 8 to 865 participants in lab studies (M = 72, SD = 125).
- The authors identify four dimensions of trust calibration interventions: exo vs. endo, warranted vs. unwarranted, static vs. adaptive, and capabilities vs. process-oriented.
- Results across interventions were mixed: uncertainty information, confidence displays, reliability updates, and some explanations sometimes helped calibration, but the paper also reports cases of increased workload, over-trust, or null effects; users were generally sensitive to reliability, yet decreases in reliability produced steeper trust decay than increases produced trust gain.
## Methods
- Design: Scoping review following the PRISMA 2020 reporting standard; the authors state they did not intend a systematic review.
- Data/sample: Peer-reviewed empirical studies with human subjects and a true trust calibration intervention, excluding preprints, theses, simulation-only studies, purely qualitative studies, and non-empirical theory/framework papers.
- Analysis: Two-wave search across Google Scholar, major HCI conference proceedings, the ACM library, and Scopus; papers were coded for system, task, intervention, calibration measurement, trust measures, and calibration results, then grouped into categories such as task domains and calibration strategies.
## Limitations
- The paper states that its results are limited to English-language texts.
- The paper states that much of the corpus was screened by a single coder, though it reports an inter-rater agreement of Krippendorf’s alpha = .83 in the training phase.
- The text makes evident that the review is limited by its operational definition of calibration as matching system capabilities to trust, which the authors acknowledge may leave out other trust factors such as process and purpose (stated in the paper).
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
14 Valid · 8 Not valid · 81 Not checked · 78 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“We fnd this, albeit understandable due to limited resources and recruitment possibilities, problematic as experts react diferently to systems than lay people (see, e.g., [59]).”
§ Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions > Domain
What the sources say
- Supportedcites [59] 59. Stefanie Maria Jungmann, Timo Klan, Sebastian Kuhn, and Florian Jungmann. 2019. Accuracy of a Chatbot (ADA) in the diagnosis of mental disorders: com- parative case study with lay and expert users. JMIR formative research 3, 4 (2019), e13863. source
The Results (Number of Questions and Duration) report that psychotherapists took significantly longer than laypersons to complete cases using Ada. The Results (Percentage Agreement and Interrater Reliability) also report different diagnostic agreement across user groups. This supports the cited paper as an example of experts and laypeople interacting differently with a system, specifically the Ada app. (LLM, conf=0.88)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“However, need for trust in collaborative tasks might increase for situations of human-AI teaming, where the human team member is dependent on the performance of the system, such as [18,24,116] report.”
§ Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions > 4.2 The Tasks
What the sources say
- Not checked — no full text of the sourcecites [18] 18. Min Chen, Stefanos Nikolaidis, Harold Soh, David Hsu, and Siddhartha Srinivasa. 2018. Planning with trust for human-robot collaboration. In Proceedings of the 2018 ACM/IEEE international conference on human-robot interaction. 307-315. source
- Partly supportedcites [24] 24. Sylvain Daronnat, Leif Azzopardi, Martin Halvey, and Mateusz Dubiel. 2021. Inferring Trust From Users' Behaviours; Agents' Predictability Positively Af- fects Trust, Task Performance and Cognitive Load in Human-Agent Real-Time Collaboration. Frontiers in Robotics and AI 8 (2021), 194. sourceExperimental — accuracy not yet measured
The Abstract says effective human-agent collaboration requires appropriate trust and identifies agent performance as a factor influencing it. The Introduction likewise calls trust important in collaborative decision-making. Neither passage says the need for trust increases when a human teammate depends on the system’s performance. (LLM, conf=0.82)
- Not checked — no full text of the sourcecites [116] 116. Beau G Schelble, Jeremy Lopez, Claire Textor, Rui Zhang, Nathan J McNeese, Richard Pak, and Guo Freeman. 2022. Towards Ethical AI: Empirically Investi- gating Dimensions of AI Ethics, Trust Repair, and Performance in Human-AI Teaming. Human Factors (2022), 00187208221116952. source
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“Okamura and Yamada [104] implemented specifc trust calibration cues, which were displayed when participants relied too little and too much on the system.”
§ Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions > 4.4 The calibrations dimensions
What the sources say
- Could not verifycites [104] 104. Kazuo Okamura and Seiji Yamada. 2020. Adaptive trust calibration for human-AI collaboration. Plos one 15, 2 (2020), e0229132. sourceExperimental — accuracy not yet measured
The Development of PRISMA 2020 and Discussion passages concern a guideline for reporting systematic reviews, not an experiment involving trust calibration. Searches for Okamura and Yamada and for cues triggered by under- or over-reliance found no passage addressing the claim. (LLM, conf=0.77)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“A third way to measure the trust calibration was to measure over-, under-, and appropriate trust as the behavioral deviation (reliance) from the ideal behavior [27,50,55,104,110,132,137].”
§ Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions > System capabilities.
What the sources say
- Not checked — no full text of the sourcecites [27] 27. Ewart J De Visser, Samuel S Monfort, Ryan McKendrick, Melissa AB Smith, Patrick E McKnight, Frank Krueger, and Raja Parasuraman. 2016. Almost human: Anthropomorphism increases trust resilience in cognitive agents. Journal of Experimental Psychology: Applied 22, 3 (2016), 331. source
- Not checked — no full text of the sourcecites [50] 50. Sandy H Huang, Kush Bhatia, Pieter Abbeel, and Anca D Dragan. 2018. Es- tablishing appropriate trust via critical states. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 3929-3936. source
- Not checked — no full text of the sourcecites [55] 55. Theodore Jensen, Mohammad Maif Hasan Khan, and Yusuf Albayram. 2020. The role of behavioral anthropomorphism in human-automation trust calibration. In International Conference on Human-Computer Interaction. Springer, 33-53. source
- Could not verifycites [104] 104. Kazuo Okamura and Seiji Yamada. 2020. Adaptive trust calibration for human-AI collaboration. Plos one 15, 2 (2020), e0229132. sourceExperimental — accuracy not yet measured
The Discussion addresses the trustworthiness of systematic-review findings, not calibration of a person's trust or reliance behavior. Searches for over-, under-, and appropriate trust and for deviation from ideal behavior found no passage in our copy addressing that measurement method. Page numbers were not available. (LLM, conf=0.76)
- Could not verifycites [110] 110. Dalai Dos Santos Ribeiro, Gabriel Diniz Junqueira Barbosa, Marisa Do Carmo Silva, Hélio Lopes, and Simone Diniz Junqueira Barbosa. 2021. Exploring the impact of classifcation probabilities on users' trust in ambiguous instances. In 2021 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). IEEE, 1-9. sourceExperimental — accuracy not yet measured
Our copy of Ribeiro et al. (2021) is a literature review of RPA tools with AI support (Sections 1-2 define RPA, Sections 4-5 survey UiPath, Kofax, Automation Anywhere, AssistEdge, Table 1 compares AI techniques per tool, Section 6 conclusions); no passage addresses trust, trust calibration, over-/under-trust, reliance or ideal-performance reference behaviour. Three searches (trust calibration / over-trust under-trust reliance ideal behavior; trust measurement human-agent interaction metrics; trust in automation reliance human robot interaction) returned only RPA-tool and reference-list passages, the closest being Section 2 on task replacement and cost reduction. The claim therefore cannot be checked against this source, and nothing in our copy either supports or contradicts it. (LLM, conf=0.70)
- Not checked — no full text of the sourcecites [132] 132. Xinru Wang and Ming Yin. 2022. Efects of Explanations in AI-Assisted Decision Making: Principles and Comparisons. ACM Transactions on Interactive Intelligent Systems (TiiS) (2022). source
- Not checked — no full text of the sourcecites [137] 137. Fumeng Yang, Zhuanyi Huang, Jean Scholtz, and Dustin L Arendt. 2020. How do visual explanations foster end users' appropriate trust in machine learning?. In Proceedings of the 25th International Conference on Intelligent User Interfaces. Wischnewski et al. 189-201. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“Okamura and Yamada [104] could show that participants who received adaptive trust calibration cues outperformed those who continuously received information about the system's performance.”
§ Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions > 4.4 The calibrations dimensions
What the sources say
- Could not verifycites [104] 104. Kazuo Okamura and Seiji Yamada. 2020. Adaptive trust calibration for human-AI collaboration. Plos one 15, 2 (2020), e0229132. sourceExperimental — accuracy not yet measured
The Discussion addresses strategies for improving adherence to reporting guidelines, not trust-calibration cues or participant performance. Searches for Okamura and Yamada, adaptive trust calibration, and system-performance information found no passage addressing the claimed comparison. (LLM, conf=0.78)
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
70 more claims · 7 with their cited sources checked
Create a free account to see all 181 claims and to verify the 11 references still open.
- It may state two separate claims as one. 65
- Probably not a substantial claim. 53
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Because goals vary by task and context [75], aligning AI behavior with an individual's ongoing goals is crucial for effective human-AI collaboration [148].”
From What Needs Attention? Prioritizing Drivers of Developers' Trust and Adoption of Generative AI (2025) · cites this paper as [148]
In the citing paper
“Developers trusted genAI more when its actions aligned with their goals, reducing verification effort and cognitive load.”
From What Needs Attention? Prioritizing Drivers of Developers' Trust and Adoption of Generative AI (2025) · cites this paper as [148]
In the citing paper
“Trust is central to explaining resistance toward automated systems [148, 150, 151].”
From What Needs Attention? Prioritizing Drivers of Developers' Trust and Adoption of Generative AI (2025) · cites this paper as [148]
In the citing paper
“Trust is bolstered when genAI actions stay congruent with developers' objectives, reinforcing its credibility as a cognitive collaborator [148].”
From What Needs Attention? Prioritizing Drivers of Developers' Trust and Adoption of Generative AI (2025) · cites this paper as [148]
In the citing paper
“It is more consistent with the broader logic of the trust-calibration literature, in which the depth of user processing is treated as a condition for appropriately differentiated reliance (Lee and See 2004;Wischnewski et al. 2023;Kaplan et al. 2023).”
From Epistemic Trust in Generative AI as an Information Source: Development and Validation of the Trust in AI-Epistemic Scale (TAI-E) (2026) · cites this paper as Wischnewski et al. 2023
Citations
Cites
The PRISMA 2020 statement: An updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · BMJ · doi:10.1136/bmj.n71 · Cited by 108,283
An integrative model of organizational trust
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995) · The Academy of Management Review · doi:10.2307/258792 · Cited by 10,736
The PRISMA 2020 statement: an updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J., Bossuyt, P., Boutron, I., Hoffmann, T., mulrow, C., Shamseer, L., Tetzlaff, J., Akl, E., Brennan, S. E., et al. (2020) · doi:10.31222/osf.io/v7gm2_v1 · Cited by 8,090
Affect and Cognition Based Trust as Foundations for Interpersonal Cooperation in Organizations
McAllister, D. J. (1995) · Academy of Management Journal · doi:10.2307/256727 · Cited by 7,346 · cited 2×
“As McAllister [88] suggested, "the amount of knowledge necessary for trust is somewhat in between total knowledge and total ignorance" (p. 26).”
BackgroundTrust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 8×
“To avoid any such loss due to over-or under-trust, trust needs to be calibrated [74] or warranted [53], which describes “how well trust matches the true capabilities of the automation” [74, p. 57] (see also [28]).”
BackgroundMethodInfluentialDeveloping and validating trust measures for e-commerce: an integrative typology
McKnight, D. H., Choudhury, V., & Kacmar, C. (2002) · Information Systems Research · doi:10.1287/isre.13.3.334.81 · Cited by 5,550 · cited 2×
“However, we also noticed that only a few studies diferentiated between diferent subconstructs of trust (performance, process, purpose) which is inherently included in many scales (e.g., [91]) but authors did not make use of this information.”
BackgroundMethodExplanation in Artificial Intelligence: Insights from the Social Sciences
Miller, T. (2017) · arXiv · doi:10.48550/arxiv.1706.07269 · Cited by 5,338
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
The PRISMA 2020 statement: an updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J., Bossuyt, P., Boutron, I., Hoffmann, T., mulrow, C., Shamseer, L., Tetzlaff, J., Akl, E., Brennan, S. E., et al. (2020) · doi:10.31222/osf.io/v7gm2 · Cited by 5,177
Humans and automation: Use, misuse, disuse, abuse
Parasuraman, R., & Riley, V. (1997) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872097778543886 · Cited by 4,554 · cited 2×
“Hence, many studies aim to adjust trust bestowed in a system to refect the trustworthiness of systems, arriving at the concept of calibrated trust [74, 106].”
BackgroundThe PRISMA 2020 statement: An updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.03.001 · Cited by 4,265
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232 · cited 2×
“sity to trust and their attitudes towards automation, both factors which afect trust [30, 41].”
BackgroundBaier, A. (1986) · Ethics · doi:10.1086/292745 · Cited by 2,567
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159 · cited 1×
“systems which act intelligently based on available data, powered by deep learning algorithms, automation can guide credit scoring and risk assessments [42], transportation [52, 128], medical diagnostics [38], hiring decisions [78], and judicial sentencing [46] and recidivism [21].”
BackgroundAlgorithm appreciation: People prefer algorithmic to human judgment
Logg, J. M., Minson, J. A., & Moore, D. A. (2019) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2018.12.005 · Cited by 2,061
Foundations for an Empirically Determined Scale of Trust in Automated Systems
Jian, J.-Y., Bisantz, A. M., & Drury, C. G. (2000) · International Journal of Cognitive Ergonomics · doi:10.1207/s15327566ijce0401_04 · Cited by 1,799
Trust, control strategies and allocation of function in human-machine systems
LEE, J., & MORAY, N. (1992) · Ergonomics · doi:10.1080/00140139208967392 · Cited by 1,459 · cited 7×
“In many cases, trust measures were accompanied by subcon-structs of trust such as perceptions of system accuracy [90, 143], predictability [68, 73], and reliability [15, 68, 107, 133], or trust related measures such as perceived usability [4], frustration [4], perceived humanness [26, 27, 55, 58],…”
BackgroundMethodInfluentialBuçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021) · arXiv · doi:10.48550/arxiv.2102.09692 · Cited by 1,124
Zhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372852 · Cited by 895 · cited 9×
“inform users about the specifc capabilities and limitations of a system, such as information about a system’s confdence [90, 104, 145] or general information about a system’s reliability [61, 69].”
BackgroundInfluentialRise of Machine Agency: A Framework for Studying the Psychology of Human–AI Interaction (HAII)
Sundar, S. S. (2020) · Journal of Computer-Mediated Communication · doi:10.1093/jcmc/zmz026 · Cited by 871 · cited 1×
“Such machine agency competes with human agency and is likely perceived diferently from less agentic systems [120].”
BackgroundOn Defining Artificial Intelligence
Wang, P. (2019) · Journal of Artificial General Intelligence · doi:10.2478/jagi-2019-0002 · Cited by 740
Manipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445315 · Cited by 728 · cited 3×
“Poursabzi-Sangdeh et al. [108] found that providing users with more information (through increased interpretability and transparency) decreased the user’s capabilities to detect and correct mistakes by the system.”
BackgroundManipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Vaughan, J. W., & Wallach, H. (2018) · arXiv · doi:10.48550/arxiv.1802.07810 · Cited by 711
Almost human: Anthropomorphism increases trust resilience in cognitive agents.
de Visser, E. J., Monfort, S. S., McKendrick, R., Smith, M. A. B., McKnight, P. E., Krueger, F., & Parasuraman, R. (2016) · Journal of Experimental Psychology: Applied · doi:10.1037/xap0000092 · Cited by 614 · cited 8×
“De Visser and colleagues [27], for example, compared a human agent with an avatar and a computer.”
BackgroundMethodResultInfluentialSimilarities and differences between human–human and human–automation trust: an integrative review
Madhavan, P., & Wiegmann, D. A. (2007) · Theoretical Issues in Ergonomics Science · doi:10.1080/14639220500337708 · Cited by 577 · cited 3×
“Yet, Madhavan and Wickens [85] found that the perceived expertise was also inconclusive if the system operated well.”
BackgroundFormalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI
Jacovi, A., Marasović, A., Miller, T., & Goldberg, Y. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445923 · Cited by 545 · cited 4×
“However, unlike Lee and See [74], Jacovi and colleagues [53] have argued that calibrated trust "is trust that is caused by trustworthiness (to some contract)" (p.”
BackgroundInfluentialUnderstanding the Effect of Accuracy on Trust in Machine Learning Models
Yin, M., Wortman Vaughan, J., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300509 · Cited by 527
Robotic Process Automation and Artificial Intelligence in Industry 4.0 – A Literature review
Ribeiro, J., Lima, R., Eckhardt, T., & Paiva, S. (2021) · Procedia Computer Science · doi:10.1016/j.procs.2021.01.104 · Cited by 489 · cited 1×
“From previous work, we know that robots are increasingly equipped with artifcial intelligence (see e.g., [111]).”
BackgroundTowards a Theory of Longitudinal Trust Calibration in Human–Robot Teams
de Visser, E. J., Peeters, M. M. M., Jung, M. F., Kohn, S., Shaw, T. H., Pak, R., & Neerincx, M. A. (2020) · International Journal of Social Robotics · doi:10.1007/s12369-019-00596-x · Cited by 447 · cited 1×
“To avoid any such loss due to over-or under-trust, trust needs to be calibrated [74] or warranted [53], which describes “how well trust matches the true capabilities of the automation” [74, p. 57] (see also [28]).”
BackgroundIntelligent agent transparency in human–agent teaming for multi-UxV management
Mercado, J. E., Rupp, M. A., Chen, J. Y. C., Barnes, M. J., Barber, D., & Procci, K. (2016) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720815621206 · Cited by 403 · cited 2×
“Screening tasks [17, 33, 44, 63, 81, 82, 127, 130, 135, 139, 140], gathering of information [58], Military mission planning [92], human-AI collaboration for search and destroy missions [116]”
BackgroundTrust in Artificial Intelligence: Meta-Analytic Findings
Kaplan, A. D., Kessler, T. T., Brill, J. C., & Hancock, P. A. (2023) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208211013988 · Cited by 397
Lai, V., & Tan, C. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287590 · Cited by 377 · cited 3×
“[71, 109, 110, 132, 141] and eight which specifcally referred to artifcial intelligence [11, 12, 43, 83, 100, 118, 144, 145].”
BackgroundTheoretical Considerations and Development of a Questionnaire to Measure Trust in Automation
Körber, M. (2019) · Advances in Intelligent Systems and Computing · doi:10.1007/978-3-319-96074-6_2 · Cited by 317 · cited 1×
“[49] Situational Trust Scale for Automated Driving, a scale used in Muir [99], the Usability and Trust Questionnaire developed by Chen [16], the Robot Trust Questionnaire by Schäfer [114], a scale to assess trust in human-robot collaborations [13], and Körber’s trust in automation scale [65].”
MethodAI enabled applications towards intelligent transportation
Iyer, L. S. (2021) · Transportation Engineering · doi:10.1016/j.treng.2021.100083 · Cited by 315 · cited 1×
“…sophisticated systems which act intelligently based on available data, powered by deep learning algorithms, automation can guide credit scoring and risk assessments [42], transportation [52, 128], medical diagnostics [38], hiring decisions [78], and judicial sentencing [46] and recidi-vism [21].”
BackgroundAI-based computer-aided diagnosis (AI-CAD): the latest review to read first
Fujita, H. (2020) · Radiological Physics and Technology · doi:10.1007/s12194-019-00552-4 · Cited by 300
Helldin, T., Falkman, G., Riveiro, M., & Davidsson, S. (2013) · Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications · doi:10.1145/2516540.2516554 · Cited by 286 · cited 7×
“For example, Helldin et al. [47] let participants drive an autonomous vehicle in a driving simulator but manipulated the weather conditions (sight was afected by snow) or the terrain of the course (steep climbs & tight turns), which, in turn, afected the system’s reliability.”
BackgroundInfluentialKraus, J., Scholz, D., Stiegemeier, D., & Baumann, M. (2020) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720819853686 · Cited by 276 · cited 8×
“…measures were accompanied by subcon-structs of trust such as perceptions of system accuracy [90, 143], predictability [68, 73], and reliability [15, 68, 107, 133], or trust related measures such as perceived usability [4], frustration [4], perceived humanness [26, 27, 55, 58], perceived…”
BackgroundMethodResultInfluentialGill, H., Boies, K., Finegan, J. E., & McNally, J. (2005) · Journal of Business and Psychology · doi:10.1007/s10869-004-2229-8 · Cited by 270 · cited 1×
“In the context of trust, this point might be of particular importance as previous studies have shown that individuals difer in their propensity to trust and their attitudes towards automation, both factors which afect trust [30, 41].”
Resultde Vries, P., Midden, C., & Bouwhuis, D. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00039-9 · Cited by 264 · cited 1×
“[29], Lee & Moray [73], and Lewandowsky et al.”
BackgroundAffective Processes in Human–Automation Interactions
Merritt, S. M. (2011) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720811411912 · Cited by 260 · cited 1×
“We found 16 studies (see, e.g., [26, 68, 69, 84, 107]) which used the Trust in Automation Scale by Jian et al. [57], at least two [10, 94] studies used a scale developed by Merritt et al. [93], and at least fve studies (see e.g., [4, 130]) used the trust predisposition scale by McKnight et al. [91].”
MethodThe dynamics of trust: Comparing humans to automation.
Lewandowsky, S., Mundy, M., & Tan, G. P. A. (2000) · Journal of Experimental Psychology: Applied · doi:10.1037//1076-898x.6.2.104 · Cited by 240
Adaptive trust calibration for human-AI collaboration
Okamura, K., & Yamada, S. (2020) · PLOS ONE · doi:10.1371/journal.pone.0229132 · Cited by 239 · cited 9×
“For example, systems were used to aid users screen bags for dangerous objects [94], detect system malfunctions [89] or potholes [104], improve the production of orange juice pasteurization [73], play games [24], diagnose [100], discover number patterns [26], and classify plants and animals [110,…”
BackgroundMethodInfluentialThe Development of a Scale to Evaluate Trust in Industrial Human-robot Collaboration
Charalambous, G., Fletcher, S., & Webb, P. (2016) · International Journal of Social Robotics · doi:10.1007/s12369-015-0333-8 · Cited by 228 · cited 1×
“…al. [49] Situational Trust Scale for Automated Driving, a scale used in Muir [99], the Usability and Trust Questionnaire developed by Chen [16], the Robot Trust Questionnaire by Schäfer [114], a scale to assess trust in human-robot collaborations [13], and Körber’s trust in automation scale [65].”
MethodPlanning with Trust for Human-Robot Collaboration
Chen, M., Nikolaidis, S., Soh, H., Hsu, D., & Srinivasa, S. (2018) · Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/3171221.3171264 · Cited by 221 · cited 5×
“Similary, Chen and colleagues [18] developed a computational model, the Partially observable Markov decision process (POMDP), which integrates the extend users trust the system into the system’s decision making by, among others, inferring users’ trust through interactions.”
BackgroundInfluentialTrust calibration within a human-robot team: Comparing automatically generated explanations
Wang, N., Pynadath, D. V., & Hill, S. G. (2016) · 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2016.7451741 · Cited by 220 · cited 5×
“Some authors work with correlations to get a better sense of perceived trustworthiness and system capabilities [5, 12, 24, 39, 66, 67, 81, 94, 130, 139, 142].”
BackgroundMethodInfluentialMcGuirl, J. M., & Sarter, N. B. (2006) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872006779166334 · Cited by 212 · cited 13×
“questionnaires to advance their quantitative trust assessments [84, 90].”
BackgroundMethodInfluentialEvaluating Effects of User Experience and System Transparency on Trust in Automation
Yang, X. J., Unhelkar, V. V., Li, K., & Shah, J. A. (2017) · Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/2909824.3020230 · Cited by 208 · cited 7×
“In addition, some studies also employed linear trust meters [137], or visual analog scales [139].”
BackgroundInfluentialHow do visual explanations foster end users' appropriate trust in machine learning?
Yang, F., Huang, Z., Scholtz, J., & Arendt, D. L. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377480 · Cited by 200 · cited 7×
“…Income prediction [40, 145], number pattern discovery [26, 27], pothole detection [104], classif-cation of plants/animals/images [11, 75, 103, 110, 137], meal preparation [12], online dating Others [109, 141], hotel reviews [71], clearing tables [18], fnd housing [43, 124], navigation through…”
BackgroundMethodResultInfluentialLeichtmann, B., Humer, C., Hinterreiter, A., Streit, M., & Mara, M. (2023) · Computers in Human Behavior · doi:10.1016/j.chb.2022.107539 · Cited by 200 · cited 2×
“[75] indicated that merely educating users about how a system functions was not as helpful as including explanations.”
BackgroundDesigning Interactive Systems Conference 2021
(2021) · doi:10.1145/3461778 · Cited by 200
Explainable and Interpretable Models in Computer Vision and Machine Learning
(2018) · The Springer Series on Challenges in Machine Learning · doi:10.1007/978-3-319-98131-4 · Cited by 199
Deep Learning in Music Recommendation Systems
Schedl, M. (2019) · Frontiers in Applied Mathematics and Statistics · doi:10.3389/fams.2019.00044 · Cited by 189
Automated diagnostic aids: The effects of aid reliability on users' trust and reliance
Wiegmann, D. A., Rich, A., & Zhang, H. (2001) · Theoretical Issues in Ergonomics Science · doi:10.1080/14639220110110306 · Cited by 176 · cited 4×
“One study found, for example, that users reacted rather slowly to changes in reliability and that such changes depended on the direction of reliability change [133].”
BackgroundInfluentialYeh, M., & Wickens, C. D. (2001) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872001775898269 · Cited by 175 · cited 7×
“…with AV [48], observe AVs [64, 119], drive in driving simulator [87, 96, 136] Screening tasks [17, 33, 44, 63, 81, 82, 127, 130, 135, 139, 140], gathering of information [58], Military mission planning [92], human-AI collaboration for search and destroy missions [116] Improving…”
BackgroundInfluentialNaiseh, M., Al-Thani, D., Jiang, N., & Ali, R. (2023) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102941 · Cited by 170
Kunze, A., Summerskill, S. J., Marshall, R., & Filtness, A. J. (2019) · Ergonomics · doi:10.1080/00140139.2018.1547842 · Cited by 165 · cited 6×
“Yet, uncertainty cues also increased workload in some cases [70] but not in others [92] and led to instances of over-trust when the cues were not reliable [140].”
BackgroundInfluentialChristoforakos, L., Gallucci, A., Surmava-Große, T., Ullrich, D., & Diefenbach, S. (2021) · Frontiers in Robotics and AI · doi:10.3389/frobt.2021.640444 · Cited by 154 · cited 3×
“Likewise, some studies [22, 68, 98, 100], used the Human-Computer Trust Questionnaire by Madsen and Gregor [86], whereas others [3, 56, 83] used a scale developed by Mayer and colleagues [95].”
BackgroundResultSituational Trust Scale for Automated Driving (STS-AD): Development and Initial Validation
Holthausen, B. E., Wintersberger, P., Walker, B. N., & Riener, A. (2020) · International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI) · doi:10.1145/3409120.3410637 · Cited by 153 · cited 1×
“Other scales that were used are Holthausen’s et al. [49] Situational Trust Scale for Automated Driving, a scale used in Muir [99], the Usability and Trust Questionnaire developed by Chen [16], the Robot Trust Questionnaire by Schäfer [114], a scale to assess trust in human-robot collaborations…”
MethodToward a Revolution in Transportation Operations: AI for Complex Systems
Wang, F.-Y. (2008) · IEEE Intelligent Systems · doi:10.1109/mis.2008.112 · Cited by 149 · cited 1×
“systems which act intelligently based on available data, powered by deep learning algorithms, automation can guide credit scoring and risk assessments [42], transportation [52, 128], medical diagnostics [38], hiring decisions [78], and judicial sentencing [46] and recidivism [21].”
BackgroundLiem, C. C. S., Langer, M., Demetriou, A. M., Hiemstra, A. M. F., Wicaksana, A. S., Born, M. P., & König, C. J. (2018) · The Springer series on challenges in machine learning · doi:10.1007/978-3-319-98131-4_9 · Cited by 144
Khastgir, S., Birrell, S., Dhadyalla, G., & Jennings, P. (2018) · Transportation Research Part C: Emerging Technologies · doi:10.1016/j.trc.2018.07.001 · Cited by 134 · cited 4×
“, Level 3 - roughly 75% automation), except for one study, which included a highly automated system (Level 4) [61].”
BackgroundInfluentialYang, X. J., Schemanske, C., & Searle, C. (2023) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208211034716 · Cited by 132 · cited 1×
“trust game [3, 23], collaboration game [24], fanker task [25], image recognition [138, 144], Gaming fnance game [62], frst-person shooter game [67, 129], Pong [50]”
BackgroundJungmann, S. M., Klan, T., Kuhn, S., & Jungmann, F. (2019) · JMIR Formative Research · doi:10.2196/13863 · Cited by 127
Timing Is Key for Robot Trust Repair
Robinette, P., Howard, A. M., & Wagner, A. R. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-25554-5_57 · Cited by 121 · cited 2×
“In contrast to that, Robinette et al. [112] and Schelble et al. [116] found no efects of apologies on trust repair.”
BackgroundResultNourani, M., King, J., & Ragan, E. (2020) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v8i1.7469 · Cited by 116 · cited 2×
“Income prediction [40, 145], number pattern discovery [26, 27], pothole detection [104], classifcation of plants/animals/images [11, 75, 103, 110, 137], meal preparation [12], online dating Others [109, 141], hotel reviews [71], clearing tables [18], fnd housing [43, 124], navigation through buildings [101, 112], long…”
BackgroundOvertrust in External Cues of Automated Vehicles: An Experimental Investigation
Holländer, K., Wintersberger, P., & Butz, A. (2019) · Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications · doi:10.1145/3342197.3344528 · Cited by 109 · cited 3×
“Responding to take-over requests [2, 6, 7, 47, 61, 68–70, 77, 84, 98], collusion avoidance [8], Transportation managing (air) trafc [31, 118], pedestrians interaction with AV [48], observe AVs [64, 119], drive in driving simulator [87, 96, 136]”
BackgroundIndividual Differences in the Calibration of Trust in Automation
Pop, V. L., Shrewsbury, A., & Durso, F. T. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814564422 · Cited by 109
Explainable Active Learning (XAL): Toward AI Explanations as Interfaces for Machine Teachers
Ghai, B., Liao, Q. V., Zhang, Y., Bellamy, R., & Mueller, K. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3432934 · Cited by 106
The World is not Enough: Trust in Cognitive Agents
de Visser, E. J., Krueger, F., McKnight, P., Scheid, S., Smith, M., Chalk, S., & Parasuraman, R. (2012) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1071181312561062 · Cited by 105
Kim, T., & Song, H. (2021) · Telematics and Informatics · doi:10.1016/j.tele.2021.101595 · Cited by 101
Lu, Y., & Sarter, N. (2019) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2019.2930980 · Cited by 97
Rechkemmer, A., & Yin, M. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501967 · Cited by 97
News Recommendation Systems - Accomplishments, Challenges & Future Directions
Feng, C., Khan, M., Rahman, A. U., & Ahmad, A. (2020) · IEEE Access · doi:10.1109/access.2020.2967792 · Cited by 96
Schelble, B. G., Lopez, J., Textor, C., Zhang, R., McNeese, N. J., Pak, R., & Freeman, G. (2024) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208221116952 · Cited by 96
Establishing Appropriate Trust via Critical States
Huang, S. H., Bhatia, K., Abbeel, P., & Dragan, A. D. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593649 · Cited by 94 · cited 5×
“One subclass of system failures was the manipulation of error types by, for example, Huang et al. [50], who showed participants either a correct action, false-negatives, weak false-positives, false positives, or an incorrect action.”
BackgroundMethodInfluentialExplanations and trust: What happens to trust when a robot partner does something unexpected?
Lyons, J. B., Hamdan, I. A., & Vo, T. Q. (2023) · Computers in Human Behavior · doi:10.1016/j.chb.2022.107473 · Cited by 93
User Trust Dynamics: An Investigation Driven by Differences in System Performance
Yu, K., Berkovsky, S., Taib, R., Conway, D., Zhou, J., & Chen, F. (2017) · Proceedings of the 22nd International Conference on Intelligent User Interfaces · doi:10.1145/3025171.3025219 · Cited by 90 · cited 3×
“In fact, we noticed that only a few studies recruited experts to test their hypotheses [31, 66, 89, 90, 100, 113, 137, 140, 142].”
BackgroundCredit Scoring: A Review on Support Vector Machines and Metaheuristic Approaches
Goh, R. Y., & Lee, L. S. (2019) · Advances in Operations Research · doi:10.1155/2019/1974794 · Cited by 87
Transparency in HRI: Trust and Decision Making in the Face of Robot Errors
Nesset, B., Robb, D. A., Lopes, J., & Hastie, H. (2021) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/3434074.3447183 · Cited by 85 · cited 1×
“…[25], image recognition [138, 144], Gaming fnance game [62], frst-person shooter game [67, 129], Pong [50] Medicine diagnostics [100], robot triage [102] Income prediction [40, 145], number pattern discovery [26, 27], pothole detection [104], classif-cation of plants/animals/images [11, 75, 103,…”
BackgroundYou Complete Me: Human-AI Teams and Complementary Expertise
Zhang, Q., Lee, M. L., & Carter, S. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517791 · Cited by 85
Merritt, S. M., Lee, D., Unnerstall, J. L., & Huber, K. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814561675 · Cited by 83 · cited 8×
“[94] also discuss the possible limitation of this procedure: “Individuals with a great deal of trust sensitivity (meaning that their trust changes dramatically as actual reliability changes) might actually either overreact or under-”
BackgroundMethodInfluentialTrust repair in human-agent teams: the effectiveness of explanations and expressing regret
Kox, E. S., Kerstholt, J. H., Hueting, T. F., & de Vries, P. W. (2021) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-021-09515-9 · Cited by 83 · cited 3×
“…trust game [3, 23], collaboration game [24], fanker task [25], image recognition [138, 144], Gaming fnance game [62], frst-person shooter game [67, 129], Pong [50] Medicine diagnostics [100], robot triage [102] Income prediction [40, 145], number pattern discovery [26, 27], pothole detection…”
BackgroundNo Risk No Trust: Investigating Perceived Risk in Highly Automated Driving
Li, M., Holthausen, B. E., Stuck, R. E., & Walker, B. N. (2019) · Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications · doi:10.1145/3342197.3344525 · Cited by 80
Do You Still Trust Me? Human-Robot Trust Repair Strategies
Esterwood, C., & Robert, L. P. (2021) · 2021 30th IEEE International Conference on Robot & Human Interactive Communication (RO-MAN) · doi:10.1109/ro-man50785.2021.9515365 · Cited by 75 · cited 4×
“However, Esterwood and Robert [34] could show that promising no errors in the future as well as explanations for why the error happened showed greater efects than mere apologies (see also results by [136]).”
BackgroundInfluentialEffects of Explanations in AI-Assisted Decision Making: Principles and Comparisons
Wang, X., & Yin, M. (2022) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3519266 · Cited by 74
Learning From the Slips of Others: Neural Correlates of Trust in Automated Agents
de Visser, E. J., Beatty, P. J., Estepp, J. R., Kohn, S., Abubshait, A., Fedota, J. R., & McDonald, C. G. (2018) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2018.00309 · Cited by 73
Theoretical considerations and development of a questionnaire to measure trust in automation
Körber, M. (2018) · doi:10.31234/osf.io/nfc45 · Cited by 71
Do I trust my machine teammate?: an investigation from perception to decision
Yu, K., Berkovsky, S., Taib, R., Zhou, J., & Chen, F. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302277 · Cited by 71 · cited 4×
“In many cases, trust measures were accompanied by subconstructs of trust such as perceptions of system accuracy [90, 143], predictability [68, 73], and reliability [15, 68, 107, 133], or trust related measures such as perceived usability [4], frustration [4], perceived humanness [26, 27, 55, 58], perceived intelligenc…”
BackgroundInfluentialBehavioral Measurement of Trust in Automation: The Trust Fall
Miller, D., Johns, M., Mok, B., Gowda, N., Sirkin, D., Lee, K., & Ju, W. (2016) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931213601422 · Cited by 71 · cited 1×
“Responding to take-over requests [2, 6, 7, 47, 61, 68–70, 77, 84, 98], collusion avoidance [8], Transportation managing (air) trafc [31, 118], pedestrians interaction with AV [48], observe AVs [64, 119], drive in driving simulator [87, 96, 136]”
BackgroundAzevedo-Sa, H., Zhao, H., Esterwood, C. T., Yang, X. J., Tilbury, D. M., & Robert, L. (2021) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2021.102973 · Cited by 70
Capable but Amoral? Comparing AI and Human Expert Collaboration in Ethical Decision Making
Tolmeijer, S., Christen, M., Kandul, S., Kneer, M., & Bernstein, A. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517732 · Cited by 70
Ullrich, D., Butz, A., & Diefenbach, S. (2021) · Frontiers in Robotics and AI · doi:10.3389/frobt.2021.554578 · Cited by 69 · cited 1×
“…141], hotel reviews [71], clearing tables [18], fnd housing [43, 124], navigation through buildings [101, 112], long-distance management of robots [126] studies of this survey, it was difcult to gain insights into what powered the automated systems, as many authors merely described their systems…”
BackgroundToward Adaptive Trust Calibration for Level 2 Driving Automation
Akash, K., Jain, N., & Misu, T. (2020) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3382507.3418885 · Cited by 68 · cited 2×
“Most predominantly, we found this functionality in the transportation domain for autonomous vehicles [2, 47, 61, 68– 70 , 84 ] but also Unlike cooperative tasks, users are no longer in the position to choose whether they want to rely on or reject a system’s decision/action but instead users are…”
BackgroundThe Impact of Training on Human–Autonomy Team Communications and Trust Calibration
Johnson, C. J., Demir, M., McNeese, N. J., Gorman, J. C., Wolff, A. T., & Cooke, N. J. (2023) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208211047323 · Cited by 68 · cited 8×
“workload [58], and understanding of the system [132].”
BackgroundInfluentialSecond Chance for a First Impression? Trust Development in Intelligent System Interaction
Tolmeijer, S., Gadiraju, U., Ghantasala, R., Gupta, A., & Bernstein, A. (2021) · Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization · doi:10.1145/3450613.3456817 · Cited by 67 · cited 1×
“Income prediction [40, 145], number pattern discovery [26, 27], pothole detection [104], classifcation of plants/animals/images [11, 75, 103, 110, 137], meal preparation [12], online dating Others [109, 141], hotel reviews [71], clearing tables [18], fnd housing [43, 124], navigation through buildings [101, 112], long…”
BackgroundTrust Repair Strategies with Self-Driving Vehicles: An Exploratory Study
Kohn, S. C., Quinn, D., Pak, R., de Visser, E. J., & Shaw, T. H. (2018) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931218621254 · Cited by 64 · cited 2×
“…61, 68–70, 77, 84, 98], collusion avoidance [8], Transportation managing (air) trafc [31, 118], pedestrians interaction with AV [48], observe AVs [64, 119], drive in driving simulator [87, 96, 136] Screening tasks [17, 33, 44, 63, 81, 82, 127, 130, 135, 139, 140], gathering of information [58],…”
BackgroundEffect of automation failure type on trust development in driving automation systems
Mishler, S., & Chen, J. (2022) · Applied Ergonomics · doi:10.1016/j.apergo.2022.103913 · Cited by 62 · cited 1×
“Likewise, some studies [22, 68, 98, 100], used the Human-Computer Trust Questionnaire by Madsen and Gregor [86], whereas others [3, 56, 83] used a scale developed by Mayer and colleagues [95].”
ResultAn improved approach for movie recommendation system
(2017) · doi:10.1109/i-smac.2017.8058367 · Cited by 59 · cited 1×
“Moreover, automated systems help people decide what to wear [125], what to watch [1] or listen to [115], which news to read [37], and whom to date [134].”
BackgroundM. Faas, S., Kraus, J., Schoenhals, A., & Baumann, M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445738 · Cited by 57 · cited 7×
“We found 16 studies (see, e.g., [26, 68, 69, 84, 107]) which used the Trust in Automation Scale by Jian et al. [57], at least two [10, 94] studies used a scale developed by Merritt et al. [93], and at least fve studies (see e.g., [4, 130]) used the trust predisposition scale by McKnight et al. [91].”
BackgroundMethodInfluentialGupta, A., Basu, D., Ghantasala, R., Qiu, S., & Gadiraju, U. (2022) · Proceedings of the ACM Web Conference 2022 · doi:10.1145/3485447.3512248 · Cited by 57 · cited 3×
“[71, 109, 110, 132, 141] and eight which specifcally referred to artifcial intelligence [11, 12, 43, 83, 100, 118, 144, 145].”
BackgroundTextor, C., Zhang, R., Lopez, J., Schelble, B. G., McNeese, N. J., Freeman, G., Pak, R., Tossell, C., & de Visser, E. J. (2022) · Journal of Cognitive Engineering and Decision Making · doi:10.1177/15553434221113964 · Cited by 57
Robot Capability and Intention in Trust-Based Decisions Across Tasks
Xie, Y., Bodala, I. P., Ong, D. C., Hsu, D., & Soh, H. (2019) · 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2019.8673084 · Cited by 57 · cited 1×
“Screening tasks [17, 33, 44, 63, 81, 82, 127, 130, 135, 139, 140], gathering of information [58], Military mission planning [92], human-AI collaboration for search and destroy missions [116]”
BackgroundLanger, M., Hunsicker, T., Feldkamp, T., König, C. J., & Grgić-Hlača, N. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517527 · Cited by 55
Chen, Y., Zahedi, F. M., Abbasi, A., & Dobolyi, D. (2021) · Information & Management · doi:10.1016/j.im.2020.103394 · Cited by 53 · cited 6×
“Focusing more on the context than on the user or the system, Chen et al. [19] found that trustworthiness perceptions also depended on the severity of the outcome, with more severe outcomes reducing the perceived trustworthiness.”
BackgroundInfluentialChen, J., Mishler, S., Hu, B., Li, N., & Proctor, R. W. (2018) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2018.05.010 · Cited by 50 · cited 7×
“…measures were accompanied by subcon-structs of trust such as perceptions of system accuracy [90, 143], predictability [68, 73], and reliability [15, 68, 107, 133], or trust related measures such as perceived usability [4], frustration [4], perceived humanness [26, 27, 55, 58], perceived…”
BackgroundResultInfluentialApplication of a System-Wide Trust Strategy when Supervising Multiple Autonomous Agents
Walliser, J. C., de Visser, E. J., & Shaw, T. H. (2016) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931213601031 · Cited by 49 · cited 3×
“…118], pedestrians interaction with AV [48], observe AVs [64, 119], drive in driving simulator [87, 96, 136] Screening tasks [17, 33, 44, 63, 81, 82, 127, 130, 135, 139, 140], gathering of information [58], Military mission planning [92], human-AI collaboration for search and destroy missions [116]…”
BackgroundResultDaronnat, S., Azzopardi, L., Halvey, M., & Dubiel, M. (2021) · Frontiers in Robotics and AI · doi:10.3389/frobt.2021.642201 · Cited by 45 · cited 5×
“…[5], moving objects [34, 35, 45], demand Production forecasting [39], harvesting [113], quality checks [142] trust game [3, 23], collaboration game [24], fanker task [25], image recognition [138, 144], Gaming fnance game [62], frst-person shooter game [67, 129], Pong [50] Medicine diagnostics…”
BackgroundMethodInfluentialTwo Routes to Trust Calibration: Effects of Reliability and Brand Information on Trust in Automation
Kraus, J. M., Forster, Y., Hergeth, S., & Baumann, M. (2019) · International Journal of Mobile Human Computer Interaction · doi:10.4018/ijmhci.2019070101 · Cited by 42 · cited 10×
“However, other factors have also been identifed to increase the trust users bestow in a system, such as a system’s reputation [69] or anthropomorphism [26, 55].”
BackgroundMethodResultInfluentialAn Investigation of Drivers’ Dynamic Situational Trust in Conditionally Automated Driving
Ayoub, J., Avetisyan, L., Makki, M., & Zhou, F. (2021) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2021.3131676 · Cited by 42 · cited 1×
“…[89, 90], crime prevention [10], recidivism prediction [132], watch a video of a house search [66, 83] Responding to take-over requests [2, 6, 7, 47, 61, 68–70, 77, 84, 98], collusion avoidance [8], Transportation managing (air) trafc [31, 118], pedestrians interaction with AV [48], observe…”
BackgroundChen, J., Mishler, S., & Hu, B. (2021) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2021.3051137 · Cited by 40 · cited 2×
“Yet, others found positive efects of feedback [14, 15, 94].”
BackgroundAn Intelligent Personalized Fashion Recommendation System
Tu, Q., & Dong, L. (2010) · International Conference on Communications, Circuits and Systems · doi:10.1109/icccas.2010.5581949 · Cited by 39 · cited 1×
“Moreover, automated systems help people decide what to wear [125], what to watch [1] or listen to [115], which news to read [37], and whom to date [134].”
BackgroundTrust Mediating Reliability–Reliance Relationship in Supervisory Control of Human–Swarm Interactions
Hussein, A., Elsawah, S., & Abbass, H. A. (2020) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720819879273 · Cited by 39 · cited 1×
“Task Screening of dangerous objects/subjects [4, 14, 15, 19, 36, 51, 54–56, 85, 94, 107, 117], Security & Safety detection of system malfunctions [89, 90], crime prevention [10], recidivism prediction [132], watch a video of a house search [66, 83] Responding to take-over requests [2, 6, 7, 47, 61,…”
BackgroundCan AI become Reliable Source to Support Human Decision Making in a Court Scene?
Hayashi, Y., & Wakabayashi, K. (2017) · Companion of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing · doi:10.1145/3022198.3026338 · Cited by 37 · cited 1×
“…sophisticated systems which act intelligently based on available data, powered by deep learning algorithms, automation can guide credit scoring and risk assessments [42], transportation [52, 128], medical diagnostics [38], hiring decisions [78], and judicial sentencing [46] and recidi-vism [21].”
BackgroundHaving the Right Attitude: How Attitude Impacts Trust Repair in Human—Robot Interaction
Esterwood, C., & Robert, L. P. (2022) · 2022 17th ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri53351.2022.9889535 · Cited by 36
The World is not Enough: Trust in Cognitive Agents: (572172013-055)
de Visser, E. J., Krueger, F., McKnight, P., Scheid, S., Smith, M., Chalk, S., & Parasuraman, R. (2012) · PsycEXTRA Dataset · doi:10.1037/e572172013-055 · Cited by 34
Detecting Human Trust Calibration in Automation: A Convolutional Neural Network Approach
Choo, S., & Nam, C. (2022) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2021.3137015 · Cited by 34 · cited 4×
“the system’s accuracy [19, 94], the system’s ability [132], and the system’s credibility [20].”
BackgroundInfluentialBobko, P., Hirshfield, L., Eloy, L., Spencer, C., Doherty, E., Driscoll, J., & Obolsky, H. (2023) · Theoretical Issues in Ergonomics Science · doi:10.1080/1463922x.2022.2086644 · Cited by 33 · cited 4×
“well [10, 55] Interestingly, we noticed that a large domain, medicine/healthcare, commonly characterized as a high-stakes”
BackgroundMethodInfluentialInteraction of Automation Visibility and Information Quality in Flight Deck Information Automation
Dorneich, M. C., Dudley, R., Letsu-Dake, E., Rogers, W., Whitlow, S. D., Dillard, M. C., & Nelson, E. T. (2017) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2017.2717939 · Cited by 31 · cited 2×
“In fact, we noticed that only a few studies recruited experts to test their hypotheses [31, 66, 89, 90, 100, 113, 137, 140, 142].”
BackgroundInvestigating the Effects of (Empty) Promises on Human-Automation Interaction and Trust Repair
Albayram, Y., Jensen, T., Khan, M. M. H., Fahim, M. A. A., Buck, R., & Coman, E. (2020) · Proceedings of the 8th International Conference on Human-Agent Interaction · doi:10.1145/3406499.3415064 · Cited by 31 · cited 2×
“In many cases, trust measures were accompanied by subconstructs of trust such as perceptions of system accuracy [90, 143], predictability [68, 73], and reliability [15, 68, 107, 133], or trust related measures such as perceived usability [4], frustration [4], perceived humanness [26, 27, 55, 58], perceived intelligenc…”
BackgroundMethodTrust Dynamics and Verbal Assurances in Human Robot Physical Collaboration
Alhaji, B., Prilla, M., & Rausch, A. (2021) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2021.703504 · Cited by 29 · cited 3×
“Some authors work with correlations to get a better sense of perceived trustworthiness and system capabilities [5, 12, 24, 39, 66, 67, 81, 94, 130, 139, 142].”
BackgroundWhen Should a Robot Apologize? Understanding How Timing Affects Human-Robot Trust Repair
Nayyar, M., & Wagner, A. R. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-05204-1_26 · Cited by 28 · cited 1×
“Income prediction [40, 145], number pattern discovery [26, 27], pothole detection [104], classifcation of plants/animals/images [11, 75, 103, 110, 137], meal preparation [12], online dating Others [109, 141], hotel reviews [71], clearing tables [18], fnd housing [43, 124], navigation through buildings [101, 112], long…”
BackgroundHaving the Right Attitude: How Attitude Impacts Trust Repair in Human—Robot Interaction
Esterwood, C., & Robert, L. (2022) · IEEE/ACM International Conference on Human-Robot Interaction · doi:10.5555/3523760.3523806 · Cited by 28
Jensen, T., Khan, M. M. H., Fahim, M. A. A., & Albayram, Y. (2021) · Designing Interactive Systems Conference 2021 · doi:10.1145/3461778.3462102 · Cited by 26 · cited 1×
“Likewise, some studies [22, 68, 98, 100], used the Human-Computer Trust Questionnaire by Madsen and Gregor [86], whereas others [3, 56, 83] used a scale developed by Mayer and colleagues [95].”
ResultJensen, T., Albayram, Y., Khan, M. M. H., Fahim, M. A. A., Buck, R., & Coman, E. (2019) · Proceedings of the 2019 on Designing Interactive Systems Conference · doi:10.1145/3322276.3322349 · Cited by 24
The Transfer of Human Trust in Robot Capabilities across Tasks
Soh, H., Pan, S. L., Chen, M., & Hsu, D. (2018) · Robotics: Science and Systems · doi:10.15607/rss.2018.xiv.033 · Cited by 23
Seong, Y., Bisantz, A. M., & Bisantz, A. M. (2002) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120204600344 · Cited by 22 · cited 6×
“Moreover, providing cognitive feedback information[117] or forcing users to critically assess the trustworthiness of a system, [12] improved the trust calibration.”
BackgroundMethodInfluentialKox, E. S., Siegling, L. B., & Kerstholt, J. H. (2022) · International Journal of Social Robotics · doi:10.1007/s12369-022-00871-4 · Cited by 22 · cited 2×
“Some authors work with correlations to get a better sense of perceived trustworthiness and system capabilities [5, 12, 24, 39, 66, 67, 81, 94, 130, 139, 142].”
BackgroundTrust: from the Philosophical to the Commercial
Tallant, J., & Donati, D. (2019) · Philosophy of Management · doi:10.1007/s40926-019-00107-y · Cited by 18 · cited 1×
“Reliance is used for inanimate objects [9], for which, when reliance fails, we do not feel emotions of betrayal [121].”
BackgroundLlinas, J., Bisantz, A. M., Drury, C., Seong, Y., & Jian, J.-Y. (1998) · Defense Technical Information Center (DTIC) · doi:10.21236/ada370937 · Cited by 18
How Do Visual Explanations Foster End Users' Appropriate Trust in Machine Learning?Preprint
Yang, F. (2021) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/s9q8t · Cited by 18
Chen, J. Y. C., Barnes, M. J., & Kenny, C. (2011) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/1957656.1957793 · Cited by 17 · cited 1×
“Screening tasks [17, 33, 44, 63, 81, 82, 127, 130, 135, 139, 140], gathering of information [58], Military mission planning [92], human-AI collaboration for search and destroy missions [116]”
BackgroundHuman-Robot Trust Assessment Using Top-Down Visual Tracking After Robot Task Execution Mistakes
Hald, K., Rehm, M., & Moeslund, T. B. (2021) · IEEE RO-MAN · doi:10.1109/ro-man50785.2021.9515501 · Cited by 15
“Reliance and automation for Human-AI collaborative data labeling conflict resolution,”
Brachman, M., Ashktorab, Z., Desmond, M., Duesterwald, E., Dugan, C., Joshi, N. N., Pan, Q., & Sharma, A. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555212 · Cited by 14 · cited 2×
“[71, 109, 110, 132, 141] and eight which specifcally referred to artifcial intelligence [11, 12, 43, 83, 100, 118, 144, 145].”
Background(2021) · doi:10.1145/3461778.3461997 · Cited by 14 · cited 2×
“[36] suggested that emotional reactions towards a system can partly explain the resulting trust attitude.”
BackgroundTrust Miscalibration Is Sometimes Necessary: An Empirical Study and a Computational Model
Collins, M. G., & Juvina, I. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.690089 · Cited by 14
Chen, J. Y. C. (2009) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/1514095.1514128 · Cited by 12 · cited 9×
“Similary, Chen and colleagues [18] developed a computational model, the Partially observable Markov decision process (POMDP), which integrates the extend users trust the system into the system’s decision making by, among others, inferring users’ trust through interactions.”
MethodInfluentialTrust Repair in Performance, Process, and Purpose Factors of Human-Robot Ttust
Alarcon, G. M., Gibson, A. M., & Jessup, S. A. (2020) · 2020 IEEE International Conference on Human-Machine Systems (ICHMS) · doi:10.1109/ichms49158.2020.9209453 · Cited by 12 · cited 2×
“Likewise, some studies [22, 68, 98, 100], used the Human-Computer Trust Questionnaire by Madsen and Gregor [86], whereas others [3, 56, 83] used a scale developed by Mayer and colleagues [95].”
BackgroundResultThe Role of Behavioral Anthropomorphism in Human-Automation Trust Calibration
Jensen, T., Khan, M. M. H., & Albayram, Y. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-50334-5_3 · Cited by 12 · cited 10×
“However, other factors have also been identifed to increase the trust users bestow in a system, such as a system’s reputation [69] or anthropomorphism [26, 55].”
BackgroundInfluentialGuznov, S., Nelson, A., Lyons, J., & Dycus, D. (2015) · Communications in Computer and Information Science · doi:10.1007/978-3-319-21383-5_103 · Cited by 11 · cited 2×
“[44] found that increased workload led to more reliance on a system.”
BackgroundDo We Blindly Trust Self-Driving Cars
Andersen, K. E., Köslich, S., Pedersen, B. K. M. K., Weigelin, B. C., & Jensen, L. C. (2017) · Proceedings of the Companion of the 2017 ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/3029798.3038428 · Cited by 9 · cited 3×
“Ribeiro et al. [110] found that displaying class probabilities did not help users to appropriately assess a system’s trustworthiness (see also [6]).”
BackgroundManchon, J. B., Beaufort, R., Bueno, M., & Navarro, J. (2022) · Information · doi:10.3390/info13100480 · Cited by 8 · cited 1×
“Responding to take-over requests [2, 6, 7, 47, 61, 68–70, 77, 84, 98], collusion avoidance [8], Transportation managing (air) trafc [31, 118], pedestrians interaction with AV [48], observe AVs [64, 119], drive in driving simulator [87, 96, 136]”
BackgroundKluck, M., Kohn, S. C., Walliser, J. C., de Visser, E. J., & Shaw, T. H. (2018) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931218621253 · Cited by 8 · cited 1×
“Screening tasks [17, 33, 44, 63, 81, 82, 127, 130, 135, 139, 140], gathering of information [58], Military mission planning [92], human-AI collaboration for search and destroy missions [116]”
BackgroundEffect of Shared Information on Trust and Reliance in a Demand Forecasting Task
Gao, J., & Lee, J. D. (2006) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120605000302 · Cited by 8
Leichtmann, B., Humer, C., Hinterreiter, A., et al. (2022) · doi:10.31219/osf.io/n4w6u · Cited by 8
McGuirl, J. M., & Sarter, N. B. (2003) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120304700360 · Cited by 8
Lu, Y., & Sarter, N. (2019) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1071181319631345 · Cited by 5 · cited 6×
“To achieve an alignment after an interaction, some studies provide feedback about the user and system performance (see e.g., [5, 14, 82]).”
BackgroundInfluentialWang, L., Jamieson, G. A., & Hollands, J. G. (2008) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120805200420 · Cited by 5
Exploring the impact of classification probabilities on users' trust in ambiguous instances
Dos Santos Ribeiro, D., Barbosa, G. D. J., Do Carmo Silva, M., Lopes, H., & Barbosa, S. D. J. (2021) · 2021 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl/hcc51201.2021.9576291 · Cited by 4 · cited 5×
“Income prediction [40, 145], number pattern discovery [26, 27], pothole detection [104], classifcation of plants/animals/images [11, 75, 103, 110, 137], meal preparation [12], online dating Others [109, 141], hotel reviews [71], clearing tables [18], fnd housing [43, 124], navigation through buildings [101, 112], long…”
BackgroundMethodInfluentialSanchez, J., Fisk, A. D., & Rogers, W. A. (2006) · 2006 9th International Conference on Control, Automation, Robotics and Vision · doi:10.1109/icarcv.2006.345296 · Cited by 2 · cited 2×
“that experts were generally less willing to rely on automation than lay users [113].”
BackgroundHow Different Explanations Impact Trust Calibration: The Case of Clinical Decision Support SystemsPreprint
Naiseh, M., Al‐Thani, D., Jiang, N., & Ali, R. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4098528 · Cited by 2
Yang, X. J., Schemanske, C., & Searle, C. (2021) · arXiv · doi:10.48550/arxiv.2107.07374
Corrigendum: Trust Dynamics and Verbal Assurances in Human Robot Physical Collaboration
Alhaji, B., Prilla, M., & Rausch, A. (2021) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2021.761184
Research works printed but not linked: 7
- 33. Na Du, Kevin Y Huang, and X Jessie Yang. 2020. Not all information is equal: ef- fects of disclosing diferent types of likelihood information on trust, compliance and reliance, and task performance in human-automation teaming. Human factors 62, 6 (2020), 987-1001.
- 60. Alexandra D Kaplan, Theresa T Kessler, J Christopher Brill, and PA Hancock. 2021. Trust in artifcial intelligence: Meta-analytic fndings. Human Factors (2021), 00187208211013988.
- 86. M. Madsen and S. Gregor. 2000. Measuring Human-Computer Trust. In Proceed- ings of 11th Australasian Conference on Information Systems. 6e8.
- 99. Bonnie Marlene Muir. 2002. Operators' trust in and use of automatic controllers in a supervisory process control task. (2002).
- 118. Mona SharifHeravi, John R Taylor, Christopher J Stanton, Sandra Lambeth, and Christopher Shanahan. 2020. It'sa Disaster! Factors Afecting Trust Devel- opment and Repair Following Agent Task Failure. In Proceedings of the 2020 Australasian Conference on Robotics and Automation (ACRA 2020), 8-10 December 2020, Brisbane, Queensland.
- 134. Yihan Wu and Ryan M Kelly. 2020. Online Dating Meets Artifcial Intelli- gence: How the Perception of Algorithmically Generated Profle Text Impacts Attractiveness and Trust. In 32nd Australian Conference on Human-Computer Interaction. 444-453.
- 136. Jin Xu and Ayanna Howard. 2022. Evaluating the Impact of Emotional Apology on Human-Robot Trust. In 2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN). IEEE, 1655-1661.
Cited by
Psychological factors underlying attitudes toward AI tools
De Freitas, J., Agarwal, S., Schmitt, B., & Haslam, N. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01734-2 · Cited by 221
“Consequently, research has sought to determine the psychological factors driving attitudes toward AI tools, and how to overcome AI resistance, so that user trust is calibrated to the system’s capabilities 8 .”
BackgroundThe role of explainable AI in the context of the AI Act
Panigutti, C., Hamon, R., Hupont, I., Fernandez Llorca, D., Fano Yela, D., Junklewitz, H., Scalzo, S., Mazzini, G., Sanchez, I., Soler Garrido, J., et al. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594069 · Cited by 137
“Many factors play a role in the relationship between trust and AI explanations [91, 119].”
BackgroundHow Do We Assess the Trustworthiness of AI? Introducing the Trustworthiness Assessment Model (TrAM)
Schlicker, N., Baum, K., Uhde, A., Sterz, S., Hirsch, M. C., & Langer, M. (2022) · doi:10.31234/osf.io/qhwvx · Cited by 79
A Systematic Review of Trust Assessments in Human–Robot Interaction
Campagna, G., & Rehm, M. (2024) · ACM Transactions on Human-Robot Interaction · doi:10.1145/3706123 · Cited by 102
Ma, S., Wang, X., Lei, Y., Shi, C., Yin, M., & Ma, X. (2024) · arXiv · doi:10.48550/arxiv.2403.09552 · Cited by 56
Metadata
- Authors
- Wischnewski, M., Krämer, N., & Müller, E.
- Affiliations
- Research Center for Trustworthy Data Science and Security, Germany
- University of Duisburg-Essen (DE)
- TU Dortmund University (DE)
- Venue
- Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2023-04-19
- Pages
- 1-16
- OpenAlex
- W4366548373
- Cited by
- 146
- References
- 111
- Citations by year
- 2026: 73 · 2025: 43 · 2024: 23 · 2023: 7
- FWCI
- 72.97
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by)
- Universitätsbibliographie, Universität Duisburg-Essen (submittedVersion)
- OSF Preprints (OSF Preprints) (submittedVersion)
- PsyArXiv (OSF Preprints) (submittedVersion, cc-by)
- PsyArXiv (OSF Preprints) (submittedVersion)
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Social Sciences › Decision Sciences › Information Systems and Management › Technology Adoption and User Behaviour
- Keywords
- trust calibration, trust in automation, trust measurement, human-automation interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.