The DOI matches its Crossref record; 43 of the 44 references it cites were found in a registry or an index; none of the 22 sources its claims cite is checked for support yet.
Adaptive trust calibration for human-AI collaboration
Open accessAbstract
Safety and efficiency of human-AI collaboration often depend on how humans could appropriately calibrate their trust towards the AI agents. Over-trusting the autonomous system sometimes causes serious safety issues. Although many studies focused on the importance of system transparency in keeping proper trust calibration, the research in detecting and mitigating improper trust calibration remains very limited. To fill these research gaps, we propose a method of adaptive trust calibration that consists of a framework for detecting the inappropriate calibration status by monitoring the user's reliance behavior and cognitive cues called "trust calibration cues" to prompt the user to reinitiate trust calibration. We evaluated our framework and four types of trust calibration cues in an online experiment using a drone simulator. A total of 116 participants performed pothole inspection tasks by using the drone's automatic inspection, the reliability of which could fluctuate depending upon the weather conditions. The participants needed to decide whether to rely on automatic inspection or to do the inspection manually. The results showed that adaptively presenting simple cues could significantly promote trust calibration during over-trust.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
The paper proposes adaptive trust calibration: a behavior-based framework detects over-trust/under-trust and presents “trust calibration cues” (TCCs) only when needed. In an online drone-simulator experiment, adaptively presented TCCs significantly increased manual choices and improved sensitivity d′ during over-trust, whereas continuous reliability information alone did not.
Key findings
- Of 194 online participants, 116 successfully inspected at least 15 checkpoints; they completed 1,740 inspections with a correct rate of 0.74, choosing automatic inspection 1,236 times (0.71) and manual inspection 504 times (0.29).
- A two-factor mixed ANOVA found a significant main effect of weather period on manual rates [F(1, 111) = 51.69, p < 0.01] and a significant interaction between TCC group and period [F(4, 111) = 4.86, p < 0.01]. The NoTCC group did not significantly change manual rates between good and bad weather, while all TCC groups did: visual [F(1, 17) = 9.20, p < 0.01], audio [F(1, 21) = 5.54, p = 0.03], verbal [F(1, 28) = 62.9, p < 0.001], and anthropomorphic [F(1, 18) = 8.55, p < 0.01].
- In bad weather, the verbal TCC group had a significantly higher manual rate than both NoTCC [t(111) = 4.77, adj.p < 0.01] and anthropomorphic TCC [t(111) = 2.74, adj.p = 0.04].
- Sensitivity d′ in bad weather was significantly higher for the verbal group (0.92, SE 0.14) than for NoTCC [t(111) = 2.97, adj.p = 0.04, Cohen’s d = 0.75]; accuracy declined in bad weather but did not differ significantly among groups, though the verbal group had the highest mean accuracy (0.72, SE 0.04).
Methods
- Online experiment using a 3D drone simulator built with CesiumJS and Bing Map API. Participants performed a pothole inspection task over 30 checkpoints (10 with potholes, 20 without), choosing between automatic inspection and manual inspection; they were randomly assigned to NoTCC, visual, audio, verbal, or anthropomorphic TCC groups.
- Automatic inspection reliability Pauto was manipulated: good weather Pauto = 90% and d′ = 1.8; bad weather Pauto = 50% and d′ = 0.1. The framework detected over-trust by observing reliance behavior and comparing Pman and Pauto, then presented a TCC when over-trust was detected.
- Main analyses were two-factor mixed ANOVAs with TCC group as between-subjects and checkpoint period as within-subjects, plus Holm–Bonferroni-adjusted post hoc comparisons and t-tests for Pman.
Limitations
- Stated: the framework focuses only on performance-related factors, ignoring other automation benefits such as faster completion, lighter workload, or fewer risks; it evaluates only an over-trust scenario; it does not guarantee convergence of behavior; and it uses only a pothole inspection/reconnaissance task and four TCC types.
- Stated: correct answers were not shown during the task, and automatic-inspection results were shown only when participants chose automatic inspection, not manual inspection; future work should test combinations of feedback information.
- Evident: attrition was high—78 of 194 participants failed to complete the task within the time limit because they moved the drone away from the checkpoints—so the analyzed sample of 116 may not represent all recruited participants.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
29 Own result · 26 Own interpretation · 11 Not checked yet · 6 Own method · 3 No basis given
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The participants of the group without TCC did not change their behavior when they were in over-trust status, even if the system reliability information was continuously presented.”
§ Adaptive trust calibration for human-AI collaboration > Introduction
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We found that adaptively presented TCCs significantly affected the choice behavior of the participants.”
§ Adaptive trust calibration for human-AI collaboration > Introduction
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“while TCC rates in the anthro. TCC group did not follow the decreasing trend.”
§ Adaptive trust calibration for human-AI collaboration > Results > TCC rates
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“and that Pman in the bad weather period was significantly larger than Pauto = 0.50 [Mean = 0.86, t(200) = 17.79, p < 0.01, Cohen’s d = 1.25].”
§ Adaptive trust calibration for human-AI collaboration > Results > Performance: Sensitivity d′ and accuracy
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- The authors state this as an assumption: Manual accuracy can be used to estimate Pman.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“One-sample t-tests showed that Pman in the good weather period was significantly smaller than Pauto = 0.90 [Mean = 0.81, t(71) = −2.17, p = 0.03, Cohen’s d = 0.26]”
§ Adaptive trust calibration for human-AI collaboration > Results > Performance: Sensitivity d′ and accuracy
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- The authors state this as an assumption: Manual accuracy can be used to estimate Pman.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A significant interaction was found between the two factors [F(4, 111) = 4.86, p < 0.01, ].”
§ Adaptive trust calibration for human-AI collaboration > Results > Manual rates
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“and the anthro. group (t(111) = 2.74, adj.p = 0.04).”
§ Adaptive trust calibration for human-AI collaboration > Results > Manual rates
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“a Welch’s t-test indicated that there was no significant difference between Pman in the good weather period and in the bad one [t(114) = −1.08, p = 0.28, Cohen’s d = 0.16].”
§ Adaptive trust calibration for human-AI collaboration > Results > Performance: Sensitivity d′ and accuracy
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- The authors state this as an assumption: Manual accuracy can be used to estimate Pman.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“No other differences between the groups were found to be significant.”
- Also stated in: § Effects of TCCs to change the participants’ behavior · plain
“The result, however, showed that its manual choice rate was not significantly larger than the one of NoTCc group.”
§ Adaptive trust calibration for human-AI collaboration > Results > Manual rates
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Effects of TCCs to change the participants’ behavior · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“TCC rates in the verbal TCC group were higher in the early part of the period and gradually decreased.”
§ Adaptive trust calibration for human-AI collaboration > Results > TCC rates
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 16
- It may state two separate claims as one. 16
- The quote only partly states this claim. 1
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The participants did the manual inspection 504 times (the choice rate = 0.29).”
§ Adaptive trust calibration for human-AI collaboration > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“however, the verbal group showed the highest mean value (0.72 (SE 0.04)), and other groups with TCCs also had better accuracy values (0.6 (SE 0.04)) than the NoTCC group.”
§ Adaptive trust calibration for human-AI collaboration > Results > Performance: Sensitivity d′ and accuracy
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The visual and audio TCC groups also showed a similar trend”
§ Adaptive trust calibration for human-AI collaboration > Results > TCC rates
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The quote only partly states this claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“They inspected the total of 1,740 CKPs from the 1st CKP to the 15th CKP, and the results of 1,282 inspections were correct, making the correct answer rate 0.74.”
§ Adaptive trust calibration for human-AI collaboration > Results
plainn = 16- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In the bad weather period, post-hoc comparisons indicated that the sensitivity d′ of the verbal group, which was the highest (0.92 (SE 0.14) among all the groups, was significantly higher than the sensitivity d′ of the NoTCC group [t(111) = 2.97, adj.p = 0.04, Cohen’s d = 0.75].”
§ Adaptive trust calibration for human-AI collaboration > Results > Performance: Sensitivity d′ and accuracy
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Automatic inspection was selected 1,236 times (the choice rate = 0.71).”
§ Adaptive trust calibration for human-AI collaboration > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Though there was no statistical difference in accuracy among the groups, all the groups with TCCs also showed better accuracy than the NoTCC group.”
§ Adaptive trust calibration for human-AI collaboration > Discussion > Performance
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“To solve this dilemma that could perpetuate an inappropriate status of trust calibration, a new trigger would be necessary in instances of over-trust or under-trust.”
§ Adaptive trust calibration for human-AI collaboration > Introduction > Our approach: Adaptive trust calibration
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
In the paper
“Calibration can only occur in response to new evidence that may change the users’ prevailing recognition, while no new evidence can be learned without changing the current behavior first [5].”
§ Adaptive trust calibration for human-AI collaboration > Introduction > Our approach: Adaptive trust calibration
What the sources say
- Not checked yetcites [5] Adams BD, Bruyn LE, Houde S. Trust in automated systems literature review. Defence Research and Development Canada; 2003. Report No. CR-2003-096.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Regarding online experiments in general, Crump et al. [32] showed that the data collected online using a web-browser seemed mostly in line with laboratory results, so long as the experiment methods were solid.”
§ Adaptive trust calibration for human-AI collaboration > Materials and methods > Participants
What the sources say
- Not checked yetcites [32] MJC Crump, 2013, Evaluating amazon’s mechanical turk as a tool for experimental behavioral research, PLoS ONE source
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This approach is to mitigate over-trust or under-trust, in contrast with the traditional approach of trying to maintain appropriate trust calibration with continuous system transparency.”
§ Adaptive trust calibration for human-AI collaboration > Introduction > Our approach: Adaptive trust calibration
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
It may state two separate claims as one.
In the paper
“Behaviors might not be observable if there is no interaction; nevertheless, behavioral measures are practical and can easily be used as a basis for modeling and prediction [25].”
§ Adaptive trust calibration for human-AI collaboration > Introduction > Measuring trust
What the sources say
- Not checked yetcites [25] K Drnec, 2016, From trust in automation to decision neuroscience: applying cognitive neuroscience methods to understand and improve interaction decisions involved in human automation interaction, Frontires In Human Neuroscience
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Instead of directly measuring Ptrust or , the first inequalities of (1) and (2) can be estimated by observing the user’s reliance behavior; thus, the trust calibration status can be detected, if the second inequalities of Pman and Pauto can be estimated.”
§ Adaptive trust calibration for human-AI collaboration > Introduction > Our approach: Adaptive trust calibration
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“194 participants joined the experiment online.”
§ Adaptive trust calibration for human-AI collaboration > Materials and methods > Participants
plainn = 194- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“As for the successful participants, their ages ranged from 20 to 69 years old (M = 43.25, SD = 14.01), 66 participants were male and 50 were females.”
§ Adaptive trust calibration for human-AI collaboration > Results
plainn = 16- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“28 were in the NoTCC group, 18 in the visual group, 22 in the audio group, 29 in the verbal group, and 19 in the anthro. group.”
§ Adaptive trust calibration for human-AI collaboration > Results
plainn = 16- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Seventy eight participants unintentionally moved the drone far from the area where the CKPs were located, and they failed to complete the tasks within the time limit.”
§ Adaptive trust calibration for human-AI collaboration > Results
plainn = 78- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“One hundred sixteen participants successfully inspected 15 CKPs or more within the time limit.”
§ Adaptive trust calibration for human-AI collaboration > Results
plainn = 116- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Participants were randomly assigned to one of five groups with the corresponding TCCs: NoTCC group (without TCCs), visual group (with the visual sign TCC), audio group (with the audio TCC), verbal group (with the verbal TCC), and anthro. group (with the anthropomorphic TCC).”
§ Adaptive trust calibration for human-AI collaboration > Materials and methods > Procedure
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“TCCs were given right after the behavior only if the participants were judged to over-trust, so that it would be easier for the participants to understand the implication of the cues and to move forward in the trust calibration process.”
§ Adaptive trust calibration for human-AI collaboration > Discussion > Adaptive method vs. continuous method
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
In the paper
“However, automation could be beneficial beyond a better performance, such as for faster task completion, lighter workloads, fewer risks, etc. [3]”
§ Adaptive trust calibration for human-AI collaboration > Discussion > Limitations and future research
What the sources say
- Not checked yetcites [3] R Parasuraman, 1997, Humans and automation: Use, misuse, disuse, abuse, Human Factors source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Previous studies emphasized the importance of the system transparency for proper trust calibration.”
§ Adaptive trust calibration for human-AI collaboration > Conclusion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These results indicated that adaptively presenting TCCs promoted appropriate trust calibration leading to the better performance in the bad weather period.”
§ Adaptive trust calibration for human-AI collaboration > Discussion > Performance
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“It was obvious that the participants recognized these visually impressive TCCs; however, the results suggested that just being salient on the screen was not enough for some of the participants to change their choice behavior [31, 40].”
§ Adaptive trust calibration for human-AI collaboration > Discussion > Effects of TCCs to change the participants’ behavior
What the sources say
- Not checked yetcites [31] KR Laughery, 2014, A three-stage model summarizes product warning and environmental sign research, Safety Science source
- Not checked yetcites [40] VC Conzola, 2001, A Communication—Human Information Processing (C–HIP) approach to warning effectiveness in the workplace, Journal of Risk Research source
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The proposed method deals with users’ behaviors to choose Auto or Manual and does not guarantee the convergence of the behaviors, which is the result of trust calibration by users.”
§ Adaptive trust calibration for human-AI collaboration > Discussion > Limitations and future research
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“Therefore, trust as the observable and rational choice behavior can be a product of believing that these benefits will outweigh the costs [5].”
§ Adaptive trust calibration for human-AI collaboration > Discussion > Limitations and future research
What the sources say
- Not checked yetcites [5] Adams BD, Bruyn LE, Houde S. Trust in automated systems literature review. Defence Research and Development Canada; 2003. Report No. CR-2003-096.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The feedback information given for each inspection was very important for the participants to make decisions.”
§ Adaptive trust calibration for human-AI collaboration > Discussion > Limitations and future research
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The pothole inspection task in the experiment is a remote sensing task, and it would be quite difficult for the system to know the correct answer (ground truth) at the time of each inspection in practical situations because the only information available is the image data and the results of automatic recognition.”
§ Adaptive trust calibration for human-AI collaboration > Discussion > Limitations and future research
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The authors state this as an assumption.
Pauto fluctuates more widely than Pman under changing weather conditions.
“Therefore, we assumed that Pauto would fluctuate more widely than Pman under changing weather conditions.”
The paper relies on this without support.
Trust is a latent construct and cannot be directly measured; measuring trust experimentally is difficult in general.
“Trust is a latent construct and cannot be directly measured; measuring trust experimentally is difficult in general.”
The paper relies on this without support.
Users often fail to calibrate their trust in the agent and end up in over-trust or under-trust.
“While an agent’s reliability changes for various reasons in an environment, the users often fail to calibrate their trust in the agent and end up in the status called over-trust or under-trust.”
The paper relies on this without support.
Self-report measures are too intrusive to be viable in applied settings.
“Most previous literature used self-report measures such as subjective rating with questionnaires [18] or rating scales [19, 20]; however, self-report measures are too intrusive to be viable in applied settings.”
The paper relies on this without support.
Physiological and neural measures usually require special hardware.
“Although these measures can be used for dynamic tracking of trust, they usually require special hardware.”
The paper relies on this without support.
TCC rates are a valid indicator of over-trust or trust calibration status.
“TCC rates in the verbal TCC group were higher in the early part of the period and gradually decreased. This indicates that over-trust decreased during this period.”
The authors state this as an assumption.
Manual accuracy can be used to estimate Pman.
“The results of the manual accuracy were consistent with our assumption to estimate Pman − Pauto in this experiment.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Notwithstanding a clear agenda on the issues that we should explore further, we still have limited evidence-based knowledge of the context, design, and psychological mechanisms that would craft an optimal human–ai team15–17.”
From Experimental evidence of effective human–AI collaboration in medical decision-making (2022) · cites this paper as [16]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“They also found that giving additional trigger signals was effective in resolving this problem [16].”
From Dynamic Selection of Reliance Calibration Cues With AI Reliance Model (2023) · cites this paper as [16]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Avoiding misplaced trust in AI results is not essentially a technical challenge (Okamura and Yamada 2020).”
From Keeping the organization in the loop: a socio-technical extension of human-centered artificial intelligence (2023) · cites this paper as Okamura and Yamada 2020
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“In their experiment, participants did not change their over-reliance in spite of a reliability indicator being continuously displayed to them.”
From Dynamic Selection of Reliance Calibration Cues With AI Reliance Model (2023) · cites this paper as [16]
The cited source says what the claim attributes to it.
In the citing paper
“Various interventions have been explored to foster trust calibration, such as displaying confidence scores that indicate the likelihood of a model's outputs being correct to end users [124], and designing adaptive monitoring systems that visually prompt users to calibrate their trust when over-trust or under-trust is detected during human-AI collaboration [87]”
From Understanding the Practices, Perceptions, and (Dis)Trust of Generative AI among Instructors: A Mixed-methods Study in the U.S. Higher Education (2025) · cites this paper as [87]
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- ConsistentF(1, 111) = 51.69reported p < .01 · recomputed p < .001
“F(1, 111) = 51.69, p < .01”
- ConsistentF(4, 111) = 4.86reported p < .01 · recomputed p = .001
“F(4, 111) = 4.86, p < .01”
- ConsistentF(1, 17) = 9.2reported p < .01 · recomputed p = .008
“F(1, 17) = 9.20, p < .01”
- ConsistentF(1, 21) = 5.54reported p = .03 · recomputed p = .028
“F(1, 21) = 5.54, p = .03”
- ConsistentF(1, 28) = 62.9reported p < .001 · recomputed p < .001
“F(1, 28) = 62.9, p < .001”
- ConsistentF(1, 18) = 8.55reported p < .01 · recomputed p = .009
“F(1, 18) = 8.55, p < .01”
- Consistentt(114) = -1.08reported p = .28 · recomputed p = .282
“t(114) = −1.08, p = .28”
- Consistentt(71) = -2.17reported p = .03 · recomputed p = .033
“t(71) = −2.17, p = .03”
- Consistentt(200) = 17.79reported p < .01 · recomputed p < .001
“t(200) = 17.79, p < .01”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Adaptive trust calibration for human-AI collaboration > Results
“TCCs were not presented in the period from the 7th CKP to the 9th CKP”
§ Adaptive trust calibration for human-AI collaboration > Results
“| Visual TCC | 0.78 (0.10) | 0.67 (0.11) | 0.56 (0.12) | 0.67 (0.11) | 0.67 (0.11) | 0.50 (0.12) | 0.56 (0.12) |”
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605
Humans and automation: Use, misuse, disuse, abuse
Parasuraman, R., & Riley, V. (1997) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872097778543886 · Cited by 4,554
A model for types and levels of human interaction with automation
Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/3468.844354 · Cited by 4,195 · cited 1×
“A possible explanation for the result could be made by discussing models for the trust process [1, 27, 41].”
BackgroundTrust in Automation: Integrating Empirical Evidence on Factors That Influence Trust
Hoff, K. A., & Bashir, M. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814547570 · Cited by 2,817
Calculation of signal detection theory measures
Stanislaw, H., & Todorov, N. (1999) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03207704 · Cited by 2,673
A Meta-Analysis of Factors Affecting Trust in Human-Robot Interaction
Hancock, P. A., Billings, D. R., Schaefer, K. E., Chen, J. Y. C., de Visser, E. J., & Parasuraman, R. (2011) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720811417254 · Cited by 2,051 · cited 1×
“[10] examined factors that affect trust by applying meta-analysis methods to existing empirical studies.”
MethodFoundations for an Empirically Determined Scale of Trust in Automated Systems
Jian, J.-Y., Bisantz, A. M., & Drury, C. G. (2000) · International Journal of Cognitive Ergonomics · doi:10.1207/s15327566ijce0401_04 · Cited by 1,799
Evaluating amazon’s mechanical turk as a tool for experimental behavioral research
Crump, M. J. C., McDonnell, J. V., & Gureckis, T. M. (2013) · PLoS ONE · doi:10.1371/journal.pone.0057410 · Cited by 1,617
The mind in the machine: Anthropomorphism increases trust in an autonomous vehicle
Waytz, A., Heafner, J., & Epley, N. (2014) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2014.01.005 · Cited by 1,439
Research works printed but not linked: 1
- Adams BD, Bruyn LE, Houde S. Trust in automated systems literature review. Defence Research and Development Canada; 2003. Report No. CR-2003-096.
Cited by
Sorted by citations
Trust in AI: progress, challenges, and future directions
Afroogh, S., Akbari, A., Malone, E., Kargar, M., & Alambeigi, H. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-024-04044-8 · Cited by 718
Six Human-Centered Artificial Intelligence Grand Challenges
Ozmen Garibay, O., Winslow, B., Andolina, S., Antona, M., Bodenschatz, A., Coursaris, C., Falco, G., Fiore, S. M., Garibay, I., Grieman, K., et al. (2023) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2022.2153320 · Cited by 501
Experimental evidence of effective human–AI collaboration in medical decision-making
Reverberi, C., Rigon, T., Solari, A., Hassan, C., Cherubini, P., GI Genius CADx Study Group, Antonelli, G., Awadie, H., Bernhofer, S., Carballal, S., et al. (2022) · Scientific Reports · doi:10.1038/s41598-022-18751-2 · Cited by 281
Trust in Robots: Challenges and Opportunities
Kok, B. C., & Soh, H. (2020) · Current Robotics Reports · doi:10.1007/s43154-020-00029-y · Cited by 245
Deciding Fast and Slow: The Role of Cognitive Biases in AI-assisted Decision-making
Rastogi, C., Zhang, Y., Wei, D., Varshney, K. R., Dhurandhar, A., & Tomsett, R. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3512930 · Cited by 216
Exploring automation bias in human–AI collaboration: a review and implications for explainable AI
Romeo, G., & Conti, D. (2026) · AI & SOCIETY · doi:10.1007/s00146-025-02422-7 · Cited by 209
Mehrotra, S., Degachi, C., Vereschak, O., Jonker, C. M., & Tielman, M. L. (2024) · ACM Journal on Responsible Computing · doi:10.1145/3696449 · Cited by 219
Herrmann, T., & Pfeiffer, S. (2023) · AI & SOCIETY · doi:10.1007/s00146-022-01391-5 · Cited by 173
“2018), • giving prompts that help counter the human tendency to place too much trust in automated systems (Okamura and Yamada 2020), • and integrating knowledge management with learning (Li and Herd 2017).”
BackgroundWischnewski, M., Krämer, N., & Müller, E. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581197 · Cited by 146
“For example, systems were used to aid users screen bags for dangerous objects [94], detect system malfunctions [89] or potholes [104], improve the production of orange juice pasteurization [73], play games [24], diagnose [100], discover number patterns [26], and classify plants and animals [110,…”
BackgroundMethodInfluentialSimón, C., Revilla, E., & Sáenz, M. J. (2024) · Journal of Business Research · doi:10.1016/j.jbusres.2024.114783 · Cited by 139
Beyond AI-powered context-aware services: the role of human–AI collaboration
Jiang, N., Liu, X., Liu, H., Lim, E. T. K., Tan, C., & Gu, J. (2022) · Industrial Management & Data Systems · doi:10.1108/imds-03-2022-0152 · Cited by 113
A Review on Human–Machine Trust Evaluation: Human-Centric and Machine-Centric Perspectives
Gebru, B., Zeleke, L., Blankson, D., Nabil, M., Nateghi, S., Homaifar, A., & Tunstel, E. (2022) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2022.3144956 · Cited by 101
Fostering Collective Intelligence in Human–AI Collaboration: Laying the Groundwork for COHUMAIN
Gupta, P., Nguyen, T. N., González, C., & Woolley, A. W. (2023) · Topics in Cognitive Science · doi:10.1111/tops.12679 · Cited by 108
Improving Human-AI Collaboration With Descriptions of AI Behavior
Cabrera, Á. A., Perer, A., & Hong, J. I. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579612 · Cited by 87
“Recent methods have explored improving people’s mental models, including tutorials explaining a task [33], tuning a model to better match human expectations [41], or adaptive trust calibration [45].”
MethodHow do we assess the trustworthiness of AI? Introducing the trustworthiness assessment model (TrAM)
Schlicker, N., Baum, K. L., Uhde, A., Sterz, S., Hirsch, M. C., & Langer, M. (2025) · Computers in Human Behavior · doi:10.1016/j.chb.2025.108671 · Cited by 106
Real-Time Trust Prediction in Conditionally Automated Driving Using Physiological Measures
Ayoub, J., Avetisyan, L., Yang, X. J., & Zhou, F. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3295783 · Cited by 91
Who needs explanation and when? Juggling explainable AI and user epistemic uncertainty
Jiang, J., Kahai, S. S., & Yang, M. H. (2022) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102839 · Cited by 92
Caldwell, S., Sweetser, P., O’Donnell, N., Knight, M. J., Aitchison, M., Gedeon, T., Johnson, D. M., Brereton, M. F., Gallagher, M. R., & Conroy, D. (2022) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3514257 · Cited by 92
Riveiro, M., & Thill, S. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103507 · Cited by 93
Human-in-the-loop machine learning: Reconceptualizing the role of the user in interactive approaches
Gómez–Carmona, O., Casado–Mansilla, D., López–de–Ipiña, D., & García-Zubía, J. (2024) · Internet of Things · doi:10.1016/j.iot.2023.101048 · Cited by 84
Understanding User Reliance on AI in Assisted Decision-Making
Cao, S., & Huang, C.-M. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555572 · Cited by 71
The effects of explanations on automation bias
Vered, M., Livni, T., Howe, P. D. L., Miller, T., & Sonenberg, L. (2023) · Artificial Intelligence · doi:10.1016/j.artint.2023.103952 · Cited by 83
Lyu, W., Zhang, S., Chung, T., Sun, Y., & Zhang, Y. (2025) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2025.100383 · Cited by 68
Empirical Evaluations of Framework for Adaptive Trust Calibration in Human-AI Cooperation
Okamura, K., & Yamada, S. (2020) · IEEE Access · doi:10.1109/access.2020.3042556 · Cited by 66
Self-citationMa, S., Zhou, T., Nie, F., & Ma, X. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517482 · Cited by 63
Understanding the Evolvement of Trust Over Time within Human-AI Teams
Duan, W., Zhou, S., Scalia, M. J., Yin, X., Weng, N., Zhang, R., Freeman, G., McNeese, N. J., Gorman, J. C., & Tolston, M. T. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3687060 · Cited by 62
Integrity-based Explanations for Fostering Appropriate Trust in AI Agents
Mehrotra, S., Jorge, C. C., Jonker, C. M., & Tielman, M. L. (2024) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3610578 · Cited by 51
Psychological Traits and Appropriate Reliance: Factors Shaping Trust in AI
Küper, A., & Krämer, N. (2024) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2024.2348216 · Cited by 49
Kahr, P. K., Rooks, G., Willemsen, M. C., & Snijders, C. C. P. (2024) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3686164 · Cited by 49
Cao, S., Liu, A., & Huang, C. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637318 · Cited by 63
Kahr, P. K., Rooks, G., Willemsen, M. C., & Snijders, C. C. P. (2023) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3581641.3584058 · Cited by 51
An Investigation of Drivers’ Dynamic Situational Trust in Conditionally Automated Driving
Ayoub, J., Avetisyan, L., Makki, M., & Zhou, F. (2021) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2021.3131676 · Cited by 42
Does More Advice Help? The Effects of Second Opinions in AI-Assisted Decision Making
Lu, Z., Wang, D., & Yin, M. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3653708 · Cited by 53
Carter, O. B. J., Loft, S. D., & Visser, T. A. W. (2023) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208231218156 · Cited by 42
Impact of Model Interpretability and Outcome Feedback on Trust in AI
Ahn, D., Almaatouq, A., Gulabani, M., & Hosanagar, K. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642780 · Cited by 46
Influence of agent’s self-disclosure on human empathy
Tsumura, T., & Yamada, S. (2023) · PLoS ONE · doi:10.1371/journal.pone.0283955 · Cited by 38
Self-citationSakamoto, T., Harada, Y., & Shimizu, T. (2024) · JMIR Formative Research · doi:10.2196/58666 · Cited by 36
Chang, W.-C., & Hasanzadeh, S. (2024) · Journal of Computing in Civil Engineering · doi:10.1061/jccee5.cpeng-5656 · Cited by 37
Machine Replacement: A Mind-Role Fit Perspective
Yam, K. C., Eng, A., & Gray, K. J. (2024) · Annual Review of Organizational Psychology and Organizational Behavior · doi:10.1146/annurev-orgpsych-030223-044504 · Cited by 34
A Survey on Efficient Vision‐Language Models
Shinde, G., Ravi, A., Dey, E., Sakib, S., Rampure, M., & Roy, N. (2025) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.70036 · Cited by 46
Göndöcs, D., Horváth, S., & Dörfler, V. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103622 · Cited by 39
Küper, A., Lodde, G. C., Livingstone, E., Schadendorf, D., & Krämer, N. (2025) · Journal of Medical Internet Research · doi:10.2196/58660 · Cited by 31
“Thus, it is important to correctly calibrate trust and thereby adjust expectations about the trustworthiness of systems [26,50,51].”
Zhu, Y., Wang, T., Wang, C., Quan, W., & Tang, M. (2023) · Applied Sciences · doi:10.3390/app132412989 · Cited by 35
Islam, M. M., & Miraz, A. I. (2022) · doi:10.63125/k22shk74 · Cited by 35
Eckhardt, S., Kühl, N., Dolata, M., & Schwabe, G. (2025) · ACM Computing Surveys · doi:10.1145/3776528 · Cited by 38
Trusting AI: does uncertainty visualization affect decision-making?
Reyes, J., Batmaz, A. U., & Kersten-Oertel, M. (2025) · Frontiers in Computer Science · doi:10.3389/fcomp.2025.1464348 · Cited by 27
You, Me, and the AI: The Role of Third‐Party Human Teammates for Trust Formation Toward AI Teammates
Erengin, T., Briker, R., & de Jong, S. B. (2025) · Journal of Organizational Behavior · doi:10.1002/job.2857 · Cited by 33
Hasan, R. (2025) · American Journal of Interdisciplinary Studies · doi:10.63125/0fd1yb74 · Cited by 25
Tenhundfeld, N. L., Demir, M., & de Visser, E. J. (2022) · Journal of Cognitive Engineering and Decision Making · doi:10.1177/15553434221096261 · Cited by 26
Influence of Anthropomorphic Agent on Human Empathy Through Games
Tsumura, T., & Yamada, S. (2023) · IEEE Access · doi:10.1109/access.2023.3269301 · Cited by 23
Self-citationJiang, W., Dong-qin, L., & Liu, C. (2025) · PLoS ONE · doi:10.1371/journal.pone.0326558 · Cited by 26
Human-AI Use Patterns for Decision-Making in Disaster Scenarios: A Systematic Review
Domfeh, E. A., & Dancy, C. L. (2025) · IEEE International Symposium on Technology and Society (ISTAS) · doi:10.1109/istas65609.2025.11269624 · Cited by 21
Calibrating Reliance on Automated Advice: Transparency and Trust Calibration Feedback
Tatasciore, M., & Loft, S. (2025) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2025.2487861 · Cited by 20
How Do We Assess the Trustworthiness of AI? Introducing the Trustworthiness Assessment Model (TrAM)
Schlicker, N., Baum, K., Uhde, A., Sterz, S., Hirsch, M. C., & Langer, M. (2022) · doi:10.31234/osf.io/qhwvx · Cited by 79
Choudhury, A., & Elkefi, S. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.966174 · Cited by 22
Herse, S., Vitale, J., & Williams, M. (2023) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2022.2150691 · Cited by 21
Clinical needs and preferences for AI-based explanations in clinical simulation training
Kollerup, N. K., Johansen, S. S., Tolsgaard, M. G., Friis, M. L., Skov, M. B., & van Berkel, N. (2024) · Behaviour and Information Technology · doi:10.1080/0144929x.2024.2334852 · Cited by 22
Belmonte, L. M., García, A. S., Morales, R., de la Vara, J. L., de la Rosa, F. L., & Fernández‐Caballero, A. (2021) · Sensors · doi:10.3390/s21030908 · Cited by 19
A Computational Model of Coupled Human Trust and Self-confidence Dynamics
Williams, K. J., Yuh, M. S., & Jain, N. (2023) · ACM Transactions on Human-Robot Interaction · doi:10.1145/3594715 · Cited by 18
Cai, J., Fu, X., Gu, Z., & Wu, R. (2023) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2023.2212224 · Cited by 18
Dynamic and quantitative trust modeling and real-time estimation in human-machine co-driving process
Hu, C., Huang, S., Zhou, Y. L., Ge, S., Yi, B., Zhang, X., & Wu, X. (2024) · Transportation Research Part F Traffic Psychology and Behaviour · doi:10.1016/j.trf.2024.08.001 · Cited by 20
Ogunsina, K., & DeLaurentis, D. A. (2021) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2021.104600 · Cited by 20
How Humans Comply With a (Potentially) Faulty Robot: Effects of Multidimensional Transparency
Simon, L., Guerin, C., Rauffet, P., Chauvin, C., & Martin, É. (2023) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2023.3273773 · Cited by 17
Calibrating Trust in Human-Drone Cooperative Navigation
Okamura, K., & Yamada, S. (2020) · IEEE RO-MAN · doi:10.1109/ro-man47096.2020.9223509 · Cited by 17
Self-citationIMPACTS Homeostasis Trust Management System: Optimizing Trust in Human-AI Teams
Hou, M., Banbury, S. P., Cain, B., Fang, S. X., Willoughby, H., Foley, L., Tunstel, E. W., & Rudas, I. J. (2024) · ACM Computing Surveys · doi:10.1145/3649446 · Cited by 19
Khac, L. T. D., & Leyer, M. (2025) · Technology in Society · doi:10.1016/j.techsoc.2025.103064 · Cited by 17
Singh, A. P., & Szajnfarber, Z. (2025) · Systems Engineering · doi:10.1002/sys.70024 · Cited by 20
Reliance and Automation for Human-AI Collaborative Data Labeling Conflict Resolution
Brachman, M., Ashktorab, Z., Desmond, M. J., Duesterwald, E., Dugan, C., Joshi, N. N., Pan, Q., & Sharma, A. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555212 · Cited by 18
Panditharathna, R., Liu, Y., Bergamo, F. V. D. M., Appiah, D., Trim, P. R. J., & Lee, Y. (2024) · Big Data and Cognitive Computing · doi:10.3390/bdcc8110165 · Cited by 16
ContractMind: Trust-calibration interaction design for AI contract review tools
Zeng, J., Chen, K., Wang, R., Li, Y., Fan, M., Wu, K., Qi, X., & Wang, L. (2024) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2024.103411 · Cited by 17
Gao, Q., Chen, L., Shi, Y., Luo, Y., Shen, M., & Gao, Z. (2024) · Transportation Research Part F Traffic Psychology and Behaviour · doi:10.1016/j.trf.2024.09.019 · Cited by 17
Tsumura, T., & Yamada, S. (2024) · Frontiers in Computer Science · doi:10.3389/fcomp.2024.1461131 · Cited by 15
Self-citationRemote research methods for Human–AI–Robot Teaming
Lematta, G. J., Corral, C. C., Buchanan, V., Johnson, C. J., Mudigonda, A., Scholcover, F., Wong, M. E., Ezenyilimba, A., Baeriswyl, M., Kim, J., et al. (2021) · Human Factors and Ergonomics in Manufacturing & Service Industries · doi:10.1002/hfm.20929 · Cited by 15
Digitalization in psychology: A bit of challenge and a byte of success
Ostermann, T., Röer, J. P., & Tomasik, M. J. (2021) · Patterns · doi:10.1016/j.patter.2021.100334 · Cited by 16
A Recent Review on Subjective and Objective Assessment of Trust in Human Autonomy Teaming
Jui, J. J., Hettiarachchi, I. T., Bhatti, A., Farghaly, M. R. M., & Creighton, D. (2025) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2025.3589981 · Cited by 13
Dynamic calibration of trust and trustworthiness in AI-enabled systems
Liebherr, M., Enkel, E., Law, E. L., Mousavi, M. R., Sammartino, M., & Sieberg, P. M. (2026) · International Journal on Software Tools for Technology Transfer · doi:10.1007/s10009-026-00840-6 · Cited by 13
Li, M., Guo, F., Li, Z., Ma, H., & Duffy, V. G. (2024) · Ergonomics · doi:10.1080/00140139.2024.2343954 · Cited by 14
AI in project teams: how trust calibration reconfigures team's collaboration and performance
Dawarka, V., Doargajudhur, M. S., & Dutot, V. (2026) · International Journal of Managing Projects in Business · doi:10.1108/ijmpb-07-2025-0285 · Cited by 13
Valentine, D. C., Smit, I., & Kim, E. (2021) · Proceedings of the Design Society · doi:10.1017/pds.2021.114 · Cited by 14
Breckner, K., Neumayr, T., Mara, M., Streit, M., & Augstein, M. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2482742 · Cited by 14
To Solicit or Not to Solicit? Impact of AI Assistance Delivery Mechanisms on Decision-Making
Nair, P., Gressel, G., Anand, M. N., & Achuthan, K. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2536617 · Cited by 15
Trust Miscalibration Is Sometimes Necessary: An Empirical Study and a Computational Model
Collins, M. G., & Juvina, I. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.690089 · Cited by 12
Reinforcement Learning for Human-AI Collaboration: Challenges, Mechanisms, and Methods
Li, W., Liu, H., Huang, K., & Hussain, A. (2025) · Cognitive Computation · doi:10.1007/s12559-025-10500-7 · Cited by 11
User Trust in Assisted Decision-Making Using Miniaturized Near-Infrared Spectroscopy
Jiang, W., Sarsenbayeva, Z., van Berkel, N., Wang, C., Yu, D., Wei, J., Gonçalves, J., & Kostakos, V. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445710 · Cited by 11
Gemalmaz, M. A., & Yin, M. (2022) · Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3514094.3534201 · Cited by 10
Sonographer interaction with artificial intelligence: collaboration or conflict?
Day, T. G., Matthew, J., Budd, S., Hajnal, J. V., Simpson, J. M., Razavi, R., & Kainz, B. (2023) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.26238 · Cited by 10
Trust dampening and trust promoting: A dual-pathway of trust calibration in human-robot interaction
HUANG, X., & Li, Y. (2024) · Advances in Psychological Science · doi:10.3724/sp.j.1042.2024.00527 · Cited by 10
Practising Appropriate Trust in Human-Centred AI Design
Degachi, C., Mehrotra, S., Yurrita, M., Niforatos, E., & Tielman, M. L. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3650825 · Cited by 10
Chang, W.-C., Garcia, N. F. G., & Hasanzadeh, S. (2025) · Journal of Computing in Civil Engineering · doi:10.1061/jccee5.cpeng-6302 · Cited by 10
Ma, S., Sun, M., & Ma, X. (2022) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3571813 · Cited by 12
Dynamic Selection of Reliance Calibration Cues With AI Reliance Model
Fukuchi, Y., & Yamada, S. (2023) · IEEE Access · doi:10.1109/access.2023.3339548 · Cited by 16
“A user’s decision on whether to assign a task to themselves or an AI is also used as an indicator of human trust [16], [17], and vPred-RC follows this approach.”
BackgroundMethodSelf-citationHow Can I Signal You To Trust Me: Investigating AI Trust Signalling in Clinical Self-Assessments
Kollerup, N. K., Wester, J., Skov, M. B., & van Berkel, N. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3661612 · Cited by 13
Exploring the Drivers and Outcomes of Generative AI Usage: The Role of Metacognition
Lee, Y., Ahn, H., & Jo, H. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2526592 · Cited by 9
Trust(worthiness) Issues with Trust in Human–Robot Interaction
Onnasch, L., Roesler, E., Robert, L., & de Visser, E. J. (2025) · ACM Transactions on Human-Robot Interaction · doi:10.1145/3778865 · Cited by 9
Xue, C., Zhang, H., & Cao, H. (2024) · Systems Science & Control Engineering · doi:10.1080/21642583.2024.2343301 · Cited by 9
Park, K., Han, S. H., & Kwahk, J. (2021) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2021.1886482 · Cited by 9
Jiang, W., Li, Y., Hu, X., & Ma, D. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1652130 · Cited by 9
‘ChatGPT can make mistakes’ warnings fail: A randomized controlled trial
Kıyak, Y. S., Çoşkun, Ö., & Budakoğlu, I. İ. (2025) · Medical Education · doi:10.1111/medu.70056 · Cited by 7
Towards detecting the level of trust in the skills of a virtual assistant from the user’s speech
Gauder, L., Pepino, L., Riera, P., Brussino, S., Vidal, J., Gravano, A., & Ferrer, L. (2023) · Computer Speech & Language · doi:10.1016/j.csl.2023.101487 · Cited by 7
Herse, S., Vitale, J., & Williams, M. (2023) · International Journal of Social Robotics · doi:10.1007/s12369-023-01041-w · Cited by 7
Metadata
- Authors
- Okamura, K., & Yamada, S.
- Affiliations
- The Graduate University for Advanced Studies, SOKENDAI (JP)
- National Institute of Informatics (JP)
- Corresponding author
- Okamura, Kazuo
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2020-02-21
- Volume
- 15
- Issue
- 2
- Pages
- e0229132
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC7034851
- OpenAlex
- W2954555759
- Semantic Scholar
- acc10c9ebf9456d71474d79a61b2f0af3d3b29b2
- Cited by
- 239
- References
- 43
- Citations by year
- 2026: 65 · 2025: 56 · 2024: 46 · 2023: 31 · 2022: 18 · 2021: 16 · 2020: 7
- FWCI
- 19.67
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- The Japanese Society for Artificial Intelligence
- Funders
- Japan Society for the Promotion of Science (26118005)
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- trust calibration, human-AI collaboration, cognitive cues, adaptive prompts, system transparency, trust in AI, drone inspection, human-AI decision-making
- MeSH terms
- Adult, Aged, Artificial Intelligence, Calibration, Female, Humans, Male, Man-Machine Systems, Middle Aged, Surveys and Questionnaires, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
