CoChat
Sign Up
Cited by 248Human-Automation Interaction and SafetySocial Robot Interaction and HRIUsability and User Interface Design

Adaptive trust calibration for human-AI collaboration

Open access
Okamura, Kazuo, Yamada, Seiji·2020·PLOS ONE
DOI 10.1371/journal.pone.0229132PMC PMC7034851
Open this paper in CoChat — free account

Abstract

Safety and efficiency of human-AI collaboration often depend on how humans could appropriately calibrate their trust towards the AI agents. Over-trusting the autonomous system sometimes causes serious safety issues. Although many studies focused on the importance of system transparency in keeping proper trust calibration, the research in detecting and mitigating improper trust calibration remains very limited. To fill these research gaps, we propose a method of adaptive trust calibration that consists of a framework for detecting the inappropriate calibration status by monitoring the user's reliance behavior and cognitive cues called "trust calibration cues" to prompt the user to reinitiate trust calibration. We evaluated our framework and four types of trust calibration cues in an online experiment using a drone simulator. A total of 116 participants performed pothole inspection tasks by using the drone's automatic inspection, the reliability of which could fluctuate depending upon the weather conditions. The participants needed to decide whether to rely on automatic inspection or to do the inspection manually. The results showed that adaptively presenting simple cues could significantly promote trust calibration during over-trust.

AI snapshot

Written from the paper’s full text.

TL;DR

The paper proposes adaptive trust calibration: a behavior-based framework detects over-trust/under-trust and presents “trust calibration cues” (TCCs) only when needed. In an online drone-simulator experiment, adaptively presented TCCs significantly increased manual choices and improved sensitivity d′ during over-trust, whereas continuous reliability information alone did not.

Key findings

  • Of 194 online participants, 116 successfully inspected at least 15 checkpoints; they completed 1,740 inspections with a correct rate of 0.74, choosing automatic inspection 1,236 times (0.71) and manual inspection 504 times (0.29).
  • A two-factor mixed ANOVA found a significant main effect of weather period on manual rates [F(1, 111) = 51.69, p < 0.01] and a significant interaction between TCC group and period [F(4, 111) = 4.86, p < 0.01]. The NoTCC group did not significantly change manual rates between good and bad weather, while all TCC groups did: visual [F(1, 17) = 9.20, p < 0.01], audio [F(1, 21) = 5.54, p = 0.03], verbal [F(1, 28) = 62.9, p < 0.001], and anthropomorphic [F(1, 18) = 8.55, p < 0.01].
  • In bad weather, the verbal TCC group had a significantly higher manual rate than both NoTCC [t(111) = 4.77, adj.p < 0.01] and anthropomorphic TCC [t(111) = 2.74, adj.p = 0.04].
  • Sensitivity d′ in bad weather was significantly higher for the verbal group (0.92, SE 0.14) than for NoTCC [t(111) = 2.97, adj.p = 0.04, Cohen’s d = 0.75]; accuracy declined in bad weather but did not differ significantly among groups, though the verbal group had the highest mean accuracy (0.72, SE 0.04).

Methods

  • Online experiment using a 3D drone simulator built with CesiumJS and Bing Map API. Participants performed a pothole inspection task over 30 checkpoints (10 with potholes, 20 without), choosing between automatic inspection and manual inspection; they were randomly assigned to NoTCC, visual, audio, verbal, or anthropomorphic TCC groups.
  • Automatic inspection reliability Pauto was manipulated: good weather Pauto = 90% and d′ = 1.8; bad weather Pauto = 50% and d′ = 0.1. The framework detected over-trust by observing reliance behavior and comparing Pman and Pauto, then presented a TCC when over-trust was detected.
  • Main analyses were two-factor mixed ANOVAs with TCC group as between-subjects and checkpoint period as within-subjects, plus Holm–Bonferroni-adjusted post hoc comparisons and t-tests for Pman.

Limitations

  • Stated: the framework focuses only on performance-related factors, ignoring other automation benefits such as faster completion, lighter workload, or fewer risks; it evaluates only an over-trust scenario; it does not guarantee convergence of behavior; and it uses only a pothole inspection/reconnaissance task and four TCC types.
  • Stated: correct answers were not shown during the task, and automatic-inspection results were shown only when participants chose automatic inspection, not manual inspection; future work should test combinations of feedback information.
  • Evident: attrition was high—78 of 194 participants failed to complete the task within the time limit because they moved the drone away from the checkpoints—so the analyzed sample of 116 may not represent all recruited participants.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1371/journal.pone.0229132

Cites

43 of 44 references verified1 citation errors · 0 not checkable

Statistics

Claims the paper makes

0 of 22 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Participants in the group without TCC did not change their behavior when they were in over-trust status, even if the system reliability information was continuously presented.

    “The participants of the group without TCC did not change their behavior when they were in over-trust status, even if the system reliability information was continuously presented.”
    Quote in the paperSubstantial claim
  • Findingown data

    Adaptively presented TCCs significantly affected the choice behavior of the participants.

    “We found that adaptively presented TCCs significantly affected the choice behavior of the participants.”
    Quote in the paperSubstantial claim
  • Findingown data

    TCC rates in the anthro TCC group did not follow the decreasing trend.

    “while TCC rates in the anthro. TCC group did not follow the decreasing trend.”
    Quote in the paperNot a substantial claim
  • Findingown data

    Pman in the bad weather period was significantly larger than Pauto = 0.50 (Mean = 0.86, t(200) = 17.79, p < 0.01, Cohen's d = 1.25).

    “and that Pman in the bad weather period was significantly larger than Pauto = 0.50 [Mean = 0.86, t(200) = 17.79, p < 0.01, Cohen’s d = 1.25].”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: Manual accuracy can be used to estimate Pman.
  • Findingown data

    Pman in the good weather period was significantly smaller than Pauto = 0.90 (Mean = 0.81, t(71) = -2.17, p = 0.03, Cohen's d = 0.26).

    “One-sample t-tests showed that Pman in the good weather period was significantly smaller than Pauto = 0.90 [Mean = 0.81, t(71) = −2.17, p = 0.03, Cohen’s d = 0.26]”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: Manual accuracy can be used to estimate Pman.
  • Classified as probably not a substantial claim. 16
  • May combine two claims. 16
  • The quote only partly states this claim. 1

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Manual inspection was done 504 times (choice rate = 0.29).

    “The participants did the manual inspection 504 times (the choice rate = 0.29).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The verbal group had the highest mean accuracy (0.72, SE 0.04), and other TCC groups had better accuracy values (0.6, SE 0.04) than the NoTCC group.

    “however, the verbal group showed the highest mean value (0.72 (SE 0.04)), and other groups with TCCs also had better accuracy values (0.6 (SE 0.04)) than the NoTCC group.”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    The visual and audio TCC groups also showed a similar decreasing trend in TCC rates.

    “The visual and audio TCC groups also showed a similar trend”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The participants inspected a total of 1,740 CKPs from the 1st to the 15th CKP, and 1,282 inspections were correct, making the correct answer rate 0.74.

    “They inspected the total of 1,740 CKPs from the 1st CKP to the 15th CKP, and the results of 1,282 inspections were correct, making the correct answer rate 0.74.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In the bad weather period, the verbal group had the highest sensitivity d' (0.92, SE 0.14) and it was significantly higher than the NoTCC group (t(111) = 2.97, adj.p = 0.04, Cohen's d = 0.75).

    “In the bad weather period, post-hoc comparisons indicated that the sensitivity d′ of the verbal group, which was the highest (0.92 (SE 0.14) among all the groups, was significantly higher than the sensitivity d′ of the NoTCC group [t(111) = 2.97, adj.p = 0.04, Cohen’s d = 0.75].”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Automatic inspection was selected 1,236 times (choice rate = 0.71).

    “Automatic inspection was selected 1,236 times (the choice rate = 0.71).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    All groups with TCCs showed better accuracy than the NoTCC group, though there was no statistical difference in accuracy among the groups.

    “Though there was no statistical difference in accuracy among the groups, all the groups with TCCs also showed better accuracy than the NoTCC group.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    A new trigger is necessary to solve the dilemma that could perpetuate an inappropriate status of trust calibration.

    “To solve this dilemma that could perpetuate an inappropriate status of trust calibration, a new trigger would be necessary in instances of over-trust or under-trust.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Calibration can only occur in response to new evidence that may change the users' prevailing recognition, while no new evidence can be learned without changing the current behavior first.

    “Calibration can only occur in response to new evidence that may change the users’ prevailing recognition, while no new evidence can be learned without changing the current behavior first [5].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [5] Adams BD, Bruyn LE, Houde S. Trust in automated systems literature review. Defence Research and Development Canada; 2003. Report No. CR-2003-096.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Online data collection using a web-browser seems mostly in line with laboratory results, so long as the experiment methods are solid.

    “Regarding online experiments in general, Crump et al. [32] showed that the data collected online using a web-browser seemed mostly in line with laboratory results, so long as the experiment methods were solid.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [32] MJC Crump, 2013, Evaluating amazon’s mechanical turk as a tool for experimental behavioral research, PLoS ONE source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    The proposed approach mitigates over-trust or under-trust, in contrast with the traditional approach of trying to maintain appropriate trust calibration with continuous system transparency.

    “This approach is to mitigate over-trust or under-trust, in contrast with the traditional approach of trying to maintain appropriate trust calibration with continuous system transparency.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Behavioral measures are practical and can easily be used as a basis for modeling and prediction.

    “Behaviors might not be observable if there is no interaction; nevertheless, behavioral measures are practical and can easily be used as a basis for modeling and prediction [25].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [25] K Drnec, 2016, From trust in automation to decision neuroscience: applying cognitive neuroscience methods to understand and improve interaction decisions involved in human automation interaction, Frontires In Human Neuroscience

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    The proposed framework can detect trust calibration status by observing the user's reliance behavior if Pman and Pauto can be estimated.

    “Instead of directly measuring Ptrust or , the first inequalities of (1) and (2) can be estimated by observing the user’s reliance behavior; thus, the trust calibration status can be detected, if the second inequalities of Pman and Pauto can be estimated.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    194 participants joined the experiment online.

    “194 participants joined the experiment online.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    Among successful participants, ages ranged from 20 to 69 years old (M = 43.25, SD = 14.01), 66 were male and 50 were female.

    “As for the successful participants, their ages ranged from 20 to 69 years old (M = 43.25, SD = 14.01), 66 participants were male and 50 were females.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The successful participants were distributed across groups as follows: 28 in NoTCC, 18 in visual, 22 in audio, 29 in verbal, and 19 in anthro.

    “28 were in the NoTCC group, 18 in the visual group, 22 in the audio group, 29 in the verbal group, and 19 in the anthro. group.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    78 participants failed to complete the tasks within the time limit.

    “Seventy eight participants unintentionally moved the drone far from the area where the CKPs were located, and they failed to complete the tasks within the time limit.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    116 participants successfully inspected 15 CKPs or more within the time limit.

    “One hundred sixteen participants successfully inspected 15 CKPs or more within the time limit.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Participants were randomly assigned to one of five groups with corresponding TCCs.

    “Participants were randomly assigned to one of five groups with the corresponding TCCs: NoTCC group (without TCCs), visual group (with the visual sign TCC), audio group (with the audio TCC), verbal group (with the verbal TCC), and anthro. group (with the anthropomorphic TCC).”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    TCCs were given right after the behavior only if participants were judged to over-trust, so it would be easier for participants to understand the implication of the cues and to move forward in the trust calibration process.

    “TCCs were given right after the behavior only if the participants were judged to over-trust, so that it would be easier for the participants to understand the implication of the cues and to move forward in the trust calibration process.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Automation could be beneficial beyond better performance, such as for faster task completion, lighter workloads, and fewer risks.

    “However, automation could be beneficial beyond a better performance, such as for faster task completion, lighter workloads, fewer risks, etc. [3]”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [3] R Parasuraman, 1997, Humans and automation: Use, misuse, disuse, abuse, Human Factors source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    Previous studies emphasized the importance of system transparency for proper trust calibration.

    “Previous studies emphasized the importance of the system transparency for proper trust calibration.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    Adaptively presenting TCCs promoted appropriate trust calibration leading to better performance in the bad weather period.

    “These results indicated that adaptively presenting TCCs promoted appropriate trust calibration leading to the better performance in the bad weather period.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    Participants recognized the visually impressive TCCs, but just being salient on the screen was not enough for some participants to change their choice behavior.

    “It was obvious that the participants recognized these visually impressive TCCs; however, the results suggested that just being salient on the screen was not enough for some of the participants to change their choice behavior [31, 40].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [31] KR Laughery, 2014, A three-stage model summarizes product warning and environmental sign research, Safety Science source

      Not checked in the free run.

    • cites [40] VC Conzola, 2001, A Communication—Human Information Processing (C–HIP) approach to warning effectiveness in the workplace, Journal of Risk Research source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationnone given

    The proposed method deals with users' behaviors to choose Auto or Manual and does not guarantee the convergence of the behaviors.

    “The proposed method deals with users’ behaviors to choose Auto or Manual and does not guarantee the convergence of the behaviors, which is the result of trust calibration by users.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Trust as observable and rational choice behavior can be a product of believing that benefits will outweigh costs.

    “Therefore, trust as the observable and rational choice behavior can be a product of believing that these benefits will outweigh the costs [5].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [5] Adams BD, Bruyn LE, Houde S. Trust in automated systems literature review. Defence Research and Development Canada; 2003. Report No. CR-2003-096.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    The feedback information given for each inspection was very important for the participants to make decisions.

    “The feedback information given for each inspection was very important for the participants to make decisions.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisereasoning

    The pothole inspection task is a remote sensing task, and it would be difficult for the system to know the correct answer at the time of each inspection in practical situations.

    “The pothole inspection task in the experiment is a remote sensing task, and it would be quite difficult for the system to know the correct answer (ground truth) at the time of each inspection in practical situations because the only information available is the image data and the results of automatic recognition.”
    Quote in the paperSubstantial claim
28 of 28

  • The authors state this as an assumption.

    Pauto fluctuates more widely than Pman under changing weather conditions.

    “Therefore, we assumed that Pauto would fluctuate more widely than Pman under changing weather conditions.”

  • The paper relies on this without support.

    Trust is a latent construct and cannot be directly measured; measuring trust experimentally is difficult in general.

    “Trust is a latent construct and cannot be directly measured; measuring trust experimentally is difficult in general.”

  • The paper relies on this without support.

    Users often fail to calibrate their trust in the agent and end up in over-trust or under-trust.

    “While an agent’s reliability changes for various reasons in an environment, the users often fail to calibrate their trust in the agent and end up in the status called over-trust or under-trust.”

  • The paper relies on this without support.

    Self-report measures are too intrusive to be viable in applied settings.

    “Most previous literature used self-report measures such as subjective rating with questionnaires [18] or rating scales [19, 20]; however, self-report measures are too intrusive to be viable in applied settings.”

  • The paper relies on this without support.

    Physiological and neural measures usually require special hardware.

    “Although these measures can be used for dynamic tracking of trust, they usually require special hardware.”

  • The paper relies on this without support.

    TCC rates are a valid indicator of over-trust or trust calibration status.

    “TCC rates in the verbal TCC group were higher in the early part of the period and gradually decreased. This indicates that over-trust decreased during this period.”

  • The authors state this as an assumption.

    Manual accuracy can be used to estimate Pman.

    “The results of the manual accuracy were consistent with our assumption to estimate Pman − Pauto in this experiment.”
7 of 7

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • There is still limited evidence-based knowledge of the context, design, and psychological mechanisms that would craft an optimal human-AI team.

    “Notwithstanding a clear agenda on the issues that we should explore further, we still have limited evidence-based knowledge of the context, design, and psychological mechanisms that would craft an optimal human–ai team15–17.”

    From Experimental evidence of effective human–AI collaboration in medical decision-making (2022) · cites this paper as [16]

    The Discussion’s “Limitations and future research” section calls for further experiments on different tasks, effective cue designs, and dispositional and situational trust. That supports a narrower claim that aspects of human–AI trust calibration need more evidence; it does not establish a general lack of evidence-based knowledge about all the context, design, and psychological mechanisms needed for an optimal human–AI team. No page number is available in our copy. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • Giving additional trigger signals was effective in resolving the problem of not paying attention to continuously displayed information.

    “They also found that giving additional trigger signals was effective in resolving this problem [16].”

    From Dynamic Selection of Reliance Calibration Cues With AI Reliance Model (2023) · cites this paper as [16]

    The Discussion (‘Adaptive method vs. continuous method’; page unavailable) reports that participants continued to over-trust automation despite a continuously displayed reliability indicator, while participants given trust calibration cues changed their behavior. This supports the effectiveness of additional cues, but the paper does not establish that participants were not paying attention to the display; it suggests they may not have understood the indicator’s implications. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • Avoiding misplaced trust in AI results is not essentially a technical challenge.

    “Avoiding misplaced trust in AI results is not essentially a technical challenge (Okamura and Yamada 2020).”

    From Keeping the organization in the loop: a socio-technical extension of human-centered artificial intelligence (2023) · cites this paper as Okamura and Yamada 2020

    The Introduction describes misplaced trust as a failure to calibrate users’ trust to an AI agent’s reliability, and the Conclusion reports that cognitive cues helped participants recover from over-trust. Neither passage says that avoiding misplaced trust is not essentially a technical challenge; the paper also proposes a detection framework and user-interface intervention. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • Participants did not change their over-reliance in spite of a reliability indicator being continuously displayed to them.

    “In their experiment, participants did not change their over-reliance in spite of a reliability indicator being continuously displayed to them.”

    From Dynamic Selection of Reliance Calibration Cues With AI Reliance Model (2023) · cites this paper as [16]

  • Various interventions have been explored to foster trust calibration, such as displaying confidence scores and designing adaptive monitoring systems.

    “Various interventions have been explored to foster trust calibration, such as displaying confidence scores that indicate the likelihood of a model's outputs being correct to end users [124], and designing adaptive monitoring systems that visually prompt users to calibrate their trust when over-trust or under-trust is detected during human-AI collaboration [87]”

    From Understanding the Practices, Perceptions, and (Dis)Trust of Generative AI among Instructors: A Mixed-methods Study in the U.S. Higher Education (2025) · cites this paper as [87]

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • ConsistentF(1, 111) = 51.69reported p < .01 · recomputed p < .001

    “F(1, 111) = 51.69, p < .01”

  • ConsistentF(4, 111) = 4.86reported p < .01 · recomputed p = .001

    “F(4, 111) = 4.86, p < .01”

  • ConsistentF(1, 17) = 9.2reported p < .01 · recomputed p = .008

    “F(1, 17) = 9.20, p < .01”

  • ConsistentF(1, 21) = 5.54reported p = .03 · recomputed p = .028

    “F(1, 21) = 5.54, p = .03”

  • ConsistentF(1, 28) = 62.9reported p < .001 · recomputed p < .001

    “F(1, 28) = 62.9, p < .001”

  • ConsistentF(1, 18) = 8.55reported p < .01 · recomputed p = .009

    “F(1, 18) = 8.55, p < .01”

  • Consistentt(114) = -1.08reported p = .28 · recomputed p = .282

    “t(114) = −1.08, p = .28”

  • Consistentt(71) = -2.17reported p = .03 · recomputed p = .033

    “t(71) = −2.17, p = .03”

  • Consistentt(200) = 17.79reported p < .01 · recomputed p < .001

    “t(200) = 17.79, p < .01”

9 of 9

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • CKP9 has nonzero TCC rates despite a stated no-presentation period

    Minor

    The text says no TCCs were presented from CKP7 through CKP9, but Table 1 reports nonzero TCC presentation rates at CKP9 for all four TCC groups.

1 of 1

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. Adams BD, Bruyn LE, Houde S. Trust in automated systems literature review. Defence Research and Development Canada; 2003. Report No. CR-2003-096.

Cited by

Sorted by citations

Related

Authors
Okamura, K., & Yamada, S.
Affiliations
  • The Graduate University for Advanced Studies, SOKENDAI (JP)
  • National Institute of Informatics (JP)
Corresponding author
Okamura, Kazuo
ORCID
Venue
PLOS ONE
Publisher
Public Library of Science (PLoS)
Published
2020-02-21
Volume
15
Issue
2
Pages
e0229132
ISSN
1932-6203 (ISSN-L)
DOI
10.1371/journal.pone.0229132
PMC
PMC7034851
OpenAlex
W2954555759
Semantic Scholar
acc10c9ebf9456d71474d79a61b2f0af3d3b29b2
Cited by
239
References
43
Citations by year
2026: 65 · 2025: 56 · 2024: 46 · 2023: 31 · 2022: 18 · 2021: 16 · 2020: 7
FWCI
19.67
Type
journal article
Language
en
Open access
Licence
Locations
  • PLoS ONE (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • The Japanese Society for Artificial Intelligence
Funders
Japan Society for the Promotion of Science (26118005)
Topics
  • Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
  • Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
Keywords
trust calibration, human-AI collaboration, cognitive cues, adaptive prompts, system transparency, trust in AI, drone inspection, human-AI decision-making
MeSH terms
Adult, Aged, Artificial Intelligence, Calibration, Female, Humans, Male, Man-Machine Systems, Middle Aged, Surveys and Questionnaires, Young Adult

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review