CoChat
Sign Up
Cited by 71Human-Automation Interaction and SafetyGaze Tracking and Assistive TechnologySocial Robot Interaction and HRI

Understanding User Reliance on AI in Assisted Decision-Making

Open access
Cao, Shiye, Huang, Chien-Ming·2022·Proceedings of the ACM on Human-Computer Interaction
DOI 10.1145/3555572
Open this paper in CoChat — free accountOpen PDF

Abstract

Proper calibration of human reliance on AI is fundamental to achieving complementary performance in AI-assisted human decision-making. Most previous works focused on assessing user reliance, and more broadly trust, retrospectively, through user perceptions and task-based measures. In this work, we explore the relationship between eye gaze and reliance under varying task difficulties and AI performance levels in a spatial reasoning task. Our results show a strong positive correlation between percent gaze duration on the AI suggestion and user AI task agreement, as well as user perceived reliance. Moreover, user agency is preserved particularly when the task is easy and when AI performance is low or inconsistent. Our results also reveal nuanced differences between reliance and trust. We discuss the potential of using eye gaze to gauge human reliance on AI in real-time, enabling adaptive AI assistance for optimal human-AI team performance.

AI snapshot

Written from the paper’s full text.

TL;DR

This study examines whether eye gaze can serve as a real-time behavioral measure of user reliance on AI during an AI-assisted spatial reasoning task. It finds that percent gaze duration on the AI suggestion is positively correlated with user-AI agreement and perceived reliance, but not with perceived trust, while users often preserve agency when tasks are easy or AI performance is low/inconsistent.

Key findings

  • Percent gaze duration on the AI suggestion was positively correlated with user-AI agreement in both task views: bottom-view \(r(172)=.19, p=.011\); top-view \(r(171)=.19, p=.012\).
  • Percent gaze duration on the AI suggestion was positively correlated with perceived reliance: bottom-view \(r(172)=.37, p<.001\); top-view \(r(171)=.30, p<.001\). It was not significantly correlated with perceived trust: bottom-view \(r(172)=.04, p=.596\); top-view \(r(171)=.07, p=.334\).
  • AI accuracy level significantly affected user-AI agreement: high performance \(M=0.98, SD=0.08\) vs. medium \(M=0.90, SD=0.19\), \(p=.014\); medium vs. low \(M=0.55, SD=0.18\), \(p<.001\); high vs. low, \(p<.001\). Perceived reliance was higher with high performance AI \(M=4.68, SD=1.99\) than low performance AI \(M=2.53, SD=1.57\), \(p<.001\).
  • Task difficulty affected gaze duration: bottom-view \(M=0.17, SD=0.11\) vs. top-view \(M=0.13, SD=0.12\), \(F(1,32.42)=16.18, p<.001\); phase 2 \(M=0.12, SD=0.12\) vs. phase 3 \(M=0.17, SD=0.11\), \(F(1,634.5)=22.95, p<.001\). Final response accuracy in top-view tasks was 0.98, 0.98, and 0.94 for high, medium, and low AI performance, respectively; in bottom-view tasks it was 0.98, 0.84, and 0.75.

Methods

  • A 2×3 mixed factorial in-person study: task difficulty (top-view vs. bottom-view) was within-subjects, and AI accuracy (high, medium, low) was between-subjects. Participants completed 10 spatial reasoning trials with an AI suggestion appearing after four user clicks; eye gaze was recorded with a Gazepoint GP3 remote eye tracker (60 Hz).
  • 42 participants were recruited; data from 7 were excluded (2 left the room, 5 moved too much/eyes out of range for more than five trials), leaving 35 participants and 347 analyzed trials. Participants were assigned to high (\(n=14\)), medium (\(n=10\)), or low (\(n=11\)) AI performance.
  • Analyses used two-way and three-way repeated measures ANOVA with participants as a random effect, Tukey’s HSD for pairwise comparisons, and Pearson or Spearman correlations for gaze-reliance relationships.

Limitations

  • Stated by the paper: the spatial reasoning task is non-critical and low-stakes; results come from a one-session experiment, so it is unclear whether trust-reliance differences persist across multiple sessions; and the study focused only on gaze duration, not other gaze behaviors such as temporal gaze-shift patterns.
  • Evident from the text: participants were recruited by convenience sampling from the local community and student group mailing lists, with a mean age of 23.69 years (\(SD=6.13\)), which may limit generalizability. Also, 7 of 42 participants were excluded due to leaving the room or eye-tracking failure, which may introduce selection or data-quality concerns.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3555572

Cites

62 of 62 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

0 of 26 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    There was no significant learning effect over the ten task trials.

    “Therefore, we did not observe a significant learning effect over the ten task trials.”
    Quote in the paperNot a substantial claim
  • Findingown data

    For the top-view task, there was a positive correlation between percent gaze duration on AI and perceived reliance (r(171)=.30, p<.001).

    “Among the top-view tasks, there was also a positive correlation between the two variables, 𝑟 (171) = .30,𝑝 < .001.”
    Quote in the paperSubstantial claim
  • Findingown data

    Percent gaze duration on the AI suggestion was significantly higher in phase 3 (M=0.17, SD=0.11) than in phase 2 (M=0.12, SD=0.12) (F(1,634.5)=22.95, p<.001).

    “However, there was a significant difference in the participants' percent gaze duration on the AI suggestion in phase 2 of the task (𝑀 = 0.12, 𝑆𝐷 = 0.12) and phase 3 of the task (𝑀 = 0.17, 𝑆𝐷 = 0.11), 𝐹 (1, 634.5) = 22.95, 𝑝 < .001.”
    Quote in the paperSubstantial claim
  • Findingown data

    The difference between the reported perceived reliance level of participants with the medium performance AI and that of participants with the low performance AI was not significant (p=.079).

    “The difference between the reported perceived reliance level of participants with the medium performance AI and that of participants with the low performance AI was not significant, 𝑝 = .079.”
    Quote in the paperNot a substantial claim
  • Findingown data

    There was no significant difference in percent gaze duration on the AI suggestion under varying AI accuracy levels (F(2,32.14)=0.74, p=.487).

    “The test did not find a significant difference in the participants' percent gaze duration on the AI suggestion under varying AI accuracy levels, 𝐹 (2, 32.14) = 0.74, 𝑝 = .487.”
    Quote in the paperNot a substantial claim
  • The quote only partly states this claim. 2
  • May combine two claims. 27
  • Classified as probably not a substantial claim. 33

  • The quote only partly states this claim.

    Findingown data

    Participants with the high performance AI reported significantly higher perceived reliance levels (M=4.68, SD=1.99) than participants with medium performance AI (M=3.58, SD=1.92), p=.043.

    “Participants with the high performance AI reported significantly higher perceived reliance levels than participants with medium performance AI (𝑀 = 3.58, 𝑆𝐷 = 1.92), 𝑝 = .043.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    There was a significant difference in percent gaze duration on the AI suggestion between the bottom-view task in phase 2 (M=0.17, SD=0.10) and the top-view task in phase 2 (M=0.09, SD=0.12), p<.001, and between the top-view task in phase 2 and the top-view task in phase 3, p<.001.

    “However, we found a significant difference between percent gaze duration on the AI suggestion for the bottom-view task in phase 2 and the top-view task in phase 2 (𝑀 = 0.09, 𝑆𝐷 = 0.12), 𝑝 < .001,and between the top-view task in phase 2 and the top-view task in phase 3, 𝑝 < .001.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    There was no significant difference in task accuracy between participants with the high performance AI and the medium performance AI (p=.165), nor between the medium performance AI and the low performance AI (p=.270).

    “No significant difference in performance was observed in participants with the high performance AI and participants with the medium performance AI (𝑀 = 0.90, 𝑆𝐷 = 0.20), 𝑝 = .165,nor in participants with the medium performance AI and participants with the low performance AI, 𝑝 = .270.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    User agency is preserved particularly when the task is easy and when AI performance is low or inconsistent.

    “Moreover, user agency is preserved particularly when the task is easy and when AI performance is low or inconsistent.”
    • Also stated in: § 1 INTRODUCTION · plain
      “3) preserved user agency when the task is easy and AI performance is low or inconsistent.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    There was no interaction effect on percent gaze duration on the AI suggestion from AI accuracy level and task difficulty (F(2,32.47)=1.25, p=.301), nor from AI accuracy level, task phase, and task difficulty (F(2,625.4)=1.06, p=.347).

    “No interaction effect on the participants' percent gaze duration on the AI suggestion was found from AI accuracy level and task difficulty (𝐹 (2, 32.47) = 1.25, 𝑝 = .301),nor the AI accuracy level, task phase, and task difficulty (𝐹 (2, 625.4) = 1.06, 𝑝 = .347).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Duration on AI is positively correlated with perceived reliance and user agreement with the AI response.

    “In our results, we found that duration on AI is positively correlated with perceived reliance (Fig. 5, b) and user agreement with the AI response.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    There was no significant difference in confidence between participants with the high performance AI and the medium performance AI (p=.152), nor between the medium performance AI and the low performance AI (p=.138).

    “However, there was no significant difference in the reported confidence level in participants with the high performance AI and participants with the medium performance AI (𝑀 = 5.67, 𝑆𝐷 = 1.33), 𝑝 = .152.Furthermore, the difference in the reported confidence level in the participants with the medium performance AI and the participants with the low performance AI were not significant, 𝑝 = .138.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Participants with the high performance AI (M=6.19, SD=1.26) reported higher trust than participants with the medium performance AI (M=4.29, SD=1.71, p<.001); participants with the medium performance AI reported higher trust than participants with the low performance AI (M=2.30, SD=1.19, p<.001); and participants with the high performance AI reported higher trust than participants with the low

    “Pairwise comparisons using Tukey's HSD test (see Table 5 in Appendix) showed that participants with the high performance AI (𝑀 = 6.19, 𝑆𝐷 = 1.26) reported that they trusted the AI more than participants with the medium performance AI (𝑀 = 4.29, 𝑆𝐷 = 1.71), 𝑝 < .001;participants with the medium performance AI reported that they trusted the AI more than participants with the low performance AI (𝑀 =”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    In this work, gaze duration on AI is strongly correlated with perceived reliance and user-AI task agreement.

    “In this work, we show how gaze duration on AI is strongly correlated with perceived reliance and user-AI task agreement.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Participants tend to maintain agency, particularly when the task is easy and when AI performance is low or inconsistent.

    “We also find that participants tend to maintain agency, particularly when the task is easy and when AI performance is low or inconsistent.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    Physiological measures would allow for real-time measurements of trust and reliance, but they require special equipment, thereby making them difficult to generalize for everyday use.

    “While physiological measures would allow for real-time measurements of trust and reliance, they require special equipment, thereby making them difficult to generalize for everyday use.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Prior research has shown how eye gaze may provide insights into human decision-making and collaboration; for instance, gaze duration is indicative of people's preferences in selective decision-making.

    “Prior research has shown how eye gaze may provide insights into human decision-making and collaboration ( [28]); for instance, gaze duration is indicative of people's preferences in selective decision-making [72].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [28]

      Not checked in the free run.

    • cites [72]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Prior research was able to manipulate user agency in decision-making successfully through manipulating the level of machine assistance.

    “Indeed, prior research was able to manipulate user agency in decision-making successfully through manipulating the level of machine assistance [35].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [35] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Prior studies have found that in experimental tasks such as text classification, deception detection, and treatment selection, even though a human-AI team often outperforms the individual human's performance, its team performance is inferior to that of the AI alone.

    “However, prior studies have found that in experimental tasks such as text classification [4], deception detection [33,34], and treatment selection [31], even though a human-AI team often outperforms the individual human's performance, its team performance is inferior to that of the AI alone.”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [4] source

      Not checked in the free run.

    • cites [33] source

      Not checked in the free run.

    • cites [34] source

      Not checked in the free run.

    • cites [31]

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    Self-reported measures and task measures are only used to analyze user reliance and trust retrospectively, and therefore are inadequate for timely, adaptive calibration of reliance and trust in human-AI collaboration.

    “These measures are only used to analyze user reliance and trust retrospectively, and therefore are inadequate for timely, adaptive calibration of reliance and trust in human-AI collaboration.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Most prior works assess reliance and trust in human-AI interaction through self-reported measures (perceived reliance and trust) and task measures (user AI agreement fraction).

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Data from two participants were excluded because they left the experiment room in the middle of the study.

    “Data from two participants were excluded because they left the experiment room in the middle of the study.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Each participant alternated between doing a top-view task and a bottom-view task to decrease the learning effect throughout the experiment.

    “Each participant alternated between doing a top-view task and a bottom-view task in hopes to decrease the learning effect throughout the experiment.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The study was approved by the institutional review board (IRB).

    “The study was approved by our institutional review board (IRB).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S1 (asserted).

    Premisecitation

    Greater user reliance on decision aids occurs when a task is more difficult.

    “This hypothesis is informed by previous work that showed greater user reliance on decision aids when a task is more difficult [7,57].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [7]

      Not checked in the free run.

    • cites [57] source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    Participants were randomly assigned to a particular AI accuracy level by the system; neither the participant nor the experimenter were made aware of the accuracy level of the AI.

    “In the beginning of an experiment, a participant was randomly assigned to a particular AI accuracy level by the system; neither the participant nor the experimenter were made aware of the accuracy level of the AI.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    The gaze bias effect suggests that preference for the AI suggestion and adoption of the AI suggestion means longer gaze duration on the AI suggestion.

    “Furthermore, the gaze bias effect suggests that preference for the AI suggestion and adoption of the AI suggestion means longer gaze duration on the AI suggestion [58].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [58] source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning

    The AI suggestion was not shown to the user until the user entered a response for four different squares on the user response grid, regardless of correctness, to encourage participants to form an opinion before seeing the AI suggestion.

    “An AI suggestion was not shown to the user until the user entered a response for four different squares on the user response grid, regardless of the correctness (color and location) of the blocks they entered.”
    • Also stated in: § 3.3.3 Timing of an AI Suggestion. · plain
      “This design choice was to encourage the participants to form an opinion about the task answer before they saw the AI suggestion so they did not solely rely on the AI.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The eye tracker calibration procedure was repeated until all points were successfully calibrated and captured by the eye tracker for both of the participants' eyes.

    “The calibration procedure was repeated until all points were successfully calibrated and captured by the eye tracker for both of the participants' eyes.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S1 (asserted).

    Premisecitation

    The observed accuracy of a model can affect people's reliance on the model.

    “This hypothesis is based on previous findings suggesting that the observed accuracy of a model can affect people's reliance on the model [85].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [85] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims. The quote only partly states this claim.

    Study design and conductreasoning

    The final data analysis included a total of 347 trials from 35 participants.

    “As a result, our data analysis included a total of 347 trials.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationown data

    Our findings point toward a possibility of enabling real-time adaptive human-AI collaboration through gaze awareness.

    “Our findings point toward a possibility of enabling real-time adaptive human-AI collaboration through gaze awareness.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    When the user has too much trust for the AI, the human-AI team performance may be bounded by the AI capability, which can be problematic especially in critical task domains since AI systems are not error-free.

    “In the latter case, the human-AI team performance may be bounded by the AI capability, which can be problematic especially in critical task domains since AI systems are not error-free.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: AI systems are not error-free.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Data from five participants were excluded because they moved too much and their eyes were out of range for more than five task trials.

    “Data from another five participants were excluded because they moved too much during the experiment and their eyes were completely out of range and unable to be tracked by the gaze tracker for more than five task trials.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    When the user has little trust for the AI assistance, they do not properly rely on it; when the user has too much trust for the AI, they over-rely on it.

    “When the user has little trust for the AI assistance, they do not properly rely on it.When the user has too much trust for the AI, they over-rely on it [9,32,50].”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [9]

      Not checked in the free run.

    • cites [32] source

      Not checked in the free run.

    • cites [50]

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    The results presented in this paper are based on a one-session experimental study.

    “Second, the results presented in this paper are based on a one-session experimental study.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    The empirical findings have design implications for enabling real-time adaptive human-AI collaboration.

    “Our empirical findings have design implications for enabling real-time adaptive human-AI collaboration.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    Reliance is how much the user thinks the AI was involved in the formation of their final response, whereas trust has more emotional value and is about how much the user is willing to use the AI in the formation of their final response.

    “One possible explanation is that reliance is how much the user think the AI was involved in the formation of their final response, whereas trust has more emotional value and is about how much the user is willingly to use the AI in the formation of their final response.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationown data

    The relationship between eye gaze and user reliance on AI opens the possibility of real-time adaptive human-AI collaboration through gaze monitoring.

    “This relationship between eye gaze and user reliance on AI opens the possibility of real-time adaptive human-AI collaboration through gaze monitoring.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Prior work [57] also showed that the higher the level of expertise the participant has in the task, the smaller the effect of task difficulty on user reliance.

    “The prior work also showed the higher the level of expertise the participant has in the task, the smaller the effect of task difficulty on user reliance.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [57] source

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    Participants had enough expertise in the task to make an accurate judgement of the true accuracy of the AI, as reflected in their agreement ratio and perceived trust decreasing when AI performance was low.

    “User expertise in the task was also reflected in their ability to accurately differentiate between varying AI performance, as evidenced by their agreement ratio with the AI (Fig. 2, b) and perceived trust in AI (Fig. 4, c) decreased when AI performance was low, showing that the participants had enough expertise in the task to make an accurate judgement of the true accuracy of the AI.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    An AI agent may, based on observed gaze behavior, provide timely feedback to help its user better calibrate their reliance on the AI for more optimal task performance.

    “As an example, an AI agent may based on observed gaze behavior provide timely feedback to help its user better calibrate their reliance on the AI for more optimal task performance.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    A holistic view of the user state can further allow the AI to adaptively assist the user in complex decision making.

    “A holistic view of the user state can further allow the AI to adaptively assist the user in complex decision making.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    When the task is more difficult, people are more hesitant towards trusting the AI, even though more trust in the AI would lead to a higher performance in this case.

    “As a result, when the task is more difficult, they are more hesitant towards trusting the AI, even though more trust in the AI would lead to a higher performance in this case.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: In the bottom-view task, more trust in the AI would lead to higher performance.

  • May combine two claims.

    Interpretationown data

    Even though participants were not informed of AI performance, they were able to figure out the quality of the AI based on the quality of its suggestions and adjust their trust and reliance accordingly.

    “Furthermore, this result shows that even though the participants were not informed the AI performance, they were able to figure out the quality of the AI based on the quality of its suggestions and adjust their trust and reliance accordingly.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Real-time gaze attention information (fixation location and pattern) can be used to guide AI behavior to direct the user's attention to places where the user might have overlooked.

    “For example, real-time gaze attention information (fixation location and pattern) can be used to guide AI behavior to direct the user's attention to places where the user might have overlooked.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    The gaze cascade model explains the positive correlation: as users spend more time looking at the AI suggestion, their preference for it increases, forming a positive feedback that leads them to agree more with the AI suggestion.

    “This result follows the gaze cascade model, after the AI suggestion appears, as the user spends more time looking at the AI suggestion, their preference for the AI suggestion increases, which increases the amount of time they spend with the AI suggestion, which in turn increases user exposure to the AI suggestion, forming a positive feedback that leads them to agree more with the AI suggestion.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The gaze cascade model applies to user-AI interaction.

  • May combine two claims.

    Interpretationown data

    Participants were able to maintain their agency and did not blindly follow nor ignore the AI suggestion.

    “This result reflects that the participants were able to maintain their agency and did not blindly follow nor ignore the AI suggestion.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationcitation

    The user's cognitive load as estimated by pupil diameter and gaze entropy can be used to regulate the amount of information provided by the AI.

    “The user's cognitive load as estimated by pupil diameter and gaze entropy can be used to regulate the amount of information provided by the AI.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    The results suggest nuanced differences between perception-based reliance, task-based reliance, and behavior-based reliance, illustrating the complexity of the reliance construct.

    “These results suggest nuanced differences between perception-based reliance, task-based reliance, and behavior-based reliance, illustrating the complexity of the reliance construct.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    This work contributes an initial exploration of how eye behaviors may be incorporated into the computational characterization and estimation of the holistic user state.

    “All in all, this work contributes an initial exploration of how eye behaviors may be incorporated into the computational characterization and estimation of the holistic user state and paves the way for enabling productive human-AI collaboration.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    When the AI is inconsistent regarding its performance (the medium performance AI makes minor errors in four out of ten trials), people are more cautious towards trust for the AI.

    “One potential explanation is that when the AI is inconsistent regarding its performance (the medium performance AI makes minor errors in four out of ten trials), people are more cautious towards trust for the AI.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    This work paves the way for enabling productive human-AI collaboration.

    “All in all, this work contributes an initial exploration of how eye behaviors may be incorporated into the computational characterization and estimation of the holistic user state and paves the way for enabling productive human-AI collaboration.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Prior work [57] showed that increase in task complexity and difficulty heightens reliance.

    “This finding contradicts the results from prior work [57], which showed that increase in task complexity and difficulty heightens reliance.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [57] source

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    The behavioral metric of gaze duration allows for real-time estimation of user reliance, which can be used to enable adaptive human-AI collaboration.

    “In addition to helping understand user reliance on AI, this behavioral metric allows for real-time estimation of user reliance, which can be used to enable adaptive human-AI collaboration.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    This work points toward the possibility of gaze-aware, adaptive human-AI collaboration.

    “Altogether, this work offers an empirical understanding of how people consider AI suggestions during spatial reasoning through gaze behavior and points toward the possibility of gaze-aware, adaptive human-AI collaboration.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationown data

    The results suggest nuanced differences between the conceptions of user trust and reliance.

    “Additionally, our results suggest nuanced differences between the conceptions of user trust and reliance.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Prior research in human-robot collaboration has demonstrated how a robot can anticipate its human collaborator's task needs based on the observation of the collaborator's eye gaze and provide anticipatory robot assistance [29].

    “Prior research in the domain of human-robot collaboration has demonstrated how a robot can anticipate its human collaborator's task needs based on the observation of the collaborator's eye gaze and provide anticipatory robot assistance [29].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [29] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The study only focused on using gaze duration in understanding human-AI collaboration.

    “Finally, we only focused on using gaze duration in understanding the complex process of human-AI collaboration.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning

    Participants' interaction with the AI and their level of reliance on the AI may be different if they were in a higher stakes scenario.

    “The participants' interaction with the AI and their level of reliance on the AI may be different if they were in a higher stakes scenario.”
    Quote in the paperNot a substantial claim
54 of 54

  • The paper relies on this without support.

    AI systems are not error-free.

    “since AI systems are not error-free.”

  • The paper relies on this without support.

    Most prior works assess reliance and trust in human-AI interaction through self-reported measures (perceived reliance and trust) and task measures (user AI agreement fraction).

    “However, most prior works assess reliance and trust in human-AI interaction through self-reported measures (perceived reliance and trust) and task measures (user AI agreement fraction).”

  • The paper relies on this without support.

    Proper calibration of human reliance on AI is fundamental to achieving complementary performance in AI-assisted human decision-making.

    “Proper calibration of human reliance on AI is fundamental to achieving complementary performance in AI-assisted human decision-making.”

  • The paper relies on this without support.

    The bottom-view task requires one more level of mental rotation than the top-view task.

    “We expected bottom-view tasks to be more difficult as they required one more level mental rotation than top-view tasks.”

  • The paper relies on this without support.

    In the bottom-view task, more trust in the AI would lead to higher performance.

    “even though more trust in the AI would lead to a higher performance in this case.”

  • The paper relies on this without support.

    The gaze cascade model applies to user-AI interaction.

    “This result follows the gaze cascade model, after the AI suggestion appears, as the user spends more time looking at the AI suggestion, their preference for the AI suggestion increases, which increases the amount of time they spend with the AI suggestion, which in turn increases user exposure to the AI suggestion, forming a positive feedback that leads them to agree more with the AI suggestion.”
6 of 6

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • Consistentχ²(6) = 25.6reported p < .001 · recomputed p < .001

    “χ 2 (6, N = 70) = 25.60,p < .001”

  • ConsistentF(1, 68) = 147.67reported p < .001 · recomputed p < .001

    “F (1, 68) = 147.67,p < .001”

  • ConsistentF(1, 31.91) = 11.06reported p = .002 · recomputed p = .002

    “F (1, 31.91)= 11.06,p = .002”

  • ConsistentF(2, 32.02) = 6.64reported p = .004 · recomputed p = .004

    “F (2, 32.02) = 6.64, p = .004”

  • ConsistentF(2, 31.91) = 3.25reported p = .052 · recomputed p = .052

    “F (2, 31.91)= 3.25, p = .052”

  • ConsistentF(2, 32.15) = 124.58reported p < .001 · recomputed p < .001

    “F (2, 32.15) = 124.58,p < .001”

  • ConsistentF(1, 32.02) = 1.19reported p = .283 · recomputed p = .283

    “F (1, 32.02) = 1.19, p = .283”

  • ConsistentF(2, 32) = 5.58reported p = .008 · recomputed p = .008

    “F (2, 32) = 5.58, p = .008”

  • ConsistentF(2, 32.11) = 12.73reported p < .001 · recomputed p < .001

    “F (2, 32.11) = 12.73, p < .001”

  • ConsistentF(1, 31.8) = 3.89reported p = .057 · recomputed p = .057

    “F (1, 31.8)= 3.89, p = .057”

  • ConsistentF(2, 31.78) = 0.31reported p = .739 · recomputed p = .736

    “F (2, 31.78)= 0.31, p = .739”

  • ConsistentF(1, 32.42) = 16.18reported p < .001 · recomputed p < .001

    “F (1, 32.42) = 16.18,p < .001”

  • ConsistentF(2, 32.14) = 0.74reported p = .487 · recomputed p = .485

    “F (2, 32.14) = 0.74, p = .487”

  • ConsistentF(1, 634.5) = 22.95reported p < .001 · recomputed p < .001

    “F (1, 634.5) = 22.95, p < .001”

  • ConsistentF(1, 625.1) = 17.11reported p < .001 · recomputed p < .001

    “F (1, 625.1) = 17.11, p < .001”

  • ConsistentF(2, 634.7) = 3.98reported p = .019 · recomputed p = .019

    “F (2, 634.7) = 3.98, p = .019”

  • ConsistentF(2, 32.47) = 1.25reported p = .301 · recomputed p = .300

    “F (2, 32.47) = 1.25, p = .301”

  • ConsistentF(2, 625.4) = 1.06reported p = .347 · recomputed p = .347

    “F (2, 625.4) = 1.06, p = .347”

  • Consistentr(172) = 0.19reported p = .011 · recomputed p = .012

    “r (172) = .19,p = .011”

  • Consistentr(171) = 0.19reported p = .012 · recomputed p = .012

    “r (171) = .19,p = .012”

  • Consistentr(172) = 0.37reported p < .001 · recomputed p < .001

    “r (172) = .37,p < .001”

  • Consistentr(171) = 0.3reported p < .001 · recomputed p < .001

    “r (171) = .30,p < .001”

  • ConsistentF(2, 31.98) = 111.46reported p < .001 · recomputed p < .001

    “F (2, 31.98)= 111.46,p < .001”

  • ConsistentF(1, 32.2) = 1.48reported p = .233 · recomputed p = .233

    “F (1, 32.2) = 1.48, p = .233”

  • ConsistentF(2, 32.18) = 4.36reported p = .046 · recomputed p = .021

    “F (2, 32.18) = 4.36, p = .046”

    The paper states a correction for multiple tests

  • ConsistentF(2, 32.07) = 109.22reported p < .001 · recomputed p < .001

    “F (2, 32.07) = 109.22,p < .001”

  • ConsistentF(1, 31.65) = 1.71reported p = .2 · recomputed p = .200

    “F (1, 31.65)= 1.71, p = .200”

  • ConsistentF(2, 31.64) = 4.09reported p = .026 · recomputed p = .026

    “F (2, 31.64)= 4.09, p = .026”

  • ConsistentF(2, 32.07) = 8.39reported p = .001 · recomputed p = .001

    “F (2, 32.07) = 8.39, p = .001”

  • ConsistentF(1, 32.27) = 30.01reported p < .001 · recomputed p < .001

    “F (1, 32.27) = 30.01,p < .001”

  • ConsistentF(2, 32.25) = 1.6reported p = .217 · recomputed p = .218

    “F (2, 32.25) = 1.60, p = .217”

  • Consistentχ²(36) = 173.93reported p < .001 · recomputed p < .001

    “χ 2 (36, N = 347) = 173.93,p < .001”

  • Consistentr(172) = 0.04reported p = .596 · recomputed p = .600

    “r (172) = .04,p = .596”

  • Consistentr(171) = 0.07reported p = .334 · recomputed p = .360

    “r (171) = .07,p = .334”

34 of 34

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Table 2 caption names the wrong interaction

    Minor

    The results refer to Table 2 for the interaction between AI accuracy level and task phase, and the table lists AI performance by phase. Its caption instead says it reports the interaction between task difficulty and phase.

  • GSR is described as both unrelated to trust and predictive of it

    Minor

    The related-work section says no study has observed a relationship between GSR and trust in machines, then cites a study in which GSR predicted whether a user would trust an AI suggestion.

2 of 2

Cites

Flagged first, then by citations

Research works printed but not linked: 26

  1. Nora Balfe , Sarah Sharples , and John R Wilson . 2018. Understanding is key: An analysis of factors pertaining to trust in a real-world automation system. Human factors , Vol. 60 , 4 ( 2018 ), 477--495. Nora Balfe, Sarah Sharples, and John R Wilson. 2018. Understanding is key: An analysis of factors pertaining to trust in a real-world automation system. Human factors, Vol. 60, 4 (2018), 477--495.
  2. James H Bertera and Keith Rayner . 2000. Eye movements and the span of the effective stimulus in visual search. Perception & psychophysics , Vol. 62 , 3 ( 2000 ), 576--585. James H Bertera and Keith Rayner. 2000. Eye movements and the span of the effective stimulus in visual search. Perception & psychophysics, Vol. 62, 3 (2000), 576--585.
  3. D Birchall . 2015. Spatial ability in radiologists: a necessary prerequisite? The British journal of radiology , Vol. 88 , 1049 ( 2015 ), 20140511. D Birchall. 2015. Spatial ability in radiologists: a necessary prerequisite? The British journal of radiology, Vol. 88, 1049 (2015), 20140511.
  4. Eric Bogert , Aaron Schecter , and Richard T Watson . 2021. Humans rely more on algorithms than social influence as a task becomes more difficult. Scientific reports , Vol. 11 , 1 ( 2021 ), 1--9. Eric Bogert, Aaron Schecter, and Richard T Watson. 2021. Humans rely more on algorithms than social influence as a task becomes more difficult. Scientific reports, Vol. 11, 1 (2021), 1--9.
  5. Béatrice Cahour and Jean-Francc ois Forzy . 2009. Does projection into use improve trust and exploration? An example with a cruise control system. Safety science , Vol. 47 , 9 ( 2009 ), 1260--1270. Béatrice Cahour and Jean-Francc ois Forzy. 2009. Does projection into use improve trust and exploration? An example with a cruise control system. Safety science, Vol. 47, 9 (2009), 1260--1270.
  6. Alexandra Frischen , Andrew P Bayliss , and Steven P Tipper . 2007. Gaze cueing of attention: visual attention, social cognition, and individual differences. Psychological bulletin , Vol. 133 , 4 ( 2007 ), 694. Alexandra Frischen, Andrew P Bayliss, and Steven P Tipper. 2007. Gaze cueing of attention: visual attention, social cognition, and individual differences. Psychological bulletin, Vol. 133, 4 (2007), 694.
  7. Mary M Hayhoe . 2017. Vision and action. Annual review of vision science , Vol. 3 ( 2017 ), 389--413. Mary M Hayhoe. 2017. Vision and action. Annual review of vision science, Vol. 3 (2017), 389--413.
  8. Chien-Ming Huang , Sean Andrist , Allison Sauppé , and Bilge Mutlu . 2015. Using gaze patterns to predict task intent in collaboration. Frontiers in psychology , Vol. 6 ( 2015 ), 1049. Chien-Ming Huang, Sean Andrist, Allison Sauppé, and Bilge Mutlu. 2015. Using gaze patterns to predict task intent in collaboration. Frontiers in psychology, Vol. 6 (2015), 1049.
  9. Shigao Huang , Jie Yang , Simon Fong , and Qi Zhao . 2020. Artificial intelligence in cancer diagnosis and prognosis: Opportunities and challenges. Cancer letters , Vol. 471 ( 2020 ), 61--71. Shigao Huang, Jie Yang, Simon Fong, and Qi Zhao. 2020. Artificial intelligence in cancer diagnosis and prognosis: Opportunities and challenges. Cancer letters, Vol. 471 (2020), 61--71.
  10. Maia Jacobs , Melanie F Pradier , Thomas H McCoy , Roy H Perlis , Finale Doshi-Velez , and Krzysztof Z Gajos . 2021. How machine-learning recommendations influence clinician treatment selections: the example of antidepressant selection. Translational psychiatry , Vol. 11 , 1 ( 2021 ), 1--9. Maia Jacobs, Melanie F Pradier, Thomas H McCoy, Roy H Perlis, Finale Doshi-Velez, and Krzysztof Z Gajos. 2021. How machine-learning recommendations influence clinician treatment selections: the example of antidepressant selection. Translational psychiatry, Vol. 11, 1 (2021), 1--9.
  11. 10.1145/3287560.3287590 · DOI · Paper page
  12. John D Lee and Katrina A See . 2004. Trust in automation: Designing for appropriate reliance. Human factors , Vol. 46 , 1 ( 2004 ), 50--80. John D Lee and Katrina A See. 2004. Trust in automation: Designing for appropriate reliance. Human factors, Vol. 46, 1 (2004), 50--80.
  13. Zhiyuan ?Jerry" Lin , Jongbin Jung , Sharad Goel , and Jennifer Skeem . 2020. The limits of human predictions of recidivism. Science advances , Vol. 6 , 7 ( 2020 ), eaaz0652. Zhiyuan ?Jerry" Lin, Jongbin Jung, Sharad Goel, and Jennifer Skeem. 2020. The limits of human predictions of recidivism. Science advances, Vol. 6, 7 (2020), eaaz0652.
  14. Han Liu , Vivian Lai , and Chenhao Tan . 2021. Understanding the Effect of Out-of-distribution Examples and Interactive Explanations on Human-AI Decision Making. arXiv preprint arXiv:2101.05303 ( 2021 ). Han Liu, Vivian Lai, and Chenhao Tan. 2021. Understanding the Effect of Out-of-distribution Examples and Interactive Explanations on Human-AI Decision Making. arXiv preprint arXiv:2101.05303 (2021). · arXiv
  15. Maria Madsen and Shirley Gregor . 2000. Measuring human-computer trust . In 11th australasian conference on information systems , Vol. 53 . Citeseer , 6--8. Maria Madsen and Shirley Gregor. 2000. Measuring human-computer trust. In 11th australasian conference on information systems, Vol. 53. Citeseer, 6--8.
  16. Sina Mohseni , Fan Yang , Shiva Pentyala , Mengnan Du , Yi Liu , Nic Lupfer , Xia Hu , Shuiwang Ji , and Eric Ragan . 2020. Machine Learning Explanations to Prevent Overtrust in Fake News Detection. arXiv preprint arXiv:2007.12358 ( 2020 ). Sina Mohseni, Fan Yang, Shiva Pentyala, Mengnan Du, Yi Liu, Nic Lupfer, Xia Hu, Shuiwang Ji, and Eric Ragan. 2020. Machine Learning Explanations to Prevent Overtrust in Fake News Detection. arXiv preprint arXiv:2007.12358 (2020). · arXiv
  17. Mahsan Nourani , Chiradeep Roy , Jeremy E Block , Donald R Honeycutt , Tahrima Rahman , Eric Ragan , and Vibhav Gogate . 2021 . Anchoring Bias Affects Mental Model Formation and User Reliance in Explainable AI Systems. In 26th International Conference on Intelligent User Interfaces. 340--350 . Mahsan Nourani, Chiradeep Roy, Jeremy E Block, Donald R Honeycutt, Tahrima Rahman, Eric Ragan, and Vibhav Gogate. 2021. Anchoring Bias Affects Mental Model Formation and User Reliance in Explainable AI Systems. In 26th International Conference on Intelligent User Interfaces. 340--350.
  18. Michael I Posner and Steven E Petersen . 1990. The attention system of the human brain. Annual review of neuroscience , Vol. 13 , 1 ( 1990 ), 25--42. Michael I Posner and Steven E Petersen. 1990. The attention system of the human brain. Annual review of neuroscience, Vol. 13, 1 (1990), 25--42.
  19. Keith Rayner . 1998. Eye movements in reading and information processing: 20 years of research. Psychological bulletin , Vol. 124 , 3 ( 1998 ), 372. Keith Rayner. 1998. Eye movements in reading and information processing: 20 years of research. Psychological bulletin, Vol. 124, 3 (1998), 372.
  20. Toshiki Saito , Ryunosuke Sudo , and Yuji Takano . 2020. The gaze bias effect in toddlers: Preliminary evidence for the developmental study of visual decision-making. Developmental science , Vol. 23 , 6 ( 2020 ), e12969. Toshiki Saito, Ryunosuke Sudo, and Yuji Takano. 2020. The gaze bias effect in toddlers: Preliminary evidence for the developmental study of visual decision-making. Developmental science, Vol. 23, 6 (2020), e12969.
  21. D Selvathi and A Aarthy Poornila . 2017 . Breast cancer detection in mammogram images using deep learning technique . Middle-East Journal of Scientific Research , Vol. 25 , 2 (2017), 417 -- 426 . D Selvathi and A Aarthy Poornila. 2017. Breast cancer detection in mammogram images using deep learning technique. Middle-East Journal of Scientific Research, Vol. 25, 2 (2017), 417--426.
  22. Mona SharifHeravi , John R Taylor , Christopher J Stanton , Sandra Lambeth , and Christopher Shanahan . 2020 . It's a Disaster! Factors Affecting Trust Development and Repair Following Agent Task Failure . In Proceedings of the 2020 Australasian Conference on Robotics and Automation (ACRA 2020 ), 8--10 December 2020, Brisbane, Queensland. Mona SharifHeravi, John R Taylor, Christopher J Stanton, Sandra Lambeth, and Christopher Shanahan. 2020. It's a Disaster! Factors Affecting Trust Development and Repair Following Agent Task Failure. In Proceedings of the 2020 Australasian Conference on Robotics and Automation (ACRA 2020), 8--10 December 2020, Brisbane, Queensland.
  23. Shinsuke Shimojo , Claudiu Simion , Eiko Shimojo , and Christian Scheier . 2003. Gaze bias both reflects and influences preference. Nature neuroscience , Vol. 6 , 12 ( 2003 ), 1317--1322. Shinsuke Shimojo, Claudiu Simion, Eiko Shimojo, and Christian Scheier. 2003. Gaze bias both reflects and influences preference. Nature neuroscience, Vol. 6, 12 (2003), 1317--1322.
  24. Claudiu Simion and Shinsuke Shimojo . 2006. Early interactions between orienting, visual sampling and decision making in facial preference. Vision research , Vol. 46 , 20 ( 2006 ), 3331--3335. Claudiu Simion and Shinsuke Shimojo. 2006. Early interactions between orienting, visual sampling and decision making in facial preference. Vision research, Vol. 46, 20 (2006), 3331--3335.
  25. David H Uttal and Cheryl A Cohen . 2012. Spatial thinking and STEM education: When, why, and how? In Psychology of learning and motivation . Vol. 57 . Elsevier , 147--181. David H Uttal and Cheryl A Cohen. 2012. Spatial thinking and STEM education: When, why, and how? In Psychology of learning and motivation. Vol. 57. Elsevier, 147--181.
  26. Chuhao Wu , Jackie Cha , Jay Sulek , Tian Zhou , Chandru P Sundaram , Juan Wachs , and Denny Yu. 2020. Eye-tracking metrics predict perceived workload in robotic surgical skills training. Human factors , Vol. 62 , 8 ( 2020 ), 1365--1386. Chuhao Wu, Jackie Cha, Jay Sulek, Tian Zhou, Chandru P Sundaram, Juan Wachs, and Denny Yu. 2020. Eye-tracking metrics predict perceived workload in robotic surgical skills training. Human factors, Vol. 62, 8 (2020), 1365--1386.

Cited by

Sorted by citations

Related

Authors
Cao, S., & Huang, C.-M.
Affiliations
Johns Hopkins University (US)
ORCID
Venue
Proceedings of the ACM on Human-Computer Interaction
Publisher
Association for Computing Machinery (ACM)
Published
2022-11-07
Volume
6
Issue
CSCW2
Pages
1-23
ISSN
2573-0142 (ISSN-L)
DOI
10.1145/3555572
OpenAlex
W4309618893
Semantic Scholar
4fbd0b378d814be3bf65607ddf55cf0cc4209ad8
Cited by
71
References
57
Citations by year
2026: 25 · 2025: 25 · 2024: 16 · 2023: 5
FWCI
10.87
Type
journal article
Language
en
Open access
Licence
Locations
Proceedings of the ACM on Human-Computer Interaction (publishedVersion, cc-by)
Funders
National Science Foundation (1840088)
Topics
  • Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
  • Physical Sciences › Computer Science › Human-Computer Interaction › Gaze Tracking and Assistive Technology
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
Keywords
AI-assisted decision-making, gaze, trust in AI, user agency, spatial reasoning, task difficulty, gaze duration, human-AI teaming

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-10.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-10.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review