The DOI matches its DataCite record; 67 of the 67 references it cites were found in a registry or an index; none of the 140 sources its claims cite is checked for support yet.
From Trust to Appropriate Reliance: Measurement Constructs in Human-AI Decision-Making
Open accessAbstract
While human-AI decision-making research has primarily used trust measurements to assess the practical usage of AI systems by their end-users, recent empirical evidence suggests that trust measurements do not inform users' appropriate reliance on AI systems. While examining the human-AI decision-making literature, in this work, we review empirical studies that assess people's appropriate reliance on AI advice, differentiating measurements and constructs of appropriate reliance from trust and mere reliance. Our analysis of literature shows that constructs for human-AI appropriate reliance are still fragmented in research. We present three views on appropriate reliance, namely Traditional, Appropriateness, and Dominance, as discussed in research. Using these views, we evaluate objective metrics reported in studies and argue for their consensus to facilitate the comparison across empirical research. We also discuss how studies employ objective metrics and examine their validity in application contexts. Our work contributes to the critical body of research on exploring objective metrics for assessing humans' appropriate reliance on AI advice.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This review analyzes 22 empirical studies of human-AI appropriate reliance, distinguishing objective reliance metrics from trust and mere reliance. It finds that appropriate-reliance constructs remain fragmented across three views—Traditional, Appropriateness, and Dominance—and that no consensus exists on standardized metrics.
Key findings
- Across 729 initial search results, 22 eligible studies were included; most were published between 2023 and 2025. The Traditional view was applied in around 10 studies (45%), the Appropriateness view in 10 studies (45%), and the Dominance view in 2 studies.
- Objective metrics varied: accuracy was most common (16 studies), followed by over-reliance (13), RAIR (10), RSR (9), under-reliance (8), agreement fraction (8), switch fraction (8), and Accuracy-wid (5). Subjective metrics included confidence (8), Trust in Automation (5), perceived trust (4), mental demand (5), usefulness (4), and understanding (4).
- Most studies used sequential/two-step decision protocols: 90% (20 studies) versus 18% (4 studies) using concurrent protocols. Interventions showed mixed effects: cognitive forcing, deductive-XAI, second opinion, and cognitive effort were positive; task difficulty, poor impression, self-assessment, and domain knowledge were negative; example-based XAI, feature-based XAI, and showing AI uncertainty had no effect.
- Experimental designs relied heavily on lay users or crowd workers (20 of 22 studies), with a mean sample of 309.18 participants (SD = 273.93); 45% did not use tasks relevant to users’ expertise, and 7 studies (33%) used simulated rather than actual AI systems.
Methods
- Systematic literature review following the PRISMA framework, with searches in SCOPUS (N = 369) and ACM Digital Library (N = 360), totaling 729 results; lower bound set to 2018.
- Inclusion criteria required peer-reviewed full papers, human-subject decision-making studies, quantitative methods, and objective metrics of appropriate reliance (including over- and under-reliance); secondary studies, short/workshop papers, books/chapters, and non-reviewed work were excluded.
- The first author coded papers across evaluation metrics, experimental designs, and intervention methods; codes were collated into groups and themes. Snowballing of included studies’ references found no new matching results.
Limitations
- Stated: the focus on appropriate reliance measurement may have excluded studies using appropriate trust interchangeably; searches were limited to SCOPUS and ACM DL, though IEEE Xplore and Google Scholar were also checked without adding eligible results; other libraries such as AIS could be explored; screening bias and subjective interpretation may affect results.
- Evident from the text: the corpus is small (22 studies), mostly recent and from top HCI venues, and excludes non-peer-reviewed or secondary work; constructs and metrics remain fragmented with limited consensus.
- Evident from the text: most studies use classification-style binary decisions, lay/crowd workers, and one-off interactions; under-reliance is less studied than over-reliance, and metrics for generative AI, changing reliance over time, and user manipulation/contestation of AI advice are underexplored.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 of 140 cited sources checked · 0 supported · 0 flagged
- Findingown data
Constructs for human-AI appropriate reliance are still fragmented in research.
“Our analysis of literature shows that constructs for human-AI appropriate reliance are still fragmented in research.”
- Also stated in: § 1 Introduction · plain
“Such under-exploration provides a fragmented understanding of human-AI appropriate reliance research.”
Quote in the paperSubstantial claim- Rests on an assumption: The 22 papers selected through the PRISMA protocol are representative of the empirical human-AI appropriate reliance literature.
- Rests on an assumption: The search keywords and databases (SCOPUS and ACM DL) cover the relevant literature on human-AI appropriate reliance.
- Also stated in: § 1 Introduction · plain
- Findingown data
The literature mainly explores three views to assess users' appropriate reliance on AI advice: Traditional, Appropriateness, and Dominance.
“We present three views on appropriate reliance, namely Traditional, Appropriateness, and Dominance, as discussed in research.”
- Also stated in: § Key insights. · plain
“The literature mainly explores three main views to assess users' appropriate reliance on AI advice.”
- Also stated in: § 1 Introduction · plain
“In research,”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [10]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [72]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Rests on an assumption: The 22 papers selected through the PRISMA protocol are representative of the empirical human-AI appropriate reliance literature.
- Rests on an assumption: The search keywords and databases (SCOPUS and ACM DL) cover the relevant literature on human-AI appropriate reliance.
- Also stated in: § Key insights. · plain
- Findingcitation
No consensus exists for metrics for measuring appropriate reliance.
“In addition, no consensus exists for metrics for measuring appropriate reliance, which is used interchangeably with appropriate trust, which includes a degree of belief and personal feeling about the system alongside its actual performance [54].”
- Also stated in: § Key insights. · hedged
“Studies have also used extensions of these views to understand users' appropriate reliance on AI advice, but the research on consensus and validity of these views across domains remains limited.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [54]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Rests on an assumption: The 22 papers selected through the PRISMA protocol are representative of the empirical human-AI appropriate reliance literature.
- Rests on an assumption: The search keywords and databases (SCOPUS and ACM DL) cover the relevant literature on human-AI appropriate reliance.
- Also stated in: § Key insights. · hedged
- Findingnone given
Subjective metrics are mostly self-assessed attributes.
“These metrics are mostly self-assessed attributes and are also used to understand the impact on user performance, but they do not necessarily help measure actual reliance behavior.”
Quote in the paperNot a substantial claim - Findingcitation
There is a growing prevalence of objective measurements compared to evaluations reported in existing reviews.
“showing the growing prevalence of objective measurements compared to evaluations reported in existing reviews [26,43].”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [26]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [43]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Rests on an assumption: The 22 papers selected through the PRISMA protocol are representative of the empirical human-AI appropriate reliance literature.
- Findingnone given
The literature reports the use of a variety of objective and subjective metrics to evaluate users' reliance and appropriate reliance, as well as their perceptions towards AI.
“The literature reports the use of a variety of objective and subjective metrics to evaluate the users' reliance and appropriate reliance, as well as their perceptions towards AI.”
Quote in the paperSubstantial claim - Findingcitation
Metrics also measure how users engage with AI systems to capture whether they can appropriately assess the AI advice or not.
“Metrics also measure how users engage with AI systems [7,83] to capture whether they can appropriately assess the AI advice or not.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [7]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [83]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Findingnone given
Overall, studies show mixed results regarding the improvement of reliance on AI.
“Overall, studies show mixed results regarding the improvement of reliance on AI (see Table 3), and cite limitations in forming appropriate human-AI reliance.”
Quote in the paperSubstantial claim - Findingcitation
Some studies use other constructs that may partially conform with appropriate reliance, such as user agreeableness or switching decisions with AI advice, considering correctness of final decisions.
“Some studies also use other constructs that may partially conform with appropriate reliance, such as user agreeableness, or switching decisions with AI advice [7,8], considering the correctness of final decisions.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [7]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [8]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Rests on an assumption: The 22 papers selected through the PRISMA protocol are representative of the empirical human-AI appropriate reliance literature.
- Findingreasoning
Subjective metrics are also used to understand the impact on user performance.
“These metrics are mostly self-assessed attributes and are also used to understand the impact on user performance, but they do not necessarily help measure actual reliance behavior.”
Quote in the paperNot a substantial claim - Findingcitation
The traditional view does not fully drill down on the behavior users depict during assessment.
“Though this view has objective metrics for over-and under-reliance, it does not fully drill down on the behavior users depict during the assessment [7,86,94].”
Quote in the paperSubstantial claim0 of 3 sources checked- cites [7]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [86]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [94]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Rests on an assumption: The 22 papers selected through the PRISMA protocol are representative of the empirical human-AI appropriate reliance literature.
- Findingreasoning
The use of two-stage decision-making has become a standard in recent studies because an initial human decision is essential to capture for the employed metrics.
“The use of two-stage decision-making has become a standard in recent studies, as an initial human decision is essential to capture for the employed metrics for capturing appropriate reliance.”
Quote in the paperNot a substantial claim- Rests on an assumption: The 22 papers selected through the PRISMA protocol are representative of the empirical human-AI appropriate reliance literature.
- Findingcitation
Recent studies use over-reliance (13 studies) and under-reliance (8 studies) as separate metrics.
“Recent studies explore the metrics for capturing appropriate reliance by applying traditional metrics like over-reliance [7] (13 studies) and underreliance [11] (8 studies) as separate metrics, which capture when users blindly agree with incorrect advice and neglect correct advice, respectively.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [7]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [11]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Rests on an assumption: The 22 papers selected through the PRISMA protocol are representative of the empirical human-AI appropriate reliance literature.
- Findingown data
Around 18% of studies (4 studies) use a concurrent process.
“Though not very common, around around 18% of the studies (4 studies), often as part of multi-study experiments, use concurrent process.”
Quote in the paperNot a substantial claim- Rests on an assumption: The 22 papers selected through the PRISMA protocol are representative of the empirical human-AI appropriate reliance literature.
- Findingnone given
Subjective impressions such as trust and confidence do not necessarily reveal whether users can build an appropriate reliance on AI.
“However, subjective impressions (such as trust and confidence) do not necessarily reveal whether users can build an appropriate reliance on AI.”
Quote in the paperSubstantial claim - Findingown data
Most selected studies were published between 2023 and 2025.
“Most selected studies were published between 2023 and 2025, showing the recent conceptualization of appropriate reliance.”
Quote in the paperNot a substantial claim- Rests on an assumption: The 22 papers selected through the PRISMA protocol are representative of the empirical human-AI appropriate reliance literature.
- Findingcitation
Studies use relative AI reliance (RAIR) (10 studies) and relative self-reliance (RSR) (9 studies) as metrics aligned with the appropriateness view.
“In terms of metrics aligned with the appropriateness [73] view, studies also use relative AI reliance (RAIR) (10 studies) and relative self-reliance (RSR) (9 studies), as defined in Section 4.1.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [73]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Rests on an assumption: The 22 papers selected through the PRISMA protocol are representative of the empirical human-AI appropriate reliance literature.
- Findingown data
90% of studies (20) in the corpus mainly use a two-step (or multi-step) process where humans first make decisions and then receive AI advice.
“Therefore, 90% of the studies (20) in our corpus mainly use a two-step (or multi-step) process where humans first make the decisions, investigating the task themselves, and then are provided with the AI advice, subsequently having the option to revise their decision.”
Quote in the paperSubstantial claim- Rests on an assumption: The 22 papers selected through the PRISMA protocol are representative of the empirical human-AI appropriate reliance literature.
- Findingcitation
Accuracy-wid (accuracy with initial disagreement) is used in 5 studies to capture appropriate reliance when users disagree with AI advice initially.
“Likewise, Accuracy-wid [33] (accuracy with initial disagreement -5 studies) is used to capture users' appropriate reliance when they disagree with the AI advice initially.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [33]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Rests on an assumption: The 22 papers selected through the PRISMA protocol are representative of the empirical human-AI appropriate reliance literature.
- Findingreasoning
Subjective metrics do not necessarily help measure actual reliance behavior.
“These metrics are mostly self-assessed attributes and are also used to understand the impact on user performance, but they do not necessarily help measure actual reliance behavior.”
Quote in the paperSubstantial claim - Findingreasoning
There is still limited consensus on common measurements of humans' appropriate reliance on AI across the studies due to variance in using metrics to measure users' performance.
“Due to the variance in using metrics to measure users' performance, there is still limited consensus on common measurements of humans' appropriate reliance on AI across the studies.”
Quote in the paperSubstantial claim - Findingown data
Over-reliance, RAIR, RSR, and under-reliance are commonly used as appropriate reliance metrics.
“In terms of objective metrics, we find that over-reliance, RAIR, RSR, and under-reliance are commonly used as appropriate reliance metrics.”
Quote in the paperSubstantial claim - Findingcitation
The distinction between measurements and constructs for reliance and appropriate reliance has been overlooked in recent work.
“The distinction between measurements and the constructs for reliance and appropriate reliance has been overlooked in recent work [26,43].”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [26]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [43]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Rests on an assumption: The 22 papers selected through the PRISMA protocol are representative of the empirical human-AI appropriate reliance literature.
- Rests on an assumption: The search keywords and databases (SCOPUS and ACM DL) cover the relevant literature on human-AI appropriate reliance.
- Findingnone given
Switch fractions and agreement fractions are used for capturing user behavior without the focus on appropriate reliance.
“Other metrics, such as switch fractions and agreement fractions, are used, but for capturing user behavior without the focus on appropriate reliance.”
Quote in the paperNot a substantial claim - Findingnone given
Several subjective metrics only measure perception or feedback, for instance, to capture trust and confidence for gauging users' perception of AI systems.
“In terms of subjective metrics, we find several metrics that only measure perception or feedback, for instance, to capture trust and confidence for gauging the users' perception of AI systems.”
Quote in the paperSubstantial claim - Findingreasoning
The combination of objective and subjective metrics does not necessarily lead to better predictions of reliance.
“The combination of objective and subjective metrics does not necessarily lead to better predictions of reliance; for instance, users can express high trust towards a system that aligns with them, without any objective indication of appropriate reliance.”
Quote in the paperSubstantial claim - Findingcitation
10 recent studies (45%) apply the appropriateness view for measuring users' reliance on AI advice.
“As this view was introduced in 2023 [73], 10 recent studies (45%) apply it for measuring users' reliance on AI advice.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [73]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Rests on an assumption: The 22 papers selected through the PRISMA protocol are representative of the empirical human-AI appropriate reliance literature.
- Findingreasoning
The traditional view's measurements mainly focus on the accuracy of final decisions, i.e., adopting correct AI advice and rejecting incorrect AI advice.
“The alignment is considered holistically instead of evaluating users' ability to differentiate decision-making outcomes, and measurements mainly focus on the accuracy of final decisions, i.e., adopting correct AI advice and rejecting incorrect AI advice.”
Quote in the paperSubstantial claim- Rests on an assumption: The 22 papers selected through the PRISMA protocol are representative of the empirical human-AI appropriate reliance literature.
- Findingnone given
Research on consensus and validity of the three views across domains remains limited.
“Studies have also used extensions of these views to understand users' appropriate reliance on AI advice, but the research on consensus and validity of these views across domains remains limited.”
Quote in the paperSubstantial claim- Rests on an assumption: The 22 papers selected through the PRISMA protocol are representative of the empirical human-AI appropriate reliance literature.
- Findingcitation
Understanding appropriate reliance and its measurements and constructs has been underexplored.
“In research, the metrics of measuring trust and reliance are studied [26,92]; still, understanding appropriate reliance and its measurements and constructs has been underexplored.”
- Also stated in: § Our Contribution. · plain
“Despite recent research emphasizing the need to differentiate subjective trust from objective reliance, limited work has focused on evaluating the relevant assessment methods used to study human-AI appropriate reliance [26,43,73].”
Quote in the paperSubstantial claim0 of 4 sources checked- cites [26]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [92]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [43]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [73]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Rests on an assumption: The 22 papers selected through the PRISMA protocol are representative of the empirical human-AI appropriate reliance literature.
- Rests on an assumption: The search keywords and databases (SCOPUS and ACM DL) cover the relevant literature on human-AI appropriate reliance.
- Also stated in: § Our Contribution. · plain
- Findingcitation
Studies have used several interventions, such as cognitive forcing or frictional AI, with metrics to capture users' over-reliance.
“Studies have used several interventions (e.g., cognitive forcing [7] or frictional AI [9]) with metrics to capture users' over-reliance.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [7]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [9]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Findingnone given
Studies often supplement the objective metrics of human reliance on AI with various subjective metrics aimed at capturing user perceptions towards AI.
“Studies often supplement the objective metrics of human reliance on AI with various subjective metrics, aiming at capturing user perceptions towards AI.”
Quote in the paperNot a substantial claim - Findingnone given
Studies cite limitations in forming appropriate human-AI reliance.
“Overall, studies show mixed results regarding the improvement of reliance on AI (see Table 3), and cite limitations in forming appropriate human-AI reliance.”
Quote in the paperSubstantial claim - Findingown data
The final corpus comprised 22 papers that employed objective metrics of appropriate reliance.
“After applying the criteria, we selected 22 papers that employed objective metrics of appropriate reliance as per the evaluation.”
Quote in the paperSubstantial claim- Rests on an assumption: The 22 papers selected through the PRISMA protocol are representative of the empirical human-AI appropriate reliance literature.
- Findingreasoning
Decision accuracy does not directly distinguish whether reliance was appropriate at a granular level.
“However, it does not directly distinguish whether reliance was appropriate at a granular level, and only evaluates whether the final decision is correct or not.”
Quote in the paperSubstantial claim- Rests on an assumption: The 22 papers selected through the PRISMA protocol are representative of the empirical human-AI appropriate reliance literature.
- Findingown data
The initial analysis of SCOPUS results showed the prevalence of the topic studied at ACM venues, particularly within human-AI research.
“Our initial analysis of SCOPUS results showed the prevalence of the topic studied at ACM venues, particularly within human-AI research.”
Quote in the paperSubstantial claim- Rests on an assumption: The search keywords and databases (SCOPUS and ACM DL) cover the relevant literature on human-AI appropriate reliance.
- Findingcitation
Around 5 studies measure AI's effect on accuracy as a means to test if providing AI advice can improve users' reliance.
“Along with accuracy, around 5 studies also measure AI's effect on accuracy [7] as a means to test if providing AI advice (against no advice) can improve users' reliance.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [7]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Rests on an assumption: The 22 papers selected through the PRISMA protocol are representative of the empirical human-AI appropriate reliance literature.
- Findingnone given
Several applied interventions have either a negative or, at best, a neutral effect on appropriate reliance.
“Studies also indicate that several applied interventions have either a negative or, at best, a neutral effect on appropriate reliance.”
Quote in the paperSubstantial claim - Findingnone given
Studies have measured humans' appropriate reliance on AI in various application domains.
“Studies have measured humans' appropriate reliance on AI in various application domains.”
Quote in the paperSubstantial claim - Findingnone given
The majority of the studies employed novice participants.
“However, the majority of the studies employed novice participants, and task scenarios may not be comparable across studies, thereby limiting the generality of the used metrics.”
Quote in the paperSubstantial claim - Findingown data
Studies recruit on average approximately 309 people (mean = 309.18, std = 273.93).
“Studies recruit on average approximately 309 people (mean = 309.18,std = 273.93).”
Quote in the paperNot a substantial claim - Findingcitation
Capturing users' reliance in scenarios where system performance and users' reliance change over time is less explored.
“Capturing users' reliance in scenarios where system performance and users' reliance change over time is less explored [76,89].”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [76]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [89]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Findingown data
Most (20 out of 22) studies use lay users or crowd workers as the study subjects performing the tasks.
“Studies show the application of diverse tasks, but most (20 out of 22) of them use lay users or crowd workers as the study subjects performing the tasks.”
Quote in the paperSubstantial claim - Findingown data
There is a nascent but growing interest in empirical research to evaluate users' appropriate reliance on AI using objective metrics.
“Our analysis shows that there is a nascent but growing interest in empirical research to evaluate users' appropriate reliance on AI using objective metrics.”
Quote in the paperSubstantial claim - Findingnone given
Certain methods, such as cognitive forcing and deliberation, show positive effects in building appropriate reliance.
“Certain methods, such as cognitive forcing and deliberation, show positive effects in building appropriate reliance.”
Quote in the paperNot a substantial claim - Findingnone given
Studies often do not relate user behavioral patterns (engagement, skills, expertise) and task complexity (easy to hard) with metrics used to capture appropriate reliance.
“However, studies often do not relate user behavioral patterns (engagement, skills, expertise) and task complexity (easy to hard) with metrics used to capture appropriate reliance.”
Quote in the paperSubstantial claim - Findingcitation
Less research work has focused on measuring users' appropriate reliance on AI.
“While existing work has focused on evaluating calibrated or appropriate trust using different metrics [54,92], less research work has focused on measuring users' appropriate reliance on AI.”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [54]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [92]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Findingown data
Around 50% (11) of the examined studies acknowledge the limitations of recruiting laypeople and using tasks that are not relevant to their expertise or stakes.
“Around 50% (11) of the examined studies acknowledge the limitations of recruiting laypeople and using tasks that are not relevant to their expertise or stakes.”
Quote in the paperNot a substantial claim - Findingcitation
Task difficulty is often not considered in measuring humans' appropriate reliance on AI advice.
“Task difficulty is often not considered in measuring humans' appropriate reliance on AI advice [68]; however, it can significantly impact user behavior with AI.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [68]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Findingnone given
Many studies (45%) did not use tasks that were relevant to the users' expertise.
“Many studies (45%) did not use tasks that were relevant to the users' expertise.”
Quote in the paperSubstantial claim - Findingnone given
Metrics to assess reliance and appropriate reliance have been mostly studied in classification-related scenarios where outputs have fixed ground truth and are considered binary (accept or reject AI advice).
“Metrics to assess reliance and appropriate reliance have been mostly studied in classification-related scenarios where outputs have fixed ground truth and are considered binary (accept or reject AI advice).”
Quote in the paperNot a substantial claim - Findingcitation
Human-AI reliance research has under-explored the users' ability to manipulate, contest, or change the AI assistance, which can be an important factor in measuring their reliance.
“Human-AI reliance research has under-explored the users' ability to manipulate, contest, or change the AI assistance, which can be an important factor in measuring their reliance [63].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [63]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Findingcitation
Research uses metrics that measure users' reliance on one-off interactions.
“Research uses metrics that measure users' reliance on one-off interactions [34,72].”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [34]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [72]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Findingcitation
Around 7 studies (33%) do not construct real AI interventions and instead use simulated approaches.
“However, around 7 studies (33%) do not construct real AI interventions, and, instead, use simulated [18] approaches, trying to replicate what users would believe or experience while interacting with AI-assisted decision support.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [18]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Findingnone given
Task scenarios may not be comparable across studies.
“However, the majority of the studies employed novice participants, and task scenarios may not be comparable across studies, thereby limiting the generality of the used metrics.”
Quote in the paperSubstantial claim - Findingnone given
The operationalization of certain methods in tasks is seldom measured in practice.
“However, their operationalization in tasks is seldom measured in practice.”
Quote in the paperSubstantial claim - Findingreasoning
Intervention-specific behaviors are often under-substantiated.
“In addition, intervention-specific behaviors are often under-substantiated.”
Quote in the paperNot a substantial claim - Findingcitation
Users' expertise or skills with AI systems are mostly captured as fixed subjective self-reports.
“Users' expertise or skills with AI systems are also an important factor to measure [8], which is mostly captured as fixed subjective self-reports.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [8]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Findingcitation
Studies focused on over-reliance somewhat overlook the aspect of under-reliance.
“Broadly, studies focused on over-reliance and somewhat overlook the aspect of under-reliance, i.e., when users do not use the AI advice [82,83].”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [82]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [83]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Findingown data
The included studies have a high focus on example-based and feature-based XAI.
“Diverse explanation methods are used with a high focus on example-based and feature-based XAI.”
Quote in the paperNot a substantial claim - Findingown data
Most included studies favor multi-stage decisions.
“Studies mostly use empirical evaluations with the crowd-worker population, and favor the multi-stage decisions.”
Quote in the paperNot a substantial claim - Findingown data
Most included studies use empirical evaluations.
“Studies mostly use empirical evaluations with the crowd-worker population, and favor the multi-stage decisions.”
Quote in the paperNot a substantial claim - Findingown data
Most included studies use crowd-worker populations.
“Studies mostly use empirical evaluations with the crowd-worker population, and favor the multi-stage decisions.”
Quote in the paperNot a substantial claim - Findingown data
There are three common views in research for conceptualizing reliance: traditional, appropriateness, and dominance.
“Building upon the definitions, our analysis showed that there are three common views in research for conceptualizing reliance, namely, traditional, appropriateness, and dominance.”
Quote in the paperSubstantial claim - Premisecitation
Trust measurements do not necessarily inform or enhance users' appropriate reliance on AI systems.
“While human-AI decision-making research has primarily used trust measurements to assess the practical usage of AI systems by their end-users, recent empirical evidence suggests that trust measurements do not inform users' appropriate reliance on AI systems.”
- Also stated in: § 1 Introduction · hedged
“trust measures do not necessarily enhance reliance [72]”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [72]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Also stated in: § 1 Introduction · hedged
- Premisecitation
People are less likely to over-rely on the AI system's advice when the benefit of obtaining the correct answer substantially outweighs the risk of over-reliance, compared with situations in which the benefit-risk tradeoff is low.
“people are less likely to over-rely on the AI system's advice when the benefit of obtaining the correct answer substantially outweighs the risk of over-reliance”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [86]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Premisecitation
The concept of imperfect explanations can exist regardless of the correctness of AI systems.
“The concept of imperfect explanations [57] can exist regardless of the correctness of AI systems.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [57]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Premisecitation
Explanations of AI systems have often been reported to cause over-reliance.
“Explanations of AI systems help users in understanding AI advice, but they have often been reported to cause over-reliance [7,83].”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [7]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [83]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Premisecitation
Explanations of AI systems help users in understanding AI advice.
“Explanations of AI systems help users in understanding AI advice, but they have often been reported to cause over-reliance [7,83].”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [7]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [83]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Premisecitation
If users can analytically engage with AI advice and spend time understanding it, they may be able to reduce their over- or under-reliance on AI.
“For instance, if users can analytically engage with AI advice and spend time understanding it, they may be able to reduce their over-or under-reliance on AI [86].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [86]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Premisecitation
The traditional view assesses users' overall behavior (performance) with decision support for measuring over-reliance or under-reliance on AI.
“The traditional view [86] assesses users' overall behavior (performance) with decision support for measuring over-reliance or under-reliance on AI.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [86]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationnone given
Users can express high trust towards a system that aligns with them without any objective indication of appropriate reliance.
“The combination of objective and subjective metrics does not necessarily lead to better predictions of reliance; for instance, users can express high trust towards a system that aligns with them, without any objective indication of appropriate reliance.”
Quote in the paperSubstantial claim - Premisecitation
The point at which AI advice is given (concurrent or sequential) affects the choice of metrics.
“The point at which AI advice is given, i.e., using a concurrent or sequential (multistep) protocol, also affects the choice of metrics as both protocols slightly differ in the way they use reliance metrics [72].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [72]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationreasoning
Metrics may exhibit different outcomes when applied in domain-specific settings.
“Metrics may exhibit different outcomes when applied in domain-specific settings.”
Quote in the paperNot a substantial claim - Premisecitation
Different people trust AI differently.
“Different people trust AI differently [83], and capturing their traits and real-time behavior can help define metrics that can be applied to study personality factors in human-AI appropriate reliance.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [83]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationcitation
Assessing or measuring whether users reflect in their decision-making can provide more insights into their reliance behavior.
“Hence, assessing or measuring whether users reflect in their decision-making can provide more insights into their reliance behavior [7].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [7]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationreasoning
Using fictitious or non-representative tasks, especially when performed with crowd workers in low-stakes or irrelevant contexts, may not represent real-world situations.
“Hence, using fictitious or non-representative tasks, especially when performed with crowd workers in low-stakes or irrelevant contexts (about the task at hand), may not represent real-world situations.”
Quote in the paperSubstantial claim - Interpretationcitation
Experiments with non-realistic tasks and a lack of users' expertise on those tasks may fail to assess users' reliance appropriately.
“Experiments with non-realistic tasks and a lack of users' expertise on those tasks may fail to assess users' reliance appropriately [27,87].”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [27]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [87]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationcitation
Measuring objective performance on being able to discriminate correct/incorrect AI advice provides a realistic notion of users' appropriate reliance on AI advice, compared to using a mix of objective and subjective metrics.
“Measuring objective performance on being able to discriminate correct/incorrect AI advice provides a realistic notion of users' appropriate reliance on AI advice, compared to using a mix of objective and subjective metrics [92].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [92]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Premisecitation
Evaluations with crowd workers only capture the general notion of human-AI decision-making.
“While evaluations with crowd workers are appealing when assessing target users, they only capture the general notion of human-AI decision-making [23].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [23]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationown data
Reliance on lay users and crowd-workers limits actual evaluations of AI systems with relevant stakeholders.
“Such reliance on lay users and crowd-workers limits actual evaluations of AI systems with relevant stakeholders.”
Quote in the paperSubstantial claim - Interpretationreasoning
The lack of uniform evaluations may considerably undermine the applicability of results from studies across tasks and participants' expertise.
“The lack of uniform evaluations may considerably undermine the applicability of results from studies across tasks and participants' expertise.”
Quote in the paperSubstantial claim - Premisecitation
When users lack expertise in a task, they tend to inappropriately rely more heavily on AI advice, especially as the task becomes more complex.
“For instance, when users lack expertise in a task, they tend to (inappropriately) rely more heavily on AI advice, especially as the task becomes more complex [19,69].”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [19]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [69]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationcitation
Comparing whether existing metrics produce similar results in different task settings for a similar population can inform their validity for measuring appropriate reliance across settings.
“Comparing whether existing metrics produce similar results in different task settings for a similar population can inform their validity for measuring appropriate reliance across settings [68].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [68]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Premisecitation
Non-realistic AI systems risk fully replicating a realistic AI and human behavior.
“Although this is a well-accepted strategy to evaluate systems in research, non-realistic AI systems risk fully replicating a realistic AI and human behavior [83].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [83]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Premisecitation
Measuring users' competence and skill can inform reliance behavior.
“Buçinca et al. [8] highlighted that measuring users' competence and skill can also inform reliance behavior, as AI support often fails to address the knowledge gaps that users seek to fill [8].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [8]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Premisecitation
Evaluating AI systems in their actual form through human subject experiments that involve realistic tasks can be important to assess reliance.
“The literature acknowledges that evaluating AI systems in their actual form through human subject experiments that involve realistic tasks can be important to assess reliance [23].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [23]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Premisecitation
People's reliance on the AI is impacted by the value of the decision and the effort it takes to make correct decisions.
“Hence, people's reliance on the AI is also impacted by the value of the decision and the effort it takes to make correct decisions [69], which needs to be measured.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [69]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationcitation
Capturing people's traits and real-time behavior can help define metrics that can be applied to study personality factors in human-AI appropriate reliance.
“Different people trust AI differently [83], and capturing their traits and real-time behavior can help define metrics that can be applied to study personality factors in human-AI appropriate reliance.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [83]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationreasoning
The systems or situations in which study experiments are conducted may influence the metrics and results for human reliance on AI.
“The systems or situations in which study experiments are conducted may also influence the metrics and results for human reliance on AI.”
Quote in the paperSubstantial claim - Interpretationreasoning
Measurements in such experimental settings may not be realistic representations of actual context-specific scenarios.
“Therefore, it may be possible that measurements in such experimental settings may not be realistic representations of actual context-specific scenarios.”
Quote in the paperSubstantial claim - Premisecitation
Task complexity increases information overload pertaining to the user's perception and capabilities.
“For instance, task complexity increases information overload pertaining to the user's perception and capabilities [61], which also should be taken into account for measuring reliance.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [61]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationcitation
The use of subjective metrics reflects a lack of a common framework for selecting metrics using a standardized method.
“Their use reflects a lack of a common framework for selecting metrics using a standardized method [26].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [26]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationcitation
Measurements may miss capturing users' reliance if they provide low-effort results.
“It can also be argued that measurements may miss capturing users' reliance if they provide low-effort results [31].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [31]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Probably not a substantial claim. 76
- It may state two separate claims as one. 35
Probably not a substantial claim. The paper’s conclusions may not depend on it.
Findingnone givenThe accuracy metric is used in both cases.
“The accuracy metric is used in both cases.”
Quote in the paperNot a substantial claimIt may state two separate claims as one.
FindingcitationDecision accuracy is a commonly used metric (16 studies) that measures the correctness of final decisions.
“In selected studies, decision accuracy [11] is a commonly used (16 studies) metric that measures the correctness of final decisions.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [11]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Rests on an assumption: The 22 papers selected through the PRISMA protocol are representative of the empirical human-AI appropriate reliance literature.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
FindingcitationWeight of advice (WOA) is used as a collective metric to measure if users change their decision due to AI's assistance or exertion towards a correct decision.
“Cabitza et al. [9] also use weight of advice (WOA) as a collective metric to measure if the users change their decision due to AI's assistance or exertion towards a correct decision.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [9]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Rests on an assumption: The 22 papers selected through the PRISMA protocol are representative of the empirical human-AI appropriate reliance literature.
It may state two separate claims as one.
FindingcitationCommonly used subjective metrics in the examined studies include confidence in decisions (8 studies), trust in automation (5 studies), perceived trust (4 studies), mental demand (5 studies), usefulness (4 studies), and understanding of advice (4 studies).
“Commonly used subjective metrics include the users' confidence [94] in their decisions (8 studies), Trust in Automation [31] (5 studies) or perceived trust [34] (4 studies), mental demand [7] (5 studies), usefulness [50] (4 studies), understanding [14] of advice (4 studies), and others [12,52,83].”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites [94]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [31]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [34]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
It may state two separate claims as one.
Findingown dataAround 90% (20) of studies rely on assessing and reporting user interaction on these metrics using empirical analysis, while a few (2 studies) use mixed methods.
“Around 90% (20) of the studies in our corpus rely on assessing and reporting user interaction on these metrics using empirical analysis, while a few (2 studies) use mixed methods, showing the growing prevalence of objective measurements compared to evaluations reported in existing reviews [26,43].”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [26]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [43]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Rests on an assumption: The 22 papers selected through the PRISMA protocol are representative of the empirical human-AI appropriate reliance literature.
It may state two separate claims as one.
Findingown dataAround 10 (45%) of studies in the corpus apply the traditional view for capturing users' reliance on AI advice, mostly in earlier research studies.
“Around 10 (45%) studies in our corpus apply the traditional view for capturing users' reliance on AI advice, mostly being applied in earlier research studies.”
Quote in the paperNot a substantial claim- Rests on an assumption: The 22 papers selected through the PRISMA protocol are representative of the empirical human-AI appropriate reliance literature.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
Findingnone givenSurvey feedback is used to provide deeper insights about human behavior in reliance strategies.
“Survey feedback is also used to provide deeper insights about human behavior in reliance strategies.”
Quote in the paperNot a substantial claimProbably not a substantial claim. The paper’s conclusions may not depend on it.
Findingnone givenSome cases include reliance development over multi-stage decisions, where users explore additional information in intermediary steps before making decisions.
“Some cases include reliance development over multi-stage decisions, where users explore additional information in intermediary steps before making decisions.”
Quote in the paperNot a substantial claim- Rests on an assumption: The 22 papers selected through the PRISMA protocol are representative of the empirical human-AI appropriate reliance literature.
Probably not a substantial claim.
FindingcitationThe appropriateness view uses relative self-reliance (RSR) and relative AI reliance (RAIR) to measure users' independent and AI-dependent decisions.
“Schemmer et al. [73] define the appropriateness view as attempting to understand the users' relative self-reliance (RSR) and relative AI reliance (RAIR).”
- Also stated in: § Key insights. · plain
“The appropriateness view uses relative AI reliance (RAIR) and relative self-reliance (RSR) to measure users' independent and AI-dependent decisions.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [73]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Also stated in: § Key insights. · plain
Probably not a substantial claim. The paper’s conclusions may not depend on it.
Findingown dataThe search collected 729 results across the queried databases.
“In total, we collected 729 search results across the queried databases.”
Quote in the paperNot a substantial claim- Rests on an assumption: The search keywords and databases (SCOPUS and ACM DL) cover the relevant literature on human-AI appropriate reliance.
It may state two separate claims as one.
FindingcitationMost studies rely on fixed reliance metrics, and user reliance development over time is under-explored.
“Still, most studies rely on fixed reliance metrics (i.e., for each interaction), and user reliance development over time is under-explored [83].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [83]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Rests on an assumption: The 22 papers selected through the PRISMA protocol are representative of the empirical human-AI appropriate reliance literature.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
Findingnone givenSubjective metrics focus on measuring perceived usefulness and understanding of AI advice to capture users' experiences with the system.
“Subjective metrics focus on measuring the perceived usefulness and understanding of AI advice to capture users' experiences with the system.”
Quote in the paperNot a substantial claimProbably not a substantial claim. The paper’s conclusions may not depend on it.
Findingnone givenThe dominance view measures the benevolent or detrimental exertion of technology for final decisions.
“The dominance view measures the benevolent or detrimental exertion of technology for final decisions.”
Quote in the paperNot a substantial claim- Rests on an assumption: The 22 papers selected through the PRISMA protocol are representative of the empirical human-AI appropriate reliance literature.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
Findingnone givenThe selection of studies comes from top venues, i.e., CHI, IUI, PACMHCI, and others.
“The selection of studies comes from top venuesfoot_1 i.e., CHI, IUI, PACMHCI, and others.”
Quote in the paperNot a substantial claim- Rests on an assumption: The 22 papers selected through the PRISMA protocol are representative of the empirical human-AI appropriate reliance literature.
It may state two separate claims as one.
FindingcitationExisting reviews evaluated metrics for overall reliance and have overlooked their distinction from metrics needed for capturing appropriate reliance explicitly.
“Existing reviews [26,43] evaluated metrics for overall reliance and have overlooked their distinction from metrics needed for capturing appropriate reliance explicitly.”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [26]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [43]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Rests on an assumption: The 22 papers selected through the PRISMA protocol are representative of the empirical human-AI appropriate reliance literature.
It may state two separate claims as one.
Findingnone givenThe traditional view is commonly studied without explicit definitions and mainly originates from concepts of demonstrated (or appropriate) trust, measured through objective behavior.
“This view is commonly studied without explicit definitions, as it mainly originates from concepts of demonstrated (or appropriate) trust, measured through users' objective behavior.”
Quote in the paperNot a substantial claim- Rests on an assumption: The 22 papers selected through the PRISMA protocol are representative of the empirical human-AI appropriate reliance literature.
It may state two separate claims as one.
FindingreasoningNeither agreement fraction nor switch fraction captures whether reliance was appropriate.
“Neither of these metrics, however, captures whether the reliance was appropriate, i.e., the correctness of user decisions in cases when AI advice was correct or incorrect.”
Quote in the paperSubstantial claim- Rests on an assumption: The 22 papers selected through the PRISMA protocol are representative of the empirical human-AI appropriate reliance literature.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
FindingcitationMorrison et al. explored the metrics for capturing whether users can differentiate between correct and incorrect explanations of AI systems.
“Morrison et al. [57] explored the metrics for capturing whether users can differentiate between correct and incorrect explanations of AI systems.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [57]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim.
Findingnone givenStudies still use fragmented concepts and metrics of appropriate reliance.
“In practice, studies still use fragmented concepts and metrics of appropriate reliance.”
Quote in the paperSubstantial claimProbably not a substantial claim. The paper’s conclusions may not depend on it.
Findingnone givenCurrent research in human-AI decision-making focuses on assessing users' trust and reliance on AI systems through diverse metrics.
“Current research in human-AI decision-making focuses on assessing users' trust and reliance on AI systems through diverse metrics.”
Quote in the paperNot a substantial claimProbably not a substantial claim.
Findingnone givenMetrics for users' deskilling and agency loss are rarely used in studies.
“Over-reliance on AI can also lead users to lose their skills in the tasks, which should also be assessed by incorporating metrics for users' deskilling [8] and agency loss [63], which are rarely used in studies.”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [8]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [63]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
Findingown dataStudies have employed various applications in various high- and low-stakes domains.
“Studies have employed various applications in various high-and low-stakes domains, as depicted in (Table 5).”
Quote in the paperNot a substantial claimIt may state two separate claims as one.
Findingown dataThe field is yet to settle on metrics to measure users' appropriate reliance on AI, which is often mixed with measuring mere reliance.
“However, our analysis shows that the field is yet to settle on metrics to measure users' appropriate reliance on AI [9,43], which is often mixed with measuring mere reliance [26].”
Quote in the paperSubstantial claim0 of 3 sources checked- cites [9]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [43]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [26]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
It may state two separate claims as one.
Findingnone givenMany studies do not employ real AI systems; rather, they rely on simulated approaches, which further limit their operationalization and appropriate assessment across contexts.
“Many studies do not employ real AI systems; rather, they rely on simulated approaches, which further limit their operationalization and appropriate assessment across contexts.”
Quote in the paperNot a substantial claimIt may state two separate claims as one.
Findingnone givenStudies introduce methods to enhance user engagement, deliberation, and other capacities to assess AI systems, but they barely measure how users engage with such interventions.
“For example, studies introduce methods to enhance user engagement, deliberation, and other capacities to assess AI systems; however, they barely measure how users engage with such interventions.”
Quote in the paperNot a substantial claimIt may state two separate claims as one.
FindingcitationSubjective metrics may only test users' perception of AI systems and do not validate their reliance.
“Subjective metrics may only test users' perception of AI systems and do not validate their reliance.”
Quote in the paperNot a substantial claimIt may state two separate claims as one.
FindingcitationStudies show limitations in generalizing results across domains, as most metrics are not standardized.
“Hence, studies show limitations in generalizing results across domains, as most metrics are not standardized [76].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [76]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim. The paper may give no reason why it holds.
Findingnone givenThe vast majority of studies have previously explored measuring trust.
“We focused on research in human-AI appropriate reliance measurement, where the vast majority of studies have previously explored measuring trust.”
Quote in the paperNot a substantial claimIt may state two separate claims as one.
Findingnone givenThere are clear gaps in the constructs explored in the literature, as well as in measurements used in application contexts.
“We found that there are clear gaps in the constructs explored in the literature, as well as in measurements used in application contexts.”
Quote in the paperSubstantial claimProbably not a substantial claim. The paper’s conclusions may not depend on it.
Findingown dataThe review included 22 studies.
“Total | 22”
Quote in the paperNot a substantial claimProbably not a substantial claim. The paper’s conclusions may not depend on it.
Findingown dataThe included studies use diverse explanation methods.
“Diverse explanation methods are used with a high focus on example-based and feature-based XAI.”
Quote in the paperNot a substantial claimProbably not a substantial claim. The paper’s conclusions may not depend on it.
Findingown dataMost included studies explore binary or multi-class decisions.
“Studies mostly explore binary or multi-class decisions.”
Quote in the paperNot a substantial claimProbably not a substantial claim.
Study design and conductown dataStudies were included only if they evaluated human-subject studies using quantitative methods and objective metrics to measure appropriate reliance, including over- and under-reliance.
“(3) Human-AI Appropriate Reliance Measurement with Objective Metrics: We performed a full-text screening on the remaining articles, removing articles that did not evaluate 1) human-subject studies using quantitative methods, and 2) objective metrics to measure appropriate reliance (including over-and under-reliance).”
Quote in the paperSubstantial claimProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
PremisecitationDominance can negatively mislead users to make more mistakes or over-rely on technology, resulting in detrimental reliance.
“However, the dominance can also negatively mislead users to make more mistakes, or over-rely on the technology (AI) [9], resulting in detrimental reliance.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [9]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
Study design and conductown dataSome studies included subjective measures along with objective metrics and were not excluded.
“Some studies also included subjective measures along with objective metrics, and those were not excluded.”
Quote in the paperNot a substantial claimProbably not a substantial claim. It may be too vague to check against evidence.
InterpretationreasoningReliance should be considered separately as a behavioral construct to understand how humans adopt AI advice and whether such adoption is appropriate.
“Therefore, it is important to consider reliance separately as a behavioral construct to understand how humans adopt AI advice in their decision-making and whether such adoption is appropriate, i.e., correct reliance on AI advice [9,86].”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [9]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [86]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim.
PremisecitationSchemmer et al. argue that metrics in the traditional view can have a major disadvantage of not knowing whether reliance on AI advice stems from correct discrimination or simply overlap of human and AI decisions.
“Schemmer et al. [73] argue that the metrics in the traditional view can have a major disadvantage of not knowing whether the reliance on AI advice stems from a correct discrimination of the AI decisions by humans or simply by an overlap of human and AI decisions.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [73]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
It may state two separate claims as one.
PremisecitationHumans often over- or under-rely on AI, leading to inferior performance on decision-making tasks.
“yet humans often over-or under-rely on AI, leading to inferior performance on decision-making tasks [4,28].”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [4]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [28]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
PremisecitationMorrison et al. expand the appropriateness view to include the dimension of incorrect explanations.
“Extensions of the discussed views are also explored, for instance, Morrison et al. [57] expand the appropriateness view from Schemmer et al. [73], including the dimension of incorrect explanations, i.e., where AI predictions can be correct, but their explanations can be incorrect or vice versa.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [57]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [73]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
It may state two separate claims as one.
InterpretationreasoningConcurrent approaches lack switching or a change of decision after taking advice, hence measuring fewer objective metrics.
“For instance, concurrent approaches lack switching or a change of decision after taking the advice, hence, measuring fewer objective metrics.”
Quote in the paperNot a substantial claimProbably not a substantial claim.
Study design and conductreasoningStudies were included only if they involved human subjects and human-AI decision-making contexts.
“(2) Human-AI Decision-Making: From the remaining articles, we screened for studies with human subjects and human-AI decision-making contexts.We used title and abstract screening to remove articles without human subjects, decision-making context, and those that only explored theoretical aspects.”
Quote in the paperSubstantial claimProbably not a substantial claim.
Study design and conductcitationThe review followed a systematic protocol (PRISMA) to identify and analyze literature.
“We followed a systematic protocol (i.e., the PRISMA [58] framework, as summarized in Figure 1) to identify and analyze literature, as adopted in prior studies [63,80].”
Quote in the paperSubstantial claim0 of 3 sources checked- cites [58]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [63]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [80]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
PremisecitationPeople weigh the potential benefits of cognitive effort against its perceived cost, a phenomenon referred to as cognitive-effort-discounting.
“In practice, people weigh the potential benefits of cognitive effort against its perceived cost [42], a phenomenon also referred to as cognitive-effort-discounting [90].”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [42]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [90]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
InterpretationreasoningExplanations of AI systems are only one of many components that impact reliance measurement.
“Besides, explanations of AI systems are only one of the many components that impact reliance measurement, as explained in the following sections.”
Quote in the paperNot a substantial claimProbably not a substantial claim. The paper’s conclusions may not depend on it.
Study design and conductcitationThe review protocol and coding are provided as an anonymized OSF repository for evaluation.
“The review protocol and coding are provided as an anonymized Open Science Framework (OSF) repository [2] for the reader's evaluation.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [2]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
It may state two separate claims as one.
PremisereasoningExplanations of AI systems may oversimplify or incorrectly portray underlying decisions, leading to under- or over-reliance on AI advice.
“For instance, explanations of AI systems may oversimplify or incorrectly portray the underlying decisions, leading to under-or over-reliance on the AI advice.”
Quote in the paperSubstantial claimProbably not a substantial claim. The paper’s conclusions may not depend on it.
PremisecitationThe traditional view focuses on the alignment between the perceived and actual performance of the system.
“This view focuses on the alignment between the perceived and actual performance of the system [54,92].”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [54]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [92]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
Study design and conductown dataSnowballing on relevant references of included studies found no new results matching the criteria.
“In another iteration, we looked at the relevant references of included studies to identify more papers using snowballing [93] (by reviewing their titles and abstracts); however, no new results matching the criteria were found.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [93]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
PremisereasoningThe dominance view aims to capture how exertion by technology can influence users' appropriate reliance and is often studied in high-stakes domains to understand beneficial and detrimental effects of AI advice and presentation biases.
“This view aims to capture how exertion by technology (AI) can influence users' appropriate reliance and is often studied in high-stakes domains to understand the beneficial and detrimental effects of using AI advice, as well as biases induced by the way AI advice is presented.”
Quote in the paperNot a substantial claimProbably not a substantial claim. The paper’s conclusions may not depend on it.
PremisecitationThe dominance view captures the dominance that technology exerts on users, which can be detrimental or beneficial in overall decision-making.
“Another important but slightly less applied view is defined by Cabitza et al. [9], called the dominance view.The dominance view captures the dominance that technology exerts on users, which can be detrimental or beneficial in overall decision-making [81].”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [9]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [81]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim. It may state two separate claims as one.
Study design and conductown dataThe first author coded the papers across dimensions of evaluation metrics, experimental designs, and intervention methods, and codes were collated to create groups and themes.
“The first author coded the papers across dimensions of evaluation metrics, experimental designs, and intervention methods used in studies.Afterwards, codes were collated to create groups and themes to report analysis and discussions.”
Quote in the paperSubstantial claim- Rests on an assumption: The single-author coding of papers into dimensions and themes was reliable.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
Study design and conductown dataThe inclusion and exclusion criteria were applied sequentially and discussed for consensus among the authors.
“We applied the above-mentioned criteria sequentially and discussed for consensus among ourselves.”
Quote in the paperNot a substantial claimProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
PremisereasoningThe dominating influence of AI can make users take a more subservient role and defer their decision-making to the technology.
“The dominating influence of AI can make users take a more subservient role (leading to confirmation bias or automation complacency), and defer their decision-making to the technology (AI).”
Quote in the paperNot a substantial claimProbably not a substantial claim. It may state two separate claims as one.
Study design and conductnone givenStudies were included only if they were peer-reviewed full conference or journal papers, excluding short/workshop papers, books/chapters, non-reviewed work, and secondary studies.
“(1) Peer-Reviewed: We only included full conference and journal papers.Short and workshop papers, books/chapters, and non-reviewed work (e.g., from arXiv) were excluded.In addition, secondary studies such as surveys and literature reviews were also excluded.”
Quote in the paperSubstantial claimProbably not a substantial claim. It may be too vague to check against evidence.
Interpretationnone givenInvestigating users' actual behavior with AI systems instead of only capturing their trust or agreement with AI is essential to test systems' appropriate use in realistic contexts.
“Investigating users' actual behavior with AI systems instead of only capturing their trust or agreement with AI is essential to test systems' appropriate use in realistic contexts.”
Quote in the paperNot a substantial claimProbably not a substantial claim. The paper’s conclusions may not depend on it.
PremisecitationPartial explanations align with a larger body of research on deliberation that encourages users to reflect on their decisions.
“Partial explanations align with a larger body of research on deliberation that encourages users to reflect on their decisions [50].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [50]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim. It may be too vague to check against evidence.
InterpretationreasoningEstablishing definitions and terminologies for appropriate reliance is integral.
“Establishing definitions and terminologies for appropriate reliance is integral.”
Quote in the paperNot a substantial claimIt may state two separate claims as one.
InterpretationcitationCapturing other demographics such as education and literacy, and more importantly domain expertise, are central to defining and evaluating accurate metrics for human-AI appropriate reliance.
“Capturing other demographics (e.g., education, literacy) [34], and more importantly, domain expertise are central to defining and evaluating accurate metrics for human-AI appropriate reliance.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [34]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim. It may state two separate claims as one.
Interpretationnone givenAssessing users' expertise and the difficulty of the task are important.
“In particular, assessing users' expertise (i.e., an estimation of their skills) and the difficulty of the task are important.”
Quote in the paperSubstantial claimProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
InterpretationcitationAssessing appropriate reliance is also important in situations where users are given the agency to contest and adjust an AI advice.
“Assessing appropriate reliance is also important in situations where users are given the agency to contest and adjust an AI advice [22].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [22]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
It may state two separate claims as one.
PremisecitationRunning complex tasks with crowd workers may not be entirely representative and therefore may not report actual measurement of appropriate reliance on AI advice.
“Existing work [24,65] also highlights that running complex tasks with crowd workers may not be entirely representative and therefore may not report actual measurement of appropriate reliance on AI advice.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [24]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [65]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
InterpretationcitationOptimal measurements may come from studies that apply metrics in realistic evaluations instead of one-off experiments with crowd workers and proxy tasks.
“Optimal measurements may come from studies that apply metrics in realistic evaluations instead of one-off experiments with crowd workers and proxy tasks [23].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [23]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
PremisecitationActive solicitation can influence reliance, for instance, by getting a second opinion to form a consensus.
“Lu et al. [49] pointed out that active solicitation can also influence reliance, for instance, by getting a second opinion to form a consensus.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [49]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
PremisecitationResearch suggests that when the task is difficult, the design should help users make better decisions through reflections.
“Research suggests that when the task is difficult, the design should help users make better decisions through reflections [38,77].”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [38]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [77]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim. It may be too vague to check against evidence.
InterpretationcitationDespite ensuring crowd workers can execute tasks, it is useful to employ reliance measurements with domain users for decision-making tasks.
“Despite ensuring crowd workers can execute tasks [23], it is also useful to employ reliance measurements with domain users for decision-making tasks.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [23]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
PremisecitationAI support often fails to address the knowledge gaps that users seek to fill.
“Buçinca et al. [8] highlighted that measuring users' competence and skill can also inform reliance behavior, as AI support often fails to address the knowledge gaps that users seek to fill [8].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [8]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim. It may be too vague to check against evidence.
PremisecitationThere is a need for evaluations of metrics for measuring trust and reliance distinctively.
“As highlighted in recent literature [9,26], there is a need for evaluations of metrics for measuring trust and reliance distinctively.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [9]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [26]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim. It may be too vague to check against evidence.
InterpretationreasoningEqual consideration of both over-reliance and under-reliance constructs is important for providing insights from both perspectives in user studies.
“Hence, equal consideration of both constructs is important for providing insights from both perspectives (over-and under-reliance) in the user studies.”
Quote in the paperNot a substantial claimProbably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.
InterpretationreasoningSuch a distinction will help clarify the need for more refined and standardized metrics for appropriate, over-, and under-reliance, as well as consolidate the research agenda in this domain.
“Such a distinction will help clarify the need for more refined and standardized metrics for appropriate, over-, and under-reliance, as well as consolidate the research agenda in this domain.”
Quote in the paperNot a substantial claimProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. It may state two separate claims as one.
InterpretationcitationInvestigating reliance in setups where AI systems can be manipulated can induce more insights and elicit additional metrics for measurement.
“Real AI systems may have the flexibility to adapt AI advice [24], and investigating reliance in setups where AI systems can be manipulated can induce more insights and elicit additional metrics for measurement.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [24]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim. It may be too vague to check against evidence.
InterpretationreasoningFor an accurate assessment of metrics, the system implementations should also be uniform across studies.
“Therefore, for an accurate assessment of metrics, the system implementations should also be uniform across studies.”
Quote in the paperNot a substantial claimProbably not a substantial claim. The paper’s conclusions may not depend on it.
InterpretationcitationMeasuring users' interaction with partial explanations can also be useful.
“Similarly, measuring users' interaction with partial explanations can also be useful [49,50].”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [49]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [50]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim.
PremisecitationOver-reliance on AI can lead users to lose their skills in the tasks.
“Over-reliance on AI can also lead users to lose their skills in the tasks, which should also be assessed by incorporating metrics for users' deskilling [8] and agency loss [63], which are rarely used in studies.”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [8]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [63]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim. It may state two separate claims as one.
InterpretationcitationVarying task complexity and users' expertise can elicit different reliance behavior, which needs to be measured across domains to validate metrics.
“Varying task complexity and users' expertise can elicit different reliance behavior, which needs to be measured across domains to validate metrics [68].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [68]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
It may state two separate claims as one.
InterpretationcitationGenerative AI outputs often do not have a fixed ground truth and may require different metrics, which are highly underexplored in research.
“For instance, generative AI outputs often do not have a fixed ground truth and may require different metrics, which are highly underexplored in research [37].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [37]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim.
InterpretationcitationConceptually, it is important to distinguish between metrics for appropriate reliance, such as over-reliance, under-reliance, and relative reliance, from the existing metrics of accuracy, agreement, or switching.
“Conceptually, it is important to distinguish between metrics for appropriate reliance, such as over-reliance, under-reliance, and relative reliance, from the existing metrics of accuracy, agreement, or switching, among others [26,43].”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [26]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [43]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim.
InterpretationreasoningUsers' reliance may change while interacting with an AI system after a few iterations.
“For instance, users' reliance may change while interacting with an AI system after a few iterations.”
Quote in the paperSubstantial claimProbably not a substantial claim. The paper’s conclusions may not depend on it.
PremisecitationReal AI systems may have the flexibility to adapt AI advice.
“Real AI systems may have the flexibility to adapt AI advice [24], and investigating reliance in setups where AI systems can be manipulated can induce more insights and elicit additional metrics for measurement.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [24]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.
InterpretationreasoningMetrics for reliance and appropriate reliance have a fine distinction, which needs to be more explicit in measurements and related constructs.
“We further argue that metrics for reliance and appropriate reliance also have a fine distinction, which needs to be more explicit in measurements and related constructs.”
Quote in the paperNot a substantial claimProbably not a substantial claim. It may be too vague to check against evidence.
InterpretationreasoningThis consolidation can ensure that future empirical studies use standardized and validated metrics for measurement.
“This consolidation can ensure that future empirical studies use standardized and validated metrics for measurement.”
Quote in the paperNot a substantial claimProbably not a substantial claim. The paper’s conclusions may not depend on it.
PremisecitationEnhancing users' skills with AI systems is important for improving engagement and human decision-making.
“They [8] showed that enhancing users' skills with AI systems is important for improving engagement and human decision-making.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [8]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim. It may be too vague to check against evidence.
InterpretationreasoningThere is a need to unify the traditional, appropriateness, and dominance views as frameworks.
“Although these views focus on objective metrics for measuring appropriate reliance on AI, there is a need to unify them as frameworks.”
Quote in the paperNot a substantial claimIt may state two separate claims as one.
PremisecitationSome tasks can be assigned to rely on AI advice easily (easy or low-stakes), while others may require more consideration.
“Some tasks can be assigned to rely on AI advice easily (easy or low-stakes), while others may require more consideration [82,83].”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [82]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [83]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
InterpretationcitationMetrics for appropriate reliance can also be aligned to measure user engagement with AI.
“Metrics for appropriate reliance can also be aligned to measure user engagement with AI [17,55].”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [17]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [55]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
Study design and conductnone givenResults may be affected by screening bias and subjective interpretation during the application of inclusion and exclusion criteria.
“In addition, despite careful evaluation, we acknowledge that results may be affected by screening bias and subjective interpretation during the application of inclusion and exclusion criteria.”
Quote in the paperNot a substantial claimProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
Interpretationnone givenDue to over-reliance, users may not be able to assess the system's incorrect advice when the system underperforms.
“However, due to over-reliance, users may not be able to assess the system's incorrect advice when the system underperforms.”
Quote in the paperNot a substantial claimProbably not a substantial claim. The paper’s conclusions may not depend on it.
Interpretationnone givenGenerative AI outputs can have varying levels of correctness and value for different user groups.
“Such outputs can have varying levels of correctness and value for different user groups, providing several opportunities for future research to explore.”
Quote in the paperNot a substantial claimProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. It may state two separate claims as one.
PremisecitationEngaging users with AI can help them contest, take ownership of their decisions, and establish critical engagement or mindful usage.
“Engaging users with AI can help them contest, take ownership of their decisions, and establish critical engagement or mindful usage [35].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [35]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
InterpretationreasoningThis focus may have resulted in eliminating studies that use appropriate trust interchangeably with appropriate reliance.
“This focus may have resulted in eliminating studies that use appropriate trust interchangeably with appropriate reliance.”
Quote in the paperNot a substantial claimProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
InterpretationcitationAssessing users' domain expertise and self-confidence about the task is essential for eliciting a realistic behavior.
“Assessing users' domain expertise and self-confidence about the task is essential for eliciting a realistic behavior [51].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [51]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
InterpretationcitationReliance on AI may also depend on situational/contextual factors, cognitive traits, or individuality for more realistic evaluations.
“Reliance on AI may also depend on situational/contextual factors, cognitive traits, or individuality [20,82,83] for more realistic evaluations.”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites [20]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [82]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [83]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
InterpretationcitationMeasurements can enable capturing users' reliance on AI with superior performance to themselves, and self-reliance when AI underperforms.
“For instance, measurements can enable capturing users' reliance on AI with superior performance to themselves, and self-reliance when AI underperforms [52].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [52]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
InterpretationreasoningMeasuring how users take ownership of decisions while building appropriate reliance can inform more about their ability to evaluate AI.
“Measuring how users take ownership of decisions while building appropriate reliance can inform more about their ability to evaluate AI.”
Quote in the paperNot a substantial claimProbably not a substantial claim. The authors may not put it forward as true. It may be too vague to check against evidence. It may state two separate claims as one.
InterpretationcitationResearch needs to explore metrics for appropriate reliance with generative AI, as the mode of adopting AI advice transitions from binary to open choice.
“Research also needs to explore metrics for appropriate reliance with generative AI [20,41], as the mode of adopting AI advice transitions from binary to open choice [37].”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites [20]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [41]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [37]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim. It may be too vague to check against evidence.
InterpretationcitationUnderstanding and modeling why inappropriate reliance happens can inform more realistic assessment metrics for appropriate reliance.
“Understanding and modeling why inappropriate reliance happens (i.e., due to poor self-assessment, literacy, bias, or task stakes) can inform more realistic assessment metrics for appropriate reliance [29].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [29]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim. It may be too vague to check against evidence.
InterpretationreasoningSuch understanding can inform intervention design to facilitate and measure appropriate reliance effectively.
“Such understanding can inform intervention design to facilitate and measure appropriate reliance effectively.”
Quote in the paperNot a substantial claimProbably not a substantial claim. The paper’s conclusions may not depend on it.
InterpretationreasoningSome work on human-AI reliance may exist outside the covered databases.
“While our search was comprehensive, some work on human-AI reliance may exist outside the covered databases.”
Quote in the paperNot a substantial claim
The paper relies on this without support.
The 22 papers selected through the PRISMA protocol are representative of the empirical human-AI appropriate reliance literature.
“After applying the criteria, we selected 22 papers that employed objective metrics of appropriate reliance as per the evaluation.”
The paper relies on this without support.
The search keywords and databases (SCOPUS and ACM DL) cover the relevant literature on human-AI appropriate reliance.
“First, we searched the ACM Digital Library (DL) with keywords on”
The paper relies on this without support.
The single-author coding of papers into dimensions and themes was reliable.
“The first author coded the papers across dimensions of evaluation metrics, experimental designs, and intervention methods used in studies.Afterwards, codes were collated to create groups and themes to report analysis and discussions.”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Switch-fraction study count differs from Table 4
MinorThe text reports switch fraction in eight studies, but Table 4 lists seven study references for that measure: [31-34], [50], [52], and [69].
“switch fraction (8 studies) [31]”
“Switch Fraction [31-34, 50, 52, 69]”
Citations
Cites
The PRISMA 2020 statement: An updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · BMJ · doi:10.1136/bmj.n71 · Cited by 108,283
Literature review as a research methodology: An overview and guidelines
Snyder, H. (2019) · Journal of Business Research · doi:10.1016/j.jbusres.2019.07.039 · Cited by 11,524
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605
Wohlin, C. (2014) · Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering · doi:10.1145/2601248.2601268 · Cited by 4,234
Towards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121
Trust in Automation: Integrating Empirical Evidence on Factors That Influence Trust
Hoff, K. A., & Bashir, M. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814547570 · Cited by 2,817
Algorithm appreciation: People prefer algorithmic to human judgment
Logg, J. M., Minson, J. A., & Moore, D. A. (2019) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2018.12.005 · Cited by 2,061
Ali, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser, J., Díaz-Rodríguez, N., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101805 · Cited by 1,762
Complacency and Bias in Human Use of Automation: An Attentional Integration
Parasuraman, R., & Manzey, D. H. (2010) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720810376055 · Cited by 1,710
The role of trust in automation reliance
Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G., & Beck, H. P. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00038-7 · Cited by 1,276
Buçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021) · arXiv · doi:10.48550/arxiv.2102.09692 · Cited by 1,124
Trust in AI and Its Role in the Acceptance of AI Technologies
Choung, H., David, P., & Ross, A. (2023) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2022.2050543 · Cited by 963
Dellermann, D., Ebel, P., Söllner, M., & Leimeister, J. M. (2019) · Business & Information Systems Engineering · doi:10.1007/s12599-019-00595-2 · Cited by 826
Dahlbäck, N., Jönsson, A., & Ahrenberg, L. (1993) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/169891.169968 · Cited by 784
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445717 · Cited by 757
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. S. (2020) · arXiv · doi:10.48550/arxiv.2006.14779 · Cited by 754
Longo, L., Brcic, M., Cabitza, F., Choi, J., Confalonieri, R., Ser, J. D., Guidotti, R., Hayashi, Y., Herrera, F., Holzinger, A., et al. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102301 · Cited by 626
Technology, Humanness, and Trust: Rethinking Trust in Technology
Marshall University, Lankton, N., McKnight, D. H., Michigan State University, Tripp, J., & Baylor University (2015) · Journal of the Association for Information Systems · doi:10.17705/1jais.00411 · Cited by 553
Explanations Can Reduce Overreliance on AI Systems During Decision-Making
Vasconcelos, H., Jörke, M., Grunde-McLaughlin, M., Gerstenberg, T., Bernstein, M. S., & Krishna, R. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579605 · Cited by 402
The Principles and Limits of Algorithm-in-the-Loop Decision Making
Green, B., & Chen, Y. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359152 · Cited by 392
A Review of User Interface Design for Interactive Machine Learning
Dudley, J. J., & Kristensson, P. O. (2018) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3185517 · Cited by 343
How to Evaluate Trust in AI-Assisted Decision Making? A Survey of Empirical Methodologies
Vereschak, O., Bailly, G., & Caramiaux, B. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3476068 · Cited by 229
Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations
Chen, V., Liao, Q. V., Wortman Vaughan, J., & Bansal, G. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610219 · Cited by 220
Mehrotra, S., Degachi, C., Vereschak, O., Jonker, C. M., & Tielman, M. L. (2024) · ACM Journal on Responsible Computing · doi:10.1145/3696449 · Cited by 219
Deciding Fast and Slow: The Role of Cognitive Biases in AI-assisted Decision-making
Rastogi, C., Zhang, Y., Wei, D., Varshney, K. R., Dhurandhar, A., & Tomsett, R. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3512930 · Cited by 216
Appropriate Reliance on AI Advice: Conceptualization and the Effect of Explanations
Schemmer, M., Kuehl, N., Benz, C., Bartos, A., & Satzger, G. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584066 · Cited by 208
Lai, V., Chen, C., Smith-Renner, A., Liao, Q. V., & Tan, C. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594087 · Cited by 204
How do visual explanations foster end users' appropriate trust in machine learning?
Yang, F., Huang, Z., Scholtz, J., & Arendt, D. L. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377480 · Cited by 200
From explainable to interactive AI: A literature review on current trends in human-AI interaction
Raees, M., Meijerink, I., Lykourentzou, I., Khan, V.-J., & Papangelis, K. (2024) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2024.103301 · Cited by 180
Self-citationMiller, T. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594001 · Cited by 172
How and why humans trust: A meta-analysis and elaborated model
Hancock, P. A., Kessler, T. T., Kaplan, A. D., Stowers, K. L., Brill, J. C., Billings, D. R., Schaefer, K. E., & Szalma, J. L. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1081086 · Cited by 168
Kim, S. S. Y., Liao, Q. V., Vorvoreanu, M., Ballard, S., & Vaughan, J. W. (2024) · The 2024 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3630106.3658941 · Cited by 147
Wischnewski, M., Krämer, N., & Müller, E. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581197 · Cited by 146
Ma, S., Lei, Y., Wang, X., Zheng, C., Shi, C., Yin, M., & Ma, X. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581058 · Cited by 143
How Cognitive Biases Affect XAI-assisted Decision-making: A Systematic Review
Bertrand, A., Belloum, R., Eagan, J. R., & Maxwell, W. (2022) · Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3514094.3534164 · Cited by 129
Knowing About Knowing: An Illusion of Human Competence Can Hinder Appropriate Reliance on AI Systems
He, G., Kuiper, L., & Gadiraju, U. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581025 · Cited by 107
Explanations, Fairness, and Appropriate Reliance in Human-AI Decision-Making
Schoeffer, J., De‐Arteaga, M., & Kühl, N. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642621 · Cited by 105
Ma, S., Wang, X., Lei, Y., Shi, C., Yin, M., & Ma, X. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642671 · Cited by 92
Is Conversational XAI All You Need? Human-AI Decision Making With a Conversational XAI Assistant
He, G., Aishwarya, N., & Gadiraju, U. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712133 · Cited by 87
The Impact of Imperfect XAI on Human-AI Decision-Making
Morrison, K., Spitzer, P., Turri, V., Feng, M., Kühl, N., & Perer, A. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3641022 · Cited by 86
Chiang, C.-W., Lu, Z., Li, Z., & Yin, M. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581015 · Cited by 72
Buçinca, Z., Swaroop, S., Paluch, A. E., Doshi‐Velez, F., & Gajos, K. Z. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713229 · Cited by 62
Exploring the effects of human-centered AI explanations on trust and reliance
Scharowski, N., Perrig, S. A. C., Svab, M., Opwis, K., & Brühlmann, F. (2023) · Frontiers in Computer Science · doi:10.3389/fcomp.2023.1151150 · Cited by 57
He, G., Buijsman, S., & Gadiraju, U. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610067 · Cited by 56
Does More Advice Help? The Effects of Second Opinions in AI-Assisted Decision Making
Lu, Z., Wang, D., & Yin, M. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3653708 · Cited by 53
Cabitza, F., Campagner, A., Angius, R., Natali, C., & Reverberi, C. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581095 · Cited by 51
Accuracy-Time Tradeoffs in AI-Assisted Decision Making under Time Pressure
Swaroop, S., Buçinca, Z., Gajos, K. Z., & Doshi-Velez, F. (2024) · Proceedings of the 29th International Conference on Intelligent User Interfaces · doi:10.1145/3640543.3645206 · Cited by 49
Kahr, P. K., Rooks, G., Willemsen, M. C., & Snijders, C. C. P. (2024) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3686164 · Cited by 49
Ma, S., Chen, Q., Wang, X., Zheng, C., Peng, Z., Yin, M., & Ma, X. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713423 · Cited by 48
Effects of AI and Logic-Style Explanations on Users’ Decisions Under Different Levels of Uncertainty
Cau, F. M., Hauptmann, H., Spano, L. D., & Tintarev, N. (2023) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3588320 · Cited by 44
A Missing Piece in the Puzzle: Considering the Role of Task Complexity in Human-AI Decision Making
Salimzadeh, S., He, G., & Gadiraju, U. (2023) · Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization · doi:10.1145/3565472.3592959 · Cited by 42
Effect of Explanation Conceptualisations on Trust in AI-assisted Credibility Assessment
Pareek, S., van Berkel, N., Velloso, E., & Goncalves, J. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3686922 · Cited by 42
Salimzadeh, S., He, G., & Gadiraju, U. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641905 · Cited by 41
Eckhardt, S., Kühl, N., Dolata, M., & Schwabe, G. (2025) · ACM Computing Surveys · doi:10.1145/3776528 · Cited by 38
Evaluating the impact of human explanation strategies on human-AI visual decision-making
Morrison, K., Shin, D., Holstein, K., & Perer, A. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579481 · Cited by 37
AI Reliance and Decision Quality: Fundamentals, Interdependence, and the Effects of Interventions
Schoeffer, J., Jakubik, J., Vössing, M., Kühl, N., & Satzger, G. (2025) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.15873 · Cited by 36
de Jong, S., Paananen, V., Tag, B., & van Berkel, N. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710946 · Cited by 32
A Decision Theoretic Framework for Measuring AI Reliance
Guo, Z., Wu, Y., Hartline, J. D., & Hullman, J. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658901 · Cited by 27
To Err Is AI! Debugging as an Intervention to Facilitate Appropriate Reliance on AI Systems
He, G., Bharos, A., & Gadiraju, U. (2024) · ACM Conference on Hypertext and Social Media (HT) · doi:10.1145/3648188.3675130 · Cited by 21
Reflection machines: increasing meaningful human control over Decision Support Systems
Cornelissen, N. A. J., van Eerdt, R. J. M., Schraffenberger, H., & Haselager, P. (2022) · Ethics and Information Technology · doi:10.1007/s10676-022-09645-y · Cited by 20
Personalising AI Assistance Based on Overreliance Rate in AI-Assisted Decision Making
Swaroop, S., Buçinca, Z., Gajos, K. Z., & Doshi-Velez, F. (2025) · Proceedings of the 30th International Conference on Intelligent User Interfaces · doi:10.1145/3708359.3712128 · Cited by 20
Trust and Reliance in XAI -- Distinguishing Between Attitudinal and Behavioral Measures
Scharowski, N., Perrig, S. A. C., von Felten, N., & Brühlmann, F. (2022) · arXiv · doi:10.48550/arxiv.2203.12318 · Cited by 18
Reliance and Automation for Human-AI Collaborative Data Labeling Conflict Resolution
Brachman, M., Ashktorab, Z., Desmond, M. J., Duesterwald, E., Dugan, C., Joshi, N. N., Pan, Q., & Sharma, A. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555212 · Cited by 18
The Influence of Curiosity Traits and On-Demand Explanations in AI-Assisted Decision-Making
Cau, F. M., & Spano, L. D. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712165 · Cited by 13
Trust to Reliance: Measurement Constructs for Human-AI Appropriate Reliance
Raees, M., & Papagelis, K. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798835 · Cited by 8
Self-citationRaees, M., Khan, V.-J., Lykourentzou, I., & Papangelis, K. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791467 · Cited by 5
Self-citationTowards Understanding Human-AI Reliance Patterns Through Explanation Styles
Casolin, E. R., & Salim, F. D. (2024) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3675094.3678996 · Cited by 2
Flagged first, then by citations
Research works printed but not linked: 25
- Annette Baier. 1986. Trust and antitrust. ethics 96, 2 (1986), 231-260.
- Sam Corbett-Davies, Emma Pierson, Avi Feller, Sharad Goel, and Aziz Huq. 2017. Algorithmic decision making and the cost of fairness. In Proceedings of the 23rd acm sigkdd international conference on knowledge discovery and data mining. 797-806.
- Devleena Das and Sonia Chernova. 2020. Leveraging rationales to improve human task performance. In Proceedings of the 25th international conference on intelligent user interfaces. 510-518.
- Berkeley J Dietvorst, Joseph P Simmons, and Cade Massey. 2015. Algorithm aversion: people erroneously avoid algorithms after seeing them err. Journal of experimental psychology: General 144, 1 (2015), 114.
- Susanne Gaube, Harini Suresh, Martina Raue, Alexander Merritt, Seth J Berkowitz, Eva Lermer, Joseph F Coughlin, John V Guttag, Errol Colak, and Marzyeh Ghassemi. 2021. Do as AI say: susceptibility in deployment of clinical decision-aids. NPJ digital medicine 4, 1 (2021), 31.
- Maren Hinrichs, Thi Bich Diep Bui, and Stefan Schneegass. 2024. Exploring the Effects of User Input and Decision Criteria Control on Trust in a Decision Support Tool for Spare Parts Inventory Management. In Proceedings of the International Conference on Mobile and Ubiquitous Multimedia. 313-323.
- Angel Hsing-Chi Hwang, Q Vera Liao, Su Lin Blodgett, Alexandra Olteanu, and Adam Trischler. 2025. 'It was 80% me, 20% AI': Seeking Authenticity in Co-Writing with Large Language Models. Proceedings of the ACM on Human-Computer Interaction 9, 2 (2025), 1-41.
- Daniel Kahneman and Gary Klein. 2009. Conditions for intuitive expertise: a failure to disagree. American psychologist 64, 6 (2009), 515.
- Wouter Kool and Matthew Botvinick. 2018. Mental labour. Nature human behaviour 2, 12 (2018), 899-908.
- Benedikt Leichtmann, Christina Humer, Andreas Hinterreiter, Marc Streit, and Martina Mara. 2023. Effects of Explainable Artificial Intelligence on trust and human behavior in a high-risk decision task. Computers in Human Behavior 139 (2023), 107539.
- Scott Mayer McKinney, Marcin Sieniek, Varun Godbole, Jonathan Godwin, Natasha Antropova, Hutan Ashrafian, Trevor Back, Mary Chesus, Greg S Corrado, Ara Darzi, et al. 2020. International evaluation of an AI system for breast cancer screening. Nature 577, 7788 (2020), 89-94.
- Alison Parkes. 2017. The effect of individual and task characteristics on decision aid reliance. Behaviour & Information Technology 36, 2 (2017), 165-177.
- Gonzalo Ramos, Christopher Meek, Patrice Simard, Jina Suh, and Soroush Ghorashi. 2020. Interactive machine teaching: a human-centered approach to building machine-learned models. Human-Computer Interaction 35, 5-6 (2020), 413-451.
- Giuseppe Romeo and Daniela Conti. 2026. Exploring automation bias in human-AI collaboration: a review and implications for explainable AI. AI & SOCIETY 41, 1 (2026), 259-278.
- Max Schemmer, Andrea Bartos, Philipp Spitzer, Patrick Hemmer, Niklas Kühl, Jonas Liebschner, and Gerhard Satzger. 2023. Towards effective human-AI decision-making: The role of human learning in appropriate reliance on AI advice. In Forty-Fourth International Conference on Information Systems (Hyderabad, India) (ICIS 2023 Proceedings). AISEL, 1-17. https://aisel.aisnet.org/icis2023/hti/hti/14
- Nadine Schlicker and Markus Langer. 2021. Towards warranted trust: A model on the relation between actual and perceived system trustworthiness. In Proceedings of mensch und computer 2021. 325-329.
- Phoebe Sengers, Kirsten Boehner, Shay David, and Joseph'Jofish' Kaye. 2005. Reflective design. In Proceedings of the 4th decennial conference on Critical computing: between sense and sensibility. 49-58.
- Keng Siau and Weiyu Wang. 2018. Building trust in artificial intelligence, machine learning, and robotics. Cutter business technology journal 31, 2 (2018), 47-53.
- Janet A Sniezek and Timothy Buckley. 1995. Cueing and cognitive conflict in judge-advisor decision making. Organizational behavior and human decision processes 62, 2 (1995), 159-174.
- Amos Tversky and Daniel Kahneman. 1981. The framing of decisions and the psychology of choice. science 211, 4481 (1981), 453-458.
- Amos Tversky and Daniel Kahneman. 1989. Rational choice and the framing of decisions. In Multiple criteria decision making and risk analysis using microcomputers. Springer, 81-126.
- Tiffany C Veinot, Hannah Mitchell, and Jessica S Ancker. 2018. Good intentions are not enough: how informatics interventions can worsen inequality. Journal of the American Medical Informatics Association 25, 8 (2018), 1080-1088.
- Xinru Wang and Ming Yin. 2023. Watch out for updates: Understanding the effects of model explanation updates in ai-assisted decision making. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1-19.
- Andrew Westbrook, Daria Kester, and Todd S Braver. 2013. What is the subjective cost of cognitive effort? Load, trait, and aging effects revealed by economic preference. PloS one 8, 7 (2013), e68210.
- Earl L Wiener. 1981. Complacency: Is the term useful for air safety. In Proceedings of the 26th corporate aviation safety seminar, Vol. 117. 116-125.
Cited by
Sorted by citations
Farrag, S. E., & Saber, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.15678 · Cited by 1
Impact of Expertise and Trust on AI-assisted WorkPreprint
Ma, Q., Khape, P., Pang, R., Koedinger, K., Wu, T., Suri, S., & Iqbal, S. T. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.14660 · Cited by 1
The Belief Update Gate: Separating Inertia from Learning in Human–AI Interaction
Biswas, S., Erlei, A., & Gadiraju, U. (2026) · ACM Conference on Human-AI Complementarity and Alignment · doi:10.1145/3834580.3838740
Zhang, Z., Li, S., Chen, J., Wu, J., Liu, X., Yao, N., Pang, S., Zou, B., Sun, Y., Zhang, Y., et al. (2026) · National Science Review · doi:10.1093/nsr/nwag688
Lecce, M., Falchi, G., & Cesarotti, V. (2026) · Preprints.org · doi:10.20944/preprints202611.0172.v1
Metadata
- Authors
- Raees, M., & Papangelis, K.
- Affiliations
- Rochester Institute of Technology (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2026
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2604.23896
- OpenAlex
- W7156795363
- Cited by
- 5
- References
- 66
- Citations by year
- 2026: 5
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- human-AI decision-making, trust in AI, measurement constructs, objective metrics, AI advice, trust measurement, measurement validity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
