Appropriate Reliance on AI Advice: Conceptualization and the Effect of Explanations
Open accessThe DOI matches its Crossref record; 83 of the 83 references it cites were found in a registry or an index; 23 of the 39 sources its claims cite were checked for support.
Abstract
AI advice is becoming increasingly popular, e.g., in investment and medical treatment decisions. As this advice is typically imperfect, decision-makers have to exert discretion as to whether actually follow that advice: they have to “appropriately” rely on correct and turn down incorrect advice. However, current research on appropriate reliance still lacks a common definition as well as an operational measurement concept. Additionally, no in-depth behavioral experiments have been conducted that help understand the factors influencing this behavior. In this paper, we propose Appropriateness of Reliance (AoR) as an underlying, quantifiable two-dimensional measurement concept. We develop a research model that analyzes the effect of providing explanations for AI advice. In an experiment with 200 participants, we demonstrate how these explanations influence the AoR, and, thus, the effectiveness of AI advice. Our work contributes fundamental concepts for the analysis of reliance behavior and the purposeful design of AI advisors.
Overview
CoChat Summary
TL;DR
The paper proposes Appropriateness of Reliance (AoR) as a two-dimensional metric—relative AI reliance (RAIR) and relative self-reliance (RSR)—for measuring whether decision-makers appropriately follow correct AI advice and reject incorrect AI advice. In a 200-participant hotel-review experiment, feature-importance explanations significantly increased RAIR but not RSR, and the human-AI team did not reach complementary team performance, so appropriate reliance was not displayed.
Key findings
- AoR is defined as the tuple of RAIR and RSR; appropriate reliance (AR) is defined as any AoR tuple that leads to complementary team performance (CTP).
- In the control condition, RAIR was 29.59% and RSR was 71.87%; with feature-importance explanations, RAIR rose significantly to 38.87% (t = -1.95, p = 0.05), while RSR did not change significantly (69.45%; t = 0.61, p = 0.54).
- Explanations significantly increased the change in self-confidence (t = -2.33, p = 0.02), which partially mediated the effect on RAIR; explanations had no significant effect on trust (H4 not supported), but trust increased RAIR and decreased RSR (H5a and H5b supported).
- CTP was not reached: down-sampled AI performance was 50% in both conditions; human accuracy was 55.50% (control) and 53.85% (feature importance), and AI-assisted accuracy was 53.94% (control) and 56.30% (feature importance), with human-AI team performance not significantly different from human accuracy.
Methods
- Between-subject online experiment with 200 participants recruited via Prolific; one participant in the feature-importance condition was excluded for a failed manipulation check, and SEM analyses used a final sample of 199.
- Participants classified 16 deceptive or genuine hotel reviews sequentially: first without AI advice and with a confidence rating, then after receiving AI advice (SVM with 86% accuracy) either alone or with LIME feature-importance explanations, then revised their decision and confidence.
- Analysis used descriptive t-tests and structural equation modeling (SEM) with the MLR estimator in lavaan, measuring RAIR, RSR, trust, and change in self-confidence.
Limitations
- The paper states that deception detection is difficult for humans, that generalizability is limited by the choice of explanations, that the concept is limited to classification tasks, and that the sequential task setup may change the task, induce anchoring, reduce over-reliance, and may not always be possible or desired in real-world settings; crowd workers also limit generalizability.
- Evident from the text: the sample was relatively young (mean age 27.5, SD 8.5) and drawn from an online platform, and the experimental AI’s down-sampled performance was 50% in both conditions, which may constrain conclusions about CTP and appropriate reliance.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures




![Fig. 5 Online experiment graphical user interface for the feature importance condition. The ground truth of the exemplarily shown hotel review is 'fake'. The design of the interface is adapted from Lai et al. [42].](https://figures.cochat.ai/paper-figures/aa69c3bc1d844572931891d4e00e8f4d/appropriate-reliance-on-ai-advice-conceptualization-and-the-effect-of-figure-5.png)


Verification
Claims the paper makes
17 Valid · 3 Not valid · 10 Not checked · 54 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“The sequential task setup could induce an anchoring effect which prevents the human to more actively take the AI into account [9].”
What the sources say
- Disputedcites [9] [9] Zana Buçinca, Maja Barbara Malaya, and Krzysztof Z Gajos. 2021. To trust or to think: cognitive forcing functions can reduce overreliance on AI in AI-assisted decision-making. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1 (2021), 1-21. - sourceExperimental — accuracy not yet measured
Section 3.2.2 (p. 7) describes a sequence in which people decide before seeing the AI and hypothesizes that it will motivate them to engage with the AI explanation, not prevent engagement. Section 2.2 (p. 4) instead associates anchoring with showing the AI recommendation before the person makes an independent decision. (LLM, conf=0.84)
Checked by a reasoning model — may be wrong.
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
Every source the paper cites for this claim was checked and supports it.
In the paper
“Understanding is crucial in building trust [23].”
What the sources say
- Supportedcites [23] [23] Leilani H Gilpin, Cecilia Testart, Nathaniel Fruchter, and Julius Adebayo. 2019. Explaining explanations to society. arXiv preprint arXiv:1901.06560 (2019). - source
The Discussion and Conclusion (p. 5) says that truly trusting AI systems requires people to feel confident they understand how decisions are made. It also says they need recourse, so understanding is crucial but not sufficient on its own. (LLM, conf=0.92)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Confidence in one's own decision is a key mechanism underlying advice acceptance [14, 73].”
What the sources say
- Supportedcites [14] [14] Leah Chong, Guanglu Zhang, Kosa Goucher-Lambert, Kenneth Kotovsky, and Jonathan Cagan. 2022. Human confidence in artificial intelligence and in themselves: The evolution and impact of confidence on adoption of AI advice. Computers in Human Behavior 127 (2022), 107018. - source
The Discussion reports that self-confidence significantly contributes to whether people accept AI suggestions and describes it as a driving factor. The Conclusion likewise reports a significant impact of self-confidence on accepting or rejecting those suggestions. (LLM, conf=0.90)
- Supportedcites [73] [73] Xiuxin Wang and Xiufang Du. 2018. Why does advice discounting occur? The combined roles of confidence and trust. Frontiers in Psychology 9 (2018), 2381. - source
The Introduction calls confidence in one’s own judgment a key mechanism underlying advice taking. The General Discussion reports that advice discounting decreased when confidence decreased and that confidence helped predict advice taking. Page numbers are not provided in our copy. (LLM, conf=0.94)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“We would like to emphasize that deception detection is a difficult task for humans [42, 43].”
What the sources say
- Supportedcites [42] [42] Vivian Lai, Han Liu, and Chenhao Tan. 2020. " Why is' Chicago'deceptive?" Towards Building Model-Driven Tutorials for Humans. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems . 1-13. - source
The Deception detection section (p. 3) says it is well recognized that humans struggle to detect deception. The Discussion (p. 10) likewise describes deception detection as potentially too challenging for humans. (LLM, conf=0.94)
- Supportedcites [43] [43] Vivian Lai and Chenhao Tan. 2019. On human predictions with explanations and predictions of machine learning models: A case study on deception detection. In Proceedings of the conference on fairness, accountability, and transparency . 29-38. - source
The paper explicitly calls deception detection “a challenging task for humans” in both its introductory discussion (p. 2) and Related Work (p. 3). This directly supports the citing sentence’s paraphrase. (LLM, conf=0.95)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“While this may sound disappointing at first, it also shows that the claim that explanations would reduce overreliance [6, 9] does not seem to hold for all kinds of tasks.”
What the sources say
- Supportedcites [6] [6] Gagan Bansal, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, and Daniel Weld. 2021. Does the whole exceed its parts? the effect of ai explanations on complementary team performance. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems . 1-16. - source
The Introduction (p. 2) reports that explanations increased reliance on AI recommendations even when the AI was incorrect. Section 6.2 (p. 12) describes this as increased blind trust rather than appropriate reliance, supporting the qualified claim that explanations do not reduce overreliance in all tasks. (LLM, conf=0.93)
- Supportedcites [9] [9] Zana Buçinca, Maja Barbara Malaya, and Krzysztof Z Gajos. 2021. To trust or to think: cognitive forcing functions can reduce overreliance on AI in AI-assisted decision-making. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1 (2021), 1-21. - source
The Introduction (p. 2) says explanations were intended to reduce overreliance but have not had substantial success. The Discussion (p. 16) reports that, in this paper’s decision-making task, participants using simple explainable AI still followed incorrect suggestions. This supports the qualified claim that explanations do not reduce overreliance in all tasks. (LLM, conf=0.93)
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
29 more claims · 5 with their cited sources checked
Create a free account to see all 84 claims and to verify the 2 references still open.
- It may state two separate claims as one. 21
- Probably not a substantial claim. 33
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Research on human-AI decision making suggests that reliance is appropriate when people accept correct AI output and reject incorrect output; thus, the relevant outcome is not simply whether and how often students defer to AI, but how well their reliance tracks the quality of AI suggestions (Raees et al., 2026;Schemmer et al., 2023;Bansal et al., 2021).”
From Think Before You Accept: Can Written Justification Reduce Uncritical Uptake of AI Writing Suggestions? (2026) · cites this paper as Schemmer et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“Research on human-AI decision making suggests that reliance is appropriate when people accept correct AI output and reject incorrect output; thus, the relevant outcome is not simply whether and how often students defer to AI, but how well their reliance tracks the quality of AI suggestions (Raees et al., 2026;Schemmer et al., 2023;Bansal et al., 2021).”
From Think Before You Accept: Can Written Justification Reduce Uncritical Uptake of AI Writing Suggestions? (2026) · cites this paper as Schemmer et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“in contrast to prior work which has evaluated reliance in action space only [3, 28, 35].”
From A Decision Theoretic Framework for Measuring AI Reliance (2024) · cites this paper as [28]
Citations
Cites
Hu, L., & Bentler, P. M. (1999) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705519909540118 · Cited by 108,788
lavaan : An R Package for Structural Equation Modeling
Rosseel, Y. (2012) · Journal of Statistical Software · doi:10.18637/jss.v048.i02 · Cited by 27,955
Comparative fit indexes in structural models.
Bentler, P. M. (1990) · Psychological Bulletin · doi:10.1037/0033-2909.107.2.238 · Cited by 25,266
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
The future of employment: How susceptible are jobs to computerisation?
Frey, C. B., & Osborne, M. A. (2017) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2016.08.019 · Cited by 8,727
Trust and TAM in Online Shopping: An Integrated Model1
Gefen, D., Karahanna, E., & Straub, D. W. (2003) · MIS Quarterly · doi:10.2307/30036519 · Cited by 8,635 · cited 1×
“We use four items based on [15, 17, 21, 22].”
An Examination of the Nature of Trust in Buyer-Seller Relationships
Doney, P. M., & Cannon, J. P. (1997) · Journal of Marketing · doi:10.2307/1251829 · Cited by 6,822 · cited 1×
“In our study, we define trust as a belief in the integrity, benevolence, trustworthiness, and predictability of the AI advisor following [15, 17, 21, 49].”
BackgroundDeterminants of Long-Term Orientation in Buyer-Seller Relationships
Ganesan, S. (1994) · Journal of Marketing · doi:10.1177/002224299405800201 · Cited by 6,741 · cited 1×
“In our study, we define trust as a belief in the integrity, benevolence, trustworthiness, and predictability of the AI advisor following [15, 17, 21, 49].”
BackgroundTrust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 10×
“Trust influences reliance, but does not fully determine it [45].”
BackgroundInfluentialPeeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · cited 3×
“Most researchers that studied AR so far have proposed to use explanations of AI as a means for AR [1, 43, 78].”
Background
Research works printed but not linked: 3
- [29] Patrick Hemmer, Max Schemmer, Michael Vössing, and Niklas Kühl. 2021. Human-AI Complementarity in Hybrid Intelligence Systems: A Structured Literature Review. PACIS 2021 Proceedings (2021). -
- [66] Keng Siau and Weiyu Wang. 2018. Building trust in artificial intelligence, machine learning, and robotics. Cutter business technology journal 31, 2 (2018), 47-53. -
- [68] Philipp Spitzer, Niklas Kühl, and Marc Goutier. 2022. Training Novices: The Role of Human-AI Collaboration and Knowledge Transfer. Workshop on Human-Machine Collaboration and Teaming (HM-CaT 2022). The 39th International Conference on Machine Learning, Baltimore, Maryland, USA (2022). -
Cited by
Gomez, C., Cho, S. M., Ke, S., Huang, C.-M., & Unberath, M. (2025) · Frontiers in Computer Science · doi:10.3389/fcomp.2024.1521066 · Cited by 209
Enhancing AI-Assisted Group Decision Making through LLM-Powered Devil's Advocate
Chiang, C.-W., Lu, Z., Li, Z., & Yin, M. (2024) · Proceedings of the 29th International Conference on Intelligent User Interfaces · doi:10.1145/3640543.3645199 · Cited by 142
“For example, many studies specifically examine whether people exhibit any over-reliance or under-reliance on AI recommendations [11, 15, 23, 33, 37, 45, 53, 56, 72, 76, 84, 98, 99, 105, 109, 110].”
BackgroundMehrotra, S., Degachi, C., Vereschak, O., Jonker, C. M., & Tielman, M. L. (2024) · ACM Journal on Responsible Computing · doi:10.1145/3696449 · Cited by 144
Kim, S. S. Y., Liao, Q. V., Vorvoreanu, M., Ballard, S., & Vaughan, J. W. (2024) · The 2024 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3630106.3658941 · Cited by 147
Explainable AI improves task performance in human–AI collaboration
Senoner, J., Schallmoser, S., Kratzwald, B., Feuerriegel, S., & Netland, T. H. (2024) · Scientific Reports · doi:10.1038/s41598-024-82501-9 · Cited by 204
“Unlike previous works on studying task performance in human–AI collaboration 41–44 , we (i) conducted experiments of two real-world job tasks in manufacturing and medicine and (ii) recruited domain experts for those tasks, i.e., factory workers and radiologists.”
Fok, R., & Weld, D. S. (2024) · AI Magazine · doi:10.1002/aaai.12182 · Cited by 81
Wang, F., Li, N., Cheung, A. C. K., & Wong, G. K. W. (2025) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-025-00547-9 · Cited by 69
“On the one hand, reliance involved three forms: over-reliance, appropriate reliance, and under-reliance (Schemmer et al., 2023).”
HILL: A Hallucination Identifier for Large Language Models
Leiser, F., Eckhardt, S., Leuthe, V., Knaeble, M., Mädche, A., Schwabe, G., & Sunyaev, A. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642428 · Cited by 68
“The goal for designers and developers of AI-based systems should be to achieve appropriate reliance of the users (i.e., a user follows correct and does not follow incorrect output) [25, 41].”
BackgroundExplanations, Fairness, and Appropriate Reliance in Human-AI Decision-Making
Schoeffer, J., De-Arteaga, M., & Kühl, N. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642621 · Cited by 68
“From a fairness perspective, the goal is to minimize such disparities, so as to equalize the burden of being misclassified and, as a result, being excluded from exposure to relevant opportunities between men and women.”
BackgroundMethodInfluentialThe Impact of Imperfect XAI on Human-AI Decision-Making
Morrison, K., Spitzer, P., Turri, V., Feng, M., Kühl, N., & Perer, A. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3641022 · Cited by 60
“The original dataset consists of 11,788 images of 200 different bird species and is split into 5,994 training and 5,794 test images.”
BackgroundMethodInfluentialPeng, L., Li, D., Zhang, Z., Zhang, T., Huang, A., Yang, S., & Hu, Y. (2024) · International Journal of Industrial Ergonomics · doi:10.1016/j.ergon.2024.103629 · Cited by 59
Visser, R., Peters, T. M., Scharlau, I., & Hammer, B. (2025) · Cognitive Systems Research · doi:10.1016/j.cogsys.2025.101357 · Cited by 58
“…Guesmi et al., 2023, 2021; Wang & Yin, 2023), user satisfaction (Ehsan & Riedl, 2020), user confidence (Guesmi et al., 2023; Morrison et al., 2024; Schemmer et al., 2023; Thaler & Schmid, 2021; Wanner et al., 2020; Zhou et al., 2018), perceived ease of use (Cramer et al., 2008; Guesmi et al.,…”
InfluentialMa, S., Wang, X., Lei, Y., Shi, C., Yin, M., & Ma, X. (2024) · arXiv · doi:10.48550/arxiv.2403.09552 · Cited by 56
Bashkirova, A., & Krpan, D. (2024) · Computers in Human Behavior: Artificial Humans · doi:10.1016/j.chbah.2024.100066 · Cited by 55
Fok, R., & Weld, D. S. (2023) · arXiv · doi:10.48550/arxiv.2305.07722 · Cited by 54
Ma, S., Wang, X., Lei, Y., Shi, C., Yin, M., & Ma, X. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642671 · Cited by 54
“In recent HCI studies, the focus has shifted from merely increasing trust in AI to facilitating appropriate trust and reliance [6, 8, 47, 70, 78, 79, 85, 87, 89].”
InfluentialA Critical Survey on Fairness Benefits of Explainable AI
Deck, L., Schoeffer, J., De‐Arteaga, M., & Kühl, N. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658990 · Cited by 144
Psychological Traits and Appropriate Reliance: Factors Shaping Trust in AI
Küper, A., & Krämer, N. (2024) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2024.2348216 · Cited by 49
“Schemmer et al. (2023) conceptualized relative AI reliance and relative self-reliance.”
InfluentialMa, S., Chen, Q., Wang, X., Zheng, C., Peng, Z., Yin, M., & Ma, X. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713423 · Cited by 48
“Research in AI-assisted decision-making encompasses various objectives, such as team performance, decision fairness, efficacy, efficiency, understanding of AI, trust, reliance on AI, and subjective experience [6, 16, 17, 25, 26, 35, 43, 57, 67, 83, 90, 90, 91, 112, 130, 131, 138, 141].”
Is Conversational XAI All You Need? Human-AI Decision Making With a Conversational XAI Assistant
He, G., Aishwarya, N., & Gadiraju, U. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712133 · Cited by 115
“Such appropriate reliance requires a comprehensive understanding of the AI system and its underlying rationale alongside the AI advice [14, 67, 102], which play important roles in calibrating user trust and reliance behaviors [116, 130].”
Meakin, L. (2024) · Information Technology and Libraries · doi:10.5860/ital.v43i3.17246 · Cited by 45
“19 Research has shown that humans are susceptible to a variety of misjudgments and biases when seeking advice from machines and that recommendation-dependent preferences can create inefficiencies when the decision-maker is too responsive to the recommendation.”
BackgroundEffect of Explanation Conceptualisations on Trust in AI-assisted Credibility Assessment
Pareek, S., van Berkel, N., Velloso, E., & Goncalves, J. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3686922 · Cited by 42
“This phenomenon where users are able to discern the accuracy of AI advice and rely on it only when warranted is termed appropriate reliance [98].”
BackgroundDoes More Advice Help? The Effects of Second Opinions in AI-Assisted Decision Making
Lu, Z., Wang, D., & Yin, M. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3653708 · Cited by 39
To Rely or Not to Rely? Evaluating Interventions for Appropriate Reliance on Large Language Models
Bo, J. Y., Wan, S., & Anderson, A. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714097 · Cited by 36
Cabitza, F., Fregosi, C., Campagner, A., & Natali, C. (2024) · Communications in Computer and Information Science · doi:10.1007/978-3-031-63803-9_14 · Cited by 35
“Moreover, contrary to initial beliefs that explanations might reduce system over-reliance, findings indicate they might actually increase it [4], leading users to rely too much on incorrect AI recommendations [26].”
ResultDiebel, C., Goutier, M., Adam, M., & Benlian, A. (2025) · Business & Information Systems Engineering · doi:10.1007/s12599-024-00918-y · Cited by 93
Baron, S. (2025) · Philosophy & Technology · doi:10.1007/s13347-024-00837-6 · Cited by 53
“As in the more general case of reliance, explainability can promote reliance on AI (see, e.g., Scharowski et al., 2023; Schemmer et al., 2023; Schoeffer et al., 2024).”
Buçinca, Z., Swaroop, S., Paluch, A. E., Doshi‐Velez, F., & Gajos, K. Z. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713229 · Cited by 76
“With AI systems redefining workflows and the way tasks are carried out, questions surrounding their effect on people’s motivation about the tasks for which they receive assistance are becoming more pressing [12].”
Autonomy by Design: Preserving Human Autonomy in AI Decision-Support
Buijsman, S., Carter, S. E., & Bermúdez, J. P. (2025) · Philosophy & Technology · doi:10.1007/s13347-025-00932-2 · Cited by 53
“This can lead to either under-reliance on the system, as people decide that it is better to trust their own faculties, with lower performance as a result (He et al., 2023); or it can lead to overreliance, often accompanied by automation bias, where users adopt the AI’s recommendation outputs even…”
Göndöcs, D., Horváth, S., & Dörfler, V. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103622 · Cited by 52
A Decision Theoretic Framework for Measuring AI Reliance
Guo, Z., Wu, Y., Hartline, J. D., & Hullman, J. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658901 · Cited by 27
“Future work could extend our definition to continuous recommendation spaces.”
BackgroundKüper, A., Lodde, G. C., Livingstone, E., Schadendorf, D., & Krämer, N. (2025) · Journal of Medical Internet Research · doi:10.2196/58660 · Cited by 31
“In both cases, a value close to 1 indicates a more appropriate level of trust, according to Schemmer et al [48].”
Reicherts, L., Zhang, Z. T., von Oswald, E., Liu, Y., Rogers, Y., & Hassib, M. E. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713295 · Cited by 64
The effects of generative AI’s human-like competencies on clinical decision-making
Spatscheck, N., Schaschek, M., & Winkelmann, A. (2024) · Journal of Decision System · doi:10.1080/12460125.2024.2430731 · Cited by 37
(2025) · doi:10.1007/s12525-025-00862-z · Cited by 23
“…networks may obscure rather than clarify decision logic, which threatens appropriate reliance and user trust in marketplace settings (Ray, 2023; Schemmer et al., 2023 By integrating these perspectives, the IS community can shape MAAI not simply as a technical evolution, but as a paradigmatic…”
He, G., Demartini, G., & Gadiraju, U. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713218 · Cited by 23
“Such user reliance patterns are denoted as appropriate reliance [87, 88], which is the key to achieving complementary team performance.”
Enhancing Intuitive Decision-Making and Reliance Through Human–AI Collaboration: A Review
Xu, G., Murthy, S. V., & Jia, B. (2025) · Informatics · doi:10.3390/informatics12040135 · Cited by 49
Lawless, C., Schoeffer, J., Le, L., Rowan, K., Sen, S., Hill, C. S., Suh, J., & Sarrafzadeh, B. (2024) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3685053 · Cited by 65
“Future work is needed to do larger scale analysis of the system’s usability with personal contextual scheduling tasks.”
Reassuring, Misleading, Debunking: Comparing Effects of XAI Methods on Human Decisions
Humer, C., Hinterreiter, A., Leichtmann, B., Mara, M., & Streit, M. (2024) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3665647 · Cited by 31
“Appropriate user reliance in the AI system is one of the key factors that influence joint performance in an AI-assisted decision-making task [39, 75].”
InfluentialBuilding Appropriate Mental Models: What Users Know and Want to Know about an Agentic AI Chatbot
Brachman, M., Kunde, S., Miller, S., Fucs, A., Dempsey, S., Jabbour, J., & Geyer, W. (2025) · Proceedings of the 30th International Conference on Intelligent User Interfaces · doi:10.1145/3708359.3712071 · Cited by 18
Generative AI and Creativity: A Systematic Literature Review and Meta-Analysis
Holzner, N., Maier, S., & Feuerriegel, S. (2025) · arXiv · doi:10.48550/arxiv.2505.17241 · Cited by 143
“Ultimately, such insights could inform frameworks with a notion of ‘appropriate reliance’ (see [53] for an overview on the notion of appropriate reliance in AI advice), that is, when it is most beneficial for an idea to originate from the human, the GenAI system, or through their collaboration.”
Dogru, E. Ö., & Krämer, N. C. (2025) · Journal of Decision Systems · doi:10.1080/12460125.2025.2593251 · Cited by 15
“Appropriate Reliance was calculated according to Schemmer et al. (2023) using RAIR and RSR. RAIR describes the ratio of cases in which participants correctly adjusted their initially incorrect answers based on accurate AI advice to the total number of cases where they had the opportunity to adjust…”
InfluentialJeon, H., Yoon, S., Lee, K., Kim, S. H., Kim, E. H., Cho, S., Ko, Y., Yang, S., Dabbish, L., Zimmerman, J., et al. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714206 · Cited by 36
“Such misuse could raise ethical concerns about diminished human agency, as over-reliance on AI threatens to undermine the critical role of personal judgment and self-directed decision-making [42, 83].”
From Text to Trust: Empowering AI-assisted Decision Making with Adaptive LLM-powered Analysis
Li, Z., Zhu, H., Lu, Z., Xiao, Z., & Yin, M. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713133 · Cited by 38
“To address this problem, previous research [2, 78, 79] proposed to display explanations generated by post-hoc explainable AI (XAI) meth-ods [29, 56, 72] along with the AI model’s decision recommendations to assist people in evaluating AI’s reliability and identifying optimal strategies for relying…”
Prabhudesai, S., Kasi, A. P., Mansingh, A., Antar, A. D., Shen, H., & Banović, N. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713714 · Cited by 35
Human control redressed: Comparing AI and human predictability in a real-effort task
Kandul, S., Micheli, V., Beck, J., Burri, T., Fleuret, F., Kneer, M., & Christen, M. (2023) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2023.100290 · Cited by 22
Measuring the effect of mental workload and explanations on appropriate AI reliance using EEG
Zhang, Z. T., Argın, S. K., Bilen, M. B., Urgun, D., Deniz, S. M., Liu, Y., & Hassib, M. E. (2024) · Behaviour and Information Technology · doi:10.1080/0144929x.2024.2431055 · Cited by 24
“Moreover, to be able to use the appropriateness of reliance measurement concept by Schemmer et al. [14], we used a setup where participants made an initial decision independently from the AI.”
InfluentialOn the Interdependence of Reliance Behavior and Accuracy in AI-Assisted Decision-Making
Schoeffer, J., Jakubik, J., Voessing, M., Kuehl, N., & Satzger, G. (2023) · Frontiers in Artificial Intelligence and Applications · doi:10.3233/faia230074 · Cited by 15
Self-citationMuijlwijk, H., Willemsen, M. C., Smyth, B., & IJsselsteijn, W. A. (2024) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3640543.3645205 · Cited by 20
“Over the last years various solutions have been proposed to improve AI-assisted decision making by calibrating trust and foster ’appropriate reliance’ [50].”
BackgroundEckhardt, S., Kühl, N., Dolata, M., & Schwabe, G. (2025) · ACM Computing Surveys · doi:10.1145/3776528 · Cited by 45
Beyond Recommendations: From Backward to Forward AI Support of Pilots' Decision-Making Process
Zhang, Z. T., Feger, S. S., Dullenkopf, L., Liao, R., Süsslin, L., Liu, Y., & Butz, A. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3687024 · Cited by 16
Memduhoğlu, A., Fulman, N., Polat, N., & Atas, T. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.130171 · Cited by 17
Don’t Be Fooled: The Misinformation Effect of Explanations in Human–AI Collaboration
Spitzer, P., Holstein, J., Morrison, K., Holstein, K., Satzger, G., & Kühl, N. (2026) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2025.2574511 · Cited by 12
Self-citationLiel, Y., & Zalmanson, L. (2025) · Journal of Management Information Systems · doi:10.1080/07421222.2025.2561390 · Cited by 24
“Algorithmic overreliance—a phenomenon where people follow algorithmic recommendations even when those recommendations are flawed [15, 70, 77]—represents a persistent challenge in human-algorithm interaction.”
Kahr, P. K., Rooks, G., Snijders, C. C. P., & Willemsen, M. C. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713099 · Cited by 26
User Engagement and Beyond: A Conceptual Framework for Engagement in Information Systems Research
Benz, C., Riefle, L., & Satzger, G. (2024) · Communications of the Association for Information Systems · doi:10.17705/1cais.05412 · Cited by 23
Self-citationCooperative Multi-Objective Bayesian Design Optimization
Mo, G., Dudley, J. J., Chan, L., Liao, Y.-C., Oulasvirta, A., & Kristensson, P. O. (2024) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3657643 · Cited by 14
“This echoes an overarching problem of establishing and maintaining appropriate trust in human-AI interaction [12, 31].”
Why AI may undermine phronesis and what to do about it
Tsai, C., & Ku, H.-L. (2024) · AI and Ethics · doi:10.1007/s43681-024-00617-0 · Cited by 17
“Most research on AI reliance defines appropriate reliance in decision-making as adhering to correct AI advice and overriding wrong AI advice (see [26]). opportunities for individuals to develop their phronesis.”
Biswas, S., Erlei, A., & Gadiraju, U. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713201 · Cited by 20
“In this paper, we focus on the latter two aspects, as both appropriate reliance and consumer attitudes have been previously shown to be important facets in human-AI interaction [30, 42, 47, 98, 102, 124].”
To Solicit or Not to Solicit? Impact of AI Assistance Delivery Mechanisms on Decision-Making
Nair, P., Gressel, G., Anand, M. N., & Achuthan, K. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2536617 · Cited by 17
“Schemmer et al. introduced Appropriateness of Reliance (AoR) as a two-dimensional measure of human discretion in following AI advice and demonstrated how explanations can significantly influence reliance behavior and AI effectiveness (Schemmer et al., 2023).”
InfluentialA comprehensive introspection on AI risks: taxonomy, challenges, and future directions
Ray, P. P. (2026) · Iran Journal of Computer Science · doi:10.1007/s42044-025-00370-5 · Cited by 17
“In high-stakes scenarios—such as medical diagnostics or financial risk assessments—the margin for error narrows, making accurate calibration of trust a critical prerequisite for effective human–AI collaboration [69].”
Ashktorab, Z., Desmond, M., Pan, Q., Johnson, J., Brachman, M., Dugan, C., Danilevsky, M., & Geyer, W. (2025) · Proceedings of the 4th Annual Symposium on Human-Computer Interaction for Work · doi:10.1145/3729176.3729179 · Cited by 10
Воронцова, А., Тарасенко, С., Duranowski, W., Durasiewicz, A., Soss, J., & Біловол, А. (2025) · Problems and Perspectives in Management · doi:10.21511/ppm.23(1).2025.08 · Cited by 14
BleacherBot: AI Agent as a Sports Co-Viewing Partner
Kim, K., Song, H., Ryu, J., Oh, C., & Suh, B. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714178 · Cited by 15
“Additionally, Schemmer et al. [100] propose that careful attention to data privacy protocols could help ensure compliance with data protection regulations while maintaining transparency about data usage.”
The Amplifying Effect of Explainability in AI-assisted Decision-making in Groups
Duarte, R. D. B., Abreu, M. C., Campos, J., & Paiva, A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713534 · Cited by 19
“This evaluation helps to understand the human-AI interaction in decision-making tasks [51].”
InfluentialHuman-AI Collaboration in Decision Making: An Initial Reliability Study and Methodology
Biloborodova, T., & Skarga-Bandurova, I. (2023) · IEEE International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS) · doi:10.1109/idaacs58523.2023.10348928 · Cited by 12
“In a separate study, Schemmer et al. [12] introduced a measurement concept of appropriateness of reliance and a research model that analyses the effect of providing explanations for AI advice.”
BackgroundThe Influence of Curiosity Traits and On-Demand Explanations in AI-Assisted Decision-Making
Cau, F. M., & Spano, L. D. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712165 · Cited by 15
Raees, M., Khan, V.-J., & Papangelis, K. (2025) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3699682.3728320 · Cited by 9
Trang, T. T. N., & Thắng, P. C. (2025) · Journal of Consumer Behaviour · doi:10.1002/cb.70045 · Cited by 17
Cabitza, F., Campagner, A., Fregosi, C., Cameli, M., Gallazzi, E., Sconfienza, L. M., & Tontini, G. E. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757601 · Cited by 13
The effect of medical explanations from large language models on diagnostic accuracy in radiology
Spitzer, P., Hendriks, D., Rudolph, J., Schlaeger, S., Ricke, J., Kühl, N., Hoppe, B. F., & Feuerriegel, S. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02619-0 · Cited by 21
Ma, S., Chen, Q., Wang, X., Zheng, C., Peng, Z., Yin, M., & Ma, X. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4772689 · Cited by 8
Authorship Drift: How Self-Efficacy and Trust Evolve During LLM-Assisted Writing
Park, Y. S., Arvi, N. A. P., Kim, S., & Kim, J. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790276 · Cited by 8
Zipperling, D., Deck, L., Lanzl, J., & Kühl, N. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732084 · Cited by 14
“When confronted with an AI recommendation, human decision-makers can over-rely , often stemming from a phenomenon labeled “au-tomation bias” [3, 18, 55, 69], under-rely , often stemming from a phenomenon labeled “algorithm aversion” [22], or appropriately rely [50, 68].”
Ghasemaghaei, M. (2026) · Journal of Management Information Systems · doi:10.1080/07421222.2025.2602397 · Cited by 10
“The extent to which individuals adhere to and implement the recommendations provided by AI systems in their decision-making processes [18, 64].”
Zhang, Z. T., Buchner, F., Liu, Y., & Butz, A. (2024) · Proceedings of Mensch und Computer 2024 · doi:10.1145/3670653.3670660 · Cited by 6
“However, many studies have found that explanations do not effectively improve appropriate reliance [3, 33, 38, 41] or even increase overreliance on AI [2, 4, 17, 21, 34], i.e., people become more likely to adopt detrimental AI recommendations.”
Towards Understanding Persuasive and Personalized Engagement for Human-AI Reliance
Raees, M., Khan, V.-J., & Papangelis, K. (2025) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3708319.3733643 · Cited by 10
“Despite the significance of these varied human factors, most implementations lack adaptation of AI assistance to user traits, latent qualities, and real-time engagement behavior [4, 26].”
InfluentialHolstein, J., Böcking, L., Spitzer, P., Kühl, N., Vössing, M., & Satzger, G. (2025) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3762813 · Cited by 17
Self-citationMendel, T., Mandal, S., Nov, O., & Wiesenfeld, B. M. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757524 · Cited by 10
Schäfer, U., Sipos, L., & Müller-Birn, C. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757451 · Cited by 13
Passi, S., Dhanorkar, S., & Vorvoreanu, M. (2025) · Springer eBooks · doi:10.1007/978-981-95-3658-0_98 · Cited by 15
Raees, M., Khan, V.-J., Lykourentzou, I., & Papangelis, K. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791467 · Cited by 5
“Research increasingly highlights the need to gain a foundational understanding of appropriate reliance on AI advice [13, 41, 81, 132] to improve human-AI decision-making scenarios.”
Influential“Garbage In, Garbage Out”: Mitigating Human Biases in Data Entry by Means of Artificial Intelligence
Eckhardt, S., Knaeble, M., Bucher, A. M., Staehelin, D., Dolata, M., Agotai, D., & Schwabe, G. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-42286-7_2 · Cited by 9
A short-form AI literacy intervention can reduce over-reliance on AIPreprint
Morrill, J., & Noetel, M. (2023) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/hv9qc · Cited by 10
Chen, R., Wang, R., Sadeh, N., & Fang, F. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732050 · Cited by 13
Provoking critical thinking: Using counter-arguments in online discussion summarisation
Li, S., Han, L., & Demartini, G. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104258 · Cited by 9
Zhu, J., Kempermann, M., Cannanure, V. K., Hartland, A., Navarrete, R. M., Carteny, G., Braun, D., & Weber, I. G. (2025) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3719160.3736611 · Cited by 8
“Trustworthiness in an LLM chatbot can be assessed based on its competence and value alignment with users [40], ideally corresponding to appropriate levels of user reliance on the technology [52].”
Kuhl, U., & Bush, A. (2025) · Communications in computer and information science · doi:10.1007/978-3-032-08333-3_12 · Cited by 12
“Users must develop accurate mental models of AI capabilities through experience [35].”
Being Pragmatic About Reliance and Trust in Artificial Intelligence
Ferrario, A. (2025) · Minds and Machines · doi:10.1007/s11023-025-09754-9 · Cited by 14
“First, we show that appropriate reliance is the first objective of human-AI interactions; for instance, within the context of AI supporting decision-making, it can be clearly formalized in terms of the performance of human-AI teams (Schemmer et al., 2023).”
InfluentialCan I Trust You? Exploring the Impact of Misleading AI Suggestions on User's Trust
Pretolesi, D., Zechner, O., Schrom-Feiertag, H., & Tscheligi, M. (2024) · IEEE International Conference on Metrology for eXtended Reality, Artificial Intelligence and Neural Engineering (MetroXRAINE) · doi:10.1109/metroxraine62247.2024.10796402 · Cited by 7
From Trust to Appropriate Reliance: Measurement Constructs in Human-AI Decision-Making
Raees, M., & Papangelis, K. (2026) · arXiv · doi:10.48550/arxiv.2604.23896 · Cited by 5
“While addressing these challenges, users may be able to build reliance on AI, but it cannot necessarily always be appropriate, i.e., when AI provides incorrect advice presented as correct, and users accept this advice without being able to recognize it as incorrect [73].”
InfluentialDong, A., Li, P., Chen, Y., Gibson, S., Zhao, L., & He, M. (2026) · Sustainability · doi:10.3390/su18115313 · Cited by 9
Bridging Domain Expertise and AI through Data Understanding
Holstein, J. (2024) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3640544.3645248 · Cited by 4
“In the latter, human experts receive AI suggestions and must evaluate whether to follow these suggestions or overwrite them with their own decisions [13].”
BackgroundSpitzer, P., Celis, S., Martin, D., Kühl, N., & Satzger, G. (2024) · Mensch und Computer (MuC) · doi:10.1145/3670653.3677488 · Cited by 8
“As our study builds the foundation for further exploration, future research can focus on measuring the decision-making behavior in terms of appropriate reliance [22, 23].”
Self-citationGender, knowledge, and trust in artificial intelligence: A classroom-based randomized experiment.
Dávila, A., El Fassi, I., Oyon, D., & Rudolf, N. (2025) · Scientific Reports · doi:10.1038/s41598-025-25002-7 · Cited by 3
Bruijnes, M., Grimmelikhuijsen, S., & Robeer, M. (2025) · Springer eBooks · doi:10.1007/978-3-031-84748-6_4 · Cited by 7
Understanding the Effects of Explaining Predictive but Unintuitive Features in Human-XAI Interaction
Qu, J., Arguello, J., & Wang, Y. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732021 · Cited by 8
Ogawa, N., Okafuji, Y., Hatada, Y., & Baba, J. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757686 · Cited by 11
Biases and Ethical Considerations in AI-Driven Knowledge Management
Gafni, R., Kantsepolsky, B., & Sherman, S. (2026) · Emerald Group Publishing Limited eBooks · doi:10.1108/978-1-80592-391-620261013 · Cited by 3
Exploring human-swarm interaction dynamics in cyber-physical systems: A physiological approach
Distefano, J. P., Chowdhury, S., & Esfahani, E. T. (2023) · Journal of Integrated Design and Process Science · doi:10.1177/10920617241292155 · Cited by 4
“…loops (Chen et al., 2011; de Visser & Parasuraman, 2011; Prati et al., 2021; Su et al., 2022) or to calibrate the level of feed-back, ensuring that operators are informed yet not over-whelmed by information (Camilleri et al., 2022; Chen et al., 2018; Lyons & Havig, 2014; Schemmer et al., 2023).”
Metadata
- Authors
- Schemmer, M., Kuehl, N., Benz, C., Bartos, A., & Satzger, G.
- Affiliations
- Karlsruhe Institute of Technology (DE)
- Venue
- Proceedings of the 28th International Conference on Intelligent User Interfaces
- Publisher
- ACM
- Published
- 2023-03-27
- Pages
- 410-422
- OpenAlex
- W4360991389
- Semantic Scholar
- 95f751e8535a6d7f8fe733ab4729f738ef0566c4
- Cited by
- 208
- References
- 43
- Citations by year
- 2026: 93 · 2025: 71 · 2024: 37 · 2023: 6
- FWCI
- 36.59
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM International Conference on Intelligent User Interfaces (IUI) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by-nc-nd)
- Repository KITopen (Karlsruhe Institute of Technology) (submittedVersion, cc-by-nc-nd)
- arXiv (Cornell University)
- KITopen
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Keywords
- AI advice, AI explanations, human-AI decision-making, trust in AI, measurement concepts, investment decision-making, treatment decision-making
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.