The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
Do People Engage Cognitively with AI? Impact of AI Assistance on Incidental Learning
Open accessAbstract
When people receive advice while making difficult decisions, they often make better decisions in the moment and also increase their knowledge in the process. However, such incidental learning can only occur when people cognitively engage with the information they receive and process this information thoughtfully. How do people process the information and advice they receive from AI, and do they engage with it deeply enough to enable learning? To answer these questions, we conducted three experiments in which individuals were asked to make nutritional decisions and received simulated AI recommendations and explanations. In the first experiment, we found that when people were presented with both a recommendation and an explanation before making their choice, they made better decisions than they did when they received no such help, but they did not learn. In the second experiment, participants first made their own choice, and only then saw a recommendation and an explanation from AI; this condition also resulted in improved decisions, but no learning. However, in our third experiment, participants were presented with just an AI explanation but no recommendation and had to arrive at their own decision. This condition led to both more accurate decisions and learning gains. We hypothesize that learning gains in this condition were due to deeper engagement with explanations needed to arrive at the decisions. This work provides some of the most direct evidence to date that it may not be sufficient to include explanations together with AI-generated recommendation to ensure that people engage carefully with the AI-provided information. This work also presents one technique that enables incidental learning and, by implication, can help people process AI recommendations and explanations more carefully.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Previous studies have attributed the cause of overreliance on AI to a lack of cognitive engagement [10,32,55].”
From Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations (2023) · cites this paper as [32]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“It is possible that when not thinking aloud, especially in a lowstakes setting, people will be less cognitively engaged with AI explanations [32] and some effects found in the study will not be observed.”
From Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations (2023) · cites this paper as [32]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Similarly, Gajos and Mamykina (2022) found that people achieved learning gains from interacting with AI explanations when they critically engaged with AI explanations.”
From The role of critical thinking on undergraduates' reliance behaviours on generative AI in problem‐solving (2025) · cites this paper as Gajos & Mamykina, 2022
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The actual impact of explanations can be influenced by various factors, including task difficulty [44], decision-maker expertise, decision design [45], and the cognitive load required to comprehend the explanation [46] and also the type of explanations and the way it is presented.”
From AI Trust: Can Explainable AI Enhance Warranted Trust? (2023) · cites this paper as [45]
The cited source says what the claim attributes to it.
In the citing paper
“The results indicate that active cognitive processes involved in evaluating AI explanations are key to effective collaboration between humans and AI.”
From The role of critical thinking on undergraduates' reliance behaviours on generative AI in problem‐solving (2025) · cites this paper as Gajos & Mamykina, 2022
In the citing paper
“or perhaps even worse, people follow them blindly, even when the recommendations are wrong.”
From Explainable AI is Dead, Long Live Explainable AI!: Hypothesis-driven Decision Support using Evaluative AI (2023) · cites this paper as [6]
In the citing paper
“Their results showed that cognitive forcing slightly mitigated overreliance, compared to feature-based and uncertainty judgements, particularly disregarding incorrect AI recommendations, but with no statistically significant differences in task performance.”
From Explainable AI is Dead, Long Live Explainable AI!: Hypothesis-driven Decision Support using Evaluative AI (2023) · cites this paper as [6]
In the citing paper
“However, recent research has shown that the recommendation-driven explainable/interpretable AI has little effect on decision making [1, 6-8, 28, 32], although this is not always the case [22,23,41].”
From Explainable AI is Dead, Long Live Explainable AI!: Hypothesis-driven Decision Support using Evaluative AI (2023) · cites this paper as [6]
In the citing paper
“Similarly, in experiments on a general task with general participants, Buçinca et al. [1] and Gajos and Mamykina [6] show that explainability did not mitigate over-reliance, and in some cases it increased over-reliance.”
From Explainable AI is Dead, Long Live Explainable AI!: Hypothesis-driven Decision Support using Evaluative AI (2023) · cites this paper as [6]
In the citing paper
“Interestingly, despite cognitive forcing functions being more effective, they were least preferred by participants.”
From Explainable AI is Dead, Long Live Explainable AI!: Hypothesis-driven Decision Support using Evaluative AI (2023) · cites this paper as [6]
In the citing paper
“Cognitive forcing takes a similar approach, but current cognitive forcing techniques are recommendation-driven [1,6].”
From Explainable AI is Dead, Long Live Explainable AI!: Hypothesis-driven Decision Support using Evaluative AI (2023) · cites this paper as [6]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Re-epithelialization and immune cell behaviour in an ex vivo human skin model
Rakita, A., Nikolić, N., Mildner, M., Matiasek, J., & Elbe‐Bürger, A. (2020) · Scientific Reports · doi:10.1038/s41598-019-56847-4 · Cited by 8,234
Cacioppo, J. T., & Petty, R. E. (1982) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.42.1.116 · Cited by 5,359
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Petty, R. E., & Cacioppo, J. T. (1986) · doi:10.1007/978-1-4612-4964-1 · Cited by 5,219
The Elaboration Likelihood Model of Persuasion
Petty, R. E., & Cacioppo, J. T. (1986) · Communication and Persuasion · doi:10.1007/978-1-4612-4964-1_1 · Cited by 3,202
The Efficient Assessment of Need for Cognition
Cacioppo, J. T., Petty, R. E., & Feng Kao, C. (1984) · Journal of Personality Assessment · doi:10.1207/s15327752jpa4803_13 · Cited by 2,913
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372873 · Cited by 2,245
Measuring cognitive and psychological engagement: Validation of the Student Engagement Instrument
Appleton, J. J., Christenson, S. L., Kim, D., & Reschly, A. L. (2006) · Journal of School Psychology · doi:10.1016/j.jsp.2006.04.002 · Cited by 1,858
Active‐Constructive‐Interactive: A Conceptual Framework for Differentiating Learning Activities
Michelene, T. H. (2009) · Topics in Cognitive Science · doi:10.1111/j.1756-8765.2008.01005.x · Cited by 1,799
Statistical Power Analysis for the Behavioral Sciences
Wassertheil, S., & Cohen, J. (1970) · Biometrics · doi:10.2307/2529115 · Cited by 1,383
Research works printed but not linked: 15
- John R Anderson and Lynne M Reder . 1979. An elaborative processing explanation of depth of processing. L .; S. Cermak and FIM Craik, Eds., Levels of Processing in Human Memory (Erlbam , 1979 ) (1979), 385–404. John R Anderson and Lynne M Reder. 1979. An elaborative processing explanation of depth of processing. L.; S. Cermak and FIM Craik, Eds., Levels of Processing in Human Memory (Erlbam, 1979) (1979), 385–404.
- David W Bates , David Levine , Ania Syrowatka , Masha Kuznetsova , Kelly Jean Thomas Craig , Angela Rui, Gretchen Purcell Jackson, and Kyu Rhee. 2021 . The potential of artificial intelligence to improve patient safety: a scoping review. NPJ digital medicine 4, 1 (2021), 1–8. David W Bates, David Levine, Ania Syrowatka, Masha Kuznetsova, Kelly Jean Thomas Craig, Angela Rui, Gretchen Purcell Jackson, and Kyu Rhee. 2021. The potential of artificial intelligence to improve patient safety: a scoping review. NPJ digital medicine 4, 1 (2021), 1–8.
- Michelene TH Chi and Ruth Wylie . 2014. The ICAP framework: Linking cognitive engagement to active learning outcomes. Educational psychologist 49, 4 ( 2014 ), 219–243. Michelene TH Chi and Ruth Wylie. 2014. The ICAP framework: Linking cognitive engagement to active learning outcomes. Educational psychologist 49, 4 (2014), 219–243.
- Amit Dhurandhar Pin-Yu Chen Ronny Luss Chun-Chen Tu Paishun Ting Karthikeyan Shanmugam and Payel Das. 2018. Explanations based on the missing: Towards contrastive explanations with pertinent negatives. In Advances in Neural Information Processing Systems. 592–603. Amit Dhurandhar Pin-Yu Chen Ronny Luss Chun-Chen Tu Paishun Ting Karthikeyan Shanmugam and Payel Das. 2018. Explanations based on the missing: Towards contrastive explanations with pertinent negatives. In Advances in Neural Information Processing Systems. 592–603.
- Catherine O Fritz , Peter E Morris , and Jennifer J Richler . 2012 . Effect size estimates: current use, calculations, and interpretation.Journal of experimental psychology : General 141 , 1 (2012), 2 . Catherine O Fritz, Peter E Morris, and Jennifer J Richler. 2012. Effect size estimates: current use, calculations, and interpretation.Journal of experimental psychology: General 141, 1 (2012), 2.
- Laura Germine , Ken Nakayama , Bradley C Duchaine , Christopher F Chabris , Garga Chatterjee , and Jeremy B Wilmer . 2012. Is the Web as good as the lab? Comparable performance from Web and lab in cognitive/perceptual experiments. Psychonomic bulletin & review 19, 5 ( 2012 ), 847–857. Laura Germine, Ken Nakayama, Bradley C Duchaine, Christopher F Chabris, Garga Chatterjee, and Jeremy B Wilmer. 2012. Is the Web as good as the lab? Comparable performance from Web and lab in cognitive/perceptual experiments. Psychonomic bulletin & review 19, 5 (2012), 847–857.
- Ben Green . 2021. The Flaws of Policies Requiring Human Oversight of Government Algorithms. Available at SSRN ( 2021 ). Ben Green. 2021. The Flaws of Policies Requiring Human Oversight of Government Algorithms. Available at SSRN (2021).
- Barbara A Greene , Raymond B Miller , H Michael Crowson , Bryan L Duke , and Kristine L Akey . 2004. Predicting high school students ’ cognitive engagement and achievement: Contributions of classroom perceptions and motivation. Contemporary educational psychology 29, 4 ( 2004 ), 462–482. Barbara A Greene, Raymond B Miller, H Michael Crowson, Bryan L Duke, and Kristine L Akey. 2004. Predicting high school students’ cognitive engagement and achievement: Contributions of classroom perceptions and motivation. Contemporary educational psychology 29, 4 (2004), 462–482.
- Ece Kamar. 2016. Directions in Hybrid Intelligence: Complementing AI Systems with Human Intelligence.. In IJCAI. 4070–4073. Ece Kamar. 2016. Directions in Hybrid Intelligence: Complementing AI Systems with Human Intelligence.. In IJCAI. 4070–4073.
- Ece Kamar , Severin Hacker , and Eric Horvitz . 2012 . Combining human and machine intelligence in large-scale crowdsourcing . In Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems-Volume 1. International Foundation for Autonomous Agents and Multiagent Systems, 467–474 . Ece Kamar, Severin Hacker, and Eric Horvitz. 2012. Combining human and machine intelligence in large-scale crowdsourcing. In Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems-Volume 1. International Foundation for Autonomous Agents and Multiagent Systems, 467–474.
- Benjamin Krarup , Michael Cashmore , Daniele Magazzeni , and Tim Miller . 2019 . Model-based contrastive explanations for explainable planning . In ICAPS 2019 Workshop on Explainable AI Planning (XAIP). Benjamin Krarup, Michael Cashmore, Daniele Magazzeni, and Tim Miller. 2019. Model-based contrastive explanations for explainable planning. In ICAPS 2019 Workshop on Explainable AI Planning (XAIP).
- Stefan Münzer , Hubert D Zimmer , and Jörg Baus . 2012. Navigation assistance: A trade-off between wayfinding support and configural learning support.Journal of experimental psychology: applied 18, 1 ( 2012 ), 18. Stefan Münzer, Hubert D Zimmer, and Jörg Baus. 2012. Navigation assistance: A trade-off between wayfinding support and configural learning support.Journal of experimental psychology: applied 18, 1 (2012), 18.
- Jerome I Rotgans and Henk G Schmidt . 2011. Cognitive engagement in the problem-based learning classroom. Advances in health sciences education 16, 4 ( 2011 ), 465–479. Jerome I Rotgans and Henk G Schmidt. 2011. Cognitive engagement in the problem-based learning classroom. Advances in health sciences education 16, 4 (2011), 465–479.
- Maciej Tomczak and Ewa Tomczak . 2014. The need to report effect size estimates revisited. An overview of some recommended measures of effect size. Trends in sport sciences 1, 21 ( 2014 ), 19–25. Maciej Tomczak and Ewa Tomczak. 2014. The need to report effect size estimates revisited. An overview of some recommended measures of effect size. Trends in sport sciences 1, 21 (2014), 19–25.
- Bas Van Fraassen . 1988. The pragmatic theory of explanation. Theories of explanation 8 ( 1988 ), 135–155. Bas Van Fraassen. 1988. The pragmatic theory of explanation. Theories of explanation 8 (1988), 135–155.
Cited by
Sorted by citations
Items 101–129 of 129
Intelligent Decision Support in Personal Health: Personalized Health Coaching in Type 2 Diabetes
Mamykina, L., Mitchell, E. G., Desai, P. M., & Albers, D. J. (2024) · Cognitive informatics in biomedicine and healthcare · doi:10.1007/978-3-031-69947-4_17
Self-citationNault, K., Nieto-Taborda, M., & Ruhi, U. (2025) · doi:10.1109/icaie64856.2025.11158527
Zahnoiko, V. (2025) · State Formation · doi:10.26565/1992-2337-2025-1-30 · Cited by 2
Onari, M. A., Baer, G. T., Zhang, C., Grau, I., Nobile, M. S., & Zhang, Y. (2025) · Communications in computer and information science · doi:10.1007/978-3-032-08317-3_15 · Cited by 1
AI-Driven Digital Learning Recognition: Evaluating User Perceptions of Beedik Implementation
Sasmoko, Indrianti, Y., Manalu, S. R., Hartono, H., Bakar, A. Y. A., & Hermanus, D. R. (2025) · International Conference on ICT for Smart Society (ICISS) · doi:10.1109/iciss66954.2025.11389255
Double-edged sword effect of AI distrust on individual creativity: empirical evidence from China
Li, H., Guo, C., & Ali, R. Z. (2026) · Asia Pacific Business Review · doi:10.1080/13602381.2026.2634713
MIRAGE: Misleading Impacts Resulting from AI Generated Explanations (Workshop)
Daly, E., Ehsan, U., Quercia, D., & Stumpf, S. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742414.3794955
Beyond Information Amount: Rethinking Transparency through Active Reasoning Demand
Zhang, X., Ye, Y., Ma, Y., Zhang, Z. T., & Butz, A. M. (2026) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/3757279.3785623
Explainability for machine learning models : from data adaptability to user perception
Delaunay, J. (2023) · theses.fr (ABES) · doi:10.70675/933366c4z576ez464cz9fcfz24627f86dd2f
Bertrand, A. (2024) · theses.fr (ABES) · doi:10.70675/216d3647z78bcz4c68z9ba9z2d0a029cf1b4 · Cited by 1
Cheatle, C., & Banks, A. P. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798384
Faster but Not Smarter? Temporal Constraints and User Compliance with AI Explanations
Shahu, A., Greinecker, N., & Wintersberger, P. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798697
Guided Reflection in AI-Assisted Decision-Making: Effects on AI Overreliance and Decision Accuracy
Li, S., Li, J., Li, H., Zhu, H., & Li, X. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790632 · Cited by 2
Human-AI Narrative Synthesis to Foster Shared Understanding in Civic Decision-Making
Overney, C., Jiang, H., Haider, U., Moe, C., Mangat, J., Pantano, F., McMillian, E. G., Riggins, P., & Gillani, N. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791570 · Cited by 2
"It Would Take Too Much to Explain": Task-Contingent AI Use in Design Education
Dines, N., & Mokryn, O. (2026) · Annual Symposium on HCI Education (EduCHI) · doi:10.1145/3803869.3803887
WikiCoach: Scaffolding Learning for Novices in Online Apprenticeship Communities
Zhou, M., Zhao, M., & Terveen, L. (2026) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3774935.3806185 · Cited by 1
Explanation Demand as a Trait, Not a State: Implications for User Modeling in Adaptive Systems
Ingesson, E., & Riveiro, M. (2026) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3774935.3806160
Charan, J., Singla, L., Girdharwal, N., Gautam, U., Kaur, G., & Sharma, A. (2025) · International Conference on Intelligent & Innovative Practices in Engineering & Management (IIPEM) · doi:10.1109/iipem65914.2025.11548294
Arun, S. K., & Fischer, J. E. (2026) · Annual Symposium on Human-Computer Interaction for Work (CHIWORK) · doi:10.1145/3808045.3808049
Bergh, C. H., Vassar, A., Banks, N., Xu, J., & Renzella, J. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-29773-0_7
Yang, M.-Z., Kagawa, R., & Baba, Y. (2026) · Conference on Fairness, Accountability and Transparency · doi:10.1145/3805689.3812288
Wang, Z., & Hu, K. (2026) · ACM Transactions on Social Computing · doi:10.1145/3843742
Martinelli, C. V., Vidali, A., Carnevale, V., Wang, C.-H., Ercoli, A., & Giordano, A. (2026) · doi:10.2196/preprints.110829
Generative AI for Metacognition in AI Agents
Goel, A. K., Sushri, S., Alico, N. E., Luu, J., Khan, R. M., & Rugaber, S. (2026) · Springer eBooks · doi:10.1007/978-3-032-24819-0_23
Li, Z., Ma, Z., He, S., Lu, Z., Xiao, Z., & Yin, M. (2026) · ACM Conference on Human-AI Complementarity and Alignment · doi:10.1145/3834580.3838738
From Potential to Reality: Gaps in Realizing Human–AI Complementarity in Educational Tasks
Jiang, B., Wang, J., Lu, Y., Liu, J., & Jiang, Y. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2732482
an, L. B., Hoang, P. N., Nguyen, G., & Huynh, H. (2026) · Education + Training · doi:10.1108/et-06-2025-0427
Fan, L., Feng, A., Li, Y., & Zhao, Z. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2737593
Metadata
- Authors
- Gajos, K. Z., & Mamykina, L.
- Affiliations
- Harvard University (US)
- Columbia University (US)
- Venue
- 27th International Conference on Intelligent User Interfaces
- Publisher
- ACM
- Published
- 2022-03-22
- Pages
- 794-806
- OpenAlex
- W4220961556
- Cited by
- 127
- References
- 50
- Citations by year
- 2026: 34 · 2025: 47 · 2024: 22 · 2023: 21 · 2022: 3
- FWCI
- 12.86
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM International Conference on Intelligent User Interfaces (IUI) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (IIS-2107391)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Keywords
- incidental learning, AI assistance, AI explanations, AI recommendations, decision-making, cognitive engagement, advice taking, human-AI interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
