Cited by 129Explainable Artificial Intelligence (XAI)Human-Automation Interaction and SafetyDecision-Making and Behavioral Economics

The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.

Do People Engage Cognitively with AI? Impact of AI Assistance on Incidental Learning

Open access
Gajos, Krzysztof Z., Mamykina, Lena·2022·27th International Conference on Intelligent User Interfaces

Abstract

When people receive advice while making difficult decisions, they often make better decisions in the moment and also increase their knowledge in the process. However, such incidental learning can only occur when people cognitively engage with the information they receive and process this information thoughtfully. How do people process the information and advice they receive from AI, and do they engage with it deeply enough to enable learning? To answer these questions, we conducted three experiments in which individuals were asked to make nutritional decisions and received simulated AI recommendations and explanations. In the first experiment, we found that when people were presented with both a recommendation and an explanation before making their choice, they made better decisions than they did when they received no such help, but they did not learn. In the second experiment, participants first made their own choice, and only then saw a recommendation and an explanation from AI; this condition also resulted in improved decisions, but no learning. However, in our third experiment, participants were presented with just an AI explanation but no recommendation and had to arrive at their own decision. This condition led to both more accurate decisions and learning gains. We hypothesize that learning gains in this condition were due to deeper engagement with explanations needed to arrive at the decisions. This work provides some of the most direct evidence to date that it may not be sufficient to include explanations together with AI-generated recommendation to ensure that people engage carefully with the AI-provided information. This work also presents one technique that enables incidental learning and, by implication, can help people process AI recommendations and explanations more carefully.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3490099.3511138

Cites

57 of 57 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 50 references

Research works printed but not linked: 15

  1. John R Anderson and Lynne M Reder . 1979. An elaborative processing explanation of depth of processing. L .; S. Cermak and FIM Craik, Eds., Levels of Processing in Human Memory (Erlbam , 1979 ) (1979), 385–404. John R Anderson and Lynne M Reder. 1979. An elaborative processing explanation of depth of processing. L.; S. Cermak and FIM Craik, Eds., Levels of Processing in Human Memory (Erlbam, 1979) (1979), 385–404.
  2. David W Bates , David Levine , Ania Syrowatka , Masha Kuznetsova , Kelly Jean Thomas Craig , Angela Rui, Gretchen Purcell Jackson, and Kyu Rhee. 2021 . The potential of artificial intelligence to improve patient safety: a scoping review. NPJ digital medicine 4, 1 (2021), 1–8. David W Bates, David Levine, Ania Syrowatka, Masha Kuznetsova, Kelly Jean Thomas Craig, Angela Rui, Gretchen Purcell Jackson, and Kyu Rhee. 2021. The potential of artificial intelligence to improve patient safety: a scoping review. NPJ digital medicine 4, 1 (2021), 1–8.
  3. Michelene TH Chi and Ruth Wylie . 2014. The ICAP framework: Linking cognitive engagement to active learning outcomes. Educational psychologist 49, 4 ( 2014 ), 219–243. Michelene TH Chi and Ruth Wylie. 2014. The ICAP framework: Linking cognitive engagement to active learning outcomes. Educational psychologist 49, 4 (2014), 219–243.
  4. Amit Dhurandhar Pin-Yu Chen Ronny Luss Chun-Chen Tu Paishun Ting Karthikeyan Shanmugam and Payel Das. 2018. Explanations based on the missing: Towards contrastive explanations with pertinent negatives. In Advances in Neural Information Processing Systems. 592–603. Amit Dhurandhar Pin-Yu Chen Ronny Luss Chun-Chen Tu Paishun Ting Karthikeyan Shanmugam and Payel Das. 2018. Explanations based on the missing: Towards contrastive explanations with pertinent negatives. In Advances in Neural Information Processing Systems. 592–603.
  5. Catherine O Fritz , Peter E Morris , and Jennifer J Richler . 2012 . Effect size estimates: current use, calculations, and interpretation.Journal of experimental psychology : General 141 , 1 (2012), 2 . Catherine O Fritz, Peter E Morris, and Jennifer J Richler. 2012. Effect size estimates: current use, calculations, and interpretation.Journal of experimental psychology: General 141, 1 (2012), 2.
  6. Laura Germine , Ken Nakayama , Bradley C Duchaine , Christopher F Chabris , Garga Chatterjee , and Jeremy B Wilmer . 2012. Is the Web as good as the lab? Comparable performance from Web and lab in cognitive/perceptual experiments. Psychonomic bulletin & review 19, 5 ( 2012 ), 847–857. Laura Germine, Ken Nakayama, Bradley C Duchaine, Christopher F Chabris, Garga Chatterjee, and Jeremy B Wilmer. 2012. Is the Web as good as the lab? Comparable performance from Web and lab in cognitive/perceptual experiments. Psychonomic bulletin & review 19, 5 (2012), 847–857.
  7. Ben Green . 2021. The Flaws of Policies Requiring Human Oversight of Government Algorithms. Available at SSRN ( 2021 ). Ben Green. 2021. The Flaws of Policies Requiring Human Oversight of Government Algorithms. Available at SSRN (2021).
  8. Barbara A Greene , Raymond B Miller , H Michael Crowson , Bryan L Duke , and Kristine L Akey . 2004. Predicting high school students ’ cognitive engagement and achievement: Contributions of classroom perceptions and motivation. Contemporary educational psychology 29, 4 ( 2004 ), 462–482. Barbara A Greene, Raymond B Miller, H Michael Crowson, Bryan L Duke, and Kristine L Akey. 2004. Predicting high school students’ cognitive engagement and achievement: Contributions of classroom perceptions and motivation. Contemporary educational psychology 29, 4 (2004), 462–482.
  9. Ece Kamar. 2016. Directions in Hybrid Intelligence: Complementing AI Systems with Human Intelligence.. In IJCAI. 4070–4073. Ece Kamar. 2016. Directions in Hybrid Intelligence: Complementing AI Systems with Human Intelligence.. In IJCAI. 4070–4073.
  10. Ece Kamar , Severin Hacker , and Eric Horvitz . 2012 . Combining human and machine intelligence in large-scale crowdsourcing . In Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems-Volume 1. International Foundation for Autonomous Agents and Multiagent Systems, 467–474 . Ece Kamar, Severin Hacker, and Eric Horvitz. 2012. Combining human and machine intelligence in large-scale crowdsourcing. In Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems-Volume 1. International Foundation for Autonomous Agents and Multiagent Systems, 467–474.
  11. Benjamin Krarup , Michael Cashmore , Daniele Magazzeni , and Tim Miller . 2019 . Model-based contrastive explanations for explainable planning . In ICAPS 2019 Workshop on Explainable AI Planning (XAIP). Benjamin Krarup, Michael Cashmore, Daniele Magazzeni, and Tim Miller. 2019. Model-based contrastive explanations for explainable planning. In ICAPS 2019 Workshop on Explainable AI Planning (XAIP).
  12. Stefan Münzer , Hubert D Zimmer , and Jörg Baus . 2012. Navigation assistance: A trade-off between wayfinding support and configural learning support.Journal of experimental psychology: applied 18, 1 ( 2012 ), 18. Stefan Münzer, Hubert D Zimmer, and Jörg Baus. 2012. Navigation assistance: A trade-off between wayfinding support and configural learning support.Journal of experimental psychology: applied 18, 1 (2012), 18.
  13. Jerome I Rotgans and Henk G Schmidt . 2011. Cognitive engagement in the problem-based learning classroom. Advances in health sciences education 16, 4 ( 2011 ), 465–479. Jerome I Rotgans and Henk G Schmidt. 2011. Cognitive engagement in the problem-based learning classroom. Advances in health sciences education 16, 4 (2011), 465–479.
  14. Maciej Tomczak and Ewa Tomczak . 2014. The need to report effect size estimates revisited. An overview of some recommended measures of effect size. Trends in sport sciences 1, 21 ( 2014 ), 19–25. Maciej Tomczak and Ewa Tomczak. 2014. The need to report effect size estimates revisited. An overview of some recommended measures of effect size. Trends in sport sciences 1, 21 (2014), 19–25.
  15. Bas Van Fraassen . 1988. The pragmatic theory of explanation. Theories of explanation 8 ( 1988 ), 135–155. Bas Van Fraassen. 1988. The pragmatic theory of explanation. Theories of explanation 8 (1988), 135–155.

Cited by

Sorted by citations

Metadata

Authors
Gajos, K. Z., & Mamykina, L.
Affiliations
  • Harvard University (US)
  • Columbia University (US)
Venue
27th International Conference on Intelligent User Interfaces
Publisher
ACM
Published
2022-03-22
Pages
794-806
OpenAlex
W4220961556
Cited by
127
References
50
Citations by year
2026: 34 · 2025: 47 · 2024: 22 · 2023: 21 · 2022: 3
FWCI
12.86
Type
conference paper
Language
en
Open access
Locations
  • ACM International Conference on Intelligent User Interfaces (IUI) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
National Science Foundation (IIS-2107391)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
  • Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
Keywords
incidental learning, AI assistance, AI explanations, AI recommendations, decision-making, cognitive engagement, advice taking, human-AI interaction

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review