CoChat
Sign Up
PreprintCited by 3Intelligent Tutoring Systems and Adaptive LearningInnovative Teaching and Learning MethodsVisual and Cognitive Learning Processes

The Power of Nudging: Exploring Three Interventions for Metacognitive Skills Instruction across Intelligent Tutoring Systems

Open access
Abdelshiheed, Mark, Hostetter, John Wesley, Shabrina, Preya, Barnes, Tiffany, Chi, Min·2023·arXiv
DOI 10.48550/arxiv.2303.11965arXiv 2303.11965
Open this paper in CoChat — free accountOpen PDF

Abstract

Deductive domains are typical of many cognitive skills in that no single problem-solving strategy is always optimal for solving all problems. It was shown that students who know how and when to use each strategy (StrTime) outperformed those who know neither and stick to the default strategy (Default). In this work, students were trained on a logic tutor that supports a default forward-chaining and a backward-chaining (BC) strategy, then a probability tutor that only supports BC. We investigated three types of interventions on teaching the Default students how and when to use which strategy on the logic tutor: Example, Nudge and Presented. Meanwhile, StrTime students received no interventions. Overall, our results show that Nudge outperformed their Default peers and caught up with StrTime on both tutors.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2303.11965

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • In the context of solving logical puzzles, intelligent tutors should offer backwards-oriented workflows rather than focusing on forward-oriented workflows.

    “For example, [1] and [164] suggest that in the context of solving logical puzzles, intelligent tutors should offer backwards-oriented workflows (e.g., using prompts to encourage thinking about the negation of the actual solution), rather than focusing on forward-oriented workflows.”

    From The Metacognitive Demands and Opportunities of Generative AI (2024) · cites this paper as [1]

    The “Two Tutors and Our Interventions” section describes a logic tutor that offers backward chaining, including prompts to switch from its default forward-chaining approach; backward chaining can involve negating the conclusion. But the Abstract says no single strategy is always optimal, and the paper investigates teaching students when to use each strategy. It does not recommend backward-oriented workflows in place of a focus on forward-oriented ones generally. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 31

  1. Abdelshiheed, M., Maniktala, M., Barnes, T., & Chi, M. (2022). Assessing competency using metacognition and motivation: The role of time-awareness in preparation for future learning. In Design recommendations for intelligent tutoring systems (Vol. 9, pp. 121-131).
  2. Abdelshiheed, M., Maniktala, M., Ju, S., Jain, A., Barnes, T., & Chi, M. (2021). Preparing unprepared students for future learning. In Proceedings of the 43rd annual conference of the cognitive science society (Vol. 43).
  3. Abdelshiheed, M., Zhou, G., Maniktala, M., Barnes, T., & Chi, M. (2020). Metacognition and motivation: The role of time-awareness in preparation for future learning. In Proceedings of the 42nd annual conference of the cognitive science society (Vol. 42).
  4. Belenky, D. M., & Nokes, T. J. (2009). Examining the role of manipulatives and metacognition on engagement, learning, and transfer. The Journal of Problem Solving, 2(2), 6.
  5. Belenky, D. M., & Nokes-Malach, T. J. (2012). Motivation and transfer: The role of mastery-approach goals in prepa- ration for future learning. Journal of the Learning Sciences, 21(3), 399-432.
  6. Bransford, J. D., & Schwartz, D. L. (1999). Rethinking trans- fer: A simple proposal with multiple implications. Review of research in education, 24(1), 61-100.
  7. Cardelle-Elawar, M. (1992). Effects of teaching metacogni- tive skills to students with low mathematics ability. Teach- ing and teacher education, 8(2), 109-121.
  8. Chi, M., & VanLehn, K. (2007). The impact of explicit strat- egy instruction on problem-solving behaviors across intel- ligent tutoring systems. In Proceedings of the annual meet- ing of the cognitive science society (Vol. 29).
  9. Chi, M., & VanLehn, K. (2008). Eliminating the gap between the high and low students through meta-cognitive strategy instruction. In Intelligent tutoring systems (pp. 603-613).
  10. Chi, M., & VanLehn, K. (2010). Meta-cognitive strategy instruction in intelligent tutoring systems: How, when, and why. Educational Technology & Society, 13(1), 25-39.
  11. Chi, M. T. (2009). Active-constructive-interactive: A concep- tual framework for differentiating learning activities. Top- ics in cognitive science, 1(1), 73-105.
  12. Chi, M. T., & Wylie, R. (2014). The icap framework: Linking cognitive engagement to active learning outcomes. Educa- tional psychologist, 49(4), 219-243.
  13. de Boer, H., et al. (2018). Long-term effects of metacognitive strategy instruction on student academic performance: A meta-analysis. Educational Research Review, 24, 98-115.
  14. Fazio, L. K., et al. (2016). Strategy use and strategy choice in fraction magnitude comparison. Journal of Experimental Psychology: Learning, Memory, and Cognition, 42(1), 1.
  15. Fellman, D., Jylkkä, J., Waris, O., Soveri, A., Ritakallio, L., Haga, S., . . . Laine, M. (2020). The role of strategy use in working memory training outcomes. Journal of Memory and Language, 110, 104064.
  16. Goldstein, D. G., et al. (2008). Nudge your customers toward better choices. Harvard Business Review, 86(12), 99-105.
  17. Gosling, S. D., Rentfrow, P. J., & Swann Jr, W. B. (2003). A very brief measure of the big-five personality domains. Journal of Research in personality, 37(6), 504-528.
  18. Lee, K. R., & Oxford, R. (2008). Understanding efl learners' strategy use and strategy awareness. The Asian EFL Jour- nal Quarterly March 2008 Volume 10, Issue, 10(1), 7-32.
  19. Priest, A., & Lindsay, R. (1992). New light on novice-expert differences in physics problem solving. British journal of Psychology, 83(3), 389-405.
  20. Richey, J. E., et al. (2015). Transfer effects of prompted and self-reported analogical comparison and self-explanation. In Proceedings of the annual meeting of the cognitive sci- ence society (Vol. 37).
  21. Russell, S. J., & Norvig, P. (2020). Artificial intelligence: a modern approach (4th ed.). Pearson.
  22. Schraw, G., & Gutierrez, A. P. (2015). Metacognitive strat- egy instruction that highlights the role of monitoring and control processes. In Metacognition: Fundaments, appli- cations, and trends (pp. 3-16). Springer.
  23. Schwartz, D. L., & Martin, T. (2004). Inventing to prepare for future learning: The hidden efficiency of encouraging orig- inal student production in statistics instruction. Cognition and instruction, 22(2), 129-184.
  24. Simon, C., & Tagliabue, M. (2018). Feeding the behavioral revolution: Contributions of behavior analysis to nudging and vice versa. Journal of Behavioral Economics for Pol- icy, 2(1), 91-97.
  25. Smith, N. C., et al. (2013). Choice without awareness: Eth- ical and policy implications of defaults. Journal of Public Policy & Marketing, 32(2), 159-172.
  26. Spörer, N., et al. (2009). Improving students' reading com- prehension skills: Effects of strategy instruction and recip- rocal teaching. Learning and instruction, 19(3), 272-286.
  27. Teng, F. (2020). The benefits of metacognitive reading strat- egy awareness instruction for young learners of english as a second language. Literacy, 54(1), 29-39.
  28. Vanlehn, K. (2006). The behavior of tutoring systems. In- ternational journal of artificial intelligence in education, 16(3), 227-265.
  29. Winne, P. H., & Azevedo, R. (2014). Metacognition. In R. K. Sawyer (Ed.), The cambridge handbook of the learn- ing sciences (2nd ed.). Cambridge University Press.
  30. Zepeda, C. D., et al. (2015). Direct instruction of metacogni- tion benefits adolescent science learning, transfer, and mo- tivation: An in vivo study. Journal of Educational Psychol- ogy, 107(4), 954.
  31. Zimmerman, B. J. (1990). Self-regulated learning and aca- demic achievement: An overview. Educational psycholo- gist, 25(1), 3-17.

Cited by

Sorted by citations

Authors
Abdelshiheed, M., Hostetter, J. W., Shabrina, P., Barnes, T., & Chi, M.
Affiliations
North Carolina State University (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2023
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2303.11965
arXiv
2303.11965
OpenAlex
W7094358784
Cited by
3
References
0
Citations by year
2024: 1 · 2023: 1 · 2022: 1
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • eScholarship (California Digital Library) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
National Science Foundation (2013502, CAREER, 1660878, 1651909)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
  • Social Sciences › Psychology › Developmental and Educational Psychology › Innovative Teaching and Learning Methods
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Visual and Cognitive Learning Processes
Keywords
nudging, intelligent tutoring systems, strategy selection, forward chaining, backward chaining, deductive reasoning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review