CoChat

Metacognitive Overload!: Positive and Negative Effects of Metacognitive Prompts in an Intelligent Tutoring System

Open access
McCarthy, Kathryn S., Likens, Aaron D., Johnson, Amy M., Guerrero, Tricia A., McNamara, Danielle S. · 2018 · International Journal of Artificial Intelligence in Education
Cited by 49

The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.

Abstract

Research suggests that promoting metacognitive awareness can increase performance in, and learning from, intelligent tutoring systems (ITSs). The current work examines the effects of two metacognitive prompts within iSTART, a reading comprehension strategy ITS in which students practice writing quality self-explanations. In addition to comparing iSTART practice to a no-training control, those in the iSTART condition ( n = 116) were randomly assigned to a 2 (performance threshold: off, on) × 2(self-assessment: off, on) design. The performance threshold notified students when their average self-explanation score was below an experimenter-set threshold and the self-assessment prompted students to estimate their self-explanation score on the current trial. Students who practiced with iSTART had higher posttest self-explanation scores and inference comprehension scores on a transfer test than students in the no training control, replicating previous benefits for iSTART. However, there were no effects of either metacognitive prompt on these learning outcomes. In-system self-explanation scores indicated that the metacognitive prompts were detrimental to performance relative to standard iSTART practice. This study did not find benefits of metacognitive prompts in enhancing performance during practice or after the completion of training. Such findings support the idea that improving reading comprehension strategies comes from deliberate practice with actionable feedback rather than explicit metacognitive supports.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s40593-018-0164-5

Cites

12 of 12 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 12 references

Research works printed but not linked: 19

  1. Azevedo, 2005, Computer environments as metacognitive tools for enhancing learning, Educational Psychologist, 40, 193, 10.1207/s15326985ep4004_1 · DOI · Paper page
  2. Azevedo, 2005, Scaffolding self-regulated learning and metacognition–implications for the design of computer-based scaffolds, Instructional Science, 33, 367, 10.1007/s11251-005-1272-9 · DOI · Paper page
  3. Azevedo, R., Martin, S. A., Taub, M., Mudrick, N. V., Millar, G. C., & Grafsgaard, J. F. (2016, June). Are pedagogical agents’ external regulation effective in fostering learning with intelligent tutoring systems?. In International Conference on Intelligent Tutoring Systems (pp. 197-207). Springer, Cham. · DOI
  4. Bannert, 2009, Effects of a metacognitive support device in learning environments, Computers in Human Behavior, 25, 829, 10.1016/j.chb.2008.07.002 · DOI · Paper page
  5. Bates, 2015, Fitting linear mixed-effects models using lme4, Journal of Statistical Software, 67, 1, 10.18637/jss.v067.i01 · DOI · Paper page
  6. Falchikov, 1989, Student self-assessment in higher education: A meta-analysis, Review of Educational Research, 59, 395, 10.3102/00346543059004395 · DOI · Paper page
  7. Hagaman, 2008, The effects of the paraphrasing strategy on the reading comprehension of middle school students at risk for failure in reading, Remedial and Special Education, 29, 222, 10.1177/0741932507311638 · DOI
  8. Haller, 1988, Can comprehension be taught? A quantitative synthesis of “metacognitive” studies, Educational Researcher, 17, 5
  9. Healy, 1993, The long-term retention of knowledge and skills, Psychology of Learning and Motivation, 30, 135, 10.1016/S0079-7421(08)60296-0 · DOI
  10. Kirby, 1987, Metacognitive awareness about reading and its relation to reading ability, Journal of Psychoeducational Assessment, 5, 119, 10.1177/073428298700500204 · doi:10.1177/073428298700500204
  11. Kurby, 2012, Changing how students process and comprehend texts with computer-based self-explanation training, Journal of Educational Computing Research, 47, 429, 10.2190/EC.47.4.e · DOI
  12. Mathan, 2005, Fostering the intelligent novice: Learning from errors with metacognitive tutoring, Educational Psychologist, 40, 257, 10.1207/s15326985ep4004_7 · DOI · Paper page
  13. McNamara, 2004, iSTART: Interactive strategy trainer for active reading and thinking, Behavior Research Methods, Instruments, & Computers, 36, 222, 10.3758/BF03195567 · DOI · Paper page
  14. McNamara, 2006, Improving adolescent students' reading comprehension with iSTART, Journal of Educational Computing Research, 34, 147, 10.2190/1RU5-HDTJ-A5C8-JVWE · DOI · Paper page
  15. Palinscar, 1984, Reciprocal teaching of comprehension-fostering and comprehension-monitoring activities, Cognition and Instruction, 1, 117, 10.1207/s1532690xci0102_1 · DOI · Paper page
  16. Roll, 2011, Improving students’ help-seeking skills using metacognitive feedback in an intelligent tutoring system, Learning and Instruction, 21, 267, 10.1016/j.learninstruc.2010.07.004 · DOI · Paper page
  17. Schraw, 1994, The effect of metacognitive knowledge on local and global monitoring, Contemporary Educational Psychology, 19, 143, 10.1006/ceps.1994.1013 · DOI · Paper page
  18. Thiede, 2017, How restudy decisions affect overall comprehension for seventh-grade students, British Journal of Educational Psychology, 87, 590, 10.1111/bjep.12166 · DOI
  19. Varner, 2013, Evaluative misalignment of 10th-grade student and teacher criteria for essay quality: An automated textual analysis, Journal of Writing Research, 5, 35, 10.17239/jowr-2013.05.01.2 · DOI

Cited by

See all 49

Metadata

Authors
McCarthy, K. S., Likens, A. D., Johnson, A. M., Guerrero, T. A., & McNamara, D. S.
Venue
International Journal of Artificial Intelligence in Education
Published
2018-02-21
Volume
28
Issue
3
Pages
420-438
ISSN
  • 1560-4292 (ISSN-L)
  • 1560-4306
OpenAlex
W2792490886
Cited by
49
References
12
FWCI
1.84
Type
journal article
Language
en
Open access
hybrid
Licence
Funders
Institute of Education Sciences (R305A130124)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Metacognitive Overload!: Positive and Negative Effects of Metacognitive Prompts in an Intelligent Tutoring System. https://app.cochat.ai/papers/10.1007/s40593-018-0164-5
The paper
McCarthy, K. S., Likens, A. D., Johnson, A. M., Guerrero, T. A., & McNamara, D. S. (2018). Metacognitive Overload!: Positive and Negative Effects of Metacognitive Prompts in an Intelligent Tutoring System. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-018-0164-5

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.