Metacognitive Overload!: Positive and Negative Effects of Metacognitive Prompts in an Intelligent Tutoring System
Open accessThe DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Abstract
Research suggests that promoting metacognitive awareness can increase performance in, and learning from, intelligent tutoring systems (ITSs). The current work examines the effects of two metacognitive prompts within iSTART, a reading comprehension strategy ITS in which students practice writing quality self-explanations. In addition to comparing iSTART practice to a no-training control, those in the iSTART condition ( n = 116) were randomly assigned to a 2 (performance threshold: off, on) × 2(self-assessment: off, on) design. The performance threshold notified students when their average self-explanation score was below an experimenter-set threshold and the self-assessment prompted students to estimate their self-explanation score on the current trial. Students who practiced with iSTART had higher posttest self-explanation scores and inference comprehension scores on a transfer test than students in the no training control, replicating previous benefits for iSTART. However, there were no effects of either metacognitive prompt on these learning outcomes. In-system self-explanation scores indicated that the metacognitive prompts were detrimental to performance relative to standard iSTART practice. This study did not find benefits of metacognitive prompts in enhancing performance during practice or after the completion of training. Such findings support the idea that improving reading comprehension strategies comes from deliberate practice with actionable feedback rather than explicit metacognitive supports.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Metacognition and cognitive monitoring: A new area of cognitive–developmental inquiry
Flavell, J. H. (1979) · American Psychologist · doi:10.1037/0003-066x.34.10.906 · Cited by 11,282 · cited 2×
“Metacognition, or Bthinking about thinking,^ refers to processes related to evaluating what one knows (Flavell 1979).”
BackgroundThe role of deliberate practice in the acquisition of expert performance
Ericsson, K. A., Krampe, R. T., & Tesch-Römer, C. (1993) · Psychological Review · doi:10.1037/0033-295x.100.3.363 · Cited by 9,564 · cited 2×
“A third hypothesis is grounded in theories of skill acquisition (Ericsson et al. 1993; Healy et al. 1993).”
BackgroundThe Role of Goal Orientation in Self-Regulated Learning
Pintrich, P. R. (2000) · Handbook of Self-Regulation · doi:10.1016/b978-012109890-2/50043-3 · Cited by 5,173 · cited 2×
“Unfortunately, students often fail to engage in metacognitive reflection (Pintrich 2000; Varner et al. 2013; Zimmerman 2008; Zimmerman and Schunk 2001) and often inaccurately assess their own learning (Maki 1998).”
BackgroundZimmerman, B. J. (2008) · American Educational Research Journal · doi:10.3102/0002831207312909 · Cited by 3,735 · cited 2×
“Unfortunately, students often fail to engage in metacognitive reflection (Pintrich 2000; Varner et al. 2013; Zimmerman 2008; Zimmerman and Schunk 2001) and often inaccurately assess their own learning (Maki 1998).”
BackgroundHandbook of Metacognition in Education
Hacker, D. J., Dunlosky, J., & Graesser, A. C. (2009) · doi:10.4324/9780203876428 · Cited by 1,509
McNamara, D. S., Kintsch, E., Songer, N. B., & Kintsch, W. (1996) · Cognition and Instruction · doi:10.1207/s1532690xci1401_1 · Cited by 1,341 · cited 1×
“In contrast, inference questions probe for deeper comprehension as they require the reader to connect information across two or more sentences in the text to derive the answer (See McNamara et al. 1996, for additional information on the construction of these texts and questions).”
BackgroundSelf-citationSelf-Regulated Learning and Academic Achievement
(2013) · doi:10.4324/9781410601032 · Cited by 1,325
Eliciting Self‐Explanations Improves Understanding
Chi, M. T. H., De Leeuw, N., Chiu, M., & Lavancher, C. (1994) · Cognitive Science · doi:10.1207/s15516709cog1803_3 · Cited by 757
SERT: Self-Explanation Reading Training
McNamara, D. S. (2004) · Discourse Processes · doi:10.1207/s15326950dp3801_1 · Cited by 519 · cited 5×
“Comprehension monitoring allows students to evaluate their understanding of the text and to repair comprehension problems by using other strategies (McNamara, 2004; McNamara et al., 2007).”
BackgroundMethodInfluentialSelf-citationDevolder, A., van Braak, J., & Tondeur, J. (2012) · Journal of Computer Assisted Learning · doi:10.1111/j.1365-2729.2011.00476.x · Cited by 258 · cited 1×
“Indeed, in a variety of studies, self-explanation is identified as a type of metacognitive prompt (see Devolder et al. 2012).”
Background
Research works printed but not linked: 19
- Azevedo, 2005, Computer environments as metacognitive tools for enhancing learning, Educational Psychologist, 40, 193, 10.1207/s15326985ep4004_1 · DOI · Paper page
- Azevedo, 2005, Scaffolding self-regulated learning and metacognition–implications for the design of computer-based scaffolds, Instructional Science, 33, 367, 10.1007/s11251-005-1272-9 · DOI · Paper page
- Azevedo, R., Martin, S. A., Taub, M., Mudrick, N. V., Millar, G. C., & Grafsgaard, J. F. (2016, June). Are pedagogical agents’ external regulation effective in fostering learning with intelligent tutoring systems?. In International Conference on Intelligent Tutoring Systems (pp. 197-207). Springer, Cham. · DOI
- Bannert, 2009, Effects of a metacognitive support device in learning environments, Computers in Human Behavior, 25, 829, 10.1016/j.chb.2008.07.002 · DOI · Paper page
- Bates, 2015, Fitting linear mixed-effects models using lme4, Journal of Statistical Software, 67, 1, 10.18637/jss.v067.i01 · DOI · Paper page
- Falchikov, 1989, Student self-assessment in higher education: A meta-analysis, Review of Educational Research, 59, 395, 10.3102/00346543059004395 · DOI · Paper page
- Hagaman, 2008, The effects of the paraphrasing strategy on the reading comprehension of middle school students at risk for failure in reading, Remedial and Special Education, 29, 222, 10.1177/0741932507311638 · DOI
- Haller, 1988, Can comprehension be taught? A quantitative synthesis of “metacognitive” studies, Educational Researcher, 17, 5
- Healy, 1993, The long-term retention of knowledge and skills, Psychology of Learning and Motivation, 30, 135, 10.1016/S0079-7421(08)60296-0 · DOI
- Kirby, 1987, Metacognitive awareness about reading and its relation to reading ability, Journal of Psychoeducational Assessment, 5, 119, 10.1177/073428298700500204 · doi:10.1177/073428298700500204
- Kurby, 2012, Changing how students process and comprehend texts with computer-based self-explanation training, Journal of Educational Computing Research, 47, 429, 10.2190/EC.47.4.e · DOI
- Mathan, 2005, Fostering the intelligent novice: Learning from errors with metacognitive tutoring, Educational Psychologist, 40, 257, 10.1207/s15326985ep4004_7 · DOI · Paper page
- McNamara, 2004, iSTART: Interactive strategy trainer for active reading and thinking, Behavior Research Methods, Instruments, & Computers, 36, 222, 10.3758/BF03195567 · DOI · Paper page
- McNamara, 2006, Improving adolescent students' reading comprehension with iSTART, Journal of Educational Computing Research, 34, 147, 10.2190/1RU5-HDTJ-A5C8-JVWE · DOI · Paper page
- Palinscar, 1984, Reciprocal teaching of comprehension-fostering and comprehension-monitoring activities, Cognition and Instruction, 1, 117, 10.1207/s1532690xci0102_1 · DOI · Paper page
- Roll, 2011, Improving students’ help-seeking skills using metacognitive feedback in an intelligent tutoring system, Learning and Instruction, 21, 267, 10.1016/j.learninstruc.2010.07.004 · DOI · Paper page
- Schraw, 1994, The effect of metacognitive knowledge on local and global monitoring, Contemporary Educational Psychology, 19, 143, 10.1006/ceps.1994.1013 · DOI · Paper page
- Thiede, 2017, How restudy decisions affect overall comprehension for seventh-grade students, British Journal of Educational Psychology, 87, 590, 10.1111/bjep.12166 · DOI
- Varner, 2013, Evaluative misalignment of 10th-grade student and teacher criteria for essay quality: An automated textual analysis, Journal of Writing Research, 5, 35, 10.17239/jowr-2013.05.01.2 · DOI
Cited by
AI technologies for education: Recent research & future directions
Zhang, K., & Aslan, A. B. (2021) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2021.100025 · Cited by 923
Guo, L. (2022) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12650 · Cited by 141
Soofi, A. A., & Uddin, M. (2019) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2019.0100312 · Cited by 37
“2018 [26] Others Find metacognitive prompt in ITS iSTART (Interactive Strategy Training for Active Reading and Thinking) , NLP Web based Experimental study (pretest and posttest)”
BackgroundThomann, H., & Deutscher, V. (2025) · Educational Research Review · doi:10.1016/j.edurev.2025.100686 · Cited by 40
Joo, S., Han, I., & Park, I. (2026) · Educational Psychology · doi:10.1080/01443410.2026.2618717 · Cited by 5
Metadata
- Authors
- McCarthy, K. S., Likens, A. D., Johnson, A. M., Guerrero, T. A., & McNamara, D. S.
- Venue
- International Journal of Artificial Intelligence in Education
- Published
- 2018-02-21
- Volume
- 28
- Issue
- 3
- Pages
- 420-438
- ISSN
- 1560-4292 (ISSN-L)
- 1560-4306
- OpenAlex
- W2792490886
- Cited by
- 49
- References
- 12
- FWCI
- 1.84
- Type
- journal article
- Language
- en
- Open access
- hybrid
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- creativecommons.org/licenses/by-nc-nd/4.0/ (vor, 2798 d)
- Funders
- Institute of Education Sciences (R305A130124)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Metacognitive Overload!: Positive and Negative Effects of Metacognitive Prompts in an Intelligent Tutoring System. https://app.cochat.ai/papers/10.1007/s40593-018-0164-5
- The paper
- McCarthy, K. S., Likens, A. D., Johnson, A. M., Guerrero, T. A., & McNamara, D. S. (2018). Metacognitive Overload!: Positive and Negative Effects of Metacognitive Prompts in an Intelligent Tutoring System. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-018-0164-5
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.