Metacognitive scaffolds improve self-judgments of accuracy in a medical intelligent tutoring system
Open accessThe DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
In this study, we examined the effect of two metacognitive scaffolds on the accuracy of confidence judgments made while diagnosing dermatopathology slides in SlideTutor. Thirty-one (N = 31) first- to fourth-year pathology and dermatology residents were randomly assigned to one of the two scaffolding conditions. The cases used in this study were selected from the domain of nodular and diffuse dermatitides. Both groups worked with a version of SlideTutor that provided immediate feedback on their actions for 2 h before proceeding to solve cases in either the Considering Alternatives or Playback condition. No immediate feedback was provided on actions performed by participants in the scaffolding mode. Measurements included learning gains (pre-test and post-test), as well as metacognitive performance, including Goodman–Kruskal Gamma correlation, bias, and discrimination. Results showed that participants in both conditions improved significantly in terms of their diagnostic scores from pre-test to post-test. More importantly, participants in the Considering Alternatives condition outperformed those in the Playback condition in the accuracy of their confidence judgments and the discrimination of the correctness of their assertions while solving cases. The results suggested that presenting participants with their diagnostic decision paths and highlighting correct and incorrect paths helps them to become more metacognitively accurate in their confidence judgments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Role of Goal Orientation in Self-Regulated Learning
Pintrich, P. R. (2000) · Handbook of Self-Regulation · doi:10.1016/b978-012109890-2/50043-3 · Cited by 5,173 · cited 2×
“Pintrich (2000) defines SRL as a constructive process where learners set goals based on their prior knowledge and experience and their current learning context.”
BackgroundVanLEHN, K. (2011) · Educational Psychologist · doi:10.1080/00461520.2011.611369 · Cited by 2,323
Studying as Self-Regulated Learning
(1998) · Metacognition in Educational Theory and Practice · doi:10.4324/9781410602350-19 · Cited by 1,589 · cited 1×
“Active monitoring and paying attention to the playback of one’s problem-solving steps would lead to a change in the learner’s internal standards (Winne and Hadwin 1998, 2008) toward becoming more accurate in future self-assessments of performance.”
BackgroundDiagnostic error in internal medicine
Graber, M. L., Franklin, N., & Gordon, R. (2005) · Archives of Internal Medicine · doi:10.1001/archinte.165.13.1493 · Cited by 1,549
Handbook of Metacognition in Education
Hacker, D. J., Dunlosky, J., & Graesser, A. C. (2009) · doi:10.4324/9780203876428 · Cited by 1,509 · cited 2×
“After the learner sets goals, she monitors both her domain knowledge and learning resources (Azevedo and Witherspoon 2009).”
BackgroundSelf-Regulated Learning and Academic Achievement
(2013) · doi:10.4324/9781410601032 · Cited by 1,325
A comparison of current measures of the accuracy of feeling-of-knowing predictions
Nelson, T. O. (1984) · Psychological Bulletin · doi:10.1037/0033-2909.95.1.109 · Cited by 976
Chi, M. T. H., Siler, S. A., Jeong, H., Yamauchi, T., & Hausmann, R. G. (2001) · Cognitive Science · doi:10.1207/s15516709cog2504_1 · Cited by 855
Eliciting Self‐Explanations Improves Understanding
Chi, M. T. H., De Leeuw, N., Chiu, M., & Lavancher, C. (1994) · Cognitive Science · doi:10.1207/s15516709cog1803_3 · Cited by 757
Exploring the Assistance Dilemma in Experiments with Cognitive Tutors
Koedinger, K. R., & Aleven, V. (2007) · Educational Psychology Review · doi:10.1007/s10648-007-9049-0 · Cited by 632 · cited 2×
“ITSs support ‘‘leaning-by-doing,’’ and provide individualized support, point out errors, and organize content to cater to the needs of the individual when the teacher has limited time to spend (Corbett et al. 2002; Koedinger and Aleven 2007; VanLehn 2006, 2011).”
Background
Cited by
van de Pol, J., Volman, M., Oort, F., & Beishuizen, J. (2015) · Instructional Science · doi:10.1007/s11251-015-9351-z · Cited by 220
“The metaphor of scaffolding is derived from mother–child observations and has been applied to many other contexts, such as computer environments (Azevedo and Hadwin 2005; Cuevas et al. 2002; Feyzi-Behnagh et al. 2013; Rasku-Puttonen et al. 2003; Simons and Klein 2007), tutoring settings (e.”
MethodDual-process cognitive interventions to enhance diagnostic reasoning: a systematic review
Lambe, K. A., O'Reilly, G., Kelly, B. D., & Curristan, S. (2016) · BMJ Quality & Safety · doi:10.1136/bmjqs-2015-004417 · Cited by 215
Fontaine, G., Cossette, S., Maheu-Cadotte, M.-A., Mailhot, T., Deschênes, M.-F., Mathieu-Dupuis, G., Côté, J., Gagnon, M.-P., & Dubé, V. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-025252 · Cited by 193
Metadata
- Authors
- Feyzi-Behnagh, R., Azevedo, R., Legowski, E., Reitmeyer, K., Tseytlin, E., & Crowley, R. S.
- Venue
- Instructional Science
- Published
- 2013-04-11
- Volume
- 42
- Issue
- 2
- Pages
- 159-181
- ISSN
- 0020-4277 (ISSN-L)
- 1573-1952
- OpenAlex
- W2033127427
- Cited by
- 58
- References
- 14
- FWCI
- 4.05
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Metacognitive scaffolds improve self-judgments of accuracy in a medical intelligent tutoring system. https://app.cochat.ai/papers/10.1007/s11251-013-9275-4
- The paper
- Feyzi-Behnagh, R., Azevedo, R., Legowski, E., Reitmeyer, K., Tseytlin, E., & Crowley, R. S. (2014). Metacognitive scaffolds improve self-judgments of accuracy in a medical intelligent tutoring system. Instructional Science. https://doi.org/10.1007/s11251-013-9275-4
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.