A stochastic detection and retrieval model for the study of metacognition
The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
Abstract
We present a signal detection-like model termed the stochastic detection and retrieval model (SDRM) for use in studying metacognition. Focusing on paradigms that relate retrieval (e.g., recall or recognition) and confidence judgments, the SDRM measures (1) variance in the retrieval process, (2) variance in the confidence process, (3) the extent to which different sources of information underlie each response, (4) simple bias (i.e., increasing or decreasing confidence criteria across conditions), and (5) metacognitive bias (i.e., contraction or expansion of the confidence criteria across conditions). In the metacognition literature, gamma correlations have been used to measure the accuracy of confidence judgments. However, gamma cannot distinguish between the first 3 attributes, and it cannot measure either form of bias. In contrast, the SDRM can distinguish among the attributes, and it can measure both forms of bias. In this way, the SDRM can be used to test competing process theories by determining the attribute that best accounts for a change across conditions. To demonstrate the SDRM's usefulness, we investigated judgments of learning (JOLs) followed by cued-recall. Through a series of nested and non-nested model comparisons applied to a new experiment, the SDRM determined that a reduction in variance during the confidence process is the most likely explanation of the delayed-JOL effect, and a stronger relation between information underlying JOLs and recall is the most likely explanation of the testing-JOL effect. Following a brief discussion of implications for JOL theories, we conclude with a broader discussion of how the SDRM can benefit metacognition research.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“A similar bias-free approach to modeling metacognitive accuracy is the “Stochastic Detection and Retrieval Model” (SDRM) introduced by Jang et al. (2012). The SDRM not only measures metacognitive accuracy, but is also able to model different potential causes of metacognitive inaccuracy.”
From How to measure metacognition (2014) · cites this paper as Jang et al. (2012)
In the citing paper
“many of these “classic” findings in the metacognition rely on measures such as G (Rhodes and Tauber, 2011) that may be confounded by bias and performance effects (although see Jang et al., 2012).”
From How to measure metacognition (2014) · cites this paper as Jang et al., 2012
Citations
Cites
Estimating the Dimension of a Model
Schwarz, G. (1978) · The Annals of Statistics · doi:10.1214/aos/1176344136 · Cited by 47,033
(1966) · Medical Entomology and Zoology · Cited by 10,293
Detection Theory: A User's Guide
Macmillan, N. A., & Creelman, C. D. (1991) · Cited by 5,415
Metacognition: Knowing about Knowing
(1994) · The MIT Press eBooks · doi:10.7551/mitpress/4561.001.0001 · Cited by 1,720
Monitoring and control processes in the strategic regulation of memory accuracy.
Koriat, A., & Goldsmith, M. (1996) · Psychological Review · doi:10.1037//0033-295x.103.3.490 · Cited by 1,065
A comparison of current measures of the accuracy of feeling-of-knowing predictions
Nelson, T. O. (1984) · Psychological Bulletin · doi:10.1037/0033-2909.95.1.109 · Cited by 976
Why Investigate Metacognition?
Nelson, T. O., & Narens, L. (1994) · Metacognition · doi:10.7551/mitpress/4561.003.0003 · Cited by 750
Indices of discrimination or diagnostic accuracy: Their ROCs and implied models.
Swets, J. A. (1986) · Psychological Bulletin · doi:10.1037//0033-2909.99.1.100 · Cited by 582
The mismeasure of memory: When retrieval fluency is misleading as a metamnemonic index.
Benjamin, A. S., Bjork, R. A., & Schwartz, B. L. (1998) · Journal of Experimental Psychology: General · doi:10.1037//0096-3445.127.1.55 · Cited by 529
Swets, J. A. (1986) · Psychological Bulletin · doi:10.1037//0033-2909.99.2.181 · Cited by 508
Galvin, S. J., Podd, J. V., Drga, V., & Whitmore, J. (2003) · Psychonomic Bulletin & Review · doi:10.3758/bf03196546 · Cited by 405
Multinomial processing models of source monitoring.
Batchelder, W. H., & Riefer, D. M. (1990) · Psychological Review · doi:10.1037//0033-295x.97.4.548 · Cited by 403
Memory and metamemory: a study of the feeling-of-knowing phenomenon in amnesic patients
Shimamura, A. P., & Squire, L. R. (1986) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/0278-7393.12.3.452 · Cited by 294
Koriat, A., Sheffer, L., & Ma’ayan, H. (2002) · Journal of Experimental Psychology General · doi:10.1037//0096-3445.131.2.147 · Cited by 285
Accounts of the confidence-accuracy relation in recognition memory
Busey, T. A., Tunnicliff, J., Loftus, G. R., & Loftus, E. F. (2000) · Psychonomic Bulletin & Review · doi:10.3758/bf03210724 · Cited by 277
Assessing model mimicry using the parametric bootstrap
Wagenmakers, E., Ratcliff, R., Gómez, P., & Iverson, G. (2004) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2003.11.004 · Cited by 184
Masson, M. E. J., & Rotello, C. M. (2009) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/a0014876 · Cited by 173
Two Types of ROC Curves and Definitions of Parameters
Clarke, F. R., Birdsall, T. G., & Tanner, W. P. (1959) · The Journal of the Acoustical Society of America · doi:10.1121/1.1907764 · Cited by 168
When Predictions Create Reality: Judgments of Learning May Alter What They Are Intended to Assess
Spellman, B. A., & Bjork, R. A. (1992) · Psychological Science · doi:10.1111/j.1467-9280.1992.tb00680.x · Cited by 166
Judgments of Knowing: The Influence of Retrieval Practice
King, J. F., Zechmeister, E. B., & Shaughnessy, J. J. (1980) · The American Journal of Psychology · doi:10.2307/1422236 · Cited by 166
Similarity relations in recognition
Tulving, E. (1981) · Journal of Verbal Learning and Verbal Behavior · doi:10.1016/s0022-5371(81)90129-8 · Cited by 165 · cited 1×
“This perspective is consistent with results in recognition memory research suggesting that different memory strength distributions underlie old/new decisions and confidence judgments (e.g., Busey, Tunnicliff, Loftus, & Loftus, 2000; Tulving, 1981; Van Zandt, 2000).”
ResultRatcliff, R., McKoon, G., & Tindall, M. (1994) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037//0278-7393.20.4.763 · Cited by 156
Metamemory: Monitoring future recallability during study.
Lovelace, E. A. (1984) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037//0278-7393.10.4.756 · Cited by 138
Signal detection with criterion noise: Applications to recognition memory.
Benjamin, A. S., Diaz, M., & Wee, S. (2009) · Psychological Review · doi:10.1037/a0014351 · Cited by 136
Delaying judgments of learning affects memory, not metamemory
Kimball, D. R., & Metcalfe, J. (2003) · Memory & Cognition · doi:10.3758/bf03196445 · Cited by 128
Schunn, C. D., Reder, L. M., Nhouyvanisvong, A., & Richards, D. R. (1997) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037//0278-7393.23.1.3 · Cited by 126
Statement verification: A stochastic model of judgment and response.
Wallsten, T. S., & González-Vallejo, C. (1994) · Psychological Review · doi:10.1037//0033-295x.101.3.490 · Cited by 125
Self-citationItem Recognition Memory and the Receiver Operating Characteristic.
Heathcote, A. (2003) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/0278-7393.29.6.1210 · Cited by 123
A Revised Methodology for Research on Metamemory: Pre-judgment Recall And Monitoring (PRAM).
Nelson, T. O., Narens, L., & Dunlosky, J. (2004) · Psychological Methods · doi:10.1037/1082-989x.9.1.53 · Cited by 122
Type I error rates and power analyses for single-point sensitivity measures
Rotello, C. M., Masson, M. E. J., & Verde, M. F. (2008) · Perception & Psychophysics · doi:10.3758/pp.70.2.389 · Cited by 83
Effects of alcohol intoxication on metamemory and on retrieval from long-term memory.
Nelson, T. O., McSpadden, M., Fromme, K., & Marlatt, G. A. (1986) · Journal of Experimental Psychology: General · doi:10.1037//0096-3445.115.3.247 · Cited by 82
Knowledge of one's own responding and the relation of such knowledge to learning
Bisanz, G. L., Vesonder, G., & Voss, J. F. (1978) · Journal of Experimental Child Psychology · doi:10.1016/0022-0965(78)90042-5 · Cited by 74
Memory for remembered events: An assessment of output monitoring in free recall
Gardiner, J. M., & Klee, H. (1976) · Journal of Verbal Learning and Verbal Behavior · doi:10.1016/0022-5371(76)90021-9 · Cited by 53
A Model for Stochastic Drift in Memory Strength to Account for Judgments of Learning.
Sikström, S., & Jönsson, F. U. (2005) · Psychological Review · doi:10.1037/0033-295x.112.4.932 · Cited by 36
Memory for remembered events: Contrasting recall and recognition
Klee, H., & GARDINER, J. (1976) · Journal of Verbal Learning and Verbal Behavior · doi:10.1016/s0022-5371(76)90042-6 · Cited by 32
Jang, Y., Wixted, J. T., & Huber, D. E. (2011) · Psychonomic Bulletin & Review · doi:10.3758/s13423-011-0096-7 · Cited by 30
Self-citationCan decision criteria interchange locations? Some positive evidence.
Treisman, M., & Faulkner, A. (1985) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037/0096-1523.11.2.187 · Cited by 17
On Indices of Signal and Response Discriminability
Pollack, I. (1959) · The Journal of the Acoustical Society of America · doi:10.1121/1.1907802 · Cited by 17
Robinson, J. A., & Kulp, R. A. (1970) · Journal of Verbal Learning and Verbal Behavior · doi:10.1016/s0022-5371(70)80012-3 · Cited by 14
(2005)
Cited by
Fleming, S. M., & Lau, H. C. (2014) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2014.00443 · Cited by 1,408
Self-Evaluation of Decision-Making: A General Bayesian Framework for Metacognitive Computation
Fleming, S. M., & Daw, N. D. (2017) · Psychological Review · doi:10.1037/rev0000045 · Cited by 584
HMeta-d: hierarchical Bayesian estimation of metacognitive efficiency from confidence ratings
Fleming, S. M. (2017) · Neuroscience of Consciousness · doi:10.1093/nc/nix007 · Cited by 309
Distinguishing the Roles of Dorsolateral and Anterior PFC in Visual Metacognition
Shekhar, M., & Rahnev, D. (2018) · The Journal of Neuroscience · doi:10.1523/jneurosci.3484-17.2018 · Cited by 154
Mazancieux, A., Fleming, S. M., Souchay, C., & Moulin, C. J. A. (2020) · Journal of Experimental Psychology: General · doi:10.1037/xge0000746 · Cited by 131
Metadata
- Authors
- Jang, Y., Wallsten, T. S., & Huber, D. E.
- Affiliations
- University of California San Diego (US)
- ORCID
- Huber, David E.
- Venue
- Psychological Review
- Published
- 2012
- Volume
- 119
- Issue
- 1
- Pages
- 186-200
- ISSN
- 0033-295X (ISSN-L)
- 1939-1471
- OpenAlex
- W2051602937
- Cited by
- 59
- References
- 40
- Citations by year
- 2026: 3 · 2025: 1 · 2024: 5 · 2023: 9 · 2022: 3 · 2021: 5 · 2020: 8 · 2019: 4 · 2018: 6 · 2017: 4 · 2016: 2 · 2015: 3 · 2014: 4 · 2013: 1 · 2012: 1
- FWCI
- 0.97
- Type
- journal article
- Language
- en
- Update notices
- correction (2012-01-09)
- Open access
- closed
- Locations
- Psychological Review (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institute of Mental Health (RMH081084A)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory Processes and Influences
- Social Sciences › Psychology › Developmental and Educational Psychology › Educational Strategies and Epistemologies
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Cognitive Functions and Memory
- Keywords
- metacognition, confidence judgments, metacognitive bias, signal detection theory, judgments of learning, cued recall, recognition memory
- MeSH terms
- Cognition, Cues, Data Interpretation, Statistical, Humans, Judgment, Learning, Models, Psychological, Mental Recall, Stochastic Processes, Time Factors, Signal Detection, Psychological, Recognition, Psychology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.