The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Large-scale analysis of test–retest reliabilities of self-regulation measures
Open accessAbstract
The ability to regulate behavior in service of long-term goals is a widely studied psychological construct known as self-regulation. This wide interest is in part due to the putative relations between self-regulation and a range of real-world behaviors. Self-regulation is generally viewed as a trait, and individual differences are quantified using a diverse set of measures, including self-report surveys and behavioral tasks. Accurate characterization of individual differences requires measurement reliability, a property frequently characterized in self-report surveys, but rarely assessed in behavioral tasks. We remedy this gap by ( i ) providing a comprehensive literature review on an extensive set of self-regulation measures and ( ii ) empirically evaluating test–retest reliability of this battery in a new sample. We find that dependent variables (DVs) from self-report surveys of self-regulation have high test–retest reliability, while DVs derived from behavioral tasks do not. This holds both in the literature and in our sample, although the test–retest reliability estimates in the literature are highly variable. We confirm that this is due to differences in between-subject variability. We also compare different types of task DVs (e.g., model parameters vs. raw response times) in their suitability as individual difference DVs, finding that certain model parameters are as stable as raw DVs. Our results provide greater psychometric footing for the study of self-regulation and provide guidance for future studies of individual differences in this domain.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Nonetheless, custom- designed behavioural tasks can be less effective at measuring typical rather than extraordinary performance 54 .”
From Predictors and consequences of intellectual humility (2022) · cites this paper as [54]
In the citing paper
“This has partially been done for the cognitive control example (Henderson et al. 2012), but in general, standard laboratory tasks are not well optimized to capture this kind of endophenotype (Enkavi et al. 2019).”
From Deep brain stimulation for psychiatric disorders: From focal brain targets to cognitive networks (2021) · cites this paper as Enkavi et al. 2019
Citations
Cites
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148
Statistical Methods for Rates and Proportions
Fleiss, J. L., Levin, B., & Paik, M. C. (2003) · Wiley Series in Probability and Statistics · doi:10.1002/0471445428 · Cited by 18,646
Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., Howerter, A., & Wager, T. D. (2000) · Cognitive Psychology · doi:10.1006/cogp.1999.0734 · Cited by 15,798
An Integrative Theory of Prefrontal Cortex Function
Miller, E. K., & Cohen, J. D. (2001) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.24.1.167 · Cited by 12,803
Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393980 · Cited by 9,996
In search of how people change: Applications to addictive behaviors.
Prochaska, J. O., DiClemente, C. C., & Norcross, J. C. (1992) · American Psychologist · doi:10.1037/0003-066x.47.9.1102 · Cited by 8,596
The proof and measurement of association between two things
Spearman, C. (1904) · The American Journal of Psychology · doi:10.2307/1412159 · Cited by 8,039
Measures of Reliability in Sports Medicine and Science
Hopkins, W. G. (2000) · Sports Medicine · doi:10.2165/00007256-200030010-00001 · Cited by 4,635
Ratcliff, R. (1978) · Psychological Review · doi:10.1037/0033-295x.85.2.59 · Cited by 4,513
Running experiments on Amazon Mechanical Turk
Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010) · Judgment and Decision Making · doi:10.1017/s1930297500002205 · Cited by 3,767
Cited by
Self-regulation in childhood as a predictor of future outcomes: A meta-analytic review.
Robson, D. A., Allen, M. S., & Howard, S. J. (2020) · Psychological Bulletin · doi:10.1037/bul0000227 · Cited by 759
Uncovering the structure of self-regulation through data-driven ontology discovery
Eisenberg, I. W., Bissett, P. G., Zeynep Enkavi, A., Li, J., MacKinnon, D. P., Marsch, L. A., & Poldrack, R. A. (2019) · Nature Communications · doi:10.1038/s41467-019-10301-1 · Cited by 474
Self-citationMeasuring Adaptive Control in Conflict Tasks
Braem, S., Bugg, J. M., Schmidt, J. R., Crump, M. J. C., Weissman, D. H., Notebaert, W., & Egner, T. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.07.002 · Cited by 358
Predictors and consequences of intellectual humility
Porter, T., Elnakouri, A., Meyers, E. A., Shibayama, T., Jayawickreme, E., & Grossmann, I. (2022) · Nature Reviews Psychology · doi:10.1038/s44159-022-00081-9 · Cited by 248
Reasoning supports utilitarian resolutions to moral dilemmas across diverse measures.
Patil, I., Zucchelli, M. M., Kool, W., Campbell, S., Fornasier, F., Calò, M., Silani, G., Cikara, M., & Cushman, F. (2021) · Journal of Personality and Social Psychology · doi:10.1037/pspp0000281 · Cited by 133
Metadata
- Authors
- Enkavi, A. Z., Eisenberg, I. W., Bissett, P. G., Mazza, G. L., MacKinnon, D. P., Marsch, L. A., & Poldrack, R. A.
- Affiliations
- Stanford University (US)
- Mayo Clinic (US)
- Arizona State University (US)
- Dartmouth College (US)
- ORCID
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2019-03-06
- Volume
- 116
- Issue
- 12
- Pages
- 5472-5477
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC6431228
- OpenAlex
- W2920590181
- Cited by
- 489
- References
- 52
- Citations by year
- 2026: 48 · 2025: 60 · 2024: 65 · 2023: 63 · 2022: 81 · 2021: 64 · 2020: 73 · 2019: 34
- FWCI
- 13.16
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- HHS | National Institutes of Health (UH2DA041713)
- HHS | NIH | National Institute on Drug Abuse (P30DA029926)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Psychology › Applied Psychology › Behavioral Health and Interventions
- Social Sciences › Psychology › Social Psychology › Grit, Self-Efficacy, and Motivation
- Keywords
- self-regulation, test-retest reliability, individual differences, behavioral tasks, self-report questionnaire, measurement reliability, between-subject variability, reaction time (psychology), model parameters, trait measurement
- MeSH terms
- Self-Control, Humans, Individuality, Psychological Tests, Reproducibility of Results, Models, Statistical, Self Report
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Large-scale analysis of test–retest reliabilities of self-regulation measures. https://app.cochat.ai/papers/10.1073/pnas.1818430116
- The paper
- Enkavi, A. Z., Eisenberg, I. W., Bissett, P. G., Mazza, G. L., MacKinnon, D. P., Marsch, L. A., & Poldrack, R. A. (2019). Large-scale analysis of test–retest reliabilities of self-regulation measures. Proceedings of the National Academy of Sciences. https://doi.org/10.1073/pnas.1818430116
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.