Why Are Self-Report and Behavioral Measures Weakly Correlated?
Open accessPublished version of a preprint
Abstract
Accumulating evidence indicates weak correlations between self-report and behavioral measures of the same construct. We suggest that these weak correlations result from the poor reliability of many behavioral measures and the distinct response processes involved in the two measurement types. We also describe how researchers can benefit from appropriate use of these measures.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Laboratory and neuropsychological measures of flexibility have high construct validity but do not always match real-world flexible behaviour reported by self- or informant-report questionnaires.
“laboratory-based measures and neuropsychological tests have high construct validity but may not always converge with real-world flexible behaviours as indexed using self-report or informant-report questionnaires, which typically have greater ecological validity21.”
From Cognitive and behavioural flexibility: Neural mechanisms and clinical considerations (2021) · cites this paper as [21]
Informant-report questionnaires typically have greater ecological validity, while laboratory-based measures can provide greater construct validity for assessing executive function and cognitive flexibility.
“EF ability and cognitive flexibility can be assessed using informant-report questionnaires, which typically have greater ecological validity, or laboratory-based measures, which can provide greater construct validity(5).”
From Brain Mechanisms Supporting Flexible Cognition and Behavior in Adolescents With Autism Spectrum Disorder (2021) · cites this paper as [5]
Self-report and behavioral measures of the same construct are only weakly correlated, possibly because behavioral measures tap responses during highly structured situations and index maximal performance, whereas self-report assesses behavior across unstructured real-life situations.
“More generally, self-report and behavioral measures of the same construct are only weakly correlated, possibly due to the fact that behavioral measures tap responses during highly structured situations and typically index maximal performance, whereas self-report assesses how individuals behave across unstructured real-life situations (5).”
From Brain Mechanisms Supporting Flexible Cognition and Behavior in Adolescents With Autism Spectrum Disorder (2021) · cites this paper as [5]
Measures with high reliability such as informant or self-report may better predict individual differences in real-life outcomes, whereas behavioral measures sensitive to within-person experimental manipulations may be important for studying processes that underlie task performance.
“Measures with high reliability such as informant- or self-report may better predict individual differences in real-life outcomes, whereas behavioral measures that are sensitive to within-person experimental manipulations may be important for studying processes that underlie task performance (5).”
From Brain Mechanisms Supporting Flexible Cognition and Behavior in Adolescents With Autism Spectrum Disorder (2021) · cites this paper as [5]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Emotional intelligence: An integrative meta-analysis and cascading model
Joseph, D. L., & Newman, D. A. (2010) · Journal of Applied Psychology · doi:10.1037/a0017286 · Cited by 1,722 · cited 2× · see citing works
“For example, the average correlation between self-report and behavioral measures of self-control [1,2], emotional intelligence [3], empathy [4], risk preference [5], and creativity [6] ranged from 0 to 0.”
BackgroundThe reliability paradox: Why robust cognitive tasks do not produce reliable individual differences
Hedge, C., Powell, G., & Sumner, P. (2018) · Behavior Research Methods · doi:10.3758/s13428-017-0935-1 · Cited by 1,457 · see citing works
Karr, J. E., Areshenkoff, C. N., Rast, P., Hofer, S. M., Iverson, G. L., & Garcia-Barrera, M. A. (2018) · Psychological Bulletin · doi:10.1037/bul0000160 · Cited by 637 · see citing works
Noble, S., Scheinost, D., & Constable, R. T. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.116157 · Cited by 560 · see citing works
Risk preference shares the psychometric structure of major psychological traits
Frey, R., Pedroni, A., Mata, R., Rieskamp, J., & Hertwig, R. (2017) · Science Advances · doi:10.1126/sciadv.1701381 · Cited by 517 · see citing works
Uncovering the structure of self-regulation through data-driven ontology discovery
Eisenberg, I. W., Bissett, P. G., Enkavi, A. Z., Li, J. K., MacKinnon, D. P., Marsch, L. A., & Poldrack, R. A. (2019) · Nature Communications · doi:10.1038/s41467-019-10301-1 · Cited by 473 · see citing works
The Implicit Association Test: A Method in Search of a Construct
Schimmack, U. (2019) · Perspectives on Psychological Science · doi:10.1177/1745691619863798 · Cited by 358 · see citing works
A psychometrics of individual differences in experimental tasks
Rouder, J. N., & Haaf, J. M. (2019) · Psychonomic Bulletin & Review · doi:10.3758/s13423-018-1558-y · Cited by 298 · see citing works
Murphy, B. A., & Lilienfeld, S. O. (2019) · Psychological Assessment · doi:10.1037/pas0000732 · Cited by 220 · see citing works
Saunders, B., Milyavskaya, M., Etz, A. J., Randles, D., & Inzlicht, M. (2018) · Collabra Psychology · doi:10.1525/collabra.134 · Cited by 139 · see citing works
Self-citationPark, N. K., Chun, M. Y.-S., & Lee, J. (2016) · Creativity Research Journal · doi:10.1080/10400419.2016.1125259 · Cited by 105 · see citing works
Individuals with dark traits have the ability but not the disposition to empathize
Kajonius, P., & Björkman, T. (2019) · Personality and Individual Differences · doi:10.1016/j.paid.2019.109716 · Cited by 64 · see citing works
Powell, D., McMinn, D., & Allan, J. (2017) · Health Psychology · doi:10.1037/hea0000471 · Cited by 56 · see citing works
Executive function and dietary intake in youth: A systematic review of the literature
Egbert, A. H., Creber, C., Loren, D. M., & Bohnert, A. M. (2019) · Appetite · doi:10.1016/j.appet.2019.04.013 · Cited by 34 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Cognitive and behavioural flexibility: Neural mechanisms and clinical considerations
Uddin, L. Q. (2021) · Nature Reviews Neuroscience · doi:10.1038/s41583-021-00428-w · Cited by 698 · see citing works
Integrating Models of Self-Regulation
Inzlicht, M., Werner, K. M., Briskin, J. L., & Roberts, B. W. (2021) · Annual Review of Psychology · doi:10.1146/annurev-psych-061020-105721 · Cited by 594 · see citing works
Self-citationWilliams, Z. J., He, J. L., Cascio, C. J., & Woynaroski, T. G. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.11.030 · Cited by 175 · see citing works
Gado, S., Kempen, R., Lingelbach, K., & Bipp, T. (2022) · Psychology Learning & Teaching · doi:10.1177/14757257211037149 · Cited by 153 · see citing works
Psychiatric Symptoms Across the Menstrual Cycle in Adult Women: A Comprehensive Review
Handy, A. B., Greenfield, S. F., Yonkers, K. A., & Payne, L. A. (2022) · Harvard Review of Psychiatry · doi:10.1097/hrp.0000000000000329 · Cited by 153 · see citing works
Uddin, L. Q. (2021) · Biological Psychiatry · doi:10.1016/j.biopsych.2020.05.010 · Cited by 116 · see citing works
Weigard, A., & Sripada, C. (2021) · Biological Psychiatry Global Open Science · doi:10.1016/j.bpsgos.2021.02.001 · Cited by 43 · see citing works
Howlett, C. A., Miles, S., Berryman, C., Phillipou, A., & Moseley, G. L. (2023) · Australian Journal of Psychology · doi:10.1080/00049530.2023.2174684 · Cited by 22 · see citing works
- Authors
- Dang, J., King, K. M., & Inzlicht, M.
- Affiliations
- Uppsala University (SE)
- University of Washington (US)
- University of Toronto (CA)
- Corresponding author
- Dang, Junhua
- Venue
- Trends in Cognitive Sciences
- Publisher
- Elsevier BV
- Published
- 2020-04
- Volume
- 24
- Issue
- 4
- Pages
- 267-269
- ISSN
- 1364-6613 (ISSN-L)
- PMC
- PMC7977810
- OpenAlex
- W3006723314
- Cited by
- 634
- References
- 14
- Citations by year
- 2026: 91 · 2025: 114 · 2024: 124 · 2023: 88 · 2022: 87 · 2021: 79 · 2020: 50 · 2019: 1
- FWCI
- 48.31
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Locations
- Trends in Cognitive Sciences (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- PsyArXiv (OSF Preprints) (submittedVersion, cc-by)
- PsyArXiv (OSF Preprints) (submittedVersion)
- Funders
- National Institute on Drug Abuse
- Social Sciences and Humanities Research Council of Canada
- Swedish Research Council
- Natural Sciences and Engineering Research Council of Canada
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Psychology › Applied Psychology › Psychological Testing and Assessment
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
- Keywords
- self-report measures, behavioral measures, construct validity, measurement reliability, response processes, weak correlation
- MeSH terms
- Humans, Reproducibility of Results, Self Report
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
