Correlations in Social Neuroscience Aren't Voodoo: Commentary on Vul et al. (2009)
Open accessThe DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Abstract
Vul, Harris, Winkielman, and Pashler (2009) , (this issue) claim that many brain—personality correlations in fMRI studies are “likely … spurious” (p. 274), and “should not be believed” (p. 285). Several of their conclusions are incorrect. First, they incorrectly claim that whole-brain regressions use an invalid and “nonindependent” two-step inferential procedure, a determination based on a survey sent to researchers that only included nondiagnostic questions about the descriptive process of plotting one's data. We explain how whole-brain regressions are a valid single-step method of identifying brain regions that have reliable correlations with individual difference measures. Second, they claim that large correlations from whole-brain regression analyses may be the result of noise alone. We provide a simulation to demonstrate that typical fMRI sample sizes will only rarely produce large correlations in the absence of any true effect. Third, they claim that the reported correlations are inflated to the point of being “implausibly high.” Though biased post hoc correlation estimates are a well-known consequence of conducting multiple tests, Vul et al. make inaccurate assumptions when estimating the theoretical ceiling of such correlations. Moreover, their own “meta-analysis suggests that the magnitude of the bias is approximately .12—a rather modest bias.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Empathy for Pain Involves the Affective but not Sensory Components of Pain
Singer, T., Seymour, B., O'Doherty, J., Kaube, H., Dolan, R. J., & Frith, C. D. (2004) · Science · doi:10.1126/science.1093535 · Cited by 3,929 · cited 2×
“Given that a number of these studies replicate the Singer et al. (2004) findings, it again raises questions about the selective inclusion of studies in their review.”
Does Rejection Hurt? An fMRI Study of Social Exclusion
Eisenberger, N. I., Lieberman, M. D., & Williams, K. D. (2003) · Science · doi:10.1126/science.1089134 · Cited by 3,849 · cited 4×
“Consider, for example, the correlation (r5 .88) between a social distress measure and activation in the dorsal anterior cingulate cortex during a social pain manipulation (Eisenberger et al., 2003) that is singled out by Vul et al. from the first page of their article.”
InfluentialSelf-citationPsychological and neural mechanisms of the affective dimension of pain
Price, D. D. (2000) · Science · doi:10.1126/science.288.5472.1769 · Cited by 2,092 · cited 1×
“In each of these pain-related studies, significant correlations were reported between individual difference measures and activity in the dorsal anterior cingulate cortex, a region central to the experience of pain (Price, 2000).”
Neuroimaging studies of working memory:
Wager, T. D., & Smith, E. E. (2003) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.3.4.255 · Cited by 1,889
Self-citationEmpathic neural responses are modulated by the perceived fairness of others
Singer, T., Seymour, B., O'Doherty, J. P., Stephan, K. E., Dolan, R. J., & Frith, C. D. (2006) · Nature · doi:10.1038/nature04271 · Cited by 1,726 · cited 3×
“In Study 31 (Singer et al., 2006), four nonindependent correlations that are described in the text were not included, though they were listed numerically in the supplementary materials (as indicated in the main text).”
Puzzlingly High Correlations in fMRI Studies of Emotion, Personality, and Social Cognition
Vul, E., Harris, C., Winkielman, P., & Pashler, H. (2009) · Perspectives on Psychological Science · doi:10.1111/j.1745-6924.2009.01125.x · Cited by 1,475 · cited 1×
“Commentary on Vul et al. (2009)”
Kober, H., Barrett, L. F., Joseph, J., Bliss-Moreau, E., Lindquist, K., & Wager, T. D. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2008.03.059 · Cited by 1,144 · cited 1×
“The same point is made by metaanalyses of the neuroimaging literature on emotion, which clearly show ‘‘hot spots’’ of consistently replicated activity across laboratories and task variants (Kober et al., 2008; Wager et al., 2008).”
Self-citationInterference resolution: Insights from a meta-analysis of neuroimaging tasks
NEE, D. E., WAGER, T. D., & JONIDES, J. (2007) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.7.1.1 · Cited by 798
Self-citationWhen inclusion costs and ostracism pays, ostracism still hurts.
van Beest, I., & Williams, K. D. (2006) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.91.5.918 · Cited by 567
Neural correlates of social cooperation and non-cooperation as a function of psychopathy
Rilling, J. K., Glenn, A. L., Jairam, M. R., Pagnoni, G., Goldsmith, D. R., Elfenbein, H. A., & Lilienfeld, S. O. (2007) · Biological Psychiatry · doi:10.1016/j.biopsych.2006.07.021 · Cited by 408
Cited by
Puzzlingly High Correlations in fMRI Studies of Emotion, Personality, and Social Cognition
Vul, E., Harris, C., Winkielman, P., & Pashler, H. (2009) · Perspectives on Psychological Science · doi:10.1111/j.1745-6924.2009.01125.x · Cited by 1,475
Type I and Type II error concerns in fMRI research: re-balancing the scale
Lieberman, M. D., & Cunningham, W. A. (2009) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsp052 · Cited by 1,342
“4 It should be noted that others have run simulations suggesting that much larger cluster sizes are necessary (Vul et al., 2009; c.f. Lieberman et al, 2009), but these analyses included all of the voxels that are outside of the brain which are never actually tested.”
BackgroundSelf-citationFrontocingulate Dysfunction in Depression: Toward Biomarkers of Treatment Response
Pizzagalli, D. A. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.166 · Cited by 945
Keil, A., Debener, S., Gratton, G., Junghöfer, M., Kappenman, E. S., Luck, S. J., Luu, P., Miller, G. A., & Yee, C. M. (2014) · Psychophysiology · doi:10.1111/psyp.12147 · Cited by 699
Prefrontal–striatal pathway underlies cognitive regulation of craving
Kober, H., Mende-Siedlecki, P., Kross, E. F., Weber, J., Mischel, W., Hart, C. L., & Ochsner, K. N. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1007779107 · Cited by 695
Metadata
- Authors
- Lieberman, M. D., Berkman, E. T., & Wager, T. D.
- Venue
- Perspectives on Psychological Science
- Publisher
- SAGE Publications
- Published
- 2009-05
- Volume
- 4
- Issue
- 3
- Pages
- 299-307
- ISSN
- 1745-6916 (ISSN-L)
- 1745-6924
- OpenAlex
- W1978137990
- Cited by
- 135
- References
- 11
- FWCI
- 7.77
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Correlations in Social Neuroscience Aren't Voodoo: Commentary on Vul et al. (2009). https://app.cochat.ai/papers/10.1111/j.1745-6924.2009.01128.x
- The paper
- Lieberman, M. D., Berkman, E. T., & Wager, T. D. (2009). Correlations in Social Neuroscience Aren't Voodoo: Commentary on Vul et al. (2009). Perspectives on Psychological Science. https://doi.org/10.1111/j.1745-6924.2009.01128.x
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.