The DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Type I and Type II error concerns in fMRI research: re-balancing the scale
Open accessAbstract
Statistical thresholding (i.e. P-values) in fMRI research has become increasingly conservative over the past decade in an attempt to diminish Type I errors (i.e. false alarms) to a level traditionally allowed in behavioral science research. In this article, we examine the unintended negative consequences of this single-minded devotion to Type I errors: increased Type II errors (i.e. missing true effects), a bias toward studying large rather than small effects, a bias toward observing sensory and motor processes rather than complex cognitive and affective processes and deficient meta-analyses. Power analyses indicate that the reductions in acceptable P-values over time are producing dramatic increases in the Type II error rate. Moreover, the push for a mapwide false discovery rate (FDR) of 0.05 is based on the assumption that this is the FDR in most behavioral research; however, this is an inaccurate assessment of the conventions in actual behavioral research. We report simulations demonstrating that combined intensity and cluster size thresholds such as P < 0.005 with a 10 voxel extent produce a desirable balance between Types I and II error rates. This joint threshold produces high but acceptable Type II error rates and produces a FDR that is comparable to the effective FDR in typical behavioral science articles (while a 20 voxel extent threshold produces an actual FDR of 0.05 with relatively common imaging parameters). We recommend a greater focus on replication and meta-analysis rather than emphasizing single studies as the unit of analysis for establishing scientific truth. From this perspective, Type I errors are self-erasing because they will not replicate, thus allowing for more lenient thresholding to avoid Type II errors.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Results were considered significant at an uncorrected threshold of p < .001 with a minimum cluster size of 10 contiguous voxels to balance between Type 1 and Type 2 errors (Lieberman and Cunningham 2009)”
From Chronic Childhood Peer Rejection is Associated with Heightened Neural Responses to Social Exclusion During Adolescence (2016) · cites this paper as Lieberman and Cunningham 2009
Experimental — accuracy not yet measuredIn the citing paper
“Next, we performed a regression analysis with OXT methylation values entered as the predictor variable and contrast estimates (emotion perspectivetaking > shape match) as the criterion variable using a threshold of P < 0.005, voxel extent of k = 60, which is sufficient to preserve the balance between sensitivity and false-positive rates (55,56).”
From Epigenetic modification of OXT and human sociability (2016) · cites this paper as [55]
In the citing paper
“However, lowering the threshold to an uncorrected voxel height of p < 0.005 with an extent cluster size of at least 20 voxels (according to Scott-Van Zeeland et al., 2010), which still produces desirable balance between Types I and II errors comparable to false discovery rate (Lieberman and Cunningham, 2009), the ASD group, compared to TDC and ADHD, showed diminished activation in ventral striatum”
From Differentiating neural reward responsiveness in autism versus ADHD (2014) · cites this paper as Lieberman and Cunningham, 2009
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
AFNI: Software for analysis and visualization of functional magnetic resonance neuroimages
Cox, R. W. (1996) · Computers and Biomedical Research · doi:10.1006/cbmr.1996.0014 · Cited by 10,913
Thresholding of statistical maps in functional neuroimaging using the false discovery rate
Genovese, C. R., Lazar, N. A., & Nichols, T. (2002) · NeuroImage · doi:10.1006/nimg.2001.1037 · Cited by 4,923
Forman, S. D., Cohen, J. D., Fitzgerald, M., Eddy, W. F., Mintun, M. A., & Noll, D. C. (1995) · Magnetic Resonance in Medicine · doi:10.1002/mrm.1910330508 · Cited by 3,202
Building Memories: Remembering and Forgetting of Verbal Experiences as Predicted by Brain Activity
Wagner, A. D., Schacter, D. L., Rotte, M., Koutstaal, W., Maril, A., Dale, A. M., Rosen, B. R., & Buckner, R. L. (1998) · Science · doi:10.1126/science.281.5380.1188 · Cited by 1,644
Puzzlingly High Correlations in fMRI Studies of Emotion, Personality, and Social Cognition
Vul, E., Harris, C., Winkielman, P., & Pashler, H. (2009) · Perspectives on Psychological Science · doi:10.1111/j.1745-6924.2009.01125.x · Cited by 1,475 · cited 1×
“It should be noted that others have run simulations suggesting that much larger cluster sizes are necessary ( Vul et al., 2009; c.f.”
BackgroundThe Arrangement of Field Experiments
Fisher, R. A. (1992) · Springer series in statistics · doi:10.1007/978-1-4612-4380-9_8 · Cited by 977
Subjective Construal, Social Inference, and Human Misunderstanding
Griffin, D. W., & Ross, L. (1991) · Advances in Experimental Social Psychology · doi:10.1016/s0065-2601(08)60333-0 · Cited by 404
Correlations in Social Neuroscience Aren't Voodoo: Commentary on Vul et al. (2009)
Lieberman, M. D., Berkman, E. T., & Wager, T. D. (2009) · Perspectives on Psychological Science · doi:10.1111/j.1745-6924.2009.01128.x · Cited by 135
Self-citationHow thinking controls feeling: A social cognitive neuroscience approach
Ochsner, K. N. (2006) · PsycEXTRA Dataset · doi:10.1037/e633962013-138 · Cited by 14
Cited by
Sorted by citations
Cluster failure: why fMRI inferences for spatial extent have inflated false-positive rates
Eklund, A., Nichols, T. E., & Knutsson, H. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1602413113 · Cited by 3,487
Scanning the horizon: towards transparent and reproducible neuroimaging research
Poldrack, R. A., Baker, C. I., Durnez, J., Gorgolewski, K. J., Matthews, P. M., Munafò, M. R., Nichols, T. E., Poline, J.-B., Vul, E., & Yarkoni, T. (2017) · Nature Reviews Neuroscience · doi:10.1038/nrn.2016.167 · Cited by 1,410
Fox, K. C. R., Nijeboer, S., Dixon, M. L., Floman, J. L., Ellamil, M., Rumak, S. P., Sedlmeier, P., & Christoff, K. (2014) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2014.03.016 · Cited by 804
Ten simple rules for neuroimaging meta-analysis
Müller, V. I., Cieslik, E. C., Laird, A. R., Fox, P. T., Radua, J., Mataix-Cols, D., Tench, C. R., Yarkoni, T., Nichols, T. E., Turkeltaub, P. E., et al. (2018) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2017.11.012 · Cited by 805
Inflammation-Induced Anhedonia: Endotoxin Reduces Ventral Striatum Responses to Reward
Eisenberger, N. I., Berkman, E. T., Inagaki, T. K., Rameson, L. T., Mashal, N. M., & Irwin, M. R. (2010) · Biological Psychiatry · doi:10.1016/j.biopsych.2010.06.010 · Cited by 596
Metadata
- Authors
- Lieberman, M. D., & Cunningham, W. A.
- Affiliations
- University of California, Los Angeles (US)
- Venue
- Social Cognitive and Affective Neuroscience
- Publisher
- Oxford University Press (OUP)
- Published
- 2009-12-01
- Volume
- 4
- Issue
- 4
- Pages
- 423-428
- ISSN
- 1749-5016 (ISSN-L)
- 1749-5024
- PMC
- PMC2799956
- OpenAlex
- W2117101773
- Cited by
- 1,342
- References
- 9
- Citations by year
- 2026: 33 · 2025: 35 · 2024: 51 · 2023: 50 · 2022: 62 · 2021: 73 · 2020: 119 · 2019: 92 · 2018: 94 · 2017: 113 · 2016: 111 · 2015: 119 · 2014: 126 · 2013: 97 · 2012: 80
- FWCI
- 26.55
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Social Cognitive and Affective Neuroscience (publishedVersion, cc-by-nc)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Functional Brain Connectivity Studies
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Advanced MRI Techniques and Applications
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- type I error, type II error, false discovery rate, power analysis, effect size bias, replication, meta-analysis, cognitive and affective processes
- MeSH terms
- Brain, Brain Mapping, Data Interpretation, Statistical, False Positive Reactions, Humans, Image Interpretation, Computer-Assisted, Magnetic Resonance Imaging, Oxygen, Research Design, Meta-Analysis as Topic, Reproducibility of Results, Sample Size
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
