Cited by 1,342Functional Brain Connectivity StudiesAdvanced MRI Techniques and ApplicationsMeta-analysis and systematic reviews

Type I and Type II error concerns in fMRI research: re-balancing the scale

Open access
Lieberman, Matthew D., Cunningham, William A.·2009·Social Cognitive and Affective Neuroscience

Abstract

Statistical thresholding (i.e. P-values) in fMRI research has become increasingly conservative over the past decade in an attempt to diminish Type I errors (i.e. false alarms) to a level traditionally allowed in behavioral science research. In this article, we examine the unintended negative consequences of this single-minded devotion to Type I errors: increased Type II errors (i.e. missing true effects), a bias toward studying large rather than small effects, a bias toward observing sensory and motor processes rather than complex cognitive and affective processes and deficient meta-analyses. Power analyses indicate that the reductions in acceptable P-values over time are producing dramatic increases in the Type II error rate. Moreover, the push for a mapwide false discovery rate (FDR) of 0.05 is based on the assumption that this is the FDR in most behavioral research; however, this is an inaccurate assessment of the conventions in actual behavioral research. We report simulations demonstrating that combined intensity and cluster size thresholds such as P < 0.005 with a 10 voxel extent produce a desirable balance between Types I and II error rates. This joint threshold produces high but acceptable Type II error rates and produces a FDR that is comparable to the effective FDR in typical behavioral science articles (while a 20 voxel extent threshold produces an actual FDR of 0.05 with relatively common imaging parameters). We recommend a greater focus on replication and meta-analysis rather than emphasizing single studies as the unit of analysis for establishing scientific truth. From this perspective, Type I errors are self-erasing because they will not replicate, thus allowing for more lenient thresholding to avoid Type II errors.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1093/scan/nsp052

Cites

9 of 9 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Results were considered significant at an uncorrected threshold of p < .001 with a minimum cluster size of 10 contiguous voxels to balance between Type 1 and Type 2 errors.

    “Results were considered significant at an uncorrected threshold of p < .001 with a minimum cluster size of 10 contiguous voxels to balance between Type 1 and Type 2 errors (Lieberman and Cunningham 2009)”

    From Chronic Childhood Peer Rejection is Associated with Heightened Neural Responses to Social Exclusion During Adolescence (2016) · cites this paper as Lieberman and Cunningham 2009

    The Abstract says that combining an intensity threshold with a 10-voxel cluster extent can balance Type I and Type II errors. But the threshold it identifies is P < 0.005, not the cited sentence’s uncorrected P < 0.001. The 'Avoiding Type I Errors' section likewise compares P < 0.005 with 10 voxels against P < 0.001 alone; it does not establish the claimed combination. (LLM, conf=0.89)

    Experimental — accuracy not yet measured
  • A threshold of P < 0.005 and a voxel extent of k = 60 were deemed adequate to maintain a balance between sensitivity and false-positive rates.

    “Next, we performed a regression analysis with OXT methylation values entered as the predictor variable and contrast estimates (emotion perspectivetaking > shape match) as the criterion variable using a threshold of P < 0.005, voxel extent of k = 60, which is sufficient to preserve the balance between sensitivity and false-positive rates (55,56).”

    From Epigenetic modification of <i>OXT</i> and human sociability (2016) · cites this paper as [55]

  • Lowering the threshold to an uncorrected voxel height of p < 0.005 with an extent cluster size of at least 20 voxels, the ASD group, compared to TDC and ADHD, showed diminished activation in ventral striatum in response to monetary reward.

    “However, lowering the threshold to an uncorrected voxel height of p < 0.005 with an extent cluster size of at least 20 voxels (according to Scott-Van Zeeland et al., 2010), which still produces desirable balance between Types I and II errors comparable to false discovery rate (Lieberman and Cunningham, 2009), the ASD group, compared to TDC and ADHD, showed diminished activation in ventral striatum”

    From Differentiating neural reward responsiveness in autism versus ADHD (2014) · cites this paper as Lieberman and Cunningham, 2009

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Lieberman, M. D., & Cunningham, W. A.
Affiliations
University of California, Los Angeles (US)
Venue
Social Cognitive and Affective Neuroscience
Published
2009-12-01
Volume
4
Issue
4
Pages
423-428
ISSN
  • 1749-5016 (ISSN-L)
  • 1749-5024
OpenAlex
W2117101773
Cited by
1,342
References
9
Citations by year
2026: 33 · 2025: 35 · 2024: 51 · 2023: 50 · 2022: 62 · 2021: 73 · 2020: 119 · 2019: 92 · 2018: 94 · 2017: 113 · 2016: 111 · 2015: 119 · 2014: 126 · 2013: 97 · 2012: 80
FWCI
26.55
Type
journal article
Language
en
Open access
Locations
  • Social Cognitive and Affective Neuroscience (publishedVersion, cc-by-nc)
  • Europe PMC (PubMed Central) (publishedVersion)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Functional Brain Connectivity Studies
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Advanced MRI Techniques and Applications
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Keywords
type I error, type II error, false discovery rate, power analysis, effect size bias, replication, meta-analysis, cognitive and affective processes
MeSH terms
Brain, Brain Mapping, Data Interpretation, Statistical, False Positive Reactions, Humans, Image Interpretation, Computer-Assisted, Magnetic Resonance Imaging, Oxygen, Research Design, Meta-Analysis as Topic, Reproducibility of Results, Sample Size

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review