Meta-analysis of functional neuroimaging data: current and future directions
Open accessThe DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Abstract
Meta-analysis is an increasingly popular and valuable tool for summarizing results across many neuroimaging studies. It can be used to establish consensus on the locations of functional regions, test hypotheses developed from patient and animal studies and develop new hypotheses on structure-function correspondence. It is particularly valuable in neuroimaging because most studies do not adequately correct for multiple comparisons; based on statistical thresholds used, we estimate that roughly 10-20% of reported activations in published studies are false positives. In this article, we briefly summarize some of the most popular meta-analytic approaches and their limitations, and we outline a revised multilevel approach with increased validity for establishing consistency across studies. We also discuss multivariate methods by which meta-analysis can be used to develop and test hypotheses about co-activity of brain regions. Finally, we argue that meta-analyses can make a uniquely valuable contribution to predicting psychological states from patterns of brain activity, and we briefly discuss some methods for making such predictions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“individual neuroimaging studies are often underpowered and exhibit relatively high false positive rate2-4”
From Large-scale automated synthesis of human functional neuroimaging data. (2011) · cites this paper as [3]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Further, each contrast is weighted by the sample size and study quality (i.e.whether the study used fixed or random effects model; Wager et al., 2007;Kober and Wager, 2010).”
From The inhibition of automatic imitation: a meta-analysis and synthesis of fMRI studies (2018) · cites this paper as Wager et al., 2007
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Applied Linear Statistical Models
Ziegel, E. R., Neter, J., Wasserman, W., & Kutner, M. (1992) · Technometrics · doi:10.2307/1269588 · Cited by 9,907
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068 · cited 1×
“Thus, if peak coordinates are used in meta-analysis, the problem of unreported activation coordinates (the ‘file drawer problem’) (Rosenthal, 1979) is greatly reduced, though not completely eliminated.”
BackgroundApplied Linear Statistical Models
(1997) · Technometrics · doi:10.1080/00401706.1997.10485141 · Cited by 7,185
Nonparametric permutation tests for functional neuroimaging: A primer with examples
Nichols, T. E., & Holmes, A. P. (2002) · Human Brain Mapping · doi:10.1002/hbm.1058 · Cited by 6,256
Talairach, J., Tournoux, P., & Rayport, M. (1988) · Thieme eBooks · Cited by 5,058
Thresholding of statistical maps in functional neuroimaging using the false discovery rate
Genovese, C. R., Lazar, N. A., & Nichols, T. (2002) · NeuroImage · doi:10.1006/nimg.2001.1037 · Cited by 4,923 · cited 2×
“In recent implementations of the ALE approach (Laird et al., 2005), ALE statistics are subjected to false discovery rate (FDR) correction (Genovese et al., 2002), which ensures that at P 0.05 corrected, on average, 5% of the reported voxels are false positives (i.e. no peak truly falls within that…”
MethodInfluentialMeta-Analytic Procedures for Social Research
Rosenthal, R. (1991) · doi:10.4135/9781412984997 · Cited by 4,635 · cited 1×
“Meta-analyses in other research areas often combine effect sizes of a single effect to test for consistency (Rosenthal, 1991).”
BackgroundMeta-Analytic Procedures for Social Research.
Metcalfe, A., & Rosenthal, R. (1994) · The Statistician · doi:10.2307/2348600 · Cited by 4,249
Forman, S. D., Cohen, J. D., Fitzgerald, M., Eddy, W. F., Mintun, M. A., & Noll, D. C. (1995) · Magnetic Resonance in Medicine · doi:10.1002/mrm.1910330508 · Cited by 3,202 · cited 2×
“005 with an eight-voxel extent that was reported to achieve ‘corrected’ false-positive control in simulations based on a single dataset (Forman et al., 1995).”
BackgroundInfluentialThe language-as-fixed-effect fallacy: A critique of language statistics in psychological research
Clark, H. H. (1973) · Journal of Verbal Learning and Verbal Behavior · doi:10.1016/s0022-5371(73)80014-3 · Cited by 2,222 · cited 2×
“The pitfalls of treating random effects as fixed effects have been widely discussed in statistics (Neter et al., 1996; Shayle et al., 2006) and in psychology (Clark, 1973).”
BackgroundBoth of Us Disgusted in My Insula
Wicker, B., Keysers, C., Plailly, J., Royet, J.-P., Gallese, V., & Rizzolatti, G. (2003) · Neuron · doi:10.1016/s0896-6273(03)00679-2 · Cited by 2,185
Subcortical and cortical brain activity during the feeling of self-generated emotions
Damasio, A. R., Grabowski, T. J., Bechara, A., Damasio, H., Ponto, L. L. B., Parvizi, J., & Hichwa, R. D. (2000) · Nature Neuroscience · doi:10.1038/79871 · Cited by 2,117 · cited 2×
“studies (Damasio et al., 2000; Liberzon et al., 2000; Wicker et al., 2003) included in recent meta-analyses of emotion (Wager et al.”
BackgroundAssessing the significance of focal activations using their spatial extent
Friston, K. J., Worsley, K. J., Frackowiak, R. S. J., Mazziotta, J. C., & Evans, A. C. (1994) · Human Brain Mapping · doi:10.1002/hbm.460010306 · Cited by 1,907 · cited 2×
“Because of the need to maintain statistical power with small samples and often hundreds of thousands of comparisons in each study, uncorrected or inadequately corrected statistical thresholds are the rule, and false positives are endemic in neuroimaging research.”
BackgroundMethodNeuroimaging studies of working memory:
Wager, T. D., & Smith, E. E. (2003) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.3.4.255 · Cited by 1,889
Self-citationMeta-analysis of the functional neuroanatomy of single-word reading: method and validation
Turkeltaub, P. E., Eden, G. F., Jones, K. M., & Zeffiro, T. A. (2002) · NeuroImage · doi:10.1006/nimg.2002.1131 · Cited by 1,604 · cited 4×
“, 2004)], activation likelihood estimate (ALE) analysis (Turkeltaub et al., 2002; Laird et al., 2005), and a new multilevel variant of KDA designed to address several important statistical shortcomings of the other two models (Wager et al.”
BackgroundInfluentialControlling the familywise error rate in functional neuroimaging: a comparative review
Nichols, T., & Hayasaka, S. (2003) · Statistical Methods in Medical Research · doi:10.1191/0962280203sm341ra · Cited by 1,218 · cited 3×
“Based on smoothness estimates for studies reviewed in Nichols and Hayasaka (2003), at this threshold, we might expect roughly one false positive for every two comparison maps, or about 150 false positive activations.2 However, many thresholds were more liberal, as shown in the inset of Figure 1.”
BackgroundMethodInfluentialWager, T. D., Phan, K. L., Liberzon, I., & Taylor, S. F. (2003) · NeuroImage · doi:10.1016/s1053-8119(03)00078-8 · Cited by 1,150 · cited 4×
“coordinates: Kernel density analysis [KDA; (Wager et al., 2003; Wager et al., 2004)], activation likelihood estimate (ALE) analysis (Turkeltaub et al.”
BackgroundMethodInfluentialSelf-citationFunctional neuroanatomy of emotions: A meta-analysis
MURPHY, F. C., NIMMO-SMITH, I., & LAWRENCE, A. D. (2003) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.3.3.207 · Cited by 1,135
ALE meta-analysis: controlling the false discovery rate and performing statistical contrasts
Laird, A. R., Fox, P. M., Price, C. J., Glahn, D. C., Uecker, A. M., Lancaster, J. L., Turkeltaub, P. E., Kochunov, P., & Fox, P. T. (2005) · Human Brain Mapping · doi:10.1002/hbm.20136 · Cited by 868 · cited 3×
“, 2004)], activation likelihood estimate (ALE) analysis (Turkeltaub et al., 2002; Laird et al., 2005), and a new multilevel variant of KDA designed to address several important statistical shortcomings of the other two models (Wager et al.”
BackgroundMethodInfluentialInterference resolution: Insights from a meta-analysis of neuroimaging tasks
NEE, D. E., WAGER, T. D., & JONIDES, J. (2007) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.7.1.1 · Cited by 798 · cited 1×
“Kosslyn and Thompson (2003) used logistic regression to examine the factors that influence whether studies find early visual activation in mental imagery, and Nee et al. (2007) used it to examine the stages of response conflict associated with activation in several regions during ‘conflict’ tasks.”
BackgroundSelf-citationNeuroimaging studies of shifting attention: a meta-analysis
Wager, T. D., Jonides, J., & Reading, S. (2004) · NeuroImage · doi:10.1016/j.neuroimage.2004.03.052 · Cited by 654 · cited 5×
“Wager et al., 2004) identified parietal regions of interest using KDA analysis on attention-switching tasks.”
BackgroundMethodInfluentialSelf-citationWhen is early visual cortex activated during visual mental imagery?
Kosslyn, S. M., & Thompson, W. L. (2003) · Psychological Bulletin · doi:10.1037/0033-2909.129.5.723 · Cited by 515 · cited 2×
“This can be accomplished by assuming that the reported peaks are spatially independent (the location of each peak is independent of the others), though this assumption is questionable, because studies tend to report multiple peaks from each significantly activated region (we’ll refer to them here…”
BackgroundSupport vector machines for temporal classification of block design fMRI data
LACONTE, S., STROTHER, S., CHERKASSKY, V, ANDERSON, J., & HU, X. (2005) · NeuroImage · doi:10.1016/j.neuroimage.2005.01.048 · Cited by 379 · cited 2×
“A host of methods for predicting task status given multivariate brain activity patterns may be productively applied to meta-analytic data to achieve these ends, including Fisher’s linear and quadratic discriminant analysis, support-vector machines (LaConte et al., 2005; Pessoa and Padmala, 2007), and Bayesian classifi…”
MethodQuantification of frontal and temporal lobe brain-imaging findings in schizophrenia: a meta-analysis
Davidson, L. L., & Heinrichs, R. W. (2003) · Psychiatry Research: Neuroimaging · doi:10.1016/s0925-4927(02)00118-x · Cited by 232 · cited 1×
“Some authors have used measures of effect size directly and performed a traditional meta-analysis comparing effect sizes across studies (Davidson and Heinrichs, 2003; Zakzanis et al., 2003; Thornton et al., 2006).”
Method(2008) · Cited by 225
Self-citationZakzanis, K. K., Graham, S. J., & Campbell, Z. (2003) · Neuropsychology Review · doi:10.1023/a:1022318921994 · Cited by 174 · cited 1×
“Some authors have used measures of effect size directly and performed a traditional meta-analysis comparing effect sizes across studies (Davidson and Heinrichs, 2003; Zakzanis et al., 2003; Thornton et al., 2006).”
MethodLiberzon, I., Taylor, S. F., Fig, L. M., Decker, L. R., Koeppe, R. A., & Minoshima, S. (2000) · Neuropsychopharmacology · doi:10.1016/s0893-133x(00)00157-3 · Cited by 162 · cited 2×
“Keywords: PET; fMRI; meta-analysis; neuroimaging; analysis methods”
BackgroundDecoding Near-Threshold Perception of Fear from Distributed Single-Trial Brain Activation
Pessoa, L., & Padmala, S. (2006) · Cerebral Cortex · doi:10.1093/cercor/bhk020 · Cited by 104 · cited 2×
“A host of methods for predicting task status given multivariate brain activity patterns may be productively applied to meta-analytic data to achieve these ends, including Fisher’s linear and quadratic discriminant analysis, support-vector machines (LaConte et al., 2005; Pessoa and Padmala, 2007), and Bayesian classifi…”
MethodThe Language-as-Fixed-Effect Fallacy: A Critique of Language Statistics in Psychological Research
Banks, W. P., Carlsmith, J. M., Clark, E. V., & Herrmann, D. J. (2005) · Cited by 73
Thornton, A. E., Snellenberg, J. X. V., Sepehry, A. A., & Honer, W. G. (2006) · Journal of Psychopharmacology · doi:10.1177/0269881105057002 · Cited by 68 · cited 1×
“Some authors have used measures of effect size directly and performed a traditional meta-analysis comparing effect sizes across studies (Davidson and Heinrichs, 2003; Zakzanis et al., 2003; Thornton et al., 2006).”
MethodMining for Associations Between Text and Brain Activation in a Functional Neuroimaging Database
Nielsen, F. Å., Hansen, L. K., & Balslev, D. (2004) · Neuroinformatics · doi:10.1385/ni:2:4:369 · Cited by 49
Research works printed but not linked: 1
- Shayle, 2006, Variance Components
Cited by
Large-scale automated synthesis of human functional neuroimaging data.
Yarkoni, T., Poldrack, R. A., Nichols, T. E., Van Essen, D. C., & Wager, T. D. (2011) · Nature Methods · doi:10.1038/nmeth.1635 · Cited by 3,686
“Because individual neuroimaging studies are often underpowered and have relatively high false positive rate”
Self-citationEtkin, A., & Wager, T. D. (2007) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2007.07030504 · Cited by 3,391
“Furthermore, unlike other methods that rely on z-score weighting, our meta-analysis approach is better suited to assess the replicability of activation across studies (63).”
MethodInfluentialSelf-citationThayer, J. F., Åhs, F., Fredrikson, M., Sollers, J. J., & Wager, T. D. (2012) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2011.11.009 · Cited by 3,224
“Unlike some methods based only on reported activation coordinates, MKDA treats contrast maps (not peaks) as the unit of analysis, and therefore is suitable for evaluating the consistency of activation across studies (Kober et al., 2008; Wager et al., 2007; Wager et al., 2009a).”
MethodSelf-citationDysfunction of the prefrontal cortex in addiction: neuroimaging findings and clinical implications
Goldstein, R. Z., & Volkow, N. D. (2011) · Nature Reviews Neuroscience · doi:10.1038/nrn3119 · Cited by 2,749
The brain basis of emotion: a meta-analytic review
Lindquist, K. A., Wager, T. D., Kober, H., Bliss-Moreau, E., & Barrett, L. F. (2012) · Behavioral and Brain Sciences · doi:10.1017/s0140525x11000446 · Cited by 2,320
“First, metaanalysis can weed out false positives (which are frequent in neuroimaging studies; see Wager et al. 2007; Yarkoni 2009) and focus interpretation on the activations that are consistently observed.”
Self-citation
Metadata
- Authors
- Wager, T. D., Lindquist, M., & Kaplan, L.
- Affiliations
- Columbia University (US)
- Venue
- Social Cognitive and Affective Neuroscience
- Publisher
- Oxford University Press (OUP)
- Published
- 2007-06-01
- Volume
- 2
- Issue
- 2
- Pages
- 150-158
- ISSN
- 1749-5016 (ISSN-L)
- 1749-5024
- PMC
- PMC2555451
- OpenAlex
- W2137449272
- Cited by
- 434
- References
- 31
- Citations by year
- 2026: 12 · 2025: 25 · 2024: 22 · 2023: 18 · 2022: 34 · 2021: 23 · 2020: 28 · 2019: 28 · 2018: 27 · 2017: 29 · 2016: 19 · 2015: 24 · 2014: 19 · 2013: 27 · 2012: 22
- FWCI
- 1.29
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Social Cognitive and Affective Neuroscience (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Functional Brain Connectivity Studies
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Keywords
- false positives, multiple testing correction, multilevel meta-analysis, multivariate meta-analysis, functional brain regions
- MeSH terms
- Forecasting, Humans, Magnetic Resonance Imaging
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.