The DOI matches its Crossref record; 75 of the 76 references it cites were found in a registry or an index.
Behavior, sensitivity, and power of activation likelihood estimation characterized by massive empirical simulation
Open accessAbstract
Given the increasing number of neuroimaging publications, the automated knowledge extraction on brain-behavior associations by quantitative meta-analyses has become a highly important and rapidly growing field of research. Among several methods to perform coordinate-based neuroimaging meta-analyses, Activation Likelihood Estimation (ALE) has been widely adopted. In this paper, we addressed two pressing questions related to ALE meta-analysis: i) Which thresholding method is most appropriate to perform statistical inference? ii) Which sample size, i.e., number of experiments, is needed to perform robust meta-analyses? We provided quantitative answers to these questions by simulating more than 120,000 meta-analysis datasets using empirical parameters (i.e., number of subjects, number of reported foci, distribution of activation foci) derived from the BrainMap database. This allowed to characterize the behavior of ALE analyses, to derive first power estimates for neuroimaging meta-analyses, and to thus formulate recommendations for future ALE studies. We could show as a first consequence that cluster-level family-wise error (FWE) correction represents the most appropriate method for statistical inference, while voxel-level FWE correction is valid but more conservative. In contrast, uncorrected inference and false-discovery rate correction should be avoided. As a second consequence, researchers should aim to include at least 20 experiments into an ALE meta-analysis to achieve sufficient power for moderate effects. We would like to note, though, that these calculations and recommendations are specific to ALE and may not be extrapolated to other approaches for (neuroimaging) meta-analysis.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, new studies have since been published, and existing ALE meta-analyses do not meet the current standards for type I error control (Eickhoff et al., 2016).”
From Inhibitory control in obsessive compulsive disorder: a systematic review and activation likelihood estimation meta-analysis of functional magnetic resonance imaging studies (2022) · cites this paper as Eickhoff et al., 2016
In the citing paper
“Following current best-practice guidelines, we assessed the significance of convergence of activation foci using a voxel-level threshold of and a cluster-level familywise error-corrected threshold of , and performed ALE meta-analyses only when at least 17 experiments reported activation foci for a contrast (Eickhoff et al., 2016, Tahmasian et al., 2019).”
From Inhibitory control in obsessive compulsive disorder: a systematic review and activation likelihood estimation meta-analysis of functional magnetic resonance imaging studies (2022) · cites this paper as Eickhoff et al., 2016
In the citing paper
“Notably, this null-finding occurred in a well-powered meta-analysis: the sample of 35 experiments is considered adequate by present guidelines for neuroimaging meta-analyses (Müller et al., 2018, Tahmasian et al., 2019) and gave us more than 80 % power to detect a convergence of activation differences in areas that were identified in just 7 (20 %) of the experiments (Eickhoff et al., 2016)”
From Inhibitory control in obsessive compulsive disorder: a systematic review and activation likelihood estimation meta-analysis of functional magnetic resonance imaging studies (2022) · cites this paper as Eickhoff et al., 2016
In the citing paper
“which has been shown to be inappropriate for neuroimaging data (Chumbley and Friston, 2009) and to entail both low sensitivity and high susceptibility to spurious, false positive findings (Eickhoff et al., 2016).”
From Inhibitory control in obsessive compulsive disorder: a systematic review and activation likelihood estimation meta-analysis of functional magnetic resonance imaging studies (2022) · cites this paper as Eickhoff et al., 2016
In the citing paper
“Brem et al. (2012); Eng et al. (2015) and Menzies et al. (2008) performed meta-analyses on activation differences obtained from less than the recommended minimum of 17 fMRI experiments (Eickhoff et al., 2016).”
From Inhibitory control in obsessive compulsive disorder: a systematic review and activation likelihood estimation meta-analysis of functional magnetic resonance imaging studies (2022) · cites this paper as Eickhoff et al., 2016
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. (1995) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.2307/2983304 · Cited by 22,453
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · cited 1×
“Moreover, it may be discussed whether indeed something like “the” true location of an effect exists, given recent accounts of distributed coding and representation (Bzdok et al., 2015; Haxby, 2012; Kriegeskorte, 2009; Rissman and Wagner, 2012).”
Thresholding of statistical maps in functional neuroimaging using the false discovery rate
Genovese, C. R., Lazar, N. A., & Nichols, T. (2002) · NeuroImage · doi:10.1006/nimg.2001.1037 · Cited by 4,923 · cited 2×
“In turn, a higher number of significant voxels at the target location allow for more false positive voxels when the false positive rate is controlled (Genovese et al., 2002).”
Self-citationLarge-scale automated synthesis of human functional neuroimaging data.
Yarkoni, T., Poldrack, R. A., Nichols, T. E., Van Essen, D. C., & Wager, T. D. (2011) · Nature Methods · doi:10.1038/nmeth.1635 · Cited by 3,686 · cited 1×
“…topics from cognitive (Chase et al., 2015; Molenberghs et al., 2012), affective (Kohn et al., 2014; Lamm et al., 2011) and motor (King et al., 2014; Yuan and Brown, 2015) neuroscience to the neurobiology of neurological (Herz et al., 2014; Rehme et al., 2012) and psychiatric (Bludau et al., 2015;…”
Self-citationAn Introduction to the Bootstrap.
Buckland, S. T., Efron, B., & Tibshirani, R. J. (1994) · Biometrics · doi:10.2307/2532810 · Cited by 3,236
Mazziotta, J. C., Toga, A. W., Evans, A. C., Fox, P. T., Lancaster, J. L., Zilles, K., Woods, R. P., Paus, T., Simpson, G. V., Pike, G. B., et al. (2001) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2001.0915 · Cited by 2,417 · cited 1×
“Particularly ALE has been applied to a wide range of different topics from cognitive (Chase et al., 2015; Molenberghs et al., 2012), affective (Kohn et al., 2014; Lamm et al., 2011) and motor (King et al., 2014; Yuan and Brown, 2015) neuroscience to the neurobiology of neurological (Herz et al.,…”
Self-citationLamm, C., Decety, J., & Singer, T. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2010.10.014 · Cited by 2,016
Eickhoff, S. B., Laird, A. R., Grefkes, C., Wang, L. E., Zilles, K., & Fox, P. T. (2009) · Human Brain Mapping · doi:10.1002/hbm.20718 · Cited by 1,924 · cited 5×
“More specifically, this is quite likely to happen if one experiment is based on a substantially larger N than the remaining ones and hence features a tighter Gaussian leading to higher voxel-wise probabilities around the reported foci (Eickhoff et al., 2009b).”
InfluentialSelf-citationMeta-analysis of the functional neuroanatomy of single-word reading: method and validation
Turkeltaub, P. E., Eden, G. F., Jones, K. M., & Zeffiro, T. A. (2002) · NeuroImage · doi:10.1006/nimg.2002.1131 · Cited by 1,604 · cited 4×
“Empirically, it has repeatedly been noted that significant effects may be largely driven by even a single experiment if the total number of experiments is relatively low (Bzdok et al., 2011; Raemaekers et al., 2007; Turkeltaub et al., 2012; Wager et al., 2007).”
InfluentialLogothetis, N. K., & Wandell, B. A. (2004) · Annual Review of Physiology · doi:10.1146/annurev.physiol.66.082602.092845 · Cited by 1,568 · cited 1×
“Such structured integration is enabled by community-wide standards of spatial normalization (Amunts et al., 2014; Evans et al., 2012; Mazziotta et al., 2001) and the convention of reporting peak locations in stereotaxic reference spaces.”
Flagged first, then by citations
1 of the 76 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Variability in the analysis of a single neuroimaging dataset by many teams
(2020) · doi:10.1038/s41586-020-2314-9 · Cited by 1,097
Self-citationNeural correlates of action: Comparing meta-analyses of imagery, observation, and execution
Hardwick, R. M., Caspers, S., Eickhoff, S. B., & Swinnen, S. P. (2018) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2018.08.003 · Cited by 948
Self-citationTen simple rules for neuroimaging meta-analysis
Müller, V. I., Cieslik, E. C., Laird, A. R., Fox, P. T., Radua, J., Mataix-Cols, D., Tench, C. R., Yarkoni, T., Nichols, T. E., Turkeltaub, P. E., et al. (2018) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2017.11.012 · Cited by 805
Self-citationMcTeague, L. M., Huemer, J., Carreon, D. M., Jiang, Y., Eickhoff, S. B., & Etkin, A. (2017) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2017.16040400 · Cited by 635
“Domain-specific ALE (as well as domain-by-disorder analyses) was performed for contrasts with at least 20 experiments (31), and these are reported in the online data supplement (see Figures S12 and S13 and Tables S19 and S20).”
BackgroundMethodInfluentialSelf-citationZhou, H.-X., Chen, X., Shen, Y.-Q., Li, L., Chen, N.-X., Zhu, Z.-C., Castellanos, F. X., & Yan, C.-G. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2019.116287 · Cited by 656
“Moreover, the limited number of typical rumination/distraction studies provides equivocal statistical power for meta-analysis (Eickhoff et al., 2016).”
Background
Metadata
- Authors
- Eickhoff, S. B., Nichols, T. E., Laird, A. R., Hoffstaedter, F., Amunts, K., Fox, P. T., Bzdok, D., & Eickhoff, C. R.
- Affiliations
- Forschungszentrum Jülich (DE)
- University of Warwick (GB)
- Florida International University (US)
- The University of Texas at San Antonio Health Science Center (US)
- Institut national de recherche en sciences et technologies du numérique (FR)
- Corresponding author
- Eickhoff, Simon B.
- ORCID
- Venue
- NeuroImage
- Publisher
- Elsevier BV
- Published
- 2016-08
- Volume
- 137
- Pages
- 70-85
- ISSN
- 1053-8119 (ISSN-L)
- PMC
- PMC4981641
- OpenAlex
- W2349406327
- Cited by
- 683
- References
- 76
- Citations by year
- 2026: 61 · 2025: 68 · 2024: 58 · 2023: 83 · 2022: 92 · 2021: 93 · 2020: 72 · 2019: 61 · 2018: 46 · 2017: 35 · 2016: 13
- FWCI
- 32.88
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- NeuroImage (publishedVersion)
- PubMed (publishedVersion)
- Warwick Research Archive Portal (University of Warwick) (acceptedVersion)
- PubMed Central (acceptedVersion)
- RWTH Publications (RWTH Aachen) (submittedVersion)
- Funders
- Deutsche Forschungsgemeinschaft (EI 816/4-1, LA 3071/3-1, EI 816/6-1)
- National Institute of Mental Health (R01-MH074457)
- Helmholtz Portfolio Theme “Supercomputing and Modeling for the Human Brain” and the European Union Seventh Framework Program (604102)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Functional Brain Connectivity Studies
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Advanced Neuroimaging Techniques and Applications
- Keywords
- activation likelihood estimation, coordinate-based meta-analysis, statistical thresholding, false discovery rate, sample size, statistical power, brain-behavior relationship, BrainMap
- MeSH terms
- Brain, Data Interpretation, Statistical, Humans, Image Interpretation, Computer-Assisted, Sensitivity and Specificity, Meta-Analysis as Topic, Reproducibility of Results, Likelihood Functions, Neuroimaging
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
