The DOI matches its Crossref record; 75 of the 76 references it cites were found in a registry or an index.
Behavior, sensitivity, and power of activation likelihood estimation characterized by massive empirical simulation
Open accessAbstract
Given the increasing number of neuroimaging publications, the automated knowledge extraction on brain-behavior associations by quantitative meta-analyses has become a highly important and rapidly growing field of research. Among several methods to perform coordinate-based neuroimaging meta-analyses, Activation Likelihood Estimation (ALE) has been widely adopted. In this paper, we addressed two pressing questions related to ALE meta-analysis: i) Which thresholding method is most appropriate to perform statistical inference? ii) Which sample size, i.e., number of experiments, is needed to perform robust meta-analyses? We provided quantitative answers to these questions by simulating more than 120,000 meta-analysis datasets using empirical parameters (i.e., number of subjects, number of reported foci, distribution of activation foci) derived from the BrainMap database. This allowed to characterize the behavior of ALE analyses, to derive first power estimates for neuroimaging meta-analyses, and to thus formulate recommendations for future ALE studies. We could show as a first consequence that cluster-level family-wise error (FWE) correction represents the most appropriate method for statistical inference, while voxel-level FWE correction is valid but more conservative. In contrast, uncorrected inference and false-discovery rate correction should be avoided. As a second consequence, researchers should aim to include at least 20 experiments into an ALE meta-analysis to achieve sufficient power for moderate effects. We would like to note, though, that these calculations and recommendations are specific to ALE and may not be extrapolated to other approaches for (neuroimaging) meta-analysis.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, new studies have since been published, and existing ALE meta-analyses do not meet the current standards for type I error control (Eickhoff et al., 2016).”
From Inhibitory control in obsessive compulsive disorder: a systematic review and activation likelihood estimation meta-analysis of functional magnetic resonance imaging studies (2022) · cites this paper as Eickhoff et al., 2016
In the citing paper
“Following current best-practice guidelines, we assessed the significance of convergence of activation foci using a voxel-level threshold of and a cluster-level familywise error-corrected threshold of , and performed ALE meta-analyses only when at least 17 experiments reported activation foci for a contrast (Eickhoff et al., 2016, Tahmasian et al., 2019).”
From Inhibitory control in obsessive compulsive disorder: a systematic review and activation likelihood estimation meta-analysis of functional magnetic resonance imaging studies (2022) · cites this paper as Eickhoff et al., 2016
In the citing paper
“Notably, this null-finding occurred in a well-powered meta-analysis: the sample of 35 experiments is considered adequate by present guidelines for neuroimaging meta-analyses (Müller et al., 2018, Tahmasian et al., 2019) and gave us more than 80 % power to detect a convergence of activation differences in areas that were identified in just 7 (20 %) of the experiments (Eickhoff et al., 2016)”
From Inhibitory control in obsessive compulsive disorder: a systematic review and activation likelihood estimation meta-analysis of functional magnetic resonance imaging studies (2022) · cites this paper as Eickhoff et al., 2016
In the citing paper
“which has been shown to be inappropriate for neuroimaging data (Chumbley and Friston, 2009) and to entail both low sensitivity and high susceptibility to spurious, false positive findings (Eickhoff et al., 2016).”
From Inhibitory control in obsessive compulsive disorder: a systematic review and activation likelihood estimation meta-analysis of functional magnetic resonance imaging studies (2022) · cites this paper as Eickhoff et al., 2016
In the citing paper
“Brem et al. (2012); Eng et al. (2015) and Menzies et al. (2008) performed meta-analyses on activation differences obtained from less than the recommended minimum of 17 fMRI experiments (Eickhoff et al., 2016).”
From Inhibitory control in obsessive compulsive disorder: a systematic review and activation likelihood estimation meta-analysis of functional magnetic resonance imaging studies (2022) · cites this paper as Eickhoff et al., 2016
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. (1995) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.2307/2983304 · Cited by 22,453
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · cited 1×
“Moreover, it may be discussed whether indeed something like “the” true location of an effect exists, given recent accounts of distributed coding and representation (Bzdok et al., 2015; Haxby, 2012; Kriegeskorte, 2009; Rissman and Wagner, 2012).”
Thresholding of statistical maps in functional neuroimaging using the false discovery rate
Genovese, C. R., Lazar, N. A., & Nichols, T. (2002) · NeuroImage · doi:10.1006/nimg.2001.1037 · Cited by 4,923 · cited 2×
“In turn, a higher number of significant voxels at the target location allow for more false positive voxels when the false positive rate is controlled (Genovese et al., 2002).”
Self-citationLarge-scale automated synthesis of human functional neuroimaging data.
Yarkoni, T., Poldrack, R. A., Nichols, T. E., Van Essen, D. C., & Wager, T. D. (2011) · Nature Methods · doi:10.1038/nmeth.1635 · Cited by 3,686 · cited 1×
“…topics from cognitive (Chase et al., 2015; Molenberghs et al., 2012), affective (Kohn et al., 2014; Lamm et al., 2011) and motor (King et al., 2014; Yuan and Brown, 2015) neuroscience to the neurobiology of neurological (Herz et al., 2014; Rehme et al., 2012) and psychiatric (Bludau et al., 2015;…”
Self-citationAn Introduction to the Bootstrap.
Buckland, S. T., Efron, B., & Tibshirani, R. J. (1994) · Biometrics · doi:10.2307/2532810 · Cited by 3,236
Mazziotta, J. C., Toga, A. W., Evans, A. C., Fox, P. T., Lancaster, J. L., Zilles, K., Woods, R. P., Paus, T., Simpson, G. V., Pike, G. B., et al. (2001) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2001.0915 · Cited by 2,417 · cited 1×
“Particularly ALE has been applied to a wide range of different topics from cognitive (Chase et al., 2015; Molenberghs et al., 2012), affective (Kohn et al., 2014; Lamm et al., 2011) and motor (King et al., 2014; Yuan and Brown, 2015) neuroscience to the neurobiology of neurological (Herz et al.,…”
Self-citationLamm, C., Decety, J., & Singer, T. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2010.10.014 · Cited by 2,016
Eickhoff, S. B., Laird, A. R., Grefkes, C., Wang, L. E., Zilles, K., & Fox, P. T. (2009) · Human Brain Mapping · doi:10.1002/hbm.20718 · Cited by 1,924 · cited 5×
“More specifically, this is quite likely to happen if one experiment is based on a substantially larger N than the remaining ones and hence features a tighter Gaussian leading to higher voxel-wise probabilities around the reported foci (Eickhoff et al., 2009b).”
InfluentialSelf-citationMeta-analysis of the functional neuroanatomy of single-word reading: method and validation
Turkeltaub, P. E., Eden, G. F., Jones, K. M., & Zeffiro, T. A. (2002) · NeuroImage · doi:10.1006/nimg.2002.1131 · Cited by 1,604 · cited 4×
“Empirically, it has repeatedly been noted that significant effects may be largely driven by even a single experiment if the total number of experiments is relatively low (Bzdok et al., 2011; Raemaekers et al., 2007; Turkeltaub et al., 2012; Wager et al., 2007).”
InfluentialLogothetis, N. K., & Wandell, B. A. (2004) · Annual Review of Physiology · doi:10.1146/annurev.physiol.66.082602.092845 · Cited by 1,568 · cited 1×
“Such structured integration is enabled by community-wide standards of spatial normalization (Amunts et al., 2014; Evans et al., 2012; Mazziotta et al., 2001) and the convention of reporting peak locations in stereotaxic reference spaces.”
Flagged first, then by citations
1 of the 76 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Variability in the analysis of a single neuroimaging dataset by many teams
(2020) · doi:10.1038/s41586-020-2314-9 · Cited by 1,097
Self-citationNeural correlates of action: Comparing meta-analyses of imagery, observation, and execution
Hardwick, R. M., Caspers, S., Eickhoff, S. B., & Swinnen, S. P. (2018) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2018.08.003 · Cited by 948
Self-citationTen simple rules for neuroimaging meta-analysis
Müller, V. I., Cieslik, E. C., Laird, A. R., Fox, P. T., Radua, J., Mataix-Cols, D., Tench, C. R., Yarkoni, T., Nichols, T. E., Turkeltaub, P. E., et al. (2018) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2017.11.012 · Cited by 805
Self-citationMcTeague, L. M., Huemer, J., Carreon, D. M., Jiang, Y., Eickhoff, S. B., & Etkin, A. (2017) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2017.16040400 · Cited by 635
“Domain-specific ALE (as well as domain-by-disorder analyses) was performed for contrasts with at least 20 experiments (31), and these are reported in the online data supplement (see Figures S12 and S13 and Tables S19 and S20).”
BackgroundMethodInfluentialSelf-citationZhou, H.-X., Chen, X., Shen, Y.-Q., Li, L., Chen, N.-X., Zhu, Z.-C., Castellanos, F. X., & Yan, C.-G. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2019.116287 · Cited by 656
“Moreover, the limited number of typical rumination/distraction studies provides equivocal statistical power for meta-analysis (Eickhoff et al., 2016).”
BackgroundSchurz, M., Raduà, J., Tholen, M. G., Maliske, L., Margulies, D. S., Mars, R. B., Sallet, J., & Kanske, P. (2020) · Psychological Bulletin · doi:10.1037/bul0000303 · Cited by 702
“The SDM method is based on the positive features from existing peak probability methods for meta-analysis, such as activation likelihood estimation (ALE, Eickhoff et al., 2016) or multilevel kernel density analysis (MKDA, Wager, Lindquist, & Kaplan, 2007).”
MethodReward Processing in Depression: A Conceptual and Meta-Analytic Review Across fMRI and EEG Studies
Keren, H., O’Callaghan, G., Vidal-Ribas, P., Buzzell, G. A., Brotman, M. A., Leibenluft, E., Pan, P. M., Meffert, L., Kaiser, A., Wolke, S., et al. (2018) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2018.17101124 · Cited by 521
“Because of the small number of studies, we combined loss anticipation and feedback (88), and this may have diluted effects.”
BackgroundGray, J. P., Müller, V. I., Eickhoff, S. B., & Fox, P. T. (2020) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2019.19050560 · Cited by 449
“(20), the minimum number of experiments required for robust ALE analysis is 17.”
BackgroundMethodInfluentialSelf-citationResting‐state network dysfunction in Alzheimer's disease: A systematic review and meta‐analysis
Badhwar, A., Tam, A., Dansereau, C. L., Orban, P., Hoffstaedter, F., & Bellec, P. (2017) · Alzheimer s & Dementia Diagnosis Assessment & Disease Monitoring · doi:10.1016/j.dadm.2017.03.007 · Cited by 453
“, scanner make and model, scanning parameters), and processing choices [13,20,61].”
Self-citationOldham, S., Murawski, C., Fornito, A., Youssef, G. J., Yücel, M., & Lorenzetti, V. (2018) · Human Brain Mapping · doi:10.1002/hbm.24184 · Cited by 449
“A false-discovery rate (FDR) threshold, which has often been used in previous meta-analyses, was not used as this method is highly susceptible to false-positives (Chumbley & Friston, 2009; Eickhoff et al., 2012, 2016).”
BackgroundMethodResultInfluentialMorawetz, C., Bode, S., Derntl, B., & Heekeren, H. R. (2017) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.11.014 · Cited by 458
Common Dysfunction of Large-Scale Neurocognitive Networks Across Psychiatric Disorders
Sha, Z., Wager, T. D., Mechelli, A., & He, Y. (2019) · Biological Psychiatry · doi:10.1016/j.biopsych.2018.11.011 · Cited by 367
Large-scale functional neural network correlates of response inhibition: an fMRI meta-analysis
Zhang, R., Geng, X., & Lee, T. M. C. (2017) · Brain Structure and Function · doi:10.1007/s00429-017-1443-x · Cited by 398
McTeague, L. M., Rosenberg, B. M., Lopez, J. W., Carreon, D. M., Huemer, J., Jiang, Y., Chick, C. F., Eickhoff, S. B., & Etkin, A. (2020) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2019.18111271 · Cited by 388
“Subparcellation of the dorsomedial prefrontal cortex has further specified that the observed cluster of convergence comprises two smaller contiguous regions, with the left strongly connected to the fronto-parietal network and the right strongly connected to the salience network (23, 43) (see Figure S21 in the online s…”
BackgroundMethodSelf-citationAltered Brain Activity in Unipolar Depression Revisited: Meta-analyses of Neuroimaging Studies
Müller, V. I., Cieslik, E. C., Serbanescu, I., Laird, A. R., Fox, P. T., & Eickhoff, S. B. (2017) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2016.2783 · Cited by 304
Self-citationImplementation errors in the GingerALE Software: Description and recommendations
Eickhoff, S. B., Laird, A. R., Fox, P. M., Lancaster, J. L., & Fox, P. T. (2016) · Human Brain Mapping · doi:10.1002/hbm.23342 · Cited by 270
Self-citationHein, T. C., & Monk, C. S. (2017) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.12651 · Cited by 257
“The cluster-wise family-wise error (FWE) correction was employed to correct for multiple comparisons at a significance threshold of p .05 (Eickhoff et al., 2016).”
Berboth, S., & Morawetz, C. (2021) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2021.107767 · Cited by 277
Samea, F., Soluki, S., Nejati, V., Zarei, M., Cortese, S., Eickhoff, S. B., Tahmasian, M., & Eickhoff, C. R. (2019) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2019.02.011 · Cited by 245
“Furthermore, we probed for convergence among task-fMRI studies when focusing only on specific cognitive domains and found a sufficient number of experiments (> 17) (Eickhoff et al., 2016) for inhibitory control and attention, but also performed a joint analyses on the remaining cognitive tasks (memory, timing, reasoni…”
InfluentialSelf-citationNg, T. H., Alloy, L. B., & Smith, D. V. (2019) · Translational Psychiatry · doi:10.1038/s41398-019-0644-x · Cited by 244
Arithmetic in the developing brain: A review of brain imaging studies
Peters, L., & Smedt, B. D. (2017) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2017.05.002 · Cited by 259
Multiple large-scale neural networks underlying emotion regulation
Morawetz, C., Riedel, M. C., Salo, T., Berboth, S., Eickhoff, S. B., Laird, A. R., & Kohn, N. (2020) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.07.001 · Cited by 226
Self-citationNeuroimaging the menstrual cycle: A multimodal systematic review
Dubol, M., Epperson, C. N., Sacher, J., Pletzer, B. A., Derntl, B., Lanzenberger, R. R., Poromaa, I. S., & Comasco, E. (2020) · Frontiers in Neuroendocrinology · doi:10.1016/j.yfrne.2020.100878 · Cited by 204
“…of different study designs and limited sample size, no quantitative meta-analysis was performed on the reviewed studies, which did not meet the requirements ensuring robust results from an Activation Likelihood Estimation coordinate-based meta-analysis (Eickhoff et al., 2016; Muller et al., 2018).”
BackgroundNovel findings from 2,838 Adult Brains on Sex Differences in Gray Matter Brain Volume
Lotze, M., Domin, M., Gerlach, F. H., Gaser, C., Lueders, E., Schmidt, C. O., & Neumann, N. (2019) · Scientific Reports · doi:10.1038/s41598-018-38239-2 · Cited by 252
Gray matter asymmetries in aging and neurodegeneration: a review and meta-analysis.
Minkova, L., Habich, A., Peter, J., Kaller, C. P., Eickhoff, S. B., & Klöppel, S. (2017) · Human Brain Mapping · doi:10.1002/hbm.23772 · Cited by 194
“Finally, as recommended by a recent simulation study [Eickhoff et al., 2016], the resulting P-values were thresholded at P 0.05, cluster-level family-wise error (cFWE) r 5893 r corrected (cluster-forming threshold at voxel-level P 0.001) and transformed into Z-scores for display.”
BackgroundMethodInfluentialSelf-citationCortese, S., Aoki, Y. Y., Itahashi, T., Castellanos, F. X., & Eickhoff, S. B. (2020) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2020.08.014 · Cited by 199
Self-citationDisrupted Dorsal Mid-Insula Activation During Interoception Across Psychiatric Disorders
Nord, C. L., Lawson, R. P., & Dalgleish, T. (2021) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2020.20091340 · Cited by 171
“Approximately 17-20 experimental datasets are needed for ALE to be adequately powered to detect small effects, and ensure results are not driven by single experiments (47).”
BackgroundMethodComputing the Social Brain Connectome Across Systems and States
Alcalá-López, D., Smallwood, J., Jefferies, E., Overwalle, F. V., Vogeley, K. T., Mars, R. B., Turetsky, B. I., Laird, A. R., Fox, P. T., Eickhoff, S. B., et al. (2017) · Cerebral Cortex · doi:10.1093/cercor/bhx121 · Cited by 183
“this significance testing procedure was found beneficial for quantitative meta-analysis experiments based on the ALE algorithm in a recent systematic evaluation (Eickhoff et al. 2016).”
MethodInfluentialSelf-citationYaple, Z. A., Stevens, W. D., & Arsalidou, M. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.03.074 · Cited by 190
“…statistical maps were thresholded at p 0.05 using a cluster-level correction for multiple comparisons and a cluster forming threshold at p 0.001 (Eickhoff et al., 2017), rather than false discovery rate that is not appropriate for inference on topological features (Eickhoff et al., 2016).”
MethodInfluentialJaniri, D., Moser, D. A., Doucet, G. E., Luber, M. J., Rasgon, A., Lee, W. H., Murrough, J. W., Sani, G., Eickhoff, S. B., & Frangou, S. (2020) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2019.3351 · Cited by 173
Self-citationDefinition and characterization of an extended multiple-demand network
Camilleri, J. A., Müller, V. I., Fox, P. M., Laird, A. R., Hoffstaedter, F., Kalenscher, T., & Eickhoff, S. B. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.10.020 · Cited by 162
Self-citationNeural signatures of trust in reciprocity: A coordinate‐based meta‐analysis
Bellucci, G., Chernyak, S. V., Goodyear, K., Eickhoff, S. B., & Krueger, F. (2017) · Human Brain Mapping · doi:10.1002/hbm.23451 · Cited by 160
“These analyses confirmed that our results meet criteria for robust, unbiased results reflecting “true” convergence [Eickhoff et al., 2016].”
MethodResultInfluentialSelf-citationCan't get it off my brain: Meta-analysis of neuroimaging studies on perseverative cognition
Makovac, E., Fagioli, S., Rae, C. L., Critchley, H., & Ottaviani, C. (2019) · Psychiatry Research Neuroimaging · doi:10.1016/j.pscychresns.2019.111020 · Cited by 165
Acar, F., Seurinck, R., Eickhoff, S. B., & Moerkerke, B. (2018) · PLOS ONE · doi:10.1371/journal.pone.0208177 · Cited by 154
“The authors of the ALE algorithm advise to employ a voxel- or cluster-level family-wise error correction (FWE) [28].”
BackgroundMethodResultInfluentialSelf-citationHartwigsen, G., Neef, N. E., Camilleri, J. A., Margulies, D. S., & Eickhoff, S. B. (2018) · Cerebral Cortex · doi:10.1093/cercor/bhy049 · Cited by 167
“The resulting individual coactivation profiles were then compared between voxels to identify clusters of voxels with similar coactivation patterns (Eickhoff et al. 2016b).”
MethodSelf-citationThe neural correlates of visual imagery: A co-ordinate-based meta-analysis
Winlove, C. I. P., Milton, F., Ranson, J., Fulford, J., MacKisack, M., Macpherson, F., & Zeman, A. (2018) · Cortex · doi:10.1016/j.cortex.2017.12.014 · Cited by 150
“05), an approach which provides the best compromise between sensitivity and specificity (Eickhoff et al., 2016).”
BackgroundInfluentialLangner, R., Leiberg, S., Hoffstaedter, F., & Eickhoff, S. B. (2018) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2018.04.022 · Cited by 156
Self-citationWalenski, M., Europa, E., Caplan, D. N., & Thompson, C. K. (2019) · Human Brain Mapping · doi:10.1002/hbm.24523 · Cited by 159
“Adank (2012b) reports a cluster in this region for language production in a meta-analysis based on 21 studies (we had 15); recent recommendations are for 20 studies to enable adequate power in cluster-based meta-analysis (Eickhoff et al., 2016).”
Jauniaux, J., Khatibi, A., Rainville, P., & Jackson, P. L. (2019) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsz055 · Cited by 154
Janacsek, K., Shattuck, K. F., Tagarelli, K. M., Lum, J. A. G., Turkeltaub, P. E., & Ullman, M. T. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.116387 · Cited by 142
“…if about one-third of the included experiments show the effect of interest (corresponding to an effect size of 0.33), 17 experiments are sufficient to achieve a desired power of 80% using the cFWE correction, though, in general, at least about 20 experiments are recommended (Eickhoff et al., 2016).”
InfluentialArioli, M., Cattaneo, Z., Ricciardi, E., & Canessa, N. (2021) · Human Brain Mapping · doi:10.1002/hbm.25570 · Cited by 151
“The number of studies included in most of these meta-analyses is in line with the recent prescriptions for ALE (Eickhoff et al., 2016; Muller et al., 2018), suggesting a minimum of 17 studies to ensure that results would not be driven by single experiments (see also Wu et al., 2019).”
ResultPractical recommendations to conduct a neuroimaging meta-analysis for neuropsychiatric disorders
Tahmasian, M., Sepehry, A. A., Samea, F., Khodadadifar, T., Soltaninejad, Z., Javaheripour, N., Khazaie, H., Zarei, M., Eickhoff, S. B., & Eickhoff, C. R. (2019) · Human Brain Mapping · doi:10.1002/hbm.24746 · Cited by 126
“It has been shown that voxel-wise FDR correction is not an optimal method and increases the chance of reporting the spurious clusters (Chumbley & Friston, 2009; Eickhoff et al., 2016).”
BackgroundMethodInfluentialSelf-citationNeural correlates of formal thought disorder: An activation likelihood estimation meta‐analysis
Wensing, T., Cieslik, E. C., Müller, V. I., Hoffstaedter, F., Eickhoff, S. B., & Nickl‐Jockschat, T. (2017) · Human Brain Mapping · doi:10.1002/hbm.23706 · Cited by 135
“…twofold: first, given the large variations in experimental designs and phenotype in view of an already rather small number of included studies, additional ALE meta-analyses tackling these questions would be unfeasible for the sake of statistical reliability and robustness [Eickhoff et al., 2016].”
BackgroundMethodInfluentialSelf-citationThe Neural Correlates of Time: A Meta-analysis of Neuroimaging Studies
Nani, A., Manuello, J., Liloia, D., Duca, S., Costa, T., & Cauda, F. (2019) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01459 · Cited by 109
“These two parameters are pivotal for the implementation of the ALE method and the identification of the effect (Eickhoff et al., 2016).”
BackgroundMethodLi, Y., Kong, F., Luo, Y., Zeng, S., Lan, J., & You, X. (2019) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2019.00128 · Cited by 130
Fallon, N. B., Roberts, C. A., & Stančák, A. (2020) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsaa090 · Cited by 138
“05) and a minimum cluster size of 200 mm3 as previously recommended (Eickhoff et al., 2016).”
BackgroundMethodInfluentialListening under difficult conditions: An activation likelihood estimation meta‐analysis
Alain, C., Du, Y., Bernstein, L. J., Barten, T. R. M., & Banai, K. (2018) · Human Brain Mapping · doi:10.1002/hbm.24031 · Cited by 127
“…and linguistic complexity—because those are the most commonly used paradigms and thus have a sufficiently large number of neuroimaging studies to assess patterns of brain activation (Eickhoff, Bzdok, Laird, Kurth, & Fox, 2012; Eickhoff, Laird, Fox, Lancaster, & Fox, 2017; Eickhoff et al., 2016).”
BackgroundMethodWhere is the “where” in the brain? A meta‐analysis of neuroimaging studies on spatial cognition
Cona, G., & Scarpazza, C. (2019) · Human Brain Mapping · doi:10.1002/hbm.24496 · Cited by 131
“001) (Eickhoff et al., 2016) according with the recent guidelines for coordinate-based meta-analysis (Muller et al.”
MethodInfluentialHuman Pregenual Anterior Cingulate Cortex: Structural, Functional, and Connectional Heterogeneity
Palomero‐Gallagher, N., Hoffstaedter, F., Mohlberg, H., Eickhoff, S. B., Amunts, K., & Zilles, K. (2018) · Cerebral Cortex · doi:10.1093/cercor/bhy124 · Cited by 126
Self-citationTahmasian, M., Noori, K., Samea, F., Zarei, M., Spiegelhalder, K., Eickhoff, S. B., Someren, E. J. W. V., Khazaie, H., & Eickhoff, C. R. (2018) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2018.07.004 · Cited by 120
Self-citationDo I feel or do I know? Neuroimaging meta-analyses on the multiple facets of empathy
Kogler, L., Müller, V. I., Werminghausen, E., Eickhoff, S. B., & Derntl, B. (2020) · Cortex · doi:10.1016/j.cortex.2020.04.031 · Cited by 134
“Analyses with a higher number of included experiments can have a higher power (Eickhoff et al., 2016).”
BackgroundMethodSelf-citationSutherland, M. T., Riedel, M. C., Flannery, J. S., Yanes, J. A., Fox, P. T., Stein, E. A., & Laird, A. R. (2016) · Behavioral and Brain Functions · doi:10.1186/s12993-016-0100-5 · Cited by 114
Self-citationXu, A., Larsen, B., Baller, E. B., Scott, J. C., Sharma, V., Adebimpe, A., Basbaum, A. I., Dworkin, R. H., Edwards, R. R., Woolf, C. J., et al. (2020) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.01.004 · Cited by 116
“, 2016) as well as an influx of articles standardizing rigorous procedures for meta-analyses (Eickhoff et al., 2016; Müller et al., 2018) suggest that an update is warranted.”
BackgroundSelf-citationSocial exclusion reliably engages the default network: A meta-analysis of Cyberball
Mwilambwe-Tshilobo, L., & Spreng, R. N. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.117666 · Cited by 116
Affordance processing in segregated parieto-frontal dorsal stream sub-pathways
Sakreida, K., Alkhasli, I., Thill, S., Menz, M. M., Jirak, D., Eickhoff, C. R., Tom Ziemke, B., Eickhoff, S. B., Borghi, A. M., & Binkofski, F. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.07.032 · Cited by 110
Self-citationFunctional neuroimaging of human postural control: a systematic review with meta-analysis.
Dijkstra, B. W., Bekkers, E., Gilat, M., de Rond, V., Hardwick, R., & Nieuwboer, A. (2020) · Neuroscience and Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.04.028 · Cited by 107
Doucet, G. E., Janiri, D., Howard, R., O’Brien, M., Andrews‐Hanna, J. R., & Frangou, S. (2020) · European Psychiatry · doi:10.1192/j.eurpsy.2020.57 · Cited by 118
“Diagnosis-specific ALE analyses were only performed for schizophrenia (number of studies: 31) and MDD (number of studies: 22); the data for all other disorders were insufficient as the minimum number of studies for robust results is currently considered to be 17 [40].”
BackgroundNeurobiological and Cognitive Profile of Young Binge Drinkers: a Systematic Review and Meta-Analysis
Lees, B., Mewton, L., Stapinski, L. A., Squeglia, L. M., Rae, C. D., & Teesson, M. (2019) · Neuropsychology Review · doi:10.1007/s11065-019-09411-w · Cited by 109
“…(Muller et al., 2018) that requires all included experiments use the same search coverage (i.e., the same brain coverage, EEG and ERP components, neurocognitive domains) and that there were a sufficient number of studies included in the analysis (i.e., >17–20 experiments; Eickhoff et al., 2016).”
A view behind the mask of sanity: meta-analysis of aberrant brain activity in psychopaths
Poeppl, T. B., Donges, M. R., Mokros, A., Rupprecht, R., Fox, P. T., Laird, A. R., Bzdok, D., Langguth, B., & Eickhoff, S. B. (2018) · Molecular Psychiatry · doi:10.1038/s41380-018-0122-5 · Cited by 124
Self-citationSchulze, L., Schulze, A., Renneberg, B., Schmahl, C., & Niedtfeld, I. (2019) · Biological Psychiatry: Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2018.11.004 · Cited by 102
Minimum statistical standards for submissions to Neuroimage: Clinical
Roiser, J. P., Linden, D. E. J., Gorno‐Tempini, M., Moran, R., Dickerson, B. C., & Grafton, S. T. (2016) · NeuroImage Clinical · doi:10.1016/j.nicl.2016.08.002 · Cited by 106
The search for the number form area: A functional neuroimaging meta-analysis
Yeo, D. J. S., Wilkey, E. D., & Price, G. R. (2017) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2017.04.027 · Cited by 108
Dricu, M., & Frühholz, S. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.10.020 · Cited by 102
Papitto, G., Friederici, A. D., & Zaccarella, E. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.116321 · Cited by 101
Convergence of semantics and emotional expression within the IFG pars orbitalis
Belyk, M., Brown, S., Lim, J., & Kotz, S. A. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.04.020 · Cited by 105
Resting-state abnormalities in Autism Spectrum Disorders: A meta-analysis
Lau, W. K., Leung, M.-K., & Lau, B. W.-M. (2019) · Scientific Reports · doi:10.1038/s41598-019-40427-7 · Cited by 111
Neural signatures of prosocial behaviors
Bellucci, G., Camilleri, J. A., Eickhoff, S. B., & Krüeger, F. (2020) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.07.006 · Cited by 111
“This average non-linear contribution to the ALE value was computed from the ratio of the ALE values at the location of the cluster with and without each contributing experiment (Eickhoff et al., 2016).”
MethodSelf-citationBerretz, G., Packheiser, J., Kumsta, R., Wolf, O. T., & Ocklenburg, S. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2021.01.001 · Cited by 115
N -back Working Memory Task: Meta-analysis of Normative fMRI Studies With Children
Yaple, Z. A., & Arsalidou, M. (2018) · Child Development · doi:10.1111/cdev.13080 · Cited by 115
“A minimum of 17–20 experiments is currently recommended to ensure the robustness of each cluster and sufficient statistical power (Eickhoff et al., 2016, 2017).”
BackgroundKlugah‐Brown, B., Di, X., Zweerings, J., Mathiak, K., Becker, B., & Biswal, B. (2020) · Human Brain Mapping · doi:10.1002/hbm.25085 · Cited by 86
“In line with recent recommendations for the application of cluster-based correction methods (Eickhoff et al., 2016) an initial cluster forming threshold of p < .”
MethodBellucci, G., Feng, C., Camilleri, J. A., Eickhoff, S. B., & Krüeger, F. (2018) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2018.06.024 · Cited by 99
“Finally, it has to be noted that we did not include contrasts for reciprocity decisions in the one-shot TG, distrust decisions in the iterative TG and decisions to propose offers in the one-shot UG, because we could not find enough published studies reporting contrasts for these decisions to reach a reasonable number…”
InfluentialSelf-citationMultiple dimensions underlying the functional organization of the language network
Hodgson, V. J., Ralph, M. A. L., & Jackson, R. L. (2021) · NeuroImage · doi:10.1016/j.neuroimage.2021.118444 · Cited by 104
Li, Q., Yang, G., Li, Z., Qi, Y., Cole, M. W., & Liu, X. (2017) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2017.09.032 · Cited by 96
Cognitive benefits of exercise interventions: an fMRI activation likelihood estimation meta-analysis
Yu, Q., Herold, F., Becker, B., Klugah‐Brown, B., Zhang, Y., Perrey, S., Veronese, N., Müller, N. G., Kramer, A. F., & Zou, L. (2021) · Brain Structure and Function · doi:10.1007/s00429-021-02247-2 · Cited by 104
Feng, C., Eickhoff, S. B., Li, T., Wang, L., Becker, B., Camilleri, J. A., Hétu, S., & Luo, Y. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2021.03.025 · Cited by 106
Self-citationGanesan, S., Beyer, E., Moffat, B. A., Dam, N. T. V., Lorenzetti, V., & Zalesky, A. (2022) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2022.104846 · Cited by 95
Xu, P., Peng, S., Luo, Y., & Gong, G. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2021.05.023 · Cited by 102
Stimuli, presentation modality, and load‐specific brain activity patterns during n‐back task
Mencarelli, L., Neri, F., Momi, D., Menardi, A., Rossi, S., Rossi, A., & Santarnecchi, E. (2019) · Human Brain Mapping · doi:10.1002/hbm.24633 · Cited by 94
“For example, based on a recent simulation study (Eickhoff et al., 2016), it is known that the results of a meta-analysis with a low number of papers could be driven by few experiments (as evidenced by our analysis, see Supporting Information Figure S5).”
BackgroundRodríguez-Nieto, G., Seer, C., Sidlauskaite, J., Vleugels, L., Van Roy, A., Hardwick, R., & Swinnen, S. (2022) · NeuroImage · doi:10.1016/j.neuroimage.2022.119665 · Cited by 89
BrainMap VBM: An environment for structural meta‐analysis
Vanasse, T. J., Fox, P. M., Barron, D. S., Robertson, M., Eickhoff, S. B., Lancaster, J. L., & Fox, P. T. (2018) · Human Brain Mapping · doi:10.1002/hbm.24078 · Cited by 82
“…likelihood estimation (ALE; performed with GingerALE software), multi-level kernel density analysis (MKDA) and signed differential mapping (SDM) (Eickhoff et al., 2016; Radua & Mataix-Cols, 2012; Turkel-taub, Eden, Jones, & Zeffiro, 2002; Wager, Lindquist, Nichols, Kober, & Van Snellenberg,…”
MethodInfluentialSelf-citationA meta-analysis of fMRI studies of language comprehension in children
Enge, A., Friederici, A. D., & Skeide, M. A. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.116858 · Cited by 91
“The cluster-wise FWE-corrected threshold was preferred in accordance with a recent simulation study by Eickhoff et al. (2016).”
BackgroundMethodInfluentialZhao, Y., Yang, L., Gong, G., Cao, Q., & Liu, J. (2021) · Progress in Neuro-Psychopharmacology and Biological Psychiatry · doi:10.1016/j.pnpbp.2021.110477 · Cited by 87
Tagarelli, K. M., Shattuck, K. F., Turkeltaub, P. E., & Ullman, M. T. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.02.061 · Cited by 92
Differential vulnerability of the cerebellum in healthy ageing and Alzheimer’s disease
Gellersen, H. M., Guell, X., & Sami, S. A. K. (2021) · NeuroImage Clinical · doi:10.1016/j.nicl.2021.102605 · Cited by 85
“In the case of this analysis, we chose a p .05 for cluster-level family-wise error correction (i.e. the cluster-level inference option on GingerALE), which provides the best compromise between sensitivity and specificity (Eickhoff et al., 2016).”
Wang, J., Zhang, J., Zang, Y., & Wu, T. (2018) · GigaScience · doi:10.1093/gigascience/giy071 · Cited by 80
“First, it has been proposed that at least 20 experiments should be included in a meta-analysis [72].”
Brain structural alterations are distributed following functional, anatomic and genetic connectivity
Cauda, F., Nani, A., Manuello, J., Premi, E., Palermo, S., Tatu, K., Duca, S., Fox, P. T., & Costa, T. (2018) · Brain · doi:10.1093/brain/awy252 · Cited by 77
“However, a minimum amount of 20 experiments is usually thought to be sufficient to resolve this issue (Eickhoff et al., 2016), so that analyses based on large databases, as the one used here, should not be so much biased as to produce invalid results.”
BackgroundSelf-citationJavaheripour, N., Shahdipour, N., Noori, K., Zarei, M., Camilleri, J. A., Laird, A. R., Fox, P. T., Eickhoff, S. B., Eickhoff, C. R., Rosenzweig, I., et al. (2019) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2019.03.008 · Cited by 85
“Of note, there was not enough experiment per task to perform a statistically sound CBMA [27].”
BackgroundMethodInfluentialSelf-citationNeural correlates of sense of agency in motor control: a neuroimaging meta-analysis
Zito, G. A., Wiest, R., & Aybek, S. (2020) · PLOS ONE · doi:10.1371/journal.pone.0234321 · Cited by 79
“05 corrected with Family-Wise Error (FWE), and 5000 permutation tests to correct for multiple comparisons [49].”
MethodSindermann, L., Redlich, R., Opel, N., Böhnlein, J., Dannlowski, U., & Leehr, E. J. (2021) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2021.07.022 · Cited by 80
Arioli, M., & Canessa, N. (2019) · Human Brain Mapping · doi:10.1002/hbm.24627 · Cited by 82
Zheng, C. J., Drunen, S. V., & Egorova, N. (2022) · Translational Psychiatry · doi:10.1038/s41398-022-01949-3 · Cited by 77
“In fact, both our ALE analyses of pain in depression and depression in pain likely are low in statistical power [23] and should be interpreted with caution.”
BackgroundEres, R., Louis, W. R., & Molenberghs, P. (2017) · Social Neuroscience · doi:10.1080/17470919.2017.1357657 · Cited by 85
Luo, Y., Eickhoff, S. B., Hétu, S., & Feng, C. (2017) · Human Brain Mapping · doi:10.1002/hbm.23854 · Cited by 82
“In addition, the average non-linear contribution of each experiment for each cluster was calculated from the fraction of the ALE values at the cluster with and without the experiment in question [Eickhoff et al., 2016].”
BackgroundMethodSelf-citationLiloia, D., Brasso, C., Cauda, F., Mancuso, L., Nani, A., Manuello, J., Costa, T., Duca, S., & Rocca, P. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2021.01.010 · Cited by 71
“g-HR = 6 experiments; c-HR = 4; RDSZ = 7; ChSZ = 7; Table S3) was insufficient to achieve robust ALE estimates (Eickhoff et al., 2016).”
BackgroundMethodResultInfluentialFunctional neural correlates of psychopathy: a meta-analysis of MRI data
Deming, P., & Koenigs, M. R. (2020) · Translational Psychiatry · doi:10.1038/s41398-020-0816-8 · Cited by 81
Naghibi, N., Jahangiri, N., Khosrowabadi, R., Eickhoff, C. R., Eickhoff, S. B., Coull, J. T., & Tahmasian, M. (2024) · Neuropsychology Review · doi:10.1007/s11065-023-09588-1 · Cited by 69
“However, we could not investigate those methodological impacts because the number of studies in some subcategories (e.g. PET studies) fell below the minimum number required to control the excessive contribution of individual experiments (Eickhoff et al., 2016).”
BackgroundMethodInfluentialSelf-citationImpact of early life adversities on human brain functioning: A coordinate-based meta-analysis
Kraaijenvanger, E. J., Pollok, T. M., Monninger, M., Kaiser, A., Brandeis, D., Banaschewski, T., & Holz, N. E. (2020) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.03.008 · Cited by 83
Brain responses to social norms: Meta‐analyses of f MRI studies
Zinchenko, O. O., & Arsalidou, M. (2017) · Human Brain Mapping · doi:10.1002/hbm.23895 · Cited by 74
“Although there are no available methods for performing robustness analyses with GingerALE, simulations of ALE analyses have been performed to test sensitivity, ensuing cluster sizes, number of incidental clusters, and statistical power (Eickhoff et al., 2016).”
BackgroundMethodMultimodal evaluation of the amygdala’s functional connectivity
Kerestes, R., Chase, H., Phillips, M., Ladouceur, C., & Eickhoff, S. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2016.12.023 · Cited by 75
“experiments (Eickhoff et al., 2016), we normalized all MACM maps to a sum of one.”
MethodSelf-citationDisner, S. G., Marquardt, C. A., Mueller, B. A., Burton, P. C., & Sponheim, S. R. (2017) · Human Brain Mapping · doi:10.1002/hbm.23886 · Cited by 70
“…that the more conservative cluster-level FWE correction employed in the present analysis was recommended following massive empirical simulation (Eickhoff et al., 2016), we are confident that the presently reported ROIs reflect the most reliable meta-analytic outcomes available based on the…”
Method
Metadata
- Authors
- Eickhoff, S. B., Nichols, T. E., Laird, A. R., Hoffstaedter, F., Amunts, K., Fox, P. T., Bzdok, D., & Eickhoff, C. R.
- Affiliations
- Forschungszentrum Jülich (DE)
- University of Warwick (GB)
- Florida International University (US)
- The University of Texas at San Antonio Health Science Center (US)
- Institut national de recherche en sciences et technologies du numérique (FR)
- Corresponding author
- Eickhoff, Simon B.
- ORCID
- Venue
- NeuroImage
- Publisher
- Elsevier BV
- Published
- 2016-08
- Volume
- 137
- Pages
- 70-85
- ISSN
- 1053-8119 (ISSN-L)
- PMC
- PMC4981641
- OpenAlex
- W2349406327
- Cited by
- 683
- References
- 76
- Citations by year
- 2026: 61 · 2025: 68 · 2024: 58 · 2023: 83 · 2022: 92 · 2021: 93 · 2020: 72 · 2019: 61 · 2018: 46 · 2017: 35 · 2016: 13
- FWCI
- 32.88
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- NeuroImage (publishedVersion)
- PubMed (publishedVersion)
- Warwick Research Archive Portal (University of Warwick) (acceptedVersion)
- PubMed Central (acceptedVersion)
- RWTH Publications (RWTH Aachen) (submittedVersion)
- Funders
- Deutsche Forschungsgemeinschaft (EI 816/4-1, LA 3071/3-1, EI 816/6-1)
- National Institute of Mental Health (R01-MH074457)
- Helmholtz Portfolio Theme “Supercomputing and Modeling for the Human Brain” and the European Union Seventh Framework Program (604102)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Functional Brain Connectivity Studies
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Advanced Neuroimaging Techniques and Applications
- Keywords
- activation likelihood estimation, coordinate-based meta-analysis, statistical thresholding, false discovery rate, sample size, statistical power, brain-behavior relationship, BrainMap
- MeSH terms
- Brain, Data Interpretation, Statistical, Humans, Image Interpretation, Computer-Assisted, Sensitivity and Specificity, Meta-Analysis as Topic, Reproducibility of Results, Likelihood Functions, Neuroimaging
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
