The DOI matches its Crossref record; 46 of the 48 references it cites were found in a registry or an index.
Assessing robustness against potential publication bias in Activation Likelihood Estimation (ALE) meta-analyses for fMRI
Open accessAbstract
The importance of integrating research findings is incontrovertible and procedures for coordinate-based meta-analysis (CBMA) such as Activation Likelihood Estimation (ALE) have become a popular approach to combine results of fMRI studies when only peaks of activation are reported. As meta-analytical findings help building cumulative knowledge and guide future research, not only the quality of such analyses but also the way conclusions are drawn is extremely important. Like classical meta-analyses, coordinate-based meta-analyses can be subject to different forms of publication bias which may impact results and invalidate findings. The file drawer problem refers to the problem where studies fail to get published because they do not obtain anticipated results (e.g. due to lack of statistical significance). To enable assessing the stability of meta-analytical results and determine their robustness against the potential presence of the file drawer problem, we present an algorithm to determine the number of noise studies that can be added to an existing ALE fMRI meta-analysis before spatial convergence of reported activation peaks over studies in specific regions is no longer statistically significant. While methods to gain insight into the validity and limitations of results exist for other coordinate-based meta-analysis toolboxes, such as Galbraith plots for Multilevel Kernel Density Analysis (MKDA) and funnel plots and egger tests for seed-based d mapping, this procedure is the first to assess robustness against potential publication bias for the ALE algorithm. The method assists in interpreting meta-analytical results with the appropriate caution by looking how stable results remain in the presence of unreported information that may differ systematically from the information that is included. At the same time, the procedure provides further insight into the number of studies that drive the meta-analytical results. We illustrate the procedure through an example and test the effect of several parameters through extensive simulations. Code to generate noise studies is made freely available which enables users to easily use the algorithm when interpreting their results.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“By pooling together numerous studies, ALE also allows for an increased power to detect true effects (Acar et al., 2018).”
From Bridging phenomenology and neural mechanisms of inner speech: ALE meta-analysis on egocentricity and spontaneity in a dual-mechanistic framework (2023) · cites this paper as Acar et al., 2018
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722 · cited 1×
“The Egger regression test estimates funnel plot asymmetry and can therefore be used as an indicator for the presence of publication bias [32].”
MethodDuval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099 · cited 1×
“In classical meta-analysis techniques (based on summarizing effect sizes), this is for instance possible with the trim and fill method [41], that looks at the asymmetry present in a funnel plot and tries to fill the gap.”
MethodThe file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068 · cited 3×
“The original FSN [21] represents the required number of findings with an effect of zero to change the conclusions of the meta-analysis under the assumption that these findings are still in the “file drawer”.”
BackgroundInfluentialSterne, J. A. C., Sutton, A. J., Ioannidis, J. P. A., Terrin, N., Jones, D. R., Lau, J., Carpenter, J., Rucker, G., Harbord, R. M., Schmid, C. H., et al. (2011) · BMJ · doi:10.1136/bmj.d4002 · Cited by 7,709
Some Methods for Strengthening the Common χ 2 Tests
Cochran, W. G. (1954) · Biometrics · doi:10.2307/3001616 · Cited by 3,718
What we can do and what we cannot do with fMRI
Logothetis, N. K. (2008) · Nature · doi:10.1038/nature06976 · Cited by 3,612 · cited 1×
“Functional magnetic resonance imaging (fMRI) continues to contribute greatly to the knowledge about the location of cognitive functions in the brain [1].”
BackgroundPublication Bias in Meta‐Analysis: Prevention, Assessment and Adjustments
(2005) · doi:10.1002/0470870168 · Cited by 2,368
Eickhoff, S. B., Laird, A. R., Grefkes, C., Wang, L. E., Zilles, K., & Fox, P. T. (2009) · Human Brain Mapping · doi:10.1002/hbm.20718 · Cited by 1,924 · cited 1×
“We select a meta-analysis on finger tapping [7,38] from the BrainMap database that was used to demonstrate the working of GingerALE [15–17].”
MethodSelf-citationPublication bias in meta-analysis: Prevention, assessment and adjustments
Viechtbauer, W. (2007) · Psychometrika · doi:10.1007/s11336-006-1450-y · Cited by 1,508 · cited 3×
“We illustrate the FSN with an example from the book “Publication Bias in Meta-Analysis” [20].”
BackgroundMethodInfluentialActivation Likelihood Estimation meta-analysis revisited
Eickhoff, S., Bzdok, D., Laird, A., Kurth, F., & Fox, P. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.09.017 · Cited by 1,448
Self-citation
Research works printed but not linked: 3
- J Radua, 2012, Meta-analytic methods for neuroimaging data explained, Biology of Mood & Anxiety Disorders, 2, 1
- SG Costafreda, 2012, Parametric coordinate-based meta-analysis: Valid effect size meta-analysis of studies with differing statistical thresholds, Journal of Neuroscience Methods, 210, 291, 10.1016/j.jneumeth.2012.07.016 · DOI
- A Laird, 2008, Brainmap: The social evolution of a functional neuroimaging database, Neuroinformatics, 65
Cited by
Sorted by citations
Gray, J. P., Müller, V. I., Eickhoff, S. B., & Fox, P. T. (2020) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2019.19050560 · Cited by 449
“(15) to estimate the robustness of identified results against unpublished neuroimaging findings.”
BackgroundMethodSelf-citationPando‐Naude, V., Toxto, S., Fernandez‐Lozano, S., Parsons, C. E., Alcauter, S., & Garza‐Villarreal, E. A. (2021) · Translational Psychiatry · doi:10.1038/s41398-020-01128-2 · Cited by 195
Current Challenges in Translational and Clinical fMRI and Future Directions
Specht, K. (2020) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2019.00924 · Cited by 130
The Neural Correlates of Time: A Meta-analysis of Neuroimaging Studies
Nani, A., Manuello, J., Liloia, D., Duca, S., Costa, T., & Cauda, F. (2019) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01459 · Cited by 109
“Noise experiments are then created randomly by combining values of sample size and amount of reported foci per experiment taken from the respective real distributions (Acar et al., 2018).”
BackgroundMethodInfluentialAntoniou, G., Lambourg, E. J., Steele, J. D., & Colvin, L. (2023) · British Journal of Anaesthesia · doi:10.1016/j.bja.2023.03.008 · Cited by 133
“Thus, traditional measures of heterogeneity are not applicable.(141,143,144) Second, the limited number of neuro-”
BackgroundFunctional neuroimaging of human postural control: a systematic review with meta-analysis.
Dijkstra, B. W., Bekkers, E., Gilat, M., de Rond, V., Hardwick, R., & Nieuwboer, A. (2020) · Neuroscience and Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.04.028 · Cited by 107
McGrath, L. M., & Stoodley, C. J. (2019) · Journal of Neurodevelopmental Disorders · doi:10.1186/s11689-019-9287-8 · Cited by 87
“Such random noise would diminish the statistical evidence for convergence of true effects across studies [129].”
BackgroundCan't get it off my brain: Meta-analysis of neuroimaging studies on perseverative cognition
Makovac, E., Fagioli, S., Rae, C. L., Critchley, H., & Ottaviani, C. (2019) · Psychiatry Research Neuroimaging · doi:10.1016/j.pscychresns.2019.111020 · Cited by 165
A meta-analysis of fMRI studies of language comprehension in children
Enge, A., Friederici, A. D., & Skeide, M. A. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.116858 · Cited by 91
“Only for cluster 4 (left superior frontal gyrus), the value of fail-safe N ¼ 115 slightly exceeded the desired upper boundary (in this case, fail-safe N ¼ 113), potentially indicating that this cluster was driven by a very small number of experiments (Acar et al., 2018).”
BackgroundMethodResultInfluentialPollok, T. M., Kaiser, A., Kraaijenvanger, E. J., Monninger, M., Brandeis, D., Banaschewski, T., Eickhoff, S. B., & Holz, N. E. (2022) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2022.104589 · Cited by 80
Self-citationFicco, L., Mancuso, L., Manuello, J., Teneggi, A., Liloia, D., Duca, S., Costa, T., Kovács, G., & Cauda, F. (2021) · Scientific Reports · doi:10.1038/s41598-021-95603-5 · Cited by 82
The empirical replicability of task-based fMRI as a function of sample size
Bossier, H., Roels, S. P., Seurinck, R., Banaschewski, T., Barker, G. J., Bokde, A. L. W., Quinlan, E. B., Desrivières, S., Flor, H., Grigis, A., et al. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.116601 · Cited by 79
“…false (Ioannidis, 2005), (2) an overestimation of the reported effect sizes compared to the true effect size (Cremers et al., 2017) and (3) an increased risk of missed effects (i.e. false negatives) which potentially induces publication bias (Acar et al., 2018; Rosenthal, 1979; Sterling, 1959).”
BackgroundSelf-citationAn ALE meta-analytic review of top-down and bottom-up processing of music in the brain
Pando-Naude, V., Patyczek, A., Bonetti, L., & Vuust, P. (2021) · Scientific Reports · doi:10.1038/s41598-021-00139-3 · Cited by 62
An ALE meta-analytic review of musical expertise
Criscuolo, A., Pando‐Naude, V., Bonetti, L., Vuust, P., & Brattico, E. (2022) · Scientific Reports · doi:10.1038/s41598-022-14959-4 · Cited by 76
Chen, Y., Chaudhary, S., & Li, C. R. (2022) · NeuroImage · doi:10.1016/j.neuroimage.2022.119764 · Cited by 62
“3 Evaluation of publication bias We performed a “Fail-Safe N (FSN)” analysis to evaluate potential publication bias (Acar et al., 2018).”
MethodInfluentialLotter, L., Kohl, S. H., Gerloff, C., Bell, L., Niephaus, A., Kruppa, J. A., Dukart, J., Schulte-Rüther, M., Reindl, V., & Konrad, K. (2022) · bioRxiv · doi:10.1101/2022.07.26.501562 · Cited by 51
“Clusterwise robustness against publication bias was estimated as the fail-safe-N (Acar et al., 2018).”
Solly, J. E., Hook, R. W., Grant, J. E., Cortese, S., & Chamberlain, S. R. (2021) · Molecular Psychiatry · doi:10.1038/s41380-021-01315-7 · Cited by 63
Tasks activating the default mode network map multiple functional systems
Mancuso, L., Cavuoti-Cabanillas, S., Liloia, D., Manuello, J., Buzi, G., Duca, S., Cauda, F., & Costa, T. (2022) · Brain Structure and Function · doi:10.1007/s00429-022-02467-0 · Cited by 53
“The Fail-Safe algorithm (https:// github. com/ Neuro Stat/ Gener ateNu ll) proposed by Acar et al. (2018) generates a given number of experiments having random foci, to be added to the dataset to be tested.”
Schumer, M. C., Chase, H. W., Rozovsky, R., Eickhoff, S. B., & Phillips, M. L. (2023) · Molecular Psychiatry · doi:10.1038/s41380-023-01974-8 · Cited by 56
“Emotional tasks were then further separated into valence classes for post-hoc meta-analyses: Negative valence was operationalized as stimuli representing or invoking fear, sadness, anger, disgust, pain, loss, or punishment ; positive valence was operatio-nalized as stimuli representing or invoking…”
BackgroundSelf-citationPereira-Sánchez, V., Franco, A. R., Vieira, D. L., de Castro‐Manglano, P., Soutullo, C. A., Milham, M. P., & Castellanos, F. X. (2020) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2020.10.013 · Cited by 55
Estimating the prevalence of missing experiments in a neuroimaging meta-analysis
Samartsidis, P., Montagna, S., Laird, A., Fox, P., Johnson, T., & Nichols, T. E. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1448 · Cited by 47
Schizophrenia-related abnormalities in the triple network: a meta-analysis of working memory studies
Wu, D., & Jiang, T. (2019) · Brain Imaging and Behavior · doi:10.1007/s11682-019-00071-1 · Cited by 114
The Cerebellum in Frontotemporal Dementia: a Meta-Analysis of Neuroimaging Studies
Chen, Y., Kumfor, F., Landín-Romero, R., Irish, M., & Piguet, O. (2019) · Neuropsychology Review · doi:10.1007/s11065-019-09414-7 · Cited by 47
Distinct Convergent Brain Alterations in Sleep Disorders and Sleep Deprivation
Reimann, G. M., Hoseini, A., Koçak, M., Beste, M., Küppers, V., Rosenzweig, I., Elmenhorst, D., Pires, G. N., Laird, A. R., Fox, P. T., et al. (2025) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2025.0488 · Cited by 48
Self-citationCattarinussi, G., Kubera, K. M., Hirjak, D., Wolf, R. C., & Sambataro, F. (2022) · Biological Psychiatry · doi:10.1016/j.biopsych.2022.02.960 · Cited by 56
Roberts, C. A., Giesbrecht, T., Fallon, N., Thomas, A. C., Mela, D., & Kirkham, T. (2020) · Journal of NutriLife · doi:10.1093/jn/nxaa071 · Cited by 42
“These algorithms use kernel techniques for assessing spatial uncertainty around the reported peaks (36).”
MethodKahoot! Game‐based digital learning platform: A comprehensive meta‐analysis
Özdemir, O. (2025) · Journal of Computer Assisted Learning · doi:10.1111/jcal.13084 · Cited by 41
Parsing variability in borderline personality disorder: a meta-analysis of neuroimaging studies
Degasperi, G., Cristea, I., Rosa, E. D., Costa, C., & Gentili, C. (2021) · Translational Psychiatry · doi:10.1038/s41398-021-01446-z · Cited by 40
A coordinate-based meta-analysis of white matter alterations in patients with alcohol use disorder
Spindler, C., Mallien, L., Trautmann, S., Alexander, N., & Muehlhan, M. (2022) · Translational Psychiatry · doi:10.1038/s41398-022-01809-0 · Cited by 36
“To estimate the robustness of the results to potential null findings, we used a calculation of the Fail-Safe N (FSN) adapted for ALE as described by Acar and colleagues [36].”
MethodHumor and emotion: Quantitative meta analyses of functional neuroimaging studies
Farkas, A. H., Trotti, R. L., Edge, E. A., Huang, L.-Y., Kasowski, A., Thomas, O. F., Chlan, E., Granros, M., Patel, K., & Sabatinelli, D. (2021) · Cortex · doi:10.1016/j.cortex.2021.02.023 · Cited by 37
Zugman, A., Jett, L., Antonacci, C., Winkler, A. M., & Pine, D. S. (2023) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2023.102773 · Cited by 54
Spindler, C., Trautmann, S., Alexander, N., Bröning, S., Bartscher, S., Stuppe, M., & Muehlhan, M. (2021) · Scientific Reports · doi:10.1038/s41598-021-84804-7 · Cited by 43
Arioli, M., Cattaneo, Z., Parimbelli, S., & Canessa, N. (2023) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsad003 · Cited by 27
“Following Acar et al. (2018), we pre-specified lower and upper boundaries for the FSN of each cluster based on the following considerations.”
MethodResultInfluentialIs Migraine Associated to Brain Anatomical Alterations? New Data and Coordinate-Based Meta-analysis
Masson, R., Demarquay, G., David Meunier, A., Lévêque, Y., Hannoun, S., Bidet‐Caulet, A., & Caclin, A. (2021) · Brain Topography · doi:10.1007/s10548-021-00824-6 · Cited by 38
Widespread and lateralized social brain activity for processing dynamic facial expressions
Sato, W., Kochiyama, T., Uono, S., Sawada, R., Kubota, Y., Yoshimura, S., & Toichi, M. (2019) · Human Brain Mapping · doi:10.1002/hbm.24629 · Cited by 35
Goodwill, A., Low, L., Fox, P., Fox, P., Poon, K., Bhowmick, S., & Chen, S. A. A. (2023) · Brain Imaging and Behavior · doi:10.1007/s11682-022-00754-2 · Cited by 30
Althubeati, S., Avery, A., Tench, C. R., Lobo, D. N., Salter, A. M., & Eldeghaidy, S. (2022) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2022.104603 · Cited by 28
“As this tool is not specifically designed for neuroimaging studies, the risk of bias might be underestimated (Acar et al., 2018).”
MethodPratts, J., Pobric, G., & Yao, B. (2023) · NeuroImage · doi:10.1016/j.neuroimage.2023.120399 · Cited by 34
“This consists of re-running the GingerALE analysis whilst iteratively adding an increasing number of randomly-generated null-result studies (Acar et al., 2018).”
BackgroundMethodInfluentialEmotion regulation: From neural circuits to a transdiagnostic perspective
Morawetz, C., Hemetsberger, F. J., Laird, A. R., & Kohn, N. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105960 · Cited by 27
“Being data-driven, metaanalyses rely on published findings, which can introduce publication bias (Acar et al., 2018; Jennings and Van Horn, 2012a; Lin and Chu, 2018).”
Bedini, M., Olivetti, E., Avesani, P., & Baldauf, D. (2023) · Brain Structure and Function · doi:10.1007/s00429-023-02641-y · Cited by 35
“It is not clear to what extent these IFJ activations can be replicated, and there may be a file drawer problem in this sample, which is typically an issue in ALE meta-analyses (Acar et al. 2018).”
Mancuso, L., Fornito, A., Costa, T., Ficco, L., Liloia, D., Manuello, J., Duca, S., & Cauda, F. (2020) · bioRxiv · doi:10.1101/2020.05.18.101436 · Cited by 25
“As our COA-O dataset of experiments is relatively small, we assessed the robustness of our results to the injection of noise with a modified Fail-safe technique (Acar et al., 2018).”
BackgroundMethodInfluentialA Meta-Analysis of fMRI Studies of Semantic Cognition in Children
Enge, A., Rahman, R. A., & Skeide, M. A. (2021) · bioRxiv · doi:10.1101/2021.05.17.442947 · Cited by 24
Yeung, A. W. K., Robertson, M., Uecker, A. M., Fox, P. T., & Eickhoff, S. B. (2022) · Human Brain Mapping · doi:10.1002/hbm.26177 · Cited by 25
“One of the commonest CBMA approach in the literature is the activation likelihood estimation (ALE) approach (Acar et al., 2018; Samartsidis et al., 2020; Tahmasian et al., 2019; Yeung et al., 2019).”
BackgroundMethodSelf-citationReappraisal and empathic perspective-taking – More alike than meets the eyes
Morawetz, C., Berboth, S., Kohn, N., Jackson, P. L., & Jauniaux, J. (2022) · NeuroImage · doi:10.1016/j.neuroimage.2022.119194 · Cited by 30
Chaudhary, S., Zhang, S., Zhornitsky, S., Chen, Y., Chao, H., & Li, C. (2023) · NeuroImage · doi:10.1016/j.neuroimage.2023.120207 · Cited by 22
Liloia, D., Cauda, F., Uddin, L. Q., Manuello, J., Mancuso, L., Keller, R., Nani, A., & Costa, T. (2022) · Biological Psychiatry Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2022.01.007 · Cited by 23
Chaudhary, S., Zhornitsky, S., Chao, H., van Dyck, C., & Li, C. (2021) · Journal of Alzheimer's Disease · doi:10.3233/jad-215316 · Cited by 22
“Briefly, we used R program to generate “noise” studies from the list of foci, sample size, and the distribution of sample sizes, as in the five studies included in meta-analysis [49].”
Cattarinussi, G., Grimaldi, D. A., & Sambataro, F. (2023) · Schizophrenia Bulletin · doi:10.1093/schbul/sbad044 · Cited by 21
The major-minor mode dichotomy in music perception
Carraturo, G., Pando‐Naude, V., Costa, M., Vuust, P., Bonetti, L., & Brattico, E. (2024) · Physics of Life Reviews · doi:10.1016/j.plrev.2024.11.017 · Cited by 35
“To address potential publication bias and assess the robustness of the results, the Fail-Safe N analysis (FSN) was performed [49].”
Hill-Bowen, L. D., Riedel, M. C., Salo, T., Flannery, J. S., Poudel, R., Laird, A. R., & Sutherland, M. T. (2022) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2022.109625 · Cited by 18
Li, J., Kuang, S., Liu, Y., Wu, Y., & Li, H. (2024) · Human Brain Mapping · doi:10.1002/hbm.26702 · Cited by 21
“To assess the robustness of identified clusters against potential publication bias, we performed the Fail-Safe N (FSN) method adapted for ALE analyses as described by Acar et al. (2018).”
MethodMattavelli, G. C., Gorrino, I., Tornaghi, D., & Canessa, N. (2023) · Cortex · doi:10.1016/j.cortex.2023.10.008 · Cited by 21
Parcellation‐based tractographic modeling of the salience network through meta‐analysis
Briggs, R. G., Young, I. M., Dadario, N. B., Fonseka, R. D., Hormovas, J., Allan, P. G., Larsen, M. L., Lin, Y., Tanglay, O., Maxwell, B. D., et al. (2022) · Brain and Behavior · doi:10.1002/brb3.2646 · Cited by 24
Neural correlates of morphological processing: An activation likelihood estimation meta-analysis.
Bulut, T. (2022) · Cortex; a journal devoted to the study of the nervous system and behavior · doi:10.1016/j.cortex.2022.02.010 · Cited by 18
Balgova, E., Diveica, V., Jackson, R., & Binney, R. J. (2024) · bioRxiv · doi:10.1101/2023.08.16.553506 · Cited by 17
“Nonetheless, meta-analyses performed on small sample sizes are susceptible to potential publication bias, and caution should be given to interpretation of results from samples with less than 30 studies (Acar et al., 2018).”
Cognitive reappraisal of food craving and emotions: a coordinate-based meta-analysis of fMRI studies
Gerosa, M., Canessa, N., Morawetz, C., & Mattavelli, G. C. (2023) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsad077 · Cited by 19
“…whether the FSN for each cluster lies between two pre-specified lower and upper boundaries, thus revealing that meta-analytic results are sufficiently robust against the file-drawer publication bias, as well as driven by at least the desired minimum of contributing studies (Acar et al ., 2018).”
BackgroundMethodResultInfluentialDisentangling emotional signals in the brain: an ALE meta-analysis of vocal affect perception
Mauchand, M., & Zhang, S. (2022) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-022-01030-y · Cited by 17
“No cluster survived the minimum FSN of 2k null studies (Acar et al., 2018) in any of the emotion-specific analyses.”
BackgroundMethodFrahm, L., Cieslik, E. C., Hoffstaedter, F., Satterthwaite, T. D., Fox, P., Langner, R., & Eickhoff, S. (2022) · Human Brain Mapping · doi:10.1002/hbm.25898 · Cited by 17
“Originally introduced in 2002 (Turkeltaub et al., 2002), ALE has since undergone extensive evaluations and improvements in regard to statistical soundness (Acar et al., 2018; Eickhoff et al., 2009; Eickhoff et al., 2012; Turkeltaub et al., 2012).”
BackgroundSelf-citationFinding specificity in structural brain alterations through Bayesian reverse inference
Cauda, F., Nani, A., Liloia, D., Manuello, J., Premi, E., Duca, S., Fox, P. T., & Costa, T. (2020) · Human Brain Mapping · doi:10.1002/hbm.25105 · Cited by 24
“The fail-safe analysis has showed that the BF results obtained from the anatomical alteration data of AD and SCZ are rather solid, as they survive after relevant injections of noise and they are not, therefore, strongly affected by publication bias (Acar et al., 2018).”
BackgroundMethodUniversality and language specificity of brain Reading networks: A developmental perspective
Zhang, J., Chen, J., & Ding, G. (2024) · Developmental Science · doi:10.1111/desc.13379 · Cited by 16
Toward Coordinate-based Cognition Dictionaries: A BrainMap and Neurosynth Demo
Lu, Q., Towne, J. M., Lock, M., Jiang, C., Cheng, Z., Habes, M., Zuo, X., & Zang, Y. (2022) · Neuroscience · doi:10.1016/j.neuroscience.2022.02.016 · Cited by 15
Groenendijk, I., Mehnert, U., Groen, J., Clarkson, B., Scheepe, J., & Blok, B. (2020) · PLoS ONE · doi:10.1371/journal.pone.0246042 · Cited by 16
“As this tool is not specifically designed for neuroimaging studies, the risk of bias might be underestimated [62].”
MethodInvariant structural and functional brain regions associated with tinnitus: A meta-analysis
Moring, J. C., Husain, F. T., Gray, J. P., Franklin, C. G., Peterson, A. L., Resick, P. A., Garrett, A. S., Esquivel, C. R., & Fox, P. T. (2022) · PLoS ONE · doi:10.1371/journal.pone.0276140 · Cited by 16
“[22] developed the fail-safe N method in order to account for publication bias by introducing noise into the ALE algorithm.”
MethodGrey matter abnormalities in Tourette syndrome: an activation likelihood estimation meta-analysis
Wen, F., Yan, J., Yu, L., Wang, F., Liu, J.-R., Li, Y., & Cui, Y.-H. (2020) · BMC Psychiatry · doi:10.1186/s12888-021-03187-1 · Cited by 15
Boyland, E., Maden, M., Coates, A. E., Masterson, T. D., Alblas, M. C., Bruce, A. S., & Roberts, C. A. (2024) · Obesity Reviews · doi:10.1111/obr.13643 · Cited by 14
Standardized Effect Sizes and Image-Based Meta-Analytical Approaches for fMRI Data
Bossier, H., Nichols, T. E., & Moerkerke, B. (2019) · doi:10.1101/865881 · Cited by 14
“Most of these methods and diagnostic tools focus on coordinate-based meta-analyses that combine results over studies using only the 3D coordinates of peak activation from the individual studies (Acar et al., 2018; Albajes-Eizagirre and Radua, 2018; Eickhoff et al., 2009, 2012; Turkeltaub et al., 2002, 2012; Radua et a…”
MethodSelf-citationGan, R., Qiu, Y.-D., Liao, J., Zhang, Y.-T., Wu, J., Peng, X., Lee, T. M., & Huang, R. (2024) · Neuroscience and Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105918 · Cited by 14
Fascher, M., Nowaczynski, S., Spindler, C., Strobach, T., & Muehlhan, M. (2023) · Psychopharmacology · doi:10.1007/s00213-023-06498-1 · Cited by 16
“To estimate robustness of our results against this type of publications bias, we performed a fail-safe-N (FSN) calculation explicitly adapted to ALE meta-analyses by Acar et al. (2018).”
Watanuki, S. (2022) · Frontiers in Neuroscience · doi:10.3389/fnins.2022.984647 · Cited by 17
“, 2009) among the CBMA methods because the ALE method is the most popular among them (Acar et al., 2018).”
MethodEstimating the prevalence of missing experiments in a neuroimaging meta-analysisPreprint
Samartsidis, P., Montagna, S., Laird, A. R., Fox, P. T., Johnson, T. D., & Nichols, T. E. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/225425 · Cited by 9
Wang, L., Li, T., Gu, R., & Feng, C. (2024) · NeuroImage · doi:10.1016/j.neuroimage.2024.120824 · Cited by 13
“A higher FSN indicates more stable results and hence a higher robustness (Acar et al., 2018).”
Bortolini, T., Laport, M. C., Latgé-Tovar, S., Fischer, R. D., Zahn, R., de Oliveira‐Souza, R., & Moll, J. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105584 · Cited by 22
“The ALE method is probably the most popular CBMA algorithm for fMRI experiments (Acar et al., 2018).”
MethodInfluentialZhang, F., Rakhimbekova, A., Lashley, T., & Madl, T. (2022) · Progress in neurobiology · doi:10.2139/ssrn.4195324 · Cited by 13
“It has been reported that cerebellar grey matter atrophy occurs in FTD (Acar et al., 2018).”
Liloia, D., Crocetta, A., Cauda, F., Duca, S., Costa, T., & Manuello, J. (2022) · Brain Sciences · doi:10.3390/brainsci12101367 · Cited by 16
“While these CBMA methods test the spatial convergence across coordinates, our PSI-SDM approach conducts standard univariate voxel-wise tests [38,79].”
InfluentialSaward, J. B., Ellis, E. G., Cobden, A. L., & Caeyenberghs, K. (2022) · Brain Imaging and Behavior · doi:10.1007/s11682-022-00655-4 · Cited by 13
Ma, W., Tian, M.-J., Yao, Q.-L., Li, Q., Tang, F., Xiao, C., Shi, J., & Chen, J. (2021) · CNS Neuroscience & Therapeutics · doi:10.1111/cns.13775 · Cited by 13
Newton‐Fenner, A., Hewitt, D., Henderson, J. R., Roberts, H., Mari, T., Gu, Y., Gorelkina, O., Giesbrecht, T., Fallon, N. B., Roberts, C. A., et al. (2023) · PLoS ONE · doi:10.1371/journal.pone.0286969 · Cited by 19
The road not taken: Common and distinct neural correlates of regret and relief
Varma, M. M., Chowdhury, A., & Yu, R. (2023) · NeuroImage · doi:10.1016/j.neuroimage.2023.120413 · Cited by 14
“A separate ALE meta-analysis was run for each adjusted maximum FSN.”
BackgroundMethodInfluentialFrahm, L., Satterthwaite, T. D., Fox, P., Langner, R., & Eickhoff, S. (2023) · NeuroImage · doi:10.1016/j.neuroimage.2023.120383 · Cited by 11
“Activation Likelihood Estimation (ALE) is a widely used approach for coordinate-based neuroimaging meta-analysis and has been under constant methodological development and review since its inception in 2002 (Acar et al., 2018; Eickhoff et al., 2009, Turkeltaub et al., 2002, 2012).”
Self-citationMasharipov, R. S., Knyazeva, I., Nikolaev, Y., Korotkov, A. D., Didur, M., Cherednichenko, D. V., & Kireev, M. V. (2021) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2021.738342 · Cited by 12
“This publishing bias is also known as the ‘file-drawer problem’ (Rosenthal, 1979; Ioannidis et al., 2014; de Winter and Dodou, 2015; for evidence in fMRI studies, see Jennings and Van Horn, 2012; Acar et al., 2018; David et al., 2018; Samartsidis et al., 2020).”
BackgroundVolkers, R., Giesen, E., van der Heiden, M., Kerperien, M., Lange, S., Kurt, E., van Dongen, R. V., Schutter, D., Vissers, K., & Henssen, D. (2020) · Neuromodulation (Malden, Mass.) · doi:10.1111/ner.13119 · Cited by 11
Zhao, M., Li, R., Xiang, S., & Liu, N. (2024) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsae068 · Cited by 13
“To account for the file drawer problem, which is a type of publication bias that refers to unpublished experiments and can potentially influence results and the validity of findings (Hayasaka et al. 2018, Samartsidis et al. 2020), the Fail-Safe N (FSN) analysis was performed.”
BackgroundFan, C., Wu, M., Liu, H., Chen, X., Zhen, G., Zhao, X., Zhou, J., & Jiang, Z. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105735 · Cited by 10
Appetitive and aversive cue reactivities differentiate neural subtypes of alcohol drinkers
Chen, Y., & Li, C. R. (2023) · Addiction Neuroscience · doi:10.1016/j.addicn.2023.100089 · Cited by 9
“We also performed a “Fail-Safe N (FSN)” analysis to evaluate potential publication bias [1].”
MethodLi, C., Hu, Q., Zhang, D., Hoffstaedter, F., Bauer, A., & Elmenhorst, D. (2022) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2022.104643 · Cited by 10
Bridging the Brain and Data Sciences
Horn, J. D. V. (2020) · Big Data · doi:10.1089/big.2020.0065 · Cited by 11
“53 However, users may need to consider biases associated with ‘‘file drawer problems’’ where only statistically significant results from a literature results are included 54 and, thus, may suffer from ‘‘so-called’’ publication bias.”
Linguistic network in early deaf individuals: A neuroimaging meta-analysis
Yang, T., Fan, X., Hou, B., Wang, J., & Chen, X. (2024) · NeuroImage · doi:10.1016/j.neuroimage.2024.120720 · Cited by 13
“Briefly, this procedure required the iterative introduction of null experiments resembling the original experiments into the original dataset and retesting the significance of the results (Acar et al., 2018).”
MethodA Minimum Bayes Factor Based Threshold for Activation Likelihood Estimation
Costa, T., Liloia, D., Cauda, F., Fox, P. T., Mutta, F. D., Duca, S., & Manuello, J. (2023) · Neuroinformatics · doi:10.1007/s12021-023-09626-6 · Cited by 11
“…unrelated cognitive domains (see Table S2 for a description of this dataset), and on two simulated data-sets generated with the Fail-safe R script (Acar et al. (2018); https:// github. com/ Neuro Stat/ Gener ateNu ll) and built with 21 experiments each with foci randomly placed across the brain.”
MethodWatanuki, S., & Akama, H. (2022) · Heliyon · doi:10.1016/j.heliyon.2022.e09702 · Cited by 7
“First, it is the most popular quantitative meta-analysis method (Acar et al., 2018).”
BackgroundMeda, N., Baggio, M., Collantoni, E., & Sambataro, F. (2025) · European Neuropsychopharmacology · doi:10.1016/j.euroneuro.2025.01.009 · Cited by 8
Fascher, M., Nowaczynski, S., & Muehlhan, M. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105712 · Cited by 8
“We adhered to the parsimonious protocol and the code for this FSN-calculation procedure provided by Acar and colleagues (Acar et al., 2018).”
Dugré, J., Hopfer, C., & Winters, D. E. (2025) · Aggression and Violent Behavior · doi:10.1016/j.avb.2025.102035 · Cited by 7
“Since the ALE algorithm does not contain effect sizes, artificial studies were randomly generated from the original sample (e.g., number of peaks and sample size) and added to the original meta-analysis (see Acar et al., 2018); implementation by Enge et al., 2020 and Bortolini et al., 2024).”
Altered anticipatory brain responses in eating disorders: A neuroimaging meta‐analysis
Yu, X., & Desrivières, S. (2023) · European Eating Disorders Review · doi:10.1002/erv.2967 · Cited by 8
“Fail‐Safe N (FSN) quantifies the number of null studies (studies with negative results or contra‐evi-dence) that are needed to change current results (Acar et al., 2018; Rosenthal, 1979).”
BackgroundLi, J., Liu, Z., Du, Z., Zhu, N., Qiu, X., & Xu, X. (2021) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2021.774656 · Cited by 7
“Therefore, the present study used activation likelihood estimation (ALE) meta-analysis, which is a quantitative meta-analytic method that has been widely utilized to determine the stereoscopic brain coordinates that have been consistently active across studies (Turkeltaub et al., 2002; Acar et al., 2018).”
MethodTeghil, A., Wiener, M., & Boccia, M. (2025) · Neuropsychology Review · doi:10.1007/s11065-025-09662-w · Cited by 3
Presti, S. L., Origlia, S., Gianelli, C., & Canessa, N. (2023) · Human Brain Mapping · doi:10.1002/hbm.26312 · Cited by 7
“Following Acar et al. (2018), we prespecified lower and upper boundaries for the FSN of each cluster based on the following considerations.”
BackgroundMethodResultInfluentialWang, L., Liu, Q., Zheng, Z., Chen, W., Shen, Y., Li, T., & Feng, C. (2025) · Translational Psychiatry · doi:10.1038/s41398-025-03760-2 · Cited by 6
“The amount of contra-evidence that the ALE can tolerate (i.e., FSN) was calculated by introducing null pseudo-studies as noise into the ALE cohort to assess the robustness of ALE clusters [58, 59].”
Towne, J. M., Eslami, V., Barron, D. S., Cavazos, J. E., & Fox, P. T. (2025) · NeuroImage Clinical · doi:10.1016/j.nicl.2025.103908 · Cited by 5
“Finally, publication bias, where studies with statistically significant results are more likely to be published, can lead to an overestimation of the true effect size (Acar et al., 2018).”
Yang, Y., Wang, C., Shi, J., & Zou, Z. (2024) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2024.109003 · Cited by 6
The neural networks of touch observation
Schaefer, M., Kuehn, E., Schweitzer, F. M., & Muehlhan, M. (2023) · Imaging Neuroscience · doi:10.1162/imag_a_00065 · Cited by 6
“However, we tried to address these issues by using the most recent guidelines ( Acar et al., 2018; Müller et al., 2018) for our procedures to make our results reliable.”
BackgroundMethodInfluential
Metadata
- Authors
- Acar, F., Seurinck, R., Eickhoff, S. B., & Moerkerke, B.
- Affiliations
- Ghent University (BE)
- Forschungszentrum Jülich (DE)
- Corresponding author
- Acar, Freya
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2018-11-30
- Volume
- 13
- Issue
- 11
- Pages
- e0208177
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC6267999
- OpenAlex
- W2902120147
- Cited by
- 154
- References
- 48
- Citations by year
- 2026: 22 · 2025: 19 · 2024: 18 · 2023: 28 · 2022: 27 · 2021: 19 · 2020: 13 · 2019: 7
- FWCI
- 7.44
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Ghent University Academic Bibliography (Ghent University) (submittedVersion, other-oa)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- JuSER (Forschungszentrum Jülich) (submittedVersion, other-oa)
- Funders
- Fonds Wetenschappelijk Onderzoek (G.0149.14)
- National Institute of Mental Health (R01-MH074457)
- Horizon 2020 (7202070 (HBP SGA1))
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Life Sciences › Neuroscience › Cognitive Neuroscience › Functional Brain Connectivity Studies
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- publication bias, file drawer problem, activation likelihood estimation, coordinate-based meta-analysis, noise study, robustness analysis, spatial convergence
- MeSH terms
- Algorithms, Brain, Brain Mapping, Humans, Magnetic Resonance Imaging, Meta-Analysis as Topic, Likelihood Functions, Publication Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
