Assessing robustness against potential publication bias in activation likelihood estimation (ALE) meta-analyses for fMRI

Acar, Freya; Seurinck, Ruth; Eickhoff, Simon B.; Moerkerke, Beatrijs

doi:10.1371/journal.pone.0208177

Cited by 154Meta-analysis and systematic reviewsFunctional Brain Connectivity StudiesPsychology Research and Bibliometrics

Assessing robustness against potential publication bias in activation likelihood estimation (ALE) meta-analyses for fMRI

Open access

Acar, Freya, Seurinck, Ruth, Eickhoff, Simon B., Moerkerke, Beatrijs·2018·PLOS ONE

DOI 10.1371/journal.pone.0208177PMC PMC6267999

Open this paper in CoChat — free account

Abstract

The importance of integrating research findings is incontrovertible and procedures for coordinate-based meta-analysis (CBMA) such as Activation Likelihood Estimation (ALE) have become a popular approach to combine results of fMRI studies when only peaks of activation are reported. As meta-analytical findings help building cumulative knowledge and guide future research, not only the quality of such analyses but also the way conclusions are drawn is extremely important. Like classical meta-analyses, coordinate-based meta-analyses can be subject to different forms of publication bias which may impact results and invalidate findings. The file drawer problem refers to the problem where studies fail to get published because they do not obtain anticipated results (e.g. due to lack of statistical significance). To enable assessing the stability of meta-analytical results and determine their robustness against the potential presence of the file drawer problem, we present an algorithm to determine the number of noise studies that can be added to an existing ALE fMRI meta-analysis before spatial convergence of reported activation peaks over studies in specific regions is no longer statistically significant. While methods to gain insight into the validity and limitations of results exist for other coordinate-based meta-analysis toolboxes, such as Galbraith plots for Multilevel Kernel Density Analysis (MKDA) and funnel plots and egger tests for seed-based d mapping, this procedure is the first to assess robustness against potential publication bias for the ALE algorithm. The method assists in interpreting meta-analytical results with the appropriate caution by looking how stable results remain in the presence of unreported information that may differ systematically from the information that is included. At the same time, the procedure provides further insight into the number of studies that drive the meta-analytical results. We illustrate the procedure through an example and test the effect of several parameters through extensive simulations. Code to generate noise studies is made freely available which enables users to easily use the algorithm when interpreting their results.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1371/journal.pone.0208177

Cites

45 of 45 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

ALE allows for increased power to detect true effects.
“By pooling together numerous studies, ALE also allows for an increased power to detect true effects (Acar et al., 2018).”
From Bridging phenomenology and neural mechanisms of inner speech: ALE meta-analysis on egocentricity and spontaneity in a dual-mechanistic framework (2023) · cites this paper as Acar et al., 2018

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722 · cited 1× · see citing works
“The Egger regression test estimates funnel plot asymmetry and can therefore be used as an indicator for the presence of publication bias [32].”
Method
Trim and Fill: A Simple Funnel‐Plot–Based Method of Testing and Adjusting for Publication Bias in Meta‐Analysis
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099 · cited 1× · see citing works
“In classical meta-analysis techniques (based on summarizing effect sizes), this is for instance possible with the trim and fill method [41], that looks at the asymmetry present in a funnel plot and tries to fill the gap.”
Method
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068 · cited 3× · see citing works
“The original FSN [21] represents the required number of findings with an effect of zero to change the conclusions of the meta-analysis under the assumption that these findings are still in the “file drawer”.”
BackgroundInfluential
Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials
Sterne, J. A. C., Sutton, A. J., Ioannidis, J. P. A., Terrin, N., Jones, D. R., Lau, J., Carpenter, J., Rucker, G., Harbord, R. M., Schmid, C. H., et al. (2011) · BMJ · doi:10.1136/bmj.d4002 · Cited by 7,709 · see citing works
Some Methods for Strengthening the Common χ 2 Tests
Cochran, W. G. (1954) · Biometrics · doi:10.2307/3001616 · Cited by 3,718 · see citing works
What we can do and what we cannot do with fMRI
Logothetis, N. K. (2008) · Nature · doi:10.1038/nature06976 · Cited by 3,612 · cited 1× · see citing works
“Functional magnetic resonance imaging (fMRI) continues to contribute greatly to the knowledge about the location of cognitive functions in the brain [1].”
Background
Coordinate‐based activation likelihood estimation meta‐analysis of neuroimaging data: A random‐effects approach based on empirical estimates of spatial uncertainty
Eickhoff, S. B., Laird, A. R., Grefkes, C., Wang, L. E., Zilles, K., & Fox, P. T. (2009) · Human Brain Mapping · doi:10.1002/hbm.20718 · Cited by 1,924 · cited 1× · see citing works
“We select a meta-analysis on finger tapping [7,38] from the BrainMap database that was used to demonstrate the working of GingerALE [15–17].”
MethodSelf-citation
Publication bias in meta-analysis: Prevention, assessment and adjustments
Viechtbauer, W. (2007) · Psychometrika · doi:10.1007/s11336-006-1450-y · Cited by 1,507 · cited 3× · see citing works
“We illustrate the FSN with an example from the book “Publication Bias in Meta-Analysis” [20].”
BackgroundMethodInfluential
Activation Likelihood Estimation meta-analysis revisited
Eickhoff, S., Bzdok, D., Laird, A., Kurth, F., & Fox, P. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.09.017 · Cited by 1,448 · see citing works
Self-citation
Minimizing Within-Experiment and Within-Group Effects in Activation Likelihood Estimation Meta-Analyses
Turkeltaub, P., Eickhoff, S., Laird, A. R., Fox, M., Wiener, M., & Fox, P. (2011) · Human Brain Mapping · doi:10.1002/hbm.21186 · Cited by 1,114 · cited 1× · see citing works
“Therefore, in this example all contrasts that stem from the same study with the same set of subjects are reported as one experiment [8].”
ResultSelf-citation
Voxel-wise meta-analysis of grey matter changes in obsessive-compulsive disorder
Radua, J., & Mataix-Cols, D. (2009) · British Journal of Psychiatry · doi:10.1192/bjp.bp.108.055046 · Cited by 863 · cited 2× · see citing works
“Seed-based d mapping also allows to perform sensitivity analyses using jackknife-analyses [33] or leave-one-out analyses [34] to study how replicable the original findings are.”
Method
Publication bias and clinical trials
Dickersin, K., Chan, S., Chalmersx, T. C., Sacks, H. S., & Smith, H. (1987) · Controlled Clinical Trials · doi:10.1016/0197-2456(87)90155-3 · Cited by 715 · cited 1× · see citing works
“Regardless of the area of research, statistically significant findings are more likely to get published than studies with non-significant results [19] implying that the latter remain largely hidden.”
Background
Impact of covert duplicate publication on meta-analysis: a case study
Tramèr, M. R., Reynolds, D. J. M., Moore, R. A., & McQuay, H. J. (1997) · BMJ · doi:10.1136/bmj.315.7109.635 · Cited by 702 · cited 1× · see citing works
“Using correlated data by reporting multiple experiments on the same set of subjects is problematic and induces bias [39,40].”
Background
Behavior, sensitivity, and power of activation likelihood estimation characterized by massive empirical simulation
Eickhoff, S. B., Nichols, T. E., Laird, A. R., Hoffstaedter, F., Amunts, K., Fox, P. T., Bzdok, D., & Eickhoff, C. R. (2016) · NeuroImage · doi:10.1016/j.neuroimage.2016.04.072 · Cited by 683 · cited 10× · see citing works
“The authors of the ALE algorithm advise to employ a voxel- or cluster-level family-wise error correction (FWE) [28].”
BackgroundMethodResultInfluentialSelf-citation
Mapping context and content: the BrainMap model
Fox, P., & Lancaster, J. (2002) · Nature Reviews Neuroscience · doi:10.1038/nrn789 · Cited by 518 · see citing works
The secret lives of experiments: Methods reporting in the fMRI literature
Carp, J. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.07.004 · Cited by 473 · cited 2× · see citing works
“fMRI studies exhibit great heterogeneity, stemming from the way experiments are constructed, carried out, analyzed and reported [2].”
Background
Publication Bias in Research Synthesis: Sensitivity Analysis Using A Priori Weight Functions.
Vevea, J. L., & Woods, C. M. (2005) · Psychological Methods · doi:10.1037/1082-989x.10.4.428 · Cited by 461 · cited 1× · see citing works
“The method of Vevea and Woods [42] corrects for publication bias by adjusting weights assigned to different studies based on study characteristics that could influence likeliness of publication, such as statistical significance.”
Method
Meta-analysis of functional neuroimaging data: current and future directions
Wager, T. D., Lindquist, M., & Kaplan, L. (2007) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsm015 · Cited by 434 · cited 1× · see citing works
“Using correlated data by reporting multiple experiments on the same set of subjects is problematic and induces bias [39,40].”
Background
BrainMap
Laird, A., Lancaster, J., & Fox, P. (2007) · Neuroinformatics · doi:10.1385/ni:3:1:065 · Cited by 379 · see citing works
Distributed solar photovoltaic array location and extent dataset for remote sensing object identification
Bradbury, K., Saboo, R., Johnson, T. L., Malof, J. M., Devarajan, A., Zhang, W.-M., Collins, L., & Newell, R. (2016) · Scientific Data · doi:10.1038/sdata.2016.106 · Cited by 370 · see citing works
Brainmap taxonomy of experimental design: Description and evaluation
Fox, P., Laird, A., Fox, S. P., Fox, P., Uecker, A. M., Crank, M., Koenig, S., & Lancaster, J. (2005) · Human Brain Mapping · doi:10.1002/hbm.20141 · Cited by 330 · see citing works
Publication Bias (The "File-Drawer Problem") in Scientific Inference
Scargle, J. D. (1999) · arXiv · doi:10.48550/arxiv.physics/9909033 · Cited by 313 · see citing works
Magnitude of teacher expectancy effects on pupil IQ as a function of the credibility of expectancy induction: A synthesis of findings from 18 experiments.
Raudenbush, S. W. (1984) · Journal of Educational Psychology · doi:10.1037/0022-0663.76.1.85 · Cited by 302 · cited 1× · see citing works
“The analysis is based on a dataset from Raudenbush on teacher expectancy, a paradigm where the influence of teacher expectancy on intelligence test performance is measured [44].”
Method
Excess Significance Bias in the Literature on Brain Volume Abnormalities
Ioannidis, J. P. A. (2011) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2011.28 · Cited by 299 · cited 1× · see citing works
“In neuroimaging literature, it has been shown that there is an excess of positive findings in literature [24, 25].”
Background
Meta-analytic methods for neuroimaging data explained
Radua, J., & Mataix-Cols, D. (2012) · Biology of Mood & Anxiety Disorders · doi:10.1186/2045-5380-2-6 · Cited by 246 · see citing works
Evaluating the consistency and specificity of neuroimaging data using meta-analysis
Wager, T. D., Lindquist, M. A., Nichols, T. E., Kober, H., & Van Snellenberg, J. X. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2008.10.061 · Cited by 242 · cited 10× · see citing works
“Different algorithms have been developed to conduct a meta- PLOS ONE | https://doi.org/10.1371/journal.pone.0208177 November 30, 2018 4 / 23 analysis on peak locations and in this section, we will briefly explain the three most used, seedbased d mapping, the ALE algorithm and MKDA.”
MethodInfluential
White Matter Development in Adolescence: Diffusion Tensor Imaging and Meta-Analytic Results
Peters, B. D., Szeszko, P. R., Raduà, J., Ikuta, T., Gruner, P., DeRosse, P., Zhang, J., Giorgio, A., Qiu, D., Tapert, S. F., et al. (2012) · Schizophrenia Bulletin · doi:10.1093/schbul/sbs054 · Cited by 203 · cited 1× · see citing works
“Seed-based d mapping also allows to perform sensitivity analyses using jackknife-analyses [33] or leave-one-out analyses [34] to study how replicable the original findings are.”
Method
Multimodal Voxel-Based Meta-Analysis of White Matter Abnormalities in Obsessive–Compulsive Disorder
Raduà, J., Grau, M., van den Heuvel, O. A., de Schotten, M. T., Stein, D. J., Canales-Rodríguez, E. J., Catani, M., & Mataix-Cols, D. (2014) · Neuropsychopharmacology · doi:10.1038/npp.2014.5 · Cited by 164 · see citing works
The American Soldier
Eysenck, H., Stouffer, S., Suchman, E., DeVinney, L. C., Star, S. A., Williams, R., Lumsdaine, A. A., Lumsdaine, M. H., Smith, M. B., Janis, I., et al. (1950) · British Journal of Sociology · doi:10.2307/586897 · Cited by 133 · see citing works
Potential Reporting Bias in fMRI Studies of the Brain
David, S. P., Ware, J. J., Chu, I. M., Loftus, P. D., Fusar-Poli, P., Radua, J., Munafò, M. R., & Ioannidis, J. P. A. (2013) · PLoS ONE · doi:10.1371/journal.pone.0070104 · Cited by 130 · cited 1× · see citing works
“In neuroimaging literature, it has been shown that there is an excess of positive findings in literature [24, 25].”
Background
Potential Reporting Bias in Neuroimaging Studies of Sex Differences
David, S., Naudet, F., Laude, J. R., Raduà, J., Fusar-Poli, P., Chu, I., Stefanick, M., & Ioannidis, J. (2018) · Scientific Reports · doi:10.1038/s41598-018-23976-1 · Cited by 123 · cited 1× · see citing works
“specifically mention selective outcome and analysis reporting bias, manipulation of parameters (such as lenient thresholding) and under-reporting of null results (file drawer) [26].”
Background
Publication bias in neuroimaging research: Implications for meta-analyses
Jennings, R., & Horn, J. V. V. (2011) · Neuroinformatics · doi:10.1007/s12021-011-9125-y · Cited by 103 · cited 2× · see citing works
“While such methods could be translated to neuroimaging studies as shown in [27] for a single outcome such as brain volume or correlation between regions or to corrections at voxel-level when effect sizes are used as in seed-based d-mapping [33], extending this to settings where only peak locations instead of effect si…”
BackgroundMethod
Sharing brain mapping statistical results with the neuroimaging data model
Maumet, C., Auer, T., Bowring, A., Chen, G., Das, S., Flandin, G., Ghosh, S. S., Glatard, T., Gorgolewski, K. J., Helmer, K., et al. (2016) · Scientific Data · doi:10.1038/sdata.2016.102 · Cited by 98 · cited 1× · see citing works
“000 voxels in the brain, however, the most prevailing trend to report results remains providing only the coordinates of statistically significant local maxima (called peaks or foci) [6].”
Background
The Usefulness of the "Fail-Safe" Statistic in Meta-Analysis
Carson, K. P., Schriesheim, C., & Kinicki, A. (1990) · Educational and Psychological Measurement · doi:10.1177/0013164490502001 · Cited by 96 · cited 2× · see citing works
“The measure provides an addition to the interpretation of results because it helps in the assessment of the confidence that can be placed in them [18] by showing how stable they remain in the presence of unreported information that may differ systematically from the information that is included.”
Background
Meta Analysis of Functional Neuroimaging Data via Bayesian Spatial Point Processes
Kang, J., Johnson, T., Nichols, T. E., & Wager, T. (2011) · Journal of the American Statistical Association · doi:10.1198/jasa.2011.ap09735 · Cited by 68 · see citing works
How does under-reporting of negative and inconclusive results affect the false-positive rate in meta-analysis? A simulation study
Kicinski, M. (2014) · BMJ Open · doi:10.1136/bmjopen-2014-004831 · Cited by 61 · see citing works
Modeling motor connectivity using TMS/PET and structural equation modeling
Laird, A., Robbins, J. M., Li, K., Price, L. R., Cykowski, M., Narayana, S., Laird, R. W., Franklin, C., & Fox, P. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2008.01.065 · Cited by 55 · cited 1× · see citing works
“We select a meta-analysis on finger tapping [7,38] from the BrainMap database that was used to demonstrate the working of GingerALE [15–17].”
Method
Post-hoc power estimation for topological inference in fMRI
Durnez, J., Moerkerke, B., & Nichols, T. E. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2013.07.072 · Cited by 36 · cited 1× · see citing works
“Furthermore, fMRI studies are relatively expensive which often limits the size of studies [2] leading to low statistical power to detect true activation [3].”
BackgroundSelf-citation
Meta-analysis of functional neuroimaging data using Bayesian nonparametric binary regression
Yue, Y., Lindquist, M., & Loh, J. (2012) · The Annals of Applied Statistics · doi:10.1214/11-aoas523 · Cited by 34 · see citing works
A BAYESIAN HIERARCHICAL SPATIAL POINT PROCESS MODEL FOR MULTI-TYPE NEUROIMAGING META-ANALYSIS
Kang, J., Nichols, T. E., Wager, T., & Johnson, T. (2014) · Annals of Applied Statistics · doi:10.1214/14-aoas757 · Cited by 33 · see citing works
Fail-Safe Ns for One- Versus Two-Tailed Tests Lead to Different Conclusions About Publication Bias
Hsu, L. (2002) · Understanding Statistics · doi:10.1207/s15328031us0102_02 · Cited by 24 · cited 1× · see citing works
“This was later adapted by Hsu [23] to the maximum number of studies with nonsignificant findings that can be added to a meta-analysis before the conclusions no longer hold.”
Background
A randomised controlled trial to compare the clinical and cost-effectiveness of prism glasses, visual search training and standard care in patients with hemianopia following stroke: a protocol
Rowe, F. J., Barton, P. G., Bedson, E., Breen, R., Conroy, E., Cwiklinski, E., Dodridge, C., Drummond, A., García-Fiñana, M., Howard, C., et al. (2014) · BMJ Open · doi:10.1136/bmjopen-2014-005885 · Cited by 21 · see citing works
Estimating the number of missing experiments in a neuroimaging meta-analysisPreprint
Samartsidis, P., Montagna, S., Laird, A., Fox, P., Johnson, T., & Nichols, T. E. (2017) · bioRxiv · doi:10.1101/225425 · Cited by 14 · cited 4× · see citing works
“A recent paper estimates the number of studies or experiments with no foci that remain unnoticed in fMRI meta-analyses [37].”
BackgroundMethodResultInfluential
A Generalized Publication Bias ModelPreprint
Schönemann, P., & Scargle, J. (2008) · arXiv (Cornell University) · doi:10.6129/cjp.2008.5001.02 · Cited by 14 · see citing works
Time-warped growth processes, with applications to the modeling of boom-bust cycles in house prices
Peng, J., Paul, D., & Müller, H.-G. (2014) · doi:10.1214/14-aoas740 · Cited by 8

Flagged first, then by citations

3 of the 48 references couldn’t be matched to a known paper.

Research works printed but not linked: 4

[4]Sutton A, Jones D, Abrams K, Sheldon T, Song F. Methods for Meta-Analysis in Medical Research. John Wiley, London; 2000.
[10]Radua J, Mataix-Cols D. Meta-analytic methods for neuroimaging data explained. Biology of Mood & Anxiety Disorders. 2012; 2(6): 1– 11. 22737993 PMC3384225
[14]Costafreda SG. Parametric coordinate-based meta-analysis: Valid effect size meta-analysis of studies with differing statistical thresholds. Journal of Neuroscience Methods. 2012; 210: 291– 300. 10.1016/j.jneumeth.2012.07.016 22878178 · DOI
[15]Laird A, Lancaster J, Fox P. Brainmap: The social evolution of a functional neuroimaging database. Neuroinformatics. 2008: 65– 78. 15897617

Sorted by citations

Multimodal Abnormalities of Brain Structure and Function in Major Depressive Disorder: A Meta-Analysis of Neuroimaging Studies
Gray, J. P., Müller, V. I., Eickhoff, S. B., & Fox, P. T. (2020) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2019.19050560 · Cited by 449 · see citing works
“(15) to estimate the robustness of identified results against unpublished neuroimaging findings.”
BackgroundMethodSelf-citation
Gray and white matter morphology in substance use disorders: a neuroimaging systematic review and meta-analysis
Pando-Naude, V., Toxto, S., Fernandez-Lozano, S., Parsons, C. E., Alcauter, S., & Garza-Villarreal, E. (2020) · bioRxiv · doi:10.1038/s41398-020-01128-2 · Cited by 118 · see citing works
Current Challenges in Translational and Clinical fMRI and Future Directions
Specht, K. (2020) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2019.00924 · Cited by 112 · see citing works
The Neural Correlates of Time: A Meta-analysis of Neuroimaging Studies
Nani, A., Manuello, J., Liloia, D., Duca, S., Costa, T., & Cauda, F. (2019) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01459 · Cited by 109 · see citing works
“Noise experiments are then created randomly by combining values of sample size and amount of reported foci per experiment taken from the respective real distributions (Acar et al., 2018).”
BackgroundMethodInfluential
The effect of adverse childhood experiences on chronic pain and major depression in adulthood: a systematic review and meta-analysis
Antoniou, G., Lambourg, E., Steele, J., & Colvin, L. (2023) · British Journal of Anaesthesia · doi:10.1016/j.bja.2023.03.008 · Cited by 108 · see citing works
“Thus, traditional measures of heterogeneity are not applicable.(141,143,144) Second, the limited number of neuro-”
Background
Functional neuroimaging of human postural control: a systematic review with meta-analysis.
Dijkstra, B. W., Bekkers, E., Gilat, M., de Rond, V., Hardwick, R., & Nieuwboer, A. (2020) · Neuroscience and Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.04.028 · Cited by 107 · see citing works
Are there shared neural correlates between dyslexia and ADHD? A meta-analysis of voxel-based morphometry studies
McGrath, L. M., & Stoodley, C. J. (2019) · Journal of Neurodevelopmental Disorders · doi:10.1186/s11689-019-9287-8 · Cited by 78 · see citing works
“Such random noise would diminish the statistical evidence for convergence of true effects across studies [129].”
Background
Can't get it off my brain: Meta-analysis of neuroimaging studies on perseverative cognition.
Makovac, E., Fagioli, S., Rae, C., Critchley, H., & Ottaviani, C. (2019) · Psychiatry research. Neuroimaging · doi:10.1016/j.pscychresns.2019.111020 · Cited by 84 · see citing works
A meta-analysis of fMRI studies of language comprehension in children
Enge, A., Friederici, A., & Skeide, M. A. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.116858 · Cited by 67 · see citing works
“Only for cluster 4 (left superior frontal gyrus), the value of fail-safe N ¼ 115 slightly exceeded the desired upper boundary (in this case, fail-safe N ¼ 113), potentially indicating that this cluster was driven by a very small number of experiments (Acar et al., 2018).”
BackgroundMethodResultInfluential
Neurostructural traces of early life adversities: A meta-analysis exploring age- and adversity-specific effects
Pollok, T. M., Kaiser, A., Kraaijenvanger, E. J., Monninger, M., Brandeis, D., Banaschewski, T., Eickhoff, S. B., & Holz, N. E. (2022) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2022.104589 · Cited by 68 · see citing works
Self-citation
Disentangling predictive processing in the brain: a meta-analytic study in favour of a predictive network
Ficco, L., Mancuso, L., Manuello, J., Teneggi, A., Liloia, D., Duca, S., Costa, T., Kovács, G., & Cauda, F. (2021) · Scientific Reports · doi:10.1038/s41598-021-95603-5 · Cited by 77 · see citing works
The empirical replicability of task-based fMRI as a function of sample size
Bossier, H., Roels, S. P., Seurinck, R., Banaschewski, T., Barker, G. J., Bokde, A. L. W., Quinlan, E. B., Desrivières, S., Flor, H., Grigis, A., et al. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.116601 · Cited by 76 · see citing works
“…false (Ioannidis, 2005), (2) an overestimation of the reported effect sizes compared to the true effect size (Cremers et al., 2017) and (3) an increased risk of missed effects (i.e. false negatives) which potentially induces publication bias (Acar et al., 2018; Rosenthal, 1979; Sterling, 1959).”
BackgroundSelf-citation
An ALE meta-analytic review of top-down and bottom-up processing of music in the brain
Pando-Naude, V., Patyczek, A., Bonetti, L., & Vuust, P. (2021) · Scientific Reports · doi:10.1038/s41598-021-00139-3 · Cited by 62 · see citing works
An ALE meta-analytic review of musical expertise
Criscuolo, A., Pando-Naude, V., Bonetti, L., Vuust, P., & Brattico, E. (2022) · Scientific Reports · doi:10.1038/s41598-022-14959-4 · Cited by 55 · see citing works
Shared and distinct neural activity during anticipation and outcome of win and loss: A meta-analysis of the monetary incentive delay task
Chen, Y., Chaudhary, S., & Li, C. (2022) · bioRxiv · doi:10.1016/j.neuroimage.2022.119764 · Cited by 54 · see citing works
“3 Evaluation of publication bias We performed a “Fail-Safe N (FSN)” analysis to evaluate potential publication bias (Acar et al., 2018).”
MethodInfluential
Revealing the Neurobiology Underlying Interpersonal Neural Synchronization with Multimodal Data FusionPreprint
Lotter, L., Kohl, S. H., Gerloff, C., Bell, L., Niephaus, A., Kruppa, J. A., Dukart, J., Schulte-Rüther, M., Reindl, V., & Konrad, K. (2022) · bioRxiv · doi:10.1101/2022.07.26.501562 · Cited by 51 · see citing works
“Clusterwise robustness against publication bias was estimated as the fail-safe-N (Acar et al., 2018).”
Structural gray matter differences in Problematic Usage of the Internet: a systematic review and meta-analysis
Solly, J., Hook, R., Grant, J., Cortese, S., & Chamberlain, S. (2021) · Molecular Psychiatry · doi:10.1038/s41380-021-01315-7 · Cited by 51 · see citing works
Tasks activating the default mode network map multiple functional systems
Mancuso, L., Cavuoti-Cabanillas, S., Liloia, D., Manuello, J., Buzi, G., Cauda, F., & Costa, T. (2022) · Brain Structure and Function · doi:10.1007/s00429-022-02467-0 · Cited by 49 · see citing works
“The Fail-Safe algorithm (https:// github. com/ Neuro Stat/ Gener ateNu ll) proposed by Acar et al. (2018) generates a given number of experiments having random foci, to be added to the dataset to be tested.”
Prefrontal, parietal, and limbic condition-dependent differences in bipolar disorder: a large-scale meta-analysis of functional neuroimaging studies
Schumer, M. C., Chase, H., Rozovsky, R., Eickhoff, S., & Phillips, M. (2023) · Molecular Psychiatry · doi:10.1038/s41380-023-01974-8 · Cited by 47 · see citing works
“Emotional tasks were then further separated into valence classes for post-hoc meta-analyses: Negative valence was operationalized as stimuli representing or invoking fear, sadness, anger, disgust, pain, loss, or punishment ; positive valence was operatio-nalized as stimuli representing or invoking…”
BackgroundSelf-citation
Systematic Review: Medication Effects on Brain Intrinsic Functional Connectivity in Patients With Attention-Deficit/Hyperactivity Disorder
Pereira-Sánchez, V., Franco, A. R., Vieira, D. L., de Castro‐Manglano, P., Soutullo, C. A., Milham, M. P., & Castellanos, F. X. (2020) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2020.10.013 · Cited by 53 · see citing works
Estimating the prevalence of missing experiments in a neuroimaging meta-analysis
Samartsidis, P., Montagna, S., Laird, A., Fox, P., Johnson, T., & Nichols, T. E. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1448 · Cited by 47 · see citing works
Schizophrenia-related abnormalities in the triple network: a meta-analysis of working memory studies
Wu, D., & Jiang, T. (2019) · Brain Imaging and Behavior · doi:10.1007/s11682-019-00071-1 · Cited by 45 · see citing works
The Cerebellum in Frontotemporal Dementia: a Meta-Analysis of Neuroimaging Studies
Chen, Y., Kumfor, F., Landín-Romero, R., Irish, M., & Piguet, O. (2019) · Neuropsychology Review · doi:10.1007/s11065-019-09414-7 · Cited by 46 · see citing works
Distinct Convergent Brain Alterations in Sleep Disorders and Sleep Deprivation
Reimann, G. M., Hoseini, A., Koçak, M., Beste, M., Küppers, V., Rosenzweig, I., Elmenhorst, D., Pires, G., Laird, A. R., Fox, P. T., et al. (2025) · JAMA psychiatry · doi:10.1001/jamapsychiatry.2025.0488 · Cited by 42 · see citing works
Self-citation
Neural Correlates of the Risk for Schizophrenia and Bipolar Disorder: A Meta-analysis of Structural and Functional Neuroimaging Studies.
Cattarinussi, G., Kubera, K., Hirjak, D., Wolf, R. C., & Sambataro, F. (2022) · Biological Psychiatry · doi:10.1016/j.biopsych.2022.02.960 · Cited by 42 · see citing works
A Systematic Review and Activation Likelihood Estimation Meta-Analysis of fMRI Studies on Sweet Taste in Humans
Roberts, C. A., Giesbrecht, T., Fallon, N., Thomas, A. C., Mela, D., & Kirkham, T. (2020) · Journal of NutriLife · doi:10.1093/jn/nxaa071 · Cited by 42 · see citing works
“These algorithms use kernel techniques for assessing spatial uncertainty around the reported peaks (36).”
Method
Kahoot! Game‐based digital learning platform: A comprehensive meta‐analysis
Özdemir, O. (2025) · Journal of Computer Assisted Learning · doi:10.1111/jcal.13084 · Cited by 41 · see citing works
Parsing variability in borderline personality disorder: a meta-analysis of neuroimaging studies
Degasperi, G., Cristea, I., Rosa, E. D., Costa, C., & Gentili, C. (2021) · Translational Psychiatry · doi:10.1038/s41398-021-01446-z · Cited by 40 · see citing works
A coordinate-based meta-analysis of white matter alterations in patients with alcohol use disorder
Spindler, C., Mallien, L., Trautmann, S., Alexander, N., & Muehlhan, M. (2022) · Translational Psychiatry · doi:10.1038/s41398-022-01809-0 · Cited by 36 · see citing works
“To estimate the robustness of the results to potential null findings, we used a calculation of the Fail-Safe N (FSN) adapted for ALE as described by Acar and colleagues [36].”
Method
Humor and emotion: Quantitative meta analyses of functional neuroimaging studies
Farkas, A. H., Trotti, R. L., Edge, E. A., Huang, L.-Y., Kasowski, A., Thomas, O. F., Chlan, E., Granros, M., Patel, K., & Sabatinelli, D. (2021) · Cortex · doi:10.1016/j.cortex.2021.02.023 · Cited by 37 · see citing works
A Systematic Review and Meta-Analysis of Resting-state fMRI in Anxiety Disorders: Need for Data Sharing to Move the Field Forward
Zugman, A., Jett, L., Antonacci, C., Winkler, A. M., & Pine, D. S. (2023) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2023.102773 · Cited by 35 · see citing works
Meta-analysis of grey matter changes and their behavioral characterization in patients with alcohol use disorder
Spindler, C., Trautmann, S., Alexander, N., Bröning, S., Bartscher, S., Stuppe, M., & Muehlhan, M. (2021) · Scientific Reports · doi:10.1038/s41598-021-84804-7 · Cited by 34 · see citing works
Relational vs representational social cognitive processing: a coordinate-based meta-analysis of neuroimaging data
Arioli, M., Cattaneo, Z., Parimbelli, S., & Canessa, N. (2023) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsad003 · Cited by 33 · see citing works
“Following Acar et al. (2018), we pre-specified lower and upper boundaries for the FSN of each cluster based on the following considerations.”
MethodResultInfluential
Is Migraine Associated to Brain Anatomical Alterations? New Data and Coordinate-Based Meta-analysis
Masson, R., Demarquay, G., David Meunier, A., Lévêque, Y., Hannoun, S., Bidet‐Caulet, A., & Caclin, A. (2021) · Brain Topography · doi:10.1007/s10548-021-00824-6 · Cited by 36 · see citing works
Widespread and lateralized social brain activity for processing dynamic facial expressions
Sato, W., Kochiyama, T., Uono, S., Sawada, R., Kubota, Y., Yoshimura, S., & Toichi, M. (2019) · Human Brain Mapping · doi:10.1002/hbm.24629 · Cited by 31 · see citing works
Meta-analytic connectivity modelling of functional magnetic resonance imaging studies in autism spectrum disorders
Goodwill, A., Low, L., Fox, P., Fox, P., Poon, K., Bhowmick, S., & Chen, S. A. A. (2023) · Brain Imaging and Behavior · doi:10.1007/s11682-022-00754-2 · Cited by 30 · see citing works
Mapping brain activity of gut-brain signaling to appetite and satiety in healthy adults: A systematic review and functional neuroimaging meta-analysis
Althubeati, S., Avery, A., Tench, C. R., Lobo, D. N., Salter, A. M., & Eldeghaidy, S. (2022) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2022.104603 · Cited by 28 · see citing works
“As this tool is not specifically designed for neuroimaging studies, the risk of bias might be underestimated (Acar et al., 2018).”
Method
Bridging phenomenology and neural mechanisms of inner speech: ALE meta-analysis on egocentricity and spontaneity in a dual-mechanistic framework
Pratts, J., Pobric, G., & Yao, B. (2023) · NeuroImage · doi:10.1016/j.neuroimage.2023.120399 · Cited by 27 · see citing works
“This consists of re-running the GingerALE analysis whilst iteratively adding an increasing number of randomly-generated null-result studies (Acar et al., 2018).”
BackgroundMethodInfluential
Emotion regulation: From neural circuits to a transdiagnostic perspective.
Morawetz, C., Hemetsberger, F. J., Laird, A. R., & Kohn, N. (2024) · Neuroscience and Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105960 · Cited by 27 · see citing works
“Being data-driven, metaanalyses rely on published findings, which can introduce publication bias (Acar et al., 2018; Jennings and Van Horn, 2012a; Lin and Chu, 2018).”
Accurate localization and coactivation profiles of the frontal eye field and inferior frontal junction: an ALE and MACM fMRI meta-analysis
Bedini, M., Olivetti, E., Avesani, P., & Baldauf, D. (2023) · Brain Structure and Function · doi:10.1007/s00429-023-02641-y · Cited by 34 · see citing works
“It is not clear to what extent these IFJ activations can be replicated, and there may be a file drawer problem in this sample, which is typically an issue in ALE meta-analyses (Acar et al. 2018).”
A meta-analytic approach to mapping co-occurrent grey matter volume increases and decreases in psychiatric disorders
Mancuso, L., Fornito, A., Costa, T., Ficco, L., Liloia, D., Manuello, J., Duca, S., & Cauda, F. (2020) · bioRxiv · doi:10.1101/2020.05.18.101436 · Cited by 25
“As our COA-O dataset of experiments is relatively small, we assessed the robustness of our results to the injection of noise with a modified Fail-safe technique (Acar et al., 2018).”
BackgroundMethodInfluential
A Meta-Analysis of fMRI Studies of Semantic Cognition in Children
Enge, A., Rahman, R. A., & Skeide, M. A. (2021) · bioRxiv · doi:10.1101/2021.05.17.442947 · Cited by 24
Trends in the sample size, statistics, and contributions to the BrainMap database of activation likelihood estimation meta‐analyses: An empirical study of 10‐year data
Yeung, A. W. K., Robertson, M., Uecker, A. M., Fox, P., & Eickhoff, S. (2022) · Human Brain Mapping · doi:10.1002/hbm.26177 · Cited by 23 · see citing works
“One of the commonest CBMA approach in the literature is the activation likelihood estimation (ALE) approach (Acar et al., 2018; Samartsidis et al., 2020; Tahmasian et al., 2019; Yeung et al., 2019).”
BackgroundMethodSelf-citation
Reappraisal and empathic perspective-taking - More alike than meets the eyes
Morawetz, C., Berboth, S., Kohn, N., Philip, L., & Jauniaux, J. (2022) · NeuroImage · doi:10.1016/j.neuroimage.2022.119194 · Cited by 23 · see citing works
Age-related reduction in trait anxiety: Behavioral and neural evidence of automaticity in negative facial emotion processing
Chaudhary, S., Zhang, S., Zhornitsky, S., Chen, Y., Chao, H., & Li, C. (2023) · NeuroImage · doi:10.1016/j.neuroimage.2023.120207 · Cited by 22 · see citing works
Revealing the Selectivity of Neuroanatomical Alteration in Autism Spectrum Disorder via Reverse Inference
Liloia, D., Cauda, F., Uddin, L. Q., Manuello, J., Mancuso, L., Keller, R., Nani, A., & Costa, T. (2022) · Biological Psychiatry Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2022.01.007 · Cited by 23 · see citing works
Cerebral volumetric correlates of apathy in Alzheimer’s disease and cognitively normal older adults: meta-analysis, label-based review, and study of an independent cohort
Chaudhary, S., Zhornitsky, S., Chao, H., van Dyck, C., & Li, C. (2021) · Journal of Alzheimer's Disease · doi:10.3233/jad-215316 · Cited by 22 · see citing works
“Brieﬂy, we used R program to generate “noise” studies from the list of foci, sample size, and the distribution of sample sizes, as in the ﬁve studies included in meta-analysis [49].”
Spontaneous Brain Activity Alterations in First-Episode Psychosis: A Meta-analysis of Functional Magnetic Resonance Imaging Studies
Cattarinussi, G., Grimaldi, D. A., & Sambataro, F. (2023) · Schizophrenia Bulletin · doi:10.1093/schbul/sbad044 · Cited by 21 · see citing works
The major-minor mode dichotomy in music perception.
Carraturo, G., Pando-Naude, V., Costa, M., Vuust, P., Bonetti, L., & Brattico, E. (2024) · Physics of Life Reviews · doi:10.1016/j.plrev.2024.11.017 · Cited by 20 · see citing works
“To address potential publication bias and assess the robustness of the results, the Fail-Safe N analysis (FSN) was performed [49].”
Convergent gray matter alterations across drugs of abuse and network-level implications: A meta-analysis of structural MRI studies.
Hill-Bowen, L. D., Riedel, M., Salo, T., Flannery, J. S., Poudel, R., Laird, A., & Sutherland, M. (2022) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2022.109625 · Cited by 19 · see citing works
Structural and functional brain alterations in subthreshold depression: A multimodal coordinate‐based meta‐analysis
Li, J., Kuang, S., Liu, Y., Wu, Y., & Li, H. (2024) · Human Brain Mapping · doi:10.1002/hbm.26702 · Cited by 20 · see citing works
“To assess the robustness of identified clusters against potential publication bias, we performed the Fail-Safe N (FSN) method adapted for ALE analyses as described by Acar et al. (2018).”
Method
Cognitive and motor impulsivity in the healthy brain, and implications for eating disorders and obesity: A coordinate-based meta-analysis and systematic review.
Mattavelli, G., Gorrino, I., Tornaghi, D., & Canessa, N. (2023) · Cortex; a journal devoted to the study of the nervous system and behavior · doi:10.1016/j.cortex.2023.10.008 · Cited by 18 · see citing works
Parcellation‐based tractographic modeling of the salience network through meta‐analysis
Briggs, R. G., Young, I., Dadario, N., Fonseka, R., Hormovas, J., Allan, P. G., Larsen, M. L., Lin, Y.-H., Tanglay, O., Maxwell, B. D., et al. (2022) · Brain and Behavior · doi:10.1002/brb3.2646 · Cited by 18 · see citing works
Neural correlates of morphological processing: An activation likelihood estimation meta-analysis.
Bulut, T. (2022) · Cortex; a journal devoted to the study of the nervous system and behavior · doi:10.1016/j.cortex.2022.02.010 · Cited by 18 · see citing works
Overlapping Neural Correlates Underpin Theory of Mind and Semantic Cognition: Evidence from a Meta-Analysis of 344 Functional Neuroimaging Studies
Balgova, E., Diveica, V., Jackson, R., & Binney, R. J. (2024) · bioRxiv · doi:10.1101/2023.08.16.553506 · Cited by 17
“Nonetheless, meta-analyses performed on small sample sizes are susceptible to potential publication bias, and caution should be given to interpretation of results from samples with less than 30 studies (Acar et al., 2018).”
Cognitive reappraisal of food craving and emotions: a coordinate-based meta-analysis of fMRI studies
Gerosa, M., Canessa, N., Morawetz, C., & Mattavelli, G. (2023) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsad077 · Cited by 17 · see citing works
“…whether the FSN for each cluster lies between two pre-specified lower and upper boundaries, thus revealing that meta-analytic results are sufficiently robust against the file-drawer publication bias, as well as driven by at least the desired minimum of contributing studies (Acar et al ., 2018).”
BackgroundMethodResultInfluential
Disentangling emotional signals in the brain: an ALE meta-analysis of vocal affect perception
Mauchand, M., & Zhang, S. (2022) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-022-01030-y · Cited by 17 · see citing works
“No cluster survived the minimum FSN of 2k null studies (Acar et al., 2018) in any of the emotion-specific analyses.”
BackgroundMethod
Evaluation of thresholding methods for activation likelihood estimation meta‐analysis via large‐scale simulations
Frahm, L., Cieslik, E. C., Hoffstaedter, F., Satterthwaite, T. D., Fox, P., Langner, R., & Eickhoff, S. (2022) · Human Brain Mapping · doi:10.1002/hbm.25898 · Cited by 17 · see citing works
“Originally introduced in 2002 (Turkeltaub et al., 2002), ALE has since undergone extensive evaluations and improvements in regard to statistical soundness (Acar et al., 2018; Eickhoff et al., 2009; Eickhoff et al., 2012; Turkeltaub et al., 2012).”
BackgroundSelf-citation
Finding specificity in structural brain alterations through Bayesian reverse inference
Cauda, F., Nani, A., Liloia, D., Manuello, J., Premi, E., Duca, S., Fox, P. T., & Costa, T. (2020) · Human Brain Mapping · doi:10.1002/hbm.25105 · Cited by 23 · see citing works
“The fail-safe analysis has showed that the BF results obtained from the anatomical alteration data of AD and SCZ are rather solid, as they survive after relevant injections of noise and they are not, therefore, strongly affected by publication bias (Acar et al., 2018).”
BackgroundMethod
Universality and language specificity of brain Reading networks: A developmental perspective
Zhang, J., Chen, J., & Ding, G. (2024) · Developmental Science · doi:10.1111/desc.13379 · Cited by 16 · see citing works
Toward coordinate-based cognition dictionaries: A brainmap and neurosynth demo.
Lu, Q.-Y., Towne, J. M., Lock, M., Jiang, C., Cheng, Z.-X., Habes, M., Zuo, X., & Zang, Y. (2022) · Neuroscience · doi:10.1016/j.neuroscience.2022.02.016 · Cited by 16 · see citing works
A systematic review and activation likelihood estimation meta-analysis of the central innervation of the lower urinary tract: Pelvic floor motor control and micturition
Groenendijk, I., Mehnert, U., Groen, J., Clarkson, B., Scheepe, J., & Blok, B. (2020) · PLoS ONE · doi:10.1371/journal.pone.0246042 · Cited by 16 · see citing works
“As this tool is not specifically designed for neuroimaging studies, the risk of bias might be underestimated [62].”
Method
Invariant structural and functional brain regions associated with tinnitus: A meta-analysis
Moring, J., Husain, F., Gray, J. P., Franklin, C., Peterson, A., Resick, P., Garrett, A., Esquivel, C. R., & Fox, P. T. (2022) · PLoS ONE · doi:10.1371/journal.pone.0276140 · Cited by 15 · see citing works
“[22] developed the fail-safe N method in order to account for publication bias by introducing noise into the ALE algorithm.”
Method
Grey matter abnormalities in Tourette syndrome: an activation likelihood estimation meta-analysis
Wen, F., Yan, J., Yu, L., Wang, F., Liu, J.-R., Li, Y., & Cui, Y.-H. (2020) · BMC Psychiatry · doi:10.1186/s12888-021-03187-1 · Cited by 15 · see citing works
Food and non‐alcoholic beverage marketing in children and adults: A systematic review and activation likelihood estimation meta‐analysis of functional magnetic resonance imaging studies
Boyland, E., Maden, M., Coates, A. E., Masterson, T. D., Alblas, M. C., Bruce, A. S., & Roberts, C. A. (2024) · Obesity Reviews · doi:10.1111/obr.13643 · Cited by 14 · see citing works
Standardized Effect Sizes and Image-Based Meta-Analytical Approaches for fMRI Data
Bossier, H., Nichols, T. E., & Moerkerke, B. (2019) · doi:10.1101/865881 · Cited by 14 · see citing works
“Most of these methods and diagnostic tools focus on coordinate-based meta-analyses that combine results over studies using only the 3D coordinates of peak activation from the individual studies (Acar et al., 2018; Albajes-Eizagirre and Radua, 2018; Eickhoff et al., 2009, 2012; Turkeltaub et al., 2002, 2012; Radua et a…”
MethodSelf-citation
Mapping the mentalizing brain: An ALE meta-analysis to differentiate the representation of social scenes and ages on theory of mind.
Gan, R., Qiu, Y.-D., Liao, J., Zhang, Y.-T., Wu, J., Peng, X., Lee, T. M., & Huang, R. (2024) · Neuroscience and Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105918 · Cited by 14 · see citing works
Neural underpinnings of response inhibition in substance use disorders: weak meta-analytic evidence for a widely used construct
Fascher, M., Nowaczynski, S., Spindler, C., Strobach, T., & Muehlhan, M. (2023) · Psychopharmacology · doi:10.1007/s00213-023-06498-1 · Cited by 14 · see citing works
“To estimate robustness of our results against this type of publications bias, we performed a fail-safe-N (FSN) calculation explicitly adapted to ALE meta-analyses by Acar et al. (2018).”
Neural mechanisms of brand love relationship dynamics: Is the development of brand love relationships the same as that of interpersonal romantic love relationships?
Watanuki, S. (2022) · Frontiers in Neuroscience · doi:10.3389/fnins.2022.984647 · Cited by 14 · see citing works
“, 2009) among the CBMA methods because the ALE method is the most popular among them (Acar et al., 2018).”
Method
Estimating the number of missing experiments in a neuroimaging meta-analysisPreprint
Samartsidis, P., Montagna, S., Laird, A., Fox, P., Johnson, T., & Nichols, T. E. (2017) · bioRxiv · doi:10.1101/225425 · Cited by 14 · see citing works
Large-scale meta-analyses and network analyses of neural substrates underlying human escalated aggression
Wang, L., Li, T., Gu, R.-L., & Feng, C. (2024) · NeuroImage · doi:10.1016/j.neuroimage.2024.120824 · Cited by 13 · see citing works
“A higher FSN indicates more stable results and hence a higher robustness (Acar et al., 2018).”
The extended neural architecture of human attachment: An fMRI coordinate-based meta-analysis of affiliative studies
Bortolini, T., Laport, M. C., Latgé-Tovar, S., Fischer, R. D., Zahn, R., de Oliveira‐Souza, R., & Moll, J. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105584 · Cited by 20 · see citing works
“The ALE method is probably the most popular CBMA algorithm for fMRI experiments (Acar et al., 2018).”
MethodInfluential
Brain regions show different metabolic and protein arginine methylation phenotypes in frontotemporal dementias and Alzheimer's disease.Preprint
Zhang, F., Rakhimbekova, A., Lashley, T., & Madl, T. (2022) · Progress in neurobiology · doi:10.2139/ssrn.4195324 · Cited by 13 · see citing works
“It has been reported that cerebellar grey matter atrophy occurs in FTD (Acar et al., 2018).”
Seeking Overlapping Neuroanatomical Alterations between Dyslexia and Attention-Deficit/Hyperactivity Disorder: A Meta-Analytic Replication Study
Liloia, D., Crocetta, A., Cauda, F., Duca, S., Costa, T., & Manuello, J. (2022) · Brain Sciences · doi:10.3390/brainsci12101367 · Cited by 16 · see citing works
“While these CBMA methods test the spatial convergence across coordinates, our PSI-SDM approach conducts standard univariate voxel-wise tests [38,79].”
Influential
Mapping cognitive deficits in cancer patients after chemotherapy: An Activation Likelihood Estimation meta-analysis of task-related fMRI studies
Saward, J. B., Ellis, E. G., Cobden, A. L., & Caeyenberghs, K. (2022) · Brain Imaging and Behavior · doi:10.1007/s11682-022-00655-4 · Cited by 13 · see citing works
Neuroimaging alterations in dementia with Lewy bodies and neuroimaging differences between dementia with Lewy bodies and Alzheimer's disease: An activation likelihood estimation meta‐analysis
Ma, W., Tian, M.-J., Yao, Q.-L., Li, Q., Tang, F., Xiao, C., Shi, J., & Chen, J. (2021) · CNS Neuroscience & Therapeutics · doi:10.1111/cns.13775 · Cited by 13 · see citing works
Economic value in the Brain: A meta-analysis of willingness-to-pay using the Becker-DeGroot-Marschak auction
Newton-Fenner, A., Hewitt, D., Henderson, J., Roberts, H., Mari, T., Gu, Y.-Q., Gorelkina, O., Giesbrecht, T., Fallon, N., Roberts, C. A., et al. (2023) · PLoS ONE · doi:10.1371/journal.pone.0286969 · Cited by 12 · see citing works
The road not taken: Common and distinct neural correlates of regret and relief
Varma, M. M., Chowdhury, A., & Yu, R. (2023) · NeuroImage · doi:10.1016/j.neuroimage.2023.120413 · Cited by 10 · see citing works
“A separate ALE meta-analysis was run for each adjusted maximum FSN.”
BackgroundMethodInfluential
ALE meta-analyses of voxel-based morphometry studies: Parameter validation via large-scale simulations
Frahm, L., Satterthwaite, T. D., Fox, P., Langner, R., & Eickhoff, S. (2023) · NeuroImage · doi:10.1016/j.neuroimage.2023.120383 · Cited by 11 · see citing works
“Activation Likelihood Estimation (ALE) is a widely used approach for coordinate-based neuroimaging meta-analysis and has been under constant methodological development and review since its inception in 2002 (Acar et al., 2018; Eickhoff et al., 2009, Turkeltaub et al., 2002, 2012).”
Self-citation
Providing Evidence for the Null Hypothesis in Functional Magnetic Resonance Imaging Using Group-Level Bayesian Inference
Masharipov, R., Knyazeva, I., Nikolaev, Y., Korotkov, A., Didur, M., Cherednichenko, D., & Kireev, M. (2021) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2021.738342 · Cited by 11 · see citing works
“This publishing bias is also known as the ‘file-drawer problem’ (Rosenthal, 1979; Ioannidis et al., 2014; de Winter and Dodou, 2015; for evidence in fMRI studies, see Jennings and Van Horn, 2012; Acar et al., 2018; David et al., 2018; Samartsidis et al., 2020).”
Background
Invasive Motor Cortex Stimulation Influences Intracerebral Structures in Patients With Neuropathic Pain: An Activation Likelihood Estimation Meta‐Analysis of Imaging Data
Volkers, R., Giesen, E., van der Heiden, M., Kerperien, M., Lange, S., Kurt, E., van Dongen, R. V., Schutter, D., Vissers, K., & Henssen, D. (2020) · Neuromodulation (Malden, Mass.) · doi:10.1111/ner.13119 · Cited by 11 · see citing works
Two different mirror neuron pathways for social and non-social actions? A meta-analysis of fMRI studies
Zhao, M., Li, R., Xiang, S., & Liu, N. (2024) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsae068 · Cited by 10 · see citing works
“To account for the file drawer problem, which is a type of publication bias that refers to unpublished experiments and can potentially influence results and the validity of findings (Hayasaka et al. 2018, Samartsidis et al. 2020), the Fail-Safe N (FSN) analysis was performed.”
Background
Effects of meditation on neural responses to pain: A systematic review and meta-analysis of fMRI studies
Fan, C., Wu, M., Liu, H., Chen, X., Zhen, G., Zhao, X., Zhou, J., & Jiang, Z. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105735 · Cited by 9 · see citing works
Appetitive and aversive cue reactivities differentiate neural subtypes of alcohol drinkers
Chen, Y., & Li, C. (2023) · Addiction Neuroscience · doi:10.1016/j.addicn.2023.100089 · Cited by 10 · see citing works
“We also performed a “Fail-Safe N (FSN)” analysis to evaluate potential publication bias [1].”
Method
Neural correlates of affective control regions induced by common therapeutic strategies in major depressive disorders: an Activation Likelihood Estimation meta-analysis study.
Li, C.-H., Hu, Q.-L., Zhang, D., Hoffstaedter, F., Bauer, A., & Elmenhorst, D. (2022) · Neuroscience and Biobehavioral Reviews · doi:10.1016/j.neubiorev.2022.104643 · Cited by 10 · see citing works
Bridging the Brain and Data Sciences
Horn, J. D. V. (2020) · Big Data · doi:10.1089/big.2020.0065 · Cited by 11 · see citing works
“53 However, users may need to consider biases associated with ‘‘ﬁle drawer problems’’ where only statistically signiﬁcant results from a literature results are included 54 and, thus, may suffer from ‘‘so-called’’ publication bias.”
Linguistic network in early deaf individuals: A neuroimaging meta-analysis
Yang, T., Fan, X., Hou, B., Wang, J., & Chen, X. (2024) · NeuroImage · doi:10.1016/j.neuroimage.2024.120720 · Cited by 9 · see citing works
“Briefly, this procedure required the iterative introduction of null experiments resembling the original experiments into the original dataset and retesting the significance of the results (Acar et al., 2018).”
Method
A Minimum Bayes Factor Based Threshold for Activation Likelihood Estimation
Costa, T., Liloia, D., Cauda, F., Fox, P. T., Mutta, F. D., Duca, S., & Manuello, J. (2023) · Neuroinformatics · doi:10.1007/s12021-023-09626-6 · Cited by 11 · see citing works
“…unrelated cognitive domains (see Table S2 for a description of this dataset), and on two simulated data-sets generated with the Fail-safe R script (Acar et al. (2018); https:// github. com/ Neuro Stat/ Gener ateNu ll) and built with 21 experiments each with foci randomly placed across the brain.”
Method
Neural substrates of brand equity: applying a quantitative meta-analytical method for neuroimage studies
Watanuki, S., & Akama, H. (2022) · Heliyon · doi:10.1016/j.heliyon.2022.e09702 · Cited by 9 · see citing works
“First, it is the most popular quantitative meta-analysis method (Acar et al., 2018).”
Background
Whole-brain functional neuroimaging correlates of cognitive flexibility impairments in people with mental disorders: A transdiagnostic coordinate-based meta-analysis.
Meda, N., Baggio, M., Collantoni, E., & Sambataro, F. (2025) · European Neuropsychopharmacology · doi:10.1016/j.euroneuro.2025.01.009 · Cited by 8 · see citing works
Substance use disorders are characterised by increased voxel-wise intrinsic measures in sensorimotor cortices: An ALE meta-analysis.
Fascher, M., Nowaczynski, S., & Muehlhan, M. (2024) · Neuroscience and Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105712 · Cited by 8 · see citing works
“We adhered to the parsimonious protocol and the code for this FSN-calculation procedure provided by Acar and colleagues (Acar et al., 2018).”
The dark sides of the brain: A systematic review and meta-analysis of functional neuroimaging studies on trait aggression,
Dugré, J., Hopfer, C., & Winters, D. E. (2025) · Aggression and Violent Behavior · doi:10.1016/j.avb.2025.102035 · Cited by 7 · see citing works
“Since the ALE algorithm does not contain effect sizes, artificial studies were randomly generated from the original sample (e.g., number of peaks and sample size) and added to the original meta-analysis (see Acar et al., 2018); implementation by Enge et al., 2020 and Bortolini et al., 2024).”
Altered anticipatory brain responses in eating disorders: A neuroimaging meta‐analysis
Yu, X., & Desriviéres, S. (2023) · European eating disorders review · doi:10.1002/erv.2967 · Cited by 7 · see citing works
“Fail‐Safe N (FSN) quantifies the number of null studies (studies with negative results or contra‐evi-dence) that are needed to change current results (Acar et al., 2018; Rosenthal, 1979).”
Background
Cortical Activation During Finger Tapping Task Performance in Parkinson's Disease Is Influenced by Priming Conditions: An ALE Meta-Analysis
Li, J., Liu, Z., Du, Z., Zhu, N., Qiu, X., & Xu, X. (2021) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2021.774656 · Cited by 7 · see citing works
“Therefore, the present study used activation likelihood estimation (ALE) meta-analysis, which is a quantitative meta-analytic method that has been widely utilized to determine the stereoscopic brain coordinates that have been consistently active across studies (Turkeltaub et al., 2002; Acar et al., 2018).”
Method
Cortical Gradients Support Mental Time Travel into the Past and Future: Evidence from Activation Likelihood Estimation Meta-analysis
Teghil, A., Wiener, M., & Boccia, M. (2025) · Neuropsychology Review · doi:10.1007/s11065-025-09662-w · Cited by 3 · see citing works
Cognition, body, and mind: A three‐in‐one coordinate‐based fMRI meta‐analysis on cognitive, physical, and meditative trainings
Presti, S. L., Origlia, S., Gianelli, C., & Canessa, N. (2023) · Human Brain Mapping · doi:10.1002/hbm.26312 · Cited by 6 · see citing works
“Following Acar et al. (2018), we prespecified lower and upper boundaries for the FSN of each cluster based on the following considerations.”
BackgroundMethodResultInfluential
Common neural dysfunction in psychiatric disorders: Insights from a meta-analysis of resting-state fMRI studies
Wang, L., Liu, Q.-X., Zheng, Z.-Y., Chen, W.-Q., Shen, Y.-X., Li, T., & Feng, C.-L. (2025) · Translational Psychiatry · doi:10.1038/s41398-025-03760-2 · Cited by 5 · see citing works
“The amount of contra-evidence that the ALE can tolerate (i.e., FSN) was calculated by introducing null pseudo-studies as noise into the ALE cohort to assess the robustness of ALE clusters [58, 59].”
Neuroimaging signatures of mesial temporal lobe epilepsy: A coordinate-based meta-analysis of structural and resting-state functional imaging literature
Towne, J. M., Lami, V., Barron, D. S., Cavazos, J., & Fox, P. T. (2025) · NeuroImage: Clinical · doi:10.1016/j.nicl.2025.103908 · Cited by 5 · see citing works
“Finally, publication bias, where studies with statistically significant results are more likely to be published, can lead to an overestimation of the true effect size (Acar et al., 2018).”
Joyful Growth vs. Compulsive Hedonism: A Meta-Analysis of Brain Activation on Romantic Love and Addictive Disorders.
Yang, Y., Wang, C., Shi, J.-N., & Zou, Z.-L. (2024) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2024.109003 · Cited by 5 · see citing works
The neural networks of touch observation
Schaefer, M., Kuehn, E., Schweitzer, F., & Muehlhan, M. (2024) · Imaging Neuroscience · doi:10.1162/imag_a_00065 · Cited by 5 · see citing works
“However, we tried to address these issues by using the most recent guidelines ( Acar et al., 2018; Müller et al., 2018) for our procedures to make our results reliable.”
BackgroundMethodInfluential

Authors: Acar, F., Seurinck, R., Eickhoff, S. B., & Moerkerke, B.
Affiliations: Ghent University (BE)
Forschungszentrum Jülich (DE)
Corresponding author: Acar, Freya
ORCID: Acar, Freya
Seurinck, Ruth
Eickhoff, Simon B.
Moerkerke, Beatrijs
Venue: PLOS ONE
Publisher: Public Library of Science (PLoS)
Published: 2018-11-30
Volume: 13
Issue: 11
Pages: e0208177
ISSN: 1932-6203 (ISSN-L)
DOI: 10.1371/journal.pone.0208177
PMC: PMC6267999
OpenAlex: W2902120147
Cited by: 154
References: 48
Citations by year: 2026: 22 · 2025: 19 · 2024: 18 · 2023: 28 · 2022: 27 · 2021: 19 · 2020: 13 · 2019: 7 · 2017: 1
FWCI: 7.44
Type: journal article
Language: en
Open access: gold
doi.org
Licence: cc-by
creativecommons.org/licenses/by/4.0/ (vor)
Locations: PLoS ONE (publishedVersion, cc-by)
Europe PMC (PubMed Central) (publishedVersion, cc-by)
PubMed Central (publishedVersion, cc-by)
PubMed (publishedVersion)
Ghent University Academic Bibliography (Ghent University) (submittedVersion, other-oa)
RePEc: Research Papers in Economics (submittedVersion)
DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
JuSER (Forschungszentrum Jülich) (submittedVersion, other-oa)
Funders: Fonds Wetenschappelijk Onderzoek (G.0149.14)
National Institute of Mental Health (R01-MH074457)
Horizon 2020 (7202070 (HBP SGA1))
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Life Sciences › Neuroscience › Cognitive Neuroscience › Functional Brain Connectivity Studies
Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
Keywords: publication bias, file drawer problem, activation likelihood estimation, coordinate-based meta-analysis, noise study, robustness analysis, spatial convergence
MeSH terms: Algorithms, Brain, Brain Mapping, Humans, Magnetic Resonance Imaging, Meta-Analysis as Topic, Likelihood Functions, Publication Bias

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Assessing robustness against potential publication bias in activation likelihood estimation (ALE) meta-analyses for fMRI

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Self-check

Cites

Cited by

Record

About these checks