The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Implementation errors in the GingerALE Software: Description and recommendations
Open accessAbstract
Neuroscience imaging is a burgeoning, highly sophisticated field the growth of which has been fostered by grant‐funded, freely distributed software libraries that perform voxel‐wise analyses in anatomically standardized three‐dimensional space on multi‐subject, whole‐brain, primary datasets. Despite the ongoing advances made using these non‐commercial computational tools, the replicability of individual studies is an acknowledged limitation. Coordinate‐based meta‐analysis offers a practical solution to this limitation and, consequently, plays an important role in filtering and consolidating the enormous corpus of functional and structural neuroimaging results reported in the peer‐reviewed literature. In both primary data and meta‐analytic neuroimaging analyses, correction for multiple comparisons is a complex but critical step for ensuring statistical rigor. Reports of errors in multiple‐comparison corrections in primary‐data analyses have recently appeared. Here, we report two such errors in GingerALE, a widely used, US National Institutes of Health (NIH)‐funded, freely distributed software package for coordinate‐based meta‐analysis. These errors have given rise to published reports with more liberal statistical inferences than were specified by the authors. The intent of this technical report is threefold. First, we inform authors who used GingerALE of these errors so that they can take appropriate actions including re‐analyses and corrective publications. Second, we seek to exemplify and promote an open approach to error management. Third, we discuss the implications of these and similar errors in a scientific environment dependent on third‐party software. Hum Brain Mapp 38:7–11, 2017. © 2016 Wiley Periodicals, Inc.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Jenkinson, M., Beckmann, C. F., Behrens, T. E. J., Woolrich, M. W., & Smith, S. M. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2011.09.015 · Cited by 11,314
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515 · cited 1×
“…software to make inferences regarding the structural and functional organization of the human brain in development, in adulthood, in aging, and in a wide variety of neurologic, psychiatric and systemic conditions in an ongoing and programmatic manner [Bandettini, 2012; Rosen and Savoy, 2012].”
MethodPower failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · cited 1×
“…variations between studies apparently reporting on the same effect in the same condition), and the indirect nature of the neuroimaging measures used vis-(cid:2) a-vis the inferred neuronal physiology and pathology [Button et al., 2013, Carp, 2012; Glatard et al., 2015; Rottschy et al., 2013].”
BackgroundThresholding of statistical maps in functional neuroimaging using the false discovery rate
Genovese, C. R., Lazar, N. A., & Nichols, T. (2002) · NeuroImage · doi:10.1006/nimg.2001.1037 · Cited by 4,923
Cluster failure: why fMRI inferences for spatial extent have inflated false-positive rates
Eklund, A., Nichols, T. E., & Knutsson, H. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1602413113 · Cited by 3,487
Promoting an open research culture
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., et al. (2015) · Science · doi:10.1126/science.aab2374 · Cited by 2,708
Eickhoff, S. B., Laird, A. R., Grefkes, C., Wang, L. E., Zilles, K., & Fox, P. T. (2009) · Human Brain Mapping · doi:10.1002/hbm.20718 · Cited by 1,924
Self-citationComparing Functional (PET) Images: The Assessment of Significant Change
Friston, K. J., Frith, C. D., Liddle, P. F., & Frackowiak, R. S. J. (1991) · Journal of Cerebral Blood Flow & Metabolism · doi:10.1038/jcbfm.1991.122 · Cited by 1,625
Meta-analysis of the functional neuroanatomy of single-word reading: method and validation
Turkeltaub, P. E., Eden, G. F., Jones, K. M., & Zeffiro, T. A. (2002) · NeuroImage · doi:10.1006/nimg.2002.1131 · Cited by 1,604
Activation likelihood estimation meta-analysis revisited
Eickhoff, S. B., Bzdok, D., Laird, A. R., Kurth, F., & Fox, P. T. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2011.09.017 · Cited by 1,443
Self-citation
Cited by
Ten simple rules for neuroimaging meta-analysis
Müller, V. I., Cieslik, E. C., Laird, A. R., Fox, P. T., Radua, J., Mataix-Cols, D., Tench, C. R., Yarkoni, T., Nichols, T. E., Turkeltaub, P. E., et al. (2018) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2017.11.012 · Cited by 805
Self-citationOldham, S., Murawski, C., Fornito, A., Youssef, G., Yücel, M., & Lorenzetti, V. (2018) · Human Brain Mapping · doi:10.1002/hbm.24184 · Cited by 425
Large-scale functional neural network correlates of response inhibition: an fMRI meta-analysis
Zhang, R., Geng, X., & Lee, T. M. C. (2017) · Brain Structure and Function · doi:10.1007/s00429-017-1443-x · Cited by 398
“It has been suggested that the replicability of meta-analyses including less than 30 studies is limited, and the conclusions drawn may, therefore, be questionable (Eickhoff et al. 2016b).”
BackgroundMethodInfluentialAltered Brain Activity in Unipolar Depression Revisited: Meta-analyses of Neuroimaging Studies
Müller, V. I., Cieslik, E. C., Serbanescu, I., Laird, A. R., Fox, P. T., & Eickhoff, S. B. (2017) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2016.2783 · Cited by 304
Self-citationThe neural correlates of semantic control revisited
Jackson, R. L. (2021) · NeuroImage · doi:10.1016/j.neuroimage.2020.117444 · Cited by 316
Brain areas associated with numbers and calculations in children: Meta-analyses of fMRI studies
Arsalidou, M., Pawliw-Levac, M., Sadeghi, M., & Pascual-Leone, J. (2018) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2017.08.002 · Cited by 267
Samea, F., Soluki, S., Nejati, V., Zarei, M., Cortese, S., Eickhoff, S. B., Tahmasian, M., & Eickhoff, C. R. (2019) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2019.02.011 · Cited by 245
“, 2008) used a version of GingerALE, which was later discovered to have a bug leading to inadequately liberal thresholds that were substantially different from the requested level and did not control for multiple comparisons (Eickhoff et al., 2017).”
Self-citationNg, T. H., Alloy, L. B., & Smith, D. V. (2019) · Translational Psychiatry · doi:10.1038/s41398-019-0644-x · Cited by 244
Hung, Y., Gaillard, S. L., Yarmak, P., & Arsalidou, M. (2018) · Human Brain Mapping · doi:10.1002/hbm.24232 · Cited by 212
Fouragnan, E., Retzler, C., & Philiastides, M. G. (2018) · Human Brain Mapping · doi:10.1002/hbm.24047 · Cited by 176
Powers, J. P., & LaBar, K. S. (2019) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2018.04.023 · Cited by 173
Vijayakumar, N., Cheng, T. W., & Pfeifer, J. H. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.02.050 · Cited by 161
“…of p < 0.05, 5000 thresholding permutations, and an uncorrected p-value of < 0.001 (Note: this version of the software has addressed a previous error in GingerALE that resulted in inferences on uncorrected cluster-level p values as opposed to FWE corrected clusters; Eickhoff et al., 2016).”
MethodInfluentialA supramodal role of the basal ganglia in memory and motor inhibition: meta-analytic evidence
Guo, Y., Schmitz, T. W., Mur, M., Ferreira, C. S., & Anderson, M. C. (2018) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2017.11.033 · Cited by 133
Biomarkers of Pathological Dissociation: A Systematic Review
Roydeva, M. I., & Reinders, A. A. T. S. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.11.019 · Cited by 116
Li, Q., Yang, G., Li, Z., Qi, Y., Cole, M. W., & Liu, X. (2017) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2017.09.032 · Cited by 96
Progress toward openness, transparency, and reproducibility in cognitive neuroscience
Gilmore, R. O., Diaz, M. T., Wyble, B. A., & Yarkoni, T. (2017) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.13325 · Cited by 95
Feng, C., Becker, B., Huang, W., Wu, X., Eickhoff, S. B., & Chen, T. (2018) · NeuroImage · doi:10.1016/j.neuroimage.2018.02.023 · Cited by 58
Self-citationZheng, L., Miao, M., & Gan, Y. (2022) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2022.104721 · Cited by 41
Neural correlates of working memory training: An fMRI meta-analysis
Zhang, Y., Fu, J., & Zhao, X. (2024) · NeuroImage · doi:10.1016/j.neuroimage.2024.120885 · Cited by 20
Vartanian, O., Replete, V., Saint, S. A., Lam, Q., Forbes, S., Beaudoin, M. E., Brunyé, T. T., Bryant, D. J., Feltman, K. A., Heaton, K. J., et al. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.868001 · Cited by 15
Metadata
- Authors
- Eickhoff, S. B., Laird, A. R., Fox, P. M., Lancaster, J. L., & Fox, P. T.
- Venue
- Human Brain Mapping
- Publisher
- Wiley
- Published
- 2016-08-11
- Volume
- 38
- Issue
- 1
- Pages
- 7-11
- ISSN
- 1065-9471 (ISSN-L)
- 1097-0193
- OpenAlex
- W2516407199
- Cited by
- 269
- References
- 36
- FWCI
- 18.41
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.