The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Potential Reporting Bias in fMRI Studies of the Brain
Open accessAbstract
BACKGROUND: Functional magnetic resonance imaging (fMRI) studies have reported multiple activation foci associated with a variety of conditions, stimuli or tasks. However, most of these studies used fewer than 40 participants. METHODOLOGY: After extracting data (number of subjects, condition studied, number of foci identified and threshold) from 94 brain fMRI meta-analyses (k = 1,788 unique datasets) published through December of 2011, we analyzed the correlation between individual study sample sizes and number of significant foci reported. We also performed an analysis where we evaluated each meta-analysis to test whether there was a correlation between the sample size of the meta-analysis and the number of foci that it had identified. Correlation coefficients were then combined across all meta-analyses to obtain a summary correlation coefficient with a fixed effects model and we combine correlation coefficients, using a Fisher's z transformation. PRINCIPAL FINDINGS: There was no correlation between sample size and the number of foci reported in single studies (r = 0.0050) but there was a strong correlation between sample size and number of foci in meta-analyses (r = 0.62, p<0.001). Only studies with sample sizes <45 identified larger (>40) numbers of foci and claimed as many discovered foci as studies with sample sizes ≥ 45, whereas meta-analyses yielded a limited number of foci relative to the yield that would be anticipated from smaller single studies. CONCLUSIONS: These results are consistent with possible reporting biases affecting small fMRI studies and suggest the need to promote standardized large-scale evidence in this field. It may also be that small studies may be analyzed and reported in ways that may generate a larger number of claimed foci or that small fMRI studies with inconclusive, null, or not very promising results may not be published at all.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“may avoid reporting biases that are likely to affect certain brain regions, especially smaller subcortical regions that are poorly represented in commonly used stereotaxic atlases of the human brain.15,16”
From Neural signatures of human fear conditioning: an updated and extended meta-analysis of fMRI studies (2016) · cites this paper as [15]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
Sterne, J. A. C., Sutton, A. J., Ioannidis, J. P. A., Terrin, N., Jones, D. R., Lau, J., Carpenter, J., Rucker, G., Harbord, R. M., Schmid, C. H., et al. (2011) · BMJ · doi:10.1136/bmj.d4002 · Cited by 7,709
Self-citationPower failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · cited 1×
“Moreover, a recent paper by authors of the present paper (MM & JPAI) reported power analyses of single studies included in 49 metaanalyses and found that almost 50% of studies had an average power lower than 20% [24].”
ResultSelf-citationPublication bias in clinical research
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991) · The Lancet · doi:10.1016/0140-6736(91)90201-y · Cited by 3,109 · cited 1×
“While it is reasonable that particular regions of the brain are indeed activated during a given task, it is also possible that positive findings are more likely to be reported, either because investigators are more likely to report them or because journals are more likely to publish positive findings [14,15,16].”
BackgroundEickhoff, S. B., Laird, A. R., Grefkes, C., Wang, L. E., Zilles, K., & Fox, P. T. (2009) · Human Brain Mapping · doi:10.1002/hbm.20718 · Cited by 1,924
The case of the misleading funnel plot
Lau, J., Ioannidis, J. P. A., Terrin, N., Schmid, C. H., & Olkin, I. (2006) · BMJ · doi:10.1136/bmj.333.7568.597 · Cited by 1,766
Self-citationMeta-analysis of the functional neuroanatomy of single-word reading: method and validation
Turkeltaub, P. E., Eden, G. F., Jones, K. M., & Zeffiro, T. A. (2002) · NeuroImage · doi:10.1006/nimg.2002.1131 · Cited by 1,604
Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias
Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A.-W., Cronin, E., Decullier, E., Easterbrook, P. J., Von Elm, E., Gamble, C., et al. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003081 · Cited by 1,546
Self-citationActivation Likelihood Estimation meta-analysis revisited
Eickhoff, S., Bzdok, D., Laird, A., Kurth, F., & Fox, P. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.09.017 · Cited by 1,448
ALE meta-analysis of action observation and imitation in the human brain
Caspers, S., Zilles, K., Laird, A. R., & Eickhoff, S. B. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2009.12.112 · Cited by 1,386
Voxel-wise meta-analysis of grey matter changes in obsessive-compulsive disorder
Radua, J., & Mataix-Cols, D. (2009) · British Journal of Psychiatry · doi:10.1192/bjp.bp.108.055046 · Cited by 863
How Many Subjects Constitute a Study?
Friston, K. J., Holmes, A. P., & Worsley, K. J. (1999) · NeuroImage · doi:10.1006/nimg.1999.0439 · Cited by 814 · cited 2×
“Some authors have proposed optimal sample sizes of 16-32 subjects per group [19,20,21], suggesting that between-subject comparison studies of n,30 are too small even by liberal estimates.”
BackgroundRadua, J., Mataix-Cols, D., Phillips, M. L., El-Hage, W., Kronhaus, D. M., Cardoner, N., & Surguladze, S. (2012) · European Psychiatry · doi:10.1016/j.eurpsy.2011.04.001 · Cited by 765
An exploratory test for an excess of significant findings
Ioannidis, J. P., & Trikalinos, T. A. (2007) · Clinical Trials · doi:10.1177/1740774507079441 · Cited by 751
Self-citationHandbook of Functional MRI Data Analysis
Poldrack, R. A., Mumford, J. A., & Nichols, T. E. (2011) · Cambridge University Press eBooks · doi:10.1017/cbo9780511895029 · Cited by 729
Estimating sample size in functional MRI (fMRI) neuroimaging studies: Statistical power analyses
Desmond, J. E., & Glover, G. H. (2002) · Journal of Neuroscience Methods · doi:10.1016/s0165-0270(02)00121-8 · Cited by 716
Naturalizing aesthetics: Brain areas for aesthetic appraisal across sensory modalities
Brown, S., Gao, X., Tisdelle, L., Eickhoff, S. B., & Liotti, M. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.06.012 · Cited by 376
Ten ironic rules for non-statistical reviewers
Friston, K. J. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.04.018 · Cited by 368
Neuroanatomy of vulnerability to psychosis: a voxel-based meta-analysis.
Fusar-Poli, P., Borgwardt, S., Crescini, A., Deste, G., Kempton, M., Lawrie, S., Guire, P. M., & Sacchetti, E. (2011) · Neuroscience and Biobehavioral Reviews · doi:10.1016/j.neubiorev.2010.12.005 · Cited by 357
Self-citationExcess Significance Bias in the Literature on Brain Volume Abnormalities
Ioannidis, J. P. A. (2011) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2011.28 · Cited by 299
Self-citationNeuroanatomical Maps of Psychosis Onset: Voxel-wise Meta-Analysis of Antipsychotic-Naive VBM Studies
Fusar‐Poli, P., Raduà, J., McGuire, P. K., & Borgwardt, S. (2011) · Schizophrenia Bulletin · doi:10.1093/schbul/sbr134 · Cited by 291
Self-citationMeta-analytic methods for neuroimaging data explained
Raduà, J., & Mataix‐Cols, D. (2012) · Biology of Mood & Anxiety Disorders · doi:10.1186/2045-5380-2-6 · Cited by 250
Brooks, S. J., Savov, V., Allzén, E., Benedict, C., Fredriksson, R., & Schiöth, H. B. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.09.077 · Cited by 222
Amanzio, M., Benedetti, F., Porro, C. A., Palermo, S., & Cauda, F. (2013) · Human Brain Mapping · doi:10.1002/hbm.21471 · Cited by 202 · cited 1×
“and the assigned stimuli or condition [6,7,8,9,10,11,12,13].”
ALE meta-analysis on facial judgments of trustworthiness and attractiveness
Bzdok, D., Langner, R., Caspers, S., Kurth, F., Habel, U., Zilles, K., Laird, A., & Eickhoff, S. B. (2011) · Brain Structure and Function · doi:10.1007/s00429-010-0287-4 · Cited by 183
Functional neuroimaging in schizophrenia: diagnosis and drug discovery
McGuire, P. K., Howes, O. D., Stone, J. M., & Fusar‐Poli, P. (2008) · Trends in Pharmacological Sciences · doi:10.1016/j.tips.2007.11.005 · Cited by 157
Self-citationConverging Evidence for the Advantage of Dynamic Facial Expressions
Arsalidou, M., Morris, D. M., & Taylor, M. J. (2011) · Brain Topography · doi:10.1007/s10548-011-0171-4 · Cited by 139
Gountouna, V., Job, D. E., McIntosh, A. M., Moorhead, T. W. J., Lymer, G. K. L., Whalley, H. C., Hall, J., Waiter, G. D., Brennan, D., McGonigle, D. J., et al. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.07.026 · Cited by 128
Albrecht, J., Kopietz, R., Frasnelli, J., Wiesmann, M., Hummel, T., & Lundström, J. (2009) · Brain Research Reviews · doi:10.1016/j.brainresrev.2009.11.001 · Cited by 121
Multisite reliability of cognitive BOLD data
Brown, G. G., Mathalon, D. H., Stern, H. S., Ford, J. M., Mueller, B. A., Greve, D. N., McCarthy, G., Voyvodic, J. T., Glover, G. H., Diaz, M. T., et al. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.09.076 · Cited by 83
Faro, S. H., Mohamed, F. B., & Haughton, V. (2006) · Springer eBooks · doi:10.1007/0-387-34665-1 · Cited by 51
Ironing out the statistical wrinkles in “ten ironic rules”
Lindquist, M. A., Caffo, B. C., & Crainiceanu, C. M. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.02.056 · Cited by 48
Kavvoura, F. K., McQueen, M. B., Khoury, M. J., Tanzi, R. E., Bertram, L., & Ioannidis, J. P. A. (2008) · American Journal of Epidemiology · doi:10.1093/aje/kwn206 · Cited by 48
Self-citationNo evidence for structural brain changes in young adolescents at ultra high risk for psychosis
Ziermans, T. B., Durston, S., Sprong, M., Nederveen, H., van Haren, N. E. M., Schnack, H. G., Lahuis, B., Schothorst, P. F., & van Engeland, H. (2009) · Schizophrenia Research · doi:10.1016/j.schres.2009.04.013 · Cited by 35
Brain functional abnormality in schizo-affective disorder: an fMRI study
Madre, M., Pomarol‐Clotet, E., McKenna, P. J., Raduà, J., Ortiz‐Gil, J., Panicali, F., Goikolea, J. M., Vieta, E., Sarró, S., Salvador, R. M., et al. (2012) · Psychological Medicine · doi:10.1017/s0033291712000943 · Cited by 16
Cited by
Sorted by citations
Fractionating theory of mind: a meta-analysis of functional brain imaging studies
Schurz, M., Radua, J., Aichhorn, M., Richlan, F., & Perner, J. (2014) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2014.01.009 · Cited by 1,656
Scanning the horizon: towards transparent and reproducible neuroimaging research
Poldrack, R. A., Baker, C. I., Durnez, J., Gorgolewski, K. J., Matthews, P. M., Munafò, M. R., Nichols, T. E., Poline, J.-B., Vul, E., & Yarkoni, T. (2017) · Nature Reviews Neuroscience · doi:10.1038/nrn.2016.167 · Cited by 1,410
Neural signatures of human fear conditioning: an updated and extended meta-analysis of fMRI studies
Fullana, M. A., Harrison, B. J., Soriano-Mas, C., Vervliet, B., Cardoner, N., Àvila-Parcet, A., & Radua, J. (2016) · Molecular Psychiatry · doi:10.1038/mp.2015.88 · Cited by 702
Publication and other reporting biases in cognitive sciences: detection, prevalence, and prevention
Ioannidis, J. P. A., Munafò, M. R., Fusar-Poli, P., Nosek, B. A., & David, S. P. (2014) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2014.02.010 · Cited by 471
Self-citationSzucs, D., & Ioannidis, J. P. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.117164 · Cited by 453
Self-citation
Metadata
- Authors
- David, S. P., Ware, J. J., Chu, I. M., Loftus, P. D., Fusar-Poli, P., Radua, J., Munafò, M. R., & Ioannidis, J. P. A.
- Affiliations
- SRI International (US)
- University of Bristol (GB)
- Stanford Medicine (US)
- Corresponding author
- David, Sean P.
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2013-07-25
- Volume
- 8
- Issue
- 7
- Pages
- e70104
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC3723634
- OpenAlex
- W1975996475
- Cited by
- 130
- References
- 38
- Citations by year
- 2026: 6 · 2025: 8 · 2024: 14 · 2023: 7 · 2022: 6 · 2021: 6 · 2020: 12 · 2019: 9 · 2018: 15 · 2017: 11 · 2016: 15 · 2015: 11 · 2014: 9 · 2013: 1
- FWCI
- 13.22
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Research Portal (King's College London) (publishedVersion, other-oa)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Bristol Research (University of Bristol) (submittedVersion)
- Funders
- United Kingdom Clinical Research Collaboration
- Cancer Research UK
- National Institute for Health and Care Research
- British Heart Foundation
- Medical Research Council
- Economic and Social Research Council
- National Institute on Drug Abuse (DA-027331, R21DA027331, DA-017441, R01DA017441)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Life Sciences › Neuroscience › Cognitive Neuroscience › Functional Brain Connectivity Studies
- Social Sciences › Education › Academic integrity and plagiarism
- Keywords
- reporting bias, functional MRI, sample size, meta-analysis, publication bias, negative results, brain activation, statistical significance thresholds
- MeSH terms
- Brain, Humans, Magnetic Resonance Imaging, Research Design, Statistics as Topic, Databases, Bibliographic, Publication Bias, Sample Size
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
