Publication Bias in Neuroimaging Research: Implications for Meta-Analyses
Open accessThe DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Abstract
Neuroimaging and the neurosciences have made notable advances in sharing activation results through detailed databases, making meta-analysis of the published research faster and easier. However, the effect of publication bias in these fields has not been previously addressed or accounted for in the developed meta-analytic methods. In this article, we examine publication bias in functional magnetic resonance imaging (fMRI) for tasks involving working memory in the frontal lobes (Brodmann Areas 4, 6, 8, 9, 10, 37, 45, 46, and 47). Seventy-four studies were selected from the literature and the effect of publication bias was examined using a number of regression-based techniques. Pearson’s r correlation coefficient and Cohen’s d effect size estimates were computed for the activation in each study and compared to the study sample size using Egger’s regression, Macaskill’s regression, and the ‘Trim and Fill’ method. Evidence for publication bias was identified in this body of literature (p < 0.01 for each test), generally, though was neither task- nor sub-region-dependent. While we focused our analysis on this subgroup of brain mapping studies, we believe our findings generalize to the brain imaging literature as a whole and databases seeking to curate their collective results. While neuroimaging databases of summary effects are of enormous value to the community, the potential publication bias should be considered when performing meta-analyses based on database contents.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Publication bias in clinical research
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991) · The Lancet · doi:10.1016/0140-6736(91)90201-y · Cited by 3,109 · cited 1×
“Such bias can occur despite the fact that studies with significant results may not appear to be superior to studies having null results with respect to quality of design (Easterbrook et al. 1991).”
ResultDeeks, J. J., Macaskill, P., & Irwig, L. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.01.016 · Cited by 2,910 · cited 1×
“If publication bias is not present, then the points should form a symmetrical inverted funnel around the overall estimate of the effect, with results from smaller studies scattered more widely about the mean effect at the bottom of the graph (see Deeks et al. 2005 for examples).”
Publication and related bias in meta-analysis
Sterne, J. A. C., Gavaghan, D., & Egger, M. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00242-0 · Cited by 2,282 · cited 4×
“While the Egger method has been shown to be highly sensitive with strong statistical power (Sterne et al. 2000), it also tends to have a relatively high false positive rate, although it can be subject to low power when metaanalytically examining results from only small numbers of studies (Peters et al.”
BackgroundInfluentialTemporal dynamics of brain activation during a working memory task
Cohen, J. D., Perlstein, W. M., Braver, T. S., Nystrom, L. E., Noll, D. C., Jonides, J., & Smith, E. E. (1997) · Nature · doi:10.1038/386604a0 · Cited by 1,876
Puzzlingly High Correlations in fMRI Studies of Emotion, Personality, and Social Cognition
Vul, E., Harris, C., Winkielman, P., & Pashler, H. (2009) · Perspectives on Psychological Science · doi:10.1111/j.1745-6924.2009.01125.x · Cited by 1,475 · cited 2×
“Vul et al. (2009) examined how nonindependent region of interest (ROI) analyses (where the correlations are both the selection criteria and the secondary statistics) inflate the results presented in functional imaging research, also known as non-independence or circular analysis.”
BackgroundThe Existence of Publication Bias and Risk Factors for Its Occurrence
Dickersin, K. (1990) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1990.03440100097014 · Cited by 1,417 · cited 2×
“…the tendency of researchers, journal editors, and corporate entities to manage the reporting of experimental findings that are positive (i.e. " significant " findings) differently from findings that are negative (i.e. supporting the null hypothesis) or are otherwise inconclusive (Dickersin 1990).”
ResultA comparison of methods to detect publication bias in meta‐analysis
Macaskill, P., Walter, S. D., & Irwig, L. (2001) · Statistics in Medicine · doi:10.1002/sim.698 · Cited by 1,286 · cited 2×
“Macaskill’s Regression A method which draws on the idea of the funnel plot more systematically is Macaskill’s regression method, also known as the funnel plot regression method (Macaskill et al. 2001).”
MethodFunctional neuroanatomy of emotions: A meta-analysis
MURPHY, F. C., NIMMO-SMITH, I., & LAWRENCE, A. D. (2003) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.3.3.207 · Cited by 1,135 · cited 2×
“In recent years meta-analyses have become increasingly popular in neuroimaging as large databases of structural and functional brain imaging data have been created and employed to aggregate results from across individual studies (Murphy et al. 2003; Neumann et al. 2008; Fusar- Poli et al. 2009).”
MethodPublication bias in clinical trials due to statistical significance or direction of trial results
Hopewell, S., Loudon, K., Clarke, M. J., Oxman, A. D., & Dickersin, K. (2009) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000006.pub3 · Cited by 898 · cited 1×
“Typical reasons for non-publication of non-findings has been attributed to loss of interest in the study in question by the researcher once a null effect has been observed (Hopewell et al. 2009).”
BackgroundHow Many Subjects Constitute a Study?
Friston, K. J., Holmes, A. P., & Worsley, K. J. (1999) · NeuroImage · doi:10.1006/nimg.1999.0439 · Cited by 814 · cited 1×
“This practice has been defended by Friston et al. (1999), who have argued that fixed-effects analyses are adequately serviced through voxel-wise general linear models and conjunction-based analyses based upon samples of at least 6 subjects, whereas only experiments comparing two or more samples…”
Method
Cited by
Modelling neural correlates of working memory: A coordinate-based meta-analysis
Rottschy, C., Langner, R., Dogan, I., Reetz, K., Laird, A. R., Schulz, J. B., Fox, P. T., & Eickhoff, S. B. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2011.11.050 · Cited by 951
Langner, R., & Eickhoff, S. B. (2013) · Psychological Bulletin · doi:10.1037/a0030694 · Cited by 746
Publication and other reporting biases in cognitive sciences: detection, prevalence, and prevention
Ioannidis, J. P. A., Munafò, M. R., Fusar-Poli, P., Nosek, B. A., & David, S. P. (2014) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2014.02.010 · Cited by 471
Neuroimaging of the periaqueductal gray: State of the field
Linnman, C., Moulton, E. A., Barmettler, G., Becerra, L., & Borsook, D. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2011.11.095 · Cited by 393
Berboth, S., & Morawetz, C. (2021) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2021.107767 · Cited by 254
Metadata
- Authors
- Jennings, R. G., & Van Horn, J. D.
- Venue
- Neuroinformatics
- Published
- 2011-06-04
- Volume
- 10
- Issue
- 1
- Pages
- 67-80
- ISSN
- 1539-2791 (ISSN-L)
- 1559-0089
- OpenAlex
- W2080937370
- Cited by
- 102
- References
- 31
- FWCI
- 4.05
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Publication Bias in Neuroimaging Research: Implications for Meta-Analyses. https://app.cochat.ai/papers/10.1007/s12021-011-9125-y
- The paper
- Jennings, R. G., & Van Horn, J. D. (2012). Publication Bias in Neuroimaging Research: Implications for Meta-Analyses. Neuroinformatics. https://doi.org/10.1007/s12021-011-9125-y
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.