Scanning the horizon: towards transparent and reproducible neuroimaging research
Open accessPublished version of a preprint
Abstract
Functional neuroimaging techniques have transformed our ability to probe the neurobiological basis of behaviour and are increasingly being applied by the wider neuroscience community. However, concerns have recently been raised that the conclusions that are drawn from some human neuroimaging studies are either spurious or not generalizable. Problems such as low statistical power, flexibility in data analysis, software errors and a lack of direct replication apply to many fields, but perhaps particularly to functional MRI. Here, we discuss these problems, outline current and suggested best practices, and describe how we think the field should evolve to produce the most meaningful and reliable answers to neuroscientific questions.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The study is limited by the lack of an independent sample by which to test these biomarkers' predictive utility.
“First, while the study implicates novel biomarkers of depression vulnerability, it is limited by the lack of an independent sample by which to test these biomarkers’ predictive utility (Poldrack et al., 2017).”
From Static and treatment-responsive brain biomarkers of depression relapse vulnerability following prophylactic psychotherapy: Evidence from a randomized control trial (2022) · cites this paper as Poldrack et al., 2017
Imprecise assessment can interfere with predictor measurement, such as behavioural markers with high variability or neuroimaging markers with poor reproducibility.
“Imprecise assessment can also interfere with predictor measurement, such as behavioural markers with high variability39 or neuroimaging markers with poor reproducibility.40”
From Precision clinical trials: a framework for getting to precision medicine for neurobehavioural disorders (2021) · cites this paper as [40]
Lack of reproducible findings across studies may be the result of low sample sizes, inflated false-positive rates due to analytic choices, and heterogeneity inherent to groups of interest.
“The neuroscience field is becoming increasingly aware of the lack of reproducible findings across studies, which may be the result of low sample sizes, inflated false-positive rates due to analytic choices, and heterogeneity inherent to groups of interest (Poldrack et al., 2017).”
From Investigating functional brain network integrity using a traditional and novel categorical scheme for neurodevelopmental disorders (2019) · cites this paper as Poldrack et al., 2017
Positive results reported from underpowered studies have a greater likelihood of being false.
“However, it is less appreciated that positive results reported from underpowered studies also have a greater likelihood of being false (Poldrack et al., 2017; Ioannidis, 2005).”
From Investigating functional brain network integrity using a traditional and novel categorical scheme for neurodevelopmental disorders (2019) · cites this paper as Poldrack et al., 2017
The exorbitant number of researcher degrees of freedom available in fMRI analyses lead to inflated false-positive rates.
“Further, the exorbitant number of researcher degrees of freedom available in fMRI analyses, including choice of preprocessing pipeline, analysis method, group matching procedures, thresholding procedures, and multiple comparison correction lead to inflated false-positive rates (Poldrack et al., 2017).”
From Investigating functional brain network integrity using a traditional and novel categorical scheme for neurodevelopmental disorders (2019) · cites this paper as Poldrack et al., 2017
The study was restricted to a smaller sample of participants than is generally recommended for clinical functional MRI research and for group comparisons in general functional MRI studies.
“Finally, we were restricted to a smaller sample of participants in the present study than is generally recommended for clinical functional MRI research (Carter et al., 2008) and for group comparisons in general functional MRI studies (Poldrack et al., 2017).”
From Distinct processing of ambiguous speech in people with non-clinical auditory verbal hallucinations (2017) · cites this paper as Poldrack et al., 2017
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Desikan, R. S., Ségonne, F., Fischl, B., Quinn, B. T., Dickerson, B. C., Blacker, D., Buckner, R. L., Dale, A. M., Maguire, R. P., Hyman, B. T., et al. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.01.021 · Cited by 13,324 · see citing works
Complex network measures of brain connectivity: Uses and interpretations
Rubinov, M., & Sporns, O. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2009.10.003 · Cited by 11,522 · see citing works
Jenkinson, M., Beckmann, C. F., Behrens, T. E. J., Woolrich, M. W., & Smith, S. M. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2011.09.015 · Cited by 11,314 · see citing works
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505 · see citing works
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515 · see citing works
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · see citing works
Self-citationSimmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567 · see citing works
The WU-Minn Human Connectome Project: An overview
Van Essen, D. C., Smith, S. M., Barch, D. M., Behrens, T. E. J., Yacoub, E., & Ugurbil, K. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.05.041 · Cited by 5,651 · see citing works
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · see citing works
Large-scale automated synthesis of human functional neuroimaging data.
Yarkoni, T., Poldrack, R. A., Nichols, T. E., Van Essen, D. C., & Wager, T. D. (2011) · Nature Methods · doi:10.1038/nmeth.1635 · Cited by 3,686 · see citing works
Self-citationWhat we can do and what we cannot do with fMRI
Logothetis, N. K. (2008) · Nature · doi:10.1038/nature06976 · Cited by 3,612 · see citing works
Cluster failure: why fMRI inferences for spatial extent have inflated false-positive rates
Eklund, A., Nichols, T. E., & Knutsson, H. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1602413113 · Cited by 3,487 · see citing works
Self-citationStatistical Parametric Mapping: The Analysis of Functional Brain Images
Penny, W. D., Friston, K. J., Ashburner, J., Kiebel, S. J., & Thomas, N. (2007) · UCL Discovery (University College London) · Cited by 3,339 · see citing works
Raise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979 · see citing works
Toward discovery science of human brain function
Biswal, B. B., Mennes, M., Zuo, X.-N., Gohel, S., Kelly, C., Smith, S. M., Beckmann, C. F., Adelstein, J. S., Buckner, R. L., Colcombe, S., et al. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0911855107 · Cited by 2,970 · see citing works
Promoting an open research culture
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., et al. (2015) · Science · doi:10.1126/science.aab2374 · Cited by 2,708 · see citing works
Self-citationBeyond mind-reading: multi-voxel pattern analysis of fMRI data
Norman, K. A., Polyn, S. M., Detre, G. J., & Haxby, J. V. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.07.005 · Cited by 2,271 · see citing works
Can cognitive processes be inferred from neuroimaging data?
POLDRACK, R. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2005.12.004 · Cited by 2,230 · see citing works
Self-citationHARKing: Hypothesizing after the results are known
Kerr, N. L. (1998) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0203_4 · Cited by 1,989 · see citing works
EPIC- InterAct Consortium, Burgess, S., Scott, R. A., Timpson, N. J., Davey Smith, G., & Thompson, S. G. (2015) · European Journal of Epidemiology · doi:10.1007/s10654-015-0011-z · Cited by 1,803 · see citing works
Gorgolewski, K. J., Auer, T., Calhoun, V. D., Craddock, R. C., Das, S., Duff, E. P., Flandin, G., Ghosh, S. S., Glatard, T., Halchenko, Y. O., et al. (2016) · Scientific Data · doi:10.1038/sdata.2016.44 · Cited by 1,775 · see citing works
Self-citationComparing Functional (PET) Images: The Assessment of Significant Change
Friston, K. J., Frith, C., Liddle, P. F., & Frackowiak, R. S. J. (1991) · Journal of Cerebral Blood Flow & Metabolism · doi:10.1038/jcbfm.1991.122 · Cited by 1,626 · see citing works
Function in the human connectome: task-fMRI and individual differences in behavior
Barch, D. M., Burgess, G. C., Harms, M. P., Petersen, S. E., Schlaggar, B. L., Corbetta, M., Glasser, M. F., Curtiss, S., Dixit, S., Feldt, C., et al. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.05.033 · Cited by 1,529 · see citing works
Self-citationType I and Type II error concerns in fMRI research: re-balancing the scale
Lieberman, M. D., & Cunningham, W. A. (2009) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsp052 · Cited by 1,342 · see citing works
Matching Categorical Object Representations in Inferior Temporal Cortex of Man and Monkey
Kriegeskorte, N., Mur, M., Ruff, D. A., Kiani, R., Bodurka, J., Esteky, H., Tanaka, K., & Bandettini, P. A. (2008) · Neuron · doi:10.1016/j.neuron.2008.10.043 · Cited by 1,334 · see citing works
The Difference Between “Significant” and “Not Significant” is not Itself Statistically Significant
Gelman, A., & Stern, H. (2006) · The American Statistician · doi:10.1198/000313006x152649 · Cited by 1,316 · see citing works
Policy: NIH plans to enhance reproducibility
Collins, F. S., & Tabak, L. A. (2014) · Nature · doi:10.1038/505612a · Cited by 1,247 · see citing works
Controlling the familywise error rate in functional neuroimaging: a comparative review
Nichols, T. E., & Hayasaka, S. (2003) · Statistical Methods in Medical Research · doi:10.1191/0962280203sm341ra · Cited by 1,210 · see citing works
Self-citationThe Statistical Crisis in Science
Gelman, A., & Loken, E. (2014) · American Scientist · doi:10.1511/2014.111.460 · Cited by 963 · see citing works
Erroneous analyses of interactions in neuroscience: A problem of significance
Nieuwenhuis, S., Forstmann, B. U., & Wagenmakers, E.-J. (2011) · Nature Neuroscience · doi:10.1038/nn.2886 · Cited by 864 · see citing works
The ENIGMA Consortium: large-scale collaborative analyses of neuroimaging and genetic data
the Alzheimer’s Disease Neuroimaging Initiative, E. C., Thompson, P. M., Stein, J. L., Medland, S. E., Hibar, D. P., Vasquez, A. A., Renteria, M. E., Toro, R., Jahanshad, N., Schumann, G., et al. (2014) · Brain Imaging and Behavior · doi:10.1007/s11682-013-9269-5 · Cited by 838 · see citing works
Self-citationInferring mental states from neuroimaging data: from reverse inference to large-scale decoding
Poldrack, R. A. (2011) · Neuron · doi:10.1016/j.neuron.2011.11.001 · Cited by 821 · see citing works
Self-citationBest practices in data analysis and sharing in neuroimaging using MRI
Nichols, T. E., Das, S., Eickhoff, S. B., Evans, A. C., Glatard, T., Hanke, M., Kriegeskorte, N., Milham, M. P., Poldrack, R. A., Poline, J.-B., et al. (2017) · Nature Neuroscience · doi:10.1038/nn.4500 · Cited by 714 · see citing works
Self-citationIdentification of common variants associated with human hippocampal and intracranial volumes
Stein, J., Medland, S., Vásquez, A., Hibar, D., Senstad, R. E., Winkler, A., Toro, R., Appel, K., Barteček, R., Bergmann, Ø., et al. (2012) · Nature Genetics · doi:10.1038/ng.2250 · Cited by 568 · see citing works
Self-citationGorgolewski, K. J., Varoquaux, G. P., Rivera, G., Schwarz, Y., Ghosh, S., Maumet, C., Sochat, V., Nichols, T. E., Poldrack, R. A., Poline, J., et al. (2015) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2015.00008 · Cited by 554 · see citing works
Self-citationGuidelines for reporting an fMRI study
Poldrack, R. A., Fletcher, P. C., Henson, R. N., Worsley, K. J., Brett, M., & Nichols, T. E. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2007.11.048 · Cited by 550 · see citing works
Self-citationYarkoni, T. (2009) · Perspectives on Psychological Science · doi:10.1111/j.1745-6924.2009.01127.x · Cited by 549 · see citing works
Self-citationMeta-research: Evaluation and Improvement of Research Methods and Practices
Ioannidis, J. P. A., Fanelli, D., Dunne, D. D., & Goodman, S. N. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002264 · Cited by 495 · see citing works
Validating cluster size inference: random field and permutation methods
Hayasaka, S. (2003) · NeuroImage · doi:10.1016/j.neuroimage.2003.08.003 · Cited by 491 · see citing works
The principled control of false positives in neuroimaging
Bennett, C. M., Wolford, G. L., & Miller, M. B. (2009) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsp053 · Cited by 486 · see citing works
Long-term neural and physiological phenotyping of a single human
Poldrack, R. A., Laumann, T. O., Koyejo, S., Gregory, B., Hover, A. M., Chen, M.-Y., Gorgolewski, K. J., Luci, J. J., Joo, S. J., Boyd, R. L., et al. (2015) · Nature Communications · doi:10.1038/ncomms9885 · Cited by 480 · see citing works
Self-citationThe secret lives of experiments: Methods reporting in the fMRI literature
Carp, J. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.07.004 · Cited by 473 · see citing works
An open science resource for establishing reliability and reproducibility in functional connectomics
Zuo, X., Anderson, J. S., Bellec, P., Birn, R. M., Biswal, B. B., Blautzik, J., Breitner, J. C. S., Buckner, R. L., Calhoun, V. D., Castellanos, F. X., et al. (2014) · Scientific Data · doi:10.1038/sdata.2014.49 · Cited by 441 · see citing works
Self-citationMeta-analysis of functional neuroimaging data: current and future directions
Wager, T. D., Lindquist, M., & Kaplan, L. (2007) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsm015 · Cited by 434 · see citing works
Bennett, C., Miller, M., & Wolford, G. (2009) · NeuroImage · doi:10.1016/s1053-8119(09)71202-9 · Cited by 400 · see citing works
The infeasibility of quantifying the reliability of life-critical real-time software
Butler, R. W., & Finelli, G. B. (1993) · IEEE Transactions on Software Engineering · doi:10.1109/32.210303 · Cited by 399 · see citing works
On the Plurality of (Methodological) Worlds: Estimating the Analytic Flexibility of fMRI Experiments
Carp, J. (2012) · Frontiers in Neuroscience · doi:10.3389/fnins.2012.00149 · Cited by 381 · see citing works
AFNI: What a long strange trip it's been
Cox, R. W. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.08.056 · Cited by 344 · see citing works
Toward open sharing of task-based fMRI data: the OpenfMRI project
Poldrack, R. A., Barch, D. M., Mitchell, J. P., Wager, T. D., Wagner, A. D., Devlin, J. T., Cumba, C., Koyejo, S., & Milham, M. P. (2013) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2013.00012 · Cited by 341 · see citing works
Self-citationThe False-positive to False-negative Ratio in Epidemiologic Studies
Ioannidis, J. P. A., Tarone, R., & McLaughlin, J. K. (2011) · Epidemiology · doi:10.1097/ede.0b013e31821b506e · Cited by 321 · see citing works
Making data sharing work: The FCP/INDI experience
Mennes, M., Biswal, B., Castellanos, F., & Milham, M. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.10.064 · Cited by 286 · see citing works
Progress and challenges in probing the human brain
Poldrack, R. A., & Farah, M. J. (2015) · Nature · doi:10.1038/nature15692 · Cited by 278 · see citing works
Self-citationMumford, J. A., & Nichols, T. E. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2007.07.061 · Cited by 252 · see citing works
Self-citationMu suppression – A good measure of the human mirror neuron system?
Hobson, H., & Bishop, D. (2016) · Cortex · doi:10.1016/j.cortex.2016.03.019 · Cited by 252 · see citing works
D. Chambers, C., 1 Cardiff University Brain Research Imaging Centre, S. O. P., Feredoes, E., D. Muthukumaraswamy, S., & J. Etchells, P. (2014) · AIMS Neuroscience · doi:10.3934/neuroscience.2014.1.4 · Cited by 213 · see citing works
Devlin, J. T., & Poldrack, R. A. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2006.09.055 · Cited by 195 · see citing works
Self-citationGil, Y., David, C. H., Demir, İ., Essawy, B. T., Fulweiler, R. W., Goodall, J. L., Karlstrom, L., Lee, H., Mills, H. J., Oh, J., et al. (2016) · Earth and Space Science · doi:10.1002/2015ea000136 · Cited by 182 · see citing works
A multivariate distance-based analytic framework for connectome-wide association studies
Shehzad, Z. E., Kelly, C. A. M., Reiss, P. T., Craddock, R. C., Emerson, J. W., McMahon, K. L., Copland, D. A., Castellanos, F. X., & Milham, M. P. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.02.024 · Cited by 178 · see citing works
Registered Reports: Realigning incentives in scientific publishing
Chambers, C., Dienes, Z. P., McIntosh, R. D., Rotshtein, P., & Willmes, K. (2015) · Cortex · doi:10.1016/j.cortex.2015.03.022 · Cited by 171 · see citing works
A purely confirmatory replication study of structural brain-behavior correlations
Boekel, W., Wagenmakers, E., Belay, L., Verhagen, J., Brown, S. D., & Forstmann, B. U. (2015) · Cortex · doi:10.1016/j.cortex.2014.11.019 · Cited by 167 · see citing works
Best Practices in Data Analysis and Sharing in Neuroimaging using MRIPreprint
Nichols, T. E., Das, S., Eickhoff, S. B., et al. (2016) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/054262 · Cited by 140 · see citing works
Self-citationWaskom, M., Kumaran, D., Gordon, A. M., Rissman, J., & Wagner, A. D. (2014) · Journal of Neuroscience · doi:10.1523/jneurosci.5282-13.2014 · Cited by 136 · see citing works
Hierarchical competitions subserving multi-attribute choice
Hunt, L. T., Dolan, R. J., & Behrens, T. E. J. (2014) · Nature Neuroscience · doi:10.1038/nn.3836 · Cited by 132 · see citing works
Potential Reporting Bias in fMRI Studies of the Brain
David, S. P., Ware, J. J., Chu, I. M., Loftus, P. D., Fusar-Poli, P., Radua, J., Munafò, M. R., & Ioannidis, J. P. A. (2013) · PLoS ONE · doi:10.1371/journal.pone.0070104 · Cited by 130 · see citing works
Self-referential forces are sufficient to explain different dendritic morphologies
Memelli, H., Torben-Nielsen, B., & Kozloski, J. (2013) · Front. Neuroinform. · doi:10.3389/fninf.2013.00001 · Cited by 127 · see citing works
Candidate and non-candidate genes in behavior genetics
Jonathan Flint, C., & Munafo, M. R. (2012) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.07.005 · Cited by 85 · see citing works
AFNI and Clustering: False Positive Rates ReduxPreprint
Cox, R. W., Reynolds, R. C., & Taylor, P. A. (2016) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/065862 · Cited by 81 · see citing works
Ten Simple Rules for Reducing Overoptimistic Reporting in Methodological Computational Research
Boulesteix, A. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004191 · Cited by 74 · see citing works
Power and sample size calculations for fMRI studies based on the prevalence of active peaksPreprint
Durnez, J., Degryse, J., Moerkerke, B., Seurinck, R., Sochat, V., Poldrack, R. A., & Nichols, T. E. (2016) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/049429 · Cited by 69 · see citing works
Self-citationTong, Y., Chen, Q., Nichols, T. E., Rasetti, R., Callicott, J. H., Berman, K. F., Weinberger, D. R., & Mattay, V. S. (2016) · PLoS ONE · doi:10.1371/journal.pone.0151391 · Cited by 55 · see citing works
Self-citationChurchill, N. W., Yourganov, G., Oder, A., Tam, F., Graham, S. J., & Strother, S. C. (2012) · PLoS ONE · doi:10.1371/journal.pone.0031147 · Cited by 52 · see citing works
Fast Bayesian whole-brain fMRI analysis with spatial 3D priors
Sidén, P., Eklund, A., Bolin, D., et al. (2016) · NeuroImage · doi:10.1016/j.neuroimage.2016.11.040 · Cited by 42 · see citing works
Distribution of the height of local maxima of Gaussian random fields
Cheng, D., & Schwartzman, A. (2014) · Extremes · doi:10.1007/s10687-014-0211-z · Cited by 36 · see citing works
Why shared data should not be acknowledged on the author byline
Torsten Rohlfing, B., & Poline, J.-B. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.09.080 · Cited by 34 · see citing works
Self-citationPredicting intrinsic brain activity
Craddock, R. C., Milham, M. P., & LaConte, S. M. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.05.072 · Cited by 33 · see citing works
Austin, M. A., Hair, M. S., & Fullerton, S. M. (2012) · American Journal of Epidemiology · doi:10.1093/aje/kwr441 · Cited by 31 · see citing works
Guo, Q., Parlar, M. E., Truong, W., Hall, G. B., Thabane, L., McKinnon, M. C., Goeree, R. A., & Pullenayegum, E. (2014) · PLoS ONE · doi:10.1371/journal.pone.0094412 · Cited by 30 · see citing works
Heininga, V. E., Oldehinkel, A. J., Veenstra, R., & Nederhof, E. (2015) · PLoS ONE · doi:10.1371/journal.pone.0125383 · Cited by 23 · see citing works
Using small numbers of subjects in fMRI-based research
Savoy, R. L. (2006) · IEEE Engineering in Medicine and Biology Magazine · doi:10.1109/memb.2006.1607669 · Cited by 15 · see citing works
Muthukumaraswamy, S. D., Routley, B. C., Droog, W., Singh, K. D., & Hamandi, K. (2016) · Cortex · doi:10.1016/j.cortex.2016.03.004 · Cited by 12 · see citing works
Worsley (1996) · doi:10.1002/(sici)1097-0193(1996)4:1
Flagged first, then by citations
Cited by
Sorted by citations
Yan, C.-G., Chen, X., Li, L., Castellanos, F. X., Bai, T.-J., Bo, Q.-J., Cao, J., Chen, G.-M., Chen, N.-X., Chen, W., et al. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1900390116 · Cited by 895 · see citing works
Cross-validation failure: Small sample sizes lead to large error bars
Varoquaux, G. (2018) · NeuroImage · doi:10.1016/j.neuroimage.2017.06.061 · Cited by 897 · see citing works
“Overall, exploring larger datasets is a promising future for neuroimaging (Poldrack et al., 2017).”
BackgroundWicherts, J. M., Veldkamp, C. L. S., Augusteijn, H. E. M., Bakker, M., van Aert, R. C. M., & van Assen, M. A. L. M. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.01832 · Cited by 754 · see citing works
Wen, J., Thibeau-Sutre, E., Diaz-Melo, M., Samper-González, J., Routier, A., Bottani, S., Dormont, D., Durrleman, S., Burgos, N., & Colliot, O. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101694 · Cited by 809 · see citing works
Dergaa, I., Chamari, K., Zmijewski, P., & Ben Saad, H. (2023) · Biology of Sport · doi:10.5114/biolsport.2023.125623 · Cited by 796 · see citing works
Elliott, M. L., Knodt, A. R., Ireland, D., Morris, M. L., Poulton, R., Ramrakha, S., Sison, M. L., Moffitt, T. E., Caspi, A., & Hariri, A. R. (2020) · Psychological Science · doi:10.1177/0956797620916786 · Cited by 714 · see citing works
Toward a universal decoder of linguistic meaning from brain activation
Pereira, F., Lou, B., Pritchett, B., Ritter, S., Gershman, S. J., Kanwisher, N., Botvinick, M., & Fedorenko, E. (2018) · Nature Communications · doi:10.1038/s41467-018-03068-4 · Cited by 324 · see citing works
Chen, X., Lu, B., & Yan, C. (2018) · Human Brain Mapping · doi:10.1002/hbm.23843 · Cited by 340 · see citing works
Sui, J., Jiang, R., Bustillo, J., & Calhoun, V. (2020) · Biological Psychiatry · doi:10.1016/j.biopsych.2020.02.016 · Cited by 309 · see citing works
Ng, T. H., Alloy, L. B., & Smith, D. V. (2019) · Translational Psychiatry · doi:10.1038/s41398-019-0644-x · Cited by 244 · see citing works
Stomach-brain synchrony reveals a novel, delayed-connectivity resting-state network in humans
Rebollo, I., Devauchelle, A.-D., Béranger, B., & Tallon-Baudry, C. (2018) · eLife · doi:10.7554/elife.33321 · Cited by 194 · see citing works
Data leakage inflates prediction performance in connectome-based machine learning models
Rosenblatt, M., Tejavibulya, L., Jiang, R., Noble, S., & Scheinost, D. (2024) · Nature Communications · doi:10.1038/s41467-024-46150-w · Cited by 236 · see citing works
Lack of selectivity for syntax relative to word meanings throughout the language network
Fedorenko, E., Blank, I. A., Siegelman, M., & Mineroff, Z. (2020) · Cognition · doi:10.1016/j.cognition.2020.104348 · Cited by 170 · see citing works
Cluster success: fMRI inferences for spatial extent have acceptable false-positive rates.
Slotnick, S. D. (2017) · Cognitive Neuroscience · doi:10.1080/17588928.2017.1319350 · Cited by 184 · see citing works
A distributed fMRI-based signature for the subjective experience of fear
Zhou, F., Zhao, W., Qi, Z., Geng, Y., Yao, S., Kendrick, K. M., Wager, T. D., & Becker, B. (2021) · Nature Communications · doi:10.1038/s41467-021-26977-3 · Cited by 155 · see citing works
Assessing inter individual differences with task related functional neuroimaging
Lebreton, M., Bavard, S., Daunizeau, J., & Palminteri, S. (2019) · Nature Human Behaviour · doi:10.1038/s41562-019-0681-8 · Cited by 160 · see citing works
Are specific learning disorders truly specific, and are they disorders?
Peters, L., & Ansari, D. (2019) · Trends in Neuroscience and Education · doi:10.1016/j.tine.2019.100115 · Cited by 151 · see citing works
Gee, D. G. (2021) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2021.21090944 · Cited by 148 · see citing works
Reproducible and replicable pain research: a critical review
Lee, H., Lamb, S. E., Bagg, M. K., Toomey, E., Cashin, A. G., & Moseley, G. L. (2018) · Pain · doi:10.1097/j.pain.0000000000001254 · Cited by 150 · see citing works
The role of the opioid system in decision making and cognitive control: a review
van Steenbergen, H., Eikemo, M., & Leknes, S. (2019) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-019-00710-6 · Cited by 118 · see citing works
Artificial intelligence for diagnostic and prognostic neuroimaging in dementia: a systematic review
Borchert, R. J., Azevedo, T., Badhwar, A., Bernal, J., Betts, M., Bruffaerts, R., Burkhart, M. C., Dewachter, I., Gellersen, H. M., Low, A., et al. (2023) · Alzheimer's & Dementia · doi:10.1002/alz.13412 · Cited by 125 · see citing works
Grace, S. A., Rossell, S. L., Heinrichs, M., Kordsachia, C., & Labuschagne, I. (2018) · Psychoneuroendocrinology · doi:10.1016/j.psyneuen.2018.05.031 · Cited by 120 · see citing works
Martins, D., Rademacher, L., Gabay, A. S., Taylor, R., Richey, J. A., Smith, D. V., Goerlich, K. S., Nawijn, L., Cremers, H. R., Wilson, R., et al. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.12.034 · Cited by 119 · see citing works
Interoception primes emotional processing: multimodal evidence from neurodegeneration
Salamone, P. C., Legaz, A., Sedeño, L., Moguilner, S., Fraile-Vazquez, M., Campo, C. G., Fittipaldi, S., Yoris, A., Miranda, M., Birba, A., et al. (2021) · The Journal of Neuroscience · doi:10.1523/jneurosci.2578-20.2021 · Cited by 103 · see citing works
Distinct processing of ambiguous speech in people with non-clinical auditory verbal hallucinations
Alderson-Day, B., Lima, C. F., Evans, S., Krishnan, S., Shanmugalingam, P., Fernyhough, C., & Scott, S. K. (2017) · Brain · doi:10.1093/brain/awx206 · Cited by 108 · see citing works
Conwell, C., Prince, J. S., Kay, K. N., Alvarez, G. A., & Konkle, T. (2024) · Nature Communications · doi:10.1038/s41467-024-53147-y · Cited by 100 · see citing works
Progress toward openness, transparency, and reproducibility in cognitive neuroscience
Gilmore, R. O., Diaz, M. T., Wyble, B. A., & Yarkoni, T. (2017) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.13325 · Cited by 95 · see citing works
Self-citationA multidimensional and multi-feature framework for cardiac interoception
Fittipaldi, S., Abrevaya, S., Fuente, A. D. L., Pascariello, G. O., Hesse, E., Birba, A., Salamone, P., Hildebrandt, M., Martí, S. A., Pautassi, R. M., et al. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.116677 · Cited by 90 · see citing works
van der Velden, A. M., Scholl, J., Elmholdt, E.-M., Fjorback, L. O., Harmer, C. J., Lazar, S. W., O’Toole, M. S., Smallwood, J., Roepstorff, A., & Kuyken, W. (2023) · Biological Psychiatry · doi:10.1016/j.biopsych.2022.06.038 · Cited by 83 · see citing works
Neuroimaging in Parkinson’s disease dementia: connecting the dots
Weil, R. S., Hsu, J. K., Darby, R. R., Soussand, L., & Fox, M. D. (2019) · Brain Communications · doi:10.1093/braincomms/fcz006 · Cited by 71 · see citing works
Allostatic interoceptive overload in frontotemporal dementia
Birba, A., Santamaría-García, H., Prado, P., Cruzat, J., Ballesteros, A. S., Legaz, A., Fittipaldi, S., Duran-Aniotz, C., Slachevsky, A., Santibañez, R., et al. (2022) · Biological Psychiatry · doi:10.1016/j.biopsych.2022.02.955 · Cited by 70 · see citing works
Yoris, A., Abrevaya, S., Esteves, S., Salamone, P., Lori, N., Martorell, M., Legaz, A., Alifano, F., Petroni, A., Sánchez, R., et al. (2018) · Human Brain Mapping · doi:10.1002/hbm.23933 · Cited by 66 · see citing works
Moguilner, S., García, A. M., Perl, Y. S., Tagliazucchi, E., Piguet, O., Kumfor, F., Reyes, P., Matallana, D., Sedeño, L., & Ibáñez, A. (2021) · NeuroImage · doi:10.1016/j.neuroimage.2020.117522 · Cited by 63 · see citing works
Dysregulated resting state functional connectivity and obesity: A systematic review
Syan, S. K., McIntyre-Wood, C., Minuzzi, L., Hall, G., McCabe, R. E., & MacKillop, J. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2021.08.019 · Cited by 66 · see citing works
Schaefer, J. D., Hamdi, N. R., Malone, S. M., Vrieze, S., Wilson, S., McGue, M., & Iacono, W. G. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2013180118 · Cited by 64 · see citing works
Odriozola, P., & Gee, D. G. (2021) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2020.20020232 · Cited by 62 · see citing works
Hosseini, E. A., Schrimpf, M., Zhang, Y., Bowman, S., Zaslavsky, N., & Fedorenko, E. (2024) · Neurobiology of Language · doi:10.1162/nol_a_00137 · Cited by 63 · see citing works
MacQueen, G. M., Hassel, S., Arnott, S. R., Addington, J., Bowie, C. R., Bray, S. L., Davis, A. D., Downar, J., Foster, J. A., Frey, B. N., et al. (2019) · Journal of Psychiatry and Neuroscience · doi:10.1503/jpn.180036 · Cited by 57 · see citing works
Temporal irreversibility of large-scale brain dynamics in Alzheimer's disease
Cruzat, J., Herzog, R., Prado, P., Sanz-Perl, Y., Gonzalez-Gomez, R., Moguilner, S., Kringelbach, M. L., Deco, G., Tagliazucchi, E., & Ibañez, A. (2023) · The Journal of Neuroscience · doi:10.1523/jneurosci.1312-22.2022 · Cited by 57 · see citing works
Adhikari, B. M., Jahanshad, N., Shukla, D., Turner, J., Grotegerd, D., Dannlowski, U., Kugel, H., Engelen, J., Dietsche, B., Krug, A., et al. (2019) · Brain Imaging and Behavior · doi:10.1007/s11682-018-9941-x · Cited by 53 · see citing works
Puberty and functional brain development in humans: convergence in findings?
Dai, J., & Scherf, K. S. (2019) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2019.100690 · Cited by 51 · see citing works
Lenze, E. J., Nicol, G. E., Barbour, D. L., Kannampallil, T., Wong, A. W. K., Piccirillo, J., Drysdale, A. T., Sylvester, C. M., Haddad, R., Miller, J. P., et al. (2021) · Journal of Psychiatry and Neuroscience · doi:10.1503/jpn.200042 · Cited by 43 · see citing works
Private–public mappings in human prefrontal cortex
Bang, D., Ershadmanesh, S., Nili, H., & Fleming, S. M. (2020) · eLife · doi:10.7554/elife.56477 · Cited by 40 · see citing works
Abrevaya, S., Fittipaldi, S., García, A. M., Dottori, M., Santamaria-Garcia, H., Birba, A., Yoris, A., Hildebrandt, M. K., Salamone, P., De la Fuente, A., et al. (2020) · Psychosomatic Medicine · doi:10.1097/psy.0000000000000868 · Cited by 34 · see citing works
Multimodal mechanisms of human socially reinforced learning across neurodegenerative diseases
Legaz, A., Abrevaya, S., Dottori, M., González Campo, C., Birba, A., Martorell Caro, M., Aguirre, J., Slachevsky, A., Aranguiz, R., Serrano, C., et al. (2022) · Brain · doi:10.1093/brain/awab345 · Cited by 34 · see citing works
Dajani, D. R., Burrows, C. A., Odriozola, P., Baez, A., Nebel, M. B., Mostofsky, S. H., & Uddin, L. Q. (2019) · NeuroImage: Clinical · doi:10.1016/j.nicl.2019.101678 · Cited by 33 · see citing works
“The neuroscience field is becoming increasingly aware of the lack of reproducible findings across studies, which may be the result of low sample sizes, inflated falsepositive rates due to analytic choices, and heterogeneity inherent to groups of interest (Poldrack et al., 2017).”
Müller, F., Liechti, M. E., Lang, U. E., & Borgwardt, S. (2018) · Progress in Brain Research · doi:10.1016/bs.pbr.2018.08.004 · Cited by 31 · see citing works
Funch Uhre, V., Melissa Larsen, K., Marc Herz, D., Baaré, W., Katrine Pagsberg, A., & Roman Siebner, H. (2022) · NeuroImage: Clinical · doi:10.1016/j.nicl.2022.103268 · Cited by 18 · see citing works
Farb, N. A. S., Desormeau, P., Anderson, A. K., & Segal, Z. V. (2022) · NeuroImage: Clinical · doi:10.1016/j.nicl.2022.102969 · Cited by 16 · see citing works
“biomarkers of depression vulnerability, it is limited by the lack of an independent sample by which to test these biomarkers’ predictive utility (68).”
Hazelton, J. L., Della Bella, G., Barttfeld, P., Dottori, M., Gonzalez-Gomez, R., Migeot, J., Moguilner, S., Legaz, A., Hernandez, H., Prado, P., et al. (2025) · eBioMedicine · doi:10.1016/j.ebiom.2025.105614 · Cited by 14 · see citing works
Standardized Effect Sizes and Image-Based Meta-Analytical Approaches for fMRI Data
Bossier, H., Nichols, T. E., & Moerkerke, B. (2019) · doi:10.1101/865881 · Cited by 14 · see citing works
Self-citationThe Cognitive Neuroscience of Psychological Treatment Action in Depression and Anxiety
Young, K. S., & Craske, M. G. (2018) · Current Behavioral Neuroscience Reports · doi:10.1007/s40473-018-0137-x · Cited by 13 · see citing works
- Authors
- Poldrack, R. A., Baker, C. I., Durnez, J., Gorgolewski, K. J., Matthews, P. M., Munafò, M. R., Nichols, T. E., Poline, J.-B., Vul, E., & Yarkoni, T.
- Affiliations
- Stanford University (US)
- National Institutes of Health (US)
- Institut national de recherche en sciences et technologies du numérique (FR)
- Hammersmith Hospital (GB)
- University of Warwick (GB)
- University of California, Berkeley (US)
- University of California San Diego (US)
- The University of Texas at Austin (US)
- Corresponding author
- Poldrack, Russell A.
- ORCID
- Venue
- Nature Reviews Neuroscience
- Published
- 2017-01-05
- Volume
- 18
- Issue
- 2
- Pages
- 115-126
- ISSN
- 1471-003X (ISSN-L)
- 1471-0048
- PMC
- PMC6910649
- OpenAlex
- W2571093196
- Cited by
- 1,410
- References
- 81
- Citations by year
- 2026: 123 · 2025: 117 · 2024: 188 · 2023: 141 · 2022: 157 · 2021: 159 · 2020: 141 · 2019: 172 · 2018: 125 · 2017: 81 · 2016: 4
- FWCI
- 15.87
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature reviews. Neuroscience (publishedVersion)
- PubMed (publishedVersion)
- Warwick Research Archive Portal (University of Warwick) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Bristol Research (University of Bristol) (acceptedVersion, other-oa)
- HAL (submittedVersion)
- Explore Bristol Research (submittedVersion, other-oa)
- Spiral (Imperial College London) (submittedVersion)
- Funders
- Laura and John Arnold Foundation
- Wellcome Trust (1U54MH091657, 100309/Z/12/Z., 100309/A/12/Z)
- United Kingdom Clinical Research Collaboration
- Cancer Research UK
- National Institute for Health and Care Research (NF-SI-0514-10022)
- British Heart Foundation
- Imperial College London
- Imperial College Healthcare NHS Trust
- National Institutes of Health (5u54mh091657-05, 100309/Z/12/Z, 1U54‐MH‐091657, EB019936)
- McDonnell Center for Systems Neuroscience (1U54MH091657)
- Medical Research Council (MR/K023195/1, 12013/6, 1U54MH091657, MR/M024903/1, MC_UU_12013/6, MR/N026934/1, MC_PC_17114)
- Engineering and Physical Sciences Research Council (1U54MH091657, 100309/Z/12/Z)
- Economic and Social Research Council (ES/G007489/1)
- National Institute of Mental Health (1U54 MH091657, U54-MH091657, ZIAMH002909, R01MH096906)
- National Institute on Drug Abuse (1U54MH091657)
- National Institute of Neurological Disorders and Stroke (R01NS075066)
- National Institute of Biomedical Imaging and Bioengineering (1U54MH091657, R01EB015611, EB019936, P41EB019936)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Functional Brain Connectivity Studies
- Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Advanced MRI Techniques and Applications
- Keywords
- reproducibility, transparency, functional MRI, statistical power, software errors, direct replication, research reliability, best practices, neuroimaging research
- MeSH terms
- Humans, Magnetic Resonance Imaging, Software, Statistics as Topic, Reproducibility of Results, Practice Guidelines as Topic, Functional Neuroimaging
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
