The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.
The secret lives of experiments: Methods reporting in the fMRI literature
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Statistical Power Analysis for the Behavioral Sciences
Muller, K. (1989) · Technometrics · doi:10.1080/00401706.1989.10488618 · Cited by 99,167
Statistical power analysis for the behavioral sciences
(1990) · Computers, Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,269
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040296 · Cited by 23,020
Tzourio-Mazoyer, N., Landeau, B., Papathanassiou, D., Crivello, F., Etard, O., Delcroix, N., Mazoyer, B., & Joliot, M. (2002) · NeuroImage · doi:10.1006/nimg.2001.0978 · Cited by 16,319 · cited 1×
“Among these studies, 50.0% reported using the Talairach atlas (Talairach and Tournoux, 1988), 11.5% used the Automatic Anatomical Labeling toolbox of Tzourio-Mazoyer et al. (2002), and 11.5% used the Duvernoy atlas (Duvernoy et al., 1999).”
InfluentialCo-Planar Stereotaxic Atlas of the Human Brain
Talairach, J., & Tournoux, P. (1988) · Medical Entomology and Zoology · Cited by 11,798
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Co-planar stereotaxic atlas of the human brain
Laitinen, L. (1989) · Clinical Neurology and Neurosurgery · doi:10.1016/0303-8467(89)90128-5 · Cited by 7,565
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · The Lancet · doi:10.1016/s0140-6736(00)04337-3 · Cited by 5,373
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 7×
“Further, analysis strategies vary widely across studies, suggesting the potential for bias (Ioannidis, 2005b).”
Influential
Research works printed but not linked: 1
- Dale, 1999, Optimal experimental design for event-related fMRI, Hum. Brain Mapp., 8, 109, 10.1002/(SICI)1097-0193(1999)8:2/3<109::AID-HBM7>3.0.CO;2-W · doi:10.1002/(sici)1097-0193(1999)8:2/3<109::aid-hbm7>3.0.co;2-w
Cited by
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641
fMRIPrep: A robust preprocessing pipeline for functional MRI
Esteban, O., Markiewicz, C. J., Blair, R. W., Moodie, C. A., Isik, A. I., Erramuzpe, A., Kent, J. D., Goncalves, M., DuPre, E., Snyder, M., et al. (2019) · Nature Methods · doi:10.1038/s41592-018-0235-4 · Cited by 3,905
MEG and EEG data analysis with MNE-Python
Gramfort, A. (2013) · Frontiers in Neuroscience · doi:10.3389/fnins.2013.00267 · Cited by 3,784
Cluster failure: why fMRI inferences for spatial extent have inflated false-positive rates
Eklund, A., Nichols, T. E., & Knutsson, H. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1602413113 · Cited by 3,487
“The tested parameter combinations, given in Table 1, are common in the fMRI field according to a recent review (26).”
InfluentialA manifesto for reproducible science
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397
Cognitive Reappraisal of Emotion: A Meta-Analysis of Human Neuroimaging Studies
Buhle, J. T., Silvers, J. A., Wager, T. D., Lopez, R., Onyemekwu, C., Kober, H., Weber, J., & Ochsner, K. N. (2014) · Cerebral Cortex · doi:10.1093/cercor/bht154 · Cited by 1,896
Scanning the horizon: towards transparent and reproducible neuroimaging research
Poldrack, R. A., Baker, C. I., Durnez, J., Gorgolewski, K. J., Matthews, P. M., Munafò, M. R., Nichols, T. E., Poline, J.-B., Vul, E., & Yarkoni, T. (2017) · Nature Reviews Neuroscience · doi:10.1038/nrn.2016.167 · Cited by 1,410
Reducing waste from incomplete or unusable reports of biomedical research
Glasziou, P., Altman, D. G., Bossuyt, P., Boutron, I., Clarke, M., Julious, S., Michie, S., Moher, D., & Wager, E. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62228-x · Cited by 1,319
Cluster-extent based thresholding in fMRI analyses: Pitfalls and recommendations
Woo, C.-W., Krishnan, A., & Wager, T. D. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2013.12.058 · Cited by 1,253
“1A; Friston et al., 1994; also see Carp, 2012).”
BackgroundPolicy: NIH plans to enhance reproducibility
Collins, F. S., & Tabak, L. A. (2014) · Nature · doi:10.1038/505612a · Cited by 1,247
Cross-validation failure: Small sample sizes lead to large error bars
Varoquaux, G. (2018) · NeuroImage · doi:10.1016/j.neuroimage.2017.06.061 · Cited by 897
“However, the success of meta-analysis in fMRI suggests that pooling data can be beneficial, whether it is by assembling a small number of well-matched studies or over a wider coverage of the literature (Laird et al., 2005; Costafreda, 2009).”
BackgroundHallquist, M. N., Hwang, K., & Luna, B. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.05.116 · Cited by 753
Many Analysts, One Data Set: Making Transparent How Variations in Analytic Choices Affect Results
Silberzahn, R., Uhlmann, E. L., Martin, D. P., Anselmi, P., Aust, F., Awtrey, E., Bahník, Š., Bai, F., Bannard, C., Bonnier, E., et al. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245917747646 · Cited by 784
Szucs, D., & Ioannidis, J. P. A. (2017) · PLOS Biology · doi:10.1371/journal.pbio.2000797 · Cited by 761
Eickhoff, S. B., Nichols, T. E., Laird, A. R., Hoffstaedter, F., Amunts, K., Fox, P. T., Bzdok, D., & Eickhoff, C. R. (2016) · NeuroImage · doi:10.1016/j.neuroimage.2016.04.072 · Cited by 683
“Furthermore, the substantial analytic flexibility of neuroimaging pipelines also leads to low reliability and reproducibility of neuroimaging findings (Carp, 2012b; Glatard et al., 2015; Wager et al., 2009).”
Best practices in data analysis and sharing in neuroimaging using MRI
Nichols, T. E., Das, S., Eickhoff, S. B., Evans, A. C., Glatard, T., Hanke, M., Kriegeskorte, N., Milham, M. P., Poldrack, R. A., Poline, J.-B., et al. (2017) · Nature Neuroscience · doi:10.1038/nn.4500 · Cited by 714
Deriving reproducible biomarkers from multi-site resting-state data: An Autism-based example
Abraham, A., Milham, M. P., Di Martino, A., Craddock, R. C., Samaras, D., Thirion, B., & Varoquaux, G. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2016.10.045 · Cited by 701
Best practice guidance for linear mixed-effects models in psychological science
Meteyard, L., & Davies, R. A. I. (2020) · Journal of Memory and Language · doi:10.1016/j.jml.2020.104092 · Cited by 698
Donoho, D. (2017) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.2017.1384734 · Cited by 695
Methods for cleaning the BOLD fMRI signal
Caballero-Gaudes, C., & Reynolds, R. C. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2016.12.018 · Cited by 542
Ventromedial Prefrontal Cortex Is Critical for the Regulation of Amygdala Activity in Humans
Motzkin, J. C., Philippi, C. L., Wolf, R. C., Baskaya, M. K., & Koenigs, M. (2015) · Biological Psychiatry · doi:10.1016/j.biopsych.2014.02.014 · Cited by 454
Publication and other reporting biases in cognitive sciences: detection, prevalence, and prevention
Ioannidis, J. P. A., Munafò, M. R., Fusar-Poli, P., Nosek, B. A., & David, S. P. (2014) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2014.02.010 · Cited by 471
Szucs, D., & Ioannidis, J. P. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.117164 · Cited by 453
Pre-registration in social psychology—A discussion and suggested template
van 't Veer, A. E., & Giner-Sorolla, R. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2016.03.004 · Cited by 439
Deficient approaches to human neuroimaging
Stelzer, J., Lohmann, G., Mueller, K., Buschmann, T., & Turner, R. (2014) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2014.00462 · Cited by 412
Evaluation of a decided sample size in machine learning applications
Rajput, D., Wang, W.-J., & Chen, C.-C. (2023) · BMC Bioinformatics · doi:10.1186/s12859-023-05156-9 · Cited by 411
“The results of small sample studies are particularly vulnerable to minor analytical manipulations that produce false-negative results [5, 6].”
BackgroundHow Bayes factors change scientific practice
Dienes, Z. (2016) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2015.10.003 · Cited by 401
Small sample sizes reduce the replicability of task-based fMRI studies
Turner, B. O., Paul, E. J., Miller, M. B., & Barbey, A. K. (2018) · Communications Biology · doi:10.1038/s42003-018-0073-z · Cited by 387
On the Plurality of (Methodological) Worlds: Estimating the Analytic Flexibility of fMRI Experiments
Carp, J. (2012) · Frontiers in Neuroscience · doi:10.3389/fnins.2012.00149 · Cited by 381
Self-citationITK: enabling reproducible research and open science
McCormick, M., Liu, X., Jomier, J., Marion, C., & Ibanez, L. (2014) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2014.00013 · Cited by 376
Eliot, L., Ahmed, A., Khan, H., & Patel, J. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2021.02.026 · Cited by 360
Chen, X., Lu, B., & Yan, C. (2018) · Human Brain Mapping · doi:10.1002/hbm.23843 · Cited by 340
The impact of traditional neuroimaging methods on the spatial localization of cortical areas
Coalson, T. S., Van Essen, D. C., & Glasser, M. F. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1801582115 · Cited by 306
Interpreting and Utilising Intersubject Variability in Brain Function
Seghier, M. L., & Price, C. J. (2018) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.03.003 · Cited by 291
When Null Hypothesis Significance Testing Is Unsuitable for Research: A Reassessment
Szucs, D., & Ioannidis, J. P. A. (2017) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2017.00390 · Cited by 298
A Hitchhiker's Guide to Functional Magnetic Resonance Imaging
Soares, J. M., Magalhães, R., Moreira, P. S., Sousa, A., Ganz, E., Sampaio, A., Alves, V., Marques, P., & Sousa, N. (2016) · Frontiers in Neuroscience · doi:10.3389/fnins.2016.00515 · Cited by 272
fMRI clustering and false-positive rates
Cox, R. W., Chen, G., Glen, D. R., Reynolds, R. C., & Taylor, P. A. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1614961114 · Cited by 219
“1 of [1] show quite good cluster results for stricter per-voxel p-values (which [6] found to be predominantly used in FMRI analyses) and for event-related stimuli (emphasizing the importance of good experimental design): FPR inflation was often .”
MethodDean, R. T., & Dunsmuir, W. T. M. (2016) · Behavior Research Methods · doi:10.3758/s13428-015-0611-2 · Cited by 215
“This may also be one component of the reasons for the unreliability of results in fields such as neuroimaging (Button et al., 2013; Carp, 2012), which we will not consider directly here.”
ResultTAPAS: An Open-Source Software Package for Translational Neuromodeling and Computational Psychiatry
Frässle, S., Aponte, E. A., Bollmann, S., Brodersen, K. H., Do, C. T., Harrison, O. K., Harrison, S. J., Heinzle, J., Iglesias, S., Kasper, L., et al. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.680811 · Cited by 169
Acar, F., Seurinck, R., Eickhoff, S. B., & Moerkerke, B. (2018) · PLOS ONE · doi:10.1371/journal.pone.0208177 · Cited by 154
“fMRI studies exhibit great heterogeneity, stemming from the way experiments are constructed, carried out, analyzed and reported [2].”
BackgroundThe neural correlates of visual imagery: A co-ordinate-based meta-analysis
Winlove, C. I. P., Milton, F., Ranson, J., Fulford, J., MacKisack, M., Macpherson, F., & Zeman, A. (2018) · Cortex · doi:10.1016/j.cortex.2017.12.014 · Cited by 150
Personality Neuroscience and the Five Factor Model
Allen, T. A., & DeYoung, C. G. (2016) · Oxford Handbooks Online · doi:10.1093/oxfordhb/9780199352487.013.26 · Cited by 143
Measuring neural representations with fMRI: practices and pitfalls
Davis, T., & Poldrack, R. A. (2013) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.12156 · Cited by 132
Neuroscience and Sex/Gender: Looking Back and Forward
Hines, M. (2020) · The Journal of Neuroscience · doi:10.1523/jneurosci.0750-19.2019 · Cited by 117
Using fMRI to study reward processing in humans: past, present, and future
Wang, K. S., Smith, D. V., & Delgado, M. R. (2016) · Journal of Neurophysiology · doi:10.1152/jn.00333.2015 · Cited by 105
McNaughton, N., DeYoung, C. G., & Corr, P. J. (2016) · Neuroimaging Personality, Social Cognition, and Character · doi:10.1016/b978-0-12-800935-2.00002-6 · Cited by 123
Philippi, C. L., Motzkin, J. C., Pujara, M. S., & Koenigs, M. (2015) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2015.10.005 · Cited by 93
Calibrating the Scientific Ecosystem Through Meta-Research
Hardwicke, T. E., Serghiou, S., Janiaud, P., Danchev, V., Crüwell, S., Goodman, S. N., & Ioannidis, J. P. A. (2020) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-031219-041104 · Cited by 98
Progress toward openness, transparency, and reproducibility in cognitive neuroscience
Gilmore, R. O., Diaz, M. T., Wyble, B. A., & Yarkoni, T. (2017) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.13325 · Cited by 95
Eklund, A., Knutsson, H., & Nichols, T. E. (2019) · Human Brain Mapping · doi:10.1002/hbm.24350 · Cited by 82
Reliable individual differences in fine-grained cortical functional architecture
Feilong, M., Nastase, S. A., Guntupalli, J. S., & Haxby, J. V. (2018) · NeuroImage · doi:10.1016/j.neuroimage.2018.08.029 · Cited by 68
Barch, D. M., & Yarkoni, T. (2013) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-013-0201-7 · Cited by 67
Neural and behavioral correlates of negative self‐focused thought associated with depression
Philippi, C. L., Cornejo, M. D., Frost, C. P., Walsh, E. C., Hoks, R. M., Birn, R., & Abercrombie, H. C. (2018) · Human Brain Mapping · doi:10.1002/hbm.24003 · Cited by 60
Ventromedial Prefrontal Cortex Lesions Alter Neural and Physiological Correlates of Anticipation
Motzkin, J. C., Philippi, C. L., Wolf, R. C., Baskaya, M. K., & Koenigs, M. (2014) · Journal of Neuroscience · doi:10.1523/jneurosci.1446-14.2014 · Cited by 44
“Allstatisticalmapswerefamilywiseerror(FWE)cor-rectedformultiplecomparisonsacrossthewholebrainattheclusterlevel ( P FWE (cid:7) 0.05),usingaheightthresholdof p (cid:7) 0.001(Formanetal.,1995; Carp, 2012).”
InfluentialFunch Uhre, V., Melissa Larsen, K., Marc Herz, D., Baaré, W., Katrine Pagsberg, A., & Roman Siebner, H. (2022) · NeuroImage: Clinical · doi:10.1016/j.nicl.2022.103268 · Cited by 18
Standardized Effect Sizes and Image-Based Meta-Analytical Approaches for fMRI Data
Bossier, H., Nichols, T. E., & Moerkerke, B. (2019) · doi:10.1101/865881 · Cited by 14
The inhibition of automatic imitation: a meta-analysis and synthesis of fMRI studies
Darda, K. M., & Ramsey, R. (2018) · bioRxiv · doi:10.1101/334938 · Cited by 43
Metadata
- Authors
- Carp, J.
- Venue
- NeuroImage
- Publisher
- Elsevier BV
- Published
- 2012-10
- Volume
- 63
- Issue
- 1
- Pages
- 289-300
- ISSN
- 1053-8119 (ISSN-L)
- OpenAlex
- W2053239079
- Cited by
- 457
- References
- 56
- FWCI
- 25.12
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- Funders
- Department of Defense
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.