Is the statistic value all we should care about in neuroimaging?
Open accessThe DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Abstract
Here we address an important issue that has been embedded within the neuroimaging community for a long time: the absence of effect estimates in results reporting in the literature. The statistic value itself, as a dimensionless measure, does not provide information on the biophysical interpretation of a study, and it certainly does not represent the whole picture of a study. Unfortunately, in contrast to standard practice in most scientific fields, effect (or amplitude) estimates are usually not provided in most results reporting in the current neuroimaging publications and presentations. Possible reasons underlying this general trend include: 1) lack of general awareness, 2) software limitations, 3) inaccurate estimation of the BOLD response, and 4) poor modeling due to our relatively limited understanding of FMRI signal components. However, as we discuss here, such reporting damages the reliability and interpretability of the scientific findings themselves, and there is in fact no overwhelming reason for such a practice to persist. In order to promote meaningful interpretation, cross validation, reproducibility, meta and power analyses in neuroimaging, we strongly suggest that, as part of good scientific practice, effect estimates should be reported together with their corresponding statistic values. We provide several easily adaptable recommendations for facilitating this process.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · cited 2×
“…most research-oriented investigations place a heavily-lopsided emphasis on the false positive rate controllability, sensitivity (or power) may also be a primary focus under some circumstances, such as pre-surgical detection, where the efficiency is usually less than 10% (Button et al., 2013).”
BackgroundThe ASA Statement on p -Values: Context, Process, and Purpose
Wasserstein, R. L., & Lazar, N. A. (2016) · The American Statistician · doi:10.1080/00031305.2016.1154108 · Cited by 6,306
Distributed and overlapping representations of faces and objects in ventral temporal cortex
Haxby, J. V., Gobbini, M. I., Furey, M. L., Ishai, A., Schouten, J. L., & Pietrini, P. (2001) · Science · doi:10.1126/science.1063736 · Cited by 3,940
Experimental evidence of massive-scale emotional contagion through social networks
Kramer, A. D. I., Guillory, J. E., & Hancock, J. T. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1320040111 · Cited by 3,687
Large-scale automated synthesis of human functional neuroimaging data.
Yarkoni, T., Poldrack, R. A., Nichols, T. E., Van Essen, D. C., & Wager, T. D. (2011) · Nature Methods · doi:10.1038/nmeth.1635 · Cited by 3,686
Statistical methods in psychology journals: Guidelines and explanations.
Wilkinson, L., & Task Force on Statistical Inference (1999) · American Psychologist · doi:10.1037/0003-066x.54.8.594 · Cited by 2,791
Publication Bias in Meta‐Analysis: Prevention, Assessment and Adjustments
(2005) · doi:10.1002/0470870168 · Cited by 2,368
Intersubject synchronization of cortical activity during natural vision
Hasson, U., Nir, Y., Levy, I., Fuhrmann, G., & Malach, R. (2004) · Science · doi:10.1126/science.1089506 · Cited by 1,839 · cited 1×
“Similarly, for naturalistic scanning, one measure is the “inter-subject correlation” (ISC) at a region that features the synchronization or similarity among subjects (Hasson et al., 2004).”
BackgroundScientific method: Statistical errors
Nuzzo, R. (2014) · Nature · doi:10.1038/506150a · Cited by 1,536 · cited 1×
“Recently there have been a number of discussions about the use and misuse of p-values in the scientific community (e.g., Wasserstein and Lazar, 2016; Nuzzo, 2014), and others have been more critical of the “cult” or “obsession” of statistical significance (e.g., Ziliak and McCloskey, 2009).”
BackgroundThe Difference Between “Significant” and “Not Significant” is not Itself Statistically Significant
Gelman, A., & Stern, H. (2006) · The American Statistician · doi:10.1198/000313006x152649 · Cited by 1,407
Type I and Type II error concerns in fMRI research: re-balancing the scale
Lieberman, M. D., & Cunningham, W. A. (2009) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsp052 · Cited by 1,342 · cited 1×
“Statistical inference should not be a binary decision, and the inclusion of effect estimates allows for a consistent approach to avoid this and to achieve a balance between false positives and false negatives (Lieberman and Cunningham, 2009).”
BackgroundDeep supervised, but not unsupervised, models may explain IT cortical representation
Khaligh-Razavi, S.-M., & Kriegeskorte, N. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003915 · Cited by 1,204
Voxel-wise meta-analysis of grey matter changes in obsessive-compulsive disorder
Radua, J., & Mataix-Cols, D. (2009) · British Journal of Psychiatry · doi:10.1192/bjp.bp.108.055046 · Cited by 863 · cited 3×
“…2016) would be more robust than other alternatives such as label- and coordinate-based approaches (e.g., coordinates only: activation likelihood estimation, Eickhoff et al. 2012; coordinates and Gaussianized Z-values: Radua and Mataix-Cols, 2009; Costafreda et al., 2009; Yarkoni et al., 2011).”
BackgroundBest practices in data analysis and sharing in neuroimaging using MRI
Nichols, T. E., Das, S., Eickhoff, S. B., Evans, A. C., Glatard, T., Hanke, M., Kriegeskorte, N., Milham, M. P., Poldrack, R. A., Poline, J.-B., et al. (2017) · Nature Neuroscience · doi:10.1038/nn.4500 · Cited by 714
Guidelines for reporting an fMRI study
Poldrack, R. A., Fletcher, P. C., Henson, R. N., Worsley, K. J., Brett, M., & Nichols, T. E. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2007.11.048 · Cited by 550 · cited 1×
“Some effort has been devoted to promote the standardization of the reporting process in neuroimaging analysis (e.g., Poldrack et al., 2008; Carp, 2012; Nichols et al., 2016), though the important issue of reporting effect estimates has not been paid much attention.”
BackgroundUludağ, K., Müller-Bierl, B., & Uğurbil, K. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.05.051 · Cited by 450
Type S error rates for classical and Bayesian single and multiple comparison procedures
Gelman, A., & Tuerlinckx, F. (2000) · Computational Statistics · doi:10.1007/s001800000040 · Cited by 380 · cited 1×
“…of failure or success, respectively, to detect the effect), but it would also be impossible to assess two other useful but less known errors (Gelman and Tuerlinckx, 2000): type M (tendency to over- or under-estimate the effect magnitude) and type S (likelihood of obtaining the incorrect…”
BackgroundFirst results from psychology’s largest reproducibility test
Baker, M. (2015) · Nature · doi:10.1038/nature.2015.17433 · Cited by 54 · cited 2×
“In recent surveys, about 60% of published experiments failed to survive replication in psychology (Baker, 2015) and about 40% in economics (Bohannon, 2016), and the situation with neuroimaging is likely not much better (Griffanti et al.”
Cited by
FMRI Clustering in AFNI: False-Positive Rates Redux
Cox, R. W., Chen, G., Glen, D. R., Reynolds, R. C., & Taylor, P. A. (2017) · Brain Connectivity · doi:10.1089/brain.2016.0475 · Cited by 862
Self-citationMeasuring shared responses across subjects using intersubject correlation
Nastase, S. A., Gazzola, V., Hasson, U., & Keysers, C. (2019) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsz037 · Cited by 407
A Hitchhiker's Guide to Functional Magnetic Resonance Imaging
Soares, J. M., Magalhães, R., Moreira, P. S., Sousa, A., Ganz, E., Sampaio, A., Alves, V., Marques, P., & Sousa, N. (2016) · Frontiers in Neuroscience · doi:10.3389/fnins.2016.00515 · Cited by 272
Mothers with substance addictions show reduced reward responses when viewing their own infant's face
Kim, S., Iyengar, U., Mayes, L. C., Potenza, M. N., Rutherford, H. J. V., & Strathearn, L. (2017) · Human Brain Mapping · doi:10.1002/hbm.23731 · Cited by 71
Rosch, K. S., Mostofsky, S. H., & Nebel, M. B. (2018) · Journal of Neurodevelopmental Disorders · doi:10.1186/s11689-018-9254-9 · Cited by 69
Metadata
- Authors
- Chen, G., Taylor, P. A., & Cox, R. W.
- Venue
- NeuroImage
- Publisher
- Elsevier BV
- Published
- 2017-02
- Volume
- 147
- Pages
- 952-959
- ISSN
- 1053-8119 (ISSN-L)
- OpenAlex
- W2952384350
- Cited by
- 139
- References
- 18
- FWCI
- 8.81
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.