Evaluating Alternative Correction Methods for Multiple Comparison in Functional Neuroimaging Research
Open accessPublished version of a preprint
The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Abstract
A significant challenge for fMRI research is statistically controlling for false positives without omitting true effects. Although a number of traditional methods for multiple comparison correction exist, several alternative tools have been developed that do not rely on strict parametric assumptions, but instead implement alternative methods to correct for multiple comparisons. In this study, we evaluated three of these methods, Statistical non-Parametric Mapping (SnPM), 3DClustSim, and Threshold Free Cluster Enhancement (TFCE), by examining which method produced the most consistent outcomes even when spatially-autocorrelated noise was added to the original images. We assessed the false alarm rate and hit rate of each method after noise was applied to the original images.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures

Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Kass, R. E., & Raftery, A. E. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476572 · Cited by 12,558
AFNI: Software for analysis and visualization of functional magnetic resonance neuroimages
Cox, R. W. (1996) · Computers and Biomedical Research · doi:10.1006/cbmr.1996.0014 · Cited by 10,913
Nonparametric permutation tests for functional neuroimaging: A primer with examples
Nichols, T. E., & Holmes, A. P. (2002) · Human Brain Mapping · doi:10.1002/hbm.1058 · Cited by 6,256
SMITH, S., & NICHOLS, T. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2008.03.061 · Cited by 5,250
Thresholding of statistical maps in functional neuroimaging using the false discovery rate
Genovese, C. R., Lazar, N. A., & Nichols, T. (2002) · NeuroImage · doi:10.1006/nimg.2001.1037 · Cited by 4,923
An fMRI Investigation of Emotional Engagement in Moral Judgment
Greene, J. D., Sommerville, R. B., Nystrom, L. E., Darley, J. M., & Cohen, J. D. (2001) · Science · doi:10.1126/science.1062872 · Cited by 4,561
Permutation inference for the general linear model
Winkler, A. M., Ridgway, G. R., Webster, M. A., Smith, S. M., & Nichols, T. E. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.01.060 · Cited by 3,563 · cited 5×
“Third, TFCE is currently implemented in the randomize function in FSL [18].”
MethodInfluentialCluster failure: why fMRI inferences for spatial extent have inflated false-positive rates
Eklund, A., Nichols, T. E., & Knutsson, H. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1602413113 · Cited by 3,487 · cited 5×
“We utilized the spatially-autocorrelated noise since the noise in fMRI images, such as the physiological noise, is likely to be spatially correlated according to previous research [7,34].”
BackgroundMethodInfluentialRedefine statistical significance
Benjamin, D. J., Berger, J. O., Johannesson, M., Nosek, B. A., Wagenmakers, E.-J., Berk, R., Bollen, K. A., Brembs, B., Brown, L., Camerer, C., et al. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0189-z · Cited by 2,772
The Neural Bases of Cognitive Conflict and Control in Moral Judgment
Greene, J. D., Nystrom, L. E., Engell, A. D., Darley, J. M., & Cohen, J. D. (2004) · Neuron · doi:10.1016/j.neuron.2004.09.027 · Cited by 2,620
Bayesian inference for psychology. Part II: Example applications with JASP
Wagenmakers, E.-J., Love, J., Marsman, M., Jamil, T., Ly, A., Verhagen, J., Selker, R., Gronau, Q. F., Dropmann, D., Boutin, B., et al. (2018) · Psychonomic Bulletin & Review · doi:10.3758/s13423-017-1323-7 · Cited by 1,715
Bayesian inference for psychology. Part I: Theoretical advantages and practical ramifications
Wagenmakers, E.-J., Marsman, M., Jamil, T., Ly, A., Verhagen, J., Love, J., Selker, R., Gronau, Q. F., Šmíra, M., Epskamp, S., et al. (2018) · Psychonomic Bulletin & Review · doi:10.3758/s13423-017-1343-3 · Cited by 1,627
Type I and Type II error concerns in fMRI research: re-balancing the scale
Lieberman, M. D., & Cunningham, W. A. (2009) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsp052 · Cited by 1,342 · cited 1×
“Furthermore, the measurement of small effect sizes and experiments that often have low statistical power are major issues in the fields [10].”
BackgroundCluster-extent based thresholding in fMRI analyses: Pitfalls and recommendations
Woo, C.-W., Krishnan, A., & Wager, T. D. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2013.12.058 · Cited by 1,253 · cited 3×
“Given this finding, TFCE is likely to show greater selectivity, even with noise, compared with other thresholding methods [12,33].”
BackgroundMethodControlling the familywise error rate in functional neuroimaging: a comparative review
Nichols, T., & Hayasaka, S. (2003) · Statistical Methods in Medical Research · doi:10.1191/0962280203sm341ra · Cited by 1,218
Validity and Reliability in Social Science Research
California State University, L. A., & Drost, E. A. (2011) · Education Research and Perspectives · doi:10.70953/erpv38.11005 · Cited by 977
FMRI Clustering in AFNI: False-Positive Rates Redux
Cox, R. W., Chen, G., Glen, D. R., Reynolds, R. C., & Taylor, P. A. (2017) · Brain Connectivity · doi:10.1089/brain.2016.0475 · Cited by 862
Inferring mental states from neuroimaging data: from reverse inference to large-scale decoding
Poldrack, R. A. (2011) · Neuron · doi:10.1016/j.neuron.2011.11.001 · Cited by 821 · cited 1×
“Scholars in the field often examine the neural correlates of psychological processes that are not well defined and may be less easily observable than, for example, sensory-motor processes in which there is often less variability between trials and between subjects [9].”
BackgroundEstimating sample size in functional MRI (fMRI) neuroimaging studies: Statistical power analyses
Desmond, J. E., & Glover, G. H. (2002) · Journal of Neuroscience Methods · doi:10.1016/s0165-0270(02)00121-8 · Cited by 716 · cited 1×
“When noise is added to the original images, it is likely to increase the standard error value in each voxel during the t-test [48], so the increased standard error value decreases the possibility of surviving thresholding, particularly among voxels with pseudo-t values that are close to the initial pseudo-t thresholdi…”
MethodEmpirical Analyses of BOLD fMRI Statistics
Zarahn, E., Aguirre, G. K., & D'Esposito, M. (1997) · NeuroImage · doi:10.1006/nimg.1997.0263 · Cited by 693
Mapping sources of correlation in resting state FMRI, with artifact detection and removal
Jo, H. J., Saad, Z. S., Simmons, W. K., Milbury, L. A., & Cox, R. W. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.04.246 · Cited by 515
Bennett, C., Miller, M., & Wolford, G. (2009) · NeuroImage · doi:10.1016/s1053-8119(09)71202-9 · Cited by 400 · cited 1×
“Because the analysis of fMRI data involves comparisons between more than a hundred thousand brain voxels, failing to control the false positive rate could result in misleading findings [1].”
BackgroundPernet, C. R., Latinus, M., Nichols, T. E., & Rousselet, G. A. (2015) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2014.08.003 · Cited by 300
Relationship between respiration, end-tidal CO2, and BOLD signals in resting-state fMRI
Chang, C., & Glover, G. H. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.04.048 · Cited by 292
Kragel, P. A., Kano, M., Van Oudenhove, L., Ly, H. G., Dupont, P., Rubio, A., Delon-Martin, C., Bonaz, B. L., Manuck, S. B., Gianaros, P. J., et al. (2018) · Nature Neuroscience · doi:10.1038/s41593-017-0051-7 · Cited by 272 · cited 2×
“For the present study, we used a reprocessed dataset that was created from the original dataset by a group of neuroimaging researchers who intended to illuminate the generalizability of the neural circuitry underlying higher order cognition [26].”
MethodIntellect as distinct from openness: Differences revealed by fMRI of working memory.
DeYoung, C. G., Shamosh, N. A., Green, A. E., Braver, T. S., & Gray, J. R. (2009) · Journal of Personality and Social Psychology · doi:10.1037/a0016615 · Cited by 248
Detection of neural activity in functional MRI using canonical correlation analysis
Friman, O., Cedefamn, J., Lundberg, P., Borga, M., & Knutsson, H. (2001) · Magnetic Resonance in Medicine · doi:10.1002/1522-2594(200102)45:2<323::aid-mrm1041>3.0.co;2-# · Cited by 240
fMRI clustering and false-positive rates
Cox, R. W., Chen, G., Glen, D. R., Reynolds, R. C., & Taylor, P. A. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1614961114 · Cited by 219
Multiple testing corrections, nonparametric methods, and random field theory
Nichols, T. E. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.04.014 · Cited by 204
Accurate autocorrelation modeling substantially improves fMRI reliability
Olszowy, W., Aston, J., Rua, C., & Williams, G. B. (2019) · Nature Communications · doi:10.1038/s41467-019-09230-w · Cited by 164
Effect of spatial smoothing on physiological noise in high-resolution fMRI
Triantafyllou, C., Hoge, R. D., & Wald, L. L. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.04.182 · Cited by 147
Brechtel, K., Dahl, D. B., Machann, J., Bachmann, O. P., Wenzel, I., Maier, T., Claussen, C. D., H�ring, H. U., Jacob, S., & Schick, F. (2001) · Magnetic Resonance in Medicine · doi:10.1002/1522-2594(200102)45:2<179::aid-mrm1023>3.0.co;2-d · Cited by 108
Han, H., & Park, J. (2018) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2018.00001 · Cited by 103 · cited 4×
“The hit rate is defined as “the ratio of voxels marked as active from both analyses to voxels marked as active from the analysis of the original images” [22].”
BackgroundMethodInfluentialSelf-citationHan, H., Chen, J., Jeong, C., & Glover, G. H. (2016) · Behavioural Brain Research · doi:10.1016/j.bbr.2016.01.001 · Cited by 70
Self-citationHan, H. (2017) · Journal of Moral Education · doi:10.1080/03057240.2016.1262834 · Cited by 67
Self-citationCox, R. W. (2019) · Brain Connectivity · doi:10.1089/brain.2019.0666 · Cited by 59
Han, H., Park, J., & Thoma, S. J. (2018) · Journal of Moral Education · doi:10.1080/03057240.2018.1463204 · Cited by 48
Self-citationHan, H., & Glenn, A. L. (2018) · Social Neuroscience · doi:10.1080/17470919.2017.1324521 · Cited by 46
Self-citationSlotnick, S. D. (2016) · Cognitive Neuroscience · doi:10.1080/17588928.2016.1273892 · Cited by 43
A defense of using resting-state fMRI as null data for estimating false positive rates
Nichols, T. E., Eklund, A., & Knutsson, H. E. (2017) · Cognitive Neuroscience · doi:10.1080/17588928.2017.1287069 · Cited by 15
Hahn, A. D., & Rowe, D. B. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.09.082 · Cited by 10
Gopinath, K., Krishnamurthy, V., Lacey, S., & Sathian, K. (2017) · Brain Connectivity · doi:10.1089/brain.2017.0522 · Cited by 10
Bayesian fMRI-SPM EvaluationPreprint
Han, H. (2018) · OSF Preprints (OSF Preprints) · Cited by 1
Self-citationGlover (2000) · doi:10.1002/1522-2594(200007)44:1<162::aid-mrm23>3.0.co;2-e
Cited by
Neural correlates of conceptual-level fear generalization in posttraumatic stress disorder
Mid-Atlantic MIRECC Workgroup, Morey, R. A., Haswell, C. C., Stjepanović, D., Dunsmoor, J. E., & LaBar, K. S. (2020) · Neuropsychopharmacology · doi:10.1038/s41386-020-0661-8 · Cited by 67
Henn, A. T., Larsen, B., Frahm, L., Xu, A., Adebimpe, A., Scott, J. C., Linguiti, S., Sharma, V., Basbaum, A. I., Corder, G. F., et al. (2022) · Pain · doi:10.1097/j.pain.0000000000002681 · Cited by 56
Rostowsky, K. A., Irimia, A., & Initiative, A. D. N. (2021) · GeroScience · doi:10.1007/s11357-021-00355-9 · Cited by 52
Steward, T., Martínez‐Zalacaín, I., Mestre‐Bach, G., Sánchez, I., Riesco, N., Jiménez‐Múrcia, S., Fernández-Formoso, J. A., Heras, M. V. D. L., Custal, N., Menchón, J. M., et al. (2020) · Psychological Medicine · doi:10.1017/s0033291720002457 · Cited by 47
Neural basis of positive and negative emotion regulation in remitted depression
van Kleef, R. S., Marsman, J.-B. C., van Valen, E., Bockting, C., Aleman, A., & van Tol, M. (2022) · NeuroImage Clinical · doi:10.1016/j.nicl.2022.102988 · Cited by 38
Metadata
- Authors
- Han, H., Glenn, A. L., & Dawson, K. J.
- Affiliations
- University of Alabama (US)
- Corresponding author
- Han, Hyemin
- Venue
- Brain Sciences
- Publisher
- MDPI AG
- Published
- 2019-08-12
- Volume
- 9
- Issue
- 8
- Pages
- 198
- ISSN
- 2076-3425 (ISSN-L)
- PMC
- PMC6721788
- OpenAlex
- W2967024440
- Cited by
- 34
- References
- 43
- Citations by year
- 2026: 4 · 2025: 2 · 2024: 2 · 2023: 3 · 2022: 6 · 2021: 13 · 2020: 3 · 2019: 1
- FWCI
- 3.34
- Type
- journal article
- Language
- en
- Open access
- gold
- dx.doi.org
- Licence
- Locations
- Brain Sciences (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- PsyArXiv (OSF Preprints) (submittedVersion)
- PsyArXiv (OSF Preprints) (submittedVersion, cc-by)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Functional Brain Connectivity Studies
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Advanced MRI Techniques and Applications
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Advanced Neuroimaging Techniques and Applications
- Keywords
- multiple testing correction, false positives, functional MRI, threshold-free cluster enhancement, spatial autocorrelation, false alarm rate, hit rate, non-parametric methods
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.