Circular analysis in systems neuroscience: the dangers of double dipping
Open accessAbstract
A neuroscientific experiment typically generates a large amount of data, of which only a small fraction is analyzed in detail and presented in a publication. However, selection among noisy measurements can render circular an otherwise appropriate analysis and invalidate results. Here we argue that systems neuroscience needs to adjust some widespread practices to avoid the circularity that can arise from selection. In particular, 'double dipping', the use of the same dataset for selection and selective analysis, will give distorted descriptive statistics and invalid statistical inference whenever the results statistics are not inherently independent of the selection criteria under the null hypothesis. To demonstrate the problem, we apply widely used analyses to noise data known to not contain the experimental effects in question. Spurious effects can appear in the context of both univariate activation analysis and multivariate pattern-information analysis. We suggest a policy for avoiding circularity.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
An across-run cross-validation approach ensures that data used for defining fROIs were independent of data used for estimating response.
“For these localizer analyses, an across-run cross-validation approach was used (Nieto-Castañón & Fedorenko, 2012) to ensure that data used for defining fROIs were independent of data used for estimating response (Kriegeskorte, Simmons, Bellgowan, & Baker, 2009).”
From Functional Specificity and Sex Differences in the Neural Circuits Supporting the Inhibition of Automatic Imitation (2018) · cites this paper as Kriegeskorte, Simmons, Bellgowan, & Baker, 2009
Selecting subjects based on the same data used to demonstrate the absence of awareness is comparable to double-dipping and likely to confound the results with regression to the mean.
“This practice is problematic because selecting subjects based on the same data used to demonstrate the absence of awareness is comparable to “double-dipping” (Kriegeskorte et al. 2009) and likely to confound the results with regression to the mean (Shanks 2017, Stein et al. 2024).”
From Evidence for unconscious priming using a Bayesian single-subject approach (2026) · cites this paper as Kriegeskorte et al. 2009
Many findings of cognitive neuroscience provide a solid basis on which to build.
“Despite methodological challenges10–11, many of the findings of cognitive neuroscience provide a solid basis on which to build.”
From Cognitive computational neuroscience (2018) · cites this paper as [11]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Seel, N. M. (2012) · Springer eBooks · doi:10.1007/978-1-4419-1428-6_5195 · Cited by 19,525 · see citing works
Neural networks for pattern recognition
Bishop, C. (1994) · Choice Reviews Online · doi:10.5860/choice.31-5500 · Cited by 18,537 · see citing works
AFNI: Software for analysis and visualization of functional magnetic resonance neuroimages
Cox, R. W. (1996) · Computers and Biomedical Research · doi:10.1006/cbmr.1996.0014 · Cited by 10,913 · see citing works
Nonparametric permutation tests for functional neuroimaging: A primer with examples
Nichols, T. E., & Holmes, A. P. (2002) · Human Brain Mapping · doi:10.1002/hbm.1058 · Cited by 6,256 · see citing works
Thresholding of statistical maps in functional neuroimaging using the false discovery rate
Genovese, C. R., Lazar, N. A., & Nichols, T. (2002) · NeuroImage · doi:10.1006/nimg.2001.1037 · Cited by 4,923 · see citing works
Distributed and overlapping representations of faces and objects in ventral temporal cortex
Haxby, J. V., Gobbini, M. I., Furey, M. L., Ishai, A., Schouten, J. L., & Pietrini, P. (2001) · Science · doi:10.1126/science.1063736 · Cited by 3,940 · see citing works
Abe, S. (2001) · Springer eBooks · doi:10.1007/978-1-4471-0285-4 · Cited by 3,740 · see citing works
Forman, S. D., Cohen, J. D., Fitzgerald, M., Eddy, W. F., Mintun, M. A., & Noll, D. C. (1995) · Magnetic Resonance in Medicine · doi:10.1002/mrm.1910330508 · Cited by 3,202 · see citing works
Pattern recognition and machine learning
(2007) · Choice Reviews Online · doi:10.5860/choice.44-5091 · Cited by 2,782 · see citing works
Analysis of fMRI Time-Series Revisited
Friston, K. J., Holmes, A. P., Poline, J., Grasby, P. M., Williams, S., Frackowiak, R. S. J., & Turner, R. (1995) · NeuroImage · doi:10.1006/nimg.1995.1007 · Cited by 2,504 · see citing works
Immanuel, I. (1998) · Felix Meiner Verlag eBooks · doi:10.28937/978-3-7873-2112-4 · Cited by 2,308 · see citing works
Beyond mind-reading: multi-voxel pattern analysis of fMRI data
Norman, K. A., Polyn, S. M., Detre, G. J., & Haxby, J. V. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.07.005 · Cited by 2,271 · see citing works
Analysis of fMRI Time-Series Revisited—Again
Worsley, K. J., & Friston, K. J. (1995) · NeuroImage · doi:10.1006/nimg.1995.1023 · Cited by 2,198 · see citing works
Information-based functional brain mapping
Kriegeskorte, N., Goebel, R., & Bandettini, P. (2006) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0600244103 · Cited by 2,113 · see citing works
Self-citationPattern Classification (2nd Edition)
Duda, R. O., Hart, P. E., & Stork, D. G. (2000) · Wiley-Interscience eBooks · Cited by 2,064 · see citing works
Cover, T. M. (1965) · IEEE Transactions on Electronic Computers · doi:10.1109/pgec.1965.264137 · Cited by 2,044 · see citing works
A three-dimensional statistical analysis for CBF activation studies in human brain
Worsley, K. J., Evans, A. C., Marrett, S., & Neelin, P. (1992) · Journal of Cerebral Blood Flow & Metabolism · doi:10.1038/jcbfm.1992.127 · Cited by 2,036 · see citing works
Valid conjunction inference with the minimum statistic
Nichols, T. E., Brett, M., Andersson, J., Wager, T. D., & Poline, J. (2005) · NeuroImage · doi:10.1016/j.neuroimage.2004.12.005 · Cited by 1,885 · see citing works
Analysis of functional MRI time‐series
Friston, K. J., Jezzard, P., & Turner, R. (1994) · Human Brain Mapping · doi:10.1002/hbm.460010207 · Cited by 1,835 · see citing works
Puzzlingly High Correlations in fMRI Studies of Emotion, Personality, and Social Cognition
Vul, E., Harris, C., Winkielman, P., & Pashler, H. (2009) · Perspectives on Psychological Science · doi:10.1111/j.1745-6924.2009.01125.x · Cited by 1,475 · cited 1× · see citing works
“Untangling how data and assumptions influence neuroscientific analyses sometimes reveals that assumptions predetermine results to some extent 1–5 .”
Controlling the familywise error rate in functional neuroimaging: a comparative review
Nichols, T. E., & Hayasaka, S. (2003) · Statistical Methods in Medical Research · doi:10.1191/0962280203sm341ra · Cited by 1,210 · see citing works
Multisubject fMRI Studies and Conjunction Analyses
Friston, K. J., Holmes, A. P., Price, C. J., Büchel, C., & Worsley, K. J. (1999) · NeuroImage · doi:10.1006/nimg.1999.0484 · Cited by 1,090 · see citing works
Cox, D., & Savoy, R. L. (2003) · NeuroImage · doi:10.1016/s1053-8119(03)00049-1 · Cited by 1,057 · see citing works
Learning to Decode Cognitive States from Brain Images
Mitchell, T. M., Hutchinson, R., Niculescu, R., Pereira, F., Wang, X., Just, M., & Newman, S. (2004) · Machine-mediated learning · doi:10.1023/b:mach.0000035475.85309.1b · Cited by 765 · see citing works
Combining spatial extent and peak intensity to test for activations in functional imaging
Poline, J.-B., Worsley, K. J., Evans, A. C., & Friston, K. J. (1997) · NeuroImage · doi:10.1006/nimg.1996.0248 · Cited by 670 · see citing works
Book Review: Pattern Recognition and Machine Learning
Nasrabadi, N. M. (2007) · Journal of Electronic Imaging · doi:10.1117/1.2819119 · Cited by 636 · see citing works
Kant, I. (1911) · De Gruyter eBooks · doi:10.1515/9783112610046 · Cited by 551 · see citing works
Divide and conquer: A defense of functional localizers
Saxe, R., Brett, M., & Kanwisher, N. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2005.12.062 · Cited by 544 · see citing works
A critique of functional localisers
Friston, K. J., Rotshtein, P., Geng, J. J., Sterzer, P., & Henson, R. N. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2005.08.012 · Cited by 424 · see citing works
A Comparative Review On Aggression Konrad Lorenz The Territorial Imperative Robert Ardrey
Alexander, R. D., & Tinkle, D. W. (1968) · BioScience · doi:10.2307/1294259 · Cited by 400 · see citing works
Value-Based Modulations in Human Visual Cortex
Serences, J. T. (2008) · Neuron · doi:10.1016/j.neuron.2008.10.051 · Cited by 353 · see citing works
Microarrays, empirical Bayes and the two-groups model
Efron, B. (2006) · Cited by 337 · see citing works
Strother, S. C., Anderson, J., Hansen, L. K., Kjems, U., Kustra, R., Sidtis, J. J., Frutiger, S. A., Muley, S., LaConte, S. M., & Rottenberg, D. (2002) · NeuroImage · doi:10.1006/nimg.2001.1034 · Cited by 244 · see citing works
Strother, S. C., Anderson, J., Frutiger, S. A., Muley, S., Rottenberg, D. A., Kjems, U., & Hansen, L. K. (2000) · NeuroImage · doi:10.1016/s1053-8119(00)91523-4 · Cited by 217 · see citing works
Functional Magnetic Resonance Imaging (fMRI)
Savoy, R. L. (2002) · Elsevier eBooks · doi:10.1016/b0-12-227210-2/00146-1 · Cited by 200 · see citing works
Woolrich, M. W., Behrens, T. E. J., Beckmann, C. F., & Smith, S. M. (2005) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2004.836545 · Cited by 168 · see citing works
Strother, S. C., Conte, S. L., Hansen, L. K., Anderson, J., Zhang, J., Pulapura, S. K., & Rottenberg, D. A. (2004) · NeuroImage · doi:10.1016/j.neuroimage.2004.07.022 · Cited by 134 · see citing works
Does the fusiform face area contain subregions highly selective for nonfaces?
Baker, C. I., Hutchison, T., & Kanwisher, N. (2006) · Nature Neuroscience · doi:10.1038/nn0107-3 · Cited by 72 · see citing works
Self-citationHuman Cortical Object Recognition from a Visual Motion Flowfield
Kriegeskorte, N., Sorger, B., Naumer, M. J., et al. (2003) · Journal of Neuroscience · doi:10.1523/jneurosci.23-04-01451.2003 · Cited by 67 · see citing works
Self-citationComment: Microarrays, Empirical Bayes and the Two-Groups Model
Benjamini, Y. (2008) · Statistical Science · doi:10.1214/07-sts236b · Cited by 31 · see citing works
Circular inference in neuroscience: The dangers of double dipping
Kriegeskorte, N., Simmons, W. K., Bellgowan, P. S. F., & Baker, C. I. (2010) · Journal of Vision · doi:10.1167/8.6.88 · Cited by 30 · see citing works
Self-citationComment: Microarrays, Empirical Bayes and the Twogroups modelPreprint
Benjamini, Y. (2008) · Cited by 12 · see citing works
Microarrays, Empirical Bayes and the Two-Groups Model. Comment.
Benjamini, Y. (2008) · Statistical Science · Cited by 2 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- [4]Vul E Kanwisher N Hanson S Bunzl M Begging the question: The non-independence error in fMRI data analysis Foundations and Philosophy for Neuroimaging in press To appear in
- [21]Mitchell TM Machine Learning 1997 McGraw Hill
Cited by
Sorted by citations
Dysfunction of the prefrontal cortex in addiction: neuroimaging findings and clinical implications
Goldstein, R. Z., & Volkow, N. D. (2011) · Nature Reviews Neuroscience · doi:10.1038/nrn3119 · Cited by 2,749 · see citing works
Intrinsic and task-evoked network architectures of the human brain
Cole, M. W., Bassett, D. S., Power, J. D., Braver, T. S., & Petersen, S. E. (2014) · Neuron · doi:10.1016/j.neuron.2014.05.014 · Cited by 1,564 · see citing works
Puzzlingly High Correlations in fMRI Studies of Emotion, Personality, and Social Cognition
Vul, E., Harris, C., Winkielman, P., & Pashler, H. (2009) · Perspectives on Psychological Science · doi:10.1111/j.1745-6924.2009.01125.x · Cited by 1,475 · see citing works
Broad domain generality in focal regions of frontal and parietal cortex
Fedorenko, E., Duncan, J., & Kanwisher, N. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1315235110 · Cited by 952 · see citing works
An Agenda for Purely Confirmatory Research
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., van der Maas, H. L. J., & Kievit, R. A. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612463078 · Cited by 1,042 · see citing works
Behavioral interpretations of intrinsic connectivity networks
Laird, A. R., Fox, P. M., Eickhoff, S. B., Turner, J. A., Ray, K. L., McKay, D. R., Glahn, D. C., Beckmann, C. F., Smith, S. M., & Fox, P. T. (2011) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00077 · Cited by 1,023 · see citing works
Functional specificity in the human brain: a window into the functional architecture of the mind
Kanwisher, N. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1005062107 · Cited by 929 · see citing works
What does the amygdala contribute to social cognition
Adolphs, R. (2010) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2010.05445.x · Cited by 836 · see citing works
Wicherts, J. M., Veldkamp, C. L. S., Augusteijn, H. E. M., Bakker, M., van Aert, R. C. M., & van Assen, M. A. L. M. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.01832 · Cited by 754 · see citing works
Hein, G., Silani, G., Preuschoff, K., Batson, C. D., & Singer, T. (2010) · Neuron · doi:10.1016/j.neuron.2010.09.003 · Cited by 821 · see citing works
Establishment of Best Practices for Evidence for Prediction: A Review
Poldrack, R. A., Huckins, G., & Varoquaux, G. (2020) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2019.3671 · Cited by 738 · see citing works
Wen, J., Thibeau-Sutre, E., Diaz-Melo, M., Samper-González, J., Routier, A., Bottani, S., Dormont, D., Durrleman, S., Burgos, N., & Colliot, O. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101694 · Cited by 809 · see citing works
New method for fMRI investigations of language: defining ROIs functionally in individual subjects
Fedorenko, E., Hsieh, P.-J., Nieto-Castañón, A., Whitfield-Gabrieli, S., & Kanwisher, N. (2010) · Journal of Neurophysiology · doi:10.1152/jn.00032.2010 · Cited by 686 · see citing works
Szucs, D., & Ioannidis, J. P. A. (2017) · PLOS Biology · doi:10.1371/journal.pbio.2000797 · Cited by 761 · see citing works
Best practices in data analysis and sharing in neuroimaging using MRI
Nichols, T. E., Das, S., Eickhoff, S. B., Evans, A. C., Glatard, T., Hanke, M., Kriegeskorte, N., Milham, M. P., Poldrack, R. A., Poline, J.-B., et al. (2017) · Nature Neuroscience · doi:10.1038/nn.4500 · Cited by 714 · see citing works
Self-citationDecoding Neural Representational Spaces Using Multivariate Pattern Analysis
Haxby, J. V., Connolly, A. C., & Guntupalli, J. S. (2014) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-062012-170325 · Cited by 717 · see citing works
Global Connectivity of Prefrontal Cortex Predicts Cognitive Control and Intelligence
Cole, M. W., Yarkoni, T., Repovs, G., Anticevic, A., & Braver, T. S. (2012) · Journal of Neuroscience · doi:10.1523/jneurosci.0536-12.2012 · Cited by 613 · see citing works
Rathore, S., Habes, M., Iftikhar, M. A., Shacklett, A., & Davatzikos, C. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.03.057 · Cited by 622 · see citing works
Compassion training alters altruism and neural responses to suffering.
Weng, H. Y., Fox, A. S., Shackman, A. J., Stodola, D. E., Caldwell, J. Z. K., Olson, M. C., Rogers, G. M., & Davidson, R. J. (2013) · Psychological Science · doi:10.1177/0956797612469537 · Cited by 602 · see citing works
A Review of Feature Reduction Techniques in Neuroimaging
Mwangi, B., Tian, T. S., & Soares, J. C. (2014) · Neuroinformatics · doi:10.1007/s12021-013-9204-3 · Cited by 559 · see citing works
Oxytocin Attenuates Amygdala Reactivity to Fear in Generalized Social Anxiety Disorder
Labuschagne, I., Phan, K. L., Wood, A., Angstadt, M., Chua, P., Heinrichs, M., Stout, J. C., & Nathan, P. J. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2010.123 · Cited by 549 · see citing works
Cognitive computational neuroscience
Kriegeskorte, N., & Douglas, P. K. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0210-5 · Cited by 535 · see citing works
Self-citationPalhano-Fontes, F., Andrade, K. C., Tofoli, L. F., Santos, A. C., Crippa, J. A. S., Hallak, J. E. C., Ribeiro, S., & de Araujo, D. B. (2015) · PLOS ONE · doi:10.1371/journal.pone.0118143 · Cited by 454 · see citing works
Desbordes, G., Negi, L. T., Pace, T. W. W., Wallace, B. A., Raison, C. L., & Schwartz, E. L. (2012) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2012.00292 · Cited by 458 · see citing works
Heller, A. S., Johnstone, T., Shackman, A. J., Light, S. N., Peterson, M. J., Kolden, G. G., Kalin, N. H., & Davidson, R. J. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0910651106 · Cited by 444 · see citing works
Saygin, A. P., Chaminade, T., Ishiguro, H., Driver, J., & Frith, C. (2012) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsr025 · Cited by 452 · see citing works
BCILAB: a platform for brain–computer interface development
Kothe, C. A., & Makeig, S. (2013) · Journal of Neural Engineering · doi:10.1088/1741-2560/10/5/056014 · Cited by 412 · see citing works
A Primer on Pattern-Based Approaches to fMRI: Principles, Pitfalls, and Perspectives
Haynes, J.-D. (2015) · Neuron · doi:10.1016/j.neuron.2015.05.025 · Cited by 402 · see citing works
Bennett, C., Miller, M., & Wolford, G. (2009) · NeuroImage · doi:10.1016/s1053-8119(09)71202-9 · Cited by 400 · see citing works
Decoding Patterns of Human Brain Activity
Tong, F., & Pratte, M. S. (2012) · Annual Review of Psychology · doi:10.1146/annurev-psych-120710-100412 · Cited by 375 · see citing works
A convergent functional architecture of the insula emerges across imaging modalities
Kelly, C., Toro, R., Di Martino, A., Cox, C. L., Bellec, P., Castellanos, F. X., & Milham, M. P. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.03.021 · Cited by 374 · see citing works
The problem of pseudoreplication in neuroscientific studies: is it affecting your analysis?
Lazic, S. E. (2010) · BMC Neuroscience · doi:10.1186/1471-2202-11-5 · Cited by 381 · see citing works
Reward Reduces Conflict by Enhancing Attentional Control and Biasing Visual Cortical Processing
Padmala, S., & Pessoa, L. (2011) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00011 · Cited by 362 · see citing works
The Architecture of Reward Value Coding in the Human Orbitofrontal Cortex
Sescousse, G., Redouté, J., & Dreher, J.-C. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.3501-10.2010 · Cited by 353 · see citing works
Evaluating the evidence for biotypes of depression: Methodological replication and extension of
Dinga, R., Schmaal, L., Penninx, B. W. J. H., van Tol, M. J., Veltman, D. J., van Velzen, L., Mennes, M., van der Wee, N. J. A., & Marquand, A. F. (2019) · NeuroImage Clinical · doi:10.1016/j.nicl.2019.101796 · Cited by 313 · see citing works
Striatal Prediction Error Modulates Cortical Coupling
den Ouden, H. E. M., Daunizeau, J., Roiser, J., Friston, K. J., & Stephan, K. E. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.4458-09.2010 · Cited by 304 · see citing works
van Marle, H. J. F., Hermans, E. J., Qin, S., & Fernández, G. (2009) · Biological Psychiatry · doi:10.1016/j.biopsych.2009.05.014 · Cited by 317 · see citing works
Acute Sleep Deprivation Enhances the Brain's Response to Hedonic Food Stimuli: An fMRI Study
Benedict, C., Brooks, S. J., O'Daly, O. G., Almèn, M. S., Morell, A., Åberg, K., Gingnell, M., Schultes, B., Hallschmid, M., Broman, J.-E., et al. (2012) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2011-2759 · Cited by 308 · see citing works
Chaddock-Heyman, L., Erickson, K. I., Voss, M. W., Knecht, A. M., Pontifex, M. B., Castelli, D. M., Hillman, C. H., & Kramer, A. F. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00072 · Cited by 305 · see citing works
Steinbeis, N., Bernhardt, B. C., & Singer, T. (2012) · Neuron · doi:10.1016/j.neuron.2011.12.027 · Cited by 298 · see citing works
Abnormally high degree connectivity of the orbitofrontal cortex in obsessive-compulsive disorder
Beucke, J. C., Sepulcre, J., Talukdar, T., Linnman, C., Zschenderlein, K., Endrass, T., Kaufmann, C., & Kathmann, N. (2013) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2013.173 · Cited by 280 · see citing works
Early neglect is associated with alterations in white matter integrity and cognitive functioning
Hanson, J. L., Adluru, N., Chung, M. K., Alexander, A. L., Davidson, R. J., & Pollak, S. D. (2013) · Child Development · doi:10.1111/cdev.12069 · Cited by 276 · see citing works
When Null Hypothesis Significance Testing Is Unsuitable for Research: A Reassessment
Szucs, D., & Ioannidis, J. P. A. (2017) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2017.00390 · Cited by 298 · see citing works
The neural code of reward anticipation in human orbitofrontal cortex
Kahnt, T., Heinzle, J., Park, S. Q., & Haynes, J.-D. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0912838107 · Cited by 274 · see citing works
Global Prefrontal and Fronto-Amygdala Dysconnectivity in Bipolar I Disorder with Psychosis History
Anticevic, A., Brumbaugh, M. S., Winkler, A. M., Lombardo, L. E., Barrett, J., Corlett, P. R., Kober, H., Gruber, J., Repovs, G., Cole, M. W., et al. (2013) · Biological Psychiatry · doi:10.1016/j.biopsych.2012.07.031 · Cited by 263 · see citing works
Filimon, F. (2010) · The Neuroscientist · doi:10.1177/1073858410375468 · Cited by 265 · see citing works
Adolescents, adults and rewards: Comparing motivational neurocircuitry recruitment using fMRI
Bjork, J. M., Smith, A. R., Chen, G., & Hommer, D. W. (2010) · PLoS ONE · doi:10.1371/journal.pone.0011440 · Cited by 271 · see citing works
Predicting treatment response in social anxiety disorder from functional magnetic resonance imaging.
Doehrmann, O., Ghosh, S. S., Polli, F. E., Reynolds, G. O., Horn, F., Keshavan, A., Triantafyllou, C., Saygin, Z. M., Whitfield-Gabrieli, S., Hofmann, S. G., et al. (2013) · JAMA Psychiatry · doi:10.1001/2013.jamapsychiatry.5 · Cited by 262 · see citing works
Kokal, I., Engel, A., Kirschner, S., & Keysers, C. (2011) · PLoS ONE · doi:10.1371/journal.pone.0027272 · Cited by 254 · see citing works
Single-Unit Stability Using Chronically Implanted Multielectrode Arrays
Dickey, A. S., Suminski, A., Amit, Y., & Hatsopoulos, N. G. (2009) · Journal of Neurophysiology · doi:10.1152/jn.90920.2008 · Cited by 234 · see citing works
Young, J., Modat, M., Cardoso, M. J., Mendelson, A., Cash, D., & Ourselin, S. (2013) · NeuroImage Clinical · doi:10.1016/j.nicl.2013.05.004 · Cited by 249 · see citing works
Nieto-Castañón, A., & Fedorenko, E. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.06.065 · Cited by 247 · see citing works
Uncertainty during anticipation modulates neural responses to aversion in human insula and amygdala
Sarinopoulos, I., Grupe, D. W., Mackiewicz, K. L., Herrington, J. D., Lor, M., Steege, E. E., & Nitschke, J. B. (2010) · Cerebral Cortex · doi:10.1093/cercor/bhp155 · Cited by 241 · see citing works
Lawrence, N. S., Hinton, E. C., Parkinson, J. A., & Lawrence, A. D. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.06.070 · Cited by 242 · see citing works
Ten common statistical mistakes to watch out for when writing or reviewing a manuscript
Makin, T. R., & Orban de Xivry, J.-J. (2019) · eLife · doi:10.7554/elife.48175 · Cited by 238 · see citing works
Beijers, L., Wardenaar, K. J., van Loo, H. M., & Schoevers, R. A. (2019) · Molecular Psychiatry · doi:10.1038/s41380-019-0385-5 · Cited by 240 · see citing works
The impact of study design on pattern estimation for single-trial multivariate pattern analysis
Mumford, J. A., Davis, T., & Poldrack, R. A. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.09.026 · Cited by 237 · see citing works
Episodic reinstatement in the medial temporal lobe
Staresina, B. P., Henson, R. N. A., Kriegeskorte, N., & Alink, A. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.4156-12.2012 · Cited by 228 · see citing works
Self-citationFonzo, G. A., Simmons, A. N., Thorp, S. R., Norman, S. B., Paulus, M. P., & Stein, M. B. (2010) · Biological Psychiatry · doi:10.1016/j.biopsych.2010.04.028 · Cited by 235 · see citing works
Methods in cognitive pupillometry: Design, preprocessing, and statistical analysis
Mathôt, S., & Vilotijević, A. (2022) · Behavior Research Methods · doi:10.3758/s13428-022-01957-7 · Cited by 235 · see citing works
Neural correlate of the construction of sentence meaning
Fedorenko, E., Scott, T. L., Brunner, P., Coon, W. G., Pritchett, B., Schalk, G., & Kanwisher, N. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1612132113 · Cited by 199 · see citing works
Kinetics and dose dependency of intranasal oxytocin effects on amygdala reactivity
Spengler, F. B., Schultz, J., Scheele, D., Essel, M., Maier, W., Heinrichs, M., & Hurlemann, R. (2017) · Biological Psychiatry · doi:10.1016/j.biopsych.2017.04.015 · Cited by 229 · see citing works
Rütgen, M., Seidel, E.-M., Silani, G., Riečanský, I., Hummer, A., Windischberger, C., Petrovic, P., & Lamm, C. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1511269112 · Cited by 208 · see citing works
Nummenmaa, L., Hirvonen, J., Hannukainen, J. C., Immonen, H., Lindroos, M. M., Salminen, P., & Nuutila, P. (2012) · PLoS ONE · doi:10.1371/journal.pone.0031089 · Cited by 220 · see citing works
Similarities and differences in perceiving threat from dynamic faces and bodies. An fMRI study
Kret, M. E., Pichon, S., Grèzes, J., & de Gelder, B. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2010.08.012 · Cited by 204 · see citing works
Lack of selectivity for syntax relative to word meanings throughout the language network
Fedorenko, E., Blank, I. A., Siegelman, M., & Mineroff, Z. (2020) · Cognition · doi:10.1016/j.cognition.2020.104348 · Cited by 170 · see citing works
Self-related awareness and emotion regulation
Herwig, U., Kaffenberger, T., Jäncke, L., & Brühl, A. B. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2009.12.089 · Cited by 189 · see citing works
Weber, M. J., Messing, S. B., Rao, H., Detre, J. A., & Thompson‐Schill, S. L. (2014) · Human Brain Mapping · doi:10.1002/hbm.22429 · Cited by 183 · see citing works
Kim, H., Shimojo, S., & O'Doherty, J. P. (2011) · Cerebral Cortex · doi:10.1093/cercor/bhq145 · Cited by 176 · see citing works
Reduced Structural Connectivity of a Major Frontolimbic Pathway in Generalized Anxiety Disorder
Tromp, D. P. M., Grupe, D. W., Oathes, D. J., McFarlin, D. R., Hernandez, P. J., Kral, T. R. A., Lee, J. E., Adams, M., Alexander, A. L., & Nitschke, J. B. (2012) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2011.2178 · Cited by 184 · see citing works
A power calculation guide for fMRI studies
Mumford, J. A. (2012) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nss059 · Cited by 171 · see citing works
I tried a bunch of things: The dangers of unexpected overfitting in classification of brain data
Hosseini, M., Powell, M., Collins, J., Callahan-Flintoft, C., Jones, W., Bowman, H., & Wyble, B. (2020) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.09.036 · Cited by 165 · see citing works
Interactions between cognition and motivation during response inhibition
Padmala, S., & Pessoa, L. (2010) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2009.10.017 · Cited by 162 · see citing works
A new fun and robust version of an fMRI localizer for the frontotemporal language system
Scott, T. L., Gallée, J., & Fedorenko, E. (2017) · Cognitive Neuroscience · doi:10.1080/17588928.2016.1201466 · Cited by 125 · see citing works
Chau, B. K. H., Sallet, J., Papageorgiou, G. K., Noonan, M. P., Bell, A. H., Walton, M. E., & Rushworth, M. F. S. (2015) · Neuron · doi:10.1016/j.neuron.2015.08.018 · Cited by 159 · see citing works
Pereira, M. G., de Oliveira, L., Erthal, F. S., Joffily, M., Mocaiber, I. F., Volchan, E., & Pessoa, L. (2010) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/cabn.10.1.94 · Cited by 151 · see citing works
Human Scalp Electroencephalography Reveals that Repetition Suppression Varies with Expectation
Summerfield, C., Wyart, V., Johnen, V. M., & de Gardelle, V. (2011) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2011.00067 · Cited by 145 · see citing works
Multivoxel neurofeedback selectively modulates confidence without changing perceptual performance
Cortese, A., Amano, K., Koizumi, A., Kawato, M., & Lau, H. (2016) · Nature Communications · doi:10.1038/ncomms13669 · Cited by 157 · see citing works
Savitz, J., Frank, M. B., Victor, T., Bebak, M., Marino, J. H., Bellgowan, P. S. F., McKinney, B. A., Bodurka, J., Kent Teague, T., & Drevets, W. C. (2013) · Brain, Behavior, and Immunity · doi:10.1016/j.bbi.2012.10.007 · Cited by 146 · see citing works
Self-citationNeural basis of contagious itch and why some people are more prone to it
Holle, H., Warne, K., Seth, A. K., Critchley, H. D., & Ward, J. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1216160109 · Cited by 145 · see citing works
Impact of state anxiety on the interaction between threat monitoring and cognition
Choi, J. M., Padmala, S., & Pessoa, L. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2011.08.102 · Cited by 137 · see citing works
Hedden, T., & Gabrieli, J. D. E. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.01.089 · Cited by 146 · see citing works
Batch effects removal for microbiome data via conditional quantile regression
Ling, W., Lu, J., Zhao, N., Lulla, A., Plantinga, A. M., Fu, W., Zhang, A., Liu, H., Song, H., Li, Z., et al. (2022) · Nature Communications · doi:10.1038/s41467-022-33071-9 · Cited by 136 · see citing works
How to control for confounds in decoding analyses of neuroimaging data
Snoek, L., Miletić, S., & Scholte, H. S. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2018.09.074 · Cited by 143 · see citing works
Christensen, J. C., Estepp, J. R., Wilson, G. F., & Russell, C. A. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2011.07.091 · Cited by 144 · see citing works
Indovina, I., Riccelli, R., Chiarella, G., Petrolo, C., Augimeri, A., Giofrè, L., Lacquaniti, F., Staab, J. P., & Passamonti, L. (2015) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2015.00334 · Cited by 140 · see citing works
Automatic versus Choice-Dependent Value Representations in the Human Brain
Grueschow, M., Polania, R., Hare, T. A., & Ruff, C. C. (2015) · Neuron · doi:10.1016/j.neuron.2014.12.054 · Cited by 130 · see citing works
Kassraian-Fard, P., Matthis, C., Balsters, J. H., Maathuis, M. H., & Wenderoth, N. (2016) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2016.00177 · Cited by 132 · see citing works
Wang, Y., Wang, J., Jia, Y., Zhong, S., Zhong, M., Sun, Y., Niu, M., Zhao, L., Zhao, L., Pan, J., et al. (2017) · Translational Psychiatry · doi:10.1038/tp.2017.117 · Cited by 133 · see citing works
Krebs, R. M., Heipertz, D., Schuetze, H., & Duzel, E. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.06.038 · Cited by 135 · see citing works
Multivariate data analysis for neuroimaging data: overview and application to Alzheimer’s disease
Initiative, T. A. D. N., Habeck, C., & Stern, Y. (2010) · Cell Biochemistry and Biophysics · doi:10.1007/s12013-010-9093-0 · Cited by 127 · see citing works
Identifying Neuroimaging Markers of Motor Disability in Acute Stroke by Machine Learning Techniques
Rehme, A. K., Volz, L. J., Feis, D.-L., Bomilcar-Focke, I., Liebig, T., Eickhoff, S. B., Fink, G. R., & Grefkes, C. (2015) · Cerebral Cortex · doi:10.1093/cercor/bhu100 · Cited by 117 · see citing works
Gershman, S. J., Pesaran, B., & Daw, N. D. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.2469-09.2009 · Cited by 118 · see citing works
Integrative frontal-parietal dynamics supporting cognitive control
Nee, D. E. (2021) · eLife · doi:10.7554/elife.57244 · Cited by 112 · see citing works
Cooper, J. C., Dunne, S., Furey, T., & O'Doherty, J. P. (2012) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00114 · Cited by 101 · see citing works
Goldstein-Piekarski, A. N., Staveland, B. R., Ball, T. M., Yesavage, J., Korgaonkar, M. S., & Williams, L. M. (2018) · Translational Psychiatry · doi:10.1038/s41398-018-0100-3 · Cited by 108 · see citing works
Real-world stress resilience is associated with the responsivity of the locus coeruleus
Grueschow, M., Stenz, N., Thörn, H., Ehlert, U., Breckwoldt, J., Brodmann Maeder, M., Exadaktylos, A. K., Bingisser, R., Ruff, C. C., & Kleim, B. (2021) · Nature Communications · doi:10.1038/s41467-021-22509-1 · Cited by 116 · see citing works
Using fMRI to study reward processing in humans: past, present, and future
Wang, K. S., Smith, D. V., & Delgado, M. R. (2016) · Journal of Neurophysiology · doi:10.1152/jn.00333.2015 · Cited by 105 · see citing works
Madore, K. P., Szpunar, K. K., Addis, D. R., & Schacter, D. L. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1612278113 · Cited by 104 · see citing works
Subjective confidence reflects representation of Bayesian probability in cortex
Geurts, L. S., Cooke, J. R. H., van Bergen, R. S., & Jehee, J. F. M. (2022) · Nature Human Behaviour · doi:10.1038/s41562-021-01247-w · Cited by 104 · see citing works
- Authors
- Kriegeskorte, N., Simmons, W. K., Bellgowan, P. S. F., & Baker, C. I.
- Affiliations
- National Institutes of Health (US)
- Corresponding author
- Kriegeskorte, Nikolaus
- Baker, Chris I
- Venue
- Nature Neuroscience
- Published
- 2009-04-26
- Volume
- 12
- Issue
- 5
- Pages
- 535-540
- ISSN
- 1097-6256 (ISSN-L)
- 1546-1726
- PMC
- PMC2841687
- OpenAlex
- W2015866962
- Cited by
- 2,696
- References
- 43
- Citations by year
- 2026: 139 · 2025: 102 · 2024: 116 · 2023: 136 · 2022: 130 · 2021: 166 · 2020: 169 · 2019: 178 · 2018: 172 · 2017: 201 · 2016: 178 · 2015: 164 · 2014: 190 · 2013: 157 · 2012: 152
- FWCI
- 79.69
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Nature Neuroscience (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institutes of Health
- National Institute of Mental Health (ZIAMH002909)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Functional Brain Connectivity Studies
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Applications
- Life Sciences › Neuroscience › Cognitive Neuroscience › Undergraduate Neuroscience Education and Research
- Keywords
- double dipping, data selection, spurious results, statistical inference validity, neuroimaging, electrophysiology, multivariate pattern analysis, noise data
- MeSH terms
- Animals, Data Collection, Data Interpretation, Statistical, Humans, Image Processing, Computer-Assisted, Magnetic Resonance Imaging, Neurosciences, Reproducibility of Results, Selection Bias, Artifacts, Systems Biology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
