Introduction to the special issue on reliability and replication in cognitive and affective neuroscience research
Open accessThe claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Using a common analytic approach across multiple informants, time points, and developmental domains can address concerns about the robustness of scientific findings.
“By using a common approach to analyze data from multiple informants, time points, and developmental domains, we can address concerns about the robustness of scientific findings (20,21).”
From A Comprehensive Multilevel Analysis of the Bucharest Early Intervention Project: Causal Effects on Recovery From Early Severe Deprivation (2023) · cites this paper as [20]
The introduction (page not supplied) links improved statistical analysis, assessment of analytic choices, and aggregation across datasets to more reliable and robust findings. It does not discuss applying one common approach across multiple informants, time points, and developmental domains; its references to domains concern cognitive and affective neuroscience, not developmental domains. (LLM, conf=0.85)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505 · see citing works
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · cited 3× · see citing works
“, 2011), and ubiquitous low power (Button et al., 2013).”
BackgroundSimmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567 · see citing works
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · see citing works
Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
John, L. K., Loewenstein, G., & Prelec, D. (2012) · Psychological Science · doi:10.1177/0956797611430953 · Cited by 2,239 · see citing works
Estimating sample size in functional MRI (fMRI) neuroimaging studies: Statistical power analyses
Desmond, J. E., & Glover, G. H. (2002) · Journal of Neuroscience Methods · doi:10.1016/s0165-0270(02)00121-8 · Cited by 716 · see citing works
Is the Replicability Crisis Overblown? Three Arguments Examined
Pashler, H., & Harris, C. R. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612463401 · Cited by 697 · see citing works
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2012) · doi:10.2139/ssrn.2160588 · Cited by 680 · see citing works
Guidelines for reporting an fMRI study
Poldrack, R. A., Fletcher, P. C., Henson, R. N., Worsley, K. J., Brett, M., & Nichols, T. E. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2007.11.048 · Cited by 550 · see citing works
Yarkoni, T. (2009) · Perspectives on Psychological Science · doi:10.1111/j.1745-6924.2009.01127.x · Cited by 549 · see citing works
Self-citationThe secret lives of experiments: Methods reporting in the fMRI literature
Carp, J. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.07.004 · Cited by 473 · see citing works
The (mis)reporting of statistical results in psychology journals
Bakker, M., & Wicherts, J. M. (2011) · Behavior Research Methods · doi:10.3758/s13428-011-0089-5 · Cited by 356 · see citing works
Ten ironic rules for non-statistical reviewers
Friston, K. J. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.04.018 · Cited by 347 · see citing works
A peculiar prevalence of <i>p</i> values just below .05
Masicampo, E. J., & Lalande, D. R. (2012) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470218.2012.711335 · Cited by 266 · see citing works
Vul, E., & Pashler, H. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.01.027 · Cited by 147 · see citing works
Ioannidis, J. P. A. (2011) · Scientific American · doi:10.1038/scientificamerican0611-16 · Cited by 67 · see citing works
Ioannidis, J. P. A. (2011) · PubMed · Cited by 56 · see citing works
Barbero (2014) · doi:10.1016/j · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
The Extent and Consequences of P-Hacking in Science
Head, M. L., Holman, L., Lanfear, R., Kahn, A. T., & Jennions, M. D. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002106 · Cited by 1,388 · see citing works
“There is increasing concern that many published results are false positives [1,2] (but see [3]).”
BackgroundUnreliability as a threat to understanding psychopathology: The cautionary tale of attentional bias.
Rodebaugh, T. L., Scullin, R. B., Langer, J. K., Dixon, D. J., Huppert, J. D., Bernstein, A., Zvielli, A., & Lenze, E. J. (2016) · Journal of Abnormal Psychology · doi:10.1037/abn0000184 · Cited by 312 · see citing works
Larson, M. J., & Carbine, K. A. (2016) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2016.06.015 · Cited by 148 · see citing works
Fibromyalgia Is Associated with Decreased Connectivity Between Pain- and Sensorimotor Brain Areas
Flodin, P., Martinsen, S., Löfgren, M., Bilevičiūtė-Ljungar, I., Kosek, E., & Fransson, P. (2014) · Brain Connectivity · doi:10.1089/brain.2014.0274 · Cited by 124 · see citing works
Chen, X., Chen, N.-X., Shen, Y., Li, H., Li, L., Lu, B., Zhu, Z.-C., Fan, Z., & Yan, C. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.117185 · Cited by 110 · see citing works
Resting state connectivity of the bed nucleus of the stria terminalis at ultra‐high field
Torrisi, S., O’Connell, K., Davis, A., Reynolds, R. C., Balderston, N. L., Fudge, J. L., Grillon, C., & Monique Ernst, D. (2015) · Human Brain Mapping · doi:10.1002/hbm.22899 · Cited by 98 · see citing works
Flodin, P., Martinsen, S., Altawil, R., Waldheim, E., Lampa, J., Kosek, E., & Fransson, P. (2016) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2016.00107 · Cited by 89 · see citing works
Power-up: A Reanalysis of 'Power Failure' in Neuroscience Using Mixture Modeling
Nord, C. L., Valton, V., Wood, J., & Roiser, J. P. (2017) · Journal of Neuroscience · doi:10.1523/jneurosci.3592-16.2017 · Cited by 87 · see citing works
Towards robust and replicable sex differences in the intrinsic brain function of autism
Floris, D. L., Filho, J. O. A., Lai, M., Giavasis, S., Oldehinkel, M., Mennes, M., Charman, T., Tillmann, J., Dumas, G., Ecker, C., et al. (2021) · Molecular Autism · doi:10.1186/s13229-021-00415-z · Cited by 77 · see citing works
The reliability of continuous brain responses during naturalistic listening to music
Burunat, I., Toiviainen, P., Alluri, V., Bogert, B., Ristaniemi, T., Sams, M. E., & Brattico, E. (2015) · NeuroImage · doi:10.1016/j.neuroimage.2015.09.005 · Cited by 64 · see citing works
Neuroimaging of individual differences: A latent variable modeling perspective
Cooper, S. R., Jackson, J. J., Barch, D. M., & Braver, T. S. (2019) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2018.12.022 · Cited by 57 · see citing works
Self-citationA critique on neuroscientific methodologies in organizational behavior and management studies
Lindebaum, D., & Jordan, P. J. (2014) · Journal of Organizational Behavior · doi:10.1002/job.1940 · Cited by 57 · see citing works
Effects of single-session transcranial direct current stimulation on reactive response inhibition
Friehs, M. A., Frings, C., & Hartwigsen, G. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2021.07.013 · Cited by 56 · see citing works
Tseng, W., Thomas, L. A., Harkins, E., Pine, D. S., Leibenluft, E., & Brotman, M. A. (2015) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsv087 · Cited by 44 · see citing works
The “Reproducibility Crisis:” Might the Methods Used Frequently in Behavior-Analysis Research Help?
Branch, M. (2018) · Perspectives on Behavior Science · doi:10.1007/s40614-018-0158-5 · Cited by 39 · see citing works
Multimodal neural correlates of cognitive control in the Human Connectome Project
Lerman-Sinkoff, D. B., Jing Sui （Beijing Normal University）， my correct affiliation is beijing normal university, N. Q. U. O. S. A. T., Rachakonda, S., Kandala, S., Calhoun, V. D., & Barch, D. M. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.08.081 · Cited by 42 · see citing works
Self-citationHedonic Hotspots Regulate Cingulate-driven Adaptation to Cognitive Demands
van Steenbergen, H., Band, G. P. H., Hommel, B., Rombouts, S. A. R. B., & Nieuwenhuis, S. (2014) · Cerebral Cortex · doi:10.1093/cercor/bht416 · Cited by 41 · see citing works
Computational Language Modeling and the Promise of In Silico Experimentation
Jain, S., Vo, V. A., Wehbe, L., & Huth, A. G. (2023) · Neurobiology of Language · doi:10.1162/nol_a_00101 · Cited by 37 · see citing works
Adolescent maturation of the relationship between cortical gyrification and cognitive ability
Chung, Y. S., Hyatt, C. J., & Stevens, M. C. G. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.06.082 · Cited by 40 · see citing works
At the core of reasoning: Dissociating deductive and non‐deductive load
Coetzee, J. P., & Monti, M. M. (2018) · Human Brain Mapping · doi:10.1002/hbm.23979 · Cited by 40 · see citing works
Human torque is not present in chimpanzee brain
Li, X., Crow, T. J., Hopkins, W. D., Gong, Q., & Roberts, N. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.10.017 · Cited by 37 · see citing works
Do Neuroscience Journals Accept Replications? A Survey of Literature
Yeung, A. W. K. (2017) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2017.00468 · Cited by 35 · see citing works
Cognitive Control Promotes Either Honesty or Dishonesty, Depending on One's Moral Default
Speer, S. P. H., Smidts, A., & Boksem, M. A. S. (2021) · Journal of Neuroscience · doi:10.1523/jneurosci.0666-21.2021 · Cited by 36 · see citing works
Grand Field Challenges for Cognitive Neuroergonomics in the Coming Decade
Gramann, K., McKendrick, R. D., Baldwin, C. L., Roy, R. N., Jeunet, C., Mehta, R. K., & Vecchiato, G. (2021) · Frontiers in Neuroergonomics · doi:10.3389/fnrgo.2021.643969 · Cited by 34 · see citing works
Hillman, C., Fontana, B. D., Amstislavskaya, T. G., Gorbunova, M. A., Altenhofen, S., Barthelson, K., Bastos, L. M., Borba, J. V., Bonan, C. D., Brennan, C. H., et al. (2025) · Lab Animal · doi:10.1038/s41684-025-01548-x · Cited by 34 · see citing works
Prior, M., Hibberd, R., Asemota, N., & Thornton, J. (2017) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.14506 · Cited by 32 · see citing works
King, L. S., Guyon-Harris, K. L., Valadez, E. A., Radulescu, A., Fox, N. A., Nelson, C. A., Zeanah, C. H., & Humphreys, K. L. (2023) · American Journal of Psychiatry · doi:10.1176/appi.ajp.20220672 · Cited by 29 · see citing works
Open-source raspberry Pi-based operant box for translational behavioral testing in rodents
Buscher, N., Ojeda, A., Francoeur, M. J., Hulyalkar, S., Claros, C., Tang, T., Terry, A., Gupta, A., Fakhraei, L., & Ramanathan, D. (2020) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2020.108761 · Cited by 29 · see citing works
Hervais‐Adelman, A., Moser‐Mercer, B., & Golestani, N. (2018) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2018.00022 · Cited by 28 · see citing works
What Has Replication Ever Done for Us? Insights from Neuroimaging of Speech Perception
Evans, S. (2017) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2017.00041 · Cited by 21 · see citing works
Replicability in Brain Imaging
Kelly, R. E., & Hoptman, M. J. (2022) · Brain Sciences · doi:10.3390/brainsci12030397 · Cited by 19 · see citing works
Advancing the Cognitive Science of Religion through Replication and Open Science
Hoogeveen, S., & van Elk, M. (2020) · Journal for the Cognitive Science of Religion · doi:10.1558/jcsr.39039 · Cited by 19 · see citing works
Do tasks matter in task switching? Dissociating domain-general from context-specific brain activity
Muhle-Karbe, P. S., Baene, W. D., & Braß, M. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.05.058 · Cited by 21 · see citing works
Psychologists’ psychologies of psychologists in a time of crisis.
Morawski, J. G. (2020) · History of Psychology · doi:10.1037/hop0000140 · Cited by 20 · see citing works
Analytical Transparency and Reproducibility in Human Neuroimaging Studies
Picciotto, M. R. (2018) · Journal of Neuroscience · doi:10.1523/jneurosci.0424-18.2018 · Cited by 17 · see citing works
The Brain Disorders Debate, Chekhov, and Mental Health Humanities
Valtonen, J., & Lewis, B. E. (2023) · Journal of Medical Humanities · doi:10.1007/s10912-023-09786-1 · Cited by 15 · see citing works
Investigating the Roles of Neuroscience and Knowledge Management in Higher Education
Kasemsap, K. (2017) · Advances in educational marketing, administration, and leadership book series · doi:10.4018/978-1-5225-0672-0.ch006 · Cited by 17 · see citing works
Takagi, Y., Hirayama, J.-I., & Tanaka, S. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.116036 · Cited by 11 · see citing works
White matter correlates of sensorimotor synchronization in persistent developmental stuttering
Jossinger, S., Sares, A. G., Zislis, A., Sury, D., Gracco, V. L., & Ben‐Shachar, M. (2021) · Journal of Communication Disorders · doi:10.1016/j.jcomdis.2021.106169 · Cited by 11 · see citing works
Reporting checklists in neuroimaging: promoting transparency, replicability, and reproducibility
Ekhtiari, H., Zare-Bidoky, M., Sangchooli, A., Valyan, A., Abi‐Dargham, A., Cannon, D. M., Carter, C. S., Garavan, H., George, T. P., Ghobadi‐Azbari, P., et al. (2024) · Neuropsychopharmacology · doi:10.1038/s41386-024-01973-5 · Cited by 13 · see citing works
Towards robust and replicable sex differences in the intrinsic brain function of autismPreprint
Floris, D. L., Filho, J. O. A., Lai, M., Giavasis, S., Oldehinkel, M., Mennes, M., Charman, T., Tillmann, J., Dumas, G., Ecker, C., et al. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.06.09.142471 · Cited by 10 · see citing works
Cade, A. E., Jones, K., Holt, K. R., Penkar, A. M., & Haavik, H. (2021) · Brain Sciences · doi:10.3390/brainsci11081047 · Cited by 9 · see citing works
Neuroanatomical correlates of extraversion
Li, M., Wei, D., Yang, W., Zhang, J., & Qiu, J. (2019) · Neuroreport · doi:10.1097/wnr.0000000000001306 · Cited by 8 · see citing works
Reduced Threat-Related Neural Efficiency: A Possible Biomarker for Pediatric Anxiety Disorders
Linke, J. O., Naim, R., Haller, S. P., Khosravi, P., Scheinberg, B., Byrne, M. E., Harrewijn, A., Leibenluft, E., Brotman, M. A., Winkler, A. M., et al. (2025) · American Journal of Psychiatry · doi:10.1176/appi.ajp.20241043 · Cited by 7 · see citing works
Patients participating to neurobiological research in early psychosis: A selected subgroup?
Golay, P., Baumann, P., Jenni, R., Kim, Q., & Conus, P. (2018) · Schizophrenia Research · doi:10.1016/j.schres.2018.04.033 · Cited by 6 · see citing works
Hormones and close relationship processes
Edelstein, R. S., & Chin, K. A. (2018) · Routledge eBooks · doi:10.4324/9781315200439-17 · Cited by 6 · see citing works
Neural Correlates of Social Influence Among Cannabis Users
Gilman, J. M. (2017) · Current Addiction Reports · doi:10.1007/s40429-017-0141-3 · Cited by 6 · see citing works
Transcending humanness or: Doing the right thing for science
Inzlicht, M. (2018) · Cortex · doi:10.1016/j.cortex.2018.11.032 · Cited by 5 · see citing works
Működési zavarok és megújulási törekvések a tudományos pszichológiában, a 21. század elején
Ágnes, S. (2018) · Magyar Pszichológiai Szemle · doi:10.1556/0016.2018.73.4.3 · Cited by 3 · see citing works
What Explains Consciousness? Or...What Consciousness Explains?
Dulany, D. E. (2014) · Mens Sana monographs · doi:10.4103/0973-1229.130283 · Cited by 2 · see citing works
Verhoeven, L. T. W., Li, X., Ogunniran, M. O., Wu, R., Chakraborty, A., Vickers, E., Gupta, L., & Macintyre, T. (2022) · UNESCO MGIEP eBooks · doi:10.56383/nfal3758 · Cited by 1 · see citing works
Development of aspect in children’s narrations in German Sign Language
Kolbe, V. (2025) · Sign Language & Linguistics · doi:10.1075/sll.24017.kol · Cited by 1 · see citing works
Licata, A. E., Bouffier, M., & Borghesani, V. (2026) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2026.106591 · Cited by 1 · see citing works
Neuroimaging studies of extra-sensory perception: A systematic and critical review
Acunzo, D. J., Denton, A. H., Weiler, M., & Kelly, E. F. (2026) · NeuroImage · doi:10.1016/j.neuroimage.2026.121937 · Cited by 1 · see citing works
Elsayed, M., Deng, W. Q., Belisario, K. L., & MacKillop, J. M. (2026) · Cerebral Cortex · doi:10.1093/cercor/bhag037 · Cited by 1 · see citing works
Establishing Effective Measurement Strategy for Mental Health ResearchPreprint
Mattek, S. (2022) · doi:10.31234/osf.io/ud3qt · Cited by 1 · see citing works
Bos, M. G. N. (2014) · UvA-DARE (University of Amsterdam)
Tompson, S. (2016) · Deep Blue (University of Michigan)
Individual Differences in Social Cognition and Behavior: a Personality Psychology Framework
Blain, S. D. (2021) · University of Minnesota Digital Conservancy (University of Minnesota)
Lies, Damned Lies, and Crafty Questionnaire Design
Kampen, J. K., van Dam, Y. K., & Platje, J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-662-66597-8_4
Magnetic Resonance Imaging (MRI)
Hervais‐Adelman, A., & Babcock, L. (2025) · Research methods in applied linguistics · doi:10.1075/rmal.10.13her
How Robust Are fMRI- and EEG-Based Representational Similarity Analysis?
Sarma, S. S., Boruah, G., & Srivastava, N. (2025) · Computational Brain & Behavior · doi:10.1007/s42113-025-00257-5
Floris, D. L., Filho, J. O. A., Lai, M.-C., Giavasis, S., Oldehinkel, M., Mennes, M., Charman, T., Tillmann, J., Dumas, G., Ecker, C., et al. (2021) · Figshare · doi:10.6084/m9.figshare.14140167
Floris, D. L., Filho, J. O. A., Lai, M.-C., Giavasis, S., Oldehinkel, M., Mennes, M., Charman, T., Tillmann, J., Dumas, G., Ecker, C., et al. (2021) · Figshare · doi:10.6084/m9.figshare.14140161
Floris, D. L., Filho, J. O. A., Lai, M.-C., Giavasis, S., Oldehinkel, M., Mennes, M., Charman, T., Tillmann, J., Dumas, G., Ecker, C., et al. (2021) · Figshare · doi:10.6084/m9.figshare.14140164
The Hypothesis Race Model for evaluation of research findingsPreprint
Kelly, R. E. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.05.28.728385
- Authors
- Barch, D. M., & Yarkoni, T.
- Affiliations
- Washington University in St. Louis (US)
- University of Colorado Boulder (US)
- Corresponding author
- Barch, Deanna M.
- Venue
- Cognitive, Affective, & Behavioral Neuroscience
- Published
- 2013-08-07
- Volume
- 13
- Issue
- 4
- Pages
- 687-689
- ISSN
- 1530-7026 (ISSN-L)
- 1531-135X
- OpenAlex
- W2064628090
- Cited by
- 67
- References
- 17
- Citations by year
- 2026: 4 · 2025: 5 · 2024: 1 · 2023: 4 · 2022: 4 · 2021: 10 · 2020: 5 · 2019: 3 · 2018: 8 · 2017: 8 · 2016: 5 · 2015: 4 · 2014: 6
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Cognitive Affective & Behavioral Neuroscience (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Life Sciences › Neuroscience › Cognitive Neuroscience › Functional Brain Connectivity Studies
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- replication crisis, reliability statistics, functional neuroimaging, cognitive processes, emotional processing, statistical power, sample size, false positives, false-negative results, questionable research practices, p-hacking, reporting practices, stimulus presentation timing, effect size inflation, statistical analysis methods, conflicts of interest, reproducibility, reproducible research, reproducible analysis
- MeSH terms
- Affect, Cognition, Humans, Neurosciences, Research, Reproducibility of Results
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
