Nonreplicable publications are cited more than replicable ones

Serra-Garcia, Marta; Gneezy, Uri

doi:10.1126/sciadv.abd1705

Cited by 177scientometrics and bibliometrics researchPsychology Research and BibliometricsAcademic Publishing and Open Access

Nonreplicable publications are cited more than replicable ones

Open access

Serra-Garcia, Marta, Gneezy, Uri·2021·Science Advances

DOI 10.1126/sciadv.abd1705PMC PMC8139580

Open this paper in CoChat — free account

Abstract

We use publicly available data to show that published papers in top psychology, economics, and general interest journals that fail to replicate are cited more than those that replicate. This difference in citation does not change after the publication of the failure to replicate. Only 12% of postreplication citations of nonreplicable findings acknowledge the replication failure. Existing evidence also shows that experts predict well which papers will be replicated. Given this prediction, why are nonreplicable papers accepted for publication in the first place? A possible answer is that the review team faces a trade-off. When the results are more "interesting," they apply lower standards regarding their reproducibility.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1126/sciadv.abd1705

Cites

26 of 26 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Non-replicable findings are cited more than replicable ones.
“Non-replicable findings are cited more than replicable ones,1”
From Leakage and the reproducibility crisis in machine-learning-based science (2023) · cites this paper as [1]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

1 retracted

RETRACTED: Declines in Religiosity Predict Increases in Violent Crime—but Not Among Countries With Relatively High Average IQ
Clark, C. J., Winegard, B., Beardslee, J., Baumeister, R. F., & Shariff, A. (2020) · Psychological Science · doi:10.1177/0956797619897915 · Cited by 16 · see citing works
Retracted
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515 · see citing works
Generalized Linear Models
Nelder, J. A., & Wedderburn, R. W. M. (1972) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2344614 · Cited by 7,321 · see citing works
False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567 · see citing works
Regression Analysis of Count Data: Author Index
Cameron, A., & Trivedi, P. (1998) · Cambridge University Press eBooks · doi:10.1017/cbo9780511814365 · Cited by 6,406 · see citing works
Redefine statistical significance
Benjamin, D. J., Berger, J. O., Johannesson, M., Nosek, B. A., Wagenmakers, E.-J., Berk, R., Bollen, K. A., Brembs, B., Brown, L., Camerer, C., et al. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0189-z · Cited by 2,772 · see citing works
Promoting an open research culture
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., et al. (2015) · Science · doi:10.1126/science.aab2374 · Cited by 2,708 · see citing works
Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015
Camerer, C. F., Dreber, A., Holzmeister, F., Ho, T.-H., Huber, J., Johannesson, M., Kirchler, M., Nave, G., Nosek, B. A., Pfeiffer, T., et al. (2018) · Nature Human Behaviour · doi:10.1038/s41562-018-0399-z · Cited by 1,521 · see citing works
Evaluating replicability of laboratory experiments in economics
Camerer, C. F., Dreber, A., Forsell, E., Ho, T.-H., Huber, J., Johannesson, M., Kirchler, M., Almenberg, J., Altmejd, A., Chan, T., et al. (2016) · Science · doi:10.1126/science.aaf0918 · Cited by 1,314 · see citing works
Investigating Variation in Replicability: A “Many Labs” Replication Project
Klein, R. A., Ratliff, K. A., Vianello, M., Adams, R. B., Bahník, Š., Bernstein, M. J., Bocian, K., Brandt, M. J., Brooks, B., Brumbaugh, C. C., et al. (2014) · Social Psychology · doi:10.1027/1864-9335/a000178 · Cited by 1,208 · see citing works
Many Labs 2: Investigating variation in replicability across samples and settings
Klein, R. A., Vianello, M., Hasselman, F., Adams, B. G., Adams, R. B., Alper, S., Aveyard, M., Axt, J. R., Babalola, M. T., Bahník, Š., et al. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245918810225 · Cited by 944 · see citing works
Specification curve analysis
Simonsohn, U., Simmons, J. P., & Nelson, L. D. (2020) · Nature Human Behaviour · doi:10.1038/s41562-020-0912-z · Cited by 651 · see citing works
Many Labs 3: Evaluating participant pool quality across the academic semester via replication
Ebersole, C. R., Atherton, O. E., Belanger, A. L., Skulborstad, H. M., Allen, J. M., Banks, J. B., Baranski, E., Bernstein, M. J., Bonfiglio, D. B. V., Boucher, L., et al. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2015.10.012 · Cited by 468 · see citing works
Measuring the effectiveness of scientific gatekeeping
Siler, K., Lee, K. P., & Bero, L. A. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1418218112 · Cited by 338 · see citing works
Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics
Brodeur, A., Cook, N., & Heyes, A. (2020) · American Economic Review · doi:10.1257/aer.20190687 · Cited by 333 · see citing works
Using prediction markets to estimate the reproducibility of scientific research
Dreber, A., Pfeiffer, T., Almenberg, J., Isaksson, S., Wilson, B., Chen, Y., Nosek, B. A., & Johannesson, M. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1516179112 · Cited by 331 · see citing works
The Retraction Penalty: Evidence from the Web of Science
Lu, S. F., Jin, G. Z., Uzzi, B., & Jones, B. (2013) · Scientific Reports · doi:10.1038/srep03146 · Cited by 191 · see citing works
Specification curve analysisPreprint
Schweinsberg, M., Amireh, H., Tierney, W., Mohamed, Z., & Uhlmann, E. L. (2019) · Cited by 138 · see citing works
What Do Editors Maximize? Evidence from Four Economics Journals
Card, D., & DellaVigna, S. (2020) · The Review of Economics and Statistics · doi:10.1162/rest_a_00839 · Cited by 123 · see citing works
Estimating the deep replicability of scientific findings using human and artificial intelligence
Yang, Y., Youyou, W., & Uzzi, B. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1909046117 · Cited by 99 · see citing works
Predicting replication outcomes in the Many Labs 2 study
Forsell, E., Viganola, D., Pfeiffer, T., Almenberg, J., Wilson, B., Chen, Y.-L., Nosek, B. A., Johannesson, M., & Dreber, A. (2018) · Journal of Economic Psychology · doi:10.1016/j.joep.2018.10.009 · Cited by 97 · see citing works
Predicting the replicability of social science lab experiments
Altmejd, A., Dreber, A., Forsell, E., Huber, J., Imai, T., Johannesson, M., Kirchler, M., Nave, G., & Camerer, C. F. (2019) · PLoS ONE · doi:10.1371/journal.pone.0225826 · Cited by 89 · see citing works
Specification Searching and Significance Inflation Across Time, Methods and Disciplines
Vivalt, E. (2019) · Oxford Bulletin of Economics and Statistics · doi:10.1111/obes.12289 · Cited by 58 · see citing works
Association between contextual dependence and replicability in psychology may be spurious
Inbar, Y. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1608676113 · Cited by 56 · see citing works
Science Fictions: How Fraud, Bias, Negligence, and Hype Undermine the Search for Truth
Ritchie, S. J. (2020) · Cited by 50 · see citing works
Probabilistic forecasting of replication studies
Pawel, S., & Held, L. (2020) · PLoS ONE · doi:10.1371/journal.pone.0231416 · Cited by 32 · see citing works

Flagged first, then by citations

Research works printed but not linked: 2

[25]A. C. Cameron, P. K. Trivedi, Regression Analysis of Count Data (Cambridge Univ. Press, ed. 2, 1998).
[27]Measuring your Research Impact: Journal Citation Reports (April 9, 2020); https://ucsd.libguides.com/ResearchImpact/JCR.

Sorted by citations

Leakage and the reproducibility crisis in machine-learning-based science
Kapoor, S., & Narayanan, A. (2023) · Patterns · doi:10.1016/j.patter.2023.100804 · Cited by 1,387 · see citing works
Social physics
Jusup, M., Holme, P., Kanazawa, K., Takayasu, M., Romić, I., Wang, Z., Geček, S., Lipić, T., Boris Podobnik, A., Wang, L., et al. (2022) · Physics Reports · doi:10.1016/j.physrep.2021.10.005 · Cited by 662 · see citing works
REFORMS: Consensus-based Recommendations for Machine-learning-based Science
Kapoor, S., Cantrell, E. M., Peng, K., Pham, T. H., Bail, C. A., Gundersen, O. E., Hofman, J. M., Hullman, J. R., Lones, M. A., Malik, M. M., et al. (2024) · Science Advances · doi:10.1126/sciadv.adk3452 · Cited by 192 · see citing works
The psychological distance of climate change is overestimated
van Valkengoed, A. M., Steg, L., & Perlaviciute, G. (2023) · One Earth · doi:10.1016/j.oneear.2023.03.006 · Cited by 127 · see citing works
Citation counts and journal impact factors do not capture some indicators of research quality in the behavioural and brain sciences
Dougherty, M. R. P., & Horne, Z. (2022) · Royal Society Open Science · doi:10.1098/rsos.220334 · Cited by 137 · see citing works
Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equations
McGreivy, N., & Hakim, A. H. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00897-5 · Cited by 117 · see citing works
Open and reproducible neuroimaging: From study inception to publication
Niso, G., Botvinik‐Nezer, R., Appelhoff, S., de la Vega, A. I., Estéban, O., Etzel, J. A., Finc, K., Ganz, M., Gau, R., Halchenko, Y. O., et al. (2022) · NeuroImage · doi:10.1016/j.neuroimage.2022.119623 · Cited by 104 · see citing works
Beyond the “Third Wave of Positive Psychology”: Challenges and Opportunities for Future Research
Wissing, M. P. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.795067 · Cited by 99 · see citing works
A discipline-wide investigation of the replicability of Psychology papers over the past two decades
Youyou, W., Yang, Y., & Uzzi, B. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2208863120 · Cited by 87 · see citing works
The pandemic that shocked managers across the world: The impact of the COVID-19 crisis on leadership behavior
Garretsen, H., Stoker, J. I., Soudis, D., & Wendt, H. (2022) · The Leadership Quarterly · doi:10.1016/j.leaqua.2022.101630 · Cited by 73 · see citing works
Ten Points to Improve Reproducibility and Translation of Animal Research
Spanagel, R. (2022) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2022.869511 · Cited by 72 · see citing works
The performance of socially responsible investments: A meta‐analysis
Hornuf, L., & Yüksel, G. (2023) · European Financial Management · doi:10.1111/eufm.12439 · Cited by 66 · see citing works
Where Are the Self-Correcting Mechanisms in Science?
Vazire, S., & Holcombe, A. O. (2022) · Review of General Psychology · doi:10.1177/10892680211033912 · Cited by 81 · see citing works
Sodium and Health: Old Myths and a Controversy Based on Denial
Cappuccio, F. P., Campbell, N. R. C., He, F. J., Jacobson, M. F., MacGregor, G. A., Antman, E. M., Appel, L. J., Arcand, J., Blanco‐Metzler, A., Cook, N. R., et al. (2022) · Current Nutrition Reports · doi:10.1007/s13668-021-00383-z · Cited by 62 · see citing works
To Err is Human; To Reproduce Takes Time
Scott, S. L., Gunnoe, T. B., Fornasiero, P., & Crudden, C. M. (2022) · ACS Catalysis · doi:10.1021/acscatal.2c00967 · Cited by 58 · see citing works
Ten steps toward a better personality science – how quality may be rewarded more in research evaluation
Leising, D., Thielmann, I., Glöckner, A., Gärtner, A., & Schönbrodt, F. D. (2022) · Personality Science · doi:10.5964/ps.6029 · Cited by 63 · see citing works
Ground truth tracings (GTT): On the epistemic limits of machine learning
Kang, E. B. (2023) · Big Data & Society · doi:10.1177/20539517221146122 · Cited by 47 · see citing works
Credibility crisis in agricultural economics
Ferraro, P. J., & Shukla, P. (2022) · Applied Economic Perspectives and Policy · doi:10.1002/aepp.13323 · Cited by 44 · see citing works
Data leakage jeopardizes ecological applications of machine learning
Stock, A., Gregr, E. J., & Chan, K. M. A. (2023) · Nature Ecology & Evolution · doi:10.1038/s41559-023-02162-1 · Cited by 46 · see citing works
Computational Reproducibility in Finance: Evidence from 1,000 Tests
Pérignon, C., Akmansoy, O., Hurlin, C., Dreber, A., Holzmeister, F., Huber, J., Johannesson, M., Kirchler, M., Menkveld, A. J., Razen, M., et al. (2024) · Review of Financial Studies · doi:10.1093/rfs/hhae029 · Cited by 37 · see citing works
Discontinuous Distribution of Test Statistics Around Significance Thresholds in Empirical Accounting Studies
Chang, X., Gao, H., & Li, W. (2024) · Journal of Accounting Research · doi:10.1111/1475-679x.12579 · Cited by 35 · see citing works
The Effect of Replications on Citation Patterns: Evidence From a Large-Scale Reproducibility Project
Schafmeister, F. (2021) · Psychological Science · doi:10.1177/09567976211005767 · Cited by 35 · see citing works
Extraordinary Claims in the Literature on High-Intensity Interval Training (HIIT): I. Bonafide Scientific Revolution or a Looming Crisis of Replication and Credibility?
Panteleimon Ekkekakis, A., Swinton, P. A., & Tiller, N. B. (2023) · Sports Medicine · doi:10.1007/s40279-023-01880-7 · Cited by 35 · see citing works
Self-correction in science: The effect of retraction on the frequency of citations
Kühberger, A., Streit, D., & Scherndl, T. (2022) · PLoS ONE · doi:10.1371/journal.pone.0277814 · Cited by 33 · see citing works
The p-Value Debate and Statistical (Mal)practice – Implications for the Agricultural and Food Economics Community
Heckelei, T., Hüttel, S., Rommel, J., & Rommel, J. (2023) · German Journal of Agricultural Economics · doi:10.30430/gjae.2023.0231 · Cited by 38 · see citing works
The quantitative paradigm and the nature of the human mind. The replication crisis as an epistemological crisis of quantitative psychology in view of the ontic nature of the psyche
Mayrhofer, R., Büchner, I. C., & Hevesi, J. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2024.1390233 · Cited by 33 · see citing works
Publishing quantitative careers research: challenges and recommendations
Schreurs, B. H. J., Duff, A. J., Blanc, P. M. L., & Stone, T. H. (2021) · Career Development International · doi:10.1108/cdi-08-2021-0217 · Cited by 30 · see citing works
Who is Alyx? A new behavioral biometric dataset for user identification in XR
Räck, C., Fernando, T., Yalcin, M., Andreas Hotho, A., & Latoschik, M. E. (2023) · Frontiers in Virtual Reality · doi:10.3389/frvir.2023.1272234 · Cited by 27 · see citing works
Mass Reproducibility and Replicability: A New HopePreprint
Brodeur, A., Mikola, D., & Cook, N. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4790780 · Cited by 26 · see citing works
Is Psychological Science Self-Correcting? Citations Before and After Successful and Failed Replications
von Hippel, P. T. (2022) · Perspectives on Psychological Science · doi:10.1177/17456916211072525 · Cited by 25 · see citing works
Cytotoxicity, mutagenicity and genotoxicity of electronic cigarettes emission aerosols compared to cigarette smoke: the REPLICA project
Emma, R., Fuochi, V., Distefano, A., Partsinevelos, K., Rust, S., Zadjali, F., Tobi, M. A., Zadjali, R., Alharthi, Z., Pulvirenti, R., et al. (2023) · Scientific Reports · doi:10.1038/s41598-023-44626-1 · Cited by 25 · see citing works
Unraveling overoptimism and publication bias in ML-driven science
Saidi, P., Dasarathy, G., & Berisha, V. (2025) · Patterns · doi:10.1016/j.patter.2025.101185 · Cited by 29 · see citing works
A Practical Guide to Registered Reports for Economists
Arpinon, T., & Espinosa, R. (2022) · Journal of the Economic Science Association · doi:10.1007/s40881-022-00123-1 · Cited by 23 · see citing works
Beware ‘persuasive communication devices’ when writing and reading scientific articles
Corneille, O., Beffara, B., Carroll, H. A., Havemann, J., Henderson, E. L., Holmes, N. P., Hussey, I., IJzerman, H., Jussim, L., Lotter, L. D., et al. (2023) · eLife · doi:10.7554/elife.88654 · Cited by 24 · see citing works
How to trust a scientist
de Ridder, J. (2022) · Studies in History and Philosophy of Science Part A · doi:10.1016/j.shpsa.2022.02.003 · Cited by 23 · see citing works
Replicability in Brain Imaging
Kelly, R. E., & Hoptman, M. J. (2022) · Brain Sciences · doi:10.3390/brainsci12030397 · Cited by 19 · see citing works
Extraordinary Claims in the Literature on High-Intensity Interval Training: II. Are the Extraordinary Claims Supported by Extraordinary Evidence?
Panteleimon Ekkekakis, A., & Tiller, N. B. (2022) · Kinesiology Review · doi:10.1123/kr.2022-0003 · Cited by 21 · see citing works
Progressive and degenerative journals: on the growth and appraisal of knowledge in scholarly publishing
Dunleavy, D. J. (2022) · European Journal for Philosophy of Science · doi:10.1007/s13194-022-00492-8 · Cited by 19 · see citing works
Data Leakage and Evaluation Issues in Micro-Expression Analysis
Varanka, T., Li, Y., Peng, W., & Zhao, G. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2023.3265063 · Cited by 20 · see citing works
Clinical Promise of Brain-Phenotype Modeling
Greene, A. S., & Constable, R. T. (2023) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2023.1419 · Cited by 19 · see citing works
High-Performance and Parallel Computing Techniques Review: Applications, Challenges and Potentials to Support Net-Zero Transition of Future Grids
Al-Shafei, A., Zareipour, H., & Cao, Y. (2022) · Energies · doi:10.3390/en15228668 · Cited by 18 · see citing works
Replication value as a function of citation impact and sample size
Isager, P. M., Veer, A. V. '., & Lakens, D. (2025) · Meta-Psychology · doi:10.15626/mp.2022.3300 · Cited by 25 · see citing works
What do we know about the second moment of financial markets?
Grobys, K. (2021) · International Review of Financial Analysis · doi:10.1016/j.irfa.2021.101891 · Cited by 18 · see citing works
Incentives and the replication crisis in social sciences: A critical review of open science practices
Balafoutas, L., Celse, J., Karakostas, A., & Umashev, N. (2024) · Journal of Behavioral and Experimental Economics · doi:10.1016/j.socec.2024.102327 · Cited by 18 · see citing works
Is economics self‐correcting? Replications in the American Economic Review
Peters, J., Fiala, N., & Neubauer, F. (2024) · Economic Inquiry · doi:10.1111/ecin.13222 · Cited by 17 · see citing works
Overcoming the “Ostrich Effect”: A Narrative Review on the Incentives and Consequences of Questionable Research Practices in Kinesiology
Tiller, N. B., & Panteleimon Ekkekakis, A. (2023) · Kinesiology Review · doi:10.1123/kr.2022-0039 · Cited by 17 · see citing works
The Doer Effect at Scale: Investigating Correlation and Causation Across Seven Courses
Campenhout, R. V., Jerome, B., Dittel, J., & Johnson, B. G. (2023) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3576050.3576103 · Cited by 16 · see citing works
Learning to Resolve Social Dilemmas: A Survey
Fatima, S. S., Jennings, N. R., & Wooldridge, M. (2024) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.15167 · Cited by 16 · see citing works
One Decade Into the Replication Crisis, How Have Psychological Results Changed?
Bogdan, P. C. (2025) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459251323480 · Cited by 16 · see citing works
On zombies, struldbrugs, and other horrors of the scientific literature
Berenbaum, M. R. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2111924118 · Cited by 15 · see citing works
Ethical Publishing: How Do We Get There?
Racimo, F., Galtier, N., Herde, V. D., Bonn, N. A., Phillips, B. L., Guillemaud, T., & Bourguet, D. (2022) · Philosophy Theory and Practice in Biology · doi:10.3998/ptpbio.3363 · Cited by 17 · see citing works
Concentration of power at the editorial boards of economics journals
Ductor, L., & Visser, B. (2021) · Journal of Economic Surveys · doi:10.1111/joes.12497 · Cited by 1 · see citing works
Journal-based replication of experiments: An application to “Being Chosen to Lead”
Allan Drazen, Y., Dreber, A., Ozbay, E. Y., & Snowberg, E. (2021) · Journal of Public Economics · doi:10.1016/j.jpubeco.2021.104482 · Cited by 13 · see citing works
What’s the Big Deal? How Researchers Are Navigating Changes to Journal Access
Cooper, D., & Rieger, O. Y. (2021) · doi:10.18665/sr.315570 · Cited by 14 · see citing works
Expecting the Unexpected: Entropy and Multifractal Systems in Finance
Orlando, G., & Lampart, M. (2023) · Entropy · doi:10.3390/e25111527 · Cited by 13 · see citing works
Empfehlungen zur Bewertung wissenschaftlicher Leistungen bei Berufungsverfahren in der Psychologie
Gärtner, A., Leising, D., & Schönbrodt, F. D. (2023) · Psychologische Rundschau · doi:10.1026/0033-3042/a000630 · Cited by 15 · see citing works
Correlation versus co-fractality: Evidence from foreign-exchange-rate variances
Grobys, K. (2023) · International Review of Financial Analysis · doi:10.1016/j.irfa.2023.102531 · Cited by 12 · see citing works
Open and reproducible neuroimaging: from study inception to publicationPreprint
Niso, G., Botvinik‐Nezer, R., Appelhoff, S., de la Vega, A. I., Estéban, O., Etzel, J. A., Finc, K., Ganz, M., Gau, R., Halchenko, Y. O., et al. (2022) · doi:10.31219/osf.io/pu5vb · Cited by 12 · see citing works
We Have Sent Ourselves to Iceland (With Apologies to Iceland): Changing the Academy From Internally-Driven to Externally Partnered
Singh, G. G. (2022) · Frontiers in Sustainable Cities · doi:10.3389/frsc.2022.832506 · Cited by 11 · see citing works
Does the Market of Citations Reward Reproducible Work?
Raff, E. (2023) · ACM Conference on Reproducibility and Replicability (ACM REP) · doi:10.1145/3589806.3600041 · Cited by 12 · see citing works
Publication pressure threatens the integrity of palaeontological research
Raja, N. B., & Dunne, E. M. (2022) · Geological Curator · doi:10.55468/gc1459 · Cited by 12 · see citing works
Is forensic science in crisis?
Sikorski, M. (2022) · Synthese · doi:10.1007/s11229-022-03685-z · Cited by 12 · see citing works
Perspectives: replication is more than meets the eye
Bergkvist, L., Keyzer, F. D., & Buzeta, C. (2023) · International Journal of Advertising · doi:10.1080/02650487.2023.2245278 · Cited by 10 · see citing works
Heterogeneity in autism spectrum disorder explained by social-communicative and restricted repetitive behavior balance subgroups
Kim, S. Y., Udhnani, M. D., & Lecavalier, L. (2024) · Research in Autism · doi:10.1016/j.rasd.2024.102387 · Cited by 10 · see citing works
The failure of success: four lessons learned in five years of research on research integrity and research assessments
Bonn, N. A., Vries, R. G. D., & Pinxten, W. (2022) · BMC Research Notes · doi:10.1186/s13104-022-06191-0 · Cited by 10 · see citing works
Is peer review duration shorter for attractive manuscripts?
Zhang, G., Shang, F., Wang, L., Xie, W., Jia, P., Jiang, C., & Wang, X. (2023) · Journal of Information Science · doi:10.1177/01655515231174382 · Cited by 10 · see citing works
The pupil collaboration: A multi-lab, multi-method analysis of goal attribution in infants
Sirois, S., Brisson, J., Blaser, E., Calignano, G., Donenfeld, J., Hepach, R., Hochmann, J., Káldy, Z., Liszkowski, U., Mayer, M., et al. (2023) · Infant Behavior and Development · doi:10.1016/j.infbeh.2023.101890 · Cited by 10 · see citing works
Launching Registered Report Replications in Computer Science Education Research
Brown, N. C. C., Marinus, E., & Cheuoua, A. H. (2022) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/3501385.3543971 · Cited by 10 · see citing works
Forcing a Deterministic Frame on Probabilistic Phenomena: A Communication Blind Spot in Media Coverage of the “Replication Crisis”
Ting, C., & Greenland, S. (2024) · Science Communication · doi:10.1177/10755470241239947 · Cited by 11 · see citing works
Inaccuracy in the Scientific Record and Open Postpublication Critique
Brewin, C. R. (2023) · Perspectives on Psychological Science · doi:10.1177/17456916221141357 · Cited by 10 · see citing works
The Temporal Modulation of Nocebo Hyperalgesia in a Model of Sustained Pain
Camerone, E. M., Battista, S., Benedetti, F., Carlino, E., Sansone, L. G., Buzzatti, L., Scafoglieri, A., & Testa, M. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2022.807138 · Cited by 9 · see citing works
The Doer Effect: Replicating Findings that Doing Causes Learning
Campenhout, R. V., Johnson, B. G., & Olsen, J. (2021) · Cited by 9 · see citing works
Diastereoselectivity is in the Details: Minor Changes Yield Major Improvements to the Synthesis of Bedaquiline**
Mear, S. J., Lucas, T., Ahlqvist, G. P., Robey, J. M. S., Dietz, J., Khairnar, P. V., Maity, S., Williams, C. L., Snead, D. R., Nelson, R. C., et al. (2022) · Chemistry - A European Journal · doi:10.1002/chem.202201311 · Cited by 9 · see citing works
Failing to replicate predicts citation declines in psychology
Clark, C. J., Connor, P. R., & Isch, C. G. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2304862120 · Cited by 8 · see citing works
The Replicability Crisis and the p-Value Debate – what Are the Consequences for the Agricultural and Food Economics Community?Preprint
Heckelei, T., Hüttel, S., Rommel, J., & Odening, M. (2022) · Preprints.org · doi:10.20944/preprints202201.0311.v1 · Cited by 9 · see citing works
DeSci-Decentralized SciencePreprint
Ducrée, J., Codyre, M., Walshe, R., & Barting, S. (2022) · Preprints.org · doi:10.20944/preprints202205.0223.v1 · Cited by 8 · see citing works
No support for a causal role of primary motor cortex in construing meaning from language: An rTMS study
Solana, P., Escámez, O., Casasanto, D. J., Chica, A. B., & Santiago, J. (2024) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2024.108832 · Cited by 8 · see citing works
Animal models in preclinical evaluation of CAR-T cell therapy: Advantages and limitations
Fløe, L. V. B., Pedersen, M. G., & Møller, B. K. (2025) · Biochimica et Biophysica Acta (BBA) - Reviews on Cancer · doi:10.1016/j.bbcan.2025.189455 · Cited by 8 · see citing works
Establishing ground truth in the traumatic brain injury literature: if replication is the answer, then what are the questions?
Priestley, D. R., Staph, J., Koneru, S., Rajtmajer, S. M., Cwiek, A. P., Vervoordt, S. M., & Hillary, F. G. (2022) · Brain Communications · doi:10.1093/braincomms/fcac322 · Cited by 7 · see citing works
On the Continued Need for Replication in Media and Communication Research
Bowman, N. D. (2024) · Media and Communication · doi:10.17645/mac.7935 · Cited by 8 · see citing works
To “P” or not to “P”, that is the question: A narrative review on P value
Darling, H. (2021) · Cancer Research Statistics and Treatment · doi:10.4103/crst.crst_222_21 · Cited by 7 · see citing works
Don't hate the player, hate the game: Realigning incentive structures to promote robust science and better scientific practices in marketing
Shaw, S. D., & Nave, G. (2023) · Journal of Business Research · doi:10.1016/j.jbusres.2023.114129 · Cited by 8 · see citing works
Forking Paths in Empirical StudiesPreprint
Coqueret, G. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.3999379 · Cited by 8 · see citing works
Toward Science‐Led Publishing
Pattinson, D., & Currie, G. (2025) · Learned Publishing · doi:10.1002/leap.2012 · Cited by 20 · see citing works
On the Realized Risk of Foreign Exchange Rates: A Fractal Perspective
Fathi, M., Grobys, K., & Kolari, J. W. (2024) · Journal of risk and financial management · doi:10.3390/jrfm17020079 · Cited by 6 · see citing works
Scientific rot: Unsustainable publishing practices threatens trust in medicine
Heen, E., & Vogt, H. (2024) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.13989 · Cited by 7 · see citing works
Good Practice Guideline for Preclinical Alcohol Research: The STRINGENCY Framework
Meinhardt, M. W., Gerlach, B., & Spanagel, R. (2024) · Current topics in behavioral neurosciences · doi:10.1007/7854_2024_484 · Cited by 7 · see citing works
Citation counts and journal impact factors do not capture some indicators research quality in the behavioral and brain sciencesPreprint
Dougherty, M. R. P., & Horne, Z. (2019) · doi:10.31234/osf.io/9g5wk · Cited by 6 · see citing works
The Case for Preregistering Quasi-Experimental Program and Policy Evaluations
Dee, T. S. (2025) · Evaluation Review · doi:10.1177/0193841x251326738 · Cited by 6 · see citing works
Estimating the replicability of highly cited clinical research (2004–2018)
da Costa, G. G., Neves, K., & Amaral, O. B. (2024) · PLoS ONE · doi:10.1371/journal.pone.0307145 · Cited by 8 · see citing works
The Controversy behind the Controversies: Scientific Discourse in the Twenty-First Century
Fahnestock, J. (2025) · Rhetoric Society Quarterly · doi:10.1080/02773945.2025.2484162 · Cited by 5 · see citing works
Publishing important work that lacks validity or reproducibility – pushing frontiers or corrupting science?
Tang, B. L. (2024) · Accountability in Research · doi:10.1080/08989621.2024.2345714 · Cited by 5 · see citing works
The Erosion of Healthcare and Scientific Integrity: A Growing Concern
Brücher, B. L. D. M. (2025) · Journal of Healthcare Leadership · doi:10.2147/jhl.s506767 · Cited by 6 · see citing works
Engaging in Student-Centered Educational Data Science Through Learning Engineering
Campenhout, R. V., Jerome, B., & Johnson, B. G. (2023) · Big data management · doi:10.1007/978-981-99-0026-8_1 · Cited by 5 · see citing works
Logical Fallacies: How They Undermine Critical Thinking and How to Avoid ThemPreprint
Friedman, H. H., & Kaganovskiy, L. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4794200 · Cited by 5 · see citing works
Towards reliable data augmentation in machine learning: Practices to prevent data leakage
Gerz, F., & Jelali, M. (2026) · Smart Agricultural Technology · doi:10.1016/j.atech.2026.102462 · Cited by 4 · see citing works
Show your work: Tools for open developmental science
Gilmore, R. O. (2022) · Advances in child development and behavior · doi:10.1016/bs.acdb.2022.01.001 · Cited by 3 · see citing works
How to read a paper on the short-term impairing effects of cannabis: A selective and critical review of the literature
White, M. A., & Burns, N. R. (2022) · Drug Science Policy and Law · doi:10.1177/20503245221119046 · Cited by 4 · see citing works
Unwrapping the McDonald's model: An introduction to dynamic social theory
Alexander, T. (2023) · The Journal of American Culture · doi:10.1111/jacc.13467 · Cited by 4 · see citing works
Retractions: Updating from Complex Information
Gonçalves, D., Libgober, J., & Willis, J. (2021) · The Review of Economic Studies · doi:10.1093/restud/rdaf032 · Cited by 5 · see citing works

Authors: Serra-Garcia, M., & Gneezy, U.
Affiliations: Rady School of Management, University of California, San Diego La Jolla, CA, USA.
ORCID: Serra-Garcia, Marta
Gneezy, Uri
Venue: Science Advances
Publisher: American Association for the Advancement of Science (AAAS)
Published: 2021-05-21
Volume: 7
Issue: 21
ISSN: 2375-2548 (ISSN-L)
DOI: 10.1126/sciadv.abd1705
PMC: PMC8139580
OpenAlex: W3160678577
Cited by: 177
References: 26
Citations by year: 2026: 9 · 2025: 22 · 2024: 32 · 2023: 39 · 2022: 47 · 2021: 25 · 2020: 1 · 2019: 2
FWCI: 20.72
Type: journal article
Language: en
Open access: gold
doi.org
Licence: cc-by-nc
creativecommons.org/licenses/by-nc/4.0/ (vor)
Locations: Science Advances (publishedVersion, cc-by-nc)
PubMed Central (publishedVersion, cc-by-nc)
PubMed (publishedVersion)
eScholarship (California Digital Library) (submittedVersion, cc-by-nc-nd)
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
Keywords: replication failure, citation impact, reproducibility, peer review, publication standards, scientific credibility, psychology journals, economics journals

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Nonreplicable publications are cited more than replicable ones

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Self-check

Cites

Cited by

Related

Record

About these checks