Novelty versus Replicability: Virtues and Vices in the Reward System of Science
Abstract
The reward system of science is the priority rule. The first scientist making a new discovery is rewarded with prestige, while second runners get little or nothing. Michael Strevens, following Philip Kitcher, defends this reward system, arguing that it incentivizes an efficient division of cognitive labor. I argue that this assessment depends on strong implicit assumptions about the replicability of findings. I question these assumptions on the basis of metascientific evidence and argue that the priority rule systematically discourages replication. My analysis leads us to qualify Kitcher and Strevens’s contention that a priority-based reward system is normatively desirable for science.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The publishing landscape in contemporary science rewards novel, positive findings over replications or negative results.
“Furthermore, the publishing landscape in contemporary science rewards novel, positive findings over replications or negative results (Ioannidis 2005; Romero 2017).”
From The myth of the Bayesian brain (2025) · cites this paper as Romero 2017
Cites
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068 · cited 1× · see citing works
“…metascientific evidence in various fields suggests that many successful published findings result from questionable practices: selective reporting (Rosenthal 1979), exclusion of data points (Bones 2012; John, Loewenstein, and Prelec 2012), p-hacking (Simmons, Nelson, and Simonsohn 2011), HARKing…”
BackgroundEstimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515 · see citing works
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567 · see citing works
Stanford Encyclopedia of Philosophy
(2015) · Choice Reviews Online · doi:10.5860/choice.187763 · Cited by 5,364 · see citing works
Raise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979 · see citing works
Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
John, L. K., Loewenstein, G., & Prelec, D. (2012) · Psychological Science · doi:10.1177/0956797611430953 · Cited by 2,239 · see citing works
Science as a Process: An Evolutionary Account of the Social and Conceptual Development of Science
Hull, D. L. (1988) · DigitalGeorgetown (Georgetown University Library) · doi:10.7208/chicago/9780226360492.001.0001 · Cited by 2,052 · see citing works
HARKing: Hypothesizing after the results are known
Kerr, N. L. (1998) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0203_4 · Cited by 1,989 · see citing works
Fanelli, D. (2009) · PLoS ONE · doi:10.1371/journal.pone.0005738 · Cited by 1,911 · see citing works
Priorities in Scientific Discovery: A Chapter in the Sociology of Science
Merton, R. Κ. (1957) · American Sociological Review · doi:10.2307/2089193 · Cited by 1,615 · see citing works
Negative results are disappearing from most disciplines and countries
Fanelli, D. (2012) · Scientometrics · doi:10.1007/s11192-011-0494-7 · Cited by 1,248 · see citing works
Shall we Really do it Again? The Powerful Concept of Replication is Neglected in the Social Sciences
Schmidt, S. (2009) · Review of General Psychology · doi:10.1037/a0015108 · Cited by 1,125 · see citing works
The Arrangement of Field Experiments
Fisher, R. A. (1992) · Springer series in statistics · doi:10.1007/978-1-4612-4380-9_8 · Cited by 979 · see citing works
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., & van der Maas, H. L. J. (2011) · Journal of Personality and Social Psychology · doi:10.1037/a0022790 · Cited by 977 · see citing works
Stephan, P. E. (2012) · Harvard University Press eBooks · doi:10.4159/harvard.9780674062757 · Cited by 958 · see citing works
Scotchmer, S. (2004) · MIT Press Books · Cited by 929 · see citing works
Bem, D. J. (2011) · Journal of Personality and Social Psychology · doi:10.1037/a0021524 · Cited by 893 · see citing works
The Division of Cognitive Labor
Kitcher, P. (1990) · The Journal of Philosophy · doi:10.2307/2026796 · Cited by 772 · see citing works
Replications in Psychology Research: How Often Do They Really Occur?
Makel, M. C., Plucker, J. A., & Hegarty, B. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612460688 · Cited by 750 · see citing works
Small Telescopes: Detectability and the Evaluation of Replication Results
Simonsohn, U. (2015) · Psychological Science · doi:10.1177/0956797614567341 · Cited by 742 · see citing works
“Positive” results increase down the Hierarchy of the Sciences
Fanelli, D. (2010) · PLoS ONE · doi:10.1371/journal.pone.0010068 · Cited by 734 · see citing works
Is the Replicability Crisis Overblown? Three Arguments Examined
Pashler, H., & Harris, C. R. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612463401 · Cited by 697 · see citing works
The Logic of Scientific Discovery
Jeffrey, R. C., & Popper, K. R. (1960) · Econometrica · doi:10.2307/1907578 · Cited by 676 · see citing works
Facts Are More Important Than Novelty: Replication in the Education Sciences
Makel, M. C., & Plucker, J. A. (2014) · Educational Researcher · doi:10.3102/0013189x14545513 · Cited by 609 · see citing works
The Advancement of Science: Science without Legend, Objectivity without Illusions.
Dupré, J., & Kitcher, P. (1995) · The Philosophical Review · doi:10.2307/2186026 · Cited by 608 · see citing works
The Role of the Priority Rule in Science
Strevens, M. (2003) · The Journal of Philosophy · doi:10.5840/jphil2003100224 · Cited by 351 · see citing works
Replication research's disturbing trend
Evanschitzky, H., Baumgarth, C., Hubbard, R., & Armstrong, J. S. (2007) · Journal of Business Research · doi:10.1016/j.jbusres.2006.12.003 · Cited by 271 · see citing works
The advancement of science: science without legend, objectivity without illusions
(1993) · Choice Reviews Online · doi:10.5860/choice.31-1465 · Cited by 220 · see citing works
A Bayes factor meta-analysis of Bem’s ESP claim
Rouder, J. N., & Morey, R. D. (2011) · Psychonomic Bulletin & Review · doi:10.3758/s13423-011-0088-7 · Cited by 194 · see citing works
Ritchie, S. J., Wiseman, R. J., & French, C. C. (2012) · PLoS ONE · doi:10.1371/journal.pone.0033423 · Cited by 184 · see citing works
Correcting the past: Failures to replicate psi.
Galak, J., LeBoeuf, R. A., Nelson, L. D., & Simmons, J. P. (2012) · Journal of Personality and Social Psychology · doi:10.1037/a0029709 · Cited by 148 · see citing works
Editorial bias against replication research.
Neuliep, J. W., & Crandall, R. (1990) · Journal of social behavior and personality · Cited by 141 · see citing works
Replication studies: A neglected aspect of psychological research.
Smith, N. C. (1970) · American Psychologist · doi:10.1037/h0029774 · Cited by 133 · see citing works
Replicability, Reproducibility, and Robustness: Comments on Harry Collins
Cartwright, N. (1991) · History of Political Economy · doi:10.1215/00182702-23-1-143 · Cited by 122 · see citing works
Reviewer bias against replication research.
Neuliep, J. W., & Crandall, R. (1993) · Journal of social behavior and personality · Cited by 116 · see citing works
Must psychologists change the way they analyze their data?
Bem, D. J., Utts, J., & Johnson, W. O. (2011) · Journal of Personality and Social Psychology · doi:10.1037/a0024777 · Cited by 105 · see citing works
Robustness and idealization in models of cognitive labor
Muldoon, R., & Weisberg, M. (2010) · Synthese · doi:10.1007/s11229-010-9757-8 · Cited by 51 · see citing works
Bones, A. K. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612441216 · Cited by 48 · see citing works
Replication, replication, replication
Ritchie, S. J., Wiseman, R. J., & French, C. C. (2012) · University of Hertfordshire Research Archive (University of Hertfordshire) · Cited by 33 · see citing works
Economic Approaches to Understanding Scientific Norms
Strevens, M. (2011) · Episteme · doi:10.3366/epi.2011.0015 · Cited by 32 · see citing works
Machery, É. (2012) · Philosophy of Science · doi:10.1086/667877 · Cited by 23 · see citing works
Interesting results: Can they be replicated?
Winterdahl, M. (2015) · OMICS Journal of Radiology · Cited by 13 · see citing works
Interesting results: Can they be replicated?
Winerman, L. (2013) · PsycEXTRA Dataset · doi:10.1037/e505012013-006 · Cited by 11 · see citing works
The rule of reproducibility and its applications in experiment appraisal
Chen, X. (1994) · Synthese · doi:10.1007/bf01064532 · Cited by 5 · see citing works
Replication, replication, replication
Morley, S., Masterson, C., & Main, C. J. (2017) · Routledge eBooks · doi:10.4324/9781315412931-7 · Cited by 2 · see citing works
Flagged first, then by citations
1 of the 46 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Replicability, Robustness, and Reproducibility in Psychological Science
Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., Fidler, F., Hilgard, J., Kline Struhl, M., Nuijten, M. B., et al. (2022) · Annual Review of Psychology · doi:10.1146/annurev-psych-020821-114157 · Cited by 770 · see citing works
Self-citationPhilosophy of science and the replicability crisis
Romero, F. (2019) · Philosophy Compass · doi:10.1111/phc3.12633 · Cited by 152 · see citing works
Self-citationHeesen, R., & Bright, L. (2020) · British Journal for the Philosophy of Science · doi:10.1093/bjps/axz029 · Cited by 142 · see citing works
Rethinking Reproducibility as a Criterion for Research Quality
Leonelli, S. (2018) · Research in the history of economic thought and methodology · doi:10.1108/s0743-41542018000036b009 · Cited by 136 · see citing works
Understanding the Replication Crisis as a Base Rate Fallacy
Bird, A. (2018) · The British Journal for the Philosophy of Science · doi:10.1093/bjps/axy051 · Cited by 122 · see citing works
Feest, U. (2019) · Philosophy of Science · doi:10.1086/705451 · Cited by 92 · see citing works
Self-correction in science: Meta-analysis, bias and social structure
Bruner, J., & Holman, B. (2019) · Studies in History and Philosophy of Science Part A · doi:10.1016/j.shpsa.2019.02.001 · Cited by 61 · see citing works
Contessa, G. (2022) · Erkenntnis · doi:10.1007/s10670-021-00485-8 · Cited by 63 · see citing works
Greco, D. (2023) · doi:10.1093/oso/9780198860556.001.0001 · Cited by 52 · see citing works
Pursuit and inquisitive reasons
Fleisher, W. (2022) · Studies in History and Philosophy of Science Part A · doi:10.1016/j.shpsa.2022.04.009 · Cited by 49 · see citing works
Is novel research worth doing? Evidence from peer review at 49 journals
Teplitskiy, M., Peng, H., Blasco, A., & Lakhani, K. R. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2118046119 · Cited by 48 · see citing works
Competition for priority harms the reliability of science, but reforms can help
Tiokhin, L., Yan, M., & Morgan, T. J. H. (2021) · Nature Human Behaviour · doi:10.1038/s41562-020-01040-1 · Cited by 37 · see citing works
Stewart, S., Pennington, C. R., Silva, G., Ballou, N., Butler, J. E., Dienes, Z. P., Jay, C., Rossit, S., Samara, A., & U. K. Reproducibility Network (UKRN) Local Network Leads (2022) · BMC Research Notes · doi:10.1186/s13104-022-05949-w · Cited by 29 · see citing works
A New Account of Replication in the Experimental Life Sciences
Guttinger, S. (2019) · Philosophy of Science · doi:10.1086/703555 · Cited by 27 · see citing works
Priority and privilege in scientific discovery
Rubin, H., & Schneider, M. D. (2021) · Studies in History and Philosophy of Science Part A · doi:10.1016/j.shpsa.2021.08.005 · Cited by 23 · see citing works
Replications in Comparative Psychology
Halina, M. (2021) · Animal Behavior and Cognition · doi:10.26451/abc.08.02.13.2021 · Cited by 23 · see citing works
Unveiling the impact and dual innovation of funded research
Yang, A. J. (2023) · Journal of Informetrics · doi:10.1016/j.joi.2023.101480 · Cited by 22 · see citing works
Towards a Contextual Approach to Data Quality
Canali, S. (2020) · Data · doi:10.3390/data5040090 · Cited by 20 · see citing works
Modelling Scientific Communities
O’Connor, C. (2023) · Cambridge University Press eBooks · doi:10.1017/9781009359535 · Cited by 16 · see citing works
How (not) to measure replication
Fletcher, S. C. (2021) · European Journal for Philosophy of Science · doi:10.1007/s13194-021-00377-2 · Cited by 15 · see citing works
Australian and Italian Psychologists’ View of Replication
Agnoli, F., Fraser, H. S., Thorn, F. S., & Fidler, F. M. (2021) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459211039218 · Cited by 13 · see citing works
Scientific self-correction: the Bayesian way
Romero, F., & Sprenger, J. (2020) · Synthese · doi:10.1007/s11229-020-02697-x · Cited by 16 · see citing works
Self-citationEpistemic Functions of Replicability in Experimental Sciences: Defending the Orthodox View
Sikorski, M., & Andreoletti, M. (2023) · Foundations of Science · doi:10.1007/s10699-023-09901-4 · Cited by 13 · see citing works
The myth of the Bayesian brain
Mangalam, M. (2025) · European Journal of Applied Physiology · doi:10.1007/s00421-025-05855-6 · Cited by 14 · see citing works
“Furthermore, the publishing landscape in contemporary science rewards novel, positive findings over replications or negative results (Ioannidis 2005; Romero 2017).”
Publication bias is bad for science if not necessarily scientists
Heesen, R., & Bright, L. K. (2025) · Royal Society Open Science · doi:10.1098/rsos.240688 · Cited by 9 · see citing works
Values, bias and replicability
Sikorski, M. (2024) · Synthese · doi:10.1007/s11229-024-04573-4 · Cited by 8 · see citing works
Maneuvrier, A. (2024) · Virtual Reality · doi:10.1007/s10055-024-00993-2 · Cited by 6 · see citing works
Bertin, P., & Nera, K. (2025) · International Review of Social Psychology · doi:10.5334/irsp.979 · Cited by 5 · see citing works
(When) should psychology be a science?
Krpan, D. (2021) · Journal for the Theory of Social Behaviour · doi:10.1111/jtsb.12316 · Cited by 5 · see citing works
The Division of Replication Labor
Romero, F. (2020) · Philosophy of Science · doi:10.1086/710625 · Cited by 6 · see citing works
Self-citationEarly Experimental Psychology: How did Replication Work Before P-Hacking?
Mülberger, A. (2022) · Review of General Psychology · doi:10.1177/10892680211066468 · Cited by 4 · see citing works
What FiveThirtyEight Can Teach Us about Solving the Replication Crisis
Holman, B. (2022) · Philosophy of Science · doi:10.1017/psa.2022.50 · Cited by 4 · see citing works
On the quest for novelty in ecology
Ottaviani, G., Martínez, A., Bon, M. P., & Mammola, S. (2025) · Peer Community Journal · doi:10.24072/pcjournal.567 · Cited by 4 · see citing works
Is Novel Research Worth Doing? Evidence from Journal Peer ReviewPreprint
Teplitskiy, M., Peng, H., Blasco, A., & Lakhani, K. R. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3920711 · Cited by 3 · see citing works
Publication pressure and questionable research practices: a moderated mediation model
Gan, Y., Shi, Y. Y., & Wang, G. (2024) · Ethics & Behavior · doi:10.1080/10508422.2024.2397103 · Cited by 2 · see citing works
Games researchers play: conceptual advancement versus validation strategies
Dubois, F., & Peres‐Neto, P. R. (2022) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2022.02.003 · Cited by 2 · see citing works
Publish without bias or perish without replications
Ventura, R. (2022) · Studies in History and Philosophy of Science Part A · doi:10.1016/j.shpsa.2022.08.010 · Cited by 2 · see citing works
Online Moral Conformity Revisited
Fedorov, A. A., & Steshova, A. A. (2025) · International Journal of Psychology · doi:10.1002/ijop.70033 · Cited by 2 · see citing works
Schiekiera, L., & Niemeyer, H. (2025) · F1000Research · doi:10.12688/f1000research.160170.1 · Cited by 1 · see citing works
Greco, D. (2023) · doi:10.1093/oso/9780198860556.003.0006 · Cited by 1 · see citing works
Overcoming Limitations of Proprietary Scientific Hardware Funding
Seidle, A. G., & Pearce, J. M. (2025) · Journal of the Knowledge Economy · doi:10.1007/s13132-025-02783-w · Cited by 1 · see citing works
Max Weber’s ‘Inconvenient Facts’ and Contemporary Studies of Public Science Communication
Shipovalova, L. V. (2019) · Social Epistemology · doi:10.1080/02691728.2019.1695013 · Cited by 1 · see citing works
Science of Public Recognition and Benefits (Sprb): Team, Impact, Novelty, and DisruptionPreprint
Yang, A. J., & Deng, S. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4478791
Business and Science – Not So Different After All? A Commentary on Ontrup et al. (2023)
Langer, M. (2023) · Zeitschrift für Arbeits- und Organisationspsychologie A&O · doi:10.1026/0932-4089/a000419
Ideal and Non-Ideal Epistemology
Greco, D. (2023) · doi:10.1093/oso/9780198860556.003.0008
Greco, D. (2023) · doi:10.1093/oso/9780198860556.003.0002
Greco, D. (2023) · doi:10.1093/oso/9780198860556.003.0007
Greco, D. (2023) · doi:10.1093/oso/9780198860556.003.0004
Greco, D. (2023) · doi:10.1093/oso/9780198860556.003.0009
Greco, D. (2023) · doi:10.1093/oso/9780198860556.003.0005
Greco, D. (2023) · doi:10.1093/oso/9780198860556.003.0001
Greco, D. (2023) · doi:10.1093/oso/9780198860556.003.0003
Tourlakis, M. E. (2024) · Elsevier eBooks · doi:10.1016/b978-0-12-817218-6.00015-2
Empirische Erforschung kognitiver Fähigkeiten
Kästner, L. (2024) · J.B. Metzler eBooks · doi:10.1007/978-3-476-05416-6_42
Measuring the quality of experimental research
Ventura, R. (2025) · Synthese · doi:10.1007/s11229-024-04905-4
Challenges and actions towards a more comprehensive developmental science
Tantaros, S. (2025) · European Journal of Developmental Psychology · doi:10.1080/17405629.2025.2498756
Creating a path to meaningful public participation: the case of heritable human genome editing
Güttinger, S. (2025) · Zeitschrift für Ethik und Moralphilosophie · doi:10.1007/s42048-025-00232-6
Regaining Scientific Authority in a Post-Truth Landscape
Petzold, A. M., & Nichols, M. D. (2025) · Publications · doi:10.3390/publications13040065 · Cited by 1 · see citing works
Individual and collective epistemic virtue in science
Tunç, D. U., & Pritchard, D. (2026) · Synthese · doi:10.1007/s11229-025-05396-7
Matarese, V. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-40622-5.00015-3
- Authors
- Romero, F.
- Affiliations
- Tilburg University (NL)
- Corresponding author
- Romero, Felipe
- ORCID
- Romero, Felipe
- Venue
- Philosophy of Science
- Publisher
- Cambridge University Press (CUP)
- Published
- 2017-12
- Volume
- 84
- Issue
- 5
- Pages
- 1031-1043
- ISSN
- 0031-8248 (ISSN-L)
- 1539-767X
- OpenAlex
- W2612674204
- Cited by
- 61
- References
- 46
- Citations by year
- 2026: 2 · 2025: 11 · 2024: 5 · 2023: 16 · 2022: 8 · 2021: 8 · 2020: 3 · 2019: 6 · 2018: 2
- FWCI
- 15.21
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Philosophy of Science (publishedVersion)
- PhilPapers (PhilPapers Foundation) (submittedVersion)
- Funders
- H2020 European Research Council (640638)
- Topics
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Philosophy and History of Science
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Philosophy, Science, and History
- Social Sciences › Decision Sciences › Information Systems and Management › Research, Science, and Academia
- Keywords
- dispatching rules, replicability, novelty, division of cognitive labor, Philip Kitcher, meta-research, scientific prestige, replication
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
