Bite-size science and its undesired side effects
Abstract
Short and rapid publication of research findings has many advantages. However, there is another side of the coin that needs careful consideration. We argue that the most dangerous aspect of a shift toward "bite-size" publishing is the relationship between study size and publication bias. Findings based on a single study or a study based on a limited sample size are more likely to be false positive, because the false positive rate remains constant, whereas the true positive rate (the power) declines as sample size declines. Pressure on productivity and on novelty value further exacerbates the problem.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
From a statistical point of view this problem might not be any more serious for this field than other domains of psychology or neuroscience in general.
“From a statistical point of view this problem might not be any more serious for this field than other domains of psychology (20, 21, 31) or neuroscience in general (14).”
From Statistical and Methodological Considerations for the Interpretation of Intranasal Oxytocin Studies (2016) · cites this paper as [31]
Cites
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722 · see citing works
Molecular Structure of Nucleic Acids: A Structure for Deoxyribose Nucleic Acid
WATSON, J. D., & CRICK, F. H. C. (1953) · Nature · doi:10.1038/171737a0 · Cited by 13,584 · see citing works
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505 · see citing works
Funnel plots for detecting bias in meta-analysis
Sterne, J. A. C., & Egger, M. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00377-8 · Cited by 3,779 · see citing works
Sterne, J. A. C. (2001) · BMJ · doi:10.1136/bmj.322.7280.226 · Cited by 1,466 · see citing works
Larsen, P. O., & von Ins, M. (2010) · Scientometrics · doi:10.1007/s11192-010-0202-z · Cited by 964 · see citing works
Erroneous analyses of interactions in neuroscience: A problem of significance
Nieuwenhuis, S., Forstmann, B. U., & Wagenmakers, E.-J. (2011) · Nature Neuroscience · doi:10.1038/nn.2886 · Cited by 864 · see citing works
Do studies of statistical power have an effect on the power of studies?
Sedlmeier, P., & Gigerenzer, G. (1989) · Psychological Bulletin · doi:10.1037/0033-2909.105.2.309 · Cited by 748 · see citing works
Publication bias and clinical trials
Dickersin, K., Chan, S., Chalmersx, T. C., Sacks, H. S., & Smith, H. (1987) · Controlled Clinical Trials · doi:10.1016/0197-2456(87)90155-3 · Cited by 715 · see citing works
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2019) · CHANCE · doi:10.1080/09332480.2019.1579573 · Cited by 473 · see citing works
Sullivan, P. F. (2007) · Biological Psychiatry · doi:10.1016/j.biopsych.2006.11.010 · Cited by 335 · see citing works
Excess Significance Bias in the Literature on Brain Volume Abnormalities
Ioannidis, J. P. A. (2011) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2011.28 · Cited by 299 · see citing works
Irresponsible Authorship and Wasteful Publication
Huth, E. J. (1986) · Annals of Internal Medicine · doi:10.7326/0003-4819-104-2-257 · Cited by 184 · see citing works
Robinson, K. A., & Goodman, S. N. (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-154-1-201101040-00007 · Cited by 151 · see citing works
Read, D., & Livesay, D. (1978) · Bulletin of the Committee on Canadian Labour History / Bulletin du Comité sur l Histoire Ouvrière Canadienne · doi:10.2307/25093606 · Cited by 128 · see citing works
Short, Sweet, and Problematic? The Rise of the Short Report in Psychological Science
Ledgerwood, A., & Sherman, J. W. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691611427304 · Cited by 72 · see citing works
Bias in genetic association studies and impact factor
Munafò, M. R., Stothart, G., & Flint, J. (2009) · Molecular Psychiatry · doi:10.1038/mp.2008.77 · Cited by 60 · see citing works
Not So Fast: Inflation in Impact Factors Contributes to Apparent Improvements in Journal Quality
Neff, B. D., & Olden, J. D. (2010) · BioScience · doi:10.1525/bio.2010.60.6.9 · Cited by 51 · see citing works
End the wasteful tyranny of reviewer experiments
Ploegh, H. (2011) · Nature · doi:10.1038/472391a · Cited by 49 · see citing works
MUNAFÒ, M. R., ATTWOOD, A. S., & FLINT, J. (2008) · Psychological Medicine · doi:10.1017/s003329170800353x · Cited by 32 · see citing works
Haslam, N. (2010) · Perspectives on Psychological Science · doi:10.1177/1745691610369466 · Cited by 23 · see citing works
Publishing in the Psychological Sciences: Enhancing Journal Impact While Decreasing Author Fatigue
Park, D. C. (2009) · Perspectives on Psychological Science · doi:10.1111/j.1745-6924.2009.01103.x · Cited by 16 · see citing works
Credible genetic associations?
Munafo, M. R. (2010) · PubMed · Cited by 4 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
The Replication Recipe: What makes for a convincing replication?
Brandt, M. J., IJzerman, H., Dijksterhuis, A., Farach, F. J., Geller, J., Giner‐Sorolla, R., Grange, J. A., Perugini, M., Spies, J. R., & Veer, A. V. '. (2013) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2013.10.005 · Cited by 703 · see citing works
An Open, Large-Scale, Collaborative Effort to Estimate the Reproducibility of Psychological Science
Open Science Collaboration (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612462588 · Cited by 605 · see citing works
On doing better science: From thrill of discovery to policy implications
Antonakis, J. (2017) · The Leadership Quarterly · doi:10.1016/j.leaqua.2017.01.006 · Cited by 483 · see citing works
Deep impact: unintended consequences of journal rank
Brembs, B., Button, K., & Munafò, M. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00291 · Cited by 441 · see citing works
Artificial Intelligence and Journalism
Broussard, M., Diakopoulos, N. A., Guzman, A. L., Abebe, R., Dupagne, M., & Chuan, C. (2019) · Journalism & Mass Communication Quarterly · doi:10.1177/1077699019859901 · Cited by 418 · see citing works
Ten Simple Rules for Writing a Literature Review
Pautasso, M. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003149 · Cited by 399 · see citing works
Statistical and Methodological Considerations for the Interpretation of Intranasal Oxytocin Studies
Walum, H., Waldman, I. D., & Young, L. J. (2016) · Biological Psychiatry · doi:10.1016/j.biopsych.2015.06.016 · Cited by 345 · see citing works
“From a statistical point of view this problem might not be any more serious for this field than other domains of psychology (20, 21, 31) or neuroscience in general (14).”
BackgroundKoole, S. L., & Lakens, D. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612462586 · Cited by 281 · see citing works
“Excellence R Us”: university research and the fetishisation of excellence
Moore, S. A., Neylon, C., Eve, M. P., O'Donnell, D. P., & Pattinson, D. (2017) · Palgrave Communications · doi:10.1057/palcomms.2016.105 · Cited by 71 · see citing works
Priming Intelligent Behavior: An Elusive Phenomenon
Shanks, D. R., Newell, B. R., Lee, E. H., Balakrishnan, D., Ekelund, L., Cenac, Z., Kavvadia, F., & Moore, C. (2013) · PLoS ONE · doi:10.1371/journal.pone.0056515 · Cited by 240 · see citing works
Researchers’ Individual Publication Rate Has Not Increased in a Century
Fanelli, D., & Larivière, V. (2016) · PLOS ONE · doi:10.1371/journal.pone.0149504 · Cited by 205 · see citing works
Research Practices That Can Prevent an Inflation of False-Positive Rates
Murayama, K., Pekrun, R. H., & Fiedler, K. (2013) · Personality and Social Psychology Review · doi:10.1177/1088868313496330 · Cited by 126 · see citing works
The Structure and Conduct of a Narrative Literature Review
Pautasso, M. (2019) · A Guide to the Scientific Career · doi:10.1002/9781118907283.ch31 · Cited by 124 · see citing works
P-hacking and publication bias interact to distort meta-analytic effect size estimates
Friese, M., & Frankenbach, J. (2020) · Psychological Methods · doi:10.1037/met0000246 · Cited by 122 · see citing works
Knowledge creation in consumer research: Multiple routes, multiple criteria
Lynch, J. G., Alba, J. W., Krishna, A., Morwitz, V. G., & Gürhan‐Canlı, Z. (2012) · Journal of Consumer Psychology · doi:10.1016/j.jcps.2012.06.004 · Cited by 109 · see citing works
Article Commentary: On the Persistence of Low Power in Psychological Science
Vankov, I., Bowers, J., & Munafò, M. R. (2014) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470218.2014.885986 · Cited by 82 · see citing works
Publication Growth in Biological Sub-Fields: Patterns, Predictability and Sustainability
Pautasso, M. (2012) · Sustainability · doi:10.3390/su4123234 · Cited by 70 · see citing works
Why Selective Publication of Statistically Significant Results Can Be Effective
de Winter, J., & Happee, R. (2013) · PLoS ONE · doi:10.1371/journal.pone.0066463 · Cited by 67 · see citing works
Matz, S., Gladstone, J. J., & Stillwell, D. J. (2017) · Psychological Science · doi:10.1177/0956797617697445 · Cited by 67 · see citing works
Helfer, B., Prosser, A., Samara, M. T., Geddes, J. R., Cipriani, A., Davis, J. M., Mavridis, D., Salanti, G., & Leucht, S. (2015) · BMC Medicine · doi:10.1186/s12916-015-0317-4 · Cited by 63 · see citing works
Gervais, W. M., Jewell, J. A., Najle, M. B., & Ng, B. K. L. (2015) · Social Psychological and Personality Science · doi:10.1177/1948550615584199 · Cited by 56 · see citing works
Pay Big to Publish Fast: Academic Journal Rackets.
Truth, F. (2012) · The Journal for Critical Education Policy Studies · Cited by 52 · see citing works
Prime Numbers: Anchoring and its Implications for Theories of Behavior Priming
Newell, B. R., & Shanks, D. R. (2014) · Social Cognition · doi:10.1521/soco.2014.32.supp.88 · Cited by 38 · see citing works
Forming Impressions of Personality
Nauts, S., Langner, O., Huijsmans, I., Vonk, R., & Wigboldus, D. H. J. (2014) · Social Psychology · doi:10.1027/1864-9335/a000179 · Cited by 31 · see citing works
Tarr, M. J. (2013) · Psychological Science · doi:10.1177/0956797612474669 · Cited by 28 · see citing works
Citation of Previous Meta-analyses on the Same Topic
Li, T., & Dickersin, K. (2013) · Ophthalmology · doi:10.1016/j.ophtha.2012.11.038 · Cited by 24 · see citing works
A perspective on problems and prospects for academic publishing in Geography
Meadows, M. E., Dietz, T. J., & Vandermotten, C. (2016) · Geo Geography and Environment · doi:10.1002/geo2.16 · Cited by 22 · see citing works
The role of emotions in B2B product advertising on social media: a family business case study
Pirraglia, E., Giuliani, F., Cicco, R. D., Berardino, C. D., & Palumbo, R. (2022) · Journal of Family Business Management · doi:10.1108/jfbm-12-2021-0157 · Cited by 21 · see citing works
Quantity Over Quality? Reproducible Psychological Science from a Mixed Methods Perspective
Reischer, H. N., & Cowan, H. R. (2020) · Collabra Psychology · doi:10.1525/collabra.284 · Cited by 19 · see citing works
Park, J. H., Eisenhut, M., van der Vliet, H. J., & Shin, J. I. (2017) · Annals of Oncology · doi:10.1093/annonc/mdx024 · Cited by 19 · see citing works
p-Hacking and Publication Bias Interact to Distort Meta-Analytic Effect Size EstimatesPreprint
Friese, M., & Frankenbach, J. (2020) · doi:10.31234/osf.io/bvm58 · Cited by 15 · see citing works
Powering Reproducible Research
Button, K. S., & Munafo, M. R. (2017) · Wiley eBooks · doi:10.1002/9781119095910.ch2 · Cited by 14 · see citing works
Exploring the effects of non-monetary reimbursement for participants in HCI research
Wiseman, S., Cox, A. L., Gould, S. J. J., & Brumby, D. P. (2017) · Human Computation · doi:10.15346/hc.v4i1.1 · Cited by 13 · see citing works
Waldman, I. D., & Lilienfeld, S. O. (2015) · Psychometrika · doi:10.1007/s11336-015-9447-z · Cited by 13 · see citing works
Nicht replizieren: publizieren!?
Rost, D. H., & Bienefeld, M. A. (2019) · Zeitschrift für Pädagogische Psychologie · doi:10.1024/1010-0652/a000253 · Cited by 13 · see citing works
Back to the Beginning — The Journal is Dead, Long Live Science
Nwagwu, W. E., & Onyancha, O. B. (2015) · The Journal of Academic Librarianship · doi:10.1016/j.acalib.2015.06.005 · Cited by 11 · see citing works
Parra, F., Scherer, S. J., Benezeth, Y., Tsvetanova, P., & Tereno, S. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2019.2921311 · Cited by 10 · see citing works
Reis, D., & Friese, M. (2022) · Springer eBooks · doi:10.1007/978-3-031-04968-2_5 · Cited by 10 · see citing works
Brown, P. (2012) · EMBO Reports · doi:10.1038/embor.2012.147 · Cited by 10 · see citing works
When quotes matter: impact of outside quotes in a science press release on news judgment
Jarreau, P. (2014) · Journal of Science Communication · doi:10.22323/2.13040202 · Cited by 10 · see citing works
How Do Entomologists Consume and Produce Their Science?
Chouvenc, T., & Su, N. (2015) · American Entomologist · doi:10.1093/ae/tmv067 · Cited by 8 · see citing works
Probing Word and Sentence Embeddings for Long-distance Dependencies Effects in French and English
Merlo, P. (2019) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/w19-4817 · Cited by 8 · see citing works
The diktat of concision: When accounting for words shrinks academic knowledge
Guénin-Paracini, H., Gendron, Y., & Moralès, J. (2024) · Critical Perspectives on Accounting · doi:10.1016/j.cpa.2024.102732 · Cited by 7 · see citing works
Replicability of Social-Cognitive Priming: The Case of Weight as an Embodiment of ImportancePreprint
IJzerman, H., Padiotis, N., & Koole, S. L. (2013) · SSRN Electronic Journal · doi:10.2139/ssrn.2255726 · Cited by 7 · see citing works
An Evidence-Hierarchical Decision Aid for Ranking in Evidence-Based Medicine
Landes, J. (2020) · Boston studies in the philosophy of science · doi:10.1007/978-3-030-29179-2_11 · Cited by 6 · see citing works
Kemp, M. W., Pajic, M., Molloy, T. J., & Chapman, E. (2013) · International journal of doctoral studies · doi:10.28945/1893 · Cited by 5 · see citing works
Dix règles simples pour rédiger une revue de littérature
Pautasso, M. (2014) · Kinésithérapie la Revue · doi:10.1016/j.kine.2014.01.008 · Cited by 4 · see citing works
ECCO – A new initiative to support early-career researchers in the complement field
Poppelaars, F., da Costa, M. G., Lokki, A. I., Mallah, K., Nord, D., Reddaway, J., & Schäfer, N. (2021) · Molecular Immunology · doi:10.1016/j.molimm.2021.11.018 · Cited by 4 · see citing works
Publishing papers that matter.
Morrow, D. G. (2018) · Journal of Experimental Psychology Applied · doi:10.1037/xap0000141 · Cited by 4 · see citing works
How Cross-Discipline Understanding and Communication Can Improve Research on Multiracial Populations
Gaither, S. E., & Sims, J. P. (2022) · Social Sciences · doi:10.3390/socsci11030090 · Cited by 3 · see citing works
Preregistration: the good, the bad, and the confusingPreprint
Alain De Cheveigné, A. (2022) · doi:10.31234/osf.io/bcd9t · Cited by 2 · see citing works
Kara, E. (2023) · Anadolu Journal Of Educational Sciences International · doi:10.18039/ajesi.1240655 · Cited by 1 · see citing works
Haslam, N. (2012) · Measurement Interdisciplinary Research and Perspectives · doi:10.1080/15366367.2012.716258 · Cited by 1 · see citing works
Doctoral learning in the biomedical sciences
Kemp, M. W. (2014) · UWA Profiles and Research Repository (UWA)
Intrapersonal Externalities: When Decisions Help or Hinder your Future Self
Stillwell, D. J. (2013) · Nottingham ePrints (University of Nottingham)
Integriteit in kwantitatief, empirisch onderzoek
Hornikx, J., & Batenburg, A. (2016) · Tijdschrift voor taalbeheersing · doi:10.5117/tvt2016.2.horn
Siddaway, A. P. (2017) · Stirling Online Research Repository (University of Stirling)
Festina Lente: A Farewell from the Editor
Merlo, P. (2018) · Computational Linguistics · doi:10.1162/coli_e_00320
Bertamini, M. (2022) · Perception · doi:10.1177/03010066221135720
Self-citationAcademic Writing and Its Impediments among Academicians in a Southern Nigerian State
Ijah, R. F. O. A., Eli, S., Akoko, S., Babatunde, S., & Mato, C. N. (2025) · European Journal of Theoretical and Applied Sciences · doi:10.59324/ejtas.2025.3(1).30
- Authors
- Bertamini, M., & Munafò, M. R.
- Affiliations
- University of Liverpool (GB)
- University of Bristol
- ORCID
- Bertamini, Marco
- Venue
- Perspectives on Psychological Science
- Publisher
- SAGE Publications
- Published
- 2012-01
- Volume
- 7
- Issue
- 1
- Pages
- 67-71
- ISSN
- 1745-6916 (ISSN-L)
- 1745-6924
- OpenAlex
- W2023964994
- Cited by
- 62
- References
- 23
- Citations by year
- 2025: 1 · 2024: 1 · 2023: 1 · 2022: 5 · 2021: 1 · 2020: 3 · 2019: 7 · 2018: 2 · 2017: 7 · 2016: 3 · 2015: 6 · 2014: 6 · 2013: 12 · 2012: 7
- FWCI
- 11.30
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Perspectives on Psychological Science (publishedVersion)
- PubMed (publishedVersion)
- Bristol Research (University of Bristol) (publishedVersion)
- Explore Bristol Research (submittedVersion)
- Padua Research Archive (University of Padova) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
- Keywords
- publication bias, sample size, false positive rate, statistical power, diagnostic sensitivity, research productivity, novelty value, rapid publication
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
