Questionable research practices in ecology and evolution
Open accessPublished version of a preprint
Abstract
We surveyed 807 researchers (494 ecologists and 313 evolutionary biologists) about their use of Questionable Research Practices (QRPs), including cherry picking statistically significant results, p hacking, and hypothesising after the results are known (HARKing). We also asked them to estimate the proportion of their colleagues that use each of these QRPs. Several of the QRPs were prevalent within the ecology and evolution research community. Across the two groups, we found 64% of surveyed researchers reported they had at least once failed to report results because they were not statistically significant (cherry picking); 42% had collected more data after inspecting whether results were statistically significant (a form of p hacking) and 51% had reported an unexpected finding as though it had been hypothesised from the start (HARKing). Such practices have been directly implicated in the low rates of reproducible results uncovered by recent large scale replication studies in psychology and other disciplines. The rates of QRPs found in this study are comparable with the rates seen in psychology, indicating that the reproducibility problems discovered in psychology are also likely to be present in ecology and evolution.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Researchers are often unfamiliar with the much larger systemic bias in the publication of research and in the reliability of certain findings and even of entire fields.
“however, researchers are often unfamiliar with the much larger systemic bias in the publication of research and in the reliability of certain findings and even of entire fields [33, 38–40].”
From The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research (2020) · cites this paper as [39]
The Introduction (page not provided) describes publication bias across sciences and explains how it can inflate false positives and undermine reproducibility. But the paper does not establish that researchers are often unfamiliar with those systemic problems; its survey asks about questionable practices and scientific-integrity doubts, not familiarity with publication bias or the reliability of entire fields. (LLM, conf=0.86)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2000) · doi:10.32614/r.manuals · Cited by 355,706 · see citing works
ggplot2: Elegant Graphics for Data Analysis
Wickham, H. (2009) · doi:10.1007/978-0-387-98141-3 · Cited by 14,447 · see citing works
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505 · see citing works
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515 · see citing works
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567 · see citing works
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · see citing works
Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
John, L. K., Loewenstein, G., & Prelec, D. (2012) · Psychological Science · doi:10.1177/0956797611430953 · Cited by 2,239 · see citing works
A farewell to Bonferroni: the problems of low statistical power and publication bias
Nakagawa, S. (2004) · Behavioral Ecology · doi:10.1093/beheco/arh107 · Cited by 2,102 · see citing works
Self-citationThe preregistration revolution
Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1708274114 · Cited by 1,998 · see citing works
HARKing: Hypothesizing after the results are known
Kerr, N. L. (1998) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0203_4 · Cited by 1,989 · see citing works
Pashler, H., & Wagenmakers, E. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612465253 · Cited by 1,321 · see citing works
Negative results are disappearing from most disciplines and countries
Fanelli, D. (2012) · Scientometrics · doi:10.1007/s11192-011-0494-7 · Cited by 1,248 · see citing works
Forstmeier, W., & Schielzeth, H. (2010) · Behavioral Ecology and Sociobiology · doi:10.1007/s00265-010-1038-5 · Cited by 1,117 · see citing works
The economics of reproducibility in preclinical research
Freedman, L. P., Cockburn, I. M., & Simcoe, T. S. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002165 · Cited by 1,055 · see citing works
An Agenda for Purely Confirmatory Research
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., van der Maas, H. L. J., & Kievit, R. A. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612463078 · Cited by 1,042 · see citing works
Sterling, T. D. (1959) · Journal of the American Statistical Association · doi:10.1080/01621459.1959.10501497 · Cited by 964 · cited 2× · see citing works
“The bias exists in many sciences [22], has been documented for decades in some disciplines (e.g., in psychology, see Sterling, 1959 [23]) and may be getting stronger across science, with a detectable increase in the proportion statistically significant results over the last 25 years [24].”
BackgroundThe natural selection of bad science
Smaldino, P. E., & McElreath, R. (2016) · Royal Society Open Science · doi:10.1098/rsos.160384 · Cited by 860 · see citing works
“Positive” results increase down the Hierarchy of the Sciences
Fanelli, D. (2010) · PLoS ONE · doi:10.1371/journal.pone.0010068 · Cited by 734 · see citing works
Sterling, T. D. (1959) · Journal of the American Statistical Association · doi:10.2307/2282137 · Cited by 524 · see citing works
Detecting and avoiding likely false‐positive findings – a practical guide
Forstmeier, W., Wagenmakers, E., & Parker, T. H. (2017) · Biological Reviews · doi:10.1111/brv.12315 · Cited by 436 · see citing works
Self-citationPublic Data Archiving in Ecology and Evolution: How Well Are We Doing?
Roche, D. G., Kruuk, L. E. B., Lanfear, R., & Binning, S. A. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002295 · Cited by 303 · see citing works
Data archiving in ecology and evolution: best practices
Whitlock, M. C. (2010) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2010.11.006 · Cited by 283 · see citing works
Publication bias in ecology and evolution: an empirical assessment using the ‘trim and fill’ method
Jennions, M. D., & Møller, A. P. (2002) · Biological reviews/Biological reviews of the Cambridge Philosophical Society · doi:10.1017/s1464793101005875 · Cited by 279 · see citing works
Questionable Research Practices Revisited
Fiedler, K., & Schwarz, N. (2016) · Social Psychological and Personality Science · doi:10.1177/1948550615612150 · Cited by 279 · see citing works
The Chrysalis Effect: How Ugly Initial Results Metamorphosize Into Beautiful Articles
O’Boyle, E. H., Banks, G. C., & Gonzalez-Mulé, E. (2017) · Journal of Management · doi:10.1177/0149206314527133 · Cited by 246 · see citing works
Dynamic response of RNA editing to temperature in Drosophila
Rieder, L. E., Savva, Y. A., Reyna, M. A., Chang, Y.-J., Dorsky, J. S., Rezaei, A., & Reenan, R. A. (2015) · BMC Biology · doi:10.1186/s12915-014-0111-3 · Cited by 243 · see citing works
The Preregistration RevolutionPreprint
Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2017) · doi:10.31219/osf.io/2dxu5 · Cited by 220 · see citing works
The "File Drawer Problem" of Non-Significant Results: Does It Apply to Biological Research?
Csada, R. D., James, P. C., & Espie, R. H. M. (1996) · Oikos · doi:10.2307/3546355 · Cited by 204 · see citing works
Transparency in Ecology and Evolution: Real Problems, Real Solutions
Parker, T. H., Forstmeier, W., Koricheva, J., Fidler, F., Hadfield, J. D., Chee, Y. E., Kelly, C. D., Gurevitch, J., & Nakagawa, S. (2016) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2016.07.002 · Cited by 201 · see citing works
Self-citationQuestionable research practices among italian research psychologists
Agnoli, F., Wicherts, J. M., Veldkamp, C. L. S., Albiero, P., & Cubelli, R. (2017) · PLoS ONE · doi:10.1371/journal.pone.0172792 · Cited by 160 · see citing works
Kelly, C. D. (2006) · The Quarterly Review of Biology · doi:10.1086/506236 · Cited by 154 · see citing works
Schwab, A., Abrahamson, E., Starbuck, W. H., & Fidler, F. M. (2010) · Organization Science · doi:10.1287/orsc.1100.0557 · Cited by 129 · see citing works
Self-citationArchiving Primary Data: Solutions for Long-Term Studies
Mills, J. A., Teplitsky, C., Arroyo, B., Charmantier, A., Becker, P. H., Birkhead, T. R., Bize, P., Blumstein, D. T., Bonenfant, C., Boutin, S. A., et al. (2015) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2015.07.006 · Cited by 120 · see citing works
Self-citationNakagawa, S., & Parker, T. H. (2015) · BMC Biology · doi:10.1186/s12915-015-0196-3 · Cited by 110 · see citing works
Self-citationLeBel, E. P., Borsboom, D., Giner‐Sorolla, R., Hasselman, F., Peters, K. R., Ratliff, K. A., & Smith, C. T. (2013) · Perspectives on Psychological Science · doi:10.1177/1745691613491437 · Cited by 103 · see citing works
Metaresearch for Evaluating Reproducibility in Ecology and Evolution
Fidler, F. M., Chee, Y. E., Wintle, B. C., Burgman, M. A., Mccarthy, M. A., & Gordon, A. (2016) · BioScience · doi:10.1093/biosci/biw159 · Cited by 98 · see citing works
Self-citationElevating The Status of Code in Ecology
Mislan, K. A. S., Heer, J., & White, E. P. (2015) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2015.11.006 · Cited by 83 · see citing works
Striving for transparent and credible research: practical guidelines for behavioral ecologists
Ihle, M., Winney, I. S., Krystalli, A., & Croucher, M. (2017) · Behavioral Ecology · doi:10.1093/beheco/arx003 · Cited by 69 · see citing works
A survey of publication bias within evolutionary ecology
Cassey, P., Ewen, J. G., Blackburn, T. M., & Møller, A. P. (2004) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rsbl.2004.0218 · Cited by 48 · see citing works
Empowering peer reviewers with a checklist to improve transparency
Parker, T. H., Griffith, S. C., Bronstein, J. L., Fidler, F. M., Foster, S., Fraser, H. S., Forstmeier, W., Gurevitch, J., Koricheva, J., Seppelt, R., et al. (2018) · Nature Ecology & Evolution · doi:10.1038/s41559-018-0545-z · Cited by 43 · see citing works
Self-citationWould Ecology Fail the Repeatability Test?
Schnitzer, S. A., & Carson, W. P. (2016) · BioScience · doi:10.1093/biosci/biv176 · Cited by 35 · see citing works
Mitigating the epidemic of type I error: ecology and evolution can learn from other disciplines
Parker, T. H., & Nakagawa, S. (2014) · Frontiers in Ecology and Evolution · doi:10.3389/fevo.2014.00076 · Cited by 12 · see citing works
Self-citationPreregistration and increased transparency will benefit science
Mellor, D. T. (2017) · European Science Editing · doi:10.31219/osf.io/xsfam · Cited by 2 · see citing works
Flagged first, then by citations
2 of the 45 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- [31]Reuters T. Thomson Reuters Research Analytics Unveils 2013 Edition of Its Journal Citation Reports. PR Newswire. Ipswich, MA; 2013.
- [32]Fraser H, Parker TH, Nakagawa S, Barnett A, Fidler F. Preregistration: Questionable Research Practices in Ecology and Evolution [Internet]. Melbourne: Open Science Framework; 2017 10.17605/OSF.IO/RG2NQ PMC6047784 30011289 · DOI
Cited by
Sorted by citations
The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research
Percie du Sert, N., Hurst, V., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., et al. (2020) · PLOS Biology · doi:10.1371/journal.pbio.3000410 · Cited by 5,939 · see citing works
The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research
du Sert, N. P., Hurst, V., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., et al. (2020) · Experimental Physiology · doi:10.1113/ep088870 · Cited by 3,538 · see citing works
The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research*
Percie du Sert, N., Hurst, V., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., et al. (2020) · Journal of Cerebral Blood Flow & Metabolism · doi:10.1177/0271678x20943823 · Cited by 1,443 · see citing works
The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research
du Sert, N. P., Hurst, V., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., et al. (2020) · British Journal of Pharmacology · doi:10.1111/bph.15193 · Cited by 1,121 · see citing works
The ARRIVE guidelines 2019: updated guidelines for reporting animal researchPreprint
du Sert, N. P., Hurst, V., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., et al. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/703181 · Cited by 1,042 · see citing works
O’Dea, R. E., Lagisz, M., Jennions, M. D., Koricheva, J., Noble, D. W. A., Parker, T. H., Gurevitch, J., Page, M. J., Stewart, G. B., Moher, D., et al. (2021) · Biological reviews/Biological reviews of the Cambridge Philosophical Society · doi:10.1111/brv.12721 · Cited by 771 · see citing works
Self-citationRewriting results sections in the language of evidence
Muff, S., Nilsen, E. B., O’Hara, R. B., & Nater, C. R. (2021) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2021.10.009 · Cited by 639 · see citing works
The ARRIVE guidelines 2.0: updated guidelines for reporting animal research
du Sert, N. P., Hurst, V., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., et al. (2020) · The Journal of Physiology · doi:10.1113/jp280389 · Cited by 623 · see citing works
The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research
du Sert, N. P., Hurst, V., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., et al. (2020) · BMC Veterinary Research · doi:10.1186/s12917-020-02451-y · Cited by 509 · see citing works
The past, present and future of Registered Reports
Chambers, C. D., & Tzavella, L. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01193-7 · Cited by 499 · see citing works
An Excess of Positive Results: Comparing the Standard Psychology Literature With Registered Reports
Scheel, A. M., Schijen, M., & Lakens, D. (2020) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459211007467 · Cited by 359 · see citing works
Addressing context dependence in ecology.
Catford, J., Wilson, J. R. U., Pyšek, P., Hulme, P., & Duncan, R. (2021) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2021.09.007 · Cited by 296 · see citing works
du Sert, N. P., Hurst, V., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., et al. (2020) · BMJ Open Science · doi:10.1136/bmjos-2020-100115 · Cited by 288 · see citing works
Xie, Y., Wang, K., & Kong, Y. (2021) · Science and Engineering Ethics · doi:10.1007/s11948-021-00314-9 · Cited by 205 · see citing works
Reducing bias, increasing transparency and calibrating confidence with preregistration
Hardwicke, T. E., & Wagenmakers, E. (2023) · Nature Human Behaviour · doi:10.1038/s41562-022-01497-2 · Cited by 190 · see citing works
Lessons From Pinocchio: Cues to Deception May Be Highly Exaggerated
Luke, T. (2018) · Perspectives on Psychological Science · doi:10.1177/1745691619838258 · Cited by 135 · see citing works
The ConTraSt database for analysing and comparing empirical studies of consciousness theories
Yaron, I., Melloni, L., Pitts, M., & Mudrik, L. (2022) · Nature Human Behaviour · doi:10.1038/s41562-021-01284-5 · Cited by 147 · see citing works
The replicability crisis and public trust in psychological science
Anvari, F., & Lakens, D. (2018) · Comprehensive Results in Social Psychology · doi:10.1080/23743603.2019.1684822 · Cited by 147 · see citing works
Evidence of questionable research practices in clinical prediction models
White, N. M., Parsons, R., Collins, P. G. S., & Barnett, A. G. (2023) · BMC Medicine · doi:10.1186/s12916-023-03048-6 · Cited by 149 · see citing works
Self-citationKarakochuk, C. D., Hess, S. Y., Moorthy, D., Namaste, S. M. L., Parker, M., Rappaport, A. I., Wegmüller, R., Dary, O., & the HEmoglobin MEasurement (HEME) Working Group (2019) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.14003 · Cited by 135 · see citing works
Caldwell, A. R., Vigotsky, A. D., Nuckols, G. L., Boardley, I. D., Schmidt, J., Tenan, M. S., Škarabot, J., Radel, R., Naughton, M. R., Schoenfeld, B. J., et al. (2020) · Sports Medicine · doi:10.1007/s40279-019-01227-1 · Cited by 117 · see citing works
Modernizing the Bradford Hill criteria for assessing causal relationships in observational data
Cox, L. A. (2018) · Critical Reviews in Toxicology · doi:10.1080/10408444.2018.1518404 · Cited by 103 · see citing works
Isbell, D. R., BROWN, D., Chen, M., Derrick, D. J., Ghanem, R., Arvizu, M. N. G., Schnur, E., Zhang, M., & Plonsky, L. (2022) · Modern Language Journal · doi:10.1111/modl.12760 · Cited by 101 · see citing works
Meta-analysis challenges a textbook example of status signalling and demonstrates publication bias
Sánchez-Tójar, A., Nakagawa, S., Sánchez-Fortún, M., Martin, D., Ramani, S., Girndt, A., Bókony, V., Kempenaers, B., Liker, A., Westneat, D., et al. (2018) · eLife · doi:10.7554/elife.37385 · Cited by 76 · see citing works
Self-citationBoth Questionable and Open Research Practices Are Prevalent in Education Research
Makel, M. C., Hodges, J., Cook, B. G., & Plucker, J. A. (2021) · Educational Researcher · doi:10.3102/0013189x211001356 · Cited by 92 · see citing works
Replications in Comparative Cognition: What Should We Expect and How Can We Improve?
Farrar, B. G., Boeckle, M., & Clayton, N. S. (2020) · Animal Behavior and Cognition · doi:10.26451/abc.07.01.02.2020 · Cited by 86 · see citing works
Casagrande, S., Stier, A., Monaghan, P., Loveland, J. L., Boner, W., Lupi, S., Trevisi, R., & Hau, M. (2020) · Journal of Experimental Biology · doi:10.1242/jeb.222513 · Cited by 85 · see citing works
Where Are the Self-Correcting Mechanisms in Science?
Vazire, S., & Holcombe, A. O. (2022) · Review of General Psychology · doi:10.1177/10892680211033912 · Cited by 81 · see citing works
Why and how we should join the shift from significance testing to estimation
Berner, D., & Amrhein, V. (2022) · Journal of Evolutionary Biology · doi:10.1111/jeb.14009 · Cited by 78 · see citing works
A protocol for reproducible functional diversity analyses
Palacio, F. X., Callaghan, C. T., Cardoso, P., Hudgins, E. J., Jarzyna, M. A., Ottaviani, G., Riva, F., Graco‐Roza, C., Shirey, V. M., & Mammola, S. (2022) · Ecography · doi:10.1111/ecog.06287 · Cited by 78 · see citing works
Prevalence of visual snow syndrome in the UK
Kondziella, D., Olsen, M. H., & Dreier, J. P. (2020) · European Journal of Neurology · doi:10.1111/ene.14150 · Cited by 77 · see citing works
The power of forecasts to advance ecological theory
Lewis, A. S. L., Rollinson, C. R., Allyn, A., Ashander, J., Brodie, S. J., Brookson, C. B., Collins, E. L., Dietze, M. C., Gallinat, A. S., Juvigny‐Khenafou, N. P. D., et al. (2022) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13955 · Cited by 78 · see citing works
Journals’ instructions to authors: A cross-sectional study across scientific disciplines
Malički, M., Aalbersberg, I. J., Bouter, L., & ter Riet, G. (2019) · PLOS ONE · doi:10.1371/journal.pone.0222157 · Cited by 56 · see citing works
Nakagawa, S., Ivimey‐Cook, E. R., Grainger, M., O’Dea, R. E., Burke, S., Drobniak, S. M., Gould, E., Macartney, E. L., Martinig, A. R., Morrison, K., et al. (2023) · Nature Communications · doi:10.1038/s41467-023-37039-1 · Cited by 76 · see citing works
Self-citationThe role of replication studies in ecology
Fraser, H. S., Barnett, A., Parker, T. H., & Fidler, F. M. (2020) · Ecology and Evolution · doi:10.1002/ece3.6330 · Cited by 74 · see citing works
Self-citationStudy preregistration: An evaluation of a method for transparent reporting
Toth, A. A., Banks, G. C., Mellor, D., O’Boyle, E. H., Dickson, A., Davis, D. J., DeHaven, A., Bochantin, J., & Borns, J. (2021) · Journal of Business and Psychology · doi:10.1007/s10869-020-09695-3 · Cited by 73 · see citing works
Towards open, reliable, and transparent ecology and evolutionary biology
O’Dea, R. E., Parker, T. H., Chee, Y. E., Čulina, A., Drobniak, S. M., Duncan, D. H., Fidler, F. M., Gould, E., Ihle, M., Kelly, C. D., et al. (2021) · BMC Biology · doi:10.1186/s12915-021-01006-3 · Cited by 73 · see citing works
Self-citationEmpirical evidence of widespread exaggeration bias and selective reporting in ecology
Kimmel, K. A., Avolio, M. L., & Ferraro, P. J. (2023) · Nature Ecology & Evolution · doi:10.1038/s41559-023-02144-3 · Cited by 73 · see citing works
Creative destruction in science
Tierney, W., Hardy, J. H., Ebersole, C. R., Leavitt, K., Viganola, D., Clemente, E. G., Gordon, M., Dreber, A., Johannesson, M., Pfeiffer, T., et al. (2020) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2020.07.002 · Cited by 64 · see citing works
On the frequency, prevalence, and perceived severity of questionable research practices
Larsson, T., Plonsky, L., Sterling, S., Kytö, M., Yaw, K., & Wood, M. (2023) · Research Methods in Applied Linguistics · doi:10.1016/j.rmal.2023.100064 · Cited by 64 · see citing works
Incidence and prevalence of coma in the UK and the USA
Kondziella, D., Amiri, M., Othman, M., Beghi, E., Bodien, Y. G., Citerio, G., Giacino, J. T., Mayer, S. A., Lawson, T. N., Menon, D. K., et al. (2022) · Brain Communications · doi:10.1093/braincomms/fcac188 · Cited by 63 · see citing works
The benefits of preregistration and Registered Reports
Lakens, D., Mesquida, C., Rasti, S., & Ditroilo, M. (2024) · Evidence-Based Toxicology · doi:10.1080/2833373x.2024.2376046 · Cited by 76 · see citing works
The consequences of checking for zero‐inflation and overdispersion in the analysis of count data
Campbell, H. (2021) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13559 · Cited by 62 · see citing works
Rate and success of study replication in ecology and evolution
Kelly, C. D. (2019) · PeerJ · doi:10.7717/peerj.7654 · Cited by 62 · see citing works
Research Misconduct in Business and Management Studies: Causes, Consequences, and Possible Remedies
Tourish, D., & Craig, R. (2018) · Journal of Management Inquiry · doi:10.1177/1056492618792621 · Cited by 63 · see citing works
Bakker, B. N., Jaidka, K., Dorr, T. G., Fasching, N., & Yphtach Lelkes, B. (2021) · Journal of Communication · doi:10.1093/joc/jqab031 · Cited by 56 · see citing works
Yang, Y., Hillebrand, H., Lagisz, M., Cleasby, I. R., & Nakagawa, S. (2021) · Global Change Biology · doi:10.1111/gcb.15972 · Cited by 60 · see citing works
Self-citationEvidence synthesis for tackling research waste
Grainger, M. J., Bolam, F. C., Stewart, G. B., & Nilsen, E. B. (2020) · Nature Ecology & Evolution · doi:10.1038/s41559-020-1141-6 · Cited by 59 · see citing works
The problem with unadjusted multiple and sequential statistical testing
Albers, C. J. (2019) · Nature Communications · doi:10.1038/s41467-019-09941-0 · Cited by 59 · see citing works
Gould, E., Fraser, H. S., Parker, T., Nakagawa, S., Griffith, S., Vesk, P., Fidler, F., Hamilton, D., Abbey-Lee, R. N., Abbott, J., et al. (2020) · BMC Biology · doi:10.1186/s12915-024-02101-x · Cited by 95 · see citing works
Self-citationSampling bias exaggerates a textbook example of a trophic cascade
(2021) · doi:10.1111/ele.13915 · Cited by 56 · see citing works
Bias in science: natural and social
May, J. (2020) · Synthese · doi:10.1007/s11229-020-02937-0 · Cited by 54 · see citing works
Advances and challenges in ecological connectivity science
Liczner, A., Pither, R., Bennett, J., Bowman, J., Hall, K. R., Fletcher, R. J., Ford, A. T., Michalak, J., Rayfield, B., Wittische, J., et al. (2024) · Ecology and Evolution · doi:10.1002/ece3.70231 · Cited by 54 · see citing works
Mebane, C. A., Sumpter, J. P., Fairbrother, A., Augspurger, T. P., Canfield, T. J., Goodfellow, W. L., Guiney, P. D., LeHuray, A. P., Maltby, L. L., Mayfield, D. B., et al. (2019) · Integrated Environmental Assessment and Management · doi:10.1002/ieam.4119 · Cited by 53 · see citing works
Exploratory and confirmatory research in the open science era
Nilsen, E. B., Bowler, D. E., & Linnell, J. D. C. (2020) · Journal of Applied Ecology · doi:10.1111/1365-2664.13571 · Cited by 55 · see citing works
Bullock, J. M., McCracken, M. E., Bowes, M. J., Chapman, R. E., Graves, A. R., Hinsley, S. A., Hutchins, M. G., Nowakowski, M., Nicholls, D. J. E., Oakley, S., et al. (2021) · Agriculture Ecosystems & Environment · doi:10.1016/j.agee.2021.107582 · Cited by 51 · see citing works
Questionable Research Practices and Open Science in Quantitative Criminology
Chin, J. M., Pickett, J. T., Vazire, S., & Holcombe, A. O. (2021) · Journal of Quantitative Criminology · doi:10.1007/s10940-021-09525-6 · Cited by 59 · see citing works
Ryan, M. J., Akre, K. L., Baugh, A. T., Bernal, X. E., Lea, A. M., Leslie, C., Still, M. B., Wylie, D., & Rand, A. S. (2019) · The American Naturalist · doi:10.1086/704103 · Cited by 50 · see citing works
Making conservation science more reliable with preregistration and registered reports
Parker, T., Fraser, H. S., & Nakagawa, S. (2019) · Conservation Biology · doi:10.1111/cobi.13342 · Cited by 51 · see citing works
Self-citationA comprehensive meta-analysis of character education programs
Brown, M., McGrath, R. E., Bier, M. C., Johnson, K., & Berkowitz, M. W. (2022) · Journal of Moral Education · doi:10.1080/03057240.2022.2060196 · Cited by 46 · see citing works
Replication crisis or an opportunity to improve scientific production?
Navarro, M. D. F., Pascual‐Llobell, J., Pascual‐Soler, M., Perezgonzalez, J. D., & Berríos-Riquelme, J. (2020) · European Journal of Education · doi:10.1111/ejed.12417 · Cited by 46 · see citing works
Quantifying research waste in ecology
Purgar, M., Klanjšček, T., & Čulina, A. (2022) · Nature Ecology & Evolution · doi:10.1038/s41559-022-01820-0 · Cited by 46 · see citing works
Field, S. M., Wagenmakers, E.-J., Kiers, H. A. L., Hoekstra, R., Ernst, A. F., & van Ravenzwaaij, D. (2020) · Royal Society Open Science · doi:10.1098/rsos.181351 · Cited by 43 · see citing works
Do multiple experimenters improve the reproducibility of animal studies?
von Kortzfleisch, V. T., Ambrée, O., Karp, N. A., Meyer, N., Novak, J., Palme, R., Rosso, M., Touma, C., Würbel, H., Kaiser, S., et al. (2022) · PLoS Biology · doi:10.1371/journal.pbio.3001564 · Cited by 41 · see citing works
How expectations of confirmation influence students’ experimentation decisions in introductory labs
Smith, E. M., Stein, M. M., & Holmes, N. G. (2020) · Physical Review Physics Education Research · doi:10.1103/physrevphyseducres.16.010113 · Cited by 41 · see citing works
Four principles for improved statistical ecology
Popovic, G. C., Mason, T. J., Drobniak, S. M., Marques, T. A., Potts, J., Joo, R., Altwegg, R., Burns, C. C. I., Mccarthy, M. A., Johnston, A., et al. (2024) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.14270 · Cited by 45 · see citing works
Self-citationBehavioural lateralization in a detour test is not repeatable in fishes
Roche, D. G., Amcoff, M., Morgan, R., Sundin, J., Andreassen, A. H., Finnøen, M. H., Lawrence, M. J., Henderson, E., Norin, T., Speers‐Roesch, B., et al. (2020) · Animal Behaviour · doi:10.1016/j.anbehav.2020.06.025 · Cited by 39 · see citing works
A How-to Guide for Open-Science Practices in Special Education Research
Cook, B. G., Fleming, J. I., Hart, S. A., Lane, K. L., Therrien, W. J., van Dijk, W., & Wilson, S. E. (2021) · Remedial and Special Education · doi:10.1177/07419325211019100 · Cited by 38 · see citing works
The impact of increasing turbidity on the predator–prey interactions of freshwater fishes
Zanghi, C., & Ioannou, C. C. (2024) · Freshwater Biology · doi:10.1111/fwb.14354 · Cited by 40 · see citing works
Nakagawa, S., Lagisz, M., Yang, Y., & Drobniak, S. M. (2024) · PLoS Biology · doi:10.1371/journal.pbio.3002423 · Cited by 40 · see citing works
Self-citationPrevalence of near-death experiences in people with and without REM sleep intrusion
Kondziella, D., Dreier, J. P., & Olsen, M. H. (2019) · PeerJ · doi:10.7717/peerj.7585 · Cited by 36 · see citing works
Wiik, E., Jones, J. P. G., Pynegar, E., Bottazzi, P., Asquith, N., Gibbons, J., & Kontoleon, A. V. (2020) · Conservation Biology · doi:10.1111/cobi.13508 · Cited by 36 · see citing works
Improving norms in research culture to incentivize transparency and rigor
Mellor, D. T. (2021) · Educational Psychologist · doi:10.1080/00461520.2021.1902329 · Cited by 36 · see citing works
Rabelo, A. L. A., Farias, J. E. M., Sarmet, M. M., Joaquim, T. C. R., Hoersting, R. C., Victorino, L., Modesto, J. G., & Pilati, R. (2019) · International Journal of Psychology · doi:10.1002/ijop.12632 · Cited by 34 · see citing works
Johnsson, J. I., & Näslund, J. (2018) · Reviews in Fish Biology and Fisheries · doi:10.1007/s11160-018-9532-3 · Cited by 34 · see citing works
Public perception of the vegetative state/unresponsive wakefulness syndrome: a crowdsourced study
Kondziella, D., Cheung, M., & Dutta, A. (2019) · PeerJ · doi:10.7717/peerj.6575 · Cited by 33 · see citing works
Towards Bayesian Model-Based Demography
Bijak, J. (2021) · Methodos series · doi:10.1007/978-3-030-83039-7 · Cited by 33 · see citing works
Damestoy, T., Brachi, B., Moreira, X., Jactel, H., Plomion, C., & Castagneyrol, B. (2018) · Tree Physiology · doi:10.1093/treephys/tpy149 · Cited by 31 · see citing works
Schneider, J. W., Allum, N., Andersen, J. P., Petersen, M. B., Madsen, E. B., Mejlgaard, N., & Zachariae, R. H. (2024) · PLoS ONE · doi:10.1371/journal.pone.0304342 · Cited by 30 · see citing works
Is less more? A commentary on the practice of ‘metric hacking’ in animal social network analysis
Webber, Q., Schneider, D., & Wal, E. V. (2020) · Animal Behaviour · doi:10.1016/j.anbehav.2020.08.011 · Cited by 27 · see citing works
Journal quality criteria: Measurement and significance
Kirillova, O. V., & Тихонова, Е. В. (2022) · Science Editor and Publisher · doi:10.24069/sep-22-39 · Cited by 32 · see citing works
Four steps to strengthen connectivity modeling
Riordan‐Short, E., Pither, R., & Pither, J. (2023) · Ecography · doi:10.1111/ecog.06766 · Cited by 28 · see citing works
A Guide and Toolbox to Replicability and Open Science in Entomology
Wittman, J. T., & Aukema, B. H. (2020) · Journal of Insect Science · doi:10.1093/jisesa/ieaa036 · Cited by 26 · see citing works
Are weak dispersers more vulnerable than strong dispersers to land use intensification?
Martin, A. E., Lockhart, J. K., & Fahrig, L. (2023) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2022.0909 · Cited by 26 · see citing works
A community-led initiative for training in reproducible research
Auer, S., Haelterman, N. A. T., Weissgerber, T. L., Erlich, J. C., Susilaradeya, D., Julkowska, M., Gazda, M. A., Schwessinger, B., Jadavji, N. M., Reproducibility for Everyone Team, et al. (2021) · eLife · doi:10.7554/elife.64719 · Cited by 26 · see citing works
Testing an active intervention to deter researchers’ use of questionable research practices
Bruton, S. V., Brown, M., Sacco, D. F., & DIDLAKE, R. H. (2019) · Research Integrity and Peer Review · doi:10.1186/s41073-019-0085-3 · Cited by 23 · see citing works
The illusion of science in comparative cognitionPreprint
Farrar, B. G., & Ostojić, L. (2019) · doi:10.31234/osf.io/hduyx · Cited by 26 · see citing works
Authier, M., Galatius, A., Gilles, A., & Spitz, J. (2020) · PeerJ · doi:10.7717/peerj.9436 · Cited by 24 · see citing works
The Academic, Societal and Animal Welfare Benefits of Open Science for Animal Science
Nawroth, C., & Krause, E. T. (2022) · Frontiers in Veterinary Science · doi:10.3389/fvets.2022.810989 · Cited by 24 · see citing works
Questionable research practices following pre-registrationPreprint
Ikeda, A., Xu, H.-Q., Fuji, N., Zhu, S., & Yamada, Y. (2019) · 心理学評論 · doi:10.24602/sjpr.62.3_281 · Cited by 23 · see citing works
Multi-Scale Drivers of Immunological Variation and Consequences for Infectious Disease Dynamics
Becker, D. J., Downs, C. J., & Martin, L. B. (2019) · Integrative and Comparative Biology · doi:10.1093/icb/icz138 · Cited by 23 · see citing works
Spatiotemporal variation in drivers of parasitism in a wild wood mouse population
Sweeny, A. R., Albery, G. F., Venkatesan, S., Fenton, A., & Pedersen, A. B. (2021) · Functional Ecology · doi:10.1111/1365-2435.13786 · Cited by 22 · see citing works
Open Science and Single-Case Design Research
Cook, B. G., Johnson, A. H., Maggin, D. M., Therrien, W. J., Barton, E. E., Lloyd, J. W., Reichow, B., Talbott, E., & Travers, J. C. (2021) · Remedial and Special Education · doi:10.1177/0741932521996452 · Cited by 22 · see citing works
Norberg, J., Blenckner, T., Cornell, S., Petchey, O. L., & Hillebrand, H. (2022) · Ecology Letters · doi:10.1111/ele.13984 · Cited by 22 · see citing works
Cockburn, A. (2020) · Advances in the study of behavior · doi:10.1016/bs.asb.2019.11.002 · Cited by 22 · see citing works
The Challenges of Replicating Research on Endangered Species
Shaw, R. C., Greggor, A. L., & Plotnik, J. M. (2021) · Animal Behavior and Cognition · doi:10.26451/abc.08.02.10.2021 · Cited by 22 · see citing works
Harjes, J., Link, A., Weibulat, T., Triebel, D., & Rambold, G. (2020) · Database · doi:10.1093/database/baaa059 · Cited by 21 · see citing works
Heterogeneity estimates in a biased world
Hönekopp, J., & Linden, A. (2022) · PLoS ONE · doi:10.1371/journal.pone.0262809 · Cited by 22 · see citing works
Epistemology of Experimental Physics
Boyd, N. M. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108885676 · Cited by 20 · see citing works
Chin, J. M., & Ibaviosa, C. M. (2022) · Science & Justice · doi:10.1016/j.scijus.2022.02.006 · Cited by 20 · see citing works
- Authors
- Fraser, H., Parker, T., Nakagawa, S., Barnett, A., & Fidler, F.
- Affiliations
- The University of Melbourne (AU)
- Whitman College (US)
- UNSW Sydney (AU)
- Corresponding author
- Fraser, Hannah
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2018-07-16
- Volume
- 13
- Issue
- 7
- Pages
- e0200303
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC6047784
- OpenAlex
- W2793861965
- Cited by
- 346
- References
- 45
- Citations by year
- 2026: 17 · 2025: 29 · 2024: 42 · 2023: 51 · 2022: 37 · 2021: 51 · 2020: 57 · 2019: 45 · 2018: 15 · 2017: 2
- FWCI
- 83.45
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- EcoEvoRxiv (submittedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Minerva Access (University of Melbourne) (submittedVersion, cc-by)
- EcoEvoRxiv (submittedVersion)
- OSF Preprints (OSF Preprints) (submittedVersion)
- Funders
- Australian Research Council Future Fellowship (FT150100297)
- Topics
- Social Sciences › Safety Research › Academic integrity and plagiarism
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
- Keywords
- questionable research practices, cherry-picking, p-hacking, HARKing, research integrity, reproducibility, ecology, evolutionary biology, replication study, statistical significance
- MeSH terms
- Ecology, Biological Evolution, Humans, Surveys and Questionnaires, Research, Research Design, Research Personnel, Statistics as Topic, Reproducibility of Results, Scientific Misconduct
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
