A survey on data reproducibility and the effect of publication process on the ethical reporting of laboratory research
Open accessPublished version of a preprint
Abstract
Purpose: The successful translation of laboratory research into effective therapies is dependent upon the validity of peer-reviewed publications. However, several publications in recent years suggested that published scientific findings could be reproduced only 11% to 45% of the time. Multiple surveys attempted to elucidate the fundamental causes of data irreproducibility and underscored potential solutions, more robust experimental designs, better statistics, and better mentorship. However, no prior survey has addressed the role of the review and publication process on honest reporting. Experimental Design: We developed an anonymous online survey intended for trainees involved in bench research. The survey included questions related to mentoring/career development, research practice, integrity, and transparency, and how the pressure to publish and the publication process itself influence their reporting practices. Results: Responses to questions related to mentoring and training practices were largely positive, although an average of approximately 25% did not seem to receive optimal mentoring. A total of 39.2% revealed having been pressured by a principle investigator or collaborator to produce “positive” data. About 62.8% admitted that the pressure to publish influences the way they report data. The majority of respondents did not believe that extensive revisions significantly improved the manuscript while adding to the cost and time invested. Conclusions: This survey indicates that trainees believe that the pressure to publish affects honest reporting, mostly emanating from our system of rewards and advancement. The publication process itself affects faculty and trainees and appears to influence a shift in their ethics from honest reporting (“negative data”) to selective reporting, data falsification, or even fabrication. Clin Cancer Res; 24(14); 3447–55. ©2018 AACR.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Gratuitous data requested in revisions to gene research manuscripts could be more easily generated by paper mills than through genuine research.
“editors and peer reviewers should also ensure that revisions to gene research manuscripts do not require gratuitous data that could be more easily generated by paper mills than through genuine research.”
From Protection of the human gene research literature from contract cheating organizations known as research paper mills (2022) · cites this paper as [81]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
1,500 scientists lift the lid on reproducibility
Baker, M. (2016) · Nature · doi:10.1038/533452a · Cited by 4,209 · see citing works
Raise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979 · cited 2× · see citing works
“Respondent characteristics With our eligibility criteria of (1) being a graduate student or postdoctoral fellow and (2) performing bench science, 467 of our total 576 respondents were deemed eligible.”
BackgroundSelf-citationBelieve it or not: how much can we rely on published data on potential drug targets?
Prinz, F., Schlange, T., & Asadullah, K. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439-c1 · Cited by 1,888 · see citing works
Policy: NIH plans to enhance reproducibility
Collins, F. S., & Tabak, L. A. (2014) · Nature · doi:10.1038/505612a · Cited by 1,247 · see citing works
Peer Review: A Flawed Process at the Heart of Science and Journals
Smith, R. (2006) · Journal of the Royal Society of Medicine · doi:10.1177/014107680609900414 · Cited by 880 · see citing works
The Costs of using Unauthenticated, Over-Passaged Cell Lines: How Much more Data do we Need?
Hughes, P., Marshall, D., Reid, Y., Parkes, H., & Gelber, C. (2007) · BioTechniques · doi:10.2144/000112598 · Cited by 434 · see citing works
The declining interest in an academic career
Roach, M., & Sauermann, H. (2017) · PLoS ONE · doi:10.1371/journal.pone.0184130 · Cited by 245 · see citing works
Matosin, N., Frank, E., Engel, M., Lum, J., & Newell, K. (2014) · Disease Models & Mechanisms · doi:10.1242/dmm.015123 · Cited by 209 · see citing works
Causes for the Persistence of Impact Factor Mania
Casadevall, A., & Fang, F. C. (2014) · mBio · doi:10.1128/mbio.00064-14 · Cited by 200 · see citing works
The ghosts of HeLa: How cell line misidentification contaminates the scientific literature
Horbach, S. P. J. M., & Halffman, W. (2017) · PLOS ONE · doi:10.1371/journal.pone.0186281 · Cited by 167 · cited 1× · see citing works
“A more recent publication estimated a 20% cross-contamination rate (10, 11).”
BackgroundMobley, A. K., Linder, S. K., Braeuer, R. R., Ellis, L. M., & Zwelling, L. A. (2013) · PLoS ONE · doi:10.1371/journal.pone.0063221 · Cited by 164 · see citing works
Self-citationChatterjee, R. (2007) · Science · doi:10.1126/science.315.5814.928 · Cited by 115 · see citing works
Cancer reproducibility project releases first results
Baker, M., & Dolgin, E. (2017) · Nature · doi:10.1038/541269a · Cited by 104 · see citing works
Modelling the effects of subjective and objective decision making in scientific peer review
Park, I.-U., Peacey, M. W., & Munafò, M. R. (2014) · Nature · doi:10.1038/nature12786 · Cited by 103 · see citing works
Bailey, B., Einarson, A., & Koren, G. (2018) · The Canadian Journal of Hospital Pharmacy · doi:10.4212/cjhp.v50i2.2026 · Cited by 89 · see citing works
Mix-ups and mycoplasma: the enemies within
Drexler, H. G., Uphoff, C. C., Dirks, W. G., & MacLeod, R. A. F. (2002) · Leukemia Research · doi:10.1016/s0145-2126(01)00136-9 · Cited by 66 · see citing works
Neimark, J. (2015) · Science · doi:10.1126/science.347.6225.938 · Cited by 57 · see citing works
Visualizing detailed postdoctoral employment trends using a new career outcome taxonomy
Xu, H., Gilliam, R. S. T., Peddada, S. D., Buchold, G. M., & Collins, T. (2018) · Nature Biotechnology · doi:10.1038/nbt.4059 · Cited by 43 · see citing works
The Culture of Cell Culture Practices and Authentication—Results from a 2015 Survey
Freedman, L. P., Gibson, M. C., Wisman, R., Ethier, S. P., Soule, H. R., Reid, Y. A., & Neve, R. M. (2015) · BioTechniques · doi:10.2144/000114344 · Cited by 31 · see citing works
Graduate survey: Uncertain futures
Woolston, C. (2015) · Nature · doi:10.1038/nj7574-597a · Cited by 25 · see citing works
Negative results are published
O’Hara, R. B. (2011) · Nature · doi:10.1038/471448e · Cited by 21 · see citing works
Many junior scientists need to take a hard look at their job prospects
(2017) · Nature · doi:10.1038/550429a · Cited by 21 · see citing works
The erosion of research integrity: the need for culture change
Ellis, L. M. (2015) · The Lancet Oncology · doi:10.1016/s1470-2045(15)00085-6 · Cited by 9 · see citing works
Self-citationIt Is Time to Re‐Evaluate the Peer Review Process for Preclinical Research
Bhattacharya, R., & Ellis, L. M. (2017) · BioEssays · doi:10.1002/bies.201700185 · Cited by 9 · see citing works
Self-citationAnthropology: follow field primatologists
Nekaris, K. A. I., Nijman, V., & Godfrey, L. R. (2011) · Nature · doi:10.1038/471448c · Cited by 4 · see citing works
Flagged first, then by citations
1 of the 26 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- [15]Finn CE 2002 The Limits of Peer Review Education Week
Cited by
Sorted by citations
Requirements and reliability of AI in the medical context
Balagurunathan, Y., Mitchell, R., & Naqa, I. E. (2021) · Physica Medica · doi:10.1016/j.ejmp.2021.02.024 · Cited by 91 · see citing works
Ten simple rules towards healthier research labs
Maestre, F. T. (2019) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006914 · Cited by 74 · see citing works
Festing, M. F. W. (2020) · Scientific Reports · doi:10.1038/s41598-020-74538-3 · Cited by 58 · see citing works
Byrne, J. A., Park, Y., Richardson, R. A. K., Pathmendra, P., Sun, M., & Stoeger, T. (2022) · Nucleic Acids Research · doi:10.1093/nar/gkac1139 · Cited by 46 · see citing works
Amangeldy, B., Tasmurzayev, N., Imankulov, T., Baigarayeva, Z., Izmailov, N., Riza, T., Abdukarimov, A., Mukazhan, M., & Zhumagulov, B. (2025) · Sensors · doi:10.3390/s25175265 · Cited by 41 · see citing works
A qualitative study of Equal Co-First Authorship
Hosseini, M., & Bruton, S. V. (2020) · Accountability in Research · doi:10.1080/08989621.2020.1776122 · Cited by 33 · see citing works
Rodgers, P., & Collings, A. (2021) · eLife · doi:10.7554/elife.75830 · Cited by 33 · see citing works
Sensor Reproducibility Analysis: Challenges and Potential Solutions
Sekhar, P. K., Billey, W., Begay, M., Thomas, B., Woody, C., & Soundappan, T. (2024) · ECS Sensors Plus · doi:10.1149/2754-2726/ad9936 · Cited by 20 · see citing works
Enhancing reproducibility using interprofessional team best practices
Rolland, B., Burnside, E. S., Voils, C. I., Shah, M. N., & Brasier, A. R. (2020) · Journal of Clinical and Translational Science · doi:10.1017/cts.2020.512 · Cited by 21 · see citing works
Tagne, A. M., Cassina, N., Furgiuele, A., Storelli, E., Cosentino, M., & Marino, F. (2020) · Journal of Academic Ethics · doi:10.1007/s10805-020-09359-0 · Cited by 19 · see citing works
The Evolution and Reliability of Machine Learning Techniques for Oncology
Owida, H. A., Moh’d, B. A.-H., Turab, N. M., Al-Nabulsi, J., & Abuowaida, S. F. A. (2023) · International Journal of Online and Biomedical Engineering (iJOE) · doi:10.3991/ijoe.v19i08.39433 · Cited by 18 · see citing works
Kedron, P., Holler, J., & Bardin, S. (2023) · Annals of the American Association of Geographers · doi:10.1080/24694452.2023.2276115 · Cited by 13 · see citing works
Gonzalez, J. A. H., & Colom, M. (2025) · Frontiers in Computer Science · doi:10.3389/fcomp.2024.1491823 · Cited by 12 · see citing works
Friedrich, J., Lindauer, U., & Höllig, A. (2022) · Frontiers in Neurology · doi:10.3389/fneur.2022.834003 · Cited by 13 · see citing works
Publication pressure and scientific misconduct: why we need more open governance
Gandevia, S. C. (2018) · Spinal Cord · doi:10.1038/s41393-018-0193-9 · Cited by 12 · see citing works
Scientific Discovery at the Press of a Button: Navigating Emerging Cloud Laboratory Technology
Arias, D. S., & Taylor, R. E. (2024) · Advanced Materials Technologies · doi:10.1002/admt.202400084 · Cited by 12 · see citing works
Evaluation of reproducible and transparent research practices in pulmonology
Smith, C., Nolan, J., Tritz, D., Heavener, T. E., Pelton, J., Cook, K., & Vassar, M. B. (2020) · Pulmonology · doi:10.1016/j.pulmoe.2020.07.001 · Cited by 10 · see citing works
Issues and Challenges to Reproducibility of Cancer Research: A Commentary
da Silva, J. A. T. (2022) · Future Oncology · doi:10.2217/fon-2021-1378 · Cited by 8 · see citing works
Wirz, C. D., Sutter, C., Demuth, J. L., Mayer, K. J., Chapman, W. E., Cains, M. G., Radford, J. T., Przybylo, V. M., Evans, A., Martin, T., et al. (2024) · Earth and Space Science · doi:10.1029/2023ea003364 · Cited by 7 · see citing works
Promoting and Enabling Reproducible Data Science Through a Reproducibility Challenge
Liu, J., Carlson, J., Josh Pasek, B., Puchala, B., Rao, A., & Jagadish, H. V. (2022) · Harvard Data Science Review · doi:10.1162/99608f92.9624ea51 · Cited by 7 · see citing works
Fair credit? The impact of shared first authorship on academic career evaluation
Decius, J., & Schilbach, M. (2025) · Scientometrics · doi:10.1007/s11192-025-05262-w · Cited by 5 · see citing works
What’s Right and Wrong in Preclinical Science: A Matter of Principled Investigation
Smith, L. N. (2022) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2022.805661 · Cited by 4 · see citing works
Ten tips for teaching research integrity to early career students: A perspective over 20 years
Martinez-Campos, M. (2022) · Frontiers in Research Metrics and Analytics · doi:10.3389/frma.2022.989668 · Cited by 4 · see citing works
Scholarly Publishing and Scientific Reproducibility
Bomzon, A., & Tobin, G. (2021) · Laboratory animal science and medicine · doi:10.1007/978-3-030-66147-2_9 · Cited by 2 · see citing works
Machine learning-assisted decoding of temporal transcriptional dynamics via fluorescent timer
Irie, N., Takeda, N., Satou, Y., Araki, K., & Ono, M. (2025) · Nature Communications · doi:10.1038/s41467-025-61279-y · Cited by 3 · see citing works
Irreproducibility –The deadly sin of preclinical research in drug development
Pillai, S., Kobayashi, K., Michael, M., & Meena, A. (2020) · Journal of Pre-Clinical and Clinical Research · doi:10.26444/jpccr/131017 · Cited by 1 · see citing works
Comment on “A Roadmap for Aspiring Surgeon-Scientists in Today's Healthcare Environment”
Vega, E. A., & Ellis, L. M. (2019) · Annals of Surgery · doi:10.1097/sla.0000000000003334 · Cited by 1 · see citing works
Self-citationEvaluation of Reproducible and Transparent Research Practices in Pulmonology PublicationsPreprint
Smith, C. A., Nolan, J., Tritz, D. J., Heavener, T. E., Pelton, J., Cook, K., & Vassar, M. B. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/726505 · Cited by 1 · see citing works
Zhu, F., Zhang, Q., Chen, H., Shi, G., Chen, W. L., Zhu, Z., & Chen, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.08800 · Cited by 1 · see citing works
Workshop on reproducibility in research
KALICHMAN, M. W., & Mills, P. J. (2020) · Journal of Clinical and Translational Science · doi:10.1017/cts.2020.496
Faculty Opinions recommendation of Ten simple rules towards healthier research labs.
Hamann, T. (2019) · Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature · doi:10.3410/f.735526979.793560932
Faculty Opinions recommendation of Ten simple rules towards healthier research labs.
Amaral, L. A. N. (2019) · Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature · doi:10.3410/f.735526979.793558970
Tourish, D. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108616669.012
Faculty Opinions recommendation of Ten simple rules towards healthier research labs.
Araújo, W. L. (2019) · Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature · doi:10.3410/f.735526979.793559844
Redman, B. K. (2023) · Collaborative bioethics · doi:10.1007/978-3-031-29455-6_2
Fabrication, Falsification and Spin
Crombie, I. (2021) · doi:10.1002/9781394322183.ch6
Sangal, L., Sood, V., Haar, K., Moyana, T. S., Jinnai, Y., & Rijal, S. (2025) · Research Square · doi:10.21203/rs.3.rs-7998434/v1
Sangal, L., Sood, V., Haar, K., Silubonde, T. M., Jinnai, Y., & Rijal, S. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1755256
Palumbo, P., Kruger, J., & Robinson, C. L. (2026) · Current Pain and Headache Reports · doi:10.1007/s11916-026-01499-z
Kozula, M., DeVito, N., Onghena, P., Colombo, C., Dudda, L. A., Teno, P. M., & den Eynden, V. V. (2026) · PLoS ONE · doi:10.1371/journal.pone.0348512
Overcoming animal methods bias and its impacts on nonanimal research assessments
Trunnell, E. R., & Krebs, C. E. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-26514-3.00012-6
- Authors
- Boulbes, D. R., Costello, T., Baggerly, K., Fan, F., Wang, R., Bhattacharya, R., Ye, X., & Ellis, L. M.
- Affiliations
- 1Department of Surgical Oncology, University of Texas MD Anderson Cancer Center, Houston, Texas.
- Moffitt Cancer Center (US)
- The University of Texas MD Anderson Cancer Center (US)
- Corresponding author
- Ellis, Lee M.
- Venue
- Clinical Cancer Research
- Published
- 2018-07-15
- Volume
- 24
- Issue
- 14
- Pages
- 3447-3455
- ISSN
- 1078-0432 (ISSN-L)
- 1557-3265
- PMC
- PMC6050098
- OpenAlex
- W2797795519
- Cited by
- 41
- References
- 26
- Citations by year
- 2026: 4 · 2025: 5 · 2024: 3 · 2023: 3 · 2022: 6 · 2021: 4 · 2020: 8 · 2019: 7 · 2018: 1
- FWCI
- 9.96
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Clinical Cancer Research (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- HHS | National Institutes of Health (NIH) (CA016672)
- William C. Liedtke, Jr Chair in Cancer Research
- Topics
- Social Sciences › Safety Research › Academic integrity and plagiarism
- Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Academic Writing and Publishing
- Keywords
- research reproducibility, publication pressure, selective reporting, research integrity, research ethics, mentoring, peer review, data falsification, research transparency, research misconduct
- MeSH terms
- Humans, Professional Practice, Publications, Surveys and Questionnaires, Research, Research Personnel, Students, Reproducibility of Results, Internet, Ethics, Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
