CoChat

A checklist is associated with increased quality of reporting preclinical biomedical research: A systematic review

Open access

The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.

Abstract

Irreproducibility of preclinical biomedical research has gained recent attention. It is suggested that requiring authors to complete a checklist at the time of manuscript submission would improve the quality and transparency of scientific reporting, and ultimately enhance reproducibility. Whether a checklist enhances quality and transparency in reporting preclinical animal studies, however, has not been empirically studied. Here we searched two highly cited life science journals, one that requires a checklist at submission (Nature) and one that does not (Cell), to identify in vivo animal studies. After screening 943 articles, a total of 80 articles were identified in 2013 (pre-checklist) and 2015 (post-checklist), and included for the detailed evaluation of reporting methodological and analytical information. We compared the quality of reporting preclinical animal studies between the two journals, accounting for differences between journals and changes over time in reporting. We find that reporting of randomization, blinding, and sample-size estimation significantly improved when comparing Nature to Cell from 2013 to 2015, likely due to implementation of a checklist. Specifically, improvement in reporting of the three methodological information was at least three times greater when a mandatory checklist was implemented than when it was not. Reporting the sex of animals and the number of independent experiments performed also improved from 2013 to 2015, likely from factors not related to a checklist. Our study demonstrates that completing a checklist at manuscript submission is associated with improved reporting of key methodological information in preclinical animal studies.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1371/journal.pone.0183591

Cites

33 of 33 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig 1 Outline of the study. (A) Selection of articles: Twenty consecutive articles that met the inclusion criteria among those published beginning in January for both 2013 and 2015 in Nature (one that implemented a pre-submission checklist) and Cell (one that did not) journals. This represents articles from periods of time before and after the implementation of the checklist in May 2013. (B) Flow of the analysis: To examine whether quality of reporting has improved over time, the degree of key information reported in 2015 was compared to that in 2013 in both journals combined (Objective 1). To assess whether a checklist is associated with improved quality in reporting, we first compared the changes over time observed in Nature (④ vs. ③). If there was significant difference, we compared time “2015 vs. 2013” in Cell (② vs. ①) and Nature vs. Cell within 2013 (③ vs. ①) and 2015 (④ vs. ②) to adjust for differences between journals and changes over time in reporting (Objective 2).
Fig 1 Outline of the study. (A) Selection of articles: Twenty consecutive articles that met the inclusion criteria among those published beginning in January for both 2013 and 2015 in Nature (one that implemented a pre-submission checklist) and Cell (one that did not) journals. This represents articles from periods of time before and after the implementation of the checklist in May 2013. (B) Flow of the analysis: To examine whether quality of reporting has improved over time, the degree of key information reported in 2015 was compared to that in 2013 in both journals combined (Objective 1). To assess whether a checklist is associated with improved quality in reporting, we first compared the changes over time observed in Nature (④ vs. ③). If there was significant difference, we compared time “2015 vs. 2013” in Cell (② vs. ①) and Nature vs. Cell within 2013 (③ vs. ①) and 2015 (④ vs. ②) to adjust for differences between journals and changes over time in reporting (Objective 2).CC BY · PubMed Central
Fig 2 Distribution of reporting study designs across time. The distributions of the reporting status are presented in stacked bar graphs. The numbers inside the stacks are the number of articles corresponding to each percentage. The data for 2013 and 2015 are the total numbers of articles assessed from Cell and Nature within a given year. Fisher exact test was performed to assess the difference in reporting each methodological across time. Significant P values (< 0.05) are provided.
Fig 2 Distribution of reporting study designs across time. The distributions of the reporting status are presented in stacked bar graphs. The numbers inside the stacks are the number of articles corresponding to each percentage. The data for 2013 and 2015 are the total numbers of articles assessed from Cell and Nature within a given year. Fisher exact test was performed to assess the difference in reporting each methodological across time. Significant P values (< 0.05) are provided.CC BY · PubMed Central
Fig 3 The changes in rigorous reporting of study designs by a checklist. The numbers inside the pie charts are the number of articles corresponding to each category. P values < 0.10 using Fisher exact test are provided to compare time 2015 vs. 2013 within the intervention (Nature) or the comparison (Cell) group, or to compare intervention vs comparison group within 2013 and 2015, respectively. ≠ is shown where comparisons between the touching two groups are significantly different with P < 0.05.
Fig 3 The changes in rigorous reporting of study designs by a checklist. The numbers inside the pie charts are the number of articles corresponding to each category. P values < 0.10 using Fisher exact test are provided to compare time 2015 vs. 2013 within the intervention (Nature) or the comparison (Cell) group, or to compare intervention vs comparison group within 2013 and 2015, respectively. ≠ is shown where comparisons between the touching two groups are significantly different with P < 0.05.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 33 references

Research works printed but not linked: 1

  1. Drummond C, editor Replicability is not Reproducibility: Nor is it Good Science. Proc of the Evaluation Methods for Machine Learning Workshop at the 26th ICML; 2009; Montreal, Quebec, Canada.

Cited by

See all 132

Metadata

Authors
Han, S., Olonisakin, T. F., Pribis, J. P., Zupetic, J., Yoon, J. H., Holleran, K. M., Jeong, K., Shaikh, N., Rubio, D. M., & Lee, J. S.
Affiliations
  • University of Pittsburgh (US)
  • Baylor College of Medicine (US)
Corresponding author
Han, SeungHye
Venue
PLOS ONE
Published
2017-09-13
Volume
12
Issue
9
Pages
e0183591
ISSN
1932-6203 (ISSN-L)
OpenAlex
W2755142264
Cited by
131
References
33
Citations by year
2026: 3 · 2025: 15 · 2024: 7 · 2023: 8 · 2022: 11 · 2021: 24 · 2020: 32 · 2019: 18 · 2018: 11 · 2017: 2
FWCI
3.92
Type
journal article
Language
en
Open access
Locations
  • PLoS ONE (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Figshare (submittedVersion, cc-by)
Funders
  • National Institutes of Health (UL1TR001857, HL086884, AI119042)
  • American Heart Association (15POST25700096)
  • breathe Pennsylvania (Lung Health Research Grant)
  • Howard Hughes Medical Institute (Research Fellow Award)
Topics
  • Health Sciences › Veterinary › Small Animals › Animal testing and alternatives
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
Keywords
reporting checklist, preclinical animal studies, research reproducibility, reporting quality, randomization, blinding, sample size determination, nature, cells
MeSH terms
Data Accuracy, Animals, Drug Evaluation, Preclinical, Humans, Publications, Research Design, Reproducibility of Results, Models, Animal, Biomedical Research, Checklist

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: A checklist is associated with increased quality of reporting preclinical biomedical research: A systematic review. https://app.cochat.ai/papers/10.1371/journal.pone.0183591
The paper
Han, S., Olonisakin, T. F., Pribis, J. P., Zupetic, J., Yoon, J. H., Holleran, K. M., Jeong, K., Shaikh, N., Rubio, D. M., & Lee, J. S. (2017). A checklist is associated with increased quality of reporting preclinical biomedical research: A systematic review. PLOS ONE. https://doi.org/10.1371/journal.pone.0183591

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.