Intersection of Systematic Review Methodology with the NIHReproducibility Initiative
Open accessThe DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
In a landmark 2005 paper published in PLoS Medicine, Ioannidis posited that " most current published research findings are false " (Ioannidis 2005). Consistent with this opinion are reports that drug develop ment has been hin dered and many clinical trials wasted based on published findings from pre clinical studies that with further effort could not be reproduced (Begley and Ellis 2012). The National Institutes of Health (NIH) recently outlined a sweeping set of initia tives to address the lack of reproducibility of research findings (Collins and Tabak 2014). In this editorial we touch on current efforts to address the research repro ducibility problem and propose that systematic review methodologies, which are being developed to assess confi dence in the quality of evidence used in reaching public health decisions, could also be used to improve the reproducibly of research. Reports and editorials in the biomedical literature have increas ingly drawn attention to a disturbing lack of reproducibility of pub lished scien tific findings. Although poor reporting of key aspects of study methodology clearly contributes to the problem, other factors such as study conduct, may be equally or more important (Begley and This situa tion has prompted actions by both the private and public sectors. These include the private Reproducibility Initiative collabora tion between Plos One 2013), which among other projects is attempting to replicate key find ings from the 50 most impactful studies published in the field of cancer biology between 2010 and 2012. A major public effort is the NIH Initiative to Enhance Reproducibility and Transparency of Research Findings, which seeks to increase community awareness of the repro ducibility problem, enhance formal training of investigators in elements of proper study design, improve the review of grant applications, and increase funding stability for investigators to enable them to use more appropriate and complex study designs (Tabak 2013). One planned activity of the NIH initiative is to develop a pilot training module on research integrity as it relates to experimental biases and study design. The intention is to provide specific guidance for researchers to improve the quality of their research publications by increasing their awareness of research practices that may affect the validity of their study findings. This guidance could also be used to improve both the grant proposal and journal peerreview stages to ensure more systematic and rigorous evaluation of both proposed and completed studies. Hoojimans and RitskesHoitinga (2013) recently …
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Raise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979 · cited 2×
“Consistent with this opinion are reports that drug develop ment has been hin dered and many clinical trials wasted based on published findings from pre clinical studies that with further effort could not be reproduced (Begley and Ellis 2012).”
BackgroundPolicy: NIH plans to enhance reproducibility
Collins, F. S., & Tabak, L. A. (2014) · Nature · doi:10.1038/505612a · Cited by 1,247 · cited 1×
“The National Institutes of Health (NIH) recently outlined a sweeping set of initia tives to address the lack of reproducibility of research findings (Collins and Tabak 2014).”
BackgroundA call for transparent reporting to optimize the predictive value of preclinical research
Landis, S. C., Amara, S. G., Asadullah, K., Austin, C. P., Blumenstein, R., Bradley, E. W., Crystal, R. G., Darnell, R. B., Ferrante, R. J., Fillit, H., et al. (2012) · Nature · doi:10.1038/nature11556 · Cited by 1,196
Rooney, A. A., Boyles, A. L., Wolfe, M. S., Bucher, J. R., & Thayer, K. A. (2014) · Environmental Health Perspectives · doi:10.1289/ehp.1307972 · Cited by 434
Self-citationEvaluation of excess significance bias in animal studies of neurological diseases
Tsilidis, K. K., Panagiotou, O. A., Sena, E. S., Aretouli, E., Evangelou, E., Howells, D. W., Salman, R. A.-S., Macleod, M. R., & Ioannidis, J. P. A. (2013) · PLoS Biology · doi:10.1371/journal.pbio.1001609 · Cited by 295
Woodruff, T. J., Sutton, P., & The Navigation Guide Work Group (2011) · Health Affairs · doi:10.1377/hlthaff.2010.1219 · Cited by 134
Cited by
Olker, J. H., Elonen, C. M., Pilli, A., Anderson, A., Kinziger, B., Erickson, S., Skopinski, M., Pomplun, A., LaLone, C. A., Russom, C. L., et al. (2022) · Environmental Toxicology and Chemistry · doi:10.1002/etc.5324 · Cited by 301
Irreproducibility in Preclinical Biomedical Research: Perceptions, Uncertainties, and Knowledge Gaps
Jarvis, M. F., & Williams, M. (2016) · Trends in Pharmacological Sciences · doi:10.1016/j.tips.2015.12.001 · Cited by 129
A primer on systematic reviews in toxicology
Hoffmann, S., de Vries, R. B. M., Stephens, M. L., Beck, N. B., Dirven, H. A. A. M., Fowle, J. R., Goodman, J. E., Hartung, T., Kimber, I., Lalu, M. M., et al. (2017) · Archives of Toxicology · doi:10.1007/s00204-017-1980-3 · Cited by 85
Self-citationSamuel, G. O., Hoffmann, S., Wright, R. A., Lalu, M. M., Patlewicz, G., Becker, R. A., DeGeorge, G. L., Fergusson, D., Hartung, T., Lewis, R. J., et al. (2016) · Environment International · doi:10.1016/j.envint.2016.03.010 · Cited by 82
Walker, V. R., Boyles, A. L., Pelch, K. E., Holmgren, S. D., Shapiro, A. J., Blystone, C. R., Devito, M. J., Newbold, R. R., Blain, R., Hartman, P., et al. (2018) · Environment International · doi:10.1016/j.envint.2017.12.032 · Cited by 28
Self-citation
Metadata
- Authors
- Thayer, K. A., Wolfe, M. S., Rooney, A. A., Boyles, A. L., Bucher, J. R., & Birnbaum, L. S.
- Venue
- Environmental Health Perspectives
- Publisher
- American Chemical Society (ACS)
- Published
- 2014-07
- Volume
- 122
- Issue
- 7
- Pages
- A176-A177
- ISSN
- 0091-6765 (ISSN-L)
- 1552-9924
- OpenAlex
- W2023615889
- Cited by
- 29
- References
- 6
- FWCI
- 1.83
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.