Implementing Systematic Review at the National Toxicology Program: Status and Next Steps
Open accessThe DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.
Abstract
The National Toxicology Program (NTP), an interagency program headquartered at the National Institute of Environmental Health Sciences (NIEHS), carries out a broad range of toxicology research and testing and serves as a resource for identification of substances in our environment that are hazards for human health. One of the ways that the NTP identifies hazards is through carrying out literature-based health assessments. Approximately 2 years ago we began exploring systematic-review methodology as a means to enhance transparency and increase efficiency in summarizing and synthesizing findings from studies in our literature-based health assessments. A systematic review uses an explicit, prespecified approach to identify, select, assess, and appraise the data from studies that focus on addressing a specific scientific question (Institute of Medicine 2011). Although traditionally used to grade the quality of evidence and strength of scientific support for recommendations for clinical practice guidelines and healthcare interventions [Agency for Healthcare Research and Quality (AHRQ) 2012; Guyatt et al. 2011; Higgins and Green 2011], we—and others—were interested in how systematic review methodology might be applied to environmental health questions (Agency for Toxic Substances and Disease Registry 2012; National Research Council 2011; Silbergeld and Scherer 2013; U.S. Environmental Protection Agency 2013; Woodruff and Sutton 2011). With the establishment of the Office of Health Assessment and Translation (OHAT) in 2011, the NIEHS launched a new problem-solving resource for the NTP, particularly with respect to identification of noncancer hazards in our environment (Bucher et al. 2011). OHAT took the lead in investigating how systematic review methodology might be used by the NTP. We embraced systematic review methodology as a useful approach for providing thorough documentation of the steps, inputs, and decisions in a literature-based evaluation. However, we also recognized the necessity to extend existing systematic review methods to accommodate our need in environmental health to integrate data from multiple evidence streams (human, animal, in vitro) and focus on observational human studies rather than on the randomized clinical trials more commonly encountered in the field of health-care intervention (NTP 2012a, 2012b). In late February 2013, the NTP released the Draft OHAT Approach for Systematic Review and Evidence Integration for Literature-based Health Assessments – February 2013 [Draft OHAT Approach; Department of Health and Human Services (DHHS) 2013] for public comment; the deadline for receipt of comments is 11 June 2013. The Draft OHAT Approach adopts or adapts guidance from authoritative systematic review groups (AHRQ 2012; Guyatt et al. 2011; Higgins and Green 2011) to handle the breadth of data from human, animal, in vitro, and mechanistic studies relevant for addressing environmental health questions. In developing a draft approach, OHAT sought advice on systematic review through educational webinars and consultation with technical experts, the NTP Executive Committee, a working group of the NTP Board of Scientific Counselors, the NTP Board of Scientific Counselors, and the public. The draft approach involves a seven-step framework for incorporating systematic review methodology into OHAT literature-based health assessments. In early April of 2013, OHAT will release protocols for two case studies to illustrate application of this framework in specific evaluations. We will test our approach in these case studies to help determine whether additional refinement or revision to the Draft OHAT Approach might be needed. To help the public understand the draft approach and protocols, the NTP will hold a web-based informational meeting on 23 April 2013 to provide an overview of the framework, describe the contents of the case-study protocols, and respond to questions (DHHS 2013). Our intent is to carefully consider all public comments received on the draft approach and to present the Draft OHAT Approach to the NTP Board of Scientific Counselors at its meeting on 25–26 June 2013, with discussion by the NTP of any plans to update the document on the basis of the public’s input. Moving forward, our goal is to increase efficiency and provide greater transparency to the rigorous and objective approach that has been the hallmark of OHAT literature-based health assessments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Woodruff, T. J., Sutton, P., & The Navigation Guide Work Group (2011) · Health Affairs · doi:10.1377/hlthaff.2010.1219 · Cited by 134
Cited by
Overview of known plastic packaging-associated chemicals and their hazards
Groh, K. J., Backhaus, T., Carney-Almroth, B., Geueke, B., Inostroza, P. A., Lennquist, A., Leslie, H. A., Maffini, M., Slunge, D., Trasande, L., et al. (2019) · Science of The Total Environment · doi:10.1016/j.scitotenv.2018.10.015 · Cited by 913
Rooney, A. A., Boyles, A. L., Wolfe, M. S., Bucher, J. R., & Thayer, K. A. (2014) · Environmental Health Perspectives · doi:10.1289/ehp.1307972 · Cited by 434
Self-citationWoodruff, T. J., & Sutton, P. (2014) · Environmental Health Perspectives · doi:10.1289/ehp.1307175 · Cited by 427
“Although novel aspects of the method will require further development and validation, our findings demonstrated that improved methods of research synthesis under development at the National Toxicology Program and under consideration by the U.S. Environmental Protection Agency are fully achievable.”
MethodInfluentialPound, P., & Ritskes-Hoitinga, M. (2018) · Journal of Translational Medicine · doi:10.1186/s12967-018-1678-1 · Cited by 419
Ambient temperature and mental health: a systematic review and meta-analysis
Thompson, R., Lawrance, E. L., Roberts, L. F., Grailey, K., Ashrafian, H., Maheswaran, H., Toledano, M. B., & Darzi, A. (2023) · The Lancet Planetary Health · doi:10.1016/s2542-5196(23)00104-3 · Cited by 306
Metadata
- Authors
- Birnbaum, L. S., Thayer, K. A., Bucher, J. R., & Wolfe, M. S.
- Venue
- Environmental Health Perspectives
- Publisher
- American Chemical Society (ACS)
- Published
- 2013-04
- Volume
- 121
- Issue
- 4
- Pages
- a108-a109
- ISSN
- 0091-6765 (ISSN-L)
- 1552-9924
- OpenAlex
- W2023196891
- Cited by
- 33
- References
- 1
- FWCI
- 1.24
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.