Systematic Review and Evidence Integration for Literature-Based Environmental Health Science Assessments
Open accessThe DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index; 4 of the 28 sources its claims cite were checked for support.
Abstract
Background: Systematic-review methodologies provide objectivity and transparency to the process of collecting and synthesizing scientific evidence in reaching conclusions on specific research questions. There is increasing interest in applying these procedures to address environmental health questions. Objectives: The goal was to develop a systematic-review framework to address environmental health questions by extending approaches developed for clinical medicine to handle the breadth of data relevant to environmental health sciences (e.g., human, animal, and mechanistic studies). Methods: The Office of Health Assessment and Translation (OHAT) adapted guidance from authorities on systematic-review and sought advice during development of the OHAT Approach through consultation with technical experts in systematic review and human health assessments, as well as scientific advisory groups and the public. The method was refined by considering expert and public comments and through application to case studies. Results and Discussion: Here we present a seven-step framework for systematic review and evidence integration for reaching hazard identification conclusions: 1) problem formulation and protocol development, 2) search for and select studies for inclusion, 3) extract data from studies, 4) assess the quality or risk of bias of individual studies, 5) rate the confidence in the body of evidence, 6) translate the confidence ratings into levels of evidence, and 7) integrate the information from different evidence streams (human, animal, and “other relevant data” including mechanistic or in vitro studies) to develop hazard identification conclusions. Conclusion: The principles of systematic review can be successfully applied to environmental health questions to provide greater objectivity and transparency to the process of developing conclusions. Citation: Rooney AA, Boyles AL, Wolfe MS, Bucher JR, Thayer KA. 2014. Systematic review and evidence integration for literature-based environmental health science assessments. Environ Health Perspect 122:711–718; http://dx.doi.org/10.1289/ehp.1307972
Overview
CoChat Summary
TL;DR
This paper presents the OHAT seven-step framework for systematic review and evidence integration in environmental health assessments, extending clinical systematic-review methods to human, animal, and other relevant data. The approach is designed to improve objectivity, transparency, and consistency in hazard identification conclusions.
Key findings
- The OHAT framework is a flexible seven-step process: 1) problem formulation and protocol development, 2) search for and select studies, 3) extract data, 4) assess study quality/risk of bias, 5) rate confidence in the body of evidence, 6) translate confidence ratings into levels of evidence, and 7) integrate evidence streams to develop hazard identification conclusions.
- In applying the OHAT Approach to case studies, steps 2–4 were the most time intensive: selecting studies, extracting data, and assessing the quality of individual studies.
- The framework defines five hazard identification conclusion categories: known, presumed, suspected, not classifiable, and not identified as a hazard to humans.
- The paper states that high confidence means further research is very unlikely to change the apparent relationship, while very low confidence means further research is very likely to change it; for a conclusion of evidence of no health effect, the paper says this is reached only when there is high confidence.
Methods
- The authors developed the OHAT Approach by adapting guidance from authoritative systematic-review groups and consulting technical experts, scientific advisory groups, and the public; the method was refined through comments and application to case studies.
- The framework was built to cover human, animal, and other relevant data (including mechanistic or in vitro studies) in literature-based environmental health evaluations.
- The paper describes the method as a seven-step systematic-review and evidence-integration protocol, including prespecified procedures for study selection, data extraction, risk-of-bias assessment, confidence rating, and evidence integration.
Limitations
- The paper states that systematic-review methods may add time and investment to literature-based evaluations, and notes that steps 2–4 were the most time intensive.
- It also says that the scientific judgments used in confidence ratings are inherently subjective; this is the paper’s own stated limitation.
- Evident from the text: the framework is not a report of results from an empirical study with a sample, so there are no quantitative effect estimates or outcome findings to assess here.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
3 Valid · 0 Not valid · 16 Not checked · 52 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“It includes all of the recommended elements for conducting and reporting a systematic review [outlined in the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement (Moher et al. 2009)].”
§ Systematic Review and Evidence Integration for Literature-Based Environmental Health Science Assessments > Results
What the sources say
- Partly supportedcites Moher et al. 2009 matched to “Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement” (2009) sourceExperimental — accuracy not yet measured
Section 4 (The PRISMA Statement) confirms that PRISMA outlines recommended elements — a 27-item checklist plus a four-phase flow diagram — but frames them strictly as reporting guidance: 'The aim of the PRISMA Statement is to help authors improve the reporting of systematic reviews and meta-analyses,' and explicitly adds that 'the PRISMA checklist is not a quality assessment instrument to gauge the quality of a systematic review.' The dedicated section 'Conduct and Reporting Research Are Distinct Concepts' maintains that distinction (noting only that the two are closely intertwined for systematic reviews), so the claim's 'elements for conducting and reporting' overstates the source on the 'conducting' half; Table 1 likewise lists 'Checklist of items to include when reporting a systematic review or meta-analysis.' Whether OHAT covers all of those elements is the citing authors' own comparison, not something this source addresses. (LLM, conf=0.75)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
In the paper
“In applying the OHAT Approach to case studies (NTP 2013a, 2013f), the NTP found that steps 2–4 were the most time intensive: selecting studies, extracting data, and assessing the quality of individual studies.”
§ Systematic Review and Evidence Integration for Literature-Based Environmental Health Science Assessments > Discussion
What the sources say
- Not checked yetcites NTP 2013a, 2013f
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Therefore, these studies will usually have all four features and receive an initial rating of “high confidence.””
§ Systematic Review and Evidence Integration for Literature-Based Environmental Health Science Assessments > Results > Step 5: Rate the Confidence in the Body of Evidence
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Therefore, it is worth noting that a single well-conducted study may provide evidence of toxicity or a health effect associated with exposure to the substance in question [e.g., see Germolec (2009) and Foster (2009) for explanations of the NTP levels of evidence for determination of “toxicity” for individual studies].”
§ Systematic Review and Evidence Integration for Literature-Based Environmental Health Science Assessments > Results > Step 5: Rate the Confidence in the Body of Evidence
What the sources say
- Not checked yetcites Germolec (2009)
- Not checked yetcites Foster (2009)
hedgedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The principles of systematic review can be successfully applied to environmental health questions to provide greater objectivity and transparency to the process of developing conclusions.”
§ Systematic Review and Evidence Integration for Literature-Based Environmental Health Science Assessments > Abstract
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
7 more claims
Create a free account to see all 71 claims and to verify the 23 references still open.
- Probably not a substantial claim. 54
- It may state two separate claims as one. 16
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The study design is sometimes made explicit in the key elements (e.g. 'PICO'-type questions may be stated as 'PICOD' or 'PICOS' in the scientific literature, where 'D' (design) or 'S' (study) indicates that study design is being considered) [11].”
From Eligibility screening in evidence synthesis of environmental management topics (2017) · cites this paper as [11]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“such methods have only recently been developed and utilized in environmental health sciences [23–30].”
From A Systematic Review and Meta-Analysis of Multiple Airborne Pollutants and Autism Spectrum Disorder (2016) · cites this paper as [29]
The cited source says what the claim attributes to it.
In the citing paper
“To ensure reliability of the eligibility screening process, articles providing guidance on conducting systematic reviews in environmental research [2, 11, 12] and health research [14, 15, 18] recommend that eligibility screening should be performed where possible by at least two people.”
From Eligibility screening in evidence synthesis of environmental management topics (2017) · cites this paper as [11]
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048
Dermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,451
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.06.005 · Cited by 19,575
GRADE guidelines: 1. Introduction—GRADE evidence profiles and summary of findings tables
Guyatt, G., Oxman, A. D., Akl, E. A., Kunz, R., Vist, G., Brozek, J., Norris, S., Falck-Ytter, Y., Glasziou, P., & deBeer, H. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.04.026 · Cited by 10,681
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.06.006 · Cited by 10,578
GRADE guidelines: 3. Rating the quality of evidence
Balshem, H., Helfand, M., Schünemann, H. J., Oxman, A. D., Kunz, R., Brozek, J., Vist, G. E., Falck-Ytter, Y., Meerpohl, J., & Norris, S. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.015 · Cited by 8,285
The environment and disease: association or causation?
Hill, A. B. (2015) · Journal of the Royal Society of Medicine · doi:10.1177/0141076814562718 · Cited by 7,285
Research works printed but not linked: 2
- CLARITY Group at McMaster University. 2013. Systematic and Literature Review Resources. Instruments.Available: http://www.evidencepartners.com/resources/ [accessed 15 January 2013].
- NRC (National Research Council). 2013b. Project Title: Review of the IRIS Process.Available: http://www8.nationalacademies.org/cp/projectview.aspx?key=49458 [accessed 1 November 2013].
Cited by
Items 101–200 of 463
Hexavalent chromium and stomach cancer: a systematic review and meta-analysis
Suh, M., Wikoff, D. S., Lipworth, L. P., Goodman, M. G., Fitch, S., Mittal, L., Ring, C., & Proctor, D. M. (2019) · Critical Reviews in Toxicology · doi:10.1080/10408444.2019.1578730 · Cited by 67
Exposure to endocrine disruptors and risk of breast cancer: A systematic review
Rocha, P. R. S., Oliveira, V. D., Vasques, C. I., dos Reis, P. E. D., & Amato, A. A. (2021) · Critical Reviews in Oncology/Hematology · doi:10.1016/j.critrevonc.2021.103330 · Cited by 67
Industrial odour pollution and human health: a systematic review and meta-analysis
Guadalupe-Fernández, V., Sario, M. D., Vecchi, S., Bauleo, L., Michelozzi, P., Davoli, M., & Ancona, C. (2021) · Environmental Health · doi:10.1186/s12940-021-00774-3 · Cited by 65
Uwak, I., Olson, N., Fuentes, A., Moriarty, M., Pulczinski, J. C., Lam, J., Xu, X., Taylor, B. D., Taiwo, S., Koehler, K., et al. (2021) · Environment International · doi:10.1016/j.envint.2021.106378 · Cited by 64
OECD approaches and considerations for regulatory evaluation of endocrine disruptors
Browne, P., van der Wal, L., & Gourmelon, A. (2019) · Molecular and Cellular Endocrinology · doi:10.1016/j.mce.2019.110675 · Cited by 63
Stenson, C., Wheeler, A. J., Carver, A., Donaire-González, D., Molina, M. A., Nieuwenhuijsen, M. J., & Tham, R. (2021) · Environment International · doi:10.1016/j.envint.2021.106696 · Cited by 60
Hamed, M. A., Akhigbe, T. M., Adeogun, A. E., Adesoye, O. B., & Akhigbe, R. E. (2023) · Frontiers in Endocrinology · doi:10.3389/fendo.2023.1227836 · Cited by 59
Meyer, F., Bitsch, A., Forman, H. J., Fragoulis, A., Ghezzi, P., Henschenmacher, B., Kellner, R. L. L., Kühne, J., Ludwig, T., Sachno, D., et al. (2024) · Environment International · doi:10.1016/j.envint.2024.108940 · Cited by 59
The Emergence of Systematic Review in Toxicology
Stephens, M. L., Betts, K., Beck, N. B., Cogliano, V., Dickersin, K., Fitzpatrick, S., Freeman, J., Gray, G., Hartung, T., McPartland, J., et al. (2016) · Toxicological Sciences · doi:10.1093/toxsci/kfw059 · Cited by 54
Self-citationVogel, J. A., van Veldhuisen, E., Agnass, P., Crezee, J., Dijk, F., Verheij, J., van Gulik, T. M., Meijerink, M. R., Vroomen, L. G. P. H., van Lienden, K. P., et al. (2017) · PLoS ONE · doi:10.1371/journal.pone.0174018 · Cited by 59
(EFSA), E. F. S. A., Devos, Y., Guajardo, I. M., Glanville, J., & Waigmann, E. (2017) · EFSA Supporting Publications · doi:10.2903/sp.efsa.2017.en-1207 · Cited by 58
Zhu, X., Liu, B., Guo, C., Li, Z., Cheng, M., Zhu, X., & Wei, Y. (2022) · Environmental Pollution · doi:10.1016/j.envpol.2023.121086 · Cited by 56
Flannery, B. M., Dolan, L. C., Hoffman‐Pennesi, D., Gavelek, A., Jones, O. E., Kanwal, R., Wolpert, B. J., Gensheimer, K. F., Dennis, S. B., & Fitzpatrick, S. (2019) · Regulatory Toxicology and Pharmacology · doi:10.1016/j.yrtph.2019.104516 · Cited by 56
Evidence from Toxicology: The Most Essential Science for Prevention
Mandrioli, D., & Silbergeld, E. K. (2015) · Environmental Health Perspectives · doi:10.1289/ehp.1509880 · Cited by 56
Mortality and morbidity risks associated with floods: A systematic review and meta-analysis
Yang, Z., Huang, W., McKenzie, J. E., Yu, P., Ju, K., Wu, Y., Wen, B., Guo, Y., & Li, S. (2024) · Environmental Research · doi:10.1016/j.envres.2024.120263 · Cited by 54
Germolec, D. R., Luebke, R. W., Rooney, A. A., Shipkowski, K. A., Vandebriel, R. J., & Loveren, H. V. (2017) · Current Opinion in Toxicology · doi:10.1016/j.cotox.2017.08.002 · Cited by 54
Self-citationZwanenburg, P. R., Backer, S. F. M., Obdeijn, M. C., Lapid, O., Gans, S. L., & Boermeester, M. A. (2019) · Plastic & Reconstructive Surgery · doi:10.1097/prs.0000000000006090 · Cited by 53
A scoping review of engineering education systematic reviews
Phillips, M., Reed, J. B., Zwicky, D., & Epps, A. V. (2023) · Journal of Engineering Education · doi:10.1002/jee.20549 · Cited by 53
Beronius, A., Molander, L., Zilliacus, J., Rudén, C., & Hanberg, A. (2018) · Journal of Applied Toxicology · doi:10.1002/jat.3648 · Cited by 53
Wang, J., Cortés-Ramírez, J., Gan, T., Davies, J. M., & Hu, W. (2024) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2024.175863 · Cited by 52
Weight of Evidence for Hazard Identification: A Critical Review of the Literature
Martin, P., Bladier, C., Meek, B., Bruyère, O., Feinblatt, E., Touvier, M., Watier, L., & Makowski, D. (2018) · Environmental Health Perspectives · doi:10.1289/ehp3067 · Cited by 52
Development of an Evidence-Based Risk Assessment Framework
Krewski, D. R., Saunders-Hastings, P., Baan, R. A., Barton-Maclaren, T. S., Browne, P., Chiu, W. A., Gwinn, M., Hartung, T., Kraft, A. D., Lam, J., et al. (2022) · ALTEX · doi:10.14573/altex.2004041 · Cited by 51
Self-citationCamoni, N., Conti, G., Esteves‐Oliveira, M., Carvalho, T. S., Roccuzzo, A., Cagetti, M. G., & Campus, G. (2023) · Applied Sciences · doi:10.3390/app13179654 · Cited by 49
Zhu, S., Zhang, T., Wang, Y., Zhou, X., Wang, S., & Wang, Z. (2021) · Environmental Science and Pollution Research · doi:10.1007/s11356-021-13396-6 · Cited by 49
Pega, F., Momen, N. C., Ujita, Y., Driscoll, T., & Whaley, P. (2021) · Environment International · doi:10.1016/j.envint.2021.106605 · Cited by 49
Framework for the quantitative weight-of-evidence analysis of ‘omics data for regulatory purposes
Bridges, J., Sauer, U. G., Buesen, R., Deferme, L., Tollefsen, K. E., Tralau, T., van Ravenzwaay, B., Poole, A., & Pemberton, M. (2017) · Regulatory Toxicology and Pharmacology · doi:10.1016/j.yrtph.2017.10.010 · Cited by 48
Mundt, K. A., Gentry, P. R., Dell, L. D., Rodricks, J. V., & Boffetta, P. (2017) · Regulatory Toxicology and Pharmacology · doi:10.1016/j.yrtph.2017.11.006 · Cited by 48
Building science and radiofrequency radiation: What makes smart and healthy buildings
Clegg, F. M., Sears, M., Friesen, M., Scarato, T., Metzinger, R., Russell, C., Stadtner, A., & Miller, A. B. (2019) · Building and Environment · doi:10.1016/j.buildenv.2019.106324 · Cited by 48
Ågerstrand, M., & Beronius, A. (2015) · Environment International · doi:10.1016/j.envint.2015.10.008 · Cited by 48
Jett, D. A., Sibrizzi, C. A., Blain, R. B., Hartman, P. A., Lein, P. J., Taylor, K. W., & Rooney, A. A. (2020) · Critical Reviews in Toxicology · doi:10.1080/10408444.2020.1787330 · Cited by 47
Self-citationPacchierotti, F., Ardoino, L., Benassi, B., Consales, C., Cordelli, E., Eleuteri, P., Marino, C., Sciortino, M., Brinkworth, M. H., Chen, G., et al. (2021) · Environment International · doi:10.1016/j.envint.2021.106806 · Cited by 47
The methodological rigour of systematic reviews in environmental health
Menon, J. M. L., Struijs, F., & Whaley, P. (2022) · Critical Reviews in Toxicology · doi:10.1080/10408444.2022.2082917 · Cited by 45
Chang, C.-J., Ish, J. L., Chang, V. C., Daniel, M., Jones, R. R., & White, A. J. (2024) · American Journal of Epidemiology · doi:10.1093/aje/kwae010 · Cited by 43
The association between prenatal bisphenol A exposure and birth weight: a meta-analysis
Hu, C., Li, F., Hua, X., Jiang, W., Mao, C., & Zhang, X.-J. (2018) · Reproductive Toxicology · doi:10.1016/j.reprotox.2018.04.013 · Cited by 43
Hu, C., Qiao, J.-C., Gui, S., Xu, K., Dzhambov, A. M., & Zhang, X.-J. (2023) · Environmental Research · doi:10.1016/j.envres.2023.116064 · Cited by 42
Segal, D., Makris, S. L., Kraft, A. D., Bale, A. S., Fox, J., Gilbert, M. E., Bergfelt, D. R., Raffaele, K. C., Blain, R. B., Fedak, K. M., et al. (2015) · Regulatory Toxicology and Pharmacology · doi:10.1016/j.yrtph.2015.03.005 · Cited by 42
Aucoin, M., Cardozo, V., McLaren, M. D., Garber, A., Remy, D., Baker, J., Gratton, A., Kala, M. A., Monteiro, S., Warder, C., et al. (2021) · Metabolism Open · doi:10.1016/j.metop.2021.100115 · Cited by 42
Malloy, T. F., Zaunbrecher, V. M., Beryt, E., Judson, R., Tice, R. R., Allard, P., Blake, A., Cote, I., Godwin, H. A., Heine, L. G., et al. (2017) · Integrated Environmental Assessment and Management · doi:10.1002/ieam.1923 · Cited by 42
Self-citationBero, L., Anglemyer, A., Vesterinen, H., & Krauth, D. (2016) · Environment International · doi:10.1016/j.envint.2015.10.011 · Cited by 41
“These problems with animal research have resulted in significant debate about how to assess biases in animal studies used in systematic reviews, risk assessments and other regulatory decisions [3-5].”
Bamber, A. M., Hasanali, S. H., Nair, A. S., Watkins, S. M., Vigil, D. I., Dyke, M. V. V., McMullin, T. S., & Richardson, K. (2019) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph16122123 · Cited by 41
Progress in data interoperability to support computational toxicology and chemical safety evaluation
Watford, S., Edwards, S. W., Angrish, M. M., Judson, R., & Friedman, K. P. (2019) · Toxicology and Applied Pharmacology · doi:10.1016/j.taap.2019.114707 · Cited by 42
Henschenmacher, B., Bitsch, A., Gala, T. D. L. H., Forman, H. J., Fragoulis, A., Ghezzi, P., Kellner, R. L. L., Koch, W., Kühne, J., Sachno, D., et al. (2021) · Environment International · doi:10.1016/j.envint.2021.106932 · Cited by 41
Das, A., Giri, B. S., & Manjunatha, R. G. (2024) · Environmental Science and Pollution Research · doi:10.1007/s11356-024-35698-1 · Cited by 40
Morris, R. D., Morgan, L. L., & Davis, D. L. (2015) · IEEE Access · doi:10.1109/access.2015.2478701 · Cited by 40
Schmiedchen, K., Petri, A.-K., Drießen, S., & Bailey, W. H. (2017) · Environmental Research · doi:10.1016/j.envres.2017.09.013 · Cited by 40
Can Periodontitis Influence the Progression of Abdominal Aortic Aneurysm? A Systematic Review
Salhi, L., Rompen, E., Sakalihasan, N., Laleman, I., Teughels, W., Michel, J., & Lambert, F. (2018) · Angiology · doi:10.1177/0003319718821243 · Cited by 40
Bosch-Capblanch, X., Esu, E., Oringanje, C. M., Dongus, S., Jalilian, H., Eyers, J., Auer, C., Meremikwu, M., & Röösli, M. (2024) · Environment International · doi:10.1016/j.envint.2024.108612 · Cited by 39
Shrestha, T., Chi, C. V. Y., Cassarino, M., Foley, S., & Blasi, Z. D. (2025) · Environment International · doi:10.1016/j.envint.2024.109217 · Cited by 39
Health effects of WiFi radiation: a review based on systematic quality evaluation
Dongus, S., Jalilian, H., Schürmann, D., & Röösli, M. (2021) · Critical Reviews in Environmental Science and Technology · doi:10.1080/10643389.2021.1951549 · Cited by 39
Application of evidence-based methods to construct mechanism-driven chemical assessment frameworks
Hoffmann, S., Aiassa, E., Angrish, M. M., Beausoleil, C., Bois, F. Y., Ciccolallo, L., Craig, P. S., de Vries, R. B. M., Dorne, J. L. C. M., Druwe, I. L., et al. (2022) · ALTEX · doi:10.14573/altex.2202141 · Cited by 39
Self-citationMundt, K. A., Dell, L. D., Crawford, L., Sax, S. N., & Boffetta, P. (2017) · Journal of Occupational and Environmental Medicine · doi:10.1097/jom.0000000000001202 · Cited by 38
Kenny, R. P. W., Johnson, E. E., Adesanya, A. M., Richmond, C., Beyer, F. R., Calderón, C., Rankin, J. M., Pearce, M. S., Toledano, M. B., Craig, D., et al. (2024) · Environment International · doi:10.1016/j.envint.2024.108817 · Cited by 37
Rebooting the generally recognized as safe (GRAS) approach for food additive safety in the US
Härtung, T. (2018) · ALTEX · doi:10.14573/altex.1712181 · Cited by 37
Romeo, S., Zeni, O., Sannino, A., Lagorio, S., Biffoni, M., & Scarfì, M. R. (2021) · Environment International · doi:10.1016/j.envint.2021.106386 · Cited by 36
Lagorio, S., Blettner, M., Baaken, D., Feychting, M., Karipidis, K., Loney, T., Orsini, N., Röösli, M., Paulo, M. S., & Elwood, M. (2021) · Environment International · doi:10.1016/j.envint.2021.106828 · Cited by 36
Psychiatric comorbidities in cancer survivors across tumor subtypes: A systematic review
Bach, A., Knauer, K., Graf, J., Schäffeler, N., & Stengel, A. (2022) · World Journal of Psychiatry · doi:10.5498/wjp.v12.i4.623 · Cited by 37
Photobiomodulation effects on keratinocytes cultured in vitro: a critical review
de Abreu, P. T. R., de Arruda, J. A. A., Mesquita, R. A., Abreu, L. G., Diniz, I. M. A., & Silva, T. A. (2019) · Lasers in Medical Science · doi:10.1007/s10103-019-02813-5 · Cited by 36
Laser Treatment of Peri-Implantitis: A Systematic Review of Radiographic Outcomes
Ting, M., Alluri, L. S. C., Sulewski, J. G., Suzuki, J. B., & da Silva, A. P. B. (2022) · Dentistry Journal · doi:10.3390/dj10020020 · Cited by 36
Khreis, H., Bredell, C., Fung, K. W., Hong, L., Szybka, M., Phillips, V., Abbas, A., Lim, Y., Andersen, Z. J., Woodcock, J., et al. (2022) · Environment International · doi:10.1016/j.envint.2022.107596 · Cited by 36
Tippett, V. L., Tattersall, L., Latif, N. A., Shah, K. M., Lawson, M. A., & Gartland, A. (2022) · Oncogene · doi:10.1038/s41388-022-02529-x · Cited by 36
Guiding Principles and Key Elements for Establishing a Weight of Evidence for Chemical Assessment
OECD (2019) · OECD series on testing and assessment · doi:10.1787/69b366a9-en · Cited by 36
Using systematic reviews for hazard and risk assessment of endocrine disrupting chemicals
Beronius, A., & Vandenberg, L. N. (2015) · Reviews in Endocrine and Metabolic Disorders · doi:10.1007/s11154-016-9334-7 · Cited by 36
Environmental Chemical Assessment in Clinical Practice: Unveiling the Elephant in the Room
Bijlsma, N., & Cohen, M. M. (2016) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph13020181 · Cited by 35
Christensen, J. J., Arnesen, E., Rundblad, A., Telle‐Hansen, V. H., Narverud, I., Blomhoff, R., Bogsrud, M. P., Retterstøl, K., Ulven, S. M., & Holven, K. B. (2023) · Atherosclerosis · doi:10.1016/j.atherosclerosis.2023.117433 · Cited by 34
Photobiomodulation of human gingival fibroblasts with diode laser - A systematic review
Bakshi, P. V., Setty, S. B., & Kulkarni, M. R. (2022) · Journal of Indian Society of Periodontology · doi:10.4103/jisp.jisp_90_21 · Cited by 34
Phthalate Exposures and Placental Health in Animal Models and Humans: A Systematic Review
Seymore, T., Rivera‐Núñez, Z., Stapleton, P. A., Adibi, J. J., & Barrett, E. S. (2022) · Toxicological Sciences · doi:10.1093/toxsci/kfac060 · Cited by 34
Cooper, G. S., Lunn, R. M., Ågerstrand, M., Glenn, B. S., Kraft, A. D., Luke, A. M., & Ratcliffe, J. M. (2016) · Environment International · doi:10.1016/j.envint.2016.03.017 · Cited by 33
Arroyave, W. D., Mehta, S. S., Guha, N., Schwingl, P., Taylor, K. W., Glenn, B., Radke, E. G., Vilahur, N., Carreón, T., Nachman, R. M., et al. (2021) · Journal of Exposure Science & Environmental Epidemiology · doi:10.1038/s41370-020-0228-0 · Cited by 33
Pophof, B., Kühne, J., Schmid, G., Weiser, E., Dorn, H., Henschenmacher, B., Burns, J., Danker‐Hopfe, H., & Sauter, C. (2024) · Environment International · doi:10.1016/j.envint.2024.108899 · Cited by 33
Assessing risk of bias in toxicological studies in the era of artificial intelligence
Härtung, T., Hoffmann, S., & Whaley, P. (2025) · Archives of Toxicology · doi:10.1007/s00204-025-03978-5 · Cited by 33
Low-calorie sweeteners and health outcomes: A demonstration of rapid evidence mapping (rEM)
Lam, J., Howard, B. E., Thayer, K. A., & Shah, R. (2019) · Environment International · doi:10.1016/j.envint.2018.11.070 · Cited by 33
Self-citationKnowledge Organization Systems for Systematic Chemical Assessments
Whaley, P., Edwards, S. W., Kraft, A. D., Nyhan, K., Shapiro, A. J., Watford, S., Wattam, S., Wolffe, T. A. M., & Angrish, M. M. (2020) · Environmental Health Perspectives · doi:10.1289/ehp6994 · Cited by 33
Goudman, L., Groote, S. D., Linderoth, B., Smedt, A. D., Eldabe, S. S., Duarte, R. V., & Moens, M. (2021) · Journal of Clinical Medicine · doi:10.3390/jcm10132766 · Cited by 33
Benchikh, I., Ziani, K., González, A., & Khaled, M. B. (2024) · Critical Reviews in Toxicology · doi:10.1080/10408444.2024.2310593 · Cited by 32
Applying evidence-based methods to the development and use of adverse outcome pathways
de Vries, R. B. M. (2021) · ALTEX · doi:10.14573/altex.2101211 · Cited by 32
Norris, S. L., Aung, M. T., Chartres, N., & Woodruff, T. J. (2021) · Environmental Health · doi:10.1186/s12940-021-00794-z · Cited by 32
BPMS para la gestión: una revisión sistemática de la literatura
Martín‐Navarro, A., Sancho, M. P. L., & Medina-Garrido, J. A. (2018) · Revista española de Documentación Científica · doi:10.3989/redc.2018.3.1532 · Cited by 32
Making nutrition guidelines fit for purpose
Bero, L. A., Norris, S. L., & Lawrence, M. A. (2019) · BMJ · doi:10.1136/bmj.l1579 · Cited by 32
Bauer, K., Lau, T., Schwille‐Kiuntke, J., Schild, S., Hauner, H., Stengel, A., Zipfel, S., & Mack, I. (2020) · European Eating Disorders Review · doi:10.1002/erv.2741 · Cited by 31
Fiolet, T., Mahamat‐Saleh, Y., Frénoy, P., Kvaskoff, M., & Mancini, F. R. (2021) · Environment International · doi:10.1016/j.envint.2021.106663 · Cited by 31
Matta, K., Koual, M., Ploteau, S., Coumoul, X., Audouze, K., Bizec, B. L., Antignac, J., & Cano-Sancho, G. (2021) · Environmental Health Perspectives · doi:10.1289/ehp8421 · Cited by 31
Patwary, M. M., Sakhvidi, M. J. Z., Ashraf, S., Dadvand, P., Browning, M. H. E. M., Alam, M. A., Bell, M. L., James, P., & Astell‐Burt, T. (2024) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2024.170977 · Cited by 31
Pan, K., Xu, J., Xu, Y., Wang, C., & Yu, J. (2024) · Pharmacological Research · doi:10.1016/j.phrs.2024.107251 · Cited by 31
Sheehan, M. C., & Lam, J. (2015) · Current Environmental Health Reports · doi:10.1007/s40572-015-0062-z · Cited by 31
Guidelines for performing systematic reviews in the development of toxicity factors
Schaefer, H. R., & Myers, J. L. (2017) · Regulatory Toxicology and Pharmacology · doi:10.1016/j.yrtph.2017.10.008 · Cited by 30
A review of the evidence to support interim reference level for dietary lead exposure in adults
Dolan, L. C., Flannery, B. M., Hoffman‐Pennesi, D., Gavelek, A., Jones, O. E., Kanwal, R., Wolpert, B. J., Gensheimer, K. F., Dennis, S. B., & Fitzpatrick, S. (2020) · Regulatory Toxicology and Pharmacology · doi:10.1016/j.yrtph.2020.104579 · Cited by 30
Bosch‐Capblanch, X., Esu, E., Dongus, S., Oringanje, C. M., Jalilian, H., Eyers, J. E., Oftedal, G., Meremikwu, M. M., & Röösli, M. (2021) · Environment International · doi:10.1016/j.envint.2021.106953 · Cited by 30
Stephens, J., Moorhouse, A. J., Craenen, K., Schroeder, E., Drenos, F., & Anderson, R. M. (2024) · International Journal of Radiation Biology · doi:10.1080/09553002.2024.2306328 · Cited by 30
Sheehan, M. C., Lam, J., Navas‐Acién, A., & Chang, H. H. (2016) · Environment International · doi:10.1016/j.envint.2016.02.016 · Cited by 30
Intersection of Systematic Review Methodology with the NIHReproducibility Initiative
Thayer, K. A., Wolfe, M. S., Rooney, A. A., Boyles, A. L., Bucher, J. R., & Birnbaum, L. S. (2014) · Environmental Health Perspectives · doi:10.1289/ehp.1408671 · Cited by 29
Self-citationJohnson, E. E., Kenny, R. P. W., Adesanya, A. M., Richmond, C., Beyer, F. R., Calderón, C., Rankin, J. M., Pearce, M. S., Toledano, M. B., Craig, D., et al. (2024) · Environment International · doi:10.1016/j.envint.2024.108816 · Cited by 29
Lefebvre, T., Fréour, T., Ploteau, S., Bizec, B. L., Antignac, J., & Cano-Sancho, G. (2021) · Reproductive Toxicology · doi:10.1016/j.reprotox.2021.09.005 · Cited by 29
Rodríguez-Villamizar, L. A., Berney, C., Villa‐Roel, C., Ospina, M. B., Osornio-Vargas, Á. R., & Rowe, B. H. (2016) · Reviews on Environmental Health · doi:10.1515/reveh-2016-0005 · Cited by 29
Röösli, M., Dongus, S., Jalilian, H., Eyers, J., Esu, E., Oringanje, C. M., Meremikwu, M., & Bosch-Capblanch, X. (2024) · Environment International · doi:10.1016/j.envint.2023.108338 · Cited by 28
Walker, V. R., Boyles, A. L., Pelch, K. E., Holmgren, S. D., Shapiro, A. J., Blystone, C. R., Devito, M. J., Newbold, R. R., Blain, R., Hartman, P., et al. (2018) · Environment International · doi:10.1016/j.envint.2017.12.032 · Cited by 28
“gov/go/38159) to facilitate consideration of risk of bias across evidence streams with common terms and categories (Rooney et al., 2014).”
Self-citationScanlon, M. M., Gordon, J. L., McCoy, W. F., & Cain, M. F. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17062168 · Cited by 28
Wikoff, D. S., Lewis, R. J., Erraguntla, N. K., Franzen, A., & Foreman, J. E. (2020) · Regulatory Toxicology and Pharmacology · doi:10.1016/j.yrtph.2020.104790 · Cited by 28
Mevissen, M., Ward, J. M., Kopp‐Schneider, A., McNamee, J. P., Wood, A., Rivero, T. M., Thayer, K. A., & Straíf, K. (2022) · Environment International · doi:10.1016/j.envint.2022.107106 · Cited by 28
Self-citationLow-dose ionizing radiation and adverse birth outcomes: a systematic review and meta-analysis
Frangione, B., Hinton, P., & Villeneuve, P. J. (2022) · International Archives of Occupational and Environmental Health · doi:10.1007/s00420-022-01911-2 · Cited by 28
A perspective on In vitro developmental neurotoxicity test assay results: An expert panel review
Juberg, D. R., Fox, D. A., Forcelli, P. A., Kacew, S., Lipscomb, J. C., Saghir, S. A., Sherwin, C. M., Koenig, C. M., Hays, S. M., & Kirman, C. R. (2023) · Regulatory Toxicology and Pharmacology · doi:10.1016/j.yrtph.2023.105444 · Cited by 28
Metadata
- Authors
- Rooney, A. A., Boyles, A. L., Wolfe, M. S., Bucher, J. R., & Thayer, K. A.
- Affiliations
- National Institutes of Health (US)
- Corresponding author
- Rooney, Andrew A.
- Venue
- Environmental Health Perspectives
- Publisher
- American Chemical Society (ACS)
- Published
- 2014-04-22
- Volume
- 122
- Issue
- 7
- Pages
- 711-718
- ISSN
- 0091-6765 (ISSN-L)
- 1552-9924
- PMC
- PMC4080517
- OpenAlex
- W2066218383
- Semantic Scholar
- f30c4b27530fb3fa34898432b9850e3fbcdbde0e
- Cited by
- 434
- References
- 31
- Citations by year
- 2026: 34 · 2025: 33 · 2024: 43 · 2023: 34 · 2022: 39 · 2021: 53 · 2020: 40 · 2019: 29 · 2018: 33 · 2017: 25 · 2016: 31 · 2015: 28 · 2014: 12
- FWCI
- 8.81
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- public-domain
- creativecommons.org/licenses/by/4.0/ (vor, 1347 d)
- Locations
- Environmental Health Perspectives (publishedVersion, public-domain)
- Europe PMC (PubMed Central) (publishedVersion, public-domain)
- PubMed Central (publishedVersion, public-domain)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Environmental Science › Health, Toxicology and Mutagenesis › Health, Environment, Cognitive Aging
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- systematic reviews, evidence integration, environmental health risk assessment, hazard identification, level of evidence, confidence in evidence, risk of bias assessment, mechanistic studies, animal studies, human subjects research
- MeSH terms
- Systematic Reviews as Topic, Animals, Environmental Health, Humans, United States, Evidence-Based Medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
