Systematic Review and Evidence Integration for Literature-Based Environmental Health Science Assessments
Open accessThe DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index; 4 of the 28 sources its claims cite were checked for support.
Abstract
Background: Systematic-review methodologies provide objectivity and transparency to the process of collecting and synthesizing scientific evidence in reaching conclusions on specific research questions. There is increasing interest in applying these procedures to address environmental health questions. Objectives: The goal was to develop a systematic-review framework to address environmental health questions by extending approaches developed for clinical medicine to handle the breadth of data relevant to environmental health sciences (e.g., human, animal, and mechanistic studies). Methods: The Office of Health Assessment and Translation (OHAT) adapted guidance from authorities on systematic-review and sought advice during development of the OHAT Approach through consultation with technical experts in systematic review and human health assessments, as well as scientific advisory groups and the public. The method was refined by considering expert and public comments and through application to case studies. Results and Discussion: Here we present a seven-step framework for systematic review and evidence integration for reaching hazard identification conclusions: 1) problem formulation and protocol development, 2) search for and select studies for inclusion, 3) extract data from studies, 4) assess the quality or risk of bias of individual studies, 5) rate the confidence in the body of evidence, 6) translate the confidence ratings into levels of evidence, and 7) integrate the information from different evidence streams (human, animal, and “other relevant data” including mechanistic or in vitro studies) to develop hazard identification conclusions. Conclusion: The principles of systematic review can be successfully applied to environmental health questions to provide greater objectivity and transparency to the process of developing conclusions. Citation: Rooney AA, Boyles AL, Wolfe MS, Bucher JR, Thayer KA. 2014. Systematic review and evidence integration for literature-based environmental health science assessments. Environ Health Perspect 122:711–718; http://dx.doi.org/10.1289/ehp.1307972
Overview
CoChat Summary
TL;DR
This paper presents the OHAT seven-step framework for systematic review and evidence integration in environmental health assessments, extending clinical systematic-review methods to human, animal, and other relevant data. The approach is designed to improve objectivity, transparency, and consistency in hazard identification conclusions.
Key findings
- The OHAT framework is a flexible seven-step process: 1) problem formulation and protocol development, 2) search for and select studies, 3) extract data, 4) assess study quality/risk of bias, 5) rate confidence in the body of evidence, 6) translate confidence ratings into levels of evidence, and 7) integrate evidence streams to develop hazard identification conclusions.
- In applying the OHAT Approach to case studies, steps 2–4 were the most time intensive: selecting studies, extracting data, and assessing the quality of individual studies.
- The framework defines five hazard identification conclusion categories: known, presumed, suspected, not classifiable, and not identified as a hazard to humans.
- The paper states that high confidence means further research is very unlikely to change the apparent relationship, while very low confidence means further research is very likely to change it; for a conclusion of evidence of no health effect, the paper says this is reached only when there is high confidence.
Methods
- The authors developed the OHAT Approach by adapting guidance from authoritative systematic-review groups and consulting technical experts, scientific advisory groups, and the public; the method was refined through comments and application to case studies.
- The framework was built to cover human, animal, and other relevant data (including mechanistic or in vitro studies) in literature-based environmental health evaluations.
- The paper describes the method as a seven-step systematic-review and evidence-integration protocol, including prespecified procedures for study selection, data extraction, risk-of-bias assessment, confidence rating, and evidence integration.
Limitations
- The paper states that systematic-review methods may add time and investment to literature-based evaluations, and notes that steps 2–4 were the most time intensive.
- It also says that the scientific judgments used in confidence ratings are inherently subjective; this is the paper’s own stated limitation.
- Evident from the text: the framework is not a report of results from an empirical study with a sample, so there are no quantitative effect estimates or outcome findings to assess here.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
3 Valid · 0 Not valid · 16 Not checked · 52 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“It includes all of the recommended elements for conducting and reporting a systematic review [outlined in the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement (Moher et al. 2009)].”
§ Systematic Review and Evidence Integration for Literature-Based Environmental Health Science Assessments > Results
What the sources say
- Partly supportedcites Moher et al. 2009 matched to “Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement” (2009) sourceExperimental — accuracy not yet measured
Section 4 (The PRISMA Statement) confirms that PRISMA outlines recommended elements — a 27-item checklist plus a four-phase flow diagram — but frames them strictly as reporting guidance: 'The aim of the PRISMA Statement is to help authors improve the reporting of systematic reviews and meta-analyses,' and explicitly adds that 'the PRISMA checklist is not a quality assessment instrument to gauge the quality of a systematic review.' The dedicated section 'Conduct and Reporting Research Are Distinct Concepts' maintains that distinction (noting only that the two are closely intertwined for systematic reviews), so the claim's 'elements for conducting and reporting' overstates the source on the 'conducting' half; Table 1 likewise lists 'Checklist of items to include when reporting a systematic review or meta-analysis.' Whether OHAT covers all of those elements is the citing authors' own comparison, not something this source addresses. (LLM, conf=0.75)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
In the paper
“In applying the OHAT Approach to case studies (NTP 2013a, 2013f), the NTP found that steps 2–4 were the most time intensive: selecting studies, extracting data, and assessing the quality of individual studies.”
§ Systematic Review and Evidence Integration for Literature-Based Environmental Health Science Assessments > Discussion
What the sources say
- Not checked yetcites NTP 2013a, 2013f
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Therefore, these studies will usually have all four features and receive an initial rating of “high confidence.””
§ Systematic Review and Evidence Integration for Literature-Based Environmental Health Science Assessments > Results > Step 5: Rate the Confidence in the Body of Evidence
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Therefore, it is worth noting that a single well-conducted study may provide evidence of toxicity or a health effect associated with exposure to the substance in question [e.g., see Germolec (2009) and Foster (2009) for explanations of the NTP levels of evidence for determination of “toxicity” for individual studies].”
§ Systematic Review and Evidence Integration for Literature-Based Environmental Health Science Assessments > Results > Step 5: Rate the Confidence in the Body of Evidence
What the sources say
- Not checked yetcites Germolec (2009)
- Not checked yetcites Foster (2009)
hedgedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The principles of systematic review can be successfully applied to environmental health questions to provide greater objectivity and transparency to the process of developing conclusions.”
§ Systematic Review and Evidence Integration for Literature-Based Environmental Health Science Assessments > Abstract
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
7 more claims
Create a free account to see all 71 claims and to verify the 23 references still open.
- Probably not a substantial claim. 54
- It may state two separate claims as one. 16
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The study design is sometimes made explicit in the key elements (e.g. 'PICO'-type questions may be stated as 'PICOD' or 'PICOS' in the scientific literature, where 'D' (design) or 'S' (study) indicates that study design is being considered) [11].”
From Eligibility screening in evidence synthesis of environmental management topics (2017) · cites this paper as [11]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“such methods have only recently been developed and utilized in environmental health sciences [23–30].”
From A Systematic Review and Meta-Analysis of Multiple Airborne Pollutants and Autism Spectrum Disorder (2016) · cites this paper as [29]
The cited source says what the claim attributes to it.
In the citing paper
“To ensure reliability of the eligibility screening process, articles providing guidance on conducting systematic reviews in environmental research [2, 11, 12] and health research [14, 15, 18] recommend that eligibility screening should be performed where possible by at least two people.”
From Eligibility screening in evidence synthesis of environmental management topics (2017) · cites this paper as [11]
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048
Dermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,451
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.06.005 · Cited by 19,575
GRADE guidelines: 1. Introduction—GRADE evidence profiles and summary of findings tables
Guyatt, G., Oxman, A. D., Akl, E. A., Kunz, R., Vist, G., Brozek, J., Norris, S., Falck-Ytter, Y., Glasziou, P., & deBeer, H. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.04.026 · Cited by 10,681
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.06.006 · Cited by 10,578
GRADE guidelines: 3. Rating the quality of evidence
Balshem, H., Helfand, M., Schünemann, H. J., Oxman, A. D., Kunz, R., Brozek, J., Vist, G. E., Falck-Ytter, Y., Meerpohl, J., & Norris, S. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.015 · Cited by 8,285
The environment and disease: association or causation?
Hill, A. B. (2015) · Journal of the Royal Society of Medicine · doi:10.1177/0141076814562718 · Cited by 7,285
Research works printed but not linked: 2
- CLARITY Group at McMaster University. 2013. Systematic and Literature Review Resources. Instruments.Available: http://www.evidencepartners.com/resources/ [accessed 15 January 2013].
- NRC (National Research Council). 2013b. Project Title: Review of the IRIS Process.Available: http://www8.nationalacademies.org/cp/projectview.aspx?key=49458 [accessed 1 November 2013].
Cited by
Rochester, J. R., & Bolden, A. L. (2015) · Environmental Health Perspectives · doi:10.1289/ehp.1408989 · Cited by 1,524
Fedak, K. M., Bernal, A., Capshaw, Z. A., & Gross, S. (2015) · Emerging Themes in Epidemiology · doi:10.1186/s12982-015-0037-4 · Cited by 680
Key Characteristics of Carcinogens as a Basis for Organizing Data on Mechanisms of Carcinogenesis
Smith, M. T., Guyton, K. Z., Gibbons, C. F., Fritz, J. M., Portier, C. J., Rusyn, I., DeMarini, D. M., Caldwell, J. C., Kavlock, R. J., Lambert, P. F., et al. (2016) · Environmental Health Perspectives · doi:10.1289/ehp.1509912 · Cited by 650
Self-citationLevy, K., Woster, A. P., Goldstein, R. S., & Carlton, E. J. (2016) · Environmental Science & Technology · doi:10.1021/acs.est.5b06186 · Cited by 535
New Look at BTEX: Are Ambient Levels a Problem?
Bolden, A. L., Kwiatkowski, C. F., & Colborn, T. (2015) · Environmental Science & Technology · doi:10.1021/es505316f · Cited by 456
Woodruff, T. J., & Sutton, P. (2014) · Environmental Health Perspectives · doi:10.1289/ehp.1307175 · Cited by 427
“…we assign different types of human observational studies different ratings from the start (i.e., crosssectional studies, case–control studies, and case series or reports are rated as “low” quality, and cohort and nested case–control studies are rated as “moderate” quality) (Rooney et al. 2014).”
InfluentialGuidance on the use of the weight of evidence approach in scientific assessments
EFSA Scientific Committee, Hardy, A., Benford, D., Halldorsson, T., Jeger, M. J., Knutsen, H. K., More, S., Naegeli, H., Noteborn, H., Ockleford, C., et al. (2017) · EFSA Journal · doi:10.2903/j.efsa.2017.4971 · Cited by 415
(CONTAM), E. P. O. C. I. T. F. C., Knutsen, H. K., Alexander, J., Barregård, L., Bignami, M., Brüschweiler, B., Ceccatelli, S., Cottrill, B., Dinovi, M., Edler, L., et al. (2018) · EFSA Journal · doi:10.2903/j.efsa.2018.5333 · Cited by 342
Ambient temperature and mental health: a systematic review and meta-analysis
Thompson, R., Lawrance, E. L., Roberts, L. F., Grailey, K., Ashrafian, H., Maheswaran, H., Toledano, M. B., & Darzi, A. (2023) · The Lancet Planetary Health · doi:10.1016/s2542-5196(23)00104-3 · Cited by 306
Air pollution and development in Africa: impacts on health, the economy, and human capital
Fisher, S., Bellinger, D. C., Cropper, M. L., Kumar, P., Binagwaho, A., Koudenoukpo, J. B., Park, Y., Taghian, G., & Landrigan, P. J. (2021) · The Lancet Planetary Health · doi:10.1016/s2542-5196(21)00201-1 · Cited by 299
Rancière, F., Lyons, J. G., Loh, V. H. Y., Botton, J., Galloway, T., Wang, T., Shaw, J. E., & Magliano, D. J. (2015) · Environmental Health · doi:10.1186/s12940-015-0036-5 · Cited by 291
Thompson, R., Smith, R. B., Bou Karim, Y., Shen, C., Drummond, K., Teng, C., & Toledano, M. B. (2022) · Environment International · doi:10.1016/j.envint.2021.106905 · Cited by 280
Bero, L., Chartres, N., Diong, J., Fabbri, A., Ghersi, D., Lam, J., Lau, A., McDonald, S., Mintzes, B., Sutton, P., et al. (2018) · Systematic Reviews · doi:10.1186/s13643-018-0915-2 · Cited by 278
GRADE: Assessing the quality of evidence in environmental and occupational health
Morgan, R. L., Thayer, K. A., Bero, L., Bruce, N., Falck-Ytter, Y., Ghersi, D., Guyatt, G., Hooijmans, C., Langendam, M., Mandrioli, D., et al. (2016) · Environment International · doi:10.1016/j.envint.2016.01.004 · Cited by 277
Self-citationA Systematic Review and Meta-Analysis of Multiple Airborne Pollutants and Autism Spectrum Disorder
Lam, J., Sutton, P., Kalkbrenner, A., Windham, G., Halladay, A., Koustas, E., Lawler, C., Davidson, L., Daniels, N., Newschaffer, C., et al. (2016) · PLOS ONE · doi:10.1371/journal.pone.0161851 · Cited by 247
Developmental PBDE Exposure and IQ/ADHD in Childhood: A Systematic Review and Meta-analysis
Lam, J., Lanphear, B. P., Bellinger, D., Axelrad, D. A., McPartland, J., Sutton, P., Davidson, L., Daniels, N., Sen, S., & Woodruff, T. J. (2017) · Environmental Health Perspectives · doi:10.1289/ehp1632 · Cited by 246
In silico toxicology protocols
Myatt, G. J., Ahlberg, E., Akahori, Y., Allen, D., Amberg, A., Anger, L. T., Aptula, A., Auerbach, S., Beilke, L., Bellion, P., et al. (2018) · Regulatory Toxicology and Pharmacology · doi:10.1016/j.yrtph.2018.04.014 · Cited by 240
Ghosh, R., Causey, K., Burkart, K., Wozniak, S., Cohen, A., & Brauer, M. (2021) · PLOS Medicine · doi:10.1371/journal.pmed.1003718 · Cited by 220
Lam, J., Koustas, E., Sutton, P., Johnson, P. I., Atchley, D. S., Sen, S., Robinson, K. A., Axelrad, D. A., & Woodruff, T. J. (2014) · Environmental Health Perspectives · doi:10.1289/ehp.1307923 · Cited by 214
Toward Good Read-Across Practice (GRAP) guidance
Ball, N. (2016) · ALTEX · doi:10.14573/altex.1601251 · Cited by 205
Noyes, P. D., Friedman, K. P., Browne, P., Haselman, J. T., Gilbert, M. E., Hornung, M. W., Barone, S., Crofton, K. M., Laws, S. C., Stoker, T. E., et al. (2019) · Environmental Health Perspectives · doi:10.1289/ehp5297 · Cited by 205
Trasande, L., Zoeller, R. T., Hass, U., Kortenkamp, A., Grandjean, P., Myers, J. P., DiGangi, J., Hunt, P. M., Rudel, R., Sathyanarayana, S., et al. (2016) · Andrology · doi:10.1111/andr.12178 · Cited by 184
Air pollution exposure and adverse sleep health across the life course: A systematic review
Liu, J., Wu, T., Liu, Q., Wu, S., & Chen, J. (2020) · Environmental Pollution · doi:10.1016/j.envpol.2020.114263 · Cited by 179
Hooijmans, C. R., de Vries, R. B. M., Ritskes-Hoitinga, M., Rovers, M. M., Leeflang, M. M., IntHout, J., Wever, K. E., Hooft, L., de Beer, H., Kuijpers, T., et al. (2018) · PLOS ONE · doi:10.1371/journal.pone.0187271 · Cited by 176
Self-citationA systematic review of Mancozeb as a reproductive and developmental hazard
Runkle, J. D., Flocks, J. D., Economos, J., & Dunlop, A. L. (2016) · Environment International · doi:10.1016/j.envint.2016.11.006 · Cited by 175
Childhood Leukemia and Primary Prevention
Whitehead, T. P., Metayer, C., Wiemels, J. L., Singer, A. W., & Miller, M. D. (2016) · Current problems in pediatric and adolescent health care · doi:10.1016/j.cppeds.2016.08.004 · Cited by 174
Review of developmental origins of health and disease publications in environmental epidemiology
Heindel, J. J., Skalla, L. A., Joubert, B. R., Dilworth, C. H., & Gray, K. A. (2016) · Reproductive Toxicology · doi:10.1016/j.reprotox.2016.11.011 · Cited by 167
Symeonides, C., Aromataris, E., Mulders, Y. R., Dizon, J. M. R., Stern, C., Barker, T. H., Whitehorn, A., Pollock, D., Marin, T. S., & Dunlop, S. A. (2024) · Annals of Global Health · doi:10.5334/aogh.4459 · Cited by 165
Polycyclic aromatic hydrocarbons and female reproductive health: A scoping review
Bolden, A. L., Rochester, J. R., Schultz, K., & Kwiatkowski, C. F. (2017) · Reproductive Toxicology · doi:10.1016/j.reprotox.2017.07.012 · Cited by 163
Kirrane, E. F., Luben, T. J., Benson, A., Owens, E. O., Sacks, J. D., Dutton, S. J., Madden, M., & Nichols, J. L. (2019) · Environment International · doi:10.1016/j.envint.2019.02.027 · Cited by 161
A path forward in the debate over health impacts of endocrine disrupting chemicals
Zoeller, R. T., Bergman, Å., Becher, G., Bjerregaard, P., Bornman, R., Brandt, I., Iguchi, T., Jobling, S., Kidd, K. A., Kortenkamp, A., et al. (2014) · Environmental Health · doi:10.1186/1476-069x-13-118 · Cited by 141
Cano-Sancho, G., Salmon, A. G., & La Merrill, M. A. (2017) · Environmental Health Perspectives · doi:10.1289/ehp527 · Cited by 131
Zoeller, R. T., & Vandenberg, L. N. (2015) · Environmental Health · doi:10.1186/s12940-015-0029-4 · Cited by 151
Johnson, P. I., Koustas, E., Vesterinen, H. M., Sutton, P., Atchley, D. S., Kim, A. N., Campbell, M., Donald, J. M., Sen, S., Bero, L., et al. (2016) · Environment International · doi:10.1016/j.envint.2016.03.009 · Cited by 141
Rochester, J. R., Bolden, A. L., & Kwiatkowski, C. F. (2018) · Environment International · doi:10.1016/j.envint.2017.12.028 · Cited by 145
Wildfire exposure during pregnancy and the risk of adverse birth outcomes: A systematic review
Amjad, S., Chojecki, D., Osornio-Vargas, Á. R., & Ospina, M. B. (2021) · Environment International · doi:10.1016/j.envint.2021.106644 · Cited by 145
Luben, T. J., Nichols, J. L., Dutton, S. J., Kirrane, E. F., Owens, E. O., Datko-Williams, L., Madden, M., & Sacks, J. D. (2017) · Environment International · doi:10.1016/j.envint.2017.07.005 · Cited by 139
Emerging investigator series: use of behavioural endpoints in the regulation of chemicals
Ågerstrand, M., Arnold, K. E., Balshine, S., Brodin, T., Brooks, B. W., Maack, G., McCallum, E. S., Pyle, G. G., Saaristo, M., & Ford, A. T. (2019) · Environmental Science Processes & Impacts · doi:10.1039/c9em00463g · Cited by 139
Risk Perception: It’s Personal
Brown, V. J. (2014) · Environmental Health Perspectives · doi:10.1289/ehp.122-a276 · Cited by 135
Vandenberg, L. N., Ågerstrand, M., Beronius, A., Beausoleil, C., Bergman, Å., Bero, L. A., Bornehag, C.-G., Boyer, C. S., Cooper, G. S., Cotgreave, I., et al. (2016) · Environmental Health · doi:10.1186/s12940-016-0156-6 · Cited by 123
Self-citationAir pollution and human cognition: A systematic review and meta-analysis
Thompson, R., Smith, R. B., Karim, Y. B., Shen, C., Drummond, K., Teng, C., & Toledano, M. B. (2022) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2022.160234 · Cited by 134
Shaffer, R. M., Sellers, S. P., Baker, M. G., de Buen Kalman, R., Frostad, J., Suter, M. K., Anenberg, S. C., Balbus, J., Basu, N., Bellinger, D. C., et al. (2019) · Environmental Health Perspectives · doi:10.1289/ehp5496 · Cited by 117
Self-citationWolffe, T. A. M., Whaley, P., Halsall, C., Rooney, A. A., & Walker, V. R. (2019) · Environment International · doi:10.1016/j.envint.2019.05.065 · Cited by 119
Self-citationMaele‐Fabry, G. V., Gamet‐Payrastre, L., & Lison, D. (2018) · International Journal of Hygiene and Environmental Health · doi:10.1016/j.ijheh.2018.08.004 · Cited by 123
Kandhare, A. D., Thakurdesai, P. A., Wangikar, P. B., & Bodhankar, S. L. (2019) · Heliyon · doi:10.1016/j.heliyon.2019.e01536 · Cited by 123
A review of methods used for hazard identification and risk assessment of environmental hazards
Chartres, N., Bero, L. A., & Norris, S. L. (2018) · Environment International · doi:10.1016/j.envint.2018.11.060 · Cited by 118
Hu, C., Yang, X.-J., Gui, S., Ding, K., Huang, K. J., Fang, Y., Jiang, Z., & Zhang, X.-J. (2020) · Environmental Research · doi:10.1016/j.envres.2020.110599 · Cited by 117
Houessionon, K. G., Ouendo, E., Bouland, C., Takyi, S. A., Kêdoté, N. M., Fayomi, B., Fobil, J. N., & Basu, N. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18073517 · Cited by 117
Uses of NHANES Biomarker Data for Chemical Risk Assessment: Trends, Challenges, and Opportunities
Sobus, J. R., DeWoskin, R. S., Tan, Y., Pleil, J. D., Phillips, M. B., George, B. J., Christensen, K. L. Y., Schreinemachers, D. M., Williams, M. A., Hubal, E. A. C., et al. (2015) · Environmental Health Perspectives · doi:10.1289/ehp.1409177 · Cited by 117
Andersen, H. R., David, A., Freire, C., Fernández, M. F., D’Cruz, S. C., Reina-Pérez, I., Fini, J., & Bláha, L. (2022) · Environmental Research · doi:10.1016/j.envres.2022.113935 · Cited by 115
Rooney, A. A., Cooper, G. S., Jahnke, G. D., Lam, J., Morgan, R. L., Boyles, A. L., Ratcliffe, J. M., Kraft, A. D., Schünemann, H. J., Schwingl, P., et al. (2016) · Environment International · doi:10.1016/j.envint.2016.01.005 · Cited by 95
Self-citationParaquat and Parkinson’s disease: a systematic review and meta-analysis of observational studies
Vaccari, C., Dib, R. E., Gomaa, H. A., Lopes, L. C., & de Camargo, J. L. V. (2019) · Journal of Toxicology and Environmental Health Part B · doi:10.1080/10937404.2019.1659197 · Cited by 110
Tang, H., Cheng, Z., Li, N., Mao, S., Ma, R., He, H., Niu, Z., Chen, X., & Xiang, H. (2020) · Environmental Pollution · doi:10.1016/j.envpol.2020.115630 · Cited by 106
von Stackelberg, K., Guzy, E., Chu, T., & Henn, B. C. (2015) · Risk Analysis · doi:10.1111/risa.12425 · Cited by 104
Cano-Sancho, G., Ploteau, S., Matta, K., Adoamnei, E., Louis, G. B., Mendiola, J., Darai, E., Squifflet, J., Le Bizec, B., & Antignac, J.-P. (2019) · Environment International · doi:10.1016/j.envint.2018.11.065 · Cited by 89
Zhang, H., Ben, Y., Han, Y.-H., Zhang, Y., Li, Y., & Chen, X. (2021) · Environmental Research · doi:10.1016/j.envres.2021.112109 · Cited by 98
Whaley, P., Halsall, C., Ågerstrand, M., Aiassa, E., Benford, D., Bilotta, G., Coggon, D., Collins, C., Dempsey, C., Duarte-Davidson, R., et al. (2016) · Environment International · doi:10.1016/j.envint.2015.11.002 · Cited by 91
Self-citationArzuaga, X., Smith, M. T., Gibbons, C. F., Skakkebæk, N. E., Yost, E. E., Beverly, B., Hotchkiss, A. K., Hauser, R. B., de Souza, R. P., Schrader, S. M., et al. (2019) · Environmental Health Perspectives · doi:10.1289/ehp5045 · Cited by 96
Association between greenspace and blood pressure: A systematic review and meta-analysis
Zhao, Y., Bao, W.-W., Yang, B., Liang, J., Gui, Z., Huang, S., Chen, Y.-C., Dong, G., & Chen, Y.-J. (2022) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2021.152513 · Cited by 96
Eligibility screening in evidence synthesis of environmental management topics
Frampton, G. K., Livoreil, B., & Petrokofsky, G. (2017) · Environmental Evidence · doi:10.1186/s13750-017-0102-2 · Cited by 95
Chua, P. L., Ng, C. F. S., Tobías, A., Seposo, X., & Hashizume, M. (2021) · The Lancet Planetary Health · doi:10.1016/s2542-5196(22)00003-1 · Cited by 95
Vogel, J. A., van Veldhuisen, E., Agnass, P., Crezee, J., Dijk, F., Verheij, J., van Gulik, T. M., Meijerink, M. R., Vroomen, L. G. P. H., van Lienden, K. P., et al. (2016) · PLoS ONE · doi:10.1371/journal.pone.0166987 · Cited by 94
Osmanović, A., Halilović, S., Kurtović-Kozarić, A., & Hadžiabdić, N. (2018) · Dental Traumatology · doi:10.1111/edt.12437 · Cited by 93
Transdisciplinary approaches enhance the production of translational knowledge
Ciesielski, T. H., Aldrich, M. C., Marsit, C. J., Hiatt, R. A., & Williams, S. M. (2016) · Translational research · doi:10.1016/j.trsl.2016.11.002 · Cited by 92
Whaley, P., Aiassa, E., Beausoleil, C., Beronius, A., Bilotta, G., Boobis, A., de Vries, R., Hanberg, A., Hoffmann, S., Hunt, N., et al. (2020) · Environment International · doi:10.1016/j.envint.2020.105926 · Cited by 91
Self-citationBelesova, K., Agabiirwe, C. N., Zou, M., Phalkey, R. K., & Wilkinson, P. (2019) · Environment International · doi:10.1016/j.envint.2019.104973 · Cited by 91
Li, L., Guo, Y., Ma, S., Wen, H., Li, Y., & Qiao, J. (2023) · Environmental Research · doi:10.1016/j.envres.2023.117553 · Cited by 91
Taher, M. K., Momoli, F., Go, J., Hagiwara, S., Ramoju, S., Hu, X. F., Jensen, N., Terrell, R. M., Hemmerich, A., & Krewski, D. R. (2024) · Critical Reviews in Toxicology · doi:10.1080/10408444.2023.2295338 · Cited by 90
Riedlinger, C., Schmidt, G., Weiland, A., Stengel, A., Giel, K. E., Zipfel, S., Enck, P., & Mack, I. (2020) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2020.00195 · Cited by 72
Li, J., Zheng, X., Liu, X., Zhang, L., Zhang, S., Li, Y., Zhang, W., Li, Q., Zhao, Y., Chen, X., et al. (2023) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2023.167017 · Cited by 88
Childhood Leukemia: A Preventable Disease
Metayer, C., Dahl, G. V. H., Wiemels, J. L., & Miller, M. D. (2016) · PEDIATRICS · doi:10.1542/peds.2015-4268h · Cited by 88
Xu, Y.-X., Zhang, J., Fang-biao, T., & Sun, Y. (2022) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2022.159303 · Cited by 87
A primer on systematic reviews in toxicology
Hoffmann, S., de Vries, R. B. M., Stephens, M. L., Beck, N. B., Dirven, H. A. A. M., Fowle, J. R., Goodman, J. E., Hartung, T., Kimber, I., Lalu, M. M., et al. (2017) · Archives of Toxicology · doi:10.1007/s00204-017-1980-3 · Cited by 85
Self-citationGui, S., Qiao, J.-C., Xu, K., Li, Z., Chen, Y.-N., Wu, K., Jiang, Z., & Hu, C. (2022) · Journal of Exposure Science & Environmental Epidemiology · doi:10.1038/s41370-022-00464-3 · Cited by 85
Systematic review of conservation interventions to promote voluntary behavior change
Thomas‐Walters, L., McCallum, J., Montgomery, R., Petros, C., Wan, A. K. Y., & Veríssimo, D. (2022) · Conservation Biology · doi:10.1111/cobi.14000 · Cited by 85
Lai, K. Y., Sarkar, C., Ni, M. Y., Gallacher, J., & Webster, C. J. (2020) · Environmental Research · doi:10.1016/j.envres.2020.109637 · Cited by 84
Bianchi, S., Nottola, S. A., Torge, D., Palmerini, M. G., Necozione, S., & Macchiarelli, G. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17072580 · Cited by 83
Linkov, I., Massey, O., Keisler, J., Rusyn, I., & Hartung, T. (2015) · ALTEX · doi:10.14573/altex.1412231 · Cited by 83
Eick, S. M., Goin, D. E., Chartres, N., Lam, J., & Woodruff, T. J. (2020) · Systematic Reviews · doi:10.1186/s13643-020-01490-8 · Cited by 82
Samuel, G. O., Hoffmann, S., Wright, R. A., Lalu, M. M., Patlewicz, G., Becker, R. A., DeGeorge, G. L., Fergusson, D., Hartung, T., Lewis, R. J., et al. (2016) · Environment International · doi:10.1016/j.envint.2016.03.010 · Cited by 82
Balise, V. D., Meng, C., Cornelius-Green, J. N., Kassotis, C. D., Kennedy, R. F., & Nagel, S. C. (2016) · Fertility and Sterility · doi:10.1016/j.fertnstert.2016.07.1099 · Cited by 82
Systematic Review of Chromium and Nickel Exposure During Pregnancy and Impact on Child Outcomes
McDermott, S. W., Salzberg, D., Anderson, A. P., Shaw, T. F., & Lead, J. R. (2015) · Journal of Toxicology and Environmental Health · doi:10.1080/15287394.2015.1090939 · Cited by 81
Dirven, H. A. A. M., Vist, G. E., Bandhakavi, S., Mehta, J., Fitch, S., Pound, P., Ram, R., Kincaid, B., Leenaars, C. H. C., Chen, M., et al. (2021) · Scientific Reports · doi:10.1038/s41598-021-85708-2 · Cited by 80
Ozone exposure during early pregnancy and preterm birth: A systematic review and meta-analysis
Rappazzo, K. M., Nichols, J. L., Rice, R. B., & Luben, T. J. (2021) · Environmental Research · doi:10.1016/j.envres.2021.111317 · Cited by 80
(EFSA), E. F. S. A., Devos, Y., Muñoz‐Guajardo, I., Álvarez, F., & Glanville, J. (2019) · EFSA Supporting Publications · doi:10.2903/sp.efsa.2019.en-1614 · Cited by 80
Health Effects of Fossil Fuel–Derived Endocrine Disruptors
Woodruff, T. J. (2024) · New England Journal of Medicine · doi:10.1056/nejmra2300476 · Cited by 80
Melamine, beyond the kidney: A ubiquitous endocrine disruptor and neurotoxicant?
Bolden, A. L., Rochester, J. R., & Kwiatkowski, C. F. (2017) · Toxicology Letters · doi:10.1016/j.toxlet.2017.07.893 · Cited by 77
Petri, A.-K., Schmiedchen, K., Stunder, D., Dechent, D., Kraus, T., Bailey, W. H., & Drießen, S. (2017) · Environmental Health · doi:10.1186/s12940-017-0248-y · Cited by 75
Systematic review of the literature on triclosan and health outcomes in humans
Goodman, M. G., Naiman, D. Q., & LaKind, J. S. (2017) · Critical Reviews in Toxicology · doi:10.1080/10408444.2017.1350138 · Cited by 75
Cross-discipline evidence principles for sustainability policy
Game, E. T., Tallis, H. M., Olander, L. P., Alexander, S. M., Busch, J., Cartwright, N. D., Kalies, E. L., Masuda, Y. J., Mupepele, A., Qiu, J., et al. (2018) · Nature Sustainability · doi:10.1038/s41893-018-0141-x · Cited by 74
Self-citationExposure to light at night (LAN) and risk of breast cancer: A systematic review and meta-analysis
Lai, K. Y., Sarkar, C., Ni, M. Y., Cheung, L. W. T., Gallacher, J., & Webster, C. J. (2020) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2020.143159 · Cited by 74
Sprague, N. L., Bancalari, P., Karim, W., & Siddiq, S. (2022) · Journal of Exposure Science & Environmental Epidemiology · doi:10.1038/s41370-022-00445-6 · Cited by 74
(NTP), N. T. P. (2024) · doi:10.22427/ntp-mgraph-8 · Cited by 74
Principles and process for dealing with data and evidence in scientific assessments
(EFSA), E. F. S. A. (2015) · EFSA Journal · doi:10.2903/j.efsa.2015.4121 · Cited by 74
Dorman, D. C., Chiu, W. A., Hales, B. F., Hauser, R. B., Johnson, K. J., Mantus, E., Martel, S., Robinson, K. A., Rooney, A. A., Rudel, R. A., et al. (2018) · Journal of Toxicology and Environmental Health Part B · doi:10.1080/10937404.2018.1505354 · Cited by 73
Self-citationGhosh, R., Causey, K., Burkart, K. G., Wozniak, S. N., Cohen, A., & Brauer, M. D. (2021) · PLoS Medicine · doi:10.1371/journal.pmed.1003852 · Cited by 72
Environmental Chemicals and Autism: A Scoping Review of the Human and Animal Research
Pelch, K. E., Bolden, A. L., & Kwiatkowski, C. F. (2019) · Environmental Health Perspectives · doi:10.1289/ehp4386 · Cited by 71
Associations of Urban Built Environment with Cardiovascular Risks and Mortality: a Systematic Review
Lai, K. Y., Webster, C. J., Gallacher, J., & Sarkar, C. (2023) · Journal of Urban Health · doi:10.1007/s11524-023-00764-5 · Cited by 69
Doepker, C., Franke, K., Myers, E., Goldberger, J. J., Lieberman, H. R., O’Brien, C., Peck, J., Tenenbein, M., Weaver, C., & Wikoff, D. (2018) · Nutrients · doi:10.3390/nu10101536 · Cited by 59
Dorman, D. C., Chiu, W. A., Hales, B. F., Hauser, R. B., Johnson, K. J., Mantus, E., Martel, S., Robinson, K. A., Rooney, A. A., Rudel, R. A., et al. (2018) · Journal of Toxicology and Environmental Health Part B · doi:10.1080/10937404.2018.1514829 · Cited by 67
Self-citation
Metadata
- Authors
- Rooney, A. A., Boyles, A. L., Wolfe, M. S., Bucher, J. R., & Thayer, K. A.
- Affiliations
- National Institutes of Health (US)
- Corresponding author
- Rooney, Andrew A.
- Venue
- Environmental Health Perspectives
- Publisher
- American Chemical Society (ACS)
- Published
- 2014-04-22
- Volume
- 122
- Issue
- 7
- Pages
- 711-718
- ISSN
- 0091-6765 (ISSN-L)
- 1552-9924
- PMC
- PMC4080517
- OpenAlex
- W2066218383
- Semantic Scholar
- f30c4b27530fb3fa34898432b9850e3fbcdbde0e
- Cited by
- 434
- References
- 31
- Citations by year
- 2026: 34 · 2025: 33 · 2024: 43 · 2023: 34 · 2022: 39 · 2021: 53 · 2020: 40 · 2019: 29 · 2018: 33 · 2017: 25 · 2016: 31 · 2015: 28 · 2014: 12
- FWCI
- 8.81
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- public-domain
- creativecommons.org/licenses/by/4.0/ (vor, 1347 d)
- Locations
- Environmental Health Perspectives (publishedVersion, public-domain)
- Europe PMC (PubMed Central) (publishedVersion, public-domain)
- PubMed Central (publishedVersion, public-domain)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Environmental Science › Health, Toxicology and Mutagenesis › Health, Environment, Cognitive Aging
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- systematic reviews, evidence integration, environmental health risk assessment, hazard identification, level of evidence, confidence in evidence, risk of bias assessment, mechanistic studies, animal studies, human subjects research
- MeSH terms
- Systematic Reviews as Topic, Animals, Environmental Health, Humans, United States, Evidence-Based Medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
