Cited by 248Meta-analysis and systematic reviewsStatistical Methods in EpidemiologyAdvanced Causal Inference Techniques

The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.

A risk of bias instrument for non-randomized studies of exposures: A users' guide to its application in the context of GRADE

Open access

Morgan, Rebecca L., Thayer, Kristina A., Santesso, Nancy, Holloway, Alison C., Blain, Robyn, Eftim, Sorina E., Goldstone, Alexandra E., Ross, Pam, Ansari, Mohammed, Akl, Elie A, Filippini, Tommaso, Hansell, Anna, Meerpohl, Joerg J., Mustafa, Reem A., Verbeek, Jos, Vinceti, Marco, Whaley, Paul, Schünemann, Holger J.·2019·Environment International

DOI 10.1016/j.envint.2018.11.004PMC PMC8221004

Open this paper in CoChat — free account Open PDF

Abstract

The objective of this paper is to explain how to apply, interpret, and present the results of a new instrument to assess the risk of bias (RoB) in non-randomized studies (NRS) dealing with effects of environmental exposures on health outcomes. This instrument is modeled on the Risk Of Bias In Non-randomized Studies of Interventions (ROBINS-I) instrument. The RoB instrument for NRS of exposures assesses RoB along a standardized comparison to a randomized target experiment, instead of the study-design directed RoB approach. We provide specific guidance for the integral steps of developing a research question and target experiment, distinguishing issues of indirectness from RoB, making individual-study judgments, and performing and interpreting sensitivity analyses for RoB judgments across a body of evidence. Also, we present an approach for integrating the RoB assessments within the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) framework to assess the certainty of the evidence in the systematic review. Finally, we guide the reader through an overall assessment to support the rating of all domains that determine the certainty of a body of evidence using the GRADE approach.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1016/j.envint.2018.11.004

Cites

33 of 33 references verified0 citation errors · 0 not checkable

Statistics

Fig. F.1. Sensitivity analysis of studies with ‘Serious’ bias due to confounding.CC BY-NC-ND · PubMed Central

Fig. F.2. Sensitivity analysis of all studies.CC BY-NC-ND · PubMed Central

Fig. G.1. Sensitivity analysis of studies with ‘Serious’ bias due to confounding.CC BY-NC-ND · PubMed Central

Fig. G.2. Sensitivity analysis of all studies.CC BY-NC-ND · PubMed Central

Fig. 1. Approach for conducting an assessment using the RoB instrument for NRS of exposures and the integration into GRADE when conducting systematic reviews of exposure. GRADE: Grading of Recommendations Assessment, Development and Evaluation; PECO: population, exposure, comparator, outcome; RoB: risk of bias; SR: systematic review.CC BY-NC-ND · PubMed Central

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329
Self-citation
GRADE guidelines: 3. Rating the quality of evidence
Balshem, H., Helfand, M., Schünemann, H. J., Oxman, A. D., Kunz, R., Brozek, J., Vist, G. E., Falck-Ytter, Y., Meerpohl, J., & Norris, S. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.015 · Cited by 8,285
Self-citation
Using Big Data to Emulate a Target Trial When a Randomized Trial Is Not Available: Table 1.
Hernán, M. A., & Robins, J. M. (2016) · American Journal of Epidemiology · doi:10.1093/aje/kwv254 · Cited by 3,115
GRADE guidelines: 4. Rating the quality of evidence—study limitations (risk of bias)
Guyatt, G. H., Oxman, A. D., Vist, G., Kunz, R., Brozek, J., Alonso-Coello, P., Montori, V., Akl, E. A., Djulbegovic, B., & Falck-Ytter, Y. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.017 · Cited by 2,798
Self-citation
Smoking and Carcinoma of the Lung
Doll, R., & Hill, A. B. (1950) · BMJ · doi:10.1136/bmj.2.4682.739 · Cited by 1,999
GRADE guidelines: 8. Rating the quality of evidence—indirectness
Guyatt, G. H., Oxman, A. D., Kunz, R., Woodcock, J., Brozek, J., Helfand, M., Alonso-Coello, P., Falck-Ytter, Y., Jaeschke, R., Vist, G., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.04.014 · Cited by 1,843
Self-citation
Identifying the PECO: A framework for formulating good questions to explore the association of environmental and other exposures with health outcomes
Morgan, R. L., Whaley, P., Thayer, K. A., & Schünemann, H. J. (2018) · Environment International · doi:10.1016/j.envint.2018.07.015 · Cited by 1,424
Self-citation
GRADE guidelines: 9. Rating up the quality of evidence
Guyatt, G. H., Oxman, A. D., Sultan, S., Glasziou, P., Akl, E. A., Alonso-Coello, P., Atkins, D., Kunz, R., Brozek, J., Montori, V., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.06.004 · Cited by 1,336 · cited 6×
“These latter two domains contribute to increasing the CoE in a body of evidence of NRS in GRADE; however, within the RoB instrument for NRS of exposures they may also influence the study-level judgments (Guyatt et al., 2011c).”
BackgroundResultInfluentialSelf-citation
Untitled work
(2016) · Cochrane Database of Systematic Reviews · Cited by 985

Show all 33 references

Research works printed but not linked: 1

Thayer, 2013, Draft protocol for systematic review to evaluate the evidence for an association between bisphenol A (BPA) exposure and obesity, Natl. Toxicol. Program

Sorted by citations

A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E)
Higgins, J. P. T., Morgan, R. L., Rooney, A. A., Taylor, K. W., Thayer, K. A., Silva, R. A., Lemeris, C., Akl, E. A., Bateson, T. F., Berkman, N. D., et al. (2024) · Environment International · doi:10.1016/j.envint.2024.108602 · Cited by 1,022
“Early work by a subset of the authors considered the extent to which the ROBINS-I tool was suitable for observational studies of exposures and suggested modifications to the tool (Morgan et al., 2018; Morgan et al., 2019).”
Self-citation
Short-term exposure to particulate matter (PM10 and PM2.5), nitrogen dioxide (NO2), and ozone (O3) and all-cause and cause-specific mortality: Systematic review and meta-analysis
Orellano, P. W., Reynoso, J. I., QUARANTA, N. E., Bardach, A., & Ciapponi, A. (2020) · Environment International · doi:10.1016/j.envint.2020.105876 · Cited by 857
Long-term exposure to NO2 and O3 and all-cause and respiratory mortality: A systematic review and meta-analysis
Huangfu, P., & Atkinson, R. W. (2020) · Environment International · doi:10.1016/j.envint.2020.105998 · Cited by 548
Nurse Burnout and Patient Safety, Satisfaction, and Quality of Care: A Systematic Review and Meta-Analysis
Li, L. Z., Yang, P., Singer, S. J., Pfeffer, J., Mathur, M. B., & Shanafelt, T. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.43059 · Cited by 501
Air pollution exposure and depression: A comprehensive updated systematic review and meta-analysis
Borroni, E., Pesatori, A. C., Bollati, V., Buoli, M., & Carugno, M. (2021) · Environmental Pollution · doi:10.1016/j.envpol.2021.118245 · Cited by 287
Short-term exposure to sulphur dioxide (SO2) and all-cause and respiratory mortality: A systematic review and meta-analysis
Orellano, P. W., Reynoso, J. I., & QUARANTA, N. E. (2021) · Environment International · doi:10.1016/j.envint.2021.106434 · Cited by 261
Ambient air pollution and depression: A systematic review with meta-analysis up to 2019
Fan, S., Heinrich, J., Bloom, M. S., Zhao, T., Shi, T., Feng, W., Sun, Y., Shen, J., Yang, Z., Yang, B., et al. (2019) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2019.134721 · Cited by 261
Long-term exposure to nitrogen dioxide and mortality: A systematic review and meta-analysis
Huang, S., Li, H., Wang, M., Qian, Y., Steenland, K., Caudle, W. M., Liu, Y., Sarnat, J. A., Papatheodorou, S., & Shi, L. (2021) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2021.145968 · Cited by 236
Greenspace with overweight and obesity: A systematic review and meta‐analysis of epidemiological studies up to 2020
Luo, Y., Huang, W., Liu, X., Markevych, I., Bloom, M. S., Zhao, T., Heinrich, J., Yang, B., & Dong, G. (2020) · Obesity Reviews · doi:10.1111/obr.13078 · Cited by 171
Frailty and Physical Fitness in Elderly People: A Systematic Review and Meta-analysis
Navarrete‐Villanueva, D., Gómez‐Cabello, A., Marín-Puyalto, J., Moreno, L. A., Vicente‐Rodríguez, G., & Casajús, J. A. (2020) · Sports Medicine · doi:10.1007/s40279-020-01361-1 · Cited by 175
Cadmium exposure and risk of diabetes and prediabetes: A systematic review and dose-response meta-analysis
Filippini, T., Wise, L. A., & Vinceti, M. (2021) · Environment International · doi:10.1016/j.envint.2021.106920 · Cited by 145
Self-citation
8-Hydroxy-2′-deoxyguanosine (8-OHdG) as a biomarker of oxidative DNA damage induced by occupational exposure to nanomaterials: a systematic review
Shekaftik, S. O., & Nasirzadeh, N. (2021) · Nanotoxicology · doi:10.1080/17435390.2021.1936254 · Cited by 144
Increase of depression among children and adolescents after the onset of the COVID-19 pandemic in Europe: a systematic review and meta-analysis
Ludwig‐Walz, H., Dannheim, I., Pfadenhauer, L. M., Fegert, J. M., & Bujard, M. (2022) · Child and Adolescent Psychiatry and Mental Health · doi:10.1186/s13034-022-00546-y · Cited by 151
Gender differences in the association between unpaid labour and mental health in employed adults: a systematic review
Ervin, J., Taouk, Y., Alfonzo, L. F., Hewitt, B. A., & King, T. L. (2022) · The Lancet Public Health · doi:10.1016/s2468-2667(22)00160-8 · Cited by 130
Cadmium exposure and cardiovascular disease risk: A systematic review and dose-response meta-analysis
Verzelloni, P., Urbano, T., Wise, L. A., Vinceti, M., & Filippini, T. (2024) · Environmental Pollution · doi:10.1016/j.envpol.2024.123462 · Cited by 110
Self-citation
The impact of urinary incontinence on falls: A systematic review and meta-analysis
Moon, S., Chung, H. S., Kim, Y. J., Kim, S. J., Kwon, O., Lee, Y. G., Yu, J. M., & Cho, S. T. (2021) · PLoS ONE · doi:10.1371/journal.pone.0251711 · Cited by 118
Update of the WHO global air quality guidelines: Systematic reviews – An introduction
Velasco, R. P., & Jarosińska, D. (2022) · Environment International · doi:10.1016/j.envint.2022.107556 · Cited by 110
Alcohol Intake and Risk of Hypertension: A Systematic Review and Dose-Response Meta-Analysis of Nonexperimental Cohort Studies
Cecchini, M., Filippini, T., Whelton, P. K., Iamandii, I., Federico, S. D., Boriani, G., & Vinceti, M. (2024) · Hypertension · doi:10.1161/hypertensionaha.124.22703 · Cited by 101
Self-citation
Principles and framework for assessing the risk of bias for studies included in comparative quantitative environmental systematic reviews
Frampton, G. K., Whaley, P., Bennett, M. G., Bilotta, G. S., Dorne, J. L. C. M., Eales, J. F., James, K. L., Kohl, C., Land, M., Livoreil, B., et al. (2022) · Environmental Evidence · doi:10.1186/s13750-022-00264-0 · Cited by 108
Self-citation
Alcohol Intake and Blood Pressure Levels: A Dose-Response Meta-Analysis of Nonexperimental Cohort Studies
Federico, S. D., Filippini, T., Whelton, P. K., Cecchini, M., Iamandii, I., Boriani, G., & Vinceti, M. (2023) · Hypertension · doi:10.1161/hypertensionaha.123.21224 · Cited by 104
Self-citation
Fluoride exposure and cognitive neurodevelopment: Systematic review and dose-response meta-analysis
Veneri, F., Vinceti, M., Generali, L., Giannone, M. E., Mazzoleni, E., Birnbaum, L. S., Consolo, U., & Filippini, T. (2023) · Environmental Research · doi:10.1016/j.envres.2023.115239 · Cited by 97
Self-citation
Systematic review and meta-analysis of epidemiologic data on vaccine response in relation to exposure to five principal perfluoroalkyl substances
Crawford, L., Halperin, S. A., Dzierlenga, M. W., Skidmore, B., Linakis, M. W., Nakagawa, S., & Longnecker, M. P. (2023) · Environment International · doi:10.1016/j.envint.2023.107734 · Cited by 93
Sodium Intake and Risk of Hypertension: A Systematic Review and Dose–Response Meta-analysis of Observational Cohort Studies
Filippini, T., Malavolti, M., Whelton, P. K., & Vinceti, M. (2022) · Current Hypertension Reports · doi:10.1007/s11906-022-01182-9 · Cited by 96
Self-citation
Long-Term Exposure to Particulate Matter and Mortality: An Update of the WHO Global Air Quality Guidelines Systematic Review and Meta-Analysis
Orellano, P. W., Kasdagli, M., Velasco, R. P., & Samoli, E. (2024) · International Journal of Public Health · doi:10.3389/ijph.2024.1607683 · Cited by 98
How do employment conditions and psychosocial workplace exposures impact the mental health of young workers? A systematic review
Shields, M., Dimov, S., Kavanagh, A. M., Milner, A. J., Spittal, M. J., & King, T. L. (2021) · Social Psychiatry and Psychiatric Epidemiology · doi:10.1007/s00127-021-02077-x · Cited by 85
Association between greenspace and blood pressure: A systematic review and meta-analysis
Zhao, Y., Bao, W.-W., Yang, B., Liang, J., Gui, Z., Huang, S., Chen, Y.-C., Dong, G., & Chen, Y.-J. (2022) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2021.152513 · Cited by 80
Short-term exposure to carbon monoxide and myocardial infarction: A systematic review and meta-analysis
Lee, K. K., Spath, N., Miller, M. R., Mills, N. L., & Shah, A. S. V. (2020) · Environment International · doi:10.1016/j.envint.2020.105901 · Cited by 80
Challenges in applying the GRADE approach in public health guidelines and systematic reviews: a concept article from the GRADE Public Health Group
Boon, M. H., Thomson, H., Shaw, B., Akl, E. A., Lhachimi, S. K., López‐Alcalde, J., Klugar, M., Choi, L., Saz‐Parkinson, Z., Mustafa, R. A., et al. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.01.001 · Cited by 81
Self-citation
Light at night and risk of breast cancer: a systematic review and dose–response meta-analysis
Urbano, T., Vinceti, M., Wise, L. A., & Filippini, T. (2021) · International Journal of Health Geographics · doi:10.1186/s12942-021-00297-7 · Cited by 86
Self-citation
Leveraging real-world data to investigate multiple sclerosis disease behavior, prognosis, and treatment
Cohen, J. A., Trojano, M., Mowry, E. M., Uitdehaag, B. M. J., Reingold, S. C., & Marrie, R. A. (2019) · Multiple Sclerosis Journal · doi:10.1177/1352458519892555 · Cited by 77
Risk of bias assessments and evidence syntheses for observational epidemiologic studies of environmental and occupational exposures: strengths and limitations
Steenland, K., Schubauer-Berigan, M. K., Vermeulen, R., Lunn, R. M., Straif, K., Zahm, S., Stewart, P., Arroyave, W. D., Mehta, S. S., & Pearce, N. (2020) · Environmental Health Perspectives · doi:10.1289/ehp6980 · Cited by 69
Allergen immunotherapy: The growing role of observational and randomized trial “Real‐World Evidence”
Paoletti, G., Bona, D. D., Chu, D. K., Firinu, D., Heffler, E. M., Agache, I. O., Jutel, M., Klimek, L., Pfaar, O., Mösges, R., et al. (2021) · Allergy · doi:10.1111/all.14773 · Cited by 75
Assessing the certainty of the evidence in systematic reviews: importance, process, and use
Brignardello‐Petersen, R., & Guyatt, G. H. (2024) · American Journal of Epidemiology · doi:10.1093/aje/kwae332 · Cited by 91
Improving the trustworthiness of findings from nutrition evidence syntheses: assessing risk of bias and rating the certainty of evidence
Schwingshackl, L., Schünemann, H. J., & Meerpohl, J. J. (2020) · European Journal of Nutrition · doi:10.1007/s00394-020-02464-1 · Cited by 77
Self-citation
Evidence for Human-to-Human Transmission of Hantavirus: A Systematic Review
Toledo, J., Haby, M. M., Reveiz, L., Sosa Leon, L., Angerami, R., & Aldighieri, S. (2022) · The Journal of Infectious Diseases · doi:10.1093/infdis/jiab461 · Cited by 78
Selecting Risk of Bias Tools for Observational Studies for a Systematic Review of Anthropometric Measurements and Dental Caries among Children
Mamikutty, R., Aly, A. S., & Marhazlinda, J. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18168623 · Cited by 68
Cadmium exposure and risk of breast cancer: A meta-analysis
Flórez‐García, V., Guevara-Romero, E., Hawkins, M. M., Bautista, L., Jenson, T. E., Yu, J. S., & Kalkbrenner, A. (2022) · Environmental Research · doi:10.1016/j.envres.2022.115109 · Cited by 64
Fish intake, n-3 fatty acid body status, and risk of cognitive decline: a systematic review and a dose–response meta-analysis of observational and experimental studies
Kosti, R. I., Kasdagli, M., Kyrozis, A., Orsini, N., Λάγιου, Π., Taiganidou, F., & Naska, A. (2021) · Nutrition Reviews · doi:10.1093/nutrit/nuab078 · Cited by 63
Alcohol, Intraocular Pressure, and Open-Angle Glaucoma
Stuart, K. V., Madjedi, K. M., Luben, R., Chua, S. Y. L., Warwick, A., Chia, M. A., Pasquale, L. R., Wiggs, J. L., Kang, J. H., Hysi, P. G., et al. (2022) · Ophthalmology · doi:10.1016/j.ophtha.2022.01.023 · Cited by 66
Mental health of young informal carers: a systematic review
Alfonzo, L. F., Singh, A., Disney, G., Ervin, J., & King, T. L. (2022) · Social Psychiatry and Psychiatric Epidemiology · doi:10.1007/s00127-022-02333-8 · Cited by 63
Fluoride Exposure and Skeletal Fluorosis: a Systematic Review and Dose-response Meta-analysis
Veneri, F., Iamandii, I., Vinceti, M., Birnbaum, L. S., Generali, L., Consolo, U., & Filippini, T. (2023) · Current Environmental Health Reports · doi:10.1007/s40572-023-00412-9 · Cited by 62
Self-citation
Association of urinary or blood heavy metals and mortality from all causes, cardiovascular disease, and cancer in the general population: a systematic review and meta-analysis of cohort studies
Guo, X., Su, W., Li, N., Song, Q., Wang, H., Liang, Q., Li, Y., Lowe, S., Bentley, R., Zhou, Z., et al. (2022) · Environmental Science and Pollution Research · doi:10.1007/s11356-022-22353-w · Cited by 58
Is short-term and long-term exposure to black carbon associated with cardiovascular and respiratory diseases? A systematic review and meta-analysis based on evidence reliability
Song, X., Hu, Y., Ma, Y., Jiang, L., Wang, X., Shi, A., Zhao, J., Liu, Y., Liu, Y., Tang, J. Y., et al. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-049516 · Cited by 59
RoB-SPEO: A tool for assessing risk of bias in studies estimating the prevalence of exposure to occupational risk factors from the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury
Pega, F., Norris, S., Backes, C., Bero, L., Descatha, A., Gagliardi, D., Godderis, L., Loney, T., Modenese, A., Morgan, R., et al. (2019) · Environment International · doi:10.1016/j.envint.2019.105039 · Cited by 44
Self-citation
Electronic cigarette use and smoking cessation in cohort studies and randomized trials: A systematic review and meta-analysis
Hedman, L., Galanti, M. R., Ryk, C., Gilljam, H., & Adermark, L. (2021) · Tobacco Prevention & Cessation · doi:10.18332/tpc/142320 · Cited by 58
Exposure to the pesticide DDT and risk of diabetes and hypertension: Systematic review and meta-analysis of prospective studies
Hernández-Mariano, J. Á., Baltazar-Reyes, M. C., Salazar-Martínez, E., & Cupul‐Uicab, L. A. (2021) · International Journal of Hygiene and Environmental Health · doi:10.1016/j.ijheh.2021.113865 · Cited by 56
Maturation of GFR in Term-Born Neonates: An Individual Participant Data Meta-Analysis
Smeets, N. J. L., IntHout, J., van der Burgh, M. J. P., Schwartz, G. J., Schreuder, M. F., & de Wildt, S. N. (2022) · Journal of the American Society of Nephrology · doi:10.1681/asn.2021101326 · Cited by 54
Atrial Fibrillation and the Risk of Early‐Onset Dementia: A Systematic Review and Meta‐Analysis
Giannone, M. E., Filippini, T., Whelton, P. K., Chiari, A., Vitolo, M., Boriani, G., & Vinceti, M. (2022) · Journal of the American Heart Association · doi:10.1161/jaha.122.025653 · Cited by 55
Self-citation
Adverse intrapartum outcome in pregnancies complicated by small for gestational age and late fetal growth restriction undergoing induction of labor with Dinoprostone, Misoprostol or mechanical methods: A systematic review and meta-analysis
Familiari, A., Khalil, A. A., Rizzo, G., Odibo, A. O., Vergani, P., Buca, D. I. P., Hidaka, N., Mascio, D. D., Nwabuobi, C. K., Simeone, S., et al. (2020) · European Journal of Obstetrics & Gynecology and Reproductive Biology · doi:10.1016/j.ejogrb.2020.07.020 · Cited by 54
Is Greenness Associated with Dementia? A Systematic Review and Dose–Response Meta-analysis
Zagnoli, F., Filippini, T., Jimenez, M. P., Wise, L. A., Hatch, E. E., & Vinceti, M. (2022) · Current Environmental Health Reports · doi:10.1007/s40572-022-00365-5 · Cited by 57
Self-citation
The risk of tooth loss in patients with diabetes: A systematic review and meta‐analysis
Weijdijk, L. P. M., Žiūkaitė, L., van der Weijden, G. A., Bakker, E. W. P., & Slot, D. E. (2021) · International Journal of Dental Hygiene · doi:10.1111/idh.12512 · Cited by 53
Longitudinal association between informal unpaid caregiving and mental health amongst working age adults in high-income OECD countries: A systematic review
Ervin, J., Taouk, Y., Alfonzo, L. F., Peasgood, T., & King, T. L. (2022) · EClinicalMedicine · doi:10.1016/j.eclinm.2022.101711 · Cited by 53
Long-term NO2 exposure and mortality: A comprehensive meta-analysis
Chen, X., Qi, L., Li, S., & Duan, X. (2023) · Environmental Pollution · doi:10.1016/j.envpol.2023.122971 · Cited by 49
Determining factors associated with inhaled therapy adherence on asthma and COPD: A systematic review and meta-analysis of the global literature
Monteiro, C., Maricoto, T., Prazeres, F., Simões, P. A., & Simões, J. A. (2021) · Respiratory Medicine · doi:10.1016/j.rmed.2021.106724 · Cited by 53
Linking Physical Activity to Breast Cancer via Sex Hormones, Part 1: The Effect of Physical Activity on Sex Steroid Hormones
Swain, C. T. V., Drummond, A. E., Boing, L., Milne, R. L., English, D. R., Brown, K. A., van Roekel, E. H., Dixon‐Suen, S. C., Lynch, M. J., Moore, M. M., et al. (2021) · Cancer Epidemiology Biomarkers & Prevention · doi:10.1158/1055-9965.epi-21-0437 · Cited by 46
Prospective association between use of electronic cigarettes and use of conventional cigarettes: a systematic review and meta-analysis
Adermark, L., Galanti, M. R., Ryk, C., Gilljam, H., & Hedman, L. (2021) · ERJ Open Research · doi:10.1183/23120541.00976-2020 · Cited by 45
Electronic cigarettes and subsequent cigarette smoking in young people: A systematic review
Begh, R. A., Conde, M., Fanshawe, T. R., Kneale, D., Shahab, L., Zhu, S., Pesko, M. F., Livingstone‐Banks, J., Lindson, N., Rigotti, N. A., et al. (2025) · Addiction · doi:10.1111/add.16773 · Cited by 46
Linking Physical Activity to Breast Cancer via Sex Steroid Hormones, Part 2: The Effect of Sex Steroid Hormones on Breast Cancer Risk
Drummond, A. E., Swain, C. T. V., Brown, K. A., Dixon‐Suen, S. C., Boing, L., van Roekel, E. H., Moore, M. M., Gaunt, T. R., Milne, R. L., English, D. R., et al. (2021) · Cancer Epidemiology Biomarkers & Prevention · doi:10.1158/1055-9965.epi-21-0438 · Cited by 43
Impact of Noise Exposure on Risk of Developing Stress-Related Health Effects Related to the Cardiovascular System
Sivakumaran, K., Ritonja, J. A., Waseem, H., AlShenaibar, L., Morgan, E., Ahmadi, S. A., Denning, A., Michaud, D. S., & Morgan, R. L. (2022) · Noise and Health · doi:10.4103/nah.nah_83_21 · Cited by 39
Self-citation
Tobacco Smoking During Pregnancy Is Associated With Increased Risk of Moderate/Severe Bronchopulmonary Dysplasia: A Systematic Review and Meta-Analysis
González-Luis, G. E., van Westering-Kroon, E., Villamor-Martínez, E., Huizing, M. J., Kilani, M. A., Kramer, B. W. W., & Villamor, E. (2020) · Frontiers in Pediatrics · doi:10.3389/fped.2020.00160 · Cited by 40
Influence of Cooperative Learning Intervention on the Intrinsic Motivation of Physical Education Students—A Meta-Analysis within a Limited Range
Liu, T., & Lipowski, M. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18062989 · Cited by 38
Dietary fish and omega-3 polyunsaturated fatty acids intake and cancer survival: A systematic review and meta-analysis
Wang, Y., Liu, K., Long, T., Long, J., Li, Y., Li, J., & Cheng, L. (2022) · Critical Reviews in Food Science and Nutrition · doi:10.1080/10408398.2022.2029826 · Cited by 43
Challenges and recommendations on the conduct of systematic reviews of observational epidemiologic studies in environmental and occupational health
Arroyave, W. D., Mehta, S. S., Guha, N., Schwingl, P., Taylor, K. W., Glenn, B., Radke, E. G., Vilahur, N., Carreón, T., Nachman, R. M., et al. (2021) · Journal of Exposure Science & Environmental Epidemiology · doi:10.1038/s41370-020-0228-0 · Cited by 33
A Systematic Review and Appraisal of Epidemiological Studies on Household Fuel Use and Its Health Effects Using Demographic and Health Surveys
Odo, D. B., Yang, I. A., & Knibbs, L. D. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18041411 · Cited by 41
Gestational exposure to ambient particulate matter and preterm birth: An updated systematic review and meta-analysis
Yu, Z., Zhang, X., Zhang, J. J., Yang, F., Zhang, H., Wan, Z., Xiao, C., Zhang, H., Wang, Q., & Huang, C. (2022) · Environmental Research · doi:10.1016/j.envres.2022.113381 · Cited by 36
Neurodevelopmental effects of low dose ionizing radiation exposure: A systematic review of the epidemiological evidence
Pasqual, E., de Basea, M. B., López-Vicente, M., Thierry-Chef, I., & Cardis, E. (2020) · Environment International · doi:10.1016/j.envint.2019.105371 · Cited by 36
Development of an Evidence-Based Risk Assessment Framework
Krewski, D. R. (2022) · ALTEX · doi:10.14573/altex.2004041 · Cited by 41
GRADE GUIDANCE 38: Updated guidance for rating up certainty of evidence due to a dose-response gradient
Murad, M. H., Verbeek, J. H. A. M., Schwingshackl, L., Filippini, T., Vinceti, M., Akl, E. A., Morgan, R. L., Mustafa, R. A., Zeraatkar, D., Senerth, E., et al. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.09.011 · Cited by 35
Self-citation
Quiet, clean, green, and active: A Navigation Guide systematic review of the impacts of spatially correlated urban exposures on a range of physical health outcomes
Rugel, E. J., & Brauer, M. D. (2020) · Environmental Research · doi:10.1016/j.envres.2020.109388 · Cited by 34
Methodologically rigorous risk of bias tools for nonrandomized studies had low reliability and high evaluator burden
Jeyaraman, M. M., Rabbani, R., Copstein, L., Robson, R. C., Al‐Yousif, N., Pollock, M., Xia, J., Balijepalli, C., Hofer, K., Mansour, S., et al. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.09.033 · Cited by 32
Self-citation
Associations between multiple sclerosis and incidence of heart diseases: Systematic review and meta-analysis of observational studies
Rapp, D., Michels, S., Schöpe, J., Schwingshackl, L., Tumani, H., & Şenel, M. (2021) · Multiple Sclerosis and Related Disorders · doi:10.1016/j.msard.2021.103279 · Cited by 30
Real‐World Data and Real‐World Evidence in Regulatory Decision Making: Report Summary From the Council for International Organizations of Medical Sciences ( CIOMS ) Working Group XIII
Hennessy, S., Atsuta, Y., Hill, S., Rägo, L., & Juhaeri, J. (2025) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.70117 · Cited by 37
Prevalence of Cognitive Impairment in Individuals with Vascular Surgical Pathology: a Systematic Review and Meta-Analysis
Houghton, J. S. M., Nickinson, A., Bridgwood, B. M., Nduwayo, S., Pepper, C., Rayt, H. S., Gray, L. J., Haunton, V. J., & Sayers, R. D. (2021) · European Journal of Vascular and Endovascular Surgery · doi:10.1016/j.ejvs.2020.12.016 · Cited by 29
Educational attainment in patients with congenital heart disease: a comprehensive systematic review and meta-analysis
Cocomello, L., Dimagli, A., Biglino, G., Cornish, R., Caputo, M., & Lawlor, D. A. (2021) · BMC Cardiovascular Disorders · doi:10.1186/s12872-021-02349-z · Cited by 29
Dietary glycemic index, glycemic load and cancer risk: a meta-analysis of prospective cohort studies
Long, T., Liu, K., Long, J., Li, J., & Cheng, L. (2022) · European Journal of Nutrition · doi:10.1007/s00394-022-02797-z · Cited by 28
Urbanization, air pollution, and water pollution: Identification of potential environmental risk factors associated with amyotrophic lateral sclerosis using systematic reviews
Saucier, D., Registe, P. P. W., Bélanger, M. F., & O’Connell, C. M. (2023) · Frontiers in Neurology · doi:10.3389/fneur.2023.1108383 · Cited by 30
Draft for internal testing Scientific Committee guidance on appraising and integrating evidence from epidemiological studies for use in EFSA's scientific assessments
More, S. J., Bambidis, V., Benford, D., Bragard, C., Hernández, A. F., Bennekou, S. H., Koutsoumanis, K., Machera, K. A., Naegeli, H., Nielsen, S. S., et al. (2020) · EFSA Journal · doi:10.2903/j.efsa.2020.6221 · Cited by 26
Self-citation
Analgesic use and associated adverse events in patients with chronic kidney disease: a systematic review and meta-analysis
Lambourg, E. J., Colvin, L., Guthrie, G. D., Walker, H., & Bell, S. (2021) · British Journal of Anaesthesia · doi:10.1016/j.bja.2021.08.035 · Cited by 26
Effects of COVID-19 home confinement on sleep in children: A systematic review
Camacho-Montaño, L. R., Iranzo, Á., Martínez-Piédrola, R. M., Camacho-Montaño, L. M., Hoyas, E. H., Serrada-Tejeda, S., García‐Bravo, C., & Pérez‐de‐Heredia‐Torres, M. (2022) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2022.101596 · Cited by 27
Maternal acrylamide exposure during pregnancy and fetal growth: A systematic review and dose-response meta-analysis of epidemiological studies
Hogervorst, J. G. F., Virgolino, A., Halldorsson, T. I., Vinceti, M., Åkesson, A., Leander, K., Nawrot, T. S., Filippini, T., & Laguzzi, F. (2022) · Environmental Research · doi:10.1016/j.envres.2022.113705 · Cited by 26
Self-citation
Cadmium exposure and risk of hypertension: A systematic review and dose-response meta-analysis
Verzelloni, P., Giuliano, V., Wise, L. A., Urbano, T., Baraldi, C., Vinceti, M., & Filippini, T. (2024) · Environmental Research · doi:10.1016/j.envres.2024.120014 · Cited by 25
Self-citation
Effects of diabetes/hyperglycemia on peri‐implant biomarkers and clinical and radiographic outcomes in patients with dental implant restorations: A systematic review and meta‐analysis
Lv, X., Zou, L., Zhang, X., Zhang, X., Lai, H., & Shi, J. (2022) · Clinical Oral Implants Research · doi:10.1111/clr.14010 · Cited by 25
NUQUEST—NUtrition QUality Evaluation Strengthening Tools: development of tools for the evaluation of risk of bias in nutrition studies
Kelly, S., Greene-Finestone, L. S., Yetley, E. A., Benkhedda, K., Brooks, S. P. J., Wells, G. A., & MacFarlane, A. J. (2021) · American Journal of Clinical Nutrition · doi:10.1093/ajcn/nqab335 · Cited by 26
Assessing risk of bias in toxicological studies in the era of artificial intelligence
Härtung, T., Hoffmann, S., & Whaley, P. (2025) · Archives of Toxicology · doi:10.1007/s00204-025-03978-5 · Cited by 28
Self-citation
Residential exposure to electromagnetic fields and risk of amyotrophic lateral sclerosis: a dose–response meta-analysis
Filippini, T., Hatch, E. E., & Vinceti, M. (2021) · Scientific Reports · doi:10.1038/s41598-021-91349-2 · Cited by 26
Self-citation
Risk of Subsequent Stroke Among Patients Receiving Outpatient vs Inpatient Care for Transient Ischemic Attack
Shahjouei, S., Li, J., Koza, E., Abedi, V., Sadr, A. V., Chen, Q., Mowla, A., Griffin, P., Ranta, A. A., & Zand, R. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.36644 · Cited by 23
Circulating chemerin levels in metabolic-associated fatty liver disease: a systematic review and meta-analysis
Ren, Q., Wang, H., Zeng, Y., Xia, F., Wang, M., Li, D., Huang, W., & Xu, Y. (2022) · Lipids in Health and Disease · doi:10.1186/s12944-022-01637-7 · Cited by 23
Malocclusion characteristics amongst individuals with autism spectrum disorder: a systematic review and meta-analysis
da Motta, T. P., Owens, J., Abreu, L. G., Debossan, S. A. T., Vargas‐Ferreira, F., & Vettore, M. V. (2022) · BMC Oral Health · doi:10.1186/s12903-022-02366-0 · Cited by 22
GRADE guidelines for environmental and occupational health: A new series of articles in Environment International
Morgan, R. L., Beverly, B., Ghersi, D., Schünemann, H. J., Rooney, A. A., Whaley, P., Zhu, Y., & Thayer, K. A. (2019) · Environment International · doi:10.1016/j.envint.2019.04.016 · Cited by 22
Self-citation
Inter-rater reliability and validity of risk of bias instrument for non-randomized studies of exposures: a study protocol
Jeyaraman, M. M., Al‐Yousif, N., Robson, R. C., Copstein, L., Balijepalli, C., Hofer, K., Fazeli, M. S., Ansari, M., Tricco, A. C., Rabbani, R., et al. (2020) · Systematic Reviews · doi:10.1186/s13643-020-01291-z · Cited by 28
Self-citation
Preclinical assessment for translation to humans: The PATH approach for assessing supporting evidence for early-phase trials and innovative care
Kimmelman, J., Kane, P. B., Bicer, S., & Carlisle, B. G. (2024) · Med · doi:10.1016/j.medj.2024.07.014 · Cited by 31
Development of outcome-specific criteria for study evaluation in systematic reviews of epidemiology studies
Radke, E. G., Glenn, B. S., Galizia, A., Persad, A. S., Nachman, R. M., Bateson, T. F., Wright, J. M., Navas‐Acién, A., Arroyave, W. D., Puett, R. C., et al. (2019) · Environment International · doi:10.1016/j.envint.2019.05.078 · Cited by 22
Evaluating reliability and risk of bias of in vivo animal data for risk assessment of chemicals – Exploring the use of the SciRAP tool in a systematic review context
Waspe, J., Bui, T. T., Dishaw, L. V., Kraft, A. D., Luke, A., & Beronius, A. (2020) · Environment International · doi:10.1016/j.envint.2020.106103 · Cited by 21
Dietary intake, biomarkers and supplementation of fatty acids and risk of coronary events: a systematic review and dose-response meta-analysis of randomized controlled trials and prospective observational studies
Jayedi, A., Soltani, S., Emadi, A., Ghods, K., & Shab‐Bidar, S. (2023) · Critical Reviews in Food Science and Nutrition · doi:10.1080/10408398.2023.2251583 · Cited by 23
Maternal smoking during pregnancy and offspring body mass index and overweight: a systematic review and meta-analysis
Magalhães, E. I. D. S., Sousa, B., Lima, N. P., & Horta, B. L. (2019) · Cadernos de Saúde Pública · doi:10.1590/0102-311x00176118 · Cited by 21
Linking Physical Activity to Breast Cancer Risk via the Insulin/Insulin-like Growth Factor Signaling System, Part 2: The Effect of Insulin/Insulin-like Growth Factor Signaling on Breast Cancer Risk
Drummond, A. E., Swain, C. T. V., Milne, R. L., English, D. R., Brown, K. A., Skinner, T. L., Lay, J., van Roekel, E. H., Moore, M. M., Gaunt, T. R., et al. (2022) · Cancer Epidemiology Biomarkers & Prevention · doi:10.1158/1055-9965.epi-22-0505 · Cited by 19
Impact of Noise Exposure on Risk of Developing Stress-Related Metabolic Effects
Sivakumaran, K., Ritonja, J. A., Waseem, H., AlShenaiber, L., Morgan, E., Ahmadi, S. A., Denning, A., Michaud, D. S., & Morgan, R. L. (2022) · Noise and Health · doi:10.4103/nah.nah_21_22 · Cited by 24
Self-citation
ROBVALU: a tool for assessing risk of bias in studies about people’s values, utilities, or importance of health outcomes
Karam, S. G., Zhang, Y., Hernandez, H. P., Siebert, U., Koopman, L., Noyes, J., Tarride, J., Stevens, A., Welch, V. A., Saz‐Parkinson, Z., et al. (2024) · BMJ · doi:10.1136/bmj-2024-079890 · Cited by 19
Self-citation
Impaired lung function and associated risk factors in children born prematurely: a systematic review and meta-analysis
van Boven, M. R., Hutten, G. J., Richardson, R., Königs, M., Leemhuis, A. G., Onland, W., Terheggen‐Lagro, S. W. J., Oosterlaan, J., & van Kaam, A. H. (2024) · European Respiratory Review · doi:10.1183/16000617.0114-2024 · Cited by 19
The effects of light in children: A systematic review
Westwood, E., Smith, S. S., Mann, D., Pattinson, C. L., Allan, A. C., & Staton, S. (2023) · Journal of Environmental Psychology · doi:10.1016/j.jenvp.2023.102062 · Cited by 26

Authors: Morgan, R. L., Thayer, K. A., Santesso, N., Holloway, A. C., Blain, R., Eftim, S. E., Goldstone, A. E., Ross, P., Ansari, M., Akl, E. A., Filippini, T., Hansell, A., Meerpohl, J. J., Mustafa, R. A., Verbeek, J., Vinceti, M., Whaley, P., & Schünemann, H. J.
Affiliations: McMaster University (CA)
Environmental Protection Agency (US)
ICF International (United States) (US)
University of Ottawa (CA)
University of Modena and Reggio Emilia (IT)
Imperial College Healthcare NHS Trust (GB)
University of Freiburg (DE)
University of Kansas Medical Center (US)
Finnish Institute of Occupational Health (FI)
Boston University (US)
Lancaster University (GB)
Corresponding author: Schünemann, Holger J.
ORCID: Morgan, Rebecca L.
Thayer, Kristina A.
Santesso, Nancy
Holloway, Alison C.
Blain, Robyn
Eftim, Sorina E.
Akl, Elie A
Filippini, Tommaso
Hansell, Anna
Meerpohl, Joerg J.
Mustafa, Reem A.
Verbeek, Jos
Vinceti, Marco
Whaley, Paul
Schünemann, Holger J.
Venue: Environment International
Publisher: Elsevier BV
Published: 2019-01
Volume: 122
Pages: 168-184
ISSN: 0160-4120 (ISSN-L)
DOI: 10.1016/j.envint.2018.11.004
PMC: PMC8221004
OpenAlex: W2901788566
Cited by: 248
References: 33
Citations by year: 2026: 20 · 2025: 34 · 2024: 31 · 2023: 45 · 2022: 45 · 2021: 41 · 2020: 25 · 2019: 7
FWCI: 13.92
Type: journal article
Language: en
Open access: hybrid
doi.org
Licence: cc-by-nc-nd
www.elsevier.com/tdm/userlicense/1.0/ (tdm)
creativecommons.org/licenses/by-nc-nd/4.0/ (vor)
Locations: Environment International (publishedVersion, cc-by-nc-nd)
PubMed (publishedVersion)
Lancaster EPrints (Lancaster University) (acceptedVersion)
PubMed Central (acceptedVersion, cc-by-nc-nd)
IRIS UNIMORE (University of Modena and Reggio Emilia) (submittedVersion, cc-by-nc-nd)
DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Figshare (submittedVersion, cc-by)
Leicester Research Archive (University of Leicester) (submittedVersion, other-oa)
Funders: Intramural Research Program of the National Institute of Environmental Health Sciences
MacGRADE Centre at the McMaster University
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
Keywords: risk of bias, non-randomized studies, environmental exposure, ROBINS-I, GRADE, quality of evidence, indirectness, sensitivity analysis, systematic reviews
MeSH terms: Environmental Exposure, Humans, Random Allocation, Research Design, Bias, Risk Assessment

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

A risk of bias instrument for non-randomized studies of exposures: A users' guide to its application in the context of GRADE

Abstract

Overview

Integrity signals

Figures

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks