A risk of bias instrument for non-randomized studies of exposures: A users' guide to its application in the context of GRADE
Open accessAbstract
The objective of this paper is to explain how to apply, interpret, and present the results of a new instrument to assess the risk of bias (RoB) in non-randomized studies (NRS) dealing with effects of environmental exposures on health outcomes. This instrument is modeled on the Risk Of Bias In Non-randomized Studies of Interventions (ROBINS-I) instrument. The RoB instrument for NRS of exposures assesses RoB along a standardized comparison to a randomized target experiment, instead of the study-design directed RoB approach. We provide specific guidance for the integral steps of developing a research question and target experiment, distinguishing issues of indirectness from RoB, making individual-study judgments, and performing and interpreting sensitivity analyses for RoB judgments across a body of evidence. Also, we present an approach for integrating the RoB assessments within the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) framework to assess the certainty of the evidence in the systematic review. Finally, we guide the reader through an overall assessment to support the rating of all domains that determine the certainty of a body of evidence using the GRADE approach.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The risk of bias of included nonrandomized comparative studies was assessed using the ROBINS-E tool.
“The Risk of Bias in Non-randomized Studies—of Exposures (ROBINS-E) tool was used for nonrandomized comparative studies [19, 20].”
From Evidence for Human-to-Human Transmission of Hantavirus: A Systematic Review (2022) · cites this paper as [19]
Early work by a subset of the authors considered the extent to which the ROBINS-I tool was suitable for observational studies of exposures and suggested modifications to the tool.
“Early work by a subset of the authors considered the extent to which the ROBINS-I tool was suitable for observational studies of exposures and suggested modifications to the tool (Morgan et al., 2018; Morgan et al., 2019).”
From A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) (2024) · cites this paper as Morgan et al., 2019
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164 · see citing works
ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329 · see citing works
Self-citationGRADE guidelines: 3. Rating the quality of evidence
Balshem, H., Helfand, M., Schünemann, H. J., Oxman, A. D., Kunz, R., Brozek, J., Vist, G. E., Falck-Ytter, Y., Meerpohl, J., & Norris, S. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.015 · Cited by 8,285 · see citing works
Self-citationUsing Big Data to Emulate a Target Trial When a Randomized Trial Is Not Available: Table 1.
Hernán, M. A., & Robins, J. M. (2016) · American Journal of Epidemiology · doi:10.1093/aje/kwv254 · Cited by 3,115 · see citing works
GRADE guidelines: 4. Rating the quality of evidence—study limitations (risk of bias)
Guyatt, G. H., Oxman, A. D., Vist, G., Kunz, R., Brozek, J., Alonso-Coello, P., Montori, V., Akl, E. A., Djulbegovic, B., & Falck-Ytter, Y. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.017 · Cited by 2,798 · see citing works
Self-citationSmoking and Carcinoma of the Lung
Doll, R., & Hill, A. B. (1950) · BMJ · doi:10.1136/bmj.2.4682.739 · Cited by 1,999 · see citing works
GRADE guidelines: 8. Rating the quality of evidence—indirectness
Guyatt, G. H., Oxman, A. D., Kunz, R., Woodcock, J., Brozek, J., Helfand, M., Alonso-Coello, P., Falck-Ytter, Y., Jaeschke, R., Vist, G., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.04.014 · Cited by 1,843 · see citing works
Self-citationMorgan, R. L., Whaley, P., Thayer, K. A., & Schünemann, H. J. (2018) · Environment International · doi:10.1016/j.envint.2018.07.015 · Cited by 1,365 · see citing works
Self-citationGRADE guidelines: 9. Rating up the quality of evidence
Guyatt, G. H., Oxman, A. D., Sultan, S., Glasziou, P., Akl, E. A., Alonso-Coello, P., Atkins, D., Kunz, R., Brozek, J., Montori, V., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.06.004 · Cited by 1,336 · see citing works
Self-citation(2016) · Cochrane Database of Systematic Reviews · Cited by 985 · see citing works
Schünemann, H. J., Cuello, C., Akl, E. A., Mustafa, R. A., Meerpohl, J. J., Thayer, K., Morgan, R. L., Gartlehner, G., Kunz, R., Katikireddi, S. V., et al. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.01.012 · Cited by 911 · see citing works
Self-citationMortality in Relation to Smoking: Ten Years' Observations of British Doctors
Doll, R., & Hill, A. B. (1964) · BMJ · doi:10.1136/bmj.1.5396.1460 · Cited by 683 · see citing works
The Planning of Observational Studies of Human Populations
Cochran, W. G., & Chambers, S. (1965) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2344179 · Cited by 572 · see citing works
Mortality in Relation to Smoking: Ten Years' Observations of British Doctors
Doll, R., & Hill, A. (1964) · BMJ · doi:10.1136/bmj.1.5395.1399 · Cited by 545 · see citing works
Woodruff, T. J., & Sutton, P. (2014) · Environmental Health Perspectives · doi:10.1289/ehp.1307175 · Cited by 427 · see citing works
Proper interpretation of non-differential misclassification effects: expectations vs observations
Jurek, A. M., Greenland, S., Maldonado, G., & Church, T. R. (2005) · International Journal of Epidemiology · doi:10.1093/ije/dyi060 · Cited by 413 · see citing works
Urinary bisphenol A and obesity: NHANES 2003–2006
Carwile, J. L., & Michels, K. B. (2011) · Environmental Research · doi:10.1016/j.envres.2011.05.014 · Cited by 372 · see citing works
Pharmacokinetics of bisphenol A in humans following a single oral administration
Thayer, K. A., Doerge, D. R., Hunt, D., Schurman, S. H., Twaddle, N. C., Churchwell, M. I., Garantziotis, S., Kissling, G. E., Easterling, M. R., Bucher, J. R., et al. (2015) · Environment International · doi:10.1016/j.envint.2015.06.008 · Cited by 328 · see citing works
Self-citationRancière, F., Lyons, J. G., Loh, V. H. Y., Botton, J., Galloway, T. S., Wang, T., Shaw, J. E., & Magliano, D. J. (2015) · Environmental Health · doi:10.1186/s12940-015-0036-5 · Cited by 297 · see citing works
GRADE: Assessing the quality of evidence in environmental and occupational health
Morgan, R. L., Thayer, K. A., Bero, L. A., Bruce, N., Falck-Ytter, Y. T., Ghersi, D., Guyatt, G. H., Hooijmans, C. R., Langendam, M. W., Mandrioli, D., et al. (2016) · Environment International · doi:10.1016/j.envint.2016.01.004 · Cited by 280 · see citing works
Self-citationPrenatal and Postnatal Bisphenol A Exposure and Body Mass Index in Childhood in the CHAMACOS Cohort
Harley, K. G., Schall, R. A., Chevrier, J., Tyler, K., Aguirre, H., Bradman, A., Holland, N., Lustig, R. H., Calafat, A. M., & Eskenazi, B. (2013) · Environmental Health Perspectives · doi:10.1289/ehp.1205548 · Cited by 258 · see citing works
Blair, A., Stewart, P., Lubin, J. H., & Forastiere, F. (2007) · American Journal of Industrial Medicine · doi:10.1002/ajim.20281 · Cited by 251 · see citing works
Human exposures to bisphenol A: mismatches between data and assumptions
Vandenberg, L. N., Hunt, P. A., Myers, J. P., & Saal, F. S. V. (2013) · Reviews on Environmental Health · doi:10.1515/reveh-2012-0034 · Cited by 212 · see citing works
Spurious effects from an extraneous variable
Bross, I. D. J. (1966) · Journal of Chronic Diseases · doi:10.1016/0021-9681(66)90062-2 · Cited by 184 · see citing works
Urine Bisphenol-A Level in Relation to Obesity and Overweight in School-Age Children
Li, D., Miao, M., Zhou, Z., Wu, C., Shi, H., Liu, X., Wang, S., & Yuan, W. (2013) · PLoS ONE · doi:10.1371/journal.pone.0065399 · Cited by 159 · see citing works
Wang, H., Zhou, Y., Tang, C., Wu, J., Chen, Y., & Jiang, Q. (2012) · Environmental Health · doi:10.1186/1476-069x-11-79 · Cited by 138 · see citing works
Morgan, R. L., Thayer, K. A., Santesso, N., Holloway, A. C., Blain, R., Eftim, S. E., Goldstone, A. E., Ross, P., Guyatt, G., & Schünemann, H. J. (2018) · Environment International · doi:10.1016/j.envint.2018.08.018 · Cited by 106 · see citing works
Self-citationRooney, A. A., Cooper, G. S., Jahnke, G. D., Lam, J., Morgan, R. L., Boyles, A. L., Ratcliffe, J. M., Kraft, A. D., Schünemann, H. J., Schwingl, P. J., et al. (2016) · Environment International · doi:10.1016/j.envint.2016.01.005 · Cited by 95 · see citing works
Self-citationExposure Assessment in Environmental Epidemiology
(2015) · Oxford University Press eBooks · doi:10.1093/med/9780199378784.001.0001 · Cited by 89 · see citing works
Topics in Environmental Epidemiology
Steenland, K., & Savitz, D. A. (1998) · Epidemiology · doi:10.1097/00001648-199803000-00021 · Cited by 53 · see citing works
Cox, K. J., Porucznik, C. A., Anderson, D. J., Brozek, E., Szczotka, K. M., Bailey, N. M., Wilkins, D. G., & Stanford, J. B. (2015) · Environmental Health Perspectives · doi:10.1289/ehp.1509752 · Cited by 40 · see citing works
Kogevinas, M. (2011) · Environmental Health · doi:10.1186/1476-069x-10-s1-s3 · Cited by 35 · see citing works
Topics in Environmental Epidemiology
(1997) · doi:10.1093/oso/9780195095647.001.0001 · Cited by 29 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- Thayer K, Rooney A, Boyles A, Holmgren S, Walker V, Kissling G, National Toxicology Program, US Department of Health and Human Services, 2013. Draft protocol for systematic review to evaluate the evidence for an association between bisphenol A (BPA) exposure and obesity. Natl. Toxicol. Program.
Cited by
Sorted by citations
A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E)
Higgins, J. P. T., Morgan, R. L., Rooney, A. A., Taylor, K. W., Thayer, K. A., Silva, R. A., Lemeris, C., Akl, E. A., Bateson, T. F., Berkman, N. D., et al. (2024) · Environment International · doi:10.1016/j.envint.2024.108602 · Cited by 1,022 · see citing works
“Early work by a subset of the authors considered the extent to which the ROBINS-I tool was suitable for observational studies of exposures and suggested modifications to the tool (Morgan et al., 2018; Morgan et al., 2019).”
Self-citationOrellano, P. W., Reynoso, J. I., QUARANTA, N. E., Bardach, A., & Ciapponi, A. (2020) · Environment International · doi:10.1016/j.envint.2020.105876 · Cited by 801 · see citing works
Huangfu, P., & Atkinson, R. W. (2020) · Environment International · doi:10.1016/j.envint.2020.105998 · Cited by 535 · see citing works
Li, L. Z., Yang, P., Singer, S. J., Pfeffer, J., Mathur, M. B., & Shanafelt, T. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.43059 · Cited by 501 · see citing works
Air pollution exposure and depression: A comprehensive updated systematic review and meta-analysis
Borroni, E., Pesatori, A. C., Bollati, V., Buoli, M., & Carugno, M. (2021) · Environmental Pollution · doi:10.1016/j.envpol.2021.118245 · Cited by 272 · see citing works
Orellano, P. W., Reynoso, J. I., & QUARANTA, N. E. (2021) · Environment International · doi:10.1016/j.envint.2021.106434 · Cited by 261 · see citing works
Ambient air pollution and depression: A systematic review with meta-analysis up to 2019
Fan, S., Heinrich, J., Bloom, M. S., Zhao, T., Shi, T., Feng, W., Sun, Y., Shen, J., Yang, Z., Yang, B., et al. (2019) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2019.134721 · Cited by 256 · see citing works
Long-term exposure to nitrogen dioxide and mortality: A systematic review and meta-analysis
Huang, S., Li, H., Wang, M., Qian, Y., Steenland, K., Caudle, W. M., Liu, Y., Sarnat, J. A., Papatheodorou, S., & Shi, L. (2021) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2021.145968 · Cited by 217 · see citing works
Luo, Y., Huang, W., Liu, X., Markevych, I., Bloom, M. S., Zhao, T., Heinrich, J., Yang, B., & Dong, G. (2020) · Obesity Reviews · doi:10.1111/obr.13078 · Cited by 171 · see citing works
Frailty and Physical Fitness in Elderly People: A Systematic Review and Meta-analysis
Navarrete-Villanueva, D., Gómez-Cabello, A., Marín-Puyalto, J., Moreno, L., Vicente-Rodríguez, G., & Casajús, J. (2020) · Sports Medicine · doi:10.1007/s40279-020-01361-1 · Cited by 134 · see citing works
Filippini, T., Wise, L. A., & Vinceti, M. (2021) · Environment International · doi:10.1016/j.envint.2021.106920 · Cited by 145 · see citing works
Self-citationShekaftik, S. O., & Nasirzadeh, N. (2021) · Nanotoxicology · doi:10.1080/17435390.2021.1936254 · Cited by 144 · see citing works
Ludwig‐Walz, H., Dannheim, I., Pfadenhauer, L. M., Fegert, J. M., & Bujard, M. (2022) · Child and Adolescent Psychiatry and Mental Health · doi:10.1186/s13034-022-00546-y · Cited by 140 · see citing works
Ervin, J., Taouk, Y., Alfonzo, L. F., Hewitt, B. A., & King, T. L. (2022) · The Lancet Public Health · doi:10.1016/s2468-2667(22)00160-8 · Cited by 130 · see citing works
Verzelloni, P., Urbano, T., Wise, L. A., Vinceti, M., & Filippini, T. (2024) · Environmental Pollution · doi:10.1016/j.envpol.2024.123462 · Cited by 104 · see citing works
Self-citationThe impact of urinary incontinence on falls: A systematic review and meta-analysis
Moon, S., Chung, H. S., Kim, Y. J., Kim, S. J., Kwon, O., Lee, Y. G., Yu, J. M., & Cho, S. T. (2021) · PLoS ONE · doi:10.1371/journal.pone.0251711 · Cited by 103 · see citing works
Update of the WHO global air quality guidelines: Systematic reviews – An introduction
Velasco, R. P., & Jarosińska, D. (2022) · Environment International · doi:10.1016/j.envint.2022.107556 · Cited by 106 · see citing works
Cecchini, M., Filippini, T., Whelton, P. K., Iamandii, I., Federico, S. D., Boriani, G., & Vinceti, M. (2024) · Hypertension · doi:10.1161/hypertensionaha.124.22703 · Cited by 101 · see citing works
Self-citationFrampton, G. K., Whaley, P., Bennett, M. G., Bilotta, G. S., Dorne, J. L. C. M., Eales, J. F., James, K. L., Kohl, C., Land, M., Livoreil, B., et al. (2022) · Environmental Evidence · doi:10.1186/s13750-022-00264-0 · Cited by 103 · see citing works
Self-citationFederico, S. D., Filippini, T., Whelton, P. K., Cecchini, M., Iamandii, I., Boriani, G., & Vinceti, M. (2023) · Hypertension · doi:10.1161/hypertensionaha.123.21224 · Cited by 100 · see citing works
Self-citationFluoride exposure and cognitive neurodevelopment: Systematic review and dose-response meta-analysis
Veneri, F., Vinceti, M., Generali, L., Giannone, M. E., Mazzoleni, E., Birnbaum, L. S., Consolo, U., & Filippini, T. (2023) · Environmental Research · doi:10.1016/j.envres.2023.115239 · Cited by 97 · see citing works
Self-citationCrawford, L., Halperin, S. A., Dzierlenga, M. W., Skidmore, B., Linakis, M. W., Nakagawa, S., & Longnecker, M. P. (2023) · Environment International · doi:10.1016/j.envint.2023.107734 · Cited by 92 · see citing works
Filippini, T., Malavolti, M., Whelton, P. K., & Vinceti, M. (2022) · Current Hypertension Reports · doi:10.1007/s11906-022-01182-9 · Cited by 96 · see citing works
Self-citationOrellano, P. W., Kasdagli, M., Velasco, R. P., & Samoli, E. (2024) · International Journal of Public Health · doi:10.3389/ijph.2024.1607683 · Cited by 83 · see citing works
Shields, M., Dimov, S., Kavanagh, A. M., Milner, A. J., Spittal, M. J., & King, T. L. (2021) · Social Psychiatry and Psychiatric Epidemiology · doi:10.1007/s00127-021-02077-x · Cited by 85 · see citing works
Association between greenspace and blood pressure: A systematic review and meta-analysis
Zhao, Y., Bao, W.-W., Yang, B., Liang, J., Gui, Z., Huang, S., Chen, Y.-C., Dong, G., & Chen, Y.-J. (2022) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2021.152513 · Cited by 80 · see citing works
Lee, K. K., Spath, N., Miller, M. R., Mills, N. L., & Shah, A. S. V. (2020) · Environment International · doi:10.1016/j.envint.2020.105901 · Cited by 80 · see citing works
Boon, M. H., Thomson, H., Shaw, B., Akl, E. A., Lhachimi, S. K., López‐Alcalde, J., Klugar, M., Choi, L., Saz‐Parkinson, Z., Mustafa, R. A., et al. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.01.001 · Cited by 81 · see citing works
Self-citationLight at night and risk of breast cancer: a systematic review and dose–response meta-analysis
Urbano, T., Vinceti, M., Wise, L. A., & Filippini, T. (2021) · International Journal of Health Geographics · doi:10.1186/s12942-021-00297-7 · Cited by 77 · see citing works
Self-citationCohen, J. A., Trojano, M., Mowry, E. M., Uitdehaag, B. M. J., Reingold, S. C., & Marrie, R. A. (2019) · Multiple Sclerosis Journal · doi:10.1177/1352458519892555 · Cited by 77 · see citing works
Steenland, K., Schubauer‐Berigan, M. K., Vermeulen, R. C. H., Lunn, R. M., Straif, K., Zahm, S. H., Stewart, P. A., Arroyave, W. D., Mehta, S., & Pearce, N. E. (2020) · Environmental Health Perspectives · doi:10.1289/ehp6980 · Cited by 75 · see citing works
Allergen immunotherapy: The growing role of observational and randomized trial “Real‐World Evidence”
Paoletti, G., Bona, D. D., Chu, D. K., Firinu, D., Heffler, E. M., Agache, I. O., Jutel, M., Klimek, L., Pfaar, O., Mösges, R., et al. (2021) · Allergy · doi:10.1111/all.14773 · Cited by 75 · see citing works
Assessing the certainty of the evidence in systematic reviews: importance, process, and use
Brignardello‐Petersen, R., & Guyatt, G. H. (2024) · American Journal of Epidemiology · doi:10.1093/aje/kwae332 · Cited by 91 · see citing works
Schwingshackl, L., Schünemann, H. J., & Meerpohl, J. J. (2020) · European Journal of Nutrition · doi:10.1007/s00394-020-02464-1 · Cited by 73 · see citing works
Self-citationEvidence for Human-to-Human Transmission of Hantavirus: A Systematic Review
Toledo, J., Haby, M. M., Reveiz, L., Sosa Leon, L., Angerami, R., & Aldighieri, S. (2022) · The Journal of Infectious Diseases · doi:10.1093/infdis/jiab461 · Cited by 78 · see citing works
Mamikutty, R., Aly, A. S., & Marhazlinda, J. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18168623 · Cited by 68 · see citing works
Cadmium exposure and risk of breast cancer: A meta-analysis
Flórez‐García, V., Guevara-Romero, E., Hawkins, M. M., Bautista, L., Jenson, T. E., Yu, J. S., & Kalkbrenner, A. (2022) · Environmental Research · doi:10.1016/j.envres.2022.115109 · Cited by 64 · see citing works
Kosti, R. I., Kasdagli, M., Kyrozis, A., Orsini, N., Λάγιου, Π., Taiganidou, F., & Naska, A. (2021) · Nutrition Reviews · doi:10.1093/nutrit/nuab078 · Cited by 63 · see citing works
Alcohol, Intraocular Pressure, and Open-Angle Glaucoma
Stuart, K. V., Madjedi, K. M., Luben, R., Chua, S. Y. L., Warwick, A., Chia, M. A., Pasquale, L. R., Wiggs, J. L., Kang, J. H., Hysi, P. G., et al. (2022) · Ophthalmology · doi:10.1016/j.ophtha.2022.01.023 · Cited by 63 · see citing works
Mental health of young informal carers: a systematic review
Alfonzo, L. F., Singh, A., Disney, G., Ervin, J., & King, T. L. (2022) · Social Psychiatry and Psychiatric Epidemiology · doi:10.1007/s00127-022-02333-8 · Cited by 63 · see citing works
Fluoride Exposure and Skeletal Fluorosis: a Systematic Review and Dose-response Meta-analysis
Veneri, F., Iamandii, I., Vinceti, M., Birnbaum, L. S., Generali, L., Consolo, U., & Filippini, T. (2023) · Current Environmental Health Reports · doi:10.1007/s40572-023-00412-9 · Cited by 62 · see citing works
Self-citationGuo, X., Su, W., Li, N., Song, Q., Wang, H., Liang, Q., Li, Y., Lowe, S., Bentley, R., Zhou, Z., et al. (2022) · Environmental Science and Pollution Research · doi:10.1007/s11356-022-22353-w · Cited by 58 · see citing works
Song, X., Hu, Y., Ma, Y., Jiang, L., Wang, X., Shi, A., Zhao, J., Liu, Y., Liu, Y., Tang, J. Y., et al. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-049516 · Cited by 59 · see citing works
Pega, F., Norris, S., Backes, C., Bero, L., Descatha, A., Gagliardi, D., Godderis, L., Loney, T., Modenese, A., Morgan, R., et al. (2019) · Environment International · doi:10.1016/j.envint.2019.105039 · Cited by 44 · see citing works
Self-citationHedman, L., Galanti, M. R., Ryk, C., Gilljam, H., & Adermark, L. (2021) · Tobacco Prevention & Cessation · doi:10.18332/tpc/142320 · Cited by 58 · see citing works
Hernández-Mariano, J. Á., Baltazar-Reyes, M. C., Salazar-Martínez, E., & Cupul‐Uicab, L. A. (2021) · International Journal of Hygiene and Environmental Health · doi:10.1016/j.ijheh.2021.113865 · Cited by 56 · see citing works
Maturation of GFR in Term-Born Neonates: An Individual Participant Data Meta-Analysis
Smeets, N. J. L., IntHout, J., van der Burgh, M. J. P., Schwartz, G. J., Schreuder, M. F., & de Wildt, S. N. (2022) · Journal of the American Society of Nephrology · doi:10.1681/asn.2021101326 · Cited by 54 · see citing works
Atrial Fibrillation and the Risk of Early‐Onset Dementia: A Systematic Review and Meta‐Analysis
Giannone, M. E., Filippini, T., Whelton, P. K., Chiari, A., Vitolo, M., Boriani, G., & Vinceti, M. (2022) · Journal of the American Heart Association · doi:10.1161/jaha.122.025653 · Cited by 55 · see citing works
Self-citationFamiliari, A., Khalil, A. A., Rizzo, G., Odibo, A. O., Vergani, P., Buca, D. I. P., Hidaka, N., Mascio, D. D., Nwabuobi, C. K., Simeone, S., et al. (2020) · European Journal of Obstetrics & Gynecology and Reproductive Biology · doi:10.1016/j.ejogrb.2020.07.020 · Cited by 54 · see citing works
Is Greenness Associated with Dementia? A Systematic Review and Dose–Response Meta-analysis
Zagnoli, F., Filippini, T., Jimenez, M. P., Wise, L. A., Hatch, E. E., & Vinceti, M. (2022) · Current Environmental Health Reports · doi:10.1007/s40572-022-00365-5 · Cited by 53 · see citing works
Self-citationThe risk of tooth loss in patients with diabetes: A systematic review and meta‐analysis
Weijdijk, L. P. M., Žiūkaitė, L., van der Weijden, G. A., Bakker, E. W. P., & Slot, D. E. (2021) · International Journal of Dental Hygiene · doi:10.1111/idh.12512 · Cited by 51 · see citing works
Ervin, J., Taouk, Y., Alfonzo, L. F., Peasgood, T., & King, T. L. (2022) · EClinicalMedicine · doi:10.1016/j.eclinm.2022.101711 · Cited by 53 · see citing works
Long-term NO2 exposure and mortality: A comprehensive meta-analysis
Chen, X., Qi, L., Li, S., & Duan, X. (2023) · Environmental Pollution · doi:10.1016/j.envpol.2023.122971 · Cited by 49 · see citing works
Monteiro, C., Maricoto, T., Prazeres, F., Simões, P. A., & Simões, J. A. (2021) · Respiratory Medicine · doi:10.1016/j.rmed.2021.106724 · Cited by 53 · see citing works
Swain, C. T. V., Drummond, A. E., Boing, L., Milne, R. L., English, D. R., Brown, K. A., van Roekel, E. H., Dixon‐Suen, S. C., Lynch, M. J., Moore, M. M., et al. (2021) · Cancer Epidemiology Biomarkers & Prevention · doi:10.1158/1055-9965.epi-21-0437 · Cited by 46 · see citing works
Adermark, L., Galanti, M. R., Ryk, C., Gilljam, H., & Hedman, L. (2021) · ERJ Open Research · doi:10.1183/23120541.00976-2020 · Cited by 45 · see citing works
Electronic cigarettes and subsequent cigarette smoking in young people: A systematic review
Begh, R. A., Conde, M., Fanshawe, T. R., Kneale, D., Shahab, L., Zhu, S., Pesko, M. F., Livingstone‐Banks, J., Lindson, N., Rigotti, N. A., et al. (2025) · Addiction · doi:10.1111/add.16773 · Cited by 46 · see citing works
Drummond, A. E., Swain, C. T. V., Brown, K. A., Dixon‐Suen, S. C., Boing, L., van Roekel, E. H., Moore, M. M., Gaunt, T. R., Milne, R. L., English, D. R., et al. (2021) · Cancer Epidemiology Biomarkers & Prevention · doi:10.1158/1055-9965.epi-21-0438 · Cited by 42 · see citing works
Sivakumaran, K., Ritonja, J. A., Waseem, H., AlShenaibar, L., Morgan, E., Ahmadi, S. A., Denning, A., Michaud, D. S., & Morgan, R. L. (2022) · Noise and Health · doi:10.4103/nah.nah_83_21 · Cited by 39 · see citing works
Self-citationGonzález-Luis, G. E., van Westering-Kroon, E., Villamor-Martínez, E., Huizing, M. J., Kilani, M. A., Kramer, B. W. W., & Villamor, E. (2020) · Frontiers in Pediatrics · doi:10.3389/fped.2020.00160 · Cited by 40 · see citing works
Liu, T., & Lipowski, M. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18062989 · Cited by 38 · see citing works
Wang, Y., Liu, K., Long, T., Long, J., Li, Y., Li, J., & Cheng, L. (2022) · Critical Reviews in Food Science and Nutrition · doi:10.1080/10408398.2022.2029826 · Cited by 37 · see citing works
Arroyave, W., Mehta, S., Guha, N., Schwingl, P., Taylor, K. W., Glenn, B. S., Radke, E., Vilahur, N., Carreón, T., Nachman, R., et al. (2020) · Journal of Exposure Science and Environmental Epidemiology · doi:10.1038/s41370-020-0228-0 · Cited by 31 · see citing works
Odo, D. B., Yang, I. A., & Knibbs, L. D. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18041411 · Cited by 41 · see citing works
Yu, Z., Zhang, X., Zhang, J. J., Yang, F., Zhang, H., Wan, Z., Xiao, C., Zhang, H., Wang, Q., & Huang, C. (2022) · Environmental Research · doi:10.1016/j.envres.2022.113381 · Cited by 36 · see citing works
Pasqual, E., de Basea, M. B., López-Vicente, M., Thierry-Chef, I., & Cardis, E. (2020) · Environment International · doi:10.1016/j.envint.2019.105371 · Cited by 28 · see citing works
Development of an Evidence-Based Risk Assessment Framework
Krewski, D. R. (2022) · ALTEX · doi:10.14573/altex.2004041 · Cited by 37 · see citing works
Murad, M. H., Verbeek, J. H. A. M., Schwingshackl, L., Filippini, T., Vinceti, M., Akl, E. A., Morgan, R. L., Mustafa, R. A., Zeraatkar, D., Senerth, E., et al. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.09.011 · Cited by 33 · see citing works
Self-citationRugel, E., & Brauer, M. (2020) · Environmental Research · doi:10.1016/j.envres.2020.109388 · Cited by 29 · see citing works
Jeyaraman, M. M., Rabbani, R., Copstein, L., Robson, R. C., Al‐Yousif, N., Pollock, M., Xia, J., Balijepalli, C., Hofer, K., Mansour, S., et al. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.09.033 · Cited by 30 · see citing works
Self-citationRapp, D., Michels, S., Schöpe, J., Schwingshackl, L., Tumani, H., & Şenel, M. (2021) · Multiple Sclerosis and Related Disorders · doi:10.1016/j.msard.2021.103279 · Cited by 30 · see citing works
Hennessy, S., Atsuta, Y., Hill, S., Rägo, L., & Juhaeri, J. (2025) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.70117 · Cited by 29 · see citing works
Houghton, J. S. M., Nickinson, A., Bridgwood, B. M., Nduwayo, S., Pepper, C., Rayt, H. S., Gray, L. J., Haunton, V. J., & Sayers, R. D. (2021) · European Journal of Vascular and Endovascular Surgery · doi:10.1016/j.ejvs.2020.12.016 · Cited by 29 · see citing works
Cocomello, L., Dimagli, A., Biglino, G., Cornish, R., Caputo, M., & Lawlor, D. A. (2021) · BMC Cardiovascular Disorders · doi:10.1186/s12872-021-02349-z · Cited by 29 · see citing works
Dietary glycemic index, glycemic load and cancer risk: a meta-analysis of prospective cohort studies
Long, T., Liu, K., Long, J., Li, J., & Cheng, L. (2022) · European Journal of Nutrition · doi:10.1007/s00394-022-02797-z · Cited by 28 · see citing works
Saucier, D., Registe, P. P. W., Bélanger, M. F., & O’Connell, C. M. (2023) · Frontiers in Neurology · doi:10.3389/fneur.2023.1108383 · Cited by 29 · see citing works
More, S., Bambidis, V., Benford, D., Bragard, C., Hernandez-Jerez, A., Bennekou, S. H., Koutsoumanis, K. P., Machera, K., Naegeli, H., Nielsen, S., et al. (2020) · EFSA journal. European Food Safety Authority · doi:10.2903/j.efsa.2020.6221 · Cited by 20 · see citing works
Self-citationLambourg, E. J., Colvin, L., Guthrie, G. D., Walker, H., & Bell, S. (2021) · British Journal of Anaesthesia · doi:10.1016/j.bja.2021.08.035 · Cited by 26 · see citing works
Effects of COVID-19 home confinement on sleep in children: A systematic review
Camacho-Montaño, L. R., Iranzo, Á., Martínez-Piédrola, R. M., Camacho-Montaño, L. M., Hoyas, E. H., Serrada-Tejeda, S., García‐Bravo, C., & Pérez‐de‐Heredia‐Torres, M. (2022) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2022.101596 · Cited by 26 · see citing works
Hogervorst, J. G. F., Virgolino, A., Halldorsson, T. I., Vinceti, M., Åkesson, A., Leander, K., Nawrot, T. S., Filippini, T., & Laguzzi, F. (2022) · Environmental Research · doi:10.1016/j.envres.2022.113705 · Cited by 26 · see citing works
Self-citationCadmium exposure and risk of hypertension: A systematic review and dose-response meta-analysis
Verzelloni, P., Giuliano, V., Wise, L. A., Urbano, T., Baraldi, C., Vinceti, M., & Filippini, T. (2024) · Environmental Research · doi:10.1016/j.envres.2024.120014 · Cited by 25 · see citing works
Self-citationLv, X., Zou, L., Zhang, X., Zhang, X., Lai, H., & Shi, J. (2022) · Clinical Oral Implants Research · doi:10.1111/clr.14010 · Cited by 25 · see citing works
Kelly, S., Greene-Finestone, L. S., Yetley, E. A. D., Benkhedda, K., Brooks, S. P. J., Wells, G. A., & MacFarlane, A. J. (2021) · American Journal of Clinical Nutrition · doi:10.1093/ajcn/nqab335 · Cited by 24 · see citing works
Assessing risk of bias in toxicological studies in the era of artificial intelligence
Härtung, T., Hoffmann, S., & Whaley, P. (2025) · Archives of Toxicology · doi:10.1007/s00204-025-03978-5 · Cited by 26 · see citing works
Self-citationFilippini, T., Hatch, E. E., & Vinceti, M. (2021) · Scientific Reports · doi:10.1038/s41598-021-91349-2 · Cited by 23 · see citing works
Self-citationShahjouei, S., Li, J., Koza, E., Abedi, V., Sadr, A. V., Chen, Q., Mowla, A., Griffin, P., Ranta, A. A., & Zand, R. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.36644 · Cited by 23 · see citing works
Ren, Q., Wang, H., Zeng, Y., Xia, F., Wang, M., Li, D., Huang, W., & Xu, Y. (2022) · Lipids in Health and Disease · doi:10.1186/s12944-022-01637-7 · Cited by 23 · see citing works
da Motta, T. P., Owens, J., Abreu, L. G., Debossan, S. A. T., Vargas‐Ferreira, F., & Vettore, M. V. (2022) · BMC Oral Health · doi:10.1186/s12903-022-02366-0 · Cited by 22 · see citing works
Morgan, R. L., Beverly, B., Ghersi, D., Schünemann, H. J., Rooney, A. A., Whaley, P., Zhu, Y., & Thayer, K. A. (2019) · Environment International · doi:10.1016/j.envint.2019.04.016 · Cited by 22 · see citing works
Self-citationJeyaraman, M. M., Al‐Yousif, N., Robson, R. C., Copstein, L., Balijepalli, C., Hofer, K., Fazeli, M. S., Ansari, M., Tricco, A. C., Rabbani, R., et al. (2020) · Systematic Reviews · doi:10.1186/s13643-020-01291-z · Cited by 22 · see citing works
Self-citationKimmelman, J., Kane, P. B., Bicer, S., & Carlisle, B. G. (2024) · Med · doi:10.1016/j.medj.2024.07.014 · Cited by 21 · see citing works
Radke, E. G., Glenn, B. S., Galizia, A., Persad, A. S., Nachman, R. M., Bateson, T. F., Wright, J. M., Navas‐Acién, A., Arroyave, W. D., Puett, R. C., et al. (2019) · Environment International · doi:10.1016/j.envint.2019.05.078 · Cited by 22 · see citing works
Waspe, J., Bui, T. T., Dishaw, L. V., Kraft, A. D., Luke, A., & Beronius, A. (2020) · Environment International · doi:10.1016/j.envint.2020.106103 · Cited by 21 · see citing works
Jayedi, A., Soltani, S., Emadi, A., Ghods, K., & Shab‐Bidar, S. (2023) · Critical Reviews in Food Science and Nutrition · doi:10.1080/10408398.2023.2251583 · Cited by 22 · see citing works
Magalhães, E. I. D. S., Sousa, B., Lima, N. P., & Horta, B. L. (2019) · Cadernos de Saúde Pública · doi:10.1590/0102-311x00176118 · Cited by 21 · see citing works
Drummond, A. E., Swain, C. T. V., Milne, R. L., English, D. R., Brown, K. A., Skinner, T. L., Lay, J., van Roekel, E. H., Moore, M. M., Gaunt, T. R., et al. (2022) · Cancer Epidemiology Biomarkers & Prevention · doi:10.1158/1055-9965.epi-22-0505 · Cited by 20 · see citing works
Impact of Noise Exposure on Risk of Developing Stress-Related Metabolic Effects
Sivakumaran, K., Ritonja, J. A., Waseem, H., AlShenaiber, L., Morgan, E., Ahmadi, S. A., Denning, A., Michaud, D. S., & Morgan, R. L. (2022) · Noise and Health · doi:10.4103/nah.nah_21_22 · Cited by 20 · see citing works
Self-citationKaram, S. G., Zhang, Y., Hernandez, H. P., Siebert, U., Koopman, L., Noyes, J., Tarride, J., Stevens, A., Welch, V. A., Saz‐Parkinson, Z., et al. (2024) · BMJ · doi:10.1136/bmj-2024-079890 · Cited by 19 · see citing works
Self-citationvan Boven, M. R., Hutten, G. J., Richardson, R., Königs, M., Leemhuis, A. G., Onland, W., Terheggen‐Lagro, S. W. J., Oosterlaan, J., & van Kaam, A. H. (2024) · European Respiratory Review · doi:10.1183/16000617.0114-2024 · Cited by 19 · see citing works
The effects of light in children: A systematic review
Westwood, E., Smith, S. S., Mann, D., Pattinson, C. L., Allan, A. C., & Staton, S. (2023) · Journal of Environmental Psychology · doi:10.1016/j.jenvp.2023.102062 · Cited by 21 · see citing works
- Authors
- Morgan, R. L., Thayer, K. A., Santesso, N., Holloway, A. C., Blain, R., Eftim, S. E., Goldstone, A. E., Ross, P., Ansari, M., Akl, E. A., Filippini, T., Hansell, A., Meerpohl, J. J., Mustafa, R. A., Verbeek, J., Vinceti, M., Whaley, P., & Schünemann, H. J.
- Affiliations
- McMaster University (CA)
- Environmental Protection Agency (US)
- ICF International (United States) (US)
- University of Ottawa (CA)
- University of Modena and Reggio Emilia (IT)
- Imperial College Healthcare NHS Trust (GB)
- University of Freiburg (DE)
- University of Kansas Medical Center (US)
- Finnish Institute of Occupational Health (FI)
- Boston University (US)
- Lancaster University (GB)
- Corresponding author
- Schünemann, Holger J.
- ORCID
- Venue
- Environment International
- Publisher
- Elsevier BV
- Published
- 2019-01
- Volume
- 122
- Pages
- 168-184
- ISSN
- 0160-4120 (ISSN-L)
- PMC
- PMC8221004
- OpenAlex
- W2901788566
- Cited by
- 248
- References
- 33
- Citations by year
- 2026: 20 · 2025: 34 · 2024: 31 · 2023: 45 · 2022: 45 · 2021: 41 · 2020: 25 · 2019: 7
- FWCI
- 13.92
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Environment International (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Lancaster EPrints (Lancaster University) (acceptedVersion)
- PubMed Central (acceptedVersion, cc-by-nc-nd)
- IRIS UNIMORE (University of Modena and Reggio Emilia) (submittedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Leicester Research Archive (University of Leicester) (submittedVersion, other-oa)
- Funders
- Intramural Research Program of the National Institute of Environmental Health Sciences
- MacGRADE Centre at the McMaster University
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- risk of bias, non-randomized studies, environmental exposure, ROBINS-I, GRADE, quality of evidence, indirectness, sensitivity analysis, systematic reviews
- MeSH terms
- Environmental Exposure, Humans, Random Allocation, Research Design, Bias, Risk Assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
