Cited by 248Meta-analysis and systematic reviewsStatistical Methods in EpidemiologyAdvanced Causal Inference Techniques

The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.

A risk of bias instrument for non-randomized studies of exposures: A users' guide to its application in the context of GRADE

Open access

Morgan, Rebecca L., Thayer, Kristina A., Santesso, Nancy, Holloway, Alison C., Blain, Robyn, Eftim, Sorina E., Goldstone, Alexandra E., Ross, Pam, Ansari, Mohammed, Akl, Elie A, Filippini, Tommaso, Hansell, Anna, Meerpohl, Joerg J., Mustafa, Reem A., Verbeek, Jos, Vinceti, Marco, Whaley, Paul, Schünemann, Holger J.·2019·Environment International

DOI 10.1016/j.envint.2018.11.004PMC PMC8221004

Open this paper in CoChat — free account Open PDF

Abstract

The objective of this paper is to explain how to apply, interpret, and present the results of a new instrument to assess the risk of bias (RoB) in non-randomized studies (NRS) dealing with effects of environmental exposures on health outcomes. This instrument is modeled on the Risk Of Bias In Non-randomized Studies of Interventions (ROBINS-I) instrument. The RoB instrument for NRS of exposures assesses RoB along a standardized comparison to a randomized target experiment, instead of the study-design directed RoB approach. We provide specific guidance for the integral steps of developing a research question and target experiment, distinguishing issues of indirectness from RoB, making individual-study judgments, and performing and interpreting sensitivity analyses for RoB judgments across a body of evidence. Also, we present an approach for integrating the RoB assessments within the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) framework to assess the certainty of the evidence in the systematic review. Finally, we guide the reader through an overall assessment to support the rating of all domains that determine the certainty of a body of evidence using the GRADE approach.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1016/j.envint.2018.11.004

Cites

33 of 33 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329
Self-citation
GRADE guidelines: 3. Rating the quality of evidence
Balshem, H., Helfand, M., Schünemann, H. J., Oxman, A. D., Kunz, R., Brozek, J., Vist, G. E., Falck-Ytter, Y., Meerpohl, J., & Norris, S. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.015 · Cited by 8,285
Self-citation
Using Big Data to Emulate a Target Trial When a Randomized Trial Is Not Available: Table 1.
Hernán, M. A., & Robins, J. M. (2016) · American Journal of Epidemiology · doi:10.1093/aje/kwv254 · Cited by 3,115
GRADE guidelines: 4. Rating the quality of evidence—study limitations (risk of bias)
Guyatt, G. H., Oxman, A. D., Vist, G., Kunz, R., Brozek, J., Alonso-Coello, P., Montori, V., Akl, E. A., Djulbegovic, B., & Falck-Ytter, Y. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.017 · Cited by 2,798
Self-citation
Smoking and Carcinoma of the Lung
Doll, R., & Hill, A. B. (1950) · BMJ · doi:10.1136/bmj.2.4682.739 · Cited by 1,999
GRADE guidelines: 8. Rating the quality of evidence—indirectness
Guyatt, G. H., Oxman, A. D., Kunz, R., Woodcock, J., Brozek, J., Helfand, M., Alonso-Coello, P., Falck-Ytter, Y., Jaeschke, R., Vist, G., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.04.014 · Cited by 1,843
Self-citation
Identifying the PECO: A framework for formulating good questions to explore the association of environmental and other exposures with health outcomes
Morgan, R. L., Whaley, P., Thayer, K. A., & Schünemann, H. J. (2018) · Environment International · doi:10.1016/j.envint.2018.07.015 · Cited by 1,424
Self-citation
GRADE guidelines: 9. Rating up the quality of evidence
Guyatt, G. H., Oxman, A. D., Sultan, S., Glasziou, P., Akl, E. A., Alonso-Coello, P., Atkins, D., Kunz, R., Brozek, J., Montori, V., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.06.004 · Cited by 1,336 · cited 6×
“These latter two domains contribute to increasing the CoE in a body of evidence of NRS in GRADE; however, within the RoB instrument for NRS of exposures they may also influence the study-level judgments (Guyatt et al., 2011c).”
BackgroundResultInfluentialSelf-citation
Untitled work
(2016) · Cochrane Database of Systematic Reviews · Cited by 985

Flagged first, then by citations

Show all 33 references

Research works printed but not linked: 1

Thayer, 2013, Draft protocol for systematic review to evaluate the evidence for an association between bisphenol A (BPA) exposure and obesity, Natl. Toxicol. Program

Sorted by citations

Items 101–200 of 247

Toolkit of methodological resources to conduct systematic reviews
Figuls, M. R. I., García, L. M., Solà, I., Alonso‐Coello, P., Bonfill, X., & Zamora, J. (2020) · F1000Research · doi:10.12688/f1000research.22032.2 · Cited by 25
Linking Physical Activity to Breast Cancer Risk via Insulin/Insulin-Like Growth Factor Signaling System, Part 1: The Effect of Physical Activity on the Insulin/Insulin-Like Growth Factor Signaling System
Swain, C. T. V., Drummond, A. E., Milne, R. L., English, D. R., Brown, K. A., Chong, J. E., Skinner, T. L., van Roekel, E. H., Moore, M. M., Gaunt, T. R., et al. (2022) · Cancer Epidemiology Biomarkers & Prevention · doi:10.1158/1055-9965.epi-22-0504 · Cited by 19
Reliability of digital measurement methods on the marginal fit of fixed prostheses: A systematic review and meta-analysis of in vitro studies
Mai, H. Y., Lee, W. K., Kwon, T., & Lee, D. (2020) · Journal of Prosthetic Dentistry · doi:10.1016/j.prosdent.2020.04.011 · Cited by 17
Linking Physical Activity to Breast Cancer via Inflammation, Part 2: The Effect of Inflammation on Breast Cancer Risk
Lou, M. W. C., Drummond, A. E., Swain, C. T. V., Milne, R. L., English, D. R., Brown, K. A., van Roekel, E. H., Skinner, T. L., Moore, M. M., Gaunt, T. R., et al. (2023) · Cancer Epidemiology Biomarkers & Prevention · doi:10.1158/1055-9965.epi-22-0929 · Cited by 17
Linking Physical Activity to Breast Cancer Risk via Inflammation, Part 1: The Effect of Physical Activity on Inflammation
Swain, C. T. V., Drummond, A. E., Milne, R. L., English, D. R., Brown, K. A., Lou, M. W. C., Boing, L., Bageley, A., Skinner, T. L., van Roekel, E. H., et al. (2023) · Cancer Epidemiology Biomarkers & Prevention · doi:10.1158/1055-9965.epi-22-0928 · Cited by 17
Night shift work and breast cancer risk – 2023 update of epidemiologic evidence
Hansen, J., & Pedersen, J. E. (2024) · Journal of the National Cancer Center · doi:10.1016/j.jncc.2024.07.004 · Cited by 17
Dose‐response relationships in health risk assessment of nutritional and toxicological factors in foods: development and application of novel biostatistical methods
Vinceti, M., Filippini, T., Malavolti, M., Naska, A., Kasdagli, M., Torres, D. P. M., Lopes, C. M. M., Carvalho, C., Moreira, P. A. M., & Orsini, N. (2020) · EFSA Supporting Publications · doi:10.2903/sp.efsa.2020.en-1899 · Cited by 17
Self-citation
IARC-NCI workshop on an epidemiological toolkit to assess biases in human cancer studies for hazard identification: beyond the algorithm
Schubauer‐Berigan, M. K., Richardson, D. B., Fox, M. P., Fritschi, L., Canu, I. G., Pearce, N. E., Stayner, L. T., & de González, A. B. (2023) · Occupational and Environmental Medicine · doi:10.1136/oemed-2022-108724 · Cited by 17
Genetic, antimicrobial resistance profile and mortality rates of Acinetobacter baumannii infection in Brazil: A systematic review
Silva, M. C., Werlang, M. H. B., Vandresen, D. F., Fortes, P. C. N., Pascotto, C. R., Lúcio, L. C., & Ferreto, L. E. D. (2022) · Narra J · doi:10.52225/narra.v2i1.68 · Cited by 16
Is model-estimated PM2.5 exposure equivalent to station-observed in mortality risk assessment? A literature review and meta-analysis
Yu, W., Song, J., Li, S., & Guo, Y. (2024) · Environmental Pollution · doi:10.1016/j.envpol.2024.123852 · Cited by 17
Does ozone inhalation cause adverse metabolic effects in humans? A systematic review
LaKind, J. S., Burns, C. J., Pottenger, L. H., Naiman, D. Q., Goodman, J. E., & Marchitti, S. A. (2021) · Critical Reviews in Toxicology · doi:10.1080/10408444.2021.1965086 · Cited by 16
The Role of Hemodynamics through the Circle of Willis in the Development of Intracranial Aneurysm: A Systematic Review of Numerical Models
Shen, Y., Molenberg, R., Bokkers, R. P. H., Wei, Y., Uyttenboogaart, M., & van Dijk, J. M. C. (2022) · Journal of Personalized Medicine · doi:10.3390/jpm12061008 · Cited by 17
Anxiety among children and adolescents during the COVID-19 pandemic in Europe: a systematic review protocol
Ludwig‐Walz, H., Dannheim, I., Pfadenhauer, L. M., Fegert, J. M., & Bujard, M. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02225-1 · Cited by 15
Impact of the environment on the health: From theory to practice
Carducci, A., Agodi, A., Ancona, C., Angelini, P., Bagordo, F., Barbone, F., Birbaum, L., Carreri, V., Casuccio, A., Conti, A., et al. (2020) · Environmental Research · doi:10.1016/j.envres.2020.110517 · Cited by 16
Self-citation
A systematic review on the association between exposure to air particulate matter during pregnancy and the development of hypertensive disorders of pregnancy and gestational diabetes mellitus
Alvarado-Jiménez, D., Donzelli, G., & Morales‐Suárez‐Varela, M. (2023) · Reviews on Environmental Health · doi:10.1515/reveh-2022-0258 · Cited by 15
Ankyloglossia and malocclusion
Póvoa-Santos, L., dos Santos, R. L., Alvarenga‐Brant, R., Notaro, S. Q., Souza‐Oliveira, A. C., Occhi‐Alexandre, I. G. P., & Martins, C. C. (2023) · The Journal of the American Dental Association · doi:10.1016/j.adaj.2023.09.014 · Cited by 15
Impact of Noise Exposure on Risk of Developing Stress-Related Obstetric Health Effects
Sivakumaran, K., Ritonja, J. A., Waseem, H., AlShenaibar, L., Morgan, E., Ahmadi, S. A., Denning, A., Michaud, D. S., & Morgan, R. L. (2022) · Noise and Health · doi:10.4103/nah.nah_22_22 · Cited by 14
Self-citation
Association of greenspace with hypertension in adult: a systematic review and meta-analysis of epidemiological studies
Bu, Y., Zhang, X., Song, S., Su, H., Yu, Z., & Guo, Y. (2023) · International Journal of Environmental Health Research · doi:10.1080/09603123.2023.2259817 · Cited by 13
Exposure to Light at Night and Risk of Cancer: A Systematic Review, Meta-Analysis, and Data Synthesis
Ma, S., Alsabawi, Y., El‐Serag, H. B., & Thrift, A. P. (2024) · Cancers · doi:10.3390/cancers16152653 · Cited by 14
Scientific Committee guidance on appraising and integrating evidence from epidemiological studies for use in EFSA's scientific assessments
More, S. J., Bampidis, V., Benford, D., Bragard, C., Hernández, A. F., Bennekou, S. H., Koutsoumanis, K. P., Lambré, C., Machera, K. A., Mennes, W. C., et al. (2024) · EFSA Journal · doi:10.2903/j.efsa.2024.8866 · Cited by 13
Self-citation
Nutritional status, body composition and chemotherapy dosing in children and young people with cancer: a systematic review by the SIOP nutrition network
Lovell, A. L., Makamo, N., Veal, G. J., Bernhardt, M. B., Barr, R. D., Gala, R. M., Gordon, E., Ladas, E. J., Prasad, M., Rogers, P. C., et al. (2025) · British Journal of Cancer · doi:10.1038/s41416-025-03023-3 · Cited by 13
Artificially Sweetened Beverage Consumption and Cancer Risk: A Comprehensive Dose–Response Meta-Analysis of Prospective Studies
Yin, T., Li, J., Wang, Y., Liu, K., Long, T., & Cheng, L. (2022) · Nutrients · doi:10.3390/nu14214445 · Cited by 16
Comparison of the short‐ and long‐term prognosis of early‐onset colorectal cancer compared with later‐onset colorectal cancer: A systematic review and meta‐analysis
Jin, T., Li, X., Ji, J., Li, J., Yin, X., Xu, K. L., Wang, W., Zhang, W., Xu, X., Hu, Z., et al. (2023) · Health Science Reports · doi:10.1002/hsr2.1307 · Cited by 13
Assessing the quality of evidence in studies estimating prevalence of exposure to occupational risk factors: The QoE-SPEO approach applied in the systematic reviews from the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury
Pega, F., Momen, N. C., Gagliardi, D., Bero, L. A., Boccuni, F., Chartres, N., Descatha, A., Dzhambov, A. M., Godderis, L., Loney, T., et al. (2022) · Environment International · doi:10.1016/j.envint.2022.107136 · Cited by 15
Self-citation
Plant-based diets and risk of type 2 diabetes: systematic review and dose–response meta-analysis
Murciano, A., Malavolti, M., Fairweather‐Tait, S. J., Paduano, S., Vinceti, M., & Filippini, T. (2025) · British Journal Of Nutrition · doi:10.1017/s0007114525104157 · Cited by 17
Self-citation
A systematic review of effective quality feedback measurement tools used in clinical skills assessment
Alsahafi, A., Ling, D. L. X., Newell, M., & Kropmans, T. (2023) · MedEdPublish · doi:10.12688/mep.18940.2 · Cited by 12
Short-term exposure to sulfur dioxide and the occurrence of chronic obstructive pulmonary disease: An updated systematic review and meta-analysis based on risk of bias and certainty of evidence
Zhou, X., Wang, X., Shen, Q., Ma, J., Cai, X., Liu, H., Yan, J., Xu, H., & Wang, Y. (2024) · Ecotoxicology and Environmental Safety · doi:10.1016/j.ecoenv.2024.116888 · Cited by 12
Assessor burden, inter-rater agreement and user experience of the RoB-SPEO tool for assessing risk of bias in studies estimating prevalence of exposure to occupational risk factors: An analysis from the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury
Momen, N. C., Streicher, K. N., da Silva, D. T. C., Descatha, A., Frings‐Dresen, M. H. W., Gagliardi, D., Godderis, L., Loney, T., Mandrioli, D., Modenese, A., et al. (2021) · Environment International · doi:10.1016/j.envint.2021.107005 · Cited by 12
Self-citation
Assessments of risk of bias in systematic reviews of observational nutritional epidemiologic studies are often not appropriate or comprehensive: a methodological study
Zeraatkar, D., Kohut, A., Bhasin, A. A., Morassut, R. E., Churchill, I. F., Gupta, A. A., Lawson, D. O., Miroshnychenko, A., Sirotich, E., Aryal, K., et al. (2021) · BMJ Nutrition Prevention & Health · doi:10.1136/bmjnph-2021-000248 · Cited by 11
Effect of sleep disturbance on biomarkers related to the development of adverse health outcomes: A systematic review of the human literature
Sivakumaran, K., Ritonja, J. A., Palmer, N. R., Pasumarthi, T. S., Waseem, H., Yu, T. M., Denning, A., Michaud, D. S., & Morgan, R. L. (2022) · Journal of Sleep Research · doi:10.1111/jsr.13775 · Cited by 12
Self-citation
Physical activity and pain in people with cancer: a systematic review and meta-analysis
Peters, M., Butson, G., Mizrahi, D., Denehy, L., Lynch, B. M., & Swain, C. T. V. (2024) · Supportive Care in Cancer · doi:10.1007/s00520-024-08343-3 · Cited by 14
Real-world evidence of allergen immunotherapy
Buta, F., Paoletti, G., Bragato, M. C., Giovannini, M., Canonica, G. W., & Heffler, E. M. (2024) · Current Opinion in Allergy and Clinical Immunology · doi:10.1097/aci.0000000000001026 · Cited by 11
GRADE guidance 40: The GRADE evidence-to-decision framework for environmental and occupational health
Senerth, E., Whaley, P., Akl, E. A., Beverly, B., Alonso‐Coello, P., Rooney, A. A., Schünemann, H. J., Thayer, K. A., Tsaioun, K., & Morgan, R. L. (2025) · Environment International · doi:10.1016/j.envint.2025.109314 · Cited by 12
Self-citation
Calling for a comprehensive risk assessment of selenium in drinking water
Vinceti, M., Mazzoli, R., Wise, L. A., Veneri, F., & Filippini, T. (2025) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2025.178700 · Cited by 11
Self-citation
The role of humor in social, psychological, and physical well-being
Oliveira, R., Arriaga, P., & Barreiros, J. (2023) · Humor - International Journal of Humor Research · doi:10.1515/humor-2022-0072 · Cited by 20
Enhancing recovery and reducing inflammation: the impact of enhanced recovery after surgery recommendations on inflammatory markers in laparoscopic surgery—a scoping review
Bersot, C. D. A., Pereira, L. F. G., Goncho, V. G. V., Pereira, J. E. G., & Falcão, L. F. D. R. (2024) · Frontiers in Surgery · doi:10.3389/fsurg.2024.1450434 · Cited by 10
Exposure to disinfection by-products and risk of birth defects: A systematic review and dose-response meta-analysis
Deiana, G., Filippini, T., Dettori, M., Vinceti, M., & Azara, A. (2025) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2025.179693 · Cited by 11
Self-citation
Cancer Effects of Low to Moderate Doses of Ionizing Radiation in Young People with Cancer-Predisposing Conditions: A Systematic Review
Canet, M., Harbron, R. W., Thierry-Chef, I., & Cardis, E. (2022) · Cancer Epidemiology Biomarkers & Prevention · doi:10.1158/1055-9965.epi-22-0393 · Cited by 11
Rehospitalisation rates after long-term follow-up of patients with severe mental illness admitted for more than one year: a systematic review
Sato, S., Nakanishi, M., Ogawa, M., Abe, M., Yasuma, N., Kono, T., Igarashi, M., Iwanaga, M., Kawaguchi, T., & Yamaguchi, S. (2023) · BMC Psychiatry · doi:10.1186/s12888-023-05290-x · Cited by 9
A critical review on the toxicological and epidemiological evidence integration for assessing human health risks to environmental chemical exposures
Donzelli, G., Gehring, R., Murugadoss, S., Roos, T., Schaffert, A., & Linzalone, N. (2024) · Reviews on Environmental Health · doi:10.1515/reveh-2024-0072 · Cited by 11
A systematic review and meta-analysis of human population studies on the association between exposure to toxic environmental chemicals and left ventricular dysfunction (LVD)
Donzelli, G., Sera, F., Morales, M., Vozzi, F., Roos, T., Schaffert, A., Paparella, M., Murugadoss, S., Mertens, B., Gehring, R., et al. (2024) · Environmental Research · doi:10.1016/j.envres.2024.118429 · Cited by 10
Systems for rating bodies of evidence used in systematic reviews of air pollution exposure and reproductive and children’s health: a methodological survey
Michel, S., Atmakuri, A., & Ehrenstein, O. S. V. (2024) · Environmental Health · doi:10.1186/s12940-024-01069-z · Cited by 10
Cadmium exposure and risk of breast cancer: A dose-response meta-analysis of cohort studies
Filippini, T., Torres, D. P. M., Lopes, C., Carvalho, C., Moreira, P. A. M., Naska, A., Kasdagli, M., Malavolti, M., Orsini, N., & Vinceti, M. (2020) · Environment International · doi:10.1016/j.envint.2020.105879 · Cited by 9
Self-citation
Accuracy and Clinical Performance of Intraoral Scanners Compared to Conventional and Extraoral Impressions: An Umbrella Review
Singh, R., Mistry, G. S., Kini, A. Y., Ansari, R., Kailaje, V., & Kapoor, S. (2025) · Cureus · doi:10.7759/cureus.93202 · Cited by 8
Circulating retinol binding protein 4 levels in coronary artery disease: a systematic review and meta-analysis
Chen, H., Zhang, J., Lai, J., Zhou, Y., Lin, X., Deng, G., Zhang, Z., & Li, L. (2021) · Lipids in Health and Disease · doi:10.1186/s12944-021-01516-7 · Cited by 8
Effects of air pollution on restricted activity days: systematic review and meta-analysis
Orellano, P. W., Reynoso, J. I., & QUARANTA, N. E. (2023) · Environmental Health · doi:10.1186/s12940-023-00979-8 · Cited by 11
Systematic review of longitudinal studies on the association between cluster of health-related behaviors and tooth loss among adults
Alobaidi, F., Heidari, E., & Sabbah, W. (2023) · Acta Odontologica Scandinavica · doi:10.1080/00016357.2023.2287718 · Cited by 8
Relationship between flame retardants and respiratory health– A systematic review and meta-analysis of observational studies
Coelho, S. D., Maricoto, T., Taborda‐Barata, L., Annesi‐Maesano, I., Isobe, T., & Sousa, A. C. A. (2024) · Environmental Pollution · doi:10.1016/j.envpol.2024.123733 · Cited by 8
Toolkit of methodological resources to conduct systematic reviews
Figuls, M. R. I., García, L. M., Solà, I., Alonso‐Coello, P., Bonfill, X., & Zamora, J. (2020) · F1000Research · doi:10.12688/f1000research.22032.1 · Cited by 8
Development of a framework to structure decision-making in environmental and occupational health: A systematic review and Delphi study
Senerth, E., Whaley, P., Akl, E. A., Beverly, B., Alonso‐Coello, P., Jalil, E., Khattar, J., Palmer, N. R., Rooney, A. A., Schünemann, H. J., et al. (2024) · Environment International · doi:10.1016/j.envint.2024.109209 · Cited by 9
Self-citation
Risks of colorectal and extracolonic cancers following colorectal cancer: a systematic review and meta-analysis
Aung, Y. K., Zhang, Y., Jenkins, M. A., & Win, A. K. (2025) · JNCI Cancer Spectrum · doi:10.1093/jncics/pkaf031 · Cited by 9
Effects of environmental air pollution on anxiety and depression in adults: an updated systematic review and meta-analysis
Bereziartua, A., Jimeno-Romero, A., Subiza‐Pérez, M., & Lertxundi, A. (2026) · Environmental Research · doi:10.1016/j.envres.2026.123738 · Cited by 9
Circulating chemerin levels in preeclampsia: a systematic review and meta-analysis
Yin, Y., Xie, S., Xu, Q. S., Liao, L., Chen, H., & Zhou, R. (2023) · Lipids in Health and Disease · doi:10.1186/s12944-023-01941-w · Cited by 7
Systematic review of plasma/packed red blood cell ratio on survival in ruptured abdominal aortic aneurysms
Phillips, A. R., Tran, L. M., Foust, J. E., & Liang, N. L. (2020) · Journal of Vascular Surgery · doi:10.1016/j.jvs.2020.10.027 · Cited by 7
Customized guidance/training improved the psychometric properties of methodologically rigorous risk of bias instruments for non-randomized studies
Jeyaraman, M. M., Robson, R. C., Copstein, L., Al‐Yousif, N., Pollock, M., Xia, J., Balijepalli, C., Hofer, K., Mansour, S., Fazeli, M. S., et al. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.04.017 · Cited by 8
Self-citation
Physical activity and physical fitness among children and adolescents after the onset of the COVID-19 pandemic in the WHO European Region: a systematic review protocol
Ludwig‐Walz, H., Siemens, W., Heinisch, S., Dannheim, I., Loss, J., & Bujard, M. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-073397 · Cited by 6
Safety of Normothermic Cardiopulmonary Bypass in Pediatric Cardiac Surgery: A System Review and Meta-Analysis
Xiong, T., Pu, L., Ma, Y.-F., Zhu, Y., Cui, X., Li, H., Zhan, X., & Li, Y.-X. (2021) · Frontiers in Pediatrics · doi:10.3389/fped.2021.757551 · Cited by 7
Factors associated with prehospital and in-hospital delays in acute ischaemic stroke care in Indonesia: a systematic review
Wiyarta, E., Hidayat, R., Kurniawan, M., Sinaga, G. H. P., & Budiman, R. A. (2025) · BMJ Open · doi:10.1136/bmjopen-2024-095845 · Cited by 6
Fine particulate matter exposure and cancer risk: a systematic review and meta-analysis of prospective cohort studies
Feng, Y., Li, J., Wang, Y., Yin, T., Wang, Q., & Cheng, L. (2025) · Reviews on Environmental Health · doi:10.1515/reveh-2024-0171 · Cited by 7
The association of fluoride exposure with bone density and fracture risk: a dose-response meta-analysis
Mazzoli, R., Filippini, T., Iamandii, I., Pasquale, L. D., Veneri, F., Birnbaum, L. S., Rothman, K. J., & Vinceti, M. (2025) · Environmental Health · doi:10.1186/s12940-025-01226-y · Cited by 6
Self-citation
Is there a bidirectional relationship between workplace bullying and the risk of sickness absence? Systematic review and meta-analysis of prospective studies
Liao, X., Wang, Y., Zeng, Q., Wang, J., Yang, X., Yan, W., Wang, G., & Zeng, Y. (2023) · Occupational and Environmental Medicine · doi:10.1136/oemed-2023-108823 · Cited by 8
Health risk behaviours among migrants by duration of residence: protocol for a systematic review and meta-analysis
Berg, L., Gustafsson, N.-K. J., Honkaniemi, H., & Juárez, S. (2020) · BMJ Open · doi:10.1136/bmjopen-2020-038388 · Cited by 6
Meta-Analysis of Body Concentration of Polychlorinated Biphenyls and Prostate Cancer
Pourhassan, B., Pourbabaki, R., Omidi, F., Kalantary, S., & Beigzadeh, Z. (2022) · Toxicology and Industrial Health · doi:10.1177/07482337221129162 · Cited by 6
A systematic review of effective quality feedback measurement tools used in clinical skills assessment
Alsahafi, A., Ling, D. L. X., Newell, M., & Kropmans, T. (2022) · MedEdPublish · doi:10.12688/mep.18940.1 · Cited by 16
Association between road traffic noise exposure and heart failure: A systematic review and meta-analysis of prospective cohort studies.
Liu, M., Chen, X., Zheng, G., Zhou, B., Fang, Z., Chen, H.-Y., Liang, X.-H., & Hao, G. (2025) · Public Health · doi:10.1016/j.puhe.2025.01.029 · Cited by 4
Associations of nickel exposure with diabetes: evidence from observational studies
Xia, W., Guo, X., Xie, P., Feng, L., Wu, B., Gao, J., Ma, S., Liu, H., Sun, C., Qu, G., et al. (2023) · Environmental Science and Pollution Research · doi:10.1007/s11356-023-29423-7 · Cited by 5
Antibiotic exposure is associated with decreased risk of psychiatric disorders
Kerman, I. A., Glover, M. E., Lin, Y., West, J. L., Hanlon, A. L., Kablinger, A. S., & Clinton, S. M. (2024) · Frontiers in Pharmacology · doi:10.3389/fphar.2023.1290052 · Cited by 7
Use of GRADE in systematic reviews of health effects on pollutants and extreme temperatures: A cross-sectional survey
Song, X., Ma, Y., Tang, J. Y., Peng, J., Hu, Y., Han, Y., Fu, X., Luo, X., Li, X., Ge, L., et al. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.05.021 · Cited by 5
Proposed Risk of Bias Assessment Tool for In Vitro Antimicrobial Susceptibility Studies
Falagas, M. E., Ragias, D., Kontogiannis, D. S., Romanos, L. T., & Farazi, P. A. (2026) · Pathogens · doi:10.3390/pathogens15040396 · Cited by 4
Can renewable energy meet the surging power demand of artificial intelligence? A systematic review
Yoo, S., Choi, S., & Kim, D. (2026) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2026.117213 · Cited by 4
The Impact of Excluding Nonrandomized Studies From Systematic Reviews in Rare Diseases: “The Example of Meta-Analyses Evaluating the Efficacy and Safety of Enzyme Replacement Therapy in Patients With Mucopolysaccharidosis”
Sampayo-Cordero, M., Miguel-Huguet, B., Malfettone, A., Pérez-García, J. M., Llombart‐Cussac, A., Cortés, J., Pardo-Mateos, A., & Pérez‐López, J. (2021) · Frontiers in Molecular Biosciences · doi:10.3389/fmolb.2021.690615 · Cited by 4
Protocol for a Systematic Review and Meta-Analysis of Observational Studies on the Association of Exposure to Toxic Environmental Pollutants and Left Ventricular Dysfunction
Linzalone, N., Donzelli, G., Morales, M. A., & Vozzi, F. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19127482 · Cited by 4
Effects of exposure to night shift work on cancer risk in workers
Ritonja, J. A., Papantoniou, K., Ebenberger, A., Wagner, G., Gartlehner, G., Klerings, I., Morgan, R. L., Herkner, H., Aronson, K. J., & Schernhammer, E. S. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013466 · Cited by 5
Self-citation
What is the effect of preterm birth on permanent tooth crown dimensions? A systematic review and meta-analysis
Buhamer, S. N., Kaklamanos, E. G., Kowash, M., Hussein, I., Salami, A., & Halabi, M. A. (2021) · PLoS ONE · doi:10.1371/journal.pone.0259293 · Cited by 4
Exposure to nitrate and nitrite in drinking water and cancers
Chambers, T. J., Willink, R., Reynolds, A. N., Anglemyer, A. T., Royal, H., Rentta, N. N., Green, R., & Picetti, R. (2024) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd015822 · Cited by 3
Are asthma and allergy associated with increased root resorption following orthodontic treatment? A meta-analysis
Al-Saqi, R. K., Athanasiou, A. E., Makrygiannakis, M. A., & Kaklamanos, E. G. (2023) · PLoS ONE · doi:10.1371/journal.pone.0285309 · Cited by 4
Maternal consumption of dairy products and gestational weight gain, gestational diabetes mellitus, and preeclampsia: A systematic review
Huang, D., Li, X., Wu, Q., Ji, C., Dai, H., Li, H., Chang, Q., Xu, X., & Zhao, Y. (2023) · Journal of Functional Foods · doi:10.1016/j.jff.2023.105613 · Cited by 4
Ideal cardiovascular health metrics and risk of type 2 diabetes: A systematic review and dose-response meta-analysis of prospective cohort studies
Badrooj, N., Jayedi, A., & Shab‐Bidar, S. (2023) · Nutrition Metabolism and Cardiovascular Diseases · doi:10.1016/j.numecd.2023.07.004 · Cited by 2
Interventions to promote the health and well-being of children under 5s experiencing homelessness in high-income countries: a scoping review
Tu, Y., Sarkar, K., Svirydzenka, N., Palfreyman, Z., Parry, Y., Ankers, M. D., Parikh, P., Raghavan, R., & Lakhanpaul, M. (2024) · BMJ Open · doi:10.1136/bmjopen-2023-076492 · Cited by 4
Associations between ambient particulate matter and lung cancer: A systematic review and meta-analyses up to 2020
Huang, W., Yang, B., Zeeshan, M., & Dong, G. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-820123-7.00004-8 · Cited by 3
Depression and anxiety among children and adolescents during the COVID-19 pandemic in Europe: A systematic review protocolPreprint
Ludwig‐Walz, H., Dannheim, I., Pfadenhauer, L. M., Fegert, J. M., & Bujard, M. (2022) · Research Square · doi:10.21203/rs.3.rs-1638547/v2 · Cited by 3
Systematic review and meta-analysis of epidemiologic data on infectious disease in relation to exposure to twelve perfluoroalkyl substances (PFAS)
Rosen, E. M., Crawford, L., Hoffmann, S., Skidmore, B., Porter, A. K., Halperin, S. A., & Longnecker, M. P. (2025) · International Journal of Hygiene and Environmental Health · doi:10.1016/j.ijheh.2025.114571 · Cited by 3
Malnutrition and Sarcopenia in Patients with Neuroendocrine Tumors: A Comprehensive Review of Evidence
Poulia, K., Spyroglou, A., Violetis, O., Mastorakos, G. P., Alexandraki, K., & Papavassiliou, A. G. (2025) · Biomolecules · doi:10.3390/biom15121746 · Cited by 2
Coffee and tea consumption and risk of dementia: a dose-response meta-analysis of cohort and cohort-nested case-control studies
Mazzoleni, E., Malavolti, M., Rossetti, A. O., Mazzoli, R., Vinceti, M., & Filippini, T. (2026) · Journal of Epidemiology and Population Health · doi:10.1016/j.jeph.2026.203168 · Cited by 2
Self-citation
Long-term prognostic significance of negative CT-derived fractional flow reserve in patients with stable CAD: A meta-analysis of reconstructed time-to-event data
Javid, R. N., Tavakoli, K., Dastjerdi, P., Parastooei, B., Najafinezhad, F., Aghaei, M., Javadi, M., Rasool, M. H. U., Movahedi, N., Alsaid, A., et al. (2026) · Journal of cardiovascular computed tomography · doi:10.1016/j.jcct.2026.03.006 · Cited by 2
The prevalence of pediatric asthma hospitalizations at different stages of the COVID-19 pandemic: A systematic review and meta-analysis study protocol
Abdelrahim, R., Gao, Z., Smith, M. J., & Newhook, L. A. (2023) · PLoS ONE · doi:10.1371/journal.pone.0289538 · Cited by 2
Feeding styles and adiposity in children of 6 months– 5 years of age: Protocol for a systematic review and meta- analysis
Haridas, D. N., van Schayck, O. C. P., Babu, G. R., Nair, N. S., & Shriyan, P. (2023) · PLoS ONE · doi:10.1371/journal.pone.0292139 · Cited by 3
Recommendations for future research exploring e‐cigarette use and later cigarette smoking in young people: Results from a consultation exercise
Conde, M., Pesko, M. F., Shahab, L., Begh, R. A., Lindson, N., Jackson, S. E., Kale, D., Kneale, D., Livingstone‐Banks, J., & Hartmann‐Boyce, J. (2025) · Addiction · doi:10.1111/add.70038 · Cited by 2
Unreliable evidence from problematic risk of bias assessments: Comment on Begh et al., ‘Electronic cigarettes and subsequent cigarette smoking in young people: A systematic review’
Egger, S. J., & McKee, M. (2025) · Addiction · doi:10.1111/add.70143 · Cited by 2
The impact of the number of remaining teeth on the risk of frailty: a meta-analysis
Tada, A., & Miura, H. (2025) · Odontology · doi:10.1007/s10266-025-01154-w · Cited by 2
Drug-related risk of hospital readmission in children with chronic diseases, a systematic review
Jaberi, E., Kassaï, B., Bérard, A., Grenet, G., & Nguyen, K. A. (2022) · Therapies · doi:10.1016/j.therap.2022.09.004 · Cited by 2
A tool to assess risk of bias in studies estimating the prevalence of mental health disorders (RoB-PrevMH)Preprint
Tonia, T., Buitrago‐García, D., Peter, N. L., Mesa-Vieira, C., Li, T., Furukawa, T. A., Cipriani, A., Leucht, S., Low, N. N., & Salanti, G. (2023) · medRxiv · doi:10.1101/2023.02.01.23285335 · Cited by 2
Association of diethylhexyl phthalate exposure with serum thyroid hormone levels: a systematic review and meta-analysis
Ke, X. (2024) · American Journal of Translational Research · doi:10.62347/iums4514 · Cited by 1
Has the documentation of chest injuries and the development of systemic complications in patients with long bone fractures changed over time?–A systematic literature review and meta-analysis by the IMPACT expert group
Hatz, B. A., Klingebiel, F. K.-L., Kalbas, Y., Halvachizadeh, S., Pfeifer, R., Schemitsch, E. H., & Pape, H. (2025) · Injury · doi:10.1016/j.injury.2025.112182 · Cited by 1
Interpreting evidence on the association between multiple adverse childhood experiences and mental and physical health outcomes in adulthood: protocol for a systematic review assessing causality
Jones, L., Bellis, M. A., Butler, N., Hughes, K. E., McManus, S., & Quigg, Z. (2025) · BMJ Open · doi:10.1136/bmjopen-2024-091865 · Cited by 1
Association of feeding and parenting styles with adiposity in young children: a systematic review and meta-analysis
Haridas, D. N., Shriyan, P., Tartour, A. I., Chivese, T., van Schayck, O. C. P., Nair, N. S., & Babu, G. R. (2025) · European Journal of Pediatrics · doi:10.1007/s00431-025-06348-6 · Cited by 1
Systematic Review of Prenatal Exposure to PM2.5 and Its Chemical Components and Their Effects on Neurodevelopmental Outcomes in Neonates
Donzelli, G., Peraita‐Costa, I., Linzalone, N., & Morales‐Suárez‐Varela, M. (2025) · Atmosphere · doi:10.3390/atmos16091034 · Cited by 1
Association between DDT or its byproducts and T2DM: a systematic review and meta-analysis
Tian, W., Bai, S., Xie, T., Zha, H., Yan, Z., Zhang, S., Wu, N., Yuan, J., Luo, H., Xie, Q., et al. (2025) · Frontiers in Endocrinology · doi:10.3389/fendo.2025.1634292 · Cited by 1
Developing Guideline Recommendations and Treatment Regimens for Acupuncture: An Example from the Clinical Practice Guideline
Hui-ze, L. I. N., Li, S., Fei, Y., Zhao, H., Lu, Y., Liu, X., Wang, X., Feng, Y., Guo, S., Liu, L., et al. (2025) · Journal of Visualized Experiments · doi:10.3791/68716 · Cited by 1
A Systematic Review and Meta-Analysis of Maternal Dengue Infection and Adverse Pregnancy Outcomes
Elaveyini, U., Sivakumar, T., Krithika, C., Sridhar, C., & Hercules, A. (2025) · Cureus · doi:10.7759/cureus.96888 · Cited by 1

Authors: Morgan, R. L., Thayer, K. A., Santesso, N., Holloway, A. C., Blain, R., Eftim, S. E., Goldstone, A. E., Ross, P., Ansari, M., Akl, E. A., Filippini, T., Hansell, A., Meerpohl, J. J., Mustafa, R. A., Verbeek, J., Vinceti, M., Whaley, P., & Schünemann, H. J.
Affiliations: McMaster University (CA)
Environmental Protection Agency (US)
ICF International (United States) (US)
University of Ottawa (CA)
University of Modena and Reggio Emilia (IT)
Imperial College Healthcare NHS Trust (GB)
University of Freiburg (DE)
University of Kansas Medical Center (US)
Finnish Institute of Occupational Health (FI)
Boston University (US)
Lancaster University (GB)
Corresponding author: Schünemann, Holger J.
ORCID: Morgan, Rebecca L.
Thayer, Kristina A.
Santesso, Nancy
Holloway, Alison C.
Blain, Robyn
Eftim, Sorina E.
Akl, Elie A
Filippini, Tommaso
Hansell, Anna
Meerpohl, Joerg J.
Mustafa, Reem A.
Verbeek, Jos
Vinceti, Marco
Whaley, Paul
Schünemann, Holger J.
Venue: Environment International
Publisher: Elsevier BV
Published: 2019-01
Volume: 122
Pages: 168-184
ISSN: 0160-4120 (ISSN-L)
DOI: 10.1016/j.envint.2018.11.004
PMC: PMC8221004
OpenAlex: W2901788566
Cited by: 248
References: 33
Citations by year: 2026: 20 · 2025: 34 · 2024: 31 · 2023: 45 · 2022: 45 · 2021: 41 · 2020: 25 · 2019: 7
FWCI: 13.92
Type: journal article
Language: en
Open access: hybrid
doi.org
Licence: cc-by-nc-nd
www.elsevier.com/tdm/userlicense/1.0/ (tdm)
creativecommons.org/licenses/by-nc-nd/4.0/ (vor)
Locations: Environment International (publishedVersion, cc-by-nc-nd)
PubMed (publishedVersion)
Lancaster EPrints (Lancaster University) (acceptedVersion)
PubMed Central (acceptedVersion, cc-by-nc-nd)
IRIS UNIMORE (University of Modena and Reggio Emilia) (submittedVersion, cc-by-nc-nd)
DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Figshare (submittedVersion, cc-by)
Leicester Research Archive (University of Leicester) (submittedVersion, other-oa)
Funders: Intramural Research Program of the National Institute of Environmental Health Sciences
MacGRADE Centre at the McMaster University
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
Keywords: risk of bias, non-randomized studies, environmental exposure, ROBINS-I, GRADE, quality of evidence, indirectness, sensitivity analysis, systematic reviews
MeSH terms: Environmental Exposure, Humans, Random Allocation, Research Design, Bias, Risk Assessment

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

A risk of bias instrument for non-randomized studies of exposures: A users' guide to its application in the context of GRADE

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks