A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E)
Open accessAbstract
BACKGROUND: Observational epidemiologic studies provide critical data for the evaluation of the potential effects of environmental, occupational and behavioural exposures on human health. Systematic reviews of these studies play a key role in informing policy and practice. Systematic reviews should incorporate assessments of the risk of bias in results of the included studies. OBJECTIVE: To develop a new tool, Risk Of Bias In Non-randomized Studies - of Exposures (ROBINS-E) to assess risk of bias in estimates from cohort studies of the causal effect of an exposure on an outcome. METHODS AND RESULTS: ROBINS-E was developed by a large group of researchers from diverse research and public health disciplines through a series of working groups, in-person meetings and pilot testing phases. The tool aims to assess the risk of bias in a specific result (exposure effect estimate) from an individual observational study that examines the effect of an exposure on an outcome. A series of preliminary considerations informs the core ROBINS-E assessment, including details of the result being assessed and the causal effect being estimated. The assessment addresses bias within seven domains, through a series of 'signalling questions'. Domain-level judgements about risk of bias are derived from the answers to these questions, then combined to produce an overall risk of bias judgement for the result, together with judgements about the direction of bias. CONCLUSION: ROBINS-E provides a standardized framework for examining potential biases in results from cohort studies. Future work will produce variants of the tool for other epidemiologic study designs (e.g. case-control studies). We believe that ROBINS-E represents an important development in the integration of exposure assessment, evidence synthesis and causal inference.
AI snapshot
Written from the paper’s full text.
TL;DR
ROBINS-E is a new tool for assessing risk of bias in a specific exposure effect estimate from non-randomized cohort studies, developed by adapting ROBINS-I and RoB 2. It uses preliminary considerations, seven bias domains, signalling questions, and domain-level judgements to produce overall risk-of-bias, direction-of-bias, and threat-to-conclusions judgements.
Key findings
- ROBINS-E assesses risk of bias in a specific result from an observational cohort study of the causal effect of an exposure on an outcome, addressing seven domains: confounding, exposure measurement, selection of participants, post-exposure interventions, missing data, outcome measurement, and selection of the reported result.
- Development involved 43 researchers; proposals were discussed at a two-day meeting of 31 participants in Bristol in January 2017, and a draft was piloted in October 2019 by 28 in-person plus 3 remote participants; a revised draft was piloted in June–July 2021, and the launch version was posted in June 2022.
- Domain-level judgements include risk of bias (“Low risk of bias”, “Some concerns”, “High risk of bias”, “Very high risk of bias”), predicted direction of bias, and whether the bias threatens conclusions; the default overall risk-of-bias judgement is that of the domain with the greatest risk of bias.
- ROBINS-E does not directly assess financial conflicts of interest, and the present version addresses only follow-up (cohort) studies, with variants for other designs such as case-control studies planned.
Methods
- Design: iterative tool development through working groups, in-person meetings, webinars, and pilot testing, following principles used for RoB 2 and ROBINS-I.
- Data or sample: no empirical dataset was used; development involved 43 identified researchers, a 31-participant meeting in January 2017, a 2019 pilot with 28 in-person plus 3 remote participants, and a 2021 revised-draft pilot.
- Main analysis: not stated in the paper; the article describes the tool’s structure, domains, signalling questions, and judgement algorithms rather than conducting a statistical analysis.
Limitations
- Stated: the present version addresses only follow-up (cohort) studies, and work is ongoing to develop variants for other study designs including case-control studies.
- Stated: assessments require judgement, so subjectivity and between-assessor variability are impossible to avoid; accuracy depends on the quality of available information, and successful use requires both content and methodological expertise.
- Stated/evident: the tool has not yet had formal evaluation of its reliability and validity; the paper reports no empirical reliability or validity data, and the data availability statement says no data were used for the research described.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 27 cited sources checked · 0 supported · 0 flagged
- Findingcitation
All meta-epidemiological studies of bias identified by the working groups related to randomized trials.
“although all such studies identified related to randomized trials”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Sterne et al., 2002 source
Not checked in the free run.
- Findingreasoning
ROBINS-E does not include direct assessment of financial conflicts of interest.
“ROBINS-E does not include direct assessment of financial conflicts of interest, which may be important in interpreting the findings and credibility of studies of exposures and have been recommended for inclusion in evaluation of environmental studies (National Academies of Sciences Engineering and Medicine, 2022).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites National Academies of Sciences Engineering and Medicine, 2022
Not checked in the free run.
- Premisecitation
Different risk-of-bias tools, and different ways of addressing risk-of-bias assessments, can lead to differences in conclusions of systematic reviews.
“different tools, and different ways of addressing risk-of-bias assessments, can lead to differences in conclusions of systematic reviews”
Quote in the paperSubstantial claim0 of 2 sources checked - Premisecitation
Recently-developed tools for risk-of-bias assessment in observational epidemiologic studies have moved away from numeric scores based on checklists to tools based on assessments within specific domains of bias.
“Recently-developed tools for this purpose have moved away from numeric scores based on checklists (which often combined elements of risk of bias with other considerations such as of applicability) to tools based on assessments within specific domains of bias.”
Quote in the paperSubstantial claim0 of 3 sources checked- cites National Toxicology Program, 2015
Not checked in the free run.
- cites Woodruff and Sutton, 2014 source
Not checked in the free run.
- cites U.S. Environmental Protection Agency, 2022
Not checked in the free run.
- Premisecitation
Time-varying confounding can occur when the exposure changes over time and follow-up time is split according to exposure level.
“The second variant additionally addresses time-varying confounding, which can occur when the exposure changes over time and follow-up time is split according to exposure level (Hernán and Robins, 2020; Mansournia et al., 2017).”
Quote in the paperSubstantial claim0 of 2 sources checked- cites Hernán and Robins, 2020
Not checked in the free run.
- cites Mansournia et al., 2017
Not checked in the free run.
- Interpretationreasoning
ROBINS-E assessments may support causal inferences, if sufficient evidence at 'low' risk of bias or with 'some concerns' about risk of bias can be located.
“For this reason, ROBINS-E assessments may support causal inferences, if sufficient evidence at ‘low’ risk of bias or with ‘some concerns’ about risk of bias can be located.”
Quote in the paperSubstantial claim - Interpretationcitation
ROBINS-E can be used as part of a framework to rate the quality of the evidence for a group of studies, such as GRADE.
“ROBINS-E can be used as part of a framework to rate the quality of the evidence for a group of studies. An example of such a framework is the GRADE approach (Guyatt et al., 2011)”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Guyatt et al., 2011 source
Not checked in the free run.
- Interpretationnone given
It is impossible to exclude the possibility of uncontrolled confounding in a non-randomized study.
“It is impossible to exclude the possibility of uncontrolled confounding in a non-randomized study”
Quote in the paperNot a substantial claim - Interpretationreasoning
There are impacts of financial conflicts of interest that are not problems of internal validity but instead relate to what question is being asked in the study or analysis.
“Furthermore, there are impacts of such conflicts that are not problems of internal validity (targeted by ROBINS-E) but instead relate to what question is being asked in the study or analysis.”
Quote in the paperSubstantial claim - Interpretationcitation
The rationale for not including financial conflicts of interest in ROBINS-E is that if financial conflicts of interest lead to bias then this will operate through one of the domains already in the tool.
“The rationale, previously discussed in the context of randomized trials (Sterne, 2013), is that if financial conflicts of interest lead to bias then this will operate through one of the domains already in the tool.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Sterne, 2013 source
Not checked in the free run.
- Premisecitation
In settings with time-varying exposure, there can be confounding both by baseline factors and by time-varying prognostic factors that vary after cohort baseline and affect subsequent exposure during the exposure window.
“In these circumstances, there can be confounding both by baseline factors and by time-varying prognostic factors that vary after the cohort baseline and that affect subsequent exposure during the exposure window, known as ‘time-varying’ confounders.”
Quote in the paperSubstantial claim0 of 2 sources checked- cites Hernán and Robins, 2020
Not checked in the free run.
- cites Mansournia et al., 2017
Not checked in the free run.
- Premisecitation
It is possible to correct for bias due to post-exposure interventions by censoring at the time of the intervention and using inverse-probability-of-censoring weights.
“It is possible to correct for such bias, for example by censoring at the time of the intervention and using inverse-probability-of-censoring weights (Hernán and Robins, 2020).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Hernán and Robins, 2020
Not checked in the free run.
- Premisecitation
Conditioning on common effects of exposure and outcome distorts the association between exposure and outcome, compared with the association in the population in the absence of such conditioning.
“Conditioning on common effects distorts the association between exposure and outcome, compared with the association in the population in the absence of such conditioning.”
Quote in the paperSubstantial claim0 of 2 sources checked- cites Hernán and Robins, 2020
Not checked in the free run.
- cites Hernán et al., 2004 source
Not checked in the free run.
- May combine two claims. 9
- Classified as probably not a substantial claim. 37
May combine two claims.
Findingnone givenThe ROBINS-E instrument was based on the ROBINS-I tool for non-randomized studies of interventions and RoB 2 tool for randomized trials, and was informed by previous instruments for assessing studies of exposure.
“The instrument was based on the ROBINS-I tool for non-randomized studies of interventions (Sterne et al., 2016) and RoB 2 tool for randomized trials (Sterne et al., 2019), and was informed by previous instruments for assessing studies of exposure.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Sterne et al., 2016
Not checked in the free run.
- cites Sterne et al., 2019
Not checked in the free run.
Classified as probably not a substantial claim.
Findingnone givenROBINS-E is a new general-purpose tool for assessing risk of bias in results of non-randomized studies of exposure.
“ROBINS-E is a new general-purpose tool for assessing risk of bias in results of non-randomized studies of exposure.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Findingnone givenThe present version of ROBINS-E addresses only follow-up (cohort) studies.
“The present version addresses only follow-up (cohort) studies”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Findingnone givenImportant elements in ROBINS-E include a definition of the causal effect of interest, signalling questions to guide domain-level risk-of-bias judgements across seven domains, algorithms to derive suggested risk of bias judgements from answers to the signalling questions, and the separation of risk of bias from other important considerations in the evaluation of study sensitivity.
“Important elements in ROBINS-E include a definition of the causal effect of interest, signalling questions to guide domain-level risk-of-bias judgements across seven domains, algorithms to derive suggested risk of bias judgements from answers to the signalling questions, and the separation of risk of bias from other important considerations in the evaluation of study sensitivity.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingnone givenThese additional considerations, which fall outside the scope of the main ROBINS-E assessment, are addressed in an optional section of ROBINS-E.
“These additional considerations, which fall outside the scope of the main ROBINS-E assessment, are addressed in an optional section of ROBINS-E.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
FindingreasoningThe user may override the default overall risk-of-bias judgement.
“However, the user may override this, for example if they judge that so many domains were rated as at ‘High’ risk of bias that the overall judgement should be ‘Very high’ risk of bias.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Findingnone givenThe default ROBINS-E overall risk-of-bias judgement is that for the domain with the greatest risk of bias.
“The default ROBINS-E overall risk-of-bias judgement is that for the domain with the greatest risk of bias.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
FindingreasoningThe default judgement for the threat to conclusions is derived in a similar way to the overall risk-of-bias judgement; for example, the overall judgement is 'Yes' if there is considered to be a threat to conclusions in any of the seven bias domains.
“The default judgement for the threat to conclusions is derived in a similar way to the overall risk-of-bias judgement; for example, the overall judgement is ‘Yes’ if there is considered to be a threat to conclusions in any of the seven bias domains.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Findingnone givenROBINS-E provides a standardized framework for examining the range of biases common to observational studies of exposures.
“ROBINS-E provides a standardized framework for examining the range of biases common to observational studies of exposures.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Findingnone givenROBINS-E is designed primarily for use in the context of a systematic review.
“ROBINS-E is designed primarily for use in the context of a systematic review.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
FindingcitationThe collaborative development of the tool, between methodologists and stakeholders from fields such as environmental, occupational, chemical, clinical and public health, has highlighted the benefits of sharing ideas and research methods across communities.
“The collaborative development of the tool, between methodologists and stakeholders from fields such as environmental, occupational, chemical, clinical and public health, has highlighted the benefits of sharing ideas and research methods across communities (Hoffmann et al., 2022; Hoffmann et al., 2022).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Hoffmann et al., 2022
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingnone givenAn optional section of ROBINS-E allows users to record observations about study sensitivity, aspects of which may additionally feed into the "indirectness" domain within GRADE.
“An optional section of ROBINS-E allows users to record observations about study sensitivity, aspects of which may additionally feed into the “indirectness” domain within GRADE, as it relates to generalizability of the result of the study to the population, exposure and outcome of interest.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationSeveral tools for observational epidemiologic studies provide significant methodological advances over previously used tools.
“Several such tools for observational epidemiologic studies provide significant methodological advances over previously used tools (National Toxicology Program, 2015, 2015; Woodruff and Sutton, 2014; U.S. Environmental Protection Agency, 2022).”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites National Toxicology Program, 2015
Not checked in the free run.
- cites Woodruff and Sutton, 2014 source
Not checked in the free run.
- cites U.S. Environmental Protection Agency, 2022
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationEarly work by a subset of the authors considered the extent to which the ROBINS-I tool was suitable for observational studies of exposures and suggested modifications to the tool.
“Early work by a subset of the authors considered the extent to which the ROBINS-I tool was suitable for observational studies of exposures and suggested modifications to the tool (Morgan et al., 2018; Morgan et al., 2019).”
Quote in the paperNot a substantial claim0 of 2 sources checkedClassified as probably not a substantial claim.
PremisecitationThe ROBINS-I tool was developed to evaluate estimates of the effects of deliberate interventions aimed to alter health outcomes.
“The Risk Of Bias In Non-randomized Studies - of Interventions (ROBINS-I) tool was developed to evaluate estimates of the effects of deliberate interventions aimed to alter health outcomes (Sterne et al., 2016).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Sterne et al., 2016
Not checked in the free run.
Classified as probably not a substantial claim. May combine two claims.
InterpretationreasoningDomain-level judgements about risk of bias are derived from the answers to signalling questions and combined to produce an overall risk of bias judgement for the result, together with judgements about the direction of bias.
“Domain-level judgements about risk of bias are derived from the answers to these questions, then combined to produce an overall risk of bias judgement for the result, together with judgements about the direction of bias.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Interpretationnone givenThe extent of the potential for bias across a body of evidence for an exposure-outcome effect informs assessment of the certainty of evidence for that effect.
“The extent of the potential for bias across a body of evidence for an exposure-outcome effect informs assessment of the certainty of evidence for that effect.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
InterpretationreasoningA screening section facilitates identification of results that are at very high risk of bias, allowing the user to avoid a detailed risk of bias assessment.
“A screening section then facilitates identification of results that are at very high risk of bias, allowing the user to avoid a detailed risk of bias assessment (Fig. 1, part B).”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based).
Premisenone givenThere is a risk of bias when such selection is based on the magnitude, direction or P value of the result.
“There is a risk of bias when such selection is based on the magnitude, direction or P value of the result.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Interpretationnone givenEvaluation of risk of bias requires that the review team has expertise in both content and methodology.
“First, evaluation of risk of bias requires that the review team has expertise in both content (detailed subject-matter knowledge) and methodology (understanding of the different types of bias that may distort the results of observational studies).”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Premisenone givenBias due to post-exposure interventions may arise if the presence or level of exposure leads to administration of interventions that change the subsequent risk of the outcome.
“Bias due to post-exposure interventions may arise if the presence or level of exposure leads to administration of interventions that change the subsequent risk of the outcome.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based).
Premisenone givenBias in selection of the reported result arises when study authors select results from a multiplicity of analyses.
“Bias in selection of the reported result arises when study authors select results from a multiplicity of analyses, for example from different ways of measuring the exposure, different ways of measuring the outcome, different subsets of the full study sample or different analyses.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Interpretationnone givenThe ROBINS-E risk-of-bias assessment addresses seven domains of bias.
“The ROBINS-E risk-of-bias assessment addresses seven domains of bias (Fig. 1, Section 5).”
Quote in the paperNot a substantial claim- The authors state this as an assumption: Issues common to both randomized trials and observational studies (missing data, outcome assessment, selection of the reported result) are likely to be similar.
Probably not a substantial claim: may not pass S3 (load-bearing).
InterpretationreasoningIt is possible to envisage situations in which risk of bias for domains other than confounding is low, for example if exposure is measured without error, there are no missing data, or results are reported in accordance with a pre-specified analysis plan.
“By contrast, it is possible to envisage situations in which risk of bias for other domains is low: for example, if exposure is measured without error, there are no missing data, or results are reported in accordance with a pre-specified analysis plan.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Interpretationnone givenA series of preliminary considerations informs the core ROBINS-E assessment, including details of the result being assessed and the causal effect being estimated.
“A series of preliminary considerations informs the core ROBINS-E assessment, including details of the result being assessed and the causal effect being estimated.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
InterpretationreasoningSuch analyses require that the factors leading to the post-exposure intervention can be modelled.
“However, such analyses require that the factors leading to the post-exposure intervention can be modelled.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenROBINS-E was iteratively developed based on the results of pilot testing with a diverse group.
“Although ROBINS-E was iteratively developed based on the results of pilot testing with a diverse group”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
Premisenone givenBias arising from measurement of the outcome is a problem when error in measurement of the outcome is related to the presence or level of exposure, in which case the measurement error is differential rather than non-differential.
“Bias arising from measurement of the outcome is a problem when error in measurement of the outcome is related to the presence or level of exposure, in which case the measurement error is differential rather than non-differential.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
InterpretationreasoningROBINS-E assessments require judgements based on the available information, and it is impossible to avoid subjectivity and between-assessor variability.
“Third, ROBINS-E assessments require judgements based on the available information. It is therefore impossible to avoid subjectivity and between-assessor variability.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
InterpretationreasoningROBINS-E provides a standardized framework for examining potential biases in results from cohort studies.
“ROBINS-E provides a standardized framework for examining potential biases in results from cohort studies.”
Quote in the paperSubstantial claim- The authors state this as an assumption: Issues common to both randomized trials and observational studies (missing data, outcome assessment, selection of the reported result) are likely to be similar.
May combine two claims.
InterpretationreasoningThe question being asked in a study can be manipulated to give an illusion of presence or lack of effect, but this need not have implications for internal validity.
“This question can be manipulated to give an illusion of presence or lack or effect (for example by studying trivial levels of exposure or irrelevant outcomes), but this need not have implications for internal validity.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
InterpretationreasoningPractical considerations can impact on the successful implementation of ROBINS-E.
“Practical considerations can impact on the successful implementation of ROBINS-E.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Interpretationnone givenEach ROBINS-E domain is addressed using a series of signalling questions that aim to gather relevant information about the study and analysis being assessed.
“Each domain is addressed using a series of signalling questions that aim to gather relevant information about the study and analysis being assessed.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Interpretationnone givenThe first version of ROBINS-E targets cohort (follow-up) studies.
“The first version of ROBINS-E, described here, targets cohort (follow-up) studies.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenROBINS-E was developed by a large group of researchers from diverse research and public health disciplines through a series of working groups, in-person meetings and pilot testing phases.
“ROBINS-E was developed by a large group of researchers from diverse research and public health disciplines through a series of working groups, in-person meetings and pilot testing phases.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
InterpretationreasoningBecause risk of bias is assessed in relation to a defined causal effect of an exposure on an outcome, a low risk of bias corresponds to an absence of bias in estimating that causal effect.
“Because risk of bias is assessed in relation to a defined causal effect of an exposure on an outcome, a low risk of bias corresponds to an absence of bias in estimating that causal effect.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).
Interpretationnone givenPublication of answers to signalling questions and justification for these should make risk of bias judgements transparent.
“However, publication of answers to signalling questions and justification for these should make risk of bias judgements transparent.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
PremisecitationBias in selection of participants into the study or into the analysis arises when we condition, in the study design or in the analysis, on common effects of exposure and outcome.
“Bias in selection of participants into the study (or into the analysis) is the first of two domains that address selection bias, which arises when we condition, in the study design or in the analysis, on common effects of exposure and outcome (Hernán and Robins, 2020; Hernán et al., 2004).”
Quote in the paperSubstantial claim0 of 2 sources checked- cites Hernán and Robins, 2020
Not checked in the free run.
- cites Hernán et al., 2004 source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Interpretationnone givenThe accuracy of a risk-of-bias assessment will depend on the quality of information available about the study being assessed.
“Second, the accuracy of a risk-of-bias assessment will depend on the quality of information available about the study being assessed.”
Quote in the paperNot a substantial claim
The paper relies on this without support.
Randomized controlled trials are often not feasible, ethical, large enough or of sufficient duration to evaluate chronic health effects.
“Randomized controlled trials (RCTs) are often not feasible, ethical, large enough or of sufficient duration to evaluate chronic health effects.”
The authors state this as an assumption.
Issues common to both randomized trials and observational studies (missing data, outcome assessment, selection of the reported result) are likely to be similar.
“with the additional assumption that issues common to both study designs (missing data, outcome assessment, selection of the reported result) are likely to be similar”
The paper relies on this without support.
Pre-specified plans for analyses are uncommon for observational studies.
“However, such pre-specified plans are uncommon for observational studies and often the nature of the data need to be taken into account when selecting a suitable statistical model and method.”
The paper relies on this without support.
The nature of the data often needs to be taken into account when selecting a suitable statistical model and method.
“However, such pre-specified plans are uncommon for observational studies and often the nature of the data need to be taken into account when selecting a suitable statistical model and method.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The Risk of Bias in Non-randomized Studies of Exposures (ROBINS-E) tool was used for cohort studies.
“For cohort studies, the Risk of Bias in Non‐randomized Studies of Exposures (ROBINS‐E) tool was used.”
From Risk of cardiac, neuropsychiatric and musculoskeletal adverse events with levothyroxine: Systematic review (2026) · cites this paper as [12]
Cites
RoB 2: a revised tool for assessing risk of bias in randomised trials
Sterne, J. A. C., Savović, J., Page, M. J., Elbers, R. G., Blencowe, N. S., Boutron, I., Cates, C. J., Cheng, H.-Y., Corbett, M. S., Eldridge, S. M., et al. (2019) · BMJ · doi:10.1136/bmj.l4898 · Cited by 33,583 · see citing works
Self-citationDermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,577 · see citing works
GRADE: an emerging consensus on rating quality of evidence and strength of recommendations
Guyatt, G. H., Oxman, A. D., Vist, G. E., Kunz, R., Falck-Ytter, Y., Alonso-Coello, P., & Schünemann, H. J. (2008) · BMJ · doi:10.1136/bmj.39489.470347.ad · Cited by 23,305 · cited 1× · see citing works
“…of interest; measurement of exposure; measurement of outcomes; attrition and selective reporting; integration of ROBINS-E with the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) framework (Guyatt et al., 2008); and issues in statistical analysis (see Online Supplement).”
Self-citationROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329 · see citing works
Self-citationCochrane Handbook for Systematic Reviews of Interventions
(2019) · doi:10.1002/9781119536604 · Cited by 14,828 · see citing works
GRADE guidelines: 1. Introduction—GRADE evidence profiles and summary of findings tables
Guyatt, G., Oxman, A. D., Akl, E. A., Kunz, R., Vist, G., Brozek, J., Norris, S., Falck-Ytter, Y., Glasziou, P., & deBeer, H. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.04.026 · Cited by 10,681 · cited 1× · see citing works
“An example of such a framework is the GRADE approach (Guyatt et al., 2011) where users may start at high certainty in the “study limitations” (risk of bias) domain of GRADE and use their ROBINS-E evaluation to determine how many levels to rate down this certainty (Schunemann et al., 2018).”
Self-citationA structural approach to selection bias
Hernán, M. A., Hernández-Díaz, S., & Robins, J. M. (2004) · Epidemiology · doi:10.1097/01.ede.0000135174.63482.43 · Cited by 2,844 · cited 1× · see citing works
“Bias in selection of participants into the study (or into the analysis) is the first of two domains that address selection bias, which arises when we condition, in the study design or in the analysis, on common effects of exposure and outcome (Hernán and Robins, 2020; Hernán et al., 2004).”
Triangulation in aetiological epidemiology
Lawlor, D. A., Tilling, K., & Davey Smith, G. (2017) · International Journal of Epidemiology · doi:10.1093/ije/dyw314 · Cited by 1,129 · cited 1× · see citing works
“…together in a triangulation exercise, in which both the risks EPA A u t ho r M anu sc r i p t EPA A u t ho r M anu sc r i p t EPA A u t ho r M anu sc r i p t of bias (assessed by ROBINS-E) and differences between causal questions being addressed are considered jointly (Lawlor et al., 2016).”
Self-citationSchünemann, H. J., Cuello, C., Akl, E. A., Mustafa, R. A., Meerpohl, J. J., Thayer, K., Morgan, R. L., Gartlehner, G., Kunz, R., Katikireddi, S. V., et al. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.01.012 · Cited by 911 · cited 1× · see citing works
“An example of such a framework is the GRADE approach (Guyatt et al., 2011) where users may start at high certainty in the “study limitations” (risk of bias) domain of GRADE and use their ROBINS-E evaluation to determine how many levels to rate down this certainty (Schunemann et al., 2018).”
Self-citationWoodruff, T. J., & Sutton, P. (2014) · Environmental Health Perspectives · doi:10.1289/ehp.1307175 · Cited by 427 · cited 1× · see citing works
“Several such tools for observational epidemiologic studies provide significant methodological advances over previously used tools (National Toxicology Program, 2015, 2015; Woodruff and Sutton, 2014; U.S. Environmental Protection Agency, 2022).”
Sterne, J. A. C., Jüni, P., Schulz, K. F., Altman, D. G., Bartlett, C., & Egger, M. (2002) · Statistics in Medicine · doi:10.1002/sim.1184 · Cited by 382 · cited 1× · see citing works
“The groups considered empirical evidence of bias where this was available, for example from meta-epidemiological studies (Sterne et al., 2002), although all such studies identified related to randomized trials.”
Self-citationHandling time varying confounding in observational research
Mansournia, M. A., Etminan, M., Danaei, G., Kaufman, J. S., & Collins, P. G. S. (2017) · BMJ · doi:10.1136/bmj.j4587 · Cited by 343 · cited 1× · see citing works
“The second variant additionally addresses time-varying confounding, which can occur when the exposure changes over time and follow-up time is split according to exposure level (Hernán and Robins, 2020; Mansournia et al., 2017).”
Morgan, R. L., Thayer, K. A., Santesso, N., Holloway, A. C., Blain, R., Eftim, S. E., Goldstone, A. E., Ross, P., Ansari, M., Akl, E. A., et al. (2019) · Environment International · doi:10.1016/j.envint.2018.11.004 · Cited by 248 · cited 1× · see citing works
“Early work by a subset of the authors considered the extent to which the ROBINS-I tool was suitable for observational studies of exposures and suggested modifications to the tool (Morgan et al., 2018; Morgan et al., 2019).”
Self-citationMorgan, R. L., Thayer, K. A., Santesso, N., Holloway, A. C., Blain, R., Eftim, S. E., Goldstone, A. E., Ross, P., Guyatt, G., & Schünemann, H. J. (2018) · Environment International · doi:10.1016/j.envint.2018.08.018 · Cited by 106 · cited 1× · see citing works
“Early work by a subset of the authors considered the extent to which the ROBINS-I tool was suitable for observational studies of exposures and suggested modifications to the tool (Morgan et al., 2018; Morgan et al., 2019).”
Self-citationEick, S. M., Goin, D. E., Chartres, N., Lam, J., & Woodruff, T. J. (2020) · Systematic Reviews · doi:10.1186/s13643-020-01490-8 · Cited by 82 · cited 2× · see citing works
“Comparisons between these tools have been published previously (National Toxicology Program, 2015; Steenland et al., 2020; Eick et al., 2020; Radke et al., 2021; Eick et al., 2022) and have observed that different tools, and different ways of addressing risk-of-bias assessments, can lead to…”
Steenland, K., Schubauer‐Berigan, M. K., Vermeulen, R. C. H., Lunn, R. M., Straif, K., Zahm, S. H., Stewart, P. A., Arroyave, W. D., Mehta, S., & Pearce, N. E. (2020) · Environmental Health Perspectives · doi:10.1289/ehp6980 · Cited by 75 · cited 1× · see citing works
“Comparisons between these tools have been published previously (National Toxicology Program, 2015; Steenland et al., 2020; Eick et al., 2020; Radke et al., 2021; Eick et al., 2022) and have observed that different tools, and different ways of addressing risk-of-bias assessments, can lead to…”
Why the Cochrane Risk of Bias Tool Should not Include Funding Source as a Standard Item
Sterne, J. A. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.ed000076 · Cited by 60 · cited 1× · see citing works
“The rationale, previously discussed in the context of randomized trials (Sterne, 2013), is that if financial conflicts of interest lead to bias then this will operate through one of the domains already in the tool.”
Self-citationApplication of evidence-based methods to construct mechanism-driven chemical assessment frameworks
Hoffmann, S. (2022) · ALTEX · doi:10.14573/altex.2202141 · Cited by 30 · cited 1× · see citing works
“…collaborative development of the tool, between methodologists and stakeholders from fields such as environmental, occupational, chemical, clinical and public health, has highlighted the benefits of sharing ideas and research methods across communities (Hoffmann et al., 2022; Hoffmann et al., 2022).”
Confounders and co-interventions identified in non-randomized studies of interventions
Pufulete, M., Mahadevan, K., Johnson, T., et al. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2022.03.018 · Cited by 10 · see citing works
Self-citationHoffmann, S. (2022) · ALTEX · doi:10.14573/altex.2201131 · Cited by 8 · see citing works
Radke, E. G., Glenn, B. S., & Kraft, A. D. (2021) · Systematic Reviews · doi:10.1186/s13643-021-01783-6 · Cited by 3 · cited 2× · see citing works
“Comparisons between these tools have been published previously (National Toxicology Program, 2015; Steenland et al., 2020; Eick et al., 2020; Radke et al., 2021; Eick et al., 2022) and have observed that different tools, and different ways of addressing risk-of-bias assessments, can lead to…”
Self-citationEick, S. M., Goin, D. E., Lam, J., Woodruff, T. J., & Chartres, N. (2022) · Systematic Reviews · doi:10.1186/s13643-022-01894-8 · Cited by 2 · cited 1× · see citing works
“…these tools have been published previously (National Toxicology Program, 2015; Steenland et al., 2020; Eick et al., 2020; Radke et al., 2021; Eick et al., 2022) and have observed that different tools, and different ways of addressing risk-of-bias assessments, can lead to differences in…”
Flagged first, then by citations
8 of the 30 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- Boutron, 2019, Chapter 7: considering bias and conflicts of interest among the included studies, 177
- National Academies of Sciences Engineering and Medicine, 2022. A Review of U.S. EPA’s ORD Staff Handbook for Developing IRIS Assessments: 2020 Version. Washington DC: The National Academies Press.
Cited by
Sorted by citations
AI Chatbots for Mental Health: A Scoping Review of Effectiveness, Feasibility, and Applications
Casu, M., Triscari, S., Battiato, S., Guarnera, L., & Caponnetto, P. (2024) · Applied Sciences · doi:10.3390/app14135889 · Cited by 205 · see citing works
“The methodological quality of the included studies was assessed using the Cochrane risk-of-bias tool for randomized trials, version 2 (RoB 2) [54], and the Cochrane Risk Of Bias In Non-randomized Studies—of Exposure (ROBINS-E) [55].”
Associations between sexual health and well-being: a systematic review
Vasconcelos, P. A., Carrito, M. L., Quinta-Gomes, A. L., Patrão, A. L., Nóbrega, C. A., Costa, P. A., & Nobre, P. J. (2024) · Bulletin of the World Health Organization · doi:10.2471/blt.24.291565 · Cited by 119 · see citing works
“We used the Risk Of Bias In Non-ran-domized Studies - of Exposures tool 31 to assess the risk of bias in the included studies, as the tool indicates whether the risk of bias is substantial enough to question the impact of the exposure on the outcome.”
Life's Essential 8 and the risk of cardiovascular disease: a systematic review and meta-analysis
Sebastian, S. A., Shah, Y., Paul, H., & Arsene, C. (2024) · European Journal of Preventive Cardiology · doi:10.1093/eurjpc/zwae280 · Cited by 84 · see citing works
The Applications of Artificial Intelligence for Assessing Fall Risk: Systematic Review
González-Castro, A., Leirós‐Rodríguez, R., Prada‐García, C., & Benítez‐Andrades, J. A. (2024) · Journal of Medical Internet Research · doi:10.2196/54934 · Cited by 64 · see citing works
Luo, D., Dashti, S. G., Sawyer, S. M., & Vijayakumar, N. (2024) · EClinicalMedicine · doi:10.1016/j.eclinm.2024.102828 · Cited by 59 · see citing works
“The Risk of Bias in Non-randomized Studies of Expo-sures (ROBINS-E) tool 26,27 provides a structured approach to assessing the risk of bias in observational epidemiological studies evaluating the effect of an exposure on an outcome.”
BackgroundStorck, W., Elbaz, M., Vindis, C., Deguilhem, A., Lapeyre‐Mestre, M., & Jouanjus, É. (2025) · Heart · doi:10.1136/heartjnl-2024-325429 · Cited by 46 · see citing works
“19 The latter risk is approached systematically in seven distinct domains, that is, (1) bias due to confounding (D1), (2) bias arising from measurement of the exposure (D2), (3) bias in selection of participants into the study (or into the analysis) (D3), (4) bias due to post-exposure interventions…”
Methodological Standards for Conducting High-Quality Systematic Reviews
CASSAI, A. D., Dost, B., Tulgar, S., & Boscolo, A. (2025) · Biology · doi:10.3390/biology14080973 · Cited by 67 · see citing works
“…In Non-randomized Studies—of Interventions) [29] Non-Randomized Intervention Studies ROBINS-E (Risk Of Bias In Non-randomized Studies—of Exposures) [30] Observational Exposure Studies (Cohort, Case-Control) Newcastle-Ottawa Scale (NOS) [31] Cohort and Case-Control Studies QUADAS-2 (Quality…”
Carbone, F., Spinelli, A., Ciardiello, D., Luc, M. R., Pascale, S. D., Bertani, E., Fazio, N., & Romario, U. F. (2024) · European Journal of Cancer · doi:10.1016/j.ejca.2024.115172 · Cited by 43 · see citing works
Li, Y., Smith, R., Whitney, S. L., Seemungal, B. M., & Ellmers, T. J. (2024) · Age and Ageing · doi:10.1093/ageing/afae177 · Cited by 47 · see citing works
“We assessed the quality of included studies using the Risk of Bias in Non-randomised Studies of Exposures (ROBINS-E) tool [22] and visualised the assessments using ‘robvis’ [23].”
MethodAcevedo, P., Magana, A. J., Beneš, B., & Mousas, C. (2024) · IEEE Access · doi:10.1109/access.2024.3489233 · Cited by 43 · see citing works
“For non-randomized studies that focused on validating interventions rather than conducting comparison studies, we used the ROBINS-E (Risk of Bias in Non-randomized Studies of Exposures) tool [37].”
Cardiac rehabilitation and frailty: a systematic review and meta-analysis
MacEachern, E., Quach, J., Giacomantonio, N., Theou, O., Hillier, T. W. R., Abel-Adegbite, I., González-Lara, M., & Kehler, D. S. (2024) · European Journal of Preventive Cardiology · doi:10.1093/eurjpc/zwae239 · Cited by 39 · see citing works
Elangovan, S., Lo, J. J.-M., Xie, Y., et al. (2024) · Journal of Hospital Infection · doi:10.1016/j.jhin.2024.08.002 · Cited by 37 · see citing works
From social media to artificial intelligence: improving research on digital harms in youth
Mansfield, K. L., Ghai, S., Hakman, T., Ballou, N., Vuorre, M., & Przybylski, A. K. (2025) · The Lancet Child & Adolescent Health · doi:10.1016/s2352-4642(24)00332-8 · Cited by 46 · see citing works
The changing prevalence of ADHD? A systematic review
Martin, A. F., Rubin, G. J., Rogers, M. B., Wessely, S. C., Greenberg, N. L., Hall, C. E., Pitt, A., Logan, P. E., Lucas, R., & Brooks, S. K. (2025) · Journal of Affective Disorders · doi:10.1016/j.jad.2025.119427 · Cited by 39 · see citing works
Iqbal, S., Ramini, A., & Kaja, S. (2024) · The Ocular Surface · doi:10.1016/j.jtos.2024.12.003 · Cited by 32 · see citing works
“For non-randomized clinical studies, the Risk-of-Bias In Non-Randomized Studies of Exposure (ROBINS-E) tool was used for observational studies [12].”
Nordon, C., Simons, S. O., Marshall, J., Müllerová, H., Pollack, M. F., Bengtsson, C., Hoti, F. J., Lobier, M., Salosensaari, A., SANTOS, A. C. D. S., et al. (2024) · ERJ Open Research · doi:10.1183/23120541.01091-2024 · Cited by 33 · see citing works
“Risk of bias in each individual study was assessed according to the seven bias domains within the Risk Of Bias In Non-randomised Studies – of Exposure (ROBINS-E) tool [15].”
Baecker, L., Iyengar, U., Piccolo, M. C. D., & Mechelli, A. (2025) · The Journal of Climate Change and Health · doi:10.1016/j.joclim.2025.100446 · Cited by 29 · see citing works
Deprato, A., Garud, A., Azzolina, D., Murgia, N., Davenport, M. H., Kaul, P., Lacy, P., & Moitra, S. (2024) · Journal of Hazardous Materials · doi:10.1016/j.jhazmat.2024.137028 · Cited by 29 · see citing works
Udom, G. J., Turyahabwe, B., Aturamu, A., Aziakpono, O. M., Agbana, R. D., Joseph, O. G., Udom, N.-W. G., Mugide, N., Odey, O. P., Olot, H., et al. (2025) · Environmental Advances · doi:10.1016/j.envadv.2025.100620 · Cited by 30 · see citing works
Xie, L., Ma, W., Du, K., et al. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1525277 · Cited by 27 · see citing works
Tan, L., Stagg, L., Hanlon, E., Li, T., McGrattan, A., Siervo, M., Matu, J., Griffiths, A., & Shannon, O. M. (2024) · Nutrients · doi:10.3390/nu16101511 · Cited by 27 · see citing works
“…arising from confounding; (2) resulting from exposure measurement; (3) from selection of participants into the study/analysis; (4) arising from post-exposure interventions; (5) arising from missing data; (6) resulting from outcome measurement and (7) from selection of the reported result” [26].”
BackgroundMethodTerry, P., Heidel, R. E., Wilson, A. Q., et al. (2025) · BMJ Open Respiratory Research · doi:10.1136/bmjresp-2024-002528 · Cited by 24 · see citing works
“The risk of bias was more formally assessed using the ROBINS-E (ROB, risk of bias in non-randomised studies of exposure), 65 and the resulting table is shown in online supplemental appendix 2.”
Allostatic Load and Cardiovascular Disease: A Systematic Review
Evans, E., Jacobs, M., Fuller, D. D., Wheeler‐Hegland, K., & Ellis, C. (2025) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2025.02.016 · Cited by 24 · see citing works
Corte, K. W. D., Bosler, T., McClure, C., Buyken, A. E., LeCheminant, J. D., Schwingshackl, L., & Corte, D. D. (2025) · Advances in Nutrition · doi:10.1016/j.advnut.2025.100413 · Cited by 27 · see citing works
“To improve the validity of the fi ndings, potential risks of bias were assessed using the ROBINS-E (Risk of Bias in Non-randomized Studies of Exposures) tool [24,25].”
Vitale, E., Maistrello, L., Ciccimarra, F., Rizzo, A., Soares, A., Brunetti, O., Mollica, V., Maiorano, B. A., Maiorano, M. F. P., Monteiro, F. S. M., et al. (2026) · Cancer and Metastasis Reviews · doi:10.1007/s10555-026-10327-0 · Cited by 33 · see citing works
Pirera, E., Raimondo, D. D., D’Anna, L., & Tuttolomondo, A. (2025) · European Journal of Internal Medicine · doi:10.1016/j.ejim.2025.01.016 · Cited by 27 · see citing works
Antonini, M., Mattar, A. P. L., Pereira, T. P., Oliveira, L. L., Teixeira, M. D., Amorim, A. G., Ferraro, O., de Oliveira, L. C., Ramos, M. D. N. M., Cavalcante, F. P., et al. (2025) · Heliyon · doi:10.1016/j.heliyon.2025.e43069 · Cited by 24 · see citing works
Climate Conditions, Weather Changes, and Air Pollutants and Atopic Dermatitis
Park, M., Mohsen, S. T., Katz, T., Sood, S., Sy, S. M. T., Rochwerg, B. N., & Drucker, A. M. (2025) · JAMA Dermatology · doi:10.1001/jamadermatol.2025.1790 · Cited by 27 · see citing works
Evidence update on the cancer risk of vaping e-cigarettes: A systematic review
Kundu, A., Sachdeva, K., Feore, A., Sanchez, S., Sutton, M., Seth, S., Schwartz, R. M., & Chaiton, M. (2025) · Tobacco Induced Diseases · doi:10.18332/tid/192934 · Cited by 23 · see citing works
Green, K. L., Yang, Y., Abaraogu, U. O., Eastaugh, C., Beyer, F. R., Norman, G., & Todd, C. J. (2024) · Age and Ageing · doi:10.1093/ageing/afae104 · Cited by 21 · see citing works
Martínez‐Vizcaíno, V., Fernández‐Rodríguez, R., Reina‐Gutiérrez, S., Rodríguez‐Gutiérrez, E., Garrido‐Miguel, M., de Arenas‐Arroyo, S. N., & Torres-Costoso, A. I. (2024) · BMC Public Health · doi:10.1186/s12889-024-19383-z · Cited by 22 · see citing works
“The risk of bias assessment results obtained using the ROBINS-E tool [34], are shown in Supplementary Table S4.”
MethodDiep, C., Patel, K., Petricca, J., et al. (2024) · Anaesthesia · doi:10.1111/anae.16398 · Cited by 21 · see citing works
Comparing cancer stage at diagnosis between migrants and non-migrants: a meta-analysis
Harvey-Sullivan, A., Ali, S., Dhesi, P., et al. (2024) · British Journal of Cancer · doi:10.1038/s41416-024-02896-0 · Cited by 21 · see citing works
“41 papers met the inclusion criteria for this systematic review (Fig.”
González, I. B., Montero, J., Gómez‐Polo, C., & Pardal-Peláez, B. (2025) · Journal of Dentistry · doi:10.1016/j.jdent.2025.105707 · Cited by 31 · see citing works
Cosgrave, N., Frydenlund, J., Beirne, F., Lee, S. J., Faez, I., Cahir, C., & Williams, D. J. (2025) · Age and Ageing · doi:10.1093/ageing/afaf231 · Cited by 23 · see citing works
“This tool evaluates the ROB across seven domains: (i) confounders, (ii) participants, (iii) exposure measurement, (iv) post-exposure interventions, (v) missing data, (vi) outcome measurement and (vii) reporting [21].”
Zainal, N. H., Wang, V. Y., Garthwaite, B., & Curtiss, J. E. (2025) · Health Psychology Review · doi:10.1080/17437199.2025.2547610 · Cited by 21 · see citing works
“Fourth, with 51.3% of studies showing high bias, future work should adopt more rigorous standards (e.g., preregistration, transparent preprocessing, randomisation; Harrison et al., 2017; Higgins et al., 2024).”
InfluentialVitale, E., Rizzo, A., Maistrello, L., Cauli, O., Brunetti, O., Monteiro, F. S. M., Soares, A., Santoni, M., Mollica, V., & Massari, F. (2025) · Drug Design Development and Therapy · doi:10.2147/dddt.s560518 · Cited by 33 · see citing works
“42 The tool consisted of seven sub dimensions exploring the presence of bias in the obtained results.”
Petrucci, G., Papalia, G., Ambrosio, L., Russo, F., Marchetti, A., Marinis, M. D. D., Vadalà, G., Papalia, R., & Denaro, V. (2025) · European spine journal · doi:10.1007/s00586-025-08733-z · Cited by 24 · see citing works
The Role of Biomarkers in Temporomandibular Disorders: A Systematic Review
Soares, J. M., Carneiro, B. D., & Pozza, D. H. (2025) · International Journal of Molecular Sciences · doi:10.3390/ijms26135971 · Cited by 20 · see citing works
“…the Cochrane RoB 2 tool [18] at the outcome level visualized with the Cochrane risk of bias VISualization app 4.0 [19] and the risk of bias of other study types was evaluated with the ROBINS-E tool [20], also at the outcome level visualized with the Cochrane risk of bias VISualization app 4.0 [19].”
InfluentialMapping Worldwide Antibiotic Use in Dental Practices: A Scoping Review
Soleymani, F., Martı́nez, C. P., Makiabadi, M., et al. (2024) · Antibiotics · doi:10.3390/antibiotics13090859 · Cited by 19 · see citing works
“This research identified a slight decreasing trend in the DDD of antibiotics prescribed by dental professionals in the studied countries, with a notable increase during the COVID-19 pandemic.”
ResultGrubic, N., Johnston, A., Randhawa, V. K., Humphries, K. H., Rosella, L. C., & Maximova, K. (2024) · Canadian Journal of Cardiology · doi:10.1016/j.cjca.2024.12.022 · Cited by 20 · see citing works
Anxiety and Depression in Adults With Vestibular Disorders: A Systematic Review and Meta‐Analysis
Kim, C. H., McCray, L. R., Nguyen, S. A., Staab, J. P., Jafri, S. H., & Rizk, H. G. (2025) · The Laryngoscope · doi:10.1002/lary.70055 · Cited by 19 · see citing works
“The risk of bias was assessed according to the Risk of Bias In Non-randomized Studies—of Exposure (ROBINS-E) [17] for cohort studies and the Joanna Briggs Institute (JBI) critical appraisal checklist for cross-sectional and case–control studies [18, 19].”
Kiesswetter, E., Neuenschwander, M., Stadelmaier, J., Szczerba, E., Hofacker, L., Sedlmaier, K., Kussmann, M., Roeger, C., Hauner, H., Schlesinger, S., et al. (2024) · Current Developments in Nutrition · doi:10.1016/j.cdnut.2024.102159 · Cited by 21 · see citing works
“We used the Risk Of Bias In Non-randomized Studies - of Exposure (ROBINS-E) tool to evaluate the risk of bias [21].”
Grassi, G. M., Quarti‐Trevano, F., Cuspidi, C., Sanidas, E. A., Mancia, G., & Thomopoulos, C. (2025) · Hypertension · doi:10.1161/hypertensionaha.125.24749 · Cited by 20 · see citing works
Grasa-Ciria, D., Couto-Alfonso, S., Samatán, E., Martínez‐Jarreta, B., Cénit, M. C., & Iguacel, I. (2025) · Nutrients · doi:10.3390/nu17172894 · Cited by 18 · see citing works
“The methodological quality and risk of bias of the included studies were assessed using two complementary tools: the ROBINS-E (Risk Of Bias In Non-randomized Studies—of Exposures) tool [39] and the NIH Quality Assessment Tool [40] (Supplementary Materials S3 and S4).”
Pouliopoulou, D. V., Hawthorne, M., MacDermid, J. C., Billias, N., Miller, E., Quinn, K. L., Décary, S., Razak, F. A., Cheung, A. M., Galiatsatos, P., et al. (2025) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2025.01.471 · Cited by 19 · see citing works
Evaluation of dietary protein and amino acid requirements: a systematic review
Burstad, K. M., Lamina, T., Erickson, A., Gholizadeh, E., Namigga, H., Claussen, A. M., Slavin, J. L., Teigen, L. M., Gallant, K. M. H., Stang, J. L., et al. (2025) · American Journal of Clinical Nutrition · doi:10.1016/j.ajcnut.2025.04.017 · Cited by 19 · see citing works
Donaldson, S. I., Russell, A. M., Capria, K. L., DeJesus, A., Wang, E., Fayad, J., & Allem, J.-P. (2025) · The Lancet Public Health · doi:10.1016/s2468-2667(25)00219-1 · Cited by 20 · see citing works
Rimmer, P. J., Zhang, F., Scott, G., Kaakoush, N. O., Lewindon, P. J., Ashton, J., Kansal, S., Limbergen, J. V., Boneh, R. S., Pai, N. N., et al. (2025) · Gastroenterology · doi:10.1053/j.gastro.2025.09.014 · Cited by 21 · see citing works
A systematic review and meta-analysis of disease clusters in multimorbidity
Ferris, J. K., Fiedeldey, L., Kim, B., Clemens, F., Irvine, M. A., Hosseini, S. H., Smolina, K., & Wister, A. (2025) · Nature Communications · doi:10.1038/s41467-025-67372-6 · Cited by 18 · see citing works
Karam, S. G., Zhang, Y., Hernandez, H. P., Siebert, U., Koopman, L., Noyes, J., Tarride, J., Stevens, A., Welch, V. A., Saz‐Parkinson, Z., et al. (2024) · BMJ · doi:10.1136/bmj-2024-079890 · Cited by 19 · see citing works
Self-citationWallerer, S., Papakonstantinou, T., Morze, J., Stadelmaier, J., Kiesswetter, E., Gorenflo, L., Barbaresko, J., Szczerba, E., Neuenschwander, M., Bell, W., et al. (2024) · EClinicalMedicine · doi:10.1016/j.eclinm.2024.102807 · Cited by 19 · see citing works
“We used the Risk Of Bias In Non-randomized Studies - of Exposures (ROBINS-E) tool 20 to evaluate the RoB and visualized it using the robvis tool.”
Self-citationMaeneja, R., Rato, J. R., & Ferreira, I. S. (2025) · Children · doi:10.3390/children12050555 · Cited by 21 · see citing works
“; • the results, a narrative fi the heterogeneity study [62]. relevant Children 2025 , 12 , x FOR PEER REVIEW 7 of 23 Children 2025 2025 ,”
InfluentialLu, Z., Guo, J., Sun, J., Sun, Y., Zhang, Y., Zhang, Y., Liao, Y., Kang, Z., Feng, X., Zhao, G., et al. (2025) · Molecular Psychiatry · doi:10.1038/s41380-025-03062-5 · Cited by 18 · see citing works
“1).”
The Impact of Treatment Delay on Endometrial and Ovarian Cancer Patients: A Systematic Review
Zouzoulas, D., Karalis, T., Sofianou, I., et al. (2025) · Cancers · doi:10.3390/cancers17132076 · Cited by 16 · see citing works
“The risk of bias across the twenty-one included studies was systematically assessed using the ROBINS-E tool [7], evaluating each study across multiple bias domains including bias due to confounding, exposure measurement, participant selection, post-exposure interventions, missing data, outcome…”
Luo, Y., Cao, Y., Pan, X., Li, S., Koh, D., & Shi, Y. (2025) · Scientific Reports · doi:10.1038/s41598-025-10393-4 · Cited by 17 · see citing works
“The risk of bias in the included randomized controlled trials was systematically assessed using the second version of the Cochrane-endorsed Risk of Bias tool (RoB 2) 19 .”
Association between COVID-19 vaccination and menstruation: a state of the science review
Payne, L. A., Wise, L. A., Wesselink, A. K., et al. (2024) · BMJ Sexual & Reproductive Health · doi:10.1136/bmjsrh-2024-202274 · Cited by 16 · see citing works
Varghese, C., Lim, A. H., Daker, C., Sebaratnam, G., Gharibans, A., Andrews, C., Hasler, W. L., & O'Grady, G. (2024) · American Journal of Gastroenterology · doi:10.14309/ajg.0000000000003213 · Cited by 19 · see citing works
“Quality assessment and risk of bias was assessed using the ROBINS-E tool (21).”
Brown, J. R., Baptiste, P. J., Hajmohammadi, H., Nadarajah, R., Gale, C. P., & Wu, J. (2025) · European Journal of Preventive Cardiology · doi:10.1093/eurjpc/zwaf165 · Cited by 18 · see citing works
Alcohol consumption and frailty risk: a dose–response meta-analysis of cohort studies
Soltani, S., Jayedi, A., Ghoreishy, S. M., Mousavirad, M., Movahed, S., Jabbari, M., & Asoudeh, F. (2024) · Age and Ageing · doi:10.1093/ageing/afae199 · Cited by 15 · see citing works
Scheim, A. I., Restar, A., Zubizarreta, D., Lucas, R., Cole, S. W., Everhart, A. R., Baker, K., & Rodriguez, M. I. (2025) · Social Science & Medicine · doi:10.1016/j.socscimed.2025.118147 · Cited by 17 · see citing works
Hussey, M. R., Kornberg, T. G., Sherrick, J. M., Olson, A. M., Kind, J. A., & Pérez, A. L. (2025) · European Journal of Epidemiology · doi:10.1007/s10654-025-01271-4 · Cited by 16 · see citing works
“Finally, a bias analysis was conducted using the ROB - INS-E assessment tool for observational studies [ 64 ].”
Catapano, P., Luciano, M., Cipolla, S., D'Amico, D., Cirino, A., Corte, M. C. D., Sampogna, G., & Fiorillo, A. (2024) · Brain Behavior & Immunity - Health · doi:10.1016/j.bbih.2024.100922 · Cited by 16 · see citing works
Caponnetto, P., Lanzafame, I., Prezzavento, G. C., Rawashdeh, S., Moussa, M. A., & Fakhrou, A. A. (2025) · Journal of Addictive Diseases · doi:10.1080/10550887.2025.2473179 · Cited by 15 · see citing works
Foods, dietary patterns, and risk of vascular dementia: a systematic review
Griffiths, A., Matu, J., Tang, E. Y. H., Gregory, S., Anderson, E. L., Fairley, A., Townsend, R., Stevenson, E., Stephan, B. C. M., Siervo, M., et al. (2024) · Nutrition & Metabolism · doi:10.1186/s12986-024-00880-2 · Cited by 15 · see citing works
Guan, Z., Minnetti, M., Heymsfield, S. B., Poggiogalle, E., Prado, C. M. M., Sim, M., Stephan, B. C. M., Wells, J. C. K., Donini, L. M., & Siervo, M. (2025) · Advances in Nutrition · doi:10.1016/j.advnut.2024.100364 · Cited by 18 · see citing works
Papoutsaki, M., Katsagoni, C. N., & Papadopoulou, A. (2025) · Nutrients · doi:10.3390/nu17030487 · Cited by 19 · see citing works
Romoli, M., Urbinati, G., Tudisco, V., et al. (2025) · Neurology · doi:10.1212/wnl.0000000000213426 · Cited by 14 · see citing works
Kumar, G., Barhoun, P., Fuelscher, I. T., Zwicker, J. G., Bianco, K. M., Singh, M., Mukherjee, M., Williams, J., & Hyde, C. E. (2025) · Research in Developmental Disabilities · doi:10.1016/j.ridd.2025.104981 · Cited by 15 · see citing works
Scharré, S., Mengler, K., Schnabel‐Besson, E., Hübschmann, O. K., Tuncel, A. T., Hoffmann, G. F., Garbade, S. F., Mütze, U., & Kölker, S. (2024) · Genetics in Medicine · doi:10.1016/j.gim.2024.101303 · Cited by 15 · see citing works
Liu, Q., Welsh, P. I., Celis-Morales, C. A., Ho, F. K. W., Lees, J. S., & Mark, P. B. (2025) · Clinical Kidney Journal · doi:10.1093/ckj/sfaf003 · Cited by 17 · see citing works
“Because all the selected studies were observational studies, their qualities were independently assessed by two reviewers ( Q.L., P.W.) using the Risk Of Bias In Non-randomized Studies—of Exposure ( ROBINS-E) , which was designed for evaluating observational studies [26 ].”
Teaching trauma-informed care in undergraduate medical education: A scoping review
Bell, S. B., Miller‐Cribbs, J., Meireles, J. F. F., Clifton, S. C., Bright, R., Moon, A., Jamshed, H., Wen, F. K., & Jelley, M. (2025) · BMC Health Services Research · doi:10.1186/s12913-025-13086-w · Cited by 13 · see citing works
“Literature determined to be of very low quality by the standards of ROBINS-E (Risk of Bias in Non-Randomized Studies of Exposures) [21] was eligible for exclusion upon consensus of the authors.”
Wallerer, S., Stadelmaier, J., Floegel, E. F., Kiesswetter, E., Bantle, G., Hoffmann, G., & Schwingshackl, L. (2025) · Advances in Nutrition · doi:10.1016/j.advnut.2025.100562 · Cited by 14 · see citing works
“However, estimates that included adjustment for BMI were not prioritized, as BMI is considered a potential mediator in the causal pathway between MedDiet adherence and T2D risk, and adjusting for it could lead to overadjustment bias [17,18].”
EFSA Panel on Nutrition, N. F. A. F. A. (., Turck, D., Bohn, T., et al. (2024) · EFSA Journal · doi:10.2903/j.efsa.2024.9052 · Cited by 13 · see citing works
Etesami, E., Nikparast, A., Rahmani, J., Rezaei, M., & Ghanavati, M. (2025) · Food & Function · doi:10.1039/d4fo04741a · Cited by 13 · see citing works
The Association of Lipoprotein(a) and Stroke Recurrence: A Systematic Review and Meta-Analysis
Palaiodimou, L., Melanis, K., Stefanou, M., et al. (2025) · Journal of Stroke · doi:10.5853/jos.2024.04623 · Cited by 13 · see citing works
“10 Any disagreements were settled by consensus after discussion with the corresponding author (GT).”
Arenas, M. B., Arroyo-Sánchez, A., Torres-Parejo, Ú., & Muñoz-Negro, J. E. (2025) · International Journal of Social Psychiatry · doi:10.1177/00207640251331898 · Cited by 12 · see citing works
“…to explore the risk of bias they might induce and the overall risk of the revision, in the form of the ROBVIS (Risk-Of-Bias VISualization) R package and web app (McGuinness & Higgins, 2021), through the ROBINS-E (Risk-Of-Bias In Non-randomized Studies – of Exposure) tool (Higgins et al., 2024).”
Pérez, I. M. M., Bourhim, S., & Pérez, S. E. M. (2025) · Applied Sciences · doi:10.3390/app15115978 · Cited by 12 · see citing works
Nguyen, T. Q., Vlasenko, D., Shetty, A. N., Zhao, E., Reid, C. M., Clothier, H. J., & Buttery, J. P. (2025) · Cardiovascular Research · doi:10.1093/cvr/cvaf092 · Cited by 13 · see citing works
Gil, A. M., Correa, J. E. B., Flórez-Suárez, J. B., Fernández-Ávila, D. G., & Cucunubá, Z. M. (2025) · Seminars in Arthritis and Rheumatism · doi:10.1016/j.semarthrit.2025.152805 · Cited by 13 · see citing works
No meta-analytical effect of economic inequality on well-being or mental health
Sommet, N., Fillon, A. A., Rudmann, O., Cunha, A. R. S., & Ehsan, A. (2025) · Nature · doi:10.1038/s41586-025-09797-z · Cited by 16 · see citing works
“Following ROBINS-E, we started from a baseline of high certainty 75 .”
Magi, C. E., Aoufy, K. E., Amato, C., Longobucco, Y., Bambi, S., Vellone, E., Bonaccorsi, G., Lorini, C., Rasero, L., & Iovino, P. (2026) · Journal of Clinical Nursing · doi:10.1111/jocn.70291 · Cited by 14 · see citing works
From London buses to activity trackers: A reflection of 70 years of physical activity research
Ding, D., & Ekelund, U. (2024) · Journal of sport and health science/Journal of Sport and Health Science · doi:10.1016/j.jshs.2024.06.001 · Cited by 13 · see citing works
Various biases in systematic review and meta-analysis and their assessment
Nair, A. S., & Borkar, N. B. (2025) · Indian Journal of Anaesthesia · doi:10.4103/ija.ija_1212_24 · Cited by 13 · see citing works
Ding, J., Fu, R., Yuan, T., Brenner, H., & Hoffmeister, M. (2025) · BMC Public Health · doi:10.1186/s12889-025-21537-6 · Cited by 16 · see citing works
A Systematic Review of Regenerative Medicine Therapies for Axial Spine Pain of Facet Joint Origin
Manchikanti, L., Abd-Elsayed, A. A., Kaye, A. D., Sanapati, M. R., Pampati, V., Shekoohi, S., & Hirsch, J. A. (2025) · Current Pain and Headache Reports · doi:10.1007/s11916-025-01376-1 · Cited by 13 · see citing works
Characteristics of Hearing Loss in Type 2 Diabetes Mellitus: A Systematic Review and Meta‐Analysis
Kim, C. H., Lauritsen, K. L., Nguyen, S. A., Meyer, T. A., Cumpston, E. C., Pelic, J. C., & Labadie, R. F. (2025) · Otolaryngology · doi:10.1002/ohn.1378 · Cited by 12 · see citing works
“E) 12 for cohort studies and the Joanna Briggs Institute (JBI) critical appraisal checklist for cross ‐ sectional and case ‐ control studies.”
Bircak-Kuchtova, B., Rose, N., Geis, C., Finke, K., Pletz, M. W., Chung, H., & Fleischmann-Struzek, C. (2025) · Critical Care · doi:10.1186/s13054-025-05585-3 · Cited by 12 · see citing works
“The risk of bias in non-randomized studies was assessed using ROBINS-I-Tool, with judgments in each domain categorized as ‘very low‘, ‘low‘, ‘moderate‘ or ‘high‘ [12].”
Outcomes of Left Atrial Appendage Patency After Device Closure Detected by Cardiac CT
Jacquemyn, X., Sá, M. P., Korsholm, K., Nielsen‐Kudsk, J. E., Iriart, X., Saw, J., Alkhouli, M. A., Jain, S., Bittencourt, M. S., & Sultan, I. S. (2025) · JACC. Cardiovascular imaging · doi:10.1016/j.jcmg.2025.07.018 · Cited by 14 · see citing works
Multifaceted Antibiotic Resistance in Diabetic Foot Infections: A Systematic Review
Li, W., Sadeh, O., Chakraborty, J., et al. (2025) · Microorganisms · doi:10.3390/microorganisms13102311 · Cited by 12 · see citing works
“Studies that scored <50%, 50–75%, and >75% were deemed of high, moderate, and low risk of bias, respectively, based on a collective decision taken by the investigators in line with the ROBINS-E tool guidelines [11,12].”
Grzonka, P., Mosimann, T., Berger, S., Amacher, S. A., Baumann, S. M., Gebhard, C. E., Marchis, G. M. D., Dittrich, T. D., & Sutter, R. (2025) · Systematic Reviews · doi:10.1186/s13643-025-02797-0 · Cited by 11 · see citing works
Karakasis, P., Lefkou, E. E., Pamporis, K., Farmakis, D. T., Patoulias, D., Antoniadis, A. P., Heymans, S. R. B., Filippatos, G. S., & Fragakis, N. (2025) · European Journal of Heart Failure · doi:10.1002/ejhf.3637 · Cited by 12 · see citing works
Organizational return on investment in nursing: A systematic review
Yakusheva, O., Lee, K. A., Fial, A., & Weiss, M. (2025) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2025.105146 · Cited by 11 · see citing works
Bottosso, M., Zurlo, C., Miglietta, F., Cattelan, A. C., Iannaccone, D., Dieci, M. V., Griguolo, G., Girardi, F., & Guarneri, V. (2025) · The Breast · doi:10.1016/j.breast.2025.104544 · Cited by 12 · see citing works
“Risk of bias assessment for each included study was evaluated by two independent investigators (MB and CZ) using the ROBINS-E tool, and potential disagreements were discussed with a third reviewer (FG) [11].”
The Prognostic Value of Right Ventricle–Pulmonary Artery Coupling in Valve Interventions
Androshchuk, V., Long, E., Chehab, O., et al. (2025) · JACC: Cardiovascular Interventions · doi:10.1016/j.jcin.2025.10.024 · Cited by 10 · see citing works
Marín‐Jiménez, N., Bizzozero‐Peroni, B., Molina‐García, P., Ortega, F. B., Chaput, J., Zhang, K., Lang, J. J., McGrath, R. P., Tomkinson, G. R., Martínez-Vizcaíno, V., et al. (2026) · British Journal of Sports Medicine · doi:10.1136/bjsports-2024-109173 · Cited by 10 · see citing works
Middleton, D. R. S., Wedekind, R., Hall, A. L., et al. (2024) · Occupational and Environmental Medicine · doi:10.1136/oemed-2023-109277 · Cited by 10 · see citing works
“14 Such tools emphasise the potential for bias as determined systematically by reviewers, rather than the Monographs’ expert-based appraisal of the quality and informativeness of the exposure data for addressing the causal question about exposure to an agent and cancer.”
Obstetric Outcomes in Women on Lithium: A Systematic Review and Meta-Analysis
Callovini, T., Montanari, S., Bardi, F., Barbonetti, S., Rossi, S., Caso, R., Mandracchia, G., Margoni, S., Brugnami, A., Paolini, M., et al. (2024) · Journal of Clinical Medicine · doi:10.3390/jcm13164872 · Cited by 10 · see citing works
“The Cochrane RoB tool we used [23] does not address publication bias; however, this bias does not appear to have influenced our results, since our search strategy did not exclude studies with negative results, nor those published in languages other than English, and we directly contacted authors of…”
BackgroundWang, Y., Chang, J., Hu, P., Deng, C., Luo, Z., Zhao, J., Zhang, Z., Yi, W., Zhu, G., Zheng, G., et al. (2024) · Environmental Pollution · doi:10.1016/j.envpol.2024.124991 · Cited by 10 · see citing works
Aladawi, M., Elfil, M., Ghozy, S. A., et al. (2024) · Journal of Stroke and Cerebrovascular Diseases · doi:10.1016/j.jstrokecerebrovasdis.2024.108084 · Cited by 10 · see citing works
- Authors
- Higgins, J. P. T., Morgan, R. L., Rooney, A. A., Taylor, K. W., Thayer, K. A., Silva, R. A., Lemeris, C., Akl, E. A., Bateson, T. F., Berkman, N. D., Glenn, B. S., Hróbjartsson, A., LaKind, J. S., McAleenan, A., Meerpohl, J. J., Nachman, R. M., Obbagy, J. E., O'Connor, A., Radke, E. G., Savović, J., Schünemann, H. J., Shea, B., Tilling, K., Verbeek, J., Viswanathan, M., & Sterne, J. A. C.
- Affiliations
- University of Bristol (GB)
- McMaster University (CA)
- National Institute of Environmental Health Sciences (US)
- Environmental Protection Agency (US)
- ICF International (United States) (US)
- American University of Beirut (LB)
- RTI International (US)
- University of Southern Denmark (DK)
- University of Freiburg (DE)
- United States Department of Agriculture (US)
- Michigan State University (US)
- Cochrane Canada (CA)
- Ottawa Hospital Research Institute (CA)
- Amsterdam UMC Location University of Amsterdam (NL)
- Corresponding author
- Higgins, Julian P.T.
- ORCID
- Higgins, Julian P.T.
- Morgan, Rebecca L.
- Rooney, Andrew A.
- Taylor, Kyla W.
- Thayer, Kristina A.
- Silva, Raquel A.
- Lemeris, Courtney
- Akl, Elie A.
- Bateson, Thomas F.
- Berkman, Nancy D.
- Hróbjartsson, Asbjørn
- LaKind, Judy S.
- McAleenan, Alexandra
- Meerpohl, Joerg J.
- Nachman, Rebecca M.
- Obbagy, Julie E.
- O'Connor, Annette
- Radke, Elizabeth G.
- Savović, Jelena
- Schünemann, Holger J.
- Shea, Beverley
- Tilling, Kate
- Verbeek, Jos
- Viswanathan, Meera
- Sterne, Jonathan A.C.
- Venue
- Environment International
- Publisher
- Elsevier BV
- Published
- 2024-04
- Volume
- 186
- Pages
- 108602
- ISSN
- 0160-4120 (ISSN-L)
- PMC
- PMC11098530
- OpenAlex
- W4393132304
- Semantic Scholar
- b822b2b6998a5003935cf5815b5370685f4e5231
- Cited by
- 1,022
- References
- 30
- Citations by year
- 2026: 470 · 2025: 440 · 2024: 112
- FWCI
- 284.27
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Environment International (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Pure Amsterdam UMC (publishedVersion, cc-by)
- University of Southern Denmark Research Portal (University of Southern Denmark) (publishedVersion, cc-by-nc)
- PubMed Central (acceptedVersion, cc-by-nc)
- Europe PMC (PubMed Central) (acceptedVersion, cc-by-nc)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Bristol Research (University of Bristol) (submittedVersion, cc-by-nc-nd)
- Explore Bristol Research (submittedVersion, cc-by-nc-nd)
- Funders
- National Institute for Health and Care Research
- Cancer Research UK
- National Institute of Environmental Health Sciences
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- risk of bias assessment, non-randomized studies, cohort studies, causal inference, systematic review methodology, observational epidemiology
- MeSH terms
- Environmental Exposure, Follow-Up Studies, Humans, Cohort Studies, Bias, Epidemiologic Studies, Risk Assessment, Observational Studies as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
