The DOI matches its Crossref record; 22 of the 30 references it cites were found in a registry or an index; none of the 27 sources its claims cite is checked for support yet.
A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E)
Open accessAbstract
BACKGROUND: Observational epidemiologic studies provide critical data for the evaluation of the potential effects of environmental, occupational and behavioural exposures on human health. Systematic reviews of these studies play a key role in informing policy and practice. Systematic reviews should incorporate assessments of the risk of bias in results of the included studies. OBJECTIVE: To develop a new tool, Risk Of Bias In Non-randomized Studies - of Exposures (ROBINS-E) to assess risk of bias in estimates from cohort studies of the causal effect of an exposure on an outcome. METHODS AND RESULTS: ROBINS-E was developed by a large group of researchers from diverse research and public health disciplines through a series of working groups, in-person meetings and pilot testing phases. The tool aims to assess the risk of bias in a specific result (exposure effect estimate) from an individual observational study that examines the effect of an exposure on an outcome. A series of preliminary considerations informs the core ROBINS-E assessment, including details of the result being assessed and the causal effect being estimated. The assessment addresses bias within seven domains, through a series of 'signalling questions'. Domain-level judgements about risk of bias are derived from the answers to these questions, then combined to produce an overall risk of bias judgement for the result, together with judgements about the direction of bias. CONCLUSION: ROBINS-E provides a standardized framework for examining potential biases in results from cohort studies. Future work will produce variants of the tool for other epidemiologic study designs (e.g. case-control studies). We believe that ROBINS-E represents an important development in the integration of exposure assessment, evidence synthesis and causal inference.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
ROBINS-E is a new tool for assessing risk of bias in a specific exposure effect estimate from non-randomized cohort studies, developed by adapting ROBINS-I and RoB 2. It uses preliminary considerations, seven bias domains, signalling questions, and domain-level judgements to produce overall risk-of-bias, direction-of-bias, and threat-to-conclusions judgements.
Key findings
- ROBINS-E assesses risk of bias in a specific result from an observational cohort study of the causal effect of an exposure on an outcome, addressing seven domains: confounding, exposure measurement, selection of participants, post-exposure interventions, missing data, outcome measurement, and selection of the reported result.
- Development involved 43 researchers; proposals were discussed at a two-day meeting of 31 participants in Bristol in January 2017, and a draft was piloted in October 2019 by 28 in-person plus 3 remote participants; a revised draft was piloted in June–July 2021, and the launch version was posted in June 2022.
- Domain-level judgements include risk of bias (“Low risk of bias”, “Some concerns”, “High risk of bias”, “Very high risk of bias”), predicted direction of bias, and whether the bias threatens conclusions; the default overall risk-of-bias judgement is that of the domain with the greatest risk of bias.
- ROBINS-E does not directly assess financial conflicts of interest, and the present version addresses only follow-up (cohort) studies, with variants for other designs such as case-control studies planned.
Methods
- Design: iterative tool development through working groups, in-person meetings, webinars, and pilot testing, following principles used for RoB 2 and ROBINS-I.
- Data or sample: no empirical dataset was used; development involved 43 identified researchers, a 31-participant meeting in January 2017, a 2019 pilot with 28 in-person plus 3 remote participants, and a 2021 revised-draft pilot.
- Main analysis: not stated in the paper; the article describes the tool’s structure, domains, signalling questions, and judgement algorithms rather than conducting a statistical analysis.
Limitations
- Stated: the present version addresses only follow-up (cohort) studies, and work is ongoing to develop variants for other study designs including case-control studies.
- Stated: assessments require judgement, so subjectivity and between-assessor variability are impossible to avoid; accuracy depends on the quality of available information, and successful use requires both content and methodological expertise.
- Stated/evident: the tool has not yet had formal evaluation of its reliability and validity; the paper reports no empirical reliability or validity data, and the data availability statement says no data were used for the research described.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
20 Own interpretation · 16 Not checked yet · 14 No basis given · 2 Own method
In the paper
“although all such studies identified related to randomized trials”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Development of ROBINS-E
What the sources say
- Not checked yetcites Sterne et al., 2002 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“ROBINS-E does not include direct assessment of financial conflicts of interest, which may be important in interpreting the findings and credibility of studies of exposures and have been recommended for inclusion in evaluation of environmental studies (National Academies of Sciences Engineering and Medicine, 2022).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
What the sources say
- Not checked yetcites National Academies of Sciences Engineering and Medicine, 2022
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“different tools, and different ways of addressing risk-of-bias assessments, can lead to differences in conclusions of systematic reviews”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Introduction
What the sources say
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Recently-developed tools for this purpose have moved away from numeric scores based on checklists (which often combined elements of risk of bias with other considerations such as of applicability) to tools based on assessments within specific domains of bias.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Introduction
What the sources say
- Not checked yetcites National Toxicology Program, 2015
- Not checked yetcites Woodruff and Sutton, 2014 source
- Not checked yetcites U.S. Environmental Protection Agency, 2022
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“The second variant additionally addresses time-varying confounding, which can occur when the exposure changes over time and follow-up time is split according to exposure level (Hernán and Robins, 2020; Mansournia et al., 2017).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
What the sources say
- Not checked yetcites Hernán and Robins, 2020
- Not checked yetcites Mansournia et al., 2017
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For this reason, ROBINS-E assessments may support causal inferences, if sufficient evidence at ‘low’ risk of bias or with ‘some concerns’ about risk of bias can be located.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“ROBINS-E can be used as part of a framework to rate the quality of the evidence for a group of studies. An example of such a framework is the GRADE approach (Guyatt et al., 2011)”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
What the sources say
- Not checked yetcites Guyatt et al., 2011 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It is impossible to exclude the possibility of uncontrolled confounding in a non-randomized study”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Furthermore, there are impacts of such conflicts that are not problems of internal validity (targeted by ROBINS-E) but instead relate to what question is being asked in the study or analysis.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“The rationale, previously discussed in the context of randomized trials (Sterne, 2013), is that if financial conflicts of interest lead to bias then this will operate through one of the domains already in the tool.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
What the sources say
- Not checked yetcites Sterne, 2013 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- It may state two separate claims as one. 9
- Probably not a substantial claim. 37
None of the sources cited for this claim have been checked yet.
In the paper
“The instrument was based on the ROBINS-I tool for non-randomized studies of interventions (Sterne et al., 2016) and RoB 2 tool for randomized trials (Sterne et al., 2019), and was informed by previous instruments for assessing studies of exposure.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
What the sources say
- Not checked yetcites Sterne et al., 2016
- Not checked yetcites Sterne et al., 2019
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“ROBINS-E is a new general-purpose tool for assessing risk of bias in results of non-randomized studies of exposure.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The present version addresses only follow-up (cohort) studies”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Important elements in ROBINS-E include a definition of the causal effect of interest, signalling questions to guide domain-level risk-of-bias judgements across seven domains, algorithms to derive suggested risk of bias judgements from answers to the signalling questions, and the separation of risk of bias from other important considerations in the evaluation of study sensitivity.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“These additional considerations, which fall outside the scope of the main ROBINS-E assessment, are addressed in an optional section of ROBINS-E.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Study sensitivity and appropriateness of its design
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“However, the user may override this, for example if they judge that so many domains were rated as at ‘High’ risk of bias that the overall judgement should be ‘Very high’ risk of bias.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Overall risk of bias
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The default ROBINS-E overall risk-of-bias judgement is that for the domain with the greatest risk of bias.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Overall risk of bias
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The default judgement for the threat to conclusions is derived in a similar way to the overall risk-of-bias judgement; for example, the overall judgement is ‘Yes’ if there is considered to be a threat to conclusions in any of the seven bias domains.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Overall risk of bias
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“ROBINS-E provides a standardized framework for examining the range of biases common to observational studies of exposures.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“ROBINS-E is designed primarily for use in the context of a systematic review.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“The collaborative development of the tool, between methodologists and stakeholders from fields such as environmental, occupational, chemical, clinical and public health, has highlighted the benefits of sharing ideas and research methods across communities (Hoffmann et al., 2022; Hoffmann et al., 2022).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
What the sources say
- Not checked yetcites Hoffmann et al., 2022
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“An optional section of ROBINS-E allows users to record observations about study sensitivity, aspects of which may additionally feed into the “indirectness” domain within GRADE, as it relates to generalizability of the result of the study to the population, exposure and outcome of interest.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Several such tools for observational epidemiologic studies provide significant methodological advances over previously used tools (National Toxicology Program, 2015, 2015; Woodruff and Sutton, 2014; U.S. Environmental Protection Agency, 2022).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Introduction
What the sources say
- Not checked yetcites National Toxicology Program, 2015
- Not checked yetcites Woodruff and Sutton, 2014 source
- Not checked yetcites U.S. Environmental Protection Agency, 2022
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Early work by a subset of the authors considered the extent to which the ROBINS-I tool was suitable for observational studies of exposures and suggested modifications to the tool (Morgan et al., 2018; Morgan et al., 2019).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Development of ROBINS-E
What the sources say
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“The Risk Of Bias In Non-randomized Studies - of Interventions (ROBINS-I) tool was developed to evaluate estimates of the effects of deliberate interventions aimed to alter health outcomes (Sterne et al., 2016).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Introduction
What the sources say
- Not checked yetcites Sterne et al., 2016
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Domain-level judgements about risk of bias are derived from the answers to these questions, then combined to produce an overall risk of bias judgement for the result, together with judgements about the direction of bias.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Abstract > Methods and results:
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The extent of the potential for bias across a body of evidence for an exposure-outcome effect informs assessment of the certainty of evidence for that effect.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A screening section then facilitates identification of results that are at very high risk of bias, allowing the user to avoid a detailed risk of bias assessment (Fig. 1, part B).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Preliminary considerations
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“There is a risk of bias when such selection is based on the magnitude, direction or P value of the result.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“First, evaluation of risk of bias requires that the review team has expertise in both content (detailed subject-matter knowledge) and methodology (understanding of the different types of bias that may distort the results of observational studies).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Bias due to post-exposure interventions may arise if the presence or level of exposure leads to administration of interventions that change the subsequent risk of the outcome.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Bias in selection of the reported result arises when study authors select results from a multiplicity of analyses, for example from different ways of measuring the exposure, different ways of measuring the outcome, different subsets of the full study sample or different analyses.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The ROBINS-E risk-of-bias assessment addresses seven domains of bias (Fig. 1, Section 5).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Overview of ROBINS-E
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- The authors state this as an assumption: Issues common to both randomized trials and observational studies (missing data, outcome assessment, selection of the reported result) are likely to be similar.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“By contrast, it is possible to envisage situations in which risk of bias for other domains is low: for example, if exposure is measured without error, there are no missing data, or results are reported in accordance with a pre-specified analysis plan.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A series of preliminary considerations informs the core ROBINS-E assessment, including details of the result being assessed and the causal effect being estimated.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Abstract > Methods and results:
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, such analyses require that the factors leading to the post-exposure intervention can be modelled.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Although ROBINS-E was iteratively developed based on the results of pilot testing with a diverse group”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Bias arising from measurement of the outcome is a problem when error in measurement of the outcome is related to the presence or level of exposure, in which case the measurement error is differential rather than non-differential.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Third, ROBINS-E assessments require judgements based on the available information. It is therefore impossible to avoid subjectivity and between-assessor variability.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ROBINS-E provides a standardized framework for examining potential biases in results from cohort studies.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Abstract > Conclusion:
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- The authors state this as an assumption: Issues common to both randomized trials and observational studies (missing data, outcome assessment, selection of the reported result) are likely to be similar.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This question can be manipulated to give an illusion of presence or lack or effect (for example by studying trivial levels of exposure or irrelevant outcomes), but this need not have implications for internal validity.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Practical considerations can impact on the successful implementation of ROBINS-E.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Each domain is addressed using a series of signalling questions that aim to gather relevant information about the study and analysis being assessed.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Overview of ROBINS-E
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The first version of ROBINS-E, described here, targets cohort (follow-up) studies.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Introduction
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“ROBINS-E was developed by a large group of researchers from diverse research and public health disciplines through a series of working groups, in-person meetings and pilot testing phases.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Abstract > Methods and results:
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Because risk of bias is assessed in relation to a defined causal effect of an exposure on an outcome, a low risk of bias corresponds to an absence of bias in estimating that causal effect.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, publication of answers to signalling questions and justification for these should make risk of bias judgements transparent.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Bias in selection of participants into the study (or into the analysis) is the first of two domains that address selection bias, which arises when we condition, in the study design or in the analysis, on common effects of exposure and outcome (Hernán and Robins, 2020; Hernán et al., 2004).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
What the sources say
- Not checked yetcites Hernán and Robins, 2020
- Not checked yetcites Hernán et al., 2004 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Second, the accuracy of a risk-of-bias assessment will depend on the quality of information available about the study being assessed.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper relies on this without support.
Randomized controlled trials are often not feasible, ethical, large enough or of sufficient duration to evaluate chronic health effects.
“Randomized controlled trials (RCTs) are often not feasible, ethical, large enough or of sufficient duration to evaluate chronic health effects.”
The authors state this as an assumption.
Issues common to both randomized trials and observational studies (missing data, outcome assessment, selection of the reported result) are likely to be similar.
“with the additional assumption that issues common to both study designs (missing data, outcome assessment, selection of the reported result) are likely to be similar”
The paper relies on this without support.
Pre-specified plans for analyses are uncommon for observational studies.
“However, such pre-specified plans are uncommon for observational studies and often the nature of the data need to be taken into account when selecting a suitable statistical model and method.”
The paper relies on this without support.
The nature of the data often needs to be taken into account when selecting a suitable statistical model and method.
“However, such pre-specified plans are uncommon for observational studies and often the nature of the data need to be taken into account when selecting a suitable statistical model and method.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“For cohort studies, the Risk of Bias in Non‐randomized Studies of Exposures (ROBINS‐E) tool was used.”
From Risk of cardiac, neuropsychiatric and musculoskeletal adverse events with levothyroxine: Systematic review (2026) · cites this paper as [12]
Citations
Cites
RoB 2: a revised tool for assessing risk of bias in randomised trials
Sterne, J. A. C., Savović, J., Page, M. J., Elbers, R. G., Blencowe, N. S., Boutron, I., Cates, C. J., Cheng, H.-Y., Corbett, M. S., Eldridge, S. M., et al. (2019) · BMJ · doi:10.1136/bmj.l4898 · Cited by 34,787
Self-citationDermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,586
GRADE: an emerging consensus on rating quality of evidence and strength of recommendations
Guyatt, G. H., Oxman, A. D., Vist, G. E., Kunz, R., Falck-Ytter, Y., Alonso-Coello, P., & Schünemann, H. J. (2008) · BMJ · doi:10.1136/bmj.39489.470347.ad · Cited by 23,305 · cited 1×
“…of interest; measurement of exposure; measurement of outcomes; attrition and selective reporting; integration of ROBINS-E with the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) framework (Guyatt et al., 2008); and issues in statistical analysis (see Online Supplement).”
Self-citationROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329
Self-citationCochrane Handbook for Systematic Reviews of Interventions
(2019) · doi:10.1002/9781119536604 · Cited by 14,828
GRADE guidelines: 1. Introduction—GRADE evidence profiles and summary of findings tables
Guyatt, G., Oxman, A. D., Akl, E. A., Kunz, R., Vist, G., Brozek, J., Norris, S., Falck-Ytter, Y., Glasziou, P., & deBeer, H. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.04.026 · Cited by 10,681 · cited 1×
“An example of such a framework is the GRADE approach (Guyatt et al., 2011) where users may start at high certainty in the “study limitations” (risk of bias) domain of GRADE and use their ROBINS-E evaluation to determine how many levels to rate down this certainty (Schunemann et al., 2018).”
Self-citationA structural approach to selection bias
Hernán, M. A., Hernández-Díaz, S., & Robins, J. M. (2004) · Epidemiology · doi:10.1097/01.ede.0000135174.63482.43 · Cited by 2,844 · cited 1×
“Bias in selection of participants into the study (or into the analysis) is the first of two domains that address selection bias, which arises when we condition, in the study design or in the analysis, on common effects of exposure and outcome (Hernán and Robins, 2020; Hernán et al., 2004).”
Triangulation in aetiological epidemiology
Lawlor, D. A., Tilling, K., & Davey Smith, G. (2017) · International Journal of Epidemiology · doi:10.1093/ije/dyw314 · Cited by 1,129 · cited 1×
“…together in a triangulation exercise, in which both the risks EPA A u t ho r M anu sc r i p t EPA A u t ho r M anu sc r i p t EPA A u t ho r M anu sc r i p t of bias (assessed by ROBINS-E) and differences between causal questions being addressed are considered jointly (Lawlor et al., 2016).”
Self-citationSchünemann, H. J., Cuello, C., Akl, E. A., Mustafa, R. A., Meerpohl, J. J., Thayer, K., Morgan, R. L., Gartlehner, G., Kunz, R., Katikireddi, S. V., et al. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.01.012 · Cited by 911 · cited 1×
“An example of such a framework is the GRADE approach (Guyatt et al., 2011) where users may start at high certainty in the “study limitations” (risk of bias) domain of GRADE and use their ROBINS-E evaluation to determine how many levels to rate down this certainty (Schunemann et al., 2018).”
Self-citationWoodruff, T. J., & Sutton, P. (2014) · Environmental Health Perspectives · doi:10.1289/ehp.1307175 · Cited by 427 · cited 1×
“Several such tools for observational epidemiologic studies provide significant methodological advances over previously used tools (National Toxicology Program, 2015, 2015; Woodruff and Sutton, 2014; U.S. Environmental Protection Agency, 2022).”
Research works printed but not linked: 2
- Boutron, 2019, Chapter 7: considering bias and conflicts of interest among the included studies, 177
- National Academies of Sciences Engineering and Medicine, 2022. A Review of U.S. EPA’s ORD Staff Handbook for Developing IRIS Assessments: 2020 Version. Washington DC: The National Academies Press.
Cited by
Sorted by citations
AI Chatbots for Mental Health: A Scoping Review of Effectiveness, Feasibility, and Applications
Casu, M., Triscari, S., Battiato, S., Guarnera, L., & Caponnetto, P. (2024) · Applied Sciences · doi:10.3390/app14135889 · Cited by 186
“The methodological quality of the included studies was assessed using the Cochrane risk-of-bias tool for randomized trials, version 2 (RoB 2) [54], and the Cochrane Risk Of Bias In Non-randomized Studies—of Exposure (ROBINS-E) [55].”
Associations between sexual health and well-being: a systematic review
Vasconcelos, P. A., Carrito, M. L., Quinta-Gomes, A. L., Patrão, A. L., Nóbrega, C. A., Costa, P. A., & Nobre, P. J. (2024) · Bulletin of the World Health Organization · doi:10.2471/blt.24.291565 · Cited by 119
“We used the Risk Of Bias In Non-ran-domized Studies - of Exposures tool 31 to assess the risk of bias in the included studies, as the tool indicates whether the risk of bias is substantial enough to question the impact of the exposure on the outcome.”
Life's Essential 8 and the risk of cardiovascular disease: a systematic review and meta-analysis
Sebastian, S. A., Shah, Y., Paul, H., & Arsene, C. (2024) · European Journal of Preventive Cardiology · doi:10.1093/eurjpc/zwae280 · Cited by 84
The Applications of Artificial Intelligence for Assessing Fall Risk: Systematic Review
González-Castro, A., Leirós‐Rodríguez, R., Prada‐García, C., & Benítez‐Andrades, J. A. (2024) · Journal of Medical Internet Research · doi:10.2196/54934 · Cited by 64
Luo, D., Dashti, S. G., Sawyer, S. M., & Vijayakumar, N. (2024) · EClinicalMedicine · doi:10.1016/j.eclinm.2024.102828 · Cited by 67
“The Risk of Bias in Non-randomized Studies of Expo-sures (ROBINS-E) tool 26,27 provides a structured approach to assessing the risk of bias in observational epidemiological studies evaluating the effect of an exposure on an outcome.”
BackgroundStorck, W., Elbaz, M., Vindis, C., Deguilhem, A., Lapeyre‐Mestre, M., & Jouanjus, É. (2025) · Heart · doi:10.1136/heartjnl-2024-325429 · Cited by 51
“19 The latter risk is approached systematically in seven distinct domains, that is, (1) bias due to confounding (D1), (2) bias arising from measurement of the exposure (D2), (3) bias in selection of participants into the study (or into the analysis) (D3), (4) bias due to post-exposure interventions…”
Methodological Standards for Conducting High-Quality Systematic Reviews
CASSAI, A. D., Dost, B., Tulgar, S., & Boscolo, A. (2025) · Biology · doi:10.3390/biology14080973 · Cited by 77
“…In Non-randomized Studies—of Interventions) [29] Non-Randomized Intervention Studies ROBINS-E (Risk Of Bias In Non-randomized Studies—of Exposures) [30] Observational Exposure Studies (Cohort, Case-Control) Newcastle-Ottawa Scale (NOS) [31] Cohort and Case-Control Studies QUADAS-2 (Quality…”
Carbone, F., Spinelli, A., Ciardiello, D., Luc, M. R., Pascale, S. D., Bertani, E., Fazio, N., & Romario, U. F. (2024) · European Journal of Cancer · doi:10.1016/j.ejca.2024.115172 · Cited by 43
Li, Y., Smith, R., Whitney, S. L., Seemungal, B. M., & Ellmers, T. J. (2024) · Age and Ageing · doi:10.1093/ageing/afae177 · Cited by 47
“We assessed the quality of included studies using the Risk of Bias in Non-randomised Studies of Exposures (ROBINS-E) tool [22] and visualised the assessments using ‘robvis’ [23].”
MethodAcevedo, P., Magana, A. J., Beneš, B., & Mousas, C. (2024) · IEEE Access · doi:10.1109/access.2024.3489233 · Cited by 43
“For non-randomized studies that focused on validating interventions rather than conducting comparison studies, we used the ROBINS-E (Risk of Bias in Non-randomized Studies of Exposures) tool [37].”
Cardiac rehabilitation and frailty: a systematic review and meta-analysis
MacEachern, E., Quach, J., Giacomantonio, N., Theou, O., Hillier, T. W. R., Abel-Adegbite, I., González-Lara, M., & Kehler, D. S. (2024) · European Journal of Preventive Cardiology · doi:10.1093/eurjpc/zwae239 · Cited by 39
Elangovan, S., Lo, J. J.-M., Xie, Y., et al. (2024) · Journal of Hospital Infection · doi:10.1016/j.jhin.2024.08.002 · Cited by 37
From social media to artificial intelligence: improving research on digital harms in youth
Mansfield, K. L., Ghai, S., Hakman, T., Ballou, N., Vuorre, M., & Przybylski, A. K. (2025) · The Lancet Child & Adolescent Health · doi:10.1016/s2352-4642(24)00332-8 · Cited by 53
The changing prevalence of ADHD? A systematic review
Martin, A. F., Rubin, G. J., Rogers, M. B., Wessely, S. C., Greenberg, N. L., Hall, C. E., Pitt, A., Logan, P. E., Lucas, R., & Brooks, S. K. (2025) · Journal of Affective Disorders · doi:10.1016/j.jad.2025.119427 · Cited by 39
Iqbal, S., Ramini, A., & Kaja, S. (2024) · The Ocular Surface · doi:10.1016/j.jtos.2024.12.003 · Cited by 32
“For non-randomized clinical studies, the Risk-of-Bias In Non-Randomized Studies of Exposure (ROBINS-E) tool was used for observational studies [12].”
Nordon, C., Simons, S. O., Marshall, J., Müllerová, H., Pollack, M. F., Bengtsson, C., Hoti, F. J., Lobier, M., Salosensaari, A., SANTOS, A. C. D. S., et al. (2024) · ERJ Open Research · doi:10.1183/23120541.01091-2024 · Cited by 33
“Risk of bias in each individual study was assessed according to the seven bias domains within the Risk Of Bias In Non-randomised Studies – of Exposure (ROBINS-E) tool [15].”
Baecker, L., Iyengar, U., Piccolo, M. C. D., & Mechelli, A. (2025) · The Journal of Climate Change and Health · doi:10.1016/j.joclim.2025.100446 · Cited by 29
Deprato, A., Garud, A., Azzolina, D., Murgia, N., Davenport, M. H., Kaul, P., Lacy, P., & Moitra, S. (2024) · Journal of Hazardous Materials · doi:10.1016/j.jhazmat.2024.137028 · Cited by 30
Udom, G. J., Turyahabwe, B., Aturamu, A., Aziakpono, O. M., Agbana, R. D., Joseph, O. G., Udom, N.-W. G., Mugide, N., Odey, O. P., Olot, H., et al. (2025) · Environmental Advances · doi:10.1016/j.envadv.2025.100620 · Cited by 30
Xie, L., Ma, W., Du, K., Huang, Y., Li, A., Wang, H., Cui, H., Qiu, W., Gao, R., Qu, G., et al. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1525277 · Cited by 37
Tan, L., Stagg, L., Hanlon, E., Li, T., McGrattan, A., Siervo, M., Matu, J., Griffiths, A., & Shannon, O. M. (2024) · Nutrients · doi:10.3390/nu16101511 · Cited by 27
“…arising from confounding; (2) resulting from exposure measurement; (3) from selection of participants into the study/analysis; (4) arising from post-exposure interventions; (5) arising from missing data; (6) resulting from outcome measurement and (7) from selection of the reported result” [26].”
BackgroundMethodTerry, P., Heidel, R. E., Wilson, A. Q., & Dhand, R. (2025) · BMJ Open Respiratory Research · doi:10.1136/bmjresp-2024-002528 · Cited by 29
“The risk of bias was more formally assessed using the ROBINS-E (ROB, risk of bias in non-randomised studies of exposure), 65 and the resulting table is shown in online supplemental appendix 2.”
Allostatic Load and Cardiovascular Disease: A Systematic Review
Evans, E., Jacobs, M., Fuller, D. D., Wheeler‐Hegland, K., & Ellis, C. (2025) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2025.02.016 · Cited by 24
Corte, K. W. D., Bosler, T., McClure, C., Buyken, A. E., LeCheminant, J. D., Schwingshackl, L., & Corte, D. D. (2025) · Advances in Nutrition · doi:10.1016/j.advnut.2025.100413 · Cited by 33
“To improve the validity of the fi ndings, potential risks of bias were assessed using the ROBINS-E (Risk of Bias in Non-randomized Studies of Exposures) tool [24,25].”
Vitale, E., Maistrello, L., Ciccimarra, F., Rizzo, A., Soares, A., Brunetti, O., Mollica, V., Maiorano, B. A., Maiorano, M. F. P., Monteiro, F. S. M., et al. (2026) · Cancer and Metastasis Reviews · doi:10.1007/s10555-026-10327-0 · Cited by 33
Pirera, E., Raimondo, D. D., D’Anna, L., & Tuttolomondo, A. (2025) · European Journal of Internal Medicine · doi:10.1016/j.ejim.2025.01.016 · Cited by 27
Antonini, M., Mattar, A. P. L., Pereira, T. P., Oliveira, L. L., Teixeira, M. D., Amorim, A. G., Ferraro, O., de Oliveira, L. C., Ramos, M. D. N. M., Cavalcante, F. P., et al. (2025) · Heliyon · doi:10.1016/j.heliyon.2025.e43069 · Cited by 24
Climate Conditions, Weather Changes, and Air Pollutants and Atopic Dermatitis
Park, M., Mohsen, S. T., Katz, T., Sood, S., Sy, S. M. T., Rochwerg, B. N., & Drucker, A. M. (2025) · JAMA Dermatology · doi:10.1001/jamadermatol.2025.1790 · Cited by 30
Evidence update on the cancer risk of vaping e-cigarettes: A systematic review
Kundu, A., Sachdeva, K., Feore, A., Sanchez, S., Sutton, M., Seth, S., Schwartz, R. M., & Chaiton, M. (2025) · Tobacco Induced Diseases · doi:10.18332/tid/192934 · Cited by 23
Green, K. L., Yang, Y., Abaraogu, U. O., Eastaugh, C., Beyer, F. R., Norman, G., & Todd, C. J. (2024) · Age and Ageing · doi:10.1093/ageing/afae104 · Cited by 21
Martínez‐Vizcaíno, V., Fernández‐Rodríguez, R., Reina‐Gutiérrez, S., Rodríguez‐Gutiérrez, E., Garrido‐Miguel, M., de Arenas‐Arroyo, S. N., & Torres-Costoso, A. I. (2024) · BMC Public Health · doi:10.1186/s12889-024-19383-z · Cited by 32
“The risk of bias assessment results obtained using the ROBINS-E tool [34], are shown in Supplementary Table S4.”
MethodDiep, C., Patel, K., Petricca, J., et al. (2024) · Anaesthesia · doi:10.1111/anae.16398 · Cited by 21
Comparing cancer stage at diagnosis between migrants and non-migrants: a meta-analysis
Harvey-Sullivan, A., Ali, S., Dhesi, P., Hart, J., Painter, H., Walter, F. M., Funston, G., & Zenner, D. (2024) · British Journal of Cancer · doi:10.1038/s41416-024-02896-0 · Cited by 26
“41 papers met the inclusion criteria for this systematic review (Fig.”
González, I. B., Montero, J., Gómez‐Polo, C., & Pardal-Peláez, B. (2025) · Journal of Dentistry · doi:10.1016/j.jdent.2025.105707 · Cited by 31
Cosgrave, N., Frydenlund, J., Beirne, F., Lee, S. J., Faez, I., Cahir, C., & Williams, D. J. (2025) · Age and Ageing · doi:10.1093/ageing/afaf231 · Cited by 23
“This tool evaluates the ROB across seven domains: (i) confounders, (ii) participants, (iii) exposure measurement, (iv) post-exposure interventions, (v) missing data, (vi) outcome measurement and (vii) reporting [21].”
Zainal, N. H., Wang, V. Y., Garthwaite, B., & Curtiss, J. E. (2025) · Health Psychology Review · doi:10.1080/17437199.2025.2547610 · Cited by 26
“Fourth, with 51.3% of studies showing high bias, future work should adopt more rigorous standards (e.g., preregistration, transparent preprocessing, randomisation; Harrison et al., 2017; Higgins et al., 2024).”
InfluentialVitale, E., Rizzo, A., Maistrello, L., Cauli, O., Brunetti, O., Monteiro, F. S. M., Soares, A., Santoni, M., Mollica, V., & Massari, F. (2025) · Drug Design Development and Therapy · doi:10.2147/dddt.s560518 · Cited by 35
“42 The tool consisted of seven sub dimensions exploring the presence of bias in the obtained results.”
Petrucci, G., Papalia, G. F., Ambrosio, L., Russo, F., Marchetti, A., Marinis, M. G. D., Vadalà, G., Papalia, R., & Denaro, V. (2025) · European Spine Journal · doi:10.1007/s00586-025-08733-z · Cited by 29
The Role of Biomarkers in Temporomandibular Disorders: A Systematic Review
Soares, J. M., Carneiro, B. D., & Pozza, D. H. (2025) · International Journal of Molecular Sciences · doi:10.3390/ijms26135971 · Cited by 20
“…the Cochrane RoB 2 tool [18] at the outcome level visualized with the Cochrane risk of bias VISualization app 4.0 [19] and the risk of bias of other study types was evaluated with the ROBINS-E tool [20], also at the outcome level visualized with the Cochrane risk of bias VISualization app 4.0 [19].”
InfluentialMapping Worldwide Antibiotic Use in Dental Practices: A Scoping Review
Soleymani, F., Martı́nez, C. P., Makiabadi, M., et al. (2024) · Antibiotics · doi:10.3390/antibiotics13090859 · Cited by 19
“This research identified a slight decreasing trend in the DDD of antibiotics prescribed by dental professionals in the studied countries, with a notable increase during the COVID-19 pandemic.”
ResultGrubic, N., Johnston, A., Randhawa, V. K., Humphries, K. H., Rosella, L. C., & Maximova, K. (2024) · Canadian Journal of Cardiology · doi:10.1016/j.cjca.2024.12.022 · Cited by 24
Anxiety and Depression in Adults With Vestibular Disorders: A Systematic Review and Meta‐Analysis
Kim, C. H., McCray, L. R., Nguyen, S. A., Staab, J. P., Jafri, S. H., & Rizk, H. G. (2025) · The Laryngoscope · doi:10.1002/lary.70055 · Cited by 19
“The risk of bias was assessed according to the Risk of Bias In Non-randomized Studies—of Exposure (ROBINS-E) [17] for cohort studies and the Joanna Briggs Institute (JBI) critical appraisal checklist for cross-sectional and case–control studies [18, 19].”
Kiesswetter, E., Neuenschwander, M., Stadelmaier, J., Szczerba, E., Hofacker, L., Sedlmaier, K., Kussmann, M., Roeger, C., Hauner, H., Schlesinger, S., et al. (2024) · Current Developments in Nutrition · doi:10.1016/j.cdnut.2024.102159 · Cited by 22
“We used the Risk Of Bias In Non-randomized Studies - of Exposure (ROBINS-E) tool to evaluate the risk of bias [21].”
Grassi, G. M., Quarti‐Trevano, F., Cuspidi, C., Sanidas, E. A., Mancia, G., & Thomopoulos, C. (2025) · Hypertension · doi:10.1161/hypertensionaha.125.24749 · Cited by 20
Grasa-Ciria, D., Couto-Alfonso, S., Samatán, E., Martínez‐Jarreta, B., Cénit, M. C., & Iguacel, I. (2025) · Nutrients · doi:10.3390/nu17172894 · Cited by 19
“The methodological quality and risk of bias of the included studies were assessed using two complementary tools: the ROBINS-E (Risk Of Bias In Non-randomized Studies—of Exposures) tool [39] and the NIH Quality Assessment Tool [40] (Supplementary Materials S3 and S4).”
Pouliopoulou, D. V., Hawthorne, M., MacDermid, J. C., Billias, N., Miller, E., Quinn, K. L., Décary, S., Razak, F. A., Cheung, A. M., Galiatsatos, P., et al. (2025) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2025.01.471 · Cited by 19
Evaluation of dietary protein and amino acid requirements: a systematic review
Burstad, K. M., Lamina, T., Erickson, A., Gholizadeh, E., Namigga, H., Claussen, A. M., Slavin, J. L., Teigen, L. M., Gallant, K. M. H., Stang, J. L., et al. (2025) · American Journal of Clinical Nutrition · doi:10.1016/j.ajcnut.2025.04.017 · Cited by 19
Donaldson, S. I., Russell, A. M., Capria, K. L., DeJesus, A., Wang, E., Fayad, J., & Allem, J.-P. (2025) · The Lancet Public Health · doi:10.1016/s2468-2667(25)00219-1 · Cited by 22
Rimmer, P. J., Zhang, F., Scott, G., Kaakoush, N. O., Lewindon, P. J., Ashton, J., Kansal, S., Limbergen, J. V., Boneh, R. S., Pai, N. N., et al. (2025) · Gastroenterology · doi:10.1053/j.gastro.2025.09.014 · Cited by 25
A systematic review and meta-analysis of disease clusters in multimorbidity
Ferris, J. K., Fiedeldey, L., Kim, B., Clemens, F., Irvine, M. A., Hosseini, S. H., Smolina, K., & Wister, A. (2025) · Nature Communications · doi:10.1038/s41467-025-67372-6 · Cited by 22
Karam, S. G., Zhang, Y., Hernandez, H. P., Siebert, U., Koopman, L., Noyes, J., Tarride, J., Stevens, A., Welch, V. A., Saz‐Parkinson, Z., et al. (2024) · BMJ · doi:10.1136/bmj-2024-079890 · Cited by 19
Self-citationWallerer, S., Papakonstantinou, T., Morze, J., Stadelmaier, J., Kiesswetter, E., Gorenflo, L., Barbaresko, J., Szczerba, E., Neuenschwander, M., Bell, W., et al. (2024) · EClinicalMedicine · doi:10.1016/j.eclinm.2024.102807 · Cited by 22
“We used the Risk Of Bias In Non-randomized Studies - of Exposures (ROBINS-E) tool 20 to evaluate the RoB and visualized it using the robvis tool.”
Self-citationMaeneja, R., Rato, J. R., & Ferreira, I. S. (2025) · Children · doi:10.3390/children12050555 · Cited by 42
“; • the results, a narrative fi the heterogeneity study [62]. relevant Children 2025 , 12 , x FOR PEER REVIEW 7 of 23 Children 2025 2025 ,”
InfluentialLu, Z., Guo, J., Sun, J., Sun, Y., Zhang, Y., Zhang, Y., Liao, Y., Kang, Z., Feng, X., Zhao, G., et al. (2025) · Molecular Psychiatry · doi:10.1038/s41380-025-03062-5 · Cited by 18
“1).”
The Impact of Treatment Delay on Endometrial and Ovarian Cancer Patients: A Systematic Review
Zouzoulas, D., Karalis, T., Sofianou, I., et al. (2025) · Cancers · doi:10.3390/cancers17132076 · Cited by 16
“The risk of bias across the twenty-one included studies was systematically assessed using the ROBINS-E tool [7], evaluating each study across multiple bias domains including bias due to confounding, exposure measurement, participant selection, post-exposure interventions, missing data, outcome…”
Luo, Y., Cao, Y., Pan, X., Li, S., Koh, D., & Shi, Y. (2025) · Scientific Reports · doi:10.1038/s41598-025-10393-4 · Cited by 17
“The risk of bias in the included randomized controlled trials was systematically assessed using the second version of the Cochrane-endorsed Risk of Bias tool (RoB 2) 19 .”
Association between COVID-19 vaccination and menstruation: a state of the science review
Payne, L. A., Wise, L. A., Wesselink, A. K., Wang, S., Missmer, S. A., & Edelman, A. B. (2024) · BMJ Sexual & Reproductive Health · doi:10.1136/bmjsrh-2024-202274 · Cited by 19
Varghese, C., Lim, A. H., Daker, C., Sebaratnam, G., Gharibans, A. A., Andrews, C. N., Hasler, W. L., & O’Grady, G. (2024) · The American Journal of Gastroenterology · doi:10.14309/ajg.0000000000003213 · Cited by 21
“Quality assessment and risk of bias was assessed using the ROBINS-E tool (21).”
Brown, J. R., Baptiste, P. J., Hajmohammadi, H., Nadarajah, R., Gale, C. P., & Wu, J. (2025) · European Journal of Preventive Cardiology · doi:10.1093/eurjpc/zwaf165 · Cited by 18
Alcohol consumption and frailty risk: a dose–response meta-analysis of cohort studies
Soltani, S., Jayedi, A., Ghoreishy, S. M., Mousavirad, M., Movahed, S., Jabbari, M., & Asoudeh, F. (2024) · Age and Ageing · doi:10.1093/ageing/afae199 · Cited by 15
Scheim, A. I., Restar, A., Zubizarreta, D., Lucas, R., Cole, S. W., Everhart, A. R., Baker, K., & Rodriguez, M. I. (2025) · Social Science & Medicine · doi:10.1016/j.socscimed.2025.118147 · Cited by 28
Hussey, M. R., Kornberg, T. G., Sherrick, J. M., Olson, A. M., Kind, J. A., & Pérez, A. L. (2025) · European Journal of Epidemiology · doi:10.1007/s10654-025-01271-4 · Cited by 17
“Finally, a bias analysis was conducted using the ROB - INS-E assessment tool for observational studies [ 64 ].”
Catapano, P., Luciano, M., Cipolla, S., D'Amico, D., Cirino, A., Corte, M. C. D., Sampogna, G., & Fiorillo, A. (2024) · Brain Behavior & Immunity - Health · doi:10.1016/j.bbih.2024.100922 · Cited by 16
Caponnetto, P., Lanzafame, I., Prezzavento, G. C., Rawashdeh, S., Moussa, M. A., & Fakhrou, A. A. (2025) · Journal of Addictive Diseases · doi:10.1080/10550887.2025.2473179 · Cited by 18
Foods, dietary patterns, and risk of vascular dementia: a systematic review
Griffiths, A., Matu, J., Tang, E. Y. H., Gregory, S., Anderson, E. L., Fairley, A., Townsend, R., Stevenson, E., Stephan, B. C. M., Siervo, M., et al. (2024) · Nutrition & Metabolism · doi:10.1186/s12986-024-00880-2 · Cited by 16
Guan, Z., Minnetti, M., Heymsfield, S. B., Poggiogalle, E., Prado, C. M. M., Sim, M., Stephan, B. C. M., Wells, J. C. K., Donini, L. M., & Siervo, M. (2025) · Advances in Nutrition · doi:10.1016/j.advnut.2024.100364 · Cited by 18
Papoutsaki, M., Katsagoni, C. N., & Papadopoulou, A. (2025) · Nutrients · doi:10.3390/nu17030487 · Cited by 20
Romoli, M., Urbinati, G., Tudisco, V., et al. (2025) · Neurology · doi:10.1212/wnl.0000000000213426 · Cited by 14
Kumar, G., Barhoun, P., Fuelscher, I. T., Zwicker, J. G., Bianco, K. M., Singh, M., Mukherjee, M., Williams, J., & Hyde, C. E. (2025) · Research in Developmental Disabilities · doi:10.1016/j.ridd.2025.104981 · Cited by 15
Scharré, S., Mengler, K., Schnabel‐Besson, E., Hübschmann, O. K., Tuncel, A. T., Hoffmann, G. F., Garbade, S. F., Mütze, U., & Kölker, S. (2024) · Genetics in Medicine · doi:10.1016/j.gim.2024.101303 · Cited by 15
Liu, Q., Welsh, P. I., Celis-Morales, C. A., Ho, F. K. W., Lees, J. S., & Mark, P. B. (2025) · Clinical Kidney Journal · doi:10.1093/ckj/sfaf003 · Cited by 17
“Because all the selected studies were observational studies, their qualities were independently assessed by two reviewers ( Q.L., P.W.) using the Risk Of Bias In Non-randomized Studies—of Exposure ( ROBINS-E) , which was designed for evaluating observational studies [26 ].”
Teaching trauma-informed care in undergraduate medical education: A scoping review
Bell, S. B., Miller‐Cribbs, J., Meireles, J. F. F., Clifton, S. C., Bright, R., Moon, A., Jamshed, H., Wen, F. K., & Jelley, M. (2025) · BMC Health Services Research · doi:10.1186/s12913-025-13086-w · Cited by 15
“Literature determined to be of very low quality by the standards of ROBINS-E (Risk of Bias in Non-Randomized Studies of Exposures) [21] was eligible for exclusion upon consensus of the authors.”
Wallerer, S., Stadelmaier, J., Floegel, E. F., Kiesswetter, E., Bantle, G., Hoffmann, G., & Schwingshackl, L. (2025) · Advances in Nutrition · doi:10.1016/j.advnut.2025.100562 · Cited by 15
“However, estimates that included adjustment for BMI were not prioritized, as BMI is considered a potential mediator in the causal pathway between MedDiet adherence and T2D risk, and adjusting for it could lead to overadjustment bias [17,18].”
EFSA Panel on Nutrition, N. F. A. F. A. (., Turck, D., Bohn, T., et al. (2024) · EFSA Journal · doi:10.2903/j.efsa.2024.9052 · Cited by 13
Etesami, E., Nikparast, A., Rahmani, J., Rezaei, M., & Ghanavati, M. (2025) · Food & Function · doi:10.1039/d4fo04741a · Cited by 13
The Association of Lipoprotein(a) and Stroke Recurrence: A Systematic Review and Meta-Analysis
Palaiodimou, L., Melanis, K., Stefanou, M., et al. (2025) · Journal of Stroke · doi:10.5853/jos.2024.04623 · Cited by 13
“10 Any disagreements were settled by consensus after discussion with the corresponding author (GT).”
Arenas, M. B., Arroyo-Sánchez, A., Torres-Parejo, Ú., & Muñoz-Negro, J. E. (2025) · International Journal of Social Psychiatry · doi:10.1177/00207640251331898 · Cited by 12
“…to explore the risk of bias they might induce and the overall risk of the revision, in the form of the ROBVIS (Risk-Of-Bias VISualization) R package and web app (McGuinness & Higgins, 2021), through the ROBINS-E (Risk-Of-Bias In Non-randomized Studies – of Exposure) tool (Higgins et al., 2024).”
Pérez, I. M. M., Bourhim, S., & Pérez, S. E. M. (2025) · Applied Sciences · doi:10.3390/app15115978 · Cited by 12
Nguyen, T. Q., Vlasenko, D., Shetty, A. N., Zhao, E., Reid, C. M., Clothier, H. J., & Buttery, J. P. (2025) · Cardiovascular Research · doi:10.1093/cvr/cvaf092 · Cited by 13
Gil, A. M., Correa, J. E. B., Flórez-Suárez, J. B., Fernández-Ávila, D. G., & Cucunubá, Z. M. (2025) · Seminars in Arthritis and Rheumatism · doi:10.1016/j.semarthrit.2025.152805 · Cited by 13
No meta-analytical effect of economic inequality on well-being or mental health
Sommet, N., Fillon, A. A., Rudmann, O., Cunha, A. R. S., & Ehsan, A. (2025) · Nature · doi:10.1038/s41586-025-09797-z · Cited by 19
“Following ROBINS-E, we started from a baseline of high certainty 75 .”
Magi, C. E., Aoufy, K. E., Amato, C., Longobucco, Y., Bambi, S., Vellone, E., Bonaccorsi, G., Lorini, C., Rasero, L., & Iovino, P. (2026) · Journal of Clinical Nursing · doi:10.1111/jocn.70291 · Cited by 17
From London buses to activity trackers: A reflection of 70 years of physical activity research
Ding, D., & Ekelund, U. (2024) · Journal of sport and health science/Journal of Sport and Health Science · doi:10.1016/j.jshs.2024.06.001 · Cited by 13
Various biases in systematic review and meta-analysis and their assessment
Nair, A. S., & Borkar, N. B. (2025) · Indian Journal of Anaesthesia · doi:10.4103/ija.ija_1212_24 · Cited by 14
Ding, J., Fu, R., Yuan, T., Brenner, H., & Hoffmeister, M. (2025) · BMC Public Health · doi:10.1186/s12889-025-21537-6 · Cited by 17
A Systematic Review of Regenerative Medicine Therapies for Axial Spine Pain of Facet Joint Origin
Manchikanti, L., Abd-Elsayed, A. A., Kaye, A. D., Sanapati, M. R., Pampati, V., Shekoohi, S., & Hirsch, J. A. (2025) · Current Pain and Headache Reports · doi:10.1007/s11916-025-01376-1 · Cited by 13
Characteristics of Hearing Loss in Type 2 Diabetes Mellitus: A Systematic Review and Meta‐Analysis
Kim, C. H., Lauritsen, K. L., Nguyen, S. A., Meyer, T. A., Cumpston, E. C., Pelic, J. C., & Labadie, R. F. (2025) · Otolaryngology · doi:10.1002/ohn.1378 · Cited by 13
“E) 12 for cohort studies and the Joanna Briggs Institute (JBI) critical appraisal checklist for cross ‐ sectional and case ‐ control studies.”
Bircak-Kuchtova, B., Rose, N., Geis, C., Finke, K., Pletz, M. W., Chung, H., & Fleischmann-Struzek, C. (2025) · Critical Care · doi:10.1186/s13054-025-05585-3 · Cited by 14
“The risk of bias in non-randomized studies was assessed using ROBINS-I-Tool, with judgments in each domain categorized as ‘very low‘, ‘low‘, ‘moderate‘ or ‘high‘ [12].”
Outcomes of Left Atrial Appendage Patency After Device Closure Detected by Cardiac CT
Jacquemyn, X., Sá, M. P., Korsholm, K., Nielsen‐Kudsk, J. E., Iriart, X., Saw, J., Alkhouli, M. A., Jain, S., Bittencourt, M. S., & Sultan, I. S. (2025) · JACC. Cardiovascular imaging · doi:10.1016/j.jcmg.2025.07.018 · Cited by 14
Multifaceted Antibiotic Resistance in Diabetic Foot Infections: A Systematic Review
Li, W., Sadeh, O., Chakraborty, J., et al. (2025) · Microorganisms · doi:10.3390/microorganisms13102311 · Cited by 12
“Studies that scored <50%, 50–75%, and >75% were deemed of high, moderate, and low risk of bias, respectively, based on a collective decision taken by the investigators in line with the ROBINS-E tool guidelines [11,12].”
Grzonka, P., Mosimann, T., Berger, S., Amacher, S. A., Baumann, S. M., Gebhard, C. E., Marchis, G. M. D., Dittrich, T. D., & Sutter, R. (2025) · Systematic Reviews · doi:10.1186/s13643-025-02797-0 · Cited by 11
Karakasis, P., Lefkou, E. E., Pamporis, K., Farmakis, D. T., Patoulias, D., Antoniadis, A. P., Heymans, S. R. B., Filippatos, G. S., & Fragakis, N. (2025) · European Journal of Heart Failure · doi:10.1002/ejhf.3637 · Cited by 13
Organizational return on investment in nursing: A systematic review
Yakusheva, O., Lee, K. A., Fial, A., & Weiss, M. (2025) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2025.105146 · Cited by 11
Bottosso, M., Zurlo, C., Miglietta, F., Cattelan, A. C., Iannaccone, D., Dieci, M. V., Griguolo, G., Girardi, F., & Guarneri, V. (2025) · The Breast · doi:10.1016/j.breast.2025.104544 · Cited by 12
“Risk of bias assessment for each included study was evaluated by two independent investigators (MB and CZ) using the ROBINS-E tool, and potential disagreements were discussed with a third reviewer (FG) [11].”
The Prognostic Value of Right Ventricle–Pulmonary Artery Coupling in Valve Interventions
Androshchuk, V., Long, E., Chehab, O., et al. (2025) · JACC: Cardiovascular Interventions · doi:10.1016/j.jcin.2025.10.024 · Cited by 10
Marín‐Jiménez, N., Bizzozero‐Peroni, B., Molina‐García, P., Ortega, F. B., Chaput, J., Zhang, K., Lang, J. J., McGrath, R. P., Tomkinson, G. R., Martínez-Vizcaíno, V., et al. (2026) · British Journal of Sports Medicine · doi:10.1136/bjsports-2024-109173 · Cited by 11
Middleton, D. R. S., Wedekind, R., Hall, A. L., et al. (2024) · Occupational and Environmental Medicine · doi:10.1136/oemed-2023-109277 · Cited by 10
“14 Such tools emphasise the potential for bias as determined systematically by reviewers, rather than the Monographs’ expert-based appraisal of the quality and informativeness of the exposure data for addressing the causal question about exposure to an agent and cancer.”
Obstetric Outcomes in Women on Lithium: A Systematic Review and Meta-Analysis
Callovini, T., Montanari, S., Bardi, F., Barbonetti, S., Rossi, S., Caso, R., Mandracchia, G., Margoni, S., Brugnami, A., Paolini, M., et al. (2024) · Journal of Clinical Medicine · doi:10.3390/jcm13164872 · Cited by 10
“The Cochrane RoB tool we used [23] does not address publication bias; however, this bias does not appear to have influenced our results, since our search strategy did not exclude studies with negative results, nor those published in languages other than English, and we directly contacted authors of…”
BackgroundWang, Y., Chang, J., Hu, P., Deng, C., Luo, Z., Zhao, J., Zhang, Z., Yi, W., Zhu, G., Zheng, G., et al. (2024) · Environmental Pollution · doi:10.1016/j.envpol.2024.124991 · Cited by 11
Aladawi, M., Elfil, M., Ghozy, S. A., Najdawi, Z. R., Ghaith, H. S., Alzayadneh, M., Rabinstein, A. A., & Hawkes, M. A. (2024) · Journal of Stroke and Cerebrovascular Diseases · doi:10.1016/j.jstrokecerebrovasdis.2024.108084 · Cited by 10
Metadata
- Authors
- Higgins, J. P. T., Morgan, R. L., Rooney, A. A., Taylor, K. W., Thayer, K. A., Silva, R. A., Lemeris, C., Akl, E. A., Bateson, T. F., Berkman, N. D., Glenn, B. S., Hróbjartsson, A., LaKind, J. S., McAleenan, A., Meerpohl, J. J., Nachman, R. M., Obbagy, J. E., O'Connor, A., Radke, E. G., Savović, J., Schünemann, H. J., Shea, B., Tilling, K., Verbeek, J., Viswanathan, M., & Sterne, J. A. C.
- Affiliations
- University of Bristol (GB)
- McMaster University (CA)
- National Institute of Environmental Health Sciences (US)
- Environmental Protection Agency (US)
- ICF International (United States) (US)
- American University of Beirut (LB)
- RTI International (US)
- University of Southern Denmark (DK)
- University of Freiburg (DE)
- United States Department of Agriculture (US)
- Michigan State University (US)
- Cochrane Canada (CA)
- Ottawa Hospital Research Institute (CA)
- Amsterdam UMC Location University of Amsterdam (NL)
- Corresponding author
- Higgins, Julian P.T.
- ORCID
- Higgins, Julian P.T.
- Morgan, Rebecca L.
- Rooney, Andrew A.
- Taylor, Kyla W.
- Thayer, Kristina A.
- Silva, Raquel A.
- Lemeris, Courtney
- Akl, Elie A.
- Bateson, Thomas F.
- Berkman, Nancy D.
- Hróbjartsson, Asbjørn
- LaKind, Judy S.
- McAleenan, Alexandra
- Meerpohl, Joerg J.
- Nachman, Rebecca M.
- Obbagy, Julie E.
- O'Connor, Annette
- Radke, Elizabeth G.
- Savović, Jelena
- Schünemann, Holger J.
- Shea, Beverley
- Tilling, Kate
- Verbeek, Jos
- Viswanathan, Meera
- Sterne, Jonathan A.C.
- Venue
- Environment International
- Publisher
- Elsevier BV
- Published
- 2024-04
- Volume
- 186
- Pages
- 108602
- ISSN
- 0160-4120 (ISSN-L)
- PMC
- PMC11098530
- OpenAlex
- W4393132304
- Semantic Scholar
- b822b2b6998a5003935cf5815b5370685f4e5231
- Cited by
- 1,022
- References
- 30
- Citations by year
- 2026: 470 · 2025: 440 · 2024: 112
- FWCI
- 284.27
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Environment International (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Pure Amsterdam UMC (publishedVersion, cc-by)
- University of Southern Denmark Research Portal (University of Southern Denmark) (publishedVersion, cc-by-nc)
- PubMed Central (acceptedVersion, cc-by-nc)
- Europe PMC (PubMed Central) (acceptedVersion, cc-by-nc)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Bristol Research (University of Bristol) (submittedVersion, cc-by-nc-nd)
- Explore Bristol Research (submittedVersion, cc-by-nc-nd)
- Funders
- National Institute for Health and Care Research
- Cancer Research UK
- National Institute of Environmental Health Sciences
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- risk of bias assessment, non-randomized studies, cohort studies, causal inference, systematic review methodology, observational epidemiology
- MeSH terms
- Environmental Exposure, Follow-Up Studies, Humans, Cohort Studies, Bias, Epidemiologic Studies, Risk Assessment, Observational Studies as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
