The DOI matches its Crossref record; 22 of the 30 references it cites were found in a registry or an index; none of the 27 sources its claims cite is checked for support yet.
A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E)
Open accessAbstract
BACKGROUND: Observational epidemiologic studies provide critical data for the evaluation of the potential effects of environmental, occupational and behavioural exposures on human health. Systematic reviews of these studies play a key role in informing policy and practice. Systematic reviews should incorporate assessments of the risk of bias in results of the included studies. OBJECTIVE: To develop a new tool, Risk Of Bias In Non-randomized Studies - of Exposures (ROBINS-E) to assess risk of bias in estimates from cohort studies of the causal effect of an exposure on an outcome. METHODS AND RESULTS: ROBINS-E was developed by a large group of researchers from diverse research and public health disciplines through a series of working groups, in-person meetings and pilot testing phases. The tool aims to assess the risk of bias in a specific result (exposure effect estimate) from an individual observational study that examines the effect of an exposure on an outcome. A series of preliminary considerations informs the core ROBINS-E assessment, including details of the result being assessed and the causal effect being estimated. The assessment addresses bias within seven domains, through a series of 'signalling questions'. Domain-level judgements about risk of bias are derived from the answers to these questions, then combined to produce an overall risk of bias judgement for the result, together with judgements about the direction of bias. CONCLUSION: ROBINS-E provides a standardized framework for examining potential biases in results from cohort studies. Future work will produce variants of the tool for other epidemiologic study designs (e.g. case-control studies). We believe that ROBINS-E represents an important development in the integration of exposure assessment, evidence synthesis and causal inference.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
ROBINS-E is a new tool for assessing risk of bias in a specific exposure effect estimate from non-randomized cohort studies, developed by adapting ROBINS-I and RoB 2. It uses preliminary considerations, seven bias domains, signalling questions, and domain-level judgements to produce overall risk-of-bias, direction-of-bias, and threat-to-conclusions judgements.
Key findings
- ROBINS-E assesses risk of bias in a specific result from an observational cohort study of the causal effect of an exposure on an outcome, addressing seven domains: confounding, exposure measurement, selection of participants, post-exposure interventions, missing data, outcome measurement, and selection of the reported result.
- Development involved 43 researchers; proposals were discussed at a two-day meeting of 31 participants in Bristol in January 2017, and a draft was piloted in October 2019 by 28 in-person plus 3 remote participants; a revised draft was piloted in June–July 2021, and the launch version was posted in June 2022.
- Domain-level judgements include risk of bias (“Low risk of bias”, “Some concerns”, “High risk of bias”, “Very high risk of bias”), predicted direction of bias, and whether the bias threatens conclusions; the default overall risk-of-bias judgement is that of the domain with the greatest risk of bias.
- ROBINS-E does not directly assess financial conflicts of interest, and the present version addresses only follow-up (cohort) studies, with variants for other designs such as case-control studies planned.
Methods
- Design: iterative tool development through working groups, in-person meetings, webinars, and pilot testing, following principles used for RoB 2 and ROBINS-I.
- Data or sample: no empirical dataset was used; development involved 43 identified researchers, a 31-participant meeting in January 2017, a 2019 pilot with 28 in-person plus 3 remote participants, and a 2021 revised-draft pilot.
- Main analysis: not stated in the paper; the article describes the tool’s structure, domains, signalling questions, and judgement algorithms rather than conducting a statistical analysis.
Limitations
- Stated: the present version addresses only follow-up (cohort) studies, and work is ongoing to develop variants for other study designs including case-control studies.
- Stated: assessments require judgement, so subjectivity and between-assessor variability are impossible to avoid; accuracy depends on the quality of available information, and successful use requires both content and methodological expertise.
- Stated/evident: the tool has not yet had formal evaluation of its reliability and validity; the paper reports no empirical reliability or validity data, and the data availability statement says no data were used for the research described.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
20 Own interpretation · 16 Not checked yet · 14 No basis given · 2 Own method
In the paper
“although all such studies identified related to randomized trials”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Development of ROBINS-E
What the sources say
- Not checked yetcites Sterne et al., 2002 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“ROBINS-E does not include direct assessment of financial conflicts of interest, which may be important in interpreting the findings and credibility of studies of exposures and have been recommended for inclusion in evaluation of environmental studies (National Academies of Sciences Engineering and Medicine, 2022).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
What the sources say
- Not checked yetcites National Academies of Sciences Engineering and Medicine, 2022
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“different tools, and different ways of addressing risk-of-bias assessments, can lead to differences in conclusions of systematic reviews”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Introduction
What the sources say
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Recently-developed tools for this purpose have moved away from numeric scores based on checklists (which often combined elements of risk of bias with other considerations such as of applicability) to tools based on assessments within specific domains of bias.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Introduction
What the sources say
- Not checked yetcites National Toxicology Program, 2015
- Not checked yetcites Woodruff and Sutton, 2014 source
- Not checked yetcites U.S. Environmental Protection Agency, 2022
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“The second variant additionally addresses time-varying confounding, which can occur when the exposure changes over time and follow-up time is split according to exposure level (Hernán and Robins, 2020; Mansournia et al., 2017).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
What the sources say
- Not checked yetcites Hernán and Robins, 2020
- Not checked yetcites Mansournia et al., 2017
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For this reason, ROBINS-E assessments may support causal inferences, if sufficient evidence at ‘low’ risk of bias or with ‘some concerns’ about risk of bias can be located.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“ROBINS-E can be used as part of a framework to rate the quality of the evidence for a group of studies. An example of such a framework is the GRADE approach (Guyatt et al., 2011)”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
What the sources say
- Not checked yetcites Guyatt et al., 2011 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It is impossible to exclude the possibility of uncontrolled confounding in a non-randomized study”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Furthermore, there are impacts of such conflicts that are not problems of internal validity (targeted by ROBINS-E) but instead relate to what question is being asked in the study or analysis.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“The rationale, previously discussed in the context of randomized trials (Sterne, 2013), is that if financial conflicts of interest lead to bias then this will operate through one of the domains already in the tool.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
What the sources say
- Not checked yetcites Sterne, 2013 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- It may state two separate claims as one. 9
- Probably not a substantial claim. 37
None of the sources cited for this claim have been checked yet.
In the paper
“The instrument was based on the ROBINS-I tool for non-randomized studies of interventions (Sterne et al., 2016) and RoB 2 tool for randomized trials (Sterne et al., 2019), and was informed by previous instruments for assessing studies of exposure.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
What the sources say
- Not checked yetcites Sterne et al., 2016
- Not checked yetcites Sterne et al., 2019
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“ROBINS-E is a new general-purpose tool for assessing risk of bias in results of non-randomized studies of exposure.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The present version addresses only follow-up (cohort) studies”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Important elements in ROBINS-E include a definition of the causal effect of interest, signalling questions to guide domain-level risk-of-bias judgements across seven domains, algorithms to derive suggested risk of bias judgements from answers to the signalling questions, and the separation of risk of bias from other important considerations in the evaluation of study sensitivity.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“These additional considerations, which fall outside the scope of the main ROBINS-E assessment, are addressed in an optional section of ROBINS-E.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Study sensitivity and appropriateness of its design
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“However, the user may override this, for example if they judge that so many domains were rated as at ‘High’ risk of bias that the overall judgement should be ‘Very high’ risk of bias.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Overall risk of bias
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The default ROBINS-E overall risk-of-bias judgement is that for the domain with the greatest risk of bias.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Overall risk of bias
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The default judgement for the threat to conclusions is derived in a similar way to the overall risk-of-bias judgement; for example, the overall judgement is ‘Yes’ if there is considered to be a threat to conclusions in any of the seven bias domains.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Overall risk of bias
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“ROBINS-E provides a standardized framework for examining the range of biases common to observational studies of exposures.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“ROBINS-E is designed primarily for use in the context of a systematic review.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“The collaborative development of the tool, between methodologists and stakeholders from fields such as environmental, occupational, chemical, clinical and public health, has highlighted the benefits of sharing ideas and research methods across communities (Hoffmann et al., 2022; Hoffmann et al., 2022).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
What the sources say
- Not checked yetcites Hoffmann et al., 2022
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“An optional section of ROBINS-E allows users to record observations about study sensitivity, aspects of which may additionally feed into the “indirectness” domain within GRADE, as it relates to generalizability of the result of the study to the population, exposure and outcome of interest.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Several such tools for observational epidemiologic studies provide significant methodological advances over previously used tools (National Toxicology Program, 2015, 2015; Woodruff and Sutton, 2014; U.S. Environmental Protection Agency, 2022).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Introduction
What the sources say
- Not checked yetcites National Toxicology Program, 2015
- Not checked yetcites Woodruff and Sutton, 2014 source
- Not checked yetcites U.S. Environmental Protection Agency, 2022
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Early work by a subset of the authors considered the extent to which the ROBINS-I tool was suitable for observational studies of exposures and suggested modifications to the tool (Morgan et al., 2018; Morgan et al., 2019).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Development of ROBINS-E
What the sources say
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“The Risk Of Bias In Non-randomized Studies - of Interventions (ROBINS-I) tool was developed to evaluate estimates of the effects of deliberate interventions aimed to alter health outcomes (Sterne et al., 2016).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Introduction
What the sources say
- Not checked yetcites Sterne et al., 2016
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Domain-level judgements about risk of bias are derived from the answers to these questions, then combined to produce an overall risk of bias judgement for the result, together with judgements about the direction of bias.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Abstract > Methods and results:
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The extent of the potential for bias across a body of evidence for an exposure-outcome effect informs assessment of the certainty of evidence for that effect.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A screening section then facilitates identification of results that are at very high risk of bias, allowing the user to avoid a detailed risk of bias assessment (Fig. 1, part B).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Preliminary considerations
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“There is a risk of bias when such selection is based on the magnitude, direction or P value of the result.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“First, evaluation of risk of bias requires that the review team has expertise in both content (detailed subject-matter knowledge) and methodology (understanding of the different types of bias that may distort the results of observational studies).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Bias due to post-exposure interventions may arise if the presence or level of exposure leads to administration of interventions that change the subsequent risk of the outcome.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Bias in selection of the reported result arises when study authors select results from a multiplicity of analyses, for example from different ways of measuring the exposure, different ways of measuring the outcome, different subsets of the full study sample or different analyses.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The ROBINS-E risk-of-bias assessment addresses seven domains of bias (Fig. 1, Section 5).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Overview of ROBINS-E
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- The authors state this as an assumption: Issues common to both randomized trials and observational studies (missing data, outcome assessment, selection of the reported result) are likely to be similar.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“By contrast, it is possible to envisage situations in which risk of bias for other domains is low: for example, if exposure is measured without error, there are no missing data, or results are reported in accordance with a pre-specified analysis plan.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A series of preliminary considerations informs the core ROBINS-E assessment, including details of the result being assessed and the causal effect being estimated.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Abstract > Methods and results:
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, such analyses require that the factors leading to the post-exposure intervention can be modelled.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Although ROBINS-E was iteratively developed based on the results of pilot testing with a diverse group”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Bias arising from measurement of the outcome is a problem when error in measurement of the outcome is related to the presence or level of exposure, in which case the measurement error is differential rather than non-differential.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Third, ROBINS-E assessments require judgements based on the available information. It is therefore impossible to avoid subjectivity and between-assessor variability.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ROBINS-E provides a standardized framework for examining potential biases in results from cohort studies.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Abstract > Conclusion:
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- The authors state this as an assumption: Issues common to both randomized trials and observational studies (missing data, outcome assessment, selection of the reported result) are likely to be similar.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This question can be manipulated to give an illusion of presence or lack or effect (for example by studying trivial levels of exposure or irrelevant outcomes), but this need not have implications for internal validity.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Practical considerations can impact on the successful implementation of ROBINS-E.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Each domain is addressed using a series of signalling questions that aim to gather relevant information about the study and analysis being assessed.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Overview of ROBINS-E
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The first version of ROBINS-E, described here, targets cohort (follow-up) studies.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Introduction
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“ROBINS-E was developed by a large group of researchers from diverse research and public health disciplines through a series of working groups, in-person meetings and pilot testing phases.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Abstract > Methods and results:
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Because risk of bias is assessed in relation to a defined causal effect of an exposure on an outcome, a low risk of bias corresponds to an absence of bias in estimating that causal effect.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, publication of answers to signalling questions and justification for these should make risk of bias judgements transparent.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Bias in selection of participants into the study (or into the analysis) is the first of two domains that address selection bias, which arises when we condition, in the study design or in the analysis, on common effects of exposure and outcome (Hernán and Robins, 2020; Hernán et al., 2004).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
What the sources say
- Not checked yetcites Hernán and Robins, 2020
- Not checked yetcites Hernán et al., 2004 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Second, the accuracy of a risk-of-bias assessment will depend on the quality of information available about the study being assessed.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper relies on this without support.
Randomized controlled trials are often not feasible, ethical, large enough or of sufficient duration to evaluate chronic health effects.
“Randomized controlled trials (RCTs) are often not feasible, ethical, large enough or of sufficient duration to evaluate chronic health effects.”
The authors state this as an assumption.
Issues common to both randomized trials and observational studies (missing data, outcome assessment, selection of the reported result) are likely to be similar.
“with the additional assumption that issues common to both study designs (missing data, outcome assessment, selection of the reported result) are likely to be similar”
The paper relies on this without support.
Pre-specified plans for analyses are uncommon for observational studies.
“However, such pre-specified plans are uncommon for observational studies and often the nature of the data need to be taken into account when selecting a suitable statistical model and method.”
The paper relies on this without support.
The nature of the data often needs to be taken into account when selecting a suitable statistical model and method.
“However, such pre-specified plans are uncommon for observational studies and often the nature of the data need to be taken into account when selecting a suitable statistical model and method.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“For cohort studies, the Risk of Bias in Non‐randomized Studies of Exposures (ROBINS‐E) tool was used.”
From Risk of cardiac, neuropsychiatric and musculoskeletal adverse events with levothyroxine: Systematic review (2026) · cites this paper as [12]
Citations
Cites
RoB 2: a revised tool for assessing risk of bias in randomised trials
Sterne, J. A. C., Savović, J., Page, M. J., Elbers, R. G., Blencowe, N. S., Boutron, I., Cates, C. J., Cheng, H.-Y., Corbett, M. S., Eldridge, S. M., et al. (2019) · BMJ · doi:10.1136/bmj.l4898 · Cited by 34,787
Self-citationDermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,586
GRADE: an emerging consensus on rating quality of evidence and strength of recommendations
Guyatt, G. H., Oxman, A. D., Vist, G. E., Kunz, R., Falck-Ytter, Y., Alonso-Coello, P., & Schünemann, H. J. (2008) · BMJ · doi:10.1136/bmj.39489.470347.ad · Cited by 23,305 · cited 1×
“…of interest; measurement of exposure; measurement of outcomes; attrition and selective reporting; integration of ROBINS-E with the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) framework (Guyatt et al., 2008); and issues in statistical analysis (see Online Supplement).”
Self-citationROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329
Self-citationCochrane Handbook for Systematic Reviews of Interventions
(2019) · doi:10.1002/9781119536604 · Cited by 14,828
GRADE guidelines: 1. Introduction—GRADE evidence profiles and summary of findings tables
Guyatt, G., Oxman, A. D., Akl, E. A., Kunz, R., Vist, G., Brozek, J., Norris, S., Falck-Ytter, Y., Glasziou, P., & deBeer, H. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.04.026 · Cited by 10,681 · cited 1×
“An example of such a framework is the GRADE approach (Guyatt et al., 2011) where users may start at high certainty in the “study limitations” (risk of bias) domain of GRADE and use their ROBINS-E evaluation to determine how many levels to rate down this certainty (Schunemann et al., 2018).”
Self-citationA structural approach to selection bias
Hernán, M. A., Hernández-Díaz, S., & Robins, J. M. (2004) · Epidemiology · doi:10.1097/01.ede.0000135174.63482.43 · Cited by 2,844 · cited 1×
“Bias in selection of participants into the study (or into the analysis) is the first of two domains that address selection bias, which arises when we condition, in the study design or in the analysis, on common effects of exposure and outcome (Hernán and Robins, 2020; Hernán et al., 2004).”
Triangulation in aetiological epidemiology
Lawlor, D. A., Tilling, K., & Davey Smith, G. (2017) · International Journal of Epidemiology · doi:10.1093/ije/dyw314 · Cited by 1,129 · cited 1×
“…together in a triangulation exercise, in which both the risks EPA A u t ho r M anu sc r i p t EPA A u t ho r M anu sc r i p t EPA A u t ho r M anu sc r i p t of bias (assessed by ROBINS-E) and differences between causal questions being addressed are considered jointly (Lawlor et al., 2016).”
Self-citationSchünemann, H. J., Cuello, C., Akl, E. A., Mustafa, R. A., Meerpohl, J. J., Thayer, K., Morgan, R. L., Gartlehner, G., Kunz, R., Katikireddi, S. V., et al. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.01.012 · Cited by 911 · cited 1×
“An example of such a framework is the GRADE approach (Guyatt et al., 2011) where users may start at high certainty in the “study limitations” (risk of bias) domain of GRADE and use their ROBINS-E evaluation to determine how many levels to rate down this certainty (Schunemann et al., 2018).”
Self-citationWoodruff, T. J., & Sutton, P. (2014) · Environmental Health Perspectives · doi:10.1289/ehp.1307175 · Cited by 427 · cited 1×
“Several such tools for observational epidemiologic studies provide significant methodological advances over previously used tools (National Toxicology Program, 2015, 2015; Woodruff and Sutton, 2014; U.S. Environmental Protection Agency, 2022).”
Research works printed but not linked: 2
- Boutron, 2019, Chapter 7: considering bias and conflicts of interest among the included studies, 177
- National Academies of Sciences Engineering and Medicine, 2022. A Review of U.S. EPA’s ORD Staff Handbook for Developing IRIS Assessments: 2020 Version. Washington DC: The National Academies Press.
Cited by
Sorted by citations
Items 501–600 of 966
Chowdhury, A., Sheikh, F., Azab, S., et al. (2026) · BMJ Open · doi:10.1136/bmjopen-2025-109126 · Cited by 1
Batu, N. I., Kamika, I. A., & Malefetse, T. J. (2026) · Journal of Microbiological Methods · doi:10.1016/j.mimet.2026.107414 · Cited by 2
Novo, G., Stabile, F. A., Lisi, D. D., et al. (2026) · Cardio-Oncology · doi:10.1186/s40959-025-00435-1 · Cited by 1
Interventions to increase vaccine uptake among socially excluded groups: A systematic review
Sidebotham, E., Caldwell, D. M., Davies, S. R., et al. (2026) · Vaccine · doi:10.1016/j.vaccine.2026.128314 · Cited by 1
Santagata, D., Subri, F., Vegliach, C., et al. (2026) · Thrombosis and Haemostasis · doi:10.1055/a-2810-4737 · Cited by 1
Nguyen, B. A. H., Quon, S., Hazan, E., et al. (2026) · Journal of the Canadian Association of Gastroenterology · doi:10.1093/jcag/gwag004 · Cited by 1
Henderson, L. A., Mitchell, T. J., Steele, C. W., et al. (2026) · Critical Reviews in Oncology/Hematology · doi:10.1016/j.critrevonc.2026.105215 · Cited by 1
Sguazzini, E., Montagna, C., Nayfeh, T., et al. (2026) · Frontiers in Cellular and Infection Microbiology · doi:10.3389/fcimb.2026.1716483 · Cited by 1
Berton, G. G., Rodrigues, A. C. L. F., Mattei, N. M., Calza, F. D., Dias, R. H., Menegat, É. L., Brkanitch, I. D. A., LAGO, J. A., & Hadley, G. (2026) · Fluids and Barriers of the CNS · doi:10.1186/s12987-025-00757-3 · Cited by 2
Bakhshimoghaddam, F., Chaharlang, R., Shirani, M., Ghamari, M., & Rastaghi, S. (2026) · BMC Cancer · doi:10.1186/s12885-026-15677-0 · Cited by 2
Elwenspoek, M. M. C., O’Donnell, R., Manzano, C. L., Buss, L., Dawson, S., Charlwood, K., Harding, T. A., Stokes, C., Palma, F., Hay, A. D., et al. (2026) · The Lancet Primary Care · doi:10.1016/j.lanprc.2025.100099 · Cited by 1
Borca, C.-I., Ivan, C.-S., Fira-Mlădinescu, C., Margan, R., Margan, M.-M., Cîndrea, A. C., Balasa-Virzob, C.-R., Vlaicu, B., & David, V.-L. (2026) · Children · doi:10.3390/children13030326 · Cited by 1
Empathy and burnout in healthcare professionals: protocol for a systematic review and meta-analysis
Edwards, R. J., Nockels, K., & Howick, J. (2026) · BMJ Open · doi:10.1136/bmjopen-2025-108207 · Cited by 1
Abdelrazek, K. M., Soliman, F. E. E., Hassan, M., Ewida, A. H., & Ibrahim, W. (2026) · African Journal of Emergency Medicine · doi:10.1016/j.afjem.2025.100942 · Cited by 1
Zouzoulas, D., Karalis, T., Sofianou, I., et al. (2026) · Medical Sciences · doi:10.3390/medsci14010120 · Cited by 1
Gutierrez-Del-Val, G., Serrano-Fernandez, V., Mazoteras-Pardo, V., et al. (2026) · Scientific Reports · doi:10.1038/s41598-026-42949-3 · Cited by 1
Gupta, D. K., Mustaque, A. M., & Kumar, R. (2026) · Cureus · doi:10.7759/cureus.104825 · Cited by 1
Combating distorted information through third person perception advances in a meta-analysis
Li, X., Lyu, W., Salleh, S. M., & Ahmad, A. L. (2026) · Current Psychology · doi:10.1007/s12144-026-09212-4 · Cited by 1
Khadivi, Y., Tavallaeinejad, P., Hashemipour, M., et al. (2026) · Metabolism · doi:10.1016/j.metabol.2026.156591 · Cited by 1
Treatment of Central Serous Chorioretinopathy With Topical Difluprednate: A Literature Review
Chatterjee, S. K., & Khuddus, N. (2026) · Cureus · doi:10.7759/cureus.105298 · Cited by 1
Merola, E., Dore, M. P., & Fanciulli, G. (2026) · Cancers · doi:10.3390/cancers18060964 · Cited by 1
Sebastian, S. A., Ayyalu, T., Bimal, T., Patel, B., Kittleson, M. M., Carbone, S., & Yehya, A. (2026) · Disease-a-Month · doi:10.1016/j.disamonth.2026.102099 · Cited by 2
Wan, C., Ge, M., Li, W., et al. (2026) · Korean Circulation Journal · doi:10.4070/kcj.2025.0443 · Cited by 1
Gardella, B., Spinillo, A., Dominoni, M., Tiranini, L., Cassani, C., & Nappi, R. E. (2026) · The Journal of Sexual Medicine · doi:10.1093/jsxmed/qdag072 · Cited by 1
Air pollution and adverse birth outcomes: A systematic review of fine critical windows
Boitel, A., Cot, S., Pujol, S., Martin, B., Mottet, N., Gauthier-Manuel, H., Mauny, F., & Bernard, N. (2026) · Environmental Research · doi:10.1016/j.envres.2026.124294 · Cited by 3
Autoimmunity following SARS-CoV-2 infection and vaccination: Systematic review and meta-analysis
Anouar, O., Elghali, M., Zanina, Y., Changuel, M., & Sakly, N. (2026) · Clinical Immunology · doi:10.1016/j.clim.2026.110702 · Cited by 1
Qannas, F. A., Nurgozhina, M., Owen, A. J., Gašević, D., Polkinghorne, K., Bongetti, E. K., Chen, Y., & Wild, H. (2026) · Nutrition Reviews · doi:10.1093/nutrit/nuag037 · Cited by 1
Mah, A. E., Ranjbar, D., Chai, B. S., et al. (2026) · Plastic Surgery · doi:10.1177/22925503261436341 · Cited by 1
Radiographic paraspinal muscle size in adult spinal deformity: a systematic review and meta-analysis
Yamanouchi, K., de Oliveira, R. G., Younus, I., Nguyen, K. T., Reed, T. F., Bansal, A., Nemani, V. M., Leveque, J.-C. A., & Louie, P. K. (2026) · Spine Deformity · doi:10.1007/s43390-026-01341-x · Cited by 2
Optimizing infrastructure for better education: a TOPSIS-based study of Moroccan universities
Kaaouas, A., Kaaouas, N., Alami, I., et al. (2026) · Frontiers in Education · doi:10.3389/feduc.2026.1752186 · Cited by 1
Burma, J. S., Nel, J. H., Cheung, H., Hofforth, D., Zala, D., Hayden, A., Ronksley, P. E., Carty, C. L., Chang, M., Gross, A. L., et al. (2026) · Frontiers in Neuroendocrinology · doi:10.1016/j.yfrne.2026.101250 · Cited by 2
Carvalho, R. A. M., Borges, D. F., Casalta-Lopes, J., et al. (2026) · Epilepsy & Behavior · doi:10.1016/j.yebeh.2026.111035 · Cited by 1
Varma, A., Bhakta, S., Dohle, E., et al. (2026) · Age and Ageing · doi:10.1093/ageing/afag080 · Cited by 1
Eshraghi, R., Bahrami, A., Kosari, M., MD-MPH, K. V., Keyhani, A., Ebrahimi, P., Amini‐Salehi, E., Amani‐Beni, R., Afkhami, S., Yazdani, M. S., et al. (2026) · Nutrition Metabolism and Cardiovascular Diseases · doi:10.1016/j.numecd.2026.104764 · Cited by 2
Gil‐Lespinard, M., Iglesias‐Vázquez, L., & Jakszyn, P. G. (2026) · European Journal of Nutrition · doi:10.1007/s00394-026-03948-2 · Cited by 2
Liver Enzyme Biomarkers Before or in Early Pregnancy as Predictors for Gestational Diabetes Mellitus
Pratama, S., Aulia, R. D., Jazilah, I., et al. (2026) · Journal of the ASEAN Federation of Endocrine Societies · doi:10.15605/jafes.041.01.5807 · Cited by 1
Fortune, A., Gallagher, N., Okanlawon, A., Murphy, J., Lee, A. R., Gladstone, M., & Sinha, I. P. (2026) · Archives of Disease in Childhood · doi:10.1136/archdischild-2025-329096 · Cited by 1
Shah, S., Chen, Y., owen, A., Ekegren, C. L., Talic, S., Ilić, D., & Gašević, D. (2026) · American Journal of Health Promotion · doi:10.1177/08901171261438402 · Cited by 5
Bonell, A., Qureshi, A., Flouris, A. D., Tsoutsoubi, L., Sferruzzi-Perri, A. N., Das, J. K., Hirst, J. E., & Ioannou, L. G. (2026) · The Lancet Planetary Health · doi:10.1016/j.lanplh.2026.101431 · Cited by 6
Asghari, S., Rezaee, S., Khaneshi, N., et al. (2026) · BMC Oral Health · doi:10.1186/s12903-026-08420-5 · Cited by 1
Ge, Z., Marck, C. H., Gupta, A., et al. (2026) · Obesity Reviews · doi:10.1111/obr.70150 · Cited by 1
Marcos-Pardo, P. J., Mateo‐Orcajada, A., Vale, R. G. D. S., & Vaquero‐Cristóbal, R. (2026) · European Journal of Investigation in Health Psychology and Education · doi:10.3390/ejihpe16050066 · Cited by 2
Sawesi, S., Sabbineni, H., Shagamreddy, R. R., & Rashrash, B. (2026) · Journal of Multidisciplinary Healthcare · doi:10.2147/jmdh.s581251 · Cited by 5
Johnstone, S., Schenkel, A., Canazzi, S., Eiden, R. D., & Hawk, L. W. (2026) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntag118 · Cited by 2
Zhou, Y., Zhang, X., Gui, W., et al. (2026) · Journal of Thrombosis and Thrombolysis · doi:10.1007/s11239-026-03330-3 · Cited by 1
Glymphatic system dysfunction in gliomas: A systematic review and meta-analysis of DTI-ALPS index
Dehaghi, F. T., Nasab, A. S., Arabi, N., et al. (2026) · Journal of Neuroradiology · doi:10.1016/j.neurad.2026.101572 · Cited by 1
Weese, J. S., Soltero‐Rivera, M., Weese, H. E., et al. (2026) · Journal of Small Animal Practice · doi:10.1111/jsap.70147 · Cited by 1
Pérez-Manchón, D., Lozano-Hernández, C., Mata-González, G., Arias-Arias, Á. J., Mayoral-Gonzalo, N., Córdoba, M. G. D. Q., & Ruiz‐Zaldibar, C. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1839569 · Cited by 1
Calabrò, R. S., Calderone, A., Nunzio, A. M. D., Cerasa, A., & Quartarone, A. (2026) · Building and Environment · doi:10.1016/j.buildenv.2026.114849 · Cited by 1
Liguoro, I., D'Agostini, M., Franco, F., et al. (2026) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/clinem/dgag238 · Cited by 1
Environmental and health implications of emerging pollutants in landfill leachate: a review
Macaulay, O. A., Sobola, A. O., & Obayori, O. S. (2026) · Discover Hazards · doi:10.1007/s44475-026-00042-3 · Cited by 1
Fridén, M., Mai, M., Olsen, A., Dahm, C. C., & Ibsen, D. B. (2026) · BMC Medicine · doi:10.1186/s12916-026-05028-y · Cited by 4
Pang, K. H., Tong, K. S., Muneer, A., & Alnajjar, H. M. (2026) · International Journal of Impotence Research · doi:10.1038/s41443-026-01315-7 · Cited by 2
Davis, R., Wood, L., Wilkes, L., Chandler, H. L., Anderson, E. L., & Hiscox, L. V. (2026) · NeuroImage Clinical · doi:10.1016/j.nicl.2026.104035 · Cited by 1
Go, A. R., Sacueza, L. R., & Regencia, Z. J. G. (2026) · Discover Public Health · doi:10.1186/s12982-026-02532-7 · Cited by 1
Behzad, H., Asham, H., Beheshtirouy, S., et al. (2024) · Perfusion · doi:10.1177/02676591241257371 · Cited by 1
Obesity in rheumatoid arthritis: impact on response to biologic therapies
Irshad, Z., & Gullick, N. (2024) · Exploration of Musculoskeletal Diseases · doi:10.37349/emd.2024.00050 · Cited by 1
Saadi, S. M., Hasan, B., Kanaan, A., Abusalih, M. E., Tarakji, Z., Sadek, M., Basha, A. S., Firwana, M., Wang, Z., & Murad, M. H. (2024) · Cochrane Evidence Synthesis and Methods · doi:10.1002/cesm.12094 · Cited by 3
Awor, S., Bongomin, F., Kaggwa, M. M., et al. (2024) · Systematic Reviews · doi:10.1186/s13643-024-02662-6 · Cited by 1
Vitus, E. S., Mann, S., Lees, C. W., et al. (2024) · Gastro Hep Advances · doi:10.1016/j.gastha.2024.10.021 · Cited by 1
Pritha, A. N., Medha, T. N., Pasmay, A. A., Mamun, M. M. A., Afroze, F., & Chisti, M. J. (2024) · Journal of Global Health · doi:10.7189/jogh-14-04207 · Cited by 2
Supit, W., Supit, T., & Sherryn, S. (2024) · UroPrecision · doi:10.1002/uro2.92 · Cited by 1
Benz, F., Hanna, S., Harvey, N. C., Vollmer, B., Köster, M., Villar, J., & Fernandes, M. (2025) · Pediatric Investigation · doi:10.1002/ped4.12460 · Cited by 1
Khalaf, R. K. S., Akaraci, S., Baldwin, F. D., Geary, R. S., Kolamunnage‐Dona, R., Hunter, R. F., & Rodgers, S. (2025) · Environmental Research Health · doi:10.1088/2752-5309/ada8ce · Cited by 1
Nag, R., Bhola, P., Janardhana, N., Warriner, L., Prabhu, S. G., Arelingaiah, M., Krupa, A., Jayalaxmi, K. P., & Hugh‐Jones, S. (2025) · School Psychology International · doi:10.1177/01430343241308549 · Cited by 1
Wagner, C.-S., Lithopoulos, A., Shorr, R., et al. (2025) · Infectious Diseases Now · doi:10.1016/j.idnow.2025.105050 · Cited by 1
New Icebergs in Evidence‐Based Medicine
de Alencar, J. N. (2025) · Journal of Evidence-Based Medicine · doi:10.1111/jebm.70028 · Cited by 1
Villalba, N., Byrne, K., Abdelmageed, S., Votoupal, M., Lam, S. K., & García, R. M. (2025) · BMC Public Health · doi:10.1186/s12889-025-22407-x · Cited by 1
Impact of Antidepressants on Male Fertility and Seminal Parameters: A Systematic Review
dos Santos, L. O. D. P., da Silva, P. R. Q., Gonçalves, N., Perini, H. F., & Siervo, G. E. M. D. L. (2025) · Reproductive Sciences · doi:10.1007/s43032-025-01858-y · Cited by 1
Sakazaki, A., Lui, A., Wang, M., Ngo, K., Lugue, M. T., Aligireddy, H., Nguyen, M., Castro, C., Park, K. S., & Murakami, S. (2025) · Cureus · doi:10.7759/cureus.83806 · Cited by 1
Meireles, A. S., da Silva, G. C. P. S. M., Linhares, D. G., et al. (2025) · Sport Sciences for Health · doi:10.1007/s11332-025-01459-x · Cited by 1
Hedin, E., Tungsanga, S., Feng, Y., Kung, J. Y. C., Okpechi, I. G., Dart, A. B., Thompson, S., Morgan, C., Erickson, R. L., Chaturvedi, S., et al. (2025) · BMJ Open · doi:10.1136/bmjopen-2024-090615 · Cited by 1
Schlesinger, S., Conrad, J., Amini, A. M., Buyken, A. E., Egert, S., Haardt, J., Kalotai, N., Kroke, A., Schwingshackl, L., & On behalf of the German Nutrition Society (2025) · European Journal of Nutrition · doi:10.1007/s00394-025-03744-4 · Cited by 1
Jiang, J. Y., Li, A., Romero, A., Flowers, C. R., & Nze, C. C. (2025) · The Oncologist · doi:10.1093/oncolo/oyaf180 · Cited by 1
Su, L., & Jin, Y. (2025) · European Journal of Preventive Cardiology · doi:10.1093/eurjpc/zwaf362 · Cited by 1
Pakpahan, C., Lionardi, S., Handini, L. P., Lionardi, S. K., Alvianto, S., Rinaldi, F. X., & Iskandar, A. F. (2025) · International Journal of Sexual Health · doi:10.1080/19317611.2025.2527881 · Cited by 1
Updating the Scientific Advisory Committee on Nutrition’s Framework for the evaluation of evidence
Singh, M., Elsom, R. L., Cullum, A., de Souza, R. J., Greenwood, D. C., Mela, D. J., Ong, K. K., Young, I. S., & Lovegrove, J. A. (2025) · British Journal Of Nutrition · doi:10.1017/s0007114525104054 · Cited by 1
Stanaway, F., Khalatbari‐Soltani, S., Zhu, L., Sreeharan, T., Noguchi, N., Pathak, A., Mathieu, E., & Bhopal, R. S. (2025) · The Lancet Regional Health - Europe · doi:10.1016/j.lanepe.2025.101498 · Cited by 1
Fuge, T. G., Miller, E. R., & Ersado, T. L. (2025) · Research Square · doi:10.21203/rs.3.rs-6671884/v1 · Cited by 1
Health Impacts of Micro- and Nanoplastics in Humans: Systematic Review of In Vivo EvidencePreprint
Anna, H. A. T., Tang, D. H., Yong, C. T., Ng, J. J., & Ho, A. F. W. (2025) · medRxiv · doi:10.1101/2025.07.10.25331209 · Cited by 2
Cheo, H. M., Lee, F., Lee, A., Ong, X., Koh, L. P., Chan, M. Y., Wee, I. J. Y., Ong, J. S. P., & Sia, C. (2026) · medRxiv · doi:10.64898/2026.01.02.26343368 · Cited by 1
Restar, A., Rodriguez, M. I., Dusic, E., et al. (2026) · Research Square · doi:10.21203/rs.3.rs-9887690/v1 · Cited by 1
Iniesta‐Sepúlveda, M., & Ríos, A. (2024) · Cirugía Española · doi:10.1016/j.ciresp.2024.04.013
A Systematic Review of Cortical Excitability during Dual-Task in Post-Stroke Patients
Jung, S., & An, C.-S. (2024) · Physical therapy rehabilitation science · doi:10.14474/ptrs.2024.13.2.71
Evaluation of Systematic Reviews and Meta-analyses
Dziadkowiec, O. (2024) · Journal of Obstetric, Gynecologic & Neonatal Nursing · doi:10.1016/j.jogn.2024.07.006
Evidence Communication Rules for Policy (ECR-P) critical appraisal toolPreprint
Danopoulos, E., Aston, J. A. D., Shah, A., & Schneider, C. R. (2024) · doi:10.31235/osf.io/shwfu
Opioid consumption following foot and ankle surgery: a systematic review
DeMaio, C. V., Nayfa, R., Reasnor, T. D., Fox, J. A., Burrow, Z., Thompson, S. F., & Haleem, A. M. (2024) · Journal of the Foot & Ankle · doi:10.30795/jfootankle.2024.v18.1766
Holis, R. V., van Dalem, J., Munshi, R., et al. (2024) · HRB Open Research · doi:10.12688/hrbopenres.13959.1
Jiang, T., Cai, Z., Liu, C., & Zhang, B. (2024) · PLoS ONE · doi:10.1371/journal.pone.0310752
The Seasonality of Childhood Bone and Joint Infection with Focus on Kingella kingae
Hunter, S., & Crawford, H. (2024) · JBJS Reviews · doi:10.2106/jbjs.rvw.24.00149
Ronca, D. B., Mesquita, L. O., Oliveira, D. B., Figueiredo, A. C. M. G., Wen, J., Song, M., & de Carvalho, K. M. B. (2024) · medRxiv · doi:10.1101/2024.11.20.24317635
RapidAI for Stroke Detection and AI Implementation Review
CDA-AMC (2024) · Canadian Journal of Health Technologies · doi:10.51731/cjht.2024.1032 · Cited by 1
Sebastian, S. A., Vaja, H., Shah, Y. R., et al. (2024) · SN Comprehensive Clinical Medicine · doi:10.1007/s42399-024-01772-5
Systematic Review and Meta-analysis in Digestive Cancer Research
Gwon, S. H., & Son, N. (2024) · Journal of Digestive Cancer Research · doi:10.52927/jdcr.2024.12.3.207
Noguez, C. A., Contreras‐Ramos, A., Melo, M. E. L., et al. (2025) · REC CardioClinics · doi:10.1016/j.rccl.2024.12.003
Martins, É. L. C., Constantino, P., & de Oliveira, G. L. A. (2025) · BMC Public Health · doi:10.1186/s12889-024-20859-1
Gong, S., Jacob, B. M., Harris, Á., Raval, K., Clarke, N., Doyle, F., Smith, A. D., Cotter, S., Walsh, K. R., & Redmond, P. (2025) · HRB Open Research · doi:10.12688/hrbopenres.13926.1
Carvalho, P. B., Maffazioli, G. D. N., Simões, R. S., Macchione, R. F., de Medeiros, S. F., Soares, J. M., Baracat, E. C., & Maciel, G. A. R. (2025) · medRxiv · doi:10.1101/2025.01.25.25321110
Bhanot, A., Hawton, K., Perry, R. A., Miller-Molloy, F., Blest, F., Easter, S., Hamilton-Shield, J. P., & Candler, T. P. (2025) · Diabetic Medicine · doi:10.1111/dme.70005
Stanhope, E. N., Thomsen, S. N., Turner, J. E., et al. (2025) · JCSM Communications · doi:10.1002/rco2.115
Metadata
- Authors
- Higgins, J. P. T., Morgan, R. L., Rooney, A. A., Taylor, K. W., Thayer, K. A., Silva, R. A., Lemeris, C., Akl, E. A., Bateson, T. F., Berkman, N. D., Glenn, B. S., Hróbjartsson, A., LaKind, J. S., McAleenan, A., Meerpohl, J. J., Nachman, R. M., Obbagy, J. E., O'Connor, A., Radke, E. G., Savović, J., Schünemann, H. J., Shea, B., Tilling, K., Verbeek, J., Viswanathan, M., & Sterne, J. A. C.
- Affiliations
- University of Bristol (GB)
- McMaster University (CA)
- National Institute of Environmental Health Sciences (US)
- Environmental Protection Agency (US)
- ICF International (United States) (US)
- American University of Beirut (LB)
- RTI International (US)
- University of Southern Denmark (DK)
- University of Freiburg (DE)
- United States Department of Agriculture (US)
- Michigan State University (US)
- Cochrane Canada (CA)
- Ottawa Hospital Research Institute (CA)
- Amsterdam UMC Location University of Amsterdam (NL)
- Corresponding author
- Higgins, Julian P.T.
- ORCID
- Higgins, Julian P.T.
- Morgan, Rebecca L.
- Rooney, Andrew A.
- Taylor, Kyla W.
- Thayer, Kristina A.
- Silva, Raquel A.
- Lemeris, Courtney
- Akl, Elie A.
- Bateson, Thomas F.
- Berkman, Nancy D.
- Hróbjartsson, Asbjørn
- LaKind, Judy S.
- McAleenan, Alexandra
- Meerpohl, Joerg J.
- Nachman, Rebecca M.
- Obbagy, Julie E.
- O'Connor, Annette
- Radke, Elizabeth G.
- Savović, Jelena
- Schünemann, Holger J.
- Shea, Beverley
- Tilling, Kate
- Verbeek, Jos
- Viswanathan, Meera
- Sterne, Jonathan A.C.
- Venue
- Environment International
- Publisher
- Elsevier BV
- Published
- 2024-04
- Volume
- 186
- Pages
- 108602
- ISSN
- 0160-4120 (ISSN-L)
- PMC
- PMC11098530
- OpenAlex
- W4393132304
- Semantic Scholar
- b822b2b6998a5003935cf5815b5370685f4e5231
- Cited by
- 1,022
- References
- 30
- Citations by year
- 2026: 470 · 2025: 440 · 2024: 112
- FWCI
- 284.27
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Environment International (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Pure Amsterdam UMC (publishedVersion, cc-by)
- University of Southern Denmark Research Portal (University of Southern Denmark) (publishedVersion, cc-by-nc)
- PubMed Central (acceptedVersion, cc-by-nc)
- Europe PMC (PubMed Central) (acceptedVersion, cc-by-nc)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Bristol Research (University of Bristol) (submittedVersion, cc-by-nc-nd)
- Explore Bristol Research (submittedVersion, cc-by-nc-nd)
- Funders
- National Institute for Health and Care Research
- Cancer Research UK
- National Institute of Environmental Health Sciences
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- risk of bias assessment, non-randomized studies, cohort studies, causal inference, systematic review methodology, observational epidemiology
- MeSH terms
- Environmental Exposure, Follow-Up Studies, Humans, Cohort Studies, Bias, Epidemiologic Studies, Risk Assessment, Observational Studies as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
