The DOI matches its Crossref record; 22 of the 30 references it cites were found in a registry or an index; none of the 27 sources its claims cite is checked for support yet.
A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E)
Open accessAbstract
BACKGROUND: Observational epidemiologic studies provide critical data for the evaluation of the potential effects of environmental, occupational and behavioural exposures on human health. Systematic reviews of these studies play a key role in informing policy and practice. Systematic reviews should incorporate assessments of the risk of bias in results of the included studies. OBJECTIVE: To develop a new tool, Risk Of Bias In Non-randomized Studies - of Exposures (ROBINS-E) to assess risk of bias in estimates from cohort studies of the causal effect of an exposure on an outcome. METHODS AND RESULTS: ROBINS-E was developed by a large group of researchers from diverse research and public health disciplines through a series of working groups, in-person meetings and pilot testing phases. The tool aims to assess the risk of bias in a specific result (exposure effect estimate) from an individual observational study that examines the effect of an exposure on an outcome. A series of preliminary considerations informs the core ROBINS-E assessment, including details of the result being assessed and the causal effect being estimated. The assessment addresses bias within seven domains, through a series of 'signalling questions'. Domain-level judgements about risk of bias are derived from the answers to these questions, then combined to produce an overall risk of bias judgement for the result, together with judgements about the direction of bias. CONCLUSION: ROBINS-E provides a standardized framework for examining potential biases in results from cohort studies. Future work will produce variants of the tool for other epidemiologic study designs (e.g. case-control studies). We believe that ROBINS-E represents an important development in the integration of exposure assessment, evidence synthesis and causal inference.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
ROBINS-E is a new tool for assessing risk of bias in a specific exposure effect estimate from non-randomized cohort studies, developed by adapting ROBINS-I and RoB 2. It uses preliminary considerations, seven bias domains, signalling questions, and domain-level judgements to produce overall risk-of-bias, direction-of-bias, and threat-to-conclusions judgements.
Key findings
- ROBINS-E assesses risk of bias in a specific result from an observational cohort study of the causal effect of an exposure on an outcome, addressing seven domains: confounding, exposure measurement, selection of participants, post-exposure interventions, missing data, outcome measurement, and selection of the reported result.
- Development involved 43 researchers; proposals were discussed at a two-day meeting of 31 participants in Bristol in January 2017, and a draft was piloted in October 2019 by 28 in-person plus 3 remote participants; a revised draft was piloted in June–July 2021, and the launch version was posted in June 2022.
- Domain-level judgements include risk of bias (“Low risk of bias”, “Some concerns”, “High risk of bias”, “Very high risk of bias”), predicted direction of bias, and whether the bias threatens conclusions; the default overall risk-of-bias judgement is that of the domain with the greatest risk of bias.
- ROBINS-E does not directly assess financial conflicts of interest, and the present version addresses only follow-up (cohort) studies, with variants for other designs such as case-control studies planned.
Methods
- Design: iterative tool development through working groups, in-person meetings, webinars, and pilot testing, following principles used for RoB 2 and ROBINS-I.
- Data or sample: no empirical dataset was used; development involved 43 identified researchers, a 31-participant meeting in January 2017, a 2019 pilot with 28 in-person plus 3 remote participants, and a 2021 revised-draft pilot.
- Main analysis: not stated in the paper; the article describes the tool’s structure, domains, signalling questions, and judgement algorithms rather than conducting a statistical analysis.
Limitations
- Stated: the present version addresses only follow-up (cohort) studies, and work is ongoing to develop variants for other study designs including case-control studies.
- Stated: assessments require judgement, so subjectivity and between-assessor variability are impossible to avoid; accuracy depends on the quality of available information, and successful use requires both content and methodological expertise.
- Stated/evident: the tool has not yet had formal evaluation of its reliability and validity; the paper reports no empirical reliability or validity data, and the data availability statement says no data were used for the research described.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 of 27 cited sources checked · 0 supported · 0 flagged
In the paper
“although all such studies identified related to randomized trials”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Development of ROBINS-E
What the sources say
- Not checked yetcites Sterne et al., 2002 source
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIn the paper
“ROBINS-E does not include direct assessment of financial conflicts of interest, which may be important in interpreting the findings and credibility of studies of exposures and have been recommended for inclusion in evaluation of environmental studies (National Academies of Sciences Engineering and Medicine, 2022).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
What the sources say
- Not checked yetcites National Academies of Sciences Engineering and Medicine, 2022
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesNone of the sources cited for this claim have been checked yet.
In the paper
“different tools, and different ways of addressing risk-of-bias assessments, can lead to differences in conclusions of systematic reviews”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Introduction
What the sources say
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesNone of the sources cited for this claim have been checked yet.
In the paper
“Recently-developed tools for this purpose have moved away from numeric scores based on checklists (which often combined elements of risk of bias with other considerations such as of applicability) to tools based on assessments within specific domains of bias.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Introduction
What the sources say
- Not checked yetcites National Toxicology Program, 2015
- Not checked yetcites Woodruff and Sutton, 2014 source
- Not checked yetcites U.S. Environmental Protection Agency, 2022
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesNone of the sources cited for this claim have been checked yet.
In the paper
“The second variant additionally addresses time-varying confounding, which can occur when the exposure changes over time and follow-up time is split according to exposure level (Hernán and Robins, 2020; Mansournia et al., 2017).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
What the sources say
- Not checked yetcites Hernán and Robins, 2020
- Not checked yetcites Mansournia et al., 2017
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For this reason, ROBINS-E assessments may support causal inferences, if sufficient evidence at ‘low’ risk of bias or with ‘some concerns’ about risk of bias can be located.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIn the paper
“ROBINS-E can be used as part of a framework to rate the quality of the evidence for a group of studies. An example of such a framework is the GRADE approach (Guyatt et al., 2011)”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
What the sources say
- Not checked yetcites Guyatt et al., 2011 source
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It is impossible to exclude the possibility of uncontrolled confounding in a non-randomized study”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes? Based: unclearA conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Furthermore, there are impacts of such conflicts that are not problems of internal validity (targeted by ROBINS-E) but instead relate to what question is being asked in the study or analysis.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIn the paper
“The rationale, previously discussed in the context of randomized trials (Sterne, 2013), is that if financial conflicts of interest lead to bias then this will operate through one of the domains already in the tool.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
What the sources say
- Not checked yetcites Sterne, 2013 source
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- It may state two separate claims as one. 9
- Probably not a substantial claim. 37
None of the sources cited for this claim have been checked yet.
In the paper
“The instrument was based on the ROBINS-I tool for non-randomized studies of interventions (Sterne et al., 2016) and RoB 2 tool for randomized trials (Sterne et al., 2019), and was informed by previous instruments for assessing studies of exposure.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
What the sources say
- Not checked yetcites Sterne et al., 2016
- Not checked yetcites Sterne et al., 2019
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“ROBINS-E is a new general-purpose tool for assessing risk of bias in results of non-randomized studies of exposure.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The present version addresses only follow-up (cohort) studies”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Important elements in ROBINS-E include a definition of the causal effect of interest, signalling questions to guide domain-level risk-of-bias judgements across seven domains, algorithms to derive suggested risk of bias judgements from answers to the signalling questions, and the separation of risk of bias from other important considerations in the evaluation of study sensitivity.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“These additional considerations, which fall outside the scope of the main ROBINS-E assessment, are addressed in an optional section of ROBINS-E.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Study sensitivity and appropriateness of its design
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“However, the user may override this, for example if they judge that so many domains were rated as at ‘High’ risk of bias that the overall judgement should be ‘Very high’ risk of bias.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Overall risk of bias
Strength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The default ROBINS-E overall risk-of-bias judgement is that for the domain with the greatest risk of bias.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Overall risk of bias
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The default judgement for the threat to conclusions is derived in a similar way to the overall risk-of-bias judgement; for example, the overall judgement is ‘Yes’ if there is considered to be a threat to conclusions in any of the seven bias domains.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Overall risk of bias
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“ROBINS-E provides a standardized framework for examining the range of biases common to observational studies of exposures.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“ROBINS-E is designed primarily for use in the context of a systematic review.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
In the paper
“The collaborative development of the tool, between methodologists and stakeholders from fields such as environmental, occupational, chemical, clinical and public health, has highlighted the benefits of sharing ideas and research methods across communities (Hoffmann et al., 2022; Hoffmann et al., 2022).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
What the sources say
- Not checked yetcites Hoffmann et al., 2022
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“An optional section of ROBINS-E allows users to record observations about study sensitivity, aspects of which may additionally feed into the “indirectness” domain within GRADE, as it relates to generalizability of the result of the study to the population, exposure and outcome of interest.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Several such tools for observational epidemiologic studies provide significant methodological advances over previously used tools (National Toxicology Program, 2015, 2015; Woodruff and Sutton, 2014; U.S. Environmental Protection Agency, 2022).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Introduction
What the sources say
- Not checked yetcites National Toxicology Program, 2015
- Not checked yetcites Woodruff and Sutton, 2014 source
- Not checked yetcites U.S. Environmental Protection Agency, 2022
Strength:boosted· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Early work by a subset of the authors considered the extent to which the ROBINS-I tool was suitable for observational studies of exposures and suggested modifications to the tool (Morgan et al., 2018; Morgan et al., 2019).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Development of ROBINS-E
What the sources say
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“The Risk Of Bias In Non-randomized Studies - of Interventions (ROBINS-I) tool was developed to evaluate estimates of the effects of deliberate interventions aimed to alter health outcomes (Sterne et al., 2016).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Introduction
What the sources say
- Not checked yetcites Sterne et al., 2016
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Domain-level judgements about risk of bias are derived from the answers to these questions, then combined to produce an overall risk of bias judgement for the result, together with judgements about the direction of bias.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Abstract > Methods and results:
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The extent of the potential for bias across a body of evidence for an exposure-outcome effect informs assessment of the certainty of evidence for that effect.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A screening section then facilitates identification of results that are at very high risk of bias, allowing the user to avoid a detailed risk of bias assessment (Fig. 1, part B).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Preliminary considerations
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“There is a risk of bias when such selection is based on the magnitude, direction or P value of the result.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“First, evaluation of risk of bias requires that the review team has expertise in both content (detailed subject-matter knowledge) and methodology (understanding of the different types of bias that may distort the results of observational studies).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Bias due to post-exposure interventions may arise if the presence or level of exposure leads to administration of interventions that change the subsequent risk of the outcome.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Bias in selection of the reported result arises when study authors select results from a multiplicity of analyses, for example from different ways of measuring the exposure, different ways of measuring the outcome, different subsets of the full study sample or different analyses.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The ROBINS-E risk-of-bias assessment addresses seven domains of bias (Fig. 1, Section 5).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Overview of ROBINS-E
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
- The authors state this as an assumption: Issues common to both randomized trials and observational studies (missing data, outcome assessment, selection of the reported result) are likely to be similar.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“By contrast, it is possible to envisage situations in which risk of bias for other domains is low: for example, if exposure is measured without error, there are no missing data, or results are reported in accordance with a pre-specified analysis plan.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
Strength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A series of preliminary considerations informs the core ROBINS-E assessment, including details of the result being assessed and the causal effect being estimated.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Abstract > Methods and results:
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, such analyses require that the factors leading to the post-exposure intervention can be modelled.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Although ROBINS-E was iteratively developed based on the results of pilot testing with a diverse group”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Bias arising from measurement of the outcome is a problem when error in measurement of the outcome is related to the presence or level of exposure, in which case the measurement error is differential rather than non-differential.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Third, ROBINS-E assessments require judgements based on the available information. It is therefore impossible to avoid subjectivity and between-assessor variability.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ROBINS-E provides a standardized framework for examining potential biases in results from cohort studies.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Abstract > Conclusion:
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
- The authors state this as an assumption: Issues common to both randomized trials and observational studies (missing data, outcome assessment, selection of the reported result) are likely to be similar.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This question can be manipulated to give an illusion of presence or lack or effect (for example by studying trivial levels of exposure or irrelevant outcomes), but this need not have implications for internal validity.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Practical considerations can impact on the successful implementation of ROBINS-E.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
Strength:plain· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Each domain is addressed using a series of signalling questions that aim to gather relevant information about the study and analysis being assessed.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Overview of ROBINS-E
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The first version of ROBINS-E, described here, targets cohort (follow-up) studies.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Introduction
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“ROBINS-E was developed by a large group of researchers from diverse research and public health disciplines through a series of working groups, in-person meetings and pilot testing phases.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Abstract > Methods and results:
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Because risk of bias is assessed in relation to a defined causal effect of an exposure on an outcome, a low risk of bias corresponds to an absence of bias in estimating that causal effect.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, publication of answers to signalling questions and justification for these should make risk of bias judgements transparent.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
Strength:plain· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Bias in selection of participants into the study (or into the analysis) is the first of two domains that address selection bias, which arises when we condition, in the study design or in the analysis, on common effects of exposure and outcome (Hernán and Robins, 2020; Hernán et al., 2004).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
What the sources say
- Not checked yetcites Hernán and Robins, 2020
- Not checked yetcites Hernán et al., 2004 source
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Second, the accuracy of a risk-of-bias assessment will depend on the quality of information available about the study being assessed.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper relies on this without support.
Randomized controlled trials are often not feasible, ethical, large enough or of sufficient duration to evaluate chronic health effects.
“Randomized controlled trials (RCTs) are often not feasible, ethical, large enough or of sufficient duration to evaluate chronic health effects.”
The authors state this as an assumption.
Issues common to both randomized trials and observational studies (missing data, outcome assessment, selection of the reported result) are likely to be similar.
“with the additional assumption that issues common to both study designs (missing data, outcome assessment, selection of the reported result) are likely to be similar”
The paper relies on this without support.
Pre-specified plans for analyses are uncommon for observational studies.
“However, such pre-specified plans are uncommon for observational studies and often the nature of the data need to be taken into account when selecting a suitable statistical model and method.”
The paper relies on this without support.
The nature of the data often needs to be taken into account when selecting a suitable statistical model and method.
“However, such pre-specified plans are uncommon for observational studies and often the nature of the data need to be taken into account when selecting a suitable statistical model and method.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“For cohort studies, the Risk of Bias in Non‐randomized Studies of Exposures (ROBINS‐E) tool was used.”
From Risk of cardiac, neuropsychiatric and musculoskeletal adverse events with levothyroxine: Systematic review (2026) · cites this paper as [12]
Citations
Cites
RoB 2: a revised tool for assessing risk of bias in randomised trials
Sterne, J. A. C., Savović, J., Page, M. J., Elbers, R. G., Blencowe, N. S., Boutron, I., Cates, C. J., Cheng, H.-Y., Corbett, M. S., Eldridge, S. M., et al. (2019) · BMJ · doi:10.1136/bmj.l4898 · Cited by 34,787
Self-citationDermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,586
GRADE: an emerging consensus on rating quality of evidence and strength of recommendations
Guyatt, G. H., Oxman, A. D., Vist, G. E., Kunz, R., Falck-Ytter, Y., Alonso-Coello, P., & Schünemann, H. J. (2008) · BMJ · doi:10.1136/bmj.39489.470347.ad · Cited by 23,305 · cited 1×
“…of interest; measurement of exposure; measurement of outcomes; attrition and selective reporting; integration of ROBINS-E with the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) framework (Guyatt et al., 2008); and issues in statistical analysis (see Online Supplement).”
Self-citationROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329
Self-citationCochrane Handbook for Systematic Reviews of Interventions
(2019) · doi:10.1002/9781119536604 · Cited by 14,828
GRADE guidelines: 1. Introduction—GRADE evidence profiles and summary of findings tables
Guyatt, G., Oxman, A. D., Akl, E. A., Kunz, R., Vist, G., Brozek, J., Norris, S., Falck-Ytter, Y., Glasziou, P., & deBeer, H. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.04.026 · Cited by 10,681 · cited 1×
“An example of such a framework is the GRADE approach (Guyatt et al., 2011) where users may start at high certainty in the “study limitations” (risk of bias) domain of GRADE and use their ROBINS-E evaluation to determine how many levels to rate down this certainty (Schunemann et al., 2018).”
Self-citationA structural approach to selection bias
Hernán, M. A., Hernández-Díaz, S., & Robins, J. M. (2004) · Epidemiology · doi:10.1097/01.ede.0000135174.63482.43 · Cited by 2,844 · cited 1×
“Bias in selection of participants into the study (or into the analysis) is the first of two domains that address selection bias, which arises when we condition, in the study design or in the analysis, on common effects of exposure and outcome (Hernán and Robins, 2020; Hernán et al., 2004).”
Triangulation in aetiological epidemiology
Lawlor, D. A., Tilling, K., & Davey Smith, G. (2017) · International Journal of Epidemiology · doi:10.1093/ije/dyw314 · Cited by 1,129 · cited 1×
“…together in a triangulation exercise, in which both the risks EPA A u t ho r M anu sc r i p t EPA A u t ho r M anu sc r i p t EPA A u t ho r M anu sc r i p t of bias (assessed by ROBINS-E) and differences between causal questions being addressed are considered jointly (Lawlor et al., 2016).”
Self-citationSchünemann, H. J., Cuello, C., Akl, E. A., Mustafa, R. A., Meerpohl, J. J., Thayer, K., Morgan, R. L., Gartlehner, G., Kunz, R., Katikireddi, S. V., et al. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.01.012 · Cited by 911 · cited 1×
“An example of such a framework is the GRADE approach (Guyatt et al., 2011) where users may start at high certainty in the “study limitations” (risk of bias) domain of GRADE and use their ROBINS-E evaluation to determine how many levels to rate down this certainty (Schunemann et al., 2018).”
Self-citationWoodruff, T. J., & Sutton, P. (2014) · Environmental Health Perspectives · doi:10.1289/ehp.1307175 · Cited by 427 · cited 1×
“Several such tools for observational epidemiologic studies provide significant methodological advances over previously used tools (National Toxicology Program, 2015, 2015; Woodruff and Sutton, 2014; U.S. Environmental Protection Agency, 2022).”
Flagged first, then by citations
8 of the 30 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- Boutron, 2019, Chapter 7: considering bias and conflicts of interest among the included studies, 177
- National Academies of Sciences Engineering and Medicine, 2022. A Review of U.S. EPA’s ORD Staff Handbook for Developing IRIS Assessments: 2020 Version. Washington DC: The National Academies Press.
Cited by
Sorted by citations
Items 401–500 of 966
Labrèche, F., Prud’homme, P., Gagnon, M., Dupré, N., & Gravel, S. (2026) · Occupational and Environmental Medicine · doi:10.1136/oemed-2025-110662 · Cited by 2
Joseph, T., Gao, C., Nadarajah, R., Al-Lehebi, R. O., Alves, L., Berg, D. D., Couper, A., Datta, I., Eleyan, L., Haris, M., et al. (2026) · EClinicalMedicine · doi:10.1016/j.eclinm.2026.104131 · Cited by 2
A Systematic Review of Cortical Excitability during Dual-Task in Post-Stroke Patients
Jung, S., & An, C.-S. (2024) · Physical therapy rehabilitation science · doi:10.14474/ptrs.2024.13.2.213 · Cited by 2
AlJaafary, A. A., Alahmari, R., & Alhumaid, N. A. (2024) · Cureus · doi:10.7759/cureus.67721 · Cited by 2
Smida, T., Dayal, S., Bardes, J. M., et al. (2024) · Prehospital Emergency Care · doi:10.1080/10903127.2024.2408628 · Cited by 2
Predictors of perinatal mortality in Ghana: a systematic review protocol
Punguyire, D., Abiiro, G. A., & Koray, M. H. (2024) · BMJ Open · doi:10.1136/bmjopen-2023-080527 · Cited by 2
Rijnhart, J. J. M., Rabbers, A., & Rizzuto, S. (2024) · International Journal of Epidemiology · doi:10.1093/ije/dyae147 · Cited by 2
Strazda, A. E., Laouris, P., Luxhøj, C. M., et al. (2024) · EJC Skin Cancer · doi:10.1016/j.ejcskn.2024.100276 · Cited by 2
Increasing the utility of epidemiologic studies as key evidence in chemical risk assessment
Schaefer, H. R., Vincent, M., Burns, C. J., & Lange, S. S. (2024) · Toxicological Sciences · doi:10.1093/toxsci/kfae134 · Cited by 2
Chichagi, F. (2024) · American Journal of Cardiovascular Disease · doi:10.62347/joym3506 · Cited by 2
Schwartz, F. D., Pride, T. M., Stewart, S. H., Mackinnon, S. P., & Robinson, M. (2025) · BMJ Open · doi:10.1136/bmjopen-2023-082276 · Cited by 2
Azene, Z. N., MacPhail, C. L., & Smithers, L. G. (2025) · PLoS ONE · doi:10.1371/journal.pone.0318585 · Cited by 3
Vacharaksa, K., Thongrakard, T., & Vacharaksa, A. (2025) · Health Policy · doi:10.1016/j.healthpol.2025.105262 · Cited by 2
Purnomo, H. D., Permatadewi, C. O., Hutami, H. T., et al. (2025) · Applied Sciences · doi:10.3390/app15042111 · Cited by 2
Moloney, L. M., Chacón, V., Devarakonda, S. L. S., Scollard, T., Jones, S., Rozga, M., & Handu, D. (2025) · Journal of the Academy of Nutrition and Dietetics · doi:10.1016/j.jand.2025.03.005 · Cited by 3
Brunwasser, S. M., Warner, A. K., Rosas-Salazar, C., & Wu, P. (2025) · Journal of Allergy and Clinical Immunology · doi:10.1016/j.jaci.2025.04.002 · Cited by 2
Joshi, U., Shah, T., Gohil, V. S., et al. (2025) · Cureus · doi:10.7759/cureus.82874 · Cited by 2
Suicidal risk in patients with aggression in schizophrenia: a systematic review
Bravve, L. V., Kaydan, M. A., & Kostyuk, G. P. (2025) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2025.1560699 · Cited by 2
Seafood During Pregnancy and Lactation and Child Neurocognitive Development: A Systematic Review
O’Connor, L. E., Spill, M. K., Saha, S., Balalian, A. A., Davis, J. S., & MacFarlane, A. J. (2025) · Advances in Nutrition · doi:10.1016/j.advnut.2025.100414 · Cited by 3
Schepanski, S. N., Batta, F., Schröter, M., Seifert, G., & Koch, A. K. (2025) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2025.1603700 · Cited by 2
Risk of bias in effect estimates is real but still largely ignored in environmental research
Konno, K., Gibbons, J., Lewis, R. A., & Pullin, A. S. (2025) · Next research. · doi:10.1016/j.nexres.2025.100565 · Cited by 2
Treatment-related cognitive changes in multiple myeloma: A systematic review and meta-analysis
Patel, S., Parrish, C., Seymour, F., & Burke, M. R. (2025) · Journal of Geriatric Oncology · doi:10.1016/j.jgo.2025.102321 · Cited by 2
Development and application of meta-analysis in environmental health research
Li, Q., Liang, X., Yang, R., Yin, N., & Faiola, F. (2025) · Critical Reviews in Toxicology · doi:10.1080/10408444.2025.2530972 · Cited by 3
Suicidality among people with mild cognitive impairment: a systematic review
Sánchez-Gil, A., Pérez, J., Navarro‐López, V., & González, J. L. S. (2025) · BMC Psychiatry · doi:10.1186/s12888-025-07228-x · Cited by 2
Goetschi, A. N., Verloo, H., Schönenberger, N., Wernli, U., & Meyer‐Massetti, C. (2025) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.70253 · Cited by 2
da Silva, A. B. N., de Oliveira, G. M., Ruelas, M. G., Gadelha, M., Santos, A. C. F. D. F., & Zamora, F. V. (2025) · Biogerontology · doi:10.1007/s10522-025-10308-w · Cited by 3
Prevalence and Risk of Dementia Among Adults Who Have Experienced Homelessness, a Meta‐Analysis
Hoang, P., Huang, Y. Q., Jay, M., Berg, V., van Ballegooie, C., Reppas‐Rindlisbacher, C. E., Lane, N. E., Alston, J., & Rochon, P. A. (2025) · Journal of the American Geriatrics Society · doi:10.1111/jgs.70072 · Cited by 2
Talia, A. J., Busuttil, N. A., Hotchen, A., Kendal, A. R., & Brown, R. (2025) · Orthopaedic Journal of Sports Medicine · doi:10.1177/23259671251364261 · Cited by 2
Mu, L., Larvin, H., Nadarajah, R., Gale, C. P., & Wu, J. (2025) · Heart · doi:10.1136/heartjnl-2025-326376 · Cited by 2
Utility of the Social Vulnerability Index in Addressing Breast Cancer Disparities: A Meta‐Analysis
Nguyen, A. T., Li, R. A., Ontiveros, N. C., et al. (2025) · Journal of Surgical Oncology · doi:10.1002/jso.70080 · Cited by 2
Asiller, Ö. Ö., Fresilli, S., Damiani, D., et al. (2025) · Journal of Cardiothoracic and Vascular Anesthesia · doi:10.1053/j.jvca.2025.09.017 · Cited by 2
Artioli, T., Batista, L. B., Franchini, K., & de Alencar, J. N. (2025) · Arquivos Brasileiros de Cardiologia · doi:10.36660/abc.20250244i · Cited by 2
French, C. E., Troy, D. M., Dawson, S., Dalili, M. N., Hickman, M., & Thomas, K. H. (2024) · BJGP Open · doi:10.3399/bjgpo.2024.0122 · Cited by 2
Risk factors for invasive pneumococcal disease in adults: a systematic review and meta-analysisPreprint
Ditzel, K., Giardina, F., ten Oever, J., et al. (2025) · medRxiv · doi:10.1101/2025.03.13.25323815 · Cited by 3
Assessing the risk of bias in studies included in systematic reviews and meta-analyses
Iniesta‐Sepúlveda, M., & Ríos, A. (2024) · Cirugía Española (English Edition) · doi:10.1016/j.cireng.2024.04.016 · Cited by 1
Sudarman, J. P., Sayudo, I. F., Putri, M. M., et al. (2024) · Jurnal Plastik Rekonstruksi · doi:10.14228/jprjournal.v11i1.370 · Cited by 1
Evidence Communication Rules for Policy (ECR-P) critical appraisal tool
Danopoulos, E., Aston, J. A. D., Shah, A., & Schneider, C. R. (2025) · Systematic Reviews · doi:10.1186/s13643-025-02757-8 · Cited by 1
Hove, A. R., Makuzo, L. T., Shanu, R. N., Chaka, P. M., Munyoro, I., Sithole, P., Mugova, I. H., Murape, M., Muchinani, H., Doyce, S. R., et al. (2025) · BMJ Open · doi:10.1136/bmjopen-2024-091685 · Cited by 1
Jones, M. L., Leonard, A. F. C., Bethel, A. C., Lamb, E., Gaze, W. H., Taylor, T., Singer, A. C., Ukoumunne, O. C., & Garside, R. (2025) · Environment International · doi:10.1016/j.envint.2025.109371 · Cited by 3
Almeida, L. H. O., Banca, V. L., Matsumoto, O., de Sá, A. S., Vigano, A. V. P., Kalman, G., Gonzalez, F. F., Lima, G. H. V., Murachovsky, J., & Ikemoto, R. Y. (2025) · JSES International · doi:10.1016/j.jseint.2025.03.007 · Cited by 1
Mai, V., Girardi, L., de Wit, K., Castellucci, L. A., Aaron, S. D., Couturaud, F., Fergusson, D. A., & Gal, G. L. (2025) · ERJ Open Research · doi:10.1183/23120541.00202-2025 · Cited by 1
Nalwanga, D., Kinengyere, A. A., Kiggwe, A., Negash, A. A., Ocan, M., Nakalembe, L., Briend, A., Maitland, K., Musiime, V., & Karamagi, C. (2025) · BMJ Open · doi:10.1136/bmjopen-2024-091766 · Cited by 1
Elwenspoek, M. M. C., O’Donnell, R., Jackson, J., Dawson, S., Charlwood, K., Hay, A. D., Watson, J., & Whiting, P. F. (2025) · British Journal of General Practice · doi:10.3399/bjgp.2024.0744 · Cited by 1
Loh, C. T. I., Lee, Z., Yunos, N. M., Atan, R., Heyland, D. K., Stoppe, C., & Hasan, M. S. (2025) · Journal of Parenteral and Enteral Nutrition · doi:10.1002/jpen.2767 · Cited by 1
Siegel, N., Rotärmel, A., Polychronidis, G., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/72703 · Cited by 1
Hentzen, A. H. R., & Holm, D. E. (2025) · Animal Production Science · doi:10.1071/an24245 · Cited by 1
Mosavari, H., Heydary, F., Khalili, P., Ghasemi, S., Mehrnia, N., Salehimoghaddam, F., Saif, F., Jaliliyan, A., Haghighikian, M., Shalbafan, M., et al. (2025) · Obesity Surgery · doi:10.1007/s11695-025-08052-8 · Cited by 1
de Souza, D. S. M., Goulardins, J. B., Câmara, P. C. M. A., et al. (2025) · Ocular Immunology and Inflammation · doi:10.1080/09273948.2025.2532044 · Cited by 1
Umbrio, L. R. N., Pham, H. M., Barmak, A. B., Assery, N. M., & Kotsailidi, E. A. (2025) · Evidence-Based Dentistry · doi:10.1038/s41432-025-01179-8 · Cited by 1
Racial and ethnic disparities in postpartum pain: a systematic review
Craig, R., Nour, S. E., Blake, L., Carvalho, B., Kua, J., & O’Carroll, J. (2025) · International Journal of Obstetric Anesthesia · doi:10.1016/j.ijoa.2025.104736 · Cited by 2
Estes-Schmalzl, K. J., & Lefebvre, K. M. (2025) · Topics in Geriatric Rehabilitation · doi:10.1097/tgr.0000000000000477 · Cited by 1
Rose, D. U. D., Liotto, N., Maggiora, E., et al. (2025) · Nutrition Reviews · doi:10.1093/nutrit/nuaf110 · Cited by 1
Peripheral biomarkers in compulsive sexual behavior disorder: a systematic review
Ferreira, B. D. O., Teixeira, B. J., Talib, L. L., & Scanavino, M. D. T. (2025) · Sexual Medicine Reviews · doi:10.1093/sxmrev/qeaf046 · Cited by 1
Lima, N. L., Mezzari, M. H. D. S., Júnior, L. C. I. C., Grüdtner, D. O. M., Maggi, B. G., de Azevedo, D. K., de Souza, J. C. S., Júnior, W. J. M. B., Rodrigues, Y. G., & Isolan, G. R. (2025) · Neuroradiology · doi:10.1007/s00234-025-03722-x · Cited by 2
Proficiency in Esophageal Peroral Endoscopic Myotomy
Wu, Y., Grieme, A., Garza, M., et al. (2025) · Journal of Clinical Gastroenterology · doi:10.1097/mcg.0000000000002240 · Cited by 1
How to become a data synthesis expert in IBD: a clinician’s guide to cutting through the noise
Gordon, M. A. (2025) · Frontline Gastroenterology · doi:10.1136/flgastro-2024-103031 · Cited by 1
Abubacker, R., Yuvaraj, P., Rajasekaran, A. K., Ramalingaiah, A. H., Bhaskarapillai, B., Sukumar, S., Prakash, H., & Kesavan, M. (2025) · Asian Journal of Psychiatry · doi:10.1016/j.ajp.2025.104679 · Cited by 1
Nguyen, B. A. H., & Quon, S. (2025) · Digestive Diseases and Sciences · doi:10.1007/s10620-025-09392-2 · Cited by 1
Lawler, K., Behie, A. M., & Richardson, A. (2025) · International Journal of Hygiene and Environmental Health · doi:10.1016/j.ijheh.2025.114670 · Cited by 4
Reischl‐Hajiabadi, A. T., Garbade, S. F., Gleich, F., Schnabel‐Besson, E., Posset, R., Zielonka, M., Hoffmann, G. F., Kölker, S., & Mütze, U. (2025) · Journal of Inherited Metabolic Disease · doi:10.1002/jimd.70090 · Cited by 2
Labenbacher, S., Hammer, S., Moser, A., Schreiber, N., & Bornemann‐Cimenti, H. (2025) · Journal of Advanced Nursing · doi:10.1111/jan.70264 · Cited by 1
The impact of obesity on influenza Vaccine immunogenicity: A systematic review
Clarke, M. F., McMillan, M., Giles, L. C., Riley, K., Andraweera, P. H., Richmond, P., Mathew, S. M., & Marshall, H. S. (2025) · Vaccine · doi:10.1016/j.vaccine.2025.127794 · Cited by 1
Geraci, G., Calabrese, V., Ferrara, P., Rosa, G. R. M. L., Cuttone, G., Via, L. L., Sinatra, N., Sorce, A., Carollo, C., Mulè, G., et al. (2025) · JMIR Research Protocols · doi:10.2196/79071 · Cited by 1
Causes and Treatments of Recurrent Implantation Failure: A Systematic Review and Meta-Analysis
Bulletti, C., Pinborg, A. B., Franasiak, J. M., Aghajanova, L., Løssl, K., Altmäe, S., Busnelli, A., Berrettini, M., Bulletti, F. M., & Ata, B. (2025) · F1000Research · doi:10.12688/f1000research.170152.1 · Cited by 2
Jreij, G., Crone, C., Wiley, A. P., Kankaria, A. A., Lin, M. S., Mayorga‐Carlin, M., Sahoo, S., Sorkin, J. D., & Lal, B. K. (2025) · Journal of Vascular Surgery Venous and Lymphatic Disorders · doi:10.1016/j.jvsv.2025.102343 · Cited by 3
Burke, C. A., Fillipo, R., Epplein, M., et al. (2025) · Arthritis Care & Research · doi:10.1002/acr.25665 · Cited by 1
Tan, G., Chen, H., & Leuthardt, E. C. (2025) · Neuromodulation Technology at the Neural Interface · doi:10.1016/j.neurom.2025.08.001 · Cited by 6
Daga-Guijarro, N., Diciolla, N. S., & Yuste-Sánchez, M. J. (2025) · Cancer Treatment Reviews · doi:10.1016/j.ctrv.2025.103027 · Cited by 1
Changes in child maltreatment occurrence during the COVID-19 pandemic: A systematic review
Rubeis, V. D., Tonmyr, L., Rahman, S., Pagaduan, J. E., Drysdale, M., Morissette, K., MacMillan, H. L., Aylward, E., Nanziba, F., Powell, S., et al. (2025) · Child Abuse & Neglect · doi:10.1016/j.chiabu.2025.107744 · Cited by 1
Cardiovascular Health of Single Mothers: A Systematic Review
Borrowman, J. D., Pageau, L. M., Cameron, N. A., Carnethon, M. R., & Fields, N. D. (2025) · Journal of the American Heart Association · doi:10.1161/jaha.125.043296 · Cited by 1
Sexual function in axial spondyloarthritis: a systematic review and meta-analysis
Somer, M., Battista, S., Grange, J.-F., Prior, Y., Parsons, H. M., Packham, J., Haywood, K. L., Giardulli, B., & Prior, J. A. (2025) · Lara D. Veeken · doi:10.1093/rheumatology/keaf578 · Cited by 2
Microbiology of Pressure Ulcers: A Systematic Review
Shelepenko, D., Jardim, B., Almeida, L. M., Azevedo, J. M., Catalão, I., Sítima, M., & Tomé, G. (2025) · Cureus · doi:10.7759/cureus.95912 · Cited by 1
Maatouk, M., Khalifa, M. B., Essid, N., et al. (2025) · Surgery Open Science · doi:10.1016/j.sopen.2025.10.010 · Cited by 1
Danso‐Appiah, A., Kunfah, S. M. P., Nartey, K., & Asiamah, M. (2025) · Systematic Reviews · doi:10.1186/s13643-025-02830-2 · Cited by 3
Effect of GLP ‐1 receptor agonists on idiopathic intracranial hypertension: A systematic review
de Oliveira, H. M., Ruelas, M. G., Fonseca, P. E. O., Diaz, C. A. V., de Paula, G. O., da Costa, P. R. F., Parker, T., & Kahle, K. T. (2025) · Headache The Journal of Head and Face Pain · doi:10.1111/head.70005 · Cited by 2
Tuono, R. D. M., Louokdom, J. S., Njopwouo, M. S., & Tagny, C. T. (2025) · Thrombosis Journal · doi:10.1186/s12959-025-00804-x · Cited by 1
Kundu, A., Feore, A., Abu-Zarour, N., Sanchez, S., Sutton, M., Sachdeva, K., Seth, S., Schwartz, R. M., & Chaiton, M. (2025) · Tobacco Induced Diseases · doi:10.18332/tid/209954 · Cited by 2
Kaddoura, R., Al-Tamimi, H., & Pieles, G. E. (2025) · World Journal of Cardiology · doi:10.4330/wjc.v17.i11.107835 · Cited by 1
Liao, Z., Zhao, S., Fang, S., Ren, J., Wang, S., Kong, L., & Fang, M. (2025) · iScience · doi:10.1016/j.isci.2025.114178 · Cited by 1
Neonatal phototherapy and cancer risk: a systematic review and meta-analysis
Freeman, S. J., Keown‐Stoneman, C. D. G., Ghobrial, M., Jegathesan, T., & Sgro, M. D. (2025) · Frontiers in Pediatrics · doi:10.3389/fped.2025.1667636 · Cited by 2
The Relationship Between Nasal Septal Deviation and Rhinosinusitis: A Systematic Review
Shulkin, A., Katz, A., & Tewfik, M. A. (2025) · Journal of Otolaryngology - Head and Neck Surgery · doi:10.1177/19160216251390316 · Cited by 1
Chen, Q. J., Mozaffari, H., Ananthapavan, J., Smith, B. T., Wong, G. W. K., & Jessri, M. (2025) · Systematic Reviews · doi:10.1186/s13643-025-02978-x · Cited by 2
Wiradinata, W., Aditya, M. R., Subali, A. D., et al. (2025) · European Journal of Pediatrics · doi:10.1007/s00431-025-06657-w · Cited by 1
Sheervalilou, M., Ghanei, M., & Arabfard, M. (2025) · BMC Infectious Diseases · doi:10.1186/s12879-025-12272-9 · Cited by 2
Mweetwa, M. N., Kelly, P. M., & Posma, J. M. (2026) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2025.0038 · Cited by 2
Gebretsadik, G. G., Yang, B., Glenn, A. J., et al. (2025) · The journal of nutrition health & aging · doi:10.1016/j.jnha.2025.100754 · Cited by 1
Place, A., Rodrigues, T. S., Naimo, P. S., Lee, M. G. Y., Batchelor, R., Weinberg, L., Miles, L. F., Lefkovits, J., & Koshy, A. N. (2025) · Anaesthesia · doi:10.1111/anae.70084 · Cited by 2
Song, J., O’Brien, F., Eustace‐Cook, J., & O’Donnell, S. (2025) · PLoS ONE · doi:10.1371/journal.pone.0338896 · Cited by 1
Somma, D., Finocchi, A., Campagnini, S., Marra, E., Pedrini, C., Pinna, S., Szczepanska, M. A., Fasano, A., Cecchi, F., Falotico, E., et al. (2025) · Journal of NeuroEngineering and Rehabilitation · doi:10.1186/s12984-025-01839-x · Cited by 2
Chronotype Profile in Children: A Systematic Review
Silva, R. B. D. M., Schmidt, H., dos Santos, G. D., Leocadio-Miguel, M. A., & Louzada, F. M. (2025) · Sleep and Vigilance · doi:10.1007/s41782-025-00324-8 · Cited by 2
Self-citationImpact of Clear Aligners on the Temporomandibular Joint: A Systematic Review
Sicilia, L., Miranda, G. H. N., Fattori, L., et al. (2025) · Orthodontics and Craniofacial Research · doi:10.1111/ocr.70094 · Cited by 1
Invasive Neuromodulation in Lennox-Gastaut Syndrome: A Systematic Review and Meta-Analysis
de Oliveira, H. M., Fonseca, P. E. O., Ruelas, M. G., de Paula, G. O., Diaz, C. A. V., da Costa, P. R. F., de Oliveira, R., & Pilitsis, J. G. (2026) · Neuromodulation Technology at the Neural Interface · doi:10.1016/j.neurom.2025.11.014 · Cited by 2
Moghadam, A. J., Sadr, H., Asadi, A., et al. (2026) · Neurological Sciences · doi:10.1007/s10072-025-08666-x · Cited by 1
Alterations in intramuscular connective tissue in hypertonic muscle: a scoping review
Zhao, X., Sun, Y., Fede, C., et al. (2026) · Frontiers in Physiology · doi:10.3389/fphys.2025.1720927 · Cited by 1
Naveda, A. M. P., Oliva, H. N. P., Ghadimi, D. J., et al. (2026) · PLoS ONE · doi:10.1371/journal.pone.0340595 · Cited by 1
Pamporis, K., Tsiachris, D., Jaïs, P., et al. (2026) · Heart Rhythm · doi:10.1016/j.hrthm.2026.01.034 · Cited by 1
Ramji, S., Daniel, S., Ansari, A. S., et al. (2026) · International Ophthalmology · doi:10.1007/s10792-026-03944-9 · Cited by 2
Oskouei, B. S., Rahimi, P., Azimzadeh, N., et al. (2026) · Clinical Nutrition Open Science · doi:10.1016/j.nutos.2026.100632 · Cited by 1
Prevalence of limited health literacy in blue-collar workers: a systematic review protocol
Rodrigues, G. A. P., Duong, P., Lüthi, F., et al. (2026) · BMJ Open · doi:10.1136/bmjopen-2025-111163 · Cited by 1
Liu, T., Zhang, Y. L., & Chao, M. (2026) · Mass Communication & Society · doi:10.1080/15205436.2026.2620702 · Cited by 1
Metadata
- Authors
- Higgins, J. P. T., Morgan, R. L., Rooney, A. A., Taylor, K. W., Thayer, K. A., Silva, R. A., Lemeris, C., Akl, E. A., Bateson, T. F., Berkman, N. D., Glenn, B. S., Hróbjartsson, A., LaKind, J. S., McAleenan, A., Meerpohl, J. J., Nachman, R. M., Obbagy, J. E., O'Connor, A., Radke, E. G., Savović, J., Schünemann, H. J., Shea, B., Tilling, K., Verbeek, J., Viswanathan, M., & Sterne, J. A. C.
- Affiliations
- University of Bristol (GB)
- McMaster University (CA)
- National Institute of Environmental Health Sciences (US)
- Environmental Protection Agency (US)
- ICF International (United States) (US)
- American University of Beirut (LB)
- RTI International (US)
- University of Southern Denmark (DK)
- University of Freiburg (DE)
- United States Department of Agriculture (US)
- Michigan State University (US)
- Cochrane Canada (CA)
- Ottawa Hospital Research Institute (CA)
- Amsterdam UMC Location University of Amsterdam (NL)
- Corresponding author
- Higgins, Julian P.T.
- ORCID
- Higgins, Julian P.T.
- Morgan, Rebecca L.
- Rooney, Andrew A.
- Taylor, Kyla W.
- Thayer, Kristina A.
- Silva, Raquel A.
- Lemeris, Courtney
- Akl, Elie A.
- Bateson, Thomas F.
- Berkman, Nancy D.
- Hróbjartsson, Asbjørn
- LaKind, Judy S.
- McAleenan, Alexandra
- Meerpohl, Joerg J.
- Nachman, Rebecca M.
- Obbagy, Julie E.
- O'Connor, Annette
- Radke, Elizabeth G.
- Savović, Jelena
- Schünemann, Holger J.
- Shea, Beverley
- Tilling, Kate
- Verbeek, Jos
- Viswanathan, Meera
- Sterne, Jonathan A.C.
- Venue
- Environment International
- Publisher
- Elsevier BV
- Published
- 2024-04
- Volume
- 186
- Pages
- 108602
- ISSN
- 0160-4120 (ISSN-L)
- PMC
- PMC11098530
- OpenAlex
- W4393132304
- Semantic Scholar
- b822b2b6998a5003935cf5815b5370685f4e5231
- Cited by
- 1,022
- References
- 30
- Citations by year
- 2026: 470 · 2025: 440 · 2024: 112
- FWCI
- 284.27
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Environment International (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Pure Amsterdam UMC (publishedVersion, cc-by)
- University of Southern Denmark Research Portal (University of Southern Denmark) (publishedVersion, cc-by-nc)
- PubMed Central (acceptedVersion, cc-by-nc)
- Europe PMC (PubMed Central) (acceptedVersion, cc-by-nc)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Bristol Research (University of Bristol) (submittedVersion, cc-by-nc-nd)
- Explore Bristol Research (submittedVersion, cc-by-nc-nd)
- Funders
- National Institute for Health and Care Research
- Cancer Research UK
- National Institute of Environmental Health Sciences
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- risk of bias assessment, non-randomized studies, cohort studies, causal inference, systematic review methodology, observational epidemiology
- MeSH terms
- Environmental Exposure, Follow-Up Studies, Humans, Cohort Studies, Bias, Epidemiologic Studies, Risk Assessment, Observational Studies as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
