The DOI matches its Crossref record; 22 of the 30 references it cites were found in a registry or an index; none of the 27 sources its claims cite is checked for support yet.
A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E)
Open accessAbstract
BACKGROUND: Observational epidemiologic studies provide critical data for the evaluation of the potential effects of environmental, occupational and behavioural exposures on human health. Systematic reviews of these studies play a key role in informing policy and practice. Systematic reviews should incorporate assessments of the risk of bias in results of the included studies. OBJECTIVE: To develop a new tool, Risk Of Bias In Non-randomized Studies - of Exposures (ROBINS-E) to assess risk of bias in estimates from cohort studies of the causal effect of an exposure on an outcome. METHODS AND RESULTS: ROBINS-E was developed by a large group of researchers from diverse research and public health disciplines through a series of working groups, in-person meetings and pilot testing phases. The tool aims to assess the risk of bias in a specific result (exposure effect estimate) from an individual observational study that examines the effect of an exposure on an outcome. A series of preliminary considerations informs the core ROBINS-E assessment, including details of the result being assessed and the causal effect being estimated. The assessment addresses bias within seven domains, through a series of 'signalling questions'. Domain-level judgements about risk of bias are derived from the answers to these questions, then combined to produce an overall risk of bias judgement for the result, together with judgements about the direction of bias. CONCLUSION: ROBINS-E provides a standardized framework for examining potential biases in results from cohort studies. Future work will produce variants of the tool for other epidemiologic study designs (e.g. case-control studies). We believe that ROBINS-E represents an important development in the integration of exposure assessment, evidence synthesis and causal inference.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
ROBINS-E is a new tool for assessing risk of bias in a specific exposure effect estimate from non-randomized cohort studies, developed by adapting ROBINS-I and RoB 2. It uses preliminary considerations, seven bias domains, signalling questions, and domain-level judgements to produce overall risk-of-bias, direction-of-bias, and threat-to-conclusions judgements.
Key findings
- ROBINS-E assesses risk of bias in a specific result from an observational cohort study of the causal effect of an exposure on an outcome, addressing seven domains: confounding, exposure measurement, selection of participants, post-exposure interventions, missing data, outcome measurement, and selection of the reported result.
- Development involved 43 researchers; proposals were discussed at a two-day meeting of 31 participants in Bristol in January 2017, and a draft was piloted in October 2019 by 28 in-person plus 3 remote participants; a revised draft was piloted in June–July 2021, and the launch version was posted in June 2022.
- Domain-level judgements include risk of bias (“Low risk of bias”, “Some concerns”, “High risk of bias”, “Very high risk of bias”), predicted direction of bias, and whether the bias threatens conclusions; the default overall risk-of-bias judgement is that of the domain with the greatest risk of bias.
- ROBINS-E does not directly assess financial conflicts of interest, and the present version addresses only follow-up (cohort) studies, with variants for other designs such as case-control studies planned.
Methods
- Design: iterative tool development through working groups, in-person meetings, webinars, and pilot testing, following principles used for RoB 2 and ROBINS-I.
- Data or sample: no empirical dataset was used; development involved 43 identified researchers, a 31-participant meeting in January 2017, a 2019 pilot with 28 in-person plus 3 remote participants, and a 2021 revised-draft pilot.
- Main analysis: not stated in the paper; the article describes the tool’s structure, domains, signalling questions, and judgement algorithms rather than conducting a statistical analysis.
Limitations
- Stated: the present version addresses only follow-up (cohort) studies, and work is ongoing to develop variants for other study designs including case-control studies.
- Stated: assessments require judgement, so subjectivity and between-assessor variability are impossible to avoid; accuracy depends on the quality of available information, and successful use requires both content and methodological expertise.
- Stated/evident: the tool has not yet had formal evaluation of its reliability and validity; the paper reports no empirical reliability or validity data, and the data availability statement says no data were used for the research described.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 of 27 cited sources checked · 0 supported · 0 flagged
In the paper
“although all such studies identified related to randomized trials”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Development of ROBINS-E
What the sources say
- Not checked yetcites Sterne et al., 2002 source
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIn the paper
“ROBINS-E does not include direct assessment of financial conflicts of interest, which may be important in interpreting the findings and credibility of studies of exposures and have been recommended for inclusion in evaluation of environmental studies (National Academies of Sciences Engineering and Medicine, 2022).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
What the sources say
- Not checked yetcites National Academies of Sciences Engineering and Medicine, 2022
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesNone of the sources cited for this claim have been checked yet.
In the paper
“different tools, and different ways of addressing risk-of-bias assessments, can lead to differences in conclusions of systematic reviews”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Introduction
What the sources say
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesNone of the sources cited for this claim have been checked yet.
In the paper
“Recently-developed tools for this purpose have moved away from numeric scores based on checklists (which often combined elements of risk of bias with other considerations such as of applicability) to tools based on assessments within specific domains of bias.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Introduction
What the sources say
- Not checked yetcites National Toxicology Program, 2015
- Not checked yetcites Woodruff and Sutton, 2014 source
- Not checked yetcites U.S. Environmental Protection Agency, 2022
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesNone of the sources cited for this claim have been checked yet.
In the paper
“The second variant additionally addresses time-varying confounding, which can occur when the exposure changes over time and follow-up time is split according to exposure level (Hernán and Robins, 2020; Mansournia et al., 2017).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
What the sources say
- Not checked yetcites Hernán and Robins, 2020
- Not checked yetcites Mansournia et al., 2017
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For this reason, ROBINS-E assessments may support causal inferences, if sufficient evidence at ‘low’ risk of bias or with ‘some concerns’ about risk of bias can be located.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIn the paper
“ROBINS-E can be used as part of a framework to rate the quality of the evidence for a group of studies. An example of such a framework is the GRADE approach (Guyatt et al., 2011)”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
What the sources say
- Not checked yetcites Guyatt et al., 2011 source
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It is impossible to exclude the possibility of uncontrolled confounding in a non-randomized study”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes? Based: unclearA conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Furthermore, there are impacts of such conflicts that are not problems of internal validity (targeted by ROBINS-E) but instead relate to what question is being asked in the study or analysis.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIn the paper
“The rationale, previously discussed in the context of randomized trials (Sterne, 2013), is that if financial conflicts of interest lead to bias then this will operate through one of the domains already in the tool.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
What the sources say
- Not checked yetcites Sterne, 2013 source
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- It may state two separate claims as one. 9
- Probably not a substantial claim. 37
None of the sources cited for this claim have been checked yet.
In the paper
“The instrument was based on the ROBINS-I tool for non-randomized studies of interventions (Sterne et al., 2016) and RoB 2 tool for randomized trials (Sterne et al., 2019), and was informed by previous instruments for assessing studies of exposure.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
What the sources say
- Not checked yetcites Sterne et al., 2016
- Not checked yetcites Sterne et al., 2019
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“ROBINS-E is a new general-purpose tool for assessing risk of bias in results of non-randomized studies of exposure.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The present version addresses only follow-up (cohort) studies”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Important elements in ROBINS-E include a definition of the causal effect of interest, signalling questions to guide domain-level risk-of-bias judgements across seven domains, algorithms to derive suggested risk of bias judgements from answers to the signalling questions, and the separation of risk of bias from other important considerations in the evaluation of study sensitivity.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“These additional considerations, which fall outside the scope of the main ROBINS-E assessment, are addressed in an optional section of ROBINS-E.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Study sensitivity and appropriateness of its design
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“However, the user may override this, for example if they judge that so many domains were rated as at ‘High’ risk of bias that the overall judgement should be ‘Very high’ risk of bias.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Overall risk of bias
Strength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The default ROBINS-E overall risk-of-bias judgement is that for the domain with the greatest risk of bias.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Overall risk of bias
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The default judgement for the threat to conclusions is derived in a similar way to the overall risk-of-bias judgement; for example, the overall judgement is ‘Yes’ if there is considered to be a threat to conclusions in any of the seven bias domains.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Overall risk of bias
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“ROBINS-E provides a standardized framework for examining the range of biases common to observational studies of exposures.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“ROBINS-E is designed primarily for use in the context of a systematic review.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
In the paper
“The collaborative development of the tool, between methodologists and stakeholders from fields such as environmental, occupational, chemical, clinical and public health, has highlighted the benefits of sharing ideas and research methods across communities (Hoffmann et al., 2022; Hoffmann et al., 2022).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
What the sources say
- Not checked yetcites Hoffmann et al., 2022
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“An optional section of ROBINS-E allows users to record observations about study sensitivity, aspects of which may additionally feed into the “indirectness” domain within GRADE, as it relates to generalizability of the result of the study to the population, exposure and outcome of interest.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Several such tools for observational epidemiologic studies provide significant methodological advances over previously used tools (National Toxicology Program, 2015, 2015; Woodruff and Sutton, 2014; U.S. Environmental Protection Agency, 2022).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Introduction
What the sources say
- Not checked yetcites National Toxicology Program, 2015
- Not checked yetcites Woodruff and Sutton, 2014 source
- Not checked yetcites U.S. Environmental Protection Agency, 2022
Strength:boosted· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Early work by a subset of the authors considered the extent to which the ROBINS-I tool was suitable for observational studies of exposures and suggested modifications to the tool (Morgan et al., 2018; Morgan et al., 2019).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Development of ROBINS-E
What the sources say
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“The Risk Of Bias In Non-randomized Studies - of Interventions (ROBINS-I) tool was developed to evaluate estimates of the effects of deliberate interventions aimed to alter health outcomes (Sterne et al., 2016).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Introduction
What the sources say
- Not checked yetcites Sterne et al., 2016
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Domain-level judgements about risk of bias are derived from the answers to these questions, then combined to produce an overall risk of bias judgement for the result, together with judgements about the direction of bias.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Abstract > Methods and results:
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The extent of the potential for bias across a body of evidence for an exposure-outcome effect informs assessment of the certainty of evidence for that effect.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A screening section then facilitates identification of results that are at very high risk of bias, allowing the user to avoid a detailed risk of bias assessment (Fig. 1, part B).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Preliminary considerations
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“There is a risk of bias when such selection is based on the magnitude, direction or P value of the result.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“First, evaluation of risk of bias requires that the review team has expertise in both content (detailed subject-matter knowledge) and methodology (understanding of the different types of bias that may distort the results of observational studies).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Bias due to post-exposure interventions may arise if the presence or level of exposure leads to administration of interventions that change the subsequent risk of the outcome.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Bias in selection of the reported result arises when study authors select results from a multiplicity of analyses, for example from different ways of measuring the exposure, different ways of measuring the outcome, different subsets of the full study sample or different analyses.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The ROBINS-E risk-of-bias assessment addresses seven domains of bias (Fig. 1, Section 5).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Overview of ROBINS-E
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
- The authors state this as an assumption: Issues common to both randomized trials and observational studies (missing data, outcome assessment, selection of the reported result) are likely to be similar.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“By contrast, it is possible to envisage situations in which risk of bias for other domains is low: for example, if exposure is measured without error, there are no missing data, or results are reported in accordance with a pre-specified analysis plan.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
Strength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A series of preliminary considerations informs the core ROBINS-E assessment, including details of the result being assessed and the causal effect being estimated.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Abstract > Methods and results:
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, such analyses require that the factors leading to the post-exposure intervention can be modelled.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Although ROBINS-E was iteratively developed based on the results of pilot testing with a diverse group”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Bias arising from measurement of the outcome is a problem when error in measurement of the outcome is related to the presence or level of exposure, in which case the measurement error is differential rather than non-differential.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Third, ROBINS-E assessments require judgements based on the available information. It is therefore impossible to avoid subjectivity and between-assessor variability.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ROBINS-E provides a standardized framework for examining potential biases in results from cohort studies.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Abstract > Conclusion:
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
- The authors state this as an assumption: Issues common to both randomized trials and observational studies (missing data, outcome assessment, selection of the reported result) are likely to be similar.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This question can be manipulated to give an illusion of presence or lack or effect (for example by studying trivial levels of exposure or irrelevant outcomes), but this need not have implications for internal validity.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Practical considerations can impact on the successful implementation of ROBINS-E.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
Strength:plain· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Each domain is addressed using a series of signalling questions that aim to gather relevant information about the study and analysis being assessed.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Overview of ROBINS-E
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The first version of ROBINS-E, described here, targets cohort (follow-up) studies.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Introduction
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“ROBINS-E was developed by a large group of researchers from diverse research and public health disciplines through a series of working groups, in-person meetings and pilot testing phases.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Abstract > Methods and results:
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Because risk of bias is assessed in relation to a defined causal effect of an exposure on an outcome, a low risk of bias corresponds to an absence of bias in estimating that causal effect.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, publication of answers to signalling questions and justification for these should make risk of bias judgements transparent.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
Strength:plain· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Bias in selection of participants into the study (or into the analysis) is the first of two domains that address selection bias, which arises when we condition, in the study design or in the analysis, on common effects of exposure and outcome (Hernán and Robins, 2020; Hernán et al., 2004).”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Bias domains
What the sources say
- Not checked yetcites Hernán and Robins, 2020
- Not checked yetcites Hernán et al., 2004 source
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Second, the accuracy of a risk-of-bias assessment will depend on the quality of information available about the study being assessed.”
§ A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E) > Discussion
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper relies on this without support.
Randomized controlled trials are often not feasible, ethical, large enough or of sufficient duration to evaluate chronic health effects.
“Randomized controlled trials (RCTs) are often not feasible, ethical, large enough or of sufficient duration to evaluate chronic health effects.”
The authors state this as an assumption.
Issues common to both randomized trials and observational studies (missing data, outcome assessment, selection of the reported result) are likely to be similar.
“with the additional assumption that issues common to both study designs (missing data, outcome assessment, selection of the reported result) are likely to be similar”
The paper relies on this without support.
Pre-specified plans for analyses are uncommon for observational studies.
“However, such pre-specified plans are uncommon for observational studies and often the nature of the data need to be taken into account when selecting a suitable statistical model and method.”
The paper relies on this without support.
The nature of the data often needs to be taken into account when selecting a suitable statistical model and method.
“However, such pre-specified plans are uncommon for observational studies and often the nature of the data need to be taken into account when selecting a suitable statistical model and method.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“For cohort studies, the Risk of Bias in Non‐randomized Studies of Exposures (ROBINS‐E) tool was used.”
From Risk of cardiac, neuropsychiatric and musculoskeletal adverse events with levothyroxine: Systematic review (2026) · cites this paper as [12]
Citations
Cites
RoB 2: a revised tool for assessing risk of bias in randomised trials
Sterne, J. A. C., Savović, J., Page, M. J., Elbers, R. G., Blencowe, N. S., Boutron, I., Cates, C. J., Cheng, H.-Y., Corbett, M. S., Eldridge, S. M., et al. (2019) · BMJ · doi:10.1136/bmj.l4898 · Cited by 34,787
Self-citationDermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,586
GRADE: an emerging consensus on rating quality of evidence and strength of recommendations
Guyatt, G. H., Oxman, A. D., Vist, G. E., Kunz, R., Falck-Ytter, Y., Alonso-Coello, P., & Schünemann, H. J. (2008) · BMJ · doi:10.1136/bmj.39489.470347.ad · Cited by 23,305 · cited 1×
“…of interest; measurement of exposure; measurement of outcomes; attrition and selective reporting; integration of ROBINS-E with the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) framework (Guyatt et al., 2008); and issues in statistical analysis (see Online Supplement).”
Self-citationROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329
Self-citationCochrane Handbook for Systematic Reviews of Interventions
(2019) · doi:10.1002/9781119536604 · Cited by 14,828
GRADE guidelines: 1. Introduction—GRADE evidence profiles and summary of findings tables
Guyatt, G., Oxman, A. D., Akl, E. A., Kunz, R., Vist, G., Brozek, J., Norris, S., Falck-Ytter, Y., Glasziou, P., & deBeer, H. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.04.026 · Cited by 10,681 · cited 1×
“An example of such a framework is the GRADE approach (Guyatt et al., 2011) where users may start at high certainty in the “study limitations” (risk of bias) domain of GRADE and use their ROBINS-E evaluation to determine how many levels to rate down this certainty (Schunemann et al., 2018).”
Self-citationA structural approach to selection bias
Hernán, M. A., Hernández-Díaz, S., & Robins, J. M. (2004) · Epidemiology · doi:10.1097/01.ede.0000135174.63482.43 · Cited by 2,844 · cited 1×
“Bias in selection of participants into the study (or into the analysis) is the first of two domains that address selection bias, which arises when we condition, in the study design or in the analysis, on common effects of exposure and outcome (Hernán and Robins, 2020; Hernán et al., 2004).”
Triangulation in aetiological epidemiology
Lawlor, D. A., Tilling, K., & Davey Smith, G. (2017) · International Journal of Epidemiology · doi:10.1093/ije/dyw314 · Cited by 1,129 · cited 1×
“…together in a triangulation exercise, in which both the risks EPA A u t ho r M anu sc r i p t EPA A u t ho r M anu sc r i p t EPA A u t ho r M anu sc r i p t of bias (assessed by ROBINS-E) and differences between causal questions being addressed are considered jointly (Lawlor et al., 2016).”
Self-citationSchünemann, H. J., Cuello, C., Akl, E. A., Mustafa, R. A., Meerpohl, J. J., Thayer, K., Morgan, R. L., Gartlehner, G., Kunz, R., Katikireddi, S. V., et al. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.01.012 · Cited by 911 · cited 1×
“An example of such a framework is the GRADE approach (Guyatt et al., 2011) where users may start at high certainty in the “study limitations” (risk of bias) domain of GRADE and use their ROBINS-E evaluation to determine how many levels to rate down this certainty (Schunemann et al., 2018).”
Self-citationWoodruff, T. J., & Sutton, P. (2014) · Environmental Health Perspectives · doi:10.1289/ehp.1307175 · Cited by 427 · cited 1×
“Several such tools for observational epidemiologic studies provide significant methodological advances over previously used tools (National Toxicology Program, 2015, 2015; Woodruff and Sutton, 2014; U.S. Environmental Protection Agency, 2022).”
Flagged first, then by citations
8 of the 30 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- Boutron, 2019, Chapter 7: considering bias and conflicts of interest among the included studies, 177
- National Academies of Sciences Engineering and Medicine, 2022. A Review of U.S. EPA’s ORD Staff Handbook for Developing IRIS Assessments: 2020 Version. Washington DC: The National Academies Press.
Cited by
Sorted by citations
Items 101–200 of 966
Azher, S., McGrath, R., Kamalabadi, Y. M., Tsakos, G., Sim, F., & Singh, A. (2024) · Quality of Life Research · doi:10.1007/s11136-024-03795-w · Cited by 9
Melatonin secretion across puberty: A systematic review and meta-analysis
Andersen, C. C., Kjær, E. K. R., Vase, C. B., Mathiasen, R., Debes, N. M., Jørgensen, N. R., & Jennum, P. (2025) · Psychoneuroendocrinology · doi:10.1016/j.psyneuen.2025.107281 · Cited by 10
Fostitsch, A. J., Schwarzer, G., Buchgeister, M., Surbeck, W., Lahmann, C., Spiegelhalder, K., Frase, L., & Spieler, D. (2025) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2025.102073 · Cited by 12
Racial and ethnic disparity in obstetric anaesthesia: a systematic review and meta‐analysis
Craig, R., Nour, S. E., Xuereb, L. B., Gibbs, A., Suriyakumar, S., Blake, L., David, A. L., Lee, A. J., Nzelu, D., Moonesinghe, R., et al. (2025) · Anaesthesia · doi:10.1111/anae.16581 · Cited by 12
Risk Factors for Adverse Pregnancy Outcomes in Reduced Fetal Movement: An IPD Meta‐Analysis
Lu, Y., Palin, V., & Heazell, A. E. P. (2025) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.18132 · Cited by 11
Chiao-Ming, C., Lin, C., & Hsieh, Y. (2025) · Pain Physician · doi:10.36076/ppj.2025.28.83 · Cited by 10
Harris, M., Harvie, M., & Renehan, A. G. (2025) · Cancers · doi:10.3390/cancers17091451 · Cited by 10
Sustainable diets and cancer: a systematic review and meta-analysis
Kasper, M., al Masri, M., Kuhn, T. E., Rohrmann, S., Wirnitzer, K., Leitzmann, M. F., & Jochem, C. (2025) · EClinicalMedicine · doi:10.1016/j.eclinm.2025.103215 · Cited by 10
Triolo, G., Ivaldi, D., Lombardo, R., Quartarone, A., & Buono, V. L. (2025) · Sensors · doi:10.3390/s25123732 · Cited by 10
Liu, J., Lin, S., Zhao, Y.-S., et al. (2025) · Annals of Medicine · doi:10.1080/07853890.2025.2547324 · Cited by 10
First trimester maternal infections and offspring congenital heart defects: a meta-analysis
Su, H., Guo, E., Woodward, M., He, J., Waterboer, T., Schuermans, A., Bruaene, A. V. D., Troost, E., Meester, P. D., Morten, K., et al. (2025) · European Heart Journal · doi:10.1093/eurheartj/ehaf564 · Cited by 11
Fonda, F., Orso, D., Comisso, I., Crignis, E. D., Meroi, F., Galioto, B., Bove, T., & Rocca, G. D. (2025) · BMC Anesthesiology · doi:10.1186/s12871-025-03518-9 · Cited by 11
An umbrella review of meta-analyses on the low-FODMAP diet in IBS
Bogdanowska-Charkiewicz, D., Malinowska, U., & Daniluk, J. (2026) · Frontiers in Nutrition · doi:10.3389/fnut.2025.1714281 · Cited by 9
Rebamipide as an Adjunctive Therapy for Gastrointestinal Diseases: An Umbrella Review
Maev, I. V., Расимовна, Х. А., Andreev, D. N., Zaborovsky, A. V., Kucheryavyy, Y. A., Sokolov, F. S., & Beliy, P. A. (2026) · Pharmaceuticals · doi:10.3390/ph19010144 · Cited by 11
Parizad, R., Hatwal, J., Sadeghi, M. T., Brar, A. S., Desai, R., Batta, A., & Mohan, B. (2026) · Vascular Health and Risk Management · doi:10.2147/vhrm.s578698 · Cited by 13
Capozzi, V. A., Incognito, G. G., Scarpelli, E., Palumbo, M. A., Randazzo, C. L., Pino, A., Verde, M. L., Ronsini, C., Riemma, G., Gaiano, M., et al. (2024) · Journal of Personalized Medicine · doi:10.3390/jpm14040351 · Cited by 9
Orso, D., Sabbadin, M., Bacchetti, G., Simeoni, G., & Bove, T. (2024) · Journal of Cardiothoracic and Vascular Anesthesia · doi:10.1053/j.jvca.2024.08.014 · Cited by 10
Cognitive Training During Midlife: A Systematic Review and Meta-Analysis
Zhu, C., Arunogiri, S., Li, Q., Thomas, E. H. X., & Gurvich, C. (2024) · Neuropsychology Review · doi:10.1007/s11065-024-09649-z · Cited by 11
Saltaouras, G., Kyrkili, A., Bathrellou, E., Georgoulis, M., Yannakoulia, M., Bountziouka, V., Smrke, U., Dimitrakopoulos, G., & Kontogianni, M. D. (2024) · Children · doi:10.3390/children11091100 · Cited by 13
Paucity of gastrointestinal plasma cells in common variable immunodeficiency
Marsden, J. W. N., Laclé, M. M., Severs, M., et al. (2024) · Current Opinion in Allergy and Clinical Immunology · doi:10.1097/aci.0000000000001040 · Cited by 9
Nau, T., Cutts, S., & Naidoo, N. S. (2025) · EFORT Open Reviews · doi:10.1530/eor-22-0088 · Cited by 9
Nocturia, Sleep Quality, and Mortality: A Systematic Review and Meta-Analysis
Lavadia, A. C., Kim, J. H., Yun, S. W., & Noh, T. I. (2025) · The World Journal of Men s Health · doi:10.5534/wjmh.240237 · Cited by 11
Gutiérrez‐Ramírez, L., García-Dionisio, S. L., Feo‐Ortega, S., González‐Cervera, J., Tejera‐Muñoz, A., Lucendo, A. J., & Arias, Á. (2025) · Helicobacter · doi:10.1111/hel.70038 · Cited by 11
Shettigar, D., Sukumar, S., Pradhan, A., et al. (2025) · Radiography · doi:10.1016/j.radi.2025.102955 · Cited by 9
Lei, I. I., Agache, A., Robertson, A., et al. (2025) · BMJ Open Gastroenterology · doi:10.1136/bmjgast-2025-001800 · Cited by 9
The putative effects of orexin receptor antagonists on pain and sleep in humans: A systematic review
Babiloni, A. H., Sangalli, L., Puertas, F. J., Morin, C. M., Lavigne, G. J., & Fabbro, C. D. (2025) · Sleep Medicine · doi:10.1016/j.sleep.2025.106633 · Cited by 9
Jiménez, M. D. C. F.-F., & Storz, M. A. (2025) · Current Nutrition Reports · doi:10.1007/s13668-025-00688-3 · Cited by 10
Dastjerdi, P., Mohammadi, N. S. H., Anaraki, N., Rahmati, S., Nikfar, R., Momeni, S., Zafarmandi, S., Saeidi, S., Askari, M. K., Hosseini, S., et al. (2025) · Diabetology & Metabolic Syndrome · doi:10.1186/s13098-025-01900-8 · Cited by 11
Verbeek, J. H. A. M., Zeeb, H., van Deventer, E., et al. (2025) · Environment International · doi:10.1016/j.envint.2025.109751 · Cited by 10
Self-citationThoerig, R. C., O’Connor, L. E., Spill, M. K., et al. (2025) · American Journal of Clinical Nutrition · doi:10.1016/j.ajcnut.2025.07.039 · Cited by 9
Value of pelvic examination in women with pelvic organ prolapse: A systematic review
Pizzoferrato, A., Sallée, C., Thubert, T., Fauconnier, A., & Deffieux, X. (2024) · International Journal of Gynecology & Obstetrics · doi:10.1002/ijgo.15697 · Cited by 9
Cao, L., Su, J., Tian, F., Zhou, Y., Liu, S., & Lou, F. (2024) · Journal of Paediatrics and Child Health · doi:10.1111/jpc.16668 · Cited by 8
Loef, B., Boer, J. M. A., Beekman, M., Campman, S. L., Hoogendijk, E. O., Huider, F., Pagen, D. M. E., Splinter, M. J., van der Velde, J. H. P. M., Boomsma, D. I., et al. (2024) · International Journal of Obesity · doi:10.1038/s41366-024-01660-x · Cited by 8
Glioblastoma invasion patterns from a clinical perspective—a systematic review
Percuoco, V., Herlin, E., Prada, F. U., Riva, M., Pessina, F., Staartjes, V. E., Pepa, G. M. D., & Menna, G. (2024) · Neurosurgical Review · doi:10.1007/s10143-024-02944-6 · Cited by 8
Effect of neighborhood socioeconomic disadvantage on 30‐day readmissions: A systematic review
Garrett, L., Muhammad, A., & Kulshreshtha, A. (2024) · Journal of Hospital Medicine · doi:10.1002/jhm.13581 · Cited by 9
Kim, A. Y., Ahn, B. “., Kim, Y., Oh, J. W., Park, J., Jung, H. W., & Kim, W. J. (2025) · Archives of Gerontology and Geriatrics · doi:10.1016/j.archger.2025.105773 · Cited by 9
Effectiveness of interventions to improve vaccine efficacy: a systematic review and meta-analysis
Aviraj, K. S., Wasnik, A., Gupta, L., Ranjan, A., & Suresh, H. (2025) · Systematic Reviews · doi:10.1186/s13643-025-02856-6 · Cited by 10
Association of cognitive impairment and peripheral artery disease (PAD): A systematic review
Cheshire, B. L., Messeder, S. J., Pepper, C., Beishon, L., Sayers, R. D., & Houghton, J. S. M. (2025) · Vascular Medicine · doi:10.1177/1358863x251336736 · Cited by 11
Motsamai, P. F. T., & Onyenankeya, K. (2025) · Social Sciences & Humanities Open · doi:10.1016/j.ssaho.2025.101589 · Cited by 19
Chye, D. M., Rodrigues, T. S., Quarto, L. J. G., Young, N., Hamilton, G. W., Burrell, L. M., Teh, A. W., Lim, H. S., & Koshy, A. N. (2025) · Heart Rhythm · doi:10.1016/j.hrthm.2025.05.062 · Cited by 8
Coelho, D. R. A., Nicoloro‐SantaBarbara, J. M., Majd, M., Vieira, W. F., Laney, M. D., Lydston, M., Martins‐Filho, O. A., Bahia‐Oliveira, L. M. G., Salvi, J. D., Cassano, P., et al. (2025) · Acta Psychiatrica Scandinavica · doi:10.1111/acps.13824 · Cited by 8
Mantalovas, S., Karakousis, V. A., Sevva, C. D., et al. (2025) · Cancers · doi:10.3390/cancers17121934 · Cited by 8
Tian, E., Wilson, O. W. A., Wojcik, K. M., Butera, G., Schneider, J., Rogers, L. Q., Demark‐Wahnefried, W., & Jayasekera, J. C. (2025) · Quality of Life Research · doi:10.1007/s11136-025-03992-1 · Cited by 9
Schofield, J., Conti, A. A., Khan, F., & Baldacchino, A. (2025) · European Journal of Preventive Cardiology · doi:10.1093/eurjpc/zwaf500 · Cited by 9
Risk conditions for invasive pneumococcal disease in adults: a systematic review and meta-analysis
Ditzel, K., Giardina, F., ten Oever, J., & Cremers, A. J. H. (2025) · EClinicalMedicine · doi:10.1016/j.eclinm.2025.103522 · Cited by 14
Impact of risk perception on emergency information seeking behavior: a meta-analysis
Gong, W., Li, Z., & Tang, J. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1646584 · Cited by 9
Health impacts of micro- and nanoplastics in humans: systematic review of in vivo evidence
Anna, H. A. T., Ng, J. J., Tang, D. H., Yong, C. T., & Ho, A. F. W. (2026) · Environmental Health · doi:10.1186/s12940-026-01282-y · Cited by 12
Rong, C. W. W., Salleh, H., Nishio, H., & Lee, M. (2024) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2024.174348 · Cited by 10
Lai, Q., Coppola, A., Mrzljak, A., & Berković, M. C. (2024) · Biomedicines · doi:10.3390/biomedicines12112419 · Cited by 8
Bloodstream Infections in the Neonatal Intensive Care Unit: A Systematic Review of the Literature
Vogiantzi, G., Metallinou, D., Τίγκα, Μ., et al. (2024) · Cureus · doi:10.7759/cureus.68057 · Cited by 7
Olivas-Martínez, A., Ventura, P. S., Fernández, M. F., & Freire, C. (2024) · International Journal of Hygiene and Environmental Health · doi:10.1016/j.ijheh.2024.114487 · Cited by 7
Balalian, A. A., Spill, M. K., Thoerig, R. C., et al. (2024) · Advances in Nutrition · doi:10.1016/j.advnut.2024.100350 · Cited by 7
Chen, Q.-Y., Keum, N., & Giovannucci, E. L. (2025) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djaf098 · Cited by 9
Ronca, D. B., Mesquita, L. O., Oliveira, D. B., Figueiredo, A. C. M. G., Wen, J., Song, M., & de Carvalho, K. M. B. (2025) · PLoS ONE · doi:10.1371/journal.pone.0314892 · Cited by 8
Does breastfeeding increase the risk of early childhood caries (ECC)? A systematic review
Alexaki, F., Kostopoulou, M., Koleventi, K., & Lygidakis, N. N. (2025) · European Archives of Paediatric Dentistry · doi:10.1007/s40368-025-01051-4 · Cited by 8
Agache, I. O., Salazar, J., Rodriguez‐Tanta, Y., et al. (2025) · Allergy · doi:10.1111/all.16611 · Cited by 7
Taylor, H. E., Salf, J. C., Roper-Marchand, C. R., Senaratne, D. N. S., Smith, B. H., Colvin, L., & George, J. (2025) · British Journal of Anaesthesia · doi:10.1016/j.bja.2025.06.006 · Cited by 9
Ferreira, N. B., Ponte, A., Grande, A. C., et al. (2025) · Sleep Medicine · doi:10.1016/j.sleep.2025.106705 · Cited by 7
Guidance for umbrella reviews of observational studies: A scoping review
Zhou, C., Fabiano, N., Gupta, A. A., Wong, S., Cobey, K. D., Moher, D., Ebrahimzadeh, S., Ng, J. Y., Dragioti, E., Shin, J. I., et al. (2025) · JCPP Advances · doi:10.1002/jcv2.70017 · Cited by 7
Self-citationLyu, D., Zhuang, R., Li, J., Wu, Y., Di, Y., Song, M., Ma, L., Li, J., & Zhang, Y. (2025) · PLoS ONE · doi:10.1371/journal.pone.0337091 · Cited by 9
Sleep alterations in substance use disorders: a systematic review and meta-analysis
Oliva, H. N. P., Prudente, T. P., Naveda, A. M. P., Monteiro-Júnior, R. S., Potenza, M. N., MORGAN, P. T., & Angarita, G. A. (2026) · EClinicalMedicine · doi:10.1016/j.eclinm.2025.103723 · Cited by 9
Olson, M. J., Withrow, D., Koelbel, M., Southworth, G., Phan, A., Wright, K. P., Whisner, C. M., & Petrov, M. E. R. (2026) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2026.102256 · Cited by 11
Risk Factors and Interventions for Suicide in Huntington’s Disease—A Systematic Review
Grimaldi, A., Veneziani, I., Culicetto, L., Quartarone, A., & Buono, V. L. (2024) · Journal of Clinical Medicine · doi:10.3390/jcm13123437 · Cited by 7
Sun, K., Choi, Y. T., Yu, C. C.-W., Nelson, E. A. S., Goh, J., Dai, S., & Hui, L. L. (2024) · Sports Health A Multidisciplinary Approach · doi:10.1177/19417381241271547 · Cited by 9
Belletti, A., Ursoleo, J. D., Piazza, E., et al. (2024) · Artificial Organs · doi:10.1111/aor.14864 · Cited by 7
Secondary Smoking and Early Childhood Caries: A Systematic Review and Meta-Analysis
Luo, B. W., Sun, I. G., Chan, S. S. C., et al. (2024) · International Dental Journal · doi:10.1016/j.identj.2024.08.013 · Cited by 7
Fadel, A. M. A., Jakobsen, K. K., Jensen, L. H., et al. (2025) · Oral Oncology · doi:10.1016/j.oraloncology.2025.107311 · Cited by 7
Methodology of the Updated and Expanded Australasian Bronchiolitis Guideline
Loveys, K., Tavender, E. J., Babl, F. E., Cotterell, E., Haskell, L., O’Brien, S., Oakley, E. A., Wilson, C. L., Borland, M. L., & Dalziel, S. R. (2025) · Journal of Paediatrics and Child Health · doi:10.1111/jpc.70145 · Cited by 7
Agache, I., Balbin‐Ramon, G. J., Saenz, F. K. F., Sola‐Arnau, I., Alonso‐Coello, P., Haahtela, T., Traidl‐Hoffmann, C., O'Mahony, L., Damialis, A., Lauerma, A., et al. (2025) · Allergy · doi:10.1111/all.16653 · Cited by 6
“The SR used Risk of Bias in Non-randomized Studies of Exposures (ROBINS-E) tool [50].”
Communicating blood test results in primary care: a mixed-methods systematic review
Nankervis, H., Huntley, A. L., Whiting, P. F., Hamilton, W. T., Singh, H., Dawson, S., O’Donnell, R., Sprackman, J., Montague, A. F., & Watson, J. (2024) · British Journal of General Practice · doi:10.3399/bjgp.2024.0338 · Cited by 7
Tewari, J., Qidwai, K., Roy, S., Saxena, M., Rana, A., Tewari, A., Tewari, V., & Maheshwari, A. (2024) · Egyptian Heart Journal · doi:10.1186/s43044-024-00597-w · Cited by 5
Stadelmaier, J., Bantle, G., Gorenflo, L., Kiesswetter, E., Nikolakopoulou, A., & Schwingshackl, L. (2025) · BMC Medicine · doi:10.1186/s12916-025-03860-2 · Cited by 6
Aronica, R., Ostinelli, E. G., Austin, C., Oliver, D., McGuire, P. K., Brambilla, P., Torous, J. B., & Cipriani, A. (2025) · BMJ Mental Health · doi:10.1136/bmjment-2024-301337 · Cited by 7
Winder, C., Lodhia, A., Basso, M., & Kadosh, K. C. (2025) · Frontiers in Neuroscience · doi:10.3389/fnins.2025.1540180 · Cited by 7
Culicetto, L., Latella, D., Buono, V. L., Orecchio, F., Murdaca, A. M., Quartarone, A., & Marino, S. (2025) · Journal of Personalized Medicine · doi:10.3390/jpm15030092 · Cited by 7
Rashidian, P., Parsaei, M., Hantoushzadeh, S., & Salmanian, B. (2025) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-025-07444-z · Cited by 11
Evaluation of Urinary miRNA in Renal Cell Carcinoma: A Systematic Review
Cochetti, G. G., Guadagni, L., Paladini, A., et al. (2025) · Cancers · doi:10.3390/cancers17081336 · Cited by 6
Slough, E., Pitt-Francis, A., Belli, A., Ahmed, Z., Pietro, V. D., & Stevens, A. R. (2025) · Molecular Neurobiology · doi:10.1007/s12035-025-05053-7 · Cited by 6
Risk of Pancreatic Cancer and Precancerous Lesions in Patients With Chronic Pancreatitis
Wen, A., Pan, S. Y., Dadgar, K., & Yaghoobi, M. (2025) · Journal of Clinical Gastroenterology · doi:10.1097/mcg.0000000000002187 · Cited by 6
Pathological Processes Among Content Creators on Social Media: Scoping Review
Martínez-Aguirre, S., Sanz‐Valero, J., & Ronda, E. (2025) · JMIR Public Health and Surveillance · doi:10.2196/76708 · Cited by 6
Lopez, R. F. R., Barrera, D. L., Muñoz, M. C. A., & Díaz, M. (2025) · Cureus · doi:10.7759/cureus.90021 · Cited by 7
Raghavan, S., Isaac, T. S., Arya, D., Flores, G. C., Shanmugam, R., Na, A., Desiraju, B. K., Venugopal, V., Thiruvengadam, R., Kshetrapal, P., et al. (2025) · BMJ Paediatrics Open · doi:10.1136/bmjpo-2025-003428 · Cited by 6
Андреев, Д. Н., Расимовна, Х. А., Maev, I. V., Bordin, D. S., Zaborovsky, A. V., Kucheryavyy, Y. A., Sokolov, F. S., & Beliy, P. A. (2025) · Journal of Personalized Medicine · doi:10.3390/jpm15100458 · Cited by 8
Pérez-Wiesner, M., Bühler, K., López‐Moreno, J. A., & López‐Salmerón, M. D. (2025) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph22101598 · Cited by 15
Allaw, S., Khabaz, K., Yu, Q., & Ahmed, O. S. (2025) · Journal of Vascular and Interventional Radiology · doi:10.1016/j.jvir.2025.10.027 · Cited by 6
Zheng, W.-H., Hu, Y., Yu, D.-X., et al. (2025) · Frontiers in Nutrition · doi:10.3389/fnut.2025.1655488 · Cited by 6
Intra-articular therapies for synovial joint dysfunction: a comprehensive integrative review
Neves, L. B. M., Lima, B. C., de Mello, R. C., et al. (2025) · Oral Surgery Oral Medicine Oral Pathology and Oral Radiology · doi:10.1016/j.oooo.2025.10.026 · Cited by 6
Kawashima, A., Noguchi, T., Furukawa, T., Unesoko, A., Togashi, S., & Sato, K. (2025) · Palliative Medicine · doi:10.1177/02692163251403395 · Cited by 7
Valdesalici, A., Sella, E., Domenicucci, R., Ghisi, M., & Borella, E. (2026) · Psychology of sport and exercise · doi:10.1016/j.psychsport.2026.103079 · Cited by 7
Khaled, K., Abdulbaki, N., Almilaji, O., Casey, C., & Tsofliou, F. (2026) · Journal of Human Nutrition and Dietetics · doi:10.1111/jhn.70217 · Cited by 8
Slurink, I. A. L., Vogtschmidt, Y. D., Brummel, B., Smeets, T., Kupper, N. H. M., & Soedamah‐Muthu, S. S. (2024) · Current Developments in Nutrition · doi:10.1016/j.cdnut.2024.104470 · Cited by 6
Pouliopoulou, D. V., Billias, N., MacDermid, J. C., Miller, E., O’Brien, K. K., Quinn, K. L., Malvankar‐Mehta, M. S., Pereira, T., Cheung, A. M., Razak, F. A., et al. (2024) · EClinicalMedicine · doi:10.1016/j.eclinm.2024.102993 · Cited by 6
Meta-analysis of antifungal resistance patterns of Aspergillus species in Iran
Mianrood, I. B., Ahmadian, M., Farjami, M., Gharabagh, F. J., Khodavaisy, S., & Darazam, I. A. (2025) · Journal of Infection and Public Health · doi:10.1016/j.jiph.2025.102838 · Cited by 6
Adil, S., Gordon, M. A., Hathagoda, W., Kuruppu, C., Benninga, M. A., & Rajindrajith, S. (2024) · BMJ Paediatrics Open · doi:10.1136/bmjpo-2024-003069 · Cited by 7
Chami-Peña, S., Caballero-Vázquez, A., Membrive‐Jiménez, M. J., Gómez‐Urquiza, J. L., Romero‐Béjar, J. L., Caballero-Mateos, A. M., & la Fuente, G. A. C. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm14041230 · Cited by 5
An Evaluation of Serum miRNA in Renal Cell Carcinoma: A Systematic Review
Cochetti, G. G., Guadagni, L., Paladini, A., et al. (2025) · Cancers · doi:10.3390/cancers17050816 · Cited by 5
Suicidal Ideation and Behaviors in Adults With Tinnitus: A Systematic Review and Meta‐Analysis
McCray, L. R., Scharner, M. K., Nguyen, S. A., Rizk, H. G., Meyer, T. A., Labadie, R. F., Uhde, T. W., & Dixon, P. R. (2025) · The Laryngoscope · doi:10.1002/lary.32118 · Cited by 36
A Systematic Review of Sacroiliac Joint Injections of Platelet-Rich Plasma (Prp) and Stem Cells
Manchikanti, L., Kaye, A. D., Abd-Elsayed, A. A., et al. (2025) · Current Pain and Headache Reports · doi:10.1007/s11916-025-01377-0 · Cited by 5
Risk Factors for Suicide Among Head and Neck Cancer Patients: A Systematic Review and Meta‐Analysis
Nassar, S. I., Durgham, R. A., Nguyen, S. A., Newman, J. G., Kejner, A. E., & Albergotti, W. G. (2025) · Head & Neck · doi:10.1002/hed.28145 · Cited by 7
da Rocha, W. S., Reis-Silva, A., Coelho-Oliveira, A. C., et al. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm14103425 · Cited by 5
Metadata
- Authors
- Higgins, J. P. T., Morgan, R. L., Rooney, A. A., Taylor, K. W., Thayer, K. A., Silva, R. A., Lemeris, C., Akl, E. A., Bateson, T. F., Berkman, N. D., Glenn, B. S., Hróbjartsson, A., LaKind, J. S., McAleenan, A., Meerpohl, J. J., Nachman, R. M., Obbagy, J. E., O'Connor, A., Radke, E. G., Savović, J., Schünemann, H. J., Shea, B., Tilling, K., Verbeek, J., Viswanathan, M., & Sterne, J. A. C.
- Affiliations
- University of Bristol (GB)
- McMaster University (CA)
- National Institute of Environmental Health Sciences (US)
- Environmental Protection Agency (US)
- ICF International (United States) (US)
- American University of Beirut (LB)
- RTI International (US)
- University of Southern Denmark (DK)
- University of Freiburg (DE)
- United States Department of Agriculture (US)
- Michigan State University (US)
- Cochrane Canada (CA)
- Ottawa Hospital Research Institute (CA)
- Amsterdam UMC Location University of Amsterdam (NL)
- Corresponding author
- Higgins, Julian P.T.
- ORCID
- Higgins, Julian P.T.
- Morgan, Rebecca L.
- Rooney, Andrew A.
- Taylor, Kyla W.
- Thayer, Kristina A.
- Silva, Raquel A.
- Lemeris, Courtney
- Akl, Elie A.
- Bateson, Thomas F.
- Berkman, Nancy D.
- Hróbjartsson, Asbjørn
- LaKind, Judy S.
- McAleenan, Alexandra
- Meerpohl, Joerg J.
- Nachman, Rebecca M.
- Obbagy, Julie E.
- O'Connor, Annette
- Radke, Elizabeth G.
- Savović, Jelena
- Schünemann, Holger J.
- Shea, Beverley
- Tilling, Kate
- Verbeek, Jos
- Viswanathan, Meera
- Sterne, Jonathan A.C.
- Venue
- Environment International
- Publisher
- Elsevier BV
- Published
- 2024-04
- Volume
- 186
- Pages
- 108602
- ISSN
- 0160-4120 (ISSN-L)
- PMC
- PMC11098530
- OpenAlex
- W4393132304
- Semantic Scholar
- b822b2b6998a5003935cf5815b5370685f4e5231
- Cited by
- 1,022
- References
- 30
- Citations by year
- 2026: 470 · 2025: 440 · 2024: 112
- FWCI
- 284.27
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Environment International (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Pure Amsterdam UMC (publishedVersion, cc-by)
- University of Southern Denmark Research Portal (University of Southern Denmark) (publishedVersion, cc-by-nc)
- PubMed Central (acceptedVersion, cc-by-nc)
- Europe PMC (PubMed Central) (acceptedVersion, cc-by-nc)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Bristol Research (University of Bristol) (submittedVersion, cc-by-nc-nd)
- Explore Bristol Research (submittedVersion, cc-by-nc-nd)
- Funders
- National Institute for Health and Care Research
- Cancer Research UK
- National Institute of Environmental Health Sciences
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- risk of bias assessment, non-randomized studies, cohort studies, causal inference, systematic review methodology, observational epidemiology
- MeSH terms
- Environmental Exposure, Follow-Up Studies, Humans, Cohort Studies, Bias, Epidemiologic Studies, Risk Assessment, Observational Studies as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
