The DOI matches its Crossref record, with a small difference in title or year; 33 of the 33 references it cites were found in a registry or an index.
ROB-ME: a tool for assessing risk of bias due to missing evidence in systematic reviews with meta-analysis
Open accessAbstract
Various methods are available to help users assess whether selective non-publication of studies or selective non-reporting of study results has occurred, but not its impact on a meta-analysis. This limitation of existing methods leaves users to decide their own approach for judging the risk of bias in a meta-analysis result. In this paper, Page and colleagues describe the ROB-ME (risk of bias due to missing evidence) tool, a structured approach for assessing the risk of bias that arises when entire studies, or particular results within studies, are missing from a meta-analysis because of the P value, magnitude, or direction of the study results. The tool is anticipated to help authors and users of systematic reviews identify meta-analyses at high risk of bias and interpret results appropriately.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“We were not able to assess publication bias as none of the outcomes studied included more than 10 studies, which is a pre-requisite according to Cochrane's recommendations.83”
From Interventions to improve adherence to lipid-lowering drugs: a systematic review and meta-analysis (2025) · cites this paper as [83]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
RoB 2: a revised tool for assessing risk of bias in randomised trials
Sterne, J. A. C., Savović, J., Page, M. J., Elbers, R. G., Blencowe, N. S., Boutron, I., Cates, C. J., Cheng, H.-Y., Corbett, M. S., Eldridge, S. M., et al. (2019) · BMJ · doi:10.1136/bmj.l4898 · Cited by 34,787
Self-citationThe Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
Self-citationROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329
Self-citationSterne, J. A. C., Sutton, A. J., Ioannidis, J. P. A., Terrin, N., Jones, D. R., Lau, J., Carpenter, J., Rucker, G., Harbord, R. M., Schmid, C. H., et al. (2011) · BMJ · doi:10.1136/bmj.d4002 · Cited by 7,709
Self-citationSynthesis without meta-analysis (SWiM) in systematic reviews: reporting guideline
Campbell, M., McKenzie, J. E., Sowden, A., Katikireddi, S. V., Brennan, S. E., Ellis, S., Hartmann-Boyce, J., Ryan, R., Shepperd, S., Thomas, J., et al. (2020) · BMJ · doi:10.1136/bmj.l6890 · Cited by 4,628 · cited 1×
“32 33 We also plan to develop an interactive online version of the tool to facilitate use and prepare a bank of worked examples for educational purposes.”
MethodSelf-citationPeters, J. L., Sutton, A. J., Jones, D. R., Abrams, K. R., & Rushton, L. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.11.010 · Cited by 2,053
Self-citationThe GRADE Working Group clarifies the construct of certainty of evidence
Hultcrantz, M., Rind, D., Akl, E. A., Treweek, S., Mustafa, R. A., Iorio, A., Alper, B. S., Meerpohl, J. J., Murad, M. H., Ansari, M. T., et al. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.05.006 · Cited by 853
Self-citationCompleting ‘Summary of findings’ tables and grading the certainty of the evidence
Schünemann, H. J., Higgins, J. P., Vist, G. E., Glasziou, P., Akl, E. A., Skoetz, N., Guyatt, G. H., & on behalf of the Cochrane GRADEing Methods Group (formerly Applicability and Recommendations Methods Group) and the Cochrane Statistical Methods Group (2019) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9781119536604.ch14 · Cited by 669
Self-citationSynthesizing and presenting findings using other methods
McKenzie, J. E., & Brennan, S. E. (2019) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9781119536604.ch12 · Cited by 423
Self-citationAssessing risk of bias due to missing results in a synthesis
Page, M. J., Higgins, J. P., & Sterne, J. A. (2019) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9781119536604.ch13 · Cited by 402
Self-citation
Cited by
Sorted by citations
Systematic review and meta-analysis of preclinical studies
Ineichen, B. V., Held, U., Salanti, G., Macleod, M. R., & Wever, K. E. (2024) · Nature Reviews Methods Primers · doi:10.1038/s43586-024-00347-x · Cited by 28
Siafis, S., Chiocchia, V., Macleod, M. R., Austin, C., Homiar, A., Tinsdeall, F., Friedrich, C., Ramage, F., Kennett, J., Nomura, N., et al. (2024) · Wellcome Open Research · doi:10.12688/wellcomeopenres.21302.1 · Cited by 24
Triggers, Types, and Treatments for Kounis Syndrome: A Systematic Review
PÉREZ, E., Santaularia-Tomas, M., Martin-Dorantes, M. A., Villarreal-Jimenez, E., Olivera-Mar, A., Sanchez-Felix, E., Pérez-Navarrete, A., Millet-Herrera, J. L., Huchim-Mendez, O., Alejos-Briceño, R., et al. (2025) · Clinics and Practice · doi:10.3390/clinpract15030059 · Cited by 21
Milojevic, M., Freemantle, N., Hayanga, J. W. A., Kelly, R. F., Myers, P. O., Petersen, R. H., Opitz, I., Sabik, J. F., & Bakaeen, F. G. (2024) · Journal of Thoracic and Cardiovascular Surgery · doi:10.1016/j.jtcvs.2024.07.048 · Cited by 18
Lai, N. M., Lim, Y. S., Chaiyakunapruk, N., Lee, S. W. H., Liabsuetrakul, T., Sabbir, M., Mahmud, S. M. H., Siddiqi, U. R., Ahmed, T., Hasan, F. U. L., et al. (2025) · Child and Adolescent Mental Health · doi:10.1111/camh.70022 · Cited by 30
Metadata
- Authors
- Page, M. J., Sterne, J. A. C., Boutron, I., Hróbjartsson, A., Kirkham, J. J., Li, T., Lundh, A., Mayo-Wilson, E., McKenzie, J. E., Stewart, L. A., Sutton, A. J., Bero, L., Dunn, A. G., Dwan, K., Elbers, R. G., Kanukula, R., Meerpohl, J. J., Turner, E. H., & Higgins, J. P. T.
- Affiliations
- Monash University (AU)
- National Institute for Health and Care Research (GB)
- Inserm (FR)
- University of Southern Denmark (DK)
- Manchester Academic Health Science Centre (GB)
- University of Colorado Anschutz (US)
- Department of Epidemiology, UNC Gillings School of Global Public Health, Chapel Hill, NC, USA
- University of York (GB)
- University of Leicester (GB)
- Center for Bioethics and Humanities, University of Colorado, Aurora, CO, USA
- The University of Sydney (AU)
- Liverpool School of Tropical Medicine (GB)
- Erasmus MC (NL)
- University of Freiburg (DE)
- Oregon Health & Science University (US)
- Corresponding author
- Page, Matthew J
- ORCID
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2023-11-20
- Volume
- 383
- Pages
- e076754
- ISSN
- 1756-1833 (ISSN-L)
- OpenAlex
- W4388824619
- Cited by
- 134
- References
- 28
- Citations by year
- 2026: 63 · 2025: 50 · 2024: 17 · 2023: 4
- FWCI
- 9.50
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.bmj.com/company/legal-information/terms-conditions/legal-information/tdm-licencepolicy (tdm)
- Locations
- BMJ (publishedVersion)
- PubMed (publishedVersion)
- EUR Research Repository (Erasmus University Rotterdam) (publishedVersion, cc-by-nc-nd)
- Bristol Research (University of Bristol) (publishedVersion, cc-by)
- University of Southern Denmark Research Portal (University of Southern Denmark) (publishedVersion)
- LSTM Online Archive (Liverpool School of Tropical Medicine) (acceptedVersion)
- Research Explorer (The University of Manchester) (submittedVersion, other-oa)
- HAL (submittedVersion)
- Adelaide Research & Scholarship (AR&S) (University of Adelaide) (submittedVersion)
- Figshare (submittedVersion, cc-by)
- HighWire Press Open Archive (submittedVersion)
- Explore Bristol Research (submittedVersion, cc-by)
- Funders
- University Hospitals Bristol NHS Foundation Trust
- NIHR Bristol Biomedical Research Centre
- Australian Research Council
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Health Sciences › Nursing › Issues, ethics and legal aspects › Pharmaceutical industry and healthcare
- Keywords
- risk of bias, absence of evidence, publication bias, meta-analysis, systematic reviews, bias assessment tools
- MeSH terms
- Systematic Reviews as Topic, Humans, Research Design, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
