The DOI matches its Crossref record, with a small difference in title or year; 33 of the 33 references it cites were found in a registry or an index.
ROB-ME: a tool for assessing risk of bias due to missing evidence in systematic reviews with meta-analysis
Open accessAbstract
Various methods are available to help users assess whether selective non-publication of studies or selective non-reporting of study results has occurred, but not its impact on a meta-analysis. This limitation of existing methods leaves users to decide their own approach for judging the risk of bias in a meta-analysis result. In this paper, Page and colleagues describe the ROB-ME (risk of bias due to missing evidence) tool, a structured approach for assessing the risk of bias that arises when entire studies, or particular results within studies, are missing from a meta-analysis because of the P value, magnitude, or direction of the study results. The tool is anticipated to help authors and users of systematic reviews identify meta-analyses at high risk of bias and interpret results appropriately.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“We were not able to assess publication bias as none of the outcomes studied included more than 10 studies, which is a pre-requisite according to Cochrane's recommendations.83”
From Interventions to improve adherence to lipid-lowering drugs: a systematic review and meta-analysis (2025) · cites this paper as [83]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
RoB 2: a revised tool for assessing risk of bias in randomised trials
Sterne, J. A. C., Savović, J., Page, M. J., Elbers, R. G., Blencowe, N. S., Boutron, I., Cates, C. J., Cheng, H.-Y., Corbett, M. S., Eldridge, S. M., et al. (2019) · BMJ · doi:10.1136/bmj.l4898 · Cited by 34,787
Self-citationThe Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
Self-citationROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329
Self-citationSterne, J. A. C., Sutton, A. J., Ioannidis, J. P. A., Terrin, N., Jones, D. R., Lau, J., Carpenter, J., Rucker, G., Harbord, R. M., Schmid, C. H., et al. (2011) · BMJ · doi:10.1136/bmj.d4002 · Cited by 7,709
Self-citationSynthesis without meta-analysis (SWiM) in systematic reviews: reporting guideline
Campbell, M., McKenzie, J. E., Sowden, A., Katikireddi, S. V., Brennan, S. E., Ellis, S., Hartmann-Boyce, J., Ryan, R., Shepperd, S., Thomas, J., et al. (2020) · BMJ · doi:10.1136/bmj.l6890 · Cited by 4,628 · cited 1×
“32 33 We also plan to develop an interactive online version of the tool to facilitate use and prepare a bank of worked examples for educational purposes.”
MethodSelf-citationPeters, J. L., Sutton, A. J., Jones, D. R., Abrams, K. R., & Rushton, L. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.11.010 · Cited by 2,053
Self-citationThe GRADE Working Group clarifies the construct of certainty of evidence
Hultcrantz, M., Rind, D., Akl, E. A., Treweek, S., Mustafa, R. A., Iorio, A., Alper, B. S., Meerpohl, J. J., Murad, M. H., Ansari, M. T., et al. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.05.006 · Cited by 853
Self-citationCompleting ‘Summary of findings’ tables and grading the certainty of the evidence
Schünemann, H. J., Higgins, J. P., Vist, G. E., Glasziou, P., Akl, E. A., Skoetz, N., Guyatt, G. H., & on behalf of the Cochrane GRADEing Methods Group (formerly Applicability and Recommendations Methods Group) and the Cochrane Statistical Methods Group (2019) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9781119536604.ch14 · Cited by 669
Self-citationSynthesizing and presenting findings using other methods
McKenzie, J. E., & Brennan, S. E. (2019) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9781119536604.ch12 · Cited by 423
Self-citationAssessing risk of bias due to missing results in a synthesis
Page, M. J., Higgins, J. P., & Sterne, J. A. (2019) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9781119536604.ch13 · Cited by 402
Self-citation
Cited by
Sorted by citations
Items 101–135 of 135
Lee, Y., & Kim, S. J. (2026) · ANZ Journal of Surgery · doi:10.1111/ans.70839
Functional neuroimaging of Cannabidiol in stress and anxiety: a systematic review
Rutledge, O., Goyette, R., Wang, K. L., Park, M. S., & Gabrieli, J. D. E. (2026) · Frontiers in Neuroimaging · doi:10.3389/fnimg.2026.1860919
Klier, K., Masur, L., & Düking, P. (2026) · Frontiers in Sports and Active Living · doi:10.3389/fspor.2026.1872518
Kowalski, J. (2025) · Przegląd Psychologiczny · doi:10.31648/przegldpsychologiczny.12958
Rating Down for Publication Bias is Rare in Cochrane Reviews: A Descriptive Study
Toews, M., Minozzi, S., Stegeman, B. H., Cinquini, M., Meerpohl, J. J., Färnqvist, K., Bruschettini, M., & Toews, I. (2026) · Cochrane Evidence Synthesis and Methods · doi:10.1002/cesm.70089
Self-citationDevelopment of a tool to assess the risk of bias in statistical simulation studies: study protocol
Arnup, S. J., Turner, S., Page, M. J., Korevaar, E., McDonald, S. J., Higgins, J. P. T., & McKenzie, J. E. (2026) · F1000Research · doi:10.12688/f1000research.184100.1
Self-citationKang, Y., Habibula, S., Heffernan, C., Farmer, C., Meléndez‐Torres, G. J., Nurse, J., & Fellmeth, G. L. (2026) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd016376
Leveille, C. F., Lewandowski, N., Tsang, Q. K., Churchill, I. F., Zhu, X. M., Gallo, L., Chin, B. H., & Coroneos, C. J. (2026) · Aesthetic Surgery Journal · doi:10.1093/asj/sjag162
Iftikhar, R., Khan, M., Mahnoor, M., Awais, K. A. D., Risalat, A., Gilani, M., Shamshad, G. U., Malik, S. S., Skoetz, N., Monsef, I., et al. (2026) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd016225
Gaston, S. A., Zhou, W., Singh, R., Shields, T. C., Livinski, A. A., Redline, S. S., Chaput, J., Spruyt, K., Han, F., Owens, J. A., et al. (2026) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2026.102350
Ahmed, S., McCollin, S. P., Martí-Carvajal, A. J., Akus, M., Baron, D. W., Razzaq, S. A., Scudder, P. N., Honigsberg, E. J., Nasir, S., Peng, K., et al. (2026) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd016380
Ketamine and esketamine for postoperative analgesia in gynaecological surgery
Marchesi, S., Liljencrantz, J., Balintescu, A., Soumpasis, I., Sinervo, E., Makari, S., Bentzer, P., & Bjurström, M. F. (2026) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd016377
Pinheiro, V. H. G., Guesser, T. A. I., Cuesta‐Vargas, A., & Alberton, C. L. (2026) · Rehabilitation Oncology · doi:10.1097/01.reo.0000000000000424
Renal Safety of Topical Diclofenac in Adults with Increased Renal Risk: A Systematic Review
Coșovanu, E. O., Coșovanu, E. T., Balan, T. A., Foia, C. I., Tartau, C. G., Lunguleac, T., Stana, A.-B., Petroaie, A. D., Giuşcă, S. E., Coman, A. E., et al. (2026) · Medicina · doi:10.3390/medicina62081606
Chen, Z., Johansson, M., Albarqouni, L., Woloshin, S., & Jørgensen, K. J. (2026) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd016234
Nweke, M., & Pillay, J. D. (2026) · Heliyon · doi:10.1016/j.heliyon.2026.e45288
Keshtegar, S., Lashkarbolouki, A., Keshtehgar, H., Tayebi, P., & Shirafkan, H. (2026) · Artery Research · doi:10.1007/s44200-026-00124-z
Maiocco, G., Deantoni, S., Maggiora, E., Coscia, A., & Cresi, F. (2026) · BMJ Open · doi:10.1136/bmjopen-2026-117715
Liu, X., Bao, J., Yu, Z., Zheng, W., Huang, Y., Zhang, Y., & Jiang, X. (2026) · Antioxidants · doi:10.3390/antiox15101219
Beyond the hype: an empirical assessment of quantum computing in bioinformatics
Arcieri, M., Zuliani, P., Carotenuto, G., Carrus, M., & Castrignanò, T. (2026) · Frontiers in High Performance Computing · doi:10.3389/fhpcp.2026.1923360
Lipoprotein and apolipoprotein levels in age-related macular degeneration: A meta-analysis
Arsoudi, V., Chontos, T., Voutsas, S., Petrou, P., Tyrlis, K., Karavidas, I., Kandarakis, S., Chondrou, S., Diamanti, S., Angelidis, C. D., et al. (2026) · European Journal of Ophthalmology · doi:10.1177/11206721261492347
Lo, W. P., Baker, S. G., Bertolaccini, L., Bujkiewicz, S., Bunce, C., Buyse, M. E., Ciani, O., Christensen, R., DANIELS, J. P., Gandini, S., et al. (2026) · BMJ · doi:10.1136/bmj-2026-100618
Furukawa, Y., Siafis, S., Schneider‐Thoma, J., Nomura, N., Fares‐Otero, N. E., Illing, S., Storosum, B. W. C., Zantvoord, J. B., Schoretsanitis, G., Schwarz, C., et al. (2026) · The Lancet Psychiatry · doi:10.1016/s2215-0366(26)00280-4
Ganjre, S., Borah, L., & Khardenavis, A. (2026) · Cureus · doi:10.7759/cureus.117270
Prevalence of metabolic syndrome among patients with Alzheimer disease
Lichahi, M. A., Petkov, I., Smerat, A., Alsarhan, H. W., Letafatkar, N., Abbasnejad-Ardehroudkenari, M., Mohammadi, S., Jafari, M., Ghodous, S., & Amini‐Salehi, E. (2026) · Medicine · doi:10.1097/md.0000000000050670
Yao, Z., Wang, X., Zhang, H., Huang, Y., Yang, Z., Chen, C., Zheng, B., & Li, H. (2026) · Lung Cancer · doi:10.1016/j.lungcan.2026.109642
Saxena, R., Varshney, S., Khan, M. S., Bansal, M., Syed, R., Sanjaybhai, P. K., Singh, K., & Paul, R. (2026) · Cureus · doi:10.7759/cureus.117388
Liebetrau, B., Madsen, A. R., Möller, S., Olsen, M. H., Wiberg, S. C., & Mølstrøm, S. (2026) · Acta Anaesthesiologica Scandinavica · doi:10.1111/aas.70352
Paganin, W., Procenesi, L., Gorea, F., & Lorenzo, G. D. (2026) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2026.10.014
Wu, Y., Wei, J., Chang, C., Li, M., Xu, R., Dong, S., & Jiang, M. (2026) · Frontiers in Oncology · doi:10.3389/fonc.2026.1965694
Zygomatic implant fixed rehabilitation: a systematic review and narrative synthesis
Rodrigues, A., Nader, S. A., Singh, B. P., Atsü, S., & Emami, E. (2026) · Journal of Dentistry · doi:10.1016/j.jdent.2026.107122
D’Souza, P., Macdowall, W. G., Burchett, H., Kwan, I., & Sutcliffe, K. (2026) · Cochrane Evidence Synthesis and Methods · doi:10.1002/cesm.70107
Thermal stability and storage of adrenaline
Loftus, M. J., Lambert, P., Silk, R., Ford, L. S., Metzendorf, M., Thomson, D., Zubrinich, C. M., Cumpston, M., & Supported by the Cochrane Planetary Health Thematic Group (2026) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd016409
Alviz-Meza, A., Valencia-Arias, A., Díaz, F., Motta, B., & Hernández, G. (2026) · Machine Learning and Knowledge Extraction · doi:10.3390/make8110349 · Cited by 1
Nweke, M., & Pillay, J. D. (2026) · International Journal of Hypertension · doi:10.1155/ijhy/7268884
Metadata
- Authors
- Page, M. J., Sterne, J. A. C., Boutron, I., Hróbjartsson, A., Kirkham, J. J., Li, T., Lundh, A., Mayo-Wilson, E., McKenzie, J. E., Stewart, L. A., Sutton, A. J., Bero, L., Dunn, A. G., Dwan, K., Elbers, R. G., Kanukula, R., Meerpohl, J. J., Turner, E. H., & Higgins, J. P. T.
- Affiliations
- Monash University (AU)
- National Institute for Health and Care Research (GB)
- Inserm (FR)
- University of Southern Denmark (DK)
- Manchester Academic Health Science Centre (GB)
- University of Colorado Anschutz (US)
- Department of Epidemiology, UNC Gillings School of Global Public Health, Chapel Hill, NC, USA
- University of York (GB)
- University of Leicester (GB)
- Center for Bioethics and Humanities, University of Colorado, Aurora, CO, USA
- The University of Sydney (AU)
- Liverpool School of Tropical Medicine (GB)
- Erasmus MC (NL)
- University of Freiburg (DE)
- Oregon Health & Science University (US)
- Corresponding author
- Page, Matthew J
- ORCID
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2023-11-20
- Volume
- 383
- Pages
- e076754
- ISSN
- 1756-1833 (ISSN-L)
- OpenAlex
- W4388824619
- Cited by
- 134
- References
- 28
- Citations by year
- 2026: 63 · 2025: 50 · 2024: 17 · 2023: 4
- FWCI
- 9.50
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.bmj.com/company/legal-information/terms-conditions/legal-information/tdm-licencepolicy (tdm)
- Locations
- BMJ (publishedVersion)
- PubMed (publishedVersion)
- EUR Research Repository (Erasmus University Rotterdam) (publishedVersion, cc-by-nc-nd)
- Bristol Research (University of Bristol) (publishedVersion, cc-by)
- University of Southern Denmark Research Portal (University of Southern Denmark) (publishedVersion)
- LSTM Online Archive (Liverpool School of Tropical Medicine) (acceptedVersion)
- Research Explorer (The University of Manchester) (submittedVersion, other-oa)
- HAL (submittedVersion)
- Adelaide Research & Scholarship (AR&S) (University of Adelaide) (submittedVersion)
- Figshare (submittedVersion, cc-by)
- HighWire Press Open Archive (submittedVersion)
- Explore Bristol Research (submittedVersion, cc-by)
- Funders
- University Hospitals Bristol NHS Foundation Trust
- NIHR Bristol Biomedical Research Centre
- Australian Research Council
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Health Sciences › Nursing › Issues, ethics and legal aspects › Pharmaceutical industry and healthcare
- Keywords
- risk of bias, absence of evidence, publication bias, meta-analysis, systematic reviews, bias assessment tools
- MeSH terms
- Systematic Reviews as Topic, Humans, Research Design, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
