CoChat

ROB-MEN: a tool to assess risk of bias due to missing evidence in network meta-analysis

Open access
Chiocchia, Virginia, Nikolakopoulou, Adriani, Higgins, Julian P. T., Page, Matthew J., Papakonstantinou, Theodoros, Cipriani, Andrea, Furukawa, Toshi A., Siontis, George C. M., Egger, Matthias, Salanti, Georgia · 2021 · BMC Medicine
Cited by 129

The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.

Abstract

Background: Selective outcome reporting and publication bias threaten the validity of systematic reviews and meta-analyses and can affect clinical decision-making. A rigorous method to evaluate the impact of this bias on the results of network meta-analyses of interventions is lacking. We present a tool to assess the Risk Of Bias due to Missing Evidence in Network meta-analysis (ROB-MEN). Methods: ROB-MEN first evaluates the risk of bias due to missing evidence for each of the possible pairwise comparison that can be made between the interventions in the network. This step considers possible bias due to the presence of studies with unavailable results ( within-study assessment of bias ) and the potential for unpublished studies ( across-study assessment of bias ). The second step combines the judgements about the risk of bias due to missing evidence in pairwise comparisons with (i) the contribution of direct comparisons to the network meta-analysis estimates, (ii) possible small-study effects evaluated by network meta-regression, and (iii) any bias from unobserved comparisons. Then, a level of “low risk”, “some concerns”, or “high risk” for the bias due to missing evidence is assigned to each estimate, which is our tool’s final output. Results: We describe the methodology of ROB-MEN step-by-step using an illustrative example from a published NMA of non-diagnostic modalities for the detection of coronary artery disease in patients with low risk acute coronary syndrome. We also report a full application of the tool on a larger and more complex published network of 18 drugs from head-to-head studies for the acute treatment of adults with major depressive disorder. Conclusions: ROB-MEN is the first tool for evaluating the risk of bias due to missing evidence in network meta-analysis and applies to networks of all sizes and geometry. The use of ROB-MEN is facilitated by an R Shiny web application that produces the Pairwise Comparisons and ROB-MEN Table and is incorporated in the reporting bias domain of the CINeMA framework and software.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s12916-021-02166-3

Cites

18 of 18 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 18 references

Cited by

See all 129

Metadata

Authors
Chiocchia, V., Nikolakopoulou, A., Higgins, J. P. T., Page, M. J., Papakonstantinou, T., Cipriani, A., Furukawa, T. A., Siontis, G. C. M., Egger, M., & Salanti, G.
Venue
BMC Medicine
Published
2021-11-23
Volume
19
Issue
1
Pages
304-304
ISSN
1741-7015 (ISSN-L)
OpenAlex
W3215732646
Cited by
129
References
18
FWCI
8.76
Type
journal article
Language
en
Open access
gold

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: ROB-MEN: a tool to assess risk of bias due to missing evidence in network meta-analysis. https://app.cochat.ai/papers/10.1186/s12916-021-02166-3
The paper
Chiocchia, V., Nikolakopoulou, A., Higgins, J. P. T., Page, M. J., Papakonstantinou, T., Cipriani, A., Furukawa, T. A., Siontis, G. C. M., Egger, M., & Salanti, G. (2021). ROB-MEN: a tool to assess risk of bias due to missing evidence in network meta-analysis. BMC Medicine. https://doi.org/10.1186/s12916-021-02166-3

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.