CoChat

A new improved graphical and quantitative method for detecting bias in meta-analysis

Furuya-Kanamori, Luis, Barendregt, Jan J., Doi, Suhail A.R. · 2018 · International Journal of Evidence-Based Healthcare

The DOI matches its Crossref record; 38 of the 40 references it cites were found in a registry or an index.

Abstract

Detection of publication and related biases remains suboptimal and threatens the validity and interpretation of meta-analytical findings. When bias is present, it usually differentially affects small and large studies manifesting as an association between precision and effect size and therefore visual asymmetry of conventional funnel plots. This asymmetry can be quantified and Egger's regression is, by far, the most widely used statistical measure for quantifying funnel plot asymmetry. However, concerns have been raised about both the visual appearance of funnel plots and the sensitivity of Egger's regression to detect such asymmetry, particularly when the number of studies is small. In this article, we propose a new graphical method, the Doi plot, to visualize asymmetry and also a new measure, the LFK index, to detect and quantify asymmetry of study effects in Doi plots. We demonstrate that the visual representation of asymmetry was better for the Doi plot when compared with the funnel plot. We also show that the diagnostic accuracy of the LFK index in discriminating between asymmetry due to simulated publication bias versus chance or no asymmetry was also better with the LFK index which had areas under the receiver operating characteristic curve of 0.74-0.88 with simulations of meta-analyses with five, 10, 15, and 20 studies. The Egger's regression result had lower areas under the receiver operating characteristic curve values of 0.58-0.75 across the same simulations. The LFK index also had a higher sensitivity (71.3-72.1%) than the Egger's regression result (18.5-43.0%). We conclude that the methods proposed in this article can markedly improve the ability of researchers to detect bias in meta-analysis.

Overview

Integrity signals

Registry

Crossref · 10.1097/xeb.0000000000000141

Cites

38 of 38 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 38 references

Research works printed but not linked: 3

  1. Cates, 2000, Lung cancer and passive smoking. Scales for visual test of publication bias are unfair, BMJ, 321, 1222
  2. Seagroatt, 1998, Bias in meta-analysis detected by a simple, graphical test. Test had 10% false positive rate, BMJ, 316, 470
  3. Balint, 2016, Which endovenous ablation method does offer a better long-term technical success in the treatment of the incompetent great saphenous vein?, Rev Vasc, 24, 649

Cited by

See all 1,160

Metadata

Authors
Furuya-Kanamori, L., Barendregt, J. J., & Doi, S. A. R.
Affiliations
  • EpiGear (Australia) (AU)
  • Australian National University (AU)
Corresponding author
Barendregt, Jan J.
Venue
International Journal of Evidence-Based Healthcare
Published
2018-12
Volume
16
Issue
4
Pages
195-203
ISSN
  • 1744-1595 (ISSN-L)
  • 1744-1609
OpenAlex
W2799972028
Cited by
1,160
References
40
Citations by year
2026: 206 · 2025: 286 · 2024: 218 · 2023: 150 · 2022: 116 · 2021: 97 · 2020: 49 · 2019: 31 · 2018: 7
FWCI
35.03
Type
journal article
Language
en
Open access
closed
Locations
  • International Journal of Evidence-Based Healthcare (publishedVersion)
  • PubMed (publishedVersion)
  • ANU Open Research (Australian National University) (submittedVersion)
  • Qatar University QSpace (Qatar University) (submittedVersion)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Keywords
publication bias, funnel plot asymmetry, small-study effects, diagnostic accuracy
MeSH terms
Computer Simulation, ROC Curve, Meta-Analysis as Topic, Models, Statistical, Bias, Publication Bias

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: A new improved graphical and quantitative method for detecting bias in meta-analysis. https://app.cochat.ai/papers/10.1097/xeb.0000000000000141
The paper
Furuya-Kanamori, L., Barendregt, J. J., & Doi, S. A. R. (2018). A new improved graphical and quantitative method for detecting bias in meta-analysis. International Journal of Evidence-Based Healthcare. https://doi.org/10.1097/xeb.0000000000000141

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.