Simulation evaluation of statistical properties of methods for indirect and mixed treatment comparisons
Open accessThe DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Indirect treatment comparison (ITC) and mixed treatment comparisons (MTC) have been increasingly used in network meta-analyses. This simulation study comprehensively investigated statistical properties and performances of commonly used ITC and MTC methods, including simple ITC (the Bucher method), frequentist and Bayesian MTC methods. METHODS: A simple network of three sets of two-arm trials with a closed loop was simulated. Different simulation scenarios were based on different number of trials, assumed treatment effects, extent of heterogeneity, bias and inconsistency. The performance of the ITC and MTC methods was measured by the type I error, statistical power, observed bias and mean squared error (MSE). RESULTS: When there are no biases in primary studies, all ITC and MTC methods investigated are on average unbiased. Depending on the extent and direction of biases in different sets of studies, ITC and MTC methods may be more or less biased than direct treatment comparisons (DTC). Of the methods investigated, the simple ITC method has the largest mean squared error (MSE). The DTC is superior to the ITC in terms of statistical power and MSE. Under the simulated circumstances in which there are no systematic biases and inconsistencies, the performances of MTC methods are generally better than the performance of the corresponding DTC methods. For inconsistency detection in network meta-analysis, the methods evaluated are on average unbiased. The statistical power of commonly used methods for detecting inconsistency is very low. CONCLUSIONS: The available methods for indirect and mixed treatment comparisons have different advantages and limitations, depending on whether data analysed satisfies underlying assumptions. To choose the most valid statistical methods for research synthesis, an appropriate assessment of primary studies included in evidence network is required.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Assessing Risk of Bias in Included Studies
Higgins, J. P., & Altman, D. G. (2008) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9780470712184.ch8 · Cited by 3,396 · cited 1×
“Bias in a clinical trial can be defined as a systematic difference between the estimated effect size and the true effect size [27].”
BackgroundAssessing the quality of controlled clinical trials
Jüni, P., Altman, D. G., & Egger, M. (2001) · BMJ · doi:10.1136/bmj.323.7303.42 · Cited by 3,309 · cited 1×
“Empirical evidence indicated the existence of bias in randomised controlled trials [33-35], particularly in trials that had outcomes subjectively measured without appropriate blinding [36,37].”
BackgroundWood, L., Egger, M., Gluud, L. L., Schulz, K. F., Jüni, P., Altman, D. G., Gluud, C., Martin, R. M., Wood, A. J. G., & Sterne, J. A. C. (2008) · BMJ · doi:10.1136/bmj.39465.451748.ad · Cited by 2,538 · cited 1×
“Empirical evidence indicated the existence of bias in randomised controlled trials [33-35], particularly in trials that had outcomes subjectively measured without appropriate blinding [36,37].”
BackgroundChecking consistency in mixed treatment comparison meta-analysis
Dias, S., Welton, N. J., Caldwell, D. M., & Ades, A. E. (2010) · Statistics in Medicine · doi:10.1002/sim.3767 · Cited by 2,521 · cited 3×
“With the inconsistency Bayesian meta-analysis (IBMA), the estimate of dBC is naturally available, and dInd BC can be easily estimated based on dAB and dAC, as by the “nodesplitting” method [17,24].”
BackgroundMethodNetwork meta‐analysis for indirect treatment comparisons
Lumley, T. (2002) · Statistics in Medicine · doi:10.1002/sim.1201 · Cited by 1,206 · cited 5×
“; licensee BioMed Central L Commons Attribution License (http://creativec reproduction in any medium, provided the or indirectly compare multiple treatments and to combine evidence from direct and indirect comparisons in mixed treatment comparison (MTC) or network meta-analysis [5-9].”
BackgroundMethodInfluentialEvaluation of networks of randomized trials
Salanti, G., Higgins, J. P., Ades, A., & Ioannidis, J. P. (2008) · Statistical Methods in Medical Research · doi:10.1177/0962280207080643 · Cited by 1,078
Risk of bias versus quality assessment of randomised controlled trials: cross sectional study
Hartling, L., Ospina, M., Liang, Y., Dryden, D. M., Hooton, N., Krebs Seida, J., & Klassen, T. P. (2009) · BMJ · doi:10.1136/bmj.b4012 · Cited by 348
A proposed method of bias adjustment for meta-analyses of published observational studies
Thompson, S., Ekelund, U., Jebb, S., Lindroos, A. K., Mander, A., Sharp, S., Turner, R., & Wilks, D. (2011) · International Journal of Epidemiology · doi:10.1093/ije/dyq248 · Cited by 101 · cited 1×
“More complicated models could also be explored for estimating bias in evidence synthesis [39-41].”
Background
Cited by
CINeMA: an approach for assessing confidence in the results of a network meta-analysis
Nikolakopoulou, A., Higgins, J. P. T., Papakonstantinou, T., Chaimani, A., Del Giovane, C., Egger, M., & Salanti, G. (2020) · PLOS Medicine · doi:10.1371/journal.pmed.1003082 · Cited by 1,935
Evaluating the quality of evidence from a network meta-analysis
Salanti, G., Del Giovane, C., Chaimani, A., Caldwell, D. M., & Higgins, J. P. T. (2014) · PLoS ONE · doi:10.1371/journal.pone.0099682 · Cited by 1,402
“Not only are indirect comparisons integral to the methodology of network metaanalysis, but under certain conditions indirect estimates can be just as reliable as direct comparisons (and in some probably uncommon situations even more reliable) [16;17].”
BackgroundConceptual and technical challenges in network meta-analysis
Cipriani, A., Higgins, J. P. T., Geddes, J. R., & Salanti, G. (2013) · Annals of Internal Medicine · doi:10.7326/0003-4819-159-2-201307160-00008 · Cited by 1,072
“inconsistency is expected to be generally low (26) and might depend on the method used to estimate heterogeneity (27).”
MethodAdvances in the GRADE approach to rate the certainty in estimates from a network meta-analysis
Brignardello-Petersen, R., Bonner, A., Alexander, P. E., Siemieniuk, R. A., Furukawa, T. A., Rochwerg, B., Hazlewood, G. S., Alhazzani, W., Mustafa, R. A., Murad, M. H., et al. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.10.005 · Cited by 753
“However, failure of the global test to detect statistical incoherence when, for one or more of the paired comparisons, statistical incoherence exists may be a consequence of limited power of the global test [13,14].”
BackgroundA graphical tool for locating inconsistency in network meta-analyses
Krahn, U., Binder, H., & König, J. (2013) · BMC Medical Research Methodology · doi:10.1186/1471-2288-13-35 · Cited by 458
Metadata
- Authors
- Song, F., Clark, A., Bachmann, M. O., & Maas, J.
- Venue
- BMC Medical Research Methodology
- Published
- 2012-09-12
- Volume
- 12
- Issue
- 1
- Pages
- 138-138
- ISSN
- 1471-2288 (ISSN-L)
- OpenAlex
- W2136352805
- Cited by
- 92
- References
- 8
- FWCI
- 6.01
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Simulation evaluation of statistical properties of methods for indirect and mixed treatment comparisons. https://app.cochat.ai/papers/10.1186/1471-2288-12-138
- The paper
- Song, F., Clark, A., Bachmann, M. O., & Maas, J. (2012). Simulation evaluation of statistical properties of methods for indirect and mixed treatment comparisons. BMC Medical Research Methodology. https://doi.org/10.1186/1471-2288-12-138
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.